Other Quantizers#

The Ryzen AI software provides support for these additional quantizers:

  1. Vitis AI Quantizer for PyTorch/TensorFlow 2: If you require Quantization Aware Training using the original model training framework, you can use Vitis AI PyTorch/Tensorflow 2 quantizer.

  2. Olive Quantizer: If you are already familiar with the Olive framework and using it for other model transformations, quantization through Olive is also an option.

Vitis AI Quantizer for PyTorch/TensorFlow 2#

The Vitis AI Quantizer, integrated as a component of either PyTorch or TensorFlow 2, is distributed through framework-specific Docker containers.

The Vitis AI Docker containers can be installed on Ubuntu 20.04, CentOS 7.8, 7.9, 8.1, and RHEL 8.3, 8.4. For developers working on Windows 11, WSL can be used to install the Vitis AI Docker containers.

Standard Containers#

Multiple versions of the Docker container are available, each tailored to specific frameworks. Follow the Docker download and running instructions as per the following links:

Framework

Docker location

PyTorch

https://hub.docker.com/r/amdih/ryzen-ai-pytorch

TensorFlow 2

https://hub.docker.com/r/amdih/ryzen-ai-tensorflow2

GPU-Accelerated Quantization Containers#

The standard Vitis AI Docker containers do not support GPU-accelerated quantization. To create a container with GPU-accelerated quantization enabled, download the following archive and follow the instructions in the README file.

Download and build GPU Docker containers

Note

In this documentation, “NPU” is used in descriptions, while “IPU” is retained in the tool’s language, code, screenshots, and commands. This intentional distinction aligns with existing tool references and does not affect functionality. Avoid making replacements in the code.

Olive Quantizer#

Microsoft Olive framework supports quantization with Vitis AI ONNX Quantization. The Olive Quantizer can be installed by following these steps:

  1. Install the Olive Quantizer as follows:

pip install olive-ai[cpu]

  1. The current version of the Olive Quantizer is not compatible with the latest version of the pydantic library. To make it compatible, downgrade the pydantic version using the following command:

pip install pydantic==1.10.9

For additional information regarding the installation of Olive, refer to the Microsoft Olive Documentation