Alternate Quantizer Setup#
Alternate quantizers available in Ryzen-AI software platform as below
PyTorch/TensorFlow 2/TensorFlow Quantization: If the user requires Quantization Aware Training using the original model training framework, they can use Vitis AI PyTorch/Tensorflow 2/Tensorflow quantizer.
Olive Quantization: If the user is already familiar with Olive framework and using it for other model transformations, quantization through Olive is also an option.
Vitis AI PyTorch/TensorFlow 2/TensorFlow Quantization#
The Vitis AI PyTorch and TensorFlow Quantizer, which is part of the Vitis AI toolchain, require the installation of a Docker container on the host server.
The Vitis AI Docker container can be installed on Ubuntu 20.04, CentOS 7.8, 7.9, 8.1, and RHEL 8.3, 8.4. The developers working on Windows 11 can use WSL for installing Vitis AI Docker.
Multiple versions of the Docker container are available, each tailored to specific frameworks. Follow the Docker download and running instructions as per the following links:
Framework |
Docker location |
|---|---|
PyTorch |
|
TensorFlow 2 |
|
TensorFlow 1 |
The above Docker containers do not have GPU-accelerated quantization support. If you like to leverage GPU for the quantization process, you can download and build GPU Docker containers. Inside the downloaded TAR file you can find README that you can follow to build and run GPU dockers.
Olive Quantization#
Microsoft Olive framework supports quantization with Vitis AI ONNX Quantization. If you’re interested in exploring Olive Quantization as an advanced quantization method, you can follow the steps below:
Install Olive Quantization by running the following command:
pip install olive-ai[cpu]
The current Olive flow is not compatible with the latest pydantic version. To make it compatible, downgrade the pydantic version using the following command:
pip install pydantic==1.10.9
For additional information regarding the Olive installation, refer to the Microsoft Olive Documentation