Application Development#

This page captures requirements and recommendations for developers looking to create, package and distribute applications targeting NPU-enabled AMD processors.

VitisAI EP / NPU Driver Compatibility#

The VitisAI EP requires a compatible version of the NPU drivers. For each version of the VitisAI EP, compatible drivers are bounded by a minimum version and a maximum release date. NPU drivers are backward compatible with VitisAI EP released up to 3 years before. The maximum driver release date is therefore set to 3 years after the release date of the corresponding VitisAI EP.

The table below summarizes the driver requirements for the different versions of the VitisAI EP.

VitisAI EP version

Minimum NPU Driver version

Maximum NPU Driver release date

1.4

32.0.203.257

March 25th, 2028

1.3.1

32.0.203.242

January 17th, 2028

1.3

32.0.203.237

November 26th, 2027

1.2

32.0.201.204

July 30th, 2027

The application must check that NPU drivers compatible with the version of the Vitis AI EP being used are installed.

APU Types#

The Ryzen AI Software supports different types of NPU-enabled APUs. These APU types are referred to as PHX, HPT, STX and KRK.

To programmatically determine the type of the local APU, it is possible to enumerate the PCI devices and check for an instance with a matching Hardware ID.

Vendor

Device

Revision

APU Type

0x1022

0x1502

0x00

PHX or HPT

0x1022

0x17F0

0x00

STX

0x1022

0x17F0

0x10

STX

0x1022

0x17F0

0x11

STX

0x1022

0x17F0

0x20

KRK

The application must check that it is running on an AMD processor with an NPU, and that the NPU type is supported by the version of the Vitis AI EP being used.

Application Development Requirements#

ONNX-RT Session#

The application should only use the Vitis AI Execution Provider if the following conditions are met:

  • The application is running on an AMD processor with an NPU type supported by the version of the Vitis AI EP being used. See list above in this page.

  • NPU drivers compatible with the version of the Vitis AI EP being used are installed. See compatibility table above in this page.

📝 NOTE: Sample C++ code implementing the compatibility checks to be performed before using the VitisAI EP is provided here: amd/RyzenAI-SW

VitisAI EP Provider Options#

For INT8 models, the application should detect which type of APU is present (PHX/HPT/STX/KRK) and set the xclbin provider option accordingly. Refer to the section about compilation of INT8 models for details about this.

For BF16 models, the application should set the config_file provider option to use the same file as the one which was used to precompile the BF16 model. Refer to the section about compilation of BF16 models for details about this.

Cache Management#

Cache directories generated by the Vitis AI Execution Provider should not be reused across different versions of the Vitis AI EP or across different version of the NPU drivers.

The application should check the version of the Vitis AI EP and of the NPU drivers. If the application detects a version change, it should delete the cache, or create a new cache directory with a different name.

Pre-Compiled Models#

The deployment version of the VitisAI Execution Provider (EP) does not support the on-the-fly compilation of BF16 models. Applications utilizing BF16 models must include pre-compiled versions of these models. The VitisAI EP can then load the pre-compiled models and deploy them efficiently on the NPU.

Although including pre-compiled versions of INT8 models is not mandatory, it is beneficial as it reduces session creation time and enhances the end-user experience.


Application Packaging Requirements#

❗ IMPORTANT: A patched version of the %RYZEN_AI_INSTALLATION_PATH%\deployment folder is available for download at the following link: Download Here. This patched deployment folder is designed to replace the one included in the official installation of Ryzen AI 1.4. The following instructions assume that the original deployment folder has been replaced with the updated version.

A C++ application built on the Ryzen AI ONNX Runtime requires the following components to be included in its distribution package.

For INT8 models

  • DLLs:

    • %RYZEN_AI_INSTALLATION_PATH%\deployment\voe\dyn_dispatch_core.dll

    • %RYZEN_AI_INSTALLATION_PATH%\deployment\voe\onnxruntime.dll

    • %RYZEN_AI_INSTALLATION_PATH%\deployment\voe\onnxruntime_providers_shared.dll

    • %RYZEN_AI_INSTALLATION_PATH%\deployment\voe\onnxruntime_providers_vitisai.dll

    • %RYZEN_AI_INSTALLATION_PATH%\deployment\voe\onnxruntime_vitisai_ep.dll

    • %RYZEN_AI_INSTALLATION_PATH%\deployment\voe\transaction.dll

    • %RYZEN_AI_INSTALLATION_PATH%\deployment\voe\xclbin.dll

  • NPU Binary files (.xclbin) from the %RYZEN_AI_INSTALLATION_PATH%\voe-4.0-win_amd64\xclbins folder

  • Recommended but not mandatory: pre-compiled models in the form of Vitis AI EP cache folders or Onnx Runtime EP context models

For BF16 models

  • DLLs:

    • %RYZEN_AI_INSTALLATION_PATH%\deployment\voe\dyn_dispatch_core.dll

    • %RYZEN_AI_INSTALLATION_PATH%\deployment\voe\onnxruntime.dll

    • %RYZEN_AI_INSTALLATION_PATH%\deployment\voe\onnxruntime_providers_shared.dll

    • %RYZEN_AI_INSTALLATION_PATH%\deployment\voe\onnxruntime_providers_vitisai.dll

    • %RYZEN_AI_INSTALLATION_PATH%\deployment\voe\onnxruntime_vitisai_ep.dll

    • %RYZEN_AI_INSTALLATION_PATH%\deployment\voe\transaction.dll

    • %RYZEN_AI_INSTALLATION_PATH%\deployment\voe\xclbin.dll

    • %RYZEN_AI_INSTALLATION_PATH%\deployment\flexmlrt\flexmlrt.dll

  • Pre-compiled models in the form of Vitis AI EP cache folders

For Hybrid LLMs

  • DLLs:

    • %RYZEN_AI_INSTALLATION_PATH%\deployment\hybrid-llm\onnx_custom_ops.dll

    • %RYZEN_AI_INSTALLATION_PATH%\deployment\hybrid-llm\onnxruntime-genai.dll

    • %RYZEN_AI_INSTALLATION_PATH%\deployment\hybrid-llm\ryzen_mm.dll

    • %RYZEN_AI_INSTALLATION_PATH%\deployment\hybrid-llm\ryzenai_onnx_utils.dll

    • %RYZEN_AI_INSTALLATION_PATH%\deployment\voe\DirectML.dll

    • %RYZEN_AI_INSTALLATION_PATH%\deployment\voe\onnxruntime.dll

For NPU-only LLMs

  • DLLs:

    • %RYZEN_AI_INSTALLATION_PATH%\deployment\npu-llm\onnxruntime-genai.dll

    • %RYZEN_AI_INSTALLATION_PATH%\deployment\voe\onnxruntime_vitis_ai_custom_ops.dll

    • %RYZEN_AI_INSTALLATION_PATH%\deployment\voe\onnxruntime_providers_shared.dll

    • %RYZEN_AI_INSTALLATION_PATH%\deployment\voe\onnxruntime_vitisai_ep.dll

    • %RYZEN_AI_INSTALLATION_PATH%\deployment\voe\dyn_dispatch_core.dll

    • %RYZEN_AI_INSTALLATION_PATH%\deployment\voe\onnxruntime_providers_vitisai.dll

    • %RYZEN_AI_INSTALLATION_PATH%\deployment\voe\transaction.dll

    • %RYZEN_AI_INSTALLATION_PATH%\deployment\voe\onnxruntime.dll

    • %RYZEN_AI_INSTALLATION_PATH%\deployment\voe\xclbin.dll

  • VAIP LLM configuration file: %RYZEN_AI_INSTALLATION_PATH%\deployment\npu-llm\vaip_llm.json