Application Development

Application Development#

This page captures requirements and recommendations for developers looking to create, package and distribute applications targeting NPU-enabled AMD processors.

VitisAI EP / NPU Driver Compatibility#

For each version of the VitisAI EP, the EP bounds compatible drivers by a minimum version and a maximum release date.

The following table summarizes the driver requirements for the different versions of the VitisAI EP.

VitisAI EP version

Minimum NPU Driver version

Maximum NPU Driver release date

1.7

32.0.203.280

Jan 22nd, 2029

1.6

32.0.203.280

Oct 7th, 2028

1.5

32.0.203.280

July 1st, 2028

1.4.1

32.0.203.259

May 13th, 2028

1.4

32.0.203.257

March 25th, 2028

1.3.1

32.0.203.242

January 17th, 2028

1.3

32.0.203.237

November 26th, 2027

1.2

32.0.201.204

July 30th, 2027

The application must verify that the user has installed NPU drivers compatible with the version of the Vitis AI EP in use.

APU Types#

The Ryzen AI Software supports various types of NPU-enabled APUs, referred to as PHX, HPT, STX, and KRK. To programmatically determine the type of the local APU, you can enumerate the PCI devices and look for an instance with a matching Hardware ID.

Vendor

Device

Revision

APU Type

0x1022

0x1502

0x00

PHX or HPT

0x1022

0x17F0

0x00

STX

0x1022

0x17F0

0x10

STX

0x1022

0x17F0

0x11

STX

0x1022

0x17F0

0x20

KRK

The application must verify that it runs on an AMD processor with NPU, and that the Vitis AI EP version supports the NPU type.

NPU Utilities#

When deploying applications across various NPU devices, users can determine the specific type of NPU device using Python/C++ code. Based on the detected device—such as PHX, STX, KRK, or other device—users should configure the appropriate provider options as outlined in Model Compilation and Deployment.

For Python, the user can get the specific NPU type using the following example get_npu_info function in the %RYZEN_AI_INSTALLATION_PATH%\quicktest\quicktest.py

For C++, a set of APIs are provided to extract information about the NPU and check driver compatibility of the VitisAI EP with the rest of the environment. For details refer to C++ NPU Utilities

Application Development Requirements#

ONNX-RT Session#

The application should only use the Vitis AI Execution Provider if the user meets the following

  • The application is running on an AMD processor with an NPU type supported by the version of the Vitis AI EP in use. See list.

  • The user installs NPU drivers compatible with the version of the Vitis AI EP they are using. See compatibility table .

📝 NOTE: Sample C++ code that implements the compatibility checks to be performed before using the Vitis AI EP is available here

VitisAI EP Provider Options#

For INT8 models, the application should detect the type of APU present (PHX, HPT, STX, or KRK) and set the target and xclbin provider options accordingly. Refer to the section on using INT8 models for more details.

For BF16 models, the application should set the config_file provider option to the same file that the compiler used to precompile the BF16 model. Refer to the section on using BF16 models for more details.

Pre-Compiled Models#

Precompiled models load instantly and execute immediately on the NPU, significantly improving session creation time and overall end-user experience.

AMD recommends using the ONNXRuntime EP Context Cache feature for saving and reloading compiled models.

BF16 models

The deployment version of the VitisAI Execution Provider (EP) does not support the on-the-fly compilation of BF16 models. Applications utilizing BF16 models must include pre-compiled versions of these models. The VitisAI EP can then load the pre-compiled models and deploy them efficiently on the NPU.

INT8 models

The compiler recommends including pre-compiled versions of INT8 models, but it is not mandatory.


Application Packaging Requirements#

Ryzen AI 1.8.0 beta version is available for download at the following link. This beta version requires updated NPU driver.

A C++ application built on the Ryzen AI ONNX Runtime must include the following components in its distribution package:

For INT8 models

  • DLLs:

    • %RYZEN_AI_INSTALLATION_PATH%\deployment\aiecompiler_client.dll

    • %RYZEN_AI_INSTALLATION_PATH%\deployment\DirectML.dll

    • %RYZEN_AI_INSTALLATION_PATH%\deployment\dyn_dispatch_core.dll

    • %RYZEN_AI_INSTALLATION_PATH%\deployment\onnxruntime_providers_shared.dll

    • %RYZEN_AI_INSTALLATION_PATH%\deployment\onnxruntime_providers_vitisai.dll

    • %RYZEN_AI_INSTALLATION_PATH%\deployment\onnxruntime_vitis_ai_custom_ops.dll

    • %RYZEN_AI_INSTALLATION_PATH%\deployment\onnxruntime_vitisai_ep.dll

    • %RYZEN_AI_INSTALLATION_PATH%\deployment\onnxruntime.dll

  • NPU Binary files (.xclbin) from the %RYZEN_AI_INSTALLATION_PATH%\voe-4.0-win_amd64\xclbins folder

  • Recommended but not mandatory: pre-compiled models in the form of Onnx Runtime EP context models

For BF16 models

  • DLLs:

    • %RYZEN_AI_INSTALLATION_PATH%\deployment\onnxruntime.dll

    • %RYZEN_AI_INSTALLATION_PATH%\deployment\onnxruntime_providers_shared.dll

    • %RYZEN_AI_INSTALLATION_PATH%\deployment\onnxruntime_providers_vitisai.dll

    • %RYZEN_AI_INSTALLATION_PATH%\deployment\onnxruntime_vitisai_ep.dll

    • %RYZEN_AI_INSTALLATION_PATH%\deployment\dyn_dispatch_core.dll

    • %RYZEN_AI_INSTALLATION_PATH%\deployment\aiecompiler_client.dll

    • %RYZEN_AI_INSTALLATION_PATH%\deployment\vaiml.dll

  • Pre-compiled models in the form of Vitis AI EP cache folders

For LLMs

  • DLLs:

    • %RYZEN_AI_INSTALLATION_PATH%\deployment\onnxruntime-genai.dll

    • %RYZEN_AI_INSTALLATION_PATH%\deployment\onnxruntime.dll

    • %RYZEN_AI_INSTALLATION_PATH%\deployment\ryzen_mm.dll

    • %RYZEN_AI_INSTALLATION_PATH%\deployment\onnx_custom_ops.dll

    • %RYZEN_AI_INSTALLATION_PATH%\deployment\libutf8_validity.dll

    • %RYZEN_AI_INSTALLATION_PATH%\deployment\abseil_dll.dll

    • %RYZEN_AI_INSTALLATION_PATH%\deployment\DirectML.dll

Contents