Stable Diffusion Demo

Ryzen AI 1.7.1 provides preview demos of Stable Diffusion image-generation pipelines. The demos cover Image-to-Image and Text-to-Image using SD1.5, SDXL-base-1.0, Segmind-Vega, SD-Turbo (bs1), SDXL-Turbo (bs1), SD3.0 and SD3.5. These models are available for public download from Hugging Face.

Installation Steps

1. Ensure the latest version of Ryzen AI and NPU drivers are installed. See Installation Instructions.

2. The GenAI-SD folder is located in the RyzenAI installation tree. Navigate to the folder and run the following command:

cd "C:\Program Files\RyzenAI\1.7.1\GenAI-SD"

3. Activate the Conda environment for the Stable Diffusion demo packages:

conda activate ryzen-ai-1.7.1

4. The following Stable Diffusion models will be auto-downloaded from Hugging Face when running for the first time and cached locally in the GenAI-SD\models folder:

   • SD1.5

   • SD-Turbo (bs1)

   • SDXL-Turbo (bs1)

   • SDXL-base-1.0

   • Segmind-Vega

   • SD3.0 / SD3.0-ControlNet(Canny) / SD3.0-ControlNet(Pose) / SD3.0-ControlNet(Tile) / SD3.0-ControlNet(Depth)

   • SD3.5

Running the Demos

Activate the conda environment:

conda activate ryzen-ai-1.7.1

Optionally, set the NPU to high performance mode to maximize performance:

xrt-smi configure --pmode performance

Refer to the documentation on xrt-smi configure for additional information.

Image-to-Image with ControlNet

The image-to-image demo generates images based on a prompt and a control image using ControlNet (for example, Canny, pose, tile, or depth). This demo supports SD3.0 and uses 512x512 as the default resolution, which can be overridden (for example, to 1024x1024) via the -W/-H options in the CLI examples below.

To run the demo, navigate to the GenAI-SD\test directory and run the following command:

python run_sd3.py -C canny --model_id "stabilityai/stable-diffusion-3-medium-amdnpu"

The demo script uses a predefined prompt and .\ref\canny.jpg as the control image. The output image and control image are saved in the generated_images folder. You can redirect the output directory to a location where your user account has write permissions, for example: --output_path C:\Users\<username>\Documents\generated_images.

The control image can be modified and custom prompts can be provided with the --prompt option. For instance:

python run_sd3.py -C canny --model_id "stabilityai/stable-diffusion-3-medium-amdnpu" --prompt "Anime style illustration of a girl wearing a suit. A moon in sky. In the background we see a big rain approaching. text 'InstantX' on image" -n 50

The application of ControlNet can be configured with the -C option. For instance:

python run_sd3.py -C canny --model_id "stabilityai/stable-diffusion-3-medium-amdnpu" --prompt "Anime style illustration of a girl wearing a suit. A moon in sky. In the background we see a big rain approaching. text 'InstantX' on image" -H 1024 -W 1024 --control_image_path .\ref\canny.jpg -n 50
python run_sd3.py -C pose --model_id "stabilityai/stable-diffusion-3-medium-amdnpu" --prompt "Anime style illustration of a girl wearing a suit. A moon in sky. In the background we see a big rain approaching. text 'InstantX' on image" -H 1024 -W 1024 --control_image_path .\ref\pose.jpg -n 50
python run_sd3.py -C tile --model_id "stabilityai/stable-diffusion-3-medium-amdnpu" --prompt "Anime style illustration of a girl wearing a suit. A moon in sky. In the background we see a big rain approaching. text 'InstantX' on image" -H 1024 -W 1024 --control_image_path .\ref\tile.jpg -n 50
python run_sd3.py -C depth --model_id "stabilityai/stable-diffusion-3-medium-amdnpu" -H 1024 -W 1024 --control_image_path .\assets\depth.jpeg -n 50

To run the image-to-image demo of Segmind-Vega model (without ControlNet applications), run the following command:

python .\run_sd_xl.py --model_id "amd/segmind-vega-amdnpu" --control_image_path .\assets\controlimg_input_1024x1024.png --strength 0.95

Text-to-Image

The text-to-image demo generates images based on text prompts. It supports SD 1.5 (512x512), SDXL-base (1024x1024), SD-Turbo (512x512), SDXL-Turbo (512x512), Segmind-Vega (1024x1024), as well as SD 3.0 and SD 3.5 (see below for additional setup required for SD3.x models).

To run the demo, navigate to the GenAI-SD\test directory and run the following commands to run with each of the supported models:

python run_sd.py    --model_id "amd/stable-diffusion-1.5-amdnpu"
python run_sd.py    --model_id "amd/sd-turbo-amdnpu"
python run_sd_xl.py --model_id "amd/sdxl-turbo-amdnpu"
python run_sd_xl.py --model_id "amd/sdxl-base-amdnpu"
python run_sd_xl.py --model_id "amd/segmind-vega-amdnpu"

To run the sd3/sd3.5 models, you need to set the DD_PLUGINS_ROOT environment variable before running the demo. For instance:

set "DD_PLUGINS_ROOT=C:\Program Files\RyzenAI\1.7.1\GenAI-SD\lib\transaction\stx\"

Then run the following commands:

python .\run_sd3.py -C None --model_id "stabilityai/stable-diffusion-3-medium-amdnpu" -n 50
python .\run_sd3.py -C None --model_id "stabilityai/stable-diffusion-3.5-medium-amdnpu" -n 50

The demo script uses a predefined prompt for each of the models. The output images are saved in the generated_images folder.

Custom prompts can be provided with the --prompt option. For instance:

python run_sd.py --model_id "amd/stable-diffusion-1.5-amdnpu" --prompt "Photo of a ultra realistic sailing ship, dramatic light, pale sunrise, cinematic lighting, battered, low angle, trending on artstation, 4k, hyper realistic, focused, extreme details"

Running with AMD Stable Diffusion Sandbox

AMD SD Sandbox is a framework for running Stable Diffusion (SD) models accelerated by AMD Ryzen AI hardware. It provides an easy-to-use interface for evaluating, comparing, and deploying multiple SD pipelines. Please go to the AMD SD Sandbox GitHub repository for more information.