This document provides a detailed introduction for beginners on how to quickly use RKNN-Toolkit2 on a computer to perform model conversion and deploy it to a Edge 2L board using RKNPU2. The examples used in this document are integrated into the RKNN Model Zoo.

The version information we are using is as follows:

• Host PC Python: 3.8
• rknn-toolkit2: 2.3.2
• rknn model zoo: 2.3.2

It is recommended to create a new directory to store the RKNN repositories. For example, create a folder named Projects and place the RKNN-Toolkit2 and RKNN Model Zoo repositories in that directory. Refer to the following commands

$ mkdir Projects && cd Projects 

Download the RKNN-Toolkit2 repository

$ git clone -b v2.3.2 https://github.com/airockchip/rknn-toolkit2.git --depth 1

Download the RKNN Model Zoo repository

$ git clone -b v2.3.2 https://github.com/airockchip/rknn_model_zoo.git --depth 1

If the Python 3.8 environment is not installed on your system, or if there are multiple versions of Python installed, it is recommended to use Miniforge Conda to create a new Python 3.8 environment.

Download the Miniforge Conda installer from the following link:

$ wget -c https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-Linux-x86_64.sh

Then, install Miniforge Conda using the following command:

$ chmod 777 Miniforge3-Linux-x86_64.sh 
$ bash Miniforge3-Linux-x86_64.sh

In the terminal window on the computer, execute the following command to switch to the Miniforge conda base environment:

$ source ~/miniforge3/bin/activate

Create a Python 3.8 environment named py38Toolkit2.3.2 using the following command:

$ conda create -n py38Toolkit2.3.2 python=3.8

Activate the py38Toolkit2.3.2 environment. Subsequently, RKNN-Toolkit2 will be installed in this environment

$ conda activate py38Toolkit2.3.2

Install via local wheel packag.

$ cd rknn-toolkit2/rknn-toolkit2/packages/x86_64/
$ pip3 install -r requirements_cp38-2.3.2.txt
$ pip3 install ./rknn_toolkit2-2.3.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl

Switch to Python interactive mode

$ python

Import the RKNN class

$ from rknn.api import RKNN

Note: This section is applicable to development boards with the Android system. If the board is running on the Linux system, please skip this section.
Download the Android NDK from the following link:

$ wget -c https://dl.google.com/android/repository/android-ndk-r19c-linux-x86_64.zip

Extract Android NDK files

$ unzip android-ndk-r19c-linux-x86_64.zip

Note: This section is applicable to development boards with the Linux system. If the board is running on the Android system, please skip this section.
Download the GCC Cross-Compiler from the following link:

$ wget -c https://releases.linaro.org/components/toolchain/binaries/6.3-2017.05/aarch64-linux-gnu/gcc-linaro-6.3.1-2017.05-x86_64_aarch64-linux-gnu.tar.xz

Extract GCC Cross-Compiler files

$ tar -xJvf gcc-linaro-6.3.1-2017.05-x86_64_aarch64-linux-gnu.tar.xz

Switch to the rknn_model_zoo/examples/yolov5/model directory, and execute the download_model.sh script. This script will download an available YOLOv5 ONNX model and store it in the current model directory. Refer to the following commands:

$ cd Projects/rknn_model_zoo/examples/yolov5/model
$ chmod a+x download_model.sh && ./download_model.sh

Switch to the rknn_model_zoo/examples/yolov5/python directory and run the convert.py script. This script converts the original ONNX model to the RKNN model. Refer to the following commands:

$ cd Projects/rknn_model_zoo/examples/yolov5/python
$ chmod a+x convert.py && python convert.py ../model/yolov5s_relu.onnx rk3576 i8 ../model/yolov5s_relu.rknn

Switch to the rknn_model_zoo/examples/yolov5/python directory, execute the yolov5.py script, and you can run the YOLOv5 model on the development board using on-board debugging. Refer to the following command:

$ cd Projects/rknn_model_zoo/examples/yolov5/python
$ chmod a+x yolov5.py && python yolov5.py --model_path ../model/yolov5s_relu.rknn --target rk3576 --img_show

Running on a Ubuntu 24.04 system which ship the python version of 3.12.

Download the model zoo to your Edge 2L:

git clone -b v2.3.2 https://github.com/airockchip/rknn_model_zoo.git --depth 1

Install the python virtual environment:

sudo apt update
sudo apt install python3.12-venv -y

Create the python virtual environment:

python3 -m venv myenv312

Activate the python virtual environment:

source myenv312/bin/activate
(myenv312) khadas@Khadas:~$ python --version
Python 3.12.3

Install the python package from rknn-toolkit2/rknn-toolkit-lite2/packages/rknn_toolkit_lite2-2.3.2-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl

(myenv312) khadas@Khadas:~$ pip install rknn_toolkit_lite2-2.3.2-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl 

Install the runtime libraries from rknn-toolkit2/rknpu2/runtime/Linux/librknn_api/aarch64/librknnrt.so

sudo cp librknnrt.so /usr/lib/

Install the OpenCV:

(myenv312) khadas@Khadas:~$ pip install opencv-python

Copy the converted model in Model conversion to your Edge 2L:

adb push examples/yolov5/model/yolov5s_relu.rknn /home/khadas/rknn_model_zoo/examples/yolov5/model

Modify the demo code:

diff --git a/py_utils/rknn_executor.py b/py_utils/rknn_executor.py
index 21b4132..46b24d4 100644
--- a/py_utils/rknn_executor.py
+++ b/py_utils/rknn_executor.py
@@ -1,5 +1,5 @@
-from rknn.api import RKNN
-
+from rknnlite.api import RKNNLite as RKNN
+import numpy as np
 
 class RKNN_model_container():
     def __init__(self, model_path, target=None, device_id=None) -> None:
@@ -9,10 +9,7 @@ class RKNN_model_container():
         rknn.load_rknn(model_path)
 
         print('--> Init runtime environment')
-        if target==None:
-            ret = rknn.init_runtime()
-        else:
-            ret = rknn.init_runtime(target=target, device_id=device_id)
+        ret = rknn.init_runtime()
         if ret != 0:
             print('Init runtime environment failed')
             exit(ret)
@@ -33,10 +30,12 @@ class RKNN_model_container():
         else:
             inputs = [inputs]
 
+        inputs = [np.expand_dims(item, axis=0) if len(item.shape) == 3 else item for item in inputs]
+
         result = self.rknn.inference(inputs=inputs)
 
         return result
 
     def release(self):
         self.rknn.release()
-        self.rknn = None
\ No newline at end of file
+        self.rknn = None
(myenv312) khadas@Khadas:~/rknn_model_zoo$ 

Run the Yolov5 demo:

(myenv312) khadas@Khadas:~/rknn_model_zoo/examples/yolov5/python$ export QT_QPA_PLATFORM=xcb
(myenv312) khadas@Khadas:~/rknn_model_zoo/examples/yolov5/python$ python yolov5.py --model_path ../model/yolov5s_relu.rknn --target rk3576 --img_show

Deactivate the python virtual enviranment:

(myenv312) khadas@Khadas:~/rknn_model_zoo$ deactivate 

To run a RKNN C demo, you need to first compile the C/C++ source code into an executable file. After that, push the executable file, model files, input images, and other related files to the development board. Finally, execute the executable file on the development board.

In the build-android.sh script, add the following command:

$ ANDROID_NDK_PATH=Projects/android-ndk-r19c

Then, in the rknn_model_zoo directory, execute the build-android.sh script, referring to the following command:

$ cd Projects/rknn_model_zoo
$ ./build-android.sh -t rk3576 -a arm64-v8a -d yolov5

Push Files to the Edge 2L board:

$ cd Projects/rknn_model_zoo
$ adb root
$ adb push install/rk3576_android_arm64-v8a/rknn_yolov5_demo/ /data/

Run the Demo on Edge 2L board:

$ adb shell
$ cd /data/rknn_yolov5_demo/
$ export LD_LIBRARY_PATH=./lib
$ ./rknn_yolov5_demo model/yolov5s_relu.rknn model/bus.jpg

In the build-linux.sh script, add the following command:

$ GCC_COMPILER=Projects/gcc-linaro-6.3.1-2017.05-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu

Then, in the rknn_model_zoo directory, execute the build-linux.sh script, referring to the following command:

$ cd Projects/rknn_model_zoo
$ ./build-linux.sh -t rk3576 -a aarch64 -d yolov5

Push Files to the Edge 2L board:

$ cd Projects/rknn_model_zoo
$ adb push install/rk3576_linux_aarch64/rknn_yolov5_demo/ /home/khadas/rknn_yolov5_demo

Run the Demo on Edge 2L board:

$ adb shell
$ cd /home/khadas/rknn_yolov5_demo
$ export LD_LIBRARY_PATH=./lib
$ ./rknn_yolov5_demo model/yolov5s_relu.rknn model/bus.jpg

By default, the output image is saved at the path rknn_yolov5_demo/out.png. You can use the adb tool to pull it from the board to the local machine. In the local computer terminal, execute the following command:

$ adb pull /home/khadas/rknn_yolov5_demo/out.png .

RKNN Toolkit2 Documentations