We can run TFLite models by converting them into RKNN format and running them on the onboard NPU.

Clone the examples sravansenthiln1/rknn_tflite.

$ git clone https://github.com/sravansenthiln1/rknn_tflite
$ cd rknn_tflite

You can convert TFLite models to run the NPU using the convert.py conversion script from the source.

Requires: Ubuntu 22.04/20.04/18.04 x86 Host computer.

After you have cloned the source code:

$ sudo apt-get install git python3 python3-dev python3-pip
$ sudo apt-get install libxslt1-dev zlib1g-dev libglib2.0 libsm6 libgl1-mesa-glx libprotobuf-dev gcc cmake

$ git clone https://github.com/rockchip-linux/rknn-toolkit2
$ cd rknn-toolkit2
$ git checkout b25dadacc24b88eb7dfcaa47c9c525ecca89b319

$ conda create -n npu-env
$ conda activate npu-env

$ python3 --version

and run the command accordingly

$ pip3 install -r rknn-toolkit2/packages/requirements_$version-*.txt

$ pip3 install rknn-toolkit2/packages/rknn_toolkit2-*-$version-$version-linux_x86_64.whl
$ cd ../

$ python3 convert.py

eg. to convert a file such as detect_model.tflite, run

$ python3 convert.py detect_model

in the same directory, a file called detect_model.rknn will have been created.

To run it on your board, you need to install appropriate RKNN API wheel

After cloning the source code:

$ sudo apt-get install python3-pip

$ pip3 install numpy pillow opencv-python librosa sounddevice

$ git clone https://github.com/rockchip-linux/rknn-toolkit2
$ cd rknn-toolkit2
$ git checkout b25dadacc24b88eb7dfcaa47c9c525ecca89b319

$ python3 --version

and run the command accordingly

$ pip3 install rknn_toolkit_lite2/packages/rknn_toolkit_lite2-*-$version-$version-linux_aarch64.whl

$ sudo cp rknpu2/runtime/Linux/librknn_api/aarch64/librknnrt.so /usr/lib/
$ cd ../

Taking the Mobilenet v1 as example.

$ cd mobilenet_v1

$ python3 run_npu_inference.py