User Tools
Sidebar
Table of Contents
ArmNN TFLite Delegate
Khadas Edges and VIMs are powered by ARM Cortex-A CPUs and Mali GPUs.
We can make use of the available computation power for running TFLite models using ArmNN Libraries by leveraging TFLite interpreter Delegate.
The Delegate is provided by ARM as arm-software/armnn and the provided examples are made to make use of it.
This library depends on OpenCL, make sure you are using the following platforms to ensure the driver is present.
| board | Linux Kernel (BSP) | OS |
|---|---|---|
| VIM3 | 4.9 5.15 | Ubuntu 22.04 |
| VIM3L | 4.9 5.15 | Ubuntu 22.04 |
| VIM4 | 5.4 5.15 | Ubuntu 22.04 |
| Edge2 | 5.10 | Ubuntu 22.04 |
You can refer to the OpenCL doc for more info.
Get source code
Clone the examples sravansenthiln1/armnn_tflite
$ git clone https://github.com/sravansenthiln1/armnn_tflite $ cd armnn_tflite
Setup the environment
Install pip
$ sudo apt-get install python3-pip
Install necessary python packages
$ pip3 install numpy pillow
Install the TFLite runtime interpreter
$ pip3 install --extra-index-url https://google-coral.github.io/py-repo/ tflite_runtime
Download ArmNN libraries
$ wget -O ArmNN-aarch64.tgz https://github.com/ARM-software/armnn/releases/download/v23.08/ArmNN-linux-aarch64.tar.gz $ mkdir libs $ tar -xvf ArmNN-aarch64.tgz -C libs
Run Examples
Taking the Mobilenet v1 as example.
Enter the example directory
$ cd mobilenet_v1
Create library symlinks
$ sudo ln ../libs/libarmnnDelegate.so.29.0 libarmnnDelegate.so.29 $ sudo ln ../libs/libarmnn.so.33.0 libarmnn.so.33
Run the example
$ python3 run_inference.py
You can modify whether you want to use the CPU or GPU to accelerate the inference.
Change the BACKEND variable in the code to use either GpuAcc - GPU or CpuAcc - CPU backends.