Table of Contents

ArmNN TFLite Delegate

Khadas Edges and VIMs are powered by ARM Cortex-A CPUs and Mali GPUs.

We can make use of the available computation power for running TFLite models using ArmNN Libraries by leveraging TFLite interpreter Delegate.

The Delegate is provided by ARM as arm-software/armnn and the provided examples are made to make use of it.

This library depends on OpenCL, make sure you are using the following platforms to ensure the driver is present.

board Linux Kernel (BSP) OS
VIM3 4.9
5.15
Ubuntu 22.04
VIM3L 4.9
5.15
Ubuntu 22.04
VIM4 5.4
5.15
Ubuntu 22.04
Edge2 5.10 Ubuntu 22.04

You can refer to the OpenCL doc for more info.

Get source code

Clone the examples sravansenthiln1/armnn_tflite

$ git clone https://github.com/sravansenthiln1/armnn_tflite
$ cd armnn_tflite

Setup the environment

Install pip

$ sudo apt-get install python3-pip

Install necessary python packages

$ pip3 install numpy pillow

Install the TFLite runtime interpreter

$ pip3 install --extra-index-url https://google-coral.github.io/py-repo/ tflite_runtime

Download ArmNN libraries

$ wget -O ArmNN-aarch64.tgz https://github.com/ARM-software/armnn/releases/download/v23.08/ArmNN-linux-aarch64.tar.gz
$ mkdir libs
$ tar -xvf ArmNN-aarch64.tgz -C libs

Run Examples

Taking the Mobilenet v1 as example.

Enter the example directory

$ cd mobilenet_v1
$ sudo ln ../libs/libarmnnDelegate.so.29.0 libarmnnDelegate.so.29
$ sudo ln ../libs/libarmnn.so.33.0 libarmnn.so.33

Run the example

$ python3 run_inference.py

You can modify whether you want to use the CPU or GPU to accelerate the inference. Change the BACKEND variable in the code to use either GpuAcc - GPU or CpuAcc - CPU backends.