Basic information and examples about how to use Amlogic NPU SDK for VIM3

Get source: khadas/aml_npu_sdk

$ mkdir workspace && cd workspace
$ git clone --recursive https://github.com/khadas/aml_npu_sdk

Enter the SDK directory,

$ cd aml_npu_sdk
$ ls
acuity-toolkit  android_sdk  Dockerfile  docs  LICENSE  linux_sdk  README.md

The SDK contains the Android SDK, conversion and compilation tools, and manuals.

acuity-toolkit    # Conversion tool, used to convert AI models
android_sdk       # Android SDK
docs              # Conversion-related documents collection

acuity-toolkit directory contains the conversion tool,

$ cd aml_npu_sdk/acuity-toolkit
$ ls
bin  demo  python  ReadMe.txt  requirements.txt

demo directory is where we can do model conversion,

bin                   # Conversion is a collection of various tools used, most of which are not open source.
demo                  # Conversion script directory, convert AI model location
demo_hybird           # Mixed Input Conversion Tool
mulity_input_demo     # mulity input demo
python                # Used to convert the model and data corresponding to the Python API
ReadMe.txt            # ReadMe.txt file explains how to convert and use
requirements.txt      # Conversion tool dependent environment

The environment package dependencies required by the conversion tool can be installed directly on the PC or installed through the virtual environment.

$ cd aml_npu_sdk/acuity-toolkit
$ cat requirements.txt
tensorflow==2.0.0
astor==0.8.0
numpy==1.18.0
scipy==1.1.0
Pillow==5.3.0
protobuf==3.11.2
networkx>=1.11
image==1.5.5
lmdb==0.93
onnx==1.6.0
h5py==2.10.0
flatbuffers==1.10
matplotlib==2.1.0
dill==0.2.8.2
ruamel.yaml==0.15.81
ply==3.11
torch==1.2.0

Here, tensorflow==2.0.0 can be replaced by tensorflow==2.0.0a0.

The conversion script is in the acuity-toolkit/demo directory,

$ cd aml_npu_sdk/acuity-toolkit/demo
$ ls -1
data
model
0_import_model.sh
1_quantize_model.sh
2_export_case_code.sh
extractoutput.py
inference.sh

Use scripts to convert AI models.

$ cd aml_npu_sdk/acuity-toolkit/demo
$ bash 0_import_model.sh && bash 1_quantize_model.sh && bash 2_export_case_code.sh 

• 0_import_model.sh - Import model script. Now, it supports loading Tensorflow, Caffe, Tensorflow Lite, Onnx, Keras, Pytorch and Darknet model.
• 1_quantize_model.sh - Quantize model script. Now, it can quantize model in int8, int16 and uint8.
• 2_export_case_code.sh - Export model script. If you use VIM3, please set optimize to VIPNANOQI_PID0X88. If use VIM3L, please set optimize to VIPNANOQI_PID0X99.

After the conversion is completed, you can see the converted code in the xxxx_nbg_unify directory. Converted model is xxxx.nb. Here is the built-in model as an example.

$ cd aml_npu_sdk/acuity-toolkit/demo/mobilenet_tf_nbg_unify
$ ls
BUILD
makefile.linux
mobilenettf.vcxproj
main.c
mobilenet_tf.nb
nbg_meta.json
vnn_global.h
vnn_mobilenettf.h
vnn_post_process.h
vnn_pre_process.h
vnn_mobilenettf.c
vnn_post_process.c
vnn_pre_process.c

For the conversion parameters and settings, please refer to:

• khadas/aml_npu_sdk/tree/master/docs/en
• Model Transcoding and Running User Guide