NPU SDK Usage

This article mainly introduces how to convert neural network models of different platforms into model codes and data that can be run on the NPU.

Get SDK

1
2
$ mkdir workspace && cd workspace
$ git clone --recursive https://github.com/khadas/aml_npu_sdk.git

SDK Structure

Enter the SDK directory,

1
2
3
$ cd {workspace}/aml_npu_sdk
$ ls
acuity-toolkit android_sdk Dockerfile docs LICENSE linux_sdk README.md

The SDK is mainly divided into several SDK, conversion tools and compilation tools, and docs.

1
2
3
acuity-toolkit    #Conversion tool , used to convert AI models
android_sdk #Android SDK
docs #Conversion related documents collection
Note

Since all linux codes have been supports local compiled, host compilation is no longer supported. Therefore, the contents of linux_sdk have been completely removed

Docs

entre Docs directory,

1
2
3
4
$ cd {workspace}/aml_npu_sdk/docs/en
$ ls
'Amlogic NN Convolution Acceleration Tips.pdf' 'Android NN JNI Development Guide Eng.pdf' 'Neural Network Layer and Operation Support Guide .pdf'
'Amlogic NN Integration Guide Eng.pdf' 'Model_Transcoding and Running User Guide Eng.pdf' 'NN Tool FAQ Eng.pdf'

The document records a series of processes from conversion to integration, as well as some common problems

Conversion Tool

acuity-toolkit is the conversion tool directory,

1
2
3
$ cd {workspace}/aml_npu_sdk/acuity-toolkit
$ ls
bin demo python ReadMe.txt requirements.txt

The main directory of interest is demo

1
2
3
4
5
1. bin                   #Conversion is a collection of various tools used, most of which are not open source.
2. demo #Conversion script directory, convert AI model location
3. python #Used to convert the model and data corresponding to the python API
4. ReadMe.txt #ReadMe.txt file explains how to convert and use
5. requirements.txt #Conversion tool dependent environment

Dependent Installation

The environment dependency package required by the conversion tool can be installed directly on the PC or installed through the virtual environment virtualenv

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
$ cd {workspace}/aml_npu_sdk/acuity-toolkit
$ cat requirements.txt
tensorflow==2.0.0
astor==0.8.0
numpy==1.18.0
scipy==1.1.0
Pillow==5.3.0
protobuf==3.11.2
networkx>=1.11
image==1.5.5
lmdb==0.93
onnx==1.6.0
h5py==2.10.0
flatbuffers==1.10
matplotlib==2.1.0
dill==0.2.8.2
ruamel.yaml==0.15.81
ply==3.11
torch==1.2.0

Among them, tensorflow==2.0.0 can be replaced by tensorflow==2.0.0a0

Conversion Script

The conversion script is in the demo directory,

1
2
3
$ cd {workspace}/aml_npu_sdk/acuity-toolkit/demo
$ ls
0_import_model.sh 1_quantize_model.sh 2_export_case_code.sh data extractoutput.py inference.sh model

Use scripts to convert AI models

1
2
$ cd {workspace}/aml_npu_sdk/acuity-toolkit/demo
$ bash 0_import_model.sh && bash 1_quantize_model.sh && bash 2_export_case_code.sh

After the conversion is completed, you can see the converted code in the nbg_unify_xxxx directory, here is the built-in model as an example

1
2
3
4
$ cd {workspace}/aml_npu_sdk/acuity-toolkit/demo/nbg_unify_mobilenet_tf
$ ls
BUILD makefile.linux mobilenettf.vcxproj vnn_global.h vnn_mobilenettf.h vnn_post_process.h vnn_pre_process.h
main.c mobilenet_tf.nb nbg_meta.json vnn_mobilenettf.c vnn_post_process.c vnn_pre_process.c

For the setting of conversion parameters, please refer to ‘Model_Transcoding and Running User Guide Eng.pdf’ in Docs