This is an old revision of the document!
We will use a densenet model based on YCG09/chinese_ocr
git clone https://github.com/YCG09/chinese_ocr
Follow Docker official documentation to install Docker: Install Docker Engine on Ubuntu.
Then fetch the prebuilt NPU Docker Container and run it.
$ docker pull yanwyb/npu:v1 $ docker run -it --name vim4-npu1 -v $(pwd):/home/khadas/npu \ -v /etc/localtime:/etc/localtime:ro \ -v /etc/timezone:/etc/timezone:ro \ yanwyb/npu:v1
Download The conversion tool from khadas/vim4_npu_sdk.
$ git clone https://gitlab.com/khadas/vim4_npu_sdk
After training model, run the codes as follows to modify net input and output and convert model to onnx.
Keras model(.h5) can convert VIM4 model directly. If you want to convert keras model, please use model.save
to save model with weight and network structure.
import onnx from keras.models import * import keras import keras2onnx from train import get_model import densenet basemodel, model = get_model(32, 88) # input height, classes number basemodel.load_weights("models/weights_densenet-32-0.40.h5") onnx_model = keras2onnx.convert_keras(basemodel, basemodel.name, target_opset=12) onnx_model.graph.input[0].type.tensor_type.shape.dim[0].dim_value = int(1) onnx_model.graph.input[0].type.tensor_type.shape.dim[1].dim_value = int(1) onnx_model.graph.input[0].type.tensor_type.shape.dim[2].dim_value = int(32) onnx_model.graph.input[0].type.tensor_type.shape.dim[3].dim_value = int(280) onnx_model.graph.output[0].type.tensor_type.shape.dim[0].dim_value = int(1) onnx_model.graph.node.remove(onnx_model.graph.node[0]) onnx_model.graph.node[0].input[0] = "the_input" onnx.save_model(onnx_model, "./densenet_ctc.onnx")
Enter vim4_npu_sdk/demo
and modify convert_adla.sh
as follows. We should quantize the model to int16, because it is very inaccurate with int8.
#!/bin/bash ACUITY_PATH=../bin/ #ACUITY_PATH=../python/tvm/ adla_convert=${ACUITY_PATH}adla_convert if [ ! -e "$adla_convert" ]; then adla_convert=${ACUITY_PATH}adla_convert.py fi $adla_convert --model-type onnx \ --model ./model_source/densenet_ctc/densenet_ctc.onnx \ --inputs "the_input" \ --input-shapes "1,32,280" \ --dtypes "float32" \ --inference-input-type float32 \ --inference-output-type float32 \ --quantize-dtype int16 --outdir onnx_output \ --channel-mean-value "0,0,0,255" \ --source-file ./densenet_ctc_dataset.txt \ --iterations 500 \ --disable-per-channel False \ --batch-size 1 --target-platform PRODUCT_PID0XA003
Run convert_adla.sh
to generate VIM4 model. The converted model is xxx.adla
in onnx_output
.
$ bash convert_adla.sh
Clone the source code khadas/vim4_npu_applications.
$ git clone https://github.com/khadas/vim4_npu_applications
$ sudo apt update $ sudo apt install libopencv-dev python3-opencv cmake
Put densenet_ctc_int16.adla
in vim4_npu_applications/densenet_ctc/data/
.
# Compile $ cd vim4_npu_applications/densenet_ctc $ mkdir build $ cd build $ cmake .. $ make # Run $ sudo ./densenet_ctc -m ../data/densenet_ctc_int16.adla -p ../data/KhadasTeam.png
If your densenet_ctc model classes is not the same, please change data/class_str.txt
and the OBJ_CLASS_NUM
in include/postprocess.h
.