This is an old revision of the document!
We will use a DenseNet model based on YCG09/chinese_ocr
git clone https://github.com/YCG09/chinese_ocr
Follow Docker official documentation to install Docker: Install Docker Engine on Ubuntu.
Then fetch the prebuilt NPU Docker container and run it.
$ docker pull yanwyb/npu:v1 $ docker run -it --name vim4-npu1 -v $(pwd):/home/khadas/npu \ -v /etc/localtime:/etc/localtime:ro \ -v /etc/timezone:/etc/timezone:ro \ yanwyb/npu:v1
Download The conversion tool from khadas/vim4_npu_sdk.
$ git clone https://gitlab.com/khadas/vim4_npu_sdk
After training the model, run the scripts as follows to modify net input and output and convert the model to ONNX.
Keras model(.h5
) can be converted into a VIM4 model directly. If you want to convert a Keras model, please use model.save
to save the model with weight and network structure.
import onnx from keras.models import * import keras import keras2onnx from train import get_model import densenet basemodel, model = get_model(32, 88) # input height, classes number basemodel.load_weights("models/weights_densenet-32-0.40.h5") onnx_model = keras2onnx.convert_keras(basemodel, basemodel.name, target_opset=12) onnx_model.graph.input[0].type.tensor_type.shape.dim[0].dim_value = int(1) onnx_model.graph.input[0].type.tensor_type.shape.dim[1].dim_value = int(1) onnx_model.graph.input[0].type.tensor_type.shape.dim[2].dim_value = int(32) onnx_model.graph.input[0].type.tensor_type.shape.dim[3].dim_value = int(280) onnx_model.graph.output[0].type.tensor_type.shape.dim[0].dim_value = int(1) onnx_model.graph.node.remove(onnx_model.graph.node[0]) onnx_model.graph.node[0].input[0] = "the_input" onnx.save_model(onnx_model, "./densenet_ctc.onnx")
The model input is grayscale image, so before quantification, you should convert your quantification picture into grayscale image and save in npy
file.
import numpy as np import cv2 import os image_path = "./image/" save_path = "./data/" for i in os.listdir(image_path): image = cv2.imread(image_path + i, 0) image = image / 255.0 image = np.expand_dims(image, axis=0).astype(np.float32) np.save(save_path + i.split(".")[0] + ".npy", image)
Enter vim4_npu_sdk/demo
and modify convert_adla.sh
as follows.
#!/bin/bash ACUITY_PATH=../bin/ #ACUITY_PATH=../python/tvm/ adla_convert=${ACUITY_PATH}adla_convert if [ ! -e "$adla_convert" ]; then adla_convert=${ACUITY_PATH}adla_convert.py fi $adla_convert --model-type onnx \ --model ./model_source/densenet_ctc/densenet_ctc.onnx \ --inputs "the_input" \ --input-shapes "1,32,280" \ --dtypes "float32" \ --quantize-dtype int8 --outdir onnx_output \ --channel-mean-value "0,0,0,255" \ --source-file ./densenet_ctc_dataset.txt \ --iterations 500 \ --disable-per-channel False \ --inference-input-type float32 \ --inference-output-type float32 \ --batch-size 1 --target-platform PRODUCT_PID0XA003
Please prepare about 500 pictures for quantification. If the pictures size is smaller than model input size, please resize pictures to input size before quantification.
Run convert_adla.sh
to generate the VIM4 model. The converted model is xxx.adla
in onnx_output
.
$ bash convert_adla.sh
Clone the source code khadas/vim4_npu_applications.
$ git clone https://github.com/khadas/vim4_npu_applications
If your kernel version is 5.4 or earlier, please use tag ddk-1.7.5.5
. Tag ddk-2.3.6.7
is for 5.15.
$ sudo apt update $ sudo apt install libopencv-dev python3-opencv cmake
Put densenet_ctc_int16.adla
in vim4_npu_applications/densenet_ctc/data/
.
# Compile $ cd vim4_npu_applications/densenet_ctc $ mkdir build $ cd build $ cmake .. $ make # Run $ sudo ./densenet_ctc -m ../data/densenet_ctc_int16.adla -p ../data/KhadasTeam.png
If your densenet_ctc
- DenseNet-CTC model classes are not the same, please change data/class_str.txt
and the OBJ_CLASS_NUM
in include/postprocess.h
.