bubbliiiing/retinaface-pytorch

$ git clone https://github.com/bubbliiiing/retinaface-pytorch

Before training, modify retinaface-pytorch/utils/utils.py as follows.

diff --git a/utils/utils.py b/utils/utils.py
index 87bb528..4a22f2a 100644
--- a/utils/utils.py
+++ b/utils/utils.py
@@ -25,5 +25,6 @@ def get_lr(optimizer):
         return param_group['lr']
 
 def preprocess_input(image):
-    image -= np.array((104, 117, 123),np.float32)
+    image = image / 255.0
     return image

Follow Docker official documentation to install Docker: Install Docker Engine on Ubuntu.

Follow the script below to get Docker image:

docker pull numbqq/npu-vim4

Download Tool from khadas/vim4_npu_sdk.

$ git lfs install
$ git lfs clone https://gitlab.com/khadas/vim4_npu_sdk.git
$ cd vim4_npu_sdk
$ ls
adla-toolkit-binary  adla-toolkit-binary-3.1.7.4  convert-in-docker.sh  Dockerfile  docs  README.md

• adla-toolkit-binary/docs - SDK documentations
• adla-toolkit-binary/bin - SDK tools required for model conversion
• adla-toolkit-binary/demo - Conversion examples

After training the model, we should convert the PyTorch model into an ONNX model. Create the Python conversion script as follows and run.

export.py

import torch
import numpy as np
from nets.retinaface import RetinaFace
from utils.config import cfg_mnet, cfg_re50
 
model_path = "logs/Epoch150-Total_Loss6.2802.pth"
net = RetinaFace(cfg=cfg_mnet, mode='eval').eval()
device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
net.load_state_dict(torch.load(model_path, map_location=device))
 
img = torch.zeros(1, 3, 640, 640)
torch.onnx.export(net, img, "./retinaface.onnx", verbose=False, opset_version=12, input_names=['images'])

Enter vim4_npu_sdk/demo and modify convert_adla.sh as follows.

convert_adla.sh

#!/bin/bash
 
ACUITY_PATH=../bin/
#ACUITY_PATH=../python/tvm/
adla_convert=${ACUITY_PATH}adla_convert
 
 
if [ ! -e "$adla_convert" ]; then
    adla_convert=${ACUITY_PATH}adla_convert.py
fi
 
$adla_convert --model-type onnx \
        --model ./model_source/retinaface/retinaface.onnx \
        --inputs "images" \
        --input-shapes  "3,640,640"  \
        --dtypes "float32" \
        --inference-input-type float32 \
	--inference-output-type float32 \
        --quantize-dtype int8 --outdir onnx_output  \
        --channel-mean-value "0,0,0,255"  \
        --source-file ./retinaface_dataset.txt  \
        --iterations 500 \
        --disable-per-channel False \
        --batch-size 1 --target-platform PRODUCT_PID0XA003

Run convert_adla.sh to generate the VIM4 model. The converted model is xxx.adla in onnx_output.

$ bash convert_adla.sh

Clone the source code khadas/vim4_npu_applications.

$ git clone https://github.com/khadas/vim4_npu_applications

$ sudo apt update
$ sudo apt install libopencv-dev python3-opencv cmake

Put retinaface_int8.adla in vim4_npu_applications/retinaface/data/.

# Compile
$ cd vim4_npu_applications/retinaface
$ mkdir build
$ cd build
$ cmake ..
$ make
 
# Run
$ ./retinaface -m ../data/retinaface_int8.adla -p ../data/timg.jpg

Put retinaface_int8.adla in vim4_npu_applications/retinaface_cap/data/.

$ cd vim4_npu_applications/face_recognition_cap
$ mkdir build
$ cd build
$ cmake ..
$ make

MIPI Camera

$ ./retinaface_cap -m ../data/retinaface_int8.adla -t mipi

USB Camera

$ cd build
$ ./retinaface_cap -m ../data/retinaface_int8.adla -t usb -d 0

TIP: Replace 0 as the number for your camera device. Such as /dev/video5, it should be -d 5.