~~tag> NPU RetinaFace VIM4 PyTorch~~

**Doc for version ddk-3.4.7.7**

====== RetinaFace PyTorch VIM4 Demo - 5 ======

{{indexmenu_n>5}}
===== Get source code =====

[[gh>bubbliiiing/retinaface-pytorch]]

```shell
$ git clone https://github.com/bubbliiiing/retinaface-pytorch
```

Before training, modify ''retinaface-pytorch/utils/utils.py'' as follows.

```diff
diff --git a/utils/utils.py b/utils/utils.py
index 87bb528..4a22f2a 100644
--- a/utils/utils.py
+++ b/utils/utils.py
@@ -25,5 +25,6 @@ def get_lr(optimizer):
         return param_group['lr']
 
 def preprocess_input(image):
-    image -= np.array((104, 117, 123),np.float32)
+    image = image / 255.0
     return image
```

===== Convert the model =====

==== Build virtual environment ====

Follow Docker official documentation to install Docker: [[https://docs.docker.com/engine/install/ubuntu/|Install Docker Engine on Ubuntu]].

Follow the script below to get Docker image:

```shell
docker pull numbqq/npu-vim4
```

==== Get Convert Tool ====

Download Tool from [[gl>khadas/vim4_npu_sdk]].

```shell
$ git lfs install
$ git lfs clone https://gitlab.com/khadas/vim4_npu_sdk.git
$ cd vim4_npu_sdk
$ ls
adla-toolkit-binary  adla-toolkit-binary-3.1.7.4  convert-in-docker.sh  Dockerfile  docs  README.md
```

  * ''adla-toolkit-binary/docs'' - SDK documentations
  * ''adla-toolkit-binary/bin'' - SDK tools required for model conversion
  * ''adla-toolkit-binary/demo'' - Conversion examples

<WRAP important>
If your kernel is older than 241129, please use version before tag ddk-3.4.7.7.
</WRAP>

==== Convert ====

After training the model, we should convert the PyTorch model into an ONNX model. Create the Python conversion script as follows and run.

```python export.py
import torch
import numpy as np
from nets.retinaface import RetinaFace
from utils.config import cfg_mnet, cfg_re50

model_path = "logs/Epoch150-Total_Loss6.2802.pth"
net = RetinaFace(cfg=cfg_mnet, mode='eval').eval()
device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
net.load_state_dict(torch.load(model_path, map_location=device))

img = torch.zeros(1, 3, 640, 640)
torch.onnx.export(net, img, "./retinaface.onnx", verbose=False, opset_version=12, input_names=['images'])
```

Enter ''vim4_npu_sdk/demo'' and modify ''convert_adla.sh'' as follows.

```bash convert_adla.sh
#!/bin/bash
  
ACUITY_PATH=../bin/
#ACUITY_PATH=../python/tvm/
adla_convert=${ACUITY_PATH}adla_convert


if [ ! -e "$adla_convert" ]; then
    adla_convert=${ACUITY_PATH}adla_convert.py
fi

$adla_convert --model-type onnx \
        --model ./model_source/retinaface/retinaface.onnx \
        --inputs "images" \
        --input-shapes  "3,640,640"  \
        --dtypes "float32" \
        --inference-input-type float32 \
	--inference-output-type float32 \
        --quantize-dtype int8 --outdir onnx_output  \
        --channel-mean-value "0,0,0,255"  \
        --source-file ./retinaface_dataset.txt  \
        --iterations 500 \
        --disable-per-channel False \
        --batch-size 1 --target-platform PRODUCT_PID0XA003
```

Run ''convert_adla.sh'' to generate the VIM4 model. The converted model is ''xxx.adla'' in ''onnx_output''.

```shell
$ bash convert_adla.sh
```

===== Run inference on the NPU =====

==== Get source code ====

Clone the source code [[gh>khadas/vim4_npu_applications]].

```shell
$ git clone https://github.com/khadas/vim4_npu_applications
```

<WRAP important>
If your kernel is older than 241129, please use version before tag ddk-3.4.7.7.
</WRAP>

==== Install dependencies ====

```shell
$ sudo apt update
$ sudo apt install libopencv-dev python3-opencv cmake
```

==== Compile and run ====

=== Picture input demo ===

Put ''retinaface_int8.adla'' in ''vim4_npu_applications/retinaface/data/''.

```shell
# Compile
$ cd vim4_npu_applications/retinaface
$ mkdir build
$ cd build
$ cmake ..
$ make

# Run
$ ./retinaface -m ../data/retinaface_int8.adla -p ../data/timg.jpg
```

{{:products:sbc:vim4:npu:demos:retinaface-demo-output.webp?800|}}

=== Camera input demo ===

Put ''retinaface_int8.adla'' in ''vim4_npu_applications/retinaface_cap/data/''.

== Compile ==

```shell
$ cd vim4_npu_applications/face_recognition_cap
$ mkdir build
$ cd build
$ cmake ..
$ make
```

== Run==

**MIPI Camera**

```
$ ./retinaface_cap -m ../data/retinaface_int8.adla -t mipi
```

**USB Camera**
```
$ cd build
$ ./retinaface_cap -m ../data/retinaface_int8.adla -t usb -d 0
```

**TIP**: Replace 0 as the number for your camera device. Such as ''/dev/video5'', it should be ''-d 5''.