~~tag> NPU YOLO OpenCV VIM4 ~~

====== YOLOv8n OpenCV VIM4 Demo - 2 ======

{{indexmenu_n>2}}
===== Get Source Code =====

Download YOLOv8 official code [[gh>ultralytics/ultralytics]]

```shell
$ git clone https://github.com/ultralytics/ultralytics
```

Refer ''README.md'' to train a YOLOv8n model. My version ''torch==1.10.1'' and ''ultralytics==8.0.86''.

===== Convert Model =====

==== Build virtual environment ====

Follow Docker official documentation to install Docker: [[https://docs.docker.com/engine/install/ubuntu/|Install Docker Engine on Ubuntu]].

Follow the script below to get Docker image:

```shell
docker pull numbqq/npu-vim4
```

===== Get Convert Tool =====

```shell
$ git lfs install
$ git lfs clone https://gitlab.com/khadas/vim4_npu_sdk.git
$ cd vim4_npu_sdk
$ ls
adla-toolkit-binary  adla-toolkit-binary-1.2.0.9  convert-in-docker.sh  Dockerfile  docs  README.md
```

  * ''adla-toolkit-binary/docs'' - SDK documentations
  * ''adla-toolkit-binary/bin'' - SDK tools required for model conversion
  * ''adla-toolkit-binary/demo'' - Conversion examples

==== Get Model Conversion Tools ====

Get source [[gl>khadas/vim4_npu_sdk]].

```shell
$ git clone https://gitlab.com/khadas/vim4_npu_sdk
```

==== Convert ====

After training model, modify ''ultralytics/ultralytics/nn/modules/head.py'' as follows.

```diff head.py
diff --git a/ultralytics/nn/modules/head.py b/ultralytics/nn/modules/head.py
index 0b02eb3..0a6e43a 100644
--- a/ultralytics/nn/modules/head.py
+++ b/ultralytics/nn/modules/head.py
@@ -42,6 +42,9 @@ class Detect(nn.Module):
 
     def forward(self, x):
         """Concatenates and returns predicted bounding boxes and class probabilities."""
+        if torch.onnx.is_in_onnx_export():
+            return self.forward_export(x)
+
         shape = x[0].shape  # BCHW
         for i in range(self.nl):
             x[i] = torch.cat((self.cv2[i](x[i]), self.cv3[i](x[i])), 1)
@@ -80,6 +83,15 @@ class Detect(nn.Module):
             a[-1].bias.data[:] = 1.0  # box
             b[-1].bias.data[:m.nc] = math.log(5 / m.nc / (640 / s) ** 2)  # cls (.01 objects, 80 classes, 640 img)
 
+    def forward_export(self, x):
+        results = []
+        for i in range(self.nl):
+            dfl = self.cv2[i](x[i]).contiguous()
+            cls = self.cv3[i](x[i]).contiguous()
+            results.append(torch.cat([cls, dfl], 1).permute(0, 2, 3, 1))
+            # results.append(torch.cat([cls, dfl], 1))
+        return tuple(results)
+
```

<WRAP important>
If you pip-installed ultralytics package, you should modify in package.
</WRAP>

Create a python file written as follows to export ONNX model.

```python export.py
from ultralytics import YOLO
model = YOLO("./runs/detect/train/weights/best.pt")
results = model.export(format="onnx")
```

```shell
$ python export.py
```

<WRAP important>
Use [[https://netron.app/ | Netron]] to check your model output like this. If not, please check your ''head.py''.

{{:products:sbc:vim4:npu:demos:yolov8n-vim4-output.png?600|}}
</WRAP>

Enter ''vim4_npu_sdk/demo'' and modify ''convert_adla.sh'' as follows.

```sh convert_adla.sh
#!/bin/bash
  
ACUITY_PATH=../bin/
#ACUITY_PATH=../python/tvm/
adla_convert=${ACUITY_PATH}adla_convert


if [ ! -e "$adla_convert" ]; then
    adla_convert=${ACUITY_PATH}adla_convert.py
fi

$adla_convert --model-type onnx \
        --model ./model_source/yolov8n/yolov8n.onnx \
        --inputs "images" \
        --input-shapes  "3,640,640"  \
        --dtypes "float32" \
        --quantize-dtype int16 --outdir onnx_output  \
        --channel-mean-value "0,0,0,255"  \
        --source-file dataset.txt  \
        --batch-size 1 --target-platform PRODUCT_PID0XA003
```

Run ''convert_adla.sh'' to generate VIM4 model. The converted model is ''xxx.adla'' in ''onnx_output''.

```shell
$ bash convert_adla.sh
```

===== Run NPU =====

==== Get source code ====

Clone the source code from our [[gh>khadas/vim4_npu_applications]].

```shell
$ git clone https://github.com/khadas/vim4_npu_applications
```

==== Install dependencies ====

```shell
$ sudo apt update
$ sudo apt install libopencv-dev python3-opencv cmake
```

==== Compile and run ====

=== Picture input demo ===

Put ''yolov8n_int8.adla'' in ''vim4_npu_applications/yolov8n/data/''.

```shell
# Compile
$ cd vim4_npu_applications/yolov8n
$ mkdir build
$ cd build
$ cmake ..
$ make

# Run
$ sudo ./yolov8n -m ../data/yolov8n_int8.adla -p ../data/horses.jpg
```

=== Camera input demo ===

Put ''yolov8n_int8.adla'' in ''vim4_npu_applications/yolov8n_cap/data/''.

```shell
# Compile
$ cd vim4_npu_applications/yolov8n_cap
$ mkdir build
$ cd build
$ cmake ..
$ make

# Run
$ sudo ./yolov8n_cap -m ../data/yolov8n_int8.adla -d 0 -w 1920 -h 1080
```

''0'' is camera device index.

<WRAP tip >
If your **YOLOv8n** model classes are not the same as **COCO**, please change ''data/coco_80_labels_list.txt'' and the ''OBJ_CLASS_NUM'' in ''include/postprocess.h''.
</WRAP>