Khadas Docs

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
~~tag> VIM3 VIM3L Amlogic NPU OpenCV~~

====== Application Source Code ======
Guide to compiling the NPU application on the VIM3/3L.

===== Install OpenCV4 =====
Install the necessary library packages to build the OpenCV4 application.
```shell
$ sudo apt update
$ sudo apt install libopencv-dev python3-opencv
```

===== Get Source Code =====
Get the source code: [[gh>khadas/aml_npu_app]]
```shell
$ git clone https://github.com/khadas/aml_npu_app
```

===== Source Description =====
==== Base directory ====
Base directory of the cloned repository.
```shell
$ cd aml_npu_app
$ ls
DDK_6.3.2  DDK_6.3.2.3  DDK_6.3.2.5  DDK_6.3.3.4  DDK_6.4.0.3  DDK_6.4.3  detect_library  LICENSE  NN_SLT
```
^ File ^ Description ^
| DDK_xx | Library source code, xx is the version code |
| detect_library | Application layer source code. the detect_library/model_code \\ is linked to the latest version of the DDK library |
| NN_SLT | Separate source code directory for DnCnn model (no longer maintained). |
| LICENSE | LICENSE file. |
                                                                                                |

==== Detect library ====
Directory with various model specific application code.
```shell
$ cd aml_npu_app/detect_library
$ ls
densenet_ctc_demo_picture  inception  model_code  retinaface_demo_x11_usb  sample_demo_fb  sample_demo_x11  source_code  yolo_demo_fb_mipi  yolo_demo_fb_usb
yolo_demo_x11_mipi  yolo_demo_x11_usb  yolov7_tiny_demo_x11_usb  yolov8n_demo_x11_usb
```

^ File            ^ Description ^
| **model_code**  | Load model and complete pre-processing and post-processing. |
| **source_code** | Provide interfaces for demo to call model_code. |
| **Other folders** | Application source code |

==== Model code ====
Directory containing all the detection library code.

```shell
$ cd aml_npu_app/detect_library/model_code
$ ls
densenet_ctc  detect_mtcnn  detect_retinaface  detect_yoloface  detect_yolo_v2  detect_yolo_v3  detect_yolo_v3_tiny  detect_yolo_v4  detect_yolo_v7_tiny
detect_yolov8n  facenet
```

Except for ''detect_mtcnn'', each directory represents a different model, and each model will be compiled into a library.

^ File ^ Description ^
| **densenet_ctc** | Character recognition, used to identify characters. The input is only a picture. |
| **detect_retinaface** | Detect faces and the keypoints of faces. |
| **detect_yoloface** | yoloface model, used to detect faces without keypoints. |
| **detect_yolo_v2** | yolov2 model, for object detection. |
| **detect_yolo_v3** | yolov3 model, for object detection. |
| **detect_yolo_v3_tiny** | yolov3-tiny model, for object detection. |
| **detect_yolo_v4** | yolov4 model, used to detect faces. |
| **detect_yolo_v7_tiny** | yolov7_tiny model, for object detection. |
| **detect_yolov8n** | yolov8n model, for object detection. The latest yolo model. |
| **facenet** | Deprecated - No longer in use. |


Take ''detect_yolo_v3'' as an example to illustrate the structure of each directory.

```shell
$ cd aml_npu_app/detect_library/model_code/detect_yolo_v3
$ ls
build_vx.sh  include  Makefile  makefile.linux  makefile.linux.def  vnn_yolov3.c  yolo_v3.c  yolov3_process.c
```

^ File ^ Description ^
| **build_vx.sh** | Compile script. |
| **include** | The corresponding header files and all definitions will be placed in this directory. |
| **Makefile** | Makefile file. |
| **makefile.linux** | Make environment configuration file. |
| **vnn_yolov3.c** | SDK The converted model processing file is mainly used to interface with the nb file. |
| **yolo_v3.c** | Specify the called nb file, and define all the interfaces of the model call. |
| **yolov3_process.c** | Mainly defines the pre-processing and post-processing of the model. |


==== Source code ====
Directory containing the application library source code. 

```shell
$ cd aml_npu_app/detect_library/source_code
$ ls
build_vx.sh  detect.c  detect_log.c  include  Makefile  makefile.linux  makefile.linux.def
```

^ File ^ Description ^
| **build_vx.sh** | Compile script. |
| **detect.c** | Provide interface for detect and call model_code. |
| **detect_log.c** | About log level. |
| **include** | The corresponding header files and all definitions will be placed in this directory. |
| **Makefile** | Makefile file. |
| **makefile.linux** | Make environment configuration file. |


Enter the directory of the ''yolo_demo_x11_usb'':

```shell
$ cd aml_npu_app/detect_library/yolo_demo_x11_usb
$ ls
build_vx.sh  detect.h  main.cpp  makefile.linux  makefile.linux.def  makefile.target_name  nn_detect_common.h  nn_detect.h  nn_detect_utils.h
```

^ File ^ Description ^
| **build_vx.sh** | Compile script. |
| **main.cpp** | Main function. |
| **detect.h**, **nn_detect_common.h**, **nn_detect.h**, **nn_detect_utils.h** | The corresponding header files. |
| **makefile.linux** | Make environment configuration file. |
| **makefile.target_name** | The name of the compiled executable. |

===== Compile the application=====

1. Compile the model you want to use from the ''model_code'' directory. Here we shall take ''detect_yolo_v3'' as an example.

```shell
$ cd aml_npu_app/detect_library/model_code/detect_yolo_v3
$ ls
build_vx.sh  include  Makefile  makefile.linux  makefile.linux.def  vnn_yolov3.c  yolo_v3.c  yolov3_process.c
$ bash build_vx.sh
$ ls
bin_r  build_vx.sh  include  linux_build_sample.log  makefile.linux  makefile.linux.def  makefile.target_name  vnn_yolov3.c  yolo_v3.c  yolov3_process.c
```

2. The compiled library ''libnn_yolo_v3.so'' is generated in ''bin_r'' folder. Add it to the library path variable.

```shell
$ export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$(pwd)/bin_r/libnn_yolo_v3.so
```

3. You will also need to build the contents of the ''source_code'' directory.
```shell
$ cd aml_npu_app/detect_library/source_code
$ ls
build_vx.sh  detect.c  detect_log.c  include  Makefile  makefile.linux  makefile.linux.def
$ bash build_vx.sh
$ ls
bin_r  build_vx.sh  detect.c  detect_log.c  include  Makefile  makefile.linux  makefile.linux.def
```

4. The compiled library ''libnn_yolo_v3.so'' is generated in bin_r folder. Add it to the library path variable.

```shell
$ export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$(pwd)/bin_r/libnn_detect.so
```

5. Now, Build the example application.
```shell
$ cd aml_npu_app/detect_library/yolo_demo_x11_usb
$ ls
build_vx.sh  detect.h  main.cpp  makefile.linux  makefile.linux.def  makefile.target_name  nn_detect_common.h  nn_detect.h  nn_detect_utils.h
$ bash build_vx.sh
$ ls
bin_r_cv4  build_vx.sh  detect.h  linux_build_sample.log  main.cpp  makefile.linux  makefile.linux.def  makefile.target_name  nn_detect_common.h  nn_detect.h  nn_detect_utils.h
```

<WRAP important >
If you use kernel are using version 5.15 or above, please remove the red lines in ''makefile.linux'' before running ''build_vx.sh''.
</WRAP>

```diff
LIBS += -L$(VIVANTE_SDK_LIB) -lOpenVX -lOpenVXU -lGAL -lovxlib -lArchModelSw -lNNArchPerf

LIBS += -L../source_code/bin_r -lnn_detect

-#LIBS +=-L$(LIB_DIR) -lstdc++
-LIBS += -lvpcodec -lamcodec -lamadec -lamvdec -lamavutils -lrt -lpthread -lge2d -lion

#############################################################################
# Macros.
PROGRAM = 1
CUR_SOURCE = ${wildcard *.c}
#############################################################################
```

6. Create a folder named ''nn_data'' under ''bin_r_cv4'' and place the ''yolov3_88.nb'' model file in it.
```shell
$ cd bin_r_cv4
$ mkdir nn_data
$ cp -r xx/xx/yolov3_88.nb nn_data/
$ ls
detect_demo_x11_usb  main.o  nn_data
```
===== Run the application =====

Now you can run inference on captured video data using the model.
```shell
$ ./detect_demo_x11_usb -m 2 -d /dev/video1
```
==== Application setup parameters ====
=== Parameter to select the detection model ===

The parameter ''-m'' is for selecting the inference model, Here are the all the available models and their respective serial numbers.

^ Index ^ File ^
| **0** | detect_yoloface |
| **1** | detect_yolo_v2 |
| **2** | detect_yolo_v3 |
| **3** | detect_yolo_v3_tiny |
| **4** | detect_yolo_v4 |
| **13** | detect_yolo_v7_tiny |
| **14** | detect_yolov8n |
| **15** | densenet_ctc |
| **16** | detect_retinaface |


For the above example, we used the serial number ''2'' which corresponds to the ''yolov3'' model.

=== Parameter to set the input as picture ===

If you use the model that takes input as a picture, change ''-d'' to ''-p''. Here is an example with ''densenet_ctc''.
```shell
$ ./densenet_ctc_picture -m 15 -p ../KhadasTeam.png
```
Khadas Docs

User Tools

Site Tools

Page Tools
