Differences

This shows you the differences between two versions of the page.

--- products:sbc:vim3:npu:npu-app [2023/07/05 06:30]
louis
+++ products:sbc:vim3:npu:npu-app [2024/01/22 20:31]
louis
@@ Line 1: / Line 1: @@
+~~tag> VIM3 VIM3L Amlogic NPU OpenCV~~
 ====== Application Source Code ======
+Guide to compiling the NPU application on the VIM3/3L.
-===== Introduction =====
-This document mainly describes how to compile executable files from application source code.
-<WRAP important >
-Only support local compile on VIM3/3L
-Only support OpenCV4
-</WRAP>
 ===== Install OpenCV4 =====
+Install the necessary library packages to build the OpenCV4 application.
 ```shell
 $ sudo apt update
@@ Line 18: / Line 12: @@
 ===== Get Source Code =====
+Get the source code: [[gh>khadas/aml_npu_app]]
 ```shell
-$ mkdir{workspace} && cd {workspace}
 $ git clone https://github.com/khadas/aml_npu_app
 ```
 ===== Source Description =====
+==== Base directory ====
+Base directory of the cloned repository.
+```shell
+$ cd aml_npu_app
+$ ls
+DDK_6.3.2  DDK_6.3.2.3  DDK_6.3.2.5  DDK_6.3.3.4  DDK_6.4.0.3  DDK_6.4.3  detect_library  LICENSE  NN_SLT
+```
+^ File ^ Description ^
+| DDK_xx | Library source code, xx is the version code |
+| detect_library | Application layer source code. the detect_library/model_code \\ is linked to the latest version of the DDK library |
+| NN_SLT | Separate source code directory for DnCnn model (no longer maintained). |
+| LICENSE | LICENSE file. |
+                                                                                                |
+==== Detect library ====
+Directory with various model specific application code.
+```shell
+$ cd aml_npu_app/detect_library
+$ ls
+densenet_ctc_demo_picture  inception  model_code  retinaface_demo_x11_usb  sample_demo_fb  sample_demo_x11  source_code  yolo_demo_fb_mipi  yolo_demo_fb_usb
+yolo_demo_x11_mipi  yolo_demo_x11_usb  yolov7_tiny_demo_x11_usb  yolov8n_demo_x11_usb
+```
-==== Directory structure description ====
+^ File            ^ Description ^
+| **model_code**  | Load model and complete pre-processing and post-processing. |
+| **source_code** | Provide interfaces for demo to call model_code. |
+| **Other folders** | Application source code |
+==== Model code ====
+Directory containing all the detection library code.
 ```shell
-$ cd {workspace}/aml_npu_app
+$ cd aml_npu_app/detect_library/model_code
 $ ls
-DDK_6.3.2  DDK_6.3.2.3  DDK_6.3.2.5  DDK_6.3.3.4  DDK_6.4.0.3  DDK_6.4.3  detect_library  LICENSE  NN_SLT
+densenet_ctc  detect_mtcnn  detect_retinaface  detect_yoloface  detect_yolo_v2  detect_yolo_v3  detect_yolo_v3_tiny  detect_yolo_v4  detect_yolo_v7_tiny
+detect_yolov8n  facenet
 ```
-  * **DDK_xxx** - Different versions of library source code
+Except for ''detect_mtcnn'', each directory represents a different model, and each model will be compiled into a library.
-  * **detect_library** - Application layer source code. detect_library/model_code link to the latest version of DDK
-  * **NN_SLT** - Separate source code directory for DnCnn model (no longer maintained)
-  * **LICENSE** - LICENSE file
-==== Library Description ====
+^ File ^ Description ^
+| **densenet_ctc** | Character recognition, used to identify characters. The input is only a picture. |
+| **detect_retinaface** | Detect faces and the keypoints of faces. |
+| **detect_yoloface** | yoloface model, used to detect faces without keypoints. |
+| **detect_yolo_v2** | yolov2 model, for object detection. |
+| **detect_yolo_v3** | yolov3 model, for object detection. |
+| **detect_yolo_v3_tiny** | yolov3-tiny model, for object detection. |
+| **detect_yolo_v4** | yolov4 model, used to detect faces. |
+| **detect_yolo_v7_tiny** | yolov7_tiny model, for object detection. |
+| **detect_yolov8n** | yolov8n model, for object detection. The latest yolo model. |
+| **facenet** | Deprecated - No longer in use. |
+Take ''detect_yolo_v3'' as an example to illustrate the structure of each directory.
 ```shell
-$ cd {workspace}/aml_npu_app/detect_library
+$ cd aml_npu_app/detect_library/model_code/detect_yolo_v3
 $ ls
-densenet_ctc_demo_picture  inception  model_code  retinaface_demo_x11_usb  sample_demo_fb  sample_demo_x11  source_code  yolo_demo_fb_mipi  yolo_demo_fb_usb  yolo_demo_x11_mipi
+build_vx.sh  include  Makefile  makefile.linux  makefile.linux.def  vnn_yolov3.c  yolo_v3.c  yolov3_process.c
-yolo_demo_x11_usb  yolov7_tiny_demo_x11_usb  yolov8n_demo_x11_usb
 ```
-  * **model_code** - Load model and complete pre and post processing.
+^ File ^ Description ^
-  * **source_code** - Provide interfaces for demo to call model_code.
+| **build_vx.sh** | Compile script. |
-  * **Other folders** - Load input and visualize the results.
+| **include** | The corresponding header files and all definitions will be placed in this directory. |
+| **Makefile** | Makefile file. |
+| **makefile.linux** | Make environment configuration file. |
+| **vnn_yolov3.c** | SDK The converted model processing file is mainly used to interface with the nb file. |
+| **yolo_v3.c** | Specify the called nb file, and define all the interfaces of the model call. |
+| **yolov3_process.c** | Mainly defines the pre-processing and post-processing of the model. |
-Enter the directory of the library model_code, which is the directory of the DDK.
+==== Source code ====
+Directory containing the application library source code.
 ```shell
-$ cd {workspace}/aml_npu_app/detect_library/model_code
+$ cd aml_npu_app/detect_library/source_code
 $ ls
-densenet_ctc  detect_mtcnn  detect_retinaface  detect_yoloface  detect_yolo_v2  detect_yolo_v3  detect_yolo_v3_tiny  detect_yolo_v4  detect_yolo_v7_tiny  detect_yolov8n  facenet
+build_vx.sh  detect.c  detect_log.c  include  Makefile  makefile.linux  makefile.linux.def
 ```
-Except for ''detect_mtcnn'', each directory represents a different model, and each model will be compiled into a library.
+^ File ^ Description ^
+| **build_vx.sh** | Compile script. |
+| **detect.c** | Provide interface for detect and call model_code. |
+| **detect_log.c** | About log level. |
+| **include** | The corresponding header files and all definitions will be placed in this directory. |
+| **Makefile** | Makefile file. |
+| **makefile.linux** | Make environment configuration file. |
-  * **densenet_ctc** - Character recognition, used to identify characters. The input is only picture.
-  * **detect_retinaface** - Detect faces and the keypoints of faces.
-  * **detect_yoloface** - yoloface model, used to detect faces without keypoint.
-  * **detect_yolo_v2** - yolov2 model, for object detection.
-  * **detect_yolo_v3** - yolov3 model, for object detection.
-  * **detect_yolo_v3_tiny** - yolov3-tiny model, for object detection.
-  * **detect_yolo_v4** - yolov4 model, used to detect faces.
-  * **detect_yolo_v7_tiny** - yolov7_tiny model, for object detection.
-  * **detect_yolov8n** - yolov8n model, for object detection. The latest yolo model.
-  * **facenet** - Has been deprecated.
-Take ''detect_yolo_v3'' as an example to illustrate the structure of each directory.
+Enter the directory of the ''yolo_demo_x11_usb'':
 ```shell
-$ cd {workspace}/aml_npu_app/detect_library/model_code/detect_yolo_v3
+$ cd aml_npu_app/detect_library/yolo_demo_x11_usb
 $ ls
-build_vx.sh  include  Makefile  makefile.linux  vnn_yolov3.c  yolo_v3.c  yolov3_process.c
+build_vx.sh  detect.h  main.cpp  makefile.linux  makefile.linux.def  makefile.target_name  nn_detect_common.h  nn_detect.h  nn_detect_utils.h
 ```
-Main content description:
+^ File ^ Description ^
+| **build_vx.sh** | Compile script. |
+| **main.cpp** | Main function. |
+| **detect.h**, **nn_detect_common.h**, **nn_detect.h**, **nn_detect_utils.h** | The corresponding header files. |
+| **makefile.linux** | Make environment configuration file. |
+| **makefile.target_name** | The name of the compiled executable. |
-  * **build_vx.sh** - Compile script.
+===== Compile the application=====
-  * **include** - The corresponding header files and all definitions will be placed in this directory.
-  * **Makefile** - Makefile file.
-  * **makefile.linux** - Make environment configuration file.
-  * **vnn_yolov3.c** - SDK The converted model processing file is mainly used to interface with the nb file.
-  * **yolo_v3.c** - Specify the called nb file, and define all the interfaces of the model call.
-  * **yolov3_process.c** - Mainly defines the pre-processing and post-processing of the model.
-Enter the directory of the library source_code.
+. Compile the model you want to use from the ''model_code'' directory. Here we shall take ''detect_yolo_v3'' as an example.
 ```shell
-$ cd {workspace}/aml_npu_app/detect_library/model_code
+$ cd aml_npu_app/detect_library/model_code/detect_yolo_v3
 $ ls
-build_vx.sh  detect.c  detect_log.c  include  Makefile  makefile.linux
+build_vx.sh  include  Makefile  makefile.linux  makefile.linux.def  vnn_yolov3.c  yolo_v3.c  yolov3_process.c
+$ bash build_vx.sh
+$ ls
+bin_r  build_vx.sh  include  linux_build_sample.log  makefile.linux  makefile.linux.def  makefile.target_name  vnn_yolov3.c  yolo_v3.c  yolov3_process.c
 ```
-Main content description:
+. The compiled library ''libnn_yolo_v3.so'' is generated in ''bin_r'' folder. Add it to the library path variable.
-  * **build_vx.sh** - Compile script.
+```shell
-  * **detect.c** - Provide interface for detect and call model_code.
+$ export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$(pwd)/bin_r/libnn_yolo_v3.so
-  * **detect_log.c** - About log level.
+```
-  * **include** - The corresponding header files and all definitions will be placed in this directory.
-  * **Makefile** - Makefile file.
-  * **makefile.linux** - Make environment configuration file.
+. You will also need to build the contents of the ''source_code'' directory.
+```shell
+$ cd aml_npu_app/detect_library/source_code
+$ ls
+build_vx.sh  detect.c  detect_log.c  include  Makefile  makefile.linux  makefile.linux.def
+$ bash build_vx.sh
+$ ls
+bin_r  build_vx.sh  detect.c  detect_log.c  include  Makefile  makefile.linux  makefile.linux.def
+```
+. The compiled library ''libnn_yolo_v3.so'' is generated in bin_r folder. Add it to the library path variable.
+```shell
+$ export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$(pwd)/bin_r/libnn_detect.so
+```
+. Now, Build the example application.
+```shell
+$ cd aml_npu_app/detect_library/yolo_demo_x11_usb
+$ ls
+build_vx.sh  detect.h  main.cpp  makefile.linux  makefile.linux.def  makefile.target_name  nn_detect_common.h  nn_detect.h  nn_detect_utils.h
+$ bash build_vx.sh
+$ ls
+bin_r_cv4  build_vx.sh  detect.h  linux_build_sample.log  main.cpp  makefile.linux  makefile.linux.def  makefile.target_name  nn_detect_common.h  nn_detect.h  nn_detect_utils.h
+```
+<WRAP important >
+If you use kernel are using version 5.15 or above, please modify ''makefile.linux'' as follows before running ''build_vx.sh''.
+</WRAP>
+```diff
+-#LIBS +=-L$(LIB_DIR) -lstdc++
+-LIBS += -lvpcodec -lamcodec -lamadec -lamvdec -lamavutils -lrt -lpthread -lge2d -lion
+```
+. Create a folder named ''nn_data'' under ''bin_r_cv4'' and place the ''yolov3_88.nb'' model file in it.
+```shell
+$ cd bin_r_cv4
+$ mkdir nn_data
+$ cp -r xx/xx/yolov3_88.nb nn_data/
+$ ls
+detect_demo_x11_usb  main.o  nn_data
+```
+===== Run the application =====
+Now you can run inference on captured video data using the model.
+```shell
+$ ./detect_demo_x11_usb -m 2 -d /dev/video1
+```
+==== Application setup parameters ====
+=== Parameter to select the detection model ===
+The parameter ''-m'' is for selecting the inference model, Here are the all the available models and their respective serial numbers.
+^ Index ^ File ^
+| **0** | detect_yoloface |
+| **1** | detect_yolo_v2 |
+| **2** | detect_yolo_v3 |
+| **3** | detect_yolo_v3_tiny |
+| **4** | detect_yolo_v4 |
+| **13** | detect_yolo_v7_tiny |
+| **14** | detect_yolov8n |
+| **15** | densenet_ctc |
+| **16** | detect_retinaface |
+For the above example, we used the serial number ''2'' which corresponds to the ''yolov3'' model.
+=== Parameter to set the input as picture ===
+If you use the model that takes input as a picture, change ''-d'' to ''-p''. Here is an example with ''densenet_ctc''.
+```shell
+$ ./densenet_ctc_picture -m 15 -p ../KhadasTeam.png
+```

Khadas Docs

User Tools

Site Tools

Differences

Page Tools