Khadas Docs

Amazing Khadas, always amazes you!

User Tools

Site Tools

Home / products / sbc / vim3 / npu / npu-app
products:sbc:vim3:npu:npu-app

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision 		Previous revision
products:sbc:vim3:npu:npu-app [2023/07/04 23:16]
louis 		products:sbc:vim3:npu:npu-app [2024/03/12 05:26] (current)
louis
Line 1: Line 1:
 +~~tag> VIM3 VIM3L Amlogic NPU OpenCV~~
 +
 ====== Application Source Code ====== ====== Application Source Code ======
- +Guide to compiling the NPU application on the VIM3/3L.
-===== Introduction ===== +
- +
-This document mainly describes how to compile executable files from application source code. +
- +
-<WRAP important > +
-Only support local compile on VIM3/3L +
-Only support OpenCV4 +
-</WRAP>+
  
 ===== Install OpenCV4 ===== ===== Install OpenCV4 =====
 +Install the necessary library packages to build the OpenCV4 application.
 ```shell ```shell
 $ sudo apt update $ sudo apt update
Line 18: Line 12:
  
 ===== Get Source Code ===== ===== Get Source Code =====
 +Get the source code: [[gh>khadas/aml_npu_app]]
 ```shell ```shell
-$ mkdir{workspace} && cd {workspace} 
 $ git clone https://github.com/khadas/aml_npu_app $ git clone https://github.com/khadas/aml_npu_app
 ``` ```
  
 ===== Source Description ===== ===== Source Description =====
 +==== Base directory ====
 +Base directory of the cloned repository.
 +```shell
 +$ cd aml_npu_app
 +$ ls
 +DDK_6.3.2  DDK_6.3.2.3  DDK_6.3.2.5  DDK_6.3.3.4  DDK_6.4.0.3  DDK_6.4.3  detect_library  LICENSE  NN_SLT
 +```
 +^ File ^ Description ^
 +| DDK_xx | Library source code, xx is the version code |
 +| detect_library | Application layer source code. the detect_library/model_code \\ is linked to the latest version of the DDK library |
 +| NN_SLT | Separate source code directory for DnCnn model (no longer maintained). |
 +| LICENSE | LICENSE file. |
 +                                                                                                |
  
 +==== Detect library ====
 +Directory with various model specific application code.
 +```shell
 +$ cd aml_npu_app/detect_library
 +$ ls
 +densenet_ctc_demo_picture  inception  model_code  retinaface_demo_x11_usb  sample_demo_fb  sample_demo_x11  source_code  yolo_demo_fb_mipi  yolo_demo_fb_usb
 +yolo_demo_x11_mipi  yolo_demo_x11_usb  yolov7_tiny_demo_x11_usb  yolov8n_demo_x11_usb
 +```
  
-==== Directory structure description ====+^ File            ^ Description ^ 
 +| **model_code**  | Load model and complete pre-processing and post-processing. | 
 +| **source_code** | Provide interfaces for demo to call model_code. | 
 +| **Other folders** | Application source code | 
 + 
 +==== Model code ==== 
 +Directory containing all the detection library code.
  
 ```shell ```shell
-$ cd {workspace}/aml_npu_app+$ cd aml_npu_app/detect_library/model_code
 $ ls $ ls
-DDK_6.3.2  DDK_6.3.2.3  DDK_6.3.2.5  DDK_6.3.3.4  DDK_6.4.0.3  DDK_6.4.3  detect_library  LICENSE  NN_SLT+densenet_ctc  detect_mtcnn  detect_retinaface  detect_yoloface  detect_yolo_v2  detect_yolo_v3  detect_yolo_v3_tiny  detect_yolo_v4  detect_yolo_v7_tiny 
 +detect_yolov8n  facenet
 ``` ```
  
-  * **DDK_xxx** - Different versions of library source code +Except for ''detect_mtcnn'', each directory represents a different model, and each model will be compiled into a library.
-  * **detect_library** - Application layer source code. detect_library/model_code link to the latest version of DDK +
-  * **NN_SLT** - Separate source code directory for DnCnn model (no longer maintained) +
-  * **LICENSE** - LICENSE file+
  
-==== Library Description ====+^ File ^ Description ^ 
 +| **densenet_ctc** | Character recognition, used to identify characters. The input is only a picture. | 
 +| **detect_retinaface** | Detect faces and the keypoints of faces. | 
 +| **detect_yoloface** | yoloface model, used to detect faces without keypoints. | 
 +| **detect_yolo_v2** | yolov2 model, for object detection. | 
 +| **detect_yolo_v3** | yolov3 model, for object detection. | 
 +| **detect_yolo_v3_tiny** | yolov3-tiny model, for object detection. | 
 +| **detect_yolo_v4** | yolov4 model, used to detect faces. | 
 +| **detect_yolo_v7_tiny** | yolov7_tiny model, for object detection. | 
 +| **detect_yolov8n** | yolov8n model, for object detection. The latest yolo model. | 
 +| **facenet** | Deprecated - No longer in use. |
  
  
 +Take ''detect_yolo_v3'' as an example to illustrate the structure of each directory.
  
 ```shell ```shell
-$ cd {workspace}/aml_npu_app/detect_library+$ cd aml_npu_app/detect_library/model_code/detect_yolo_v3
 $ ls $ ls
-densenet_ctc_demo_picture  inception  model_code  retinaface_demo_x11_usb  sample_demo_fb  sample_demo_x11  source_code  yolo_demo_fb_mipi  yolo_demo_fb_usb  yolo_demo_x11_mipi   +build_vx.sh  include  Makefile  makefile.linux  makefile.linux.def  vnn_yolov3.c  yolo_v3.c  yolov3_process.c
-yolo_demo_x11_usb  yolov7_tiny_demo_x11_usb  yolov8n_demo_x11_usb+
 ``` ```
  
-Enter the directory of the library source code, which is the directory of the DDK.+^ File ^ Description ^ 
 +| **build_vx.sh** | Compile script. | 
 +| **include** | The corresponding header files and all definitions will be placed in this directory. | 
 +| **Makefile** | Makefile file. | 
 +| **makefile.linux** | Make environment configuration file. | 
 +| **vnn_yolov3.c** | SDK The converted model processing file is mainly used to interface with the nb file. | 
 +| **yolo_v3.c** | Specify the called nb file, and define all the interfaces of the model call. | 
 +| **yolov3_process.c** | Mainly defines the pre-processing and post-processing of the model. | 
 + 
 + 
 +==== Source code ==== 
 +Directory containing the application library source code. 
  
 ```shell ```shell
-$ cd {workspace}/aml_npu_app/detect_library/model_code+$ cd aml_npu_app/detect_library/source_code
 $ ls $ ls
-detect_mtcnn  detect_yoloface  detect_yolo_v2  detect_yolo_v3  detect_yolo_v3_tiny  detect_yolo_v4  facenet+build_vx.sh  detect.c  detect_log.c  include  Makefile  makefile.linux  makefile.linux.def
 ``` ```
  
-Except for ''detect_mtcnn'', each directory represents a different model, and each model will be compiled into a library.+^ File ^ Description ^ 
 +| **build_vx.sh** | Compile script. | 
 +| **detect.c** | Provide interface for detect and call model_code. | 
 +| **detect_log.c** | About log level. | 
 +| **include** | The corresponding header files and all definitions will be placed in this directory. | 
 +| **Makefile** | Makefile file. | 
 +| **makefile.linux** | Make environment configuration file. |
  
-  * **detect_yoloface** - yoloface model, used to detect faces 
-  * **detect_yolo_v2** - yolov2 model, for object detection 
-  * **detect_yolo_v3** - yolov3 model, for object detection 
-  * **detect_yolo_v3_tiny** - yolov3-tiny model, for object detection 
-  * **detect_yolo_v4** - yolov4 model, used to detect faces 
  
-Take ''detect_yolo_v3'' as an example to illustrate the structure of each directory.+Enter the directory of the ''yolo_demo_x11_usb'':
  
 ```shell ```shell
-$ cd {workspace}/aml_npu_app/detect_library/model_code/detect_yolo_v3+$ cd aml_npu_app/detect_library/yolo_demo_x11_usb
 $ ls $ ls
-build_vx.sh  include  Makefile  makefile.linux  vnn_yolov3.c  yolo_v3.c  yolov3_process.c+build_vx.sh  detect.h  main.cpp  makefile.linux  makefile.linux.def  makefile.target_name  nn_detect_common.h  nn_detect.h  nn_detect_utils.h
 ``` ```
  
-Main content description:+^ File ^ Description ^ 
 +| **build_vx.sh** | Compile script. | 
 +| **main.cpp** | Main function. | 
 +| **detect.h**, **nn_detect_common.h**, **nn_detect.h**, **nn_detect_utils.h** | The corresponding header files. | 
 +| **makefile.linux** | Make environment configuration file. | 
 +| **makefile.target_name** | The name of the compiled executable. |
  
-  * **build_vx.sh** - Compile script +===== Compile the application=====
-  * **include** - The corresponding header files and all definitions will be placed in this directory +
-  * **Makefile** - Makefile file +
-  * **makefile.linux** - Make environment configuration file +
-  * **vnn_yolov3.c** - SDK The converted model processing file is mainly used to interface with the nb file +
-  * **yolo_v3.c** - Specify the called nb file, and define all the interfaces of the model call +
-  * **yolov3_process.c** - Mainly defines the pre-processing and post-processing of the model+
  
 +1. Compile the model you want to use from the ''model_code'' directory. Here we shall take ''detect_yolo_v3'' as an example.
  
 +```shell
 +$ cd aml_npu_app/detect_library/model_code/detect_yolo_v3
 +$ ls
 +build_vx.sh  include  Makefile  makefile.linux  makefile.linux.def  vnn_yolov3.c  yolo_v3.c  yolov3_process.c
 +$ bash build_vx.sh
 +$ ls
 +bin_r  build_vx.sh  include  linux_build_sample.log  makefile.linux  makefile.linux.def  makefile.target_name  vnn_yolov3.c  yolo_v3.c  yolov3_process.c
 +```
  
 +2. The compiled library ''libnn_yolo_v3.so'' is generated in ''bin_r'' folder. Add it to the library path variable.
 +
 +```shell
 +$ export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$(pwd)/bin_r/libnn_yolo_v3.so
 +```
 +
 +3. You will also need to build the contents of the ''source_code'' directory.
 +```shell
 +$ cd aml_npu_app/detect_library/source_code
 +$ ls
 +build_vx.sh  detect.c  detect_log.c  include  Makefile  makefile.linux  makefile.linux.def
 +$ bash build_vx.sh
 +$ ls
 +bin_r  build_vx.sh  detect.c  detect_log.c  include  Makefile  makefile.linux  makefile.linux.def
 +```
 +
 +4. The compiled library ''libnn_yolo_v3.so'' is generated in bin_r folder. Add it to the library path variable.
 +
 +```shell
 +$ export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$(pwd)/bin_r/libnn_detect.so
 +```
 +
 +5. Now, Build the example application.
 +```shell
 +$ cd aml_npu_app/detect_library/yolo_demo_x11_usb
 +$ ls
 +build_vx.sh  detect.h  main.cpp  makefile.linux  makefile.linux.def  makefile.target_name  nn_detect_common.h  nn_detect.h  nn_detect_utils.h
 +$ bash build_vx.sh
 +$ ls
 +bin_r_cv4  build_vx.sh  detect.h  linux_build_sample.log  main.cpp  makefile.linux  makefile.linux.def  makefile.target_name  nn_detect_common.h  nn_detect.h  nn_detect_utils.h
 +```
 +
 +<WRAP important >
 +If you use kernel are using version 5.15 or above, please remove the red lines in ''makefile.linux'' before running ''build_vx.sh''.
 +</WRAP>
 +
 +```diff
 +LIBS += -L$(VIVANTE_SDK_LIB) -lOpenVX -lOpenVXU -lGAL -lovxlib -lArchModelSw -lNNArchPerf
 +
 +LIBS += -L../source_code/bin_r -lnn_detect
 +
 +-#LIBS +=-L$(LIB_DIR) -lstdc++
 +-LIBS += -lvpcodec -lamcodec -lamadec -lamvdec -lamavutils -lrt -lpthread -lge2d -lion
 +
 +#############################################################################
 +# Macros.
 +PROGRAM = 1
 +CUR_SOURCE = ${wildcard *.c}
 +#############################################################################
 +```
 +
 +6. Create a folder named ''nn_data'' under ''bin_r_cv4'' and place the ''yolov3_88.nb'' model file in it.
 +```shell
 +$ cd bin_r_cv4
 +$ mkdir nn_data
 +$ cp -r xx/xx/yolov3_88.nb nn_data/
 +$ ls
 +detect_demo_x11_usb  main.o  nn_data
 +```
 +===== Run the application =====
 +
 +Now you can run inference on captured video data using the model.
 +```shell
 +$ ./detect_demo_x11_usb -m 2 -d /dev/video1
 +```
 +==== Application setup parameters ====
 +=== Parameter to select the detection model ===
 +
 +The parameter ''-m'' is for selecting the inference model, Here are the all the available models and their respective serial numbers.
 +
 +^ Index ^ File ^
 +| **0** | detect_yoloface |
 +| **1** | detect_yolo_v2 |
 +| **2** | detect_yolo_v3 |
 +| **3** | detect_yolo_v3_tiny |
 +| **4** | detect_yolo_v4 |
 +| **13** | detect_yolo_v7_tiny |
 +| **14** | detect_yolov8n |
 +| **15** | densenet_ctc |
 +| **16** | detect_retinaface |
 +
 +
 +For the above example, we used the serial number ''2'' which corresponds to the ''yolov3'' model.
 +
 +=== Parameter to set the input as picture ===
 +
 +If you use the model that takes input as a picture, change ''-d'' to ''-p''. Here is an example with ''densenet_ctc''.
 +```shell
 +$ ./densenet_ctc_picture -m 15 -p ../KhadasTeam.png
 +```
Last modified: 2023/07/04 23:16 by louis

Page Tools

Khadas Site

Khadas Community

Khadas GitHub

Khadas Downloads