Khadas Docs

Amazing Khadas, always amazes you!

User Tools

Site Tools

Home / products / sbc / vim3 / npu / npu-app
products:sbc:vim3:npu:npu-app

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision 		Previous revision
Last revision Both sides next revision
products:sbc:vim3:npu:npu-app [2023/07/06 02:57]
louis 		products:sbc:vim3:npu:npu-app [2024/01/22 20:31]
louis
Line 1: Line 1:
 +~~tag> VIM3 VIM3L Amlogic NPU OpenCV~~
 +
 ====== Application Source Code ====== ====== Application Source Code ======
- +Guide to compiling the NPU application on the VIM3/3L.
-===== Introduction ===== +
- +
-This document mainly describes how to compile executable files from application source code. +
- +
-<WRAP important > +
-Only support local compile on VIM3/3L +
-Only support OpenCV4 +
-</WRAP>+
  
 ===== Install OpenCV4 ===== ===== Install OpenCV4 =====
 +Install the necessary library packages to build the OpenCV4 application.
 ```shell ```shell
 $ sudo apt update $ sudo apt update
Line 18: Line 12:
  
 ===== Get Source Code ===== ===== Get Source Code =====
 +Get the source code: [[gh>khadas/aml_npu_app]]
 ```shell ```shell
-$ mkdir{workspace} && cd {workspace} 
 $ git clone https://github.com/khadas/aml_npu_app $ git clone https://github.com/khadas/aml_npu_app
 ``` ```
  
 ===== Source Description ===== ===== Source Description =====
- +==== Base directory ==== 
- +Base directory of the cloned repository.
-==== Directory structure description ==== +
 ```shell ```shell
-$ cd {workspace}/aml_npu_app+$ cd aml_npu_app
 $ ls $ ls
 DDK_6.3.2  DDK_6.3.2.3  DDK_6.3.2.5  DDK_6.3.3.4  DDK_6.4.0.3  DDK_6.4.3  detect_library  LICENSE  NN_SLT DDK_6.3.2  DDK_6.3.2.3  DDK_6.3.2.5  DDK_6.3.3.4  DDK_6.4.0.3  DDK_6.4.3  detect_library  LICENSE  NN_SLT
 ``` ```
 +^ File ^ Description ^
 +| DDK_xx | Library source code, xx is the version code |
 +| detect_library | Application layer source code. the detect_library/model_code \\ is linked to the latest version of the DDK library |
 +| NN_SLT | Separate source code directory for DnCnn model (no longer maintained). |
 +| LICENSE | LICENSE file. |
 +                                                                                                |
  
-  * **DDK_xxx** - Different versions of library source code +==== Detect library ==== 
-  * **detect_library** - Application layer source code. detect_library/model_code link to the latest version of DDK +Directory with various model specific application code.
-  * **NN_SLT** - Separate source code directory for DnCnn model (no longer maintained) +
-  * **LICENSE** - LICENSE file +
- +
-==== Library Description ==== +
 ```shell ```shell
-$ cd {workspace}/aml_npu_app/detect_library+$ cd aml_npu_app/detect_library
 $ ls $ ls
 densenet_ctc_demo_picture  inception  model_code  retinaface_demo_x11_usb  sample_demo_fb  sample_demo_x11  source_code  yolo_demo_fb_mipi  yolo_demo_fb_usb densenet_ctc_demo_picture  inception  model_code  retinaface_demo_x11_usb  sample_demo_fb  sample_demo_x11  source_code  yolo_demo_fb_mipi  yolo_demo_fb_usb
Line 49: Line 41:
 ``` ```
  
-  * **model_code** - Load model and complete pre-processing and post-processing. +^ File            ^ Description ^ 
-  * **source_code** - Provide interfaces for demo to call model_code. +| **model_code**  | Load model and complete pre-processing and post-processing. | 
-  * **Other folders** - Load input and visualize the results.+| **source_code** | Provide interfaces for demo to call model_code. | 
 +| **Other folders** | Application source code |
  
-Enter the directory of the library model_code, which is the directory of the DDK.+==== Model code ==== 
 +Directory containing all the detection library code.
  
 ```shell ```shell
-$ cd {workspace}/aml_npu_app/detect_library/model_code+$ cd aml_npu_app/detect_library/model_code
 $ ls $ ls
 densenet_ctc  detect_mtcnn  detect_retinaface  detect_yoloface  detect_yolo_v2  detect_yolo_v3  detect_yolo_v3_tiny  detect_yolo_v4  detect_yolo_v7_tiny densenet_ctc  detect_mtcnn  detect_retinaface  detect_yoloface  detect_yolo_v2  detect_yolo_v3  detect_yolo_v3_tiny  detect_yolo_v4  detect_yolo_v7_tiny
Line 64: Line 58:
 Except for ''detect_mtcnn'', each directory represents a different model, and each model will be compiled into a library. Except for ''detect_mtcnn'', each directory represents a different model, and each model will be compiled into a library.
  
-  * **densenet_ctc** - Character recognition, used to identify characters. The input is only picture. +^ File ^ Description ^ 
-  * **detect_retinaface** - Detect faces and the keypoints of faces. +| **densenet_ctc** | Character recognition, used to identify characters. The input is only a picture. | 
-  * **detect_yoloface** - yoloface model, used to detect faces without keypoint. +| **detect_retinaface** | Detect faces and the keypoints of faces. | 
-  * **detect_yolo_v2** - yolov2 model, for object detection. +| **detect_yoloface** | yoloface model, used to detect faces without keypoints. | 
-  * **detect_yolo_v3** - yolov3 model, for object detection. +| **detect_yolo_v2** | yolov2 model, for object detection. | 
-  * **detect_yolo_v3_tiny** - yolov3-tiny model, for object detection. +| **detect_yolo_v3** | yolov3 model, for object detection. | 
-  * **detect_yolo_v4** - yolov4 model, used to detect faces. +| **detect_yolo_v3_tiny** | yolov3-tiny model, for object detection. | 
-  * **detect_yolo_v7_tiny** - yolov7_tiny model, for object detection. +| **detect_yolo_v4** | yolov4 model, used to detect faces. | 
-  * **detect_yolov8n** - yolov8n model, for object detection. The latest yolo model. +| **detect_yolo_v7_tiny** | yolov7_tiny model, for object detection. | 
-  * **facenet** - Has been deprecated.+| **detect_yolov8n** | yolov8n model, for object detection. The latest yolo model. | 
 +| **facenet** | Deprecated - No longer in use. | 
  
 Take ''detect_yolo_v3'' as an example to illustrate the structure of each directory. Take ''detect_yolo_v3'' as an example to illustrate the structure of each directory.
  
 ```shell ```shell
-$ cd {workspace}/aml_npu_app/detect_library/model_code/detect_yolo_v3+$ cd aml_npu_app/detect_library/model_code/detect_yolo_v3
 $ ls $ ls
 build_vx.sh  include  Makefile  makefile.linux  makefile.linux.def  vnn_yolov3.c  yolo_v3.c  yolov3_process.c build_vx.sh  include  Makefile  makefile.linux  makefile.linux.def  vnn_yolov3.c  yolo_v3.c  yolov3_process.c
 ``` ```
  
-Main content description:+^ File ^ Description ^ 
 +| **build_vx.sh** | Compile script. | 
 +| **include** | The corresponding header files and all definitions will be placed in this directory. | 
 +| **Makefile** | Makefile file. | 
 +| **makefile.linux** | Make environment configuration file. | 
 +| **vnn_yolov3.c** | SDK The converted model processing file is mainly used to interface with the nb file. | 
 +| **yolo_v3.c** | Specify the called nb file, and define all the interfaces of the model call. | 
 +| **yolov3_process.c** | Mainly defines the pre-processing and post-processing of the model. |
  
-  * **build_vx.sh** - Compile script. 
-  * **include** - The corresponding header files and all definitions will be placed in this directory. 
-  * **Makefile** - Makefile file. 
-  * **makefile.linux** - Make environment configuration file. 
-  * **vnn_yolov3.c** - SDK The converted model processing file is mainly used to interface with the nb file. 
-  * **yolo_v3.c** - Specify the called nb file, and define all the interfaces of the model call. 
-  * **yolov3_process.c** - Mainly defines the pre-processing and post-processing of the model. 
  
-Enter the directory of the source_code.+==== Source code ==== 
 +Directory containing the application library source code. 
  
 ```shell ```shell
-$ cd {workspace}/aml_npu_app/detect_library/source_code+$ cd aml_npu_app/detect_library/source_code
 $ ls $ ls
 build_vx.sh  detect.c  detect_log.c  include  Makefile  makefile.linux  makefile.linux.def build_vx.sh  detect.c  detect_log.c  include  Makefile  makefile.linux  makefile.linux.def
 ``` ```
  
-Main content description:+^ File ^ Description ^ 
 +| **build_vx.sh** | Compile script. | 
 +| **detect.c** | Provide interface for detect and call model_code. | 
 +| **detect_log.c** | About log level. | 
 +| **include** | The corresponding header files and all definitions will be placed in this directory. | 
 +| **Makefile** | Makefile file. | 
 +| **makefile.linux** | Make environment configuration file. |
  
-  * **build_vx.sh** - Compile script. 
-  * **detect.c** - Provide interface for detect and call model_code. 
-  * **detect_log.c** - About log level. 
-  * **include** - The corresponding header files and all definitions will be placed in this directory. 
-  * **Makefile** - Makefile file. 
-  * **makefile.linux** - Make environment configuration file. 
  
-Enter the directory of the yolo_demo_x11_usb.+Enter the directory of the ''yolo_demo_x11_usb'':
  
 ```shell ```shell
-$ cd {workspace}/aml_npu_app/detect_library/yolo_demo_x11_usb+$ cd aml_npu_app/detect_library/yolo_demo_x11_usb
 $ ls $ ls
 build_vx.sh  detect.h  main.cpp  makefile.linux  makefile.linux.def  makefile.target_name  nn_detect_common.h  nn_detect.h  nn_detect_utils.h build_vx.sh  detect.h  main.cpp  makefile.linux  makefile.linux.def  makefile.target_name  nn_detect_common.h  nn_detect.h  nn_detect_utils.h
 ``` ```
  
-  * **build_vx.sh** - Compile script. +^ File ^ Description ^ 
-  * **main.cpp** - Main function. +| **build_vx.sh** | Compile script. | 
-  * **detect.h**, **nn_detect_common.h**, **nn_detect.h**, **nn_detect_utils.h** - The corresponding header files. +| **main.cpp** | Main function. | 
-  * **makefile.linux** - Make environment configuration file. +| **detect.h**, **nn_detect_common.h**, **nn_detect.h**, **nn_detect_utils.h** | The corresponding header files. | 
-  * **makefile.target_name** - The name of the compiled executable.+| **makefile.linux** | Make environment configuration file. | 
 +| **makefile.target_name** | The name of the compiled executable. |
  
-===== Compile and inference =====+===== Compile the application=====
  
-Compile model_code you need. Here still take ''detect_yolo_v3'' as an example.+1. Compile the model you want to use from the ''model_code'' directory. Here we shall take ''detect_yolo_v3'' as an example.
  
 ```shell ```shell
-$ cd {workspace}/aml_npu_app/detect_library/model_code/detect_yolo_v3+$ cd aml_npu_app/detect_library/model_code/detect_yolo_v3
 $ ls $ ls
 build_vx.sh  include  Makefile  makefile.linux  makefile.linux.def  vnn_yolov3.c  yolo_v3.c  yolov3_process.c build_vx.sh  include  Makefile  makefile.linux  makefile.linux.def  vnn_yolov3.c  yolo_v3.c  yolov3_process.c
Line 137: Line 135:
 ``` ```
  
-The compiled library, ''libnn_yolo_v3.so'', is generated in bin_r folder. Put the library into /usr/lib.+2. The compiled library ''libnn_yolo_v3.so'' is generated in ''bin_r'' folder. Add it to the library path variable.
  
 ```shell ```shell
-$ sudo cp -r bin_r/libnn_yolo_v3.so /usr/lib+$ export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$(pwd)/bin_r/libnn_yolo_v3.so
 ``` ```
  
-Compile source_code and put library into /usr/lib, too.+3. You will also need to build the contents of the ''source_code'' directory.
 ```shell ```shell
-$ cd {workspace}/aml_npu_app/detect_library/source_code+$ cd aml_npu_app/detect_library/source_code
 $ ls $ ls
 build_vx.sh  detect.c  detect_log.c  include  Makefile  makefile.linux  makefile.linux.def build_vx.sh  detect.c  detect_log.c  include  Makefile  makefile.linux  makefile.linux.def
Line 151: Line 149:
 $ ls $ ls
 bin_r  build_vx.sh  detect.c  detect_log.c  include  Makefile  makefile.linux  makefile.linux.def bin_r  build_vx.sh  detect.c  detect_log.c  include  Makefile  makefile.linux  makefile.linux.def
-$ sudo cp -r bin_r/libnn_detect.so /usr/lib 
 ``` ```
  
-Compile main function.+4. The compiled library ''libnn_yolo_v3.so'' is generated in bin_r folder. Add it to the library path variable. 
 + 
 +```shell 
 +$ export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$(pwd)/bin_r/libnn_detect.so 
 +``` 
 + 
 +5. Now, Build the example application.
 ```shell ```shell
-$ cd {workspace}/aml_npu_app/detect_library/yolo_demo_x11_usb+$ cd aml_npu_app/detect_library/yolo_demo_x11_usb
 $ ls $ ls
 build_vx.sh  detect.h  main.cpp  makefile.linux  makefile.linux.def  makefile.target_name  nn_detect_common.h  nn_detect.h  nn_detect_utils.h build_vx.sh  detect.h  main.cpp  makefile.linux  makefile.linux.def  makefile.target_name  nn_detect_common.h  nn_detect.h  nn_detect_utils.h
 $ bash build_vx.sh $ bash build_vx.sh
 +$ ls
 bin_r_cv4  build_vx.sh  detect.h  linux_build_sample.log  main.cpp  makefile.linux  makefile.linux.def  makefile.target_name  nn_detect_common.h  nn_detect.h  nn_detect_utils.h bin_r_cv4  build_vx.sh  detect.h  linux_build_sample.log  main.cpp  makefile.linux  makefile.linux.def  makefile.target_name  nn_detect_common.h  nn_detect.h  nn_detect_utils.h
 ``` ```
  
-Build a folder named ''nn_data'' under bin_r_cv4 and put ''yolov3_88.nb'' into it.+<WRAP important > 
 +If you use kernel are using version 5.15 or above, please modify ''makefile.linux'' as follows before running ''build_vx.sh''. 
 +</WRAP> 
 + 
 +```diff 
 +-#LIBS +=-L$(LIB_DIR) -lstdc++ 
 +-LIBS += -lvpcodec -lamcodec -lamadec -lamvdec -lamavutils -lrt -lpthread -lge2d -lion 
 +``` 
 + 
 +6. Create a folder named ''nn_data'' under ''bin_r_cv4'' and place the ''yolov3_88.nb'' model file in it.
 ```shell ```shell
 $ cd bin_r_cv4 $ cd bin_r_cv4
 $ mkdir nn_data $ mkdir nn_data
-$ cp -r {path}/yolov3_88.nb nn_data/+$ cp -r xx/xx/yolov3_88.nb nn_data/
 $ ls $ ls
 detect_demo_x11_usb  main.o  nn_data detect_demo_x11_usb  main.o  nn_data
 ``` ```
 +===== Run the application =====
  
-Run inference.+Now you can run inference on captured video data using the model.
 ```shell ```shell
 $ ./detect_demo_x11_usb -m 2 -d /dev/video1 $ ./detect_demo_x11_usb -m 2 -d /dev/video1
 ``` ```
 +==== Application setup parameters ====
 +=== Parameter to select the detection model ===
 +
 +The parameter ''-m'' is for selecting the inference model, Here are the all the available models and their respective serial numbers.
 +
 +^ Index ^ File ^
 +| **0** | detect_yoloface |
 +| **1** | detect_yolo_v2 |
 +| **2** | detect_yolo_v3 |
 +| **3** | detect_yolo_v3_tiny |
 +| **4** | detect_yolo_v4 |
 +| **13** | detect_yolo_v7_tiny |
 +| **14** | detect_yolov8n |
 +| **15** | densenet_ctc |
 +| **16** | detect_retinaface |
 +
 +
 +For the above example, we used the serial number ''2'' which corresponds to the ''yolov3'' model.
  
-The parameter ''-m'' means the inference model and ''2'' stands for yolov3. Here are the all available model and these serial number. +=== Parameter to set the input as picture ===
-  * **0** - detect_yoloface  +
-  * **1** - detect_yolo_v2  +
-  * **2** - detect_yolo_v3  +
-  * **3** - detect_yolo_v3_tiny  +
-  * **4** - detect_yolo_v4  +
-  * **13** - detect_yolo_v7_tiny  +
-  * **14** - detect_yolov8n  +
-  * **15** - densenet_ctc  +
-  * **16** - detect_retinaface+
  
-If you use the model that input is picture, change ''-d'' to ''-p''. Here example is densenet_ctc.+If you use the model that takes input as a picture, change ''-d'' to ''-p''. Here is an example with ''densenet_ctc''.
 ```shell ```shell
 $ ./densenet_ctc_picture -m 15 -p ../KhadasTeam.png $ ./densenet_ctc_picture -m 15 -p ../KhadasTeam.png
 ``` ```
Last modified: 2024/03/12 05:26 by louis

Page Tools

Khadas Site

Khadas Community

Khadas GitHub

Khadas Downloads