Khadas Docs

Amazing Khadas, always amazes you!

User Tools

Site Tools

Home / products / sbc / vim3 / npu / npu-app
products:sbc:vim3:npu:npu-app

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision 		Previous revision
products:sbc:vim3:npu:npu-app [2023/07/06 06:22]
louis 		products:sbc:vim3:npu:npu-app [2024/03/12 05:26] (current)
louis
Line 1: Line 1:
 +~~tag> VIM3 VIM3L Amlogic NPU OpenCV~~
 +
 ====== Application Source Code ====== ====== Application Source Code ======
- +Guide to compiling the NPU application on the VIM3/3L.
-<WRAP important > +
-Only support local compile on VIM3/3L and only support OpenCV4. +
-</WRAP>+
  
 ===== Install OpenCV4 ===== ===== Install OpenCV4 =====
 +Install the necessary library packages to build the OpenCV4 application.
 ```shell ```shell
 $ sudo apt update $ sudo apt update
Line 13: Line 12:
  
 ===== Get Source Code ===== ===== Get Source Code =====
 +Get the source code: [[gh>khadas/aml_npu_app]]
 ```shell ```shell
-$ mkdir{workspace} && cd {workspace} 
 $ git clone https://github.com/khadas/aml_npu_app $ git clone https://github.com/khadas/aml_npu_app
 ``` ```
  
 ===== Source Description ===== ===== Source Description =====
 +==== Base directory ==== 
 +Base directory of the cloned repository.
 ```shell ```shell
-$ cd {workspace}/aml_npu_app+$ cd aml_npu_app
 $ ls $ ls
 DDK_6.3.2  DDK_6.3.2.3  DDK_6.3.2.5  DDK_6.3.3.4  DDK_6.4.0.3  DDK_6.4.3  detect_library  LICENSE  NN_SLT DDK_6.3.2  DDK_6.3.2.3  DDK_6.3.2.5  DDK_6.3.3.4  DDK_6.4.0.3  DDK_6.4.3  detect_library  LICENSE  NN_SLT
 ``` ```
 +^ File ^ Description ^
 +| DDK_xx | Library source code, xx is the version code |
 +| detect_library | Application layer source code. the detect_library/model_code \\ is linked to the latest version of the DDK library |
 +| NN_SLT | Separate source code directory for DnCnn model (no longer maintained). |
 +| LICENSE | LICENSE file. |
 +                                                                                                |
  
-  * **DDK_xxx** - Different versions of library source code. +==== Detect library ==== 
-  * **detect_library** - Application layer source code. detect_library/model_code link to the latest version of DDK. +Directory with various model specific application code.
-  * **NN_SLT** - Separate source code directory for DnCnn model (no longer maintained). +
-  * **LICENSE** - LICENSE file. +
 ```shell ```shell
-$ cd {workspace}/aml_npu_app/detect_library+$ cd aml_npu_app/detect_library
 $ ls $ ls
 densenet_ctc_demo_picture  inception  model_code  retinaface_demo_x11_usb  sample_demo_fb  sample_demo_x11  source_code  yolo_demo_fb_mipi  yolo_demo_fb_usb densenet_ctc_demo_picture  inception  model_code  retinaface_demo_x11_usb  sample_demo_fb  sample_demo_x11  source_code  yolo_demo_fb_mipi  yolo_demo_fb_usb
Line 39: Line 41:
 ``` ```
  
-  * **model_code** - Load model and complete pre-processing and post-processing. +^ File            ^ Description ^ 
-  * **source_code** - Provide interfaces for demo to call model_code. +| **model_code**  | Load model and complete pre-processing and post-processing. | 
-  * **Other folders** - Load input and visualize the results.+| **source_code** | Provide interfaces for demo to call model_code. | 
 +| **Other folders** | Application source code |
  
-Enter the directory of the library **model_code**, which is the directory of the DDK.+==== Model code ==== 
 +Directory containing all the detection library code.
  
 ```shell ```shell
-$ cd {workspace}/aml_npu_app/detect_library/model_code+$ cd aml_npu_app/detect_library/model_code
 $ ls $ ls
 densenet_ctc  detect_mtcnn  detect_retinaface  detect_yoloface  detect_yolo_v2  detect_yolo_v3  detect_yolo_v3_tiny  detect_yolo_v4  detect_yolo_v7_tiny densenet_ctc  detect_mtcnn  detect_retinaface  detect_yoloface  detect_yolo_v2  detect_yolo_v3  detect_yolo_v3_tiny  detect_yolo_v4  detect_yolo_v7_tiny
Line 54: Line 58:
 Except for ''detect_mtcnn'', each directory represents a different model, and each model will be compiled into a library. Except for ''detect_mtcnn'', each directory represents a different model, and each model will be compiled into a library.
  
-  * **densenet_ctc** - Character recognition, used to identify characters. The input is only picture. +^ File ^ Description ^ 
-  * **detect_retinaface** - Detect faces and the keypoints of faces. +| **densenet_ctc** | Character recognition, used to identify characters. The input is only a picture. | 
-  * **detect_yoloface** - yoloface model, used to detect faces without keypoint. +| **detect_retinaface** | Detect faces and the keypoints of faces. | 
-  * **detect_yolo_v2** - yolov2 model, for object detection. +| **detect_yoloface** | yoloface model, used to detect faces without keypoints. | 
-  * **detect_yolo_v3** - yolov3 model, for object detection. +| **detect_yolo_v2** | yolov2 model, for object detection. | 
-  * **detect_yolo_v3_tiny** - yolov3-tiny model, for object detection. +| **detect_yolo_v3** | yolov3 model, for object detection. | 
-  * **detect_yolo_v4** - yolov4 model, used to detect faces. +| **detect_yolo_v3_tiny** | yolov3-tiny model, for object detection. | 
-  * **detect_yolo_v7_tiny** - yolov7_tiny model, for object detection. +| **detect_yolo_v4** | yolov4 model, used to detect faces. | 
-  * **detect_yolov8n** - yolov8n model, for object detection. The latest yolo model. +| **detect_yolo_v7_tiny** | yolov7_tiny model, for object detection. | 
-  * **facenet** - Has been deprecated.+| **detect_yolov8n** | yolov8n model, for object detection. The latest yolo model. | 
 +| **facenet** | Deprecated - No longer in use. | 
  
 Take ''detect_yolo_v3'' as an example to illustrate the structure of each directory. Take ''detect_yolo_v3'' as an example to illustrate the structure of each directory.
  
 ```shell ```shell
-$ cd {workspace}/aml_npu_app/detect_library/model_code/detect_yolo_v3+$ cd aml_npu_app/detect_library/model_code/detect_yolo_v3
 $ ls $ ls
 build_vx.sh  include  Makefile  makefile.linux  makefile.linux.def  vnn_yolov3.c  yolo_v3.c  yolov3_process.c build_vx.sh  include  Makefile  makefile.linux  makefile.linux.def  vnn_yolov3.c  yolo_v3.c  yolov3_process.c
 ``` ```
  
-Main content description:+^ File ^ Description ^ 
 +| **build_vx.sh** | Compile script. | 
 +| **include** | The corresponding header files and all definitions will be placed in this directory. | 
 +| **Makefile** | Makefile file. | 
 +| **makefile.linux** | Make environment configuration file. | 
 +| **vnn_yolov3.c** | SDK The converted model processing file is mainly used to interface with the nb file. | 
 +| **yolo_v3.c** | Specify the called nb file, and define all the interfaces of the model call. | 
 +| **yolov3_process.c** | Mainly defines the pre-processing and post-processing of the model. |
  
-  * **build_vx.sh** - Compile script. 
-  * **include** - The corresponding header files and all definitions will be placed in this directory. 
-  * **Makefile** - Makefile file. 
-  * **makefile.linux** - Make environment configuration file. 
-  * **vnn_yolov3.c** - SDK The converted model processing file is mainly used to interface with the nb file. 
-  * **yolo_v3.c** - Specify the called nb file, and define all the interfaces of the model call. 
-  * **yolov3_process.c** - Mainly defines the pre-processing and post-processing of the model. 
  
-Enter the directory of the **source_code**.+==== Source code ==== 
 +Directory containing the application library source code. 
  
 ```shell ```shell
-$ cd {workspace}/aml_npu_app/detect_library/source_code+$ cd aml_npu_app/detect_library/source_code
 $ ls $ ls
 build_vx.sh  detect.c  detect_log.c  include  Makefile  makefile.linux  makefile.linux.def build_vx.sh  detect.c  detect_log.c  include  Makefile  makefile.linux  makefile.linux.def
 ``` ```
  
-Main content description:+^ File ^ Description ^ 
 +| **build_vx.sh** | Compile script. | 
 +| **detect.c** | Provide interface for detect and call model_code. | 
 +| **detect_log.c** | About log level. | 
 +| **include** | The corresponding header files and all definitions will be placed in this directory. | 
 +| **Makefile** | Makefile file. | 
 +| **makefile.linux** | Make environment configuration file. |
  
-  * **build_vx.sh** - Compile script. 
-  * **detect.c** - Provide interface for detect and call model_code. 
-  * **detect_log.c** - About log level. 
-  * **include** - The corresponding header files and all definitions will be placed in this directory. 
-  * **Makefile** - Makefile file. 
-  * **makefile.linux** - Make environment configuration file. 
  
-Enter the directory of the **yolo_demo_x11_usb**.+Enter the directory of the ''yolo_demo_x11_usb'':
  
 ```shell ```shell
-$ cd {workspace}/aml_npu_app/detect_library/yolo_demo_x11_usb+$ cd aml_npu_app/detect_library/yolo_demo_x11_usb
 $ ls $ ls
 build_vx.sh  detect.h  main.cpp  makefile.linux  makefile.linux.def  makefile.target_name  nn_detect_common.h  nn_detect.h  nn_detect_utils.h build_vx.sh  detect.h  main.cpp  makefile.linux  makefile.linux.def  makefile.target_name  nn_detect_common.h  nn_detect.h  nn_detect_utils.h
 ``` ```
  
-  * **build_vx.sh** - Compile script. +^ File ^ Description ^ 
-  * **main.cpp** - Main function. +| **build_vx.sh** | Compile script. | 
-  * **detect.h**, **nn_detect_common.h**, **nn_detect.h**, **nn_detect_utils.h** - The corresponding header files. +| **main.cpp** | Main function. | 
-  * **makefile.linux** - Make environment configuration file. +| **detect.h**, **nn_detect_common.h**, **nn_detect.h**, **nn_detect_utils.h** | The corresponding header files. | 
-  * **makefile.target_name** - The name of the compiled executable.+| **makefile.linux** | Make environment configuration file. | 
 +| **makefile.target_name** | The name of the compiled executable. |
  
-===== Compile and inference =====+===== Compile the application=====
  
-Compile **model_code** you need. Here still take ''detect_yolo_v3'' as an example.+1. Compile the model you want to use from the ''model_code'' directory. Here we shall take ''detect_yolo_v3'' as an example.
  
 ```shell ```shell
-$ cd {workspace}/aml_npu_app/detect_library/model_code/detect_yolo_v3+$ cd aml_npu_app/detect_library/model_code/detect_yolo_v3
 $ ls $ ls
 build_vx.sh  include  Makefile  makefile.linux  makefile.linux.def  vnn_yolov3.c  yolo_v3.c  yolov3_process.c build_vx.sh  include  Makefile  makefile.linux  makefile.linux.def  vnn_yolov3.c  yolo_v3.c  yolov3_process.c
Line 127: Line 135:
 ``` ```
  
-The compiled library, ''libnn_yolo_v3.so'', is generated in bin_r folder. Put the library into /usr/lib.+2. The compiled library ''libnn_yolo_v3.so'' is generated in ''bin_r'' folder. Add it to the library path variable.
  
 ```shell ```shell
-$ sudo cp -r bin_r/libnn_yolo_v3.so /usr/lib+$ export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$(pwd)/bin_r/libnn_yolo_v3.so
 ``` ```
  
-Compile **source_code** and put library into /usr/lib, too.+3. You will also need to build the contents of the ''source_code'' directory.
 ```shell ```shell
-$ cd {workspace}/aml_npu_app/detect_library/source_code+$ cd aml_npu_app/detect_library/source_code
 $ ls $ ls
 build_vx.sh  detect.c  detect_log.c  include  Makefile  makefile.linux  makefile.linux.def build_vx.sh  detect.c  detect_log.c  include  Makefile  makefile.linux  makefile.linux.def
Line 141: Line 149:
 $ ls $ ls
 bin_r  build_vx.sh  detect.c  detect_log.c  include  Makefile  makefile.linux  makefile.linux.def bin_r  build_vx.sh  detect.c  detect_log.c  include  Makefile  makefile.linux  makefile.linux.def
-$ sudo cp -r bin_r/libnn_detect.so /usr/lib 
 ``` ```
  
-Compile main function.+4. The compiled library ''libnn_yolo_v3.so'' is generated in bin_r folder. Add it to the library path variable. 
 + 
 +```shell 
 +$ export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$(pwd)/bin_r/libnn_detect.so 
 +``` 
 + 
 +5. Now, Build the example application.
 ```shell ```shell
-$ cd {workspace}/aml_npu_app/detect_library/yolo_demo_x11_usb+$ cd aml_npu_app/detect_library/yolo_demo_x11_usb
 $ ls $ ls
 build_vx.sh  detect.h  main.cpp  makefile.linux  makefile.linux.def  makefile.target_name  nn_detect_common.h  nn_detect.h  nn_detect_utils.h build_vx.sh  detect.h  main.cpp  makefile.linux  makefile.linux.def  makefile.target_name  nn_detect_common.h  nn_detect.h  nn_detect_utils.h
Line 154: Line 167:
 ``` ```
  
-Build a folder named ''nn_data'' under ''bin_r_cv4'' and put ''yolov3_88.nb'' into it.+<WRAP important > 
 +If you use kernel are using version 5.15 or above, please remove the red lines in ''makefile.linux'' before running ''build_vx.sh''. 
 +</WRAP> 
 + 
 +```diff 
 +LIBS += -L$(VIVANTE_SDK_LIB) -lOpenVX -lOpenVXU -lGAL -lovxlib -lArchModelSw -lNNArchPerf 
 + 
 +LIBS += -L../source_code/bin_r -lnn_detect 
 + 
 +-#LIBS +=-L$(LIB_DIR) -lstdc++ 
 +-LIBS += -lvpcodec -lamcodec -lamadec -lamvdec -lamavutils -lrt -lpthread -lge2d -lion 
 + 
 +############################################################################# 
 +# Macros. 
 +PROGRAM = 1 
 +CUR_SOURCE = ${wildcard *.c} 
 +############################################################################# 
 +``` 
 + 
 +6. Create a folder named ''nn_data'' under ''bin_r_cv4'' and place the ''yolov3_88.nb'' model file in it.
 ```shell ```shell
 $ cd bin_r_cv4 $ cd bin_r_cv4
 $ mkdir nn_data $ mkdir nn_data
-$ cp -r {path}/yolov3_88.nb nn_data/+$ cp -r xx/xx/yolov3_88.nb nn_data/
 $ ls $ ls
 detect_demo_x11_usb  main.o  nn_data detect_demo_x11_usb  main.o  nn_data
 ``` ```
 +===== Run the application =====
  
-Run inference.+Now you can run inference on captured video data using the model.
 ```shell ```shell
 $ ./detect_demo_x11_usb -m 2 -d /dev/video1 $ ./detect_demo_x11_usb -m 2 -d /dev/video1
 ``` ```
 +==== Application setup parameters ====
 +=== Parameter to select the detection model ===
 +
 +The parameter ''-m'' is for selecting the inference model, Here are the all the available models and their respective serial numbers.
 +
 +^ Index ^ File ^
 +| **0** | detect_yoloface |
 +| **1** | detect_yolo_v2 |
 +| **2** | detect_yolo_v3 |
 +| **3** | detect_yolo_v3_tiny |
 +| **4** | detect_yolo_v4 |
 +| **13** | detect_yolo_v7_tiny |
 +| **14** | detect_yolov8n |
 +| **15** | densenet_ctc |
 +| **16** | detect_retinaface |
 +
 +
 +For the above example, we used the serial number ''2'' which corresponds to the ''yolov3'' model.
  
-The parameter ''-m'' means the inference model and ''2'' stands for yolov3. Here are the all available model and these serial number. +=== Parameter to set the input as picture ===
-  * **0** - detect_yoloface  +
-  * **1** - detect_yolo_v2  +
-  * **2** - detect_yolo_v3  +
-  * **3** - detect_yolo_v3_tiny  +
-  * **4** - detect_yolo_v4  +
-  * **13** - detect_yolo_v7_tiny  +
-  * **14** - detect_yolov8n  +
-  * **15** - densenet_ctc  +
-  * **16** - detect_retinaface+
  
-If you use the model that input is picture, change ''-d'' to ''-p''. Here example is densenet_ctc.+If you use the model that takes input as a picture, change ''-d'' to ''-p''. Here is an example with ''densenet_ctc''.
 ```shell ```shell
 $ ./densenet_ctc_picture -m 15 -p ../KhadasTeam.png $ ./densenet_ctc_picture -m 15 -p ../KhadasTeam.png
 ``` ```
Last modified: 2023/07/06 06:22 by louis

Page Tools

Khadas Site

Khadas Community

Khadas GitHub

Khadas Downloads