Khadas Docs

Amazing Khadas, always amazes you!

User Tools

Site Tools

Home / products / sbc / vim3 / npu / npu-app
products:sbc:vim3:npu:npu-app

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision 		Previous revision
Last revision Both sides next revision
products:sbc:vim3:npu:npu-app [2023/09/12 01:51]
sravan [Install OpenCV4] 		products:sbc:vim3:npu:npu-app [2024/01/22 20:31]
louis
Line 12: Line 12:
  
 ===== Get Source Code ===== ===== Get Source Code =====
 +Get the source code: [[gh>khadas/aml_npu_app]]
 ```shell ```shell
-$ mkdir{workspace} && cd {workspace} 
 $ git clone https://github.com/khadas/aml_npu_app $ git clone https://github.com/khadas/aml_npu_app
 ``` ```
  
 ===== Source Description ===== ===== Source Description =====
 +==== Base directory ==== 
 +Base directory of the cloned repository.
 ```shell ```shell
-$ cd {workspace}/aml_npu_app+$ cd aml_npu_app
 $ ls $ ls
 DDK_6.3.2  DDK_6.3.2.3  DDK_6.3.2.5  DDK_6.3.3.4  DDK_6.4.0.3  DDK_6.4.3  detect_library  LICENSE  NN_SLT DDK_6.3.2  DDK_6.3.2.3  DDK_6.3.2.5  DDK_6.3.3.4  DDK_6.4.0.3  DDK_6.4.3  detect_library  LICENSE  NN_SLT
 ``` ```
 +^ File ^ Description ^
 +| DDK_xx | Library source code, xx is the version code |
 +| detect_library | Application layer source code. the detect_library/model_code \\ is linked to the latest version of the DDK library |
 +| NN_SLT | Separate source code directory for DnCnn model (no longer maintained). |
 +| LICENSE | LICENSE file. |
 +                                                                                                |
  
-  * **DDK_xxx** - Different versions of library source code. +==== Detect library ==== 
-  * **detect_library** - Application layer source code. detect_library/model_code link to the latest version of DDK. +Directory with various model specific application code.
-  * **NN_SLT** - Separate source code directory for DnCnn model (no longer maintained). +
-  * **LICENSE** - LICENSE file. +
 ```shell ```shell
-$ cd {workspace}/aml_npu_app/detect_library+$ cd aml_npu_app/detect_library
 $ ls $ ls
 densenet_ctc_demo_picture  inception  model_code  retinaface_demo_x11_usb  sample_demo_fb  sample_demo_x11  source_code  yolo_demo_fb_mipi  yolo_demo_fb_usb densenet_ctc_demo_picture  inception  model_code  retinaface_demo_x11_usb  sample_demo_fb  sample_demo_x11  source_code  yolo_demo_fb_mipi  yolo_demo_fb_usb
Line 38: Line 41:
 ``` ```
  
-  * **model_code** - Load model and complete pre-processing and post-processing. +^ File            ^ Description ^ 
-  * **source_code** - Provide interfaces for demo to call model_code. +| **model_code**  | Load model and complete pre-processing and post-processing. | 
-  * **Other folders** - Load input and visualize the results.+| **source_code** | Provide interfaces for demo to call model_code. | 
 +| **Other folders** | Application source code |
  
-Enter the directory of the library **model_code**, which is the directory of the DDK.+==== Model code ==== 
 +Directory containing all the detection library code.
  
 ```shell ```shell
-$ cd {workspace}/aml_npu_app/detect_library/model_code+$ cd aml_npu_app/detect_library/model_code
 $ ls $ ls
 densenet_ctc  detect_mtcnn  detect_retinaface  detect_yoloface  detect_yolo_v2  detect_yolo_v3  detect_yolo_v3_tiny  detect_yolo_v4  detect_yolo_v7_tiny densenet_ctc  detect_mtcnn  detect_retinaface  detect_yoloface  detect_yolo_v2  detect_yolo_v3  detect_yolo_v3_tiny  detect_yolo_v4  detect_yolo_v7_tiny
Line 53: Line 58:
 Except for ''detect_mtcnn'', each directory represents a different model, and each model will be compiled into a library. Except for ''detect_mtcnn'', each directory represents a different model, and each model will be compiled into a library.
  
-  * **densenet_ctc** - Character recognition, used to identify characters. The input is only picture. +^ File ^ Description ^ 
-  * **detect_retinaface** - Detect faces and the keypoints of faces. +| **densenet_ctc** | Character recognition, used to identify characters. The input is only a picture. | 
-  * **detect_yoloface** - yoloface model, used to detect faces without keypoint. +| **detect_retinaface** | Detect faces and the keypoints of faces. | 
-  * **detect_yolo_v2** - yolov2 model, for object detection. +| **detect_yoloface** | yoloface model, used to detect faces without keypoints. | 
-  * **detect_yolo_v3** - yolov3 model, for object detection. +| **detect_yolo_v2** | yolov2 model, for object detection. | 
-  * **detect_yolo_v3_tiny** - yolov3-tiny model, for object detection. +| **detect_yolo_v3** | yolov3 model, for object detection. | 
-  * **detect_yolo_v4** - yolov4 model, used to detect faces. +| **detect_yolo_v3_tiny** | yolov3-tiny model, for object detection. | 
-  * **detect_yolo_v7_tiny** - yolov7_tiny model, for object detection. +| **detect_yolo_v4** | yolov4 model, used to detect faces. | 
-  * **detect_yolov8n** - yolov8n model, for object detection. The latest yolo model. +| **detect_yolo_v7_tiny** | yolov7_tiny model, for object detection. | 
-  * **facenet** - Has been deprecated.+| **detect_yolov8n** | yolov8n model, for object detection. The latest yolo model. | 
 +| **facenet** | Deprecated - No longer in use. | 
  
 Take ''detect_yolo_v3'' as an example to illustrate the structure of each directory. Take ''detect_yolo_v3'' as an example to illustrate the structure of each directory.
  
 ```shell ```shell
-$ cd {workspace}/aml_npu_app/detect_library/model_code/detect_yolo_v3+$ cd aml_npu_app/detect_library/model_code/detect_yolo_v3
 $ ls $ ls
 build_vx.sh  include  Makefile  makefile.linux  makefile.linux.def  vnn_yolov3.c  yolo_v3.c  yolov3_process.c build_vx.sh  include  Makefile  makefile.linux  makefile.linux.def  vnn_yolov3.c  yolo_v3.c  yolov3_process.c
 ``` ```
  
-Main content description:+^ File ^ Description ^ 
 +| **build_vx.sh** | Compile script. | 
 +| **include** | The corresponding header files and all definitions will be placed in this directory. | 
 +| **Makefile** | Makefile file. | 
 +| **makefile.linux** | Make environment configuration file. | 
 +| **vnn_yolov3.c** | SDK The converted model processing file is mainly used to interface with the nb file. | 
 +| **yolo_v3.c** | Specify the called nb file, and define all the interfaces of the model call. | 
 +| **yolov3_process.c** | Mainly defines the pre-processing and post-processing of the model. |
  
-  * **build_vx.sh** - Compile script. 
-  * **include** - The corresponding header files and all definitions will be placed in this directory. 
-  * **Makefile** - Makefile file. 
-  * **makefile.linux** - Make environment configuration file. 
-  * **vnn_yolov3.c** - SDK The converted model processing file is mainly used to interface with the nb file. 
-  * **yolo_v3.c** - Specify the called nb file, and define all the interfaces of the model call. 
-  * **yolov3_process.c** - Mainly defines the pre-processing and post-processing of the model. 
  
-Enter the directory of the **source_code**.+==== Source code ==== 
 +Directory containing the application library source code. 
  
 ```shell ```shell
-$ cd {workspace}/aml_npu_app/detect_library/source_code+$ cd aml_npu_app/detect_library/source_code
 $ ls $ ls
 build_vx.sh  detect.c  detect_log.c  include  Makefile  makefile.linux  makefile.linux.def build_vx.sh  detect.c  detect_log.c  include  Makefile  makefile.linux  makefile.linux.def
 ``` ```
  
-Main content description:+^ File ^ Description ^ 
 +| **build_vx.sh** | Compile script. | 
 +| **detect.c** | Provide interface for detect and call model_code. | 
 +| **detect_log.c** | About log level. | 
 +| **include** | The corresponding header files and all definitions will be placed in this directory. | 
 +| **Makefile** | Makefile file. | 
 +| **makefile.linux** | Make environment configuration file. |
  
-  * **build_vx.sh** - Compile script. 
-  * **detect.c** - Provide interface for detect and call model_code. 
-  * **detect_log.c** - About log level. 
-  * **include** - The corresponding header files and all definitions will be placed in this directory. 
-  * **Makefile** - Makefile file. 
-  * **makefile.linux** - Make environment configuration file. 
  
-Enter the directory of the **yolo_demo_x11_usb**.+Enter the directory of the ''yolo_demo_x11_usb'':
  
 ```shell ```shell
-$ cd {workspace}/aml_npu_app/detect_library/yolo_demo_x11_usb+$ cd aml_npu_app/detect_library/yolo_demo_x11_usb
 $ ls $ ls
 build_vx.sh  detect.h  main.cpp  makefile.linux  makefile.linux.def  makefile.target_name  nn_detect_common.h  nn_detect.h  nn_detect_utils.h build_vx.sh  detect.h  main.cpp  makefile.linux  makefile.linux.def  makefile.target_name  nn_detect_common.h  nn_detect.h  nn_detect_utils.h
 ``` ```
  
-  * **build_vx.sh** - Compile script. +^ File ^ Description ^ 
-  * **main.cpp** - Main function. +| **build_vx.sh** | Compile script. | 
-  * **detect.h**, **nn_detect_common.h**, **nn_detect.h**, **nn_detect_utils.h** - The corresponding header files. +| **main.cpp** | Main function. | 
-  * **makefile.linux** - Make environment configuration file. +| **detect.h**, **nn_detect_common.h**, **nn_detect.h**, **nn_detect_utils.h** | The corresponding header files. | 
-  * **makefile.target_name** - The name of the compiled executable.+| **makefile.linux** | Make environment configuration file. | 
 +| **makefile.target_name** | The name of the compiled executable. |
  
-===== Compile and inference =====+===== Compile the application=====
  
-Compile **model_code** you need. Here still take ''detect_yolo_v3'' as an example.+1. Compile the model you want to use from the ''model_code'' directory. Here we shall take ''detect_yolo_v3'' as an example.
  
 ```shell ```shell
-$ cd {workspace}/aml_npu_app/detect_library/model_code/detect_yolo_v3+$ cd aml_npu_app/detect_library/model_code/detect_yolo_v3
 $ ls $ ls
 build_vx.sh  include  Makefile  makefile.linux  makefile.linux.def  vnn_yolov3.c  yolo_v3.c  yolov3_process.c build_vx.sh  include  Makefile  makefile.linux  makefile.linux.def  vnn_yolov3.c  yolo_v3.c  yolov3_process.c
Line 126: Line 135:
 ``` ```
  
-The compiled library, ''libnn_yolo_v3.so'', is generated in bin_r folder. Put the library into /usr/lib.+2. The compiled library ''libnn_yolo_v3.so'' is generated in ''bin_r'' folder. Add it to the library path variable.
  
 ```shell ```shell
-$ sudo cp -r bin_r/libnn_yolo_v3.so /usr/lib+$ export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$(pwd)/bin_r/libnn_yolo_v3.so
 ``` ```
  
-Compile **source_code** and put library into /usr/lib, too.+3. You will also need to build the contents of the ''source_code'' directory.
 ```shell ```shell
-$ cd {workspace}/aml_npu_app/detect_library/source_code+$ cd aml_npu_app/detect_library/source_code
 $ ls $ ls
 build_vx.sh  detect.c  detect_log.c  include  Makefile  makefile.linux  makefile.linux.def build_vx.sh  detect.c  detect_log.c  include  Makefile  makefile.linux  makefile.linux.def
Line 140: Line 149:
 $ ls $ ls
 bin_r  build_vx.sh  detect.c  detect_log.c  include  Makefile  makefile.linux  makefile.linux.def bin_r  build_vx.sh  detect.c  detect_log.c  include  Makefile  makefile.linux  makefile.linux.def
-$ sudo cp -r bin_r/libnn_detect.so /usr/lib 
 ``` ```
  
-Compile main function.+4. The compiled library ''libnn_yolo_v3.so'' is generated in bin_r folder. Add it to the library path variable. 
 + 
 +```shell 
 +$ export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$(pwd)/bin_r/libnn_detect.so 
 +``` 
 + 
 +5. Now, Build the example application.
 ```shell ```shell
-$ cd {workspace}/aml_npu_app/detect_library/yolo_demo_x11_usb+$ cd aml_npu_app/detect_library/yolo_demo_x11_usb
 $ ls $ ls
 build_vx.sh  detect.h  main.cpp  makefile.linux  makefile.linux.def  makefile.target_name  nn_detect_common.h  nn_detect.h  nn_detect_utils.h build_vx.sh  detect.h  main.cpp  makefile.linux  makefile.linux.def  makefile.target_name  nn_detect_common.h  nn_detect.h  nn_detect_utils.h
Line 153: Line 167:
 ``` ```
  
-Build a folder named ''nn_data'' under ''bin_r_cv4'' and put ''yolov3_88.nb'' into it.+<WRAP important > 
 +If you use kernel are using version 5.15 or above, please modify ''makefile.linux'' as follows before running ''build_vx.sh''. 
 +</WRAP> 
 + 
 +```diff 
 +-#LIBS +=-L$(LIB_DIR) -lstdc++ 
 +-LIBS += -lvpcodec -lamcodec -lamadec -lamvdec -lamavutils -lrt -lpthread -lge2d -lion 
 +``` 
 + 
 +6. Create a folder named ''nn_data'' under ''bin_r_cv4'' and place the ''yolov3_88.nb'' model file in it.
 ```shell ```shell
 $ cd bin_r_cv4 $ cd bin_r_cv4
 $ mkdir nn_data $ mkdir nn_data
-$ cp -r {path}/yolov3_88.nb nn_data/+$ cp -r xx/xx/yolov3_88.nb nn_data/
 $ ls $ ls
 detect_demo_x11_usb  main.o  nn_data detect_demo_x11_usb  main.o  nn_data
 ``` ```
 +===== Run the application =====
  
-Run inference.+Now you can run inference on captured video data using the model.
 ```shell ```shell
 $ ./detect_demo_x11_usb -m 2 -d /dev/video1 $ ./detect_demo_x11_usb -m 2 -d /dev/video1
 ``` ```
 +==== Application setup parameters ====
 +=== Parameter to select the detection model ===
 +
 +The parameter ''-m'' is for selecting the inference model, Here are the all the available models and their respective serial numbers.
 +
 +^ Index ^ File ^
 +| **0** | detect_yoloface |
 +| **1** | detect_yolo_v2 |
 +| **2** | detect_yolo_v3 |
 +| **3** | detect_yolo_v3_tiny |
 +| **4** | detect_yolo_v4 |
 +| **13** | detect_yolo_v7_tiny |
 +| **14** | detect_yolov8n |
 +| **15** | densenet_ctc |
 +| **16** | detect_retinaface |
 +
 +
 +For the above example, we used the serial number ''2'' which corresponds to the ''yolov3'' model.
  
-The parameter ''-m'' means the inference model and ''2'' stands for yolov3. Here are the all available model and these serial number. +=== Parameter to set the input as picture ===
-  * **0** - detect_yoloface  +
-  * **1** - detect_yolo_v2  +
-  * **2** - detect_yolo_v3  +
-  * **3** - detect_yolo_v3_tiny  +
-  * **4** - detect_yolo_v4  +
-  * **13** - detect_yolo_v7_tiny  +
-  * **14** - detect_yolov8n  +
-  * **15** - densenet_ctc  +
-  * **16** - detect_retinaface+
  
-If you use the model that input is picture, change ''-d'' to ''-p''. Here example is densenet_ctc.+If you use the model that takes input as a picture, change ''-d'' to ''-p''. Here is an example with ''densenet_ctc''.
 ```shell ```shell
 $ ./densenet_ctc_picture -m 15 -p ../KhadasTeam.png $ ./densenet_ctc_picture -m 15 -p ../KhadasTeam.png
 ``` ```
Last modified: 2024/03/12 05:26 by louis

Page Tools

Khadas Site

Khadas Community

Khadas GitHub

Khadas Downloads