Khadas Docs

Amazing Khadas, always amazes you!

User Tools

Site Tools

Home / products / sbc / vim3 / npu / npu-app
products:sbc:vim3:npu:npu-app

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision 		Previous revision
products:sbc:vim3:npu:npu-app [2023/09/12 02:25]
sravan [Source Description] 		products:sbc:vim3:npu:npu-app [2024/03/12 05:26] (current)
louis
Line 18: Line 18:
  
 ===== Source Description ===== ===== Source Description =====
 +==== Base directory ==== 
 +Base directory of the cloned repository.
 ```shell ```shell
 $ cd aml_npu_app $ cd aml_npu_app
Line 24: Line 25:
 DDK_6.3.2  DDK_6.3.2.3  DDK_6.3.2.5  DDK_6.3.3.4  DDK_6.4.0.3  DDK_6.4.3  detect_library  LICENSE  NN_SLT DDK_6.3.2  DDK_6.3.2.3  DDK_6.3.2.5  DDK_6.3.3.4  DDK_6.4.0.3  DDK_6.4.3  detect_library  LICENSE  NN_SLT
 ``` ```
 +^ File ^ Description ^
 +| DDK_xx | Library source code, xx is the version code |
 +| detect_library | Application layer source code. the detect_library/model_code \\ is linked to the latest version of the DDK library |
 +| NN_SLT | Separate source code directory for DnCnn model (no longer maintained). |
 +| LICENSE | LICENSE file. |
 +                                                                                                |
  
-  * **DDK_xxx** - Different versions of library source code. +==== Detect library ==== 
-  * **detect_library** - Application layer source code. detect_library/model_code link to the latest version of DDK. +Directory with various model specific application code.
-  * **NN_SLT** - Separate source code directory for DnCnn model (no longer maintained). +
-  * **LICENSE** - LICENSE file. +
 ```shell ```shell
 $ cd aml_npu_app/detect_library $ cd aml_npu_app/detect_library
Line 37: Line 41:
 ``` ```
  
-  * **model_code** - Load model and complete pre-processing and post-processing. +^ File            ^ Description ^ 
-  * **source_code** - Provide interfaces for demo to call model_code. +| **model_code**  | Load model and complete pre-processing and post-processing. | 
-  * **Other folders** - Load input and visualize the results.+| **source_code** | Provide interfaces for demo to call model_code. | 
 +| **Other folders** | Application source code |
  
-Enter the directory of the library **model_code**, which is the directory of the DDK.+==== Model code ==== 
 +Directory containing all the detection library code.
  
 ```shell ```shell
Line 52: Line 58:
 Except for ''detect_mtcnn'', each directory represents a different model, and each model will be compiled into a library. Except for ''detect_mtcnn'', each directory represents a different model, and each model will be compiled into a library.
  
-  * **densenet_ctc** - Character recognition, used to identify characters. The input is only picture. +^ File ^ Description ^ 
-  * **detect_retinaface** - Detect faces and the keypoints of faces. +| **densenet_ctc** | Character recognition, used to identify characters. The input is only a picture. | 
-  * **detect_yoloface** - yoloface model, used to detect faces without keypoint. +| **detect_retinaface** | Detect faces and the keypoints of faces. | 
-  * **detect_yolo_v2** - yolov2 model, for object detection. +| **detect_yoloface** | yoloface model, used to detect faces without keypoints. | 
-  * **detect_yolo_v3** - yolov3 model, for object detection. +| **detect_yolo_v2** | yolov2 model, for object detection. | 
-  * **detect_yolo_v3_tiny** - yolov3-tiny model, for object detection. +| **detect_yolo_v3** | yolov3 model, for object detection. | 
-  * **detect_yolo_v4** - yolov4 model, used to detect faces. +| **detect_yolo_v3_tiny** | yolov3-tiny model, for object detection. | 
-  * **detect_yolo_v7_tiny** - yolov7_tiny model, for object detection. +| **detect_yolo_v4** | yolov4 model, used to detect faces. | 
-  * **detect_yolov8n** - yolov8n model, for object detection. The latest yolo model. +| **detect_yolo_v7_tiny** | yolov7_tiny model, for object detection. | 
-  * **facenet** - Has been deprecated.+| **detect_yolov8n** | yolov8n model, for object detection. The latest yolo model. | 
 +| **facenet** | Deprecated - No longer in use. | 
  
 Take ''detect_yolo_v3'' as an example to illustrate the structure of each directory. Take ''detect_yolo_v3'' as an example to illustrate the structure of each directory.
Line 71: Line 79:
 ``` ```
  
-Main content description:+^ File ^ Description ^ 
 +| **build_vx.sh** | Compile script. | 
 +| **include** | The corresponding header files and all definitions will be placed in this directory. | 
 +| **Makefile** | Makefile file. | 
 +| **makefile.linux** | Make environment configuration file. | 
 +| **vnn_yolov3.c** | SDK The converted model processing file is mainly used to interface with the nb file. | 
 +| **yolo_v3.c** | Specify the called nb file, and define all the interfaces of the model call. | 
 +| **yolov3_process.c** | Mainly defines the pre-processing and post-processing of the model. |
  
-  * **build_vx.sh** - Compile script. 
-  * **include** - The corresponding header files and all definitions will be placed in this directory. 
-  * **Makefile** - Makefile file. 
-  * **makefile.linux** - Make environment configuration file. 
-  * **vnn_yolov3.c** - SDK The converted model processing file is mainly used to interface with the nb file. 
-  * **yolo_v3.c** - Specify the called nb file, and define all the interfaces of the model call. 
-  * **yolov3_process.c** - Mainly defines the pre-processing and post-processing of the model. 
  
-Enter the directory of the **source_code**.+==== Source code ==== 
 +Directory containing the application library source code. 
  
 ```shell ```shell
Line 89: Line 98:
 ``` ```
  
-Main content description:+^ File ^ Description ^ 
 +| **build_vx.sh** | Compile script. | 
 +| **detect.c** | Provide interface for detect and call model_code. | 
 +| **detect_log.c** | About log level. | 
 +| **include** | The corresponding header files and all definitions will be placed in this directory. | 
 +| **Makefile** | Makefile file. | 
 +| **makefile.linux** | Make environment configuration file. |
  
-  * **build_vx.sh** - Compile script. 
-  * **detect.c** - Provide interface for detect and call model_code. 
-  * **detect_log.c** - About log level. 
-  * **include** - The corresponding header files and all definitions will be placed in this directory. 
-  * **Makefile** - Makefile file. 
-  * **makefile.linux** - Make environment configuration file. 
  
-Enter the directory of the **yolo_demo_x11_usb**.+Enter the directory of the ''yolo_demo_x11_usb'':
  
 ```shell ```shell
Line 106: Line 115:
 ``` ```
  
-  * **build_vx.sh** - Compile script. +^ File ^ Description ^ 
-  * **main.cpp** - Main function. +| **build_vx.sh** | Compile script. | 
-  * **detect.h**, **nn_detect_common.h**, **nn_detect.h**, **nn_detect_utils.h** - The corresponding header files. +| **main.cpp** | Main function. | 
-  * **makefile.linux** - Make environment configuration file. +| **detect.h**, **nn_detect_common.h**, **nn_detect.h**, **nn_detect_utils.h** | The corresponding header files. | 
-  * **makefile.target_name** - The name of the compiled executable.+| **makefile.linux** | Make environment configuration file. | 
 +| **makefile.target_name** | The name of the compiled executable. |
  
-===== Compile and inference =====+===== Compile the application=====
  
 1. Compile the model you want to use from the ''model_code'' directory. Here we shall take ''detect_yolo_v3'' as an example. 1. Compile the model you want to use from the ''model_code'' directory. Here we shall take ''detect_yolo_v3'' as an example.
Line 125: Line 135:
 ``` ```
  
-2. The compiled library ''libnn_yolo_v3.so'' is generated in bin_r folder. Move the library file into the ''/usr/lib'' directory of the system.+2. The compiled library ''libnn_yolo_v3.so'' is generated in ''bin_r'' folder. Add it to the library path variable.
  
 ```shell ```shell
-$ sudo cp -r bin_r/libnn_yolo_v3.so /usr/lib+$ export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$(pwd)/bin_r/libnn_yolo_v3.so
 ``` ```
  
-3. You will also need to build the contents of the ''source_code'' directory and move the library file into ''/usr/lib'' as well.+3. You will also need to build the contents of the ''source_code'' directory.
 ```shell ```shell
 $ cd aml_npu_app/detect_library/source_code $ cd aml_npu_app/detect_library/source_code
Line 139: Line 149:
 $ ls $ ls
 bin_r  build_vx.sh  detect.c  detect_log.c  include  Makefile  makefile.linux  makefile.linux.def bin_r  build_vx.sh  detect.c  detect_log.c  include  Makefile  makefile.linux  makefile.linux.def
-$ sudo cp -r bin_r/libnn_detect.so /usr/lib 
 ``` ```
  
-4. Now, Build the example application+4. The compiled library ''libnn_yolo_v3.so'' is generated in bin_r folder. Add it to the library path variable. 
 + 
 +```shell 
 +$ export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$(pwd)/bin_r/libnn_detect.so 
 +``` 
 + 
 +5. Now, Build the example application.
 ```shell ```shell
 $ cd aml_npu_app/detect_library/yolo_demo_x11_usb $ cd aml_npu_app/detect_library/yolo_demo_x11_usb
Line 152: Line 167:
 ``` ```
  
-5. Create a folder named ''nn_data'' under ''bin_r_cv4'' and place the ''yolov3_88.nb'' model file in it.+<WRAP important > 
 +If you use kernel are using version 5.15 or above, please remove the red lines in ''makefile.linux'' before running ''build_vx.sh''. 
 +</WRAP> 
 + 
 +```diff 
 +LIBS += -L$(VIVANTE_SDK_LIB) -lOpenVX -lOpenVXU -lGAL -lovxlib -lArchModelSw -lNNArchPerf 
 + 
 +LIBS += -L../source_code/bin_r -lnn_detect 
 + 
 +-#LIBS +=-L$(LIB_DIR) -lstdc++ 
 +-LIBS += -lvpcodec -lamcodec -lamadec -lamvdec -lamavutils -lrt -lpthread -lge2d -lion 
 + 
 +############################################################################# 
 +# Macros. 
 +PROGRAM = 1 
 +CUR_SOURCE = ${wildcard *.c} 
 +############################################################################# 
 +``` 
 + 
 +6. Create a folder named ''nn_data'' under ''bin_r_cv4'' and place the ''yolov3_88.nb'' model file in it.
 ```shell ```shell
 $ cd bin_r_cv4 $ cd bin_r_cv4
Line 160: Line 194:
 detect_demo_x11_usb  main.o  nn_data detect_demo_x11_usb  main.o  nn_data
 ``` ```
 +===== Run the application =====
  
-6. Finally you can run inference on captured video data using the model.+Now you can run inference on captured video data using the model.
 ```shell ```shell
 $ ./detect_demo_x11_usb -m 2 -d /dev/video1 $ ./detect_demo_x11_usb -m 2 -d /dev/video1
 ``` ```
-==== Run-time parameters ==== +==== Application setup parameters ==== 
-=== Selecting the model ===+=== Parameter to select the detection model ===
  
 The parameter ''-m'' is for selecting the inference model, Here are the all the available models and their respective serial numbers. The parameter ''-m'' is for selecting the inference model, Here are the all the available models and their respective serial numbers.
-  * **0** - detect_yoloface  
-  * **1** - detect_yolo_v2  
-  * **2** - detect_yolo_v3  
-  * **3** - detect_yolo_v3_tiny  
-  * **4** - detect_yolo_v4  
-  * **13** - detect_yolo_v7_tiny  
-  * **14** - detect_yolov8n  
-  * **15** - densenet_ctc  
-  * **16** - detect_retinaface 
  
-For the above example, we used the serial number ''2'' which corresponds to the yolov3 model.+^ Index ^ File ^ 
 +| **0** | detect_yoloface | 
 +| **1** | detect_yolo_v2 | 
 +| **2** | detect_yolo_v3 | 
 +| **3** | detect_yolo_v3_tiny | 
 +| **4** | detect_yolo_v4 | 
 +| **13** | detect_yolo_v7_tiny | 
 +| **14** | detect_yolov8n | 
 +| **15** | densenet_ctc | 
 +| **16** | detect_retinaface | 
 + 
 + 
 +For the above example, we used the serial number ''2'' which corresponds to the ''yolov3'' model.
  
-=== Providing picture as input ===+=== Parameter to set the input as picture ===
  
-If you use the model that takes input as a picture, change ''-d'' to ''-p''. Here is an example with densenet_ctc.+If you use the model that takes input as a picture, change ''-d'' to ''-p''. Here is an example with ''densenet_ctc''.
 ```shell ```shell
 $ ./densenet_ctc_picture -m 15 -p ../KhadasTeam.png $ ./densenet_ctc_picture -m 15 -p ../KhadasTeam.png
 ``` ```
Last modified: 2023/09/12 02:25 by sravan

Page Tools

Khadas Site

Khadas Community

Khadas GitHub

Khadas Downloads