$ sudo apt update
$ sudo apt install libopencv-dev python3-opencv

$ mkdir{workspace} && cd {workspace}
$ git clone https://github.com/khadas/aml_npu_app

$ cd {workspace}/aml_npu_app
$ ls
DDK_6.3.2  DDK_6.3.2.3  DDK_6.3.2.5  DDK_6.3.3.4  DDK_6.4.0.3  DDK_6.4.3  detect_library  LICENSE  NN_SLT

• DDK_xxx - Different versions of library source code.
• detect_library - Application layer source code. detect_library/model_code link to the latest version of DDK.
• NN_SLT - Separate source code directory for DnCnn model (no longer maintained).
• LICENSE - LICENSE file.

$ cd {workspace}/aml_npu_app/detect_library
$ ls
densenet_ctc_demo_picture  inception  model_code  retinaface_demo_x11_usb  sample_demo_fb  sample_demo_x11  source_code  yolo_demo_fb_mipi  yolo_demo_fb_usb
yolo_demo_x11_mipi  yolo_demo_x11_usb  yolov7_tiny_demo_x11_usb  yolov8n_demo_x11_usb

• model_code - Load model and complete pre-processing and post-processing.
• source_code - Provide interfaces for demo to call model_code.
• Other folders - Load input and visualize the results.

Enter the directory of the library model_code, which is the directory of the DDK.

$ cd {workspace}/aml_npu_app/detect_library/model_code
$ ls
densenet_ctc  detect_mtcnn  detect_retinaface  detect_yoloface  detect_yolo_v2  detect_yolo_v3  detect_yolo_v3_tiny  detect_yolo_v4  detect_yolo_v7_tiny
detect_yolov8n  facenet

Except for detect_mtcnn, each directory represents a different model, and each model will be compiled into a library.

• densenet_ctc - Character recognition, used to identify characters. The input is only picture.
• detect_retinaface - Detect faces and the keypoints of faces.
• detect_yoloface - yoloface model, used to detect faces without keypoint.
• detect_yolo_v2 - yolov2 model, for object detection.
• detect_yolo_v3 - yolov3 model, for object detection.
• detect_yolo_v3_tiny - yolov3-tiny model, for object detection.
• detect_yolo_v4 - yolov4 model, used to detect faces.
• detect_yolo_v7_tiny - yolov7_tiny model, for object detection.
• detect_yolov8n - yolov8n model, for object detection. The latest yolo model.
• facenet - Has been deprecated.

Take detect_yolo_v3 as an example to illustrate the structure of each directory.

$ cd {workspace}/aml_npu_app/detect_library/model_code/detect_yolo_v3
$ ls
build_vx.sh  include  Makefile  makefile.linux  makefile.linux.def  vnn_yolov3.c  yolo_v3.c  yolov3_process.c

Main content description:

• build_vx.sh - Compile script.
• include - The corresponding header files and all definitions will be placed in this directory.
• Makefile - Makefile file.
• makefile.linux - Make environment configuration file.
• vnn_yolov3.c - SDK The converted model processing file is mainly used to interface with the nb file.
• yolo_v3.c - Specify the called nb file, and define all the interfaces of the model call.
• yolov3_process.c - Mainly defines the pre-processing and post-processing of the model.

Enter the directory of the source_code.

$ cd {workspace}/aml_npu_app/detect_library/source_code
$ ls
build_vx.sh  detect.c  detect_log.c  include  Makefile  makefile.linux  makefile.linux.def

Main content description:

• build_vx.sh - Compile script.
• detect.c - Provide interface for detect and call model_code.
• detect_log.c - About log level.
• include - The corresponding header files and all definitions will be placed in this directory.
• Makefile - Makefile file.
• makefile.linux - Make environment configuration file.

Enter the directory of the yolo_demo_x11_usb.

$ cd {workspace}/aml_npu_app/detect_library/yolo_demo_x11_usb
$ ls
build_vx.sh  detect.h  main.cpp  makefile.linux  makefile.linux.def  makefile.target_name  nn_detect_common.h  nn_detect.h  nn_detect_utils.h

• build_vx.sh - Compile script.
• main.cpp - Main function.
• detect.h, nn_detect_common.h, nn_detect.h, nn_detect_utils.h - The corresponding header files.
• makefile.linux - Make environment configuration file.
• makefile.target_name - The name of the compiled executable.

Compile model_code you need. Here still take detect_yolo_v3 as an example.

$ cd {workspace}/aml_npu_app/detect_library/model_code/detect_yolo_v3
$ ls
build_vx.sh  include  Makefile  makefile.linux  makefile.linux.def  vnn_yolov3.c  yolo_v3.c  yolov3_process.c
$ bash build_vx.sh
$ ls
bin_r  build_vx.sh  include  linux_build_sample.log  makefile.linux  makefile.linux.def  makefile.target_name  vnn_yolov3.c  yolo_v3.c  yolov3_process.c

The compiled library, libnn_yolo_v3.so, is generated in bin_r folder. Put the library into /usr/lib.

$ sudo cp -r bin_r/libnn_yolo_v3.so /usr/lib

Compile source_code and put library into /usr/lib, too.

$ cd {workspace}/aml_npu_app/detect_library/source_code
$ ls
build_vx.sh  detect.c  detect_log.c  include  Makefile  makefile.linux  makefile.linux.def
$ bash build_vx.sh
$ ls
bin_r  build_vx.sh  detect.c  detect_log.c  include  Makefile  makefile.linux  makefile.linux.def
$ sudo cp -r bin_r/libnn_detect.so /usr/lib

Compile main function.

$ cd {workspace}/aml_npu_app/detect_library/yolo_demo_x11_usb
$ ls
build_vx.sh  detect.h  main.cpp  makefile.linux  makefile.linux.def  makefile.target_name  nn_detect_common.h  nn_detect.h  nn_detect_utils.h
$ bash build_vx.sh
$ ls
bin_r_cv4  build_vx.sh  detect.h  linux_build_sample.log  main.cpp  makefile.linux  makefile.linux.def  makefile.target_name  nn_detect_common.h  nn_detect.h  nn_detect_utils.h

Build a folder named nn_data under bin_r_cv4 and put yolov3_88.nb into it.

$ cd bin_r_cv4
$ mkdir nn_data
$ cp -r {path}/yolov3_88.nb nn_data/
$ ls
detect_demo_x11_usb  main.o  nn_data

Run inference.

$ ./detect_demo_x11_usb -m 2 -d /dev/video1

The parameter -m means the inference model and 2 stands for yolov3. Here are the all available model and these serial number.

• 0 - detect_yoloface
• 1 - detect_yolo_v2
• 2 - detect_yolo_v3
• 3 - detect_yolo_v3_tiny
• 4 - detect_yolo_v4
• 13 - detect_yolo_v7_tiny
• 14 - detect_yolov8n
• 15 - densenet_ctc
• 16 - detect_retinaface

If you use the model that input is picture, change -d to -p. Here example is densenet_ctc.

$ ./densenet_ctc_picture -m 15 -p ../KhadasTeam.png