Guide to compiling the NPU application on the VIM3/3L.

Install the necessary library packages to build the OpenCV4 application.

$ sudo apt update
$ sudo apt install libopencv-dev python3-opencv

Get the source code: khadas/aml_npu_app

$ git clone https://github.com/khadas/aml_npu_app

Base directory of the cloned repository.

$ cd aml_npu_app
$ ls
DDK_6.3.2  DDK_6.3.2.3  DDK_6.3.2.5  DDK_6.3.3.4  DDK_6.4.0.3  DDK_6.4.3  detect_library  LICENSE  NN_SLT

Directory with various model specific application code.

$ cd aml_npu_app/detect_library
$ ls
densenet_ctc_demo_picture  inception  model_code  retinaface_demo_x11_usb  sample_demo_fb  sample_demo_x11  source_code  yolo_demo_fb_mipi  yolo_demo_fb_usb
yolo_demo_x11_mipi  yolo_demo_x11_usb  yolov7_tiny_demo_x11_usb  yolov8n_demo_x11_usb

Directory containing all the detection library code.

$ cd aml_npu_app/detect_library/model_code
$ ls
densenet_ctc  detect_mtcnn  detect_retinaface  detect_yoloface  detect_yolo_v2  detect_yolo_v3  detect_yolo_v3_tiny  detect_yolo_v4  detect_yolo_v7_tiny
detect_yolov8n  facenet

Except for detect_mtcnn, each directory represents a different model, and each model will be compiled into a library.

Take detect_yolo_v3 as an example to illustrate the structure of each directory.

$ cd aml_npu_app/detect_library/model_code/detect_yolo_v3
$ ls
build_vx.sh  include  Makefile  makefile.linux  makefile.linux.def  vnn_yolov3.c  yolo_v3.c  yolov3_process.c

Directory containing the application library source code.

$ cd aml_npu_app/detect_library/source_code
$ ls
build_vx.sh  detect.c  detect_log.c  include  Makefile  makefile.linux  makefile.linux.def

Enter the directory of the yolo_demo_x11_usb:

$ cd aml_npu_app/detect_library/yolo_demo_x11_usb
$ ls
build_vx.sh  detect.h  main.cpp  makefile.linux  makefile.linux.def  makefile.target_name  nn_detect_common.h  nn_detect.h  nn_detect_utils.h

1. Compile the model you want to use from the model_code directory. Here we shall take detect_yolo_v3 as an example.

$ cd aml_npu_app/detect_library/model_code/detect_yolo_v3
$ ls
build_vx.sh  include  Makefile  makefile.linux  makefile.linux.def  vnn_yolov3.c  yolo_v3.c  yolov3_process.c
$ bash build_vx.sh
$ ls
bin_r  build_vx.sh  include  linux_build_sample.log  makefile.linux  makefile.linux.def  makefile.target_name  vnn_yolov3.c  yolo_v3.c  yolov3_process.c

2. The compiled library libnn_yolo_v3.so is generated in bin_r folder. Add it to the library path variable.

$ export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$(pwd)/bin_r/libnn_yolo_v3.so

3. You will also need to build the contents of the source_code directory.

$ cd aml_npu_app/detect_library/source_code
$ ls
build_vx.sh  detect.c  detect_log.c  include  Makefile  makefile.linux  makefile.linux.def
$ bash build_vx.sh
$ ls
bin_r  build_vx.sh  detect.c  detect_log.c  include  Makefile  makefile.linux  makefile.linux.def

4. The compiled library libnn_yolo_v3.so is generated in bin_r folder. Add it to the library path variable.

$ export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$(pwd)/bin_r/libnn_detect.so

5. Now, Build the example application.

$ cd aml_npu_app/detect_library/yolo_demo_x11_usb
$ ls
build_vx.sh  detect.h  main.cpp  makefile.linux  makefile.linux.def  makefile.target_name  nn_detect_common.h  nn_detect.h  nn_detect_utils.h
$ bash build_vx.sh
$ ls
bin_r_cv4  build_vx.sh  detect.h  linux_build_sample.log  main.cpp  makefile.linux  makefile.linux.def  makefile.target_name  nn_detect_common.h  nn_detect.h  nn_detect_utils.h

LIBS += -L$(VIVANTE_SDK_LIB) -lOpenVX -lOpenVXU -lGAL -lovxlib -lArchModelSw -lNNArchPerf
 
LIBS += -L../source_code/bin_r -lnn_detect
 
-#LIBS +=-L$(LIB_DIR) -lstdc++
-LIBS += -lvpcodec -lamcodec -lamadec -lamvdec -lamavutils -lrt -lpthread -lge2d -lion
 
#############################################################################
# Macros.
PROGRAM = 1
CUR_SOURCE = ${wildcard *.c}
#############################################################################

6. Create a folder named nn_data under bin_r_cv4 and place the yolov3_88.nb model file in it.

$ cd bin_r_cv4
$ mkdir nn_data
$ cp -r xx/xx/yolov3_88.nb nn_data/
$ ls
detect_demo_x11_usb  main.o  nn_data

Now you can run inference on captured video data using the model.

$ ./detect_demo_x11_usb -m 2 -d /dev/video1

The parameter -m is for selecting the inference model, Here are the all the available models and their respective serial numbers.

For the above example, we used the serial number 2 which corresponds to the yolov3 model.

If you use the model that takes input as a picture, change -d to -p. Here is an example with densenet_ctc.

$ ./densenet_ctc_picture -m 15 -p ../KhadasTeam.png