This document describes how to compile and run the object detection demo with RTSP streaming on the Khadas VIM3 using its NPU (Neural Processing Unit).

Before building the project, ensure your system is up to date and the required dependencies are installed:

sudo apt update
sudo apt install cmake libopencv-dev ffmpeg gstreamer1.0-rtsp

Clone the official Khadas VIM3 NPU repository:

git clone https://github.com/khadas/vim3_npu_applications_lite.git
cd vim3_npu_applications_lite/object_detection+RTSP

Compile the project using the provided build script:

bash build_vx.sh

Before running the demo, ensure an RTSP server is running in your local network. You can use [https://github.com/bluenviron/mediamtx MediaMTX] or other RTSP servers.

wget https://github.com/bluenviron/mediamtx/releases/download/v1.11.3/mediamtx_v1.11.3_linux_amd64.tar.gz
tar -xzf mediamtx_v1.11.3_linux_amd64.tar.gz
cd mediamtx
./mediamtx

By default, MediaMTX listens on port 8554. Once started, it's ready to receive RTSP streams.

ffmpeg -re -stream_loop -1 -i input.mp4 -c:v copy -an -f rtsp rtsp://<RTSP_SERVER_IP>:8554/live/stream

Replace <RTSP_SERVER_IP> with the IP address of the machine running MediaMTX.

Navigate to the output directory and run the demo:

cd bin_r_cv4/
./object_detection+RTSP -d /dev/video50 -m ../nn_data/yolov8n_88.nb -r rtsp://192.168.31.19:8554/stream

/dev/video50                     # Camera device index (e.g., /dev/video50)
../nn_data/yolov8n_88.nb         # Path to model file
rtsp://<RTSP_SERVER_IP>:8554/... # RTSP server address

You may use one of the following RTSP servers:

• MediaMTX (recommended)
• FFmpeg (as RTSP server)
• VLC Streaming
• RTSP Simple Server

You can verify the RTSP stream using a player such as ffplay or VLC.

ffplay rtsp://<RTSP_SERVER_IP>:8554/live/stream

Open VLC and go to:

• Media → Open Network Stream
• Enter the URL
• Click Play

Replace <RTSP_SERVER_IP> with the IP address of the RTSP server (e.g., 192.168.31.19).

• Ensure the specified video device (e.g., /dev/video33) is connected and accessible.
• The RTSP server must be reachable from the VIM3 device (same LAN).
• Encoder settings (e.g., resolution, bitrate, format) can be modified in the source code.
• You can check your IP with: ip a or hostname -I