Table of Contents

New VIM4 MIPI camera usage

There are two different VIM4 versions : VIM4 and New VIM4. Both versions have different ISP IP core, so the usage of MIPI camera is also different.

Linux

Upgrade system

Please follow the documentation to upgrade your system to latest version.

Enable overlays

If you want to use the MIPI camera, you need to enable it in overlays.
khadas/khadas-linux-kernel-dt-overlays/tree/main/overlays/vim4/5.4

Enable OS08A10: /khadas/khadas-linux-kernel-dt-overlays/blob/main/overlays/vim4/5.4/vim4n-os08a10.dts 

$ cat /boot/dtb/amlogic/kvim4.dtb.overlay.env 
fdt_overlays=vim4n-os08a10

Enable IMX415: /khadas/khadas-linux-kernel-dt-overlays/blob/main/overlays/vim4/5.4/vim4n-imx415.dts 

$ cat /boot/dtb/amlogic/kvim4.dtb.overlay.env 
fdt_overlays=vim4n-imx415

And then reboot to take effect.

Video node

The video node is /dev/media0.

Record video via gstreamer

Only for 5.15 kernel.

Record video and perform H264 encoding: 

gst-launch-1.0 -e v4l2src device=/dev/media0  io-mode=mmap num-buffers=300 ! video/x-raw,format=NV21,width=1920,height=1080,framerate=30/1 ! amlvenc bitrate=8000 ! h264parse ! qtmux ! filesink location=~/test-h264.mp4

Record video and perform H265 encoding: 

gst-launch-1.0 -e v4l2src device=/dev/media0  io-mode=mmap num-buffers=300 ! video/x-raw,format=NV21,width=1920,height=1080,framerate=30/1 ! amlvenc bitrate=8000 ! h265parse ! qtmux ! filesink location=~/test-h265.mp4

Get Frames via Mediactl API

Download and build the test source code: numbqq/v4l2_media_test

5.4 kernel

$ git clone https://github.com/numbqq/v4l2_media_test
$ cd v4l2_media_test
$ git checkout e0482794337c01935593a91b621f9c42123cf175
$ g++ v4l2_test_raw.cpp -o v4l2_test_raw  -Wall -Wno-int-to-pointer-cast -Wno-pointer-to-int-cast -pthread -fPIE -lispaml -ldl -lmediaAPI -ltuning -lm -fPIC -D_FORTIFY_SOURCE=2 -O3
$ ls v4l2_test_raw
v4l2_test_raw

5.15 kernel

$ git clone https://github.com/numbqq/v4l2_media_test
$ cd v4l2_media_test
$ g++ v4l2_test_raw.cpp -o v4l2_test_raw  -Wall -Wno-int-to-pointer-cast -Wno-pointer-to-int-cast -pthread -fPIE -lispaml -ldl -lmediaAPI -ltuning -llens -lm -fPIC -D_FORTIFY_SOURCE=2 -O3
$ ls v4l2_test_raw
v4l2_test_raw

Test: 

$ ./v4l2_test_raw -p 0 -n 10 -m /dev/media0

It will capture 10 frames to file /tmp/dst_mif_0.yuv.

Preview the file:

It doesn't work on VIM4, you need to copy the file and execute the command on your host PC. 

$ ffplay -f rawvideo -pixel_format nv12 -video_size 3840x2160 /tmp/dst_mif_0.yuv

Preview video on ubuntu desktop

Ubuntu 24.04

You can use gstreamer waylandsink to preview the video on Ubuntu Wayland desktop: 

gst-launch-1.0 v4l2src device=/dev/media0 io-mode=mmap  ! video/x-raw,format=NV12,width=3840,height=2160,framerate=30/1 ! videoconvert ! waylandsink

You can use the command below to check display fps: 

gst-launch-1.0 -v v4l2src device=/dev/media0 io-mode=mmap ! video/x-raw,format=NV12,width=3840,height=2160,framerate=30/1 ! fpsdisplaysink video-sink=waylandsink sync=false text-overlay=false

Ubuntu 22.04

You can use gstreamer clutterautovideosink to preview the video on Ubuntu Wayland desktop: 

gst-launch-1.0 v4l2src device=/dev/media0 io-mode=mmap ! video/x-raw,format=NV12,width=3840,height=2160,framerate=30/1 ! clutterautovideosink

You can use the command below to check display fps: 

gst-launch-1.0 -v v4l2src device=/dev/media0 io-mode=mmap ! video/x-raw,format=NV12,width=3840,height=2160,framerate=30/1 ! fpsdisplaysink video-sink=clutterautovideosink sync=false text-overlay=true

You press Alt + F7 and then move mouse to move the windown.

Android

WIP: