~~tag> VIM4 MIPI OS08A10 IMX415 ISP IP~~ {{indexmenu_n>52}} ====== New VIM4 MIPI camera usage ====== There are two [[../configurations/identify-version#vim4-versions-differences-reference-table| different VIM4 versions ]]: VIM4 and New VIM4. Both versions have different [[/tag/isp|ISP IP core]], so the usage of MIPI camera is also different. ===== Linux ===== ==== Upgrade system ==== Please follow the [[products:sbc:vim4:configurations:upgrade-system|documentation ]]to upgrade your system to latest version. ==== Enable overlays ==== From version **1.7.5**, cameras are auto detected by default, no need any setup, you can just skip the overlay setup. If you want to use the MIPI camera, you need to enable it in [[products:sbc:vim4:configurations:device-tree-overlay|overlays]].\\ [[gh>khadas/khadas-linux-kernel-dt-overlays/tree/main/overlays/vim4/5.4]] * Add ''vim4n-os08a10'' to ''fdt_overlays'' in ''/boot/dtb/amlogic/kvim4.dtb.overlay.env'' to enable ''OS08A10'' * Add ''vim4n-imx415'' to ''fdt_overlays'' in ''/boot/dtb/amlogic/kvim4.dtb.overlay.env'' to enable ''IMX415'' Enable ''OS08A10'': [[gh>/khadas/khadas-linux-kernel-dt-overlays/blob/main/overlays/vim4/5.4/vim4n-os08a10.dts]] ``` $ cat /boot/dtb/amlogic/kvim4.dtb.overlay.env fdt_overlays=vim4n-os08a10 ``` Enable ''IMX415'': [[gh>/khadas/khadas-linux-kernel-dt-overlays/blob/main/overlays/vim4/5.4/vim4n-imx415.dts]] ``` $ cat /boot/dtb/amlogic/kvim4.dtb.overlay.env fdt_overlays=vim4n-imx415 ``` And then reboot to take effect. ==== Video node ==== The video node is ''/dev/media0''. ==== Record video via gstreamer ==== Only for 5.15 kernel. Record video and perform ''H264'' encoding: ```shell gst-launch-1.0 -e v4l2src device=/dev/media0 io-mode=dmabuf num-buffers=300 ! video/x-raw,format=NV21,width=1920,height=1080,framerate=30/1 ! amlvenc bitrate=8000 ! h264parse ! qtmux ! filesink location=~/test-h264.mp4 ``` Record video and perform ''H265'' encoding: ```shell gst-launch-1.0 -e v4l2src device=/dev/media0 io-mode=dmabuf num-buffers=300 ! video/x-raw,format=NV21,width=1920,height=1080,framerate=30/1 ! amlvenc bitrate=8000 ! h265parse ! qtmux ! filesink location=~/test-h265.mp4 ``` ==== Get Frames via Mediactl API ==== Download and build the test source code: [[gh>numbqq/v4l2_media_test]] ```shell $ git clone https://github.com/numbqq/v4l2_media_test $ cd v4l2_media_test $ git checkout e0482794337c01935593a91b621f9c42123cf175 $ g++ v4l2_test_raw.cpp -o v4l2_test_raw -Wall -Wno-int-to-pointer-cast -Wno-pointer-to-int-cast -pthread -fPIE -lispaml -ldl -lmediaAPI -ltuning -lm -fPIC -D_FORTIFY_SOURCE=2 -O3 $ ls v4l2_test_raw v4l2_test_raw ``` ```shell $ git clone https://github.com/numbqq/v4l2_media_test $ cd v4l2_media_test $ g++ v4l2_test_raw.cpp -o v4l2_test_raw -Wall -Wno-int-to-pointer-cast -Wno-pointer-to-int-cast -pthread -fPIE -lispaml -ldl -lmediaAPI -ltuning -llens -lm -fPIC -D_FORTIFY_SOURCE=2 -O3 $ ls v4l2_test_raw v4l2_test_raw ``` Test: ```shell $ ./v4l2_test_raw -p 0 -n 10 -m /dev/media0 ``` It will capture 10 frames to file ''/tmp/dst_mif_0.yuv''. Preview the file: It doesn't work on VIM4, you need to copy the file and execute the command on your host PC. ```shell $ ffplay -f rawvideo -pixel_format nv12 -video_size 3840x2160 /tmp/dst_mif_0.yuv ``` ==== Preview video on ubuntu desktop ==== You can use gstreamer ''waylandsink'' to preview the video on Ubuntu Wayland desktop: ```shell gst-launch-1.0 v4l2src device=/dev/media0 io-mode=mmap ! video/x-raw,format=NV12,width=3840,height=2160,framerate=30/1 ! videoconvert ! waylandsink ``` You can use the command below to check display fps: ```shell gst-launch-1.0 -v v4l2src device=/dev/media0 io-mode=mmap ! video/x-raw,format=NV12,width=3840,height=2160,framerate=30/1 ! fpsdisplaysink video-sink=waylandsink sync=false text-overlay=false ``` You can use gstreamer ''clutterautovideosink'' to preview the video on Ubuntu Wayland desktop: ```shell gst-launch-1.0 v4l2src device=/dev/media0 io-mode=mmap ! video/x-raw,format=NV12,width=3840,height=2160,framerate=30/1 ! clutterautovideosink ``` You can use the command below to check display fps: ```shell gst-launch-1.0 -v v4l2src device=/dev/media0 io-mode=mmap ! video/x-raw,format=NV12,width=3840,height=2160,framerate=30/1 ! fpsdisplaysink video-sink=clutterautovideosink sync=false text-overlay=true ``` You press ''Alt + F7'' and then move mouse to move the windown. ===== Android ===== WIP: