Download the PPOCR model from ppocr/model_list.md

Get source khadas/vim4_npu_sdk.

$ git lfs install
$ git lfs clone https://gitlab.com/khadas/vim4_npu_sdk.git
$ cd vim4_npu_sdk
$ ls
adla-toolkit-binary  adla-toolkit-binary-3.1.7.4  convert-in-docker.sh  Dockerfile  docs  README.md

• adla-toolkit-binary/docs - SDK documentations
• adla-toolkit-binary/bin - SDK tools required for model conversion
• adla-toolkit-binary/demo - Conversion examples

First, convert model from Paddle to ONNX. Please refer the two doc. Paddle2ONNX/blob/develop/README_en.md, PaddleOCR/blob/main/docs/ppocr/infer_deploy/paddle2onnx.en.md.

Our version is paddlepaddle==2.6.1 and paddle2onnx==1.2.11 and our covnert command.

$ paddle2onnx --model_dir ./ch_PP-OCRv4_det_infer --model_filename inference.pdmodel --params_filename inference.pdiparams --save_file ppocr_det.onnx
$ paddle2onnx --model_dir ./ch_PP-OCRv4_rec_infer --model_filename inference.pdmodel --params_filename inference.pdiparams --save_file ppocr_rec.onnx

Pull ppocr_det.onnx model and ppocr_rec model into vim4_npu_sdk/adla-toolkit-binary-3.1.7.4/python and modify ksnn_args.txt.

ppocr_det

--model-name ppocr_det 
--model-type onnx 
--model ./ppocr_det.onnx 
--inputs "x" 
--input-shapes  "3,736,736" 
--dtypes "float32" 
--quantize-dtype int8 
--outdir onnx_output 
--channel-mean-value "123.675,116.28,103.53,57.375" 
--source-file ocr_det_dataset.txt 
--iterations 1 
--batch-size 1 
--kboard VIM4 
--inference-input-type "float32" 
--inference-output-type "float32" 

ppocr_rec

--model-name ppocr_rec 
--model-type onnx 
--model ./ppocr_rec.onnx 
--inputs "x" 
--input-shapes  "3,48,320" 
--dtypes "float32" 
--quantize-dtype int8 
--outdir onnx_output 
--channel-mean-value "127.5,127.5,127.5,128" 
--source-file ocr_rec_dataset.txt 
--iterations 1 
--batch-size 1 
--kboard VIM4 
--inference-input-type "float32" 
--inference-output-type "float32" 

Download KSNN library and demo code. khadas/ksnn-vim4

$ git clone https://github.com/khadas/ksnn-vim4

If you use Ubuntu 24.04, demo must run in python virtual environment.

$ sudo apt update
$ sudo apt install python3-venv
$ python3 -m venv myenv
$ source myenv/bin/activate

$ cd ksnn-vim4/ksnn
$ sudo apt update
$ sudo apt install python3-pip
$ pip3 install ksnn_vim4-1.4.1-py3-none-any.whl
$ pip3 install shapely pyclipper Pillow

$ cd ksnn-vim4/example/ppocr
$ export QT_QPA_PLATFORM=xcb
$ python3 ppocr-picture.py --det_model ./models/VIM4/ppocr_det_int8.adla --det_library ./libs/libnn_ppocr_det.so --rec_model ./models/VIM4/ppocr_rec_int16.adla --rec_library ./libs/libnn_ppocr_rec.so --picture ./data/test.png

$ cd ksnn-vim4/example/ppocr
$ export QT_QPA_PLATFORM=xcb
$ python3 ppocr-cap.py --det_model ./models/VIM4/ppocr_det_int8.adla --det_library ./libs/libnn_ppocr_det.so --rec_model ./models/VIM4/ppocr_rec_int16.adla --rec_library ./libs/libnn_ppocr_rec.so --device 0

0 is the camera device index.