The version information we are using is as follows:
rknn-llm: release-v1.2.3

Download DeepSeek-R1-Distill-Qwen-1.5B/7B from rkllm_model_zoo,fetch code: rkllm.

Download airockchip/rknn-llm into Edge2L.

$ git clone https://github.com/airockchip/rknn-llm.git -b release-v1.2.3

$ sudo apt update
$ sudo apt install cmake

Modify rknn-llm/examples/rkllm_api_demo/deploy/build-linux.sh as follows.

--- a/examples/rkllm_api_demo/deploy/build-linux.sh
+++ b/examples/rkllm_api_demo/deploy/build-linux.sh
@@ -5,7 +5,7 @@ if [[ -z ${BUILD_TYPE} ]];then
     BUILD_TYPE=Release
 fi

-GCC_COMPILER_PATH=~/opts/gcc-arm-10.2-2020.11-x86_64-aarch64-none-linux-gnu/bin/aarch64-none-linux-gnu
+GCC_COMPILER_PATH=/usr/bin/aarch64-linux-gnu
 C_COMPILER=${GCC_COMPILER_PATH}-gcc
 CXX_COMPILER=${GCC_COMPILER_PATH}-g++
 STRIP_COMPILER=${GCC_COMPILER_PATH}-strip
@@ -33,4 +33,4 @@ cmake ../.. \
     -DCMAKE_POSITION_INDEPENDENT_CODE=ON

Run build-linux.sh to compile.

$ ./build-linux.sh

Place the model file DeepSeek-R1-Distill-Qwen-1.5B_W4A16_RK3576.rkllm or DeepSeek-R1-Distill-Qwen-7B_W4A16_RK3576.rkllm into the demo_Linux_aarch64 directory.

$ cd install/demo_Linux_aarch64
$ export LD_LIBRARY_PATH=./lib
$ export RKLLM_LOG_LEVEL=1 # print infer speed
$ ./llm_demo ./DeepSeek-R1-Distill-Qwen-1.5B_W4A16_RK3576.rkllm 2048 4096