Khadas Docs

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
====== Large Model - DeepSeek-R1-Distill-Qwen-1.5B/7B ======

This guide introduces how to run the DeepSeek-R1-Distill-Qwen-1.5B and DeepSeek-R1-Distill-Qwen-7B large language models on the Edge2L SBC.\\

==== Download the models and code ====

The version information we are using is as follows: \\
OS：Ubuntu 24.04 \\
rknn-llm: release-v1.2.3 \\

Download DeepSeek-R1-Distill-Qwen-1.5B/7B from [[https://meta.box.lenovo.com/v/link/view/ad7482f6712844b48902f07287ed3359| rkllm_model_zoo]],fetch code: rkllm.


Download [[gh>airockchip/rknn-llm]] into Edge2L.

```shell
$ git clone https://github.com/airockchip/rknn-llm.git -b release-v1.2.3
```

==== Install dependencies ====

```
$ sudo apt update
$ sudo apt install cmake
```


==== Compile the code ====

Modify ''rknn-llm/examples/rkllm_api_demo/deploy/build-linux.sh'' as follows.
```
--- a/examples/rkllm_api_demo/deploy/build-linux.sh
+++ b/examples/rkllm_api_demo/deploy/build-linux.sh
@@ -5,7 +5,7 @@ if [[ -z ${BUILD_TYPE} ]];then
     BUILD_TYPE=Release
 fi

-GCC_COMPILER_PATH=~/opts/gcc-arm-10.2-2020.11-x86_64-aarch64-none-linux-gnu/bin/aarch64-none-linux-gnu
+GCC_COMPILER_PATH=/usr/bin/aarch64-linux-gnu
 C_COMPILER=${GCC_COMPILER_PATH}-gcc
 CXX_COMPILER=${GCC_COMPILER_PATH}-g++
 STRIP_COMPILER=${GCC_COMPILER_PATH}-strip
@@ -33,4 +33,4 @@ cmake ../.. \
     -DCMAKE_POSITION_INDEPENDENT_CODE=ON
```

Run ''build-linux.sh'' to compile.

```
$ ./build-linux.sh
```

==== Run the program ====

Place the model file DeepSeek-R1-Distill-Qwen-1.5B_W4A16_RK3576.rkllm or DeepSeek-R1-Distill-Qwen-7B_W4A16_RK3576.rkllm into the demo_Linux_aarch64 directory.

```shell
$ cd install/demo_Linux_aarch64
$ export LD_LIBRARY_PATH=./lib
$ export RKLLM_LOG_LEVEL=1
$ ./llm_demo ./DeepSeek-R1-Distill-Qwen-1.5B_W4A16_RK3576.rkllm 2048 4096
```

{{local:users:william:debug:edge2l:deepseek-r1-distill-qwen-1-5b-7b.webp?800|}}
Khadas Docs

User Tools

Site Tools

Page Tools
