1、得到训练好的PT模型并利用工具转化为ONNX

best.pt

注意：你的PT文件，一定是yolov8训练出来的

下载瑞芯微官网的yolov8转化工具：

官网：

https://github.com/airockchip/ultralytics_yolov8

压缩包：

ultralytics_yolov8-main.zip

解压到指定目录：

用pycharm打开该项目并将pt模型放入项目根目录：

在根目录新建一个.py文件，内容为：

from ultralytics import YOLO


if __name__ == '__main__':
    model = YOLO("best.pt")
    model.export(format='rknn')

运行代码后会生成animal_best.onnx

2、测试转化后的onnx模型结构是否合理

测试onnx模型结构网站：

https://netron.app/

效果如图：

PS：上图的三个检测头红色方框非常重要，YOLOv8在RKNN转换过程中，是要有三个分支的，分别80×80检测头，40×40检测头，20×20检测头。每个检测头下，都有三个输出，分别是box边框输出，即图上的1×64×80×80；还有各类别输出，即1×5×80×80（这里的5就是指我在PT模型训练时的animal.yaml文件中定义了5个类别），最后是综合置信度输出，即1×1×80×80。（其余检测头同理）

3、将检测合格后的onnx模型导入到虚拟机的对应环境进行转化，环境搭建教程如下：

3-1、RKNN-Toolkit2 安装（ubuntu22.04 X86_64平台）

本地PC安装【虚拟机上】

环境要求

操作系统版本	Ubuntu18.04(x64)	Ubuntu20.04(x64)	Ubuntu22.04(x64)
Python版本	3.6/3.7	3.8/3.9	3.10/3.11

下载rknn-toolkit2

git clone https://github.com/airockchip/rknn-toolkit2.git

安装python环境

sudo apt-get update
sudo apt-get install python3 python3-dev python3-pip
sudo apt-get install libxslt1-dev zlib1g zlib1g-dev libglib2.0-0 libsm6 libgl1-mesa-glx libprotobuf-dev gcc

安装RKNN-ToolKit2依赖包

pip3 install -i https://mirrors.aliyun.com/pypi/simple/ -r rknn-toolkit2/packages/x86_64/requirements_cpxx-2.2.0.txt

# such as: 
pip3 install -i https://mirrors.aliyun.com/pypi/simple/ -r rknn-toolkit2/packages/x86_64/requirements_cp310-2.2.0.txt

根据不同的Python版本，选择安装对应的依赖包：

Python版本	RKNN-Toolkit2依赖包
3.6	requirements_cp36-2.2.0.txt
3.7	requirements_cp37-2.2.0.txt
3.8	requirements_cp38-2.2.0.txt
3.9	requirements_cp39-2.2.0.txt
3.10	requirements_cp310-2.2.0.txt
3.11	requirements_cp311-2.2.0.txt
3.12	requirements_cp312-2.2.0.txt

安装RKNN-ToolKit2

pip3 install -i https://mirrors.aliyun.com/pypi/simple/ rknn-toolkit2/packages/x86_64/rknn_toolkit2-x.x.x+xxxxxxxx-cpxx-cpxx-linux_x86_64.whl

# such as: 
pip3 install -i https://mirrors.aliyun.com/pypi/simple/ rknn-toolkit2/packages/x86_64/rknn_toolkit2-2.2.0+81f21f4d-cp310-cp310-linux_x86_64.whl

包名格式为：rknn_toolkit2-{版本号}+{commit 号}-cp{Python 版本}-cp{Python 版本}-linux_x86_64.whl，根据不同的Python版本，选择安装对应的安装包：

Python版本	RKNN-Toolkit2安装包
3.6	rknn_toolkit2-2.2.0-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
3.7	rknn_toolkit2-2.2.0-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
3.8	rknn_toolkit2-2.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
3.9	rknn_toolkit2-2.2.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
3.10	rknn_toolkit2-2.2.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
3.11	rknn_toolkit2-2.2.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
3.12	rknn_toolkit2-2.2.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl

若执行以下命令没有报错，则安装成功：

python3
from rknn.api import RKNN

3-2、下载官方rknn_model_zoo 应用示例

rknn_model_zoo 是瑞芯微官方提供的 RKNPU 支持的各种主流算法的部署示例，最新的示例支持 mobilenet 模型部署和 yolo 模型部署，本章以部署 yolov8 为例介绍 rknn_model_zoo 示例的使用。

3-2.1 导出 RKNN 模型

下载 rknn_model_zoo

git clone https://github.com/airockchip/rknn_model_zoo.git

获取 Yolov8 ONNX 模型文件。

cd /rknn_model_zoo/examples/yolov8/model
chmod a+x download_model.sh
./download_model.sh

执行 rknn_model_zoo/examples/yolov8/python 目录下的模型转换程序 convert.py，使用方法：

cd /rknn_model_zoo/examples/yolov8/python
 python3 convert.py ../model/yolov8n.onnx rk3576
# output model will be saved as ../model/yolov8.rknn

python3 convert.py <onnx_model> <TARGET_PLATFORM> <dtype(optional)> <output_rknn_path(optional)>

参数介绍：

<onnx_model>：ONNX 模型路径
<TARGET_PLATFORM>：指定 NPU 平台名，这里指rk3576
<quant_dtype> ：可选项，可以指定为i8或fp。i8表示进行量化，fp表示不量化，默认为i8
<output_rknn_path>：可选项，用于指定 RKNN 模型的保存路径，默认保存在 ONNX 模型同一目录下，名称为yolov8.rknn

检测转化后的RKNN模型输入输出结构是否符合标准

测试RKNN模型结构网站：

https://netron.app/

效果如图：

一标准输入、九输出：

3-2.2 编译和构建【C++】可执行程序，方便测试你的模型

先去ARM开发平台，下载GCC编译器

https://developer.arm.com/downloads/-/gnu-a

成功将 ONNX 模型转换成 RKNN 模型后，现在对 rknn_model_zoo/examples/yolov8 目录下的例程进行交叉编译，编译例程前需要设置如下环境变量：

export GCC_COMPILER=/home/thingskit/tools/gcc-arm-10.3-2021.07-x86_64-aarch64-none-linux-gnu/bin/aarch64-none-linux-gnu

执行 rknn_model_zoo 目录下的 build-linux.sh 脚本。该脚本将编译例程：

chmod +x ./build-linux.sh
./build-linux.sh -t rk3576 -a aarch64 -d yolov8

如果编译过程中：cmake：未找到命令

sudo apt install cmake
cmake -version

编译过程：

thingskit@thingskit-virtual-machine:~/tools/download/rknn_model_zoo$ ./build-linux.sh -t rk3576 -a aarch64 -d yolov8
./build-linux.sh -t rk3576 -a aarch64 -d yolov8
/home/thingskit/tools/gcc-arm-10.3-2021.07-x86_64-aarch64-none-linux-gnu/bin/aarch64-none-linux-gnu
===================================
BUILD_DEMO_NAME=yolov8
BUILD_DEMO_PATH=examples/yolov8/cpp
TARGET_SOC=rk3576
TARGET_ARCH=aarch64
BUILD_TYPE=Release
ENABLE_ASAN=OFF
DISABLE_RGA=OFF
DISABLE_LIBJPEG=OFF
INSTALL_DIR=/home/thingskit/tools/download/rknn_model_zoo/install/rk3576_linux_aarch64/rknn_yolov8_demo
BUILD_DIR=/home/thingskit/tools/download/rknn_model_zoo/build/build_rknn_yolov8_demo_rk3576_linux_aarch64_Release
CC=/home/thingskit/tools/gcc-arm-10.3-2021.07-x86_64-aarch64-none-linux-gnu/bin/aarch64-none-linux-gnu-gcc
CXX=/home/thingskit/tools/gcc-arm-10.3-2021.07-x86_64-aarch64-none-linux-gnu/bin/aarch64-none-linux-gnu-g++
===================================
-- The C compiler identification is GNU 10.3.1
-- The CXX compiler identification is GNU 10.3.1
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /home/thingskit/tools/gcc-arm-10.3-2021.07-x86_64-aarch64-none-linux-gnu/bin/aarch64-none-linux-gnu-gcc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /home/thingskit/tools/gcc-arm-10.3-2021.07-x86_64-aarch64-none-linux-gnu/bin/aarch64-none-linux-gnu-g++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- !!!!!!!!!!!CMAKE_SYSTEM_NAME: Linux
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
-- Check if compiler accepts -pthread
-- Check if compiler accepts -pthread - yes
-- Found Threads: TRUE  
-- Configuring done
-- Generating done
-- Build files have been written to: /home/thingskit/tools/download/rknn_model_zoo/build/build_rknn_yolov8_demo_rk3576_linux_aarch64_Release
[ 12%] Building C object utils.out/CMakeFiles/imagedrawing.dir/image_drawing.c.o
[ 12%] Building C object utils.out/CMakeFiles/imageutils.dir/image_utils.c.o
[ 18%] Building C object utils.out/CMakeFiles/fileutils.dir/file_utils.c.o
[ 25%] Building C object utils.out/CMakeFiles/audioutils.dir/audio_utils.c.o
…………
Consolidate compiler generated dependencies of target imageutils
[ 37%] Built target imageutils
Consolidate compiler generated dependencies of target rknn_yolov8_demo
[ 62%] Built target rknn_yolov8_demo
Consolidate compiler generated dependencies of target rknn_yolov8_demo_zero_copy
[ 87%] Built target rknn_yolov8_demo_zero_copy
Consolidate compiler generated dependencies of target audioutils
[100%] Built target audioutils
Install the project...
-- Install configuration: "Release"
-- Installing: /home/thingskit/tools/download/rknn_model_zoo/install/rk3576_linux_aarch64/rknn_yolov8_demo/./rknn_yolov8_demo_zero_copy
-- Set runtime path of "/home/thingskit/tools/download/rknn_model_zoo/install/rk3576_linux_aarch64/rknn_yolov8_demo/./rknn_yolov8_demo_zero_copy" to "$ORIGIN/../lib"
-- Installing: /home/thingskit/tools/download/rknn_model_zoo/install/rk3576_linux_aarch64/rknn_yolov8_demo/./rknn_yolov8_demo
-- Set runtime path of "/home/thingskit/tools/download/rknn_model_zoo/install/rk3576_linux_aarch64/rknn_yolov8_demo/./rknn_yolov8_demo" to "$ORIGIN/../lib"
-- Installing: /home/thingskit/tools/download/rknn_model_zoo/install/rk3576_linux_aarch64/rknn_yolov8_demo/model/bus.jpg
-- Installing: /home/thingskit/tools/download/rknn_model_zoo/install/rk3576_linux_aarch64/rknn_yolov8_demo/model/coco_80_labels_list.txt
-- Installing: /home/thingskit/tools/download/rknn_model_zoo/install/rk3576_linux_aarch64/rknn_yolov8_demo/model/yolov8.rknn
-- Installing: /home/thingskit/tools/download/rknn_model_zoo/install/rk3576_linux_aarch64/rknn_yolov8_demo/lib/librknnrt.so
-- Installing: /home/thingskit/tools/download/rknn_model_zoo/install/rk3576_linux_aarch64/rknn_yolov8_demo/lib/librga.so

交叉编译完成后在 rknn_model_zoo 目录下会生成一个 install 目录，包含编译出来的程序和库文件。

thingskit@thingskit-virtual-machine:~/tools/download/rknn_model_zoo$ cd install/rk3576_linux_aarch64/rknn_yolo11_demo/
thingskit@thingskit-virtual-machine:~/tools/download/rknn_model_zoo/install/rk3576_linux_aarch64/rknn_yolo11_demo$ ll
总计 1960
drwxrwxr-x 4 thingskit thingskit   4096  2月 28 17:05 ./
drwxrwxr-x 3 thingskit thingskit   4096  2月 28 17:05 ../
drwxrwxr-x 2 thingskit thingskit   4096  2月 28 17:05 lib/
drwxrwxr-x 2 thingskit thingskit   4096  2月 28 17:05 model/
-rwxr-xr-x 1 thingskit thingskit 991616  2月 28 17:05 rknn_yolov8_demo*
-rwxr-xr-x 1 thingskit thingskit 991720  2月 28 17:05 rknn_yolov8_demo_zero_copy*

运行环境已经就绪，创建压缩包【rknn_yolov8_demo.tar.gz】

tar -czvf rknn_yolov8_demo.tar.gz rknn_yolov8_demo/

识别测试包已经准备完毕，可以放在3576开发板上运行，测试。

cp rknn_yolov8_demo.tar.gz /mnt/hgfs/VM-SharedFolders/

3-2.3 运行程序【yolov8】

先将整个 rknn_yolov8_demo 目录传输至开发板，然后执行下面指令运行程序：

首先进行解压

root@myd-lr3576x-debian:/userdata/tools# tar -zxvf rknn_yolov8_demo.tar.gz 
rknn_yolov8_demo/
rknn_yolov8_demo/rknn_yolov8_demo_zero_copy
rknn_yolov8_demo/model/
rknn_yolov8_demo/model/yolov8.rknn
rknn_yolov8_demo/model/coco_80_labels_list.txt
rknn_yolov8_demo/model/bus.jpg
rknn_yolov8_demo/lib/
rknn_yolov8_demo/lib/librknnrt.so
rknn_yolov8_demo/lib/librga.so
rknn_yolov8_demo/rknn_yolov8_demo
root@myd-lr3576x-debian:/userdata/tools# ls -l
总计 129280
-rw-r--r-- 1 root   root   126252801  2月25日 11:14 Miniconda3-py39_23.11.0-2-Linux-aarch64.sh
drwxrwxr-x 4 linaro linaro      4096  2月28日 17:05 rknn_yolov8_demo
-rw-r--r-- 1 myir   myir     6123112  2月28日 17:17 rknn_yolov8_demo.tar.gz

推理完成后生成图片 out.png

执行推理：【原始程序，只有2个参数】

./rknn_yolov8_demo ./model/yolov8.rknn ./model/bus.jpg

结果为：

root@myd-lr3576x-debian:/userdata/tools/rknn_yolov8_demo# ./rknn_yolov8_demo ./model/yolov8.rknn ./model/bus.jpg 
load lable ./model/coco_80_labels_list.txt
model input num: 1, output num: 9
input tensors:
  index=0, name=images, n_dims=4, dims=[1, 640, 640, 3], n_elems=1228800, size=1228800, fmt=NHWC, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003922
output tensors:
  index=0, name=462, n_dims=4, dims=[1, 64, 80, 80], n_elems=409600, size=409600, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-44, scale=0.135466
  index=1, name=onnx::ReduceSum_476, n_dims=4, dims=[1, 80, 80, 80], n_elems=512000, size=512000, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003027
  index=2, name=480, n_dims=4, dims=[1, 1, 80, 80], n_elems=6400, size=6400, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003036
  index=3, name=487, n_dims=4, dims=[1, 64, 40, 40], n_elems=102400, size=102400, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-38, scale=0.093536
  index=4, name=onnx::ReduceSum_501, n_dims=4, dims=[1, 80, 40, 40], n_elems=128000, size=128000, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003476
  index=5, name=505, n_dims=4, dims=[1, 1, 40, 40], n_elems=1600, size=1600, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003922
  index=6, name=512, n_dims=4, dims=[1, 64, 20, 20], n_elems=25600, size=25600, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-42, scale=0.084287
  index=7, name=onnx::ReduceSum_526, n_dims=4, dims=[1, 80, 20, 20], n_elems=32000, size=32000, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003903
  index=8, name=530, n_dims=4, dims=[1, 1, 20, 20], n_elems=400, size=400, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003922
model is NHWC input fmt
model input height=640, width=640, channel=3
origin size=640x640 crop size=640x640
input image: 640 x 640, subsampling: 4:2:0, colorspace: YCbCr, orientation: 1
scale=1.000000 dst_box=(0 0 639 639) allow_slight_change=1 _left_offset=0 _top_offset=0 padding_w=0 padding_h=0
rga_api version 1.10.1_[0]
rknn_run
bus @ (95 136 553 437) 0.941
person @ (108 236 223 535) 0.890
person @ (212 240 284 509) 0.835
person @ (477 230 559 522) 0.831
person @ (79 358 117 515) 0.452
write_image path: out.png width=640 height=640 channel=3 data=0x39996620

PS：虚拟机共享目录设置

：/mnt/hgfs/VM-SharedFolders/

本文通过 YUQUE WORDPRESS 同步自语雀

IoT软件产品

IoT硬件产品

将训练好的PT模型转化为RKNN_物联网平台_边缘计算网关

1、得到训练好的PT模型并利用工具转化为ONNX

2、测试转化后的onnx模型结构是否合理

3、将检测合格后的onnx模型导入到虚拟机的对应环境进行转化，环境搭建教程如下：

3-1、RKNN-Toolkit2 安装（ubuntu22.04 X86_64平台）

本地PC安装【虚拟机上】

3-2、下载官方rknn_model_zoo 应用示例

3-2.1 导出 RKNN 模型

3-2.2 编译和构建【C++】可执行程序，方便测试你的模型

3-2.3 运行程序【yolov8】

云腾五洲-AI助理

温度超过28℃，打开空调

模拟网关+子设备MQTT接入

国内领先的智能物联产品和解决方案提供商

产品

开发者

公司