概述

云腾一体机内置一款基于 RKNN (Rockchip Neural Network) 推理引擎的 YOLOv10 模型的演示应用。

YOLO（You Only Look Once）是一个非常流行的实时目标检测算法，它通过一个神经网络进行端到端的训练，能够快速高效地检测图像中的多个目标。YOLOv10 是 YOLO 算法的一个改进版本，通常会包含一些创新和优化，使其在推理速度、准确度或硬件支持方面有所提升。

RKNN 是由 Rockchip（瑞芯微）推出的深度学习推理框架，用于在 Rockchip 的硬件平台（如 RK3588、RK3576 等）上加速人工智能推理任务。该框架支持多种常见的深度学习模型，并提供了优化和加速推理的工具，使得这些模型可以在嵌入式设备和边缘设备上高效运行。

功能

该AI推理应用可以对80种常见物体(如人、车、猫、狗等)进行有效识别，在AIoTedge通过function节点进行各项路径设置，再通过exec节点进行系统命令下发即可进行调用。

该应用识别过程为：

加载指定路径图像。
在图像中识别出多个目标。
给每个目标分配一个类别标签（如人、车、狗等）。
为每个目标输出一个边界框（bounding box），包括边界框的坐标、宽度、高度，以及目标的置信度。
输出结果并到处处理过的图像。

person          ——人
bicycle         ——自行车
car             ——汽车
motorcycle      ——摩托车
airplane        ——飞机
bus             ——公共汽车
train           ——火车
truck           ——卡车
boat            ——船
traffic light   ——交通灯
fire hydrant    ——消防栓
stop sign       ——停车标志
parking meter   ——停车收费器
bench           ——长凳
bird            ——鸟
cat             ——猫
dog             ——狗
horse           ——马
sheep           ——羊
cow             ——牛
elephant        ——大象
bear            ——熊
zebra           ——斑马
giraffe         ——长颈鹿
backpack        ——背包
umbrella        ——雨伞
handbag         ——手提包
tie             ——领带
suitcase        ——手提箱
frisbee         ——飞盘
skis            ——滑雪板
snowboard       ——滑雪板
sports ball     ——运动球
kite            ——风筝
baseball bat    ——棒球棒
baseball glove  ——棒球手套
skateboard      ——滑板
surfboard       ——冲浪板
tennis racket   ——网球拍
bottle          ——瓶子
wine glass      ——玻璃酒杯
cup             ——杯子
fork            ——叉
knife           ——刀
spoon           ——勺子
bowl            ——碗
banana          ——香蕉
apple           ——苹果
sandwich        ——三明治
orange          ——桔子
broccoli        ——西兰花
carrot          ——胡萝卜
hot dog         ——热狗
pizza           ——披萨
donut           ——甜甜圈
cake            ——蛋糕
chair           ——椅子
couch           ——沙发
potted plant    ——盆栽植物
bed             ——床
dining table    ——餐桌
toilet          ——厕所
tv              ——电视
laptop          ——笔记本电脑
mouse           ——鼠标
remote          ——遥控器
keyboard        ——键盘
cell phone      ——手机
microwave       ——微波炉
oven            ——烤箱
toaster         ——烤面包机
sink            ——洗碗槽
refrigerator    ——冰箱
book            ——书
clock           ——时钟
vase            ——花瓶
scissors        ——剪刀
teddy bear      ——玩具熊
hair drier      ——吹风机
toothbrush      ——牙刷

事件上报

将识别的结果，上报到AIoTedge平台；如果有云边协同，还可以同步到Thingskit平台。

描述：图中识别到有6个人的结果，我们将结果，以事件的形式上报到平台。

上报到平台的结果如下：

示例

1、AI识别静态图片

[{"id":"4af461b9baa0b019","type":"function","z":"fb8e39bbd95ffbcd","name":"设置名称","func":"nnvar imgImage = "/userdata/app/rknn_yolov10_demo/model/";nvar imgOut = "/userdata/app/rknn_yolov10_demo/testImage/testOut/";nvar name = "personAndCar";nvar nameOut = "outImage_";nvar ts = new Date().getTime();nvar fileNameImage = name + ".jpg";nvar fileNameOut = nameOut + ts + ".jpg";n// msg.payload = fileName;nmsg.fileNameRtsp = imgImage + fileNameImage;nmsg.fileNameOut = imgOut + fileNameOut;nmsg.rtspAndOut = imgImage + fileNameImage + " " + imgOut + fileNameOut;nreturn msg;","outputs":1,"timeout":0,"noerr":0,"initialize":"","finalize":"","libs":[],"x":360,"y":360,"wires":[["898aaed2a85c7f49"]]},{"id":"b018747c202efe37","type":"inject","z":"fb8e39bbd95ffbcd","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":210,"y":360,"wires":[["4af461b9baa0b019"]]},{"id":"898aaed2a85c7f49","type":"exec","z":"fb8e39bbd95ffbcd","command":"cd /userdata/app/rknn_yolov10_demo && ./rknn_yolov10_demo ./model/yolov10.rknn ","addpay":"rtspAndOut","append":"","useSpawn":"false","timer":"","winHide":false,"oldrc":false,"name":"YOLOv10识别图片","x":270,"y":440,"wires":[["4e51a2e1a6dc8c33","8512622ecbcdf8dc"],[],[]]},{"id":"62f5e1e9f929cbff","type":"image","z":"fb8e39bbd95ffbcd","name":"","width":"800","data":"payload","dataType":"msg","thumbnail":false,"active":true,"pass":false,"outputs":0,"x":260,"y":500,"wires":[]},{"id":"1be82dadda5ad72f","type":"file in","z":"fb8e39bbd95ffbcd","name":"","filename":"fileNameOut","filenameType":"msg","format":"utf8","chunk":false,"sendError":false,"encoding":"base64","allProps":false,"x":680,"y":460,"wires":[["62f5e1e9f929cbff"]]},{"id":"4e51a2e1a6dc8c33","type":"delay","z":"fb8e39bbd95ffbcd","name":"","pauseType":"delay","timeout":"0.5","timeoutUnits":"seconds","rate":"1","nbRateUnits":"1","rateUnits":"second","randomFirst":"1","randomLast":"5","randomUnits":"seconds","drop":false,"allowrate":false,"outputs":1,"x":480,"y":460,"wires":[["1be82dadda5ad72f"]]},{"id":"8512622ecbcdf8dc","type":"function","z":"fb8e39bbd95ffbcd","name":"function 2","func":"var result = msg.payload;n// 找到 rknn_run 的起始位置nconst startIndex = result.indexOf('rknn_run');n// 找到 write_image 的起始位置nconst endIndex = result.indexOf('write_image');nnif (startIndex!== -1 && endIndex!== -1) {n    // 截取从 rknn_run 到 write_image 之间的内容n    const resultOK = result.substring(startIndex + 'rknn_run'.length, endIndex);n    // console.log(result.trim());n    msg.payload = resultOK;n}nnreturn msg;","outputs":1,"timeout":0,"noerr":0,"initialize":"","finalize":"","libs":[],"x":480,"y":420,"wires":[["ccea2616778fa01b"]]},{"id":"ccea2616778fa01b","type":"debug","z":"fb8e39bbd95ffbcd","name":"debug 5","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","targetType":"msg","statusVal":"","statusType":"auto","x":680,"y":420,"wires":[]},{"id":"fa7d1259b49bae94","type":"comment","z":"fb8e39bbd95ffbcd","name":"AI推理-执行要求","info":"1、最快3秒执行一次。n2、可以根据你的需要修改【摄像头】地址n3、识别的结果为字符串，你可以按照要求去判断。n比如：有两个person说明图中有两个人。n剩下的交给你自由发挥了》》》n比如：机动车违章占道（识别摄像头下，不能有车）、n    离岗识别（指定时间内，摄像头下必须有人）、n    越界识别（摄像头下，不能有人）n    其他盘点，如，羊圈数羊；牛圈数牛；n————————————————————————————————————npersonttt——人nbicyclettt——自行车ncartttt——汽车nmotorcyclett——摩托车nairplanett——飞机nbustttt——公共汽车ntrainttt——火车ntruckttt——卡车nboatttt——船ntraffic lightt——交通灯nfire hydrantt——消防栓nstop signtt——停车标志nparking metert——停车收费器nbenchttt——长凳nbirdttt——鸟ncattttt——猫ndogtttt——狗nhorsettt——马nsheepttt——羊ncowtttt——牛nelephanttt——大象nbearttt——熊nzebrattt——斑马ngiraffettt——长颈鹿nbackpacktt——背包numbrellatt——雨伞nhandbagttt——手提包ntietttt——领带nsuitcasett——手提箱nfrisbeettt——飞盘nskisttt——滑雪板nsnowboardtt——滑雪板nsports balltt——运动球nkitettt——风筝nbaseball batt——棒球棒nbaseball glovet——棒球手套nskateboardtt——滑板nsurfboardtt——冲浪板ntennis rackett——网球拍nbottlettt——瓶子nwine glasstt——玻璃酒杯ncuptttt——杯子nforkttt——叉nknifettt——刀nspoonttt——勺子nbowlttt——碗nbananattt——香蕉napplettt——苹果nsandwichtt——三明治norangettt——桔子nbroccolitt——西兰花ncarrotttt——胡萝卜nhot dogttt——热狗npizzattt——披萨ndonutttt——甜甜圈ncakettt——蛋糕nchairttt——椅子ncouchttt——沙发npotted plantt——盆栽植物nbedtttt——床ndining tablet——餐桌ntoiletttt——厕所ntvtttt——电视nlaptopttt——笔记本电脑nmousettt——鼠标nremotettt——遥控器nkeyboardtt——键盘ncell phonett——手机nmicrowavett——微波炉noventtt——烤箱ntoasterttt——烤面包机nsinkttt——洗碗槽nrefrigeratort——冰箱nbookttt——书nclockttt——时钟nvasettt——花瓶nscissorstt——剪刀nteddy beartt——玩具熊nhair driertt——吹风机ntoothbrushtt——牙刷n————————————————————————————————————","x":200,"y":300,"wires":[]},{"id":"4787b0fb9b117df3","type":"comment","z":"fb8e39bbd95ffbcd","name":"AI推理案例--静态图片识别人和车","info":"nn1、右侧调试窗口，可以直观看到输出结论n   当得到结论时，可以作为事件上报到平台nn2、在页面有【image】图片预览节点。n    可以直观看到识别结果。","x":490,"y":280,"wires":[]}]

2、AI识别实时摄像帧

[{"id":"6a06e7efbcae4695","type":"exec","z":"b4c433a669c86722","command":"ffmpeg -y -i rtsp://admin:tf005188@192.168.1.128/H.264/ch1/main/av_stream -vframes 1 -qscale:v 5 ","addpay":"fileNameRtsp","append":"","useSpawn":"false","timer":"","winHide":false,"oldrc":false,"name":"配置摄像头RTSP","x":490,"y":300,"wires":[[],["80487f7723add5e9"],[]]},{"id":"80e5cd7c57dd3667","type":"function","z":"b4c433a669c86722","name":"设置名称","func":"nnvar imgRtsp = "/userdata/app/rknn_yolov10_demo/testImage/testRtsp/";nvar imgOut = "/userdata/app/rknn_yolov10_demo/testImage/testOut/";nvar name = "rtspImage_";nvar nameOut = "outImage_";nvar ts = new Date().getTime();nvar fileNameRtsp = name + ts +".jpg";nvar fileNameOut = nameOut + ts + ".jpg";n// msg.payload = fileName;nmsg.fileNameRtsp = imgRtsp + fileNameRtsp;nmsg.fileNameOut = imgOut + fileNameOut;nmsg.rtspAndOut = imgRtsp + fileNameRtsp + " " + imgOut + fileNameOut;nreturn msg;","outputs":1,"timeout":0,"noerr":0,"initialize":"","finalize":"","libs":[],"x":300,"y":300,"wires":[["6a06e7efbcae4695"]]},{"id":"3007dbcd68572b80","type":"inject","z":"b4c433a669c86722","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":150,"y":300,"wires":[["80e5cd7c57dd3667"]]},{"id":"80487f7723add5e9","type":"exec","z":"b4c433a669c86722","command":"cd /userdata/app/rknn_yolov10_demo && ./rknn_yolov10_demo ./model/yolov10.rknn ","addpay":"rtspAndOut","append":"","useSpawn":"false","timer":"","winHide":false,"oldrc":false,"name":"YOLOv10识别图片","x":190,"y":380,"wires":[["ea734de49dd41cdd","1646346e5b6faad8"],[],[]]},{"id":"453e85915f563a78","type":"image","z":"b4c433a669c86722","name":"","width":"800","data":"payload","dataType":"msg","thumbnail":false,"active":true,"pass":false,"outputs":0,"x":180,"y":440,"wires":[]},{"id":"766a02212b2c4a25","type":"file in","z":"b4c433a669c86722","name":"","filename":"fileNameOut","filenameType":"msg","format":"utf8","chunk":false,"sendError":false,"encoding":"base64","allProps":false,"x":600,"y":400,"wires":[["453e85915f563a78"]]},{"id":"ea734de49dd41cdd","type":"delay","z":"b4c433a669c86722","name":"","pauseType":"delay","timeout":"0.5","timeoutUnits":"seconds","rate":"1","nbRateUnits":"1","rateUnits":"second","randomFirst":"1","randomLast":"5","randomUnits":"seconds","drop":false,"allowrate":false,"outputs":1,"x":400,"y":400,"wires":[["766a02212b2c4a25"]]},{"id":"1646346e5b6faad8","type":"function","z":"b4c433a669c86722","name":"function 1","func":"var result = msg.payload;n// 找到 rknn_run 的起始位置nconst startIndex = result.indexOf('rknn_run');n// 找到 write_image 的起始位置nconst endIndex = result.indexOf('write_image');nnif (startIndex!== -1 && endIndex!== -1) {n    // 截取从 rknn_run 到 write_image 之间的内容n    const resultOK = result.substring(startIndex + 'rknn_run'.length, endIndex);n    // console.log(result.trim());n    msg.payload = resultOK;n}nnreturn msg;","outputs":1,"timeout":0,"noerr":0,"initialize":"","finalize":"","libs":[],"x":400,"y":360,"wires":[["7033e82531692385"]]},{"id":"7033e82531692385","type":"debug","z":"b4c433a669c86722","name":"debug 2","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","targetType":"msg","statusVal":"","statusType":"auto","x":600,"y":360,"wires":[]},{"id":"41725260c65711c6","type":"comment","z":"b4c433a669c86722","name":"AI推理-执行要求","info":"1、最快3秒执行一次。n2、可以根据你的需要修改【摄像头】地址n3、识别的结果为字符串，你可以按照要求去判断。n比如：有两个person说明图中有两个人。n剩下的交给你自由发挥了》》》n比如：机动车违章占道（识别摄像头下，不能有车）、n    离岗识别（指定时间内，摄像头下必须有人）、n    越界识别（摄像头下，不能有人）n    其他盘点，如，羊圈数羊；牛圈数牛；n————————————————————————————————————npersonttt——人nbicyclettt——自行车ncartttt——汽车nmotorcyclett——摩托车nairplanett——飞机nbustttt——公共汽车ntrainttt——火车ntruckttt——卡车nboatttt——船ntraffic lightt——交通灯nfire hydrantt——消防栓nstop signtt——停车标志nparking metert——停车收费器nbenchttt——长凳nbirdttt——鸟ncattttt——猫ndogtttt——狗nhorsettt——马nsheepttt——羊ncowtttt——牛nelephanttt——大象nbearttt——熊nzebrattt——斑马ngiraffettt——长颈鹿nbackpacktt——背包numbrellatt——雨伞nhandbagttt——手提包ntietttt——领带nsuitcasett——手提箱nfrisbeettt——飞盘nskisttt——滑雪板nsnowboardtt——滑雪板nsports balltt——运动球nkitettt——风筝nbaseball batt——棒球棒nbaseball glovet——棒球手套nskateboardtt——滑板nsurfboardtt——冲浪板ntennis rackett——网球拍nbottlettt——瓶子nwine glasstt——玻璃酒杯ncuptttt——杯子nforkttt——叉nknifettt——刀nspoonttt——勺子nbowlttt——碗nbananattt——香蕉napplettt——苹果nsandwichtt——三明治norangettt——桔子nbroccolitt——西兰花ncarrotttt——胡萝卜nhot dogttt——热狗npizzattt——披萨ndonutttt——甜甜圈ncakettt——蛋糕nchairttt——椅子ncouchttt——沙发npotted plantt——盆栽植物nbedtttt——床ndining tablet——餐桌ntoiletttt——厕所ntvtttt——电视nlaptopttt——笔记本电脑nmousettt——鼠标nremotettt——遥控器nkeyboardtt——键盘ncell phonett——手机nmicrowavett——微波炉noventtt——烤箱ntoasterttt——烤面包机nsinkttt——洗碗槽nrefrigeratort——冰箱nbookttt——书nclockttt——时钟nvasettt——花瓶nscissorstt——剪刀nteddy beartt——玩具熊nhair driertt——吹风机ntoothbrushtt——牙刷n————————————————————————————————————","x":140,"y":240,"wires":[]},{"id":"e3f1287fb8985e74","type":"exec","z":"b4c433a669c86722","command":"cd /userdata/app/rknn_yolov10_demo/testImage/testOut && sudo rm -rf out*","addpay":"","append":"","useSpawn":"false","timer":"","winHide":false,"oldrc":false,"name":"清除识别结果的文件","x":380,"y":60,"wires":[["5bd82f389bb88dcf"],[],[]]},{"id":"5e6e66459b1b604d","type":"inject","z":"b4c433a669c86722","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"600","crontab":"","once":true,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":160,"y":100,"wires":[["e3f1287fb8985e74","87c6be48d11b43cb"]]},{"id":"5bd82f389bb88dcf","type":"debug","z":"b4c433a669c86722","name":"debug 3","active":false,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","x":600,"y":60,"wires":[]},{"id":"87c6be48d11b43cb","type":"exec","z":"b4c433a669c86722","command":"cd /userdata/app/rknn_yolov10_demo/testImage/testRtsp && sudo rm -rf rtsp*","addpay":"","append":"","useSpawn":"false","timer":"","winHide":false,"oldrc":false,"name":"清除RTSP截图的文件","x":380,"y":120,"wires":[["daa3a0805c0dab73"],[],[]]},{"id":"daa3a0805c0dab73","type":"debug","z":"b4c433a669c86722","name":"debug 4","active":false,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","x":600,"y":120,"wires":[]},{"id":"29dd27750c72b770","type":"comment","z":"b4c433a669c86722","name":"清除识别产生的垃圾文件","info":"","x":170,"y":40,"wires":[]}]