RKNN Yolo11部署
RKNN部署
Anaconda安装
Anaconda 是可以便捷获取包且对包能够进行管理，同时对环境可以统一管理的发行版本。Anaconda包含了conda、Python在内的众多流行的科学计算、数据分析包。
wget https://repo.anaconda.com/archive/Anaconda3-2023.07-2-Linux-x86_64.sh
chmod +x ./Anaconda3-2023.07-2-Linux-x86_64.sh
./Anaconda3-2023.07-2-Linux-x86_64.sh
使用命令source ~/.bashrc进入anaconda环境
source ~/.bashrc
下面列出一些常用conda命令，更多命令参考conda —help
# 创建一个环境，env_name是环境名称，后面可以指定python的版本等等
conda create -n env_name ....
# 列出创建的虚拟环境
#conda env list
conda info --envs
# 激活环境，不指定env_name时，默认进入base环境
conda activate env_name
# 移除环境，其中env_name是环境名称
conda remove -n env_name --all
# 退出当前conda环境
conda deactivate
# 每次开启终端都会默认进入anaconda环境，使用下面命令后重启就不会默认启动虚拟环境
conda config --set auto_activate_base falseAnaconda配置使用国内镜像源
cat > ~/.condarc <<'EOF'
channels:
- defaults
show_channel_urls: true
default_channels:
- https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
- https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/r
- https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/msys2
custom_channels:
conda-forge: https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud
pytorch: https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud
nvidia: https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud
EOF清一下索引缓存，再创建环境
conda clean -i
conda create -n toolkit2_1.6 python=3.8 -y检查
conda config --show default_channels
Anaconda清华源的使用请参考：https://mirrors.tuna.tsinghua.edu.cn/help/anaconda
rknn-toolkit2安装
RKNN Toolkit2 开发套件运行在PC (x86_64/arm64)平台上，提供了模型转换、 量化功能、模型推理、性能和内存评估、量化精度分析、模型加密等功能。
# 创建一个名为toolkit2_3.2的环境，并指定python版本，
conda create -n toolkit2_3.2 python=3.8
conda activate toolkit2_3.2
# 拉取toolkit2源码
git clone https://github.com/airockchip/rknn-toolkit2
# 配置pip源
pip3 config set global.index-url https://pypi.tuna.tsinghua.edu.cn/simple/
# pip安装指定版本的库（当前toolkit2版本是2.3.2，请根据python版本选择文件安装）
cd rknn-toolkit2
python -m pip install -r /mnt/okaybuild/Anaconda/rknn-toolkit2/rknn-toolkit2/packages/x86_64/requirements_cp38-2.3.2.txt
# 需要根据python版本和rknn_toolkit2版本选择whl文件，例如这里创建的是python3.8环境，使用带”cp38”的whl文件。
python -m pip install /mnt/okaybuild/Anaconda/rknn-toolkit2/rknn-toolkit2/packages/x86_64/rknn_toolkit2-2.3.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl安装成功后会显示Successfully，我们也可以简单测试下,测试完成 Ctrl+D 退出
YOLO11
Ultralytics YOLO11 是新一代计算机视觉模型， 在目标检测、实例分割、图像分类、姿势估计、定向物体检测和对象跟踪等计算机视觉任务上展现了卓越的性能和准确性。
YOLO11 github地址：https://github.com/ultralytics/ultralytics
YOLO11目标检测
先在个人PC上使用Anaconda创建一个yolo11开发环境，然后简单测试yolo11
conda create -n yolo11 python=3.10
conda activate yolo11
# 配置pip源（可选）
pip config set global.index-url https://pypi.tuna.tsinghua.edu.cn/simple/
# 安装
pip install ultralytics
# 或者从源码安装
git clone https://github.com/ultralytics/ultralytics.git
cd ultralytics
pip install -e .
# 检测版本
yolo version
执行yolo命令，测试yolo11n模型
yolo predict model=yolo11n.pt source='https://ultralytics.com/images/zidane.jpg'
将会拉取yolo11n.pt文件，运行推理，结果保存在runs/detect/predict目录下：
模型转换
导出onnx模型
模型导出使用专门针对rknn优化的 ultralytics_yolo11 。 该工程在基于不影响输出结果, 不需要重新训练模型的条件下, 有以下改动:
- 文本修改输出结构, 移除后处理结构(后处理结果对于量化不友好)；
- dfl 结构在 NPU 处理上性能不佳，移至模型外部的后处理阶段，此操作大部分情况下可提升推理性能；
- 模型输出分支新增置信度的总和，用于后处理阶段加速阈值筛选。
git clone https://github.com/airockchip/ultralytics_yolo11.git
# 修改 ./ultralytics/cfg/default.yaml中model文件路径，默认为yolo11n.pt
# 安装本地 ultralytics_yolo11 仓库的基础依赖
python -m pip install -e .
python -m pip install onnx onnxslim==0.1.34 onnxruntime
python -m pip install onnxscript onnx onnxruntime
# 导出onnx模型
export PYTHONPATH=./
python ./ultralytics/engine/exporter.py
导出的onnx模型会在这个目录下
使用 NETRON 查看其模型输入输出：
转换成rknn模型
先编写一个onnx2rknn_yolo11.py脚本
#!/usr/bin/env python3
"""Convert a YOLO11 ONNX model to RKNN with RKNN-Toolkit2.
This script supports both styles below:
1. LubanCat-style positional arguments
python onnx2rknn_yolo11.py model.onnx rk3576 i8 output.rknn
2. Option-style arguments
python onnx2rknn_yolo11.py --onnx model.onnx --target rk3576 --quantize --dataset dataset.txt
"""
from __future__ import annotations
import argparse
import sys
from pathlib import Path
DEFAULT_ONNX = Path("/mnt/okaybuild/Anaconda/yolo11n.onnx")
DEFAULT_OUTPUT_DIR = Path("/mnt/okaybuild/Anaconda")
DEFAULT_TARGET = "rk3576"
DEFAULT_DTYPE = "fp"
DEFAULT_MEAN_VALUES = [[0, 0, 0]]
DEFAULT_STD_VALUES = [[255, 255, 255]]
DEFAULT_DATASET_BASENAMES = (
"dataset.txt",
"datasets.txt",
"coco_subset_20.txt",
"coco_subset_20_paths.txt",
)
I8_FP_PLATFORMS = {
"rk3562",
"rk3566",
"rk3568",
"rk3576",
"rk3588",
"rk3588s",
"rv1103",
"rv1103b",
"rv1106",
"rv1106b",
"rv1126b",
"rk2118",
}
LEGACY_U8_FP_PLATFORMS = {"rk1808", "rv1109", "rv1126"}
VALID_DTYPES = {"i8", "u8", "fp"}
def parse_args() -> argparse.Namespace:
parser = argparse.ArgumentParser(
description="Convert a YOLO11 ONNX model to RKNN with RKNN-Toolkit2.",
formatter_class=argparse.RawTextHelpFormatter,
epilog=(
"Examples:\n"
" python onnx2rknn_yolo11.py\n"
" python onnx2rknn_yolo11.py /mnt/okaybuild/Anaconda/yolo11n.onnx rk3576 fp\n"
" python onnx2rknn_yolo11.py /mnt/okaybuild/Anaconda/yolo11n.onnx rk3576 i8 /mnt/okaybuild/Anaconda/yolo11n-rk3576-i8.rknn --dataset /path/to/dataset.txt\n"
" python onnx2rknn_yolo11.py --onnx /mnt/okaybuild/Anaconda/yolo11n.onnx --target rk3576 --quantize --dataset /path/to/dataset.txt"
),
)
parser.add_argument("model_path", nargs="?", help="Input ONNX model path.")
parser.add_argument("platform", nargs="?", help="Target platform, e.g. rk3576.")
parser.add_argument("dtype", nargs="?", help="Model dtype: i8, u8, or fp.")
parser.add_argument("output_path", nargs="?", help="Output RKNN model path.")
parser.add_argument("--onnx", dest="onnx_override", default="", help=f"Input ONNX path. Default: {DEFAULT_ONNX}")
parser.add_argument("--target", dest="target_override", default="", help=f"Target platform. Default: {DEFAULT_TARGET}")
parser.add_argument("--dtype", dest="dtype_override", default="", choices=sorted(VALID_DTYPES), help="Model dtype: i8, u8, or fp.")
parser.add_argument("--output", dest="output_override", default="", help=f"Output RKNN path. Default dir: {DEFAULT_OUTPUT_DIR}")
parser.add_argument("--quantize", action="store_true", help="Shortcut for --dtype i8.")
parser.add_argument("--dataset", default="", help="Path to dataset.txt used for quantization.")
parser.add_argument("--quantized-algorithm", default="normal", choices=["normal", "mmse"], help="RKNN quantized_algorithm. Default: normal")
parser.add_argument("--quantized-dtype", default="asymmetric_quantized-8", help="Advanced RKNN quantized_dtype override. Default: asymmetric_quantized-8")
parser.add_argument("--float-dtype", default="float16", help="RKNN float_dtype for fp builds. Default: float16")
parser.add_argument("--verbose", action="store_true", help="Enable verbose RKNN logs.")
return parser.parse_args()
def choose_value(override: str, positional: str | None, default: str) -> str:
if override:
return override
if positional:
return positional
return default
def normalize_dtype(args: argparse.Namespace) -> str:
dtype = choose_value(args.dtype_override, args.dtype, "")
if not dtype:
dtype = "i8" if args.quantize else DEFAULT_DTYPE
dtype = dtype.lower().strip()
if dtype not in VALID_DTYPES:
raise ValueError(f"Invalid dtype: {dtype}. Choose from i8, u8, fp.")
if args.quantize and dtype == "fp":
raise ValueError("--quantize conflicts with dtype=fp.")
return dtype
def validate_platform_dtype(platform: str, dtype: str) -> None:
if platform in I8_FP_PLATFORMS and dtype == "u8":
raise ValueError(f"{platform} should use i8 or fp, not u8.")
if platform in LEGACY_U8_FP_PLATFORMS and dtype == "i8":
raise ValueError(f"{platform} should use u8 or fp, not i8.")
if platform in LEGACY_U8_FP_PLATFORMS:
print(f"[warn] {platform} is a legacy platform. It may require an older RKNN toolkit flow than toolkit2_3.2.")
elif platform not in I8_FP_PLATFORMS:
print(f"[warn] {platform} is not in the built-in platform list. Continuing anyway.")
def default_output_path(onnx_path: Path, target: str, dtype: str) -> Path:
return DEFAULT_OUTPUT_DIR / f"{onnx_path.stem}-{target}-{dtype}.rknn"
def find_default_dataset(onnx_path: Path) -> Path | None:
search_roots = [
Path.cwd(),
onnx_path.parent,
DEFAULT_OUTPUT_DIR,
Path(__file__).resolve().parent,
Path(__file__).resolve().parent.parent,
]
for root in search_roots:
for basename in DEFAULT_DATASET_BASENAMES:
candidate = root / basename
if candidate.is_file():
return candidate.resolve()
return None
def print_onnx_info(model_path: Path) -> None:
try:
import onnx
except ImportError:
print("[info] onnx is not installed, skip ONNX input/output inspection.")
return
try:
model = onnx.load(str(model_path))
except Exception as exc: # pragma: no cover
print(f"[warn] failed to inspect ONNX model: {exc}")
return
opsets = [item.version for item in model.opset_import]
print(f"[info] ONNX opset: {opsets}")
print("[info] inputs:")
for item in model.graph.input:
dims = [dim.dim_value or dim.dim_param or "?" for dim in item.type.tensor_type.shape.dim]
print(f" - {item.name}: {dims}")
print("[info] outputs:")
for item in model.graph.output:
dims = [dim.dim_value or dim.dim_param or "?" for dim in item.type.tensor_type.shape.dim]
print(f" - {item.name}: {dims}")
def main() -> int:
args = parse_args()
try:
dtype = normalize_dtype(args)
target = choose_value(args.target_override, args.platform, DEFAULT_TARGET).lower().strip()
validate_platform_dtype(target, dtype)
except ValueError as exc:
print(f"[error] {exc}", file=sys.stderr)
return 1
onnx_raw = choose_value(args.onnx_override, args.model_path, str(DEFAULT_ONNX))
onnx_path = Path(onnx_raw).expanduser().resolve()
if not onnx_path.is_file():
print(f"[error] ONNX model not found: {onnx_path}", file=sys.stderr)
return 1
quantize = dtype in {"i8", "u8"}
dataset_path = None
if args.dataset:
dataset_path = Path(args.dataset).expanduser().resolve()
if not dataset_path.is_file():
print(f"[error] dataset.txt not found: {dataset_path}", file=sys.stderr)
return 1
elif quantize:
dataset_path = find_default_dataset(onnx_path)
if dataset_path is not None:
print(f"[info] auto-selected dataset: {dataset_path}")
if quantize and dataset_path is None:
print("[error] Quantized build requires dataset.txt. Use --dataset /path/to/dataset.txt.", file=sys.stderr)
return 1
if not quantize and args.dataset:
print("[info] fp build selected, dataset.txt will be ignored.")
output_raw = choose_value(args.output_override, args.output_path, "")
output_path = Path(output_raw).expanduser().resolve() if output_raw else default_output_path(onnx_path, target, dtype)
output_path.parent.mkdir(parents=True, exist_ok=True)
print(f"[info] input ONNX : {onnx_path}")
print(f"[info] output RKNN: {output_path}")
print(f"[info] target : {target}")
print(f"[info] dtype : {dtype}")
print(f"[info] quantize : {quantize}")
if dataset_path is not None:
print(f"[info] dataset : {dataset_path}")
print_onnx_info(onnx_path)
try:
from rknn.api import RKNN
except ImportError as exc:
print(f"[error] failed to import RKNN: {exc}", file=sys.stderr)
print("[hint] Activate the toolkit2_3.2 conda environment first.", file=sys.stderr)
return 1
rknn = RKNN(verbose=args.verbose)
try:
print("--> Config model")
config_kwargs = {
"target_platform": target,
"mean_values": DEFAULT_MEAN_VALUES,
"std_values": DEFAULT_STD_VALUES,
"optimization_level": 3,
}
if quantize:
config_kwargs["quantized_dtype"] = args.quantized_dtype
config_kwargs["quantized_algorithm"] = args.quantized_algorithm
else:
config_kwargs["float_dtype"] = args.float_dtype
ret = rknn.config(**config_kwargs)
if ret != 0:
print(f"[error] rknn.config failed: {ret}", file=sys.stderr)
return ret
print("done")
print("--> Loading model")
ret = rknn.load_onnx(model=str(onnx_path))
if ret != 0:
print(f"[error] rknn.load_onnx failed: {ret}", file=sys.stderr)
return ret
print("done")
print("--> Building model")
if quantize:
ret = rknn.build(do_quantization=True, dataset=str(dataset_path))
else:
ret = rknn.build(do_quantization=False)
if ret != 0:
print(f"[error] rknn.build failed: {ret}", file=sys.stderr)
return ret
print("done")
print("--> Export rknn model")
ret = rknn.export_rknn(str(output_path))
if ret != 0:
print(f"[error] rknn.export_rknn failed: {ret}", file=sys.stderr)
return ret
print("done")
finally:
rknn.release()
print(f"[success] RKNN model saved to: {output_path}")
return 0
if __name__ == "__main__":
raise SystemExit(main())我用的板卡是RK3588，运行
非量化
conda activate toolkit2_3.2
python /mnt/okaybuild/Anaconda/onnx2rknn_yolo11.py /mnt/okaybuild/Anaconda/yolo11n.onnx rk3588 fp量化
conda activate toolkit2_3.2
python /mnt/okaybuild/Anaconda/onnx2rknn_yolo11.py /mnt/okaybuild/Anaconda/yolo11n.onnx rk3588 i8 /mnt/okaybuild/Anaconda/yolo11n-rk3588-i8.rknn --dataset /mnt/okaybuild/Anaconda/dataset.txt非量化转换和量化转换的区别
非量化转换 非量化的 fp 版本，本质上是浮点模型 在 RKNN 里通常会落成 FP16 优点： 精度风险更低 转换最省事 不需要 dataset.txt 适合先打通链路、排查兼容性 缺点： 一般比量化模型更大 NPU 上速度和带宽利用通常不如量化版 内存占用通常更高
量化转换 脚本里的 i8/u8 属于量化模型 会把很多浮点张量映射成 8-bit 整数 优点： 通常更快 模型更小 更适合板端长期部署 缺点： 可能掉精度 转换依赖 dataset.txt 如果校准数据选得差，效果会明显变差
量化里面的 dataset.txt 是量化校准数据列表文件，里面的内容是每行一张图片路径
量化时，RKNN 需要知道： 每一层激活值大概分布在哪个范围 用什么 scale / zero point 去把浮点映射成 int8 这个统计过程就靠 dataset.txt 里列出来的图片完成
dataset.txt 里的图片应该尽量接近真实部署场景，比如最终是拿摄像头看人、车等，那 dataset.txt 对应的图片就应该来自这些真实场景
图片不需要标注不需要 txt，不需要框，不需要类别标签，只需要原始图片文件，量化只看输入数据分布，不看监督信息。
图片就是普通图片，可以是：.jpg、.jpeg、.png等，如果最终是用 USB 摄像头看人和车，那就用这台摄像头拍一些真实场景图来做 dataset.txt
建议： 50 到 200 张通常够用 场景尽量覆盖真实输入分布 不要求人工标注 不要求和训练集完全一致 但最好和部署数据域一致
先试一个非量化的
板卡上部署测试
板卡上获取配套例程，然后将前面转换出的rknn模型放到例程model目录下
# 板卡上安装opencv，git等等相关软件
sudo apt update
sudo apt install libopencv-dev git make gcc g++ libsndfile1-dev
# 鲁班猫系统中，使用命令直接拉取教程配套例程
git clone https://gitee.com/LubanCat/lubancat_ai_manual_code.git
# 切换到例程目录
cd lubancat_ai_manual_code/example/yolo11/cpp
# 或者拉取rknn_model_zoo例程测试,实际编译操作请查看工程的README文件
# git clone https://github.com/airockchip/rknn_model_zoo.git
切换到lubancat_ai_manual_code/example/yolo11/cpp目录下，然后编译例程,我用的是RK3588，例程中yolo11_videocapture_demo将使用系统默认opencv，如果不需要，请注释CMakeLists.txt文件中的相关程序
./build-linux.sh -t rk3588
在install/rk3588_linux目录下将生成4个可执行文件，其中带zero_copy结尾的是支持零拷贝的可执行程序
下面测试yolo11_image_demo例程，该例程将读取图像并进行检测，并将结果保存在当前目录下，终端输出检测结果,首先要在板卡上准备一张图片
测试yolo11_image_demo例程，该例程将读取图像并进行检测，并将结果保存在当前目录下
./yolo11_image_demo ~/yolo11/lubancat_ai_manual_code/example/yolo11/model/yolo11n-rk3588-fp.rknn ~/yolo11/images/test.png输出有问题，问题是PC的导出环境是toolkit2.3.2，而例程的版本很旧
强制 demo 使用本地 lib/ 里的 librknnrt.so
cd ~/yolo11/lubancat_ai_manual_code/example/yolo11/cpp/install/rk3588_linux
LD_LIBRARY_PATH=$PWD/lib:$LD_LIBRARY_PATH ./yolo11_image_demo \
~/yolo11/lubancat_ai_manual_code/example/yolo11/model/yolo11n-rk3588-fp.rknn \
~/yolo11/images/test.png成功，日志打印了输出结果
测试yolo11_videocapture_demo例程，该例程将使用opencv读取摄像头或者视频文件，然后进行目标检测
使用摄像头需要修改yolo11_videocapture_demo.cc程序里面的参数，如果是打开USB摄像头,请确认摄像头的设备号，修改例程中摄像头支持的分辨率和MJPG格式等， 打开的是mipi摄像头时，opencv需要设置转换成rgb格式以及设置分辨率大小等等，我目前使用的是USB摄像头，节点是 Video20
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
/*-------------------------------------------
Includes
-------------------------------------------*/
#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <cctype>
#include <string>
#include "yolo11.h"
#include "image_utils.h"
#include "file_utils.h"
#if defined(RV1106_1103)
#include "dma_alloc.hpp"
#endif
#include <opencv2/opencv.hpp>
static const unsigned char colors[19][3] = {
{54, 67, 244},
{99, 30, 233},
{176, 39, 156},
{183, 58, 103},
{181, 81, 63},
{243, 150, 33},
{244, 169, 3},
{212, 188, 0},
{136, 150, 0},
{80, 175, 76},
{74, 195, 139},
{57, 220, 205},
{59, 235, 255},
{7, 193, 255},
{0, 152, 255},
{34, 87, 255},
{72, 85, 121},
{158, 158, 158},
{139, 125, 96}
};
/*-------------------------------------------
Main Function
-------------------------------------------*/
static bool is_number_string(const char* s)
{
if (s == NULL || *s == '\0')
{
return false;
}
for (const unsigned char* p = reinterpret_cast<const unsigned char*>(s); *p != '\0'; ++p)
{
if (!std::isdigit(*p))
{
return false;
}
}
return true;
}
static bool is_video_device_path(const char* s)
{
return s != NULL && strncmp(s, "/dev/video", 10) == 0;
}
int main(int argc, char **argv)
{
if (argc < 3 || argc > 6)
{
printf("%s <model path> <camera device id|/dev/videoX|video path> [width] [height] [fps]\n", argv[0]);
printf("Usage: %s yolov11s.rknn 0\n", argv[0]);
printf("Usage: %s yolov11s.rknn /dev/video20 1280 720 30\n", argv[0]);
printf("Usage: %s yolov11s.rknn /path/xxxx.mp4\n", argv[0]);
return -1;
}
const char *model_path = argv[1];
const char *device_path = argv[2];
int camera_width = (argc > 3) ? atoi(argv[3]) : 1280;
int camera_height = (argc > 4) ? atoi(argv[4]) : 720;
int camera_fps = (argc > 5) ? atoi(argv[5]) : 30;
int ret;
cv::Mat frame, image;
rknn_app_context_t rknn_app_ctx;
memset(&rknn_app_ctx, 0, sizeof(rknn_app_context_t));
image_buffer_t src_image;
memset(&src_image, 0, sizeof(image_buffer_t));
cv::VideoCapture cap;
bool use_camera = is_number_string(device_path) || is_video_device_path(device_path);
if (use_camera) {
if (is_number_string(device_path)) {
int camera_id = atoi(device_path);
cap.open(camera_id, cv::CAP_V4L2);
} else {
cap.open(device_path, cv::CAP_V4L2);
}
if (!cap.isOpened()) {
printf("Error: Could not open camera: %s\n", device_path);
return -1;
}
cap.set(cv::CAP_PROP_FOURCC, cv::VideoWriter::fourcc('M', 'J', 'P', 'G'));
cap.set(cv::CAP_PROP_CONVERT_RGB, 1);
cap.set(cv::CAP_PROP_FRAME_WIDTH, camera_width);
cap.set(cv::CAP_PROP_FRAME_HEIGHT, camera_height);
cap.set(cv::CAP_PROP_FPS, camera_fps);
double actual_width = cap.get(cv::CAP_PROP_FRAME_WIDTH);
double actual_height = cap.get(cv::CAP_PROP_FRAME_HEIGHT);
double actual_fps = cap.get(cv::CAP_PROP_FPS);
printf("Open camera: %s\n", device_path);
printf("Camera config requested: width=%d height=%d fps=%d MJPG\n", camera_width, camera_height, camera_fps);
printf("Camera config actual : width=%.0f height=%.0f fps=%.2f\n", actual_width, actual_height, actual_fps);
} else {
cap.open(device_path);
if (!cap.isOpened()) {
printf("Error: Could not open video file: %s\n", device_path);
return -1;
}
}
init_post_process();
ret = init_yolo11_model(model_path, &rknn_app_ctx);
if (ret != 0)
{
printf("init_yolo11_model fail! ret=%d model_path=%s\n", ret, model_path);
goto out;
}
while(true) {
if (!cap.read(frame)) {
printf("cap read frame fail!\n");
break;
}
cv::cvtColor(frame, image, cv::COLOR_BGR2RGB);
src_image.width = image.cols;
src_image.height = image.rows;
src_image.format = IMAGE_FORMAT_RGB888;
src_image.virt_addr = (unsigned char*)image.data;
// rknn推理和处理
object_detect_result_list od_results;
ret = inference_yolo11_model(&rknn_app_ctx, &src_image, &od_results);
if (ret != 0)
{
printf("init_yolov10_model fail! ret=%d\n", ret);
goto out;
}
// 画框和概率
int color_index = 0;
char text[256];
for (int i = 0; i < od_results.count; i++)
{
const unsigned char* color = colors[color_index % 19];
cv::Scalar cc(color[0], color[1], color[2]);
color_index++;
object_detect_result *det_result = &(od_results.results[i]);
printf("%s @ (%d %d %d %d) %.3f\n", coco_cls_to_name(det_result->cls_id),
det_result->box.left, det_result->box.top,
det_result->box.right, det_result->box.bottom,
det_result->prop);
sprintf(text, "%s %.1f%%", coco_cls_to_name(det_result->cls_id), det_result->prop * 100);
cv::rectangle(frame, cv::Rect(cv::Point(det_result->box.left, det_result->box.top),
cv::Point(det_result->box.right, det_result->box.bottom)), cc, 2);
int baseLine = 0;
cv::Size label_size = cv::getTextSize(text, cv::FONT_HERSHEY_SIMPLEX, 0.5, 1, &baseLine);
int x = det_result->box.left;
int y = det_result->box.top - label_size.height - baseLine;
if (y < 0)
y = 0;
if (x + label_size.width > frame.cols)
x = frame.cols - label_size.width;
cv::rectangle(frame, cv::Rect(cv::Point(x, y), cv::Size(label_size.width, label_size.height + baseLine)),cc,-1);
cv::putText(frame, text, cv::Point(x, y + label_size.height),cv::FONT_HERSHEY_SIMPLEX, 0.5, cv::Scalar(255, 255, 255));
}
// 显示结果
cv::imshow("yolo11", frame);
char c = cv::waitKey(1);
if (c == 27) { // ESC
break;
}
}
out:
deinit_post_process();
ret = release_yolo11_model(&rknn_app_ctx);
if (ret != 0)
{
printf("release_yolo11_model fail! ret=%d\n", ret);
}
return 0;
}执行命令
LD_LIBRARY_PATH=$PWD/lib:$LD_LIBRARY_PATH ./yolo11_videocapture_demo \
~/yolo11/lubancat_ai_manual_code/example/yolo11/model/yolo11n-rk3588-fp.rknn \
/dev/video20 \
1280 \
720 \
30成功
串口也有输出
使用MIPI摄像头IMX296的效果
部署自己训练的模型
将训练好的模型传到Linux环境中并进入yolo环境
修改 /mnt/okaybuild/Anaconda/ultralytics_yolo11/ultralytics/cfg/default.yaml中model文件路径，改成自己的
导出模型
查看其模型输入输出
转换成rknn模型
首先需传50-100张图片到Ubuntu供量化校准使用
然后创建一个dataset.txt文件，内容是图片的路径
修改onnx2rknn_yolo11.py脚本里面默认的 ONNX 模型路径
进入toolkit2_3.2环境中，然后使用下面命令转换
python onnx2rknn_yolo11.py --onnx /mnt/okaybuild/Anaconda/pcba.onnx --target rk3588 --quantize --dataset /mnt/okaybuild/Anaconda/dataset.txt转换完成
接下来就是部署到RK3588上
查看运行效果，先看数据分析
总的来说还是不错的，但是没有训练出来在PC跑的效果好
评论