YOLOv10 在机器人中的应用：实时目标检测与部署教程 2026

YOLOv10 在机器人视觉中的应用实战：从模型训练、优化到边缘设备部署（Jetson Nano/树莓派），涵盖目标检测、跟踪、避障、抓取，附完整代码与性能优化技巧

YOLOv10 机器视觉 机器人应用 目标检测 边缘部署 Jetson Nano 树莓派
YOLOv10 在机器人中的应用：实时目标检测与部署教程 2026

YOLOv10 在机器人中的应用：实时目标检测与部署教程 2026

YOLO（You Only Look Once）系列是实时目标检测的标杆算法。2026 年，YOLOv10 凭借更高的精度、更快的速度和更小的模型体积，成为机器人视觉系统的首选方案。

本文将系统讲解 YOLOv10 在机器人领域的应用，从模型训练、优化到边缘设备部署，涵盖目标检测、跟踪、避障、抓取等典型场景，并提供完整的代码示例和性能优化技巧。

你将学到：

  • ✅ YOLOv10 架构改进与性能优势
  • ✅ 机器人数据集准备与模型训练
  • ✅ 模型优化：量化、剪枝、TensorRT 加速
  • ✅ 边缘设备部署（Jetson Nano、树莓派、ESP32）
  • ✅ 实际应用：目标跟踪、避障、抓取定位
  • ✅ 性能优化：从 30 FPS 到 100+ FPS

前置知识：

1. YOLOv10 架构与优势

1.1 YOLO 系列演进

版本发布年份mAP@50FPS (V100)参数量核心改进
YOLOv3201855.36561.5M多尺度检测
YOLOv4202065.76564.0MCSPDarknet
YOLOv5202056.71407.0M工程优化
YOLOv7202256.816136.9ME-ELAN
YOLOv8202356.919811.2MAnchor-free
YOLOv10202458.52308.5MNMS-free

1.2 YOLOv10 核心改进

1. NMS-Free 设计

  • 移除后处理 NMS（Non-Maximum Suppression）
  • 使用一致性双重分配策略
  • 推理延迟降低 30%

2. 高效架构设计

  • 轻量级分类头（减少计算量）
  • 空间-通道分离注意力
  • 大核卷积（增强感受野）

3. 多任务优化

  • 同时支持检测、分割、姿态估计
  • 统一的特征金字塔
  • 更好的小目标检测

1.3 机器人场景优势

在 具身 AI 技术栈 中，视觉感知是最核心的模块之一。YOLOv10 的特性使其成为机器人场景的理想选择：

特性对机器人的价值
实时性快速避障、动态跟踪
高精度准确抓取、精细操作
小模型边缘设备部署
多任务一个模型完成多个感知任务
易训练快速适配新场景

2. 数据集准备

2.1 机器人常用数据集

公开数据集：

  • COCO：通用目标检测（80 类）
  • Object365：大规模物体检测（365 类）
  • Open Images：多样化场景（600+ 类）

机器人专用数据集：

  • YCB Video：物体抓取（21 类日常物体）
  • KITTI：自动驾驶（车辆、行人）
  • RoboCup：足球机器人（球、球员、球门）

2.2 自定义数据集

数据收集流程：

#!/usr/bin/env python3
"""机器人数据采集脚本"""
import cv2
import os
import json
from datetime import datetime

class RobotDataCollector:
    def __init__(self, save_dir="dataset"):
        self.save_dir = save_dir
        os.makedirs(f"{save_dir}/images", exist_ok=True)
        os.makedirs(f"{save_dir}/labels", exist_ok=True)
        
        self.cap = cv2.VideoCapture(0)
        self.annotations = []
        self.frame_count = 0
    
    def capture_frame(self):
        """采集一帧"""
        ret, frame = self.cap.read()
        if not ret:
            return None
        
        timestamp = datetime.now().strftime("%Y%m%d_%H%M%S_%f")
        filename = f"{timestamp}.jpg"
        
        # 保存图像
        cv2.imwrite(f"{self.save_dir}/images/{filename}", frame)
        
        # 显示标注界面
        annotated = frame.copy()
        cv2.putText(annotated, f"Frame: {self.frame_count}", (10, 30),
                   cv2.FONT_HERSHEY_SIMPLEX, 1, (0, 255, 0), 2)
        cv2.imshow("Data Collection", annotated)
        
        self.frame_count += 1
        return filename, frame
    
    def add_annotation(self, filename, bbox, class_id, class_name):
        """添加标注"""
        # YOLO 格式：class x_center y_center width height (归一化)
        h, w = 480, 640  # 图像尺寸
        x_center = (bbox[0] + bbox[2]) / 2 / w
        y_center = (bbox[1] + bbox[3]) / 2 / h
        width = (bbox[2] - bbox[0]) / w
        height = (bbox[3] - bbox[1]) / h
        
        # 保存标签
        label_filename = filename.replace(".jpg", ".txt")
        with open(f"{self.save_dir}/labels/{label_filename}", "w") as f:
            f.write(f"{class_id} {x_center:.6f} {y_center:.6f} {width:.6f} {height:.6f}\n")
        
        self.annotations.append({
            "image": filename,
            "bbox": bbox,
            "class_id": class_id,
            "class_name": class_name
        })
    
    def run(self):
        """主循环"""
        print("按 'c' 采集，'q' 退出")
        
        while True:
            filename, frame = self.capture_frame()
            
            key = cv2.waitKey(100)  # 10 FPS
            if key == ord('q'):
                break
            elif key == ord('c'):
                # 这里可以集成自动标注或手动标注
                print(f"采集: {filename}")
        
        self.cap.release()
        cv2.destroyAllWindows()
        
        # 保存标注元数据
        with open(f"{self.save_dir}/annotations.json", "w") as f:
            json.dump(self.annotations, f, indent=2)
        
        print(f"共采集 {len(self.annotations)} 个标注")

if __name__ == "__main__":
    collector = RobotDataCollector()
    collector.run()

数据增强策略：

# 机器人场景数据增强
import albumentations as A
from albumentations.pytorch import ToTensorV2

transform = A.Compose([
    # 几何变换
    A.HorizontalFlip(p=0.5),
    A.ShiftScaleRotate(shift_limit=0.1, scale_limit=0.2, rotate_limit=15, p=0.5),
    
    # 颜色变换（模拟不同光照）
    A.RandomBrightnessContrast(p=0.3),
    A.HueSaturationValue(p=0.3),
    
    # 噪声（模拟传感器噪声）
    A.GaussNoise(p=0.2),
    A.MotionBlur(p=0.2),
    
    # 裁剪
    A.RandomResizedCrop(height=640, width=640, scale=(0.8, 1.0), p=0.5),
    
    # 归一化
    A.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]),
    ToTensorV2()
], bbox_params=A.BboxParams(format='yolo', label_fields=['class_labels']))

2.3 数据集划分

from sklearn.model_selection import train_test_split
import os

def split_dataset(dataset_dir, train_ratio=0.8, val_ratio=0.1, test_ratio=0.1):
    """划分训练/验证/测试集"""
    images = os.listdir(f"{dataset_dir}/images")
    
    # 分层采样（确保每个类别分布均匀）
    train, temp = train_test_split(images, train_size=train_ratio, random_state=42)
    val, test = train_test_split(temp, train_size=val_ratio/(val_ratio+test_ratio), random_state=42)
    
    # 创建目录
    for split in ['train', 'val', 'test']:
        os.makedirs(f"{dataset_dir}/{split}/images", exist_ok=True)
        os.makedirs(f"{dataset_dir}/{split}/labels", exist_ok=True)
    
    # 移动文件
    import shutil
    for split, files in [('train', train), ('val', val), ('test', test)]:
        for img in files:
            src_img = f"{dataset_dir}/images/{img}"
            src_label = f"{dataset_dir}/labels/{img.replace('.jpg', '.txt')}"
            
            dst_img = f"{dataset_dir}/{split}/images/{img}"
            dst_label = f"{dataset_dir}/{split}/labels/{img.replace('.jpg', '.txt')}"
            
            shutil.move(src_img, dst_img)
            shutil.move(src_label, dst_label)
    
    print(f"Train: {len(train)}, Val: {len(val)}, Test: {len(test)}")

split_dataset("dataset")

3. 模型训练

3.1 安装 YOLOv10

# 克隆官方仓库
git clone https://github.com/THU-MIG/yolov10.git
cd yolov10

# 安装依赖
pip install -r requirements.txt
pip install -e .

# 验证安装
yolo predict model=yolov10n.pt source='ultralytics/assets/bus.jpg'

3.2 训练配置

# robot_detect.yaml
task: detect
mode: train
model: yolov10n.pt  # 预训练模型
data: robot_dataset.yaml

# 训练参数
epochs: 100
batch: 16
imgsz: 640
device: 0  # GPU

# 优化器
optimizer: AdamW
lr0: 0.001
lrf: 0.01
momentum: 0.9
weight_decay: 0.0005

# 数据增强
hsv_h: 0.015
hsv_s: 0.7
hsv_v: 0.4
degrees: 10.0
translate: 0.1
scale: 0.5
flipud: 0.0
fliplr: 0.5
mosaic: 1.0
mixup: 0.1

# 保存
project: runs/detect
name: robot_v1
save: true
save_period: 10  # 每10轮保存
# robot_dataset.yaml
path: ./dataset  # 数据集路径
train: train/images
val: val/images
test: test/images

# 类别定义（根据实际需求修改）
names:
  0: red_cube
  1: blue_cube
  2: green_cube
  3: screwdriver
  4: wrench
  5: person
  6: robot_arm
  7: obstacle

3.3 开始训练

# 命令行训练
yolo train data=robot_dataset.yaml model=yolov10n.pt epochs=100 batch=16

# Python API 训练
from ultralytics import YOLO

model = YOLO("yolov10n.pt")
results = model.train(
    data="robot_dataset.yaml",
    epochs=100,
    batch=16,
    imgsz=640,
    device=0
)

3.4 训练监控

# 实时监控训练
import matplotlib.pyplot as plt
from ultralytics.utils.plotting import plot_results

# 绘制训练曲线
plot_results("runs/detect/robot_v1/results.csv")

# 查看验证结果
model = YOLO("runs/detect/robot_v1/weights/best.pt")
metrics = model.val(data="robot_dataset.yaml")

print(f"mAP@50: {metrics.box.map50:.3f}")
print(f"mAP@50-95: {metrics.box.map:.3f}")
print(f"Precision: {metrics.box.mp:.3f}")
print(f"Recall: {metrics.box.mr:.3f}")

4. 模型优化

4.1 模型量化

INT8 量化（TensorRT）：

#!/usr/bin/env python3
"""TensorRT INT8 量化"""
import torch
from ultralytics import YOLO
from torch.utils.data import DataLoader

# 加载模型
model = YOLO("runs/detect/robot_v1/weights/best.pt")

# 准备校准数据（300 张图像）
calibration_data = []
for i in range(300):
    img = load_image(f"dataset/calibration/{i}.jpg")
    calibration_data.append(img)

# 导出为 TensorRT
model.export(
    format="engine",
    half=False,  # FP32
    int8=True,   # INT8 量化
    data="robot_dataset.yaml",
    workspace=4,  # GB
    nms=True
)

print("模型已导出为 TensorRT INT8")

性能对比：

精度模型大小推理时间 (Jetson Nano)mAP
FP3232 MB45 ms58.5
FP1616 MB28 ms58.3
INT88 MB15 ms57.8

4.2 模型剪枝

# 结构化剪枝
from ultralytics import YOLO
import torch.nn.utils.prune as prune

model = YOLO("runs/detect/robot_v1/weights/best.pt")

# 剪枝 30% 的卷积层
for name, module in model.model.named_modules():
    if isinstance(module, torch.nn.Conv2d):
        prune.l1_unstructured(module, name='weight', amount=0.3)
        prune.remove(module, 'weight')

# 微调
model.train(data="robot_dataset.yaml", epochs=20, batch=16)

# 导出剪枝后的模型
model.export(format="onnx")

4.3 ONNX 导出

# 导出为 ONNX（跨平台）
model = YOLO("runs/detect/robot_v1/weights/best.pt")

model.export(
    format="onnx",
    dynamic=True,  # 动态输入尺寸
    simplify=True,
    opset=12
)

# 验证 ONNX 模型
import onnx
model_onnx = onnx.load("best.onnx")
onnx.checker.check_model(model_onnx)
print("ONNX 模型验证通过")

5. 边缘设备部署

5.1 Jetson Nano 部署

环境准备：

# 安装 JetPack（包含 CUDA、cuDNN、TensorRT）
# 从 NVIDIA 官网下载 JetPack 4.6+

# 安装依赖
sudo apt update
sudo apt install python3-pip cmake libopenblas-dev

# 安装 PyTorch（Jetson 版本）
pip3 install torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/cu102

# 安装 TensorRT Python API
pip3 install tensorrt

# 安装 ONNX Runtime（GPU 版本）
pip3 install onnxruntime-gpu

推理代码：

#!/usr/bin/env python3
"""Jetson Nano 推理"""
import cv2
import numpy as np
import tensorrt as trt
import pycuda.driver as cuda
import pycuda.autoinit
from ultralytics import YOLO

class TensorRTInference:
    def __init__(self, engine_path):
        # 加载 TensorRT 引擎
        with open(engine_path, "rb") as f:
            runtime = trt.Runtime(trt.Logger(trt.Logger.WARNING))
            self.engine = runtime.deserialize_cuda_engine(f.read())
        
        self.context = self.engine.create_execution_context()
        
        # 分配内存
        self.inputs = []
        self.outputs = []
        self.bindings = []
        
        for binding in self.engine:
            size = trt.volume(self.engine.get_binding_shape(binding))
            dtype = trt.nptype(self.engine.get_binding_dtype(binding))
            host_mem = cuda.pagelocked_empty(size, dtype)
            device_mem = cuda.mem_alloc(host_mem.nbytes)
            
            self.bindings.append(int(device_mem))
            
            if self.engine.binding_is_input(binding):
                self.inputs.append({"host": host_mem, "device": device_mem})
            else:
                self.outputs.append({"host": host_mem, "device": device_mem})
    
    def preprocess(self, image):
        """预处理"""
        # 调整尺寸
        input_shape = self.engine.get_binding_shape(0)
        image = cv2.resize(image, (input_shape[3], input_shape[2]))
        
        # 归一化
        image = image.astype(np.float32) / 255.0
        image = np.transpose(image, (2, 0, 1))  # HWC -> CHW
        image = np.expand_dims(image, axis=0)
        
        return image
    
    def infer(self, image):
        """推理"""
        # 预处理
        input_data = self.preprocess(image)
        np.copyto(self.inputs[0]["host"], input_data.ravel())
        
        # 传输到 GPU
        for inp in self.inputs:
            cuda.memcpy_htod(inp["device"], inp["host"])
        
        # 执行推理
        self.context.execute(2, self.bindings)
        
        # 传输回 CPU
        for out in self.outputs:
            cuda.memcpy_dtoh(out["host"], out["device"])
        
        return [out["host"] for out in self.outputs]
    
    def postprocess(self, outputs, conf_threshold=0.5):
        """后处理"""
        # 解析输出（根据模型结构）
        boxes = outputs[0].reshape(-1, 6)  # x1, y1, x2, y2, conf, class
        
        # 过滤低置信度
        mask = boxes[:, 4] > conf_threshold
        boxes = boxes[mask]
        
        return boxes

# 使用示例
engine = TensorRTInference("best.engine")
cap = cv2.VideoCapture(0)

while True:
    ret, frame = cap.read()
    if not ret:
        break
    
    # 推理
    outputs = engine.infer(frame)
    boxes = engine.postprocess(outputs)
    
    # 绘制结果
    for box in boxes:
        x1, y1, x2, y2, conf, cls = box
        cv2.rectangle(frame, (int(x1), int(y1)), (int(x2), int(y2)), (0, 255, 0), 2)
        cv2.putText(frame, f"{conf:.2f}", (int(x1), int(y1)-10),
                   cv2.FONT_HERSHEY_SIMPLEX, 0.5, (0, 255, 0), 2)
    
    cv2.imshow("Detection", frame)
    if cv2.waitKey(1) == ord('q'):
        break

cap.release()
cv2.destroyAllWindows()

性能优化：

# 使用 CUDA Stream 并行处理
import pycuda.driver as cuda

class OptimizedInference(TensorRTInference):
    def __init__(self, engine_path):
        super().__init__(engine_path)
        self.stream = cuda.Stream()
    
    def infer_async(self, image_queue):
        """异步推理（流水线）"""
        while True:
            image = image_queue.get()
            if image is None:
                break
            
            # 异步传输
            input_data = self.preprocess(image)
            cuda.memcpy_htod_async(self.inputs[0]["device"], input_data.ravel(), self.stream)
            
            # 异步推理
            self.context.execute_async_v2(self.bindings, self.stream.handle)
            
            # 异步传输回
            for out in self.outputs:
                cuda.memcpy_dtoh_async(out["host"], out["device"], self.stream)
            
            self.stream.synchronize()
            
            yield [out["host"] for out in self.outputs]

5.2 树莓派部署

环境准备：

# 安装 OpenCV
sudo apt install python3-opencv

# 安装 ONNX Runtime
pip3 install onnxruntime

# 或使用 OpenVINO（Intel NCS2 加速棒）
pip3 install openvino-dev

ONNX Runtime 推理：

#!/usr/bin/env python3
"""树莓派 ONNX Runtime 推理"""
import cv2
import numpy as np
import onnxruntime as ort

class ONNXInference:
    def __init__(self, model_path):
        # 加载模型
        self.session = ort.InferenceSession(model_path)
        self.input_name = self.session.get_inputs()[0].name
        self.input_shape = self.session.get_inputs()[0].shape
        
        print(f"输入形状: {self.input_shape}")
    
    def preprocess(self, image):
        """预处理"""
        # 调整尺寸
        target_size = (self.input_shape[3], self.input_shape[2])
        image = cv2.resize(image, target_size)
        
        # 归一化
        image = image.astype(np.float32) / 255.0
        image = np.transpose(image, (2, 0, 1))
        image = np.expand_dims(image, axis=0)
        
        return image
    
    def infer(self, image):
        """推理"""
        input_data = self.preprocess(image)
        outputs = self.session.run(None, {self.input_name: input_data})
        return outputs
    
    def postprocess(self, outputs, conf_threshold=0.5):
        """后处理（简化版）"""
        # YOLOv10 输出格式
        predictions = outputs[0]
        
        # 解析边界框和置信度
        boxes = []
        for pred in predictions[0]:
            conf = pred[4]
            if conf > conf_threshold:
                x1, y1, x2, y2 = pred[:4]
                cls = np.argmax(pred[5:])
                boxes.append([x1, y1, x2, y2, conf, cls])
        
        return np.array(boxes)

# 使用示例
model = ONNXInference("best.onnx")
cap = cv2.VideoCapture(0)

while True:
    ret, frame = cap.read()
    if not ret:
        break
    
    outputs = model.infer(frame)
    boxes = model.postprocess(outputs)
    
    for box in boxes:
        x1, y1, x2, y2, conf, cls = box
        cv2.rectangle(frame, (int(x1), int(y1)), (int(x2), int(y2)), (0, 255, 0), 2)
    
    cv2.imshow("Detection", frame)
    if cv2.waitKey(1) == ord('q'):
        break

cap.release()
cv2.destroyAllWindows()

性能优化（OpenVINO）：

# 转换为 OpenVINO IR 格式
mo --input_model best.onnx --output_dir openvino_model --data_type FP16

# 使用 OpenVINO 推理
pip3 install openvino
from openvino.runtime import Core

ie = Core()
model = ie.compile_model("openvino_model/best.xml", "CPU")

# 推理
input_layer = model.input(0)
output_layer = model.output(0)

result = model.infer_new_request({input_layer: input_data})

5.3 ESP32 部署（轻量级）

模型转换（TFLite）：

# 转换为 TFLite
pip install tensorflow

# Python 转换
import tensorflow as tf

converter = tf.lite.TFLiteConverter.from_onnx_model("best.onnx")
converter.optimizations = [tf.lite.Optimize.DEFAULT]
tflite_model = converter.convert()

with open("model.tflite", "wb") as f:
    f.write(tflite_model)

ESP32 推理（简化版）：

// ESP32 + TensorFlow Lite Micro
#include <TensorFlowLite_ESP32.h>
#include "model_data.h"

tflite::MicroErrorReporter tflite_error_reporter;
tflite::MicroInterpreter* interpreter = nullptr;

void setup() {
  // 加载模型
  static tflite::MicroMutableOpResolver<10> resolver;
  resolver.AddConv2D();
  resolver.AddDepthwiseConv2D();
  resolver.AddReshape();
  
  const tflite::Model* model = tflite::GetModel(g_model);
  
  static uint8_t tensor_buffer[20 * 1024];
  static tflite::SimpleTensorAllocator allocator(tensor_buffer, sizeof(tensor_buffer));
  
  interpreter = new tflite::MicroInterpreter(
    model, resolver, &allocator, &tflite_error_reporter);
  
  interpreter->AllocateTensors();
}

void loop() {
  // 获取摄像头图像
  camera_fb_t* fb = esp_camera_fb_get();
  
  // 预处理
  TfLiteTensor* input = interpreter->input(0);
  preprocess_image(fb->buf, input->data.int8);
  
  // 推理
  interpreter->Invoke();
  
  // 后处理
  TfLiteTensor* output = interpreter->output(0);
  process_detections(output->data.int8);
  
  esp_camera_fb_return(fb);
  delay(100);  // 10 FPS
}

6. 实际应用

6.1 目标跟踪（DeepSORT）

#!/usr/bin/env python3
"""YOLOv10 + DeepSORT 目标跟踪"""
import cv2
import numpy as np
from ultralytics import YOLO
from deep_sort_realtime.deepsort_tracker import DeepSort

class RobotTracker:
    def __init__(self):
        # 检测模型
        self.detector = YOLO("best.pt")
        
        # 跟踪器
        self.tracker = DeepSort(
            max_age=30,           # 最大跟踪帧数
            n_init=3,             # 初始化所需帧数
            nms_max_overlap=0.5,  # NMS 阈值
            embedder="mobilenet"  # 特征提取器
        )
        
        # 轨迹历史
        self.track_history = {}
    
    def detect_and_track(self, frame):
        """检测 + 跟踪"""
        # 检测
        results = self.detector(frame, conf=0.5, verbose=False)
        detections = []
        
        for result in results:
            boxes = result.boxes.xyxy.cpu().numpy()
            confs = result.boxes.conf.cpu().numpy()
            classes = result.boxes.cls.cpu().numpy()
            
            for box, conf, cls in zip(boxes, confs, classes):
                detections.append(([box[0], box[1], box[2]-box[0], box[3]-box[1]], conf, int(cls)))
        
        # 跟踪
        tracks = self.tracker.update_tracks(detections, frame=frame)
        
        # 更新轨迹历史
        for track in tracks:
            if not track.is_confirmed():
                continue
            
            track_id = track.track_id
            ltrb = track.to_ltrb()
            
            if track_id not in self.track_history:
                self.track_history[track_id] = []
            
            self.track_history[track_id].append(
                ((ltrb[0]+ltrb[2])/2, (ltrb[1]+ltrb[3])/2)  # 中心点
            )
            
            # 保留最近 50 帧
            if len(self.track_history[track_id]) > 50:
                self.track_history[track_id].pop(0)
        
        return tracks
    
    def draw_tracks(self, frame, tracks):
        """绘制跟踪结果"""
        for track in tracks:
            if not track.is_confirmed():
                continue
            
            track_id = track.track_id
            ltrb = track.to_ltrb()
            
            # 绘制边界框
            cv2.rectangle(frame, (int(ltrb[0]), int(ltrb[1])),
                         (int(ltrb[2]), int(ltrb[3])), (0, 255, 0), 2)
            
            # 绘制 ID
            cv2.putText(frame, f"ID: {track_id}", (int(ltrb[0]), int(ltrb[1])-10),
                       cv2.FONT_HERSHEY_SIMPLEX, 0.5, (0, 255, 0), 2)
            
            # 绘制轨迹
            if track_id in self.track_history:
                points = np.array(self.track_history[track_id], dtype=np.int32)
                cv2.polylines(frame, [points], False, (255, 0, 0), 2)
        
        return frame

# 使用示例
tracker = RobotTracker()
cap = cv2.VideoCapture(0)

while True:
    ret, frame = cap.read()
    if not ret:
        break
    
    tracks = tracker.detect_and_track(frame)
    frame = tracker.draw_tracks(frame, tracks)
    
    cv2.imshow("Tracking", frame)
    if cv2.waitKey(1) == ord('q'):
        break

cap.release()
cv2.destroyAllWindows()

6.2 机器人避障

#!/usr/bin/env python3
"""基于 YOLOv10 的机器人避障"""
import cv2
import numpy as np
from ultralytics import YOLO
import time

class ObstacleAvoidance:
    def __init__(self):
        self.detector = YOLO("best.pt")
        
        # 相机参数（需要标定）
        self.focal_length = 600  # 像素
        self.real_width = 0.1    # 目标实际宽度（米）
        
        # 安全距离
        self.safe_distance = 0.5  # 米
        
        # 控制参数
        self.kp_linear = 0.5
        self.kp_angular = 1.0
    
    def estimate_distance(self, box_width):
        """基于边界框宽度估计距离"""
        # 针孔相机模型
        distance = (self.real_width * self.focal_length) / box_width
        return distance
    
    def detect_obstacles(self, frame):
        """检测障碍物"""
        results = self.detector(frame, conf=0.5, verbose=False)
        
        obstacles = []
        for result in results:
            boxes = result.boxes.xyxy.cpu().numpy()
            confs = result.boxes.conf.cpu().numpy()
            
            for box, conf in zip(boxes, confs):
                x1, y1, x2, y2 = box
                width = x2 - x1
                center_x = (x1 + x2) / 2
                
                # 估计距离
                distance = self.estimate_distance(width)
                
                obstacles.append({
                    'bbox': (x1, y1, x2, y2),
                    'center_x': center_x,
                    'distance': distance,
                    'confidence': conf
                })
        
        return obstacles
    
    def compute_velocity(self, obstacles, frame_width):
        """计算避障速度"""
        if not obstacles:
            # 无障碍物，前进
            return 0.3, 0.0
        
        # 找到最近的障碍物
        closest = min(obstacles, key=lambda x: x['distance'])
        
        # 判断方向
        frame_center = frame_width / 2
        obstacle_center = closest['center_x']
        
        # 计算角速度（远离障碍物）
        if obstacle_center < frame_center:
            angular = self.kp_angular * (frame_center - obstacle_center) / frame_center
        else:
            angular = -self.kp_angular * (obstacle_center - frame_center) / frame_center
        
        # 计算线速度（根据距离）
        if closest['distance'] < self.safe_distance:
            linear = 0.0  # 停止
        else:
            linear = self.kp_linear * min(1.0, closest['distance'] / 2.0)
        
        return linear, angular
    
    def run(self):
        """主循环"""
        cap = cv2.VideoCapture(0)
        
        while True:
            ret, frame = cap.read()
            if not ret:
                break
            
            # 检测障碍物
            obstacles = self.detect_obstacles(frame)
            
            # 计算速度
            linear, angular = self.compute_velocity(obstacles, frame.shape[1])
            
            # 发送控制指令（这里需要连接到机器人）
            # robot.set_velocity(linear, angular)
            
            # 可视化
            for obs in obstacles:
                x1, y1, x2, y2 = map(int, obs['bbox'])
                color = (0, 0, 255) if obs['distance'] < self.safe_distance else (0, 255, 0)
                cv2.rectangle(frame, (x1, y1), (x2, y2), color, 2)
                cv2.putText(frame, f"{obs['distance']:.2f}m", (x1, y1-10),
                           cv2.FONT_HERSHEY_SIMPLEX, 0.5, color, 2)
            
            # 显示速度
            cv2.putText(frame, f"Linear: {linear:.2f}", (10, 30),
                       cv2.FONT_HERSHEY_SIMPLEX, 0.5, (255, 255, 255), 2)
            cv2.putText(frame, f"Angular: {angular:.2f}", (10, 60),
                       cv2.FONT_HERSHEY_SIMPLEX, 0.5, (255, 255, 255), 2)
            
            cv2.imshow("Obstacle Avoidance", frame)
            if cv2.waitKey(1) == ord('q'):
                break
        
        cap.release()
        cv2.destroyAllWindows()

if __name__ == "__main__":
    avoidance = ObstacleAvoidance()
    avoidance.run()

6.3 抓取定位

抓取定位是机器人操作的核心技能。结合 宇树 Go2 ROS2 开发指南 中的相机标定方法，可以实现精准的 3D 抓取定位。

#!/usr/bin/env python3
"""YOLOv10 抓取定位"""
import cv2
import numpy as np
from ultralytics import YOLO

class GraspPlanner:
    def __init__(self):
        self.detector = YOLO("best.pt")
        
        # 相机标定参数
        self.camera_matrix = np.array([
            [600, 0, 320],
            [0, 600, 240],
            [0, 0, 1]
        ])
        self.dist_coeffs = np.zeros((4, 1))
        
        # 相机到机械臂基座的变换（需要标定）
        self.camera_to_base = np.array([
            [1, 0, 0, 0],
            [0, 1, 0, 0],
            [0, 0, 1, 0.3],  # 相机高度 0.3m
            [0, 0, 0, 1]
        ])
    
    def detect_objects(self, frame):
        """检测可抓取物体"""
        results = self.detector(frame, conf=0.6, verbose=False)
        
        objects = []
        for result in results:
            boxes = result.boxes.xyxy.cpu().numpy()
            confs = result.boxes.conf.cpu().numpy()
            classes = result.boxes.cls.cpu().numpy()
            
            for box, conf, cls in zip(boxes, confs, classes):
                x1, y1, x2, y2 = box
                center_x = (x1 + x2) / 2
                center_y = (y1 + y2) / 2
                
                objects.append({
                    'bbox': (x1, y1, x2, y2),
                    'center_2d': (center_x, center_y),
                    'confidence': conf,
                    'class_id': int(cls)
                })
        
        return objects
    
    def estimate_pose(self, obj, depth_image):
        """估计物体 3D 位姿"""
        center_x, center_y = obj['center_2d']
        
        # 从深度图获取深度值
        depth = depth_image[int(center_y), int(center_x)] / 1000.0  # mm -> m
        
        # 像素坐标转 3D 坐标
        x = (center_x - self.camera_matrix[0, 2]) * depth / self.camera_matrix[0, 0]
        y = (center_y - self.camera_matrix[1, 2]) * depth / self.camera_matrix[1, 1]
        z = depth
        
        # 转换到机械臂基座坐标系
        pos_camera = np.array([x, y, z, 1])
        pos_base = self.camera_to_base @ pos_camera
        
        return pos_base[:3]
    
    def plan_grasp(self, obj, pose_3d):
        """规划抓取位姿"""
        # 简化版：直接抓取物体中心
        grasp_pose = {
            'position': pose_3d,
            'orientation': [0, 0, 0, 1],  # 四元数
            'approach_vector': [0, 0, -1]  # 从上方接近
        }
        
        return grasp_pose
    
    def run(self):
        """主循环"""
        cap = cv2.VideoCapture(0)
        
        # 假设深度相机
        # depth_cap = cv2.VideoCapture(1)
        
        while True:
            ret, frame = cap.read()
            if not ret:
                break
            
            # 模拟深度图（实际应从深度相机获取）
            depth_image = np.random.randint(500, 1500, (480, 640), dtype=np.uint16)
            
            # 检测物体
            objects = self.detect_objects(frame)
            
            # 选择置信度最高的物体
            if objects:
                target = max(objects, key=lambda x: x['confidence'])
                
                # 估计位姿
                pose_3d = self.estimate_pose(target, depth_image)
                
                # 规划抓取
                grasp_pose = self.plan_grasp(target, pose_3d)
                
                print(f"目标: {target['class_id']}")
                print(f"3D 位置: {pose_3d}")
                print(f"抓取位姿: {grasp_pose}")
                
                # 可视化
                x1, y1, x2, y2 = map(int, target['bbox'])
                cv2.rectangle(frame, (x1, y1), (x2, y2), (0, 255, 0), 2)
                cv2.circle(frame, (int(target['center_2d'][0]), int(target['center_2d'][1])),
                          5, (0, 0, 255), -1)
            
            cv2.imshow("Grasp Planning", frame)
            if cv2.waitKey(1) == ord('q'):
                break
        
        cap.release()
        cv2.destroyAllWindows()

if __name__ == "__main__":
    planner = GraspPlanner()
    planner.run()

7. 性能优化技巧

7.1 推理加速

1. 批处理推理

# 批量推理（提高吞吐量）
images = [img1, img2, img3, img4]
results = model(images, batch=4)

2. 多线程预处理

from concurrent.futures import ThreadPoolExecutor

class ParallelInference:
    def __init__(self, model):
        self.model = model
        self.executor = ThreadPoolExecutor(max_workers=4)
    
    def preprocess_batch(self, images):
        """多线程预处理"""
        futures = [self.executor.submit(self.preprocess, img) for img in images]
        return [f.result() for f in futures]

3. CUDA Stream 并行

import torch.cuda

stream1 = torch.cuda.Stream()
stream2 = torch.cuda.Stream()

with torch.cuda.stream(stream1):
    output1 = model(input1)

with torch.cuda.stream(stream2):
    output2 = model(input2)

torch.cuda.synchronize()

7.2 内存优化

1. 模型卸载

# 将不常用的层卸载到 CPU
model.model.to('cpu')
model.model.layer1.to('cuda')  # 只保留关键层在 GPU

2. 内存池

# PyTorch 内存池
torch.cuda.memory.set_per_process_memory_fraction(0.8)  # 限制 GPU 内存使用

7.3 精度-速度权衡

优化策略速度提升精度损失适用场景
FP16 推理2x<0.5%通用
INT8 量化4x1-2%边缘设备
模型剪枝1.5x1-3%内存受限
降低输入分辨率2-4x2-5%实时性要求高
减少后处理1.2x0%所有场景

总结

YOLOv10 为机器人视觉系统提供了强大的目标检测能力。通过本文，你已掌握：

  1. ✅ YOLOv10 架构与优势
  2. ✅ 机器人数据集准备与训练
  3. ✅ 模型优化（量化、剪枝、TensorRT）
  4. ✅ 边缘设备部署（Jetson、树莓派、ESP32）
  5. ✅ 实际应用（跟踪、避障、抓取）
  6. ✅ 性能优化技巧

下一步建议：

  • 针对你的具体场景微调模型
  • 探索多模态融合（视觉 + 深度 + IMU）
  • 研究端到端学习（视觉直接到控制）
  • 关注 YOLO 系列最新进展

祝你的机器人视觉项目成功！🤖👁️

FAQ

YOLOv10 和 YOLOv8 有什么区别？

YOLOv10 最大的改进是移除了 NMS（非极大值抑制）后处理，推理延迟降低约 30%。同时采用了更高效的架构设计，在相同精度下模型体积更小，更适合边缘设备部署。

YOLOv10 能在树莓派上实时运行吗？

可以。使用 ONNX Runtime 或 OpenVINO 优化后，YOLOv10n 在树莓派 5 上可达到 5-10 FPS。如需更高帧率，建议使用 Jetson Nano 配合 TensorRT，可达 30+ FPS。

如何提升 YOLOv10 在机器人上的检测速度？

主要优化手段包括：1) INT8 量化（4x 加速）；2) TensorRT 引擎优化；3) 降低输入分辨率（如 640→320）；4) 批处理推理；5) 多线程预处理流水线。

YOLOv10 适合哪些机器人应用场景？

YOLOv10 适用于：目标检测与分类、物体抓取定位、自主避障、人员跟踪、工业质检等。特别适合需要实时响应的移动机器人和协作机器人场景。