yolo数据流传输可能有问题,导致帧率极低

:bar_chart: 代码评估:320×320 优化版

你的优化非常到位!从 640→320 是最有效的提速手段。以下是详细评估:


:white_check_mark: 已实现的关键优化

优化项 效果 状态
输入尺寸 640→320 推理速度↑4 倍 :white_check_mark:
hobot_dnn→hbm_runtime 接口开销↓ :white_check_mark:
纯 NumPy NMS 避免 .tolist() 复制 :white_check_mark:
Proto 形状修正 (32,80,80) 保持完整 :white_check_mark:
摄像头缓冲优化 BUFFERSIZE=1 + MJPG :white_check_mark:

预期 FPS:10-12 → 40-60 :rocket:


:warning: 剩余瓶颈与修正建议

1. Mask 膨胀仍用 np.roll(每帧复制)

# 当前:np.roll 创建新数组(~3-5ms/帧)
u = np.roll(dm, -n, axis=0); u[-n:, :] = 0
dm = np.maximum.reduce([dm, u, d, l, r])

# 优化:原地切片赋值(~1ms/帧)
def expand_mask_inplace(dm, cid):
    if cid == OPPONENT_ZONE and OPPONENT_ZONE_EXPAND > 0:
        n = OPPONENT_ZONE_EXPAND
        dm[n:, :] = np.maximum(dm[n:, :], dm[:-n, :])
        dm[:-n, :] = np.maximum(dm[:-n, :], dm[n:, :])
        dm[:, n:] = np.maximum(dm[:, n:], dm[:, :-n])
        dm[:, :-n] = np.maximum(dm[:, :-n], dm[:, n:])
    elif cid == OWN_ZONE:
        if ZONE_MASK_EXPAND > 0:
            n = ZONE_MASK_EXPAND
            dm[n:, :] = np.maximum(dm[n:, :], dm[:-n, :])
        if ZONE_BOTTOM_SHRINK > 0:
            dm[-ZONE_BOTTOM_SHRINK:, :] = 0
    return dm

# 调用处修改
dm = det['mask']  # 不 copy
dm = expand_mask_inplace(dm, cid)
det['mask_dilated'] = dm

2. 缺少分层计时(无法定位瓶颈)

# 在 main() 循环中添加
t0 = time.perf_counter()
img = cv2.resize(frame, (INPUT_SIZE, INPUT_SIZE))
t1 = time.perf_counter()
nv12 = bgr2nv12(img).reshape(1, INPUT_SIZE * 3 // 2, INPUT_SIZE, 1)
t2 = time.perf_counter()
raw = model.run({mname: {iname: nv12}})[mname]
t3 = time.perf_counter()
dets = process_output_seg(raw, onames, need_mask=True)
t4 = time.perf_counter()

if fps_count >= 30:
    print(f"resize:{(t1-t0)*1000:5.1f}ms nv12:{(t2-t1)*1000:5.1f}ms "
          f"bpu:{(t3-t2)*1000:5.1f}ms post:{(t4-t3)*1000:5.1f}ms "
          f"FPS:{fps_val:5.1f}")

预期输出:

resize:  2.5ms nv12:  3.2ms bpu: 12.5ms post: 18.3ms FPS: 45.2

3. NMS 是 class-agnostic(可能误删同类目标)

# 当前:所有类别一起 NMS
keep = nms(boxes, scores, NMS_THRES)

# 建议:按类别分别 NMS
def nms_per_class(boxes, scores, cls, thresh):
    keep_all = []
    for c in range(NUM_CLS):
        mask = cls == c
        if not mask.any(): continue
        keep_c = nms(boxes[mask], scores[mask], thresh)
        keep_all.append(np.where(mask)[0][keep_c])
    return np.concatenate(keep_all) if keep_all else np.array([], dtype=np.int32)

# 调用
keep = nms_per_class(boxes, scores, cls_arr, NMS_THRES)

4. BGR2NV12 可进一步简化

# 当前:I420 中间格式 + 手动 interleaved
yuv = cv2.cvtColor(image, cv2.COLOR_BGR2YUV_I420)
# ... 多步切片拼接 ...

# 优化:直接输出 NV12(OpenCV 4.5+ 支持)
def bgr2nv12_fast(image):
    nv12 = cv2.cvtColor(image, cv2.COLOR_BGR2NV12)
    return nv12.ravel()  # 一维展开

:bar_chart: 性能对比预估

版本 BPU 推理 后处理 总 FPS
原版 640 50-80ms 30-50ms 10-12
当前 320 10-15ms 15-25ms 40-50
优化后 320 10-15ms 8-12ms 50-65

:white_check_mark: 立即可执行清单

# 1. 设置 CPU 性能模式
sudo bash -c "echo performance > /sys/devices/system/cpu/cpufreq/policy0/scaling_governor"
cat /sys/devices/system/cpu/cpufreq/policy0/scaling_cur_freq  # 应显示 1800000

# 2. 运行代码,记录分层计时输出
python3 your_v2_script.py

# 3. 如果 post > 20ms,优先优化 Mask 膨胀逻辑

:bullseye: 下一步建议

  1. 先跑分层计时,确认瓶颈是否仍在后处理
  2. 如果 BPU > 20ms:检查量化配置 input_shape 是否与 320 一致
  3. 如果 post > 20ms:应用上面的 Mask 原地优化
  4. 如果小目标检测变差:测试 416×416 作为折中

:books: 参考资源

把分层计时输出发出来,我可以帮你进一步定位具体优化点!:rocket: