人体检测和跟踪使用USB摄像头发布图片，在PC端的浏览器输入http://IP 查看图像失败

应用开发 机器人开发
,
1

Boxs基础算法 FCOS目标检测算法示例订阅sensor package发布的图片，经过算法推理后发布hobot AI msg，通过websocket package实现在PC端浏览器上渲染显示sensor发布的图片和对应的AI结果。

使用USB摄像头发布图片

# 配置TogetherROS环境-
source /opt/tros/setup.bash-
# 配置USB摄像头-
export CAM_TYPE=usb-
# 启动launch文件-
ros2 launch dnn_node_example hobot_dnn_node_example.launch.py config_file:=config/fcosworkconfig.json image_width:=480 image_height:=272，实现在PC端浏览器上渲染显示sensor发布的图片，说明了摄像头与旭日X3的通信的好的

目前的_TogetherROS 版本是_

root@ubuntu:~# sudo apt list | grep hhp

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

hhp-verify/unknown,now 1.0.0 arm64 [installed,automatic]

hhp/unknown,now 1.0.5 arm64 [installed]

##########################################################################################################################退出上面的FCOS目标检测算法示例后在运行人体检测和跟踪 使用USB摄像头发布图片，使用如下的命令

# 配置TogetherROS环境-
source /opt/tros/setup.bash-
# 从TogetherROS的安装路径中拷贝出运行示例需要的配置文件。-
cp -r /opt/tros/lib/mono2d_body_detection/config/ .-
# 配置USB摄像头-
export CAM_TYPE=usb-
# 启动launch文件-
ros2 launch mono2d_body_detection hobot_mono2d_body_detection.launch.py

输出信息好的，但是在PC端的浏览器输入http://IP 后无画面输出

root@ubuntu:~# source /opt/tros/setup.bash

root@ubuntu:~# cp -r /opt/tros/lib/mono2d_body_detection/config/ .

root@ubuntu:~# export CAM_TYPE=usb

root@ubuntu:~# ros2 launch mono2d_body_detection hobot_mono2d_body_detection.launch.py

[INFO] [launch]: All log files can be found below /root/.ros/log/2022-09-04-09-36-54-763570-ubuntu-13705

[INFO] [launch]: Default logging verbosity is set to INFO

camera_type is usb

using usb cam

webserver has launch

[INFO] [hobot_usb_cam-1]: process started with pid [13730]

[INFO] [hobot_codec_republish-2]: process started with pid [13732]

[INFO] [mono2d_body_detection-3]: process started with pid [13734]

[INFO] [websocket-4]: process started with pid [13736]

[mono2d_body_detection-3] [WARN] [1662255415.231307294] [example]: This is mono2d body det example!

[mono2d_body_detection-3] [WARN] [1662255415.396168187] [mono2d_body_det]: Parameter:

[mono2d_body_detection-3] is_sync_mode_: 0

[mono2d_body_detection-3] model_file_name_: config/multitask_body_head_face_hand_kps_960x544.hbm

[mono2d_body_detection-3] is_shared_mem_sub: 1

[mono2d_body_detection-3] ai_msg_pub_topic_name: /hobot_mono2d_body_detection

[mono2d_body_detection-3] [BPU_PLAT]BPU Platform Version(1.3.1)!

[mono2d_body_detection-3] [C][13734][09-04][09:36:55:402][configuration.cpp:49][EasyDNN]EasyDNN version: 0.4.11

[mono2d_body_detection-3] [HBRT] set log level as 0. version = 3.14.5

[mono2d_body_detection-3] [DNN] Runtime version = 1.9.7_(3.14.5 HBRT)

[mono2d_body_detection-3] [WARN] [1662255415.529054269] [mono2d_body_det]: Create hbmem_subscription with topic_name: /hbmem_img

[mono2d_body_detection-3] (MOTMethod.cpp:34): MOTMethod::Init config/iou2_euclid_method_param.json

[mono2d_body_detection-3]

[mono2d_body_detection-3] (IOU2.cpp:29): IOU2 Mot::Init config/iou2_euclid_method_param.json

[mono2d_body_detection-3]

[mono2d_body_detection-3] (MOTMethod.cpp:34): MOTMethod::Init config/iou2_method_param.json

[mono2d_body_detection-3]

[mono2d_body_detection-3] (IOU2.cpp:29): IOU2 Mot::Init config/iou2_method_param.json

[mono2d_body_detection-3]

[mono2d_body_detection-3] (MOTMethod.cpp:34): MOTMethod::Init config/iou2_method_param.json

[mono2d_body_detection-3]

[mono2d_body_detection-3] (IOU2.cpp:29): IOU2 Mot::Init config/iou2_method_param.json

[mono2d_body_detection-3]

[mono2d_body_detection-3] (MOTMethod.cpp:34): MOTMethod::Init config/iou2_method_param.json

[mono2d_body_detection-3]

[mono2d_body_detection-3] (IOU2.cpp:29): IOU2 Mot::Init config/iou2_method_param.json

[mono2d_body_detection-3]

[mono2d_body_detection-3] [WARN] [1662255417.186000511] [dnn]: Task Size: 10 exceeds limit: 10

[mono2d_body_detection-3] [WARN] [1662255417.193967633] [dnn]: Task Size: 10 exceeds limit: 10

[mono2d_body_detection-3] [WARN] [1662255417.203603384] [dnn]: Task Size: 10 exceeds limit: 10

[mono2d_body_detection-3] [WARN] [1662255417.211939479] [dnn]: Task Size: 10 exceeds limit: 10

[mono2d_body_detection-3] [WARN] [1662255417.223609580] [dnn]: Task Size: 10 exceeds limit: 10

[mono2d_body_detection-3] [WARN] [1662255417.232364395] [dnn]: Task Size: 10 exceeds limit: 10

[mono2d_body_detection-3] [WARN] [1662255417.244880767] [dnn]: Task Size: 10 exceeds limit: 10

[mono2d_body_detection-3] [WARN] [1662255417.252839515] [dnn]: Task Size: 10 exceeds limit: 10

[mono2d_body_detection-3] [WARN] [1662255417.264514324] [dnn]: Task Size: 10 exceeds limit: 10

[mono2d_body_detection-3] [WARN] [1662255417.272961120] [dnn]: Task Size: 10 exceeds limit: 10

[mono2d_body_detection-3] [WARN] [1662255417.285066188] [dnn]: Task Size: 10 exceeds limit: 10

[mono2d_body_detection-3] [WARN] [1662255417.292433314] [dnn]: Task Size: 10 exceeds limit: 10

[mono2d_body_detection-3] [WARN] [1662255417.304698371] [dnn]: Task Size: 10 exceeds limit: 10

[mono2d_body_detection-3] [WARN] [1662255417.312309186] [dnn]: Task Size: 10 exceeds limit: 10

[mono2d_body_detection-3] [WARN] [1662255417.324408131] [dnn]: Task Size: 10 exceeds limit: 10

[mono2d_body_detection-3] [WARN] [1662255417.332167644] [dnn]: Task Size: 10 exceeds limit: 10

[mono2d_body_detection-3] [WARN] [1662255417.344043063] [dnn]: Task Size: 10 exceeds limit: 10

[mono2d_body_detection-3] [WARN] [1662255417.351983437] [dnn]: Task Size: 10 exceeds limit: 10

[mono2d_body_detection-3] [WARN] [1662255417.364284325] [dnn]: Task Size: 10 exceeds limit: 10

[mono2d_body_detection-3] [WARN] [1662255417.372096084] [dnn]: Task Size: 10 exceeds limit: 10

[mono2d_body_detection-3] [WARN] [1662255417.384635579] [dnn]: Task Size: 10 exceeds limit: 10

[mono2d_body_detection-3] [WARN] [1662255417.392529916] [dnn]: Task Size: 10 exceeds limit: 10

[mono2d_body_detection-3] [WARN] [1662255417.404577114] [dnn]: Task Size: 10 exceeds limit: 10

[mono2d_body_detection-3] [WARN] [1662255417.412513197] [dnn]: Task Size: 10 exceeds limit: 10

[mono2d_body_detection-3] [WARN] [1662255417.424381200] [dnn]: Task Size: 10 exceeds limit: 10

[mono2d_body_detection-3] [WARN] [1662255417.436165334] [dnn]: Task Size: 10 exceeds limit: 10

2

运行保存了，没有算法推理输出，fcos 程序运行完是正常退出的吗。设备重启下重新运行人体检测看看。这种问题没遇到过，测试完麻烦回复下。

3

今天早上我重新再次测试 使用USB摄像头发布图片，连续输出几次后有不输出了

#--------------------------------------------------------------------------------------------------------

# 配置TogetherROS环境

source /opt/tros/setup.bash

# 从TogetherROS的安装路径中拷贝出运行示例需要的配置文件。

cp -r /opt/tros/lib/mono2d_body_detection/config/ .

# 配置USB摄像头

export CAM_TYPE=usb

# 启动launch文件 ros2 launch mono2d_body_detection hobot_mono2d_body_detection.launch.py

#--------------------------------------------------------------------------------------------------------

root@ubuntu:~# source /opt/tros/loccl_setup.bash

rsource /opt/tros/setup.bash oot@ubuntu:~# cp -r /opt/tros/lib/mono2d_body_detection/config/ .

root@ubuntu:~# export CAM_TYPE=usb

root@ubuntu:~# ros2 launch mono2d_body_detection hobot_mono2d_body_detection.launch.py

[INFO] [launch]: All log files can be found below /root/.ros/log/2022-09-05-08-16-31-286750-ubuntu-5167

[INFO] [launch]: Default logging verbosity is set to INFO

camera_type is usb

using usb cam

webserver has launch

[INFO] [hobot_usb_cam-1]: process started with pid [5191]

[INFO] [hobot_codec_republish-2]: process started with pid [5193]

[INFO] [mono2d_body_detection-3]: process started with pid [5195]

[INFO] [websocket-4]: process started with pid [5197]

[mono2d_body_detection-3] [WARN] [1662336991.789373568] [example]: This is mono2d body det example!

[mono2d_body_detection-3] [WARN] [1662336991.940319893] [mono2d_body_det]: Parameter:

[mono2d_body_detection-3] is_sync_mode_: 0

[mono2d_body_detection-3] model_file_name_: config/multitask_body_head_face_hand_kps_960x544.hbm

[mono2d_body_detection-3] is_shared_mem_sub: 1

[mono2d_body_detection-3] ai_msg_pub_topic_name: /hobot_mono2d_body_detection

[mono2d_body_detection-3] [C][5195][09-05][08:16:31:950][configuration.cpp:49][EasyDNN]EasyDNN version: 0.4.11

[mono2d_body_detection-3] [BPU_PLAT]BPU Platform Version(1.3.1)!

[mono2d_body_detection-3] [HBRT] set log level as 0. version = 3.14.5

[mono2d_body_detection-3] [DNN] Runtime version = 1.9.7_(3.14.5 HBRT)

[mono2d_body_detection-3] [WARN] [1662336992.063888031] [mono2d_body_det]: Create hbmem_subscription with topic_name: /hbmem_img

[mono2d_body_detection-3] (MOTMethod.cpp:34): MOTMethod::Init config/iou2_euclid_method_param.json

[mono2d_body_detection-3]

[mono2d_body_detection-3] (IOU2.cpp:29): IOU2 Mot::Init config/iou2_euclid_method_param.json

[mono2d_body_detection-3]

[mono2d_body_detection-3] (MOTMethod.cpp:34): MOTMethod::Init config/iou2_method_param.json

[mono2d_body_detection-3]

[mono2d_body_detection-3] (IOU2.cpp:29): IOU2 Mot::Init config/iou2_method_param.json

[mono2d_body_detection-3]

[mono2d_body_detection-3] (MOTMethod.cpp:34): MOTMethod::Init config/iou2_method_param.json

[mono2d_body_detection-3]

[mono2d_body_detection-3] (IOU2.cpp:29): IOU2 Mot::Init config/iou2_method_param.json

[mono2d_body_detection-3]

[mono2d_body_detection-3] (MOTMethod.cpp:34): MOTMethod::Init config/iou2_method_param.json

[mono2d_body_detection-3]

[mono2d_body_detection-3] (IOU2.cpp:29): IOU2 Mot::Init config/iou2_method_param.json

[mono2d_body_detection-3]

[mono2d_body_detection-3] [WARN] [1662336994.093227958] [mono2d_body_det]: input fps: 34.83, out fps: 35.75

[mono2d_body_detection-3] [WARN] [1662336995.112739406] [mono2d_body_det]: input fps: 33.20, out fps: 33.37

[mono2d_body_detection-3] [WARN] [1662336996.138672466] [mono2d_body_det]: input fps: 33.37, out fps: 33.17

[mono2d_body_detection-3] [WARN] [1662336997.158819091] [mono2d_body_det]: input fps: 33.33, out fps: 33.33

[mono2d_body_detection-3] [WARN] [1662336998.177113874] [mono2d_body_det]: input fps: 33.20, out fps: 33.40

[mono2d_body_detection-3] [WARN] [1662336999.206834388] [mono2d_body_det]: input fps: 33.30, out fps: 33.01

[mono2d_body_detection-3] [WARN] [1662337000.220717271] [mono2d_body_det]: input fps: 33.20, out fps: 33.56

[mono2d_body_detection-3] [WARN] [1662337001.264396157] [mono2d_body_det]: input fps: 33.33, out fps: 32.60

[mono2d_body_detection-3] [WARN] [1662337002.277591873] [mono2d_body_det]: input fps: 33.83, out fps: 34.58

[mono2d_body_detection-3] [WARN] [1662337002.390721599] [dnn]: Task Size: 10 exceeds limit: 10

[mono2d_body_detection-3] [WARN] [1662337002.402758943] [dnn]: Task Size: 10 exceeds limit: 10

[mono2d_body_detection-3] [WARN] [1662337002.410986048] [dnn]: Task Size: 10 exceeds limit: 10

[mono2d_body_detection-3] [WARN] [1662337002.423279470] [dnn]: Task Size: 10 exceeds limit: 10

[mono2d_body_detection-3] [WARN] [1662337002.430859443] [dnn]: Task Size: 10 exceeds limit: 10

[mono2d_body_detection-3] [WARN] [1662337002.442967048] [dnn]: Task Size: 10 exceeds limit: 10

[mono2d_body_detection-3] [WARN] [1662337002.450364787] [dnn]: Task Size: 10 exceeds limit: 10

[mono2d_body_detection-3] [WARN] [1662337002.463035970] [dnn]: Task Size: 10 exceeds limit: 10

[mono2d_body_detection-3] [WARN] [1662337002.470932695] [dnn]: Task Size: 10 exceeds limit: 10

[mono2d_body_detection-3] [WARN] [1662337002.482730756] [dnn]: Task Size: 10 exceeds limit: 10

[mono2d_body_detection-3] [WARN] [1662337002.490757000] [dnn]: Task Size: 10 exceeds limit: 10

[mono2d_body_detection-3] [WARN] [1662337002.502293816] [dnn]: Task Size: 10 exceeds limit: 10

[mono2d_body_detection-3] [WARN] [1662337002.510773998] [dnn]: Task Size: 10 exceeds limit: 10

[mono2d_body_detection-3] [WARN] [1662337002.522491048] [dnn]: Task Size: 10 exceeds limit: 10

[mono2d_body_detection-3] [WARN] [1662337002.530706110] [dnn]: Task Size: 10 exceeds limit: 10

[mono2d_body_detection-3] [WARN] [1662337002.542368152] [dnn]: Task Size: 10 exceeds limit: 10

[mono2d_body_detection-3] [WARN] [1662337002.550505078] [dnn]: Task Size: 10 exceeds limit: 10

[mono2d_body_detection-3] [WARN] [1662337002.562142450] [dnn]: Task Size: 10 exceeds limit: 10

[mono2d_body_detection-3] [WARN] [1662337002.570657595] [dnn]: Task Size: 10 exceeds limit: 10

[mono2d_body_detection-3] [WARN] [1662337002.582293633] [dnn]: Task Size: 10 exceeds limit: 10

[mono2d_body_detection-3] [WARN] [1662337002.591539589] [dnn]: Task Size: 10 exceeds limit: 10

[mono2d_body_detection-3] [WARN] [1662337002.603585559] [dnn]: Task Size: 10 exceeds limit: 10

[mono2d_body_detection-3] [WARN] [1662337002.612121415] [dnn]: Task Size: 10 exceeds limit: 10

[mono2d_body_detection-3] [WARN] [1662337002.622872081] [dnn]: Task Size: 10 exceeds limit: 10

[mono2d_body_detection-3] [WARN] [1662337002.640318971] [dnn]: Task Size: 10 exceeds limit: 10

[mono2d_body_detection-3] [WARN] [1662337002.644252269] [dnn]: Task Size: 10 exceeds limit: 10

[mono2d_body_detection-3] [WARN] [1662337002.654941967] [dnn]: Task Size: 10 exceeds limit: 10

[mono2d_body_detection-3] [WARN] [1662337002.662930789] [dnn]: Task Size: 10 exceeds limit: 10

[mono2d_body_detection-3] [WARN] [1662337002.674212611] [dnn]: Task Size: 10 exceeds limit: 10

[mono2d_body_detection-3] [WARN] [1662337002.682763011] [dnn]: Task Size: 10 exceeds limit: 10

[mono2d_body_detection-3] [WARN] [1662337002.694879533] [dnn]: Task Size: 10 exceeds limit: 10

[mono2d_body_detection-3] [WARN] [1662337002.703064758] [dnn]: Task Size: 10 exceeds limit: 10

[mono2d_body_detection-3] [WARN] [1662337002.714751386] [dnn]: Task Size: 10 exceeds limit: 10

[mono2d_body_detection-3] [WARN] [1662337002.722716247] [dnn]: Task Size: 10 exceeds limit: 10

[mono2d_body_detection-3] [WARN] [1662337002.734622489] [dnn]: Task Size: 10 exceeds limit: 10

[mono2d_body_detection-3] [WARN] [1662337002.742606019] [dnn]: Task Size: 10 exceeds limit: 10

4

跑FCOS算法的时候，输出一直是正常的吗，fps是多少？

跑人体检测算法，异常的时候X3派上使用命令hrut_somstatus -n 999查询下BPU的使用情况。

5

跑FCOS算法的时候，输出一直是正常，fps55-60多

6

你使用的USB摄像头输出帧率是多少，有55-60这么大？

7

是的，我用的USB摄像头最大可以达到120帧率

8

我换了一个抵帧率的USB 摄像头也是可以的，可能是兼容性方面的问题吧

9

也不是兼容性的问题，人体检测算法的推理耗时相对较长，而WEB展示时会做图像帧和AI帧的同步。当图像输出帧率比较大时，由于缓存长度限制，图像等不到对应的AI消息，所以WEB端没有图像展示出来。

你这边使用高帧率的摄像头有什么特殊需求吗？

10

没有什么特殊的要求，只是目前手头上有这个快速usb的摄像头就用了，那我用什么方法可以解决上面的问题，抵帧率的USB进行 人体检测才10多帧率，好慢

11

摄像头发布图片的帧率是多少？人体检测算法是能够达到30fps的。

12

摄像头发布图片的帧率用什么命令来查看呢，有没有提供类似于OPencv的接口来设置摄像头的图像大小与帧率的功能呢

13

类似于这样的

  1. VideoCapture capture(0); 设置摄像头参数 不要随意修-

    capture.set(CV_CAP_PROP_FRAME_WIDTH, 1080);//宽度-

    capture.set(CV_CAP_PROP_FRAME_HEIGHT, 960);//高度-

    capture.set(CV_CAP_PROP_FPS, 30);//帧率 帧/秒-

    capture.set(CV_CAP_PROP_BRIGHTNESS, 1);//亮度-

    capture.set(CV_CAP_PROP_CONTRAST,40);//对比度 40-

    capture.set(CV_CAP_PROP_SATURATION, 50);//饱和度 50-

    capture.set(CV_CAP_PROP_HUE, 50);//色调 50-

    capture.set(CV_CAP_PROP_EXPOSURE, 50);//曝光 50 获取摄像头参数
14

不是所有的USB摄像头都支持设置输出图像帧率的，TROS的USB Node本身是有设置帧率的配置的，配置项为“framerate”, 具体参考手册：USB图像采集 — 地平线机器人平台用户手册 1.0 文档 (horizon.ai)

人体检测算法启动脚本里面没有设置USB输出帧率，也就是使用默认帧率30fps进行配置，但是实际上输出帧率大于30，说明这个USB摄像头可能不支持帧率配置。

可以在X3派上使用ros2 topic hz image来查看摄像头实际发布出来的帧率。