双目深度估计帧率低精度差

我正在使用RDK X5进行双目深度估计,所有过程均按手册操作,但运行所给的程序后发现效果远不如手册及官方视频里的,帧率只有5.6帧,(手册上写着至少17帧)而且精度非常差,请问是什么原因?

您好,参考的是 双目深度算法 | RDK DOC 这个案例吗 ,需要确定miniboot和系统镜像为最新版本并且使用案例中支持的配件 运行V2.3也只有5.6fps吗,这块发版之前我们都经过严格的QA测试,板卡运行状态正常的情况下并不会出现这种问题

2.3有20帧左右,2.0只有7帧,此外日志显示[hobot_codec_republish-4] [WARN] [1756095320.165050368] [hobot_codec_encoder]: Parameters:
[hobot_codec_republish-4] sub_topic: /StereoNetNode/stereonet_visual
[hobot_codec_republish-4] pub_topic: /image_jpeg
[hobot_codec_republish-4] channel: 1
[hobot_codec_republish-4] in_mode: ros
[hobot_codec_republish-4] out_mode: ros
[hobot_codec_republish-4] in_format: bgr8
[hobot_codec_republish-4] out_format: jpeg
[hobot_codec_republish-4] enc_qp: 10
[hobot_codec_republish-4] jpg_quality: 60
[hobot_codec_republish-4] input_framerate: 30
[hobot_codec_republish-4] output_framerate: -1
[hobot_codec_republish-4] dump_output: 0
[hobot_codec_republish-4] [WARN] [1756095320.175665763] [HobotCodecImpl]: platform x5
[mipi_cam-1] [WARN] [1756095320.826368171] [mipi_node]: dual_combine value: 1
[mipi_cam-1] [WARN] [1756095320.826578918] [mipi_node]: frame_ts_type value: sensor
[mipi_cam-1] [WARN] [1756095320.834616515] [mipi_factory]: board_type X5_RDK
[mipi_cam-1]
[mipi_cam-1] [WARN] [1756095320.844511170] [mipi_cam]: this board support mipi:
[mipi_cam-1] [WARN] [1756095320.844797083] [mipi_cam]: host 0
[mipi_cam-1] [WARN] [1756095320.844875415] [mipi_cam]: host 2
[mipi_cam-1] [WARN] [1756095320.845238660] [mipi_cam]: cal_file:/opt/tros/humble/lib/mipi_cam/config/mipi_calib.yaml
[websocket-5] [WARN] [1756095320.932190248] [websocket]:
[websocket-5] Parameter:
[websocket-5] image_topic: /image_jpeg
[websocket-5] image_type: mjpeg
[websocket-5] only_show_image: 1
[websocket-5] output_fps: 0
[stereonet_model_node-2] [BPU_PLAT]BPU Platform Version(1.3.6)!
[stereonet_model_node-2] [HBRT] set log level as 0. version = 3.15.55.0
[stereonet_model_node-2] [DNN] Runtime version = 1.24.5_(3.15.55 HBRT)
[stereonet_model_node-2] [A][DNN][packed_model.cpp:247]Model [HorizonRT] The model builder version = 1.24.3
[stereonet_model_node-2] Model info:
[stereonet_model_node-2] model_name: V22_disp96
[stereonet_model_node-2] Input count: 2
[stereonet_model_node-2] input[0]: tensorLayout: 2 tensorType: 1 validShape:(1, 3, 352, 640, ), alignedShape:(, 1, 3, 352, 640) quantiType: 0 alignedByteSize: 337920
[stereonet_model_node-2] input[1]: tensorLayout: 2 tensorType: 1 validShape:(1, 3, 352, 640, ), alignedShape:(, 1, 3, 352, 640) quantiType: 0 alignedByteSize: 337920
[stereonet_model_node-2] Output count: 2
[stereonet_model_node-2] input[0]: tensorLayout: 2 tensorType: 14 validShape:(1, 9, 88, 160, ), alignedShape:(, 1, 9, 88, 160) quantiType: 2 alignedByteSize: 506880
[stereonet_model_node-2] input[1]: tensorLayout: 2 tensorType: 11 validShape:(1, 9, 352, 640, ), alignedShape:(, 1, 9, 352, 640) quantiType: 2 alignedByteSize: 4055040
[stereonet_model_node-2] [WARN] [1756095321.430323862] [StereoNetNode]: Add StereoRectify Instance: stereo0
[stereonet_model_node-2] calib model: pinhole
[stereonet_model_node-2] Kl:
[stereonet_model_node-2] [1708.847265201933, 0, 614.9788684700537;
[stereonet_model_node-2] 0, 1517.735070095467, 349.7518197082377;
[stereonet_model_node-2] 0, 0, 1]
[stereonet_model_node-2] Dl:
[stereonet_model_node-2] [0.1003632935740837, 0.3041391315225311, -0.000435992479413011, -0.002904556477662835]
[stereonet_model_node-2] Kr:
[stereonet_model_node-2] [1702.994599894228, 0, 690.7874834867323;
[stereonet_model_node-2] 0, 1510.806377522674, 319.8175626050469;
[stereonet_model_node-2] 0, 0, 1]
[stereonet_model_node-2] Dr:
[stereonet_model_node-2] [0.07717930064799088, 0.3879633894950955, -0.0009148290674511234, 0.006538589110489651]
[stereonet_model_node-2] R, t:
[stereonet_model_node-2] [0.9997249757903384, 0.004555069409014068, -0.02300487173809759;
[stereonet_model_node-2] -0.0046040241527441, 0.9999872475491255, -0.002075500114436642;
[stereonet_model_node-2] 0.0229951243225212, 0.002180844286771015, 0.9997331984962731]
[stereonet_model_node-2] [0.0350858424927499;
[stereonet_model_node-2] 0.0004039176050205226;
[stereonet_model_node-2] 0.001767017234691505]
[stereonet_model_node-2] calib file width, height: 1280, 640
[stereonet_model_node-2] model input width, height: 640, 352
[stereonet_model_node-2] fov_scale: 0.8
[stereonet_model_node-2] resize_before_rectify: 0
[stereonet_model_node-2] width_scale, height_scale: 1, 1
[stereonet_model_node-2] [WARN] [1756095321.465714247] [StereoNetNode]: rectified fx: 961.209778, fy: 961.209778, cx: 304.827942, cy: 180.189377, base_line: :0.035133
[stereonet_model_node-2] [WARN] [1756095321.465901536] [StereoNetNode]: => rectified fx: 961.209778, fy: 961.209778, cx: 304.827942, cy: 180.189377, base_line: :0.035133
[mipi_cam-1] mipi mclk is not configed.
[mipi_cam-1] Searching camera sensor on device: /proc/device-tree/soc/cam/vcon@0 mclk_is_not_configed: 1
[mipi_cam-1] i2c bus: 6 mipi rx phy: 0
[mipi_cam-1] WARN: Sensor Name: sc230ai-30fps, Expected Chip ID: 0xCB34, Actual Chip ID Read: 0x00
[mipi_cam-1] WARN: Sensor Name: sc230ai-30fps, Expected Chip ID: 0xCB34, Actual Chip ID Read: 0x00
[mipi_cam-1] WARN: Sensor Name: sc132gs-1280p, Expected Chip ID: 0x132, Actual Chip ID Read: 0x00
[mipi_cam-1] WARN: Sensor Name: sc132gs-1280p, Expected Chip ID: 0x132, Actual Chip ID Read: 0x00
[mipi_cam-1] WARN: Sensor Name: sc132gs-1280p, Expected Chip ID: 0x132, Actual Chip ID Read: 0x00
[mipi_cam-1] WARN: Sensor Name: ov5647, Expected Chip ID: 0x5647, Actual Chip ID Read: 0x00
[mipi_cam-1] INFO: vp_sensor_detect_2 Support sensor index:0, sensor_name:imx219-30fps on mipi rx csi 0, i2c addr 0x10, config_file:linear_1920x1080_raw10_30fps_2lane.c
[mipi_cam-1] mipi mclk is not configed.
[mipi_cam-1] Searching camera sensor on device: /proc/device-tree/soc/cam/vcon@2 mclk_is_not_configed: 1
[mipi_cam-1] i2c bus: 4 mipi rx phy: 2
[mipi_cam-1] WARN: Sensor Name: sc230ai-30fps, Expected Chip ID: 0xCB34, Actual Chip ID Read: 0x00
[mipi_cam-1] WARN: Sensor Name: sc230ai-30fps, Expected Chip ID: 0xCB34, Actual Chip ID Read: 0x00
[mipi_cam-1] WARN: Sensor Name: sc132gs-1280p, Expected Chip ID: 0x132, Actual Chip ID Read: 0x00
[mipi_cam-1] WARN: Sensor Name: sc132gs-1280p, Expected Chip ID: 0x132, Actual Chip ID Read: 0x00
[mipi_cam-1] WARN: Sensor Name: sc132gs-1280p, Expected Chip ID: 0x132, Actual Chip ID Read: 0x00
[mipi_cam-1] WARN: Sensor Name: ov5647, Expected Chip ID: 0x5647, Actual Chip ID Read: 0x00
[mipi_cam-1] INFO: vp_sensor_detect_2 Support sensor index:2, sensor_name:imx219-30fps on mipi rx csi 2, i2c addr 0x10, config_file:linear_1920x1080_raw10_30fps_2lane.c
[mipi_cam-1] mipi mclk is not configed.
[mipi_cam-1] Searching camera sensor on device: /proc/device-tree/soc/cam/vcon@2 mclk_is_not_configed: 1
[mipi_cam-1] i2c bus: 4 mipi rx phy: 2
[mipi_cam-1] INFO: vp_sensor_fixed_mipi_host_1 Found sensor_name:imx219-30fps on mipi rx csi 2, i2c addr 0x10, config_file:linear_1920x1080_raw10_30fps_2lane.c
[mipi_cam-1] Kl:
[mipi_cam-1] [2563.2708978029, 0, 922.4683027050805;
[mipi_cam-1] 0, 2561.177930786101, 590.2061957576511;
[mipi_cam-1] 0, 0, 1]
[mipi_cam-1] Dl:
[mipi_cam-1] [0.1003632935740837, 0.3041391315225311, -0.000435992479413011, -0.002904556477662835, 0]
[mipi_cam-1] Kr:
[mipi_cam-1] [2554.491899841342, 0, 1036.181225230098;
[mipi_cam-1] 0, 2549.485762069512, 539.6921368960166;
[mipi_cam-1] 0, 0, 1]
[mipi_cam-1] Dr:
[mipi_cam-1] [0.07717930064799088, 0.3879633894950955, -0.0009148290674511234, 0.006538589110489651, 0]
[mipi_cam-1] R_rl:
[mipi_cam-1] [0.9997249757903384, 0.004555069409014068, -0.02300487173809759;
[mipi_cam-1] -0.0046040241527441, 0.9999872475491255, -0.002075500114436642;
[mipi_cam-1] 0.0229951243225212, 0.002180844286771015, 0.9997331984962731]
[mipi_cam-1] t_rl:
[mipi_cam-1] [0.03508584249274991;
[mipi_cam-1] 0.0004039176050205227;
[mipi_cam-1] 0.001767017234691505]
[mipi_cam-1] Kl:
[mipi_cam-1] [2563.2708978029, 0, 922.4683027050805;
[mipi_cam-1] 0, 2561.177930786101, 590.2061957576511;
[mipi_cam-1] 0, 0, 1]
[mipi_cam-1] Dl:
[mipi_cam-1] [0.1003632935740837, 0.3041391315225311, -0.000435992479413011, -0.002904556477662835, 0]
[mipi_cam-1] Kr:
[mipi_cam-1] [2554.491899841342, 0, 1036.181225230098;
[mipi_cam-1] 0, 2549.485762069512, 539.6921368960166;
[mipi_cam-1] 0, 0, 1]
[mipi_cam-1] Dr:
[mipi_cam-1] [0.07717930064799088, 0.3879633894950955, -0.0009148290674511234, 0.006538589110489651, 0]
[mipi_cam-1] R, t:
[mipi_cam-1] [0.9997249757903384, 0.004555069409014068, -0.02300487173809759;
[mipi_cam-1] -0.0046040241527441, 0.9999872475491255, -0.002075500114436642;
[mipi_cam-1] 0.0229951243225212, 0.002180844286771015, 0.9997331984962731]
[mipi_cam-1] [0.03513263165950775;
[mipi_cam-1] 0;
[mipi_cam-1] 0]
[mipi_cam-1] calib file width, height: 1280, 640
[mipi_cam-1] gdc_width_scale, gdc_height_scale: 1.5, 1.6875
[mipi_cam-1] rectify [f, cx, cy, baseline]: [1928.45, 598.862, 370.59, 0.0351326]
[mipi_cam-1]
[mipi_cam-1] [WARN] [1756095322.525114283] [mipi_cap]: gdc bin file x�A��� open failed
[mipi_cam-1]
[mipi_cam-1] [WARN] [1756095322.529583263] [mipi_cap]: X5 start gdc cal.
[mipi_cam-1]
[mipi_cam-1] [WARN] [1756095323.340010961] [mipi_cap]: gdc bin file x�A��� open failed
[mipi_cam-1]
[mipi_cam-1] [WARN] [1756095323.342695883] [mipi_cap]: X5 start gdc cal.
[mipi_cam-1]
[mipi_cam-1] [WARN] [1756095324.006017243] [mipi_cam]: [init]->cap default init success.
[mipi_cam-1]
[mipi_cam-1] [WARN] [1756095324.006271115] [mipi_cap]: get calibration camera info
[stereonet_model_node-2] [WARN] [1756095324.124195159] [StereoNetNode]: => sub rectified fx: 1928.448853, fy: 1928.448853, cx: 598.862427, cy: 370.589691, base_line: :0.035133
[stereonet_model_node-2] [WARN] [1756095324.319038895] [StereoNetNode]: inference que is full!
[stereonet_model_node-2] [WARN] [1756095324.406863229] [StereoNetNode]: pub_que is full!
[stereonet_model_node-2] [WARN] [1756095324.420299627] [StereoNetNode]: pub rectified image with topic name [~/rectified_image]
[hobot_codec_republish-4] [WARN] [1756095324.453502712] [HobotVenc]: init_pic_w_: 640, init_pic_h_: 704, alined_pic_w_: 640, alined_pic_h_: 704, aline_w_: 16, aline_h_: 16
[hobot_codec_republish-4] [WARN] [1756095325.233401572] [hobot_codec_encoder]: Sub imgRaw fps [3.58]
[hobot_codec_republish-4] [WARN] [1756095325.237365934] [hobot_codec_encoder]: Pub img fps [3.58]
[stereonet_model_node-2] [WARN] [1756095329.339949573] [StereoNetNode]: inference que is full!
[hobot_codec_republish-4] [WARN] [1756095330.252246750] [hobot_codec_encoder]: Sub imgRaw fps [25.71]
[hobot_codec_republish-4] [WARN] [1756095330.256084406] [hobot_codec_encoder]: Pub img fps [25.71]
[stereonet_model_node-2] [WARN] [1756095334.398969349] [StereoNetNode]: inference que is full!,我测得的深度很不稳定,画面一直闪,这里输入模型的图像帧率是多少?我mipi节点设置的是30帧,但似乎底层输入还是30帧,这会导致我图像不稳定吗?

明白,这块我会同步验证,存在问题会像内部研发反馈

你好,内部测试正常使用双目相机对着物体帧率是稳定的,精度问题内部在不断的优化中,后续也会不断更新新的版本做优化,现有的版本也会对速度精度做一些balance,可以按需选择