用tf训练了一个输入为512*640*2的tensor的模型[两张左右视图灰度后的merge]，转换为onnx后的模型和预处理的数据集在PC上验证过。但是转换时给出了大量超出阈值的警告，且转换后在上板验证效果也不符合。
build.yaml文件如下
# 模型转化相关的参数-
model_parameters:-
onnx_model: ‘/DispNet/models/DispNetOnnx00.onnx’-
layer_out_dump: False-
output_layout: None-
log_level: ‘info’-
working_dir: ‘model_output’-
output_model_file_prefix: ‘DispNet’-
# 模型输入相关参数, 若输入多个节点, 则应使用’;'进行分隔, 使用默认缺省设置则写None-
input_parameters:-
input_name: ‘input_1’-
input_type_rt: ‘featuremap’-
input_type_train: ‘featuremap’-
norm_type: ‘no_preprocess’-
input_shape: 1x512x640x2-
calibration_parameters:-
cal_data_dir: ‘/DispNet/Data/Data_Processed’-
preprocess_on: False-
calibration_type: ‘max’-
promoter_level: -1-
# 编译器相关参数-
compiler_parameters:-
compile_mode: ‘latency’-
debug: True-
core_num: 2-
optimize_level: ‘O3’-
makerbin log如下
2020-12-20 11:10:07,217 INFO hb_mapper_makertbin 58 Working dir: /DispNet/horizon_sh/model_output-
2020-12-20 11:10:07,217 INFO hb_mapper_makertbin 59 Start Model Convert…-
2020-12-20 11:10:07,222 WARNING hb_mapper_makertbin 95 user input_shape: [1, 512, 640, 2], proto input_shape: [0, 512, 640, 2].-
2020-12-20 11:10:07,222 WARNING mapper_conf_parser 659 Promoter feature has been removed since version 1.1.29.-
2020-12-20 11:10:07,222 WARNING mapper_conf_parser 661 Please remove ‘promoter_level’ from your yaml file-
2020-12-20 11:10:07,222 INFO hb_mapper_makertbin 334 call build params:-
{‘march’: ‘bernoulli2’, ‘debug_mode’: False, ‘save_model’: True, ‘name_prefix’: ‘DispNet’, ‘input_dict’: {‘input_1’: {‘input_shape’: [1, 512, 640, 2]}}, ‘cali_dict’: {‘calibration_type’: ‘max’, ‘calibration_loader’: {‘input_1’: <horizon_nn.data.loader.TransformLoader object at 0x7f1875593b00>}, ‘per_channel’: False, ‘max_percentile’: 1.0}, ‘hbdk_dict’: {‘compile_mode’: ‘latency’, ‘debug’: True, ‘core_num’: 2, ‘optimize_level’: ‘O3’, ‘input_source’: {‘input_1’: ‘ddr’}}, ‘layer_dict’: {‘run_on_cpu’: }}-
2020-12-20 11:10:07,233 INFO build 35 [Sun Dec 20 11:10:07 2020] Start to Horizon NN Model Convert.-
2020-12-20 11:10:07,233 INFO build 157 Parsing the input parameter:{‘input_1’: {‘input_shape’: [1, 512, 640, 2]}}-
2020-12-20 11:10:07,233 INFO build 300 Parsing the calibration parameter-
2020-12-20 11:10:07,596 INFO build 325 Parsing the hbdk parameter:{‘compile_mode’: ‘latency’, ‘debug’: True, ‘core_num’: 2, ‘optimize_level’: ‘O3’, ‘input_source’: {‘input_1’: ‘ddr’}}-
2020-12-20 11:10:07,596 INFO build 128 HorizonNN version: 0.8.4-
2020-12-20 11:10:07,596 INFO build 132 HBDK version: 3.14.2-
2020-12-20 11:10:07,596 INFO build 35 [Sun Dec 20 11:10:07 2020] Start to parse the onnx model.-
2020-12-20 11:10:07,598 INFO onnx_parser 60 Replace the input input_1’s shape [0, 512, 640, 2] to [1, 512, 640, 2]-
2020-12-20 11:10:07,612 INFO onnx_parser 101 ONNX model info:-
ONNX IR version: 6-
Opset version: 10-
Input name: input_1, [1, 512, 640, 2]-
2020-12-20 11:10:07,637 INFO build 38 [Sun Dec 20 11:10:07 2020] End to parse the onnx model.-
2020-12-20 11:10:07,637 INFO build 430 Model input names: [‘input_1’]-
2020-12-20 11:10:07,642 INFO build 807 Saving the original float model: DispNet_original_float_model.onnx.-
2020-12-20 11:10:07,642 INFO build 35 [Sun Dec 20 11:10:07 2020] Start to optimize the model.-
2020-12-20 11:10:07,658 INFO build 38 [Sun Dec 20 11:10:07 2020] End to optimize the model.-
2020-12-20 11:10:07,662 INFO build 818 Saving the optimized model: DispNet_optimized_float_model.onnx.-
2020-12-20 11:10:07,663 INFO build 35 [Sun Dec 20 11:10:07 2020] Start to calibrate the model.-
2020-12-20 11:10:07,827 INFO calibrater 126 number of calibration data samples: 40-
2020-12-20 11:10:07,857 INFO max_calibrater 70 Run calibration model with max method.-
2020-12-20 11:10:09,590 INFO build 38 [Sun Dec 20 11:10:09 2020] End to calibrate the model.-
2020-12-20 11:10:09,591 INFO build 35 [Sun Dec 20 11:10:09 2020] Start to quantize the model.-
2020-12-20 11:10:09,671 WARNING tool_utils 271 Warning: got unexpected output threshold on conv conv2d_8! value: 1.0495e+06-
2020-12-20 11:10:09,672 WARNING tool_utils 271 Warning: got unexpected input threshold on conv conv2d_9! value: 1.0495e+06-
2020-12-20 11:10:09,672 WARNING tool_utils 271 Warning: got unexpected output threshold on conv conv2d_9! value: 1.55364e+06-
2020-12-20 11:10:09,673 WARNING tool_utils 271 Warning: got unexpected input threshold on conv conv2d_10! value: 1.55364e+06-
2020-12-20 11:10:09,673 WARNING tool_utils 271 Warning: got unexpected output threshold on conv conv2d_10! value: 3.46568e+06-
2020-12-20 11:10:09,673 WARNING tool_utils 271 Warning: got unexpected input threshold on conv conv2d_11! value: 3.46568e+06-
2020-12-20 11:10:09,674 WARNING tool_utils 271 Warning: got unexpected output threshold on conv conv2d_11! value: 1.79928e+06-
2020-12-20 11:10:09,674 WARNING tool_utils 271 Warning: got unexpected input threshold on conv conv2d_12! value: 1.79928e+06-
2020-12-20 11:10:09,674 WARNING tool_utils 271 Warning: got unexpected input threshold on conv conv2d_transpose! value: 1.55364e+06-
2020-12-20 11:10:09,675 WARNING tool_utils 271 Warning: got unexpected output threshold on conv conv2d_transpose! value: 3.46568e+06-
2020-12-20 11:10:09,675 WARNING tool_utils 271 Warning: got unexpected input threshold on conv conv2d_transpose_1! value: 3.46568e+06-
2020-12-20 11:10:09,675 WARNING tool_utils 271 Warning: got unexpected output threshold on conv conv2d_transpose_1! value: 1.79928e+06-
2020-12-20 11:10:09,675 WARNING tool_utils 271 Warning: got unexpected input threshold on conv conv2d_transpose_2! value: 1.79928e+06-
2020-12-20 11:10:10,650 INFO build 38 [Sun Dec 20 11:10:10 2020] End to quantize the model.-
2020-12-20 11:10:10,709 INFO build 832 Saving the quantized model: DispNet_quantized_model.onnx.-
2020-12-20 11:10:10,709 INFO build 35 [Sun Dec 20 11:10:10 2020] Start to compile the model with march bernoulli2.-
2020-12-20 11:10:10,784 INFO hybrid_build 115 Compile submodel: functional_1_subgraph_0-
2020-12-20 11:10:10,888 INFO hbdk_cc 134 hbdk-cc parameters:{‘optimize-level’: ‘O3’, ‘optimize-target’: ‘fast’, ‘debug’: ‘debug’, ‘core-num’: ‘2’, ‘input-layout’: ‘NHWC’, ‘output-layout’: ‘NCHW’}-
2020-12-20 11:10:20,340 INFO build 38 [Sun Dec 20 11:10:20 2020] End to compile the model with march bernoulli2.-
2020-12-20 11:10:20,341 INFO node_info 58 The converted model node information:-
=======================================================================================================================================================-
Node ON Subgraph Type Cosine Similarity Threshold-
--------------------------------------------------------------------------------------------------------------------------------------------------------
conv2d BPU id(0) HzSQuantizedConv 0.999796 0.996078-
conv2d_1 BPU id(0) HzSQuantizedConv 0.998891 3.232386-
conv2d_2 BPU id(0) HzSQuantizedConv 0.996822 4.697928-
conv2d_3 BPU id(0) HzSQuantizedConv 0.993913 19.809843-
conv2d_4 BPU id(0) HzSQuantizedConv 0.990001 106.199867-
conv2d_5 BPU id(0) HzSQuantizedConv 0.984553 655.241150-
conv2d_6 BPU id(0) HzSQuantizedConv 0.979109 4436.730469-
conv2d_7 BPU id(0) HzSQuantizedConv 0.963161 29432.626953-
conv2d_8 BPU id(0) HzSQuantizedConv 0.957259 174032.421875-
conv2d_9 BPU id(0) HzSQuantizedConv 0.911744 1049499.750000-
conv2d_transpose BPU id(0) HzSQuantizedConvTranspose 0.000304 1553643.250000-
conv2d_10 BPU id(0) HzSQuantizedConv 0.975036 1553643.250000-
Resize5 BPU id(0) HzQuantizedResizeUpsample 0.975036 3465679.750000-
concatenate BPU id(0) Concat 0.555204 3465679.750000-
conv2d_transpose_1 BPU id(0) HzSQuantizedConvTranspose 0.004180 3465679.750000-
conv2d_11 BPU id(0) HzSQuantizedConv nan 3465679.750000-
Resize4 BPU id(0) HzQuantizedResizeUpsample nan 1799278.625000-
concatenate_1 BPU id(0) Concat 0.476610 1799278.625000-
conv2d_transpose_2 BPU id(0) HzSQuantizedConvTranspose 0.009597 1799278.625000-
conv2d_12 BPU id(0) HzSQuantizedConv nan 1799278.625000-
Resize3 BPU id(0) HzQuantizedResizeUpsample nan 561751.437500-
UNIT_CONV_FOR_conv2d_3/Relu_2:0_4423.2397460937… BPU id(0) HzSQuantizedConv-
concatenate_2 BPU id(0) Concat 0.551024 561751.437500-
conv2d_transpose_3 BPU id(0) HzSQuantizedConvTranspose 0.002947 561751.437500-
conv2d_13 BPU id(0) HzSQuantizedConv 0.285630 561751.437500-
Resize2 BPU id(0) HzQuantizedResizeUpsample 0.285630 167558.000000-
UNIT_CONV_FOR_conv2d_1/Relu_2:0_1319.3543701171… BPU id(0) HzSQuantizedConv-
concatenate_3 BPU id(0) Concat 0.246428 167558.000000-
conv2d_transpose_4 BPU id(0) HzSQuantizedConvTranspose -0.077105 167558.000000-
conv2d_14 BPU id(0) HzSQuantizedConv 0.004810 167558.000000-
Resize1 BPU id(0) HzQuantizedResizeUpsample 0.004810 21416.820312-
UNIT_CONV_FOR_conv2d/Relu_2:0_168.6363830566406… BPU id(0) HzSQuantizedConv-
concatenate_4 BPU id(0) Concat 0.164286 21416.820312-
conv2d_15 BPU id(0) HzSQuantizedConv 0.352382 21416.820312-
Relu CPU – Relu 0.095596 —
Resize CPU – Resize 0.095598 —
Transpose1 CPU – Transpose-
2020-12-20 11:10:20,342 INFO build 38 [Sun Dec 20 11:10:20 2020] End to Horizon NN Model Convert.-
2020-12-20 11:10:20,343 INFO hb_mapper_makertbin 385 start convert to *.bin file…-
2020-12-20 11:10:20,465 WARNING mapper_conf_parser 659 Promoter feature has been removed since version 1.1.29.-
2020-12-20 11:10:20,465 WARNING mapper_conf_parser 661 Please remove ‘promoter_level’ from your yaml file-
2020-12-20 11:10:20,466 INFO onnx2horizonrt 1557 ########################################-
2020-12-20 11:10:20,466 INFO onnx2horizonrt 1558 ----------- dependency info -------------
2020-12-20 11:10:20,466 INFO onnx2horizonrt 1559 hbdk version: 3.14.2-
2020-12-20 11:10:20,466 INFO onnx2horizonrt 1561 hbdk runtime version: 3.9.9-
2020-12-20 11:10:20,466 INFO onnx2horizonrt 1564 horizon_nn version: 0.8.4-
2020-12-20 11:10:20,467 INFO onnx2horizonrt 1566 horizon rt version:-
2020-12-20 11:10:20,467 INFO onnx2horizonrt 1568 -------- model parameters info ----------
2020-12-20 11:10:20,467 INFO onnx2horizonrt 1569 caffe_model:-
2020-12-20 11:10:20,467 INFO onnx2horizonrt 1570 prototxt:-
2020-12-20 11:10:20,467 INFO onnx2horizonrt 1571 onnx_model: /DispNet/models/DispNetOnnx00.onnx-
2020-12-20 11:10:20,467 INFO onnx2horizonrt 1572 layer_out_dump: False-
2020-12-20 11:10:20,467 INFO onnx2horizonrt 1573 output_layout: None-
2020-12-20 11:10:20,467 INFO onnx2horizonrt 1574 -------- input_parameters info ----------
2020-12-20 11:10:20,467 INFO onnx2horizonrt 1581 -------- input info : input_1 --------
2020-12-20 11:10:20,467 INFO onnx2horizonrt 1582 --input_name : input_1-
2020-12-20 11:10:20,467 INFO onnx2horizonrt 1583 --input_type_rt : featuremap-
2020-12-20 11:10:20,467 INFO onnx2horizonrt 1584 --input_type_train : featuremap-
2020-12-20 11:10:20,467 INFO onnx2horizonrt 1585 --norm_type : no_preprocess-
2020-12-20 11:10:20,467 INFO onnx2horizonrt 1586 --input_shape : 1x512x640x2-
2020-12-20 11:10:20,468 INFO onnx2horizonrt 1587 -----------------------------------
2020-12-20 11:10:20,468 INFO onnx2horizonrt 1594 -------- calibration parameters info ----------
2020-12-20 11:10:20,468 INFO onnx2horizonrt 1595 preprocess_on: False-
2020-12-20 11:10:20,468 INFO onnx2horizonrt 1596 calibration_type: max-
2020-12-20 11:10:20,468 INFO onnx2horizonrt 1597 per_channel: False-
2020-12-20 11:10:20,468 INFO onnx2horizonrt 1598 max_percentile: 1.0-
2020-12-20 11:10:20,468 INFO onnx2horizonrt 1599 ------------ compiler_parameters info --------------
2020-12-20 11:10:20,468 INFO onnx2horizonrt 1602 compile_mode: latency-
2020-12-20 11:10:20,468 INFO onnx2horizonrt 1602 debug: True-
2020-12-20 11:10:20,468 INFO onnx2horizonrt 1602 core_num: 2-
2020-12-20 11:10:20,468 INFO onnx2horizonrt 1602 optimize_level: O3-
2020-12-20 11:10:20,468 INFO onnx2horizonrt 1602 input_source: {‘input_1’: ‘ddr’}-
2020-12-20 11:10:20,468 INFO onnx2horizonrt 1603 ########################################-
2020-12-20 11:10:20,472 INFO hb_mapper_makertbin 467 Convert to runtime bin file sucessfully!-
2020-12-20 11:10:20,472 INFO hb_mapper_makertbin 468 End Model Convert-
onnx输出如下：
转换后模型输出如下：
tensor堆叠后入下：
