你好，
我在做如下模型量化转换碰到转换失败的问题，能否帮忙看下有什么分析方法和解决方法么。多谢！
1. 环境
horizon_model_convert_sample_xj3_1.1.20g
docker.hobot.cc/aitools/horizon_xj3_tc:xj3_1.1.20g
2. 模型
Simple and Lightwight Human Pose Estimation
不用gcb模块的参数，lp_net_50_256x192_without_gcb.pth.tar，转为onnx，然后来量化。
3. build.sh的配置文件
# Copyright (c) 2020 Horizon Robotics.All Rights Reserved.
# The material in this file is confidential and contains trade secrets
# of Horizon Robotics Inc. This is proprietary information owned by
# Horizon Robotics Inc. No part of this work may be disclosed,
# reproduced, copied, transmitted, or used in any way for any purpose,
# without the express written permission of Horizon Robotics Inc.
# 模型转化相关的参数
model_parameters:
# Caffe浮点网络数据模型文件
onnx_model: ‘./simple-human-pose-estimation-noGCB.onnx’
# Caffe网络描述文件
# prototxt: ‘../../01_common/modelzoo/mapper/custom_op/resnet_50_deploy_cpu.prototxt’
# 适用BPU架构
march: “bernoulli2”
# 指定模型转换过程中是否输出各层的中间结果，如果为True，则输出所有层的中间输出结果，
layer_out_dump: False
# 用于设置上板模型输出的layout, 支持NHWC和NCHW, 输入None则使用模型默认格式
output_layout: None
# 日志文件的输出控制参数，
# debug输出模型转换的详细信息
# info只输出关键信息
# warn输出警告和错误级别以上的信息
log_level: ‘debug’
# 模型转换输出的结果的存放目录
working_dir: ‘model_output-simple-human-pose-estimation-noGCB’
# 模型转换输出的用于上板执行的模型文件的名称前缀
output_model_file_prefix: ‘simple-human-pose-estimation-noGCB’
# 模型输入相关参数, 若输入多个节点, 则应使用’;'进行分隔, 使用默认缺省设置则写None
input_parameters:
# (可不填) 模型输入的节点名称, 此名称应与模型文件中的名称一致, 否则会报错, 不填则会使用模型文件中的节点名称
# input_name: data
# 网络实际执行时，输入给网络的数据格式，包括 nv12/rgbp/bgrp/yuv444_128/gray/featuremap,
# 如果输入的数据为yuv444_128, 模型训练用的是bgrp，则hb_mapper将自动插入YUV到BGRP(NCHW)转化操作
input_type_rt: ‘rgbp’
# 网络训练时输入的数据格式，可选的值为rgbp/bgrp/gray/featuremap/yuv444_128
input_type_train: ‘rgbp’
# 模型网络的输入大小, 以’x’分隔, 不填则会使用模型文件中的网络输入大小，否则会覆盖模型文件中输入大小
# input_shape: ‘’
# 网络输入的预处理方法，主要有以下几种：
# no_preprocess 不做任何操作
# data_mean 减去通道均值mean_value
# data_scale 对图像像素乘以data_scale系数
# data_mean_and_scale 减去通道均值后再乘以scale系数
norm_type: ‘data_mean_and_scale’
# 图像减去的均值, 如果是通道均值，value之间必须用空格分隔
mean_value: 103.94 116.78 123.68
# 图像预处理缩放比例，如果是通道缩放比例，value之间必须用空格分隔
scale_value: 1.0
calibration_parameters:
# 模型量化的参考图像的存放目录，图片格式支持Jpeg、Bmp等格式，输入的图片
# 应该是使用的典型场景，一般是从测试集中选择20~100张图片，另外输入
# 的图片要覆盖典型场景，不要是偏僻场景，如过曝光、饱和、模糊、纯黑、纯白等图片
# 若有多个输入节点, 则应使用’;'进行分隔
cal_data_dir: ‘./poseImgs_rgbp’
# cal_data_dir: ‘./poseImgs_bgrp’
# 如果输入的图片文件尺寸和模型训练的尺寸不一致时，并且preprocess_on为true，
# 则将采用默认预处理方法(opencv resize)，
# 将输入图片缩放或者裁减到指定尺寸，否则，需要用户提前把图片处理为训练时的尺寸
preprocess_on: False
# 模型量化的算法类型，支持kl、max，通常采用KL即可满足要求
calibration_type: ‘max’
# max_percentile: 0.9999
# per_channel: True
# calibration_type: ‘max’
# max_percentile: 0.9999
# per_channel: True
# #自定义OP相关参数
# custom_op:
# # 自定义op的校准方式, 推荐使用注册方式 register
# custom_op_method: register
# #— register ---------------
# # 自定义OP的实现文件, 多个文件可用";"分隔, 该文件可由模板生成, 详情见自定义OP相关文档
# op_register_files: sample_custom.py
# # 自定义OP实现文件所在的文件夹, 请使用相对路径
# custom_op_dir: ./custom_op
# 编译器相关参数
compiler_parameters:
# 编译策略，支持bandwidth和latency两种优化模式;
# bandwidth以优化ddr的访问带宽为目标；
# latency以优化推理时间为目标
compile_mode: ‘latency’
# 设置debug为True将打开编译器的debug模式，能够输出性能仿真的相关信息，如帧率、DDR带宽占用等
debug: False
# 编译模型指定核数，不指定默认编译单核模型, 若编译双核模型，将下边注释打开即可
# core_num: 2
# 优化等级可选范围为O0~O3
# O0不做任何优化, 编译速度最快，优化程度最低,
# O1-O3随着优化等级提高，预期编译后的模型的执行速度会更快，但是所需编译时间也会变长。
# 推荐用O2做最快验证
optimize_level: ‘O2’
4. 执行build.sh的输出
[root@32b86d5ad1a9 02_mapper]# sh 03_1-build.sh
cd $(dirname $0)
config_file=“./lpn.yaml”
model_type=“onnx”
# build model
hb_mapper makertbin --config ${config_file} \
--model-type ${model_type}
2021-05-06 18:02:18,468 INFO Start hb_mapper…
2021-05-06 18:02:18,468 INFO hb_mapper version 1.1.51
2021-05-06 18:02:18,475 INFO norm_types[i]: data_mean_and_scale
2021-05-06 18:02:18,480 INFO Working dir: /data/horizon_model_convert_sample_xj3_1.1.20g/samples/13_simple-human-pose-estimation/02_mapper/model_output-simple-human-pose-estimation-noGCB
2021-05-06 18:02:18,480 INFO Start Model Convert…
2021-05-06 18:02:18,489 WARNING model name missing, using model name from model file: [‘inputs’]
2021-05-06 18:02:18,490 INFO call build params:
{‘march’: ‘bernoulli2’, ‘debug_mode’: False, ‘save_model’: True, ‘name_prefix’: ‘simple-human-pose-estimation-noGCB’, ‘input_dict’: {‘inputs’: {‘input_shape’: [1, 3, 256, 192], ‘expected_input_type’: ‘RGB_128’, ‘original_input_type’: ‘RGB’, ‘means’: array([103.94, 116.78, 123.68]), ‘scales’: array([1.], dtype=float32)}}, ‘cali_dict’: {‘calibration_type’: ‘max’, ‘calibration_loader’: {‘inputs’: <horizon_nn.data.loader.TransformLoader object at 0x7f2b2c2dbf28>}, ‘per_channel’: False, ‘max_percentile’: 1.0}, ‘hbdk_dict’: {‘compile_mode’: ‘latency’, ‘debug’: False, ‘optimize_level’: ‘O2’, ‘input_source’: {‘inputs’: ‘ddr’}}, ‘node_dict’: {}}
2021-05-06 18:02:18,501 INFO [Thu May 6 18:02:18 2021] Start to Horizon NN Model Convert.
2021-05-06 18:02:18,501 INFO Parsing the input parameter:{‘inputs’: {‘input_shape’: [1, 3, 256, 192], ‘expected_input_type’: ‘RGB_128’, ‘original_input_type’: ‘RGB’, ‘means’: array([103.94, 116.78, 123.68]), ‘scales’: array([1.], dtype=float32)}}
2021-05-06 18:02:18,502 INFO Parsing the calibration parameter
2021-05-06 18:02:18,512 INFO Parsing the hbdk parameter:{‘compile_mode’: ‘latency’, ‘debug’: False, ‘optimize_level’: ‘O2’, ‘input_source’: {‘inputs’: ‘ddr’}}
2021-05-06 18:02:18,512 INFO HorizonNN version: 0.9.3
2021-05-06 18:02:18,512 INFO HBDK version: 3.15.4
2021-05-06 18:02:18,512 INFO [Thu May 6 18:02:18 2021] Start to parse the onnx model.
2021-05-06 18:02:18,550 INFO ONNX model info:
ONNX IR version: 4
Opset version: 10
Input name: inputs, [1, 3, 256, 192]
2021-05-06 18:02:18,591 INFO [Thu May 6 18:02:18 2021] End to parse the onnx model.
2021-05-06 18:02:18,592 INFO Model input names: [‘inputs’]
2021-05-06 18:02:18,593 INFO Create a preprocessing operator for input_name inputs with means=[103.94 116.78 123.68], std=[1.], original_input_layout=NCHW, color convert from ‘RGB’ to ‘RGB’.
2021-05-06 18:02:18,650 INFO Saving the original float model: simple-human-pose-estimation-noGCB_original_float_model.onnx.
2021-05-06 18:02:18,650 INFO [Thu May 6 18:02:18 2021] Start to optimize the model.
2021-05-06 18:02:18,819 INFO [Thu May 6 18:02:18 2021] End to optimize the model.
2021-05-06 18:02:18,868 INFO Saving the optimized model: simple-human-pose-estimation-noGCB_optimized_float_model.onnx.
2021-05-06 18:02:18,869 INFO [Thu May 6 18:02:18 2021] Start to calibrate the model.
2021-05-06 18:02:18,961 INFO number of calibration data samples: 10
2021-05-06 18:02:19,011 INFO Run calibration model with max method.
2021-05-06 18:02:19,568 INFO [Thu May 6 18:02:19 2021] End to calibrate the model.
2021-05-06 18:02:19,569 INFO [Thu May 6 18:02:19 2021] Start to quantize the model.
2021-05-06 18:02:19,772 WARNING Warning: got unexpected output threshold on conv 460! value: 1.51832e+06
2021-05-06 18:02:19,918 WARNING Warning: got unexpected input threshold on conv 463! value: 1.51832e+06
2021-05-06 18:02:19,918 WARNING Warning: got unexpected output threshold on conv 463! value: 1.23991e+06
2021-05-06 18:02:19,919 WARNING Warning: got unexpected input threshold on conv 467! value: 1.23991e+06
2021-05-06 18:02:19,919 WARNING Warning: got unexpected output threshold on conv 470! value: 2.32639e+06
2021-05-06 18:02:19,919 WARNING Warning: got unexpected input threshold on conv 473! value: 2.32639e+06
2021-05-06 18:02:19,919 WARNING Warning: got unexpected input threshold on conv 475! value: 1.23991e+06
2021-05-06 18:02:19,919 WARNING Warning: got unexpected output threshold on conv 475! value: 1.3508e+06
2021-05-06 18:02:19,919 WARNING Warning: got unexpected input threshold on conv 479! value: 1.3508e+06
2021-05-06 18:02:19,920 WARNING Warning: got unexpected output threshold on conv 479! value: 1.59059e+06
2021-05-06 18:02:19,920 WARNING Warning: got unexpected input threshold on conv 482! value: 1.59059e+06
2021-05-06 18:02:19,920 WARNING Warning: got unexpected output threshold on conv 482! value: 4.0262e+06
2021-05-06 18:02:19,920 WARNING Warning: got unexpected input threshold on conv 485! value: 4.0262e+06
2021-05-06 18:02:19,920 WARNING Warning: got unexpected output threshold on conv 485! value: 3.66549e+06
2021-05-06 18:02:19,920 WARNING Warning: got unexpected sumin threshold on conv 485! value: 1.3508e+06
2021-05-06 18:02:19,920 WARNING Warning: got unexpected input threshold on conv 489! value: 3.66549e+06
2021-05-06 18:02:19,921 WARNING Warning: got unexpected output threshold on conv 489! value: 3.00379e+06
2021-05-06 18:02:19,921 WARNING Warning: got unexpected input threshold on conv 492! value: 3.00379e+06
2021-05-06 18:02:19,921 WARNING Warning: got unexpected output threshold on conv 492! value: 7.19565e+06
2021-05-06 18:02:19,921 WARNING Warning: got unexpected input threshold on conv 495! value: 7.19565e+06
2021-05-06 18:02:19,921 WARNING Warning: got unexpected output threshold on conv 495! value: 8.47549e+06
2021-05-06 18:02:19,921 WARNING Warning: got unexpected sumin threshold on conv 495! value: 3.66549e+06
2021-05-06 18:02:19,921 WARNING Warning: got unexpected input threshold on conv 500! value: 2.47694e+07
2021-05-06 18:02:19,921 WARNING Warning: got unexpected output threshold on conv 500! value: 4.81253e+06
2021-05-06 18:02:19,922 WARNING Warning: got unexpected input threshold on conv 504! value: 7.08052e+06
2021-05-06 18:02:19,922 WARNING Warning: got unexpected input threshold on conv 499! value: 8.47549e+06
2021-05-06 18:02:19,922 WARNING Warning: got unexpected output threshold on conv 499! value: 2.47694e+07
2021-05-06 18:02:19,922 WARNING Warning: got unexpected input threshold on conv 503! value: 4.81253e+06
2021-05-06 18:02:19,922 WARNING Warning: got unexpected output threshold on conv 503! value: 7.08052e+06
2021-05-06 18:02:20,592 INFO [Thu May 6 18:02:20 2021] End to quantize the model.
2021-05-06 18:02:20,761 INFO Saving the quantized model: simple-human-pose-estimation-noGCB_quantized_model.onnx.
2021-05-06 18:02:20,761 INFO [Thu May 6 18:02:20 2021] Start to compile the model with march bernoulli2.
2021-05-06 18:02:20,901 INFO Compile submodel: torch-jit-export_subgraph_0
2021-05-06 18:02:21,105 INFO hbdk-cc parameters:{‘optimize-level’: ‘O2’, ‘input-source’: ‘ddr’, ‘optimize-target’: ‘fast’, ‘input-layout’: ‘NHWC’, ‘output-layout’: ‘NCHW’}
[==================================================] 100%
2021-05-06 18:02:29,322 INFO [Thu May 6 18:02:29 2021] End to compile the model with march bernoulli2.
2021-05-06 18:02:29,324 INFO The converted model node information:
==============================================================================================================================
Node ON Subgraph Type Cosine Similarity Threshold
------------------------------------------------------------------------------------------------------------------------------
HZ_PREPROCESS_FOR_inputs BPU id(0) HzSQuantizedPreprocess 0.999991 127.000000
329 BPU id(0) HzSQuantizedConv 0.999766 123.680000
332 BPU id(0) HzQuantizedMaxPool 0.999816 294.895599
333 BPU id(0) HzSQuantizedConv 0.999889 294.895599
336 BPU id(0) HzSQuantizedConv 0.999470 350.937164
339 BPU id(0) HzSQuantizedConv 0.999718 986.649902
343 BPU id(0) HzSQuantizedConv 0.999637 792.436951
346 BPU id(0) HzSQuantizedConv 0.998501 710.482544
349 BPU id(0) HzSQuantizedConv 0.999419 1248.289795
353 BPU id(0) HzSQuantizedConv 0.999302 1001.231384
356 BPU id(0) HzSQuantizedConv 0.999706 499.271820
359 BPU id(0) HzSQuantizedConv 0.999685 2306.238525
363 BPU id(0) HzSQuantizedConv 0.999402 2932.061279
366 BPU id(0) HzSQuantizedConv 0.999548 876.396362
369 BPU id(0) HzSQuantizedConv 0.999411 1984.778442
371 BPU id(0) HzSQuantizedConv 0.999594 2932.061279
375 BPU id(0) HzSQuantizedConv 0.999367 1076.282593
378 BPU id(0) HzSQuantizedConv 0.999139 491.580505
381 BPU id(0) HzSQuantizedConv 0.999405 3572.142334
385 BPU id(0) HzSQuantizedConv 0.999443 1663.398926
388 BPU id(0) HzSQuantizedConv 0.999749 812.110596
391 BPU id(0) HzSQuantizedConv 0.999680 3204.132324
395 BPU id(0) HzSQuantizedConv 0.999686 2589.732910
398 BPU id(0) HzSQuantizedConv 0.999727 1062.053345
401 BPU id(0) HzSQuantizedConv 0.999494 2715.701904
405 BPU id(0) HzSQuantizedConv 0.999169 3204.787109
408 BPU id(0) HzSQuantizedConv 0.999236 822.730103
411 BPU id(0) HzSQuantizedConv 0.999155 2924.552246
413 BPU id(0) HzSQuantizedConv 0.999212 3204.787109
417 BPU id(0) HzSQuantizedConv 0.999442 1375.911133
420 BPU id(0) HzSQuantizedConv 0.999467 870.686157
423 BPU id(0) HzSQuantizedConv 0.999281 3317.060059
427 BPU id(0) HzSQuantizedConv 0.998907 1099.471313
430 BPU id(0) HzSQuantizedConv 0.999294 541.977722
433 BPU id(0) HzSQuantizedConv 0.999210 2300.739258
437 BPU id(0) HzSQuantizedConv 0.998997 1300.036377
440 BPU id(0) HzSQuantizedConv 0.997258 524.971252
443 BPU id(0) HzSQuantizedConv 0.999418 22888.011719
447 BPU id(0) HzSQuantizedConv 0.999284 7766.251465
450 BPU id(0) HzSQuantizedConv 0.999593 3295.014893
453 BPU id(0) HzSQuantizedConv 0.999534 729865.125000
457 BPU id(0) HzSQuantizedConv 0.999507 207502.343750
460 BPU id(0) HzSQuantizedConv 0.994015 134960.890625
463 BPU id(0) HzSQuantizedConv 0.999425 1518323.875000
467 BPU id(0) HzSQuantizedConv 0.999391 1239912.500000
470 BPU id(0) HzSQuantizedConv 0.999396 478700.218750
473 BPU id(0) HzSQuantizedConv 0.999634 2326394.250000
475 BPU id(0) HzSQuantizedConv 0.999566 1239912.500000
479 BPU id(0) HzSQuantizedConv 0.999302 1350801.250000
482 BPU id(0) HzSQuantizedConv 0.999444 1590587.500000
485 BPU id(0) HzSQuantizedConv 0.999622 4026195.000000
489 BPU id(0) HzSQuantizedConv 0.999283 3665488.500000
492 BPU id(0) HzSQuantizedConv 0.999369 3003788.750000
495 BPU id(0) HzSQuantizedConv 0.999516 7195649.500000
499 BPU id(0) HzSQuantizedConvTranspose 0.999054 8475489.000000
500 BPU id(0) HzSQuantizedConv 0.999556 24769442.000000
503 BPU id(0) HzSQuantizedConvTranspose 0.999551 4812528.500000
504 BPU id(0) HzSQuantizedConv 0.077551 7080521.000000
outputs BPU id(0) HzSQuantizedConv 0.079697 9253.327148
2021-05-06 18:02:29,324 INFO The quantify model output:
==========================================================================
Node Cosine Similarity L1 Distance L2 Distance Chebyshev Distance
--------------------------------------------------------------------------
outputs 0.079697 166.435432 3.892239 24533.835938
2021-05-06 18:02:29,324 INFO [Thu May 6 18:02:29 2021] End to Horizon NN Model Convert.
2021-05-06 18:02:29,327 INFO start convert to *.bin file…
2021-05-06 18:02:29,454 INFO ########################################
2021-05-06 18:02:29,454 INFO ----------- dependency info ------------
2021-05-06 18:02:29,454 INFO hb mapper version: 1.1.51
2021-05-06 18:02:29,454 INFO hbdk version: 3.15.4
2021-05-06 18:02:29,454 INFO hbdk runtime version: 3.10.4
2021-05-06 18:02:29,455 INFO horizon_nn version: 0.9.3
2021-05-06 18:02:29,455 INFO -------- model parameters info ---------
2021-05-06 18:02:29,455 INFO caffe_model:
2021-05-06 18:02:29,455 INFO prototxt:
2021-05-06 18:02:29,455 INFO onnx_model: /data/horizon_model_convert_sample_xj3_1.1.20g/samples/13_simple-human-pose-estimation/02_mapper/simple-human-pose-estimation-noGCB.onnx
2021-05-06 18:02:29,455 INFO layer_out_dump: False
2021-05-06 18:02:29,455 INFO output_layout: None
2021-05-06 18:02:29,455 INFO -------- input_parameters info ---------
2021-05-06 18:02:29,455 INFO -------- input info : inputs -------
2021-05-06 18:02:29,455 INFO --input_name : inputs
2021-05-06 18:02:29,455 INFO --input_type_rt : rgbp
2021-05-06 18:02:29,455 INFO --input_type_train : rgbp
2021-05-06 18:02:29,455 INFO --norm_type : data_mean_and_scale
2021-05-06 18:02:29,455 INFO --input_shape : 1x3x256x192
2021-05-06 18:02:29,455 INFO ----------------------------------
2021-05-06 18:02:29,456 INFO -------- calibration parameters info ---------
2021-05-06 18:02:29,456 INFO preprocess_on: False
2021-05-06 18:02:29,456 INFO calibration_type: max
2021-05-06 18:02:29,456 INFO per_channel: False
2021-05-06 18:02:29,456 INFO max_percentile: 1.0
2021-05-06 18:02:29,456 INFO ------------ compiler_parameters info -------------
2021-05-06 18:02:29,456 INFO compile_mode: latency
2021-05-06 18:02:29,456 INFO debug: False
2021-05-06 18:02:29,456 INFO optimize_level: O2
2021-05-06 18:02:29,456 INFO input_source: {‘inputs’: ‘ddr’}
2021-05-06 18:02:29,456 INFO ########################################
2021-05-06 18:02:29,460 INFO Convert to runtime bin file sucessfully!
2021-05-06 18:02:29,460 INFO End Model Convert
WARNING: the onnx model’s ir_version is inconsistent with the ir_version of the parser.
# exit
# export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:./release/lib
# # model_name=“checkpoint_iter_665000”
# model_name=“checkpoint_iter_140000”
# model_output=“./model_output”
# image_file=“./crowdhuman_100imgs_bgrp/273271,1b9330008da38cd6.bgr”
# input_height=256
# input_width=456
# input_type=4 #for input type enum definition, please see readme.txt
# output_dir=“./output”
# #===================================================================
# float_model_file=“${model_output}/${model_name}_original_float_model.onnx”
# quantized_model_file=“${model_output}/${model_name}_quantized_model.onnx”
# hybrid_model_file=“${model_output}/${model_name}_hybrid_horizonrt.bin”
# mapping_file=“${model_output}/${model_name}_quantized_model_conv_output_map.json”
# input_file_path=“data_input_for_runtime.bin”
# float_output_path=$output_dir/float_dump
# quantized_output_path=$output_dir/quantized_dump
# hybrid_output_path=$output_dir/hybrid_dump
# rm -rf $output_dir && mkdir $output_dir
# rm -rf $float_output_path && rm -rf $quantized_output_path && rm -rf $hybrid_output_path
# mkdir $float_output_path && mkdir $quantized_output_path && mkdir $hybrid_output_path
# float_model_layout=“NCHW”
# quanti_model_layout=“NHWC”
# # infer float model
# hb_mapper infer --config ${config_file} \
# --model-file ${float_model_file} \
# --image-file data ${image_file} \
# --output-dir ${float_output_path} \
# --input-layout ${float_model_layout}
# # infer quantized model
# hb_mapper infer --config ${config_file} \
# --model-file ${quantized_model_file} \
# --image-file data ${image_file} \
# --output-dir ${quantized_output_path} \
# --input-layout ${quanti_model_layout}
# # infer hybrid model on simulator
# ./release/bin/dump \
# --model_file=“${hybrid_model_file}” \
# --input_file=“${input_file_path}” \
# --conv_mapping_file=“${mapping_file}” \
# --conv_dump_path=“${hybrid_output_path}” \
# --input_data_type=${input_type} \
# --input_width=${input_width} \
# --input_height=${input_height}
[root@32b86d5ad1a9 02_mapper]#
[root@32b86d5ad1a9 02_mapper]#