Pointpillars导出不含预处理模型的配置说明

前言

对于PP参考算法,导出的模型默认有预处理,如果您需要导出不含预处理的PP模型,可以按照以下步骤修改配置。这里以J5 OE 1.1.62为例介绍修改流程,不同版本可能会有微小区别。

Config

horizon_model_train_sample/scripts/configs/detection/pointpillars/pointpillars_kitti_car.py-
注释deploy_model中的pre_process部分:

# model settings
deploy_model = dict(
    type="PointPillarsDetector",
    feature_map_shape=get_feature_map_size(pc_range, voxel_size),
    is_deploy=True,
    # pre_process=dict(
    #     type="PointPillarsPreProcess",
    #     pc_range=pc_range,
    #     voxel_size=voxel_size,
    #     max_voxels_num=max_voxels_num,
    #     max_points_in_voxel=max_points_in_voxel,
    # ),
    reader=dict(

将deploy_inputs修改为PFN输入:

deploy_inputs = dict(
    voxels=torch.randn((1, 4, 12000, 100), dtype=torch.float32),
    coordinates=torch.zeros([12000, 4]).int(),
)

structure

/usr/local/lib/python3.8/dist-packages/hat/models/structures/detectors/pointpillar.py-
保留features和coors:

data = dict(  # noqa C408
                    features=example["voxels"],
                    #num_points_in_voxel=example["num_points"],
                    coors=example["coordinates"],
                    batch_size=1,
                    input_shape=self.feature_map_shape,
                )

            input_features = self.reader(
                features=data["features"],
                #num_voxels=data["num_points_in_voxel"],
                coors=data["coors"],
                horizon_preprocess=self.use_horizon_preprocess,
            )

PFNLayer

/usr/local/lib/python3.8/dist-packages/hat/models/task_modules/lidar/pillar_encoder.py-
做出以下修改:

class PillarFeatureNet(nn.Module):
    ...
     def forward(
        self,
        features: torch.Tensor,
        # num_voxels: torch.Tensor,
        coors: torch.Tensor,
        horizon_preprocess: bool = False,
    ):
        if horizon_preprocess:
            # used horizon preprocess(which support quantize),
            # skip default preprocess here.
            features = self._extract_feature(features)
        else:
            # default preprocess
            # assert num_voxels is not None, "`num_voxels` can not be None."
            # features = self._extend_dim(features, num_voxels, coors)
            features = self._extract_feature(features)
        return features

compile

运行以下命令编译不带预处理的PP模型:

python3 tools/compile_perf.py --config configs/detection/pointpillars/pointpillars_kitti_car.py --out-dir ./ --opt 3

horizon_j5_open_explorer_v1.1.68 OE环境,转换过程报错

你好,XJ3的v2.6.2版本的OE可否使用上述方法 编译PP模型,运行时报下面问题? XJ3的ai_toolchain如何编译出ai_benchmark可用的.hbm文件,可否提供些指导?

不建议在XJ3部署pp模型

ok,请问具体原因是什么?我们目前存在XJ3 激光雷达算法的需求,有什么建议?

需要自行编写大量C++代码,且性能难以保证,具体请参考别的帖子的回复。

好的,谢谢!