环境相关

python3 -m pip install onnx==1.8.1 onnx-simplifier

onnx-simplifier 在 1.8.1之后就从onnx中分离了

检查protobuf的版本一至性

$ protoc --version
libprotoc 3.7.0

$ python3 -m pip show protobuf
Name: protobuf
Version: 3.12.4
Summary: Protocol Buffers
Home-page: https://developers.google.com/protocol-buffers/
Author: 
Author-email: 
License: 3-Clause BSD License
Location: /home/aipos/.local/lib/python3.8/site-packages
Requires: setuptools, six
Required-by: onnx, onnx-simplifier, onnxruntime


yolov5s V5

问题解决

class Detect(nn.Module):
    stride = None  # strides computed during build
    onnx_dynamic = True  # ONNX export parameter

    def __init__(self, nc=80, anchors=(), ch=()):  # detection layer
        super(Detect, self).__init__()
        self.nc = nc  # number of classes
        self.no = nc + 5  # number of outputs per anchor
        self.nl = len(anchors)  # number of detection layers
        self.na = len(anchors[0]) // 2  # number of anchors
        self.grid = [torch.zeros(1)] * self.nl  # init grid
        a = torch.tensor(anchors).float().view(self.nl, -1, 2)
        self.register_buffer('anchors', a)  # shape(nl,na,2)
        self.register_buffer('anchor_grid', a.clone().view(self.nl, 1, -1, 1, 1, 2))  # shape(nl,1,na,1,1,2)
        self.m = nn.ModuleList(nn.Conv2d(x, self.no * self.na, 1) for x in ch)  # output conv

    def forward(self, x):
        # x = x.copy()  # for profiling
        z = []  # inference output
        #self.training |= self.export
        for i in range(self.nl):
            x[i] = self.m[i](x[i])  # conv
            bs, _, ny, nx = x[i].shape  # x(bs,255,20,20) to x(bs,3,20,20,85)
            x[i] = x[i].view(bs, self.na, self.no, ny, nx).permute(0, 1, 3, 4, 2).contiguous()

            if not self.training:  # inference
                if self.grid[i].shape[2:4] != x[i].shape[2:4]:
                    self.grid[i] = self._make_grid(nx, ny).to(x[i].device)

        #         y = x[i].sigmoid()
        #         y[..., 0:2] = (y[..., 0:2] * 2. - 0.5 + self.grid[i]) * self.stride[i]  # xy
        #         y[..., 2:4] = (y[..., 2:4] * 2) ** 2 * self.anchor_grid[i]  # wh
        #         z.append(y.view(bs, -1, self.no))

        # return x if self.training else (torch.cat(z, 1), x)
        return x

shape要一至

    # Input to the model
    dummy_input = torch.randn(1, 3, 1280, 1280)

    # Export the model
    torch.onnx.export(torch_model,
                      dummy_input,
                      output_onnx,  # 保存路径
                      verbose=True,
                      export_params=True,
                      opset_version=11,  # the ONNX version to export the model to
                      do_constant_folding=True,
                      training=torch.onnx.TrainingMode.EVAL,
                      input_names=['images'],
                      output_names=['output'],
                      dynamic_axes=None
                      )

编译失败与convert_tool崩溃的处理

asymmetric的问题,因为下面的mode值为nearest,所以align_corner并没起作用。

# V5
python3 ./tools/optimize/yolov5s-opt.py --input model.onnx --output model_opt.onnx --in_tensor 403 --out_tensor output,712,732 --verbose
# V6
python3 ./tools/optimize/yolov5s-opt.py --input model.onnx --output model_opt.onnx --in_tensor 403 --out_tensor output,712,732 --cut_focus --verbose

in_tensor

out_tensor

wii@wiiD:/media/wii/disk1t/Tengine$ git diff tools/convert_tool/onnx/onnx2tengine.cpp
diff --git a/tools/convert_tool/onnx/onnx2tengine.cpp b/tools/convert_tool/onnx/onnx2tengine.cpp
index 1aa3c57c..aca5f85c 100644
--- a/tools/convert_tool/onnx/onnx2tengine.cpp
+++ b/tools/convert_tool/onnx/onnx2tengine.cpp
@@ -2105,7 +2105,7 @@ static int load_resize(ir_graph_t* graph, ir_node_t* node, const onnx::NodeProto
     interp_param->width_scale = 0;
 
     std::string coordinate_transformation_mode = GetAttributeOrDefault<std::string>(onnx_node, "coordinate_transformation_mode", "half_pixel");
-    TASSERT(coordinate_transformation_mode == "half_pixel" || coordinate_transformation_mode == "align_corners");
+    TASSERT(coordinate_transformation_mode == "half_pixel" || coordinate_transformation_mode == "align_corners" || coordinate_transformation_mode == "asymmetric");
     int align_corner = (coordinate_transformation_mode == "align_corners");
 
     if (onnx_node.input_size() == 1)
wii@wiiD:/media/wii/disk1t/Tengine$ git diff tools/save_graph/tm2_op_save.cpp
diff --git a/tools/save_graph/tm2_op_save.cpp b/tools/save_graph/tm2_op_save.cpp
index 7cd66bcd..1afc4eb6 100644
--- a/tools/save_graph/tm2_op_save.cpp
+++ b/tools/save_graph/tm2_op_save.cpp
@@ -21,6 +21,7 @@
  * Copyright (c) 2019, Open AI Lab
  * Author: jingyou@openailab.com
  */
+#include <stdlib.h>
 #include <string.h>
 
 #include "tm2_op_save.hpp"
./convert_tool -f onnx -m model_opt.onnx -o model.tmfile
  1. PyTorch转ONNX格式模型

  1. 删减foucs与优化

  1. ONNX转Tengine


测试

./build/install/bin/tm_yolov5s -m model.tmfile -i ./1.jpg -r 10


精度调整到UINT8

yolov5s 注意需要 -k -c -y 参数

time ./build/install/bin/quant_tool_uint8 -m yolov5s_v5_opt.tmfile -i ./test -o yolov5s_v5_uint8.tmfile -k 1 -c 1 -y 640,640 -g 12,640,640 -w 0,0,0 -s 0.004,0.004,0.004

---- Tengine Post Training Quantization Tool ---- 

Version     : v1.2, 10:34:20 Dec 24 2021
Status      : uint8, per-layer, asymmetric
Input model : yolov5s_v5_opt.tmfile
Output model: yolov5s_v5_uint8_2.tmfile
Calib images: ./test
Scale file  : NULL
Algorithm   : 0
Dims        : 12 640 640
Mean        : 0.000 0.000 0.000
Scale       : 0.004 0.004 0.004
BGR2RGB     : ON
Center crop : ON
Letter box  : 640 640
YOLOv5 focus: ON
Thread num  : 4

[Quant Tools Info]: Step 0, load FP32 tmfile.
[Quant Tools Info]: Step 0, load FP32 tmfile done.
[Quant Tools Info]: Step 0, load calibration image files.
[Quant Tools Info]: Step 0, load calibration image files done, image num is 697.
[Quant Tools Info]: Step 1, find original calibration table.
[Quant Tools Info]: Step 1, images 00697 / 00697
[Quant Tools Info]: Step 2, find original calibration table done, output ./table_minmax.scale
[Quant Tools Info]: Thread 4, image nums 697, total time 871674.08 ms, avg time 1250.61 ms
[Quant Tools Info]: Step 3, load FP32 tmfile once again
[Quant Tools Info]: Step 3, load FP32 tmfile once again done.
[Quant Tools Info]: Step 3, load calibration table file table_minmax.scale.
[Quant Tools Info]: Step 4, optimize the calibration table.
[Quant Tools Info]: Step 4, quantize activation tensor done.
[Quant Tools Info]: Step 5, quantize weight tensor done.
[Quant Tools Info]: Step 6, save UInt8 tmfile done, yolov5s_v5_uint8_2.tmfile

---- Tengine Int8 tmfile create success, best wish for your UInt8 inference has a low accuracy loss...\(^0^)/ ----

real    21m32.009s
user    48m9.673s
sys     0m13.687s