mirrors/mmdeploy
1
0
Fork 0
mirror of https://github.com/open-mmlab/mmdeploy.git synced 2025-01-14 08:09:43 +08:00
Code Issues Projects Releases Wiki Activity
mmdeploy/docs/zh_cn/02-how-to-run/fuse_transform.md
Chen Xin 6b01a2e649
[Feature] Add option to fuse transform. (#741) 
* add collect_impl.cpp to cuda device

* add dummy compute node wich device elena

* add compiler & dynamic library loader

* add code to compile with gen code(elena)

* move folder

* fix lint

* add tracer module

* add license

* update type id

* add fuse kernel registry

* remove compilier & dynamic_library

* update fuse kernel interface

* Add elena-mmdeploy project in 3rd-party

* Fix README.md

* fix cmake file

* Support cuda device and clang format all file

* Add cudaStreamSynchronize for cudafree

* fix cudaStreamSynchronize

* rename to __tracer__

* remove unused code

* update kernel

* update extract elena script

* update gitignore

* fix ci

* Change the crop_size to crop_h and crop_w in arglist

* update Tracer

* remove cond

* avoid allocate memory

* add build.sh for elena

* remove code

* update test

* Support bilinear resize with float input

* Rename elena-mmdeploy to delete

* Introduce public submodule

* use get_ref

* update elena

* update tools

* update tools

* update fuse transform docs

* add fuse transform doc link to get_started

* fix shape in crop

* remove fuse_transform_ == true check

* remove fuse_transform_ member

* remove elena_int.h

* doesn't dump transform_static.json

* update tracer

* update CVFusion to remove compile warning

* remove mmcv version > 1.5.1 dep

* fix tests

* update docs

* add elena use option

* remove submodule of CVFusion

* update doc

* use auto

* use throw_exception(eEntryNotFound);

* update

Co-authored-by: cx <cx@ubuntu20.04>
Co-authored-by: miraclezqc <969226879@qq.com>
2022-09-05 20:29:18 +08:00

2.2 KiB
Raw Blame History

融合预处理（实验性功能）

MMDeploy提供了一些Transform融合的能力，当使用SDK进行推理时，可以通过修改pipeline.json来开启融合选项，在某些Transform的组合下可以对预处理进行加速。

若要在MMDeploy的SDK中加入融合能力，可参考CVFusion的使用。

一、使用CVFusion

有两种选择，一种是在编译mmdeploy的时候，使用我们提供的融合kernel代码，一种是自己使用CVFusion生成融合kernel的代码。

A）使用提供的kernel代码

  1. 从这里下载代码，并解压，将csrc文件夹拷贝到mmdeploy的根目录。

    elena_kernel-20220823.tar.gz

  2. 编译mmdeploy的时候，增加选项-DMMDEPLOY_ELENA_FUSION=ON

B) 使用CVFusion生成kernel

  1. 编译CVFusion

    $ git clone --recursive https://github.com/OpenComputeLab/CVFusion.git
$ cd CVFusion
$ bash build.sh

# add OpFuse to PATH
$ export PATH=`pwd`/build/examples/MMDeploy:$PATH

  2. 下载各个算法codebase

    $ tree -L 1 .
├── mmdeploy
├── mmclassification
├── mmdetection
├── mmsegmentation
├── ...

  3. 生成融合kernel

    python tools/elena/extract_transform.py ..
# 生成的代码会保存在csrc/preprocess/elena/{cpu_kernel}/{cuda_kernel}

  4. 编译mmdeploy的时候，增加选项-DMMDEPLOY_ELENA_FUSION=ON

二、模型转换

模型转换时通过--dump-info生成SDK所需文件。

$ export MODEL_CONFIG=/path/to/mmclassification/configs/resnet/resnet18_8xb32_in1k.py
$ export MODEL_PATH=https://download.openmmlab.com/mmclassification/v0/resnet/resnet18_8xb32_in1k_20210831-fbbb1da6.pth

$ python tools/deploy.py \
    configs/mmcls/classification_onnxruntime_static.py \
    $MODEL_CONFIG \
    $MODEL_PATH \
    tests/data/tiger.jpeg \
    --work-dir resnet18 \
    --device cpu \
    --dump-info

三、模型推理

若当前pipeline的预处理模块支持融合，pipeline.json中会有fuse_transform字段，表示融合开关，默认为false。当启用融合算法时，需要把false改为true