77 lines
2.2 KiB
Markdown
77 lines
2.2 KiB
Markdown
|
# Fuse Transform（Experimental）
|
|||
|
|
|
|||
|
|
MMDeploy provides ability to fuse transform for acceleration in some cases.
|
|||
|
|
|
|||
|
|
When make inference with SDK, one can edit the pipeline.json to turn on the fuse option.
|
|||
|
|
|
|||
|
|
To bring the ability of fuse transform to MMDeploy, you can refer to the use of CVFusion.
|
|||
|
|
|
|||
|
|
## 1. Use CVFusion
|
|||
|
|
|
|||
|
|
There are two ways to use CVFusion, one is to use the pre-generated kernel code, the other is to generate the code yourself.
|
|||
|
|
|
|||
|
|
A）Use pre-generated kernel code
|
|||
|
|
|
|||
|
|
i) Download the kernel code from here，unzip it and copy the csrc folder to the mmdeploy root folder.
|
|||
|
|
|
|||
|
|
[elena_kernel-20220823.tar.gz](https://github.com/open-mmlab/mmdeploy/files/9399795/elena_kernel-20220823.tar.gz)
|
|||
|
|
|
|||
|
|
ii) Add option `-DMMDEPLOY_ELENA_FUSION=ON` when compile MMDeploy.
|
|||
|
|
|
|||
|
|
B) Generate kernel code by yourself
|
|||
|
|
|
|||
|
|
i) Compile CVFusion
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
$ git clone --recursive https://github.com/OpenComputeLab/CVFusion.git
|
|||
|
|
$ cd CVFusion
|
|||
|
|
$ bash build.sh
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
```
|
|||
|
|
# add OpFuse to PATH
|
|||
|
|
$ export PATH=`pwd`/build/examples/MMDeploy:$PATH
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
ii) Download algorithm codebase
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
$ tree -L 1 .
|
|||
|
|
├── mmdeploy
|
|||
|
|
├── mmclassification
|
|||
|
|
├── mmdetection
|
|||
|
|
├── mmsegmentation
|
|||
|
|
├── ...
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
iii) Generate kernel code
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
python tools/elena/extract_transform.py ..
|
|||
|
|
# The generated code will be saved to csrc/preprocess/elena/{cpu_kernel}/{cuda_kernel}
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
iv) Add option `-DMMDEPLOY_ELENA_FUSION=ON` when compile MMDeploy.
|
|||
|
|
|
|||
|
|
## 2. Model conversion
|
|||
|
|
|
|||
|
|
Add `--dump-info` argument when convert a model, this will generate files that SDK needs.
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
$ export MODEL_CONFIG=/path/to/mmclassification/configs/resnet/resnet18_8xb32_in1k.py
|
|||
|
|
$ export MODEL_PATH=https://download.openmmlab.com/mmclassification/v0/resnet/resnet18_8xb32_in1k_20210831-fbbb1da6.pth
|
|||
|
|
|
|||
|
|
$ python tools/deploy.py \
|
|||
|
|
configs/mmcls/classification_onnxruntime_static.py \
|
|||
|
|
$MODEL_CONFIG \
|
|||
|
|
$MODEL_PATH \
|
|||
|
|
tests/data/tiger.jpeg \
|
|||
|
|
--work-dir resnet18 \
|
|||
|
|
--device cpu \
|
|||
|
|
--dump-info
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
## 3. Model Inference
|
|||
|
|
|
|||
|
|
If the model preprocess supports fusion, there will be a filed named `fuse_transform` in `pipeline.json`. It represents fusion switch and the default value `false` stands for off. One need to edit this filed to `true` to use the fuse option.
|