mirror of
https://github.com/open-mmlab/mmdeploy.git
synced 2025-01-14 08:09:43 +08:00
6b01a2e649
* add collect_impl.cpp to cuda device * add dummy compute node wich device elena * add compiler & dynamic library loader * add code to compile with gen code(elena) * move folder * fix lint * add tracer module * add license * update type id * add fuse kernel registry * remove compilier & dynamic_library * update fuse kernel interface * Add elena-mmdeploy project in 3rd-party * Fix README.md * fix cmake file * Support cuda device and clang format all file * Add cudaStreamSynchronize for cudafree * fix cudaStreamSynchronize * rename to __tracer__ * remove unused code * update kernel * update extract elena script * update gitignore * fix ci * Change the crop_size to crop_h and crop_w in arglist * update Tracer * remove cond * avoid allocate memory * add build.sh for elena * remove code * update test * Support bilinear resize with float input * Rename elena-mmdeploy to delete * Introduce public submodule * use get_ref * update elena * update tools * update tools * update fuse transform docs * add fuse transform doc link to get_started * fix shape in crop * remove fuse_transform_ == true check * remove fuse_transform_ member * remove elena_int.h * doesn't dump transform_static.json * update tracer * update CVFusion to remove compile warning * remove mmcv version > 1.5.1 dep * fix tests * update docs * add elena use option * remove submodule of CVFusion * update doc * use auto * use throw_exception(eEntryNotFound); * update Co-authored-by: cx <cx@ubuntu20.04> Co-authored-by: miraclezqc <969226879@qq.com>
77 lines
2.2 KiB
Markdown
77 lines
2.2 KiB
Markdown
# Fuse Transform（Experimental）
|
||
|
||
MMDeploy provides ability to fuse transform for acceleration in some cases.
|
||
|
||
When make inference with SDK, one can edit the pipeline.json to turn on the fuse option.
|
||
|
||
To bring the ability of fuse transform to MMDeploy, you can refer to the use of CVFusion.
|
||
|
||
## 1. Use CVFusion
|
||
|
||
There are two ways to use CVFusion, one is to use the pre-generated kernel code, the other is to generate the code yourself.
|
||
|
||
A）Use pre-generated kernel code
|
||
|
||
i) Download the kernel code from here，unzip it and copy the csrc folder to the mmdeploy root folder.
|
||
|
||
[elena_kernel-20220823.tar.gz](https://github.com/open-mmlab/mmdeploy/files/9399795/elena_kernel-20220823.tar.gz)
|
||
|
||
ii) Add option `-DMMDEPLOY_ELENA_FUSION=ON` when compile MMDeploy.
|
||
|
||
B) Generate kernel code by yourself
|
||
|
||
i) Compile CVFusion
|
||
|
||
```bash
|
||
$ git clone --recursive https://github.com/OpenComputeLab/CVFusion.git
|
||
$ cd CVFusion
|
||
$ bash build.sh
|
||
```
|
||
|
||
```
|
||
# add OpFuse to PATH
|
||
$ export PATH=`pwd`/build/examples/MMDeploy:$PATH
|
||
```
|
||
|
||
ii) Download algorithm codebase
|
||
|
||
```bash
|
||
$ tree -L 1 .
|
||
├── mmdeploy
|
||
├── mmclassification
|
||
├── mmdetection
|
||
├── mmsegmentation
|
||
├── ...
|
||
```
|
||
|
||
iii) Generate kernel code
|
||
|
||
```bash
|
||
python tools/elena/extract_transform.py ..
|
||
# The generated code will be saved to csrc/preprocess/elena/{cpu_kernel}/{cuda_kernel}
|
||
```
|
||
|
||
iv) Add option `-DMMDEPLOY_ELENA_FUSION=ON` when compile MMDeploy.
|
||
|
||
## 2. Model conversion
|
||
|
||
Add `--dump-info` argument when convert a model, this will generate files that SDK needs.
|
||
|
||
```bash
|
||
$ export MODEL_CONFIG=/path/to/mmclassification/configs/resnet/resnet18_8xb32_in1k.py
|
||
$ export MODEL_PATH=https://download.openmmlab.com/mmclassification/v0/resnet/resnet18_8xb32_in1k_20210831-fbbb1da6.pth
|
||
|
||
$ python tools/deploy.py \
|
||
configs/mmcls/classification_onnxruntime_static.py \
|
||
$MODEL_CONFIG \
|
||
$MODEL_PATH \
|
||
tests/data/tiger.jpeg \
|
||
--work-dir resnet18 \
|
||
--device cpu \
|
||
--dump-info
|
||
```
|
||
|
||
## 3. Model Inference
|
||
|
||
If the model preprocess supports fusion, there will be a filed named `fuse_transform` in `pipeline.json`. It represents fusion switch and the default value `false` stands for off. One need to edit this filed to `true` to use the fuse option.
|