mirror of
https://github.com/open-mmlab/mmdeploy.git
synced 2025-01-14 08:09:43 +08:00
36124f6205
* check in cmake * move backend_ops to csrc/backend_ops * check in preprocess, model, some codebase and their c-apis * check in CMakeLists.txt * check in parts of test_csrc * commit everything else * add readme * update core's BUILD_INTERFACE directory * skip codespell on third_party * update trt_net and ort_net's CMakeLists * ignore clion's build directory * check in pybind11 * add onnx.proto. Remove MMDeploy's dependency on ncnn's source code * export MMDeployTargets only when MMDEPLOY_BUILD_SDK is ON * remove useless message * target include directory is wrong * change target name from mmdeploy_ppl_net to mmdeploy_pplnn_net * skip install directory * update project's cmake * remove useless code * set CMAKE_BUILD_TYPE to Release by force if it isn't set by user * update custom ops CMakeLists * pass object target's source lists * fix lint end-of-file * fix lint: trailing whitespace * fix codespell hook * remove bicubic_interpolate to csrc/backend_ops/ * set MMDEPLOY_BUILD_SDK OFF * change custom ops build command * add spdlog installation command * update docs on how to checkout pybind11 * move bicubic_interpolate to backend_ops/tensorrt directory * remove useless code * correct cmake * fix typo * fix typo * fix install directory * correct sdk's readme * set cub dir when cuda version < 11.0 * change directory where clang-format will apply to * fix build command * add .clang-format * change clang-format style from google to file * reformat csrc/backend_ops * format sdk's code * turn off clang-format for some files * add -Xcompiler=-fno-gnu-unique * fix trt topk initialize * check in config for sdk demo * update cmake script and csrc's readme * correct config's path * add cuda include directory, otherwise compile failed in case of tensorrt8.2 * clang-format onnx2ncnn.cpp Co-authored-by: zhangli <lzhang329@gmail.com> Co-authored-by: grimoire <yaoqian@sensetime.com>
91 lines
3.3 KiB
Markdown
91 lines
3.3 KiB
Markdown
## ONNX Runtime Support
|
|
|
|
### Introduction of ONNX Runtime
|
|
|
|
**ONNX Runtime** is a cross-platform inferencing and training accelerator compatible with many popular ML/DNN frameworks. Check its [github](https://github.com/microsoft/onnxruntime) for more information.
|
|
|
|
### Installation
|
|
|
|
*Please note that only **onnxruntime>=1.8.1** of CPU version on Linux platform is supported by now.*
|
|
|
|
- Install ONNX Runtime python package
|
|
|
|
```bash
|
|
pip install onnxruntime==1.8.1
|
|
```
|
|
|
|
### Build custom ops
|
|
|
|
#### Prerequisite
|
|
|
|
- Download `onnxruntime-linux` from ONNX Runtime [releases](https://github.com/microsoft/onnxruntime/releases/tag/v1.8.1), extract it, expose `ONNXRUNTIME_DIR` and finally add the lib path to `LD_LIBRARY_PATH` as below:
|
|
|
|
```bash
|
|
wget https://github.com/microsoft/onnxruntime/releases/download/v1.8.1/onnxruntime-linux-x64-1.8.1.tgz
|
|
|
|
tar -zxvf onnxruntime-linux-x64-1.8.1.tgz
|
|
cd onnxruntime-linux-x64-1.8.1
|
|
export ONNXRUNTIME_DIR=$(pwd)
|
|
export LD_LIBRARY_PATH=$ONNXRUNTIME_DIR/lib:$LD_LIBRARY_PATH
|
|
```
|
|
|
|
Note:
|
|
|
|
- If you want to save onnxruntime env variables to bashrc, you could run
|
|
|
|
```bash
|
|
echo '# set env for onnxruntime' >> ~/.bashrc
|
|
echo "export ONNXRUNTIME_DIR=${ONNXRUNTIME_DIR}" >> ~/.bashrc
|
|
echo 'export LD_LIBRARY_PATH=$ONNXRUNTIME_DIR/lib:$LD_LIBRARY_PATH' >> ~/.bashrc
|
|
source ~/.bashrc
|
|
```
|
|
|
|
#### Build on Linux
|
|
|
|
```bash
|
|
cd ${MMDEPLOY_DIR} # To MMDeploy root directory
|
|
mkdir build
|
|
cd build
|
|
cmake -DMMDEPLOY_TARGET_BACKENDS=ort -DONNXRUNTIME_DIR=${ONNXRUNTIME_DIR} ..
|
|
make -j10
|
|
```
|
|
|
|
### How to convert a model
|
|
|
|
- You could follow the instructions of tutorial [How to convert model](../tutorials/how_to_convert_model.md)
|
|
|
|
### List of supported custom ops
|
|
|
|
| Operator | CPU | GPU | MMDeploy Releases |
|
|
|:-----------------------------------------------------------------------------|:---:|:---:|:------------------|
|
|
| [RoIAlign](../ops/onnxruntime.md#roialign) | Y | N | master |
|
|
| [grid_sampler](../ops/onnxruntime.md#grid_sampler) | Y | N | master |
|
|
| [MMCVModulatedDeformConv2d](../ops/onnxruntime.md#mmcvmodulateddeformconv2d) | Y | N | master |
|
|
|
|
### How to add a new custom op
|
|
|
|
#### Reminder
|
|
|
|
- The custom operator is not included in [supported operator list](https://github.com/microsoft/onnxruntime/blob/master/docs/OperatorKernels.md) in ONNX Runtime.
|
|
- The custom operator should be able to be exported to ONNX.
|
|
|
|
#### Main procedures
|
|
|
|
Take custom operator `roi_align` for example.
|
|
|
|
1. Create a `roi_align` directory in ONNX Runtime source directory `backend_ops/onnxruntime/`
|
|
2. Add header and source file into `roi_align` directory `backend_ops/onnxruntime/roi_align/`
|
|
3. Add unit test into `tests/test_ops/test_ops.py`
|
|
Check [here](../../tests/test_ops/test_ops.py) for examples.
|
|
|
|
**Finally, welcome to send us PR of adding custom operators for ONNX Runtime in MMDeploy.** :nerd_face:
|
|
|
|
### FAQs
|
|
|
|
- None
|
|
|
|
### References
|
|
|
|
- [How to export Pytorch model with custom op to ONNX and run it in ONNX Runtime](https://github.com/onnx/tutorials/blob/master/PyTorchCustomOperator/README.md)
|
|
- [How to add a custom operator/kernel in ONNX Runtime](https://github.com/microsoft/onnxruntime/blob/master/docs/AddingCustomOp.md)
|