2021-12-16 19:22:14 +08:00
< div align = "center" >
2021-12-23 20:14:35 +08:00
< img src = "resources/mmdeploy-logo.png" width = "450" / >
2022-01-13 10:19:21 +08:00
< div > < / div >
< div align = "center" >
< b > < font size = "5" > OpenMMLab website< / font > < / b >
< sup >
2022-07-19 17:57:20 +08:00
< a href = "https://openmmlab.com" >
2022-01-13 10:19:21 +08:00
< i > < font size = "4" > HOT< / font > < / i >
< / a >
< / sup >
< b > < font size = "5" > OpenMMLab platform< / font > < / b >
< sup >
< a href = "https://platform.openmmlab.com" >
< i > < font size = "4" > TRY IT OUT< / font > < / i >
< / a >
< / sup >
< / div >
< div > < / div >
2021-12-16 19:22:14 +08:00
< / div >
2022-10-24 14:15:26 +08:00
2021-12-27 11:20:26 +08:00
[](https://mmdeploy.readthedocs.io/en/latest/)
[](https://github.com/open-mmlab/mmdeploy/actions)
[](https://codecov.io/gh/open-mmlab/mmdeploy)
[](https://github.com/open-mmlab/mmdeploy/blob/master/LICENSE)
2022-06-07 18:05:51 +08:00
[](https://github.com/open-mmlab/mmdeploy/issues)
[](https://github.com/open-mmlab/mmdeploy/issues)
2021-12-27 11:20:26 +08:00
2021-10-09 14:10:42 +08:00
English | [简体中文 ](README_zh-CN.md )
2022-06-07 18:05:51 +08:00
## Introduction
MMDeploy is an open-source deep learning model deployment toolset. It is a part of the [OpenMMLab ](https://openmmlab.com/ ) project.
2021-10-09 14:10:42 +08:00
2021-12-16 19:22:14 +08:00
< div align = "center" >
2022-06-07 18:05:51 +08:00
< img src = "resources/introduction.png" >
2021-12-16 19:22:14 +08:00
< / div >
2022-06-07 18:05:51 +08:00
## Main features
2021-10-09 14:10:42 +08:00
2022-06-07 18:05:51 +08:00
### Fully support OpenMMLab models
2021-10-25 16:37:33 +08:00
2022-06-07 18:05:51 +08:00
The currently supported codebases and models are as follows, and more will be included in the future
2022-06-17 09:19:10 +08:00
- [mmcls ](docs/en/04-supported-codebases/mmcls.md )
- [mmdet ](docs/en/04-supported-codebases/mmdet.md )
- [mmseg ](docs/en/04-supported-codebases/mmseg.md )
- [mmedit ](docs/en/04-supported-codebases/mmedit.md )
- [mmocr ](docs/en/04-supported-codebases/mmocr.md )
- [mmpose ](docs/en/04-supported-codebases/mmpose.md )
- [mmdet3d ](docs/en/04-supported-codebases/mmdet3d.md )
- [mmrotate ](docs/en/04-supported-codebases/mmrotate.md )
2022-11-24 16:40:56 +08:00
- [mmaction2 ](docs/en/04-supported-codebases/mmaction2.md )
2021-10-09 14:10:42 +08:00
2022-06-07 18:05:51 +08:00
### Multiple inference backends are available
2021-10-25 16:37:33 +08:00
2022-09-27 20:05:32 +08:00
The supported Device-Platform-InferenceBackend matrix is presented as following, and more will be compatible.
2021-07-01 17:32:33 +08:00
2022-09-27 20:05:32 +08:00
The benchmark can be found from [here ](docs/en/03-benchmark/benchmark.md )
2022-12-29 16:08:29 +08:00
| Device / Platform | Linux | Windows | macOS | Android |
| ----------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------- |
| x86_64 CPU | [![Build Status][pass-backend-ort]][ci-backend-ort]ONNXRuntime< br > [![Build Status][pass-backend-pplnn]][ci-backend-pplnn]pplnn< br > [![Build Status][pass-backend-ncnn]][ci-backend-ncnn]ncnn< br > [![Build Status][pass-backend-torchscript]][ci-backend-torchscript]LibTorch< br > [![Build Status][pass-build-rknpu]][ci-build-rknpu]OpenVINO< br > [![Build Status][pass-build-tvm]][ci-build-tvm]TVM | ![][pass-no-status]ONNXRuntime< br > ![][pass-no-status]OpenVINO | - | - |
| ARM CPU | [![Build Status][pass-build-rknpu]][ci-build-rknpu]ncnn | - | - | [![Build Status][pass-build-rknpu]][ci-build-rknpu]ncnn |
| RISC-V | [![Build Status][pass-build-riscv64-gcc]][ci-build-riscv64-gcc]ncnn | - | - | - |
| NVIDIA GPU | ![Build Status][pass-no-status]ONNXRuntime< br > ![Build Status][pass-no-status]TensorRT< br > ![Build Status][pass-no-status]pplnn< br > ![Build Status][pass-no-status]LibTorch< br > ![Build Status][pass-no-status]TVM | ![Build Status][pass-no-status]ONNXRuntime< br > ![Build Status][pass-no-status]TensorRT< br > ![Build Status][pass-no-status]pplnn | - | - |
| NVIDIA Jetson | [![Build Status][pass-build-rknpu]][ci-build-rknpu]TensorRT | - | - | - |
| Huawei ascend310 | [![Build Status][pass-backend-ascend]][ci-backend-ascend]CANN | - | - | - |
| Rockchip | [![Build Status][pass-backend-rknn]][ci-backend-rknn]RKNN | - | - | - |
| Apple M1 | - | - | [![Build Status][pass-backend-coreml]][ci-backend-coreml]CoreML | - |
| Adreno GPU | - | - | - | [![Build Status][pass-backend-snpe]][ci-backend-snpe]SNPE< br > [![Build Status][pass-build-rknpu]][ci-build-rknpu]ncnn |
| Hexagon DSP | - | - | - | [![Build Status][pass-backend-snpe]][ci-backend-snpe]SNPE |
```
|
```
2021-07-10 16:16:11 +08:00
2022-06-07 18:05:51 +08:00
### Efficient and scalable C/C++ SDK Framework
2021-07-01 17:32:33 +08:00
2022-06-07 18:05:51 +08:00
All kinds of modules in the SDK can be extended, such as `Transform` for image processing, `Net` for Neural Network inference, `Module` for postprocessing and so on
2021-10-09 14:10:42 +08:00
2022-08-15 10:18:17 +08:00
## [Documentation](https://mmdeploy.readthedocs.io/en/latest/)
2021-12-21 13:59:07 +08:00
2022-08-15 10:18:17 +08:00
Please read [getting_started ](docs/en/get_started.md ) for the basic usage of MMDeploy. We also provide tutoials about:
2021-12-23 11:25:45 +08:00
2022-06-17 09:19:10 +08:00
- [Build ](docs/en/01-how-to-build/build_from_source.md )
- [Build from Docker ](docs/en/01-how-to-build/build_from_docker.md )
2022-08-26 13:53:53 +08:00
- [Build from Script ](docs/en/01-how-to-build/build_from_script.md )
2022-06-17 09:19:10 +08:00
- [Build for Linux ](docs/en/01-how-to-build/linux-x86_64.md )
2022-11-29 13:42:47 +08:00
- [Build for Windows ](docs/en/01-how-to-build/windows.md )
2022-09-05 19:55:47 +08:00
- [Build for macOS ](docs/en/01-how-to-build/macos-arm64.md )
2022-06-17 09:19:10 +08:00
- [Build for Win10 ](docs/en/01-how-to-build/windows.md )
- [Build for Android ](docs/en/01-how-to-build/android.md )
- [Build for Jetson ](docs/en/01-how-to-build/jetsons.md )
2022-08-01 11:08:55 +08:00
- [Build for SNPE ](docs/en/01-how-to-build/snpe.md )
2022-11-29 13:42:47 +08:00
- [Build for Rockchip ](docs/en/01-how-to-build/rockchip.md )
2022-10-17 11:15:29 +08:00
- [Cross Build for aarch64 ](docs/en/01-how-to-build/cross_build_ncnn_aarch64.md )
2022-06-17 09:19:10 +08:00
- User Guide
- [How to convert model ](docs/en/02-how-to-run/convert_model.md )
- [How to write config ](docs/en/02-how-to-run/write_config.md )
2022-08-15 10:18:17 +08:00
- [How to profile model ](docs/en/02-how-to-run/profile_model.md )
- [How to quantize model ](docs/en/02-how-to-run/quantize_model.md )
- [Useful tools ](docs/en/02-how-to-run/useful_tools.md )
2022-06-17 09:19:10 +08:00
- Developer Guide
2022-08-15 15:16:20 +08:00
- [Architecture ](docs/en/07-developer-guide/architecture.md )
2022-08-15 10:18:17 +08:00
- [How to support new models ](docs/en/07-developer-guide/support_new_model.md )
- [How to support new backends ](docs/en/07-developer-guide/support_new_backend.md )
- [How to partition model ](docs/en/07-developer-guide/partition_model.md )
- [How to test rewritten model ](docs/en/07-developer-guide/test_rewritten_models.md )
- [How to test backend ops ](docs/en/07-developer-guide/add_backend_ops_unittest.md )
- [How to do regression test ](docs/en/07-developer-guide/regression_test.md )
- Custom Backend Ops
- [ncnn ](docs/en/06-custom-ops/ncnn.md )
- [onnxruntime ](docs/en/06-custom-ops/onnxruntime.md )
- [tensorrt ](docs/en/06-custom-ops/tensorrt.md )
2022-06-17 09:19:10 +08:00
- [FAQ ](docs/en/faq.md )
- [Contributing ](.github/CONTRIBUTING.md )
2021-10-09 14:10:42 +08:00
2022-06-07 18:05:51 +08:00
## Benchmark and Model zoo
2021-07-01 17:32:33 +08:00
2022-06-07 18:05:51 +08:00
You can find the supported models from [here ](docs/en/03-benchmark/supported_models.md ) and their performance in the [benchmark ](docs/en/03-benchmark/benchmark.md ).
2021-12-23 11:25:45 +08:00
2021-12-16 19:22:14 +08:00
## Contributing
2021-07-01 17:32:33 +08:00
2022-06-07 18:05:51 +08:00
We appreciate all contributions to MMDeploy. Please refer to [CONTRIBUTING.md ](.github/CONTRIBUTING.md ) for the contributing guideline.
2021-08-10 17:04:58 +08:00
2021-12-16 19:22:14 +08:00
## Acknowledgement
2021-08-10 17:04:58 +08:00
2021-12-29 19:24:03 +08:00
We would like to sincerely thank the following teams for their contributions to [MMDeploy ](https://github.com/open-mmlab/mmdeploy ):
2022-06-17 09:19:10 +08:00
2021-12-29 19:24:03 +08:00
- [OpenPPL ](https://github.com/openppl-public )
- [OpenVINO ](https://github.com/openvinotoolkit/openvino )
2022-06-07 18:05:51 +08:00
- [ncnn ](https://github.com/Tencent/ncnn )
2021-07-01 17:32:33 +08:00
2021-10-27 15:03:40 +08:00
## Citation
2022-06-07 18:05:51 +08:00
If you find this project useful in your research, please consider citing:
2021-10-27 15:03:40 +08:00
```BibTeX
@misc {=mmdeploy,
2021-12-16 19:22:14 +08:00
title={OpenMMLab's Model Deployment Toolbox.},
2021-10-27 15:03:40 +08:00
author={MMDeploy Contributors},
howpublished = {\url{https://github.com/open-mmlab/mmdeploy}},
year={2021}
}
```
2022-06-07 18:05:51 +08:00
## License
This project is released under the [Apache 2.0 license ](LICENSE ).
2021-10-09 14:10:42 +08:00
## Projects in OpenMMLab
2021-07-01 17:32:33 +08:00
2021-10-09 14:10:42 +08:00
- [MMCV ](https://github.com/open-mmlab/mmcv ): OpenMMLab foundational library for computer vision.
2022-02-28 15:38:31 +08:00
- [MIM ](https://github.com/open-mmlab/mim ): MIM installs OpenMMLab packages.
2021-10-09 14:10:42 +08:00
- [MMClassification ](https://github.com/open-mmlab/mmclassification ): OpenMMLab image classification toolbox and benchmark.
- [MMDetection ](https://github.com/open-mmlab/mmdetection ): OpenMMLab detection toolbox and benchmark.
- [MMDetection3D ](https://github.com/open-mmlab/mmdetection3d ): OpenMMLab's next-generation platform for general 3D object detection.
2022-10-24 11:37:11 +08:00
- [MMYOLO ](https://github.com/open-mmlab/mmyolo ): OpenMMLab YOLO series toolbox and benchmark
2022-02-28 15:38:31 +08:00
- [MMRotate ](https://github.com/open-mmlab/mmrotate ): OpenMMLab rotated object detection toolbox and benchmark.
2021-10-09 14:10:42 +08:00
- [MMSegmentation ](https://github.com/open-mmlab/mmsegmentation ): OpenMMLab semantic segmentation toolbox and benchmark.
2022-02-28 15:38:31 +08:00
- [MMOCR ](https://github.com/open-mmlab/mmocr ): OpenMMLab text detection, recognition, and understanding toolbox.
- [MMPose ](https://github.com/open-mmlab/mmpose ): OpenMMLab pose estimation toolbox and benchmark.
- [MMHuman3D ](https://github.com/open-mmlab/mmhuman3d ): OpenMMLab 3D human parametric model toolbox and benchmark.
- [MMSelfSup ](https://github.com/open-mmlab/mmselfsup ): OpenMMLab self-supervised learning toolbox and benchmark.
- [MMRazor ](https://github.com/open-mmlab/mmrazor ): OpenMMLab model compression toolbox and benchmark.
- [MMFewShot ](https://github.com/open-mmlab/mmfewshot ): OpenMMLab fewshot learning toolbox and benchmark.
2021-10-09 14:10:42 +08:00
- [MMAction2 ](https://github.com/open-mmlab/mmaction2 ): OpenMMLab's next-generation action understanding toolbox and benchmark.
- [MMTracking ](https://github.com/open-mmlab/mmtracking ): OpenMMLab video perception toolbox and benchmark.
2022-02-28 15:38:31 +08:00
- [MMFlow ](https://github.com/open-mmlab/mmflow ): OpenMMLab optical flow toolbox and benchmark.
2021-10-09 14:10:42 +08:00
- [MMEditing ](https://github.com/open-mmlab/mmediting ): OpenMMLab image and video editing toolbox.
- [MMGeneration ](https://github.com/open-mmlab/mmgeneration ): OpenMMLab image and video generative models toolbox.
2022-02-28 15:38:31 +08:00
- [MMDeploy ](https://github.com/open-mmlab/mmdeploy ): OpenMMLab model deployment framework.
2022-11-29 13:42:47 +08:00
[ci-backend-ascend]: https://github.com/open-mmlab/mmdeploy/actions/workflows/backend-ascend.yml
[ci-backend-coreml]: https://github.com/open-mmlab/mmdeploy/actions/workflows/backend-coreml.yml
[ci-backend-ncnn]: https://github.com/open-mmlab/mmdeploy/actions/workflows/backend-ncnn.yml
[ci-backend-ort]: https://github.com/open-mmlab/mmdeploy/actions/workflows/backend-ort.yml
[ci-backend-pplnn]: https://github.com/open-mmlab/mmdeploy/actions/workflows/backend-pplnn.yml
[ci-backend-rknn]: https://github.com/open-mmlab/mmdeploy/actions/workflows/backend-rknn.yml
[ci-backend-snpe]: https://github.com/open-mmlab/mmdeploy/actions/workflows/backend-snpe.yml
[ci-backend-torchscript]: https://github.com/open-mmlab/mmdeploy/actions/workflows/backend-torchscript.yml
[ci-build-riscv64-gcc]: https://github.com/open-mmlab/mmdeploy/actions/workflows/linux-riscv64-gcc.yml
[ci-build-rknpu]: https://github.com/open-mmlab/mmdeploy/actions/workflows/linux-rknpu.yml
2022-12-12 21:19:40 +08:00
[ci-build-tvm]: https://github.com/open-mmlab/mmdeploy/actions/workflows/backend-tvm.yml
2022-12-29 16:08:29 +08:00
[pass-backend-ascend]: https://img.shields.io/github/actions/workflow/status/open-mmlab/mmdeploy/backend-ascend.yml?branch=master
[pass-backend-coreml]: https://img.shields.io/github/actions/workflow/status/open-mmlab/mmdeploy/backend-coreml.yml?branch=master
[pass-backend-ncnn]: https://img.shields.io/github/actions/workflow/status/open-mmlab/mmdeploy/backend-ncnn.yml?branch=master
[pass-backend-ort]: https://img.shields.io/github/actions/workflow/status/open-mmlab/mmdeploy/backend-ort.yml?branch=master
[pass-backend-pplnn]: https://img.shields.io/github/actions/workflow/status/open-mmlab/mmdeploy/backend-pplnn.yml?branch=master
[pass-backend-rknn]: https://img.shields.io/github/actions/workflow/status/open-mmlab/mmdeploy/backend-rknn.yml?branch=master
[pass-backend-snpe]: https://img.shields.io/github/actions/workflow/status/open-mmlab/mmdeploy/backend-snpe.yml?branch=master
[pass-backend-torchscript]: https://img.shields.io/github/actions/workflow/status/open-mmlab/mmdeploy/backend-ort.yml?branch=master
[pass-build-riscv64-gcc]: https://img.shields.io/github/actions/workflow/status/open-mmlab/mmdeploy/linux-riscv64-gcc.yml?branch=master
[pass-build-rknpu]: https://img.shields.io/github/actions/workflow/status/open-mmlab/mmdeploy/backend-rknn.yml?branch=master
[pass-build-tvm]: https://img.shields.io/github/actions/workflow/status/open-mmlab/mmdeploy/backend-tvm.yml?branch=master
[pass-no-status]: https://img.shields.io/badge/build-no%20status-lightgrey
