mirror of
https://github.com/open-mmlab/mmdeploy.git
synced 2025-01-14 08:09:43 +08:00
2b98040b26
* [Refactor] add enum class and use functions to get configuration (#40) * add task and codebase enum class * use funcitons to get config * Refactor wrappers of mmcls and mmseg (#41) * move wrappers of cls & det to apis * remove get_classes_from_config * rename onnx_helper to onnx_utils * move import to outside of class * refactor ortwrappers * Refactor build dataset and dataloader for mmseg (#44) * refactor build_dataset and build_dataloader for mmcls and mmseg * remove repeated classes * set build_dataloader with shuffle=False * [Refactor] pplwrapper and mmocr refactor (#46) * add * add pplwrapper and refactor mmocr * fix lint * remove unused arguments * apply dict input for pplwrapper and ortwrapper * add condition before import ppl and ort stuff * update ppl (#51) * Refactor return value and extract_model (#54) * remove ret_value * refactor extract_model * fix typo * resolve comments * [Refactor] Refactor model inference pipeline (#52) * move attribute_to_dict to extract_model * simplify the inference and visualization * remove unused import * [Feature] Support SRCNN in mmedit with ONNXRuntime and TensorRT (#45) * finish mmedit-ort * edit __init__ files * add noqa * add tensorrt support * 1. Rename "base.py" 2. Move srcnn.py to correct directory * fix bugs * remove figures * align to refactor-v1 * update comment in srcnn * fix lint * newfunc -> new_func * Add visualize.py split visualize() in each codebase * fix lint * fix lint * remove unnecessary code in ORTRestorer * remove .api * edit super(), remove dataset * [Refactor]: Change name of split to partition (#57) * refactor mmcls configs * refactor mmdet configs and split params * rename rest split to partition from master * remove base.py * fix init of inference class * fix mmocr init, add show_result alias Co-authored-by: AllentDan <41138331+AllentDan@users.noreply.github.com> Co-authored-by: RunningLeon <maningsheng@sensetime.com> Co-authored-by: Yifan Zhou <singlezombie@163.com>
85 lines
2.7 KiB
Markdown
85 lines
2.7 KiB
Markdown
# How to Deploy ncnn Models in deploy_prototype
|
|
|
|
This tutorial is based on Linux systems like Ubuntu-16.04.
|
|
|
|
Before starting this tutorial, you should make sure that the prerequisites mentioned by `deploy_prototype/README.md` are prepared.
|
|
|
|
## Preparation
|
|
|
|
- Download VulkanTools for the compilation of ncnn.
|
|
```bash
|
|
wget https://sdk.lunarg.com/sdk/download/1.2.176.1/linux/vulkansdk-linux-x86_64-1.2.176.1.tar.gz?Human=true -O vulkansdk-linux-x86_64-1.2.176.1.tar.gz
|
|
tar -xf vulkansdk-linux-x86_64-1.2.176.1.tar.gz
|
|
export VULKAN_SDK=$(pwd)/1.2.176.1/x86_64
|
|
|
|
- Prepare ncnn Framework
|
|
|
|
- Download ncnn source code of tag 20210507
|
|
```bash
|
|
git clone -b 20210507 git@github.com:Tencent/ncnn.git
|
|
```
|
|
- <font color=red>Make install</font> ncnn library
|
|
```bash
|
|
cd ncnn
|
|
mkdir build
|
|
cmake -DNCNN_VULKAN=ON -DNCNN_SYSTEM_GLSLANG=ON -DNCNN_BUILD_EXAMPLES=ON -DNCNN_PYTHON=ON -DNCNN_BUILD_TOOLS=ON -DNCNN_BUILD_BENCHMARK=ON -DNCNN_BUILD_TESTS=ON ..
|
|
make install
|
|
```
|
|
- Install pyncnn module
|
|
```bash
|
|
cd ncnn/python
|
|
pip install .
|
|
```
|
|
|
|
- Build ncnn backend ops of deploy_prototype
|
|
```bash
|
|
cd deploy_prototype
|
|
mkdir build
|
|
cd build
|
|
cmake -DBUILD_NCNN_OPS=ON -DNCNN_DIR=${PATH_TO_NCNN}/ncnn ..
|
|
```
|
|
The `${PATH_TO_NCNN}` refers as the root directory of ncnn source code.
|
|
- Install mmdeploy module
|
|
```bash
|
|
cd deploy_prototype
|
|
python setup.py develop
|
|
```
|
|
Or you will fail on
|
|
```
|
|
No module named mmdeploy
|
|
```
|
|
|
|
## FAQs
|
|
|
|
1. When running ncnn models for inference with custom ops, it fails and shows the error message like:
|
|
|
|
```
|
|
TypeError: register mm custom layers(): incompatible function arguments. The following argument types are supported:
|
|
1.(ar0: ncnn:Net) -> int
|
|
|
|
Invoked with: <ncnn.ncnn.Net object at 0x7f7fc4038bb0>
|
|
```
|
|
This is because of the failure to bind ncnn C++ library to pyncnn. You should build pyncnn from C++ ncnn source code, but not by `pip install`
|
|
|
|
2. When run the tools/deploy.py, it fails:
|
|
```
|
|
Undefined symbol: __cpu_model
|
|
```
|
|
This is a bug of gcc-5, you should update to `gcc >= 6`
|
|
|
|
## Performance Test
|
|
|
|
### MMCls
|
|
This table shows the performance of mmclassification models deployed on ncnn.
|
|
|
|
Dataset: ImageNet `val` dataset.
|
|
|
|
| Model | Top-1(%) | Top-5(%) |
|
|
|-------|----------|----------|
|
|
| MobileNetV2| 71.86 (71.86) | 90.42 (90.42) |
|
|
| ResNet | 69.88 (70.07) | 89.34 (89.44) |
|
|
| ResNeXt | 78.61 (78.71) | 94.17 (94.12) |
|
|
|
|
The data in the parentheses is the inference result from pytorch.
|
|
(According to: [mmcls model_zoo docs](https://github.com/open-mmlab/mmclassification/blob/master/docs/model_zoo.md))
|