polish_serving_docs
HydrogenSulfate
parent
c6525f0bf8
commit
6ed4ae5ec9
|
|
@ -1,280 +1,352 @@
|
|||
# Model Service Deployment
|
||||
|
||||
# Model Service deployment
|
||||
--------
|
||||
## Catalogue
|
||||
|
||||
- [1. Introduction](#1)
|
||||
- [2. Installation of Serving](#2)
|
||||
- [3. Service Deployment for Image Classification](#3)
|
||||
- [3.1 Model Transformation](#3.1)
|
||||
- [3.2 Service Deployment and Request](#3.2)
|
||||
- [4. Service Deployment for Image Recognition](#4)
|
||||
- [3.1 Model Transformation](#3.1)
|
||||
- [3.2 Service Deployment and Request](#3.2)
|
||||
- [3.2.1 Python Serving](#3.2.1)
|
||||
- [3.2.2 C++ Serving](#3.2.2)
|
||||
- [4. Service Deployment for Image Recognition](#4)
|
||||
- [4.1 Model Transformation](#4.1)
|
||||
- [4.2 Service Deployment and Request](#4.2)
|
||||
- [4.2.1 Python Serving](#4.2.1)
|
||||
- [4.2.2 C++ Serving](#4.2.2)
|
||||
- [5. FAQ](#5)
|
||||
|
||||
<a name="1"></a>
|
||||
## 1. Introduction
|
||||
## 1 Introduction
|
||||
[Paddle Serving](https://github.com/PaddlePaddle/Serving) aims to help deep learning developers easily deploy online prediction services, support one-click deployment of industrial-grade service capabilities, high concurrency between client and server Efficient communication and support for developing clients in multiple programming languages.
|
||||
|
||||
[Paddle Serving](https://github.com/PaddlePaddle/Serving) is designed to provide easy deployment of on-line prediction services for deep learning developers, it supports one-click deployment of industrial-grade services, highly concurrent and efficient communication between client and server, and multiple programming languages for client development.
|
||||
|
||||
This section, exemplified by HTTP deployment of prediction service, describes how to deploy model services in PaddleClas with PaddleServing. Currently, only deployment on Linux platform is supported. Windows platform is not supported.
|
||||
This section takes the HTTP prediction service deployment as an example to introduce how to use PaddleServing to deploy the model service in PaddleClas. Currently, only Linux platform deployment is supported, and Windows platform is not currently supported.
|
||||
|
||||
<a name="2"></a>
|
||||
## 2. Installation of Serving
|
||||
## 2. Serving installation
|
||||
The Serving official website recommends using docker to install and deploy the Serving environment. First, you need to pull the docker environment and create a Serving-based docker.
|
||||
|
||||
It is officially recommended to use docker for the installation and environment deployment of Serving. First, pull the docker and create a Serving-based one.
|
||||
|
||||
```
|
||||
```shell
|
||||
# start GPU docker
|
||||
docker pull paddlepaddle/serving:0.7.0-cuda10.2-cudnn7-devel
|
||||
nvidia-docker run -p 9292:9292 --name test -dit paddlepaddle/serving:0.7.0-cuda10.2-cudnn7-devel bash
|
||||
nvidia-docker exec -it test bash
|
||||
|
||||
# start CPU docker
|
||||
docker pull paddlepaddle/serving:0.7.0-devel
|
||||
docker run -p 9292:9292 --name test -dit paddlepaddle/serving:0.7.0-devel bash
|
||||
docker exec -it test bash
|
||||
```
|
||||
|
||||
Once you are in docker, install the Serving-related python packages.
|
||||
After entering docker, you need to install Serving-related python packages.
|
||||
```shell
|
||||
python3.7 -m pip install paddle-serving-client==0.7.0
|
||||
python3.7 -m pip install paddle-serving-app==0.7.0
|
||||
python3.7 -m pip install faiss-cpu==1.7.1post2
|
||||
|
||||
```
|
||||
pip3 install paddle-serving-client==0.7.0
|
||||
pip3 install paddle-serving-server==0.7.0 # CPU
|
||||
pip3 install paddle-serving-app==0.7.0
|
||||
pip3 install paddle-serving-server-gpu==0.7.0.post102 #GPU with CUDA10.2 + TensorRT6
|
||||
# For other GPU environemnt, confirm the environment before choosing which one to execute
|
||||
pip3 install paddle-serving-server-gpu==0.7.0.post101 # GPU with CUDA10.1 + TensorRT6
|
||||
pip3 install paddle-serving-server-gpu==0.7.0.post112 # GPU with CUDA11.2 + TensorRT8
|
||||
#If it is a CPU deployment environment:
|
||||
python3.7 -m pip install paddle-serving-server==0.7.0 #CPU
|
||||
python3.7 -m pip install paddlepaddle==2.2.0 # CPU
|
||||
|
||||
#If it is a GPU deployment environment
|
||||
python3.7 -m pip install paddle-serving-server-gpu==0.7.0.post102 # GPU with CUDA10.2 + TensorRT6
|
||||
python3.7 -m pip install paddlepaddle-gpu==2.2.0 # GPU with CUDA10.2
|
||||
|
||||
#Other GPU environments need to confirm the environment and then choose which one to execute
|
||||
python3.7 -m pip install paddle-serving-server-gpu==0.7.0.post101 # GPU with CUDA10.1 + TensorRT6
|
||||
python3.7 -m pip install paddle-serving-server-gpu==0.7.0.post112 # GPU with CUDA11.2 + TensorRT8
|
||||
```
|
||||
|
||||
- Speed up the installation process by replacing the source with `-i https://pypi.tuna.tsinghua.edu.cn/simple`.
|
||||
- For other environment configuration and installation, please refer to [Install Paddle Serving using docker](https://github.com/PaddlePaddle/Serving/blob/v0.7.0/doc/Install_EN.md)
|
||||
- To deploy CPU services, please install the CPU version of serving-server with the following command.
|
||||
|
||||
```
|
||||
pip install paddle-serving-server
|
||||
```
|
||||
* If the installation speed is too slow, you can change the source through `-i https://pypi.tuna.tsinghua.edu.cn/simple` to speed up the installation process.
|
||||
* For other environment configuration installation, please refer to: [Install Paddle Serving with Docker](https://github.com/PaddlePaddle/Serving/blob/v0.7.0/doc/Install_CN.md)
|
||||
|
||||
<a name="3"></a>
|
||||
## 3. Service Deployment for Image Classification
|
||||
|
||||
## 3. Image Classification Service Deployment
|
||||
|
||||
The following takes the classic ResNet50_vd model as an example to introduce how to deploy the image classification service.
|
||||
|
||||
<a name="3.1"></a>
|
||||
### 3.1 Model Transformation
|
||||
### 3.1 Model conversion
|
||||
When using PaddleServing for service deployment, you need to convert the saved inference model into a Serving model.
|
||||
- Go to the working directory:
|
||||
```shell
|
||||
cd deploy/paddleserving
|
||||
```
|
||||
- Download and unzip the inference model for ResNet50_vd:
|
||||
```shell
|
||||
# Download ResNet50_vd inference model
|
||||
wget -nc https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ResNet50_vd_infer.tar
|
||||
# Decompress the ResNet50_vd inference model
|
||||
tar xf ResNet50_vd_infer.tar
|
||||
```
|
||||
- Use the paddle_serving_client command to convert the downloaded inference model into a model format for easy server deployment:
|
||||
```shell
|
||||
# Convert ResNet50_vd model
|
||||
python3.7 -m paddle_serving_client.convert \
|
||||
--dirname ./ResNet50_vd_infer/ \
|
||||
--model_filename inference.pdmodel \
|
||||
--params_filename inference.pdiparams \
|
||||
--serving_server ./ResNet50_vd_serving/ \
|
||||
--serving_client ./ResNet50_vd_client/
|
||||
```
|
||||
The specific meaning of the parameters in the above command is shown in the following table
|
||||
| parameter | type | default value | description |
|
||||
| ----------------- | ---- | ------------------ | ------------------------------------------------------------ |
|
||||
| `dirname` | str | - | The storage path of the model file to be converted. The program structure file and parameter file are saved in this directory. |
|
||||
| `model_filename` | str | None | The name of the file storing the model Inference Program structure that needs to be converted. If set to None, use `__model__` as the default filename |
|
||||
| `params_filename` | str | None | File name where all parameters of the model to be converted are stored. It needs to be specified if and only if all model parameters are stored in a single binary file. If the model parameters are stored in separate files, set it to None |
|
||||
| `serving_server` | str | `"serving_server"` | The storage path of the converted model files and configuration files. Default is serving_server |
|
||||
| `serving_client` | str | `"serving_client"` | The converted client configuration file storage path. Default is serving_client |
|
||||
|
||||
When adopting PaddleServing for service deployment, the saved inference model needs to be converted to a Serving model. The following part takes the classic ResNet50_vd model as an example to introduce the deployment of image classification service.
|
||||
|
||||
- Enter the working directory:
|
||||
|
||||
```
|
||||
cd deploy/paddleserving
|
||||
```
|
||||
|
||||
- Download the inference model of ResNet50_vd：
|
||||
|
||||
```
|
||||
# Download and decompress the ResNet50_vd model
|
||||
wget https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ResNet50_vd_infer.tar && tar xf ResNet50_vd_infer.tar
|
||||
```
|
||||
|
||||
- Convert the downloaded inference model into a format that is readily deployable by Server with the help of paddle_serving_client.
|
||||
|
||||
```
|
||||
# Convert the ResNet50_vd model
|
||||
python3 -m paddle_serving_client.convert --dirname ./ResNet50_vd_infer/ \
|
||||
--model_filename inference.pdmodel \
|
||||
--params_filename inference.pdiparams \
|
||||
--serving_server ./ResNet50_vd_serving/ \
|
||||
--serving_client ./ResNet50_vd_client/
|
||||
```
|
||||
|
||||
After the transformation, `ResNet50_vd_serving` and `ResNet50_vd_client` will be added to the current folder in the following format:
|
||||
|
||||
```
|
||||
|- ResNet50_vd_server/
|
||||
|- __model__
|
||||
|- __params__
|
||||
|- serving_server_conf.prototxt
|
||||
|- serving_server_conf.stream.prototxt
|
||||
|- ResNet50_vd_client
|
||||
|- serving_client_conf.prototxt
|
||||
|- serving_client_conf.stream.prototxt
|
||||
```
|
||||
|
||||
Having obtained the model file, modify the alias name in `serving_server_conf.prototxt` under directory `ResNet50_vd_server` by changing `alias_name` in `fetch_var` to `prediction`.
|
||||
|
||||
**Notes**: Serving supports input and output renaming to ensure its compatibility with the deployment of different models. In this case, modifying the alias_name of the configuration file is the only step needed to complete the inference and deployment of all kinds of models. The modified serving_server_conf.prototxt is shown below:
|
||||
|
||||
```
|
||||
feed_var {
|
||||
name: "inputs"
|
||||
alias_name: "inputs"
|
||||
is_lod_tensor: false
|
||||
feed_type: 1
|
||||
shape: 3
|
||||
shape: 224
|
||||
shape: 224
|
||||
}
|
||||
fetch_var {
|
||||
name: "save_infer_model/scale_0.tmp_1"
|
||||
alias_name: "prediction"
|
||||
is_lod_tensor: true
|
||||
fetch_type: 1
|
||||
shape: -1
|
||||
}
|
||||
```
|
||||
After the ResNet50_vd inference model is converted, there will be additional `ResNet50_vd_serving` and `ResNet50_vd_client` folders in the current folder, with the following structure:
|
||||
```shell
|
||||
├── ResNet50_vd_serving/
|
||||
│ ├── inference.pdiparams
|
||||
│ ├── inference.pdmodel
|
||||
│ ├── serving_server_conf.prototxt
|
||||
│ └── serving_server_conf.stream.prototxt
|
||||
│
|
||||
└── ResNet50_vd_client/
|
||||
├── serving_client_conf.prototxt
|
||||
└── serving_client_conf.stream.prototxt
|
||||
```
|
||||
|
||||
- Serving provides the function of input and output renaming in order to be compatible with the deployment of different models. When different models are deployed in inference, you only need to modify the `alias_name` of the configuration file, and the inference deployment can be completed without modifying the code. Therefore, after the conversion, you need to modify the alias names in the files `serving_server_conf.prototxt` under `ResNet50_vd_serving` and `ResNet50_vd_client` respectively, and change the `alias_name` in `fetch_var` to `prediction`, the modified serving_server_conf.prototxt is as follows Show:
|
||||
```log
|
||||
feed_var {
|
||||
name: "inputs"
|
||||
alias_name: "inputs"
|
||||
is_lod_tensor: false
|
||||
feed_type: 1
|
||||
shape: 3
|
||||
shape: 224
|
||||
shape: 224
|
||||
}
|
||||
fetch_var {
|
||||
name: "save_infer_model/scale_0.tmp_1"
|
||||
alias_name: "prediction"
|
||||
is_lod_tensor: false
|
||||
fetch_type: 1
|
||||
shape: 1000
|
||||
}
|
||||
```
|
||||
<a name="3.2"></a>
|
||||
### 3.2 Service Deployment and Request
|
||||
|
||||
Paddleserving's directory contains the code to start the pipeline service and send prediction requests, including:
|
||||
|
||||
```
|
||||
### 3.2 Service deployment and request
|
||||
The paddleserving directory contains the code to start the pipeline service, C++ serving service and send prediction requests, including:
|
||||
```shell
|
||||
__init__.py
|
||||
config.yml # Configuration file for starting the service
|
||||
pipeline_http_client.py # Script for sending pipeline prediction requests by http
|
||||
pipeline_rpc_client.py # Script for sending pipeline prediction requests by rpc
|
||||
classification_web_service.py # Script for starting the pipeline server
|
||||
classification_web_service.py # Script to start the pipeline server
|
||||
config.yml # Configuration file to start the pipeline service
|
||||
pipeline_http_client.py # Script for sending pipeline prediction requests in http mode
|
||||
pipeline_rpc_client.py # Script for sending pipeline prediction requests in rpc mode
|
||||
run_cpp_serving.sh# Script to start C++ Serving deployment
|
||||
test_cpp_serving_client.py # Script for sending C++ serving prediction requests in rpc mode
|
||||
```
|
||||
<a name="3.2.1"></a>
|
||||
#### 3.2.1 Python Serving
|
||||
- Start the service:
|
||||
```shell
|
||||
# Start the service and save the running log in log.txt
|
||||
python3.7 classification_web_service.py &>log.txt &
|
||||
```
|
||||
|
||||
- Start the service：
|
||||
- send request:
|
||||
```shell
|
||||
# send service request
|
||||
python3.7 pipeline_http_client.py
|
||||
```
|
||||
After a successful run, the results of the model prediction will be printed in the cmd window, and the results are as follows:
|
||||
```
|
||||
{'err_no': 0, 'err_msg': '', 'key': ['label', 'prob'], 'value': ["['daisy']", '[0.9341402053833008]'], 'tensors ': []}
|
||||
```
|
||||
|
||||
```
|
||||
# Start the service and the run log is saved in log.txt
|
||||
python3 classification_web_service.py &>log.txt &
|
||||
```
|
||||
|
||||
Once the service is successfully started, a log will be printed in log.txt similar to the following 
|
||||
|
||||
- Send request：
|
||||
|
||||
```
|
||||
# Send service request
|
||||
python3 pipeline_http_client.py
|
||||
```
|
||||
|
||||
Once the service is successfully started, the prediction results will be printed in the cmd window, see the following example:
|
||||
<a name="3.2.2"></a>
|
||||
#### 3.2.2 C++ Serving
|
||||
- Start the service:
|
||||
```shell
|
||||
# Start the service, the service runs in the background, and the running log is saved in nohup.txt
|
||||
sh run_cpp_serving.sh
|
||||
```
|
||||
|
||||
- send request:
|
||||
```shell
|
||||
# send service request
|
||||
python3.7 test_cpp_serving_client.py
|
||||
```
|
||||
After a successful run, the results of the model prediction will be printed in the cmd window, and the results are as follows:
|
||||
```
|
||||
prediction: daisy, probability: 0.9341399073600769
|
||||
```
|
||||
|
||||
<a name="4"></a>
|
||||
## 4. Service Deployment for Image Recognition
|
||||
|
||||
When using PaddleServing for service deployment, the saved inference model needs to be converted to a Serving model. The following part, exemplified by the ultra-lightweight model for image recognition in PP-ShiTu, details the deployment of image recognition service.
|
||||
|
||||
|
||||
## 4. Image recognition service deployment
|
||||
In addition to the single-model deployment method introduced in [Chapter 3 Image Classification Service Deployment](#3), we will introduce how to use the detection + classification model to complete the multi-model **image recognition service deployment**
|
||||
When using PaddleServing for image recognition service deployment, **need to convert multiple saved inference models to Serving models**. The following takes the ultra-lightweight image recognition model in PP-ShiTu as an example to introduce the deployment of image recognition services.
|
||||
<a name="4.1"></a>
|
||||
## 4.1 Model Transformation
|
||||
## 4.1 Model conversion
|
||||
- Go to the working directory:
|
||||
```shell
|
||||
cd deploy/
|
||||
```
|
||||
- Download generic detection inference model and generic recognition inference model
|
||||
```shell
|
||||
# Create and enter the models folder
|
||||
mkdir models
|
||||
cd models
|
||||
# Download and unzip the generic recognition model
|
||||
wget https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/inference/general_PPLCNet_x2_5_lite_v1.0_infer.tar
|
||||
tar -xf general_PPLCNet_x2_5_lite_v1.0_infer.tar
|
||||
# Download and unzip the generic detection model
|
||||
wget https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/inference/picodet_PPLCNet_x2_5_mainbody_lite_v1.0_infer.tar
|
||||
tar -xf picodet_PPLCNet_x2_5_mainbody_lite_v1.0_infer.tar
|
||||
```
|
||||
- Convert the generic recognition inference model to the Serving model:
|
||||
```shell
|
||||
# Convert the generic recognition model
|
||||
python3.7 -m paddle_serving_client.convert \
|
||||
--dirname ./general_PPLCNet_x2_5_lite_v1.0_infer/ \
|
||||
--model_filename inference.pdmodel \
|
||||
--params_filename inference.pdiparams \
|
||||
--serving_server ./general_PPLCNet_x2_5_lite_v1.0_serving/ \
|
||||
--serving_client ./general_PPLCNet_x2_5_lite_v1.0_client/
|
||||
```
|
||||
The meaning of the parameters of the above command is the same as [#3.1 Model conversion](#3.1)
|
||||
After the conversion of the general recognition inference model is completed, there will be additional `general_PPLCNet_x2_5_lite_v1.0_serving/` and `general_PPLCNet_x2_5_lite_v1.0_client/` folders in the current folder, with the following structure:
|
||||
```shell
|
||||
├── general_PPLCNet_x2_5_lite_v1.0_serving/
|
||||
│ ├── inference.pdiparams
|
||||
│ ├── inference.pdmodel
|
||||
│ ├── serving_server_conf.prototxt
|
||||
│ └── serving_server_conf.stream.prototxt
|
||||
│
|
||||
└── general_PPLCNet_x2_5_lite_v1.0_client/
|
||||
├── serving_client_conf.prototxt
|
||||
└── serving_client_conf.stream.prototxt
|
||||
```
|
||||
|
||||
- Download inference models for general detection and general recognition
|
||||
- Modify the alias names in `serving_server_conf.prototxt` in `general_PPLCNet_x2_5_lite_v1.0_serving/` and `general_PPLCNet_x2_5_lite_v1.0_client/` directories respectively: change `alias_name` in `fetch_var` to `features`.
|
||||
The modified `serving_server_conf.prototxt` content is as follows:
|
||||
```shell
|
||||
feed_var {
|
||||
name: "x"
|
||||
alias_name: "x"
|
||||
is_lod_tensor: false
|
||||
feed_type: 1
|
||||
shape: 3
|
||||
shape: 224
|
||||
shape: 224
|
||||
}
|
||||
fetch_var {
|
||||
name: "save_infer_model/scale_0.tmp_1"
|
||||
alias_name: "features"
|
||||
is_lod_tensor: false
|
||||
fetch_type: 1
|
||||
shape: 512
|
||||
}
|
||||
```
|
||||
- Convert general detection inference model to Serving model:
|
||||
```shell
|
||||
# Convert generic detection model
|
||||
python3.7 -m paddle_serving_client.convert --dirname ./picodet_PPLCNet_x2_5_mainbody_lite_v1.0_infer/ \
|
||||
--model_filename inference.pdmodel \
|
||||
--params_filename inference.pdiparams \
|
||||
--serving_server ./picodet_PPLCNet_x2_5_mainbody_lite_v1.0_serving/ \
|
||||
--serving_client ./picodet_PPLCNet_x2_5_mainbody_lite_v1.0_client/
|
||||
```
|
||||
The meaning of the parameters of the above command is the same as [#3.1 Model conversion](#3.1)
|
||||
|
||||
```
|
||||
cd deploy
|
||||
# Download and decompress general recogntion models
|
||||
wget -P models/ https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/inference/general_PPLCNet_x2_5_lite_v1.0_infer.tar
|
||||
cd models
|
||||
tar -xf general_PPLCNet_x2_5_lite_v1.0_infer.tar
|
||||
# Download and decompress general detection models
|
||||
wget https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/inference/picodet_PPLCNet_x2_5_mainbody_lite_v1.0_infer.tar
|
||||
tar -xf picodet_PPLCNet_x2_5_mainbody_lite_v1.0_infer.tar
|
||||
```
|
||||
After the general detection inference model conversion is completed, there will be additional folders `picodet_PPLCNet_x2_5_mainbody_lite_v1.0_serving/` and `picodet_PPLCNet_x2_5_mainbody_lite_v1.0_client/` in the current folder, with the following structure:
|
||||
```shell
|
||||
├── picodet_PPLCNet_x2_5_mainbody_lite_v1.0_serving/
|
||||
│ ├── inference.pdiparams
|
||||
│ ├── inference.pdmodel
|
||||
│ ├── serving_server_conf.prototxt
|
||||
│ └── serving_server_conf.stream.prototxt
|
||||
│
|
||||
└── picodet_PPLCNet_x2_5_mainbody_lite_v1.0_client/
|
||||
├── serving_client_conf.prototxt
|
||||
└── serving_client_conf.stream.prototxt
|
||||
```
|
||||
|
||||
- Convert the inference model for recognition into a Serving model:
|
||||
|
||||
```
|
||||
# Convert the recognition model
|
||||
python3 -m paddle_serving_client.convert --dirname ./general_PPLCNet_x2_5_lite_v1.0_infer/ \
|
||||
--model_filename inference.pdmodel \
|
||||
--params_filename inference.pdiparams \
|
||||
--serving_server ./general_PPLCNet_x2_5_lite_v1.0_serving/ \
|
||||
--serving_client ./general_PPLCNet_x2_5_lite_v1.0_client/
|
||||
```
|
||||
|
||||
After the transformation, `general_PPLCNet_x2_5_lite_v1.0_serving/` and `general_PPLCNet_x2_5_lite_v1.0_serving/` will be added to the current folder. Modify the alias name in serving_server_conf.prototxt under the directory `general_PPLCNet_x2_5_lite_v1.0_serving/` by changing `alias_name` to `features` in `fetch_var`. The modified serving_server_conf.prototxt is similar to the following:
|
||||
|
||||
```
|
||||
feed_var {
|
||||
name: "x"
|
||||
alias_name: "x"
|
||||
is_lod_tensor: false
|
||||
feed_type: 1
|
||||
shape: 3
|
||||
shape: 224
|
||||
shape: 224
|
||||
}
|
||||
fetch_var {
|
||||
name: "save_infer_model/scale_0.tmp_1"
|
||||
alias_name: "features"
|
||||
is_lod_tensor: true
|
||||
fetch_type: 1
|
||||
shape: -1
|
||||
}
|
||||
```
|
||||
|
||||
- Convert the inference model for detection into a Serving model:
|
||||
|
||||
```
|
||||
# Convert the general detection model
|
||||
python3 -m paddle_serving_client.convert --dirname ./picodet_PPLCNet_x2_5_mainbody_lite_v1.0_infer/ \
|
||||
--model_filename inference.pdmodel \
|
||||
--params_filename inference.pdiparams \
|
||||
--serving_server ./picodet_PPLCNet_x2_5_mainbody_lite_v1.0_serving/ \
|
||||
--serving_client ./picodet_PPLCNet_x2_5_mainbody_lite_v1.0_client/
|
||||
```
|
||||
|
||||
After the transformation, `picodet_PPLCNet_x2_5_mainbody_lite_v1.0_serving/` and `picodet_PPLCNet_x2_5_ mainbody_lite_v1.0_client/` will be added to the current folder.
|
||||
|
||||
**Note:** The alias name in the serving_server_conf.prototxt under the directory`picodet_PPLCNet_x2_5_mainbody_lite_v1.0_serving/` requires no modification.
|
||||
|
||||
- Download and decompress the constructed search library index
|
||||
|
||||
```
|
||||
cd ../
|
||||
wget https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/data/drink_dataset_v1.0.tar && tar -xf drink_dataset_v1.0.tar
|
||||
```
|
||||
**Note:** There is no need to modify the alias name in serving_server_conf.prototxt under `picodet_PPLCNet_x2_5_mainbody_lite_v1.0_serving/` directory.
|
||||
|
||||
- Download and unzip the built retrieval library index
|
||||
```shell
|
||||
# Go back to the deploy directory
|
||||
cd ../
|
||||
# Download the built retrieval library index
|
||||
wget https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/data/drink_dataset_v1.0.tar
|
||||
# Decompress the built retrieval library index
|
||||
tar -xf drink_dataset_v1.0.tar
|
||||
```
|
||||
<a name="4.2"></a>
|
||||
## 4.2 Service Deployment and Request
|
||||
## 4.2 Service deployment and request
|
||||
**Note:** The identification service involves multiple models, and the PipeLine deployment method is used for performance reasons. The Pipeline deployment method currently does not support the windows platform.
|
||||
- go to the working directory
|
||||
```shell
|
||||
cd ./deploy/paddleserving/recognition
|
||||
```
|
||||
The paddleserving directory contains code to start the Python Pipeline service, the C++ Serving service, and send prediction requests, including:
|
||||
```shell
|
||||
__init__.py
|
||||
config.yml # The configuration file to start the python pipeline service
|
||||
pipeline_http_client.py # Script for sending pipeline prediction requests in http mode
|
||||
pipeline_rpc_client.py # Script for sending pipeline prediction requests in rpc mode
|
||||
recognition_web_service.py # Script to start the pipeline server
|
||||
run_cpp_serving.sh # Script to start C++ Pipeline Serving deployment
|
||||
test_cpp_serving_client.py # Script for sending C++ Pipeline serving prediction requests by rpc
|
||||
```
|
||||
|
||||
**Note:** Since the recognition service involves multiple models, PipeLine is adopted for better performance. This deployment method does not support the windows platform for now.
|
||||
<a name="4.2.1"></a>
|
||||
#### 4.2.1 Python Serving
|
||||
- Start the service:
|
||||
```shell
|
||||
# Start the service, run the logSave in log.txt
|
||||
python3.7 recognition_web_service.py &>log.txt &
|
||||
```
|
||||
|
||||
- Enter the working directory
|
||||
- send request:
|
||||
```shell
|
||||
python3.7 pipeline_http_client.py
|
||||
```
|
||||
After a successful run, the results of the model prediction will be printed in the cmd window, and the results are as follows:
|
||||
```log
|
||||
{'err_no': 0, 'err_msg': '', 'key': ['result'], 'value': ["[{'bbox': [345, 95, 524, 576], 'rec_docs': 'Red Bull-Enhanced', 'rec_scores': 0.79903316}]"], 'tensors': []}
|
||||
```
|
||||
|
||||
```
|
||||
cd ./deploy/paddleserving/recognition
|
||||
```
|
||||
|
||||
Paddleserving's directory contains the code to start the pipeline service and send prediction requests, including:
|
||||
|
||||
```
|
||||
__init__.py
|
||||
config.yml # Configuration file for starting the service
|
||||
pipeline_http_client.py # Script for sending pipeline prediction requests by http
|
||||
pipeline_rpc_client.py # Script for sending pipeline prediction requests by rpc
|
||||
recognition_web_service.py # Script for starting the pipeline server
|
||||
```
|
||||
|
||||
- Start the service：
|
||||
|
||||
```
|
||||
# Start the service and the run log is saved in log.txt
|
||||
python3 recognition_web_service.py &>log.txt &
|
||||
```
|
||||
|
||||
Once the service is successfully started, a log will be printed in log.txt similar to the following 
|
||||
|
||||
- Send request：
|
||||
|
||||
```
|
||||
python3 pipeline_http_client.py
|
||||
```
|
||||
|
||||
Once the service is successfully started, the prediction results will be printed in the cmd window, see the following example: 
|
||||
<a name="4.2.2"></a>
|
||||
#### 4.2.2 C++ Serving
|
||||
- Start the service:
|
||||
```shell
|
||||
# Start the service: Here, the subject detection and feature extraction services will be started at the same time in the background, and the port numbers are 9293 and 9294 respectively;
|
||||
# The running logs are saved in log_mainbody_detection.txt and log_feature_extraction.txt respectively
|
||||
sh run_cpp_serving.sh
|
||||
```
|
||||
|
||||
- send request:
|
||||
```shell
|
||||
# send service request
|
||||
python3.7 test_cpp_serving_client.py
|
||||
```
|
||||
After a successful run, the results of the model predictions are printed in the cmd window, and the results are as follows:
|
||||
```log
|
||||
[{'bbox': [345, 95, 524, 586], 'rec_docs': 'Red Bull-Enhanced', 'rec_scores': 0.8016462}]
|
||||
```
|
||||
|
||||
<a name="5"></a>
|
||||
## 5.FAQ
|
||||
## 5. FAQ
|
||||
|
||||
**Q1**： After sending a request, no result is returned or the output is prompted with a decoding error.
|
||||
**Q1**: No result is returned after the request is sent or an output decoding error is prompted
|
||||
|
||||
**A1**： Please turn off the proxy before starting the service and sending requests, try the following command:
|
||||
|
||||
```
|
||||
**A1**: Do not set the proxy when starting the service and sending the request. You can close the proxy before starting the service and sending the request. The command to close the proxy is:
|
||||
```shell
|
||||
unset https_proxy
|
||||
unset http_proxy
|
||||
```
|
||||
|
||||
For more types of service deployment, such as `RPC prediction services`, you can refer to the [github official website](https://github.com/PaddlePaddle/Serving/tree/v0.7.0/examples) of Serving.
|
||||
**Q2**: nothing happens after starting the service
|
||||
|
||||
**A2**: You can check whether the path corresponding to `model_config` in `config.yml` exists, and whether the folder name is correct
|
||||
|
||||
|
||||
For more service deployment types, such as `RPC prediction service`, you can refer to Serving's [github official website](https://github.com/PaddlePaddle/Serving/tree/v0.9.0/examples)
|
||||
|
|
|
|||
|
|
@ -39,21 +39,21 @@ docker exec -it test bash
|
|||
|
||||
进入 docker 后，需要安装 Serving 相关的 python 包。
|
||||
```shell
|
||||
pip3 install paddle-serving-client==0.7.0
|
||||
pip3 install paddle-serving-app==0.7.0
|
||||
pip3 install faiss-cpu==1.7.1post2
|
||||
python3.7 -m pip install paddle-serving-client==0.7.0
|
||||
python3.7 -m pip install paddle-serving-app==0.7.0
|
||||
python3.7 -m pip install faiss-cpu==1.7.1post2
|
||||
|
||||
#若为CPU部署环境:
|
||||
pip3 install paddle-serving-server==0.7.0 # CPU
|
||||
pip3 install paddlepaddle==2.2.0 # CPU
|
||||
python3.7 -m pip install paddle-serving-server==0.7.0 # CPU
|
||||
python3.7 -m pip install paddlepaddle==2.2.0 # CPU
|
||||
|
||||
#若为GPU部署环境
|
||||
pip3 install paddle-serving-server-gpu==0.7.0.post102 # GPU with CUDA10.2 + TensorRT6
|
||||
pip3 install paddlepaddle-gpu==2.2.0 # GPU with CUDA10.2
|
||||
python3.7 -m pip install paddle-serving-server-gpu==0.7.0.post102 # GPU with CUDA10.2 + TensorRT6
|
||||
python3.7 -m pip install paddlepaddle-gpu==2.2.0 # GPU with CUDA10.2
|
||||
|
||||
#其他GPU环境需要确认环境再选择执行哪一条
|
||||
pip3 install paddle-serving-server-gpu==0.7.0.post101 # GPU with CUDA10.1 + TensorRT6
|
||||
pip3 install paddle-serving-server-gpu==0.7.0.post112 # GPU with CUDA11.2 + TensorRT8
|
||||
python3.7 -m pip install paddle-serving-server-gpu==0.7.0.post101 # GPU with CUDA10.1 + TensorRT6
|
||||
python3.7 -m pip install paddle-serving-server-gpu==0.7.0.post112 # GPU with CUDA11.2 + TensorRT8
|
||||
```
|
||||
|
||||
* 如果安装速度太慢，可以通过 `-i https://pypi.tuna.tsinghua.edu.cn/simple` 更换源，加速安装过程。
|
||||
|
|
@ -62,233 +62,291 @@ pip3 install paddle-serving-server-gpu==0.7.0.post112 # GPU with CUDA11.2 + Tens
|
|||
<a name="3"></a>
|
||||
|
||||
## 3. 图像分类服务部署
|
||||
|
||||
下面以经典的 ResNet50_vd 模型为例，介绍如何部署图像分类服务。
|
||||
|
||||
<a name="3.1"></a>
|
||||
### 3.1 模型转换
|
||||
使用 PaddleServing 做服务化部署时，需要将保存的 inference 模型转换为 Serving 模型。下面以经典的 ResNet50_vd 模型为例，介绍如何部署图像分类服务。
|
||||
使用 PaddleServing 做服务化部署时，需要将保存的 inference 模型转换为 Serving 模型。
|
||||
- 进入工作目录：
|
||||
```shell
|
||||
cd deploy/paddleserving
|
||||
```
|
||||
- 下载 ResNet50_vd 的 inference 模型：
|
||||
```shell
|
||||
# 下载并解压 ResNet50_vd 模型
|
||||
wget https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ResNet50_vd_infer.tar && tar xf ResNet50_vd_infer.tar
|
||||
```
|
||||
- 用 paddle_serving_client 把下载的 inference 模型转换成易于 Server 部署的模型格式：
|
||||
```
|
||||
# 转换 ResNet50_vd 模型
|
||||
python3 -m paddle_serving_client.convert --dirname ./ResNet50_vd_infer/ \
|
||||
--model_filename inference.pdmodel \
|
||||
--params_filename inference.pdiparams \
|
||||
--serving_server ./ResNet50_vd_serving/ \
|
||||
--serving_client ./ResNet50_vd_client/
|
||||
```
|
||||
ResNet50_vd 推理模型转换完成后，会在当前文件夹多出 `ResNet50_vd_serving` 和 `ResNet50_vd_client` 的文件夹，具备如下格式：
|
||||
```
|
||||
|- ResNet50_vd_serving/
|
||||
|- inference.pdiparams
|
||||
|- inference.pdmodel
|
||||
|- serving_server_conf.prototxt
|
||||
|- serving_server_conf.stream.prototxt
|
||||
|- ResNet50_vd_client
|
||||
|- serving_client_conf.prototxt
|
||||
|- serving_client_conf.stream.prototxt
|
||||
```
|
||||
得到模型文件之后，需要分别修改 `ResNet50_vd_serving` 和 `ResNet50_vd_client` 下文件 `serving_server_conf.prototxt` 中的 alias 名字：将 `fetch_var` 中的 `alias_name` 改为 `prediction`
|
||||
```shell
|
||||
cd deploy/paddleserving
|
||||
```
|
||||
- 下载并解压 ResNet50_vd 的 inference 模型：
|
||||
```shell
|
||||
# 下载 ResNet50_vd inference 模型
|
||||
wget -nc https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ResNet50_vd_infer.tar
|
||||
# 解压 ResNet50_vd inference 模型
|
||||
tar xf ResNet50_vd_infer.tar
|
||||
```
|
||||
- 用 paddle_serving_client 命令把下载的 inference 模型转换成易于 Server 部署的模型格式：
|
||||
```shell
|
||||
# 转换 ResNet50_vd 模型
|
||||
python3.7 -m paddle_serving_client.convert \
|
||||
--dirname ./ResNet50_vd_infer/ \
|
||||
--model_filename inference.pdmodel \
|
||||
--params_filename inference.pdiparams \
|
||||
--serving_server ./ResNet50_vd_serving/ \
|
||||
--serving_client ./ResNet50_vd_client/
|
||||
```
|
||||
上述命令中参数具体含义如下表所示
|
||||
| 参数 | 类型 | 默认值 | 描述 |
|
||||
| ----------------- | ---- | ------------------ | ------------------------------------------------------------ |
|
||||
| `dirname` | str | - | 需要转换的模型文件存储路径，Program结构文件和参数文件均保存在此目录。 |
|
||||
| `model_filename` | str | None | 存储需要转换的模型Inference Program结构的文件名称。如果设置为None，则使用 `__model__` 作为默认的文件名 |
|
||||
| `params_filename` | str | None | 存储需要转换的模型所有参数的文件名称。当且仅当所有模型参数被保>存在一个单独的二进制文件中，它才需要被指定。如果模型参数是存储在各自分离的文件中，设置它的值为None |
|
||||
| `serving_server` | str | `"serving_server"` | 转换后的模型文件和配置文件的存储路径。默认值为serving_server |
|
||||
| `serving_client` | str | `"serving_client"` | 转换后的客户端配置文件存储路径。默认值为serving_client |
|
||||
|
||||
**备注**: Serving 为了兼容不同模型的部署，提供了输入输出重命名的功能。这样，不同的模型在推理部署时，只需要修改配置文件的 alias_name 即可，无需修改代码即可完成推理部署。
|
||||
修改后的 serving_server_conf.prototxt 如下所示:
|
||||
```
|
||||
feed_var {
|
||||
name: "inputs"
|
||||
alias_name: "inputs"
|
||||
is_lod_tensor: false
|
||||
feed_type: 1
|
||||
shape: 3
|
||||
shape: 224
|
||||
shape: 224
|
||||
}
|
||||
fetch_var {
|
||||
name: "save_infer_model/scale_0.tmp_1"
|
||||
alias_name: "prediction"
|
||||
is_lod_tensor: false
|
||||
fetch_type: 1
|
||||
shape: 1000
|
||||
}
|
||||
```
|
||||
ResNet50_vd 推理模型转换完成后，会在当前文件夹多出 `ResNet50_vd_serving` 和 `ResNet50_vd_client` 的文件夹，具备如下结构：
|
||||
```shell
|
||||
├── ResNet50_vd_serving/
|
||||
│ ├── inference.pdiparams
|
||||
│ ├── inference.pdmodel
|
||||
│ ├── serving_server_conf.prototxt
|
||||
│ └── serving_server_conf.stream.prototxt
|
||||
│
|
||||
└── ResNet50_vd_client/
|
||||
├── serving_client_conf.prototxt
|
||||
└── serving_client_conf.stream.prototxt
|
||||
```
|
||||
|
||||
- Serving 为了兼容不同模型的部署，提供了输入输出重命名的功能。让不同的模型在推理部署时，只需要修改配置文件的 `alias_name` 即可，无需修改代码即可完成推理部署。因此在转换完毕后需要分别修改 `ResNet50_vd_serving` 和 `ResNet50_vd_client` 下文件 `serving_server_conf.prototxt` 中的 alias 名字，将 `fetch_var` 中的 `alias_name` 改为 `prediction`，修改后的 serving_server_conf.prototxt 如下所示:
|
||||
```log
|
||||
feed_var {
|
||||
name: "inputs"
|
||||
alias_name: "inputs"
|
||||
is_lod_tensor: false
|
||||
feed_type: 1
|
||||
shape: 3
|
||||
shape: 224
|
||||
shape: 224
|
||||
}
|
||||
fetch_var {
|
||||
name: "save_infer_model/scale_0.tmp_1"
|
||||
alias_name: "prediction"
|
||||
is_lod_tensor: false
|
||||
fetch_type: 1
|
||||
shape: 1000
|
||||
}
|
||||
```
|
||||
<a name="3.2"></a>
|
||||
### 3.2 服务部署和请求
|
||||
paddleserving 目录包含了启动 pipeline 服务、C++ serving服务和发送预测请求的代码，包括：
|
||||
```shell
|
||||
__init__.py
|
||||
config.yml # 启动pipeline服务的配置文件
|
||||
pipeline_http_client.py # http方式发送pipeline预测请求的脚本
|
||||
pipeline_rpc_client.py # rpc方式发送pipeline预测请求的脚本
|
||||
classification_web_service.py # 启动pipeline服务端的脚本
|
||||
run_cpp_serving.sh # 启动C++ Serving部署的脚本
|
||||
test_cpp_serving_client.py # rpc方式发送C++ serving预测请求的脚本
|
||||
classification_web_service.py # 启动pipeline服务端的脚本
|
||||
config.yml # 启动pipeline服务的配置文件
|
||||
pipeline_http_client.py # http方式发送pipeline预测请求的脚本
|
||||
pipeline_rpc_client.py # rpc方式发送pipeline预测请求的脚本
|
||||
run_cpp_serving.sh # 启动C++ Serving部署的脚本
|
||||
test_cpp_serving_client.py # rpc方式发送C++ serving预测请求的脚本
|
||||
```
|
||||
<a name="3.2.1"></a>
|
||||
#### 3.2.1 Python Serving
|
||||
- 启动服务：
|
||||
```shell
|
||||
# 启动服务，运行日志保存在 log.txt
|
||||
python3 classification_web_service.py &>log.txt &
|
||||
```
|
||||
```shell
|
||||
# 启动服务，运行日志保存在 log.txt
|
||||
python3.7 classification_web_service.py &>log.txt &
|
||||
```
|
||||
|
||||
- 发送请求：
|
||||
```shell
|
||||
# 发送服务请求
|
||||
python3 pipeline_http_client.py
|
||||
```
|
||||
成功运行后，模型预测的结果会打印在 cmd 窗口中，结果如下：
|
||||
```
|
||||
{'err_no': 0, 'err_msg': '', 'key': ['label', 'prob'], 'value': ["['daisy']", '[0.9341402053833008]'], 'tensors': []}
|
||||
```
|
||||
```shell
|
||||
# 发送服务请求
|
||||
python3.7 pipeline_http_client.py
|
||||
```
|
||||
成功运行后，模型预测的结果会打印在 cmd 窗口中，结果如下：
|
||||
```
|
||||
{'err_no': 0, 'err_msg': '', 'key': ['label', 'prob'], 'value': ["['daisy']", '[0.9341402053833008]'], 'tensors': []}
|
||||
```
|
||||
|
||||
<a name="3.2.2"></a>
|
||||
#### 3.2.2 C++ Serving
|
||||
- 启动服务：
|
||||
```shell
|
||||
# 启动服务， 服务在后台运行，运行日志保存在 nohup.txt
|
||||
sh run_cpp_serving.sh
|
||||
```
|
||||
```shell
|
||||
# 启动服务， 服务在后台运行，运行日志保存在 nohup.txt
|
||||
sh run_cpp_serving.sh
|
||||
```
|
||||
|
||||
- 发送请求：
|
||||
```shell
|
||||
# 发送服务请求
|
||||
python3 test_cpp_serving_client.py
|
||||
```
|
||||
成功运行后，模型预测的结果会打印在 cmd 窗口中，结果如下：
|
||||
```
|
||||
prediction: daisy, probability: 0.9341399073600769
|
||||
```
|
||||
```shell
|
||||
# 发送服务请求
|
||||
python3.7 test_cpp_serving_client.py
|
||||
```
|
||||
成功运行后，模型预测的结果会打印在 cmd 窗口中，结果如下：
|
||||
```
|
||||
prediction: daisy, probability: 0.9341399073600769
|
||||
```
|
||||
|
||||
<a name="4"></a>
|
||||
## 4.图像识别服务部署
|
||||
使用 PaddleServing 做服务化部署时，需要将保存的 inference 模型转换为 Serving 模型。 下面以 PP-ShiTu 中的超轻量图像识别模型为例，介绍图像识别服务的部署。
|
||||
除了[第三章图像分类服务部署](#3)介绍的单模型部署方式，接下来会介绍如何使用检测+分类模型来完成多模型串联的**图像识别服务部署**
|
||||
使用 PaddleServing 做图像识别服务化部署时，**需要将保存的多个 inference 模型都转换为 Serving 模型**。 下面以 PP-ShiTu 中的超轻量图像识别模型为例，介绍图像识别服务的部署。
|
||||
<a name="4.1"></a>
|
||||
## 4.1 模型转换
|
||||
- 进入工作目录：
|
||||
```shell
|
||||
cd deploy/
|
||||
```
|
||||
- 下载通用检测 inference 模型和通用识别 inference 模型
|
||||
```
|
||||
cd deploy
|
||||
# 下载并解压通用识别模型
|
||||
wget -P models/ https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/inference/general_PPLCNet_x2_5_lite_v1.0_infer.tar
|
||||
cd models
|
||||
tar -xf general_PPLCNet_x2_5_lite_v1.0_infer.tar
|
||||
# 下载并解压通用检测模型
|
||||
wget https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/inference/picodet_PPLCNet_x2_5_mainbody_lite_v1.0_infer.tar
|
||||
tar -xf picodet_PPLCNet_x2_5_mainbody_lite_v1.0_infer.tar
|
||||
```
|
||||
- 转换识别 inference 模型为 Serving 模型：
|
||||
```
|
||||
# 转换识别模型
|
||||
python3 -m paddle_serving_client.convert --dirname ./general_PPLCNet_x2_5_lite_v1.0_infer/ \
|
||||
--model_filename inference.pdmodel \
|
||||
--params_filename inference.pdiparams \
|
||||
--serving_server ./general_PPLCNet_x2_5_lite_v1.0_serving/ \
|
||||
--serving_client ./general_PPLCNet_x2_5_lite_v1.0_client/
|
||||
```
|
||||
识别推理模型转换完成后，会在当前文件夹多出 `general_PPLCNet_x2_5_lite_v1.0_serving/` 和 `general_PPLCNet_x2_5_lite_v1.0_client/` 的文件夹。分别修改 `general_PPLCNet_x2_5_lite_v1.0_serving/` 和 `general_PPLCNet_x2_5_lite_v1.0_client/` 目录下的 serving_server_conf.prototxt 中的 alias 名字： 将 `fetch_var` 中的 `alias_name` 改为 `features`。
|
||||
修改后的 serving_server_conf.prototxt 内容如下：
|
||||
```
|
||||
feed_var {
|
||||
name: "x"
|
||||
alias_name: "x"
|
||||
is_lod_tensor: false
|
||||
feed_type: 1
|
||||
shape: 3
|
||||
shape: 224
|
||||
shape: 224
|
||||
}
|
||||
fetch_var {
|
||||
name: "save_infer_model/scale_0.tmp_1"
|
||||
alias_name: "features"
|
||||
is_lod_tensor: false
|
||||
fetch_type: 1
|
||||
shape: 512
|
||||
}
|
||||
```
|
||||
- 转换通用检测 inference 模型为 Serving 模型：
|
||||
```
|
||||
# 转换通用检测模型
|
||||
python3 -m paddle_serving_client.convert --dirname ./picodet_PPLCNet_x2_5_mainbody_lite_v1.0_infer/ \
|
||||
--model_filename inference.pdmodel \
|
||||
--params_filename inference.pdiparams \
|
||||
--serving_server ./picodet_PPLCNet_x2_5_mainbody_lite_v1.0_serving/ \
|
||||
--serving_client ./picodet_PPLCNet_x2_5_mainbody_lite_v1.0_client/
|
||||
```
|
||||
检测 inference 模型转换完成后，会在当前文件夹多出 `picodet_PPLCNet_x2_5_mainbody_lite_v1.0_serving/` 和 `picodet_PPLCNet_x2_5_mainbody_lite_v1.0_client/` 的文件夹。
|
||||
```shell
|
||||
# 创建并进入models文件夹
|
||||
mkdir models
|
||||
cd models
|
||||
# 下载并解压通用识别模型
|
||||
wget https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/inference/general_PPLCNet_x2_5_lite_v1.0_infer.tar
|
||||
tar -xf general_PPLCNet_x2_5_lite_v1.0_infer.tar
|
||||
# 下载并解压通用检测模型
|
||||
wget https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/inference/picodet_PPLCNet_x2_5_mainbody_lite_v1.0_infer.tar
|
||||
tar -xf picodet_PPLCNet_x2_5_mainbody_lite_v1.0_infer.tar
|
||||
```
|
||||
- 转换通用识别 inference 模型为 Serving 模型：
|
||||
```shell
|
||||
# 转换通用识别模型
|
||||
python3.7 -m paddle_serving_client.convert \
|
||||
--dirname ./general_PPLCNet_x2_5_lite_v1.0_infer/ \
|
||||
--model_filename inference.pdmodel \
|
||||
--params_filename inference.pdiparams \
|
||||
--serving_server ./general_PPLCNet_x2_5_lite_v1.0_serving/ \
|
||||
--serving_client ./general_PPLCNet_x2_5_lite_v1.0_client/
|
||||
```
|
||||
上述命令的参数含义与[#3.1 模型转换](#3.1)相同
|
||||
通用识别 inference 模型转换完成后，会在当前文件夹多出 `general_PPLCNet_x2_5_lite_v1.0_serving/` 和 `general_PPLCNet_x2_5_lite_v1.0_client/` 的文件夹，具备如下结构：
|
||||
```shell
|
||||
├── general_PPLCNet_x2_5_lite_v1.0_serving/
|
||||
│ ├── inference.pdiparams
|
||||
│ ├── inference.pdmodel
|
||||
│ ├── serving_server_conf.prototxt
|
||||
│ └── serving_server_conf.stream.prototxt
|
||||
│
|
||||
└── general_PPLCNet_x2_5_lite_v1.0_client/
|
||||
├── serving_client_conf.prototxt
|
||||
└── serving_client_conf.stream.prototxt
|
||||
```
|
||||
|
||||
**注意:** 此处不需要修改 `picodet_PPLCNet_x2_5_mainbody_lite_v1.0_serving/` 目录下的 serving_server_conf.prototxt 中的 alias 名字。
|
||||
- 分别修改 `general_PPLCNet_x2_5_lite_v1.0_serving/` 和 `general_PPLCNet_x2_5_lite_v1.0_client/` 目录下的 `serving_server_conf.prototxt` 中的 alias 名字：将 `fetch_var` 中的 `alias_name` 改为 `features`。
|
||||
修改后的 `serving_server_conf.prototxt` 内容如下：
|
||||
```shell
|
||||
feed_var {
|
||||
name: "x"
|
||||
alias_name: "x"
|
||||
is_lod_tensor: false
|
||||
feed_type: 1
|
||||
shape: 3
|
||||
shape: 224
|
||||
shape: 224
|
||||
}
|
||||
fetch_var {
|
||||
name: "save_infer_model/scale_0.tmp_1"
|
||||
alias_name: "features"
|
||||
is_lod_tensor: false
|
||||
fetch_type: 1
|
||||
shape: 512
|
||||
}
|
||||
```
|
||||
- 转换通用检测 inference 模型为 Serving 模型：
|
||||
```shell
|
||||
# 转换通用检测模型
|
||||
python3.7 -m paddle_serving_client.convert --dirname ./picodet_PPLCNet_x2_5_mainbody_lite_v1.0_infer/ \
|
||||
--model_filename inference.pdmodel \
|
||||
--params_filename inference.pdiparams \
|
||||
--serving_server ./picodet_PPLCNet_x2_5_mainbody_lite_v1.0_serving/ \
|
||||
--serving_client ./picodet_PPLCNet_x2_5_mainbody_lite_v1.0_client/
|
||||
```
|
||||
上述命令的参数含义与[#3.1 模型转换](#3.1)相同
|
||||
|
||||
通用检测 inference 模型转换完成后，会在当前文件夹多出 `picodet_PPLCNet_x2_5_mainbody_lite_v1.0_serving/` 和 `picodet_PPLCNet_x2_5_mainbody_lite_v1.0_client/` 的文件夹，具备如下结构：
|
||||
```shell
|
||||
├── picodet_PPLCNet_x2_5_mainbody_lite_v1.0_serving/
|
||||
│ ├── inference.pdiparams
|
||||
│ ├── inference.pdmodel
|
||||
│ ├── serving_server_conf.prototxt
|
||||
│ └── serving_server_conf.stream.prototxt
|
||||
│
|
||||
└── picodet_PPLCNet_x2_5_mainbody_lite_v1.0_client/
|
||||
├── serving_client_conf.prototxt
|
||||
└── serving_client_conf.stream.prototxt
|
||||
```
|
||||
|
||||
**注意:** 此处不需要修改 `picodet_PPLCNet_x2_5_mainbody_lite_v1.0_serving/` 目录下的 serving_server_conf.prototxt 中的 alias 名字。
|
||||
|
||||
- 下载并解压已经构建后的检索库 index
|
||||
```
|
||||
cd ../
|
||||
wget https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/data/drink_dataset_v1.0.tar && tar -xf drink_dataset_v1.0.tar
|
||||
```
|
||||
```shell
|
||||
# 回到deploy目录
|
||||
cd ../
|
||||
# 下载构建后的检索库 index
|
||||
wget https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/data/drink_dataset_v1.0.tar
|
||||
# 解压构建后的检索库 index
|
||||
tar -xf drink_dataset_v1.0.tar
|
||||
```
|
||||
<a name="4.2"></a>
|
||||
## 4.2 服务部署和请求
|
||||
**注意:** 识别服务涉及到多个模型，出于性能考虑采用 PipeLine 部署方式。Pipeline 部署方式当前不支持 windows 平台。
|
||||
- 进入到工作目录
|
||||
```shell
|
||||
cd ./deploy/paddleserving/recognition
|
||||
```
|
||||
paddleserving 目录包含启动 Python Pipeline 服务、C++ Serving 服务和发送预测请求的代码，包括：
|
||||
```
|
||||
__init__.py
|
||||
config.yml # 启动python pipeline服务的配置文件
|
||||
pipeline_http_client.py # http方式发送pipeline预测请求的脚本
|
||||
pipeline_rpc_client.py # rpc方式发送pipeline预测请求的脚本
|
||||
recognition_web_service.py # 启动pipeline服务端的脚本
|
||||
run_cpp_serving.sh # 启动C++ Pipeline Serving部署的脚本
|
||||
test_cpp_serving_client.py # rpc方式发送C++ Pipeline serving预测请求的脚本
|
||||
```
|
||||
```shell
|
||||
cd ./deploy/paddleserving/recognition
|
||||
```
|
||||
paddleserving 目录包含启动 Python Pipeline 服务、C++ Serving 服务和发送预测请求的代码，包括：
|
||||
```shell
|
||||
__init__.py
|
||||
config.yml # 启动python pipeline服务的配置文件
|
||||
pipeline_http_client.py # http方式发送pipeline预测请求的脚本
|
||||
pipeline_rpc_client.py # rpc方式发送pipeline预测请求的脚本
|
||||
recognition_web_service.py # 启动pipeline服务端的脚本
|
||||
run_cpp_serving.sh # 启动C++ Pipeline Serving部署的脚本
|
||||
test_cpp_serving_client.py # rpc方式发送C++ Pipeline serving预测请求的脚本
|
||||
```
|
||||
|
||||
<a name="4.2.1"></a>
|
||||
#### 4.2.1 Python Serving
|
||||
- 启动服务：
|
||||
```
|
||||
# 启动服务，运行日志保存在 log.txt
|
||||
python3 recognition_web_service.py &>log.txt &
|
||||
```
|
||||
```shell
|
||||
# 启动服务，运行日志保存在 log.txt
|
||||
python3.7 recognition_web_service.py &>log.txt &
|
||||
```
|
||||
|
||||
- 发送请求：
|
||||
```
|
||||
python3 pipeline_http_client.py
|
||||
```
|
||||
成功运行后，模型预测的结果会打印在 cmd 窗口中，结果如下：
|
||||
```
|
||||
{'err_no': 0, 'err_msg': '', 'key': ['result'], 'value': ["[{'bbox': [345, 95, 524, 576], 'rec_docs': '红牛-强化型', 'rec_scores': 0.79903316}]"], 'tensors': []}
|
||||
```
|
||||
```shell
|
||||
python3.7 pipeline_http_client.py
|
||||
```
|
||||
成功运行后，模型预测的结果会打印在 cmd 窗口中，结果如下：
|
||||
```log
|
||||
{'err_no': 0, 'err_msg': '', 'key': ['result'], 'value': ["[{'bbox': [345, 95, 524, 576], 'rec_docs': '红牛-强化型', 'rec_scores': 0.79903316}]"], 'tensors': []}
|
||||
```
|
||||
|
||||
<a name="4.2.2"></a>
|
||||
#### 4.2.2 C++ Serving
|
||||
- 启动服务：
|
||||
```shell
|
||||
# 启动服务： 此处会在后台同时启动主体检测和特征提取服务，端口号分别为9293和9294；
|
||||
# 运行日志分别保存在 log_mainbody_detection.txt 和 log_feature_extraction.txt中
|
||||
sh run_cpp_serving.sh
|
||||
```
|
||||
```shell
|
||||
# 启动服务： 此处会在后台同时启动主体检测和特征提取服务，端口号分别为9293和9294；
|
||||
# 运行日志分别保存在 log_mainbody_detection.txt 和 log_feature_extraction.txt中
|
||||
sh run_cpp_serving.sh
|
||||
```
|
||||
|
||||
- 发送请求：
|
||||
```shell
|
||||
# 发送服务请求
|
||||
python3 test_cpp_serving_client.py
|
||||
```
|
||||
成功运行后，模型预测的结果会打印在 cmd 窗口中，结果如下所示：
|
||||
```
|
||||
[{'bbox': [345, 95, 524, 586], 'rec_docs': '红牛-强化型', 'rec_scores': 0.8016462}]
|
||||
```
|
||||
```shell
|
||||
# 发送服务请求
|
||||
python3.7 test_cpp_serving_client.py
|
||||
```
|
||||
成功运行后，模型预测的结果会打印在 cmd 窗口中，结果如下所示：
|
||||
```log
|
||||
[{'bbox': [345, 95, 524, 586], 'rec_docs': '红牛-强化型', 'rec_scores': 0.8016462}]
|
||||
```
|
||||
|
||||
<a name="5"></a>
|
||||
## 5.FAQ
|
||||
|
||||
**Q1**： 发送请求后没有结果返回或者提示输出解码报错
|
||||
|
||||
**A1**： 启动服务和发送请求时不要设置代理，可以在启动服务前和发送请求前关闭代理，关闭代理的命令是：
|
||||
```
|
||||
```shell
|
||||
unset https_proxy
|
||||
unset http_proxy
|
||||
```
|
||||
|
||||
更多的服务部署类型，如 `RPC 预测服务` 等，可以参考 Serving 的[github 官网](https://github.com/PaddlePaddle/Serving/tree/v0.7.0/examples)
|
||||
**Q2**： 启动服务后没有任何反应
|
||||
|
||||
**A2**： 可以检查`config.yml`中`model_config`对应的路径是否存在，文件夹命名是否正确
|
||||
|
||||
|
||||
更多的服务部署类型，如 `RPC 预测服务` 等，可以参考 Serving 的[github 官网](https://github.com/PaddlePaddle/Serving/tree/v0.9.0/examples)
|
||||
|
|
|
|||
Loading…
Reference in New Issue