Merge pull request #5913 from tink2123/serving_bbox
[Serving] update readme and add bbox in resultpull/5967/head
MissPenguin
GitHub
commit
ea8ae54463
|
|
@ -31,8 +31,6 @@ PaddleOCR operating environment and Paddle Serving operating environment are nee
|
|||
|
||||
1. Please prepare PaddleOCR operating environment reference [link](../../doc/doc_ch/installation.md).
|
||||
|
||||
Download the corresponding paddle whl package according to the environment, it is recommended to install version 2.2.2
|
||||
|
||||
2. The steps of PaddleServing operating environment prepare are as follows:
|
||||
|
||||
|
||||
|
|
@ -191,6 +189,15 @@ The recognition model is the same.
|
|||
```
|
||||
## C++ Serving
|
||||
|
||||
Service deployment based on python obviously has the advantage of convenient secondary development. However, the real application often needs to pursue better performance. PaddleServing also provides a more performant C++ deployment version.
|
||||
|
||||
The C++ service deployment is the same as python in the environment setup and data preparation stages, the difference is when the service is started and the client sends requests.
|
||||
|
||||
| Language | Speed | Secondary development | Do you need to compile |
|
||||
|-----|-----|---------|------------|
|
||||
| C++ | fast | Slightly difficult | Single model prediction does not need to be compiled, multi-model concatenation needs to be compiled |
|
||||
| python | general | easy | single-model/multi-model no compilation required |
|
||||
|
||||
1. Compile Serving
|
||||
|
||||
To improve predictive performance, C++ services also provide multiple model concatenation services. Unlike Python Pipeline services, multiple model concatenation requires the pre - and post-model processing code to be written on the server side, so local recompilation is required to generate serving. Specific may refer to the official document: [how to compile Serving](https://github.com/PaddlePaddle/Serving/blob/v0.8.3/doc/Compile_EN.md)
|
||||
|
|
@ -198,12 +205,28 @@ The recognition model is the same.
|
|||
2. Run the following command to start the service.
|
||||
```
|
||||
# Start the service and save the running log in log.txt
|
||||
python3 -m paddle_serving_server.serve --model ppocrv2_det_serving ppocrv2_rec_serving --op GeneralDetectionOp GeneralRecOp --port 9293 &>log.txt &
|
||||
python3 -m paddle_serving_server.serve --model ppocrv2_det_serving ppocrv2_rec_serving --op GeneralDetectionOp GeneralInferOp --port 9293 &>log.txt &
|
||||
```
|
||||
After the service is successfully started, a log similar to the following will be printed in log.txt
|
||||
|
||||
|
||||
3. Send service request
|
||||
|
||||
Due to the need for pre and post-processing in the C++Server part, in order to speed up the input to the C++Server is only the base64 encoded string of the picture, it needs to be manually modified
|
||||
Change the feed_type field and shape field in ppocrv2_det_client/serving_client_conf.prototxt to the following:
|
||||
|
||||
```
|
||||
feed_var {
|
||||
name: "x"
|
||||
alias_name: "x"
|
||||
is_lod_tensor: false
|
||||
feed_type: 20
|
||||
shape: 1
|
||||
}
|
||||
```
|
||||
|
||||
start the client:
|
||||
|
||||
```
|
||||
python3 ocr_cpp_client.py ppocrv2_det_client ppocrv2_rec_client
|
||||
```
|
||||
|
|
|
|||
|
|
@ -6,6 +6,7 @@ PaddleOCR提供2种服务部署方式：
|
|||
- 基于PaddleHub Serving的部署：代码路径为"`./deploy/hubserving`"，使用方法参考[文档](../../deploy/hubserving/readme.md)；
|
||||
- 基于PaddleServing的部署：代码路径为"`./deploy/pdserving`"，按照本教程使用。
|
||||
|
||||
|
||||
# 基于PaddleServing的服务部署
|
||||
|
||||
本文档将介绍如何使用[PaddleServing](https://github.com/PaddlePaddle/Serving/blob/develop/README_CN.md)工具部署PP-OCR动态图模型的pipeline在线服务。
|
||||
|
|
@ -17,6 +18,8 @@ PaddleOCR提供2种服务部署方式：
|
|||
|
||||
更多有关PaddleServing服务化部署框架介绍和使用教程参考[文档](https://github.com/PaddlePaddle/Serving/blob/develop/README_CN.md)。
|
||||
|
||||
AIStudio演示案例可参考 [基于PaddleServing的OCR服务化部署实战](https://aistudio.baidu.com/aistudio/projectdetail/3630726)。
|
||||
|
||||
## 目录
|
||||
- [环境准备](#环境准备)
|
||||
- [模型转换](#模型转换)
|
||||
|
|
@ -32,8 +35,6 @@ PaddleOCR提供2种服务部署方式：
|
|||
|
||||
- 准备PaddleOCR的运行环境[链接](../../doc/doc_ch/installation.md)
|
||||
|
||||
根据环境下载对应的paddlepaddle whl包，推荐安装2.2.2版本
|
||||
|
||||
- 准备PaddleServing的运行环境，步骤如下
|
||||
|
||||
```bash
|
||||
|
|
@ -135,7 +136,7 @@ python3 -m paddle_serving_client.convert --dirname ./ch_PP-OCRv2_rec_infer/ \
|
|||
python3 pipeline_http_client.py
|
||||
```
|
||||
成功运行后，模型预测的结果会打印在cmd窗口中，结果示例为：
|
||||
|
||||
|
||||
|
||||
调整 config.yml 中的并发个数获得最大的QPS, 一般检测和识别的并发数为2：1
|
||||
```
|
||||
|
|
@ -197,9 +198,24 @@ python3 -m paddle_serving_client.convert --dirname ./ch_PP-OCRv2_rec_infer/ \
|
|||
|
||||
C++ 服务部署在环境搭建和数据准备阶段与 python 相同，区别在于启动服务和客户端发送请求时不同。
|
||||
|
||||
| 语言 | 速度 | 二次开发 | 是否需要编译 |
|
||||
|-----|-----|---------|------------|
|
||||
| C++ | 很快 | 略有难度 | 单模型预测无需编译，多模型串联需要编译 |
|
||||
| python | 一般 ｜ 容易 | 单模型/多模型 均无需编译|
|
||||
|
||||
1. 准备 Serving 环境
|
||||
|
||||
为了提高预测性能，C++ 服务同样提供了多模型串联服务。与python pipeline服务不同，多模型串联的过程中需要将模型前后处理代码写在服务端，因此需要在本地重新编译生成serving。具体可参考官方文档：[如何编译Serving](https://github.com/PaddlePaddle/Serving/blob/v0.8.3/doc/Compile_CN.md)
|
||||
为了提高预测性能，C++ 服务同样提供了多模型串联服务。与python pipeline服务不同，多模型串联的过程中需要将模型前后处理代码写在服务端，因此需要在本地重新编译生成serving。
|
||||
|
||||
首先需要下载Serving代码库, 把OCR文本检测预处理相关代码替换到Serving库中
|
||||
```
|
||||
git clone https://github.com/PaddlePaddle/Serving
|
||||
|
||||
cp -rf general_detection_op.cpp Serving/core/general-server/op
|
||||
|
||||
```
|
||||
|
||||
具体可参考官方文档：[如何编译Serving](https://github.com/PaddlePaddle/Serving/blob/v0.8.3/doc/Compile_CN.md)，注意需要开启 WITH_OPENCV 选项。
|
||||
|
||||
完成编译后，注意要安装编译出的三个whl包，并设置SERVING_BIN环境变量。
|
||||
|
||||
|
|
@ -209,12 +225,25 @@ C++ 服务部署在环境搭建和数据准备阶段与 python 相同，区别
|
|||
|
||||
```
|
||||
# 启动服务，运行日志保存在log.txt
|
||||
python3 -m paddle_serving_server.serve --model ppocrv2_det_serving ppocrv2_rec_serving --op GeneralDetectionOp GeneralRecOp --port 9293 &>log.txt &
|
||||
python3 -m paddle_serving_server.serve --model ppocrv2_det_serving ppocrv2_rec_serving --op GeneralDetectionOp GeneralInferOp --port 9293 &>log.txt &
|
||||
```
|
||||
成功启动服务后，log.txt中会打印类似如下日志
|
||||
|
||||
|
||||
3. 发送服务请求：
|
||||
|
||||
由于需要在C++Server部分进行前后处理，为了加速传入C++Server的仅仅是图片的base64编码的字符串，故需要手动修改
|
||||
ppocrv2_det_client/serving_client_conf.prototxt 中 feed_type 字段 和 shape 字段，修改成如下内容：
|
||||
```
|
||||
feed_var {
|
||||
name: "x"
|
||||
alias_name: "x"
|
||||
is_lod_tensor: false
|
||||
feed_type: 20
|
||||
shape: 1
|
||||
}
|
||||
```
|
||||
启动客户端
|
||||
```
|
||||
python3 ocr_cpp_client.py ppocrv2_det_client ppocrv2_rec_client
|
||||
```
|
||||
|
|
|
|||
|
|
@ -0,0 +1,367 @@
|
|||
// Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
#include "core/general-server/op/general_detection_op.h"
|
||||
#include "core/predictor/framework/infer.h"
|
||||
#include "core/predictor/framework/memory.h"
|
||||
#include "core/predictor/framework/resource.h"
|
||||
#include "core/util/include/timer.h"
|
||||
#include <algorithm>
|
||||
#include <iostream>
|
||||
#include <memory>
|
||||
#include <sstream>
|
||||
|
||||
/*
|
||||
#include "opencv2/imgcodecs/legacy/constants_c.h"
|
||||
#include "opencv2/imgproc/types_c.h"
|
||||
*/
|
||||
|
||||
namespace baidu {
|
||||
namespace paddle_serving {
|
||||
namespace serving {
|
||||
|
||||
using baidu::paddle_serving::Timer;
|
||||
using baidu::paddle_serving::predictor::MempoolWrapper;
|
||||
using baidu::paddle_serving::predictor::general_model::Tensor;
|
||||
using baidu::paddle_serving::predictor::general_model::Response;
|
||||
using baidu::paddle_serving::predictor::general_model::Request;
|
||||
using baidu::paddle_serving::predictor::InferManager;
|
||||
using baidu::paddle_serving::predictor::PaddleGeneralModelConfig;
|
||||
|
||||
int GeneralDetectionOp::inference() {
|
||||
VLOG(2) << "Going to run inference";
|
||||
const std::vector<std::string> pre_node_names = pre_names();
|
||||
if (pre_node_names.size() != 1) {
|
||||
LOG(ERROR) << "This op(" << op_name()
|
||||
<< ") can only have one predecessor op, but received "
|
||||
<< pre_node_names.size();
|
||||
return -1;
|
||||
}
|
||||
const std::string pre_name = pre_node_names[0];
|
||||
|
||||
const GeneralBlob *input_blob = get_depend_argument<GeneralBlob>(pre_name);
|
||||
if (!input_blob) {
|
||||
LOG(ERROR) << "input_blob is nullptr,error";
|
||||
return -1;
|
||||
}
|
||||
uint64_t log_id = input_blob->GetLogId();
|
||||
VLOG(2) << "(logid=" << log_id << ") Get precedent op name: " << pre_name;
|
||||
|
||||
GeneralBlob *output_blob = mutable_data<GeneralBlob>();
|
||||
if (!output_blob) {
|
||||
LOG(ERROR) << "output_blob is nullptr,error";
|
||||
return -1;
|
||||
}
|
||||
output_blob->SetLogId(log_id);
|
||||
|
||||
if (!input_blob) {
|
||||
LOG(ERROR) << "(logid=" << log_id
|
||||
<< ") Failed mutable depended argument, op:" << pre_name;
|
||||
return -1;
|
||||
}
|
||||
|
||||
const TensorVector *in = &input_blob->tensor_vector;
|
||||
TensorVector *out = &output_blob->tensor_vector;
|
||||
|
||||
int batch_size = input_blob->_batch_size;
|
||||
VLOG(2) << "(logid=" << log_id << ") input batch size: " << batch_size;
|
||||
|
||||
output_blob->_batch_size = batch_size;
|
||||
|
||||
std::vector<int> input_shape;
|
||||
int in_num = 0;
|
||||
void *databuf_data = NULL;
|
||||
char *databuf_char = NULL;
|
||||
size_t databuf_size = 0;
|
||||
// now only support single string
|
||||
char *total_input_ptr = static_cast<char *>(in->at(0).data.data());
|
||||
std::string base64str = total_input_ptr;
|
||||
|
||||
float ratio_h{};
|
||||
float ratio_w{};
|
||||
|
||||
cv::Mat img = Base2Mat(base64str);
|
||||
cv::Mat srcimg;
|
||||
cv::Mat resize_img;
|
||||
|
||||
cv::Mat resize_img_rec;
|
||||
cv::Mat crop_img;
|
||||
img.copyTo(srcimg);
|
||||
|
||||
this->resize_op_.Run(img, resize_img, this->max_side_len_, ratio_h, ratio_w,
|
||||
this->use_tensorrt_);
|
||||
|
||||
this->normalize_op_.Run(&resize_img, this->mean_det, this->scale_det,
|
||||
this->is_scale_);
|
||||
|
||||
std::vector<float> input(1 * 3 * resize_img.rows * resize_img.cols, 0.0f);
|
||||
this->permute_op_.Run(&resize_img, input.data());
|
||||
|
||||
TensorVector *real_in = new TensorVector();
|
||||
if (!real_in) {
|
||||
LOG(ERROR) << "real_in is nullptr,error";
|
||||
return -1;
|
||||
}
|
||||
|
||||
for (int i = 0; i < in->size(); ++i) {
|
||||
input_shape = {1, 3, resize_img.rows, resize_img.cols};
|
||||
in_num = std::accumulate(input_shape.begin(), input_shape.end(), 1,
|
||||
std::multiplies<int>());
|
||||
databuf_size = in_num * sizeof(float);
|
||||
databuf_data = MempoolWrapper::instance().malloc(databuf_size);
|
||||
if (!databuf_data) {
|
||||
LOG(ERROR) << "Malloc failed, size: " << databuf_size;
|
||||
return -1;
|
||||
}
|
||||
memcpy(databuf_data, input.data(), databuf_size);
|
||||
databuf_char = reinterpret_cast<char *>(databuf_data);
|
||||
paddle::PaddleBuf paddleBuf(databuf_char, databuf_size);
|
||||
paddle::PaddleTensor tensor_in;
|
||||
tensor_in.name = in->at(i).name;
|
||||
tensor_in.dtype = paddle::PaddleDType::FLOAT32;
|
||||
tensor_in.shape = {1, 3, resize_img.rows, resize_img.cols};
|
||||
tensor_in.lod = in->at(i).lod;
|
||||
tensor_in.data = paddleBuf;
|
||||
real_in->push_back(tensor_in);
|
||||
}
|
||||
|
||||
Timer timeline;
|
||||
int64_t start = timeline.TimeStampUS();
|
||||
timeline.Start();
|
||||
|
||||
if (InferManager::instance().infer(engine_name().c_str(), real_in, out,
|
||||
batch_size)) {
|
||||
LOG(ERROR) << "(logid=" << log_id
|
||||
<< ") Failed do infer in fluid model: " << engine_name().c_str();
|
||||
return -1;
|
||||
}
|
||||
delete real_in;
|
||||
|
||||
std::vector<int> output_shape;
|
||||
int out_num = 0;
|
||||
void *databuf_data_out = NULL;
|
||||
char *databuf_char_out = NULL;
|
||||
size_t databuf_size_out = 0;
|
||||
// this is special add for PaddleOCR postprecess
|
||||
int infer_outnum = out->size();
|
||||
for (int k = 0; k < infer_outnum; ++k) {
|
||||
int n2 = out->at(k).shape[2];
|
||||
int n3 = out->at(k).shape[3];
|
||||
int n = n2 * n3;
|
||||
|
||||
float *out_data = static_cast<float *>(out->at(k).data.data());
|
||||
std::vector<float> pred(n, 0.0);
|
||||
std::vector<unsigned char> cbuf(n, ' ');
|
||||
|
||||
for (int i = 0; i < n; i++) {
|
||||
pred[i] = float(out_data[i]);
|
||||
cbuf[i] = (unsigned char)((out_data[i]) * 255);
|
||||
}
|
||||
|
||||
cv::Mat cbuf_map(n2, n3, CV_8UC1, (unsigned char *)cbuf.data());
|
||||
cv::Mat pred_map(n2, n3, CV_32F, (float *)pred.data());
|
||||
|
||||
const double threshold = this->det_db_thresh_ * 255;
|
||||
const double maxvalue = 255;
|
||||
cv::Mat bit_map;
|
||||
cv::threshold(cbuf_map, bit_map, threshold, maxvalue, cv::THRESH_BINARY);
|
||||
cv::Mat dilation_map;
|
||||
cv::Mat dila_ele =
|
||||
cv::getStructuringElement(cv::MORPH_RECT, cv::Size(2, 2));
|
||||
cv::dilate(bit_map, dilation_map, dila_ele);
|
||||
boxes = post_processor_.BoxesFromBitmap(pred_map, dilation_map,
|
||||
this->det_db_box_thresh_,
|
||||
this->det_db_unclip_ratio_);
|
||||
|
||||
boxes = post_processor_.FilterTagDetRes(boxes, ratio_h, ratio_w, srcimg);
|
||||
|
||||
float max_wh_ratio = 0.0f;
|
||||
std::vector<cv::Mat> crop_imgs;
|
||||
std::vector<cv::Mat> resize_imgs;
|
||||
int max_resize_w = 0;
|
||||
int max_resize_h = 0;
|
||||
int box_num = boxes.size();
|
||||
std::vector<std::vector<float>> output_rec;
|
||||
for (int i = 0; i < box_num; ++i) {
|
||||
cv::Mat line_img = GetRotateCropImage(img, boxes[i]);
|
||||
float wh_ratio = float(line_img.cols) / float(line_img.rows);
|
||||
max_wh_ratio = max_wh_ratio > wh_ratio ? max_wh_ratio : wh_ratio;
|
||||
crop_imgs.push_back(line_img);
|
||||
}
|
||||
|
||||
for (int i = 0; i < box_num; ++i) {
|
||||
cv::Mat resize_img;
|
||||
crop_img = crop_imgs[i];
|
||||
this->resize_op_rec.Run(crop_img, resize_img, max_wh_ratio,
|
||||
this->use_tensorrt_);
|
||||
|
||||
this->normalize_op_.Run(&resize_img, this->mean_rec, this->scale_rec,
|
||||
this->is_scale_);
|
||||
|
||||
max_resize_w = std::max(max_resize_w, resize_img.cols);
|
||||
max_resize_h = std::max(max_resize_h, resize_img.rows);
|
||||
resize_imgs.push_back(resize_img);
|
||||
}
|
||||
int buf_size = 3 * max_resize_h * max_resize_w;
|
||||
output_rec = std::vector<std::vector<float>>(
|
||||
box_num, std::vector<float>(buf_size, 0.0f));
|
||||
for (int i = 0; i < box_num; ++i) {
|
||||
resize_img_rec = resize_imgs[i];
|
||||
|
||||
this->permute_op_.Run(&resize_img_rec, output_rec[i].data());
|
||||
}
|
||||
|
||||
// Inference.
|
||||
output_shape = {box_num, 3, max_resize_h, max_resize_w};
|
||||
out_num = std::accumulate(output_shape.begin(), output_shape.end(), 1,
|
||||
std::multiplies<int>());
|
||||
databuf_size_out = out_num * sizeof(float);
|
||||
databuf_data_out = MempoolWrapper::instance().malloc(databuf_size_out);
|
||||
if (!databuf_data_out) {
|
||||
LOG(ERROR) << "Malloc failed, size: " << databuf_size_out;
|
||||
return -1;
|
||||
}
|
||||
int offset = buf_size * sizeof(float);
|
||||
for (int i = 0; i < box_num; ++i) {
|
||||
memcpy(databuf_data_out + i * offset, output_rec[i].data(), offset);
|
||||
}
|
||||
databuf_char_out = reinterpret_cast<char *>(databuf_data_out);
|
||||
paddle::PaddleBuf paddleBuf(databuf_char_out, databuf_size_out);
|
||||
paddle::PaddleTensor tensor_out;
|
||||
tensor_out.name = "x";
|
||||
tensor_out.dtype = paddle::PaddleDType::FLOAT32;
|
||||
tensor_out.shape = output_shape;
|
||||
tensor_out.data = paddleBuf;
|
||||
out->push_back(tensor_out);
|
||||
}
|
||||
out->erase(out->begin(), out->begin() + infer_outnum);
|
||||
|
||||
int64_t end = timeline.TimeStampUS();
|
||||
CopyBlobInfo(input_blob, output_blob);
|
||||
AddBlobInfo(output_blob, start);
|
||||
AddBlobInfo(output_blob, end);
|
||||
return 0;
|
||||
}
|
||||
|
||||
cv::Mat GeneralDetectionOp::Base2Mat(std::string &base64_data) {
|
||||
cv::Mat img;
|
||||
std::string s_mat;
|
||||
s_mat = base64Decode(base64_data.data(), base64_data.size());
|
||||
std::vector<char> base64_img(s_mat.begin(), s_mat.end());
|
||||
img = cv::imdecode(base64_img, cv::IMREAD_COLOR); // CV_LOAD_IMAGE_COLOR
|
||||
return img;
|
||||
}
|
||||
|
||||
std::string GeneralDetectionOp::base64Decode(const char *Data, int DataByte) {
|
||||
const char DecodeTable[] = {
|
||||
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
|
||||
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
|
||||
0, 0, 0, 0, 0, 0, 0, 0, 0,
|
||||
62, // '+'
|
||||
0, 0, 0,
|
||||
63, // '/'
|
||||
52, 53, 54, 55, 56, 57, 58, 59, 60, 61, // '0'-'9'
|
||||
0, 0, 0, 0, 0, 0, 0, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9,
|
||||
10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, // 'A'-'Z'
|
||||
0, 0, 0, 0, 0, 0, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36,
|
||||
37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, // 'a'-'z'
|
||||
};
|
||||
|
||||
std::string strDecode;
|
||||
int nValue;
|
||||
int i = 0;
|
||||
while (i < DataByte) {
|
||||
if (*Data != '\r' && *Data != '\n') {
|
||||
nValue = DecodeTable[*Data++] << 18;
|
||||
nValue += DecodeTable[*Data++] << 12;
|
||||
strDecode += (nValue & 0x00FF0000) >> 16;
|
||||
if (*Data != '=') {
|
||||
nValue += DecodeTable[*Data++] << 6;
|
||||
strDecode += (nValue & 0x0000FF00) >> 8;
|
||||
if (*Data != '=') {
|
||||
nValue += DecodeTable[*Data++];
|
||||
strDecode += nValue & 0x000000FF;
|
||||
}
|
||||
}
|
||||
i += 4;
|
||||
} else // 回车换行,跳过
|
||||
{
|
||||
Data++;
|
||||
i++;
|
||||
}
|
||||
}
|
||||
return strDecode;
|
||||
}
|
||||
|
||||
cv::Mat
|
||||
GeneralDetectionOp::GetRotateCropImage(const cv::Mat &srcimage,
|
||||
std::vector<std::vector<int>> box) {
|
||||
cv::Mat image;
|
||||
srcimage.copyTo(image);
|
||||
std::vector<std::vector<int>> points = box;
|
||||
|
||||
int x_collect[4] = {box[0][0], box[1][0], box[2][0], box[3][0]};
|
||||
int y_collect[4] = {box[0][1], box[1][1], box[2][1], box[3][1]};
|
||||
int left = int(*std::min_element(x_collect, x_collect + 4));
|
||||
int right = int(*std::max_element(x_collect, x_collect + 4));
|
||||
int top = int(*std::min_element(y_collect, y_collect + 4));
|
||||
int bottom = int(*std::max_element(y_collect, y_collect + 4));
|
||||
|
||||
cv::Mat img_crop;
|
||||
image(cv::Rect(left, top, right - left, bottom - top)).copyTo(img_crop);
|
||||
|
||||
for (int i = 0; i < points.size(); i++) {
|
||||
points[i][0] -= left;
|
||||
points[i][1] -= top;
|
||||
}
|
||||
|
||||
int img_crop_width = int(sqrt(pow(points[0][0] - points[1][0], 2) +
|
||||
pow(points[0][1] - points[1][1], 2)));
|
||||
int img_crop_height = int(sqrt(pow(points[0][0] - points[3][0], 2) +
|
||||
pow(points[0][1] - points[3][1], 2)));
|
||||
|
||||
cv::Point2f pts_std[4];
|
||||
pts_std[0] = cv::Point2f(0., 0.);
|
||||
pts_std[1] = cv::Point2f(img_crop_width, 0.);
|
||||
pts_std[2] = cv::Point2f(img_crop_width, img_crop_height);
|
||||
pts_std[3] = cv::Point2f(0.f, img_crop_height);
|
||||
|
||||
cv::Point2f pointsf[4];
|
||||
pointsf[0] = cv::Point2f(points[0][0], points[0][1]);
|
||||
pointsf[1] = cv::Point2f(points[1][0], points[1][1]);
|
||||
pointsf[2] = cv::Point2f(points[2][0], points[2][1]);
|
||||
pointsf[3] = cv::Point2f(points[3][0], points[3][1]);
|
||||
|
||||
cv::Mat M = cv::getPerspectiveTransform(pointsf, pts_std);
|
||||
|
||||
cv::Mat dst_img;
|
||||
cv::warpPerspective(img_crop, dst_img, M,
|
||||
cv::Size(img_crop_width, img_crop_height),
|
||||
cv::BORDER_REPLICATE);
|
||||
|
||||
if (float(dst_img.rows) >= float(dst_img.cols) * 1.5) {
|
||||
cv::Mat srcCopy = cv::Mat(dst_img.rows, dst_img.cols, dst_img.depth());
|
||||
cv::transpose(dst_img, srcCopy);
|
||||
cv::flip(srcCopy, srcCopy, 0);
|
||||
return srcCopy;
|
||||
} else {
|
||||
return dst_img;
|
||||
}
|
||||
}
|
||||
|
||||
DEFINE_OP(GeneralDetectionOp);
|
||||
|
||||
} // namespace serving
|
||||
} // namespace paddle_serving
|
||||
} // namespace baidu
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 493 KiB |
|
|
@ -47,7 +47,6 @@ for img_file in os.listdir(test_img_dir):
|
|||
res_list = []
|
||||
fetch_map = client.predict(
|
||||
feed={"x": image}, fetch=["save_infer_model/scale_0.tmp_1"], batch=True)
|
||||
print("fetrch map:", fetch_map)
|
||||
one_batch_res = ocr_reader.postprocess(fetch_map, with_score=True)
|
||||
for res in one_batch_res:
|
||||
res_list.append(res[0])
|
||||
|
|
|
|||
|
|
@ -34,12 +34,28 @@ test_img_dir = args.image_dir
|
|||
for idx, img_file in enumerate(os.listdir(test_img_dir)):
|
||||
with open(os.path.join(test_img_dir, img_file), 'rb') as file:
|
||||
image_data1 = file.read()
|
||||
# print file name
|
||||
print('{}{}{}'.format('*' * 10, img_file, '*' * 10))
|
||||
|
||||
image = cv2_to_base64(image_data1)
|
||||
|
||||
for i in range(1):
|
||||
data = {"key": ["image"], "value": [image]}
|
||||
r = requests.post(url=url, data=json.dumps(data))
|
||||
print(r.json())
|
||||
result = r.json()
|
||||
print("erro_no:{}, err_msg:{}".format(result["err_no"], result["err_msg"]))
|
||||
# check success
|
||||
if result["err_no"] == 0:
|
||||
ocr_result = result["value"][0]
|
||||
try:
|
||||
for item in eval(ocr_result):
|
||||
# return transcription and points
|
||||
print("{}, {}".format(item[0], item[1]))
|
||||
except Exception as e:
|
||||
print("No results")
|
||||
continue
|
||||
|
||||
else:
|
||||
print(
|
||||
"For details about error message, see PipelineServingLogs/pipeline.log"
|
||||
)
|
||||
print("==> total number of test imgs: ", len(os.listdir(test_img_dir)))
|
||||
|
|
|
|||
|
|
@ -15,6 +15,7 @@ from paddle_serving_server.web_service import WebService, Op
|
|||
|
||||
import logging
|
||||
import numpy as np
|
||||
import copy
|
||||
import cv2
|
||||
import base64
|
||||
# from paddle_serving_app.reader import OCRReader
|
||||
|
|
@ -36,7 +37,7 @@ class DetOp(Op):
|
|||
self.filter_func = FilterBoxes(10, 10)
|
||||
self.post_func = DBPostProcess({
|
||||
"thresh": 0.3,
|
||||
"box_thresh": 0.5,
|
||||
"box_thresh": 0.6,
|
||||
"max_candidates": 1000,
|
||||
"unclip_ratio": 1.5,
|
||||
"min_size": 3
|
||||
|
|
@ -79,8 +80,10 @@ class RecOp(Op):
|
|||
raw_im = input_dict["image"]
|
||||
data = np.frombuffer(raw_im, np.uint8)
|
||||
im = cv2.imdecode(data, cv2.IMREAD_COLOR)
|
||||
dt_boxes = input_dict["dt_boxes"]
|
||||
dt_boxes = self.sorted_boxes(dt_boxes)
|
||||
self.dt_list = input_dict["dt_boxes"]
|
||||
self.dt_list = self.sorted_boxes(self.dt_list)
|
||||
# deepcopy to save origin dt_boxes
|
||||
dt_boxes = copy.deepcopy(self.dt_list)
|
||||
feed_list = []
|
||||
img_list = []
|
||||
max_wh_ratio = 0
|
||||
|
|
@ -126,25 +129,29 @@ class RecOp(Op):
|
|||
imgs[id] = norm_img
|
||||
feed = {"x": imgs.copy()}
|
||||
feed_list.append(feed)
|
||||
|
||||
return feed_list, False, None, ""
|
||||
|
||||
def postprocess(self, input_dicts, fetch_data, data_id, log_id):
|
||||
res_list = []
|
||||
rec_list = []
|
||||
dt_num = len(self.dt_list)
|
||||
if isinstance(fetch_data, dict):
|
||||
if len(fetch_data) > 0:
|
||||
rec_batch_res = self.ocr_reader.postprocess(
|
||||
fetch_data, with_score=True)
|
||||
for res in rec_batch_res:
|
||||
res_list.append(res[0])
|
||||
rec_list.append(res)
|
||||
elif isinstance(fetch_data, list):
|
||||
for one_batch in fetch_data:
|
||||
one_batch_res = self.ocr_reader.postprocess(
|
||||
one_batch, with_score=True)
|
||||
for res in one_batch_res:
|
||||
res_list.append(res[0])
|
||||
|
||||
res = {"res": str(res_list)}
|
||||
rec_list.append(res)
|
||||
result_list = []
|
||||
for i in range(dt_num):
|
||||
text = rec_list[i]
|
||||
dt_box = self.dt_list[i]
|
||||
result_list.append([text, dt_box.tolist()])
|
||||
res = {"result": str(result_list)}
|
||||
return res, None, ""
|
||||
|
||||
|
||||
|
|
|
|||
Loading…
Reference in New Issue