mmocr/projects/ABCNet/README.md

ABCNet: Real-time Scene Text Spotting with Adaptive Bezier-Curve Network

[arXiv paper] [CVPR paper]

Description

This is an implementation of ABCNet based on MMOCR, MMCV, and MMEngine.

ABCNet is a conceptually novel, efficient, and fully convolutional framework for text spotting, which address the problem by proposing the Adaptive Bezier-Curve Network (ABCNet). Our contributions are three-fold: 1) For the first time, we adaptively fit arbitrarily-shaped text by a parameterized Bezier curve. 2) We design a novel BezierAlign layer for extracting accurate convolution features of a text instance with arbitrary shapes, significantly improving the precision compared with previous methods. 3) Compared with standard bounding box detection, our Bezier curve detection introduces negligible computation overhead, resulting in superiority of our method in both efficiency and accuracy. Experiments on arbitrarily-shaped benchmark datasets, namely Total-Text and CTW1500, demonstrate that ABCNet achieves state-of-the-art accuracy, meanwhile significantly improving the speed. In particular, on Total-Text, our realtime version is over 10 times faster than recent state-of-the-art methods with a competitive recognition accuracy.

Usage

This ABCNet implementation works for inference only. Its full implementation will be available soon.

Testing commands

As of now, BezierAlign is not yet supported by MMCV, and we will use third-party MMCV with the implementation of BezierAlign. You will need to install it from the source code as follows:

git clone -b lkk/bezier_align https://github.com/Harold-lkk/mmcv.git
cd mmcv
MMCV_WITH_OPS=1 MAX_JOBS=8 python setup.py develop

In MMOCR's root directory, run the following command to test the model:

python tools/test.py projects/ABCNet/config/abcnet/abcnet_resnet50_fpn.py ${CHECKPOINT_PATH}

Results

Here we provide the baseline version of ABCNet with ResNet50 backbone.

To find more variants, please visit the official model zoo.

Name	E2E-None-Hmean	det-Hmean	Download
v1-icdar2015-finetune	0.5803	0.8818	model

Citation

If you find ABCNet useful in your research or applications, please cite ABCNet with the following BibTeX entry.

@inproceedings{liu2020abcnet,
  title     =  {{ABCNet}: Real-time Scene Text Spotting with Adaptive Bezier-Curve Network},
  author    =  {Liu, Yuliang and Chen, Hao and Shen, Chunhua and He, Tong and Jin, Lianwen and Wang, Liangwei},
  booktitle =  {Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR)},
  year      =  {2020}
}

Checklist

• Milestone 1: PR-ready, and acceptable to be one of the projects/.

  • Finish the code

  • Basic docstrings & proper citation

  • Test-time correctness

  • A full README

• Milestone 2: Indicates a successful model implementation.

  • Training-time correctness

• Milestone 3: Good to be a part of our core package!

  • Type hints and docstrings

  • Unit tests

  • Code polishing

  • Metafile.yml

• Move your modules into the core package following the codebase's file hierarchy structure.

• Refactor your modules into the core package following the codebase's file hierarchy structure.