Learning Open-World Object Proposals without Learning to Classify

Pytorch implementation for "Learning Open-World Object Proposals without Learning to Classify" (arXiv 2021)

Dahun Kim, Tsung-Yi Lin, Anelia Angelova, In So Kweon, and Weicheng Kuo.

Introduction

Humans can recognize novel objects in this image despite having never seen them before. “Is it possible to learn open-world (novel) object proposals?” In this paper we propose Object Localization Network (OLN) that learns localization cues instead of foreground vs background classification. Only trained on COCO, OLN is able to propose many novel objects (top) missed by Mask R-CNN (bottom) on an out-of-sample frame in an ego-centric video.

License

This project is released under the Apache 2.0 license.

Disclaimer

This repo is tested under Python 3.7, PyTorch 1.7.0, Cuda 11.0, and mmcv==1.2.5.

Installation

This repo is built based on mmdetection.

You can use following commands to create conda env with related dependencies.

conda create -n oln python=3.7 -y
conda activate oln
conda install pytorch=1.7.0 torchvision cudatoolkit=11.0 -c pytorch -y
pip install mmcv-full
pip install -r requirements.txt
pip install -v -e . 

Please also refer to get_started.md for more details of installation.

Prepare datasets

COCO dataset is available from official websites. It is recommended to download and extract the dataset somewhere outside the project directory and symlink the dataset root to $OLN/data as below.

object_localization_network
├── mmdet
├── tools
├── configs
├── data
│   ├── coco
│   │   ├── annotations
│   │   ├── train2017
│   │   ├── val2017
│   │   ├── test2017

Testing

Our trained models are available for download here. Rename it to latest.pth and run the following commands to test OLN on COCO dataset.

# Multi-GPU distributed testing
bash ./tools/dist_test_bbox.sh configs/oln_box/oln_box.py \
  trained_weights/latest.pth ${NUM_GPUS}

Training

# Multi-GPU distributed training
bash ./tools/dist_train.sh configs/oln_box/oln_box.py ${NUM_GPUS}

Acknowledgement

MMDetection is an open source project that is contributed by researchers and engineers from various colleges and companies. We appreciate all the contributors who implement their methods or add new features, as well as users who give valuable feedbacks. We wish that the toolbox and benchmark could serve the growing research community by providing a flexible toolkit to reimplement existing methods and develop their own new detectors.

Citation

If you use this toolbox or benchmark in your research, please cite this project.

@article{kim2021oln,
  title={Learning Open-World Object Proposals without Learning to Classify},
  author={Kim, Dahun and Lin, Tsung-Yi and Angelova, Anelia and Kweon, In So and Kuo, Weicheng},
  journal={arXiv preprint arXiv:2108.06753},
  year={2021}
}