mmsegmentation

OpenMMLab Semantic Segmentation Toolbox and Benchmark.

deeplabv3 image-segmentation medical-image-segmentation pspnet pytorch realtime-segmentation retinal-vessel-segmentation semantic-segmentation swin-transformer transformer vessel-segmentation

Go to file

Qingyun a092fea8c1 [Fix] Fix MaskFormer and Mask2Former of MMSegmentation (#2532 ) ## Motivation The DETR-related modules have been refactored in open-mmlab/mmdetection#8763, which causes breakings of MaskFormer and Mask2Former in both MMDetection (has been fixed in open-mmlab/mmdetection#9515) and MMSegmentation. This pr fix the bugs in MMSegmentation. ### TO-DO List - [x] update configs - [x] check and modify data flow - [x] fix unit test - [x] aligning inference - [x] write a ckpt converter - [x] write ckpt update script - [x] update model zoo - [x] update model link in readme - [x] update [faq.md](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/docs/en/notes/faq.md#installation) ## Tips of Fixing other implementations based on MaskXFormer of mmseg 1. The Transformer modules should be built directly. The original building with register manner has been refactored. 2. The config requires to be modified. Delete `type` and modify several keys, according to the modifications in this pr. 3. The `batch_first` is set `True` uniformly in the new implementations. Hence the data flow requires to be transposed and config of `batch_first` needs to be modified. 4. The checkpoint trained on the old implementation should be converted to be used in the new one. ### Convert script ```Python import argparse from copy import deepcopy from collections import OrderedDict import torch from mmengine.config import Config from mmseg.models import build_segmentor from mmseg.utils import register_all_modules register_all_modules(init_default_scope=True) def parse_args(): parser = argparse.ArgumentParser( description='MMSeg convert MaskXFormer model, by Li-Qingyun') parser.add_argument('Mask_what_former', type=int, help='Mask what former, can be a `1` or `2`', choices=[1, 2]) parser.add_argument('CFG_FILE', help='config file path') parser.add_argument('OLD_CKPT_FILEPATH', help='old ckpt file path') parser.add_argument('NEW_CKPT_FILEPATH', help='new ckpt file path') args = parser.parse_args() return args args = parse_args() def get_new_name(old_name: str): new_name = old_name if 'encoder.layers' in new_name: new_name = new_name.replace('attentions.0', 'self_attn') new_name = new_name.replace('ffns.0', 'ffn') if 'decoder.layers' in new_name: if args.Mask_what_former == 2: # for Mask2Former new_name = new_name.replace('attentions.0', 'cross_attn') new_name = new_name.replace('attentions.1', 'self_attn') else: # for Mask2Former new_name = new_name.replace('attentions.0', 'self_attn') new_name = new_name.replace('attentions.1', 'cross_attn') return new_name def cvt_sd(old_sd: OrderedDict): new_sd = OrderedDict() for name, param in old_sd.items(): new_name = get_new_name(name) assert new_name not in new_sd new_sd[new_name] = param assert len(new_sd) == len(old_sd) return new_sd if __name__ == '__main__': cfg = Config.fromfile(args.CFG_FILE) model_cfg = cfg.model segmentor = build_segmentor(model_cfg) refer_sd = segmentor.state_dict() old_ckpt = torch.load(args.OLD_CKPT_FILEPATH) old_sd = old_ckpt['state_dict'] new_sd = cvt_sd(old_sd) print(segmentor.load_state_dict(new_sd)) new_ckpt = deepcopy(old_ckpt) new_ckpt['state_dict'] = new_sd torch.save(new_ckpt, args.NEW_CKPT_FILEPATH) print(f'{args.NEW_CKPT_FILEPATH} has been saved!') ``` Usage: ```bash # for example python ckpt4pr2532.py 1 configs/maskformer/maskformer_r50-d32_8xb2-160k_ade20k-512x512.py original_ckpts/maskformer_r50-d32_8xb2-160k_ade20k-512x512_20221030_182724-cbd39cc1.pth cvt_outputs/maskformer_r50-d32_8xb2-160k_ade20k-512x512_20221030_182724.pth python ckpt4pr2532.py 2 configs/mask2former/mask2former_r50_8xb2-160k_ade20k-512x512.py original_ckpts/mask2former_r50_8xb2-160k_ade20k-512x512_20221204_000055-4c62652d.pth cvt_outputs/mask2former_r50_8xb2-160k_ade20k-512x512_20221204_000055.pth ``` --------- Co-authored-by: MeowZheng <meowzheng@outlook.com>		2023-02-01 18:58:21 +08:00
.circleci	[Fix] Fix MaskFormer and Mask2Former of MMSegmentation (#2532 )	2023-02-01 18:58:21 +08:00
.dev	[Refactor] Add pyupgrade pre-commit hook (#2078 )	2022-09-19 14:06:29 +08:00
.github	[Fix] Fix MaskFormer and Mask2Former of MMSegmentation (#2532 )	2023-02-01 18:58:21 +08:00
configs	[Fix] Fix MaskFormer and Mask2Former of MMSegmentation (#2532 )	2023-02-01 18:58:21 +08:00
demo	Update demo/MMSegmentation_Tutorial.ipynb	2022-11-21 11:09:08 +08:00
docker	Bump v1.0.0rc4 (#2529 )	2023-01-30 17:53:07 +08:00
docs	[Fix] Fix MaskFormer and Mask2Former of MMSegmentation (#2532 )	2023-02-01 18:58:21 +08:00
mmseg	[Refactor] Refactor fileio (#2543 )	2023-02-01 17:53:22 +08:00
projects	CodeCamp #1555[Feature] Support Mapillary Vistas Dataset (#2484 )	2023-01-20 14:25:51 +08:00
requirements	[Fix] Fix MaskFormer and Mask2Former of MMSegmentation (#2532 )	2023-02-01 18:58:21 +08:00
resources	update model doc (#2160 )	2022-10-08 21:01:07 +08:00
tests	[Fix] Fix MaskFormer and Mask2Former of MMSegmentation (#2532 )	2023-02-01 18:58:21 +08:00
tools	CodeCamp #140 [New] [Feature] Add synapse dataset and data augmentation in dev-1.x. (#2432 )	2023-01-06 16:14:54 +08:00
.gitignore	[Fix] Add data string in .gitignore file	2022-11-22 22:54:29 +08:00
.owners.yml	…
.pre-commit-config.yaml	[CI] Upgrade the version of isort to fix lint error (#2519 )	2023-01-30 10:31:12 +08:00
.readthedocs.yml	…
CITATION.cff	…
LICENSE	…
MANIFEST.in	…
README.md	Bump v1.0.0rc4 (#2529 )	2023-01-30 17:53:07 +08:00
README_zh-CN.md	Bump v1.0.0rc4 (#2529 )	2023-01-30 17:53:07 +08:00
model-index.yml	[Feature] Support Mask2former in MMSeg 1.x (#2255 )	2022-12-05 18:34:24 +08:00
requirements.txt	…
setup.cfg	Upgrade pre commit hooks (#2154 )	2022-10-08 12:04:24 +08:00
setup.py	[Enchancement] Delete mmcls in runtime.txt in dev-1.x (#2368 )	2022-12-01 16:35:22 +08:00

README.md

OpenMMLab website ^HOT OpenMMLab platform ^{TRY IT OUT}

Documentation: https://mmsegmentation.readthedocs.io/en/1.x/

English | 简体中文

Introduction

MMSegmentation is an open source semantic segmentation toolbox based on PyTorch. It is a part of the OpenMMLab project.

The 1.x branch works with PyTorch 1.6+.

Major features

Unified Benchmark

We provide a unified benchmark toolbox for various semantic segmentation methods.
Modular Design

We decompose the semantic segmentation framework into different components and one can easily construct a customized semantic segmentation framework by combining different modules.
Support of multiple methods out of box

The toolbox directly supports popular and contemporary semantic segmentation frameworks, e.g. PSPNet, DeepLabV3, PSANet, DeepLabV3+, etc.
High efficiency

The training speed is faster than or comparable to other codebases.

What's New

v1.0.0rc4 was released on 30/01/2023. Please refer to changelog.md for details and release history.

Support ISNet (ICCV'2021) in projects (#2400)
Support HSSN (CVPR'2022) in projects (#2444)

Installation

Please refer to get_started.md for installation and dataset_prepare.md for dataset preparation.

Get Started

Please see Overview for the general introduction of MMSegmentation.

Please see user guides for the basic usage of MMSegmentation. There are also advanced tutorials for in-depth understanding of mmseg design and implementation .

A Colab tutorial is also provided. You may preview the notebook here or directly run on Colab.

To migrate from MMSegmentation 1.x, please refer to migration.

Benchmark and model zoo

Results and models are available in the model zoo.

Supported backbones:

Supported methods:

Supported datasets:

Please refer to FAQ for frequently asked questions.

Contributing

We appreciate all contributions to improve MMSegmentation. Please refer to CONTRIBUTING.md for the contributing guideline.

Acknowledgement

MMSegmentation is an open source project that welcome any contribution and feedback. We wish that the toolbox and benchmark could serve the growing research community by providing a flexible as well as standardized toolkit to reimplement existing methods and develop their own new semantic segmentation methods.

Citation

If you find this project useful in your research, please consider cite:

@misc{mmseg2020,
    title={{MMSegmentation}: OpenMMLab Semantic Segmentation Toolbox and Benchmark},
    author={MMSegmentation Contributors},
    howpublished = {\url{https://github.com/open-mmlab/mmsegmentation}},
    year={2020}
}

License

This project is released under the Apache 2.0 license.

Projects in OpenMMLab

MMEngine: OpenMMLab foundational library for training deep learning models
MMCV: OpenMMLab foundational library for computer vision.
MIM: MIM installs OpenMMLab packages.
MMEval: A unified evaluation library for multiple machine learning libraries.
MMClassification: OpenMMLab image classification toolbox and benchmark.
MMDetection: OpenMMLab detection toolbox and benchmark.
MMDetection3D: OpenMMLab's next-generation platform for general 3D object detection.
MMRotate: OpenMMLab rotated object detection toolbox and benchmark.
MMYOLO: OpenMMLab YOLO series toolbox and benchmark.
MMSegmentation: OpenMMLab semantic segmentation toolbox and benchmark.
MMOCR: OpenMMLab text detection, recognition, and understanding toolbox.
MMPose: OpenMMLab pose estimation toolbox and benchmark.
MMHuman3D: OpenMMLab 3D human parametric model toolbox and benchmark.
MMSelfSup: OpenMMLab self-supervised learning toolbox and benchmark.
MMRazor: OpenMMLab model compression toolbox and benchmark.
MMFewShot: OpenMMLab fewshot learning toolbox and benchmark.
MMAction2: OpenMMLab's next-generation action understanding toolbox and benchmark.
MMTracking: OpenMMLab video perception toolbox and benchmark.
MMFlow: OpenMMLab optical flow toolbox and benchmark.
MMEditing: OpenMMLab image and video editing toolbox.
MMGeneration: OpenMMLab image and video generative models toolbox.
MMDeploy: OpenMMLab Model Deployment Framework.