## Motivation
The DETR-related modules have been refactored in
open-mmlab/mmdetection#8763, which causes breakings of MaskFormer and
Mask2Former in both MMDetection (has been fixed in
open-mmlab/mmdetection#9515) and MMSegmentation. This pr fix the bugs in
MMSegmentation.
### TO-DO List
- [x] update configs
- [x] check and modify data flow
- [x] fix unit test
- [x] aligning inference
- [x] write a ckpt converter
- [x] write ckpt update script
- [x] update model zoo
- [x] update model link in readme
- [x] update
[faq.md](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/docs/en/notes/faq.md#installation)
## Tips of Fixing other implementations based on MaskXFormer of mmseg
1. The Transformer modules should be built directly. The original
building with register manner has been refactored.
2. The config requires to be modified. Delete `type` and modify several
keys, according to the modifications in this pr.
3. The `batch_first` is set `True` uniformly in the new implementations.
Hence the data flow requires to be transposed and config of
`batch_first` needs to be modified.
4. The checkpoint trained on the old implementation should be converted
to be used in the new one.
### Convert script
```Python
import argparse
from copy import deepcopy
from collections import OrderedDict
import torch
from mmengine.config import Config
from mmseg.models import build_segmentor
from mmseg.utils import register_all_modules
register_all_modules(init_default_scope=True)
def parse_args():
parser = argparse.ArgumentParser(
description='MMSeg convert MaskXFormer model, by Li-Qingyun')
parser.add_argument('Mask_what_former', type=int,
help='Mask what former, can be a `1` or `2`',
choices=[1, 2])
parser.add_argument('CFG_FILE', help='config file path')
parser.add_argument('OLD_CKPT_FILEPATH', help='old ckpt file path')
parser.add_argument('NEW_CKPT_FILEPATH', help='new ckpt file path')
args = parser.parse_args()
return args
args = parse_args()
def get_new_name(old_name: str):
new_name = old_name
if 'encoder.layers' in new_name:
new_name = new_name.replace('attentions.0', 'self_attn')
new_name = new_name.replace('ffns.0', 'ffn')
if 'decoder.layers' in new_name:
if args.Mask_what_former == 2:
# for Mask2Former
new_name = new_name.replace('attentions.0', 'cross_attn')
new_name = new_name.replace('attentions.1', 'self_attn')
else:
# for Mask2Former
new_name = new_name.replace('attentions.0', 'self_attn')
new_name = new_name.replace('attentions.1', 'cross_attn')
return new_name
def cvt_sd(old_sd: OrderedDict):
new_sd = OrderedDict()
for name, param in old_sd.items():
new_name = get_new_name(name)
assert new_name not in new_sd
new_sd[new_name] = param
assert len(new_sd) == len(old_sd)
return new_sd
if __name__ == '__main__':
cfg = Config.fromfile(args.CFG_FILE)
model_cfg = cfg.model
segmentor = build_segmentor(model_cfg)
refer_sd = segmentor.state_dict()
old_ckpt = torch.load(args.OLD_CKPT_FILEPATH)
old_sd = old_ckpt['state_dict']
new_sd = cvt_sd(old_sd)
print(segmentor.load_state_dict(new_sd))
new_ckpt = deepcopy(old_ckpt)
new_ckpt['state_dict'] = new_sd
torch.save(new_ckpt, args.NEW_CKPT_FILEPATH)
print(f'{args.NEW_CKPT_FILEPATH} has been saved!')
```
Usage:
```bash
# for example
python ckpt4pr2532.py 1 configs/maskformer/maskformer_r50-d32_8xb2-160k_ade20k-512x512.py original_ckpts/maskformer_r50-d32_8xb2-160k_ade20k-512x512_20221030_182724-cbd39cc1.pth cvt_outputs/maskformer_r50-d32_8xb2-160k_ade20k-512x512_20221030_182724.pth
python ckpt4pr2532.py 2 configs/mask2former/mask2former_r50_8xb2-160k_ade20k-512x512.py original_ckpts/mask2former_r50_8xb2-160k_ade20k-512x512_20221204_000055-4c62652d.pth cvt_outputs/mask2former_r50_8xb2-160k_ade20k-512x512_20221204_000055.pth
```
---------
Co-authored-by: MeowZheng <meowzheng@outlook.com>
## Motivation
Use the new fileio from mmengine
https://github.com/open-mmlab/mmengine/pull/533
## Modification
1. Use `mmengine.fileio` to repalce FileClient in mmseg/datasets
2. Use `mmengine.fileio` to repalce FileClient in
mmseg/datasets/transforms
3. Use `mmengine.fileio` to repalce FileClient in mmseg/visualization
## BC-breaking (Optional)
we modify all the dataset configurations, so please use the latest config file.
## Motivation
The conflict is caused by:
The user requested torch==1.12.1+cpu
torchvision 0.13.0+cpu depends on torch==1.12.0
## Modification
modify the torch==1.12.0+cpu
* [CI] Fix CI
* fix wrong command
* remove mmcv
* fix lint error
* add pytorch install
* fix pytorch installation
* fix mmengine error
* fix mmcv installation in pr_stage
* fix docstring coverage in lint and delete cu102 in pr_stage windows
* fix lint.yml and reset test.yml
* ignore some ut in build_windows of pr_stage
* test merge stage test
* fix mmseg dependencies in pr_stage_test.yml
* delete redundant lines in pr_stage and fix mmseg dependencies in mr_stage
* fix error in merge_stage
* delete python -m in merge_stage
* fix error in merge_stage
* let mmcv installation before mmengine
* fix error of mmcv not found
* fix ut error in merge)stage_test.yml
* fix build_windows ut in metge_stage
* fix error
* fix windows error of merge_stag
* Update .github/workflows/merge_stage_test.yml
* Update .github/workflows/merge_stage_test.yml
* Update .github/workflows/merge_stage_test.yml
* fix error
* delete skip timm ut
* add requitements/optinal.txt in test.yml
* Update .github/workflows/merge_stage_test.yml
Co-authored-by: Miao Zheng <76149310+MeowZheng@users.noreply.github.com>
* Skip CI on ignoring given paths
Signed-off-by: del-zhenwu <dele.zhenwu@gmail.com>
* Fix ignoring path with using
Signed-off-by: del-zhenwu <dele.zhenwu@gmail.com>
* add TIMMBackbone and unittests
* add timm to tests requirements
* deprecate pt1.3.1
* reduce the unittests input of timm backbone
* fix ci
* fix ci
* fix ci
* fix ci
* fix ci
* fix ci
* fix ci
* fix ci
* fix ci
* remove unittests of large models of timm backbone
* generate coverage report for all unittests env
* reduce the unittests input of timm backbone
* reduce the unittests input of timm backbone
* Use docker to skip CUDA installation in CI
* Update .github/workflows/build.yml
Co-authored-by: Jerry Jiarui XU <xvjiarui0826@gmail.com>
* Remove Install system dependencies
* Add Install system dependencies for cuda usage
* Add Install system dependencies and requirements
* fix requirements error
Co-authored-by: Jerry Jiarui XU <xvjiarui0826@gmail.com>
* Upgrade CI for PyTorch1.9
* fix typo
* merge build_cu101 and build_cu102 to build_cuda
* fix substitution bug
* test
* test
* test
* test
* test
* test
* test
* test
* test
* test
* test
* test
* remove redudant items
* add python 3.9 to CI
* add python 3.9 to setup.py
* remove some versions of pytorch for python 3.9 test
* fix torch version error
* delete mim files
* update ubuntu version
* fix mmcv cuda version
* [Feature] Move 'Install MMCV' to a independent CI item.
* Merge MMCV install into MMSEG dependencies install
* Fix bug of 'Install MMCV'
* Remove duplicate CI items
* Fix torch device
* Split cpu env and gpu env into two CI project
* Fix some mmdet related bugs
* Fix mmcv-full install bug of build_cpu CI project.
* update CI for pip 20.3
* fixed 1.6 torchvision
* delete python 3.8
* fixed mmcv
* add cuda home
* change to 1.2.2
* add pip upgrade
* install cuda for all
* add missing mmcv
* switch to deprecate
谢昕辰
Qingyun
Miao Zheng
MengzhangLI
del-zhenwu
Junjun2016
sennnnn
Jerry Jiarui XU
Kai Chen