mirrors
/
mmsegmentation
mirror of https://github.com/open-mmlab/mmsegmentation.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity

Merge pull request #2251 from MeowZheng/sync-1.x 

Sync 1.x
pull/2253/head
Miao Zheng 2022-11-02 11:58:59 +08:00 committed by GitHub
parent 02d3793f0e b07bb163a1
commit e4bc52ef11
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
9 changed files with 528 additions and 219 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

14
.circleci/test.yml
View File

@ -61,7 +61,7 @@ jobs:
command: |
pip install git+https://github.com/open-mmlab/mmengine.git@main
pip install -U openmim
mim install 'mmcv >= 2.0.0rc1'
mim install 'mmcv>=2.0.0rc1'
pip install git+https://github.com/open-mmlab/mmclassification@dev-1.x
pip install -r requirements/tests.txt -r requirements/optional.txt
- run:
@ -71,9 +71,9 @@ jobs:
- run:
name: Skip timm unittests and generate coverage report
command: |
coverage run --branch --source mmseg -m pytest tests/ --ignore tests/test_models/test_backbones/test_timm_backbone.py
coverage xml
coverage report -m
python -m coverage run --branch --source mmseg -m pytest tests/ --ignore tests/test_models/test_backbones/test_timm_backbone.py
python -m coverage xml
python -m coverage report -m
build_cuda:
parameters:
torch:
@ -106,7 +106,7 @@ jobs:
command: |
docker exec mmseg pip install -e /mmengine
docker exec mmseg pip install -U openmim
docker exec mmseg mim install 'mmcv >= 2.0.0rc1'
docker exec mmseg mim install 'mmcv>=2.0.0rc1'
docker exec mmseg pip install -e /mmclassification
docker exec mmseg pip install -r requirements/tests.txt -r requirements/optional.txt
- run:
@ -128,6 +128,7 @@ workflows:
ignore:
- dev-1.x
- 1.x
- master
pr_stage_test:
when:
not:
@ -140,6 +141,7 @@ workflows:
ignore:
- dev-1.x
- 1.x
- master
- build_cpu:
name: minimum_version_cpu
torch: 1.6.0
@ -181,3 +183,5 @@ workflows:
branches:
only:
- dev-1.x
- 1.x
- master

45
.github/workflows/test_mim.yml vendored 100644
View File

@ -0,0 +1,45 @@
name: test-mim
on:
push:
paths:
- 'model-index.yml'
- 'configs/**'
pull_request:
paths:
- 'model-index.yml'
- 'configs/**'
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs:
build_cpu:
runs-on: ubuntu-18.04
strategy:
matrix:
python-version: [3.7]
torch: [1.8.0]
include:
- torch: 1.8.0
torch_version: torch1.8
torchvision: 0.9.0
steps:
- uses: actions/checkout@v2
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v2
with:
python-version: ${{ matrix.python-version }}
- name: Upgrade pip
run: pip install pip --upgrade
- name: Install PyTorch
run: pip install torch==${{matrix.torch}}+cpu torchvision==${{matrix.torchvision}}+cpu -f https://download.pytorch.org/whl/torch_stable.html
- name: Install openmim
run: pip install openmim
- name: Build and install
run: rm -rf .eggs && mim install -e .
- name: test commands of mim
run: mim search mmsegmentation>=1.0.0rc0

2
.pre-commit-config.yaml
View File

@ -48,7 +48,7 @@ repos:
name: update-model-index
description: Collect model information and update model-index.yml
entry: .dev/md2yml.py
additional_dependencies: [lxml, opencv-python, mmengine]
additional_dependencies: [mmengine, lxml, opencv-python]
language: python
files: ^configs/.*\.md$
require_serial: true

34
README.md
View File

@ -71,7 +71,7 @@ Please refer to [changelog.md](docs/en/notes/changelog.md) for details and relea
## Installation
Please refer to [get_started.md](docs/en/get_started.md#installation) for installation and [dataset_prepare.md](docs/en/user_guides/dataset_prepare.md#prepare-datasets) for dataset preparation.
Please refer to [get_started.md](docs/en/get_started.md#installation) for installation and [dataset_prepare.md](docs/en/user_guides/2_dataset_prepare.md#prepare-datasets) for dataset preparation.
## Get Started
@ -143,22 +143,22 @@ Supported methods:
Supported datasets:
- [x] [Cityscapes](https://github.com/open-mmlab/mmsegmentation/blob/1.x/docs/en/dataset_prepare.md#cityscapes)
- [x] [PASCAL VOC](https://github.com/open-mmlab/mmsegmentation/blob/1.x/docs/en/dataset_prepare.md#pascal-voc)
- [x] [ADE20K](https://github.com/open-mmlab/mmsegmentation/blob/1.x/docs/en/dataset_prepare.md#ade20k)
- [x] [Pascal Context](https://github.com/open-mmlab/mmsegmentation/blob/1.x/docs/en/dataset_prepare.md#pascal-context)
- [x] [COCO-Stuff 10k](https://github.com/open-mmlab/mmsegmentation/blob/1.x/docs/en/dataset_prepare.md#coco-stuff-10k)
- [x] [COCO-Stuff 164k](https://github.com/open-mmlab/mmsegmentation/blob/1.x/docs/en/dataset_prepare.md#coco-stuff-164k)
- [x] [CHASE_DB1](https://github.com/open-mmlab/mmsegmentation/blob/1.x/docs/en/dataset_prepare.md#chase-db1)
- [x] [DRIVE](https://github.com/open-mmlab/mmsegmentation/blob/1.x/docs/en/dataset_prepare.md#drive)
- [x] [HRF](https://github.com/open-mmlab/mmsegmentation/blob/1.x/docs/en/dataset_prepare.md#hrf)
- [x] [STARE](https://github.com/open-mmlab/mmsegmentation/blob/1.x/docs/en/dataset_prepare.md#stare)
- [x] [Dark Zurich](https://github.com/open-mmlab/mmsegmentation/blob/1.x/docs/en/dataset_prepare.md#dark-zurich)
- [x] [Nighttime Driving](https://github.com/open-mmlab/mmsegmentation/blob/1.x/docs/en/dataset_prepare.md#nighttime-driving)
- [x] [LoveDA](https://github.com/open-mmlab/mmsegmentation/blob/1.x/docs/en/dataset_prepare.md#loveda)
- [x] [Potsdam](https://github.com/open-mmlab/mmsegmentation/blob/1.x/docs/en/dataset_prepare.md#isprs-potsdam)
- [x] [Vaihingen](https://github.com/open-mmlab/mmsegmentation/blob/1.x/docs/en/dataset_prepare.md#isprs-vaihingen)
- [x] [iSAID](https://github.com/open-mmlab/mmsegmentation/blob/1.x/docs/en/dataset_prepare.md#isaid)
- [x] [Cityscapes](https://github.com/open-mmlab/mmsegmentation/blob/1.x/docs/en/user_guides/2_dataset_prepare.md#cityscapes)
- [x] [PASCAL VOC](https://github.com/open-mmlab/mmsegmentation/blob/1.x/docs/en/user_guides/2_dataset_prepare.md#pascal-voc)
- [x] [ADE20K](https://github.com/open-mmlab/mmsegmentation/blob/1.x/docs/en/user_guides/2_dataset_prepare.md#ade20k)
- [x] [Pascal Context](https://github.com/open-mmlab/mmsegmentation/blob/1.x/docs/en/user_guides/2_dataset_prepare.md#pascal-context)
- [x] [COCO-Stuff 10k](https://github.com/open-mmlab/mmsegmentation/blob/1.x/docs/en/user_guides/2_dataset_prepare.md#coco-stuff-10k)
- [x] [COCO-Stuff 164k](https://github.com/open-mmlab/mmsegmentation/blob/1.x/docs/en/user_guides/2_dataset_prepare.md#coco-stuff-164k)
- [x] [CHASE_DB1](https://github.com/open-mmlab/mmsegmentation/blob/1.x/docs/en/user_guides/2_dataset_prepare.md#chase-db1)
- [x] [DRIVE](https://github.com/open-mmlab/mmsegmentation/blob/1.x/docs/en/user_guides/2_dataset_prepare.md#drive)
- [x] [HRF](https://github.com/open-mmlab/mmsegmentation/blob/1.x/docs/en/user_guides/2_dataset_prepare.md#hrf)
- [x] [STARE](https://github.com/open-mmlab/mmsegmentation/blob/1.x/docs/en/user_guides/2_dataset_prepare.md#stare)
- [x] [Dark Zurich](https://github.com/open-mmlab/mmsegmentation/blob/1.x/docs/en/user_guides/2_dataset_prepare.md#dark-zurich)
- [x] [Nighttime Driving](https://github.com/open-mmlab/mmsegmentation/blob/1.x/docs/en/user_guides/2_dataset_prepare.md#nighttime-driving)
- [x] [LoveDA](https://github.com/open-mmlab/mmsegmentation/blob/1.x/docs/en/user_guides/2_dataset_prepare.md#loveda)
- [x] [Potsdam](https://github.com/open-mmlab/mmsegmentation/blob/1.x/docs/en/user_guides/2_dataset_prepare.md#isprs-potsdam)
- [x] [Vaihingen](https://github.com/open-mmlab/mmsegmentation/blob/1.x/docs/en/user_guides/2_dataset_prepare.md#isprs-vaihingen)
- [x] [iSAID](https://github.com/open-mmlab/mmsegmentation/blob/1.x/docs/en/user_guides/2_dataset_prepare.md#isaid)
Please refer to [FAQ](docs/en/notes/faq.md) for frequently asked questions.

4
docs/en/get_started.md
View File

@ -51,10 +51,8 @@ mim install "mmcv>=2.0.0rc1"
Case a: If you develop and run mmseg directly, install it from source:
```shell
git clone https://github.com/open-mmlab/mmsegmentation.git
git clone -b dev-1.x https://github.com/open-mmlab/mmsegmentation.git
cd mmsegmentation
git checkout dev-1.x
# branch 'dev-1.x' set up to track remote branch 'dev-1.x' from 'origin'.
pip install -v -e .
# '-v' means verbose, or more output
# '-e' means installing a project in editable mode,

237
docs/en/user_guides/1_config.md
View File

@ -226,14 +226,46 @@ load_from = None # Load checkpoint from file.
resume = False # Whether to resume from existed model.
```
These are all the configs for training and testing PSPNet, to load and parse them, we can use [Config](https://mmengine.readthedocs.io/en/latest/tutorials/config.html) implemented in [MMEngine](https://github.com/open-mmlab/mmengine)
```python
from mmengine.config import Config
cfg = Config.fromfile('configs/pspnet/pspnet_r50-d8_4xb2-40k_cityscapes-512x1024.py')
print(cfg.train_dataloader)
```
```shell
{'batch_size': 2,
'num_workers': 2,
'persistent_workers': True,
'sampler': {'type': 'InfiniteSampler', 'shuffle': True},
'dataset': {'type': 'CityscapesDataset',
'data_root': 'data/cityscapes/',
'data_prefix': {'img_path': 'leftImg8bit/train',
'seg_map_path': 'gtFine/train'},
'pipeline': [{'type': 'LoadImageFromFile'},
{'type': 'LoadAnnotations'},
{'type': 'RandomResize',
'scale': (2048, 1024),
'ratio_range': (0.5, 2.0),
'keep_ratio': True},
{'type': 'RandomCrop', 'crop_size': (512, 1024), 'cat_max_ratio': 0.75},
{'type': 'RandomFlip', 'prob': 0.5},
{'type': 'PhotoMetricDistortion'},
{'type': 'PackSegInputs'}]}}
```
`cfg` is an instance of `mmengine.config.Config`, its interface is the same as a dict object and also allows access config values as attributes. See [config tutorial](https://mmengine.readthedocs.io/en/latest/tutorials/config.html) in [MMEngine](https://github.com/open-mmlab/mmengine) for more information.
## FAQ
### Ignore some fields in the base configs
Sometimes, you may set `_delete_=True` to ignore some of the fields in base configs.
You may refer to [mmengine](https://mmengine.readthedocs.io/en/latest/tutorials/config.html) for simple illustration.
See [config tutorial](https://mmengine.readthedocs.io/en/latest/tutorials/config.html) in [MMEngine](https://github.com/open-mmlab/mmengine) for simple illustration.
In MMSegmentation, for example, to change the backbone of PSPNet with the following config.
In MMSegmentation, for example, if you would like to modify the backbone of PSPNet with the following config file `pspnet.py`:
```python
norm_cfg = dict(type='SyncBN', requires_grad=True)
@ -251,14 +283,60 @@ model = dict(
norm_eval=False,
style='pytorch',
contract_dilation=True),
decode_head=dict(...),
auxiliary_head=dict(...))
decode_head=dict(
type='PSPHead',
in_channels=2048,
in_index=3,
channels=512,
pool_scales=(1, 2, 3, 6),
dropout_ratio=0.1,
num_classes=19,
norm_cfg=norm_cfg,
align_corners=False,
loss_decode=dict(
type='CrossEntropyLoss', use_sigmoid=False, loss_weight=1.0)))
```
`ResNet` and `HRNet` use different keywords to construct.
Load and parse the config file `pspnet.py` in the code as follows:
```python
_base_ = '../pspnet/pspnet_r50-d8_4xb4-40k_cityscpaes-512x1024.py'
from mmengine.config import Config
cfg = Config.fromfile('pspnet.py')
print(cfg.model)
```
```shell
{'type': 'EncoderDecoder',
'pretrained': 'torchvision://resnet50',
'backbone': {'type': 'ResNetV1c',
'depth': 50,
'num_stages': 4,
'out_indices': (0, 1, 2, 3),
'dilations': (1, 1, 2, 4),
'strides': (1, 2, 1, 1),
'norm_cfg': {'type': 'SyncBN', 'requires_grad': True},
'norm_eval': False,
'style': 'pytorch',
'contract_dilation': True},
'decode_head': {'type': 'PSPHead',
'in_channels': 2048,
'in_index': 3,
'channels': 512,
'pool_scales': (1, 2, 3, 6),
'dropout_ratio': 0.1,
'num_classes': 19,
'norm_cfg': {'type': 'SyncBN', 'requires_grad': True},
'align_corners': False,
'loss_decode': {'type': 'CrossEntropyLoss',
'use_sigmoid': False,
'loss_weight': 1.0}}}
```
`ResNet` and `HRNet` use different keywords to construct, write a new config file `hrnet.py` as follows:
```python
_base_ = 'pspnet.py'
norm_cfg = dict(type='SyncBN', requires_grad=True)
model = dict(
pretrained='open-mmlab://msra/hrnetv2_w32',
@ -290,9 +368,54 @@ model = dict(
num_branches=4,
block='BASIC',
num_blocks=(4, 4, 4, 4),
num_channels=(32, 64, 128, 256)))),
decode_head=dict(...),
auxiliary_head=dict(...))
num_channels=(32, 64, 128, 256)))))
```
Load and parse the config file `hrnet.py` in the code as follows:
```python
from mmengine.config import Config
cfg = Config.fromfile('hrnet.py')
print(cfg.model)
```
```shell
{'type': 'EncoderDecoder',
'pretrained': 'open-mmlab://msra/hrnetv2_w32',
'backbone': {'type': 'HRNet',
'norm_cfg': {'type': 'SyncBN', 'requires_grad': True},
'extra': {'stage1': {'num_modules': 1,
'num_branches': 1,
'block': 'BOTTLENECK',
'num_blocks': (4,),
'num_channels': (64,)},
'stage2': {'num_modules': 1,
'num_branches': 2,
'block': 'BASIC',
'num_blocks': (4, 4),
'num_channels': (32, 64)},
'stage3': {'num_modules': 4,
'num_branches': 3,
'block': 'BASIC',
'num_blocks': (4, 4, 4),
'num_channels': (32, 64, 128)},
'stage4': {'num_modules': 3,
'num_branches': 4,
'block': 'BASIC',
'num_blocks': (4, 4, 4, 4),
'num_channels': (32, 64, 128, 256)}}},
'decode_head': {'type': 'PSPHead',
'in_channels': 2048,
'in_index': 3,
'channels': 512,
'pool_scales': (1, 2, 3, 6),
'dropout_ratio': 0.1,
'num_classes': 19,
'norm_cfg': {'type': 'SyncBN', 'requires_grad': True},
'align_corners': False,
'loss_decode': {'type': 'CrossEntropyLoss',
'use_sigmoid': False,
'loss_weight': 1.0}}}
```
The `_delete_=True` would replace all old keys in `backbone` field with new keys.
@ -360,15 +483,105 @@ model = dict(
## Modify config through script arguments
In the [training script](https://github.com/open-mmlab/mmsegmentation/blob/1.x/tools/train.py) and the [testing script](https://github.com/open-mmlab/mmsegmentation/blob/1.x/tools/test.py), we support the script argument `--cfg-options`, it may help users override some settings in the used config, the key-value pair in `xxx=yyy` format will be merged into config file.
For example, this is a simplified script `demo_script.py`:
```python
import argparse
from mmengine.config import Config, DictAction
def parse_args():
parser = argparse.ArgumentParser(description='Script Example')
parser.add_argument('config', help='train config file path')
parser.add_argument(
'--cfg-options',
nargs='+',
action=DictAction,
help='override some settings in the used config, the key-value pair '
'in xxx=yyy format will be merged into config file. If the value to '
'be overwritten is a list, it should be like key="[a,b]" or key=a,b '
'It also allows nested list/tuple values, e.g. key="[(a,b),(c,d)]" '
'Note that the quotation marks are necessary and that no white space '
'is allowed.')
args = parser.parse_args()
return args
def main():
args = parse_args()
cfg = Config.fromfile(args.config)
if args.cfg_options is not None:
cfg.merge_from_dict(args.cfg_options)
print(cfg)
if __name__ == '__main__':
main()
```
A example config file `demo_config.py` as follows:
```python
backbone = dict(
type='ResNetV1c',
depth=50,
num_stages=4,
out_indices=(0, 1, 2, 3),
dilations=(1, 1, 2, 4),
strides=(1, 2, 1, 1),
norm_eval=False,
style='pytorch',
contract_dilation=True)
```
Run `demo_script.py`:
```shell
python demo_script.py demo_config.py
```
```shell
Config (path: demo_config.py): {'backbone': {'type': 'ResNetV1c', 'depth': 50, 'num_stages': 4, 'out_indices': (0, 1, 2, 3), 'dilations': (1, 1, 2, 4), 'strides': (1, 2, 1, 1), 'norm_eval': False, 'style': 'pytorch', 'contract_dilation': True}}
```
Modify config through script arguments:
```shell
python demo_script.py demo_config.py --cfg-options backbone.depth=101
```
```shell
Config (path: demo_config.py): {'backbone': {'type': 'ResNetV1c', 'depth': 101, 'num_stages': 4, 'out_indices': (0, 1, 2, 3), 'dilations': (1, 1, 2, 4), 'strides': (1, 2, 1, 1), 'norm_eval': False, 'style': 'pytorch', 'contract_dilation': True}}
```
- Update values of list/tuples.
If the value to be updated is a list or a tuple. For example, the config file normally sets `sigma_range=(0, 0.04)` in `data_preprocessor` of `model`.
If the value to be updated is a list or a tuple. For example, the config file `demo_config.py` sets `strides=(1, 2, 1, 1)` in `backbone`.
If you want to change this key, you may specify in two ways:
1. `--cfg-options model.data_preprocessor.sigma_range="(0, 0.05)"`. Note that the quotation mark " is necessary to support list/tuple data types.
2. `--cfg-options model.data_preprocessor.sigma_range=0,0.05`. Note that **NO** white space is allowed in the specified value.
1. `--cfg-options backbone.strides="(1, 1, 1, 1)"`. Note that the quotation mark " is necessary to support list/tuple data types.
```shell
python demo_script.py demo_config.py --cfg-options backbone.strides="(1, 1, 1, 1)"
```
```shell
Config (path: demo_config.py): {'backbone': {'type': 'ResNetV1c', 'depth': 50, 'num_stages': 4, 'out_indices': (0, 1, 2, 3), 'dilations': (1, 1, 2, 4), 'strides': (1, 1, 1, 1), 'norm_eval': False, 'style': 'pytorch', 'contract_dilation': True}}
```
2. `--cfg-options backbone.strides=1,1,1,1`. Note that **NO** white space is allowed in the specified value.
In addition, if the original type is tuple, it will be automatically converted to list after this way.
```shell
python demo_script.py demo_config.py --cfg-options backbone.strides="(1, 1, 1, 1)"
```
```shell
Config (path: demo_config.py): {'backbone': {'type': 'ResNetV1c', 'depth': 50, 'num_stages': 4, 'out_indices': (0, 1, 2, 3), 'dilations': (1, 1, 2, 4), 'strides': [1, 1, 1, 1], 'norm_eval': False, 'style': 'pytorch', 'contract_dilation': True}}
```
```{note}
This modification of only supports modifying configuration items of string, int, float, boolean, None, list and tuple types.
More specifically, for list and tuple types, the elements inside them must also be one of the above seven types.

142
docs/en/user_guides/3_inference.md
View File

@ -2,24 +2,142 @@
MMSegmentation provides pre-trained models for semantic segmentation in [Model Zoo](../model_zoo.md), and supports multiple standard datasets, including Cityscapes, ADE20K, etc.
This note will show how to use existing models to inference on given images.
As for how to test existing models on standard datasets, please see this [guide](./4_train_test.md#Test-models-on-standard-datasets)
As for how to test existing models on standard datasets, please see this [guide](./4_train_test.md)
## Inference on given images
## Inference API
MMSegmentation provides high-level Python APIs for inference on images. Here is an example of building the model and inference on given images.
Please download the [pre-trained model](https://download.openmmlab.com/mmsegmentation/v0.5/pspnet/pspnet_r50-d8_512x1024_80k_cityscapes/pspnet_r50-d8_512x1024_80k_cityscapes_20200606_112131-2376f12b.pth) to the path specified by `checkpoint_file` first.
MMSegmentation provides several interfaces for users to easily use pre-trained models for inference.
- [mmseg.apis.init_model](#mmsegapisinit_model)
- [mmseg.apis.inference_model](#mmsegapisinference_model)
- [mmseg.apis.show_result_pyplot](#mmsegapisshow_result_pyplot)
### mmseg.apis.init_model
Initialize a segmentor from config file.
Parameters:
- config (str, `Path`, or `mmengine.Config`) - Config file path or the config object.
- checkpoint (str, optional) - Checkpoint path. If left as None, the model will not load any weights.
- device (str, optional) - CPU/CUDA device option. Default 'cuda:0'.
- cfg_options (dict, optional) - Options to override some settings in the used config.
Returns:
- nn.Module: The constructed segmentor.
Example:
```python
from mmseg.apis import init_model
from mmseg.utils import register_all_modules
config_path = 'configs/pspnet/pspnet_r50-d8_4xb2-40k_cityscapes-512x1024.py'
checkpoint_path = 'checkpoints/pspnet_r50-d8_512x1024_40k_cityscapes_20200605_003338-2966598c.pth'
# register all modules in mmseg into the registries
register_all_modules()
# initialize model without checkpoint
model = init_model(config_path)
# init model and load checkpoint
model = init_model(config_path, checkpoint_path)
# init model and load checkpoint on CPU
model = init_model(config_path, checkpoint_path, 'cpu')
```
### mmseg.apis.inference_model
Inference image(s) with the segmentor.
Parameters:
- model (nn.Module) - The loaded segmentor
- imgs (str, np.ndarray, or list\[str/np.ndarray\]) - Either image files or loaded images
Returns:
- `SegDataSample` or list\[`SegDataSample`\]: If imgs is a list or tuple, the same length list type results will be returned, otherwise return the segmentation results directly.
**Note:** [SegDataSample](https://github.com/open-mmlab/mmsegmentation/blob/1.x/mmseg/structures/seg_data_sample.py) is a data structure interface of MMSegmentation, it is used as interfaces between different components. `SegDataSample` implement the abstract data element `mmengine.structures.BaseDataElement`, please refer to data element [documentation](https://mmengine.readthedocs.io/en/latest/advanced_tutorials/data_element.html) in [MMEngine](https://github.com/open-mmlab/mmengine) for more information.
The attributes in `SegDataSample` are divided into several parts:
- `gt_sem_seg` (`PixelData`) - Ground truth of semantic segmentation.
- `pred_sem_seg` (`PixelData`) - Prediction of semantic segmentation.
- `seg_logits` (`PixelData`) - Predicted logits of semantic segmentation.
**Note** [PixelData](https://github.com/open-mmlab/mmengine/blob/main/mmengine/structures/pixel_data.py) is the data structure for pixel-level annotations or predictions, please refer to PixelData [documentation](https://mmengine.readthedocs.io/en/latest/advanced_tutorials/data_element.html) in [MMEngine](https://github.com/open-mmlab/mmengine) for more information.
Example:
```python
from mmseg.apis import init_model, inference_model
from mmseg.utils import register_all_modules
# Specify the path to model config and checkpoint file
config_file = 'configs/pspnet/pspnet_r50-d8_512x1024_80k_cityscapes.py'
checkpoint_file = 'checkpoints/pspnet_r50-d8_512x1024_80k_cityscapes_20200606_112131-2376f12b.pth'
config_path = 'configs/pspnet/pspnet_r50-d8_4xb2-40k_cityscapes-512x1024.py'
checkpoint_path = 'checkpoints/pspnet_r50-d8_512x1024_40k_cityscapes_20200605_003338-2966598c.pth'
img_path = 'demo/demo.png'
# register all modules in mmseg into the registries
register_all_modules()
# build the model from a config file and a checkpoint file
model = init_model(config_file, checkpoint_file, device='cuda:0')
# test image pair, and save the results
img = 'demo/demo.png'
result = inference_model(model, img)
model = init_model(config_path, checkpoint_path)
result = inference_model(model, img_path)
```
### mmseg.apis.show_result_pyplot
Visualize the segmentation results on the image.
Parameters:
- model (nn.Module) - The loaded segmentor.
- img (str or np.ndarray) - Image filename or loaded image.
- result (`SegDataSample`) - The prediction SegDataSample result.
- opacity (float) - Opacity of painted segmentation map. Default `0.5`, must be in `(0, 1]` range.
- title (str) - The title of pyplot figure. Default is ''.
- draw_gt (bool) - Whether to draw GT SegDataSample. Default to `True`.
- draw_pred (draws_pred) - Whether to draw Prediction SegDataSample. Default to `True`.
- wait_time (float) - The interval of show (s), 0 is the special value that means "forever". Default to `0`.
- show (bool) - Whether to display the drawn image. Default to `True`.
- save_dir (str, optional) - Save file dir for all storage backends. If it is `None`, the backend storage will not save any data.
- out_file (str, optional) - Path to output file. Default to `None`.
Returns:
- np.ndarray: the drawn image which channel is RGB.
Example:
```python
from mmseg.apis import init_model, inference_model, show_result_pyplot
from mmseg.utils import register_all_modules
config_path = 'configs/pspnet/pspnet_r50-d8_4xb2-40k_cityscapes-512x1024.py'
checkpoint_path = 'checkpoints/pspnet_r50-d8_512x1024_40k_cityscapes_20200605_003338-2966598c.pth'
img_path = 'demo/demo.png'
# register all modules in mmseg into the registries
register_all_modules()
# build the model from a config file and a checkpoint file
model = init_model(config_path, checkpoint_path, device='cuda:0')
# inference on given image
result = inference_model(model, img_path)
# display the segmentation result
vis_image = show_result_pyplot(model, img_path, result)
# save the visualization result, the output image would be found at the path `work_dirs/result.png`
vis_iamge = show_result_pyplot(model, img_path, result, out_file='work_dirs/result.png')
# Modify the time of displaying images, note that 0 is the special value that means "forever".
vis_image = show_result_pyplot(model, img_path, result, wait_time=5)
```
**Note:** If your current device doesn't have graphical user interface, it is recommended that setting `show` to `False` and specify the `out_file` or `save_dir` to save the results. If you would like to display the result on a window, no special settings are required.

263
docs/en/user_guides/4_train_test.md
View File

@ -1,51 +1,8 @@
# Tutorial 4: Train and test with existing models
This tutorial provides instruction for users to use the models provided in the [Model Zoo](../model_zoo.md) for other datasets to obtain better performance.
MMSegmentation also provides out-of-the-box tools for training models.
This section will show how to train and test models on standard datasets.
MMSegmentation supports training and testing models on a variety of devices, which are described below for single-GPU, distributed, and cluster training and testing, respectively. Through this tutorial, you will learn how to train and test using the scripts provided by MMSegmentation.
## Train models on standard datasets
### Modify training schedule
Modify the following configuration to customize the training.
```python
# training schedule for 40k
train_cfg = dict(type='IterBasedTrainLoop', max_iters=40000, val_interval=4000)
val_cfg = dict(type='ValLoop')
test_cfg = dict(type='TestLoop')
# optimizer
optimizer = dict(type='SGD', lr=0.01, momentum=0.9, weight_decay=0.0005)
optim_wrapper = dict(type='OptimWrapper', optimizer=optimizer, clip_grad=None)
# learning policy
param_scheduler = [
dict(
type='PolyLR',
eta_min=1e-4,
power=0.9,
begin=0,
end=40000,
by_epoch=False)
# basic hooks
default_hooks = dict(
timer=dict(type='IterTimerHook'),
logger=dict(type='LoggerHook', interval=50, log_metric_by_epoch=False),
param_scheduler=dict(type='ParamSchedulerHook'),
checkpoint=dict(type='CheckpointHook', by_epoch=False, interval=4000),
sampler_seed=dict(type='DistSamplerSeedHook'))
]
```
### Use pre-trained model
Users can load a pre-trained model by setting the `load_from` field of the config to the model's path or link.
The users might need to download the model weights before training to avoid the download time during training.
```python
# use the pre-trained model for the whole PSPNet
load_from = 'https://download.openmmlab.com/mmsegmentation/v0.5/pspnet/pspnet_r50-d8_512x1024_40k_cityscapes/pspnet_r50-d8_512x1024_40k_cityscapes_20200605_003338-2966598c.pth' # model path can be found in model zoo
```
## Training and testing on a single GPU
### Training on a single GPU
@ -53,32 +10,35 @@ We provide `tools/train.py` to launch training jobs on a single GPU.
The basic usage is as follows.
```shell
python tools/train.py \
${CONFIG_FILE} \
[optional arguments]
python tools/train.py ${CONFIG_FILE} [optional arguments]
```
This tool accepts several optional arguments, including:
- `--work-dir ${WORK_DIR}`: Override the working directory.
- `--amp`: Use auto mixed precision training.
- `--resume ${CHECKPOINT_FILE}`: Resume from a previous checkpoint file. If not specify, try to auto resume from the latest checkpoint in the work directory.
- `--cfg-options ${OVERRIDE_CONFIGS}`: Override some settings in the used config, the key-value pair in xxx=yyy format will be merged into config file.
- `--resume`: Resume from the latest checkpoint in the work_dir automatically.
- `--cfg-options ${OVERRIDE_CONFIGS}`: Override some settings in the used config, and the key-value pair in xxx=yyy format will be merged into the config file.
For example, '--cfg-option model.encoder.in_channels=6'. Please see this [guide](./1_config.md#Modify-config-through-script-arguments) for more details.
Below is the optional arguments for multi-gpu test:
Below are the optional arguments for the multi-gpu test:
- `--launcher`: Items for distributed job initialization launcher. Allowed choices are `none`, `pytorch`, `slurm`, `mpi`. Especially, if set to none, it will test in a non-distributed mode.
- `--local_rank`: ID for local rank. If not specified, it will be set to 0.
**Note**:
Difference between `--resume` and `load-from`:
`--resume` loads both the model weights and optimizer status, and the iteration is also inherited from the specified checkpoint.
It is usually used for resuming the training process that is interrupted accidentally.
**Note:** Difference between the argument `--resume` and the field `load_from` in the config file:
`load-from` only loads the model weights and the training iteration starts from 0. It is usually used for fine-tuning.
`--resume` only determines whether to resume from the latest checkpoint in the work_dir. It is usually used for resuming the training process that is interrupted accidentally.
### Training on CPU
`load_from` will specify the checkpoint to be loaded and the training iteration starts from 0. It is usually used for fine-tuning.
The process of training on the CPU is consistent with single GPU training if machine does not have GPU. If it has GPUs but not wanting to use it, we just need to disable GPUs before the training process.
If you would like to resume training from a specific checkpoint, you can use:
```python
python tools/train.py ${CONFIG_FILE} --resume --cfg-options load_from=${CHECKPOINT}
```
**Training on CPU**: The process of training on the CPU is consistent with single GPU training if a machine does not have GPU. If it has GPUs but not wanting to use them, we just need to disable GPUs before the training process.
```shell
export CUDA_VISIBLE_DEVICES=-1
@ -86,25 +46,46 @@ export CUDA_VISIBLE_DEVICES=-1
And then run the script [above](#training-on-a-single-gpu).
```{warning}
The process of training on the CPU is consistent with single GPU training. We just need to disable GPUs before the training process.
### Testing on a single GPU
We provide `tools/test.py` to launch training jobs on a single GPU.
The basic usage is as follows.
```shell
python tools/test.py ${CONFIG_FILE} ${CHECKPOINT_FILE} [optional arguments]
```
This tool accepts several optional arguments, including:
- `--work-dir`: If specified, results will be saved in this directory. If not specified, the results will be automatically saved to `work_dirs/{CONFIG_NAME}`.
- `--show`: Show prediction results at runtime, available when `--show-dir` is not specified.
- `--show-dir`: Directory where painted images will be saved. If specified, the visualized segmentation mask will be saved to the `work_dir/timestamp/show_dir`.
- `--wait-time`: The interval of show (s), which takes effect when `--show` is activated. Default to 2.
- `--cfg-options`: If specified, the key-value pair in xxx=yyy format will be merged into the config file.
**Testing on CPU**: The process of testing on the CPU is consistent with single GPU testing if a machine does not have GPU. If it has GPUs but not wanting to use them, we just need to disable GPUs before the training process.
```shell
export CUDA_VISIBLE_DEVICES=-1
```
And then run the script [above](#testing-on-a-single-gpu).
## Training and testing on multiple GPUs and multiple machines
### Training on multiple GPUs
OpenMMLab2.0 implements **distributed** training with `MMDistributedDataParallel`.
We provide `tools/dist_train.sh` to launch training on multiple GPUs.
The basic usage is as follows.
The basic usage is as follows:
```shell
sh tools/dist_train.sh \
${CONFIG_FILE} \
${GPU_NUM} \
[optional arguments]
sh tools/dist_train.sh ${CONFIG_FILE} ${GPU_NUM} [optional arguments]
```
Optional arguments remain the same as stated [above](#training-on-a-single-gpu)
and has additional arguments to specify the number of GPUs.
Optional arguments remain the same as stated [above](#training-on-a-single-gpu) and have additional arguments to specify the number of GPUs.
An example:
```shell
@ -113,30 +94,46 @@ An example:
sh tools/dist_train.sh configs/pspnet/pspnet_r50-d8_4xb4-80k_ade20k-512x512.py 8 --work-dir work_dirs/pspnet_r50-d8_4xb4-80k_ade20k-512x512
```
**Note**: During training, checkpoints and logs are saved in the same folder structure as the config file under `work_dirs/`. Custom work directory is not recommended since evaluation scripts infer work directories from the config file name. If you want to save your weights somewhere else, please use symlink, for example:
**Note**: During training, checkpoints and logs are saved in the same folder structure as the config file under `work_dirs/`. A custom work directory is not recommended since evaluation scripts infer work directories from the config file name. If you want to save your weights somewhere else, please use a symlink, for example:
```shell
ln -s ${YOUR_WORK_DIRS} ${MMSEG}/work_dirs
```
#### Launch multiple jobs on a single machine
### Testing on multiple GPUs
If you launch multiple jobs on a single machine, e.g., 2 jobs of 4-GPU training on a machine with 8 GPUs, you need to specify different ports (29500 by default) for each job to avoid communication conflict. Otherwise, there will be error message saying `RuntimeError: Address already in use`.
If you use `dist_train.sh` to launch training jobs, you can set the port in commands with environment variable `PORT`.
We provide `tools/dist_test.sh` to launch testing on multiple GPUs.
The basic usage is as follows.
```shell
sh tools/dist_test.sh ${CONFIG_FILE} ${CHECKPOINT_FILE} ${GPU_NUM} [optional arguments]
```
Optional arguments remain the same as stated [above](#testing-on-a-single-gpu) and have additional arguments to specify the number of GPUs.
An example:
```shell
./tools/dist_test.sh configs/pspnet/pspnet_r50-d8_4xb2-40k_cityscapes-512x1024.py \
checkpoints/pspnet_r50-d8_512x1024_40k_cityscapes_20200605_003338-2966598c.pth 4
```
### Launch multiple jobs on a single machine
If you launch multiple jobs on a single machine, e.g., 2 jobs of 4-GPU training on a machine with 8 GPUs, you need to specify different ports (29500 by default) for each job to avoid communication conflict. Otherwise, there will be an error message saying `RuntimeError: Address already in use`.
If you use `dist_train.sh` to launch training jobs, you can set the port in commands with the environment variable `PORT`.
```shell
CUDA_VISIBLE_DEVICES=0,1,2,3 PORT=29500 sh tools/dist_train.sh ${CONFIG_FILE} 4
CUDA_VISIBLE_DEVICES=4,5,6,7 PORT=29501 sh tools/dist_train.sh ${CONFIG_FILE} 4
```
### Training on multiple nodes
### Training with multiple machines
MMSegmentation relies on `torch.distributed` package for distributed training.
Thus, as a basic usage, one can launch distributed training via PyTorch's [launch utility](https://pytorch.org/docs/stable/distributed.html#launch-utility).
#### Train with multiple machines
If you launch with multiple machines simply connected with ethernet, you can simply run following commands:
If you launch with multiple machines simply connected with ethernet, you can simply run the following commands:
On the first machine:
```shell
@ -149,16 +146,20 @@ On the second machine:
NNODES=2 NODE_RANK=1 PORT=${MASTER_PORT} MASTER_ADDR=${MASTER_ADDR} sh tools/dist_train.sh ${CONFIG_FILE} ${GPUS}
```
Usually it is slow if you do not have high speed networking like InfiniBand.
Usually, it is slow if you do not have high-speed networking like InfiniBand.
#### Manage jobs with Slurm
## Manage jobs with Slurm
[Slurm](https://slurm.schedmd.com/) is a good job scheduling system for computing clusters.
### Training on a cluster with Slurm
On a cluster managed by Slurm, you can use `slurm_train.sh` to spawn training jobs. It supports both single-node and multi-node training.
The basic usage is as follows.
The basic usage is as follows:
```shell
[GPUS=${GPUS}] sh tools/slurm_train.sh ${PARTITION} ${JOB_NAME} ${CONFIG_FILE} --work-dir ${WORK_DIR}
[GPUS=${GPUS}] sh tools/slurm_train.sh ${PARTITION} ${JOB_NAME} ${CONFIG_FILE} [optional arguments]
```
Below is an example of using 4 GPUs to train PSPNet on a Slurm partition named _dev_, and set the work-dir to some shared file systems.
@ -167,8 +168,21 @@ Below is an example of using 4 GPUs to train PSPNet on a Slurm partition named _
GPUS=4 sh tools/slurm_train.sh dev pspnet configs/pspnet/pspnet_r50-d8_512x1024_40k_cityscapes.py --work-dir work_dir/pspnet
```
You can check [the source code](../../../tools/dist_train.sh) to review full arguments and environment variables.
When using Slurm, the port option need to be set in one of the following ways:
You can check [the source code](../../../tools/slurm_train.sh) to review full arguments and environment variables.
### Testing on a cluster with Slurm
Similar to the training task, MMSegmentation provides `slurm_test.sh` to launch testing jobs.
The basic usage is as follows:
```shell
[GPUS=${GPUS}] sh tools/slurm_test.sh ${PARTITION} ${JOB_NAME} ${CONFIG_FILE} ${CHECKPOINT_FILE} [optional arguments]
```
You can check [the source code](../../../tools/slurm_test.sh) to review full arguments and environment variables.
**Note:** When using Slurm, the port option needs to be set in one of the following ways:
1. Set the port through `--cfg-options`. This is more recommended since it does not change the original configs.
@ -203,90 +217,3 @@ When using Slurm, the port option need to be set in one of the following ways:
CUDA_VISIBLE_DEVICES=0,1,2,3 GPUS=4 MASTER_PORT=29500 sh tools/slurm_train.sh ${PARTITION} ${JOB_NAME} config1.py ${WORK_DIR}
CUDA_VISIBLE_DEVICES=4,5,6,7 GPUS=4 MASTER_PORT=29501 sh tools/slurm_train.sh ${PARTITION} ${JOB_NAME} config2.py ${WORK_DIR}
```
## Test models on standard datasets
We provide testing scripts for evaluating an existing model on the whole dataset.
The following testing environments are supported:
- single GPU
- CPU
- single node multiple GPU
- multiple node
Choose the proper script to perform testing depending on the testing environment.
```shell
# single-gpu testing
python tools/test.py \
${CONFIG_FILE} \
${CHECKPOINT_FILE} \
[--work-dir ${WORK_DIR}] \
[--show ${SHOW_RESULTS}] \
[--show-dir ${VISUALIZATION_DIRECTORY}] \
[--wait-time ${SHOW_INTERVAL}] \
[--cfg-options ${OVERRIDE_CONFIGS}]
# CPU testing
export CUDA_VISIBLE_DEVICES=-1
python tools/test.py \
${CONFIG_FILE} \
${CHECKPOINT_FILE} \
[--work-dir ${WORK_DIR}] \
[--show ${SHOW_RESULTS}] \
[--show-dir ${VISUALIZATION_DIRECTORY}] \
[--wait-time ${SHOW_INTERVAL}] \
[--cfg-options ${OVERRIDE_CONFIGS}]
# multi-gpu testing
bash tools/dist_test.sh \
${CONFIG_FILE} \
${CHECKPOINT_FILE} \
${GPU_NUM} \
[--work-dir ${WORK_DIR}] \
[--cfg-options ${OVERRIDE_CONFIGS}]
```
`tools/dist_test.sh` also supports multi-node testing, but relies on PyTorch's [launch utility](https://pytorch.org/docs/stable/distributed.html#launch-utility).
[Slurm](https://slurm.schedmd.com/) is a good job scheduling system for computing clusters.
On a cluster managed by Slurm, you can use `slurm_test.sh` to spawn testing jobs. It supports both single-node and multi-node testing.
```shell
[GPUS=${GPUS}] ./tools/slurm_test.sh ${PARTITION} ${JOB_NAME} \
${CONFIG_FILE} ${CHECKPOINT_FILE} \
[--work-dir ${OUTPUT_DIRECTORY}] \
[--cfg-options ${OVERRIDE_CONFIGS}]
```
Optional arguments:
- `--work-dir`: If specified, results will be saved in this directory. If not specified, the results will be automatically saved to `work_dirs/{CONFIG_NAME}`.
- `--show`: Show prediction results at runtime, available when `--show-dir` is not specified.
- `--show-dir`: If specified, the visualized segmentation mask will be saved in the specified directory.
- `--wait-time`: The interval of show (s), which takes effect when `--show` is activated. Default to 2.
- `--cfg-options`: If specified, the key-value pair in xxx=yyy format will be merged into config file.
For example: To trade speed with GPU memory, you may pass in `--cfg-options model.backbone.with_cp=True` to enable checkpoint in backbone.
Below is the optional arguments for multi-gpu test:
- `--launcher`: Items for distributed job initialization launcher. Allowed choices are `none`, `pytorch`, `slurm`, `mpi`. Especially, if set to none, it will test in a non-distributed mode.
- `--local_rank`: ID for local rank. If not specified, it will be set to 0.
Examples:
Assume that you have already downloaded the checkpoints to the directory `checkpoints/`.
1. Test PSPNet on PASCAL VOC (without saving the test results) and evaluate the mIoU.
```shell
python tools/test.py configs/pspnet/pspnet_r50-d8_4xb4-20k_voc12aug-512x512.py \
checkpoints/pspnet_r50-d8_512x512_20k_voc12aug_20200617_101958-ed5dfbd9.pth
```
Since `--work-dir` is not specified, the folder `work_dirs/pspnet_r50-d8_4xb4-20k_voc12aug-512x512` will be created automatically to save the evaluation results.
2. Test PSPNet with 4 GPUs, and evaluate the standard mIoU and cityscapes metric.
```shell
./tools/dist_test.sh configs/pspnet/pspnet_r50-d8_4xb2-40k_cityscapes-512x1024.py \
checkpoints/pspnet_r50-d8_512x1024_40k_cityscapes_20200605_003338-2966598c.pth 4
```
:::{note}
There is some gap (~0.1%) between cityscapes mIoU and our mIoU. The reason is that cityscapes average each class with class size by default.
We use the simple version without average for all datasets.
:::

6
mmseg/apis/inference.py
View File

@ -169,12 +169,16 @@ def show_result_pyplot(model: BaseSegmentor,
draw_gt (bool): Whether to draw GT SegDataSample. Default to True.
draw_pred (bool): Whether to draw Prediction SegDataSample.
Defaults to True.
wait_time (float): The interval of show (s). Defaults to 0.
wait_time (float): The interval of show (s). 0 is the special value
that means "forever". Defaults to 0.
show (bool): Whether to display the drawn image.
Default to True.
save_dir (str, optional): Save file dir for all storage backends.
If it is None, the backend storage will not save any data.
out_file (str, optional): Path to output file. Default to None.
Returns:
np.ndarray: the drawn image which channel is RGB.
"""
if hasattr(model, 'module'):
model = model.module