Commit Graph

118 Commits (test-1.2.0)

Author SHA1 Message Date
angiecao 608e319eb6
[Feature] Support Side Adapter Network (#3232)
## Motivation
Support SAN for Open-Vocabulary Semantic Segmentation
Paper: [Side Adapter Network for Open-Vocabulary Semantic
Segmentation](https://arxiv.org/abs/2302.12242)
official Code: [SAN](https://github.com/MendelXu/SAN)

## Modification
- Added the parameters of backbone vit for implementing the image
encoder of CLIP.
- Added text encoder code.
- Added segmentor multimodel encoder-decoder code for open-vocabulary
semantic segmentation.
- Added SideAdapterNetwork decode head code.
- Added config files for train and inference.
- Added tools for converting pretrained models.
- Added loss implementation for mask classification model, such as SAN,
Maskformer and remove dependency on mmdetection.
- Added test units for text encoder, multimodel encoder-decoder, san
decode head and hungarian_assigner.

## Use cases
### Convert Models
**pretrained SAN model**
The official pretrained model can be downloaded from
[san_clip_vit_b_16.pth](https://huggingface.co/Mendel192/san/blob/main/san_vit_b_16.pth)
and
[san_clip_vit_large_14.pth](https://huggingface.co/Mendel192/san/blob/main/san_vit_large_14.pth).
Use tools/model_converters/san2mmseg.py to convert offcial model into
mmseg style.
`python tools/model_converters/san2mmseg.py <MODEL_PATH> <OUTPUT_PATH>`

**pretrained CLIP model**
Use the CLIP model provided by openai to train SAN. The CLIP model can
be download from
[ViT-B-16.pt](https://openaipublic.azureedge.net/clip/models/5806e77cd80f8b59890b7e101eabd078d9fb84e6937f9e85e4ecb61988df416f/ViT-B-16.pt)
and
[ViT-L-14-336px.pt](https://openaipublic.azureedge.net/clip/models/3035c92b350959924f9f00213499208652fc7ea050643e8b385c2dac08641f02/ViT-L-14-336px.pt).
Use tools/model_converters/clip2mmseg.py to convert model into mmseg
style.
`python tools/model_converters/clip2mmseg.py <MODEL_PATH> <OUTPUT_PATH>`

### Inference
test san_vit-base-16 model on coco-stuff164k dataset
`python tools/test.py
./configs/san/san-vit-b16_coco-stuff164k-640x640.py
<TRAINED_MODEL_PATH>`

### Train
test san_vit-base-16 model on coco-stuff164k dataset
`python tools/train.py
./configs/san/san-vit-b16_coco-stuff164k-640x640.py --cfg-options
model.pretrained=<PRETRAINED_MODEL_PATH>`

## Comparision Results
### Train on COCO-Stuff164k
|                 |       | mIoU  | mAcc  | pAcc  |
| --------------- | ----- | ----- | ----- | ----- |
| san-vit-base16  | official  | 41.93 | 56.73 | 67.69 |
|                 | mmseg | 41.93 | 56.84 | 67.84 |
| san-vit-large14 | official  | 45.57 | 59.52 | 69.76 |
|                 | mmseg | 45.78 | 59.61 | 69.21 |

### Evaluate on Pascal Context
|                 |       | mIoU  | mAcc  | pAcc  |
| --------------- | ----- | ----- | ----- | ----- |
| san-vit-base16  | official  | 54.05 | 72.96 | 77.77 |
|                 | mmseg | 54.04 | 73.74 | 77.71 |
| san-vit-large14 | official  | 57.53 | 77.56 | 78.89 |
|                 | mmseg | 56.89 | 76.96 | 78.74 |

### Evaluate on Voc12Aug
|                 |       | mIoU  | mAcc  | pAcc  |
| --------------- | ----- | ----- | ----- | ----- |
| san-vit-base16  | official  | 93.86 | 96.61 | 97.11 |
|                 | mmseg | 94.58 | 97.01 | 97.38 |
| san-vit-large14 | official  | 95.17 | 97.61 | 97.63 |
|                 | mmseg | 95.58 | 97.75 | 97.79 |

---------

Co-authored-by: CastleDream <35064479+CastleDream@users.noreply.github.com>
Co-authored-by: yeedrag <46050186+yeedrag@users.noreply.github.com>
Co-authored-by: Yang-ChangHui <71805205+Yang-Changhui@users.noreply.github.com>
Co-authored-by: Xu CAO <49406546+SheffieldCao@users.noreply.github.com>
Co-authored-by: xiexinch <xiexinch@outlook.com>
Co-authored-by: 小飞猪 <106524776+ooooo-create@users.noreply.github.com>
2023-09-20 21:20:26 +08:00
Peng Lu 9c45a94cee
[Fix] fix import error raised by ldm (#3338) 2023-09-20 12:45:05 +08:00
zhen6618 56a40d78ab
Use the pytorch-grad-cam tool to visualize Class Activation Maps (CAM) (#3324)
Thanks for your contribution and we appreciate it a lot. The following
instructions would make your pull request more healthy and more easily
get feedback. If you do not understand some items, don't worry, just
make the pull request and seek help from maintainers.

## Motivation

Use the pytorch-grad-cam tool to visualize Class Activation Maps (CAM).

## Modification

Use the pytorch-grad-cam tool to visualize Class Activation Maps (CAM).

requirement: pip install grad-cam

run commad: python tools/analysis_tools/visualization_cam.py

## BC-breaking (Optional)

Does the modification introduce changes that break the
backward-compatibility of the downstream repos?
If so, please describe how it breaks the compatibility and how the
downstream projects should modify their code to keep compatibility with
this PR.

## Use cases (Optional)

If this PR introduces a new feature, it is better to list some use cases
here, and update the documentation.

## Checklist

1. Pre-commit or other linting tools are used to fix the potential lint
issues.
2. The modification is covered by complete unit tests. If not, please
add more unit test to ensure the correctness.
3. The documentation has been modified accordingly, like docstring or
example tutorials.
2023-09-20 11:08:47 +08:00
Gorgeous ebd5695104
[Fix] Update confusion_matrix.py (#3291)
## Motivation



## Modification

The confusion_matrix.py is not compatible with the current version of
mmseg.

---------

Co-authored-by: xiexinch <xiexinch@outlook.com>
2023-08-31 12:53:33 +08:00
Peng Lu 788b37f78f
[Feature] Support NYU depth estimation dataset (#3269)
Thanks for your contribution and we appreciate it a lot. The following
instructions would make your pull request more healthy and more easily
get feedback. If you do not understand some items, don't worry, just
make the pull request and seek help from maintainers.

## Motivation

Please describe the motivation of this PR and the goal you want to
achieve through this PR.

## Modification

Please briefly describe what modification is made in this PR.
1. add `NYUDataset`class
2. add script to process NYU dataset
3. add transforms for loading depth map
4. add docs & unittest

## BC-breaking (Optional)

Does the modification introduce changes that break the
backward-compatibility of the downstream repos?
If so, please describe how it breaks the compatibility and how the
downstream projects should modify their code to keep compatibility with
this PR.

## Use cases (Optional)

If this PR introduces a new feature, it is better to list some use cases
here, and update the documentation.

## Checklist

1. Pre-commit or other linting tools are used to fix the potential lint
issues.
5. The modification is covered by complete unit tests. If not, please
add more unit test to ensure the correctness.
6. If the modification has potential influence on downstream projects,
this PR should be tested with downstream projects, like MMDet or
MMDet3D.
7. The documentation has been modified accordingly, like docstring or
example tutorials.
2023-08-17 11:39:44 +08:00
谢昕辰 5fc197900f
[Fix] Update ddrnet readme (#3198) 2023-07-14 11:16:16 +08:00
Kedreamix b6ec4ab1e6
Fixes an issue in isaid.py (#3010)
## Motivation

When processing data in the isaid experiment, generated images only have
binary pixel values of 0 or 1 instead of the corresponding class values.
This causes significant interference and prevents subsequent experiments
from proceeding. After investigation, it was found that the issue was
caused by using the wrong image format for saving the images. Saving the
images as PNG resulted in binary pixel values, while saving the images
as BMP resolved the issue and correctly saved the class values.

## Modification

`img_patch.save(save_path_image, format='BMP')`

This code will save the image data as BMP format.

## BC-breaking 

Confirm that the modification does not introduce new issues and test
that the modified code successfully resolves the original problem.

---------

Co-authored-by: 谢昕辰 <xiexinch@outlook.com>
Co-authored-by: CSH <40987381+csatsurnh@users.noreply.github.com>
2023-05-30 14:44:05 +08:00
zoulinxin 77591b9e7b
[Feature] Add GDAL backend and Support LEVIR-CD Dataset (#2903)
## Motivation

For support with reading multiple remote sensing image formats, please
refer to https://gdal.org/drivers/raster/index.html.

Byte, UInt16, Int16, UInt32, Int32, Float32, Float64, CInt16, CInt32,
CFloat32 and CFloat64 are supported for reading and writing.

Support input of two images for change detection tasks, and support the
LEVIR-CD dataset.

## Modification

Add LoadSingleRSImageFromFile in 'mmseg/datasets/transforms/loading.py'.
Load a single remote sensing image for object segmentation tasks.

Add LoadMultipleRSImageFromFile in
'mmseg/datasets/transforms/loading.py'.
Load two remote sensing images for change detection tasks.

Add ConcatCDInput  in 'mmseg/datasets/transforms/transforms.py'.
Combine images that have been separately augmented for data enhancement.

Add BaseCDDataset in 'mmseg/datasets/basesegdataset.py'
Base class for datasets used in change detection tasks.

---------

Co-authored-by: xiexinch <xiexinch@outlook.com>
2023-05-08 20:09:33 +08:00
Miguel Méndez 01d40174e1
Fix typo in tools/test.py (#2961) 2023-05-05 11:30:07 +08:00
谢昕辰 23fb3a9812
[Fix] Fix accepting an unexpected argument `local-rank` in PyTorch 2.0 (#2812) 2023-03-30 16:43:15 +08:00
Miao Zheng ff95416c3b
[Features]Support dump segment predition (#2712)
## Motivation

1. It is used to save the segmentation predictions as files and upload
these files to a test server

## Modification

1. Add output_file and format only in `IoUMetric`
 
## BC-breaking (Optional)

No

## Use cases (Optional)

If this PR introduces a new feature, it is better to list some use cases
here, and update the documentation.

## Checklist

1. Pre-commit or other linting tools are used to fix the potential lint
issues.
3. The modification is covered by complete unit tests. If not, please
add more unit test to ensure the correctness.
4. If the modification has potential influence on downstream projects,
this PR should be tested with downstream projects, like MMDet or
MMDet3D.
5. The documentation has been modified accordingly, like docstring or
example tutorials.
2023-03-17 22:58:08 +08:00
谢昕辰 45fae72de5
[Feature] Support calculating FLOPs of segmentors (#2706)
## Motivation

fix compute flops problems

## Modification

Please briefly describe what modification is made in this PR.
2023-03-10 19:25:47 +08:00
jinxianwei eca61d3cda
tools/analysis_tools browse_dataset.py (#2649)
## Motivation

browse_dataset before training

## Modification

create tools/analysis_tools/browse_dataset.py
2023-02-27 12:02:48 +08:00
谢昕辰 039ba5d4ca
[Feature] Support auto import modules from registry. (#2481)
## Motivation

The registry now supports auto-import modules from the given location.

register_all_modules before running is no longer needed. The modules
will be lazy-imported during building.

- [x] This PR can be merged after
https://github.com/open-mmlab/mmengine/pull/643. The MMEngine version
should be updated.

Ref: https://github.com/open-mmlab/mmdetection/pull/9143
2023-02-23 20:33:17 +08:00
Andrew Lau 49b062e365
CodeCamp #139 [Feature] Support REFUGE dataset. (#2554)
## Motivation 
Add REFUGE datasets
Old PR: https://github.com/open-mmlab/mmsegmentation/pull/2420

---------

Co-authored-by: MengzhangLI <mcmong@pku.edu.cn>
2023-02-03 16:02:19 +08:00
王永韬 2d67e51db3
CodeCamp #140 [New] [Feature] Add synapse dataset and data augmentation in dev-1.x. (#2432)
## Motivation

Add Synapse dataset in MMSegmentation.
Old PR: https://github.com/open-mmlab/mmsegmentation/pull/2372.
2023-01-06 16:14:54 +08:00
谢昕辰 da4125587e [Refactor] Support TTA (#2184)
* tta init

* use mmcv transform

* test city

* add multiscale

* fix merge

* add softmax to post process

* add ut

* add tta pipeline to other datasets

* remove softmax

* add encoder_decoder_tta ut

* add encoder_decoder_tta ut

* rename

* rename file

* rename config

* rm aug_test

* move flip to post process

* fix channel
2022-12-30 22:52:07 +08:00
谢昕辰 230246f557
[Refactor] Add pyupgrade pre-commit hook (#2078)
* add pyupgrade hook

* run pyupgrade precommit hook
2022-09-19 14:06:29 +08:00
Miao Zheng 3388cfd475
[Refactor]Resume do not specify ckpt (#2074)
* [Refactor]Resume do not specify ckpt

* add default false value
2022-09-15 15:11:31 +08:00
Miao Zheng ef111dff8b
[Fix] browse dataset (#2005) 2022-08-31 18:08:45 +08:00
Miao Zheng 2724d47f71
[Fix] Add out-file in demo and show_result_pyplot (#2004) 2022-08-31 17:44:41 +08:00
Miao Zheng aed1886724
[Fix] benckmark script (#1991) 2022-08-29 18:51:24 +08:00
Miao Zheng 8de0050f25
[Refactor] data flow (#1956)
* [WIP] Refactor data flow

* model return

* [WIP] Refactor data flow

* support data_samples is optional

* fix benchmark

* fix base

* minors

* rebase

* fix api

* ut

* fix api inference

* comments

* docstring

* docstring

* docstring

* fix bug of slide inference

* add assert c > 1
2022-08-26 15:54:23 +08:00
Miao Zheng 50546da85c
[Fix]Remove modules from mmcv.runner and mmcv.utils (#1966)
* [WIP] mmcv-clean

* [WIP]Remove modules from mmcv.runner and mmcv.utils

* wip

* fix import mmengine

* remove ut

* loadcheckpoint in mae
2022-08-25 15:15:21 +08:00
谢昕辰 13b108dbc8 [Enhancement] Support custom runner (#1923)
* support custom runner

* fix
2022-08-19 20:50:03 +08:00
谢昕辰 671ed74332 fix logdict (#1892) 2022-08-19 20:50:02 +08:00
Miao Zheng e0499d5a77 [Fix] Fix repo based on refactoring standard (#1869)
* [Fix] Fix repo based on refactory standard

* fix ut
2022-08-19 20:50:02 +08:00
谢昕辰 1ded0a4278
[Enhancement] Replace MMCV with MMEngine in convert model scripts (#1798)
* replace mmcv with mmengine

* remove transforms
2022-07-27 17:54:37 +08:00
MengzhangLI 6873f9ece8
[Feature] Add SegVisualizer (#1792)
* [Feature] Add SegVisualizer

* change name to visualizer_example

* fix typo

* refactor folder structure
2022-07-27 16:28:09 +08:00
Miao Zheng 4de57b49c5
[Fix] Fix register_all_modules and imports (#1793)
* [Fix] Fix register_all_modules and imports

* fix transforms import
2022-07-21 22:44:42 +08:00
zhengmiao 4b76f277a6 [Refactory] MMSegmentation Content 2022-07-15 15:47:29 +00:00
zhengmiao fba91957c0 Merge branch 'xiexinchen/amp_train' into 'refactor_dev'
[Enhancement] support '--amp' option

See merge request openmmlab-enterprise/openmmlab-ce/mmsegmentation!68
2022-07-14 07:29:35 +00:00
xiexinchen.vendor 2a138b47c4 [Refactor] Fix analyze log script 2022-07-14 07:28:44 +00:00
xiexinch 741190a864 support '--amp' option 2022-07-14 14:21:32 +08:00
xiexinchen.vendor 0fc9071703 [Refactor] Fix benchmark script 2022-07-14 05:34:08 +00:00
xiexinch c1c870af31 rename init/inference_segmentor to init/inference_model 2022-07-12 16:19:59 +08:00
limengzhang.vendor f2bac79f03 [Refactor] Refactor DatasetWrapper 2022-06-27 14:36:18 +00:00
zhengmiao b23982d1b3 [Refactory] Refactory BaseSegmentor based BaseModel 2022-06-19 06:32:09 +00:00
linfangjian.vendor 3c54de06bb [Refactor] Refactor train test interface. 2022-06-09 03:01:43 +00:00
Miao Zheng 0644b3cd30 [Enhancement] Delete seed in dist_train.sh (#1519) 2022-04-28 11:35:10 +08:00
Miao Zheng f50bfe3e3d [Enhance] Add extra dataloader settings in configs (#1435)
* [Enhance] Add extra dataloader settings in configs

* val default samples

* val default samples

* del unuse

* del unused
2022-04-13 13:30:10 +08:00
Pan Zhang cba10b3f15 [fix] fix image save path bug in Windows (#1423)
* [fix] fix image save path bug in Windows

* fix lint error
2022-04-09 21:17:51 +08:00
MengzhangLI 549616888e [Fix] Fix bug in `tools/analyse_logs.py` caused by wrong plot_iter in some cases. (#1428)
* fix_analyze_logs

* fix comments

* fix comments
2022-04-05 12:47:41 +08:00
FangjianLin d1b8eae35e [Feature] Add BEiT backbone (#1404)
* [Feature] Add BEiT backbone

* fix

* fix

* fix

* fix

* add readme

* fix

* fix

* fix

* fix

* fix

* add link

* fix memory

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix test_beit.py

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix
2022-03-30 15:25:10 +08:00
Cody Wong cfc080c9b1 [Fix] make arguments effective in tools/confusion_matrix.py (#1401)
* add an argument for customizing `title' of the output figure

* fix `color_theme' arguments not passing to plot function

Signed-off-by: code14 <mob5566@gmail.com>
2022-03-21 10:51:27 +08:00
FangjianLin 415b20febd [Feature] Add multi machine `dist_train`. (#1383)
* Add training startup documentation

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix
2022-03-18 10:55:49 +08:00
wangliujun 912d109dbe Update get_flops.py (#1371)
* Update get_flops.py

argparse.ArgumentParser(description='Get the FLOPs of a segmentor')

* Update get_flops.py
2022-03-14 21:02:43 +08:00
Rockey da6bb2c8c5 [Feature]: Add diff seeds to diff ranks and set torch seed in worker_init_fn (#1362) 2022-03-10 09:41:16 +08:00
Hyeokjoon Kwon 6d124aa091 [Feature] Generating and plotting confusion matrix (#1301)
* generate and plot confusion matrix

* fix typo

* add usage and examples for confusion matrix

* deal with nan values(pick pr#7147 mmdet)

* fix md format
2022-03-03 16:34:33 +08:00
Miao Zheng 4d00877b4c Add deprecation message for deploy tool (#1321)
* Add deprecation message for deploy tool

* deploy test
2022-02-25 11:22:03 +08:00