angiecao
608e319eb6
[Feature] Support Side Adapter Network ( #3232 )
...
## Motivation
Support SAN for Open-Vocabulary Semantic Segmentation
Paper: [Side Adapter Network for Open-Vocabulary Semantic
Segmentation](https://arxiv.org/abs/2302.12242 )
official Code: [SAN](https://github.com/MendelXu/SAN )
## Modification
- Added the parameters of backbone vit for implementing the image
encoder of CLIP.
- Added text encoder code.
- Added segmentor multimodel encoder-decoder code for open-vocabulary
semantic segmentation.
- Added SideAdapterNetwork decode head code.
- Added config files for train and inference.
- Added tools for converting pretrained models.
- Added loss implementation for mask classification model, such as SAN,
Maskformer and remove dependency on mmdetection.
- Added test units for text encoder, multimodel encoder-decoder, san
decode head and hungarian_assigner.
## Use cases
### Convert Models
**pretrained SAN model**
The official pretrained model can be downloaded from
[san_clip_vit_b_16.pth](https://huggingface.co/Mendel192/san/blob/main/san_vit_b_16.pth )
and
[san_clip_vit_large_14.pth](https://huggingface.co/Mendel192/san/blob/main/san_vit_large_14.pth ).
Use tools/model_converters/san2mmseg.py to convert offcial model into
mmseg style.
`python tools/model_converters/san2mmseg.py <MODEL_PATH> <OUTPUT_PATH>`
**pretrained CLIP model**
Use the CLIP model provided by openai to train SAN. The CLIP model can
be download from
[ViT-B-16.pt](https://openaipublic.azureedge.net/clip/models/5806e77cd80f8b59890b7e101eabd078d9fb84e6937f9e85e4ecb61988df416f/ViT-B-16.pt )
and
[ViT-L-14-336px.pt](https://openaipublic.azureedge.net/clip/models/3035c92b350959924f9f00213499208652fc7ea050643e8b385c2dac08641f02/ViT-L-14-336px.pt ).
Use tools/model_converters/clip2mmseg.py to convert model into mmseg
style.
`python tools/model_converters/clip2mmseg.py <MODEL_PATH> <OUTPUT_PATH>`
### Inference
test san_vit-base-16 model on coco-stuff164k dataset
`python tools/test.py
./configs/san/san-vit-b16_coco-stuff164k-640x640.py
<TRAINED_MODEL_PATH>`
### Train
test san_vit-base-16 model on coco-stuff164k dataset
`python tools/train.py
./configs/san/san-vit-b16_coco-stuff164k-640x640.py --cfg-options
model.pretrained=<PRETRAINED_MODEL_PATH>`
## Comparision Results
### Train on COCO-Stuff164k
| | | mIoU | mAcc | pAcc |
| --------------- | ----- | ----- | ----- | ----- |
| san-vit-base16 | official | 41.93 | 56.73 | 67.69 |
| | mmseg | 41.93 | 56.84 | 67.84 |
| san-vit-large14 | official | 45.57 | 59.52 | 69.76 |
| | mmseg | 45.78 | 59.61 | 69.21 |
### Evaluate on Pascal Context
| | | mIoU | mAcc | pAcc |
| --------------- | ----- | ----- | ----- | ----- |
| san-vit-base16 | official | 54.05 | 72.96 | 77.77 |
| | mmseg | 54.04 | 73.74 | 77.71 |
| san-vit-large14 | official | 57.53 | 77.56 | 78.89 |
| | mmseg | 56.89 | 76.96 | 78.74 |
### Evaluate on Voc12Aug
| | | mIoU | mAcc | pAcc |
| --------------- | ----- | ----- | ----- | ----- |
| san-vit-base16 | official | 93.86 | 96.61 | 97.11 |
| | mmseg | 94.58 | 97.01 | 97.38 |
| san-vit-large14 | official | 95.17 | 97.61 | 97.63 |
| | mmseg | 95.58 | 97.75 | 97.79 |
---------
Co-authored-by: CastleDream <35064479+CastleDream@users.noreply.github.com>
Co-authored-by: yeedrag <46050186+yeedrag@users.noreply.github.com>
Co-authored-by: Yang-ChangHui <71805205+Yang-Changhui@users.noreply.github.com>
Co-authored-by: Xu CAO <49406546+SheffieldCao@users.noreply.github.com>
Co-authored-by: xiexinch <xiexinch@outlook.com>
Co-authored-by: 小飞猪 <106524776+ooooo-create@users.noreply.github.com>
2023-09-20 21:20:26 +08:00
谢昕辰
c55d7a721e
[Feature] support mim download dataset ( #3089 )
...
## Motivation
Please describe the motivation of this PR and the goal you want to
achieve through this PR.
## Modification
- add dataset-index.yml
## Dependencies
- [ ] https://github.com/open-mmlab/mim/pull/212
2023-06-28 17:57:22 +08:00
Lai Quang Huy
8721d44724
[Fix] Fix URL so output of mim list looks better ( #2902 )
2023-04-19 15:50:19 +08:00
MengzhangLI
90c816b6de
[Enchancement] Delete mmcls in runtime.txt in dev-1.x ( #2368 )
...
* [Enchancement] Delete mmcls in runtime.txt in dev-1.x
* fix mmcv minimal version
2022-12-01 16:35:22 +08:00
谢昕辰
230246f557
[Refactor] Add pyupgrade pre-commit hook ( #2078 )
...
* add pyupgrade hook
* run pyupgrade precommit hook
2022-09-19 14:06:29 +08:00
yancong
60f0415932
chore(setup.py): add mim extras_requires ( #2012 )
2022-09-01 17:30:54 +08:00
MengzhangLI
2f4d52f4d1
remove pytest runner and update package info ( #1201 )
2022-01-13 10:37:48 +08:00
Junjun2016
24b4761d83
[Fix] Fix symlink failure on Windows ( #1038 )
...
* fix symlink failure on Windows
* fix develop mode on windows
2021-12-08 23:19:23 -08:00
MengzhangLI
78fbaaf179
0~ ( #1090 )
2021-11-30 23:03:50 +08:00
Junjun2016
6762958ea0
[Enhancement] Upgrade CI for PyTorch1.9 ( #712 )
...
* Upgrade CI for PyTorch1.9
* fix typo
* merge build_cu101 and build_cu102 to build_cuda
* fix substitution bug
* test
* test
* test
* test
* test
* test
* test
* test
* test
* test
* test
* test
* remove redudant items
* add python 3.9 to CI
* add python 3.9 to setup.py
* remove some versions of pytorch for python 3.9 test
* fix torch version error
* delete mim files
* update ubuntu version
* fix mmcv cuda version
2021-09-08 18:43:17 -07:00
Junjun2016
42a1929779
Add MMSeg citation ( #825 )
...
* fix typo
* add citation
2021-09-01 18:38:58 -07:00
Junjun2016
2fe0bddf5e
[Dcos] Add header for files ( #796 )
...
* Add header for files
* Delete header in config files
2021-08-16 23:16:55 -07:00
谢昕辰
e0a1860369
[Feature] support mim ( #717 )
...
* support mim
* remove mim demo
2021-07-27 15:43:32 +08:00
谢昕辰
725d5aa002
[Feature] support mim ( #549 )
...
* dice loss
* format code, add docstring and calculate denominator without valid_mask
* minor change
* restore
* add metafile
* add manifest.in and add config at setup.py
* add requirements
* modify manifest
* modify manifest
* Update MANIFEST.in
* add metafile
* add metadata
* fix typo
* Update metafile.yml
* Update metafile.yml
* minor change
* Update metafile.yml
* add subfix
* fix mmshow
* add more metafile
* add config to model_zoo
* fix bug
* Update mminstall.txt
* [fix] Add models
* [Fix] Add collections
* [fix] Modify collection name
* [Fix] Set datasets to unet metafile
* [Fix] Modify collection names
* complement inference time
2021-05-31 15:07:24 -07:00
Jerry Jiarui XU
0f702f44af
Auto get version info and git hash ( #55 )
...
* Auto get version info and git hash
* bump 0.5.1 and update doc
* fixed docs
* Add change log
2020-08-11 19:23:35 +08:00
Jerry Jiarui XU
fd34179921
Fixed long description ( #16 )
2020-07-14 19:35:25 +08:00
Jerry Jiarui XU
0de0387b1f
fixed name in pypi, add badge ( #15 )
...
* fixed name in pypi, add badge
* fixed name in install.md
* update install.md
2020-07-14 14:41:52 +08:00
Kai Chen
bc285bea28
Add pypi deployment ( #11 )
...
* add pypi deployment
* remove useless jobs
* fixed typo and cleanup
* delete build.tx
* add change log
Co-authored-by: Jiarui XU <xvjiarui0826@gmail.com>
2020-07-13 20:54:32 +08:00
Jiarui XU
b2724da80b
init commit
2020-07-10 02:39:01 +08:00
