mmsegmentation/projects/hssn/README.md

# HSSN

## Description

Author: AI-Tianlong

This project implements `Deep Hierarchical Semantic Segmentation`  inference on `cityscapes` dataset

## Usage

### Prerequisites

- Python 3.8
- PyTorch 1.6 or higher
- [MMSegmentation](https://github.com/open-mmlab/mmsegmentation) v1.0.0rc5
- mmcv v2.0.0rc4
- mmengine >=0.4.0

### Dataset preparing

Preparing `cityscapes` dataset following this [Dataset Preparing Guide](https://github.com/open-mmlab/mmsegmentation/blob/master/docs/en/dataset_prepare.md#prepare-datasets)

### Testing commands

Please put [`hieraseg_deeplabv3plus_r101-d8_4xb2-80k_cityscapes-512x1024_20230112_125023-bc59a3d1.pth`](https://download.openmmlab.com/mmsegmentation/v0.5/hieraseg/hieraseg_deeplabv3plus_r101-d8_4xb2-80k_cityscapes-512x1024_20230112_125023-bc59a3d1.pth) to `mmsegmentation/checkpoints`

#### Multi-GPUs Test

```bash
# --tta optional, multi-scale test, need mmengine >=0.4.0
bash tools/dist_test.sh [configs] [model weights] [number of gpu]  --tta
```

#### Example

```shell
bash tools/dist_test.sh projects/hssn/configs/hssn/hieraseg_deeplabv3plus_r101-d8_4xb2-80l_cityscapes-512x1024.py checkpoints/hieraseg_deeplabv3plus_r101-d8_4xb2-80k_cityscapes-512x1024_20230112_125023-bc59a3d1.pth 2 --tta
```

## Results

### Cityscapes

|   Method   | Backbone | Crop Size | mIoU  | mIoU (ms+flip) |                                                                               config                                                                               |                                                                             model                                                                             |
| :--------: | :------: | :-------: | :---: | :------------: | :----------------------------------------------------------------------------------------------------------------------------------------------------------------: | :-----------------------------------------------------------------------------------------------------------------------------------------------------------: |
| DeeplabV3+ | R-101-D8 | 512x1024  | 81.61 |     82.71      | [config](https://github.com/open-mmlab/mmsegmentation/tree/main/projects/HieraSeg/configs/hieraseg/hieraseg_deeplabv3plus_r101-d8_4xb2-80l_cityscapes-512x1024.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/hieraseg/hieraseg_deeplabv3plus_r101-d8_4xb2-80k_cityscapes-512x1024_20230112_125023-bc59a3d1.pth) |

<img src="https://user-images.githubusercontent.com/50650583/210488953-e3e35ade-1132-47e1-9dfd-cf12b357ae80.png" width="50%"><img src="https://user-images.githubusercontent.com/50650583/210489746-e35ee229-3234-4292-a649-a8cd85f312ad.png" width="50%">

## Citation

This project is modified from [qhanghu/HSSN_pytorch](https://github.com/qhanghu/HSSN_pytorch)

```bibtex
@article{li2022deep,
  title={Deep Hierarchical Semantic Segmentation},
  author={Li, Liulei and Zhou, Tianfei and Wang, Wenguan and Li, Jianwu and Yang, Yi},
  journal={CVPR},
  year={2022}
}
```

## Checklist

- [x] Milestone 1: PR-ready, and acceptable to be one of the `projects/`.

  - [x] Finish the code

  - [x] Basic docstrings & proper citation

  - [x] Test-time correctness

  - [x] A full README

- [ ] Milestone 2: Indicates a successful model implementation.

  - [ ] Training-time correctness

- [ ] Milestone 3: Good to be a part of our core package!

  - [ ] Type hints and docstrings

  - [ ] Unit tests

  - [ ] Code polishing

  - [ ] Metafile.yml

- [ ] Move your modules into the core package following the codebase's file hierarchy structure.

- [ ] Refactor your modules into the core package following the codebase's file hierarchy structure.
[Fix] Rename and Fix bug of projects HieraSeg (old PR #2444) (#2565) ## Motivation Supplementary PR #2444 Fix tiny bug and add loss_by_feat() to compute loss to train. The inference process have verified to be accurate. ## Modification - modify `sep_aspp_contrast_head.py` , add `loss_by_feat()` function to train(training still has bug, will fix in future😫) - fix testing commands path error `bash tools/dist_test.sh projects/HieraSeg_project/` to `bash tools/dist_test.sh projects/HieraSeg/` at README.md 2023-02-06 18:55:22 +08:00			`# HSSN`
CodeCamp #151[Feature] Support HieraSeg on cityscapes (#2444) ## Support `HieraSeg` interface on `cityscapes` ## Motivation Support `HieraSeg` interface on cityscapes dataset Paper link : https://ieeexplore.ieee.org/document/9878466/ ``` @article{li2022deep, title={Deep Hierarchical Semantic Segmentation}, author={Li, Liulei and Zhou, Tianfei and Wang, Wenguan and Li, Jianwu and Yang, Yi}, journal={CVPR}, year={2022} } ``` ## Modification Add `HieraSeg_Projects` on `projects/` Add `sep_aspp_contrast_head` decoder head. Add `HieraSeg` config. Add `hiera_loss`, `hiera_triplet_loss_cityscape`, `tree_triplet_loss` 2023-01-12 17:12:40 +08:00
			`## Description`

			`Author: AI-Tianlong`

[Fix] Rename and Fix bug of projects HieraSeg (old PR #2444) (#2565) ## Motivation Supplementary PR #2444 Fix tiny bug and add loss_by_feat() to compute loss to train. The inference process have verified to be accurate. ## Modification - modify `sep_aspp_contrast_head.py` , add `loss_by_feat()` function to train(training still has bug, will fix in future😫) - fix testing commands path error `bash tools/dist_test.sh projects/HieraSeg_project/` to `bash tools/dist_test.sh projects/HieraSeg/` at README.md 2023-02-06 18:55:22 +08:00			This project implements `Deep Hierarchical Semantic Segmentation` inference on `cityscapes` dataset
CodeCamp #151[Feature] Support HieraSeg on cityscapes (#2444) ## Support `HieraSeg` interface on `cityscapes` ## Motivation Support `HieraSeg` interface on cityscapes dataset Paper link : https://ieeexplore.ieee.org/document/9878466/ ``` @article{li2022deep, title={Deep Hierarchical Semantic Segmentation}, author={Li, Liulei and Zhou, Tianfei and Wang, Wenguan and Li, Jianwu and Yang, Yi}, journal={CVPR}, year={2022} } ``` ## Modification Add `HieraSeg_Projects` on `projects/` Add `sep_aspp_contrast_head` decoder head. Add `HieraSeg` config. Add `hiera_loss`, `hiera_triplet_loss_cityscape`, `tree_triplet_loss` 2023-01-12 17:12:40 +08:00
			`## Usage`

			`### Prerequisites`

			`- Python 3.8`
			`- PyTorch 1.6 or higher`
[Fix] Rename and Fix bug of projects HieraSeg (old PR #2444) (#2565) ## Motivation Supplementary PR #2444 Fix tiny bug and add loss_by_feat() to compute loss to train. The inference process have verified to be accurate. ## Modification - modify `sep_aspp_contrast_head.py` , add `loss_by_feat()` function to train(training still has bug, will fix in future😫) - fix testing commands path error `bash tools/dist_test.sh projects/HieraSeg_project/` to `bash tools/dist_test.sh projects/HieraSeg/` at README.md 2023-02-06 18:55:22 +08:00			`- [MMSegmentation](https://github.com/open-mmlab/mmsegmentation) v1.0.0rc5`
			`- mmcv v2.0.0rc4`
			`- mmengine >=0.4.0`
CodeCamp #151[Feature] Support HieraSeg on cityscapes (#2444) ## Support `HieraSeg` interface on `cityscapes` ## Motivation Support `HieraSeg` interface on cityscapes dataset Paper link : https://ieeexplore.ieee.org/document/9878466/ ``` @article{li2022deep, title={Deep Hierarchical Semantic Segmentation}, author={Li, Liulei and Zhou, Tianfei and Wang, Wenguan and Li, Jianwu and Yang, Yi}, journal={CVPR}, year={2022} } ``` ## Modification Add `HieraSeg_Projects` on `projects/` Add `sep_aspp_contrast_head` decoder head. Add `HieraSeg` config. Add `hiera_loss`, `hiera_triplet_loss_cityscape`, `tree_triplet_loss` 2023-01-12 17:12:40 +08:00
			`### Dataset preparing`

[Fix] Rename and Fix bug of projects HieraSeg (old PR #2444) (#2565) ## Motivation Supplementary PR #2444 Fix tiny bug and add loss_by_feat() to compute loss to train. The inference process have verified to be accurate. ## Modification - modify `sep_aspp_contrast_head.py` , add `loss_by_feat()` function to train(training still has bug, will fix in future😫) - fix testing commands path error `bash tools/dist_test.sh projects/HieraSeg_project/` to `bash tools/dist_test.sh projects/HieraSeg/` at README.md 2023-02-06 18:55:22 +08:00			Preparing `cityscapes` dataset following this [Dataset Preparing Guide](https://github.com/open-mmlab/mmsegmentation/blob/master/docs/en/dataset_prepare.md#prepare-datasets)
CodeCamp #151[Feature] Support HieraSeg on cityscapes (#2444) ## Support `HieraSeg` interface on `cityscapes` ## Motivation Support `HieraSeg` interface on cityscapes dataset Paper link : https://ieeexplore.ieee.org/document/9878466/ ``` @article{li2022deep, title={Deep Hierarchical Semantic Segmentation}, author={Li, Liulei and Zhou, Tianfei and Wang, Wenguan and Li, Jianwu and Yang, Yi}, journal={CVPR}, year={2022} } ``` ## Modification Add `HieraSeg_Projects` on `projects/` Add `sep_aspp_contrast_head` decoder head. Add `HieraSeg` config. Add `hiera_loss`, `hiera_triplet_loss_cityscape`, `tree_triplet_loss` 2023-01-12 17:12:40 +08:00
			`### Testing commands`

[Fix] Rename and Fix bug of projects HieraSeg (old PR #2444) (#2565) ## Motivation Supplementary PR #2444 Fix tiny bug and add loss_by_feat() to compute loss to train. The inference process have verified to be accurate. ## Modification - modify `sep_aspp_contrast_head.py` , add `loss_by_feat()` function to train(training still has bug, will fix in future😫) - fix testing commands path error `bash tools/dist_test.sh projects/HieraSeg_project/` to `bash tools/dist_test.sh projects/HieraSeg/` at README.md 2023-02-06 18:55:22 +08:00			Please put [`hieraseg_deeplabv3plus_r101-d8_4xb2-80k_cityscapes-512x1024_20230112_125023-bc59a3d1.pth`](https://download.openmmlab.com/mmsegmentation/v0.5/hieraseg/hieraseg_deeplabv3plus_r101-d8_4xb2-80k_cityscapes-512x1024_20230112_125023-bc59a3d1.pth) to `mmsegmentation/checkpoints`
CodeCamp #151[Feature] Support HieraSeg on cityscapes (#2444) ## Support `HieraSeg` interface on `cityscapes` ## Motivation Support `HieraSeg` interface on cityscapes dataset Paper link : https://ieeexplore.ieee.org/document/9878466/ ``` @article{li2022deep, title={Deep Hierarchical Semantic Segmentation}, author={Li, Liulei and Zhou, Tianfei and Wang, Wenguan and Li, Jianwu and Yang, Yi}, journal={CVPR}, year={2022} } ``` ## Modification Add `HieraSeg_Projects` on `projects/` Add `sep_aspp_contrast_head` decoder head. Add `HieraSeg` config. Add `hiera_loss`, `hiera_triplet_loss_cityscape`, `tree_triplet_loss` 2023-01-12 17:12:40 +08:00
			`#### Multi-GPUs Test`

			```bash
			`# --tta optional, multi-scale test, need mmengine >=0.4.0`
			`bash tools/dist_test.sh [configs] [model weights] [number of gpu] --tta`
			```

			`#### Example`

			```shell
[Fix] Rename and Fix bug of projects HieraSeg (old PR #2444) (#2565) ## Motivation Supplementary PR #2444 Fix tiny bug and add loss_by_feat() to compute loss to train. The inference process have verified to be accurate. ## Modification - modify `sep_aspp_contrast_head.py` , add `loss_by_feat()` function to train(training still has bug, will fix in future😫) - fix testing commands path error `bash tools/dist_test.sh projects/HieraSeg_project/` to `bash tools/dist_test.sh projects/HieraSeg/` at README.md 2023-02-06 18:55:22 +08:00			`bash tools/dist_test.sh projects/hssn/configs/hssn/hieraseg_deeplabv3plus_r101-d8_4xb2-80l_cityscapes-512x1024.py checkpoints/hieraseg_deeplabv3plus_r101-d8_4xb2-80k_cityscapes-512x1024_20230112_125023-bc59a3d1.pth 2 --tta`
CodeCamp #151[Feature] Support HieraSeg on cityscapes (#2444) ## Support `HieraSeg` interface on `cityscapes` ## Motivation Support `HieraSeg` interface on cityscapes dataset Paper link : https://ieeexplore.ieee.org/document/9878466/ ``` @article{li2022deep, title={Deep Hierarchical Semantic Segmentation}, author={Li, Liulei and Zhou, Tianfei and Wang, Wenguan and Li, Jianwu and Yang, Yi}, journal={CVPR}, year={2022} } ``` ## Modification Add `HieraSeg_Projects` on `projects/` Add `sep_aspp_contrast_head` decoder head. Add `HieraSeg` config. Add `hiera_loss`, `hiera_triplet_loss_cityscape`, `tree_triplet_loss` 2023-01-12 17:12:40 +08:00			```

			`## Results`

			`### Cityscapes`

[Doc] Refine doc and fix links (#2821) ## Motivation - Create the `main` branch ## Modification Modify links from `dev-1.x` to `main` 2023-03-31 16:26:30 +08:00			`\| Method \| Backbone \| Crop Size \| mIoU \| mIoU (ms+flip) \| config \| model \|`
			`\| :--------: \| :------: \| :-------: \| :---: \| :------------: \| :----------------------------------------------------------------------------------------------------------------------------------------------------------------: \| :-----------------------------------------------------------------------------------------------------------------------------------------------------------: \|`
			`\| DeeplabV3+ \| R-101-D8 \| 512x1024 \| 81.61 \| 82.71 \| [config](https://github.com/open-mmlab/mmsegmentation/tree/main/projects/HieraSeg/configs/hieraseg/hieraseg_deeplabv3plus_r101-d8_4xb2-80l_cityscapes-512x1024.py) \| [model](https://download.openmmlab.com/mmsegmentation/v0.5/hieraseg/hieraseg_deeplabv3plus_r101-d8_4xb2-80k_cityscapes-512x1024_20230112_125023-bc59a3d1.pth) \|`
CodeCamp #151[Feature] Support HieraSeg on cityscapes (#2444) ## Support `HieraSeg` interface on `cityscapes` ## Motivation Support `HieraSeg` interface on cityscapes dataset Paper link : https://ieeexplore.ieee.org/document/9878466/ ``` @article{li2022deep, title={Deep Hierarchical Semantic Segmentation}, author={Li, Liulei and Zhou, Tianfei and Wang, Wenguan and Li, Jianwu and Yang, Yi}, journal={CVPR}, year={2022} } ``` ## Modification Add `HieraSeg_Projects` on `projects/` Add `sep_aspp_contrast_head` decoder head. Add `HieraSeg` config. Add `hiera_loss`, `hiera_triplet_loss_cityscape`, `tree_triplet_loss` 2023-01-12 17:12:40 +08:00
			`<img src="https://user-images.githubusercontent.com/50650583/210488953-e3e35ade-1132-47e1-9dfd-cf12b357ae80.png" width="50%"><img src="https://user-images.githubusercontent.com/50650583/210489746-e35ee229-3234-4292-a649-a8cd85f312ad.png" width="50%">`

			`## Citation`

			`This project is modified from [qhanghu/HSSN_pytorch](https://github.com/qhanghu/HSSN_pytorch)`

			```bibtex
			`@article{li2022deep,`
			`title={Deep Hierarchical Semantic Segmentation},`
			`author={Li, Liulei and Zhou, Tianfei and Wang, Wenguan and Li, Jianwu and Yang, Yi},`
			`journal={CVPR},`
			`year={2022}`
			`}`
			```

			`## Checklist`

			- [x] Milestone 1: PR-ready, and acceptable to be one of the `projects/`.

			`- [x] Finish the code`

			`- [x] Basic docstrings & proper citation`

			`- [x] Test-time correctness`

			`- [x] A full README`

			`- [ ] Milestone 2: Indicates a successful model implementation.`

			`- [ ] Training-time correctness`

			`- [ ] Milestone 3: Good to be a part of our core package!`

			`- [ ] Type hints and docstrings`

			`- [ ] Unit tests`

			`- [ ] Code polishing`

			`- [ ] Metafile.yml`

			`- [ ] Move your modules into the core package following the codebase's file hierarchy structure.`

			`- [ ] Refactor your modules into the core package following the codebase's file hierarchy structure.`