mmsegmentation

Siddharth Ancha 190063fbb4

[Fix] Fix reduce_zero_label in evaluation (#2504 )

## Motivation

Through this PR, I (1) fix a bug, and (2) perform some associated cleanup, and (3) add a unit test. The bug occurs during evaluation when two options -- `reduce_zero_label=True`, and custom classes are used. The bug was that the `reduce_zero_label` is not properly propagated (see details below).

## Modification

1. **Bugfix**

The bug occurs [in the initialization of `CustomDataset`](5d49918b3c/mmseg/datasets/custom.py (L108-L110)) where the `reduce_zero_label` flag is not propagated to its member `self.gt_seg_map_loader_cfg`:

```python
self.gt_seg_map_loader = LoadAnnotations(
) if gt_seg_map_loader_cfg is None else LoadAnnotations(
    **gt_seg_map_loader_cfg)
```

Because the `reduce_zero_label` flag was not being propagated, the zero label reduction was being [unnecessarily and explicitly duplicated during the evaluation](5d49918b3c/mmseg/core/evaluation/metrics.py (L66-L69)).

As pointed in a previous PR (#2500), `reduce_zero_label` must occur before applying the `label_map`. Due to this bug, the order gets reversed when both features are used simultaneously.

This has been fixed to:

```python
self.gt_seg_map_loader = LoadAnnotations(
    reduce_zero_label=reduce_zero_label, **gt_seg_map_loader_cfg)
```

2. **Cleanup**

Due to the bug fix, since both `reduce_zero_label` and `label_map` are being applied in `get_gt_seg_map_by_idx()` (i.e. `LoadAnnotations.__call__()`), the evaluation does not need perform them anymore. However, for backwards compatibility, the evaluation keeps previous input arguments.

This was pointed out for `label_map` in a previous issue (#1415) that the `label_map` should  not be applied in the evaluation. This was handled by [passing an empty dict](5d49918b3c/mmseg/datasets/custom.py (L306-L311)):

```python
# as the labels has been converted when dataset initialized
# in `get_palette_for_custom_classes ` this `label_map`
# should be `dict()`, see
# https://github.com/open-mmlab/mmsegmentation/issues/1415
# for more ditails
label_map=dict(),
reduce_zero_label=self.reduce_zero_label))
```

Similar to this, I now also set `reduce_label=False` since it is now also being handled by `get_gt_seg_map_by_idx()` (i.e. `LoadAnnotations.__call__()`).

3. **Unit test**

I've added a unit test that tests the `CustomDataset.pre_eval()` function when `reduce_zero_label=True` and custom classes are used. The test fails on the original `master` branch but passes with this fix.

## BC-breaking (Optional)

I do not anticipate this change braking any backward-compatibility.

## Checklist

- [x] Pre-commit or other linting tools are used to fix the potential lint issues.
  - _I've fixed all linting/pre-commit errors._
- [x] The modification is covered by complete unit tests. If not, please add more unit test to ensure the correctness.
  - _I've added a test that passes when the fix is introduced, and fails on the original master branch._
- [x] If the modification has potential influence on downstream projects, this PR should be tested with downstream projects, like MMDet or MMDet3D.
  - _I don't think this change affects MMDet or MMDet3D._
- [x] The documentation has been modified accordingly, like docstring or example tutorials.
  - _This change fixes an existing bug and doesn't require modifying any documentation/docstring._

2023-01-30 12:01:20 +08:00

.circleci

[Fix]Fix circile ci env (#2467 )

2023-01-11 13:59:27 +08:00

.dev

[Enhancement] .dev Python files updated to get better performance and syntax (#2020 )

2022-09-14 16:05:40 +08:00

.github

[Projects] Add 'Projects/' folder, and the first example project in 0.x (#2457 )

2023-01-11 13:59:27 +08:00

configs

Imagenet-s dataset for large-scale semantic segmentation (#2480 )