mmsegmentation/tests/test_apis/test_single_gpu.py

73 lines
1.8 KiB
Python
Raw Normal View History

[Refactor] Support progressive test with fewer memory cost (#709) * Support progressive test with fewer memory cost. * Temp code * Using processor to refactor evaluation workflow. * refactor eval hook. * Fix process bar. * Fix middle save argument. * Modify some variable name of dataset evaluate api. * Modify some viriable name of eval hook. * Fix some priority bugs of eval hook. * Depreciated efficient_test. * Fix training progress blocked by eval hook. * Depreciated old test api. * Fix test api error. * Modify outer api. * Build a sampler test api. * TODO: Refactor format_results. * Modify variable names. * Fix num_classes bug. * Fix sampler index bug. * Fix grammaly bug. * Support batch sampler. * More readable test api. * Remove some command arg and fix eval hook bug. * Support format-only arg. * Modify format_results of datasets. * Modify tool which use test apis. * support cityscapes eval * fixed cityscapes * 1. Add comments for batch_sampler; 2. Keep eval hook api same and add deprecated warning; 3. Add doc string for dataset.pre_eval; * Add efficient_test doc string. * Modify test tool to compat old version. * Modify eval hook to compat with old version. * Modify test api to compat old version api. * Sampler explanation. * update warning * Modify deploy_test.py * compatible with old output, add efficient test back * clear logic of exclusive * Warning about efficient_test. * Modify format_results save folder. * Fix bugs of format_results. * Modify deploy_test.py. * Update doc * Fix deploy test bugs. * Fix custom dataset unit tests. * Fix dataset unit tests. * Fix eval hook unit tests. * Fix some imcompatible. * Add pre_eval argument for eval hooks. * Update eval hook doc string. * Make pre_eval false in default. * Add unit tests for dataset format_results. * Fix some comments and bc-breaking bug. * Fix pre_eval set cfg field. * Remove redundant codes. Co-authored-by: Jiarui XU <xvjiarui0826@gmail.com>
2021-08-20 11:44:58 +08:00
import shutil
from unittest.mock import MagicMock
import numpy as np
import pytest
import torch
import torch.nn as nn
from torch.utils.data import DataLoader, Dataset, dataloader
from mmseg.apis import single_gpu_test
class ExampleDataset(Dataset):
def __getitem__(self, idx):
results = dict(img=torch.tensor([1]), img_metas=dict())
return results
def __len__(self):
return 1
class ExampleModel(nn.Module):
def __init__(self):
super(ExampleModel, self).__init__()
self.test_cfg = None
self.conv = nn.Conv2d(3, 3, 3)
def forward(self, img, img_metas, return_loss=False, **kwargs):
return img
def test_single_gpu():
test_dataset = ExampleDataset()
data_loader = DataLoader(
test_dataset,
batch_size=1,
sampler=None,
num_workers=0,
shuffle=False,
)
model = ExampleModel()
# Test efficient test compatibility (will be deprecated)
results = single_gpu_test(model, data_loader, efficient_test=True)
assert len(results) == 1
pred = np.load(results[0])
assert isinstance(pred, np.ndarray)
assert pred.shape == (1, )
assert pred[0] == 1
shutil.rmtree('.efficient_test')
# Test pre_eval
test_dataset.pre_eval = MagicMock(return_value=['success'])
results = single_gpu_test(model, data_loader, pre_eval=True)
assert results == ['success']
# Test format_only
test_dataset.format_results = MagicMock(return_value=['success'])
results = single_gpu_test(model, data_loader, format_only=True)
assert results == ['success']
# efficient_test, pre_eval and format_only are mutually exclusive
with pytest.raises(AssertionError):
single_gpu_test(
model,
dataloader,
efficient_test=True,
format_only=True,
pre_eval=True)