Commit Graph

331 Commits (1ab646de1707c80529fc9104cded58cee375fba9)

Author SHA1 Message Date
Mashiro c4efda4186 [Refactor] Use a real runner to test DistSamplerSeedHook (#812)
* Refactor unit test of DistSamplerSeedHook

* rename the case

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>
2023-02-22 00:08:38 +08:00
Mashiro a094976e9f [Refactor] Use a real runner to test IterTimerHook (#807)
* Refactor iter_timer_hook

* Refactor iter_timer_hook
2023-02-22 00:08:38 +08:00
Mashiro 4b781c336b [Refactor] Refactor unit test of EmptyCacheHook (#805)
* test EmptyCacheHook with runner

* Add coments
2023-02-22 00:08:38 +08:00
Mashiro 425ca99e90 [Refactor] Refactor unit test of Base hook (#806)
* refactor base hooks

* Fix CI
2023-02-22 00:08:38 +08:00
luomaoling 3dc2be05d5
[Feature] Add NPUProfilerHook to profile performance in Ascend device (#925)
* Feature NPUProfilerHook

* Feature NPUProfilerHook
2023-02-21 17:20:40 +08:00
Zaida Zhou e16dacf7e3
Refine the repr of Registry (#942)
* Refine the repr of Registry

* fix ut

* fix ut
2023-02-21 13:18:11 +08:00
Songyang Zhang aee2f6a688
[Feature] Support model complexity computation (#779)
* [Feature] Add support model complexity computation

* [Fix] fix lint error

* [Feature] update print_helper

* Update docstring

* update api, docs, fix lint

* fix lint

* update doc and add test

* update docstring

* update docstring

* update test

* Update mmengine/analysis/print_helper.py

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>

* Update mmengine/analysis/print_helper.py

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>

* Update mmengine/analysis/print_helper.py

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>

* Update mmengine/analysis/print_helper.py

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>

* Update mmengine/analysis/print_helper.py

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>

* Update mmengine/analysis/print_helper.py

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>

* Update mmengine/analysis/print_helper.py

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>

* Update mmengine/analysis/print_helper.py

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>

* Update mmengine/analysis/print_helper.py

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>

* Update mmengine/analysis/print_helper.py

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>

* Update mmengine/analysis/print_helper.py

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>

* Update mmengine/analysis/print_helper.py

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>

* Update mmengine/analysis/print_helper.py

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>

* Update mmengine/analysis/print_helper.py

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>

* Update mmengine/analysis/complexity_analysis.py

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>

* Update docs/en/advanced_tutorials/model_analysis.md

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>

* Update docs/en/advanced_tutorials/model_analysis.md

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>

* update docs

* update docs

* update docs and docstring

* update docs

* update test withj mmlogger

* Update mmengine/analysis/complexity_analysis.py

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>

* Update tests/test_analysis/test_activation_count.py

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>

* update test according to review

* Apply suggestions from code review

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>

* fix lint

* fix test

* Apply suggestions from code review

* fix API document

* Update analysis.rst

* rename variables

* minor refinement

* Apply suggestions from code review

* fix lint

* replace tabulate with existing rich

* Apply suggestions from code review

* indent

* Update mmengine/analysis/complexity_analysis.py

* Update mmengine/analysis/complexity_analysis.py

* Update mmengine/analysis/complexity_analysis.py

---------

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>
Co-authored-by: zhouzaida <zhouzaida@163.com>
2023-02-20 15:00:28 +08:00
wxDai 1d97c07068
[Docs] Fix typo shedule (#936) 2023-02-19 20:44:24 +08:00
whcao a5f48f7d99
[Bug] Fix the bug when the params in shared modules do not require grad (#903)
* fix the bug when the params in shared modules do not require grad

* test DefaultOptimWrapperConstructor when the params in shared modules do not require grad
2023-02-15 11:25:15 +08:00
Eiuyc c59899a65a
[Enhance] Beautify the print result of the registry (#922)
* beautify the print result of the registry

* remove tabulate and use rich for repr

* remove table width; text align to left

* sort _model_dict items
2023-02-14 16:32:11 +08:00
xcnick e35ed5fd2e
[Feature] Add ApexOptimWrapper (#742)
* add ApexOptimWrapper

* typo fix

* add apex amp.initialize in optim_context

* assert apex_amp

* polish code

* add parameters of apex_amp.initialize

* add docs

* polish code

* polish code

* polish code

* fix calling of apex amp load_state_dict

* polish

* add comments

* Update apex_optimizer_wrapper.py

* Update apex_optimizer_wrapper.py

---------

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>
2023-02-06 15:30:10 +08:00
takuoko d1d4609fa2
[Feature] Support using optimizers from dadaptation (#902)
* add dadaptation

* Update mmengine/optim/optimizer/builder.py

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>

* update dadaptation docs

---------

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>
2023-02-03 15:00:32 +08:00
Mashiro 6dc1d7082a
[Fix] Support update np.ScalarType data in message_hub (#898)
* Clean the commit history

* Update message_hub.py

---------

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>
2023-02-01 23:53:28 +08:00
Zaida Zhou 5753cd98e0
[Fix] backend_args should not be modified by get_file_backend (#897) 2023-02-01 14:47:22 +08:00
jbwang1997 c46f891a97
[Enhance] Substitute the environment variable in config file (#744)
* Add read environment variable function in config

* Add UT

* enable int case, split predefined and environment

* Update py config

* Add new attributes env_variables in config

* Add examples in docstring

* Fix comments

* Add tutorials

* Add en tutorials

* Refactor config docs according to comments

* Fix comments

* Change function name
2023-02-01 10:16:47 +08:00
Mashiro 2d8f2be375
[Feature] Add BaseInferencer (#874)
* [Feature] Add BaseInferencer (#773)

* Update BaseInferencer

* Fix ci

* Fix CI and rename iferencer to infer

* Fix CI

* Add renamed file

* Add test file

* Adjust interface sequence

* refine preprocess

* Update unit test

Update unit test

* Update unit test

* Fix unit test

* Fix as comment

* Minor refine

* Fix docstring and support load image from different backend

* Support load collate_fn from downstream repos, refine dispatch

* Minor refine

* Fix lint

* refine grammar

* Remove FileClient

* Refine docstring

* add rich

* Add list_models

* Add list_models

* Remove backend args

* Minor refine

* Fix typos in docs and type hints (#787)

* [Fix] Add _inputs_to_list (#795)

* Add preprocess inputs

* Add type hint

* update api/infer in index.rst

* rename preprocess_inputs to _inputs_to_list

* Fix doc format

* Update infer.py

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>

* [Fix] Fix alias type (#801)

* [Enhance] Support loading model config from checkpoint (#864)

* first commit

* [Enhance] Support build model from weight

* minor refine

* Fix type hint

* refine comments

* Update docstring

* refine as comment

* Add  method

* Refine docstring

* Fix as comment

* refine comments

* Refine warning message

* Fix unit test and refine comments

* replace MODULE2PACKAGE to MODULE2PAKCAGE

* Fix typo and syntax error in docstring

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>
Co-authored-by: Tong Gao <gaotongxiao@gmail.com>
2023-01-16 16:01:06 +08:00
LEFTeyes 0b59a90a21
[Feature] Support ReduceOnPlateauParamScheduler(#819)
* [Feature] Add ReduceOnPlateauParamScheduler and change ParamSchedulerHook

* [Feature] add ReduceOnPlateauLR and ReduceOnPlateauMomentum

* pre-commit check

* add a little docs

* change position

* fix the conflict between isort and yapf

* fix ParamSchedulerHook after_val_epoch execute without train_loop and param_schedulers built

* Apply suggestions from code review

Co-authored-by: Mashiro <57566630+HAOCHENYE@users.noreply.github.com>

* update ReduceOnPlateauParamScheduler, ReduceOnPlateauMomentum and ParamSchedulerHook

* fix get need_step_args attribute error in ParamSchedulerHook

* fix load_state_dict error for rule in ReduceOnPlateauParamScheduler

* add docs for ParamSchedulerHook and fix a few codes

* [Docs] add ReduceOnPlateauParamScheduler, ReduceOnPlateauMomentum and ReduceOnPlateauLR docs

* [Refactor] adjust the order of import

* [Fix] add init check for threshold in ReduceOnPlateauParamScheduler

* [Test] add test for ReduceOnPlateauParamScheduler, ReduceOnPlateauLR and ReduceOnPlateauMomentum

* [Fix] fix no attribute self.min_value

* [Fix] fix numerical problem in tests

* [Fix] fix error in tests

* [Fix] fix ignore first param in tests

* [Fix] fix bug in tests

* [Fix] fix bug in tests

* [Fix] fix bug in tests

* [Fix] increase coverage

* [Fix] fix count self._global_step bug and docs

* [Fix] fix tests

* [Fix] modified ParamSchedulerHook test

* Update mmengine/optim/scheduler/param_scheduler.py

Co-authored-by: Mashiro <57566630+HAOCHENYE@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Mashiro <57566630+HAOCHENYE@users.noreply.github.com>

* [Fix] modified something according to commented

* [Docs] add api for en and zh_cn

* [Fix] fix bug in test_param_scheduler_hook.py

* [Test] support more complicated test modes(less, greater, rel, abs) for ReduceOnPlateauParamScheduler

* [Docs] add docs for rule

* [Fix] fix pop from empty list bug in test

* [Fix] fix check param_schedulers is not built bug

* [Fix] fix step_args bug and without runner._train_loop bug

* [Fix] fix step_args bug and without runner._train_loop bug

* [Fix] fix scheduler type bug

* [Test] rename step_args to step_kwargs

* [Fix] remove redundancy check

* [Test] remove redundancy check

* Apply suggestions from code review

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>

* [Test] fix some defects

Co-authored-by: Mashiro <57566630+HAOCHENYE@users.noreply.github.com>
Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>
2023-01-16 11:39:03 +08:00
Ma Zerun fcd783fcb2
[Enhance] Support non-scalar type metric value. (#827)
* [Enhance] Support non-scalar type metric value.

* Refactor support.

* Fix non-scalar tags problem during validation

* Update tag processor.
2023-01-12 20:28:55 +08:00
Mashiro 925ac870e2
[Fix] Unloaded weights will not be initialized when using PretrainedIinit (#764)
* Separate init_cfgs to pretrained_cfg and other_cfgs

* Fix unit test

* update documentation

* Fix render of initialize.md

* Fix as comment

* rename initialize.md to weight_initialization.md

* add file

* fix ci

* rename weight_initialization.md to initialize.md

* Fix duplicated .md
2023-01-09 18:46:30 +08:00
Mashiro ad1b43faf2
[Fix] Fix `Config` cannot parse base config when there is `.` in tmp path (#856)
* [Fix] Fix config cannot parse tmp path like

* Add comments

* Add comments

* Apply suggestions from code review

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>
2022-12-30 14:56:14 +08:00
Zaida Zhou 646927f62f
[Enhance] Ensure metrics is not empty when saving best ckpts (#849)
* [Enhance] Ensure metrics is not empty when saving best ckpts

* fix warn to warning

* delete a unnecessary method
2022-12-28 11:34:08 +08:00
Mashiro a9b6753fbe
Make TTAModel compatible with FSDP (#611)
* Add build_runner_with_tta and PrepareTTAHook

* rename hook file

* support build tta runner with runner type

* add unit test

* Add build_runner_with_tta to index.rst

* minor refine

* Add runner test cast

* Fix unit test

* fix unit test

* tmp save

* pop None if key does not exist

* Fix is_model_wrapper and force register class in test_runner

* [Fix] Fix is_model_wrapper

* destroy group after ut

* register module in testcase

* pass through unit test

* fix as comment

* remove breakpoint

* remove mmengine/testing/runner_test_cast.py

* minor refine

* minor refine

* minor refine

* set default data preprocessor for model

* minor refine

* minor refine

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>

* fix lint

* Fix unit test

* replace  with  in ImgDataPreprocessor

* Fix as comment

* add inference tutorial in advanced tutorial

* update index.rst

* add tta example

* refine tta tutorial

* Add english tutorial

* add note for build_runner_with_tta

* Fix as comment

* add examples

* remove chinese comment

* Update docs/en/advanced_tutorials/test_time_augmentation.md

Co-authored-by: RangiLyu <lyuchqi@gmail.com>

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>
Co-authored-by: RangiLyu <lyuchqi@gmail.com>
2022-12-27 21:39:43 +08:00
BayMax_BHL 16589ce386
[Feature] Add ProfilerHook (#768)
* [Feature] Add profiler hook functionality

* [Feature] Add profiler hook functionality

* [Feature] Add profiler hook functionality

* [Feature] Add profiler hook functionality

* [Feature] Add profiler hook functionality

* [Feature] Add profiler hook functionality

* [Feature] Add profiler hook functionality

* [Feature] Add profiler hook functionality

* [Feature] Add profiler hook functionality

* [Feature] Add profiler hook functionality

* [Feature] Add profiler hook functionality

* [Feature] Add profiler hook functionality

* [Feature] Add profiler hook functionality

* [Feature] Add profiler hook functionality

* [Feature] Add profiler hook functionality

* [Feature] Add profiler hook functionality

* [Feature] Add profiler hook functionality

* [Feature] Add profiler hook functionality

* [Feature] Add profiler hook functionality

* [Feature] Add profiler hook functionality

* [Feature] Add profiler hook functionality

* [Feature] Add profiler hook functionality

* [Feature] Add profiler hook functionality

* [Feature] Add profiler hook functionality

* [Feature] Add profiler hook functionality

* [Feature] Add profiler hook functionality

* [Feature] Add profiler hook functionality

* [Feature] Add profiler hook functionality

* [Feature] Add profiler hook functionality

* [Feature] Add profiler hook functionality

* [Feature] Add profiler hook functionality

* [Feature] Add profiler hook functionality

* [Feature] Add profiler hook functionality

* [Feature] Add profiler hook functionality

* Apply suggestions from code review

* Update mmengine/hooks/profiler_hook.py

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>
2022-12-27 18:58:05 +08:00
Mashiro c382f8a5d5
[Fix] Ignore the distributed tests for macOS platform (#821)
* only test timer in linux

* Fix merge_stage_test.yml

* remove install ffmpeg

* test_ci

* test_ci

* Fix python -m pip install pip --upgrade to pip install

* don't test unittest with sleep 1 in windows

* debug with tmate

* increase timeout

* increase timeout

* skip test gloo

* skip test gloo

* fix synteax error

* skip test gloo in torch 1.13

* skip test gloo in torch 1.13

* skip testing setup_env in macOS

* skip test test_dist

* skip test setup

* restore triggered during push

* clean the code

* debug macos

* refine macOS CI

* test merge stage test

* trigger CI during push to main

* Fix as comment
2022-12-27 16:18:46 +08:00
RangiLyu e83ac944b6
[Feature] Registry supports import modules automatically. (#643)
* [Feature] Support registry auto import modules.

* update

* rebase and fix ut

* add docstring

* remove count_registered_modules

* update docstring

* resolve comments

* resolve comments

* rename ut

* fix warning

* avoid BC breaking

* update doc

* update doc

* resolve comments
2022-12-23 15:46:29 +08:00
Xiangxu-0103 591024e533
[Docs] Update docstring of structures (#840)
* Update docstring of `structures`

* update docs

* add `import torch` to `examples`
2022-12-21 20:07:18 +08:00
Zaida Zhou b44048aeb7
[Fix] Fix failed unit tests caused by the new released numpy (#838)
* [Fix] Fix failed unit tests caused by the new released numpy

* minor fix
2022-12-20 11:00:47 +08:00
Qian Zhao 4147e976a6
[Fix] ZeroRedundancyOptimizer ambiguous error with param groups when pytorch < 1.12.0 (#818)
* fix zero_optimizer error with param groups when pytorch < 1.12.0

* add docstring

* fix docstring

* add unittest

* change ut to use a valid paramwise_cfg

* modify ut

* fix as comments
2022-12-19 17:07:58 +08:00
RangiLyu 3582b4c787
[Enhance] Support setting decay multiplier for flatten parameter (#771)
* [Fix] Fix bias decay mult of depth-wise conv.

* support flatten param weight decay multiplier

* add unit tests

* REMOVE TODO

* update doc
2022-12-16 17:37:22 +08:00
Mashiro 95c71c82d0
[Fix] Only test Timer in Linux platform (#823) 2022-12-13 19:11:09 +08:00
Mashiro d876d4e0f8
[Enhance] Add support of TorchVision's Model Registration API (#793)
* enhance get_torchvision_model

* remove mmcv
2022-12-11 22:08:51 +08:00
Ma Zerun fe26c6559e
[Enhance] Support multiple `--cfg-options`. (#759) 2022-12-07 10:44:25 +08:00
Qian Zhao bd6791382f
[Fix] BaseModel & BaseDataPreprocessor `to` method to be consistent with torch.nn.Module (#783)
* fix BaseModel `to` method to be consistent with torch.nn.Module

* fix data_preprocessor as well

* fix docstring alignment

* fix docstring alignment
2022-12-05 18:04:47 +08:00
SUNYC 7e787d8ffa
[Enhance] Add a new parameter conf_path in PetrelBackend (#774)
* feat: add conf_path in PetrelBackend

* Update mmengine/fileio/backends/petrel_backend.py

Co-authored-by: Mashiro <57566630+HAOCHENYE@users.noreply.github.com>

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>
Co-authored-by: Mashiro <57566630+HAOCHENYE@users.noreply.github.com>
2022-11-29 14:23:16 +08:00
liukuikun 5f58517069
[Enchance] remove redundant attributes in BaseDataElement (#690)
* remove redundant attribute in dump or print

* optimize efficiency

Co-authored-by: HAOCHENYE <21724054@zju.edu.cn>
2022-11-24 21:36:19 +08:00
Mashiro ded73f3a56
[Enhance] Enhance compatibility of `revert_sync_batchnorm` (#695)
* [Enhance] Enhance revert_sync_batchnorm and convert_sync_batchnorm

* [Enhance] Enhance revert_sync_batchnorm and convert_sync_batchnorm

* Fix unit test

* Add coments

* Refine comments

* clean the code

* revert convert_sync_batchnorm

* revert convert_sync_batchnorm

* refine comment

* fix CI

* fix CI
2022-11-24 16:41:54 +08:00
cir7 0e6bb48b12
[Enhance] Support eta_min_ratio in CosineAnnealingParamScheduler (#725)
* [Enhance] support eta_min_ratio in CosineAnnealingParamScheduler

* [doc] fix docstring

* [Enhance] add ut for eta_min_ratio

* [doc] update docstring

* avoid bc-breaking of eta_min

* [doc] add docstring in CosineAnnealingParamScheduler and CosineAnnealingMomentum

* Apply suggestions from code review

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>
2022-11-22 20:19:16 +08:00
songyuc 6636f07cfe
[Feature] Add get_hooks_info() to print hooks messages (#672)
* Add test of get_hooks_info()

* Change to use original Runner for get_hook_info() test

* Change to test after_train_iter hooks for get_hook_info()

* Complement the stages list

* Add logging hooks information in Runner.__init__()

* Rearrange the stages list

* Restore the stages to tuple type

* Clean the unnecessary changes

* Replace  statement with TestCase's methods

* add test stages in method_stages_map

* change the hooks info into a f-string

* return list(trigger_stages) directly

* change keys of method_stages_map

* Fix previous changes to method_stages_map.keys
2022-11-22 20:02:29 +08:00
Mashiro b06234cfcd
[Enhance] Right align the log (#436)
* right allign the log

* fix as comment

* Add comments for magic number 3

* remove max_len_str

* Update mmengine/runner/log_processor.py

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>
2022-11-21 11:55:18 +08:00
Mashiro c478bdca27
[Enhance] enhance runner test case (#631)
* Add runner test cast

* Fix unit test

* fix unit test

* pop None if key does not exist

* Fix is_model_wrapper and force register class in test_runner

* [Fix] Fix is_model_wrapper

* destroy group after ut

* register module in testcase

* fix as comment

* minor refine

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>

* fix lint

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>
2022-11-21 11:54:05 +08:00
RangiLyu a9a575866f [Fix] Fix offline_evaluate index error (#630)
* [Fix] Fix offline eval dataset index error.

* update

* update
2022-11-18 01:17:12 +08:00
RangiLyu f2b0540f58 [Enhance] Raise warning for abnormal momentum (#655) 2022-11-18 01:17:12 +08:00
Yanhong Zeng 618a063ddc
[Docs] Fix docstring and documentation used for hub.get_model (#659)
* fix docstring and documentation used for hub.get_model

* fix config name in test_hub

* fix docstring mentioned faster-rcnn in utils
2022-11-02 10:16:56 +08:00
BigDong 376251961d
[Enhance] Add `ignore_keys` in ConcatDataset (#556)
* [Fix] Fix ConcatDataset error in VOCDataset

* minor fix

* minor fix

* minor fix

* add UT

* minor fix

* minor fix

* minor fix

* Update mmengine/dataset/dataset_wrapper.py

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>

* minor fix

* Update mmengine/dataset/dataset_wrapper.py

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>
2022-11-01 17:06:02 +08:00
Z-Fran 090104df21
[Fix] Fix the calculation error of eta_min in CosineRestart (#639)
* [Fix] fix CosineRestart eta_min

* add ut case

* Enhance unit test

Enhance unit test

* remove unused code

Co-authored-by: HAOCHENYE <21724054@zju.edu.cn>
2022-11-01 15:48:39 +08:00
Mashiro d1dd240796
[Fix] Fix `BaseDataPreprocessor.cast_data` cound not handle string data (#602)
* [Fix] Fix  cound not handle string data

* Minor refine

* Refine type hint

Refine type hint

* fix as comment

* Minor refine

* Update mmengine/model/base_model/data_preprocessor.py

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>
2022-10-28 15:17:27 +08:00
Hakjin Lee 0857f9fb40
[Feature] Support torch ZeroRedundancyOptimizer (#551)
* [Feature] Support torch ZeRORedundancyOptimizer

Co-authored-by: Junhwa Song <ethan9867@gmail.com>
Signed-off-by: Junhwa Song <ethan9867@gmail.com>
Signed-off-by: Hakjin Lee <nijkah@gmail.com>

* lint

* Fix saving optimizer state_dict

* Fix handling import error

* Add test case

* fix UT

* Revert "fix UT"

This reverts commit dd64538960.

* fix handling import in UT

* Fix saving zero checkpoint and delete redundant master_only

* lint

* test unittest

* Fix handling impor error

* Fix UT condition

* Edit docstrings

* Fix typo

* Skip redundant procudure in checkpoint hook

* fix typo again

* Update mmengine/optim/optimizer/zero_optimizer.py

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>

* Add api info

* lint

* Fix lint

* Handling AmpOptimWrapper case

* handling overlap_with_ddp

* Fix error

Signed-off-by: Junhwa Song <ethan9867@gmail.com>
Signed-off-by: Hakjin Lee <nijkah@gmail.com>
Co-authored-by: Junhwa Song <ethan9867@gmail.com>
Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>
2022-10-27 20:31:50 +08:00
Mashiro c1447734c2
[Enhance] Call `register_all_modules` in `Registry.get()` (#541)
* call register_all_modules  in Registry.get()

* Fix ci

* fix scope bug (scope_name -> scope), for temp sync

* Fix unit test

* Refine log information

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>

* Fix typo

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>
2022-10-24 20:15:14 +08:00
Mashiro bda92b49e7
[Feature] Add a function to mark the deprecated function. (#609)
* [Enhancement] Add a deprecated function

* Refine and add ut

* tmp save

* minor refine

* Fix unit test

* Fix unit test

* fix docstring

* Update utils.rst
2022-10-24 14:49:48 +08:00
wangjiangben-hw 601db12d38
[Feature] Support engine with NPU backend. (#572)
* init npu

* Update mmengine/optim/optimizer/amp_optimizer_wrapper.py

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>

* Update mmengine/dist/dist.py

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>

* change to is_hccl_backend

* Update mmengine/optim/optimizer/amp_optimizer_wrapper.py

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>

* add comment with AmpOptimWrapper

* Update mmengine/runner/amp.py

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>

* Update mmengine/runner/amp.py

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>

* add npu fn in base_model

* Update mmengine/optim/optimizer/amp_optimizer_wrapper.py

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>

* clean lint

* Update mmengine/optim/optimizer/amp_optimizer_wrapper.py

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>

* Update mmengine/model/base_model/base_model.py

Co-authored-by: Mashiro <57566630+HAOCHENYE@users.noreply.github.com>

* add is_npu_available

* try to fix

* Add comments

* Refine grammar

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>
Co-authored-by: Mashiro <57566630+HAOCHENYE@users.noreply.github.com>
Co-authored-by: HAOCHENYE <21724054@zju.edu.cn>
2022-10-24 11:42:29 +08:00
shenmishajing d270516fe8
[Fix]: fix add_config func is not called bug (#613)
* fix add_config func is not called bug
fix wandb add_config bug
add wandb log_code feature

* move log_code_name param to the last one
add log_code_name param to docstring

* add config only when there is a cfg

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>

* add unit test for log_code_name param of WandbVisBackend

* Update mmengine/visualization/vis_backend.py

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>

* use log_code_kwargs instead of only log_code_name
add PATH_FILTERS registry

* use log_code_kwargs instead of only log_code_name
add PATH_FILTERS registry

* fix add config unit test

* roll back to log_code_name version

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>
2022-10-24 09:58:10 +08:00
Mashiro aaba1d8871
[Fix] Fix error format of log message (#508)
* Fix error format of log message

* Fix unit test

* remove unnecessary comment
2022-10-18 18:04:15 +08:00
Mashiro abe56651db
[Fix] Make autocast compatible with mps (#587)
* [Fix] Make autocast compatible with mps

* Enhance unit test

* fix unit test

* clean the code

* fix unit test
2022-10-18 18:03:17 +08:00
Mashiro 6073d9ebd8
[Enhance] add documents for `clip_grad` , and support clip grad by value. (#513)
* [Enhance] add documents for , and support clip grad by value

* refine docstring

* fix as comment

* Fix as comment

* minor refine

* minor refine

* remove error comment for clip grad

* refine docstring
2022-10-18 18:02:46 +08:00
shenmishajing c2982723c9
[Feats]: add non_blocking feature to BaseDataPreprocessor (#618)
* add non_blocking feature to BaseDataPreprocessor

* Update mmengine/model/base_model/data_preprocessor.py

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>

* move new parameters to the last to avoid bc issue

* Update mmengine/model/base_model/data_preprocessor.py

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>

* Update mmengine/model/base_model/data_preprocessor.py

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>
2022-10-18 17:58:51 +08:00
Mashiro 62456217f9
[Feature] Add test time augmentation base model. (#538)
* First commit

* add BaseTestTimeAugModel

* Add unit test

* move loop logic to test_step

* fix ddp test

* rename model to module

* optim __init__

* Fix as comment

* Fix as comment

* make val_step should not be called

* make tta do not inherit base model

* Fix unit test

* Enhance docstring

* Fix as comment

* Fix as comment

* minor refine

* minor refine

* minor refine

* fix unit test

* minor refine

* minor refine

* minor refine

* minor refine

* minor refine

* minor refine

* fix unit test
2022-10-10 14:29:33 +08:00
Qian Zhao e73c4bf135
[Fix] some out-of-date unittests (#586)
* fix dist ut

* fix config & hub ut break due to mmdet faster-rcnn rename
2022-10-08 19:48:51 +08:00
Mashiro 89146a5b90
[Fix] Fix detect_anomalous_params (#588) 2022-10-08 19:48:35 +08:00
Zaida Zhou ed84dfd34d
[Refactor] Refactor fileio without breaking back compatibility (#533)
* [Refactor] Refactor fileio but without breaking bc

* handle compatibility

* fix format

* modify io functions

* fix ut

* fix ut

* rename method names

* refine

* refine docstring

* fix ut in windows

* update ut

* minor fix

* ensure client is not None when closing it

* add more examples for list_dir_or_file interface

* refine docstring

* refine deprecated info

* fix ut

* add a description for lmdb docstring
2022-09-26 14:30:40 +08:00
Qian Zhao c64243aa9e
[Fix] CheckpointHook behavior incorrect if given `filename_tmpl` argument (#518) 2022-09-22 12:47:45 +08:00
Mashiro 5d7527d75e
[Fix ] Fix unit test could may fail caused by `MultiProcessTestCase` (#535)
* fix merge ci

* make timeout larger

* tmp test merge

* debug

* sleep 1 in optimizer ut

* skip test clip grad

* skip test clip grad

* fix merge ci
2022-09-15 18:08:56 +08:00
Mashiro 8ee31dbc3b
[Feature] Support convert `BN` to `SyncBN` by config (#506)
* [Feature] Support convert BN to SyncBN by config

* make unit test compatible with cpu

* Fix as comment

* fix unit test

* change signature of convert_sync_batchnorm: rename sync_bn to implemention

* fix unit test

* fix unit test
2022-09-15 18:08:36 +08:00
Mashiro 6b1b8a3751
[Fix] Fix unit test in windows (#515) 2022-09-13 11:46:21 +08:00
Mashiro 0fb2b8ca8c
[Fix] Fix circle import in EMAHook (#523) 2022-09-09 19:33:27 +08:00
takuoko a6f5297727
[fix] EMAHook load state dict (#507)
* fix ema load_state_dict

* fix ema load_state_dict

* fix for test

* fix by review

* fix resume and keys
2022-09-09 11:41:12 +08:00
Zaida Zhou 19d4eba866
[Refactor] Update Github Action CI and CircleCI (#512)
* [CI] Update Github Action CI and CircleCI

* clear outdated commands in CI

* clear outdated commands in CI

* add '-r' when installing packages from txt

* update

* update circleci

* fix mismatch conflicts

* update

* install system dependencies

* update

* set env

* fix ut

* update cuda version on Windows

* update push action
2022-09-07 20:24:11 +08:00
Mashiro 2d3bff4418
[Fix] Fix unit test of `Config` will install `mmdet` and `mmcls`. (#492)
* fix unit test install mmdet and mmcls

* raise error when mmdet is not installed

* rename check_and_install to install_package

* split test case

Co-authored-by: C1rN09 <zhaoqian@pjlab.org.cn>
2022-08-31 21:18:05 +08:00
liukuikun 1fea82aad5
[Docs] update data element tutorials (#431)
* structure tutorials

* refine data element docs

* modify introduce

* fix comment

* fix comment

* fix comment
2022-08-31 20:39:08 +08:00
Qian Zhao 94412f72d8
[Fix] Exception while building registered function (#491) 2022-08-31 13:53:32 +08:00
Mashiro 33d7c81bc4
[Refactor]: Avoid creating instances with the same name (#483) 2022-08-30 16:28:18 +08:00
wxDai fba9a94f52
[Refactor] add testing utils (#475)
* add testing utils

* fix ut

* add blank line betweeen `Args` and `Returns`
2022-08-29 22:59:20 +08:00
Zaida Zhou 2f09342663
[Docs] Refine registry docs (#443)
* [Docs] Refine registry docs

* explain how to use _scope_

* refine
2022-08-29 14:50:48 +08:00
Mashiro 576e5c8f91
[Fix] Regist pytorch ddp and dp to `MODEL_WRAPPERS`, add unit test to `is_model_wrapper` (#474)
* regist pytorch ddp and dp, add unit test

* minor refine

* Support check custom wrapper

* enhance ut
2022-08-28 17:09:41 +08:00
RangiLyu 8d25dbdeda
[Fix] Fix EMAHook trigger train loop and AveragedModel sync buffer. (#467)
* [Fix] Fix EMAHook trigger train loop init during testing.

* fix sync buffer

* update ut

* fix sync buffer

* fix sync buffer
2022-08-26 14:21:56 +08:00
Mashiro 8770c6c7fc
[Refactor] Refactor data flow to make the interface more natural (#468)
* [Refactor]: modify interface of Visualizer.add_datasample (#365)

* [Refactor] Refactor data flow: refine `data_preprocessor`. (#359)

* refine data_preprocessor

* remove unused BATCH_DATA alias

* Fix type hints

* rename move_data to cast_data

* [Refactor] Refactor data flow: collate data in `collate_fn` of `DataLoader`  (#323)

* acollate data in dataloader

* fix docstring

* refine comment

* fix as comment

* refactor default collate and psedo collate

* foramt test file

* fix docstring

* fix as comment

* rename elem to data_item

* minor fix

* fix as comment

* [Refactor] Refactor data flow: `data_batch` argument of `Evaluator.process is a `dict` (#360)

* refine evaluator and metric

* compatible with new default collate

* replace default collate with pseudo

* Handle data_batch in metric

* fix unit test

* fix unit test

* fix unit test

* minor refine

* make data_batch optional

make data_batch optional

* rename outputs to predictions

* fix ut

* rename predictions to outputs

* fix docstring

* fix docstring

* fix unit test

* make outputs and data_batch to kwargs

* fix unit test

* keep signature of metric

* fix ut

* rename pred_sample arguments to data_sample(Visualizer)

* fix loop and ut

* [refactor]: Refactor model dataflow (#398)

* [Refactor] Refactor data flow: refine `data_preprocessor`. (#359)

* refine data_preprocessor

* remove unused BATCH_DATA alias

* Fix type hints

* rename move_data to cast_data

* refactor model data flow

tmp_commt

tmp commit

* make val_cfg and test_cfg optional

* roll back runner

* pass test mmdet

* fix as comment

fix as comment

fix ci in DataPreprocessor

* fix ut

* fix ut

* fix rebase main

* [Fix]: Fix test val ddp (#462)

* [Fix] Fix docstring and type hint of data flow (#463)

* Fix docstring of data flow

* change signature of hook

* fix unit test

* resolve conflicts

* fix lint
2022-08-24 22:04:55 +08:00
Zaida Zhou 7e1d7af2d9
[Refactor] Refactor code structure (#395)
* Rename data to structure

* adjust the way to import module

* adjust the way to import module

* rename Structure to Data Structures in docs api

* rename structure to structures

* support using some modules of mmengine without torch

* fix circleci config

* fix circleci config

* fix registry ut

* minor fix

* move init method from model/utils to model/weight_init.py

* move init method from model/utils to model/weight_init.py

* move sync_bn to model

* move functions depending on torch to dl_utils

* format import

* fix logging ut

* add weight init in model/__init__.py

* move get_config and get_model to mmengine/hub

* move log_processor.py to mmengine/runner

* fix ut

* Add TimeCounter in dl_utils/__init__.py
2022-08-24 19:14:07 +08:00
Zaida Zhou 486d8cda56
[Refactor] Refactor the import rule (#459)
* [Refactor] Refactor the import rule

* minor refinement

* add a comment
2022-08-23 18:58:36 +08:00
Zaida Zhou a9ad09bded
[Fix] Fix utils ut (#458) 2022-08-23 16:56:47 +08:00
liukuikun bb56cf42ab
[Visualizer] use FigureManager to manage figure to avoid affecting plt.show() outside Visualizer(#440)
* figure in Visualizer is not managed by plt

* encapsulate code and remove unused code
2022-08-22 17:16:14 +08:00
Zaida Zhou b75962a660
Migrate utils from mmcv (#447) 2022-08-22 11:51:56 +08:00
Mashiro e907931fb8
Fix unit tests (#449) 2022-08-21 14:54:24 +08:00
Zaida Zhou 429bb27972
[Refactor]: migrate LMDB backends and modifications from open-mmlab/mmcv#1764 and open-mmlab/mmcv#1747 (#426)
* Add fileio ut

* pick modification from master branch

* pick modification from mmcv #1764

* test fileio without mmcv

* remove unnecessary checking
2022-08-19 16:06:49 +08:00
Mashiro 4abf1a0454
[Enhance] Support build evaluator from list of built metric (#423)
* Support build evaluator from list of built metric

* regist evaluator

* fix as comment

* add unit test
2022-08-19 10:56:51 +08:00
Mashiro d6ad01a4cf
[Fix]: fix ci (#441) 2022-08-18 14:04:19 +08:00
Mashiro e08b9031fc
[Enhance] Support building optimizer wrapper from built Optimizer instance (#422)
* support build optimizer wrapper from built Optimizer instance

* refine comments
2022-08-17 19:17:00 +08:00
Mashiro a706bbc018
[Fix]: fix error and add unit test (#429) 2022-08-17 19:16:00 +08:00
Zaida Zhou 4637c49640
[Refactor] Rename test_hook to test_hooks (#425) 2022-08-15 11:45:07 +08:00
Mashiro 438e8e7498
BaseModel support recursively set the device of data_preprocessor (#387) 2022-08-15 11:04:24 +08:00
Zaida Zhou f98ba60629
[Enhancement] Improve unit tests of mmengine/runner (#182)
* [Enhancement] Add unit test for get_priority

* fix priority ut

* fix typo

Co-authored-by: Wenwei Zhang <40779233+ZwwWayne@users.noreply.github.com>
2022-08-15 10:57:58 +08:00
Mashiro 2708b7ed48
fix ci (#424) 2022-08-13 09:15:08 +08:00
RangiLyu 813f49bf23
[Feature] Support CosineRestartParamScheduler. (#397)
* [Feature] Support CosineRestartParamScheduler.

* add ut and docstring

* add docstring
2022-08-11 17:57:35 +08:00
Mashiro b14cbc2576
[Fix] Fix wrong epoch and iter when saving best ckpt (#400)
* fix wrong epoch andd iter when save bbest ckpt

* fix ut

* fix resume best ckpt unexpectedly

* minor refine

* fix unit test
2022-08-11 14:52:38 +08:00
Mashiro ee56f151f6
[Fix] Support training with data without `metainfo`. (#417)
* support training with data without metainfo

* clean the code

* clean the code
2022-08-11 14:51:11 +08:00
Ma Zerun 9b2a0e02da
[Enhance] Add `data_preprocessor` config as an argument of runner. (#343)
* [Enhance] Add `preprocess_cfg` as an argument of runner.

* Rename `preprocess_cfg` to `data_preprocessor`

* Fix docstring
2022-08-09 11:25:29 +08:00
Mashiro cd257c9047
[Enhance] Support log enviroment information during initiate runner (#384)
* Support log enviroment information when initiate runner

* Fix unit test

* fix as comment, save world_size

* log gpu num

* clear code and reformat log

* minor refine

* fix as comment

* minor refine

* clean the code

* clean the code

* remove save world_size in meta
2022-08-08 22:23:52 +08:00
Mashiro 7e2302388f
[Feature] add config new feature (#105) 2022-08-08 21:01:06 +08:00
Mashiro a07a063306
[Enhance] Add build function for scheduler. (#372)
* add build function for scheduler

* add unit test

add unit test

* handle convert_to_iter in build_scheduler_from_cfg

* restore deleted code

* format import

* fix lint
2022-08-08 20:34:16 +08:00
Mashiro 99de0951af
[Enhance] EMAHook support does not load checkpoint strictly (#352)
* BaseAveragedModel support load ckpt without module prefix

* refine docstring

* allow EMAHook does not load ckpt strictly

* add unit test for strict argument of EMAHook

* sync remote

* sync remote

* clean the code

* ema hook supports setting start iter

* fix unit test

* fix as comment

* fix as comment

* describe kwargs
2022-08-08 20:26:16 +08:00
LeoXing1996 08602a2385
[Enhancement] Support save best based on multi metrics (#349)
* support save best based on multi metrics

* add unit test

* resolve bugs after rebasing

* revise docstring

* revise docstring

* fix as comment

* revise as comment
2022-08-08 20:17:17 +08:00
Mashiro 6ebb7ed481
[Fix] Fix config cannot change(assign) the value of the dict in config(without base config) (#378)
* Support changing value of dict in config(without base config)

* minor refine
2022-08-08 17:07:47 +08:00