Commit Graph

95 Commits (aee6d7a769312ac51ad6056a07b5d6cda8d78ef0)
 

Author SHA1 Message Date
Zaida Zhou aee6d7a769
[Docs] Improve type hint style (#125) 2022-03-25 11:26:05 +08:00
Zaida Zhou 248ad9aee8
[Feature] Add Runner and Loop (#86)
* [Feature] Add Runner and Loop

* refine docstring

* refine docstring

* refine __init__ of Runner

* add Runner.save_checkpoint

* add comment for data_batch

* rename validation_cfg to val_cfg

* update branch

* update branch

* discard wear reference

* set seed for torch in dataloader worker

* refine comments

* reorder methods

* import module in __init__.py

* add runner.load_checkpoint

* rename attributes

* fix indent

* fix indent

* add comment for LOOPS

* add example for methods

* add example for methods

* refine examples

* add Runner.resume and refactor register_hooks

* remove type hint of runner in CheckpointHook

* rename before_run to before_train

* remove collate

* add before_run after_run for loop.run

* dummy collate_fn

* add more unit tests

* move call_hook('before_run') to runner

* minor fix

* add comments for collate_fn

* fix type hint

* refactor logic to load or resume checkpoint

* fix typo

* add comments for IterBasedTrainLoop

* refactor unit tests

* fix unit tests

* refactor

* fix setup_env

* refine

* minor fix

* Merge branch 'main' of github.com:open-mmlab/mmengine into zzd/add-runner

* minor fix

* minor fix

* refine

* refactor unit tests

* fix unit tests

* add launcher property

* add logger, message_hub and writer unit tests

* add cur_dataloader attribute for runner

* fix IterBasedTrainLoop

* add torch.no_grad decorator

* add master_only decorator for save_checkpoint

* add before_test_epoch in TestLoop

* refactor load_or_resume method

* refine docstring

* fix typo

* build_dataloader can handle dataset object

* add dump_config for runner

* collate_fn should not a lambda function

* minor fix

* refine docstring

* fix unit tests

* improve comments

* refine unit tests

* rename collate_fn to pseduo_collate

* add 'TODO' tag
2022-03-14 13:19:29 +08:00
Zaida Zhou b3d54481fd
[Fix] Fallback to use self registry if default_scope not found (#122) 2022-03-14 00:44:16 +08:00
Zaida Zhou 72cf410969
[Refactor] Refactor interface of checkpointhook (#127)
* [Refactor] Refactor interface of checkpointhook

* fix print format

* minor ifx
2022-03-13 23:39:28 +08:00
Mashiro fff4742e0b
[Enhancement] Messagehub supports update dict logs. (#120)
* Messagehub support update dict values

* Messagehub support update dict values

* fix assertion error message
2022-03-13 21:28:47 +08:00
Zaida Zhou f548c81846
[Enhancement] Handle tensor device type in sync_random_seed (#126) 2022-03-13 17:45:02 +08:00
Zaida Zhou 6d73b6cdf2
[Refactor] Use mmengine distributed in evaluator (#123)
* [Refactor] Use mmengine distributed in evaluator

* remove 'TODO' comment
2022-03-13 16:58:23 +08:00
Mashiro 4d49de7d81
[Fix] Fix LoggerHook save mutiple ranks scalar in the same json file. (#124)
* use master_only to decorator _log_train and _log_val

* fix resoloved TODO

fix resoloved TODO

fix resoloved TODO

* fix raise error typo

* ensure log item is python scalar
2022-03-13 16:56:29 +08:00
Mashiro a7961407e4
[Refactor] Refactor the interfaces of Hook and its subclassed (#117)
* Fix hook

* Fix

* Fix docs

* FIx

* Fix

* Fix as comment

* update

* Fix hook

* Fix hook

* Fix hook

* Fix itertimerhook

* Fix iter_timer_hook

* Fix

* Fix

* fix logger hook

* Fix loggerhook

* update cur_dataloader

* Fix docstring

* Fix docstring

* Fix as commet

* Fix as commet

* Fix as comment

* rename is_last_epoch, enhance and add after_val before_val .etc

* fix typo in docstring

* remove resolved TODO

* refactor docstring
2022-03-13 16:48:09 +08:00
Yuan Liu 755f8b5b59
[Refactor]: Change scheduler to param_scheduler (#121)
* [Refactor]: Change scheduler to param_scheduler

* [Fix]: Fix UT of param scheduler hook

Co-authored-by: Your <you@example.com>
2022-03-12 10:47:06 +08:00
Yining Li 61fecabea6
[Feature] Update evaluator prefix (#114)
* update evaluator prefix

* update docstring and comments

* update doc
2022-03-10 17:25:20 +08:00
liukuikun 3e0c064f49
[Feature] NaiveVisualizationHook (#98)
* [WIP] testvisualizationhook

* add TestNaiveVisualizationHook

* fix comment

* unpad

* batch imdenormalize

* fix comment

* fix comment
2022-03-10 17:22:31 +08:00
Mashiro 02ceaedb82
[Enhancement] Config support deep copy (#116)
* Config support deep copy

* Fix end of line
2022-03-10 16:01:18 +08:00
Mashiro ec3034b765
[Fix] Fix output argument of after_iter, train_after_ter and val_after_iter (#115)
* Fix hook

* Fix

* Fix docs

* FIx

* Fix

* Fix as comment
2022-03-09 23:10:19 +08:00
liukuikun 3bdd27c4e2
[Fix] add required lib for vis (#111)
* add required lib for vis

* fix comment
2022-03-09 22:11:05 +08:00
Haian Huang(深度眸) 98097bb73e
[Update] Refine visualization docs (#107)
* refine visualization

* refine visualization

* epdate

* update

* update

* update

* fix comment
2022-03-09 20:43:29 +08:00
Mashiro 2e4ae21f2b
Fix test_get_max_memory (#113) 2022-03-09 14:05:25 +08:00
RangiLyu c499d72691
fix evaluator unit test (#112) 2022-03-08 20:07:32 +08:00
Mashiro 6f69039ca9
[Feature] Add LoggerHook (#77)
* add logger hook

* update

* update

* update test

* update

* update test

* update

* update

* update

* update

* update

* Add logger hook

* Fix pre-commit

* Fix as comment

* Fix as comment

* Fix as comment

* Fix as comment

* Fix as comment

* Fix bytes

* update

* Fix as comment

* Fix as comment

* Update runner

* Fix as comment

* Fix as comment

* Fix as comment

* Fix as comment
2022-03-08 16:10:30 +08:00
RangiLyu 49b7d0ce6f
Support default_scope when building optimizer and evaluator. (#109)
* Support default_scope when building optimizer and evaluator.

* add docstring

* fix

* fix
2022-03-08 16:05:29 +08:00
RangiLyu be6f18988e
Update type hint and unit tests of evaluator. (#110)
* update type hint and ut for evaluator

* update doc

* fix
2022-03-08 16:04:46 +08:00
Yining Li cfccabc657
[Feature] Support metric prefix in BaseEvaluator (#82)
* support metric prefix in BaseEvaluator

* update docs

* use class attribute default_prefix
2022-03-08 15:12:11 +08:00
Haian Huang(深度眸) 824be950b9
Add writer (#74)
* add writer

* update

* update

* update docstr

* update unittest

* update unittest

* fix comment

* update unittest

* fix comment

* fix comment

* fix comment

* fix comment

* update
2022-03-07 22:39:25 +08:00
liukuikun 7acaca1445
Visualizer (#67)
* [WIP] Visualizer

* add draw featmap

* update docstring

* update docstring

* update docstring

* fix comment

* fix comment

* fix comment

* fix comment

* fix comment
2022-03-07 22:06:32 +08:00
Tao Gong 5d441563d0
[Refactor]: Change _prepare_data() to prepare_data() in BaseDataset (#108) 2022-03-07 21:13:58 +08:00
Zaida Zhou ed8dcb4c61
fix type hint in hooks (#106) 2022-03-07 19:35:37 +08:00
Yining Li 9f0d1a9628
fix mmdet->mmengine in build.yml (#104) 2022-03-07 18:12:15 +08:00
Yuan Liu f8a4c31037
[Fix]: Change after inter and epoch to after train iter and epoch (#100)
* [Fix]: Change after inter and epoch to after train iter and epoch

* [Fix]: Add new UT to param scheduler hook

Co-authored-by: Your <you@example.com>
2022-03-07 14:01:23 +08:00
Yuan Liu be9971781e
[Fix]: Change the type of runner in docstring to Runner (#103)
* [Fix]: Change after inter and epoch to after train iter and epoch

* [Fix]: Add new UT to param scheduler hook

* [Fix]: Change the type of runner in docstring to Runner

Co-authored-by: Your <you@example.com>
2022-03-07 14:00:05 +08:00
Yuan Liu 15abb061ef
[Fix]: Fix data batch type in base hook (#99)
* [Fix]: Fix data batch type in base hook

* [Fix]: Fix the type hint bug in checkpoint, optimizer, param scheduler hooks

Co-authored-by: Your <you@example.com>
2022-03-07 13:25:45 +08:00
liukuikun 3adf4ea6b8
[Fix] data element (#95) 2022-03-06 22:06:12 +08:00
Zaida Zhou 8b0e74d327
[Feature] Add checkpoint helper (#93)
* [Feature] Add checkpoint helper

* minor fix

* add comments and fix format

* add hub directory

* add missing docstring
2022-03-06 22:05:07 +08:00
Zaida Zhou ac666711ab
Add Priority (#92) 2022-03-06 17:17:04 +08:00
Mashiro 2d782b4998
[Feature] add logging UT and impl (#43)
* first commit

* update logging

* update MessageHub unit test and LogBuffer unitest

* update logging docs

* update logging impl and test

* update test logging

* update test

* update logging test and impl

* Fix logging test

* Update log_buffer.py

* rename statistics argumentes

* fix as comment

* Fix as comment

* Fix as comment

* Fix as comment

* Fix meta class

* Fix as comment

* Fix as comment

* Fix as comment

* Fix name declare

* Fix as comment

* Fix as comment

* Fix as comment

* Fix docstring

* Fix as comment

* Fix as comment
2022-03-06 17:09:12 +08:00
Zaida Zhou e146233546
fix documentation (#91) 2022-03-06 14:36:46 +08:00
Zaida Zhou c6a8d72c5e
[Feature] Add distributed module (#59)
* [Feature] Add distributed module

* fix IS_DIST error

* all_reduce_dict does operations in-place

* support 'mean' operation

* provide local group process

* add tmpdir argument for collect_results

* add unit tests

* refactor unit tests

* simplify steps to create multiple processes

* minor fix

* describe the different of *gather* in mmengine and pytorch

* minor fix

* add unit tests for nccl

* test nccl backend in multiple gpu

* add get_default_group function to handle different torch versions

* minor fix

* [Feature] Add distributed module

* fix IS_DIST error

* all_reduce_dict does operations in-place

* support 'mean' operation

* provide local group process

* add tmpdir argument for collect_results

* add unit tests

* refactor unit tests

* simplify steps to create multiple processes

* minor fix

* describe the different of *gather* in mmengine and pytorch

* minor fix

* add unit tests for nccl

* test nccl backend in multiple gpu

* add get_default_group function to handle different torch versions

* minor fix

* minor fix

* handle torch1.5

* handle torch1.5

* minor fix

* fix typo

* refactor unit tests

* nccl does not support gather and gather_object

* fix gather

* fix collect_results_cpu

* fix collect_results and refactor unit tests

* fix collect_results unit tests

* handle torch.cat in torch1.5

* refine docstring

* refine docstring

* fix comments

* fix comments
2022-03-05 22:03:32 +08:00
Wenwei Zhang 817eb89ac2
Add user docs for get_config/get_models (#83)
* add user docs for get_config/get_models

* resolve comments
2022-03-05 17:46:27 +08:00
Zaida Zhou fd85156412
fix type hint and format (#88) 2022-03-05 17:44:31 +08:00
liukuikun 11b38b12d6
[Fix] fix random may generate same value (#85) 2022-03-05 15:02:54 +08:00
Miao Zheng 43dcc5648f
[Fix] Typo in Logging doc (#81) 2022-03-04 19:44:31 +08:00
RangiLyu 05ff046c3b
ignore ut of unimplemented modules (#79) 2022-03-04 11:09:10 +08:00
RangiLyu 64b1d183b9
Add runner unit tests. (#68)
* add runner unit tests

* update

* update

* add test custom loop and hook

* add test model wrapper

* add test setup env

* fix typo

* fix launcher

* fix typo

* test default scope

* add logger test

* fix dataloader

* add test loop

* resolve comments

* resolve comments
2022-03-03 19:44:36 +08:00
Mashiro c87adc66a4
Fix BaseGlobalAssible in Python 3.7 (#76)
Fix CI
2022-03-03 19:43:16 +08:00
Yifei Yang 149248ce52
[Feature] Add Sync Buffer Hook (#57)
* [Feature]: Add Part3 of Hooks

* [Feature]: Add Hook

* [Fix]: Add docstring and type hint for base hook

* add sync buffer hook

* update typing hint and docs

* fix lint

* fix mypy

* fix lint

* use mock from unittest

Co-authored-by: seuyou <3463423099@qq.com>
2022-03-03 17:13:43 +08:00
Mashiro 12b916cf6f
[Feature] add base global access (#72)
* add impl

* add test

* Fix as comment

* Fix as comment

* Fix as comment

* Fix as comment
2022-03-02 23:38:04 +08:00
Yuan Liu cf239a2b17
[Feature]: Add checkpoint hook (#66)
* [Feature]: Add checkpoint hook

* [Fix]: Fix lint

* [Fix]: Delete redundant optional and give an example to our_dir

* [Feature]: Add test the last_ckpt in UT

* [Fix]: Fix docstring problem

* [Fix]: Add patch to UT

* [Feature]: Add Test case for by epoch
2022-03-02 22:01:58 +08:00
Yifei Yang 2448380339
use unittest mock (#71) 2022-03-02 21:53:40 +08:00
Yifei Yang 94ab45d07e
[Feature] Add empty cache hook (#58)
* [Feature]: Add Part3 of Hooks

* [Feature]: Add Hook

* [Fix]: Add docstring and type hint for base hook

* [Fix]: Add test case to not the last iter, inner_iter, epoch

* [Fix]: Add missing type hint

* [Feature]: Add Args and Returns in docstring

* [Fix]: Add missing colon

* [Fix]: Add optional to docstring

* [Fix]: Fix docstring problem

* [Fix]: Fix lint

* fix lint

* update typing and docs

* fix lint

* Update mmengine/hooks/empty_cache_hook.py

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>

* Update mmengine/hooks/empty_cache_hook.py

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>

* Update mmengine/hooks/empty_cache_hook.py

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>

* Update tests/test_hook/test_empty_cache_hook.py

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>

* fix lint

* fix comments

* remove test condition

Co-authored-by: seuyou <3463423099@qq.com>
Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>
2022-03-02 14:04:41 +08:00
Yuan Liu 63a3af4f8c
[Feature]: Add optimizer hook (#70)
* [Feature]: Add optimizer hook

* [Fix]: Update docstring

* [Fix]: Add call with in UT
2022-03-01 17:42:15 +08:00
RangiLyu ee95ce2488
[Doc] Add runner tutorial. (#55)
* [Doc] Add runner tutorial.

* update

* add advanced doc

* refine and add flow chart

* update

* update
2022-03-01 15:58:34 +08:00