Commit Graph

5 Commits (3dc2be05d5b79099344a62ac63fbfe2b49230e64)

Author SHA1 Message Date
xcnick e35ed5fd2e
[Feature] Add ApexOptimWrapper (#742)
* add ApexOptimWrapper

* typo fix

* add apex amp.initialize in optim_context

* assert apex_amp

* polish code

* add parameters of apex_amp.initialize

* add docs

* polish code

* polish code

* polish code

* fix calling of apex amp load_state_dict

* polish

* add comments

* Update apex_optimizer_wrapper.py

* Update apex_optimizer_wrapper.py

---------

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>
2023-02-06 15:30:10 +08:00
LEFTeyes 0b59a90a21
[Feature] Support ReduceOnPlateauParamScheduler(#819)
* [Feature] Add ReduceOnPlateauParamScheduler and change ParamSchedulerHook

* [Feature] add ReduceOnPlateauLR and ReduceOnPlateauMomentum

* pre-commit check

* add a little docs

* change position

* fix the conflict between isort and yapf

* fix ParamSchedulerHook after_val_epoch execute without train_loop and param_schedulers built

* Apply suggestions from code review

Co-authored-by: Mashiro <57566630+HAOCHENYE@users.noreply.github.com>

* update ReduceOnPlateauParamScheduler, ReduceOnPlateauMomentum and ParamSchedulerHook

* fix get need_step_args attribute error in ParamSchedulerHook

* fix load_state_dict error for rule in ReduceOnPlateauParamScheduler

* add docs for ParamSchedulerHook and fix a few codes

* [Docs] add ReduceOnPlateauParamScheduler, ReduceOnPlateauMomentum and ReduceOnPlateauLR docs

* [Refactor] adjust the order of import

* [Fix] add init check for threshold in ReduceOnPlateauParamScheduler

* [Test] add test for ReduceOnPlateauParamScheduler, ReduceOnPlateauLR and ReduceOnPlateauMomentum

* [Fix] fix no attribute self.min_value

* [Fix] fix numerical problem in tests

* [Fix] fix error in tests

* [Fix] fix ignore first param in tests

* [Fix] fix bug in tests

* [Fix] fix bug in tests

* [Fix] fix bug in tests

* [Fix] increase coverage

* [Fix] fix count self._global_step bug and docs

* [Fix] fix tests

* [Fix] modified ParamSchedulerHook test

* Update mmengine/optim/scheduler/param_scheduler.py

Co-authored-by: Mashiro <57566630+HAOCHENYE@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Mashiro <57566630+HAOCHENYE@users.noreply.github.com>

* [Fix] modified something according to commented

* [Docs] add api for en and zh_cn

* [Fix] fix bug in test_param_scheduler_hook.py

* [Test] support more complicated test modes(less, greater, rel, abs) for ReduceOnPlateauParamScheduler

* [Docs] add docs for rule

* [Fix] fix pop from empty list bug in test

* [Fix] fix check param_schedulers is not built bug

* [Fix] fix step_args bug and without runner._train_loop bug

* [Fix] fix step_args bug and without runner._train_loop bug

* [Fix] fix scheduler type bug

* [Test] rename step_args to step_kwargs

* [Fix] remove redundancy check

* [Test] remove redundancy check

* Apply suggestions from code review

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>

* [Test] fix some defects

Co-authored-by: Mashiro <57566630+HAOCHENYE@users.noreply.github.com>
Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>
2023-01-16 11:39:03 +08:00
Hakjin Lee 0857f9fb40
[Feature] Support torch ZeroRedundancyOptimizer (#551)
* [Feature] Support torch ZeRORedundancyOptimizer

Co-authored-by: Junhwa Song <ethan9867@gmail.com>
Signed-off-by: Junhwa Song <ethan9867@gmail.com>
Signed-off-by: Hakjin Lee <nijkah@gmail.com>

* lint

* Fix saving optimizer state_dict

* Fix handling import error

* Add test case

* fix UT

* Revert "fix UT"

This reverts commit dd64538960.

* fix handling import in UT

* Fix saving zero checkpoint and delete redundant master_only

* lint

* test unittest

* Fix handling impor error

* Fix UT condition

* Edit docstrings

* Fix typo

* Skip redundant procudure in checkpoint hook

* fix typo again

* Update mmengine/optim/optimizer/zero_optimizer.py

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>

* Add api info

* lint

* Fix lint

* Handling AmpOptimWrapper case

* handling overlap_with_ddp

* Fix error

Signed-off-by: Junhwa Song <ethan9867@gmail.com>
Signed-off-by: Hakjin Lee <nijkah@gmail.com>
Co-authored-by: Junhwa Song <ethan9867@gmail.com>
Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>
2022-10-27 20:31:50 +08:00
Zaida Zhou 114fa21522
[Docs] Fix API files of en docs (#525)
* [Docs] Fix api files of en docs

* minor fix
2022-09-13 14:57:18 +08:00
Zaida Zhou 3f5e08fa4a
[Docs] Make API of docs clearer (#469)
* [Docs] Make API of docs clearer

* minor fix

* fix md format
2022-08-30 16:27:20 +08:00