Hakjin Lee
|
0857f9fb40
|
[Feature] Support torch ZeroRedundancyOptimizer (#551)
* [Feature] Support torch ZeRORedundancyOptimizer
Co-authored-by: Junhwa Song <ethan9867@gmail.com>
Signed-off-by: Junhwa Song <ethan9867@gmail.com>
Signed-off-by: Hakjin Lee <nijkah@gmail.com>
* lint
* Fix saving optimizer state_dict
* Fix handling import error
* Add test case
* fix UT
* Revert "fix UT"
This reverts commit dd64538960 .
* fix handling import in UT
* Fix saving zero checkpoint and delete redundant master_only
* lint
* test unittest
* Fix handling impor error
* Fix UT condition
* Edit docstrings
* Fix typo
* Skip redundant procudure in checkpoint hook
* fix typo again
* Update mmengine/optim/optimizer/zero_optimizer.py
Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>
* Add api info
* lint
* Fix lint
* Handling AmpOptimWrapper case
* handling overlap_with_ddp
* Fix error
Signed-off-by: Junhwa Song <ethan9867@gmail.com>
Signed-off-by: Hakjin Lee <nijkah@gmail.com>
Co-authored-by: Junhwa Song <ethan9867@gmail.com>
Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>
|
2022-10-27 20:31:50 +08:00 |