Commit Graph

3 Commits (65eb39bfc6f3e6d7dcbc2fa58cac9bee17ef7cdf)

Author SHA1 Message Date
Hakjin Lee 0857f9fb40
[Feature] Support torch ZeroRedundancyOptimizer (#551)
* [Feature] Support torch ZeRORedundancyOptimizer

Co-authored-by: Junhwa Song <ethan9867@gmail.com>
Signed-off-by: Junhwa Song <ethan9867@gmail.com>
Signed-off-by: Hakjin Lee <nijkah@gmail.com>

* lint

* Fix saving optimizer state_dict

* Fix handling import error

* Add test case

* fix UT

* Revert "fix UT"

This reverts commit dd64538960.

* fix handling import in UT

* Fix saving zero checkpoint and delete redundant master_only

* lint

* test unittest

* Fix handling impor error

* Fix UT condition

* Edit docstrings

* Fix typo

* Skip redundant procudure in checkpoint hook

* fix typo again

* Update mmengine/optim/optimizer/zero_optimizer.py

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>

* Add api info

* lint

* Fix lint

* Handling AmpOptimWrapper case

* handling overlap_with_ddp

* Fix error

Signed-off-by: Junhwa Song <ethan9867@gmail.com>
Signed-off-by: Hakjin Lee <nijkah@gmail.com>
Co-authored-by: Junhwa Song <ethan9867@gmail.com>
Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>
2022-10-27 20:31:50 +08:00
Zaida Zhou 458f6d684e
[Docs] Refactor docs structure (#494)
* [WIP] Refactor-docs-structure

* [Docs] Check link and fix rst  (#498)

* check link

* fix rst

* Fix conf.py

* Fix lint

* minor refine

* minor refine

* modify zh_readme and index.rst

* add migration exampls get_started design to en

* add tutorials to en

* correct chinese doc link

* Apply suggestions from code review

* Update docs/en/advanced_tutorials/distributed.md

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>

Co-authored-by: Mashiro <57566630+HAOCHENYE@users.noreply.github.com>
2022-08-31 21:18:55 +08:00
Zaida Zhou 3f5e08fa4a
[Docs] Make API of docs clearer (#469)
* [Docs] Make API of docs clearer

* minor fix

* fix md format
2022-08-30 16:27:20 +08:00