mirrors/mmengine
1
0
Fork 0
mirror of https://github.com/open-mmlab/mmengine.git synced 2025-06-03 21:54:44 +08:00
Code Issues Projects Releases Wiki Activity
630 Commits 31 Branches 27 Tags
Commit Graph

22 Commits

Author SHA1 Message Date
Mashiro
dbae83c52f
[Enhancement] Replace warnings.warn with print_log (#961) 
* Replace warning with print_log

* Add comments for testing warning
 2023-03-06 17:25:28 +08:00
Qian Zhao
b1b1f53db2
[Fix] Fix optim_wrapper unittest for pytorch <= 1.10.0 (#975) 2023-03-02 14:14:23 +08:00
Qian Zhao
2ed8e343a0
[Feature] Enable bf16 in AmpOptimWrapper (#960) 
* support bf16 in AmpOptimWrapper

* add docstring

* modify docs

* add unittests for bf16 in AmpOptimWrapper

* fix type

* fix to pass ci

* fix ut skip logic to pass ci

* fix as comment

* add type hints

* fix docstring and add warning information

* remove check for pytorch>=1.6 in unittest

* modify unittest

* modify unittest

* remove torch.float32 && torch.float64 from valid dtypes

* fix as comments

* minor refine docstring

* fix unittest parameterized to pass CI

* fix unittest && add back torch.float32, torch.float64
 2023-03-01 21:35:18 +08:00
Zaida Zhou
fc9518e2c1
[Feature] Add Lion optimizer (#952) 2023-02-23 11:24:50 +08:00
whcao
a5f48f7d99
[Bug] Fix the bug when the params in shared modules do not require grad (#903) 
* fix the bug when the params in shared modules do not require grad

* test DefaultOptimWrapperConstructor when the params in shared modules do not require grad
 2023-02-15 11:25:15 +08:00
xcnick
e35ed5fd2e
[Feature] Add ApexOptimWrapper (#742) 
* add ApexOptimWrapper

* typo fix

* add apex amp.initialize in optim_context

* assert apex_amp

* polish code

* add parameters of apex_amp.initialize

* add docs

* polish code

* polish code

* polish code

* fix calling of apex amp load_state_dict

* polish

* add comments

* Update apex_optimizer_wrapper.py

* Update apex_optimizer_wrapper.py

---------

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>
 2023-02-06 15:30:10 +08:00
takuoko
d1d4609fa2
[Feature] Support using optimizers from dadaptation (#902) 
* add dadaptation

* Update mmengine/optim/optimizer/builder.py

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>

* update dadaptation docs

---------

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>
 2023-02-03 15:00:32 +08:00
Qian Zhao
4147e976a6
[Fix] ZeroRedundancyOptimizer ambiguous error with param groups when pytorch < 1.12.0 (#818) 
* fix zero_optimizer error with param groups when pytorch < 1.12.0

* add docstring

* fix docstring

* add unittest

* change ut to use a valid paramwise_cfg

* modify ut

* fix as comments
 2022-12-19 17:07:58 +08:00
RangiLyu
3582b4c787
[Enhance] Support setting decay multiplier for flatten parameter (#771) 
* [Fix] Fix bias decay mult of depth-wise conv.

* support flatten param weight decay multiplier

* add unit tests

* REMOVE TODO

* update doc
 2022-12-16 17:37:22 +08:00
Hakjin Lee
0857f9fb40
[Feature] Support torch ZeroRedundancyOptimizer (#551) 
* [Feature] Support torch ZeRORedundancyOptimizer

Co-authored-by: Junhwa Song <ethan9867@gmail.com>
Signed-off-by: Junhwa Song <ethan9867@gmail.com>
Signed-off-by: Hakjin Lee <nijkah@gmail.com>

* lint

* Fix saving optimizer state_dict

* Fix handling import error

* Add test case

* fix UT

* Revert "fix UT"

This reverts commit dd64538960ff7440c6020f533d43945ffc23f2d2.

* fix handling import in UT

* Fix saving zero checkpoint and delete redundant master_only

* lint

* test unittest

* Fix handling impor error

* Fix UT condition

* Edit docstrings

* Fix typo

* Skip redundant procudure in checkpoint hook

* fix typo again

* Update mmengine/optim/optimizer/zero_optimizer.py

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>

* Add api info

* lint

* Fix lint

* Handling AmpOptimWrapper case

* handling overlap_with_ddp

* Fix error

Signed-off-by: Junhwa Song <ethan9867@gmail.com>
Signed-off-by: Hakjin Lee <nijkah@gmail.com>
Co-authored-by: Junhwa Song <ethan9867@gmail.com>
Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>
 2022-10-27 20:31:50 +08:00
Mashiro
6073d9ebd8
[Enhance] add documents for clip_grad , and support clip grad by value. (#513) 
* [Enhance] add documents for , and support clip grad by value

* refine docstring

* fix as comment

* Fix as comment

* minor refine

* minor refine

* remove error comment for clip grad

* refine docstring
 2022-10-18 18:02:46 +08:00
Mashiro
5d7527d75e
[Fix ] Fix unit test could may fail caused by MultiProcessTestCase (#535) 
* fix merge ci

* make timeout larger

* tmp test merge

* debug

* sleep 1 in optimizer ut

* skip test clip grad

* skip test clip grad

* fix merge ci
 2022-09-15 18:08:56 +08:00
Zaida Zhou
7e1d7af2d9
[Refactor] Refactor code structure (#395) 
* Rename data to structure

* adjust the way to import module

* adjust the way to import module

* rename Structure to Data Structures in docs api

* rename structure to structures

* support using some modules of mmengine without torch

* fix circleci config

* fix circleci config

* fix registry ut

* minor fix

* move init method from model/utils to model/weight_init.py

* move init method from model/utils to model/weight_init.py

* move sync_bn to model

* move functions depending on torch to dl_utils

* format import

* fix logging ut

* add weight init in model/__init__.py

* move get_config and get_model to mmengine/hub

* move log_processor.py to mmengine/runner

* fix ut

* Add TimeCounter in dl_utils/__init__.py
 2022-08-24 19:14:07 +08:00
Zaida Zhou
486d8cda56
[Refactor] Refactor the import rule (#459) 
* [Refactor] Refactor the import rule

* minor refinement

* add a comment
 2022-08-23 18:58:36 +08:00
Mashiro
6b47035fdf
[Fix] Fix save scheduler state dict with optim wrapper (#375) 
* fix save scheduler state dict with optim wrapper

* remove for loop and inherit TestParameterScheduler

* remove for loop and inherit TestParameterScheduler

* minor refine
 2022-07-20 16:32:48 +08:00
Mashiro
b2ee9f8b11
[Fix] Fix loss could be nan in optimizer wrapper (#345) 
* fix optimizer wrapper counts

* fix ut
 2022-07-06 16:42:49 +08:00
Mashiro
2fd6beb972
[Fix] Fix UT of optimizer wrapper failed in pytorch1.6 (#340) 2022-06-28 10:31:14 +08:00
Mashiro
b7866021c4
[Refactor] Refactor the accumulate gradient implemention of OptimWrapper (#284) 
* merge context

* update unit test

* add docstring

* fix bug in AmpOptimWrapper

* add docstring for backward

* add warning and docstring for accumuate gradient

* fix docstring

* fix docstring

* add params_group method

* fix as comment

* fix as comment

* make default_value of loss_scale to dynamic

* Fix docstring

* decouple should update and should no sync

* rename attribute in OptimWrapper

* fix docstring

* fix comment

* fix comment

* fix as comment

* fix as comment and add unit test
 2022-06-13 23:20:53 +08:00
Mashiro
3e3866c1b9
[Feature] Add optimizer wrapper (#265) 
* Support multiple optimizers

* minor refinement

* improve unit tests

* minor fix

* Update unit tests for resuming or saving ckpt for multiple optimizers

* refine docstring

* refine docstring

* fix typo

* update docstring

* refactor the logic to build multiple optimizers

* resolve comments

* ParamSchedulers spports multiple optimizers

* add optimizer_wrapper

* fix comment and docstirng

* fix unit test

* add unit test

* refine docstring

* RuntimeInfoHook supports printing multi learning rates

* resolve comments

* add optimizer_wrapper

* fix mypy

* fix lint

* fix OptimizerWrapperDict docstring and add unit test

* rename OptimizerWrapper to OptimWrapper, OptimWrapperDict inherit OptimWrapper, and fix as comment

* Fix AmpOptimizerWrapper

* rename build_optmizer_wrapper to build_optim_wrapper

* refine optimizer wrapper

* fix AmpOptimWrapper.step, docstring

* resolve confict

* rename DefaultOptimConstructor

* fix as comment

* rename clig grad auguments

* refactor optim_wrapper config

* fix docstring of DefaultOptimWrapperConstructor

fix docstring of DefaultOptimWrapperConstructor

* add get_lr method to OptimWrapper and OptimWrapperDict

* skip some amp unit test

* fix unit test

* fix get_lr, get_momentum docstring

* refactor get_lr, get_momentum, fix as comment

* fix error message

Co-authored-by: zhouzaida <zhouzaida@163.com>
 2022-06-01 18:04:38 +08:00
RangiLyu
49b7d0ce6f
Support default_scope when building optimizer and evaluator. (#109) 
* Support default_scope when building optimizer and evaluator.

* add docstring

* fix

* fix
 2022-03-08 16:05:29 +08:00
Zaida Zhou
1e79b97444
Mock unimplemented modules and fix unit tests (#54) 
* Mock unimplemented modules and fix unit tests

* add a comment
 2022-02-25 15:24:27 +08:00
RangiLyu
7353778b7c
[Feature]: Add optimzier and constructor. (#25) 
* [Feature]: Add optimzier and constructor.

* refactor unit tests

* add optimizer doc

* add parrots wrapper

* add parrots wrapper

* solve comments

* resolve comments
 2022-02-19 14:09:37 +08:00