303f7691a1
Add cautious mars, improve test reliability by skipping grad diff for first step
2024-12-02 11:29:02 -08:00
7cf683628f
Cautious optimizer impl plus some typing cleanup.
2024-11-28 15:11:58 -08:00
aeb1ed7a15
Keep basic optim test LR range closer to before w/ updated code
2024-11-26 15:10:15 -08:00
7a165fcb62
Remove rogue import, thanks IDE :/
2024-11-26 15:10:15 -08:00
73d10ab482
Update tests, need handling for radamw with older PyTorch, need to back-off basic test LR in mars?
2024-11-26 15:10:15 -08:00
a024ab3170
Replace radam & nadam impl with torch.optim ver, rename legacy adamw, nadam, radam impl in timm. Update optim factory & tests.
2024-11-26 15:10:15 -08:00
0b5264a108
Missing optimizers in __init__.py, add bind_defaults=False for unit tests
2024-11-13 10:50:46 -08:00
dde990785e
More fixes for new factory & tests, add back adahessian
2024-11-12 20:49:01 -08:00
45490ac52f
Post merge fix reference of old param groups helper fn locations
2024-11-12 20:49:01 -08:00
ee5f6e76bb
A bit of an optimizer overhaul, added an improved factory, list_optimizers, class helper and add info classes with descriptions, arg configs
2024-11-12 20:49:01 -08:00
94e0560aba
Remove an indent level in init_group for adopt, update optim tests, adopt failing rosenbrock
2024-11-12 20:49:01 -08:00
e31e5d2d64
imports
2024-11-12 07:53:39 -08:00
68d5a64e45
extend existing unittests
2024-11-12 07:53:39 -08:00
a38c2b4475
fix bug
2023-11-16 20:27:59 -08:00
bbf0ab000c
return if cuda not available
2023-11-16 20:27:59 -08:00
e628ed7e67
device agnostic testing
2023-11-16 20:27:59 -08:00
68b2824e49
Update optim test to remove Variable/.data and fix _state_dict optim test for PyTorch 2.1 ( #1988 )
...
* Update optim test to remove Variable/.data and fix _state_dict optim test
* Attempt to run python 3.11 w/ 2.1
* Try factoring out testmarker to common var
* More fiddling
* Abandon attempt to reduce redunancy
* Another try
2023-10-12 11:53:10 -07:00
24720abe3b
Merge branch 'master' into attn_update
2021-09-13 16:51:10 -07:00
8642401e88
Swap botnet 26/50 weights/models after realizing a mistake in arch def, now figuring out why they were so low...
2021-09-05 15:17:19 -07:00
5f12de4875
Add initial AttentionPool2d that's being trialed. Fix comment and still trying to improve reliability of sgd test.
2021-09-05 12:41:14 -07:00
54e90e82a5
Another attempt at sgd momentum test passing...
2021-09-03 20:50:26 -07:00
fc894c375c
Another attempt at sgd momentum test passing...
2021-08-27 10:39:31 -07:00
708d87a813
Fix ViT SAM weight compat as weights at URL changed to not use repr layer. Fix #825 . Tweak optim test.
2021-08-27 09:20:13 -07:00
c207e02782
MOAR optimizer changes. Woo!
2021-08-18 22:20:35 -07:00
42c1f0cf6c
Fix lars tests
2021-08-18 21:05:34 -07:00
a426511c95
More optimizer cleanup. Change all to no longer use .data. Improve (b)float16 use with adabelief. Add XLA compatible Lars.
2021-08-18 17:21:56 -07:00
a6af48be64
add madgradw optimizer
2021-08-17 22:19:27 -07:00
55fb5eedf6
Remove experiment from lamb impl
2021-08-17 21:48:26 -07:00
959eaff121
Add optimizer tests and update testing to pytorch 1.9
2021-08-17 17:59:15 -07:00
Ross Wightman
mrT23
Tal
Alex Cunha
Ross Wightman