mirrors
/
pytorch-image-models
mirror of https://github.com/huggingface/pytorch-image-models.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
1,533 Commits
136 Branches
65 Tags
47 MiB
Commit Graph

30 Commits (285771972eed07068c3beccce11d9e3439132257)

Author SHA1 Message Date
Ross Wightman b1b024dfed Scheduler update, add v2 factory method, support scheduling on updates instead of just epochs. Add LR to summary csv. Add lr_base scaling calculations to train script. Fix #1168 2022-10-07 10:43:04 -07:00
Ross Wightman 2a296412be Add Adan optimizer 2022-09-23 16:05:52 -07:00
Ross Wightman 33e30f8c8b Remove layer-decay print 2022-09-18 21:33:03 -07:00
Ross Wightman 0557c8257d Fix bug introduced in non layer_decay weight_decay application. Remove debug print, fix arg desc. 2022-02-28 17:06:32 -08:00
Ross Wightman 372ad5fa0d Significant model refactor and additions: 
* All models updated with revised foward_features / forward_head interface
* Vision transformer and MLP based models consistently output sequence from forward_features (pooling or token selection considered part of 'head')
* WIP param grouping interface to allow consistent grouping of parameters for layer-wise decay across all model types
* Add gradient checkpointing support to a significant % of models, especially popular architectures
* Formatting and interface consistency improvements across models
* layer-wise LR decay impl part of optimizer factory w/ scale support in scheduler
* Poolformer and Volo architectures added
 2022-02-28 13:56:23 -08:00
Ross Wightman c207e02782 MOAR optimizer changes. Woo! 2021-08-18 22:20:35 -07:00
Ross Wightman a426511c95 More optimizer cleanup. Change all to no longer use .data. Improve (b)float16 use with adabelief. Add XLA compatible Lars. 2021-08-18 17:21:56 -07:00
Ross Wightman a6af48be64 add madgradw optimizer 2021-08-17 22:19:27 -07:00
Ross Wightman 55fb5eedf6 Remove experiment from lamb impl 2021-08-17 21:48:26 -07:00
Ross Wightman 8a9eca5157 A few optimizer comments, dead import, missing import 2021-08-17 18:01:33 -07:00
Ross Wightman ac469b50da Optimizer improvements, additions, cleanup 
* Add MADGRAD code
* Fix Lamb (non-fused variant) to work w/ PyTorch XLA
* Tweak optimizer factory args (lr/learning_rate and opt/optimizer_name), may break compat
* Use newer fn signatures for all add,addcdiv, addcmul in optimizers
* Use upcoming PyTorch native Nadam if it's available
* Cleanup lookahead opt
* Add optimizer tests
* Remove novograd.py impl as it was messy, keep nvnovograd
* Make AdamP/SGDP work in channels_last layout
* Add rectified adablief mode (radabelief)
* Support a few more PyTorch optim, adamax, adagrad
 2021-08-17 17:51:20 -07:00
Ross Wightman 1042b8a146 Add non fused LAMB optimizer option 2021-08-09 13:13:43 -07:00
Ross Wightman cd3dc4979f Fix adabelief imports, remove prints, preserve memory format is the default arg for zeros_like 2021-04-12 08:25:31 -07:00
juntang addfc7c1ac adabelief 2021-04-04 23:48:15 -04:00
Ross Wightman 37c71a5609 Some further create_optimizer_v2 tweaks, remove some redudnant code, add back safe model str. Benchmark step times per batch. 2021-04-01 22:34:55 -07:00
Ross Wightman 288682796f Update benchmark script to add precision arg. Fix some downstream (DeiT) compat issues with latest changes. Bump version to 0.4.7 2021-04-01 16:40:12 -07:00
Ross Wightman 0e16d4e9fb Add benchmark.py script, and update optimizer factory to be more friendly to use outside of argparse interface. 2021-02-23 15:38:12 -08:00
Jasha 7c56c718f3 Configure create_optimizer with args.opt_args 
Closes #301
 2020-12-08 00:03:09 -06:00
Ross Wightman 30ab4a1494 Fix issue in optim factory with sgd / eps flag. Bump version to 0.3.1 2020-10-31 18:05:30 -07:00
Ross Wightman f944242cb0 Fix #262, num_classes arg mixup. Make vision_transformers a bit closer to other models wrt get/reset classfier/forward_features. Fix torchscript for ViT. 2020-10-29 13:58:28 -07:00
Ross Wightman 477a78ed81 Fix optimizer factory regressin for optimizers like sgd/momentum that don't have an eps arg 2020-10-22 15:59:47 -07:00
Ross Wightman a4d8fea61e Add model based wd skip support. Improve cross version compat of optimizer factory. Fix #247 2020-10-13 12:49:47 -07:00
Ross Wightman 80078c47bb Add Adafactor and Adahessian optimizers, cleanup optimizer arg passing, add gradient clipping support. 2020-10-09 17:24:43 -07:00
Ross Wightman 7995295968 Merge branch 'logger' into features. Change 'logger' to '_logger'. 2020-07-27 18:00:46 -07:00
Ross Wightman 6c17d57a2c Fix some attributions, add copyrights to some file docstrings 2020-07-27 13:44:56 -07:00
Sangdoo Yun e93e571f7a Add `adamp` and 'sgdp' optimizers. 
Update requirements.txt

Update optim_factory.py

Add `adamp` optimizer

Update __init__.py

copy files of adamp & sgdp

Create adamp.py

Update __init__.py

Create sgdp.py

Update optim_factory.py

Update optim_factory.py

Update requirements.txt

Update adamp.py

Update sgdp.py

Update sgdp.py

Update adamp.py
 2020-07-25 15:33:20 -07:00
Ross Wightman e6f24e5578 Add 'momentum' optimizer (SGD w/o nesterov) for stable EfficientDet training defaults 2020-04-25 19:42:13 -07:00
Ross Wightman 64966f61f7 Add Nvidia's NovogGrad impl from Jasper (cleaner/faster than current) and Apex Fused optimizers 2019-08-29 15:21:38 -07:00
Ross Wightman fac58f609a Add RAdam, NovoGrad, Lookahead, and AdamW optimizers, a few ResNet tweaks and scheduler factory tweak. 
* Add some of the trendy new optimizers. Decent results but not clearly better than the standards.
* Can create a None scheduler for constant LR
* ResNet defaults to zero_init of last BN in residual
* add resnet50d config
 2019-08-28 00:14:10 -07:00
Ross Wightman aa4354f466 Big re-org, working towards making pip/module as 'timm' 2019-06-19 17:20:51 -07:00