Commit Graph

12 Commits (c5e0d1c700de2e39441af9b93f745aadf34be878)

Author SHA1 Message Date
Ross Wightman f670d98cb8 Make a few more layers symbolically traceable (remove from FX leaf modules)
* remove dtype kwarg from .to() calls in EvoNorm as it messed up script + trace combo
* BatchNormAct2d always uses custom forward (cut & paste from original) instead of super().forward. Fixes #1176
* BlurPool groups==channels, no need to use input.dim[1]
2022-03-24 21:43:56 -07:00
Ross Wightman 372ad5fa0d Significant model refactor and additions:
* All models updated with revised foward_features / forward_head interface
* Vision transformer and MLP based models consistently output sequence from forward_features (pooling or token selection considered part of 'head')
* WIP param grouping interface to allow consistent grouping of parameters for layer-wise decay across all model types
* Add gradient checkpointing support to a significant % of models, especially popular architectures
* Formatting and interface consistency improvements across models
* layer-wise LR decay impl part of optimizer factory w/ scale support in scheduler
* Poolformer and Volo architectures added
2022-02-28 13:56:23 -08:00
Ross Wightman abc9ba2544 Transitioning default_cfg -> pretrained_cfg. Improving handling of pretrained_cfg source (HF-Hub, files, timm config, etc). Checkpoint handling tweaks. 2022-01-25 21:54:13 -08:00
Ross Wightman 78912b6375 Updated EvoNorm implementations with some experimentation. Add FilterResponseNorm. Updated RegnetZ and ResNetV2 model defs for trials. 2021-12-01 12:09:01 -08:00
Ross Wightman 1076a65df1 Minor post FX merge cleanup 2021-11-18 19:47:07 -08:00
Alexander Soare 65d827c7a6 rename notrace registration and standardize trace_utils imports 2021-11-15 21:03:21 +00:00
Alexander Soare d2994016e9 Add try/except guards 2021-11-12 21:16:53 +00:00
Alexander Soare b25ff96768 wip - pre-rebase 2021-11-12 20:45:05 +00:00
Alexander Soare cf4561ca72 Add FX based FeatureGraphNet capability 2021-11-12 20:45:05 +00:00
Alexander Soare 0149ec30d7 wip - attempting to rebase 2021-11-12 20:45:05 +00:00
Alexander Soare 02c3a75a45 wip - make it possible to use fx graph in train and eval mode 2021-11-12 20:45:05 +00:00
Alexander Soare ab3ac3f25b Add FX based FeatureGraphNet capability 2021-11-12 20:45:05 +00:00