mirrors/pytorch-image-models
1
0
Fork 0
mirror of https://github.com/huggingface/pytorch-image-models.git synced 2025-06-03 15:01:08 +08:00
Code Issues Projects Releases Wiki Activity

Update notes in ScaledStdConv impl

Browse Source
This commit is contained in:
Ross Wightman 2021-02-18 12:44:08 -08:00
parent 678ba4e0a2
commit 8563609b28
1 changed files with 4 additions and 3 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

7
timm/models/layers/std_conv.py
View File

@ -70,6 +70,8 @@ class ScaledStdConv2d(nn.Conv2d):
Paper: `Characterizing signal propagation to close the performance gap in unnormalized ResNets` - Paper: `Characterizing signal propagation to close the performance gap in unnormalized ResNets` -
https://arxiv.org/abs/2101.08692 https://arxiv.org/abs/2101.08692
NOTE: the operations used in this impl differ slightly from the DeepMind Haiku impl. The impact is minor.
""" """
def __init__( def __init__(
@ -100,11 +102,10 @@ class ScaledStdConv2d(nn.Conv2d):
class ScaledStdConv2dSame(nn.Conv2d): class ScaledStdConv2dSame(nn.Conv2d):
"""Conv2d layer with Scaled Weight Standardization and Tensorflow-like SAME padding support """Conv2d layer with Scaled Weight Standardization and Tensorflow-like SAME padding support
NOTE: operations and default eps slightly changed from non-SAME impl to closer match Deepmind Haiku impl.
Fore the sake of completeness, numeric differences are minor with arprox .005 top-1 difference.
Paper: `Characterizing signal propagation to close the performance gap in unnormalized ResNets` - Paper: `Characterizing signal propagation to close the performance gap in unnormalized ResNets` -
https://arxiv.org/abs/2101.08692 https://arxiv.org/abs/2101.08692
NOTE: the operations used in this impl differ slightly from the DeepMind Haiku impl. The impact is minor.
""" """
def __init__( def __init__(