mirrors/pytorch-image-models
1
0
Fork 0
mirror of https://github.com/huggingface/pytorch-image-models.git synced 2025-06-03 15:01:08 +08:00
Code Issues Projects Releases Wiki Activity

Should have included Conv2d layers in original weight init. Lets see what the impact is...

Browse Source
This commit is contained in:
Ross Wightman 2021-03-18 23:15:48 -07:00
parent 4de57ccf01
commit cbcb76d72c
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
timm/models/vision_transformer.py
View File

@ -476,7 +476,7 @@ class VisionTransformer(nn.Module):
def _init_weights_original(m: nn.Module, n: str = ''):
if isinstance(m, nn.Linear):
if isinstance(m, (nn.Conv2d, nn.Linear)):
trunc_normal_(m.weight, std=.02)
if isinstance(m, nn.Linear) and m.bias is not None:
nn.init.constant_(m.bias, 0)