mirrors
/
deit
mirror of https://github.com/facebookresearch/deit
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Update README_3things.md

pull/171/head
rv 2022-06-20 19:28:45 -05:00 committed by GitHub
parent b9a8c27970
commit a6865c1a3f
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
README_3things.md
View File

@ -33,7 +33,7 @@ We propose to finetune only the attentions (flag ```--attn-only```) to adapt the
## MLP patch projection
We propose to replace the linear patch projection by an MLP patch projection (see class [hMLP_stem](models_v2.py))
We propose to replace the linear patch projection by an MLP patch projection (see class [hMLP_stem](models_v2.py)). A key advantage is that this pre-processing stem is compatible and improves mask-based self-supervised training like BeiT, MAE, iBoT and SplitMask.
<img src=".github/hmlp.png" height="190">