diff --git a/README_3things.md b/README_3things.md index df36a5c..16d41bc 100644 --- a/README_3things.md +++ b/README_3things.md @@ -33,7 +33,7 @@ We propose to finetune only the attentions (flag ```--attn-only```) to adapt the ## MLP patch projection -We propose to replace the linear patch projection by an MLP patch projection (see class [hMLP_stem](models_v2.py)) +We propose to replace the linear patch projection by an MLP patch projection (see class [hMLP_stem](models_v2.py)). A key advantage is that this pre-processing stem is compatible and improves mask-based self-supervised training like BeiT, MAE, iBoT and SplitMask.