qk_scale of timm.models.vision_transformer.Block #1999
Unanswered
For-Isolate
asked this question in
Q&A
Replies: 1 comment 1 reply
-
|
@For-Isolate yes, it was removed quite a while back now, it was only used at the very beginning when I transitioned from some initial vit weights I trained myself (with the wrong scale) to porting the original vit weights from google's jax impl. Unfortunately it propagated to some third party implementations :( |
Beta Was this translation helpful? Give feedback.
1 reply
-
|
It can safely be removed anywhere that still uses it (as far as I know) |
Beta Was this translation helpful? Give feedback.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
Has the qk_scale of timm.models.vision_transformer.Block been removed? There is still this parameter in the mae code of facebook. If I want to run this code, shall I directly remove this parameter or do I need to make any changes
Beta Was this translation helpful? Give feedback.
All reactions