Skip to content

Commit c4b8897

Browse files
committed
attention -> attn in davit for model consistency
1 parent cb57a96 commit c4b8897

File tree

1 file changed

+3
-3
lines changed

1 file changed

+3
-3
lines changed

timm/models/davit.py

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -126,9 +126,9 @@ def forward(self, x: Tensor):
126126
q, k, v = qkv.unbind(0)
127127

128128
k = k * self.scale
129-
attention = k.transpose(-1, -2) @ v
130-
attention = attention.softmax(dim=-1)
131-
x = (attention @ q.transpose(-1, -2)).transpose(-1, -2)
129+
attn = k.transpose(-1, -2) @ v
130+
attn = attn.softmax(dim=-1)
131+
x = (attn @ q.transpose(-1, -2)).transpose(-1, -2)
132132
x = x.transpose(1, 2).reshape(B, N, C)
133133
x = self.proj(x)
134134
return x

0 commit comments

Comments
 (0)