Skip to content

Commit b058afb

Browse files
authored
Arm backend: Add test for DeiT Tiny for TOSA BI (#10846)
Add test case for the DeiT Tiny model for the TOSA BI profile. At this time the output of the model differs from the reference implementation by a mean absolute error of around 2.5, which is too high. An internal ticket has been raised to resolve this issue. Signed-off-by: Martin Lindström <[email protected]>
1 parent b427188 commit b058afb

File tree

1 file changed

+17
-1
lines changed

1 file changed

+17
-1
lines changed

backends/arm/test/models/test_deit_tiny_arm.py

Lines changed: 17 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,10 @@
1111

1212
import torch
1313

14-
from executorch.backends.arm.test.tester.test_pipeline import TosaPipelineMI
14+
from executorch.backends.arm.test.tester.test_pipeline import (
15+
TosaPipelineBI,
16+
TosaPipelineMI,
17+
)
1518

1619
from timm.data import IMAGENET_INCEPTION_MEAN, IMAGENET_INCEPTION_STD
1720
from torchvision import transforms
@@ -42,3 +45,16 @@ def test_deit_tiny_tosa_MI():
4245
qtol=1,
4346
)
4447
pipeline.run()
48+
49+
50+
def test_deit_tiny_tosa_BI():
51+
pipeline = TosaPipelineBI[input_t](
52+
deit_tiny,
53+
model_inputs,
54+
aten_op=[],
55+
exir_op=[],
56+
use_to_edge_transform_and_lower=True,
57+
atol=3.0, # This needs to go down: MLETORCH-956
58+
qtol=1,
59+
)
60+
pipeline.run()

0 commit comments

Comments
 (0)