Skip to content

Commit a71935a

Browse files
Lunwen Hefacebook-github-bot
authored andcommitted
Add option to export phi-3-mini model via buck2 (#4189)
Summary: Pull Request resolved: #4189 as title ghstack-source-id: 233317596 exported-using-ghexport Reviewed By: larryliu0820 Differential Revision: D59500806 fbshipit-source-id: 850e5e71f1afbaea85993b744012c710247d3ab9
1 parent 4fa7cfc commit a71935a

File tree

1 file changed

+9
-3
lines changed

1 file changed

+9
-3
lines changed

examples/models/phi-3-mini/export_model.py renamed to examples/models/phi-3-mini/export_phi-3-mini.py

Lines changed: 9 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -19,14 +19,14 @@
1919
get_symmetric_quantization_config,
2020
XNNPACKQuantizer,
2121
)
22-
from torch.export import export
2322

2423
from transformers import Phi3ForCausalLM
2524

2625

2726
def main() -> None:
2827
torch.random.manual_seed(0)
2928

29+
# pyre-ignore: Undefined attribute [16]: Module `transformers` has no attribute `Phi3ForCausalLM`
3030
model = Phi3ForCausalLM.from_pretrained("microsoft/Phi-3-mini-4k-instruct")
3131

3232
example_inputs = (torch.randint(0, 100, (1, 100), dtype=torch.long),)
@@ -48,8 +48,14 @@ def main() -> None:
4848
model(*example_inputs)
4949
model = convert_pt2e(model, fold_quantize=False)
5050
DuplicateDynamicQuantChainPass()(model)
51-
model = export(
52-
model, example_inputs, dynamic_shapes=dynamic_shape, strict=False
51+
# TODO(lunwenh): update it to use export once
52+
# https://github.com/pytorch/pytorch/issues/128394 is resolved.
53+
model = torch.export._trace._export(
54+
model,
55+
example_inputs,
56+
dynamic_shapes=dynamic_shape,
57+
strict=False,
58+
pre_dispatch=False,
5359
)
5460

5561
edge_config = get_xnnpack_edge_compile_config()

0 commit comments

Comments
 (0)