Accept model type parameter in export_llama #5910

jackzhxng · 2024-10-05T03:37:43Z

Summary

PR chain:

pytorch-bot · 2024-10-05T03:37:47Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/5910

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 14 New Failures, 2 Cancelled Jobs

As of commit 96ba40b with merge base 8f9fb7e ():

NEW FAILURES - The following jobs have failed:

pull / test-eval_llama-mmlu-linux / linux-job (gh)
RuntimeError: Command docker exec -t cf8eca0f8b6d6c6ce2f6784b944654fd4345a0efbe03420b2bc435df3d64fe96 /exec failed with exit code 1
pull / test-eval_llama-wikitext-linux / linux-job (gh)
RuntimeError: Command docker exec -t 5bdf31cbb85eec0670d06a080c235bc91aa6c05ae8a89d4b966e6708b3a15cb4 /exec failed with exit code 1
pull / test-llama_runner_eager-linux / linux-job (gh)
RuntimeError: Command docker exec -t 144290d635a41076f80c79b7c0bb77f99c4a3817c8282bbb748667da977045aa /exec failed with exit code 1
pull / test-llama-runner-linux (bf16, buck2, custom) / linux-job (gh)
RuntimeError: Command docker exec -t 24d348dd51c0b727d3066ad97091a8eaa6ce091f29bed081d8f88f7061bd6860 /exec failed with exit code 1
pull / test-llama-runner-linux (bf16, buck2, portable) / linux-job (gh)
RuntimeError: Command docker exec -t 82128ef1b0e51c073918a468b6f2b6fa549291048e0c6cb831a4e79d5a945a43 /exec failed with exit code 1
pull / test-llama-runner-linux (bf16, cmake, custom) / linux-job (gh)
RuntimeError: Command docker exec -t 525c8f83b3b9b5690132eedd41a41bfd4090892c9e5adf65751fb9f77023d6c9 /exec failed with exit code 1
pull / test-llama-runner-linux (bf16, cmake, portable) / linux-job (gh)
RuntimeError: Command docker exec -t d9ede23b8690ae93adf54bc427c8bd9132fa9c1d7aa387b1f76a7d3d6146492d /exec failed with exit code 1
pull / test-llama-runner-linux (fp32, buck2, portable) / linux-job (gh)
RuntimeError: Command docker exec -t 25ed51e9e1d2bd17aa28f08f304f75e2a2ddd09b3dce3cc3524925d9ed4db6d0 /exec failed with exit code 1
pull / test-llama-runner-linux (fp32, buck2, xnnpack+custom) / linux-job (gh)
RuntimeError: Command docker exec -t b4a7cafa2b964e047fd24166155727b155dfeb80775d0cd29e755bcd48216ad4 /exec failed with exit code 1
pull / test-llama-runner-linux (fp32, buck2, xnnpack+custom+qe) / linux-job (gh)
RuntimeError: Command docker exec -t a3df6416f6901829cde0c550dc741f59a0607cb1025264e9f40f60b9cde80ae0 /exec failed with exit code 1
pull / test-llama-runner-linux (fp32, cmake, portable) / linux-job (gh)
RuntimeError: Command docker exec -t a4b425d8780ae9ec1fbc105eb90d04b2c6991db12f4713556a409be51822ee18 /exec failed with exit code 1
pull / test-llama-runner-linux (fp32, cmake, xnnpack+custom) / linux-job (gh)
RuntimeError: Command docker exec -t b62d86e38e2b370039fead6431a3c07d659a357dfc8702c65bf8b4d3f0307e92 /exec failed with exit code 1
pull / test-llama-runner-linux (fp32, cmake, xnnpack+custom+qe) / linux-job (gh)
RuntimeError: Command docker exec -t 17b90d72a6f3cc520a41e7cd4544d9f45800addfbfa2112530aabe65b128268a /exec failed with exit code 1
pull / test-llama-runner-qnn-linux (fp32, cmake, qnn) / linux-job (gh)
RuntimeError: Command docker exec -t 2737918da48940924781ba59ba14f884499cc7c1a48a17b530991c43a2908fba /exec failed with exit code 1

CANCELLED JOBS - The following jobs were cancelled. Please retry:

pull / test-llava-runner-linux / linux-job (gh)
##[error]The operation was canceled.
pull / unittest / macos / macos-job (gh)

This comment was automatically generated by Dr. CI and updates every 15 minutes.

Summary: For situations where the forward has non-position arguments, such as https://github.com/pytorch/torchtune/blob/3c450ef5f1fbe8237f899e942fd5222491a47ca7/torchtune/modules/transformer.py#L519 PR chain: - **YOU ARE HERE ~>** [Add kwarg example inputs to eager model base](#5765) - [Llama2 model cleanup](#5859) - [Accept model type parameter in export_llama](#5910) - [Export TorchTune llama3_2_vision in ET](#5911) - [Add et version of TorchTune MHA for swapping with custom op](#5912) Test Plan: Exported Stories110M model. ``` wget "https://huggingface.co/karpathy/tinyllamas/resolve/main/stories110M.pt" echo '{"dim": 768, "multiple_of": 32, "n_heads": 12, "n_layers": 12, "norm_eps": 1e-05, "vocab_size": 32000}' > params.json python -m examples.models.llama2.export_llama -c stories110M.pt -p params.json -X -kv ``` Differential Revision: D64027696 Pulled By: dvorjackz

Summary: For situations where the forward has non-position arguments, such as https://github.com/pytorch/torchtune/blob/3c450ef5f1fbe8237f899e942fd5222491a47ca7/torchtune/modules/transformer.py#L519 PR chain: - **YOU ARE HERE ~>** [Add kwarg example inputs to eager model base](#5765) - [Llama2 model cleanup](#5859) - [Accept model type parameter in export_llama](#5910) - [Export TorchTune llama3_2_vision in ET](#5911) - [Add et version of TorchTune MHA for swapping with custom op](#5912) Test Plan: Exported Stories110M model. ``` wget "https://huggingface.co/karpathy/tinyllamas/resolve/main/stories110M.pt" echo '{"dim": 768, "multiple_of": 32, "n_heads": 12, "n_layers": 12, "norm_eps": 1e-05, "vocab_size": 32000}' > params.json python -m examples.models.llama2.export_llama -c stories110M.pt -p params.json -X -kv ``` Reviewed By: tarun292 Differential Revision: D64027696 Pulled By: dvorjackz

Summary: For situations where the forward has non-position arguments, such as https://github.com/pytorch/torchtune/blob/3c450ef5f1fbe8237f899e942fd5222491a47ca7/torchtune/modules/transformer.py#L519 PR chain: - **YOU ARE HERE ~>** [Add kwarg example inputs to eager model base](#5765) - [Llama2 model cleanup](#5859) - [Accept model type parameter in export_llama](#5910) - [Export TorchTune llama3_2_vision in ET](#5911) - [Add et version of TorchTune MHA for swapping with custom op](#5912) Pull Request resolved: #5765 Test Plan: Exported Stories110M model. ``` wget "https://huggingface.co/karpathy/tinyllamas/resolve/main/stories110M.pt" echo '{"dim": 768, "multiple_of": 32, "n_heads": 12, "n_layers": 12, "norm_eps": 1e-05, "vocab_size": 32000}' > params.json python -m examples.models.llama2.export_llama -c stories110M.pt -p params.json -X -kv ``` Reviewed By: tarun292 Differential Revision: D64027696 Pulled By: dvorjackz fbshipit-source-id: 15ecfb458c6194159140d4c601e5443a2e524fdc

Summary: - Removes redundant steps in the Llama2 export - Factors out checkpointing to be shared with future Llama models (namely 3.2 multimodal) - Comments and orders code more clearly PR chain: - [Add kwarg example inputs to eager model base](#5765) - **YOU ARE HERE ~>** [Llama2 model cleanup](#5859) - [Accept model type parameter in export_llama](#5910) - [Export TorchTune llama3_2_vision in ET](#5911) - [Add et version of TorchTune MHA for swapping with custom op](#5912) Test Plan: Ensure export + eval is similar before and after for Stories 110M: ``` python -m examples.models.llama2.eval_llama -c <checkpoint.pth> -p <params.json> -t <tokenizer.model/bin> -d fp32 --max_seq_len 2048 --limit 1000 ``` Before: ``` wikitext: {'word_perplexity,none': 14464.645927166595, 'word_perplexity_stderr,none': 'N/A', 'byte_perplexity,none': 5.99788806086652, 'byte_perplexity_stderr,none': 'N/A', 'bits_per_byte,none': 2.5844545973083983, 'bits_per_byte_stderr,none': 'N/A', 'alias': 'wikitext'} ``` After: ``` wikitext: {'word_perplexity,none': 14464.299192404438, 'word_perplexity_stderr,none': 'N/A', 'byte_perplexity,none': 5.997861173678705, 'byte_perplexity_stderr,none': 'N/A', 'bits_per_byte,none': 2.584448130015399, 'bits_per_byte_stderr,none': 'N/A', 'alias': 'wikitext'} ``` Differential Revision: D64145852 Pulled By: dvorjackz

Summary: - Removes redundant steps in the Llama2 export - Factors out checkpointing to be shared with future Llama models (namely 3.2 multimodal) - Comments and orders code more clearly PR chain: - [Add kwarg example inputs to eager model base](#5765) - **YOU ARE HERE ~>** [Llama2 model cleanup](#5859) - [Accept model type parameter in export_llama](#5910) - [Export TorchTune llama3_2_vision in ET](#5911) - [Add et version of TorchTune MHA for swapping with custom op](#5912) Test Plan: Ensure export + eval is similar before and after for Stories 110M: ``` python -m examples.models.llama2.eval_llama -c <checkpoint.pth> -p <params.json> -t <tokenizer.model/bin> -d fp32 --max_seq_len 2048 --limit 1000 ``` Before: ``` wikitext: {'word_perplexity,none': 14464.645927166595, 'word_perplexity_stderr,none': 'N/A', 'byte_perplexity,none': 5.99788806086652, 'byte_perplexity_stderr,none': 'N/A', 'bits_per_byte,none': 2.5844545973083983, 'bits_per_byte_stderr,none': 'N/A', 'alias': 'wikitext'} ``` After: ``` wikitext: {'word_perplexity,none': 14464.299192404438, 'word_perplexity_stderr,none': 'N/A', 'byte_perplexity,none': 5.997861173678705, 'byte_perplexity_stderr,none': 'N/A', 'bits_per_byte,none': 2.584448130015399, 'bits_per_byte_stderr,none': 'N/A', 'alias': 'wikitext'} ``` Reviewed By: dbort Differential Revision: D64145852 Pulled By: dvorjackz

Summary: - Removes redundant steps in the Llama2 export - Factors out checkpointing to be shared with future Llama models (namely 3.2 multimodal) - Comments and orders code more clearly PR chain: - [Add kwarg example inputs to eager model base](#5765) - **YOU ARE HERE ~>** [Llama2 model cleanup](#5859) - [Accept model type parameter in export_llama](#5910) - [Export TorchTune llama3_2_vision in ET](#5911) - [Add et version of TorchTune MHA for swapping with custom op](#5912) Pull Request resolved: #5859 Test Plan: Ensure export + eval is similar before and after for Stories 110M: ``` python -m examples.models.llama2.eval_llama -c <checkpoint.pth> -p <params.json> -t <tokenizer.model/bin> -d fp32 --max_seq_len 2048 --limit 1000 ``` Before: ``` wikitext: {'word_perplexity,none': 14464.645927166595, 'word_perplexity_stderr,none': 'N/A', 'byte_perplexity,none': 5.99788806086652, 'byte_perplexity_stderr,none': 'N/A', 'bits_per_byte,none': 2.5844545973083983, 'bits_per_byte_stderr,none': 'N/A', 'alias': 'wikitext'} ``` After: ``` wikitext: {'word_perplexity,none': 14464.299192404438, 'word_perplexity_stderr,none': 'N/A', 'byte_perplexity,none': 5.997861173678705, 'byte_perplexity_stderr,none': 'N/A', 'bits_per_byte,none': 2.584448130015399, 'bits_per_byte_stderr,none': 'N/A', 'alias': 'wikitext'} ``` Reviewed By: malfet, dbort Differential Revision: D64145852 Pulled By: dvorjackz fbshipit-source-id: daeee834955e154e7c8262ce776bd3039991027d

facebook-github-bot · 2024-10-15T21:04:53Z

@dvorjackz has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

facebook-github-bot · 2024-10-15T22:00:02Z

@dvorjackz has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

Summary: Specify model to export in the CLI. Test Plan: Exported the stories 110M model. ``` python -m examples.models.llama.export_llama -c stories110M/stories110M.pt -p stories110M/params.json -X -kv ``` PR chain: - [Add kwarg example inputs to eager model base](#5765) - [Llama2 model cleanup](#5859) - **YOU ARE HERE ~>** [Accept model type parameter in export_llama](#5910) - [Export TorchTune llama3_2_vision in ET](#5911) - [Runner changes for TorchTune Llama3.2 vision text decoder](#6610) - [Add et version of TorchTune MHA for swapping with custom op](#5912) Differential Revision: D65612837 Pulled By: dvorjackz

Summary: Specify model to export in the CLI. Test Plan: Exported the stories 110M model. ``` python -m examples.models.llama.export_llama -c stories110M/stories110M.pt -p stories110M/params.json -X -kv ``` PR chain: - [Add kwarg example inputs to eager model base](#5765) - [Llama2 model cleanup](#5859) - **YOU ARE HERE ~>** [Accept model type parameter in export_llama](#5910) - [Export TorchTune llama3_2_vision in ET](#5911) - [Runner changes for TorchTune Llama3.2 vision text decoder](#6610) - [Add et version of TorchTune MHA for swapping with custom op](#5912) Reviewed By: helunwencser Differential Revision: D65612837 Pulled By: dvorjackz

facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Oct 5, 2024

jackzhxng force-pushed the jz/cleanup branch from b78c082 to b48f917 Compare October 7, 2024 20:55

jackzhxng force-pushed the jz/tt-llama branch from 60a21c1 to 948a18f Compare October 7, 2024 20:55

This was referenced Oct 7, 2024

Add kwarg example inputs to eager model base #5765

Closed

Add et version of TorchTune MHA for swapping with custom op #5912

Closed

Export TorchTune llama3_2_vision in ET #5911

Merged

Llama2 model cleanup #5859

Closed

jackzhxng force-pushed the jz/cleanup branch 2 times, most recently from 64c4fe1 to b116097 Compare October 8, 2024 06:42

jackzhxng force-pushed the jz/tt-llama branch 2 times, most recently from 0c46b2e to aa30bca Compare October 8, 2024 07:21

jackzhxng force-pushed the jz/cleanup branch from eebaef4 to af5dfd6 Compare October 8, 2024 17:20

jackzhxng force-pushed the jz/tt-llama branch from aa30bca to 1923d56 Compare October 8, 2024 17:21

jackzhxng force-pushed the jz/cleanup branch from af5dfd6 to d4b9d39 Compare October 8, 2024 20:09

jackzhxng force-pushed the jz/tt-llama branch from 1923d56 to 76e7cc7 Compare October 8, 2024 20:10

jackzhxng force-pushed the jz/cleanup branch from d4b9d39 to 5161153 Compare October 9, 2024 23:09

facebook-github-bot force-pushed the jz/cleanup branch from 5161153 to 27a0116 Compare October 11, 2024 01:58

facebook-github-bot force-pushed the jz/cleanup branch 2 times, most recently from ad17e72 to fb1312f Compare October 15, 2024 00:52

jackzhxng added 6 commits October 15, 2024 13:58

Add kwarg example inputs to eager model base

0b5a9a7

Create create new method for example kwarg inputs instead

a9647d2

Add kwarg example inputs to eager model base

fa3b1d2

Lint

e8715ba

Accept model type parameter in export_llama

a6f96a2

Remove future implementation

328c72c

jackzhxng force-pushed the jz/tt-llama branch from 76e7cc7 to 328c72c Compare October 15, 2024 21:03

jackzhxng changed the base branch from jz/cleanup to main October 15, 2024 21:04

Lint

ec80bba

jackzhxng added 4 commits October 25, 2024 15:07

Merge branch 'main' into jz/tt-llama

6dd47e7

Update READMEs

1825972

Change model default arg

196499a

Update eager runner and eval llama

96ba40b

jackzhxng mentioned this pull request Oct 25, 2024

Accept model type parameter in export_llama #6507

Merged

jackzhxng closed this Oct 25, 2024

jackzhxng mentioned this pull request Nov 1, 2024

Runner changes for TorchTune Llama3.2 vision text decoder #6610

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Accept model type parameter in export_llama #5910

Accept model type parameter in export_llama #5910

Uh oh!

jackzhxng commented Oct 5, 2024 •

edited

Loading

Uh oh!

pytorch-bot bot commented Oct 5, 2024 •

edited

Loading

Uh oh!

facebook-github-bot commented Oct 15, 2024

Uh oh!

facebook-github-bot commented Oct 15, 2024

Uh oh!

Uh oh!

Accept model type parameter in export_llama #5910

Accept model type parameter in export_llama #5910

Uh oh!

Conversation

jackzhxng commented Oct 5, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Uh oh!

pytorch-bot bot commented Oct 5, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/5910

❌ 14 New Failures, 2 Cancelled Jobs

Uh oh!

facebook-github-bot commented Oct 15, 2024

Uh oh!

facebook-github-bot commented Oct 15, 2024

Uh oh!

Uh oh!

jackzhxng commented Oct 5, 2024 •

edited

Loading

pytorch-bot bot commented Oct 5, 2024 •

edited

Loading