-
Notifications
You must be signed in to change notification settings - Fork 607
[ExecuTorch] Allow setting dtype to bf16 in export_llama #4985
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
bfloat16.h was a stub. I've filled it out by porting the c10 implementation, added it to ET_SWITCH and ET_FORALL macros, and hooked it up to promoteTypes. I extended the half_to_float argument to promoteTypes to also coerce bfloat16 to float because I figured anybody who wants to ignore half probably also wants to ignore bf16. Differential Revision: [D61981361](https://our.internmc.facebook.com/intern/diff/D61981361/) [ghstack-poisoned]
Adding bfloat16 support to important ops for LLMs to start. Differential Revision: [D61981356](https://our.internmc.facebook.com/intern/diff/D61981356/) [ghstack-poisoned]
Adding bfloat16 support to important ops for LLMs to start. Differential Revision: [D61981355](https://our.internmc.facebook.com/intern/diff/D61981355/) [ghstack-poisoned]
Adding bfloat16 support to important ops for LLMs to start. Differential Revision: [D61981353](https://our.internmc.facebook.com/intern/diff/D61981353/) [ghstack-poisoned]
Adding bfloat16 support to important ops for LLMs to start. Differential Revision: [D61981357](https://our.internmc.facebook.com/intern/diff/D61981357/) [ghstack-poisoned]
Adding bfloat16 support to important ops for LLMs to start. Differential Revision: [D61981364](https://our.internmc.facebook.com/intern/diff/D61981364/) [ghstack-poisoned]
Adding bfloat16 support to important ops for LLMs to start. Differential Revision: [D61981360](https://our.internmc.facebook.com/intern/diff/D61981360/) [ghstack-poisoned]
Adding bfloat16 support to important ops for LLMs to start. Differential Revision: [D61981359](https://our.internmc.facebook.com/intern/diff/D61981359/) [ghstack-poisoned]
Adding bfloat16 support to important ops for LLMs to start. Differential Revision: [D61981362](https://our.internmc.facebook.com/intern/diff/D61981362/) [ghstack-poisoned]
The LLM runner assumed that the data type could only be float or half. Suport bfloat16 and neaten up the code while we're at it. Differential Revision: [D61981354](https://our.internmc.facebook.com/intern/diff/D61981354/) **NOTE FOR REVIEWERS**: This PR has internal Meta-specific changes or comments, please review them on [Phabricator](https://our.internmc.facebook.com/intern/diff/D61981354/)! [ghstack-poisoned]
Support creating bf16 PTEs. Differential Revision: [D61981363](https://our.internmc.facebook.com/intern/diff/D61981363/) [ghstack-poisoned]
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/4985
Note: Links to docs will display an error until the docs builds have been completed. ❌ 2 New Failures, 2 Cancelled JobsAs of commit b211c00 with merge base 3c58237 ( NEW FAILURES - The following jobs have failed:
CANCELLED JOBS - The following jobs were cancelled. Please retry:
This comment was automatically generated by Dr. CI and updates every 15 minutes. |
This pull request was exported from Phabricator. Differential Revision: D61981363 |
Support creating bf16 PTEs. Differential Revision: [D61981363](https://our.internmc.facebook.com/intern/diff/D61981363/) ghstack-source-id: 240351243 Pull Request resolved: #4985
…n export_llama" Support creating bf16 PTEs. Differential Revision: [D61981363](https://our.internmc.facebook.com/intern/diff/D61981363/) [ghstack-poisoned]
Support creating bf16 PTEs. Differential Revision: [D61981363](https://our.internmc.facebook.com/intern/diff/D61981363/) [ghstack-poisoned]
This pull request was exported from Phabricator. Differential Revision: D61981363 |
Pull Request resolved: #4985 Support creating bf16 PTEs. ghstack-source-id: 240511594 @exported-using-ghexport Differential Revision: [D61981363](https://our.internmc.facebook.com/intern/diff/D61981363/)
…n export_llama" Support creating bf16 PTEs. Differential Revision: [D61981363](https://our.internmc.facebook.com/intern/diff/D61981363/) [ghstack-poisoned]
Support creating bf16 PTEs. Differential Revision: [D61981363](https://our.internmc.facebook.com/intern/diff/D61981363/) [ghstack-poisoned]
This pull request was exported from Phabricator. Differential Revision: D61981363 |
Pull Request resolved: #4985 Support creating bf16 PTEs. ghstack-source-id: 240577203 @exported-using-ghexport Differential Revision: [D61981363](https://our.internmc.facebook.com/intern/diff/D61981363/)
…n export_llama" Support creating bf16 PTEs. Differential Revision: [D61981363](https://our.internmc.facebook.com/intern/diff/D61981363/) [ghstack-poisoned]
Support creating bf16 PTEs. Differential Revision: [D61981363](https://our.internmc.facebook.com/intern/diff/D61981363/) [ghstack-poisoned]
This pull request was exported from Phabricator. Differential Revision: D61981363 |
Pull Request resolved: #4985 Support creating bf16 PTEs. ghstack-source-id: 240808278 @exported-using-ghexport Differential Revision: [D61981363](https://our.internmc.facebook.com/intern/diff/D61981363/)
…n export_llama" Support creating bf16 PTEs. Differential Revision: [D61981363](https://our.internmc.facebook.com/intern/diff/D61981363/) [ghstack-poisoned]
Support creating bf16 PTEs. Differential Revision: [D61981363](https://our.internmc.facebook.com/intern/diff/D61981363/) [ghstack-poisoned]
This pull request was exported from Phabricator. Differential Revision: D61981363 |
Pull Request resolved: #4985 Support creating bf16 PTEs. ghstack-source-id: 240866452 @exported-using-ghexport Differential Revision: [D61981363](https://our.internmc.facebook.com/intern/diff/D61981363/)
…n export_llama" Support creating bf16 PTEs. Differential Revision: [D61981363](https://our.internmc.facebook.com/intern/diff/D61981363/) [ghstack-poisoned]
Support creating bf16 PTEs. Differential Revision: [D61981363](https://our.internmc.facebook.com/intern/diff/D61981363/) [ghstack-poisoned]
This pull request was exported from Phabricator. Differential Revision: D61981363 |
Pull Request resolved: #4985 Support creating bf16 PTEs. ghstack-source-id: 241050127 @exported-using-ghexport Differential Revision: [D61981363](https://our.internmc.facebook.com/intern/diff/D61981363/)
…n export_llama" Support creating bf16 PTEs. Differential Revision: [D61981363](https://our.internmc.facebook.com/intern/diff/D61981363/) [ghstack-poisoned]
Support creating bf16 PTEs. Differential Revision: [D61981363](https://our.internmc.facebook.com/intern/diff/D61981363/) [ghstack-poisoned]
This pull request was exported from Phabricator. Differential Revision: D61981363 |
Pull Request resolved: pytorch/executorch#4985 Support creating bf16 PTEs. ghstack-source-id: 241282457 @exported-using-ghexport Differential Revision: [D61981363](https://our.internmc.facebook.com/intern/diff/D61981363/)
Stack from ghstack (oldest at bottom):
Support creating bf16 PTEs.
Differential Revision: D61981363