[executorch] Implement operator<<() for EValue #479
Closed
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add a helper utility to print `EValue`s. Doesn't format multidimensional data like PyTorch does, but we can improve that in the future if we want to. Differential Revision: [D49574853](https://our.internmc.facebook.com/intern/diff/D49574853/) [ghstack-poisoned]
facebook-github-bot
added
the
CLA Signed
This was referenced Sep 25, 2023
Add a helper utility to print `EValue`s. Doesn't format multidimensional data like PyTorch does, but we can improve that in the future if we want to. Differential Revision: [D49574853](https://our.internmc.facebook.com/intern/diff/D49574853/) [ghstack-poisoned]
Add a helper utility to print `EValue`s. Doesn't format multidimensional data like PyTorch does, but we can improve that in the future if we want to. Differential Revision: [D49574853](https://our.internmc.facebook.com/intern/diff/D49574853/) [ghstack-poisoned]
|
This pull request has been merged in bd26dbf. |
Gasoonjia
pushed a commit
that referenced
this pull request
Jul 30, 2024
* Update quantize.py to use torchao Quantizers
Summary:
Remove duplicate code for Int4WeightOnlyQuantizer and
Int8DynActInt4WeightQuantizer and use torchao API.
Test Plan:
```
python torchchat.py generate llama2 --quantize '{"linear:int4": {"groupsize": 256}, "precision": {"dtype":"float16"}, "executor":{"accelerator":"cpu"}}' --prompt "Once upon a time," --max-new-tokens 256
python torchchat.py generate llama2 --quantize '{"linear:a8w4dq": {"groupsize": 256}, "precision": {"dtype":"float16"}, "executor":{"accelerator":"cpu"}}' --prompt "Once upon a time," --max-new-tokens 256
```
Reviewers:
Subscribers:
Tasks:
Tags:
* Fix import
Summary:
Test Plan:
Reviewers:
Subscribers:
Tasks:
Tags:
* Install torchao from gh
* Explain import
Summary:
Test Plan:
Reviewers:
Subscribers:
Tasks:
Tags:
* Fix dependencies
* Test ao PR #479
* Update torchao hash
* Update torchao pin
* Fix scheduler bf16/fp16 mix error
* Incorporate torchao changes
Summary:
Test Plan:
Reviewers:
Subscribers:
Tasks:
Tags:
* update hash
* Fix GPU CI job
Summary:
Test Plan:
Reviewers:
Subscribers:
Tasks:
Tags:
* More fix
* Fix executorch CI job
* Use quant api for int4 weight only quantization
Summary:
Test Plan:
Reviewers:
Subscribers:
Tasks:
Tags:
* Fix
Summary:
Test Plan:
Reviewers:
Subscribers:
Tasks:
Tags:
* Fix again
Summary:
Test Plan:
Reviewers:
Subscribers:
Tasks:
Tags:
* Fix 3
* Fix 4
* Try something
* debug
* Only migrate 8a4w
---------
Co-authored-by: Jack Zhang <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Stack from ghstack (oldest at bottom):
Add a helper utility to print
EValues. Doesn't format multidimensional data like PyTorch does, but we can improve that in the future if we want to.Differential Revision: D49574853