Update README.md #479
Merged
Update README.md #479
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
readme update
facebook-github-bot
added
the
CLA Signed
|
README only update. Please ignore pending tests |
kirklandsign
approved these changes
Apr 25, 2024
larryliu0820
added a commit
that referenced
this pull request
Jul 17, 2024
* Update quantize.py to use torchao Quantizers
Summary:
Remove duplicate code for Int4WeightOnlyQuantizer and
Int8DynActInt4WeightQuantizer and use torchao API.
Test Plan:
```
python torchchat.py generate llama2 --quantize '{"linear:int4": {"groupsize": 256}, "precision": {"dtype":"float16"}, "executor":{"accelerator":"cpu"}}' --prompt "Once upon a time," --max-new-tokens 256
python torchchat.py generate llama2 --quantize '{"linear:a8w4dq": {"groupsize": 256}, "precision": {"dtype":"float16"}, "executor":{"accelerator":"cpu"}}' --prompt "Once upon a time," --max-new-tokens 256
```
Reviewers:
Subscribers:
Tasks:
Tags:
* Fix import
Summary:
Test Plan:
Reviewers:
Subscribers:
Tasks:
Tags:
* Install torchao from gh
* Explain import
Summary:
Test Plan:
Reviewers:
Subscribers:
Tasks:
Tags:
* Fix dependencies
* Test ao PR #479
* Update torchao hash
* Update torchao pin
* Fix scheduler bf16/fp16 mix error
* Incorporate torchao changes
Summary:
Test Plan:
Reviewers:
Subscribers:
Tasks:
Tags:
* update hash
* Fix GPU CI job
Summary:
Test Plan:
Reviewers:
Subscribers:
Tasks:
Tags:
* More fix
* Fix executorch CI job
* Use quant api for int4 weight only quantization
Summary:
Test Plan:
Reviewers:
Subscribers:
Tasks:
Tags:
* Fix
Summary:
Test Plan:
Reviewers:
Subscribers:
Tasks:
Tags:
* Fix again
Summary:
Test Plan:
Reviewers:
Subscribers:
Tasks:
Tags:
* Fix 3
* Fix 4
* Try something
* debug
* Only migrate 8a4w
---------
Co-authored-by: Jack Zhang <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
readme update