quant params from static inputs #573
Closed
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
✅ Deploy Preview for resplendent-gnome-14e531 canceled.
|
facebook-github-bot
added
the
CLA Signed
|
This pull request was exported from Phabricator. Differential Revision: D49850149 |
|
This pull request was exported from Phabricator. Differential Revision: D49850149 |
mcr229
force-pushed
the
export-D49850149
branch
2 times, most recently
from
7633904 to
729e172
Compare
|
This pull request was exported from Phabricator. Differential Revision: D49850149 |
mcr229
force-pushed
the
export-D49850149
branch
from
729e172 to
5d71d4a
Compare
|
This pull request was exported from Phabricator. Differential Revision: D49850149 |
mcr229
force-pushed
the
export-D49850149
branch
from
5d71d4a to
0254a69
Compare
|
This pull request was exported from Phabricator. Differential Revision: D49850149 |
1 similar comment
|
This pull request was exported from Phabricator. Differential Revision: D49850149 |
mcr229
force-pushed
the
export-D49850149
branch
from
0254a69 to
892e22e
Compare
Summary: Making sure we have the macOS build too. To find the hash, see https://hud.pytorch.org/hud/pytorch/pytorch/nightly for the target date. To test, check if the file like `torch-2.2.0.dev20231005-cp311-none-macosx_11_0_arm64.whl` exists at https://download.pytorch.org/whl/nightly/torch/ and run `./install_requirements.sh` on a Mac to see all the dependencies are successfully installed. Pull Request resolved: pytorch/executorch#644 Reviewed By: angelayi Differential Revision: D49973900 Pulled By: shoumikhin fbshipit-source-id: 0f1b170a77c126b0ce7a46dc9f7c7d92cbe45847
Summary: Pull Request resolved: pytorch/executorch#573 Since we allow tensor constants to be quantized inputs, we need to adjust the from_inputs api to search if this input is static or not. If it is static, then we take the first q node in get_attr --> q --> dq. If it is not static, then we just take the dq node to create the QuantParams object. In the past, we can take in static quant inputs only on weights and biases. Reviewed By: digantdesai Differential Revision: D49850149 fbshipit-source-id: 5bf3c4de63a4454fcd7d7bea2f7d113f4f33d937
|
This pull request was exported from Phabricator. Differential Revision: D49850149 |
mcr229
force-pushed
the
export-D49850149
branch
from
892e22e to
090aa75
Compare
|
This pull request has been merged in 6230f8f. |
Gasoonjia
pushed a commit
that referenced
this pull request
Jul 30, 2024
* code beautification * code beautification, move functions together * rewrite model rewriter * rewrite quantizers * weights is none check * typo * not weight -> weight is not None * fix dimensions for parallel prefill * test * typo * bfloat16 on ARM with MacOS 14 * precision for a8w4 * sdpa_kv * fixes * inline qlq definition * trial and error * qdq not working * ci * not so fast with bf16=fast * typo, and handle fast across maxcos version... * typo * type cast
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary:
Since we allow tensor constants to be quantized inputs, we need to adjust the from_inputs api to search if this input is static or not. If it is static, then we take the first q node in get_attr --> q --> dq. If it is not static, then we just take the dq node to create the QuantParams object.
In the past, we can take in static quant inputs only on weights and biases.
Differential Revision: D49850149