Defer resolution of the default value of arguments used by quantize #2738
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/2738
Note: Links to docs will display an error until the docs builds have been completed. ✅ No FailuresAs of commit 0bdf70f with merge base 45c2557 ( This comment was automatically generated by Dr. CI and updates every 15 minutes. |
facebook-github-bot
added
the
CLA Signed
|
This pull request was exported from Phabricator. Differential Revision: D55458866 |
jerryzh168
approved these changes
Mar 28, 2024
jerryzh168
reviewed
Mar 28, 2024
| "-G", | ||
| "--group_size", | ||
| type=int, | ||
| default=256, |
There was a problem hiding this comment.
since group_size doesn't make sense for other settings like fp, maybe default to None as well
There was a problem hiding this comment.
We should probably do the same for the GPTQ specific args then (calibration_*)
I'll change this PR to update all of them and the lower one to just plumbing and keeping None
Jack-Khuu
changed the title
|
This pull request was exported from Phabricator. Differential Revision: D55458866 |
Jack-Khuu
added a commit
to Jack-Khuu/executorch-1
that referenced
this pull request
Mar 28, 2024
…ytorch#2738) Summary: Pull Request resolved: pytorch#2738 Quantize() (specifically GPTQ) is the sole user of the many params, but default values are introduced early and in multiple places. This is bug prone and confusing. * For example, previously the default value of calibration tasks was [], which is not something `Int8DynActInt4WeightGPTQQuantizer` handles gracefully. This diff defers default value resolution to quantize() since that is the direct call that uses them. Differential Revision: D55458866
Jack-Khuu
force-pushed
the
export-D55458866
branch
from
4bf35d0 to
a2d0e22
Compare
|
This pull request was exported from Phabricator. Differential Revision: D55458866 |
Jack-Khuu
force-pushed
the
export-D55458866
branch
from
a2d0e22 to
d514ae3
Compare
|
This pull request was exported from Phabricator. Differential Revision: D55458866 |
Jack-Khuu
added a commit
to Jack-Khuu/executorch-1
that referenced
this pull request
Mar 28, 2024
…ytorch#2738) Summary: Pull Request resolved: pytorch#2738 Quantize() (specifically GPTQ) is the sole user of the many params, but default values are introduced early and in multiple places. This is bug prone and confusing. * For example, previously the default value of calibration tasks was [], which is not something `Int8DynActInt4WeightGPTQQuantizer` handles gracefully. This diff defers default value resolution to quantize() since that is the direct call that uses them. Differential Revision: D55458866
Jack-Khuu
force-pushed
the
export-D55458866
branch
from
d514ae3 to
4d096db
Compare
Jack-Khuu
added a commit
to Jack-Khuu/executorch-1
that referenced
this pull request
Mar 28, 2024
…ytorch#2738) Summary: Quantize() (specifically GPTQ) is the sole user of the many params, but default values are introduced early and in multiple places. This is bug prone and confusing. * For example, previously the default value of calibration tasks was [], which is not something `Int8DynActInt4WeightGPTQQuantizer` handles gracefully. This diff defers default value resolution to quantize() since that is the direct call that uses them. Differential Revision: D55458866
Jack-Khuu
force-pushed
the
export-D55458866
branch
from
4d096db to
81e9225
Compare
|
This pull request was exported from Phabricator. Differential Revision: D55458866 |
Jack-Khuu
added a commit
to Jack-Khuu/executorch-1
that referenced
this pull request
Mar 28, 2024
…ytorch#2738) Summary: Quantize() (specifically GPTQ) is the sole user of the many params, but default values are introduced early and in multiple places. This is bug prone and confusing. * For example, previously the default value of calibration tasks was [], which is not something `Int8DynActInt4WeightGPTQQuantizer` handles gracefully. This diff defers default value resolution to quantize() since that is the direct call that uses them. Differential Revision: D55458866
Jack-Khuu
force-pushed
the
export-D55458866
branch
from
81e9225 to
fde8e4a
Compare
|
This pull request was exported from Phabricator. Differential Revision: D55458866 |
Jack-Khuu
added a commit
to Jack-Khuu/executorch-1
that referenced
this pull request
Mar 28, 2024
…ytorch#2738) Summary: Quantize() (specifically GPTQ) is the sole user of the many params, but default values are introduced early and in multiple places. This is bug prone and confusing. * For example, previously the default value of calibration tasks was [], which is not something `Int8DynActInt4WeightGPTQQuantizer` handles gracefully. This diff defers default value resolution to quantize() since that is the direct call that uses them. Reviewed By: jerryzh168 Differential Revision: D55458866
Jack-Khuu
force-pushed
the
export-D55458866
branch
from
fde8e4a to
2a3e6ab
Compare
|
This pull request was exported from Phabricator. Differential Revision: D55458866 |
Summary: Previously group size wasn't being passed properly to 4b quant. This just passes it through Reviewed By: mergennachin Differential Revision: D55458352
…ytorch#2738) Summary: Quantize() (specifically GPTQ) is the sole user of the many params, but default values are introduced early and in multiple places. This is bug prone and confusing. * For example, previously the default value of calibration tasks was [], which is not something `Int8DynActInt4WeightGPTQQuantizer` handles gracefully. This diff defers default value resolution to quantize() since that is the direct call that uses them. Reviewed By: jerryzh168 Differential Revision: D55458866
Jack-Khuu
force-pushed
the
export-D55458866
branch
from
2a3e6ab to
0bdf70f
Compare
|
This pull request was exported from Phabricator. Differential Revision: D55458866 |
|
This pull request has been merged in 57e3449. |
dbort
pushed a commit
to dbort/executorch
that referenced
this pull request
Mar 30, 2024
…ytorch#2738) Summary: Pull Request resolved: pytorch#2738 Quantize() (specifically GPTQ) is the sole user of the many params, but default values are introduced early and in multiple places. This is bug prone and confusing. * For example, previously the default value of calibration tasks was [], which is not something `Int8DynActInt4WeightGPTQQuantizer` handles gracefully. This diff defers default value resolution to quantize() since that is the direct call that uses them. Reviewed By: jerryzh168 Differential Revision: D55458866 fbshipit-source-id: 1afc5a62d214409f31c43c07be66bfd69712bb74
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary:
Quantize() (specifically GPTQ) is the sole user of the many params, but default values are introduced early and in multiple places. This is bug prone and confusing.
This diff defers default value resolution to quantize() since that is the direct call that uses them.
Differential Revision: D55458866