-
Notifications
You must be signed in to change notification settings - Fork 608
Add quantize option to the coreml script #5710
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/5710
Note: Links to docs will display an error until the docs builds have been completed. ✅ No FailuresAs of commit 7c5eec5 with merge base 8b5cf96 ( This comment was automatically generated by Dr. CI and updates every 15 minutes. |
This pull request was exported from Phabricator. Differential Revision: D63476611 |
@cymbalrush @YifanShenSZ we observed the latency number is quite when loading the quantized model. Can you share some guidance on how to have the optimum latency on the |
This pull request was exported from Phabricator. Differential Revision: D63476611 |
Summary: Pull Request resolved: pytorch#5710 Add the quantizer to the export script. The commandline is ``` python executorch.examples.apple.coreml.scripts.export -m resnet18 --quantize ``` Differential Revision: D63476611
80eccc3
to
78dfc9b
Compare
Summary: Pull Request resolved: pytorch#5710 Add the quantizer to the export script. The commandline is ``` python executorch.examples.apple.coreml.scripts.export -m resnet18 --quantize ``` Reviewed By: shoumikhin Differential Revision: D63476611
78dfc9b
to
7c5eec5
Compare
This pull request was exported from Phabricator. Differential Revision: D63476611 |
This pull request has been merged in fe0e676. |
buffer = exec_prog.buffer | ||
filename = f"{model_name}_coreml_{compute_unit}.pte" | ||
data_type = "quantize" if quantize else "fp" | ||
filename = f"{model_name}_coreml_{compute_unit}_{data_type}.pte" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this line is responsible for the coreml failure in trunk https://hud.pytorch.org/pytorch/executorch/commit/77e7ad10903ed3bcf16e41d04a5e595ecaa1458b. It changes the export model file name from mv3_coreml_all.pte
to mv3_coreml_all_fp.pte
. The name is used in many different tests https://github.com/search?q=repo%3Apytorch%2Fexecutorch+mv3_coreml_all&type=code.
There might be other test models setup in the same way, not only mv3 .
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Trying to revert the change fixes the failures #5906
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ah good finding. Sorry I didn't realize the name was hardcoded. Approve the revert change.
Summary:
Add the quantizer to the export script. The commandline is
Differential Revision: D63476611