add qnn test template job #7636
Conversation
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/7636
Note: Links to docs will display an error until the docs builds have been completed. ⏳ No Failures, 1 PendingAs of commit aa7c882 with merge base 0dbe214 ( This comment was automatically generated by Dr. CI and updates every 15 minutes. |
facebook-github-bot
added
the
CLA Signed
|
This pull request was exported from Phabricator. Differential Revision: D68112936 |
cccclai
requested review from
chunit-quic,
haowhsu-quic,
shewu-quic and
winskuo-quic
.github/workflows/pull.yml
Outdated
| @@ -395,6 +395,25 @@ jobs: | |||
| # Test llama2 | |||
| PYTHON_EXECUTABLE=python bash .ci/scripts/test_llama.sh -model stories110M -build_tool "${BUILD_TOOL}" -mode "${MODE}" -dtype "${DTYPE}" -pt2e_quantize "${PT2E_QUANTIZE}" | |||
|
|
|||
| test-qnn-models-linux: | |||
| name: test-mediatek-models-linux | |||
There was a problem hiding this comment.
Might be qnn I think.
|
Thank you for this, we 'll figure out which test cases are mandatory. I think we could just have them run on the emulator, or you have some connected hardware could be used? |
We can only run emulators on the CI for now. However, @guangy10 is working on the perf benchmark with real device and he has enabled to qnn model running there. I notice that there are some real models in the test_qnn_delegate.py and we can have those running on real device. They will run periodically, but not per pull request, though should be good enough to catch failure and track down the issue. |
Summary: Pull Request resolved: pytorch#7636 Differential Revision: D68112936
cccclai
force-pushed
the
export-D68112936
branch
from
ddd9388 to
2a3fd63
Compare
|
This pull request was exported from Phabricator. Differential Revision: D68112936 |
|
@cccclai has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
|
Hi |
|
Might not be this patch, I tried a "revert" PR and get the same issue in it so it might probably be something else causing it. |
|
Ops, that is a real error, let me submit a forward fix. This is not related to this PR though |
|
The fix PR #7731 |
|
@cccclai We already have the qnn models in trunk.yml: https://github.com/pytorch/executorch/blob/main/.github/workflows/trunk.yml#L302-L323, which will be triggered on PR if files under tracked paths got modified. Why do we need to add same to pull.yml? |
oh this pr is to cover the perf op test defined in |
* add qnn test template job (#7636) Summary: Pull Request resolved: #7636 Differential Revision: D68112936 * Increase timeout --------- Co-authored-by: Huy Do <[email protected]>
Differential Revision: D68112936