Move test spec file #5218
Move test spec file #5218
Conversation
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/5218
Note: Links to docs will display an error until the docs builds have been completed. ✅ No FailuresAs of commit 7d084de with merge base de30572 ( This comment was automatically generated by Dr. CI and updates every 15 minutes. |
facebook-github-bot
added
the
CLA Signed
|
Later, could use @huydhn 's help with extracting generic model, and run it with org.pytorch.minibench/org.pytorch.minibench.BenchmarkActivity |
kirklandsign
force-pushed
the
test-spec-update
branch
from
7101a78 to
2901875
Compare
kirklandsign
force-pushed
the
test-spec-update
branch
from
2901875 to
0761dda
Compare
|
@kirklandsign has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
.github/workflows/android-perf.yml
Outdated
| @@ -291,7 +291,7 @@ jobs: | |||
| # Unlike models there are limited numbers of build flavor for apps, and the model controls whether it should build with bpe/tiktoken tokenizer. | |||
| # It's okay to build all possible apps with all possible flavors in job "build-llm-demo". However, in this job, once a model is given, there is only | |||
| # one app+flavor that could load and run the model. | |||
| android-app-archive: https://gha-artifacts.s3.amazonaws.com/${{ github.repository }}/${{ github.run_id }}/artifact/llm_demo/app-debug.apk | |||
| android-app-archive: https://gha-artifacts.s3.amazonaws.com/${{ github.repository }}/${{ github.run_id }}/artifact/minibench/app-debug.apk | |||
| android-test-archive: https://gha-artifacts.s3.amazonaws.com/${{ github.repository }}/${{ github.run_id }}/artifact/llm_demo/app-debug-androidTest.apk | |||
There was a problem hiding this comment.
There is another one here :)
There was a problem hiding this comment.
I didn't prepare for the instrumentation test for this app. So it's really a no-op if we use that. Do you think we should just no-op here?
.github/workflows/android-perf.yml
Outdated
| @@ -291,7 +291,7 @@ jobs: | |||
| # Unlike models there are limited numbers of build flavor for apps, and the model controls whether it should build with bpe/tiktoken tokenizer. | |||
| # It's okay to build all possible apps with all possible flavors in job "build-llm-demo". However, in this job, once a model is given, there is only | |||
| # one app+flavor that could load and run the model. | |||
| android-app-archive: https://gha-artifacts.s3.amazonaws.com/${{ github.repository }}/${{ github.run_id }}/artifact/llm_demo/app-debug.apk | |||
| android-app-archive: https://gha-artifacts.s3.amazonaws.com/${{ github.repository }}/${{ github.run_id }}/artifact/minibench/app-debug.apk | |||
| android-test-archive: https://gha-artifacts.s3.amazonaws.com/${{ github.repository }}/${{ github.run_id }}/artifact/llm_demo/app-debug-androidTest.apk | |||
There was a problem hiding this comment.
I didn't prepare for the instrumentation test for this app. So it's really a no-op if we use that. Do you think we should just no-op here?
There was a problem hiding this comment.
Let's put a note/comment to avoid confusion in the future
|
@kirklandsign has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
|
@kirklandsign has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
|
@kirklandsign has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
|
@kirklandsign has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
|
@kirklandsign merged this pull request in c5c121b. |
Use the new minibench for both LLM and generic model benchmarking.