Android App with MediaTek Mode

[cmodi-meta]

Purpose: Modify the existing demo Android App to support MediaTek e2e flow.

Pre-requisite: This diff should sit on top of #6208 changes. We plan to rebase to this once #6208 changes are complete in total.

Planned Changes:

• Backend selector
• UI Layout
• Enable to use MTK runner

How to Run:

1. Compile .aar using MTK Android Llama Runner #6208 . This places .aar in executorch/examples/demo-apps/android/LlamaDemo. Make sure to modify the tokenizer, embedding file and model file path in llama_runner_values.h before doing so.
2. Build+launch app using Android Studio for example. (or alternative)
3. In the Setttings widget in the app, select MTK backend and then click Load Model (ensure Llama 3 by default)
4. After model loads, type "once upon a time". Press enter.
5. Generated output will appear.

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/6304

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit 85f3304 with merge base 47bca20 ():
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[cmodi-meta]

I've confirmed that the output works (after a bit of hacky change to the if() condition in the checkForUpdateAndReloadModel() method). This was required because I initially thought that modelPath and tokenizerPath wouldn't need to be selected but this is not the case. Having paths selected is part of the trigger to load the model.

In order to prevent a one-off, I'll need to make changes to how modelPath and tokenizerPath is selected. Maybe select 1 .pte modelPath or have a json file that contains the paths.

[cmodi-meta]

I've confirmed that the output works (after a bit of hacky change to the if() condition in the checkForUpdateAndReloadModel() method). This was required because I initially thought that modelPath and tokenizerPath wouldn't need to be selected but this is not the case. Having paths selected is part of the trigger to load the model.

In order to prevent a one-off, I'll need to make changes to how modelPath and tokenizerPath is selected. Maybe select 1 .pte modelPath or have a json file that contains the paths.

I've added a temporary placeholder modelPath and tokenizerPath for the MTK backend to support model loading.

[cmodi-meta]

Open for review but TODO (once @kirklandsign makes jni changes) is to modify the LlamaModule initialization based on JNI change for how to select MediaTek backend.

[facebook-github-bot]

@cmodi-meta has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

[cccclai]

It looks to me that the mtk_llama_runner is added by you, and the rest of the changes are mostly related to android, and I'll lean on @kirklandsign to review them. I'll approve and @neuropilot-captain please let us know if you have any concern.

[facebook-github-bot]

@cmodi-meta has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

[cccclai]

@kirklandsign mentioned the code was mostly borrowed from the runner provided by MediaTek, meaning it's pretty much a fork. The risk is that once the llama runner from mediatek change, this new runner will fail. Let's describe it somewhere in the doc or code

[facebook-github-bot]

@cmodi-meta has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

[facebook-github-bot]

@cmodi-meta has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.