Qualcomm AI Engine Direct - Mimi Enablement Stage 2

[winskuo-quic]

Summary

• Support Mimi Encoder
• Support Mimi Decoder
• Support OP: CDist
• Workaround for 0D tensor scenario since QNN does not support 0D tensor

Commands to Execute the model
10 sec as a batch (1 inference for 10 sec audio):
python examples/qualcomm/oss_scripts/moshi/mimi.py -b build-android -s $DEVICE -m SM8650 --chunks_per_batch 125

80s as a batch (125 inferences for 10 sec audio)
python examples/qualcomm/oss_scripts/moshi/mimi.py -b build-android -s $DEVICE -m SM8650

Test plan

• UT for CDist
• UT for 0D tensor

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/10098

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 New Failure

As of commit 08759d6 with merge base e29a4b5 ():

NEW FAILURE - The following job has failed:

• Check Labels / Check labels (gh)
  RuntimeError: Error checking labels: PR does not have required labels

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[winskuo-quic]

Hi @cccclai, @iseeyuan, @billmguo,
This PR is to enable mimi without streaming mode.
I have listed to execution commands in the summary section.
Please have a look.
Thanks

[facebook-github-bot]

@cccclai has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

[cccclai]

Does the logic in https://github.com/pytorch/executorch/blob/main/backends/transforms/rank_0_to_rank_1.py can be applied here?

[winskuo-quic]

Hi @cccclai,
Thanks for reviewing the PR.
I believe the pass you are suggesting is to change input from 0d to 1d tensor.
However, for our case, this 0D tensor happened during select op in the middle of graph. We just removed this select op since it does not affect the logic of the graph.
The exact point where 0D tensor occurs in mimi is under moshi/quantization/core_vq.py, where is tries to create a 1D tensor and retrieve index 0.

[cccclai]

I see, I guess for this specific logic, it's more efficient to remove the select op.

A more generic way to handle 0-d tensor is to convert it to 1-d tensor. Like following

class Rank0ToRank1Pass(ExportPass):
    """
    Replace Rank-0 Tensor to Rank-1 Tensor for all the inputs.
    """

    def __init__(self) -> None:
        super().__init__()

    def call(self, graph_module: torch.fx.GraphModule) -> PassResult:
        for node in graph_module.graph.nodes:
            // we can make the pass configurable to decide what kind of node we want to reshape to 1-d tensor
            if node.op == "call_function" and node.meta["val"].shape == ():
                node.meta["val"] = node.meta["val"].reshape(1, 1)
        graph_module.recompile()
        return PassResult(graph_module, True)

[cccclai]

Thank you for enabling mimi!