Update on "use --use_sdpa_with_kv_cache for 1B/3B bf16"

We should use this option during exporting 1B/3B models as bf16 because KVCache is always fp32. Otherwise, we see regressed performance for 1B/3B in bf16 format.

Differential Revision: [D63871048](https://our.internmc.facebook.com/intern/diff/D63871048/)

[ghstack-poisoned]

Author: helunwencser

backends/arm/test/ops/test_sigmoid.py

@@ -152,12 +152,32 @@ def test_sigmoid_tosa_MI(
     def test_sigmoid_tosa_BI(self, test_name: str, test_data: torch.Tensor):
         self._test_sigmoid_tosa_BI_pipeline(self.Sigmoid(), (test_data,))
 
+    def test_add_sigmoid_tosa_MI(self):
+        self._test_sigmoid_tosa_MI_pipeline(self.AddSigmoid(), (test_data_suite[0][1],))
+
+    @unittest.skip(
+        reason="Started to fails when PyTorch 2.5->2.6 https://github.com/pytorch/executorch/issues/5832"
+    )
     def test_add_sigmoid_tosa_BI(self):
         self._test_sigmoid_tosa_BI_pipeline(self.AddSigmoid(), (test_data_suite[0][1],))
 
+    def test_sigmoid_add_tosa_MI(self):
+        self._test_sigmoid_tosa_MI_pipeline(self.SigmoidAdd(), (test_data_suite[0][1],))
+
+    @unittest.skip(
+        reason="Started to fails when PyTorch 2.5->2.6 https://github.com/pytorch/executorch/issues/5832"
+    )
     def test_sigmoid_add_tosa_BI(self):
         self._test_sigmoid_tosa_BI_pipeline(self.SigmoidAdd(), (test_data_suite[0][1],))
 
+    def test_sigmoid_add_sigmoid_tosa_MI(self):
+        self._test_sigmoid_tosa_MI_pipeline(
+            self.SigmoidAddSigmoid(), (test_data_suite[4][1], test_data_suite[3][1])
+        )
+
+    @unittest.skip(
+        reason="Started to fails when PyTorch 2.5->2.6 https://github.com/pytorch/executorch/issues/5832"
+    )
     def test_sigmoid_add_sigmoid_tosa_BI(self):
         self._test_sigmoid_tosa_BI_pipeline(
             self.SigmoidAddSigmoid(), (test_data_suite[4][1], test_data_suite[3][1])

backends/mediatek/scripts/README.md

@@ -16,7 +16,7 @@ Before you begin, ensure you have the following prerequisites installed and conf
 
 ### 2. Android NDK
 
-- **Download Android NDK**: Acquire the Android NDK from the [Android developer site](https://developer.android.com/ndk/downloads).
+- **Download Android NDK**: Acquire the Android NDK version 26.3.11579264 from the [Android developer site](https://developer.android.com/ndk/downloads).
 - **Set NDK Path**: Ensure that the `$ANDROID_NDK` environment variable is set to the path where the NDK is located.
    ```bash
    export ANDROID_NDK=<path_to_android_ndk>

examples/demo-apps/android/LlamaDemo/README.md

@@ -34,6 +34,12 @@ First it’s important to note that currently ExecuTorch provides support across
 | QNN (Qualcomm AI Accelerators)  | [link](https://github.com/pytorch/executorch/blob/main/examples/demo-apps/android/LlamaDemo/docs/delegates/qualcomm_README.md) |
 | MediaTek (MediaTek AI Accelerators)  | [link](https://github.com/pytorch/executorch/blob/main/examples/demo-apps/android/LlamaDemo/docs/delegates/mediatek_README.md)  |
 
+**WARNING** NDK r27 will cause issues like:
+  ```
+  java.lang.UnsatisfiedLinkError: dlopen failed: cannot locate symbol "_ZTVNSt6__ndk114basic_ifstreamIcNS_11char_traitsIcEEEE" referenced by "/data/app/~~F5IwquaXUZPdLpSEYA-JGA==/com.example.executorchllamademo-FSyx80gEhsQCsxz7hvS2Ew==/lib/arm64/libexecutorch.so"...
+  ```
+  Please use NDK version 26.3.11579264.
+
 ## How to Use the App
 
 This section will provide the main steps to use the app, along with a code snippet of the ExecuTorch API.

examples/demo-apps/android/LlamaDemo/docs/delegates/qualcomm_README.md

@@ -6,13 +6,13 @@ More specifically, it covers:
 2. Building and linking libraries that are required to inference on-device for Android platform using Qualcomm AI accelerators.
 3. Building the Android demo app itself.
 
-Verified on Linux CentOS, QNN SDK [v2.26](https://softwarecenter.qualcomm.com/api/download/software/qualcomm_neural_processing_sdk/v2.26.0.240828.zip), python 3.10, Android SDK r27 and r26b.
+Verified on Linux CentOS, QNN SDK [v2.26](https://softwarecenter.qualcomm.com/api/download/software/qualcomm_neural_processing_sdk/v2.26.0.240828.zip), python 3.10, Android SDK r26c.
 
 Phone verified: OnePlus 12, Samsung 24+, Samsung 23
 
 ## Prerequisites
 * Download and unzip QNN SDK [v2.26](https://softwarecenter.qualcomm.com/api/download/software/qualcomm_neural_processing_sdk/v2.26.0.240828.zip)
-* Download and unzip Android SDK [r27](https://developer.android.com/ndk/downloads)
+* Download and unzip Android SDK [r26](https://developer.android.com/ndk/downloads)
 * Android phone with Snapdragon8 Gen3 (SM8650) or Gen2 (SM8550). Gen 1 and lower SoC might be supported but not fully validated.
 * Desired Llama model weights in .PTH format. You can download them on HuggingFace ([Example](https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct)).
 
@@ -40,7 +40,7 @@ Install dependencies
 ## Setup QNN
 ```
 # Set these variables correctly for your environment
-export ANDROID_NDK_ROOT=$HOME/android-ndk-r27 # Download android SDK and unzip to home directory
+export ANDROID_NDK_ROOT=$HOME/android-ndk-r26 # Download android SDK and unzip to home directory
 export QNN_SDK_ROOT=$HOME/Your-SDK-Root #Folder contains lib
 export EXECUTORCH_ROOT=$HOME/repos/executorch
 export LD_LIBRARY_PATH=$QNN_SDK_ROOT/lib/x86_64-linux-clang/:$LD_LIBRARY_PATH

examples/demo-apps/android/LlamaDemo/docs/delegates/xnnpack_README.md

@@ -16,7 +16,11 @@ Phone verified: OnePlus 12, OnePlus 9 Pro. Samsung S23 (Llama only), Samsung S24
 
 ## Prerequisites
 * Install [Java 17 JDK](https://www.oracle.com/java/technologies/javase/jdk17-archive-downloads.html).
-* Install the [Android SDK API Level 34](https://developer.android.com/about/versions/15/setup-sdk) and [Android NDK 26.3.11579264](https://developer.android.com/studio/projects/install-ndk).
+* Install the [Android SDK API Level 34](https://developer.android.com/about/versions/15/setup-sdk) and [Android NDK 26.3.11579264](https://developer.android.com/studio/projects/install-ndk). **WARNING** NDK r27 will cause issues like:
+  ```
+  java.lang.UnsatisfiedLinkError: dlopen failed: cannot locate symbol "_ZTVNSt6__ndk114basic_ifstreamIcNS_11char_traitsIcEEEE" referenced by "/data/app/~~F5IwquaXUZPdLpSEYA-JGA==/com.example.executorchllamademo-FSyx80gEhsQCsxz7hvS2Ew==/lib/arm64/libexecutorch.so"...
+  ```
+  Please downgrade to version 26.3.11579264.
 * If you have Android Studio set up, you can install them with
   * Android Studio Settings -> Language & Frameworks -> Android SDK -> SDK Platforms -> Check the row with API Level 34.
   * Android Studio Settings -> Language & Frameworks -> Android SDK -> SDK Tools -> Check NDK (Side by side) row.
@@ -67,7 +71,7 @@ We have supported BFloat16 as a data type on the XNNPACK backend for Llama 3.2 1
 * Export Llama model and generate .pte file as below:
 
 ```
-python -m examples.models.llama2.export_llama --checkpoint <checkpoint.pth> --params <params.json> -kv -X -d bf16 --metadata '{"get_bos_id":128000, "get_eos_ids":[128009, 128001]}' --output_name="llama3_2.pte"
+python -m examples.models.llama2.export_llama --checkpoint <checkpoint.pth> --params <params.json> -kv --use_sdpa_with_kv_cache -X -d bf16 --metadata '{"get_bos_id":128000, "get_eos_ids":[128009, 128001]}' --output_name="llama3_2.pte"
 ```
 
 * Rename tokenizer for Llama 3.2 with command: `mv tokenizer.model tokenizer.bin`. We are updating the demo app to support tokenizer in original format directly.
@@ -135,6 +139,7 @@ export ANDROID_NDK=<path_to_android_ndk>
 export ANDROID_ABI=arm64-v8a
 ```
 *Note: <path_to_android_ndk> is the root for the NDK, which is usually under ~/Library/Android/sdk/ndk/XX.Y.ZZZZZ for macOS, and contains NOTICE and README.md. We use <path_to_android_ndk>/build/cmake/android.toolchain.cmake for CMake to cross-compile.*
+
 3. Build the Android Java extension code:
 ```
 pushd extension/android

examples/demo-apps/apple_ios/LLaMA/docs/delegates/xnnpack_README.md

@@ -55,7 +55,7 @@ We have supported BFloat16 as a data type on the XNNPACK backend for Llama 3.2 1
 * Export Llama model and generate .pte file as below:
 
 ```
-python -m examples.models.llama2.export_llama --checkpoint <checkpoint.pth> --params <params.json> -kv -X -d bf16 --metadata '{"get_bos_id":128000, "get_eos_ids":[128009, 128001]}' --output_name="llama3_2.pte"
+python -m examples.models.llama2.export_llama --checkpoint <checkpoint.pth> --params <params.json> -kv --use_sdpa_with_kv_cache -X -d bf16 --metadata '{"get_bos_id":128000, "get_eos_ids":[128009, 128001]}' --output_name="llama3_2.pte"
 ```
 
 For more detail using Llama 3.2 lightweight models including prompt template, please go to our official [website](https://www.llama.com/docs/model-cards-and-prompt-formats/llama3_2#-llama-3.2-lightweight-models-(1b/3b)-).

examples/models/llama2/Android3_2_1B_bf16.gif

@@ -85,10 +85,10 @@ We have verified running Llama 2 7B [mobile applications](#step-6-build-mobile-a
 ### Llama 3.2 1B and 3B
 Llama 3.2 1B and 3B performance was measured on the OnePlus 12 device. The performance measurement is expressed in terms of tokens per second using an [adb binary-based approach](#step-5-run-benchmark-on) for generating 128 tokens.
 
-|Model  | 4bit(*) via SpinQuant
-|--------| ---------------
-|1B  | 53.41 tokens/second |
-|3B | 22.98 tokens/second |
+|Model  | bf16 | 4bit(*) via SpinQuant
+|--------| ---------------------- | ---------------
+|1B  | 19.4 tokens/second | 53.41 tokens/second |
+|3B | 7.76 tokens/second | 22.98 tokens/second |
 
 (*) With SpinQuant, we currently quantize 4-bit groupwise (with groupsize 32) weight, 8bit dynamic activation of all the linear layers of the model, except embedding and output layers. The embedding and output layers are quantized as 8-bit per-channel weight and 8-bit dynamic activation.
 

examples/models/llama2/README.md

@@ -6,7 +6,7 @@
  * LICENSE file in the root directory of this source tree.
  */
 
-#include <executorch/examples/models/flamingo/cross_attention/cross_attention_mask.h>
+#include <executorch/examples/models/llama3_2_mm/cross_attention/cross_attention_mask.h>
 
 #include <algorithm>
 #include <string>

examples/models/flamingo/__init__.py renamed to examples/models/llama3_2_mm/__init__.py

@@ -6,7 +6,7 @@
  * LICENSE file in the root directory of this source tree.
  */
 
-#include <executorch/examples/models/flamingo/cross_attention/cross_attention_mask.h>
+#include <executorch/examples/models/llama3_2_mm/cross_attention/cross_attention_mask.h>
 
 #include <gtest/gtest.h>
 

examples/models/flamingo/cross_attention/cross_attention_mask.cpp renamed to examples/models/llama3_2_mm/cross_attention/cross_attention_mask.cpp

@@ -5,7 +5,7 @@
 # LICENSE file in the root directory of this source tree.
 
 import torch
-from executorch.examples.models.flamingo.preprocess.export_preprocess_lib import (
+from executorch.examples.models.llama3_2_mm.preprocess.export_preprocess_lib import (
     export_preprocess,
     get_example_inputs,
     lower_to_executorch_preprocess,

examples/models/flamingo/cross_attention/cross_attention_mask.h renamed to examples/models/llama3_2_mm/cross_attention/cross_attention_mask.h

@@ -6,7 +6,7 @@
  * LICENSE file in the root directory of this source tree.
  */
 
-#include "executorch/examples/models/flamingo/preprocess/preprocess.h"
+#include "executorch/examples/models/llama3_2_mm/preprocess/preprocess.h"
 
 #include <algorithm>
 #include <cassert>

examples/models/flamingo/cross_attention/cross_attention_mask_test.cpp renamed to examples/models/llama3_2_mm/cross_attention/cross_attention_mask_test.cpp

@@ -6,7 +6,7 @@
  * LICENSE file in the root directory of this source tree.
  */
 
-#include <executorch/examples/models/flamingo/preprocess/preprocess.h>
+#include <executorch/examples/models/llama3_2_mm/preprocess/preprocess.h>
 #include <gtest/gtest.h>
 
 using namespace ::testing;

examples/models/flamingo/cross_attention/targets.bzl renamed to examples/models/llama3_2_mm/cross_attention/targets.bzl

@@ -48,9 +48,9 @@ test_buck2_select_ops_in_list() {
     ${PYTHON_EXECUTABLE} -m examples.portable.scripts.export --model_name="add_mul"
 
     echo "Running selective build test"
-    # set max_kernel_num=21: 19 primops, add, mul
+    # set max_kernel_num=22: 19 primops, add, mul
     $BUCK run //examples/selective_build:selective_build_test \
-        --config=executorch.max_kernel_num=21 \
+        --config=executorch.max_kernel_num=22 \
         --config=executorch.select_ops=list \
         -- --model_path=./add_mul.pte
 
@@ -117,11 +117,11 @@ test_cmake_select_ops_in_list() {
 
     local example_dir=examples/selective_build
     local build_dir=cmake-out/${example_dir}
-    # set MAX_KERNEL_NUM=21: 19 primops, add, mul
+    # set MAX_KERNEL_NUM=22: 19 primops, add, mul
     rm -rf ${build_dir}
     retry cmake -DBUCK2="$BUCK" \
             -DCMAKE_BUILD_TYPE=Release \
-            -DMAX_KERNEL_NUM=21 \
+            -DMAX_KERNEL_NUM=22 \
             -DEXECUTORCH_SELECT_OPS_LIST="aten::convolution.out,\
 aten::_native_batch_norm_legit_no_training.out,aten::hardtanh.out,aten::add.out,\
 aten::mean.out,aten::view_copy.out,aten::permute_copy.out,aten::addmm.out,\

examples/models/flamingo/install_requirements.sh renamed to examples/models/llama3_2_mm/install_requirements.sh

@@ -21,8 +21,30 @@
 #include <executorch/runtime/executor/method.h>
 #include <executorch/runtime/executor/program.h>
 
-namespace torch {
-namespace executor {
+using ::executorch::aten::Tensor;
+using ::executorch::extension::BufferDataLoader;
+using ::executorch::runtime::ArrayRef;
+using ::executorch::runtime::Backend;
+using ::executorch::runtime::BackendExecutionContext;
+using ::executorch::runtime::BackendInitContext;
+using ::executorch::runtime::CompileSpec;
+using ::executorch::runtime::DelegateHandle;
+using ::executorch::runtime::Error;
+using ::executorch::runtime::EValue;
+using ::executorch::runtime::FreeableBuffer;
+using ::executorch::runtime::HierarchicalAllocator;
+using ::executorch::runtime::MemoryAllocator;
+using ::executorch::runtime::MemoryManager;
+using ::executorch::runtime::Method;
+using ::executorch::runtime::MethodMeta;
+using ::executorch::runtime::Program;
+using ::executorch::runtime::Result;
+using ::executorch::runtime::Span;
+using ::executorch::runtime::Tag;
+using ::executorch::runtime::internal::copy_tensor_data;
+
+namespace example {
+
 /**
  * ExecutorBackend is a backend to execute an executorch program via delegate.
  * In preprocess, the preprocesed bytes (delegate blob) is an executorch
@@ -51,8 +73,8 @@ class ExecutorBackend final : public ::executorch::runtime::BackendInterface {
     // will return the data directly without copying it.
     MemoryAllocator* runtime_allocator = context.get_runtime_allocator();
     auto loader = ET_ALLOCATE_INSTANCE_OR_RETURN_ERROR(
-        runtime_allocator, util::BufferDataLoader);
-    new (loader) util::BufferDataLoader(processed->data(), processed->size());
+        runtime_allocator, BufferDataLoader);
+    new (loader) BufferDataLoader(processed->data(), processed->size());
     // Can't free `processed` because the program will point into that memory.
 
     // Try loading the program.
@@ -150,7 +172,7 @@ class ExecutorBackend final : public ::executorch::runtime::BackendInterface {
       if (output.tag == Tag::Tensor) {
         Tensor t_src = output.toTensor();
         Tensor t_dst = args[num_inputs + i]->toTensor();
-        status = internal::copy_tensor_data(t_dst, t_src);
+        status = copy_tensor_data(t_dst, t_src);
       }
     }
 
@@ -165,12 +187,11 @@ class ExecutorBackend final : public ::executorch::runtime::BackendInterface {
   }
 };
 
-Error registerExecutorBackend() {
+Error register_executor_backend() {
   static auto cls = ExecutorBackend();
   static Backend backend{"ExecutorBackend", &cls};
   static auto success_with_compiler = register_backend(backend);
   return success_with_compiler;
 }
 
-} // namespace executor
-} // namespace torch
+} // namespace example

examples/models/flamingo/preprocess/__init__.py renamed to examples/models/llama3_2_mm/preprocess/__init__.py

@@ -10,10 +10,8 @@
 
 #include <executorch/runtime/core/error.h>
 
-namespace torch {
-namespace executor {
+namespace example {
 
-Error registerExecutorBackend();
+::executorch::runtime::Error register_executor_backend();
 
-} // namespace executor
-} // namespace torch
+} // namespace example

examples/models/flamingo/preprocess/export_preprocess.py renamed to examples/models/llama3_2_mm/preprocess/export_preprocess.py

@@ -10,10 +10,9 @@
 #include <executorch/runtime/backend/interface.h>
 #include <executorch/runtime/core/error.h>
 
-namespace torch {
-namespace executor {
+namespace example {
 namespace {
-static Error register_success = registerExecutorBackend();
+static ::executorch::runtime::Error register_success =
+    register_executor_backend();
 } // namespace
-} // namespace executor
-} // namespace torch
+} // namespace example

examples/models/flamingo/preprocess/export_preprocess_lib.py renamed to examples/models/llama3_2_mm/preprocess/export_preprocess_lib.py

@@ -11,7 +11,7 @@
 		03B0118E2CAC567900054791 /* DynamicTestCase.m in Sources */ = {isa = PBXBuildFile; fileRef = 03B0118C2CAC567900054791 /* DynamicTestCase.m */; };
 		03B011912CAD114E00054791 /* ResourceTestCase.m in Sources */ = {isa = PBXBuildFile; fileRef = 03B011902CAD114E00054791 /* ResourceTestCase.m */; };
 		03B2D37A2C8A515C0046936E /* GenericTests.mm in Sources */ = {isa = PBXBuildFile; fileRef = 03B2D3792C8A515C0046936E /* GenericTests.mm */; };
-		03C7FA382C8AA3EC00E6E9AE /* Models in Resources */ = {isa = PBXBuildFile; fileRef = 03C7FA322C8AA24200E6E9AE /* Models */; };
+		03C7FA382C8AA3EC00E6E9AE /* Resources in Resources */ = {isa = PBXBuildFile; fileRef = 03C7FA322C8AA24200E6E9AE /* Resources */; };
 		03DD00A92C8FE44600FE4619 /* backend_coreml.xcframework in Frameworks */ = {isa = PBXBuildFile; fileRef = 03DD00992C8FE44600FE4619 /* backend_coreml.xcframework */; };
 		03DD00AA2C8FE44600FE4619 /* kernels_custom.xcframework in Frameworks */ = {isa = PBXBuildFile; fileRef = 03DD009A2C8FE44600FE4619 /* kernels_custom.xcframework */; };
 		03DD00AF2C8FE44600FE4619 /* kernels_portable.xcframework in Frameworks */ = {isa = PBXBuildFile; fileRef = 03DD009F2C8FE44600FE4619 /* kernels_portable.xcframework */; };
@@ -50,7 +50,7 @@
 		03B0118F2CAD114E00054791 /* ResourceTestCase.h */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.h; path = ResourceTestCase.h; sourceTree = "<group>"; };
 		03B011902CAD114E00054791 /* ResourceTestCase.m */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.objc; path = ResourceTestCase.m; sourceTree = "<group>"; };
 		03B2D3792C8A515C0046936E /* GenericTests.mm */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.cpp.objcpp; path = GenericTests.mm; sourceTree = "<group>"; };
-		03C7FA322C8AA24200E6E9AE /* Models */ = {isa = PBXFileReference; lastKnownFileType = folder; path = Models; sourceTree = SOURCE_ROOT; };
+		03C7FA322C8AA24200E6E9AE /* Resources */ = {isa = PBXFileReference; lastKnownFileType = folder; path = Resources; sourceTree = SOURCE_ROOT; };
 		03DD00992C8FE44600FE4619 /* backend_coreml.xcframework */ = {isa = PBXFileReference; lastKnownFileType = wrapper.xcframework; name = backend_coreml.xcframework; path = Frameworks/backend_coreml.xcframework; sourceTree = "<group>"; };
 		03DD009A2C8FE44600FE4619 /* kernels_custom.xcframework */ = {isa = PBXFileReference; lastKnownFileType = wrapper.xcframework; name = kernels_custom.xcframework; path = Frameworks/kernels_custom.xcframework; sourceTree = "<group>"; };
 		03DD009F2C8FE44600FE4619 /* kernels_portable.xcframework */ = {isa = PBXFileReference; lastKnownFileType = wrapper.xcframework; name = kernels_portable.xcframework; path = Frameworks/kernels_portable.xcframework; sourceTree = "<group>"; };
@@ -105,7 +105,7 @@
 			children = (
 				03B2D3662C8A515A0046936E /* App */,
 				03ED6CEB2C8AAF5300F2D6EE /* Frameworks */,
-				03C7FA322C8AA24200E6E9AE /* Models */,
+				03C7FA322C8AA24200E6E9AE /* Resources */,
 				03B2D3782C8A515C0046936E /* Tests */,
 				03B0118D2CAC567900054791 /* TestUtils */,
 				03B2D3652C8A515A0046936E /* Products */,
@@ -261,7 +261,7 @@
 			isa = PBXResourcesBuildPhase;
 			buildActionMask = 2147483647;
 			files = (
-				03C7FA382C8AA3EC00E6E9AE /* Models in Resources */,
+				03C7FA382C8AA3EC00E6E9AE /* Resources in Resources */,
 			);
 			runOnlyForDeploymentPostprocessing = 0;
 		};

examples/models/flamingo/preprocess/preprocess.cpp renamed to examples/models/llama3_2_mm/preprocess/preprocess.cpp

@@ -0,0 +1,25 @@
+#!/bin/bash
+# Copyright (c) Meta Platforms, Inc. and affiliates.
+# All rights reserved.
+#
+# This source code is licensed under the BSD-style license found in the
+# LICENSE file in the root directory of this source tree.
+
+frameworks=(
+  "backend_coreml"
+  "backend_mps"
+  "backend_xnnpack"
+  "executorch"
+  "kernels_custom"
+  "kernels_optimized"
+  "kernels_portable"
+  "kernels_quantized"
+)
+
+for framework in "${frameworks[@]}"; do
+  rm -f "${f}-latest.zip"
+  rm -rf "${f}.xcframework"
+  curl -sSLO "https://ossci-ios.s3.amazonaws.com/executorch/${framework}-latest.zip" && \
+  unzip -q "${framework}-latest.zip" && \
+  rm "${framework}-latest.zip"
+done

examples/models/flamingo/preprocess/preprocess.h renamed to examples/models/llama3_2_mm/preprocess/preprocess.h

@@ -0,0 +1,4 @@
+# Resources
+
+This directory and all files in it will be copied to the bundle’s resources directory, accessible via `NSBundle.resourcePath`.
+Place any resource files you want to access at runtime here.

examples/models/flamingo/preprocess/preprocess_test.cpp renamed to examples/models/llama3_2_mm/preprocess/preprocess_test.cpp

@@ -29,8 +29,8 @@ @implementation GenericTests
 
 + (NSArray<NSString *> *)directories {
   return @[
-    @"Models",
-    @"aatp/data",
+    @"Resources",
+    @"aatp/data", // AWS Farm devides look at this subdirectory.
   ];
 }