Skip to content

Commit 665d8ff

Browse files
Riandypytorchbot
authored andcommitted
Improve iOS readme (#5877)
Summary: Pull Request resolved: #5877 Addressing feedback from D63264488 - Rewrite the XCode setup flow for better clarity. - Added section numbering and reorder some of the flow, emphasizing which steps are recommended and which are optional (For example, user can manually build the package, but it's gonna be more involved and has to deal with lib linkage, etc) - Mentioned that package dependencies has been pre-configured and clarified instructions on how to change that if needed - Added more details on which package should link against which app target - Added more details on how to confirm which packages have been linked in case the options are greyed out Reviewed By: shoumikhin Differential Revision: D63403105 fbshipit-source-id: c9867efa2d48e3e37f39b7a98b9c644390bcd3b2 (cherry picked from commit f3de2bb)
1 parent 60cc6bc commit 665d8ff

File tree

1 file changed

+81
-20
lines changed

1 file changed

+81
-20
lines changed

examples/demo-apps/apple_ios/LLaMA/docs/delegates/xnnpack_README.md

Lines changed: 81 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -79,68 +79,129 @@ python -m executorch.examples.models.llava.export_llava --pte-name llava.pte --w
7979

8080
## Configure the XCode Project
8181

82-
### Install CMake
82+
### 1. Install CMake
8383
Download and open the macOS .dmg installer at https://cmake.org/download and move the Cmake app to /Applications folder.
8484
Install Cmake command line tools:
8585

8686
```
8787
sudo /Applications/CMake.app/Contents/bin/cmake-gui --install
8888
```
8989

90+
### 2. Add ExecuTorch Runtime Package
9091

91-
### Swift Package Manager
92-
The prebuilt ExecuTorch runtime, backend, and kernels are available as a Swift PM package.
92+
There are two options to add ExecuTorch runtime package into your XCode project:
9393

94-
### Xcode
95-
Open the project in Xcode.In Xcode, go to `File > Add Package Dependencies`. Paste the URL of the ExecuTorch repo into the search bar and select it. Make sure to change the branch name to the desired ExecuTorch version, e.g., “0.3.0”, or just use the “latest” branch name for the latest stable build.
94+
- [Recommended] Prebuilt package (via Swift Package Manager)
95+
- Manually build the package locally and link them
9696

97-
Note: If you're running into any issues related to package dependencies, quit Xcode entirely, delete the whole executorch repo, clean the caches by running the command below in terminal and clone the repo again.
97+
98+
### 2.1 [Recommended] Prebuilt package (via Swift Package Manager)
99+
100+
The current XCode project is pre-configured to automatically download and link the latest prebuilt package via Swift Package Manager.
101+
102+
If you have an old ExecuTorch package cached before in XCode, or are running into any package dependencies issues (incorrect checksum hash, missing package, outdated package), close XCode and run the following command in terminal inside your ExecuTorch directory
98103

99104
```
100105
rm -rf \
101106
~/Library/org.swift.swiftpm \
102107
~/Library/Caches/org.swift.swiftpm \
103108
~/Library/Caches/com.apple.dt.Xcode \
104-
~/Library/Developer/Xcode/DerivedData
109+
~/Library/Developer/Xcode/DerivedData \
110+
examples/demo-apps/apple_ios/LLaMA/LLaMA.xcodeproj/project.xcworkspace/xcshareddata/swiftpm
105111
```
106112

107-
Link your binary with the ExecuTorch runtime and any backends or kernels used by the exported ML model. It is recommended to link the core runtime to the components that use ExecuTorch directly, and link kernels and backends against the main app target.
113+
The command above will clear all the package cache, and when you re-open the XCode project, it should re-download the latest package and link them correctly.
108114

109-
Note: To access logs, link against the Debug build of the ExecuTorch runtime, i.e., the executorch_debug framework. For optimal performance, always link against the Release version of the deliverables (those without the _debug suffix), which have all logging overhead removed.
115+
#### (Optional) Changing the prebuilt package version
116+
While we recommended using the latest prebuilt package pre-configured with the XCode project, you can also change the package version manually to your desired version.
110117

111-
For more details integrating and Running ExecuTorch on Apple Platforms, checkout this [link](https://pytorch.org/executorch/main/apple-runtime.html).
118+
Go to Project Navigator, click on LLaMA. `Project --> LLaMA --> Package Dependencies`, and update the package dependencies to any of the available options below:
112119

113-
<p align="center">
114-
<img src="https://raw.githubusercontent.com/pytorch/executorch/refs/heads/main/docs/source/_static/img/ios_demo_app_swift_pm.png" alt="iOS LLaMA App Swift PM" style="width:600px">
115-
</p>
120+
- Branch --> latest
121+
- Branch --> 0.4.0
122+
- Branch --> 0.3.0
123+
- Commit --> (Specify the commit hash, for example: `bdf3f5a1047c73ef61bb3e956d1d4528de743077`. Full list [here](https://github.com/pytorch/executorch/commits/latest/))
124+
125+
126+
### 2.2 Manually build the package locally and link them
127+
128+
Note: You should only use this step if the prebuilt package doesn't work for your usecase (For example, you require the latest PRs from main, where there are no pre-built package yet)
129+
130+
If you need to manually build the package, run the following command in your terminal
131+
```
132+
# Install a compatible version of Buck2
133+
BUCK2_RELEASE_DATE="2024-05-15"
134+
BUCK2_ARCHIVE="buck2-aarch64-apple-darwin.zst"
135+
BUCK2=".venv/bin/buck2"
136+
137+
curl -LO "https://github.com/facebook/buck2/releases/download/$BUCK2_RELEASE_DATE/$BUCK2_ARCHIVE"
138+
zstd -cdq "$BUCK2_ARCHIVE" > "$BUCK2" && chmod +x "$BUCK2"
139+
rm "$BUCK2_ARCHIVE"
140+
141+
./build/build_apple_frameworks.sh --buck2="$(realpath $BUCK2)" --coreml --custom --mps --optimized --portable --quantized --xnnpack
142+
```
143+
144+
After the build finishes successfully, the resulting frameworks can be found in the `cmake-out` directory. Copy them to your project and link them against your targets.
116145

117-
Then select which ExecuTorch framework should link against which target.
146+
The following packages should be linked in your app target `LLaMA` (left side, LLaMA --> General --> select LLaMA under "TARGETS" --> scroll down to "Frameworks, Libraries, and Embedded Content")
147+
- backend_coreml
148+
- backend_mps
149+
- backend_xnnpack
150+
- kernels_custom
151+
- kernels_optimized
152+
- kernels_portable
153+
- kernels_quantized
154+
155+
The following package should be linked in your target app `LLaMARunner` (left side, LLaMA --> General --> select LLaMARunner under "TARGETS" --> scroll down to "Frameworks and Libraries")
156+
- executorch
118157

119158
<p align="center">
120159
<img src="https://raw.githubusercontent.com/pytorch/executorch/refs/heads/main/docs/source/_static/img/ios_demo_app_choosing_package.png" alt="iOS LLaMA App Choosing package" style="width:600px">
121160
</p>
122161

123-
Click “Run” to build the app and run in on your iPhone.
162+
If you cannot add the package into your app target (it's greyed out), it might have been linked before. You can verify it by checking it from your target app `(LLaMA / LLaMARunner) --> Build Phases --> Link Binary With Libraries`.
163+
164+
165+
166+
More details on integrating and Running ExecuTorch on Apple Platforms, check out the detailed guide [here](https://pytorch.org/executorch/main/apple-runtime.html#local-build).
167+
168+
### 3. Configure Build Schemes
169+
170+
The project has two build configurations:
171+
- Debug
172+
- [Recommended] Release
173+
174+
Navigate to `Product --> Scheme --> Edit Scheme --> Info --> Build Configuration` and update the configuration to "Release".
124175

125-
## Pushing Model and Tokenizer
176+
We recommend that you only use the Debug build scheme during development, where you might need to access additional logs. Debug build has logging overhead and will impact inferencing performance, while release build has compiler optimizations enabled and all logging overhead removed.
126177

127-
### Copy the model to Simulator
178+
For more details integrating and Running ExecuTorch on Apple Platforms or building the package locally, checkout this [link](https://pytorch.org/executorch/main/apple-runtime.html).
179+
180+
### 4. Build and Run the project
181+
182+
Click the "play" button on top right of your XCode app, or navigate to `Product --> Run` to build and run the app on your phone.
183+
184+
### 5. Pushing Model and Tokenizer
185+
186+
There are two options to copy the model (.pte) and tokenizer files (.model) to your app, depending on whether you are running it on a simulator or device.
187+
188+
#### 5.1 Copy the model and tokenizer to Simulator
128189
* Drag&drop the model and tokenizer files onto the Simulator window and save them somewhere inside the iLLaMA folder.
129190
* Pick the files in the app dialog, type a prompt and click the arrow-up button.
130191

131-
### Copy the model to Device
132-
* Wire-connect the device and open the contents in Finder.
192+
#### 5.2 Copy the model and tokenizer to Device
193+
* Plug the device into your Mac and open the contents in Finder.
133194
* Navigate to the Files tab and drag & drop the model and tokenizer files onto the iLLaMA folder.
134195
* Wait until the files are copied.
135196

197+
### 6. Try out the app
136198
Open the iLLaMA app, click the settings button at the top left of the app to select the model and tokenizer files. When the app successfully runs on your device, you should see something like below:
137199

138200
<p align="center">
139201
<img src="https://raw.githubusercontent.com/pytorch/executorch/refs/heads/main/docs/source/_static/img/ios_demo_app.jpg" alt="iOS LLaMA App" style="width:300px">
140202
</p>
141203

142204

143-
144205
For Llava 1.5 models, you can select and image (via image/camera selector button) before typing prompt and send button.
145206

146207
<p align="center">

0 commit comments

Comments
 (0)