Skip to content

Commit 5a45132

Browse files
authored
[llm] Fix llama/README.md (#11543)
This PR simplifies README.md's cmake command by using preset. This pull request simplifies the build process for the Llama model and its runner by introducing a preset configuration and removing redundant build flags. Additionally, it updates the `CMakeLists.txt` file to enable specific features for tokenizers. ### Build process simplification: * [`examples/models/llama/README.md`](diffhunk://#diff-535f376de1f099ede770ee4d5b3c3193b5784c6a0342e292e667fe4ff9d1633eL272-L298): Replaced the detailed `cmake` commands with a preset configuration (`--preset llm`) for building executorch and removed redundant flags for the Llama runner build process. This streamlines the instructions and reduces complexity. ### Tokenizer configuration updates: * [`extension/llm/runner/CMakeLists.txt`](diffhunk://#diff-ab47c38904702e3d66a37419ca35a07815f7d4735f7e94330d17643b9f77ad2bR47-R48): Added `SUPPORT_REGEX_LOOKAHEAD` and `PCRE2_STATIC_PIC` settings to enable regex lookahead support and ensure position-independent code for tokenizers.
1 parent 6a4df94 commit 5a45132

File tree

3 files changed

+11
-23
lines changed

3 files changed

+11
-23
lines changed

.ci/scripts/test_llama.sh

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -156,8 +156,7 @@ cmake_install_executorch_libraries() {
156156
-DCMAKE_INSTALL_PREFIX=cmake-out \
157157
-DCMAKE_BUILD_TYPE="$CMAKE_BUILD_TYPE" \
158158
-DEXECUTORCH_BUILD_QNN="$QNN" \
159-
-DQNN_SDK_ROOT="$QNN_SDK_ROOT" \
160-
-Bcmake-out .
159+
-DQNN_SDK_ROOT="$QNN_SDK_ROOT"
161160
cmake --build cmake-out -j9 --target install --config "$CMAKE_BUILD_TYPE"
162161
}
163162

examples/models/llama/README.md

Lines changed: 3 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -269,33 +269,17 @@ You can export and run the original Llama 3 8B instruct model.
269269
270270
1. Build executorch with optimized CPU performance as follows. Build options available [here](https://github.com/pytorch/executorch/blob/main/CMakeLists.txt#L59).
271271
```
272-
cmake -DPYTHON_EXECUTABLE=python \
273-
-DCMAKE_INSTALL_PREFIX=cmake-out \
274-
-DEXECUTORCH_ENABLE_LOGGING=1 \
275-
-DCMAKE_BUILD_TYPE=Release \
276-
-DEXECUTORCH_BUILD_EXTENSION_DATA_LOADER=ON \
277-
-DEXECUTORCH_BUILD_EXTENSION_MODULE=ON \
278-
-DEXECUTORCH_BUILD_EXTENSION_TENSOR=ON \
279-
-DEXECUTORCH_BUILD_XNNPACK=ON \
280-
-DEXECUTORCH_BUILD_KERNELS_QUANTIZED=ON \
281-
-DEXECUTORCH_BUILD_KERNELS_OPTIMIZED=ON \
282-
-DEXECUTORCH_BUILD_KERNELS_CUSTOM=ON \
283-
-Bcmake-out .
272+
cmake --preset llm -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=cmake-out
284273
285274
cmake --build cmake-out -j16 --target install --config Release
286275
```
287276
Note for Mac users: There's a known linking issue with Xcode 15.1. Refer to the section of Common Issues and Mitigations below for solutions.
288277
289278
2. Build llama runner.
290279
```
291-
cmake -DPYTHON_EXECUTABLE=python \
292-
-DCMAKE_INSTALL_PREFIX=cmake-out \
280+
cmake -DCMAKE_INSTALL_PREFIX=cmake-out \
281+
-DBUILD_TESTING=OFF \
293282
-DCMAKE_BUILD_TYPE=Release \
294-
-DEXECUTORCH_BUILD_KERNELS_CUSTOM=ON \
295-
-DEXECUTORCH_BUILD_KERNELS_OPTIMIZED=ON \
296-
-DEXECUTORCH_BUILD_XNNPACK=ON \
297-
-DEXECUTORCH_BUILD_KERNELS_QUANTIZED=ON \
298-
-DSUPPORT_REGEX_LOOKAHEAD=ON
299283
-Bcmake-out/examples/models/llama \
300284
examples/models/llama
301285

extension/llm/runner/CMakeLists.txt

Lines changed: 7 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -44,6 +44,10 @@ target_include_directories(
4444
add_library(extension_llm_runner STATIC ${_extension_llm_runner__srcs})
4545

4646
# add tokenizers
47+
set(SUPPORT_REGEX_LOOKAHEAD ON)
48+
# llama/runner/CMakeLists.txt builds a shared library libllama_runner.so that
49+
# transitively depends on tokenizers. Need to build tokenizers with -fPIC.
50+
set(CMAKE_POSITION_INDEPENDENT_CODE ON)
4751
add_subdirectory(
4852
${EXECUTORCH_ROOT}/extension/llm/tokenizers
4953
${CMAKE_CURRENT_BINARY_DIR}/../../../extension/llm/tokenizers
@@ -54,8 +58,9 @@ set(runner_deps executorch_core extension_module extension_tensor tokenizers)
5458
target_link_libraries(extension_llm_runner PUBLIC ${runner_deps})
5559

5660
target_include_directories(
57-
extension_llm_runner INTERFACE ${_common_include_directories}
58-
${EXECUTORCH_ROOT}/extension/llm/tokenizers/include
61+
extension_llm_runner
62+
INTERFACE ${_common_include_directories}
63+
${EXECUTORCH_ROOT}/extension/llm/tokenizers/include
5964
)
6065

6166
if(BUILD_TESTING)

0 commit comments

Comments
 (0)