Skip to content

remove buck2 from mps readme #4232

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 9 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 3 additions & 6 deletions docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -94,21 +94,18 @@ To build the documentation locally:

You can use custom variables in your `.md` and `.rst` files. The variables take
their values from the files listed in the `./.ci/docker/ci_commit_pins/`
directory. For example, to insert a variable that specifies the latest Buck2
directory. For example, to insert a variable that specifies the latest PyTorch
version, use the following syntax:

```
The current version of Buck2 is ${executorch_version:buck2}.
The current version of PyTorch is ${executorch_version:pytorch}.
```

This will result in the following output:

<img src="./source/_static/img/s_custom_variables_extension.png" width="300">

We have the following custom variables defined in this docset:

- `${executorch_version:buck2}`
- `${executorch_version:pytorch}`
Right now we only support PyTorch version as custom variable, but will support others in the future.

You can use the variables in both regular text and code blocks.

Expand Down
Binary file modified docs/source/_static/img/s_custom_variables_extension.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
5 changes: 2 additions & 3 deletions docs/source/backend-delegates-dependencies.md
Original file line number Diff line number Diff line change
Expand Up @@ -123,9 +123,8 @@ but the URLs can't be in the requirements.txt, so not recommending this for now.
The recommended approach is to include a git submodule for a given C++
dependency in the `executorch/backends/<delegate_name>/third-party` directory.

### buck2/CMake Support
At a minimum CMake support is required. Adding buck2 support is optional, but
will make the delegate available to more ExecuTorch users.
### CMake Support
At a minimum CMake support is required.

<!---
TODO: Add more details about: complying with ET runtime build configurations;
Expand Down
4 changes: 0 additions & 4 deletions docs/source/backend-delegates-integration.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,10 +30,6 @@ should be built conditionally with an ExecuTorch build flag like
For third-party dependencies, please refer to
[this](./backend-delegates-dependencies.md).


Adding buck2 support is optional, but will make the delegate available to more
ExecuTorch users.

<!---
TODO: Add more details. Need to insert a CMake layer in `executorch/backends` to
provide some uniform abstraction across delegates.
Expand Down
35 changes: 0 additions & 35 deletions docs/source/getting-started-setup.md
Original file line number Diff line number Diff line change
Expand Up @@ -195,41 +195,6 @@ Output 0: tensor(sizes=[1], [2.])

To learn how to build a similar program, visit the [Runtime APIs Tutorial](extension-module.md).

### [Optional] Setting Up Buck2
**Buck2** is an open-source build system that some of our examples currently utilize for building and running.

However, please note that the installation of `Buck2` is optional for using ExecuTorch and we are in the process of transitioning away from `Buck2` and migrating all relevant sections to `cmake`. This section will be removed once we finish the migration.

To set up `Buck2`, You will need the following prerequisits for this section:
* The `zstd` command line tool — install by running
```bash
pip3 install zstd
```
* Version `${executorch_version:buck2}` of the `buck2` commandline tool — you can download a
prebuilt archive for your system from [the Buck2
repo](https://github.com/facebook/buck2/releases/tag/2024-05-15). Note that
the version is important, and newer or older versions may not work with the
version of the buck2 prelude used by the ExecuTorch repo.

Configure Buck2 by decompressing with the following command (filename depends
on your system, and the location of the binary can be different):

```bash
# For example, buck2-x86_64-unknown-linux-musl.zst for Linux, or buck2-aarch64-apple-darwin.zst for Mac with Apple silicon.
zstd -cdq buck2-DOWNLOADED_FILENAME.zst > /tmp/buck2 && chmod +x /tmp/buck2
```

You may want to copy the `buck2` binary into your `$PATH` so you can run it
as `buck2`.

After the installation, you can run the `add.pte` program by following `buck2` command:

```bash
/tmp/buck2 run //examples/portable/executor_runner:executor_runner -- --model_path add.pte
```

Note that the first run may take a while as it will have to complie the kernels from sources

## Next Steps

Congratulations! You have successfully exported, built, and run your first
Expand Down
42 changes: 1 addition & 41 deletions docs/source/kernel-library-custom-aten-kernel.md
Original file line number Diff line number Diff line change
Expand Up @@ -218,7 +218,7 @@ ExecuTorch does not support all of the argument types that core PyTorch supports

### Build Tool Macros

We provide build time macros to help users to build their kernel registration library. The macro takes the yaml file describing the kernel library as well as model operator metadata, and packages the generated C++ bindings into a C++ library. The macro is available on both CMake and Buck2.
We provide build time macros to help users to build their kernel registration library. The macro takes the yaml file describing the kernel library as well as model operator metadata, and packages the generated C++ bindings into a C++ library. The macro is available on CMake.


#### CMake
Expand Down Expand Up @@ -263,46 +263,6 @@ And out fallback:

The merged yaml will have the entry in functions.yaml.

#### Buck2

`executorch_generated_lib` is the macro that takes the yaml files and depends on the selective build macro `et_operator_library`. For an example:
```python
# Yaml file for kernel library
export_file(
name = "functions.yaml"
)

# Kernel library
cxx_library(
name = "add_kernel",
srcs = ["add.cpp"],
)

# Selective build artifact, it allows all operators to be registered
et_operator_library(
name = "all_ops",
include_all_ops = True, # Select all ops in functions.yaml
)

# Prepare a generated_lib
executorch_generated_lib(
name = "generated_lib",
functions_yaml_target = ":functions.yaml",
deps = [
":all_ops",
":add_kernel",
],
)

# Link generated_lib to ExecuTorch binary
cxx_binary(
name = "executorch_bin",
deps = [
":generated_lib",
],
)

```

### Custom Ops API Best Practices

Expand Down
58 changes: 3 additions & 55 deletions docs/source/kernel-library-selective-build.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Kernel Library Selective Build

_Selective build_ is a build mode on ExecuTorch that uses model metadata to guide ExecuTorch build. This build mode contains build tool APIs available on both CMake and buck2. ExecuTorch users can use selective build APIs to build an ExecuTorch runtime binary with minimal binary size by only including operators required by models.
_Selective build_ is a build mode on ExecuTorch that uses model metadata to guide ExecuTorch build. This build mode contains build tool APIs available on CMake. ExecuTorch users can use selective build APIs to build an ExecuTorch runtime binary with minimal binary size by only including operators required by models.

This document aims to help ExecuTorch users better use selective build, by listing out available APIs, providing an overview of high level architecture and showcasing examples.

Expand Down Expand Up @@ -36,17 +36,11 @@ The basic flow looks like this:

## APIs

We expose build macros for CMake and Buck2, to allow users specifying op info.

On CMake:
We expose build macros for CMake, to allow users specifying op info:

[gen_selected_ops](https://github.com/pytorch/executorch/blob/main/build/Codegen.cmake#L12)

On Buck2:

[et_operator_library](https://github.com/pytorch/executorch/blob/main/shim/xplat/executorch/codegen/codegen.bzl#L44C21-L44C21)

Both of these build macros take the following inputs:
Build macros take the following inputs:


### Select all ops
Expand All @@ -66,52 +60,6 @@ This API lets users pass in a list of operator names. Note that this API can be

## Example Walkthrough


### Buck2 example

Let’s take a look at the following build:

```
# Select a list of operators: defined in `ops`
et_operator_library(
name = "select_ops_in_list",
ops = [
"aten::add.out",
"aten::mm.out",
],
)
```
This target generates the yaml file containing op info for these two ops.

In addition to that, if we want to select all ops from a kernel library, we can do:

```
# Select all ops from a yaml file
et_operator_library(
name = "select_ops_from_yaml",
ops_schema_yaml_target = "//executorch/examples/portable/custom_ops:custom_ops.yaml",
)
```
Then in the kernel registration library we can do:
```
executorch_generated_lib(
name = "select_ops_lib",
custom_ops_yaml_target = "//executorch/examples/portable/custom_ops:custom_ops.yaml",
functions_yaml_target = "//executorch/kernels/portable:functions.yaml",
deps = [
"//executorch/examples/portable/custom_ops:custom_ops_1", # kernel library
"//executorch/examples/portable/custom_ops:custom_ops_2", # kernel library
"//executorch/kernels/portable:operators", # kernel library
":select_ops_from_yaml",
":select_ops_in_list",
],
)
```
Notice we are allowlisting both add.out, mm.out from the list, and the ones from the schema yaml (`custom_ops.yaml`).


### CMake example

In CMakeLists.txt we have the following logic:
```cmake
set(_kernel_lib)
Expand Down
28 changes: 6 additions & 22 deletions docs/source/runtime-build-and-cross-compilation.md
Original file line number Diff line number Diff line change
@@ -1,16 +1,13 @@
# Building with CMake

Although buck2 is the main build system for the ExecuTorch project, it's also
possible to build core pieces of the runtime using [CMake](https://cmake.org/)
for easier integration with other build systems. Even if you don't use CMake
directly, CMake can emit scripts for other format like Make, Ninja or Xcode.
For information, see [cmake-generators(7)](https://cmake.org/cmake/help/latest/manual/cmake-generators.7.html).
ExecuTorch uses [CMake](https://cmake.org/) as its primary build system.
Even if you don't use CMake directly, CMake can emit scripts for other format
like Make, Ninja or Xcode. For information, see [cmake-generators(7)](https://cmake.org/cmake/help/latest/manual/cmake-generators.7.html).

## Targets Built by the CMake Build System

ExecuTorch's CMake build system doesn't cover everything that the buck2 build
system covers. It can only build pieces of the runtime that are likely to be
useful to embedded systems users.
ExecuTorch's CMake build system covers the pieces of the runtime that are
likely to be useful to embedded systems users.

- `libexecutorch.a`: The core of the ExecuTorch runtime. Does not contain any
operator/kernel definitions or backend definitions.
Expand All @@ -31,17 +28,8 @@ useful to embedded systems users.

Follow the steps below to have the tools ready before using CMake to build on your machine.

1. Clone the repo and install buck2 as described in the "Building a Runtime" section
of [Setting Up ExecuTorch](getting-started-setup.md#building-a-runtime)
- `buck2` is necessary because the CMake build system runs `buck2` commands
to extract source lists from the primary build system. It will be possible
to configure the CMake system to avoid calling `buck2`, though.
2. If your system's version of python3 is older than 3.11:
1. If your system's version of python3 is older than 3.11:
- Run `pip install tomli`
- This provides an import required by a script that the CMake build system
calls to extract source lists from `buck2`. Consider doing this `pip
install` inside your Python or Conda virtual environment if you created
one already by following [Setting up ExecuTorch](getting-started-setup.md#setting-up-executorch).
3. Install CMake version 3.19 or later:
- Run `conda install cmake` or `pip install cmake`.

Expand All @@ -57,9 +45,6 @@ cd executorch

# Clean and configure the CMake build system. It's good practice to do this
# whenever cloning or pulling the upstream repo.
#
# NOTE: If your `buck2` binary is not on the PATH, you can change this line to
# say something like `-DBUCK2=/tmp/buck2` to point directly to the tool.
(rm -rf cmake-out && mkdir cmake-out && cd cmake-out && cmake ..)
```

Expand Down Expand Up @@ -140,7 +125,6 @@ Assuming Android NDK is available, run:
rm -rf cmake-android-out && mkdir cmake-android-out && cd cmake-android-out

# point -DCMAKE_TOOLCHAIN_FILE to the location where ndk is installed
# Run `which buck2`, if it returns empty (meaning the system doesn't know where buck2 is installed), pass in this flag `-DBUCK2=/path/to/buck2` pointing to buck2
cmake -DCMAKE_TOOLCHAIN_FILE=/Users/{user_name}/Library/Android/sdk/ndk/25.2.9519653/build/cmake/android.toolchain.cmake -DANDROID_ABI=arm64-v8a ..

cd ..
Expand Down
3 changes: 1 addition & 2 deletions docs/source/runtime-overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -101,8 +101,7 @@ can build it for a wide variety of target systems.
to them.
* The code is compatible with GCC and Clang, and has also been built with
several proprietary embedded toolchains.
* The repo provides both CMake and buck2 build systems to make integration
easier.
* The repo provides CMake build system to make integration easier.

#### Operating System Considerations

Expand Down
3 changes: 1 addition & 2 deletions examples/apple/mps/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,8 +28,7 @@ Once we have the model binary file, then let's run it with the ExecuTorch runtim

```bash
# Build and install executorch
cmake -DBUCK2="$BUCK" \
-DCMAKE_INSTALL_PREFIX=cmake-out \
cmake -DCMAKE_INSTALL_PREFIX=cmake-out \
-DCMAKE_BUILD_TYPE=Release \
-DEXECUTORCH_BUILD_SDK=ON \
-DEXECUTORCH_ENABLE_EVENT_TRACER=ON \
Expand Down
Loading