Skip to content

Commit 57c878e

Browse files
committed
Simplify pip install step of install_requirements.sh
Summary: It's not clear why we were using --force-reinstall; if the requested versions are different from what's currently installed, pip will handle the reinstallation. And calling pip multiple times with --force-reinstall meant that we were uninstalling and reinstalling the same packages over and over. Now we only call `pip install` once for requirements, and we do not force reinstall. On my machine, without enabling pybindings, a no-op run of this script (i.e. when all packages are up-to-date) takes 7 seconds, including the `pip install .`. Test Plan: ``` ./install_requirements.sh ``` Reviewers: Subscribers: Tasks: Tags:
1 parent 79a4ba0 commit 57c878e

File tree

1 file changed

+46
-20
lines changed

1 file changed

+46
-20
lines changed

install_requirements.sh

Lines changed: 46 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -41,35 +41,61 @@ for arg in "$@"; do
4141
esac
4242
done
4343

44-
# Install pytorch dependencies
4544
#
46-
# Note:
47-
# When getting a new version of the executorch repo (via clone, fetch, or pull),
48-
# you may need to re-install a new version for all pytorch dependencies to run the
49-
# models in executorch/examples/models.
50-
# The version in this file will be the correct version for the
51-
# corresponsing version of the repo.
45+
# Install pytorch and its dependencies.
46+
#
47+
48+
# The NIGHTLY_VERSION value will agree with the third-party/pytorch pinned
49+
# submodule commit.
50+
#
51+
# NOTE: If a newly-fetched version of the executorch repo changes the value of
52+
# NIGHTLY_VERSION, you should re-run this script to install the necessary
53+
# package versions.
5254
NIGHTLY_VERSION=dev20240324
5355

54-
TORCH_VERSION=2.4.0.${NIGHTLY_VERSION}
55-
pip install --force-reinstall --pre torch=="${TORCH_VERSION}" -i https://download.pytorch.org/whl/nightly/cpu
56+
# The pip repository that hosts nightly torch packages.
57+
TORCH_NIGHTLY_URL="https://download.pytorch.org/whl/nightly/cpu"
5658

57-
TORCH_VISION_VERSION=0.19.0.${NIGHTLY_VERSION}
58-
pip install --force-reinstall --pre torchvision=="${TORCH_VISION_VERSION}" -i https://download.pytorch.org/whl/nightly/cpu
59+
# pip packages needed by exir.
60+
EXIR_REQUIREMENTS=(
61+
torch==2.4.0.${NIGHTLY_VERSION}
62+
torchaudio==2.2.0.${NIGHTLY_VERSION} # For testing.
63+
)
5964

60-
TORCH_AUDIO_VERSION=2.2.0.${NIGHTLY_VERSION}
61-
pip install --force-reinstall --pre torchaudio=="${TORCH_AUDIO_VERSION}" -i https://download.pytorch.org/whl/nightly/cpu
65+
# pip packages needed for development.
66+
DEVEL_REQUIREMENTS=(
67+
setuptools # For building the pip package.
68+
tomli # Imported by extract_sources.py when using python < 3.11.
69+
wheel # For building the pip package archive.
70+
zstd # Imported by resolve_buck.py.
71+
)
6272

63-
TIMM_VERSION=0.6.13
64-
pip install --pre timm==${TIMM_VERSION}
73+
# pip packages needed to run examples.
74+
# TODO(dbort): Make each example publish its own requirements.txt
75+
EXAMPLES_REQUIREMENTS=(
76+
timm==0.6.13
77+
torchaudio==2.2.0.${NIGHTLY_VERSION}
78+
torchsr==1.0.4
79+
transformers==4.38.2
80+
)
6581

66-
TRANSFORMERS_VERSION=4.38.2
67-
pip install --force-reinstall --pre transformers==${TRANSFORMERS_VERSION}
82+
# Assemble the list of requirements to actually install.
83+
# TODO(dbort): Add options for reducing the number of requirements.
84+
REQUIREMENTS_TO_INSTALL=(
85+
"${EXIR_REQUIREMENTS[@]}"
86+
"${DEVEL_REQUIREMENTS[@]}"
87+
"${EXAMPLES_REQUIREMENTS[@]}"
88+
)
6889

69-
TORCHSR_VERSION=1.0.4
70-
pip install --pre torchsr==${TORCHSR_VERSION}
90+
# Install the requirements. `--extra-index-url` tells pip to look for package
91+
# versions on the provided URL if they aren't available on the default URL.
92+
pip install --extra-index-url "${TORCH_NIGHTLY_URL}" \
93+
"${REQUIREMENTS_TO_INSTALL[@]}"
94+
95+
#
96+
# Install executorch pip package.
97+
#
7198

72-
# Install ExecuTorch after dependencies are installed.
7399
EXECUTORCH_BUILD_PYBIND="$EXECUTORCH_BUILD_PYBIND" \
74100
CMAKE_ARGS="$CMAKE_ARGS" \
75101
CMAKE_BUILD_PARALLEL_LEVEL=9 \

0 commit comments

Comments
 (0)