Skip to content

Commit 13850d6

Browse files
committed
Simplify pip install step of install_requirements.sh
Summary: It's not clear why we were using --force-reinstall; if the requested versions are different from what's currently installed, pip will handle the reinstallation. And calling pip multiple times with --force-reinstall meant that we were uninstalling and reinstalling the same packages over and over. Now we only call `pip install` once for requirements, and we do not force reinstall. On my machine, without enabling pybindings, a no-op run of this script (i.e. when all packages are up-to-date) takes 7 seconds, including the `pip install .`. Test Plan: ``` ./install_requirements.sh ``` Reviewers: Subscribers: Tasks: Tags:
1 parent 2449326 commit 13850d6

File tree

1 file changed

+47
-20
lines changed

1 file changed

+47
-20
lines changed

install_requirements.sh

Lines changed: 47 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -41,35 +41,62 @@ for arg in "$@"; do
4141
esac
4242
done
4343

44-
# Install pytorch dependencies
4544
#
46-
# Note:
47-
# When getting a new version of the executorch repo (via clone, fetch, or pull),
48-
# you may need to re-install a new version for all pytorch dependencies to run the
49-
# models in executorch/examples/models.
50-
# The version in this file will be the correct version for the
51-
# corresponsing version of the repo.
45+
# Install pip packages used by code in the ExecuTorch repo.
46+
#
47+
48+
# Since ExecuTorch often uses main-branch features of pytorch, only the nightly
49+
# pip versions will have the required features. The NIGHTLY_VERSION value should
50+
# agree with the third-party/pytorch pinned submodule commit.
51+
#
52+
# NOTE: If a newly-fetched version of the executorch repo changes the value of
53+
# NIGHTLY_VERSION, you should re-run this script to install the necessary
54+
# package versions.
5255
NIGHTLY_VERSION=dev20240324
5356

54-
TORCH_VERSION=2.4.0.${NIGHTLY_VERSION}
55-
pip install --force-reinstall --pre torch=="${TORCH_VERSION}" -i https://download.pytorch.org/whl/nightly/cpu
57+
# The pip repository that hosts nightly torch packages.
58+
TORCH_NIGHTLY_URL="https://download.pytorch.org/whl/nightly/cpu"
5659

57-
TORCH_VISION_VERSION=0.19.0.${NIGHTLY_VERSION}
58-
pip install --force-reinstall --pre torchvision=="${TORCH_VISION_VERSION}" -i https://download.pytorch.org/whl/nightly/cpu
60+
# pip packages needed by exir.
61+
EXIR_REQUIREMENTS=(
62+
torch==2.4.0.${NIGHTLY_VERSION}
63+
torchaudio==2.2.0.${NIGHTLY_VERSION} # For testing.
64+
)
5965

60-
TORCH_AUDIO_VERSION=2.2.0.${NIGHTLY_VERSION}
61-
pip install --force-reinstall --pre torchaudio=="${TORCH_AUDIO_VERSION}" -i https://download.pytorch.org/whl/nightly/cpu
66+
# pip packages needed for development.
67+
DEVEL_REQUIREMENTS=(
68+
setuptools # For building the pip package.
69+
tomli # Imported by extract_sources.py when using python < 3.11.
70+
wheel # For building the pip package archive.
71+
zstd # Imported by resolve_buck.py.
72+
)
6273

63-
TIMM_VERSION=0.6.13
64-
pip install --pre timm==${TIMM_VERSION}
74+
# pip packages needed to run examples.
75+
# TODO(dbort): Make each example publish its own requirements.txt
76+
EXAMPLES_REQUIREMENTS=(
77+
timm==0.6.13
78+
torchaudio==2.2.0.${NIGHTLY_VERSION}
79+
torchsr==1.0.4
80+
transformers==4.38.2
81+
)
6582

66-
TRANSFORMERS_VERSION=4.38.2
67-
pip install --force-reinstall --pre transformers==${TRANSFORMERS_VERSION}
83+
# Assemble the list of requirements to actually install.
84+
# TODO(dbort): Add options for reducing the number of requirements.
85+
REQUIREMENTS_TO_INSTALL=(
86+
"${EXIR_REQUIREMENTS[@]}"
87+
"${DEVEL_REQUIREMENTS[@]}"
88+
"${EXAMPLES_REQUIREMENTS[@]}"
89+
)
6890

69-
TORCHSR_VERSION=1.0.4
70-
pip install --pre torchsr==${TORCHSR_VERSION}
91+
# Install the requirements. `--extra-index-url` tells pip to look for package
92+
# versions on the provided URL if they aren't available on the default URL.
93+
pip install --extra-index-url "${TORCH_NIGHTLY_URL}" \
94+
"${REQUIREMENTS_TO_INSTALL[@]}"
95+
96+
#
97+
# Install executorch pip package.
98+
#
7199

72-
# Install ExecuTorch after dependencies are installed.
73100
EXECUTORCH_BUILD_PYBIND="$EXECUTORCH_BUILD_PYBIND" \
74101
CMAKE_ARGS="$CMAKE_ARGS" \
75102
CMAKE_BUILD_PARALLEL_LEVEL=9 \

0 commit comments

Comments
 (0)