Skip to content

Commit 7c68ec6

Browse files
huydhnfacebook-github-bot
huydhn
authored and
facebook-github-bot
committed
Move MacOS jobs to trunk to reduce its load (#610)
Summary: As ET is using more MacOS runners causing a long queue there https://hud.pytorch.org/metrics, an immediate mitigation step would be to move MacOS jobs to trunk to reduce its load. I'll start working on T165554710 to consolidate MacOS jobs. Pull Request resolved: #610 Reviewed By: malfet Differential Revision: D49899056 Pulled By: huydhn fbshipit-source-id: fb7dfe32a27771463a1524a2435d90201d9eb807
1 parent 9e47a08 commit 7c68ec6

File tree

4 files changed

+124
-85
lines changed

4 files changed

+124
-85
lines changed

‎.github/workflows/pull.yml

Lines changed: 2 additions & 80 deletions
Original file line numberDiff line numberDiff line change
@@ -31,7 +31,7 @@ jobs:
3131
3232
source .ci/scripts/utils.sh
3333
# This is a simple Python script but as it tries to import executorch.examples.models,
34-
# it requires a whole bunch of Executorch dependencies on the Docker image
34+
# it requires a whole bunch of ExecuTorch dependencies on the Docker image
3535
install_pip_dependencies
3636
install_executorch
3737
@@ -62,37 +62,9 @@ jobs:
6262
DEMO_BACKEND_DELEGATION=${{ matrix.demo_backend_delegation }}
6363
6464
PYTHON_EXECUTABLE=python bash .ci/scripts/setup-linux.sh "${BUILD_TOOL}"
65-
# Build and test Executorch
65+
# Build and test ExecuTorch
6666
PYTHON_EXECUTABLE=python bash .ci/scripts/test.sh "${MODEL_NAME}" "${BUILD_TOOL}" "${XNNPACK_QUANTIZATION}" "${XNNPACK_DELEGATION}" "${DEMO_BACKEND_DELEGATION}"
6767
68-
test-models-macos:
69-
name: test-models-macos
70-
uses: pytorch/test-infra/.github/workflows/macos_job.yml@main
71-
needs: gather-models
72-
strategy:
73-
matrix: ${{ fromJSON(needs.gather-models.outputs.models) }}
74-
fail-fast: false
75-
with:
76-
runner: macos-m1-12
77-
submodules: 'true'
78-
ref: ${{ github.event_name == 'pull_request' && github.event.pull_request.head.sha || github.sha }}
79-
timeout: 90
80-
script: |
81-
WORKSPACE=$(pwd)
82-
pushd "${WORKSPACE}/pytorch/executorch"
83-
84-
MODEL_NAME=${{ matrix.model }}
85-
BUILD_TOOL=${{ matrix.build-tool }}
86-
XNNPACK_QUANTIZATION=${{ matrix.xnnpack_quantization }}
87-
XNNPACK_DELEGATION=${{ matrix.xnnpack_delegation }}
88-
DEMO_BACKEND_DELEGATION=${{ matrix.demo_backend_delegation }}
89-
90-
# Setup MacOS dependencies as there is no Docker support on MacOS atm
91-
PYTHON_EXECUTABLE=python bash .ci/scripts/setup-macos.sh "${BUILD_TOOL}"
92-
# Build and test Executorch
93-
PYTHON_EXECUTABLE=python bash .ci/scripts/test.sh "${MODEL_NAME}" "${BUILD_TOOL}" "${XNNPACK_QUANTIZATION}" "${XNNPACK_DELEGATION}" "${DEMO_BACKEND_DELEGATION}"
94-
popd
95-
9668
test-custom-ops-linux:
9769
name: test-custom-ops-linux
9870
uses: pytorch/test-infra/.github/workflows/linux_job.yml@main
@@ -117,31 +89,6 @@ jobs:
11789
# Test custom ops
11890
PYTHON_EXECUTABLE=python bash examples/custom_ops/test_custom_ops.sh "${BUILD_TOOL}"
11991
120-
test-custom-ops-macos:
121-
name: test-custom-ops-macos
122-
uses: pytorch/test-infra/.github/workflows/macos_job.yml@main
123-
strategy:
124-
matrix:
125-
include:
126-
- build-tool: buck2
127-
- build-tool: cmake
128-
fail-fast: false
129-
with:
130-
runner: macos-m1-12
131-
submodules: 'true'
132-
ref: ${{ github.event_name == 'pull_request' && github.event.pull_request.head.sha || github.sha }}
133-
script: |
134-
WORKSPACE=$(pwd)
135-
pushd "${WORKSPACE}/pytorch/executorch"
136-
137-
BUILD_TOOL=${{ matrix.build-tool }}
138-
# Setup MacOS dependencies as there is no Docker support on MacOS atm
139-
PYTHON_EXECUTABLE=python bash .ci/scripts/setup-macos.sh "${BUILD_TOOL}"
140-
141-
# Build and test custom ops
142-
PYTHON_EXECUTABLE=python bash examples/custom_ops/test_custom_ops.sh "${BUILD_TOOL}"
143-
popd
144-
14592
test-selective-build-linux:
14693
name: test-selective-build-linux
14794
uses: pytorch/test-infra/.github/workflows/linux_job.yml@main
@@ -166,31 +113,6 @@ jobs:
166113
# Test selective build
167114
PYTHON_EXECUTABLE=python bash examples/selective_build/test_selective_build.sh "${BUILD_TOOL}"
168115
169-
test-selective-build-macos:
170-
name: test-selective-build-macos
171-
uses: pytorch/test-infra/.github/workflows/macos_job.yml@main
172-
strategy:
173-
matrix:
174-
include:
175-
- build-tool: buck2
176-
- build-tool: cmake
177-
fail-fast: false
178-
with:
179-
runner: macos-m1-12
180-
submodules: 'true'
181-
ref: ${{ github.event_name == 'pull_request' && github.event.pull_request.head.sha || github.sha }}
182-
script: |
183-
WORKSPACE=$(pwd)
184-
pushd "${WORKSPACE}/pytorch/executorch"
185-
186-
BUILD_TOOL=${{ matrix.build-tool }}
187-
# Setup MacOS dependencies as there is no Docker support on MacOS atm
188-
PYTHON_EXECUTABLE=python bash .ci/scripts/setup-macos.sh "${BUILD_TOOL}"
189-
190-
# Build and test selective build
191-
PYTHON_EXECUTABLE=python bash examples/selective_build/test_selective_build.sh "${BUILD_TOOL}"
192-
popd
193-
194116
unittest:
195117
uses: ./.github/workflows/_unittest.yml
196118
with:

‎.github/workflows/trunk.yml

Lines changed: 115 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,115 @@
1+
name: trunk
2+
3+
on:
4+
push:
5+
branches:
6+
- main
7+
workflow_dispatch:
8+
9+
concurrency:
10+
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.sha }}-${{ github.event_name == 'workflow_dispatch' }}-${{ github.event_name == 'schedule' }}
11+
cancel-in-progress: true
12+
13+
jobs:
14+
gather-models:
15+
runs-on: ubuntu-22.04
16+
outputs:
17+
models: ${{ steps.gather-models.outputs.models }}
18+
steps:
19+
- uses: actions/checkout@v3
20+
with:
21+
submodules: 'true'
22+
- uses: actions/setup-python@v4
23+
with:
24+
python-version: '3.10'
25+
cache: pip
26+
- name: Extract the list of models to test
27+
id: gather-models
28+
run: |
29+
set -eux
30+
31+
source .ci/scripts/utils.sh
32+
# This is a simple Python script but as it tries to import executorch.examples.models,
33+
# it requires a whole bunch of ExecuTorch dependencies on the Docker image
34+
install_pip_dependencies
35+
install_executorch
36+
37+
PYTHONPATH="${PWD}" python .ci/scripts/gather_test_models.py
38+
39+
test-models-macos:
40+
name: test-models-macos
41+
uses: pytorch/test-infra/.github/workflows/macos_job.yml@main
42+
needs: gather-models
43+
strategy:
44+
matrix: ${{ fromJSON(needs.gather-models.outputs.models) }}
45+
fail-fast: false
46+
with:
47+
runner: macos-m1-12
48+
submodules: 'true'
49+
ref: ${{ github.event_name == 'pull_request' && github.event.pull_request.head.sha || github.sha }}
50+
timeout: 90
51+
script: |
52+
WORKSPACE=$(pwd)
53+
pushd "${WORKSPACE}/pytorch/executorch"
54+
55+
MODEL_NAME=${{ matrix.model }}
56+
BUILD_TOOL=${{ matrix.build-tool }}
57+
XNNPACK_QUANTIZATION=${{ matrix.xnnpack_quantization }}
58+
XNNPACK_DELEGATION=${{ matrix.xnnpack_delegation }}
59+
DEMO_BACKEND_DELEGATION=${{ matrix.demo_backend_delegation }}
60+
61+
# Setup MacOS dependencies as there is no Docker support on MacOS atm
62+
PYTHON_EXECUTABLE=python bash .ci/scripts/setup-macos.sh "${BUILD_TOOL}"
63+
# Build and test xecutorch
64+
PYTHON_EXECUTABLE=python bash .ci/scripts/test.sh "${MODEL_NAME}" "${BUILD_TOOL}" "${XNNPACK_QUANTIZATION}" "${XNNPACK_DELEGATION}" "${DEMO_BACKEND_DELEGATION}"
65+
popd
66+
67+
test-custom-ops-macos:
68+
name: test-custom-ops-macos
69+
uses: pytorch/test-infra/.github/workflows/macos_job.yml@main
70+
strategy:
71+
matrix:
72+
include:
73+
- build-tool: buck2
74+
- build-tool: cmake
75+
fail-fast: false
76+
with:
77+
runner: macos-m1-12
78+
submodules: 'true'
79+
ref: ${{ github.event_name == 'pull_request' && github.event.pull_request.head.sha || github.sha }}
80+
script: |
81+
WORKSPACE=$(pwd)
82+
pushd "${WORKSPACE}/pytorch/executorch"
83+
84+
BUILD_TOOL=${{ matrix.build-tool }}
85+
# Setup MacOS dependencies as there is no Docker support on MacOS atm
86+
PYTHON_EXECUTABLE=python bash .ci/scripts/setup-macos.sh "${BUILD_TOOL}"
87+
88+
# Build and test custom ops
89+
PYTHON_EXECUTABLE=python bash examples/custom_ops/test_custom_ops.sh "${BUILD_TOOL}"
90+
popd
91+
92+
test-selective-build-macos:
93+
name: test-selective-build-macos
94+
uses: pytorch/test-infra/.github/workflows/macos_job.yml@main
95+
strategy:
96+
matrix:
97+
include:
98+
- build-tool: buck2
99+
- build-tool: cmake
100+
fail-fast: false
101+
with:
102+
runner: macos-m1-12
103+
submodules: 'true'
104+
ref: ${{ github.event_name == 'pull_request' && github.event.pull_request.head.sha || github.sha }}
105+
script: |
106+
WORKSPACE=$(pwd)
107+
pushd "${WORKSPACE}/pytorch/executorch"
108+
109+
BUILD_TOOL=${{ matrix.build-tool }}
110+
# Setup MacOS dependencies as there is no Docker support on MacOS atm
111+
PYTHON_EXECUTABLE=python bash .ci/scripts/setup-macos.sh "${BUILD_TOOL}"
112+
113+
# Build and test selective build
114+
PYTHON_EXECUTABLE=python bash examples/selective_build/test_selective_build.sh "${BUILD_TOOL}"
115+
popd

‎.lintrunner.toml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -68,6 +68,8 @@ include_patterns = [
6868
exclude_patterns = [
6969
'third-party/**',
7070
'**/third-party/**',
71+
# NB: Objective-C is not supported
72+
'examples/apple/**',
7173
'examples/ios_demo_apps/**',
7274
]
7375
command = [

‎examples/apple/ios/README.md

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
1-
# iOS Demo App: Executorch Setup
1+
# iOS Demo App: ExecuTorch Setup
22

3-
This guide explains how to setup Executorch for iOS using a demo app. The app
4-
employs a MobileNet v3 model (exported to Executorch) to process live camera
3+
This guide explains how to setup ExecuTorch for iOS using a demo app. The app
4+
employs a MobileNet v3 model (exported to ExecuTorch) to process live camera
55
images.
66

77
## Pre-setup
@@ -27,7 +27,7 @@ images.
2727
ln -s /Applications/CMake.app/Contents/bin/cmake /usr/bin/cmake
2828
```
2929

30-
4. Clone Executorch repository and update submodules:
30+
4. Clone ExecuTorch repository and update submodules:
3131

3232
```bash
3333
git clone https://github.com/pytorch/executorch.git
@@ -60,7 +60,7 @@ cmake .. && cmake --build . --target flatc
6060
cd ../../..
6161
```
6262

63-
## Executorch Configuration
63+
## ExecuTorch Configuration
6464

6565
Configure the libraries for iOS:
6666

0 commit comments

Comments
 (0)