Package headers into pip wheel #5734

larryliu0820 · 2024-09-27T21:19:39Z

Summary: Changes CustomBuildPy to take a list of source directories relative to ExecuTorch root and copies all headers recursively into pip wheel during packaging.

The destination is hardcoded to executorch/include/executorch in the pip wheel.

Test Plan:

python setup.py bdist_wheel

This prints out the following lines:

creating pip-out/bdist.linux-x86_64/wheel/executorch/include/executorch/runtime
creating pip-out/bdist.linux-x86_64/wheel/executorch/include/executorch/runtime/core
copying pip-out/lib.linux-x86_64-cpython-311/executorch/include/executorch/runtime/core/array_ref.h -> pip-out/bdist.linux-x86_64/wheel/executorch/include/executorch/runtime/core
...

Then install the wheel:

pip install dist/executorch-0.5.0a0+52d5218-cp311-cp311-linux_x86_64.whl

And we can find the headers in site-packages

/data/users/larryliu/executorch/pip-out/lib.linux-x86_64-cpython-311/executorch/include/executorch/runtime/core/array_ref.h

Reviewers:

Subscribers:

Tasks:

Tags:

Summary: Add a new BaseExtension - HeaderFile. It takes a source directory relative to ExecuTorch root and copies all headers recursively into pip wheel during packaging. The destination is hardcoded to `executorch/include/executorch` in the pip wheel. Test Plan: ```bash python setup.py bdist_wheel ``` This prints out the following lines: ``` creating pip-out/bdist.linux-x86_64/wheel/executorch/include/executorch/runtime creating pip-out/bdist.linux-x86_64/wheel/executorch/include/executorch/runtime/core copying pip-out/lib.linux-x86_64-cpython-311/executorch/include/executorch/runtime/core/array_ref.h -> pip-out/bdist.linux-x86_64/wheel/executorch/include/executorch/runtime/core ... ``` Then install the wheel: ```bash pip install dist/executorch-0.5.0a0+52d5218-cp311-cp311-linux_x86_64.whl ``` And we can find the headers in site-packages ``` /data/users/larryliu/executorch/pip-out/lib.linux-x86_64-cpython-311/executorch/include/executorch/runtime/core/array_ref.h ``` Reviewers: Subscribers: Tasks: Tags:

pytorch-bot · 2024-09-27T21:19:42Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/5734

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit 32f63ed with merge base 52d5218 ():
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

facebook-github-bot · 2024-09-27T21:23:08Z

@larryliu0820 has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

setup.py

dbort

For the test plan, please also make sure that you can import and use pybindings, and that flatc is on your path and still works.

setup.py

Summary: Test Plan: Reviewers: Subscribers: Tasks: Tags:

facebook-github-bot · 2024-09-30T20:03:59Z

@larryliu0820 has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

dbort

I should have picked up on this earlier. Things are awkward because this is breaking a core aspect of the "build extension" concept.

setup.py

Summary: Test Plan: Reviewers: Subscribers: Tasks: Tags:

facebook-github-bot · 2024-10-01T18:50:25Z

@larryliu0820 has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

dbort

Way simpler, thanks! :D Please update the PR description to describe the final form of the change, but it looks great.

facebook-github-bot · 2024-10-01T20:02:54Z

@larryliu0820 merged this pull request in d62c7ad.

larryliu0820 · 2024-10-10T21:53:16Z

@pytorchbot cherry-pick --onto release/0.4 -c fixnewfeature

Summary: Changes `CustomBuildPy` to take a list of source directories relative to ExecuTorch root and copies all headers recursively into pip wheel during packaging. The destination is hardcoded to `executorch/include/executorch` in the pip wheel. Pull Request resolved: #5734 Test Plan: ```bash python setup.py bdist_wheel ``` This prints out the following lines: ``` creating pip-out/bdist.linux-x86_64/wheel/executorch/include/executorch/runtime creating pip-out/bdist.linux-x86_64/wheel/executorch/include/executorch/runtime/core copying pip-out/lib.linux-x86_64-cpython-311/executorch/include/executorch/runtime/core/array_ref.h -> pip-out/bdist.linux-x86_64/wheel/executorch/include/executorch/runtime/core ... ``` Then install the wheel: ```bash pip install dist/executorch-0.5.0a0+52d5218-cp311-cp311-linux_x86_64.whl ``` And we can find the headers in site-packages ``` /data/users/larryliu/executorch/pip-out/lib.linux-x86_64-cpython-311/executorch/include/executorch/runtime/core/array_ref.h ``` Reviewed By: dbort Differential Revision: D63561966 Pulled By: larryliu0820 fbshipit-source-id: 0c7d8983a89d11e62da006d361ca00b4a061d73c (cherry picked from commit d62c7ad)

pytorchbot · 2024-10-10T21:54:54Z

Cherry picking #5734

The cherry pick PR is at #6138 and it is recommended to link a fixnewfeature cherry pick PR with an issue. The following tracker issues are updated:

[v0.4.0] Release Tracker #5366 (comment)

Details for Dev Infra team

Raised by workflow job

Summary: Changes `CustomBuildPy` to take a list of source directories relative to ExecuTorch root and copies all headers recursively into pip wheel during packaging. The destination is hardcoded to `executorch/include/executorch` in the pip wheel. Pull Request resolved: #5734 Test Plan: ```bash python setup.py bdist_wheel ``` This prints out the following lines: ``` creating pip-out/bdist.linux-x86_64/wheel/executorch/include/executorch/runtime creating pip-out/bdist.linux-x86_64/wheel/executorch/include/executorch/runtime/core copying pip-out/lib.linux-x86_64-cpython-311/executorch/include/executorch/runtime/core/array_ref.h -> pip-out/bdist.linux-x86_64/wheel/executorch/include/executorch/runtime/core ... ``` Then install the wheel: ```bash pip install dist/executorch-0.5.0a0+52d5218-cp311-cp311-linux_x86_64.whl ``` And we can find the headers in site-packages ``` /data/users/larryliu/executorch/pip-out/lib.linux-x86_64-cpython-311/executorch/include/executorch/runtime/core/array_ref.h ``` Reviewed By: dbort Differential Revision: D63561966 Pulled By: larryliu0820 fbshipit-source-id: 0c7d8983a89d11e62da006d361ca00b4a061d73c (cherry picked from commit d62c7ad)

Package headers into pip wheel (#5734) Summary: Changes `CustomBuildPy` to take a list of source directories relative to ExecuTorch root and copies all headers recursively into pip wheel during packaging. The destination is hardcoded to `executorch/include/executorch` in the pip wheel. Pull Request resolved: #5734 Test Plan: ```bash python setup.py bdist_wheel ``` This prints out the following lines: ``` creating pip-out/bdist.linux-x86_64/wheel/executorch/include/executorch/runtime creating pip-out/bdist.linux-x86_64/wheel/executorch/include/executorch/runtime/core copying pip-out/lib.linux-x86_64-cpython-311/executorch/include/executorch/runtime/core/array_ref.h -> pip-out/bdist.linux-x86_64/wheel/executorch/include/executorch/runtime/core ... ``` Then install the wheel: ```bash pip install dist/executorch-0.5.0a0+52d5218-cp311-cp311-linux_x86_64.whl ``` And we can find the headers in site-packages ``` /data/users/larryliu/executorch/pip-out/lib.linux-x86_64-cpython-311/executorch/include/executorch/runtime/core/array_ref.h ``` Reviewed By: dbort Differential Revision: D63561966 Pulled By: larryliu0820 fbshipit-source-id: 0c7d8983a89d11e62da006d361ca00b4a061d73c (cherry picked from commit d62c7ad) Co-authored-by: Mengwei Liu <[email protected]>

larryliu0820 requested a review from dbort September 27, 2024 21:19

facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Sep 27, 2024

metascroy reviewed Sep 27, 2024

View reviewed changes

setup.py Outdated Show resolved Hide resolved

metascroy reviewed Sep 27, 2024

View reviewed changes

setup.py Outdated Show resolved Hide resolved

dbort requested changes Sep 30, 2024

View reviewed changes

Address comments

e4e2995

Summary: Test Plan: Reviewers: Subscribers: Tasks: Tags:

dbort requested changes Oct 1, 2024

View reviewed changes

setup.py Outdated Show resolved Hide resolved

Address comments

32f63ed

Summary: Test Plan: Reviewers: Subscribers: Tasks: Tags:

dbort approved these changes Oct 1, 2024

View reviewed changes

facebook-github-bot closed this in d62c7ad Oct 1, 2024

facebook-github-bot added the Merged label Oct 1, 2024

pytorchbot mentioned this pull request Oct 10, 2024

[v0.4.0] Release Tracker #5366

Closed

Package headers into pip wheel #5734

Package headers into pip wheel #5734

Uh oh!

Conversation

larryliu0820 commented Sep 27, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pytorch-bot bot commented Sep 27, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/5734

✅ No Failures

Uh oh!

facebook-github-bot commented Sep 27, 2024

Uh oh!

Uh oh!

Uh oh!

dbort left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

facebook-github-bot commented Sep 30, 2024

Uh oh!

dbort left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

facebook-github-bot commented Oct 1, 2024

Uh oh!

dbort left a comment

Choose a reason for hiding this comment

Uh oh!

facebook-github-bot commented Oct 1, 2024

Uh oh!

larryliu0820 commented Oct 10, 2024

Uh oh!

pytorchbot commented Oct 10, 2024

Cherry picking #5734

Uh oh!

Uh oh!

larryliu0820 commented Sep 27, 2024 •

edited

Loading

pytorch-bot bot commented Sep 27, 2024 •

edited

Loading