Skip to content

embed extended header inside .ptd flatbuffer section #7965

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Jan 28, 2025
Merged

embed extended header inside .ptd flatbuffer section #7965

merged 1 commit into from
Jan 28, 2025

Conversation

JacobSzwejbka
Copy link
Contributor

Summary: Embed the header inside the flatbuffer. We do this for .pte and it lets us reuse a lot of flatbuffer tools natively.

Differential Revision: D68578075

@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Jan 27, 2025 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/7965

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 New Failure, 1 Unrelated Failure

As of commit b73baeb with merge base cd51da4 (image):

NEW FAILURE - The following job has failed:

BROKEN TRUNK - The following job failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Jan 27, 2025
@facebook-github-bot
Copy link
Contributor

This pull request was exported from Phabricator. Differential Revision: D68578075

JacobSzwejbka added a commit to JacobSzwejbka/executorch-1 that referenced this pull request Jan 27, 2025
@JacobSzwejbka @facebook-github-bot
embed extended header inside .ptd flatbuffer section (pytorch#7965)
a052347 
Summary:

Embed the header inside the flatbuffer. We do this for .pte and it lets us reuse a lot of flatbuffer tools natively.

Differential Revision: D68578075
JacobSzwejbka added a commit to JacobSzwejbka/executorch-1 that referenced this pull request Jan 27, 2025
@JacobSzwejbka @facebook-github-bot
embed extended header inside .ptd flatbuffer section (pytorch#7965)
f2987eb 
Summary:

Embed the header inside the flatbuffer. We do this for .pte and it lets us reuse a lot of flatbuffer tools natively.

Differential Revision: D68578075
@facebook-github-bot
Copy link
Contributor

This pull request was exported from Phabricator. Differential Revision: D68578075

1 similar comment
@facebook-github-bot
Copy link
Contributor

This pull request was exported from Phabricator. Differential Revision: D68578075

lucylq
lucylq approved these changes Jan 27, 2025
View reviewed changes
Copy link
Contributor

@lucylq lucylq left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lgtm,

nit to change extended_header to flat_tensor_header for flat_tensor?

dbort
dbort approved these changes Jan 27, 2025
View reviewed changes
Copy link
Contributor

@dbort dbort left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good! Thanks for making this change.

A good double-check would be to try parsing a .ptd file using the flatc commandline tool, like

flatc --json --defaults-json --strict-json -o /tmp extension/flat_tensor/serialize/flat_tensor.fbs -- input.ptd

and see if it was able to create /tmp/input.json

@JacobSzwejbka @facebook-github-bot
embed extended header inside .ptd flatbuffer section (pytorch#7965)
b73baeb 
Summary:

Embed the header inside the flatbuffer. We do this for .pte and it lets us reuse a lot of flatbuffer tools natively.

Reviewed By: lucylq

Differential Revision: D68578075
@facebook-github-bot
Copy link
Contributor

This pull request was exported from Phabricator. Differential Revision: D68578075

@JacobSzwejbka
Copy link
Contributor Author

Ran the command Dave suggested on a local update to the train xor stack that makes it use .ptd and it worked (ill land that change later once the runtime can load .ptds

@facebook-github-bot facebook-github-bot merged commit 3540723 into pytorch:main Jan 28, 2025
44 of 47 checks passed
YIWENX14 pushed a commit that referenced this pull request Jan 28, 2025
@JacobSzwejbka @YIWENX14
embed extended header inside .ptd flatbuffer section
023e732 
Differential Revision: D68578075

Pull Request resolved: #7965
zonglinpeng pushed a commit to zonglinpeng/executorch that referenced this pull request Jan 30, 2025
@JacobSzwejbka
embed extended header inside .ptd flatbuffer section
5b13da2 
Differential Revision: D68578075

Pull Request resolved: pytorch#7965
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@dbort dbort dbort approved these changes

@lucylq lucylq lucylq approved these changes

Assignees
No one assigned
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. fb-exported topic: not user facing
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@JacobSzwejbka @facebook-github-bot @dbort @lucylq