Serialize constant Data outside of flatbuffer #1542
Closed
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/1542
Note: Links to docs will display an error until the docs builds have been completed. ✅ You can merge normally! (2 Unrelated Failures)As of commit 16f088e with merge base 428da4f ( FLAKY - The following jobs failed but were likely due to flakiness present on trunk:
This comment was automatically generated by Dr. CI and updates every 15 minutes. |
facebook-github-bot
added
the
CLA Signed
|
This pull request was exported from Phabricator. Differential Revision: D52498367 |
mcr229
added a commit
to mcr229/executorch
that referenced
this pull request
Jan 10, 2024
Summary: We introduce the `serialize_xnnpack_binary` method which serializees the constant data outside of the flatbuffer. It leverages the xnnheader introduced in the previous diff to store offsets and sizes for both the flatbuffer payload as well as the constant data payload. Note here we have not yet switched the delegate to use the new `serialize_xnnpack_binary` function as this new serialization also requires changes on the runtime side. This will be tested in the diff which follows. Differential Revision: D52498367
mcr229
added a commit
to mcr229/executorch
that referenced
this pull request
Jan 10, 2024
Summary: We introduce the `serialize_xnnpack_binary` method which serializees the constant data outside of the flatbuffer. It leverages the xnnheader introduced in the previous diff to store offsets and sizes for both the flatbuffer payload as well as the constant data payload. Note here we have not yet switched the delegate to use the new `serialize_xnnpack_binary` function as this new serialization also requires changes on the runtime side. This will be tested in the diff which follows. Differential Revision: D52498367
mcr229
added a commit
to mcr229/executorch
that referenced
this pull request
Jan 10, 2024
Summary: We introduce the `serialize_xnnpack_binary` method which serializees the constant data outside of the flatbuffer. It leverages the xnnheader introduced in the previous diff to store offsets and sizes for both the flatbuffer payload as well as the constant data payload. Note here we have not yet switched the delegate to use the new `serialize_xnnpack_binary` function as this new serialization also requires changes on the runtime side. This will be tested in the diff which follows. Differential Revision: D52498367
mcr229
added a commit
to mcr229/executorch
that referenced
this pull request
Jan 10, 2024
Summary: We introduce the `serialize_xnnpack_binary` method which serializees the constant data outside of the flatbuffer. It leverages the xnnheader introduced in the previous diff to store offsets and sizes for both the flatbuffer payload as well as the constant data payload. Note here we have not yet switched the delegate to use the new `serialize_xnnpack_binary` function as this new serialization also requires changes on the runtime side. This will be tested in the diff which follows. Differential Revision: D52498367
mcr229
added a commit
to mcr229/executorch
that referenced
this pull request
Jan 10, 2024
Summary: We introduce the `serialize_xnnpack_binary` method which serializees the constant data outside of the flatbuffer. It leverages the xnnheader introduced in the previous diff to store offsets and sizes for both the flatbuffer payload as well as the constant data payload. Note here we have not yet switched the delegate to use the new `serialize_xnnpack_binary` function as this new serialization also requires changes on the runtime side. This will be tested in the diff which follows. Differential Revision: D52498367
mcr229
added a commit
to mcr229/executorch
that referenced
this pull request
Jan 10, 2024
Summary: We introduce the `serialize_xnnpack_binary` method which serializees the constant data outside of the flatbuffer. It leverages the xnnheader introduced in the previous diff to store offsets and sizes for both the flatbuffer payload as well as the constant data payload. Note here we have not yet switched the delegate to use the new `serialize_xnnpack_binary` function as this new serialization also requires changes on the runtime side. This will be tested in the diff which follows. Differential Revision: D52498367
mcr229
force-pushed
the
export-D52498367
branch
from
55f9975 to
177324d
Compare
|
This pull request was exported from Phabricator. Differential Revision: D52498367 |
mcr229
force-pushed
the
export-D52498367
branch
from
177324d to
3009dfb
Compare
mcr229
added a commit
to mcr229/executorch
that referenced
this pull request
Jan 11, 2024
Summary: We introduce the `serialize_xnnpack_binary` method which serializees the constant data outside of the flatbuffer. It leverages the xnnheader introduced in the previous diff to store offsets and sizes for both the flatbuffer payload as well as the constant data payload. Note here we have not yet switched the delegate to use the new `serialize_xnnpack_binary` function as this new serialization also requires changes on the runtime side. This will be tested in the diff which follows. Differential Revision: D52498367
|
This pull request was exported from Phabricator. Differential Revision: D52498367 |
mcr229
added a commit
to mcr229/executorch
that referenced
this pull request
Jan 11, 2024
Summary: We introduce the `serialize_xnnpack_binary` method which serializees the constant data outside of the flatbuffer. It leverages the xnnheader introduced in the previous diff to store offsets and sizes for both the flatbuffer payload as well as the constant data payload. Note here we have not yet switched the delegate to use the new `serialize_xnnpack_binary` function as this new serialization also requires changes on the runtime side. This will be tested in the diff which follows. Differential Revision: D52498367
…ytorch#1523) Summary: Introducing the XNNPACKHeader to manage the flatbuffer data and constant data. Previously, we have serialized constant data along with flatbuffer. However, with large weights and large tensors in general, this takes a large amount of time and memory converting our dataclass --> json --> flatbuffer. This has become a blocker on some larger models To fix, we circumvent serializing constant tensors via flatbuffer, by appending the constant data after the flatbuffer payload. In order to do this, we need an XNNPACKHeader which will give us the flatbuffer offset, flatbuffer size, constant data offset, and constant data sizes. It will look something like this: ``` ┌───────────────────────────────────┐ │XNNPACK Header │ ├───────────────────────────────────┤ │Padding for 16 byte alignment │ ├───────────────────────────────────┤ │Flatbuffer-serialized payload data │ │ │ │ │ ├───────────────────────────────────┤ │Padding for 16 byte alignment │ ├───────────────────────────────────┤ │Constant Data │ │ │ │ │ └───────────────────────────────────┘ ``` Within the XNNPACK Header, we hold the following: - 4 bytes to offset the header magic - 4 bytes for the header magic - 2 bytes for the header length - 4 bytes for the flatbuffer offset - 4 bytes for the flatbuffer size - 4 bytes for constant data offset - 8 bytes for constant data size Reviewed By: digantdesai Differential Revision: D52497977
Summary: We introduce the `serialize_xnnpack_binary` method which serializees the constant data outside of the flatbuffer. It leverages the xnnheader introduced in the previous diff to store offsets and sizes for both the flatbuffer payload as well as the constant data payload. Note here we have not yet switched the delegate to use the new `serialize_xnnpack_binary` function as this new serialization also requires changes on the runtime side. This will be tested in the diff which follows. Differential Revision: D52498367
mcr229
force-pushed
the
export-D52498367
branch
from
3009dfb to
16f088e
Compare
|
This pull request was exported from Phabricator. Differential Revision: D52498367 |
|
This pull request has been merged in 99d0ea1. |
SS-JIA
added a commit
that referenced
this pull request
Feb 20, 2024
## Context This changeset is a mirror of #1542 and #1543 for the Vulkan delegate, which enables serialization of constant weight data outside the flatbuffer blob. I decided to package some miscellaneous codebase improvements with this changeset: * Update field names of `VkGraph` table in `schema.fbs`: * In `VulkanBackend.cpp`, use aliases for generated flatbuffer types * Move build targets to a `targets.bzl` file with a `define_common_targets()` method Differential Revision: [D53957852](https://our.internmc.facebook.com/intern/diff/D53957852/) [ghstack-poisoned]
SS-JIA
added a commit
that referenced
this pull request
Feb 20, 2024
## Context This changeset is a mirror of #1542 and #1543 for the Vulkan delegate, which enables serialization of constant weight data outside the flatbuffer blob. I decided to package some miscellaneous codebase improvements with this changeset: * Update field names of `VkGraph` table in `schema.fbs`: * In `VulkanBackend.cpp`, use aliases for generated flatbuffer types * Move build targets to a `targets.bzl` file with a `define_common_targets()` method Differential Revision: [D53957852](https://our.internmc.facebook.com/intern/diff/D53957852/) ghstack-source-id: 215772816 Pull Request resolved: #2014
SS-JIA
added a commit
that referenced
this pull request
Feb 20, 2024
## Context This changeset is a mirror of #1542 and #1543 for the Vulkan delegate, which enables serialization of constant weight data outside the flatbuffer blob. I decided to package some miscellaneous codebase improvements with this changeset: * Update field names of `VkGraph` table in `schema.fbs` and update the namespace to be simply `vkgraph`: * In `VulkanBackend.cpp`, use aliases for generated flatbuffer types * Move build targets to a `targets.bzl` file with a `define_common_targets()` method Differential Revision: [D53957852](https://our.internmc.facebook.com/intern/diff/D53957852/) [ghstack-poisoned]
SS-JIA
added a commit
that referenced
this pull request
Feb 20, 2024
## Context This changeset is a mirror of #1542 and #1543 for the Vulkan delegate, which enables serialization of constant weight data outside the flatbuffer blob. I decided to package some miscellaneous codebase improvements with this changeset: * Update field names of `VkGraph` table in `schema.fbs` and update the namespace to be simply `vkgraph`: * In `VulkanBackend.cpp`, use aliases for generated flatbuffer types * Move build targets to a `targets.bzl` file with a `define_common_targets()` method Differential Revision: [D53957852](https://our.internmc.facebook.com/intern/diff/D53957852/) ghstack-source-id: 215800831 Pull Request resolved: #2016
SS-JIA
added a commit
that referenced
this pull request
Feb 20, 2024
## Context This changeset is a mirror of #1542 and #1543 for the Vulkan delegate, which enables serialization of constant weight data outside the flatbuffer blob. I decided to package some miscellaneous codebase improvements with this changeset: * Update field names of `VkGraph` table in `schema.fbs` and update the namespace to be simply `vkgraph`: * In `VulkanBackend.cpp`, use aliases for generated flatbuffer types * Move build targets to a `targets.bzl` file with a `define_common_targets()` method Differential Revision: [D53957852](https://our.internmc.facebook.com/intern/diff/D53957852/) [ghstack-poisoned]
SS-JIA
added a commit
that referenced
this pull request
Feb 20, 2024
Pull Request resolved: #2016 ## Context This changeset is a mirror of #1542 and #1543 for the Vulkan delegate, which enables serialization of constant weight data outside the flatbuffer blob. I decided to package some miscellaneous codebase improvements with this changeset: * Update field names of `VkGraph` table in `schema.fbs` and update the namespace to be simply `vkgraph`: * In `VulkanBackend.cpp`, use aliases for generated flatbuffer types * Move build targets to a `targets.bzl` file with a `define_common_targets()` method ghstack-source-id: 215803961 @exported-using-ghexport Differential Revision: [D53957852](https://our.internmc.facebook.com/intern/diff/D53957852/)
facebook-github-bot
pushed a commit
that referenced
this pull request
Feb 21, 2024
Summary: Pull Request resolved: #2016 ## Context This changeset is a mirror of #1542 and #1543 for the Vulkan delegate, which enables serialization of constant weight data outside the flatbuffer blob. I decided to package some miscellaneous codebase improvements with this changeset: * Update field names of `VkGraph` table in `schema.fbs` and update the namespace to be simply `vkgraph`: * In `VulkanBackend.cpp`, use aliases for generated flatbuffer types * Move build targets to a `targets.bzl` file with a `define_common_targets()` method ghstack-source-id: 215803961 exported-using-ghexport Reviewed By: mcr229 Differential Revision: D53957852 fbshipit-source-id: 6d6d9c70df6f8cbd9e85e8294208839be84527eb
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary:
We introduce the
serialize_xnnpack_binarymethod which serializees the constant data outside of the flatbuffer. It leverages the xnnheader introduced in the previous diff to store offsets and sizes for both the flatbuffer payload as well as the constant data payload.Note here we have not yet switched the delegate to use the new
serialize_xnnpack_binaryfunction as this new serialization also requires changes on the runtime side. This will be tested in the diff which follows.Differential Revision: D52498367