Skip to content

incremental delivery: deduplicates all previously sent values #3842

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 9 commits into from

Conversation

yaacovCR
Copy link
Contributor

@yaacovCR yaacovCR commented Feb 5, 2023

This implementation is meant as a straw man, demonstrating that total deduplication is possible.

@IvanGoncharov @robrichard et al

Strategy

  1. A path of undefined has been replaced by a path of type Root so that we can use a WeakMap on the empty path.
  2. promiseForObject now mutates the original object so that we can stably associate every value in a response to the execution path. This function probably be renamed.
  3. The initial result tree is traversed, marking which values have been sent.
  4. Immediately prior to inclusion of any incremental payload, we iterate through the payload's result tree to exclude/mask:
    • any leaf that has already been sent
    • any value that does not include any deeper unsent leaves

Notes

Open Questions:

  1. Is it possible to associate a large bidirectional map of potential assignments of future leaf values and their paths with each asynchronous payload, such that as each incremental payload is built from the leaves, we could remove the potential assignments from all other payloads, resulting in payload tree creation after de-deduplication.
  2. How would error handling work in this approach?
  3. Would it be more performant?

@netlify
Copy link

netlify bot commented Feb 5, 2023

Deploy Preview for compassionate-pike-271cb3 ready!

Name Link
🔨 Latest commit 4891fe0
🔍 Latest deploy log https://app.netlify.com/sites/compassionate-pike-271cb3/deploys/63e15dd81bb4ea0008dd7262
😎 Deploy Preview https://deploy-preview-3842--compassionate-pike-271cb3.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site settings.

@yaacovCR yaacovCR changed the title incremental delivery: deduplicates all fields incremental delivery: deduplicates all previously sent values Feb 5, 2023
@github-actions
Copy link

github-actions bot commented Feb 5, 2023

Hi @yaacovCR, I'm @github-actions bot happy to help you with this PR 👋

Supported commands

Please post this commands in separate comments and only one per comment:

  • @github-actions run-benchmark - Run benchmark comparing base and merge commits for this PR
  • @github-actions publish-pr-on-npm - Build package from this PR and publish it on NPM

@yaacovCR
Copy link
Contributor Author

yaacovCR commented Feb 5, 2023

Also note that the current implementation relies heavily on WeakMaps that may be difficult to translate to other languages.

@yaacovCR yaacovCR force-pushed the mask-right branch 3 times, most recently from 020f6f9 to e516e33 Compare February 6, 2023 10:19
@yaacovCR
Copy link
Contributor Author

closed in favor of #3844

@yaacovCR yaacovCR closed this Feb 13, 2023
@yaacovCR yaacovCR deleted the mask-right branch September 5, 2024 11:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant