feat(replay): Save replay in sessionStorage and attempt to re-request if necessary #6698

billyvg · 2023-01-10T00:50:22Z

Save replay request parameters to sessionStorage before we await any async functions when flushing. This is cleared after each successful request. If integration is reloaded and there is data in sessionStorage, we will attempt a new request to send saved data.

This happens because when a user refreshes, we have some async tasks (e.g. compression worker) which will not resolve before the page is unloaded. However, we can retrieve the raw events from the worker and save to sessionStorage as those are all synchronous tasks.

Should help address #6533

Requires #6696, #6695, #6699

github-actions · 2023-01-10T00:59:52Z

size-limit report 📦

Path	Size
@sentry/browser - ES5 CDN Bundle (gzipped + minified)	19.82 KB (+0.01% 🔺)
@sentry/browser - ES5 CDN Bundle (minified)	61.47 KB (0%)
@sentry/browser - ES6 CDN Bundle (gzipped + minified)	18.5 KB (+0.01% 🔺)
@sentry/browser - ES6 CDN Bundle (minified)	54.77 KB (0%)
@sentry/browser - Webpack (gzipped + minified)	20.22 KB (0%)
@sentry/browser - Webpack (minified)	66.17 KB (0%)
@sentry/react - Webpack (gzipped + minified)	20.24 KB (0%)
@sentry/nextjs Client - Webpack (gzipped + minified)	47.51 KB (0%)
@sentry/browser + @sentry/tracing - ES5 CDN Bundle (gzipped + minified)	26.73 KB (-0.02% 🔽)
@sentry/browser + @sentry/tracing - ES6 CDN Bundle (gzipped + minified)	25.02 KB (+0.01% 🔺)
@sentry/replay ES6 CDN Bundle (gzipped + minified)	43.4 KB (-0.17% 🔽)
@sentry/replay - Webpack (gzipped + minified)	38.65 KB (-0.48% 🔽)
@sentry/browser + @sentry/tracing + @sentry/replay - ES6 CDN Bundle (gzipped + minified)	60.55 KB (-0.27% 🔽)

billyvg · 2023-01-10T00:59:25Z

packages/replay/src/coreHandlers/handleGlobalEvent.ts

-      // Replays have separate set of breadcrumbs, do not include breadcrumbs
-      // from core SDK
-      delete event.breadcrumbs;


Moved this to the SDK where we send the replay request (right after we prepare event), otherwise the restoration request happens before this handler is attached.

billyvg · 2023-01-10T01:01:59Z

packages/replay/src/replay.ts

+      const pendingEvent = await getPendingReplay({ useCompression });
+      if (pendingEvent) {
+        await this.sendReplayRequest(pendingEvent);


This happens before we attach any listeners at all and will block until we make this request. Note this calls the function that does not retry, but maybe it should?

billyvg · 2023-01-10T01:10:38Z

packages/replay/src/util/setFlushState.ts

+  } else if (state === FlushState.COMPLETE) {
+    WINDOW.sessionStorage.removeItem(PENDING_REPLAY_STATUS_KEY);
+  } else {
+    WINDOW.sessionStorage.setItem(PENDING_REPLAY_STATUS_KEY, state);


After reading this... this probably isn't necessary. It would let us know at which stage the user dropped off.

Lms24

Just talked with @mydea about this. I'm a little bit concerned that we might not always clear the pending replays from session storage. Which is proabaly generally a problem with everything we try to keep in session storage. We talked a little bit about using a SharedWorker instead which would live across tabs but stay alive on unloads. So we'd probably need to have a map of session ids and eventbuffers but it might be worth a try. WDYT?

Regardless, I went over the PR and code-wise I think it looks good. I'd just want to avoid the enum if possible (see comment).

Lms24 · 2023-01-18T12:08:33Z

packages/replay/src/types.ts

@@ -18,6 +18,17 @@ export interface SendReplayData {
  options: ReplayPluginOptions;
 }

+export enum FlushState {


We should generally avoid enums as they have a pretty hefty bundle size impact (more details)

We could use string constants instead, something along these lines:

export type FlushState = 'start' | 'sent-request' | 'complete' | 'error'

WDYT?

mydea · 2023-01-18T15:36:42Z

After thinking about this a bit, and trying a few things, I actually think a nicer solution for this problem would be to avoid compression when we need to send events ASAP (e.g. on unload). I'll try to write up a proof of concept to illustrate what I mean.

billyvg · 2023-01-18T15:41:23Z

@mydea yeah I had that idea in mind as well - we would avoid keeping state in session storage and segments should be small enough that it's fine to have them uncompressed.

mydea · 2023-01-18T15:43:12Z

This PR illustrates how that could work - actually it's quite straightforward, "only" thing missing is to make sure to use the finishImmediate stuff when appropriate in replay itself. WDYT?

billyvg · 2023-01-30T14:58:14Z

I'm going to close this out for now as we have improved segment quality a significant amount already. This PR will add a bit of complexity and overhead, so let's hold off unless we really deem it necessary.

billyvg changed the base branch from master to feat-replay-change-addevent-to-be-async January 10, 2023 00:50

billyvg changed the base branch from feat-replay-change-addevent-to-be-async to feat-replay-eventbuffer-save-pending-events January 10, 2023 00:53

billyvg changed the title ~~feat replay retry lost events session storage~~ feat(replay): Save replay in sessionStorage and attempt to re-request if necessary Jan 10, 2023

billyvg force-pushed the feat-replay-retry-lost-events-session-storage branch from d95e731 to aeb40b9 Compare January 10, 2023 01:04

billyvg commented Jan 10, 2023

View reviewed changes

billyvg force-pushed the feat-replay-retry-lost-events-session-storage branch 2 times, most recently from a0059a4 to 9624e4e Compare January 10, 2023 01:29

billyvg requested review from mydea and Lms24 January 10, 2023 04:08

billyvg force-pushed the feat-replay-eventbuffer-save-pending-events branch from c58b7c0 to efa539c Compare January 11, 2023 01:52

mydea mentioned this pull request Jan 11, 2023

feat(replay): Track pending events in EventBuffer #6699

Merged

billyvg force-pushed the feat-replay-eventbuffer-save-pending-events branch 2 times, most recently from be829e1 to 2756060 Compare January 11, 2023 18:18

Base automatically changed from feat-replay-eventbuffer-save-pending-events to master January 11, 2023 21:17

billyvg force-pushed the feat-replay-retry-lost-events-session-storage branch from 9624e4e to d6a5f05 Compare January 11, 2023 22:48

billyvg marked this pull request as ready for review January 11, 2023 22:50

billyvg added 3 commits January 17, 2023 21:33

update worker

f8bc998

add comments

4f7537d

change promise order

7f3fa0b

billyvg force-pushed the feat-replay-retry-lost-events-session-storage branch from d6a5f05 to 7f3fa0b Compare January 18, 2023 03:04

Lms24 reviewed Jan 18, 2023

View reviewed changes

billyvg mentioned this pull request Jan 18, 2023

feat(replay): Add non-async flush for page unloads #6854

Closed

billyvg closed this Jan 30, 2023

billyvg deleted the feat-replay-retry-lost-events-session-storage branch January 30, 2023 14:58

billyvg mentioned this pull request Feb 2, 2023

[Replay] Improve Missing Segments #6533

Closed

2 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

feat(replay): Save replay in sessionStorage and attempt to re-request if necessary #6698

feat(replay): Save replay in sessionStorage and attempt to re-request if necessary #6698

Uh oh!

billyvg commented Jan 10, 2023 •

edited

Loading

Uh oh!

github-actions bot commented Jan 10, 2023 •

edited

Loading

Uh oh!

billyvg Jan 10, 2023

Uh oh!

billyvg Jan 10, 2023

Uh oh!

billyvg Jan 10, 2023 •

edited

Loading

Uh oh!

Lms24 left a comment

Uh oh!

Lms24 Jan 18, 2023

Uh oh!

mydea commented Jan 18, 2023

Uh oh!

billyvg commented Jan 18, 2023

Uh oh!

mydea commented Jan 18, 2023

Uh oh!

billyvg commented Jan 30, 2023

Uh oh!

Uh oh!

Uh oh!

feat(replay): Save replay in sessionStorage and attempt to re-request if necessary #6698

feat(replay): Save replay in sessionStorage and attempt to re-request if necessary #6698

Uh oh!

Conversation

billyvg commented Jan 10, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Jan 10, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

size-limit report 📦

Uh oh!

billyvg Jan 10, 2023

Choose a reason for hiding this comment

Uh oh!

billyvg Jan 10, 2023

Choose a reason for hiding this comment

Uh oh!

billyvg Jan 10, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Lms24 left a comment

Choose a reason for hiding this comment

Uh oh!

Lms24 Jan 18, 2023

Choose a reason for hiding this comment

Uh oh!

mydea commented Jan 18, 2023

Uh oh!

billyvg commented Jan 18, 2023

Uh oh!

mydea commented Jan 18, 2023

Uh oh!

billyvg commented Jan 30, 2023

Uh oh!

Uh oh!

billyvg commented Jan 10, 2023 •

edited

Loading

github-actions bot commented Jan 10, 2023 •

edited

Loading

billyvg Jan 10, 2023 •

edited

Loading