feat(replay): Allow to configure URLs to capture network bodies/headers #7953

mydea · 2023-04-25T11:26:21Z

This replaces the old experimental options for enhanced replay network capture with new, official configuration:

interface ReplayNetworkOptions {
  /**
   * Capture request/response details for XHR/Fetch requests that match the given URLs.
   * The URLs can be strings or regular expressions.
   * When provided a string, we will match any URL that contains the given string.
   * You can use a Regex to handle exact matches or more complex matching.
   *
   * Only URLs matching these patterns will have bodies & additional headers captured.
   */
  networkDetailAllowUrls: (string | RegExp)[];

  /**
   * If request & response bodies should be captured.
   * Only applies to URLs matched by `networkDetailAllowUrls`.
   * Defaults to true.
   */
  networkCaptureBodies: boolean;

  /**
   * Capture the following request headers, in addition to the default ones.
   * Only applies to URLs matched by `networkDetailAllowUrls`.
   * Any headers defined here will be captured in addition to the default headers.
   */
  networkRequestHeaders: string[];

  /**
   * Capture the following response headers, in addition to the default ones.
   * Only applies to URLs matched by `networkDetailAllowUrls`.
   * Any headers defined here will be captured in addition to the default headers.
   */
  networkResponseHeaders: string[];
}

Note: This brings a small change, in that we also just capture the default headers when the URL matches. So by default, no headers are captured for any request.

Example usage:

Capture bodies & default headers for given URL:

new Replay({
  // This will capture details for any path including this
  // So this will e.g. capture https://sentry.io/api/users
  networkDetailAllowUrls: ['https://sentry.io/api']
});

Capture bodies & default headers for given exact URLs:

new Replay({
  // By providing Regex, you can do exact matching
  networkDetailAllowUrls: [
    /^http:\/\/sentry.io\/api\/users$/, 
    /^http:\/\/sentry.io\/api\/projects$/, 
  ]
});

Capture only headers, including some custom ones:

new Replay({
  networkDetailAllowUrls: ['https://sentry.io/api'],
  networkCaptureBodies: false,
  networkRequestHeaders: ['X-Custom-Header'],
  networkResponseHeaders: ['X-Custom-Header', 'X-Custom-Header-2']
});

Warnings

For any request that does not match the given URL(s), we will add a warning to the request/response meta. For example:

{
  data: {
    method: 'POST',
    statusCode: 200,
    request: {
      headers: {},
      _meta: {
        warnings: ['URL_SKIPPED'],
      },
    },
    response: {
      headers: {},
      _meta: {
        warnings: ['URL_SKIPPED'],
      },
    },
  },
}

If this is not present, and no body/headers are sent, it means that the request actually had no body/headers. We can use this in the UI to show appropriate hints to the user.

Dropped experiment

This drops the related experimental options.

Closes #7830

github-actions · 2023-04-25T11:38:40Z

size-limit report 📦

Path	Size
@sentry/browser - ES5 CDN Bundle (gzipped + minified)	21.02 KB (-0.01% 🔽)
@sentry/browser - ES5 CDN Bundle (minified)	65.66 KB (0%)
@sentry/browser - ES6 CDN Bundle (gzipped + minified)	19.56 KB (+0.01% 🔺)
@sentry/browser - ES6 CDN Bundle (minified)	58.12 KB (0%)
@sentry/browser - Webpack (gzipped + minified)	21.17 KB (0%)
@sentry/browser - Webpack (minified)	69.07 KB (0%)
@sentry/react - Webpack (gzipped + minified)	21.19 KB (0%)
@sentry/nextjs Client - Webpack (gzipped + minified)	49.04 KB (0%)
@sentry/browser + @sentry/tracing - ES5 CDN Bundle (gzipped + minified)	28.59 KB (+0.01% 🔺)
@sentry/browser + @sentry/tracing - ES6 CDN Bundle (gzipped + minified)	26.82 KB (-0.01% 🔽)
@sentry/replay ES6 CDN Bundle (gzipped + minified)	46.48 KB (+0.46% 🔺)
@sentry/replay - Webpack (gzipped + minified)	40.27 KB (+0.33% 🔺)
@sentry/browser + @sentry/tracing + @sentry/replay - ES6 CDN Bundle (gzipped + minified)	65.25 KB (+0.21% 🔺)
@sentry/browser + @sentry/replay - ES6 CDN Bundle (gzipped + minified)	58.24 KB (+0.25% 🔺)

billyvg · 2023-04-25T13:05:45Z

...er-integration-tests/suites/replay/extendNetworkBreadcrumbs/fetch/captureRequestBody/init.js

-    captureNetworkBodies: true,
-  },
+
+  networkDetailAllowUrls: ['http://localhost:7654/foo'],


Just double checking, is the port here stable?

🤷 this is also in the tests, so it would fail there as well (we check the url of the events is this in there)

billyvg · 2023-04-25T13:24:54Z

packages/replay/src/coreHandlers/util/fetchUtils.ts

@@ -78,33 +80,37 @@ async function _prepareFetchData(
  const {
    url,
    method,
-    status_code: statusCode,
+    status_code: statusCode = 0,


Should this default to 0? Does undefined vs explicit 0 have different meanings?

We've been doing this before as well (only setting it further down) - but 🤷 I'd say it's fine?

billyvg · 2023-04-25T13:27:31Z

packages/replay/src/types.ts

+  /**
+   * Capture the following request headers, in addition to the default ones.
+   * Only applies to URLs matched by `networkDetailAllowUrls`.
+   * Any headers defined here will be captured in addition to the default headers.


Should we list the default headers here?

I'd just document them in the docs, IMHO should be good enough? As this may change, I guess.

billyvg · 2023-04-25T13:30:26Z

packages/replay/src/types.ts

+   * Only applies to URLs matched by `networkDetailAllowUrls`.
+   * Any headers defined here will be captured in addition to the default headers.
+   */
+  networkRequestHeaders: string[];


Wonder if headers should be RegExp + string as well?

it's a valid point, although I think this is less of a use case probably. IMHO I'd leave this like this for now, we can always extend this later if we want to also allow regex.

Yeah we can add it later -- I was testing this locally for sentry and found myself wanting to add x-sentry-*

billyvg

ryan953 · 2023-04-25T20:56:04Z

I think we need to also have a BODY_SKIPPED warning, so we can tell the difference between 'this GET request didn't have a body' and the 'networkCaptureBodies was set to false' case.

mydea · 2023-04-26T07:16:15Z

I think we need to also have a BODY_SKIPPED warning, so we can tell the difference between 'this GET request didn't have a body' and the 'networkCaptureBodies was set to false' case.

Good point! I think I can add this in a follow up PR, though.

…IPPED` warning This actually does not exist in our SDK, instead we can make the same assumption using sdk configuration options (e.g. if they have the option enabled) and if the body is empty to determine if we should show the onboarding setup. ref: getsentry/sentry-javascript#7953 (comment) closes: getsentry/sentry-javascript#8004

billyvg · 2025-02-07T19:53:55Z

I think we need to also have a BODY_SKIPPED warning, so we can tell the difference between 'this GET request didn't have a body' and the 'networkCaptureBodies was set to false' case.

Good point! I think I can add this in a follow up PR, though.

Doing some tidying, we did not add a BODY_SKIPPED warning, but rather, added SDK options so that you can check if networkCaptureBodies is true or false.

…IPPED` warning (#84654) This actually does not exist in our SDK, instead we can make the same assumption using sdk configuration options (e.g. if they have the option enabled) and if the body is empty to determine if we should show the onboarding setup. ref: getsentry/sentry-javascript#7953 (comment) closes: getsentry/sentry-javascript#8004

mydea requested review from billyvg and Lms24 April 25, 2023 11:26

mydea self-assigned this Apr 25, 2023

mydea added Type: Feature Package: replay Issues related to the Sentry Replay SDK labels Apr 25, 2023

billyvg reviewed Apr 25, 2023

View reviewed changes

feat(replay): Allow to configure URLs to capture network bodies/headers

b4fb3d7

mydea force-pushed the fn/replay-urls branch from a662d30 to b4fb3d7 Compare April 25, 2023 13:37

billyvg approved these changes Apr 25, 2023

View reviewed changes

mydea merged commit 7d10c44 into develop Apr 26, 2023

mydea deleted the fn/replay-urls branch April 26, 2023 08:54

kahest mentioned this pull request Oct 15, 2024

Make response body accessible via hint in beforeSendTransaction callback for SentryHttpClient getsentry/sentry-dart#2293

Closed

6 tasks

billyvg mentioned this pull request Feb 5, 2025

fix(replay): Correct getOutputType to not use non-existant BODY_SKIPPED warning getsentry/sentry#84654

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

feat(replay): Allow to configure URLs to capture network bodies/headers #7953

feat(replay): Allow to configure URLs to capture network bodies/headers #7953

Uh oh!

mydea commented Apr 25, 2023

Uh oh!

github-actions bot commented Apr 25, 2023 •

edited

Loading

Uh oh!

billyvg Apr 25, 2023

Uh oh!

mydea Apr 25, 2023

Uh oh!

billyvg Apr 25, 2023

Uh oh!

mydea Apr 25, 2023

Uh oh!

billyvg Apr 25, 2023

Uh oh!

mydea Apr 26, 2023

Uh oh!

billyvg Apr 25, 2023

Uh oh!

mydea Apr 25, 2023

Uh oh!

billyvg Apr 25, 2023

Uh oh!

billyvg left a comment

Uh oh!

ryan953 commented Apr 25, 2023

Uh oh!

mydea commented Apr 26, 2023

Uh oh!

billyvg commented Feb 7, 2025

Uh oh!

Uh oh!

Uh oh!

feat(replay): Allow to configure URLs to capture network bodies/headers #7953

feat(replay): Allow to configure URLs to capture network bodies/headers #7953

Uh oh!

Conversation

mydea commented Apr 25, 2023

Example usage:

Capture bodies & default headers for given URL:

Capture bodies & default headers for given exact URLs:

Capture only headers, including some custom ones:

Warnings

Dropped experiment

Uh oh!

github-actions bot commented Apr 25, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

size-limit report 📦

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

billyvg left a comment

Choose a reason for hiding this comment

Uh oh!

ryan953 commented Apr 25, 2023

Uh oh!

mydea commented Apr 26, 2023

Uh oh!

billyvg commented Feb 7, 2025

Uh oh!

Uh oh!

github-actions bot commented Apr 25, 2023 •

edited

Loading