fix(ci): Reduce flakiness of Replay integration tests #6823
Conversation
| @@ -24,7 +24,7 @@ sentryTest('sampling', async ({ getLocalTestPath, page }) => { | |||
| await page.goto(url); | |||
|
|
|||
| await page.click('button'); | |||
| await page.waitForTimeout(200); | |||
| await page.waitForTimeout(500); | |||
There was a problem hiding this comment.
Just as a precaution, let's increase this timeout here
There was a problem hiding this comment.
What's the timeout here waiting for?
There was a problem hiding this comment.
Hmm good point actually :D My first intention was to remove flakiness, hence increase the timeout but it seems as if locally I don't need any timeout here. Removed it, let's see if CI agrees 😅
size-limit report 📦
|
Lms24
force-pushed
the
lms-fix-replay-flakes
branch
from
e9dcc90 to
f5f35ec
Compare
Lms24
force-pushed
the
lms-fix-replay-flakes
branch
from
dd82301 to
8a2f4bc
Compare
| @@ -24,7 +24,7 @@ sentryTest('sampling', async ({ getLocalTestPath, page }) => { | |||
| await page.goto(url); | |||
|
|
|||
| await page.click('button'); | |||
| await page.waitForTimeout(200); | |||
| await page.waitForTimeout(500); | |||
There was a problem hiding this comment.
What's the timeout here waiting for?
| // This waitForTimeout call should be okay, as we're not checking for any | ||
| // further network requests afterwards. |
There was a problem hiding this comment.
The below is DEFAULT_FLUSH_MAX_DELAY + 1?
There was a problem hiding this comment.
I think the idea here is just to wait for a flush cycle to ensure that nothing is sent anymore after the 400 response. Can't really use the constant here as it is not exported from the replay package. But I can add another constant here
This PR replaces a bunch of
page.waitForTimeoutcalls withpage.waitForRequestwaits which should make Replay integration tests more stable. I couldn't remove all timeout calls, as we're not always actually waiting for a request but the ones that failed mostly in CI are now gone.When relying on outgoing http requests during a test, we should try to be as specific as possible in terms of which request we're actually expecting when. Therefore, I added a
waitForReplayRequest, which ensures we're actually waiting for a replay request. Optionally, this helper takes asegmentIdto wait for a replay request of a specific replay segment. This might be useful for more involved testing scenarios.closes #6800