Skip to content

test(replay): Skip flaky test in firefox #7496

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Mar 17, 2023
Merged

test(replay): Skip flaky test in firefox #7496

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
b739033
test: Skip some flaky tests
mydea Mar 17, 2023
c6e41cd
skip some more
mydea Mar 17, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions packages/browser-integration-tests/suites/replay/errors/errorMode/test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,8 +17,9 @@ import {

sentryTest(
'[error-mode] should start recording and switch to session mode once an error is thrown',
async ({ getLocalTestPath, page }) => {
if (shouldSkipReplayTest()) {
async ({ getLocalTestPath, page, browserName }) => {
// This was sometimes flaky on firefox/webkit, so skipping for now
if (shouldSkipReplayTest() || ['firefox', 'webkit'].includes(browserName)) {
sentryTest.skip();
}

Expand Down
9 changes: 5 additions & 4 deletions packages/browser-integration-tests/suites/replay/errors/errorsInSession/test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,8 @@ import {
sentryTest(
'[session-mode] replay event should contain an error id of an error that occurred during session recording',
async ({ getLocalTestPath, page, browserName, forceFlushReplay }) => {
// TODO(replay): This is flakey on firefox where clicks are flakey
if (shouldSkipReplayTest() || ['firefox'].includes(browserName)) {
// Skipping this in firefox/webkit because it is flakey there
if (shouldSkipReplayTest() || ['firefox', 'webkit'].includes(browserName)) {
sentryTest.skip();
}

Expand Down Expand Up @@ -86,8 +86,9 @@ sentryTest(

sentryTest(
'[session-mode] replay event should not contain an error id of a dropped error while recording',
async ({ getLocalTestPath, page, forceFlushReplay }) => {
if (shouldSkipReplayTest()) {
async ({ getLocalTestPath, page, forceFlushReplay, browserName }) => {
// Skipping this in firefox/webkit because it is flakey there
if (shouldSkipReplayTest() || ['firefox', 'webkit'].includes(browserName)) {
sentryTest.skip();
}

Expand Down
12 changes: 6 additions & 6 deletions packages/browser-integration-tests/suites/replay/requests/test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,10 +5,10 @@ import { expectedFetchPerformanceSpan, expectedXHRPerformanceSpan } from '../../
import { getReplayRecordingContent, shouldSkipReplayTest, waitForReplayRequest } from '../../../utils/replayHelpers';

sentryTest('replay recording should contain fetch request span', async ({ getLocalTestPath, page, browserName }) => {
// For some reason, observing and waiting for requests in firefox is extremely flaky.
// We therefore skip this test for firefox and only test on chromium/webkit.
// For some reason, observing and waiting for requests in firefox/webkit is extremely flaky.
// We therefore skip this test for firefox and only test on chromium.
// Possibly related: https://github.com/microsoft/playwright/issues/11390
if (shouldSkipReplayTest() || browserName === 'firefox') {
if (shouldSkipReplayTest() || ['firefox', 'webkit'].includes(browserName)) {
sentryTest.skip();
}

Expand Down Expand Up @@ -48,9 +48,9 @@ sentryTest('replay recording should contain fetch request span', async ({ getLoc
});

sentryTest('replay recording should contain XHR request span', async ({ getLocalTestPath, page, browserName }) => {
// For some reason, observing and waiting for requests in firefox is extremely flaky.
// We therefore skip this test for firefox and only test on chromium/webkit.
if (shouldSkipReplayTest() || browserName === 'firefox') {
// For some reason, observing and waiting for requests in firefox/webkit is extremely flaky.
// We therefore skip this test for firefox and only test on chromium.
if (shouldSkipReplayTest() || ['firefox', 'webkit'].includes(browserName)) {
sentryTest.skip();
}

Expand Down