Skip to content

Fix flaky auth webdriver tests #7582

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Aug 24, 2023
Merged

Fix flaky auth webdriver tests #7582

merged 6 commits into from
Aug 24, 2023

Conversation

NhienLam
Copy link
Contributor

@NhienLam NhienLam commented Aug 23, 2023

We currently pin Chrome v110 on CI as tests in https://github.com/firebase/firebase-js-sdk/blob/master/packages/auth/test/integration/webdriver/redirect.test.ts became flaky on version 111.

This change fixes these flaky tests and pins Chrome v114.

Why do we pin v114 but not install the latest Chrome version?
https://chromedriver.chromium.org/downloads only goes up to version 114, so ChromeDriver installation will fail with higher versions: "No such object: chromedriver/LATEST_RELEASE_115.0.5790". According to that website, starting with version 115 these JSON endpoints should be used to determine compatible versions and download urls. Hence, we will pin v114 for now and update the script to install the latest ChromeDriver in another PR.
Internal bug: b/297380444

I've tested this locally with the latest Chrome v116. All auth webdriver tests passed 7 times out of 7 runs.
CI tests with Chrome v114 also pass.

@changeset-bot
Copy link

changeset-bot bot commented Aug 23, 2023

⚠️ No Changeset found

Latest commit: 06a7709

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@google-oss-bot
Copy link
Contributor

google-oss-bot commented Aug 23, 2023

Size Report 1

Affected Products

  • @firebase/firestore

    TypeBase (b2f6084)Merge (9acf068)Diff
    browser371 kB374 kB+3.70 kB (+1.0%)
    esm5356 kB359 kB+3.29 kB (+0.9%)
    main570 kB576 kB+5.86 kB (+1.0%)
    module371 kB374 kB+3.70 kB (+1.0%)
    react-native371 kB375 kB+3.70 kB (+1.0%)
  • @firebase/firestore-lite

    TypeBase (b2f6084)Merge (9acf068)Diff
    browser110 kB110 kB+43 B (+0.0%)
    esm5107 kB107 kB+122 B (+0.1%)
    main152 kB152 kB+43 B (+0.0%)
    module110 kB110 kB+43 B (+0.0%)
    react-native110 kB110 kB+43 B (+0.0%)
  • bundle

    TypeBase (b2f6084)Merge (9acf068)Diff
    firestore (Persistence)300 kB302 kB+1.83 kB (+0.6%)
    firestore (Query Cursors)238 kB239 kB+1.17 kB (+0.5%)
    firestore (Query)235 kB236 kB+1.17 kB (+0.5%)
    firestore (Read data once)223 kB224 kB+1.17 kB (+0.5%)
    firestore (Realtime updates)225 kB226 kB+1.17 kB (+0.5%)
    firestore (Transaction)203 kB204 kB+1.17 kB (+0.6%)
    firestore (Write data)203 kB204 kB+1.17 kB (+0.6%)
    firestore-lite (Transaction)86.1 kB86.2 kB+43 B (+0.0%)
  • firebase

    TypeBase (b2f6084)Merge (9acf068)Diff
    firebase-compat.js777 kB779 kB+1.78 kB (+0.2%)
    firebase-firestore-compat.js339 kB341 kB+1.78 kB (+0.5%)
    firebase-firestore-lite.js118 kB118 kB+43 B (+0.0%)
    firebase-firestore.js430 kB434 kB+3.60 kB (+0.8%)

Test Logs

  1. https://storage.googleapis.com/firebase-sdk-metric-reports/aIXWPkUrIH.html

@google-oss-bot
Copy link
Contributor

google-oss-bot commented Aug 23, 2023

Size Analysis Report 1

This report is too large (285,279 characters) to be displayed here in a GitHub comment. Please use the below link to see the full report on Google Cloud Storage.

Test Logs

  1. https://storage.googleapis.com/firebase-sdk-metric-reports/hPk1Q8J99c.html

@NhienLam NhienLam merged commit 5f6304d into master Aug 24, 2023
@NhienLam NhienLam deleted the flaky-webdriver branch August 24, 2023 19:51
@firebase firebase locked and limited conversation to collaborators Sep 24, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants