Skip to content

test: Make E2E tests async & parallelizable #7466

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 15 commits into from
Mar 30, 2023
Merged

test: Make E2E tests async & parallelizable #7466

merged 15 commits into from
Mar 30, 2023

Conversation

mydea
Copy link
Member

@mydea mydea commented Mar 15, 2023

This PR makes the E2E test run async in child_process.spawn vs child_process.spawnSync), so we can parallelize them.

This required rather extensive refactoring of the E2E code, as making one piece of it async requires making everything async.

For now, we run 2 jobs in parallel on CI, 5 locally. We can adjust this later.

Some notes:

  • We only log output of the child processes when a step fails, so we should have a cleaner log.
  • We have to ensure to use a unique port for each job, as otherwise this fails when running in parallel. Keep this in mind when adding a new test app - it is passed process.env.PORT with a (hopefully) unique port.
  • We consider a job to be parallelized a test-app, so if a test app has multiple jobs, they will still run in sequence.
  • We copy each app into a temporary tmp dir now before we build it, as otherwise it cannot work when multiple versions are specified. This should be cleaned up after the test run.

Closes #7439

@mydea mydea requested review from Lms24 and AbhiPrasad March 15, 2023 09:03
@mydea mydea self-assigned this Mar 15, 2023
@github-actions
Copy link
Contributor

github-actions bot commented Mar 15, 2023

size-limit report 📦

Path Size
@sentry/browser - ES5 CDN Bundle (gzipped + minified) 20.62 KB (0%)
@sentry/browser - ES5 CDN Bundle (minified) 64.4 KB (0%)
@sentry/browser - ES6 CDN Bundle (gzipped + minified) 19.16 KB (+0.02% 🔺)
@sentry/browser - ES6 CDN Bundle (minified) 56.78 KB (0%)
@sentry/browser - Webpack (gzipped + minified) 21.53 KB (0%)
@sentry/browser - Webpack (minified) 72 KB (0%)
@sentry/react - Webpack (gzipped + minified) 21.55 KB (0%)
@sentry/nextjs Client - Webpack (gzipped + minified) 52.05 KB (0%)
@sentry/browser + @sentry/tracing - ES5 CDN Bundle (gzipped + minified) 28.21 KB (+0.01% 🔺)
@sentry/browser + @sentry/tracing - ES6 CDN Bundle (gzipped + minified) 26.41 KB (+0.01% 🔺)
@sentry/replay ES6 CDN Bundle (gzipped + minified) 44.74 KB (+0.01% 🔺)
@sentry/replay - Webpack (gzipped + minified) 38.86 KB (0%)
@sentry/browser + @sentry/tracing + @sentry/replay - ES6 CDN Bundle (gzipped + minified) 63.45 KB (+0.01% 🔺)
@sentry/browser + @sentry/replay - ES6 CDN Bundle (gzipped + minified) 56.49 KB (0%)

@mydea
Copy link
Member Author

mydea commented Mar 16, 2023

No idea why this is failing on CI at build of the E2E apps with e.g.:

Build failed for test application "nextjs-13-app-dir"
  [BUILD OUTPUT] [1/4] Resolving packages...
  [BUILD OUTPUT] [2/4] Fetching packages...
  [BUILD OUTPUT] info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.
  [BUILD OUTPUT] 
  [BUILD OUTPUT] error http://localhost:4873/@sentry/utils/-/utils-7.43.0.tgz: Extracting tar content of undefined failed, the file appears to be corrupt: "ENOENT: no such file or directory, chmod '/home/runner/.cache/yarn/v6/npm-@sentry-utils-7.43.0-f2a49a3de5234bbbf2effef262d4e0f6d188be0b-integrity/node_modules/@sentry/utils/types/dsn.d.ts'"

if anybody has an idea, happy for pointers 😬

@lforst lforst self-requested a review March 29, 2023 07:14
@lforst lforst merged commit 1403e77 into develop Mar 30, 2023
@lforst lforst deleted the fn/e2e-tests branch March 30, 2023 11:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Split up e2e tests
4 participants