Skip to content

[CI][E2E] Run E2E tests on NVIDIA with prebuilt binaries in precommit #16813

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Feb 11, 2025

Conversation

ayylol
Copy link
Contributor

@ayylol ayylol commented Jan 28, 2025

Reduces time for precommit nvidia job from around 20m to around 5m

@ayylol ayylol changed the title [SYCL][E2E] Run e2e tests on NVIDIA with prebuilt binaries in precommit [SYCL][E2E] Run E2E tests on NVIDIA with prebuilt binaries in precommit Feb 6, 2025
@ayylol ayylol changed the title [SYCL][E2E] Run E2E tests on NVIDIA with prebuilt binaries in precommit [CI][E2E] Run E2E tests on NVIDIA with prebuilt binaries in precommit Feb 6, 2025
@ayylol ayylol marked this pull request as ready for review February 11, 2025 15:18
@ayylol ayylol requested review from a team as code owners February 11, 2025 15:18
@ayylol ayylol requested a review from againull February 11, 2025 15:18
@sarnex
Copy link
Contributor

sarnex commented Feb 11, 2025

Current NVIDIA job takes ~20m and with this PR it takes ~5. Great speedup!

@aelovikov-intel
Copy link
Contributor

Before (https://github.com/intel/llvm/actions/runs/13262954234/job/37025407654):

Testing Time: 1218.09s

Total Discovered Tests: 2274
  Unsupported      : 1191 (52.37%)
  Passed           : 1061 (46.66%)
  Expectedly Failed:   22 (0.97%)

After (https://github.com/intel/llvm/actions/runs/13266214639/job/37037510106?pr=16813):

Testing Time: 183.58s

Total Discovered Tests: 2274
  Unsupported      : 1199 (52.73%)
  Passed           : 1053 (46.31%)
  Expectedly Failed:   22 (0.97%)

What is the reason for 8 new UNSUPPORTEDs ? Where will we be testing those?

@ayylol
Copy link
Contributor Author

ayylol commented Feb 11, 2025

What is the reason for 8 new UNSUPPORTEDs ? Where will we be testing those?

Those extra unsupported are the following tests:

DeviceLib/math_override_test.cpp
Reduction/reduction_dynamic_span.cpp
Regression/2020-spec-constants-debug-info.cpp
Regression/optimization_level_debug_info_intopt.cpp
Regression/optimization_level_debug_info_specopt.cpp
bindless_images/examples/example_6_import_memory_and_semaphores.cpp
Basic/interop/interop_all_backends.cpp
Adapters/adapter-release.cpp

They are all marked as unsupported on the run-only side because they only have build lines. They are being ran on the build step only. The only one of these where this is incorrect behaviour is the Adapters test, since it has a single run line that executes sycl-ls, I can change that in a separate pr.

Copy link
Contributor

@YuriPlyakhin YuriPlyakhin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Joint Matrix change LGTM

@sarnex
Copy link
Contributor

sarnex commented Feb 11, 2025

Seems approved by everyone, @ayylol ready for merge?

@sarnex sarnex merged commit b97b714 into intel:sycl Feb 11, 2025
26 checks passed
@ayylol ayylol deleted the nvidia-e2e-split branch February 11, 2025 17:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants