Skip to content

[CI] Add LLVM test suite jobs to reusable workflow #5097

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 31 commits into from
Dec 19, 2021

Conversation

alexbatashev
Copy link
Contributor

@alexbatashev alexbatashev commented Dec 7, 2021

Overview of changes:

  1. Refactor cleanup into a separate action to make sure that's the last thing we do
  2. Introduce new llvm_test_suite action, that downloads and executes tests from intel/llvm-test-suite
  3. Add 4 new jobs: OCL gen9, L0 gen9, OCL x64, HIP AMD GPU
  4. Make cuda and hip args optional
  5. Upload LIT as artifact to make it usable from LLVM Test suite

General idea: one build job compiles SYCL toolchain with all available backends enabled. Test jobs wait for build to complete and download pre-built artifact to run tests in parallel on various hardware. Amount of testing can be configured from outside by specifying lts_config parameter with one or more of the following values, separated with semicolon: ocl_x64, ocl_gen9, l0_gen9, hip_amdgpu. When tests are done, results from all the jobs are uploaded to lit_results artifact, which will contain all the logs in JSON format. Also, GitHub will automatically send notification to PR owner if one of the jobs has failed.

Alexander Batashev and others added 30 commits December 7, 2021 15:32
* upstream/sycl:
  [CI] Add container users to video group (intel#5101)
  [CI] More typo fixes in Nightly build (intel#5088)
  Revert "[CI] Disable pack and upload steps (intel#5119)" (intel#5125)
  [SYCL] Translate ZE_RESULT_ERROR_INVALID_ARGUMENT error code from L0 RT (intel#5122)
  [SYCL][L0][Plugin] Call ZeCommandQueueCreate on demand (intel#5109)
  [SYCL] Switch to using blocking USM free for OpenCL GPU (intel#4928)
  [CI] Disable pack and upload steps (intel#5119)
  [SYCL] Disable submission of AssertInfoCopier for FPGA (intel#4780)
  [SYCL][SPIRV] Implement islessgreater with FOrdNotEqual instead (intel#5076)
  [SYCL] Fix typo in the name of the host-visible pool (intel#5073)
  [SYCL] Only call shutdown when DLL is being unloaded, not when process is terminating (intel#4983)
  [SYCL][CUDA][PI] Fix infinite loop when parallel_for range exceeds INT_MAX (intel#5095)
  [SYCL] Translate out-of-memory error codes from L0 RT (intel#5107)
  [SYCL] Fix a few warnings during build scripts configuration (intel#5082)
  [SYCL] Fix amdgpu openmp test (intel#5103)
  [SYCL] [FPGA] Create experimental headers for FPGA latency control (intel#5066)
  [SYCL][CUDA] Don't enqueue an event wait on same CUDA stream (intel#5099)
  Remove PR disable template (intel#5102)
  [BuildBot]Uplift CPU/FPGAEMU RT version (intel#5078)
* sycl:
  [CI] Refactor workflow files (intel#5134)
  [CI] Enable HIP and CUDA plugins in GitHub Actions builds (intel#5087)
  [SYCL] Implement queue flushing (intel#5052)
  [SYCL] Allow library mismatch for libdevice host object on Win32 (intel#5048)
  [SYCL] Remove unnecessary template parameter (intel#5127)
  [CI] Use clang-format from nightly builds (intel#5126)
@alexbatashev alexbatashev changed the title early draft of LLVM test suite testing [CI] Add LLVM test suite jobs to reusable workflow Dec 18, 2021
@alexbatashev alexbatashev marked this pull request as ready for review December 18, 2021 09:20
@alexbatashev alexbatashev requested a review from bader as a code owner December 18, 2021 09:20
Copy link
Contributor

@bader bader left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see validation results on Intel GPU. Could you enable it, please?

Comment on lines +194 to +195
- name: Pack LIT
run: tar -cJf lit.tar.xz -C $GITHUB_WORKSPACE/src/llvm/utils/lit .
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LIT can be installed with pip - https://llvm.org/docs/TestSuiteGuide.html#quickstart.
Would it be more optimal to install it into docker images used for testing?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From what I see, these are only released versions of lit. Although, it does not get updates that often, I'm still not sure if there's any kind of compatibility guarantees for tests and lit.


llvm_test_suite_l0_gen9:
name: L0 GEN9 Test Suite
needs: [build, configure]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One caveat: I think build here means "build+lit", right?
I think it might be useful to let developers run llvm-test-suite even if some LIT tests fail.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point, I'll think about it

@alexbatashev
Copy link
Contributor Author

I don't see validation results on Intel GPU. Could you enable it, please?

It's disabled for two reasons.

  1. (main) two tests are failing due to a bug in a gpu driver. I'm looking into adding a container with older driver versions, also I'm looking into ways of disabling tests based on driver version
  2. These jobs are slower than I expected. I'm also looking into ways to make them faster.

@alexbatashev alexbatashev requested a review from bader December 19, 2021 13:53
@bader bader merged commit a9ef82d into intel:sycl Dec 19, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants