Skip to content
This repository was archived by the owner on Mar 28, 2023. It is now read-only.

[SYCL] Fix failures on tests that use double datatype #1246

Merged
merged 3 commits into from
Sep 14, 2022

Conversation

aelovikov-intel
Copy link

No description provided.

@aelovikov-intel aelovikov-intel requested review from a team and Pennycook as code owners September 9, 2022 22:51
@aelovikov-intel aelovikov-intel force-pushed the fix-double branch 2 times, most recently from efb4929 to 907ca10 Compare September 9, 2022 22:53
steffenlarsen
steffenlarsen previously approved these changes Sep 12, 2022
@kbobrovs
Copy link

In one of other reviews I mentioned that test reporting "Passed" state w/o actually exercising all features it is supposed to test, can be dangerous - e.g. tests run on TGLLP will report "Passed" w/o running fp64 test cases. Alternative is to extract feature-specific test cases (e.g. fp64-specific) into a separate test which would use "REQUIRES: aspect_fp64" and report "UNSUPPORTED" on TGLLP.

@aelovikov-intel
Copy link
Author

Alternative is to extract feature-specific test cases (e.g. fp64-specific) into a separate test which would use "REQUIRES: aspect_fp64" and report "UNSUPPORTED" on TGLLP.

IMO, this is less maintainable and isn't really better. Does anybody even look at UNSUPPORTED on any regular basis? How exactly would that information be used?

@kbobrovs
Copy link

Alternative is to extract feature-specific test cases (e.g. fp64-specific) into a separate test which would use "REQUIRES: aspect_fp64" and report "UNSUPPORTED" on TGLLP.

IMO, this is less maintainable and isn't really better. Does anybody even look at UNSUPPORTED on any regular basis? How exactly would that information be used?

If nobody looks at this, this is another problem. Taking this to the extreme, a product, which passes all the tests, but all tests return "UNSUPPORTED" is not really tested and should not be released.

@v-klochkov
Copy link

Alternative is to extract feature-specific test cases (e.g. fp64-specific) into a separate test which would use "REQUIRES: aspect_fp64" and report "UNSUPPORTED" on TGLLP.

IMO, this is less maintainable and isn't really better. Does anybody even look at UNSUPPORTED on any regular basis? How exactly would that information be used?

Even though (static checks + REQUIRES) solution gives more info on which tests run or which skipped, my final vote was for device.has() because IMO using (static checks + REQUIRES) may cause quick growth of number of tests.
2 tests: is what we get after separation of fp64 test cases.
3 tests: if use same approach and separate fp16 test cases to another test.
4 tests: if we have test cases using both fp16&fp64.
That might grow exponentially with adding more independent/orthogonal features (3 or more).

@kbobrovs
Copy link

That might grow exponentially with adding more independent/orthogonal features (3 or more).

I think tests should strive to test feature alone, w/o making combination with other features. So I would not expect exponential growth of number of tests (which would actually be even worse - combinatorial explosion - if we want to test combinations).

@aelovikov-intel
Copy link
Author

In an offline discussion we've agreed to proceed with device.has approach at least short-term.

I've also addressed @AlexeySachkov 's concern in e63b15c. Can somebody please re-review/merge?

@AlexeySachkov AlexeySachkov merged commit 2da7640 into intel:intel Sep 14, 2022
myler pushed a commit to myler/llvm-test-suite that referenced this pull request Mar 22, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants