Cherry-pick: [lldb][test] Mark gtest cases as XFAIL if the test suite is XFAIL (#102986) #9253
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…vm#102986) When a test case inside of a gtest suite fails, we report a failure which causes the entire `ninja check-lldb` invocation to fail, even if the outer test case is marked as XFAIL - each test case result is reported as its own lit test run. This PR updates lit so it checks whether each test case's parent test suite is XFAIL before setting the status to FAIL. This is especially problematic because the failing tests can't manually be marked as XFAIL, due to llvm#102264. Fixes llvm#102265 ### Repro instructions 1. Modify any gtest test case to generate a failure. 2. Mark the outer lit test with XFAIL using either `--xfail-tests` flag or `LIT_XFAIL` env var. 3. Run the tests 4. Observe the lit test is XFAIL as expected, but the failed child test cases show up as separate failures. Co-authored-by: kendal <[email protected]> (cherry picked from commit 9f89d31)
|
@swift-ci please test |
|
@swift-ci please test llvm |
|
Thanks @compnerd ! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
When a test case inside of a gtest suite fails, we report a failure which causes the entire
ninja check-lldbinvocation to fail, even if the outer test case is marked as XFAIL - each test case result is reported as its own lit test run. This PR updates lit so it checks whether each test case's parent test suite is XFAIL before setting the status to FAIL.This is especially problematic because the failing tests can't manually be marked as XFAIL, due to
llvm#102264.
Fixes llvm#102265
Repro instructions
--xfail-testsflag orLIT_XFAILenv var.Co-authored-by: kendal [email protected]
(cherry picked from commit 9f89d31)