Improve CI output. #70236
Merged
Improve CI output. #70236
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
There is currently a major problem with the CI output: The information you need to see never appears in the visible log. This is because our logs are very verbose, and list (A) every test as they run it, and (B) every unsupported tests. This can be thousands of lines. (A) was introduced by me when I disabled the PTY progress bar, which doesn't play nice with log files. That change was an improvement, but I have now disabled the PTY on the builders, so we can go back to passing `-s`. When `-s` is passed but no PTY is available, it prints a long friendly progress indicator. (B) is solved here by disabling the printing of unsupported tests at the end of the test suite. While it can be useful on occasion to audit the list of unsupported tests, it's far from a common operation. Instead people want to see the log of their failure. We should upload the xml results, so if auditing is needed, it can be done using that. Hopefully this change will make it so that the test failures appear in the actual log output
llvmbot
added
the
libc++
|
@llvm/pr-subscribers-libcxx Author: Eric (EricWF) ChangesThere is currently a major problem with the CI output: The information you need to see never appears in the visible log. This is because our logs are very verbose, and list (A) every test as they run it, and (B) every unsupported tests. This can be thousands of lines. (A) was introduced by me when I disabled the PTY progress bar, which doesn't play nice with log files. That change was an improvement, but I have now disabled the PTY on the builders, so we can go back to passing (B) is solved here by disabling the printing of unsupported tests at the end of the test suite. While it can be useful on occasion to audit the list of unsupported tests, it's far from a common operation. Instead people want to see the log of their failure. We should upload the xml results, so if auditing is needed, it can be done using that. Hopefully this change will make it so that the test failures appear in the actual log output Full diff: https://github.com/llvm/llvm-project/pull/70236.diff 1 Files Affected:
diff --git a/libcxx/utils/ci/run-buildbot b/libcxx/utils/ci/run-buildbot
index 89910738f52d040..ebb255243ba9c15 100755
--- a/libcxx/utils/ci/run-buildbot
+++ b/libcxx/utils/ci/run-buildbot
@@ -137,7 +137,7 @@ function generate-cmake-base() {
-DLIBUNWIND_ENABLE_WERROR=YES \
-DLIBCXX_ENABLE_CLANG_TIDY=${ENABLE_CLANG_TIDY} \
${ENABLE_STD_MODULES} \
- -DLLVM_LIT_ARGS="-v --show-unsupported --xunit-xml-output test-results.xml --timeout=1500 --time-tests" \
+ -DLLVM_LIT_ARGS="-sv --xunit-xml-output test-results.xml --timeout=1500 --time-tests" \
"${@}"
}
@@ -372,7 +372,7 @@ bootstrapping-build)
-DLLVM_TARGETS_TO_BUILD="host" \
-DRUNTIMES_BUILD_ALLOW_DARWIN=ON \
-DLLVM_ENABLE_ASSERTIONS=ON \
- -DLLVM_LIT_ARGS="-v --show-unsupported --xunit-xml-output test-results.xml --timeout=1500 --time-tests"
+ -DLLVM_LIT_ARGS="-sv --xunit-xml-output test-results.xml --timeout=1500 --time-tests"
echo "+++ Running the libc++ and libc++abi tests"
${NINJA} -vC "${BUILD_DIR}" check-runtimes
|
zahiraam
pushed a commit
to zahiraam/llvm-project
that referenced
this pull request
Oct 26, 2023
There is currently a major problem with the CI output: The information you need to see never appears in the visible log. This is because our logs are very verbose, and list (A) every test as they run it, and (B) every unsupported tests. This can be thousands of lines. (A) was introduced by me when I disabled the PTY progress bar, which doesn't play nice with log files. That change was an improvement, but I have now disabled the PTY on the builders, so we can go back to passing `-s`. When `-s` is passed but no PTY is available, it prints a long friendly progress indicator. (B) is solved here by disabling the printing of unsupported tests at the end of the test suite. While it can be useful on occasion to audit the list of unsupported tests, it's far from a common operation. Instead people want to see the log of their failure. We should upload the xml results, so if auditing is needed, it can be done using that. Hopefully this change will make it so that the test failures appear in the actual log output
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There is currently a major problem with the CI output:
The information you need to see never appears in the visible log.
This is because our logs are very verbose, and list (A) every test as it runs, and (B) every unsupported test. This can be thousands of lines.
(A) was introduced by me when I disabled the PTY progress bar, which does not play nice with log files. That change was an improvement, but I have now disabled the PTY on the builders, so we can go back to passing
-s. When-sis passed but no PTY is available, it prints a log friendly progress indicator.(B) is solved here by disabling the printing of unsupported tests at the end of the test suite. While it can be useful on occasion to audit the list of unsupported tests, it's far from a common operation. Instead people want to see the log of their failure. We should already be uploading the results as XUnit XML, so if auditing is needed, it can be done using that.
Hopefully this change will make it so that the test failures appear in the actual log output