Skip to content

Commit 6f6af49

Browse files
authored
Clarify lit's definition of failure and conditions when it exits with exit code 1 (llvm#136190)
Documentation of when `lit` exits with code 1 is out of date. It is no longer just "FAIL or XPASS", there are more failure types: https://github.com/llvm/llvm-project/blob/b30100b87f24847afd6407b4939a184ebcf16ef9/llvm/utils/lit/lit/Test.py#L51-L55 Exit code can also be affected by `--ignore-fail` option: https://github.com/llvm/llvm-project/blob/b30100b87f24847afd6407b4939a184ebcf16ef9/llvm/utils/lit/lit/main.py#L154-L162 This PR extracts a clear definition of "failure" from the description of `--report-failures-only` option: https://github.com/llvm/llvm-project/blob/b30100b87f24847afd6407b4939a184ebcf16ef9/llvm/docs/CommandGuide/lit.rst?plain=1#L194-L196 ...puts it into "Test Status Results" section and references it when describing exit codes and `--ignore-fail` option.
1 parent 4f96ce4 commit 6f6af49

File tree

1 file changed

+8
-5
lines changed

1 file changed

+8
-5
lines changed

llvm/docs/CommandGuide/lit.rst

Lines changed: 8 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -193,7 +193,7 @@ EXECUTION OPTIONS
193193

194194
.. option:: --report-failures-only
195195

196-
Only include unresolved, timed out, failed and unexpectedly passed tests in the report.
196+
Only include failures (see :ref:`test-status-results`) in the report.
197197

198198
.. option:: --resultdb-output RESULTDB_OUTPUT
199199

@@ -384,8 +384,9 @@ ADDITIONAL OPTIONS
384384
EXIT STATUS
385385
-----------
386386

387-
:program:`lit` will exit with an exit code of 1 if there are any FAIL or XPASS
388-
results. Otherwise, it will exit with the status 0. Other exit codes are used
387+
:program:`lit` will exit with an exit code of 1 if there are any failures
388+
(see :ref:`test-status-results`) and :option:`--ignore-fail` has not been
389+
passed. Otherwise, it will exit with the status 0. Other exit codes are used
389390
for non-test related failures (for example a user error or an internal program
390391
error).
391392

@@ -461,8 +462,10 @@ Each test ultimately produces one of the following eight results:
461462

462463
**TIMEOUT**
463464

464-
The test was run, but it timed out before it was able to complete. This is
465-
considered a failure.
465+
The test was run, but it timed out before it was able to complete.
466+
467+
Unresolved (**UNRESOLVED**), timed out (**TIMEOUT**), failed (**FAIL**) and
468+
unexpectedly passed (**XPASS**) tests are considered failures.
466469

467470
Depending on the test format tests may produce additional information about
468471
their status (generally only for failures). See the :ref:`output-options`

0 commit comments

Comments
 (0)