Skip to content

Commit a8398bd

Browse files
committed
[llvm][docs] Update list of llvm-lit options
Fixes #62899 In this commit I have updated the list of options to include any missing options and re-rordered some of them to match the order in lit's --help. Where there was a larger description in this document I've used that instead of the --help description. This *does not* include --use-unique-output-file-name as this was only added recently and we are still debating whether it will be kept.
1 parent a393c92 commit a8398bd

File tree

1 file changed

+113
-49
lines changed

1 file changed

+113
-49
lines changed

llvm/docs/CommandGuide/lit.rst

Lines changed: 113 additions & 49 deletions
Original file line numberDiff line numberDiff line change
@@ -57,7 +57,11 @@ GENERAL OPTIONS
5757

5858
.. option:: -h, --help
5959

60-
Show the :program:`lit` help message.
60+
Show the :program:`lit` help message and exit.
61+
62+
.. option:: --version
63+
64+
Show :program:`lit`'s version number and exit.
6165

6266
.. option:: -j N, --workers=N
6367

@@ -108,23 +112,51 @@ OUTPUT OPTIONS
108112

109113
Enable -v, but for all tests not just failed tests.
110114

115+
.. option:: -o PATH, --output PATH
116+
117+
Write test results to the provided path.
118+
111119
.. option:: --no-progress-bar
112120

113121
Do not use curses based progress bar.
114122

123+
.. option:: --show-excluded
124+
125+
Show excluded tests.
126+
127+
.. option:: --show-skipped
128+
129+
Show skipped tests.
130+
115131
.. option:: --show-unsupported
116132

117-
Show the names of unsupported tests.
133+
Show unsupported tests.
134+
135+
.. option:: --show-pass
136+
137+
Show passed tests.
138+
139+
.. option:: --show-flakypass
140+
141+
Show passed with retry tests.
118142

119143
.. option:: --show-xfail
120144

121-
Show the names of tests that were expected to fail.
145+
Show expectedly failed tests.
122146

123147
.. _execution-options:
124148

125149
EXECUTION OPTIONS
126150
-----------------
127151

152+
.. option:: --gtest-sharding
153+
154+
Enable sharding for GoogleTest format.
155+
156+
.. option:: --no-gtest-sharding
157+
158+
Disable sharding for GoogleTest format.
159+
128160
.. option:: --path=PATH
129161

130162
Specify an additional ``PATH`` to use when searching for executables in tests.
@@ -139,32 +171,73 @@ EXECUTION OPTIONS
139171
"``valgrind``" feature that can be used to conditionally disable (or expect
140172
failure in) certain tests.
141173

142-
.. option:: --vg-arg=ARG
143-
144-
When :option:`--vg` is used, specify an additional argument to pass to
145-
:program:`valgrind` itself.
146-
147174
.. option:: --vg-leak
148175

149176
When :option:`--vg` is used, enable memory leak checks. When this option is
150177
enabled, :program:`lit` will also automatically provide a "``vg_leak``"
151178
feature that can be used to conditionally disable (or expect failure in)
152179
certain tests.
153180

181+
.. option:: --vg-arg=ARG
182+
183+
When :option:`--vg` is used, specify an additional argument to pass to
184+
:program:`valgrind` itself.
185+
186+
.. option:: --no-execute
187+
188+
Don't execute any tests (assume that they pass).
189+
190+
.. option:: --xunit-xml-output XUNIT_XML_OUTPUT
191+
192+
Write XUnit-compatible XML test reports to the specified file.
193+
194+
.. option:: --resultdb-output RESULTDB_OUTPUT
195+
196+
Write LuCI ResultDB compatible JSON to the specified file.
197+
198+
.. option:: --time-trace-output TIME_TRACE_OUTPUT
199+
200+
Write Chrome tracing compatible JSON to the specified file
201+
202+
.. option:: --timeout MAXINDIVIDUALTESTTIME
203+
204+
Maximum time to spend running a single test (in seconds). 0 means no time
205+
limit. [Default: 0]
206+
207+
.. option:: --timeout=N
208+
209+
Spend at most ``N`` seconds (approximately) running each individual test.
210+
``0`` means no time limit, and ``0`` is the default. Note that this is not an
211+
alias for :option:`--max-time`; the two are different kinds of maximums.
212+
213+
.. option:: --max-failures MAX_FAILURES
214+
215+
Stop execution after the given number of failures.
216+
217+
.. option:: --allow-empty-runs
218+
219+
Do not fail the run if all tests are filtered out.
220+
221+
.. option:: --per-test-coverage
222+
223+
Emit the necessary test coverage data, divided per test case (involves
224+
setting a unique value to LLVM_PROFILE_FILE for each RUN). The coverage
225+
data files will be emitted in the directory specified by ``config.test_exec_root``.
226+
227+
.. option:: --ignore-fail
228+
229+
Exit with status zero even if some tests fail.
230+
154231
.. option:: --skip-test-time-recording
155232

156-
Disable tracking the wall time individual tests take to execute.
233+
Do not track elapsed wall time for each test.
157234

158235
.. option:: --time-tests
159236

160237
Track the wall time individual tests take to execute and includes the results
161238
in the summary output. This is useful for determining which tests in a test
162239
suite take the most time to execute.
163240

164-
.. option:: --ignore-fail
165-
166-
Exit with status zero even if some tests fail.
167-
168241
.. _selection-options:
169242

170243
SELECTION OPTIONS
@@ -178,23 +251,6 @@ The timing data is stored in the `test_exec_root` in a file named
178251
`.lit_test_times.txt`. If this file does not exist, then `lit` checks the
179252
`test_source_root` for the file to optionally accelerate clean builds.
180253

181-
.. option:: --shuffle
182-
183-
Run the tests in a random order, not failing/slowest first. Deprecated,
184-
use :option:`--order` instead.
185-
186-
.. option:: --per-test-coverage
187-
188-
Emit the necessary test coverage data, divided per test case (involves
189-
setting a unique value to LLVM_PROFILE_FILE for each RUN). The coverage
190-
data files will be emitted in the directory specified by `config.test_exec_root`.
191-
192-
.. option:: --max-failures N
193-
194-
Stop execution after the given number ``N`` of failures.
195-
An integer argument should be passed on the command line
196-
prior to execution.
197-
198254
.. option:: --max-tests=N
199255

200256
Run at most ``N`` tests and then terminate.
@@ -205,16 +261,6 @@ The timing data is stored in the `test_exec_root` in a file named
205261
Note that this is not an alias for :option:`--timeout`; the two are
206262
different kinds of maximums.
207263

208-
.. option:: --num-shards=M
209-
210-
Divide the set of selected tests into ``M`` equal-sized subsets or
211-
"shards", and run only one of them. Must be used with the
212-
``--run-shard=N`` option, which selects the shard to run. The environment
213-
variable ``LIT_NUM_SHARDS`` can also be used in place of this
214-
option. These two options provide a coarse mechanism for partitioning large
215-
testsuites, for parallel execution on separate machines (say in a large
216-
testing farm).
217-
218264
.. option:: --order={lexical,random,smart}
219265

220266
Define the order in which tests are run. The supported values are:
@@ -228,18 +274,14 @@ The timing data is stored in the `test_exec_root` in a file named
228274
tests, all in descending execution time order. This is the default as it
229275
optimizes concurrency.
230276

231-
.. option:: --run-shard=N
277+
.. option:: --shuffle
232278

233-
Select which shard to run, assuming the ``--num-shards=M`` option was
234-
provided. The two options must be used together, and the value of ``N``
235-
must be in the range ``1..M``. The environment variable
236-
``LIT_RUN_SHARD`` can also be used in place of this option.
279+
Run the tests in a random order, not failing/slowest first. Deprecated,
280+
use :option:`--order` instead.
237281

238-
.. option:: --timeout=N
282+
.. option:: -i, --incremental
239283

240-
Spend at most ``N`` seconds (approximately) running each individual test.
241-
``0`` means no time limit, and ``0`` is the default. Note that this is not an
242-
alias for :option:`--max-time`; the two are different kinds of maximums.
284+
Run failed tests first (DEPRECATED: use ``--order=smart``).
243285

244286
.. option:: --filter=REGEXP
245287

@@ -297,6 +339,23 @@ The timing data is stored in the `test_exec_root` in a file named
297339
primary purpose is to suppress an ``XPASS`` result without modifying a test
298340
case that uses the ``XFAIL`` directive.
299341

342+
.. option:: --num-shards=M
343+
344+
Divide the set of selected tests into ``M`` equal-sized subsets or
345+
"shards", and run only one of them. Must be used with the
346+
``--run-shard=N`` option, which selects the shard to run. The environment
347+
variable ``LIT_NUM_SHARDS`` can also be used in place of this
348+
option. These two options provide a coarse mechanism for partitioning large
349+
testsuites, for parallel execution on separate machines (say in a large
350+
testing farm).
351+
352+
.. option:: --run-shard=N
353+
354+
Select which shard to run, assuming the ``--num-shards=M`` option was
355+
provided. The two options must be used together, and the value of ``N``
356+
must be in the range ``1..M``. The environment variable
357+
``LIT_RUN_SHARD`` can also be used in place of this option.
358+
300359
ADDITIONAL OPTIONS
301360
------------------
302361

@@ -313,6 +372,11 @@ ADDITIONAL OPTIONS
313372

314373
List all of the discovered tests and exit.
315374

375+
.. option:: --show-used-features
376+
377+
Show all features used in the test suite (in ``XFAIL``, ``UNSUPPORTED`` and
378+
``REQUIRES``) and exit.
379+
316380
EXIT STATUS
317381
-----------
318382

0 commit comments

Comments
 (0)