1
- # ` compiletest `
1
+ # Using ` compiletest ` commands to control test execution
2
2
3
3
## Introduction
4
4
5
- ` compiletest ` is the main test harness of the Rust test suite. It allows
5
+ ` compiletest ` is the main test harness of the Rust test suite. It allows
6
6
test authors to organize large numbers of tests (the Rust compiler has many
7
7
thousands), efficient test execution (parallel execution is supported), and
8
8
allows the test author to configure behavior and expected results of both
9
9
individual and groups of tests.
10
10
11
11
` compiletest ` tests may check test code for success, for runtime failure, or for
12
- compile-time failure. Tests are typically organized as a Rust source file with
12
+ compile-time failure. Tests are typically organized as a Rust source file with
13
13
annotations in comments before and/or within the test code, which serve to
14
14
direct ` compiletest ` on if or how to run the test, what behavior to expect,
15
- and more. If you are unfamiliar with the compiler testing framework,
16
- see [ this chapter] ( ./tests/intro.html ) for additional background.
15
+ and more. If you are unfamiliar with the compiler testing framework,
16
+ see [ this chapter] ( ./tests/intro.md ) for additional background.
17
17
18
18
The tests themselves are typically (but not always) organized into
19
19
"suites" – for example, ` run-fail ` , a folder holding tests that should compile
20
20
successfully, but return a failure (non-zero status) at runtime, ` compile-fail ` ,
21
- a folder holding tests that should fail to compile, and many more. The various
22
- suites are defined in [ ` src/tools/compiletest/src/common.rs ` ] [ common ] in the
23
- ` pub enum Mode ` declaration. And a good introduction to the different
21
+ a folder holding tests that should fail to compile, and many more. The various
22
+ suites are defined in [ ` src/tools/compiletest/src/common.rs ` ] in the
23
+ ` pub enum Mode ` declaration. And a good introduction to the different
24
24
suites of compiler tests along with details about them can be found in
25
- [ Adding new tests] ( ./tests/adding.html ) .
25
+ [ Adding new tests] ( ./tests/adding.md ) .
26
26
27
27
## Adding a new test file
28
28
29
29
Briefly, simply create your new test in the appropriate location under
30
30
[ ` src/test ` ] . No registration of test files is necessary as ` compiletest `
31
31
will scan the [ ` src/test ` ] subfolder recursively, and will execute any
32
32
Rust source files it finds as tests.
33
- See [ Adding new tests] ( ./tests/adding.html ) for a complete guide on how to add
33
+ See [ Adding new tests] ( ./tests/adding.md ) for a complete guide on how to add
34
34
new tests.
35
35
36
36
## Header Commands
37
37
38
38
Source file annotations which appear in comments near the top of the source
39
- file * before* any test code are known as header commands. These commands can
39
+ file * before* any test code are known as header commands. These commands can
40
40
instruct ` compiletest ` to ignore this test, set expectations on whether it is
41
41
expected to succeed at compiling, or what the test's return code is expected to
42
- be. Header commands (and their inline counterparts, Error Info commands) are
43
- described more fully
44
- [ here] ( ./tests/adding.html#header-commands-configuring-rustc ) .
42
+ be. Header commands and inline ` //~ ERROR ` commands are described more fully
43
+ [ here] ( ./tests/adding.md#header-commands-configuring-rustc ) .
45
44
46
45
### Adding a new header command
47
46
48
47
Header commands are defined in the ` TestProps ` struct in
49
- [ ` src/tools/compiletest/src/header.rs ` ] [ header ] . At a high level, there are
48
+ [ ` src/tools/compiletest/src/header.rs ` ] . At a high level, there are
50
49
dozens of test properties defined here, all set to default values in the
51
50
` TestProp ` struct's ` impl ` block. Any test can override this default value by
52
51
specifying the property in question as header command as a comment (` // ` ) in
@@ -77,48 +76,47 @@ fn main() -> Result<(), Box<Error>> {
77
76
#### Adding a new header command property
78
77
79
78
One would add a new header command if there is a need to define some test
80
- property or behavior on an individual, test-by-test basis. A header command
79
+ property or behavior on an individual, test-by-test basis. A header command
81
80
property serves as the header command's backing store (holds the command's
82
81
current value) at runtime.
83
82
84
83
To add a new header command property:
85
- 1. Look for the ` pub struct TestProps ` declaration in
86
- [ src/tools/compiletest/src/header.rs] [ header ] and add the new public
87
- property to the end of the declaration.
88
- 2. Look for the ` impl TestProps ` implementation block immediately following
89
- the struct declaration and initialize the new property to its default
90
- value.
84
+
85
+ 1 . Look for the ` pub struct TestProps ` declaration in
86
+ [ ` src/tools/compiletest/src/header.rs ` ] and add the new public property to
87
+ the end of the declaration.
88
+ 2 . Look for the ` impl TestProps ` implementation block immediately following
89
+ the struct declaration and initialize the new property to its default
90
+ value.
91
91
92
92
#### Adding a new header command parser
93
93
94
94
When ` compiletest ` encounters a test file, it parses the file a line at a time
95
95
by calling every parser defined in the ` Config ` struct's implementation block,
96
- also in [ ` src/tools/compiletest/src/header.rs ` ] [ header ] (note the ` Config `
97
- struct's declaration block is found in
98
- [ ` src/tools/compiletest/src/common.rs ` ] [ common ] . ` TestProps ` 's ` load_from() `
99
- method will try passing the current line of text to each parser, which, in turn
100
- typically checks to see if the line begins with a particular commented (` // ` )
101
- header command such as ` // must-compile-successfully ` or ` // failure-status ` .
102
- Whitespace after the comment marker is optional.
96
+ also in [ ` src/tools/compiletest/src/header.rs ` ] [ ] (note that the ` Config `
97
+ struct's declaration block is found in [ ` src/tools/compiletest/src/common.rs ` ] ).
98
+ ` TestProps ` 's ` load_from() ` method will try passing the current line of text to
99
+ each parser, which, in turn typically checks to see if the line begins with a
100
+ particular commented (` // ` ) header command such as ` // must-compile-successfully `
101
+ or ` // failure-status ` . Whitespace after the comment marker is optional.
103
102
104
103
Parsers will override a given header command property's default value merely by
105
104
being specified in the test file as a header command or by having a parameter
106
105
value specified in the test file, depending on the header command.
107
106
108
107
Parsers defined in ` impl Config ` are typically named ` parse_<header_command> `
109
108
(note kebab-case ` <header-command> ` transformed to snake-case
110
- ` <header_command> ` ). ` impl Config ` also defines several 'low-level' parsers
109
+ ` <header_command> ` ). ` impl Config ` also defines several 'low-level' parsers
111
110
which make it simple to parse common patterns like simple presence or not
112
111
(` parse_name_directive() ` ), header-command: parameter (s)
113
112
(` parse_name_value_directive() ` ), optional parsing only if a particular ` cfg `
114
- attribute is defined (` has_cfg_prefix() ` ) and many more. The low-level parsers
113
+ attribute is defined (` has_cfg_prefix() ` ) and many more. The low-level parsers
115
114
are found near the end of the ` impl Config ` block; be sure to look through them
116
115
and their associated parsers immediately above to see how they are used to
117
116
avoid writing additional parsing code unnecessarily.
118
117
119
118
As a concrete example, here is the implementation for the
120
- ` parse_failure_status() ` parser, in
121
- [ ` src/tools/compiletest/src/header.rs ` ] [ header ] :
119
+ ` parse_failure_status() ` parser, in [ ` src/tools/compiletest/src/header.rs ` ] :
122
120
123
121
``` diff
124
122
@@ -232,6 +232,7 @@ pub struct TestProps {
@@ -147,7 +145,7 @@ As a concrete example, here is the implementation for the
147
145
+ }
148
146
});
149
147
150
- for key in &["RUST_TEST_NOCAPTURE", "RUST_TEST_THREADS"] {
148
+ for key in &["RUST_TEST_NOCAPTURE", "RUST_TEST_THREADS"] {
151
149
@@ -488,6 +494,13 @@ impl Config {
152
150
self.parse_name_directive(line, "pretty-compare-only")
153
151
}
@@ -163,24 +161,23 @@ As a concrete example, here is the implementation for the
163
161
## Implementing the behavior change
164
162
165
163
When a test invokes a particular header command, it is expected that some
166
- behavior will change as a result. What behavior, obviously, will depend on the
167
- purpose of the header command. In the case of ` failure-status ` , the behavior
164
+ behavior will change as a result. What behavior, obviously, will depend on the
165
+ purpose of the header command. In the case of ` failure-status ` , the behavior
168
166
that changes is that ` compiletest ` expects the failure code defined by the
169
167
header command invoked in the test, rather than the default value.
170
168
171
169
Although specific to ` failure-status ` (as every header command will have a
172
170
different implementation in order to invoke behavior change) perhaps it is
173
171
helpful to see the behavior change implementation of one case, simply as an
174
- example. To implement ` failure-status ` , the ` check_correct_failure_status() `
172
+ example. To implement ` failure-status ` , the ` check_correct_failure_status() `
175
173
function found in the ` TestCx ` implementation block, located in
176
- [ ` src/tools/compiletest/src/runtest.rs ` ] ( https://github.com/rust-lang/rust/tree/master/src/tools/compiletest/src/runtest.rs ) ,
177
- was modified as per below:
174
+ [ ` src/tools/compiletest/src/runtest.rs ` ] , was modified as per below:
178
175
179
176
``` diff
180
177
@@ -295,11 +295,14 @@ impl<'test> TestCx<'test> {
181
- }
178
+ }
182
179
183
- fn check_correct_failure_status(&self, proc_res: &ProcRes) {
180
+ fn check_correct_failure_status(&self, proc_res: &ProcRes) {
184
181
- // The value the rust runtime returns on failure
185
182
- const RUST_ERR: i32 = 101;
186
183
- if proc_res.status.code() != Some(RUST_ERR) {
@@ -213,13 +210,14 @@ was modified as per below:
213
210
}
214
211
```
215
212
Note the use of ` self.props.failure_status ` to access the header command
216
- property. In tests which do not specify the failure status header command,
213
+ property. In tests which do not specify the failure status header command,
217
214
` self.props.failure_status ` will evaluate to the default value of 101 at the
218
- time of this writing. But for a test which specifies a header command of, for
215
+ time of this writing. But for a test which specifies a header command of, for
219
216
example, ` // failure-status: 1 ` , ` self.props.failure_status ` will evaluate to
220
217
1, as ` parse_failure_status() ` will have overridden the ` TestProps ` default
221
218
value, for that test specifically.
222
219
223
220
[ `src/test` ] : https://github.com/rust-lang/rust/tree/master/src/test
224
- [ header ] : https://github.com/rust-lang/rust/tree/master/src/tools/compiletest/src/header.rs
225
- [ common ] : https://github.com/rust-lang/rust/tree/master/src/tools/compiletest/src/common.rs
221
+ [ `src/tools/compiletest/src/header.rs` ] : https://github.com/rust-lang/rust/tree/master/src/tools/compiletest/src/header.rs
222
+ [ `src/tools/compiletest/src/common.rs` ] : https://github.com/rust-lang/rust/tree/master/src/tools/compiletest/src/common.rs
223
+ [ `src/tools/compiletest/src/runtest.rs` ] : https://github.com/rust-lang/rust/tree/master/src/tools/compiletest/src/runtest.rs
0 commit comments