Skip to content

Commit fdde18a

Browse files
jdenny-ornljdenny-ornl
committed
[FileCheck] Given multiple -dump-input, prefer most verbose
Problem: `FILECHECK_OPTS` was implemented so that a test runner, such as a bot, can specify FileCheck debugging options, such as `-dump-input=fail`. However, some existing test suites have FileCheck calls that already specify `-dump-input=fail` or `-dump-input=always`. Without this patch, such tests fail under such a test runner because FileCheck doesn't accept multiple occurrences of `-dump-input`. Solution: This patch permits multiple occurrences of `-dump-input` by assigning precedence to its values in the following descending order: `help`, `always`, `fail`, and `never`. That is, any occurrence of `help` always obtains help, and otherwise the behavior is similar to `-v` vs. `-vv` in that the option specifying the greatest verbosity has precedence. Rationale: My justification for the new behavior is as follows. I have not experienced use cases where, either as a test runner or as a test author, I want to **limit** the permitted debugging verbosity (except as a test author in FileCheck's or lit's test suites where the FileCheck debugging output itself is under test, but the solution there is `env FILECHECK_OPTS=`, and I imagine we should use the same solution anywhere else this need might occur). Of course, as either a test runner or test author, it is useful to **increase** debugging verbosity. Reviewed By: probinson Differential Revision: https://reviews.llvm.org/D70784
1 parent 6da7dbb commit fdde18a

File tree

2 files changed

+116
-13
lines changed

2 files changed

+116
-13
lines changed

‎llvm/test/FileCheck/dump-input-enable.txt

Lines changed: 101 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -42,33 +42,32 @@ BADVAL: {{F|f}}ile{{C|c}}heck{{.*}}: for the --dump-input option: Cannot find op
4242
; RUN: %ProtectFileCheckOutput FileCheck -dump-input=help \
4343
; RUN: | FileCheck %s -check-prefix=HELP
4444

45-
HELP-NOT: {{.}}
46-
HELP: The following description was requested by -dump-input=help
47-
HELP: try{{.*}}-color
48-
HELP-NOT: {{.}}
49-
5045
;--------------------------------------------------
5146
; Check -dump-input=never.
5247
;
5348
; Include the case without -v, which isn't covered elsewhere.
5449
;--------------------------------------------------
5550

51+
; FileCheck success, no -v => no dump, no trace.
5652
; RUN: %ProtectFileCheckOutput \
5753
; RUN: FileCheck -input-file %t.good %t.check -check-prefix=CHECK \
5854
; RUN: -match-full-lines -dump-input=never 2>&1 \
5955
; RUN: | FileCheck %s -match-full-lines -allow-empty \
6056
; RUN: -check-prefixes=NOTRACE,NODUMP
6157

58+
; FileCheck fail, no -v => no dump, no trace.
6259
; RUN: %ProtectFileCheckOutput \
6360
; RUN: not FileCheck -input-file %t.err %t.check -check-prefix=CHECK \
6461
; RUN: -match-full-lines -dump-input=never 2>&1 \
6562
; RUN: | FileCheck %s -match-full-lines -check-prefixes=NOTRACE,ERR,NODUMP
6663

64+
; FileCheck success, -v => no dump, trace.
6765
; RUN: %ProtectFileCheckOutput \
6866
; RUN: FileCheck -input-file %t.good %t.check -check-prefix=CHECK \
6967
; RUN: -match-full-lines -dump-input=never -v 2>&1 \
7068
; RUN: | FileCheck %s -match-full-lines -check-prefixes=TRACE,NODUMP
7169

70+
; FileCheck fail, -v => no dump, trace.
7271
; RUN: %ProtectFileCheckOutput \
7372
; RUN: not FileCheck -input-file %t.err %t.check -check-prefix=CHECK \
7473
; RUN: -match-full-lines -dump-input=never -v 2>&1 \
@@ -78,11 +77,13 @@ HELP-NOT: {{.}}
7877
; Check no -dump-input, which defaults to never.
7978
;--------------------------------------------------
8079

80+
; FileCheck success, -v => no dump, trace.
8181
; RUN: %ProtectFileCheckOutput \
8282
; RUN: FileCheck -input-file %t.good %t.check -check-prefix=CHECK \
8383
; RUN: -match-full-lines -v 2>&1 \
8484
; RUN: | FileCheck %s -match-full-lines -check-prefixes=TRACE,NODUMP
8585

86+
; FileCheck fail, -v => no dump, trace.
8687
; RUN: %ProtectFileCheckOutput \
8788
; RUN: not FileCheck -input-file %t.err %t.check -check-prefix=CHECK \
8889
; RUN: -match-full-lines -v 2>&1 \
@@ -94,23 +95,27 @@ HELP-NOT: {{.}}
9495
; Include the case without -v, which isn't covered elsewhere.
9596
;--------------------------------------------------
9697

98+
; FileCheck success, no -v => no dump, no trace.
9799
; RUN: %ProtectFileCheckOutput \
98100
; RUN: FileCheck -input-file %t.good %t.check -check-prefix=CHECK \
99101
; RUN: -match-full-lines -dump-input=fail 2>&1 \
100102
; RUN: | FileCheck %s -match-full-lines -allow-empty \
101103
; RUN: -check-prefixes=NOTRACE,NODUMP
102104

105+
; FileCheck fail, no -v => dump, no trace.
103106
; RUN: %ProtectFileCheckOutput \
104107
; RUN: not FileCheck -input-file %t.err %t.check -check-prefix=CHECK \
105108
; RUN: -match-full-lines -dump-input=fail 2>&1 \
106109
; RUN: | FileCheck %s -match-full-lines -check-prefixes=NOTRACE,ERR,DUMP-ERR
107110

111+
; FileCheck success, -v => no dump, no trace.
108112
; RUN: %ProtectFileCheckOutput \
109113
; RUN: FileCheck -input-file %t.good %t.check -check-prefix=CHECK \
110114
; RUN: -match-full-lines -dump-input=fail -v 2>&1 \
111115
; RUN: | FileCheck %s -match-full-lines -allow-empty \
112116
; RUN: -check-prefixes=NOTRACE,NODUMP
113117

118+
; FileCheck fail, -v => dump, no trace.
114119
; RUN: %ProtectFileCheckOutput \
115120
; RUN: not FileCheck -input-file %t.err %t.check -check-prefix=CHECK \
116121
; RUN: -match-full-lines -dump-input=fail -v 2>&1 \
@@ -121,24 +126,32 @@ HELP-NOT: {{.}}
121126
; Check -dump-input-on-failure.
122127
;--------------------------------------------------
123128

129+
; Command-line option.
130+
131+
; FileCheck success, -v => no dump, no trace.
124132
; RUN: %ProtectFileCheckOutput \
125133
; RUN: FileCheck -input-file %t.good %t.check -check-prefix=CHECK \
126134
; RUN: -match-full-lines -dump-input-on-failure -v 2>&1 \
127135
; RUN: | FileCheck %s -match-full-lines -allow-empty \
128136
; RUN: -check-prefixes=NOTRACE,NODUMP
129137

138+
; FileCheck fail, -v => dump, no trace.
130139
; RUN: %ProtectFileCheckOutput \
131140
; RUN: not FileCheck -input-file %t.err %t.check -check-prefix=CHECK \
132141
; RUN: -match-full-lines -dump-input-on-failure -v 2>&1 \
133142
; RUN: | FileCheck %s -match-full-lines \
134143
; RUN: -check-prefixes=NOTRACE,ERR,DUMP-ERR,DUMP-ERR-V
135144

145+
; FILECHECK_DUMP_INPUT_ON_FAILURE=1.
146+
147+
; FileCheck success, -v => no dump, no trace.
136148
; RUN: %ProtectFileCheckOutput FILECHECK_DUMP_INPUT_ON_FAILURE=1 \
137149
; RUN: FileCheck -input-file %t.good %t.check -check-prefix=CHECK \
138150
; RUN: -match-full-lines -v 2>&1 \
139151
; RUN: | FileCheck %s -match-full-lines -allow-empty \
140152
; RUN: -check-prefixes=NOTRACE,NODUMP
141153

154+
; FileCheck fail, -v => dump, no trace.
142155
; RUN: %ProtectFileCheckOutput FILECHECK_DUMP_INPUT_ON_FAILURE=1 \
143156
; RUN: not FileCheck -input-file %t.err %t.check -check-prefix=CHECK \
144157
; RUN: -match-full-lines -v 2>&1 \
@@ -149,23 +162,105 @@ HELP-NOT: {{.}}
149162
; Check -dump-input=always.
150163
;--------------------------------------------------
151164

165+
; FileCheck success, -v => dump, no trace.
152166
; RUN: %ProtectFileCheckOutput \
153167
; RUN: FileCheck -input-file %t.good %t.check -check-prefix=CHECK \
154168
; RUN: -match-full-lines -dump-input=always -v 2>&1 \
155169
; RUN: | FileCheck %s -match-full-lines -check-prefixes=NOTRACE,DUMP-OK
156170

171+
; FileCheck fail, -v => dump, no trace.
157172
; RUN: %ProtectFileCheckOutput \
158173
; RUN: not FileCheck -input-file %t.err %t.check -check-prefix=CHECK \
159174
; RUN: -match-full-lines -dump-input=always -v 2>&1 \
160175
; RUN: | FileCheck %s -match-full-lines \
161176
; RUN: -check-prefixes=NOTRACE,ERR,DUMP-ERR,DUMP-ERR-V
162177

178+
;--------------------------------------------------
179+
; Check multiple -dump-input options.
180+
;
181+
; This ocurrs most commonly when a test author specifies -dump-input on a
182+
; specific FileCheck call while a test runner specifies -dump-input in
183+
; FILECHECK_OPTS, but check the behavior generally.
184+
;
185+
; "help" has precedence, and then the most verbose value wins. The most
186+
; common combinations involve "fail" and "always", so test those the most.
187+
;--------------------------------------------------
188+
189+
;- - - - - - - - - - - - - - - - - - - - - - - - -
190+
; Check duplicate.
191+
;- - - - - - - - - - - - - - - - - - - - - - - - -
192+
193+
; fail, fail => fail (FileCheck fail => dump)
194+
; RUN: %ProtectFileCheckOutput \
195+
; RUN: not FileCheck -input-file %t.err %t.check -check-prefix=CHECK \
196+
; RUN: -match-full-lines -dump-input=fail -dump-input=fail -v \
197+
; RUN: 2>&1 \
198+
; RUN: | FileCheck %s -match-full-lines \
199+
; RUN: -check-prefixes=NOTRACE,ERR,DUMP-ERR,DUMP-ERR-V
200+
201+
;- - - - - - - - - - - - - - - - - - - - - - - - -
202+
; Check precedence.
203+
;- - - - - - - - - - - - - - - - - - - - - - - - -
204+
205+
; help, always => help
206+
; RUN: %ProtectFileCheckOutput \
207+
; RUN: FileCheck -input-file %t.err -color %t.check \
208+
; RUN: -dump-input=help -dump-input=always \
209+
; RUN: | FileCheck %s -check-prefix=HELP
210+
211+
; always, fail => always (FileCheck success => dump)
212+
; RUN: %ProtectFileCheckOutput \
213+
; RUN: FileCheck -input-file %t.good %t.check -check-prefix=CHECK \
214+
; RUN: -match-full-lines -dump-input=always -dump-input=fail \
215+
; RUN: -v 2>&1 \
216+
; RUN: | FileCheck %s -match-full-lines -check-prefixes=NOTRACE,DUMP-OK
217+
218+
; fail, never => fail (FileCheck fail => dump)
219+
; RUN: %ProtectFileCheckOutput \
220+
; RUN: not FileCheck -input-file %t.err %t.check -check-prefix=CHECK \
221+
; RUN: -match-full-lines -dump-input=fail -dump-input=never -v \
222+
; RUN: 2>&1 \
223+
; RUN: | FileCheck %s -match-full-lines \
224+
; RUN: -check-prefixes=NOTRACE,ERR,DUMP-ERR,DUMP-ERR-V
225+
226+
;- - - - - - - - - - - - - - - - - - - - - - - - -
227+
; Check that order doesn't matter.
228+
;- - - - - - - - - - - - - - - - - - - - - - - - -
229+
230+
; fail, always => always (FileCheck success => dump)
231+
; RUN: %ProtectFileCheckOutput \
232+
; RUN: FileCheck -input-file %t.good %t.check -check-prefix=CHECK \
233+
; RUN: -match-full-lines -dump-input=fail -dump-input=always \
234+
; RUN: -v 2>&1 \
235+
; RUN: | FileCheck %s -match-full-lines -check-prefixes=NOTRACE,DUMP-OK
236+
237+
;- - - - - - - - - - - - - - - - - - - - - - - - -
238+
; Check that FILECHECK_OPTS isn't handled differently.
239+
;- - - - - - - - - - - - - - - - - - - - - - - - -
240+
241+
; always, fail => always (FileCheck success => dump)
242+
; RUN: %ProtectFileCheckOutput FILECHECK_OPTS=-dump-input=always \
243+
; RUN: FileCheck -input-file %t.good %t.check -check-prefix=CHECK \
244+
; RUN: -match-full-lines -dump-input=fail -v 2>&1 \
245+
; RUN: | FileCheck %s -match-full-lines -check-prefixes=NOTRACE,DUMP-OK
246+
247+
; fail, always => always (FileCheck success => dump)
248+
; RUN: %ProtectFileCheckOutput FILECHECK_OPTS=-dump-input=fail \
249+
; RUN: FileCheck -input-file %t.good %t.check -check-prefix=CHECK \
250+
; RUN: -match-full-lines -dump-input=always -v 2>&1 \
251+
; RUN: | FileCheck %s -match-full-lines -check-prefixes=NOTRACE,DUMP-OK
252+
163253
; END.
164254

165255
;--------------------------------------------------
166-
; Check the output for all cases that actually process directives.
256+
; Check the output.
167257
;--------------------------------------------------
168258

259+
; HELP-NOT: {{.}}
260+
; HELP: The following description was requested by -dump-input=help
261+
; HELP: try{{.*}}-color
262+
; HELP-NOT: {{.}}
263+
169264
; Trace is sometimes suppressed.
170265
; TRACE: {{.*}}remark:{{.*}}
171266
; NOTRACE-NOT: remark:

‎llvm/utils/FileCheck/FileCheck.cpp

Lines changed: 15 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -108,24 +108,28 @@ static cl::opt<bool> DumpInputOnFailure(
108108
"FILECHECK_DUMP_INPUT_ON_FAILURE environment variable.\n"
109109
"This option is deprecated in favor of -dump-input=fail.\n"));
110110

111+
// The order of DumpInputValue members affects their precedence, as documented
112+
// for -dump-input below.
111113
enum DumpInputValue {
112114
DumpInputDefault,
113-
DumpInputHelp,
114115
DumpInputNever,
115116
DumpInputFail,
116-
DumpInputAlways
117+
DumpInputAlways,
118+
DumpInputHelp
117119
};
118120

119-
static cl::opt<DumpInputValue> DumpInput(
120-
"dump-input", cl::init(DumpInputDefault),
121+
static cl::list<DumpInputValue> DumpInputs(
122+
"dump-input",
121123
cl::desc("Dump input to stderr, adding annotations representing\n"
122-
" currently enabled diagnostics\n"),
124+
"currently enabled diagnostics. When there are multiple\n"
125+
"occurrences of this option, the <value> that appears earliest\n"
126+
"in the list below has precedence.\n"),
123127
cl::value_desc("mode"),
124128
cl::values(clEnumValN(DumpInputHelp, "help",
125129
"Explain dump format and quit"),
126-
clEnumValN(DumpInputNever, "never", "Never dump input"),
130+
clEnumValN(DumpInputAlways, "always", "Always dump input"),
127131
clEnumValN(DumpInputFail, "fail", "Dump input on failure"),
128-
clEnumValN(DumpInputAlways, "always", "Always dump input")));
132+
clEnumValN(DumpInputNever, "never", "Never dump input")));
129133

130134
typedef cl::list<std::string>::const_iterator prefix_iterator;
131135

@@ -516,6 +520,10 @@ int main(int argc, char **argv) {
516520
InitLLVM X(argc, argv);
517521
cl::ParseCommandLineOptions(argc, argv, /*Overview*/ "", /*Errs*/ nullptr,
518522
"FILECHECK_OPTS");
523+
DumpInputValue DumpInput =
524+
DumpInputs.empty()
525+
? DumpInputDefault
526+
: *std::max_element(DumpInputs.begin(), DumpInputs.end());
519527
if (DumpInput == DumpInputHelp) {
520528
DumpInputAnnotationHelp(outs());
521529
return 0;

0 commit comments

Comments
 (0)