Skip to content

[Docs] Add a section to Diagnostics.md on -verify mode #27448

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Oct 1, 2019
Merged

[Docs] Add a section to Diagnostics.md on -verify mode #27448

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 16 additions & 0 deletions docs/Diagnostics.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -104,3 +104,19 @@ Most diagnostics have no reason to change behavior under editor mode. An example
- `%error` - Represents a branch in a `%select` that should never be taken. In debug builds of the compiler this produces an assertion failure.

- `%%` - Emits a literal percent sign.

### Diagnostic Verifier ###

(This section is specific to the Swift compiler's diagnostic engine.)

If the `-verify` frontend flag is used, the Swift compiler will check emitted diagnostics against specially formatted comments in the source. This feature is used extensively throughout the test suite to ensure diagnostics are emitted with the correct message and source location.

An expected diagnostic is denoted by a comment which begins with `expected-error`, `expected-warning`, `expected-note`, or `expected-remark`. It is followed by:

- (Optional) Location information. By default, the comment will match any diagnostic emitted on the same line. However, it's possible to override this behavior and/or specify column information as well. `// expected-error@-1 ...` looks for an error on the previous line, `// expected-warning@+1:3 ...` looks for a warning on the next line at the third column, and `// expected-note@:7 ...` looks for a note on the same line at the seventh column.

- (Optional) A match count which specifies how many times the diagnostic is expected to appear. This may be a positive integer or `*`, which allows for zero or more matches. The match count must be surrounded by whitespace if present. For example, `// expected-error 2 ...` looks for two matching errors, and `// expected-warning * ...` looks for any number of matching warnings.

- (Required) The expected error message. The message should be enclosed in double curly braces and should not include the `error:`/`warning:`/`note:`/`remark:` prefix. For example, `// expected-error {{invalid redeclaration of 'y'}}` would match an error with that message on the same line. The expected message does not need to match the emitted message verbatim. As long as the expected message is a substring of the original message, they will match.

- (Optional) Expected fix-its. These are each enclosed in double curly braces and appear after the expected message. An expected fix-it consists of a column range followed by the text it's expected to be replaced with. For example, `let r : Int i = j // expected-error{{consecutive statements}} {{12-12=;}}` will match a fix-it attached to the consecutive statements error which inserts a semicolon at column 12, just after the 't' in 'Int'. The special {{none}} specifier is also supported, which will cause the diagnostic match to fail if unexpected fix-its are produced.
Copy link
Collaborator

@theblixguy theblixguy Oct 1, 2019 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I’m still confused about this - the error is consecutive statements on a line must be separated by ';', but it seems you can just enter the first few words to match it. Does it only apply to errors/warnings/notes that are unique (i.e unique prefix perhaps?) or is there something else you need to do for this to work? I think we should document it as well.

Copy link
Contributor Author

@owenv owenv Oct 1, 2019 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I referenced that briefly above: "The expected message will match so long as it's a substring of the message which is emitted.". In practice I think it's only ever used to match a prefix, so I may look into tightening that restriction if it doesn't require updating too many tests.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I clarified this a little more in the meantime to hopefully avoid confusion though.

Copy link
Collaborator

@theblixguy theblixguy Oct 1, 2019 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! I think we should match unique prefixes otherwise if you had two or more diagnostics with the same prefix then it won’t be clear which one is being referred to in the tests.

(although I haven’t seen many tests utilising this and the ones that are don’t seem to conflict with other diagnostics so I’m not sure if this is really a problem, at least for now).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's definitely just matching a substring, not a prefix. Sometimes the interesting words in a diagnostic are in the middle, and/or you don't want to copy out a whole type or something when it's not relevant to the test.

2 changes: 1 addition & 1 deletion docs/Testing.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -232,7 +232,7 @@ code for the target that is not the build machine:

* ``%target-typecheck-verify-swift``: parse and type check the current Swift file
for the target platform and verify diagnostics, like ``swift -frontend -typecheck -verify
%s``.
%s``. For further explanation of `-verify` mode, see [Diagnostics.md](Diagnostics.md).

Use this substitution for testing semantic analysis in the compiler.

Expand Down