Skip to content

[Syntax] test diagnostics in Lexer with libSyntax #14954

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Mar 3, 2018

Conversation

omochi
Copy link
Contributor

@omochi omochi commented Mar 3, 2018

To test diagnostics in Lexer with libSyntax,
add Diags arguments to swift::tokenize and swift:tokenizeWithTrivia.
"with libSyntaxmeansTriviaRetentionMode::WithTrivia` in Lexer.

I updated swift-syntax-test.cpp to passing Diags from getTokensFromFile into tokenizeWithTrivia.
It makes doFullLexRoundTrip and doDumpRawTokenSyntax are print diagnostics to stderr.

To keep compatibility,
I changed some Diags.hadAnyError() ? EXIT_FAILURE : EXIT_SUCCESS lines to just EXIT_SUCCESS.
Because, in previous Diags exists but not used so never had error.

And I add test case to check this.
To emit embedded nul character warnings,
I use little trick to make source code which is embedded nul.

This warning comes from standard Lexer logic and shared with WithTrivia and WithoutTrivia.
The purpose of this test case is checking testing of diagnostics mechanism through tokenizeWithTrivia.

In lexer_invalid_nul.swift,
tr '\132' '\0' means to change character Z to nul character.
To avoid replace character in argument of tr itself,
it uses octal form of Z , is \132.

I planned to implement some diagnostics in Lexer which already exists with WithoutTrivia but not exists WithTrivia.
This PR also prepares to this my plan.

@rintaro
Copy link
Member

rintaro commented Mar 3, 2018

@swift-ci Please smoke test

@rintaro
Copy link
Member

rintaro commented Mar 3, 2018

To keep compatibility,
I changed some Diags.hadAnyError() ? EXIT_FAILURE : EXIT_SUCCESS lines to just EXIT_SUCCESS.
Because, in previous Diags exists but not used so never had error.

It's unfortunate this Diags hadn't been used. I'm OK with this for now though.

@rintaro
Copy link
Member

rintaro commented Mar 3, 2018

CC: @shahmishal https://ci.swift.org/job/swift-PR-Linux-smoke-test/4826/console

Build timed out (after 300 minutes). Marking the build as failed.

@rintaro
Copy link
Member

rintaro commented Mar 3, 2018

@swift-ci Please smoke test Linux platform

@rintaro rintaro merged commit d125425 into swiftlang:master Mar 3, 2018
@omochi omochi deleted the test-syntax-diag branch March 7, 2018 14:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants