[Parser] Detect non-breaking space (U+00A0) and offer a fix-it #16090
Conversation
hashemi
force-pushed
the
non-breaking-spaces
branch
from
1645493 to
9d28cd5
Compare
lib/Parse/Lexer.cpp
Outdated
| @@ -716,6 +722,12 @@ static bool isRightBound(const char *tokEnd, bool isLeftBound, | |||
| else | |||
| return true; | |||
|
|
|||
| case '\302': | |||
| if (tokEnd[1] == '\240') | |||
There was a problem hiding this comment.
Could you make these '\xC2' and '\xA0'?
lib/Parse/Lexer.cpp
Outdated
| @@ -682,6 +682,12 @@ static bool isLeftBound(const char *tokBegin, const char *bufferBegin) { | |||
| else | |||
| return true; | |||
|
|
|||
| case '\240': | |||
| if (tokBegin - 1 != bufferBegin && tokBegin[-2] == '\302') | |||
lib/Parse/Lexer.cpp
Outdated
| } else if (Codepoint == 0x000000A0) { | ||
| // Non-breaking whitespace (U+00A0) | ||
| diagnose(CurPtr - 1, diag::lex_nonbreaking_space) | ||
| .fixItReplaceChars(getSourceLoc(CurPtr - 1), getSourceLoc(Tmp), " "); |
|
Also, could you add tests for fix-it in |
|
Thanks @rintaro, all suggestions incorporated. |
|
I think dedicated test file is OK. Since this test is handling invisible characters, it's more safe from accidental (automatic) edits. |
| @@ -716,6 +722,12 @@ static bool isRightBound(const char *tokEnd, bool isLeftBound, | |||
| else | |||
| return true; | |||
|
|
|||
| case '\xC2': | |||
| if (tokEnd[1] == '\xA0') | |||
There was a problem hiding this comment.
Can you confirm that this dereference is safe? Can we assume that the buffer is always null-terminated?
There was a problem hiding this comment.
Yes, it's guaranteed. https://github.com/apple/swift/blob/9ba05efb/lib/Parse/Lexer.cpp#L188
|
Thank you Ahmad! Please mark the SR issue RESOLVED. |
Detect Unicode non-breaking space character (U+00A0) in the lexer, diagnose it, and offer a fix-it turning it into a regular space. Also treat the non-breaking space as whitespace when considering whether an operator is left/right bound.
Resolves SR-7122.