Skip to content

Fix two assertion failures related to invalid UTF-8 #1286

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Jan 27, 2023

Conversation

ahoppen
Copy link
Member

@ahoppen ahoppen commented Jan 27, 2023

  • Fix an incorrect assertion in Cursor.swift
    • This assertion returned false if we reached the end of the file, while it should return true.
  • Make SourceLocationConverter handle invalid UTF-8
    • The SourceLocationConverter previously worked on String, which is lossy if the source contains invalid UTF-8. Make it work on SyntaxText and RawTriviaPiece so it’s source accurate.

While at it, also fix a few things in swift-parser-cli:

  • Show diagnostics from operator folding when running swift-parser-cli print-diags
  • Run the diagnostic formatter during round-trip testing
    • This allows us to reduce crashes in the diagnostic formatter.
  • Output reduced test case as bytes instead of a string
    • This allows us to reduce invalid UTF-8, which were lossily converted when creating a String.

This assertion returned `false` if we reached the end of the file, while it should return `true`.
This allows us to reduce crashes in the diagnostic formatter
This allows us to reduce invalid UTF-8, which were lossily converted when creating a `String`.
The `SourceLocationConverter` previously worked on `String`, which is lossy if the source contains invalid UTF-8. Make it work on `SyntaxText` and `RawTriviaPiece` so it’s source accurate.
@ahoppen ahoppen requested review from rintaro and DougGregor January 27, 2023 09:14
@ahoppen
Copy link
Member Author

ahoppen commented Jan 27, 2023

@swift-ci Please test

@DougGregor DougGregor merged commit 101eaf0 into swiftlang:main Jan 27, 2023
@ahoppen ahoppen deleted the ahoppen/invalid-utf8 branch January 27, 2023 17:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants