Introduce RawSyntaxData.ParsedToken
#597
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
rintaro
changed the title
RawSyntaxData.ParsedTokenRawSyntaxData.Payload.ParsedToken
rintaro
changed the title
RawSyntaxData.Payload.ParsedTokenRawSyntaxData.ParsedToken
|
@swift-ci Please test |
ahoppen
reviewed
Aug 16, 2022
rintaro
force-pushed
the
rawsyntax-parsedtoken
branch
4 times, most recently
from
d272eb7 to
326efb3
Compare
|
@swift-ci Please test |
ahoppen
approved these changes
Aug 17, 2022
Sources/SwiftSyntax/RawSyntax.swift
Outdated
| @@ -24,10 +24,17 @@ internal struct RawSyntaxData { | |||
| /// Token with lazy trivia parsing. | |||
| /// | |||
| /// The RawSyntax's `arena` must have a valid trivia parsing function to | |||
| /// lazily materialize the leading/trailing trivia. | |||
| /// lazily materialize the leading/trailing trivia xpieces. | |||
There was a problem hiding this comment.
Typo?
Suggested change
| /// lazily materialize the leading/trailing trivia xpieces. | |
| /// lazily materialize the leading/trailing trivia pieces. |
Sources/SwiftSyntax/RawSyntax.swift
Outdated
| struct ParsedToken { | ||
| var tokenKind: RawTokenKind | ||
|
|
||
| /// Whole of this token including leading/trailing trivia. |
There was a problem hiding this comment.
Suggested change
| /// Whole of this token including leading/trailing trivia. | |
| /// Whole text of this token including leading/trailing trivia. |
A new RawSyntax data variation. This is intended to be used for parsed tokens. Actual usage will be in follow-ups. In this variation, unlike the existing `MaterializedToken`, leading/trailing trivia are stored in a SyntaxText form, and are lazily materialized by a function registered in `SyntaxArena`. Since the parser doesn't need to materialize them while parsing, it'd gain the "parsing" performance boost, in exchange for a performance reduction when accessing trivia.
rintaro
force-pushed
the
rawsyntax-parsedtoken
branch
from
1b2c2ce to
b68716f
Compare
|
@swift-ci Please test |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
A new
RawSyntaxdata variation. This is intended to be used for parsed token. Actual usage will be in follow-ups.In this variation, unlike the existing
MaterializedToken, leading/trailing trivia are stored in aSyntaxTextform, and are lazily materialized by a function registered inSyntaxArena. Since the parser doesn't need to materialize them while parsing, it'd gain the "parsing" performance boost, in exchange for a performance reduction when accessing trivia.