Introduce RawTokenSyntax(kind:, tokenText:, ...) for a convenient way to create a token without any trivia

[akyrtzi]

This allows the parser to consistently use the RawSyntaxData.ParsedToken form for parsed tokens, instead of RawSyntaxData.MaterializedToken.

[akyrtzi]

@swift-ci Please test

[CodaFi]

This looks fine. Curious why we don't just default the two parameters of the existing initializer to []?

[akyrtzi]

This looks fine. Curious why we don't just default the two parameters of the existing initializer to []?

My motivation was to use RawSyntax.parsedToken() for these parsed expressions as well and keep RawSyntax.makeMaterializedToken() for programmatic creations.

Alternatively, in RawSyntax(kind:, text:...) I could check whether leadingTriviaPieces and trailingTriviaPieces are empty and conditionally use RawSyntax.parsedToken() in such a case (otherwise use RawSyntax.makeMaterializedToken()), WDTY?

[CodaFi]

Makes sense. Thanks!

[akyrtzi]

@swift-ci Please test

[akyrtzi]

See updated commit.

[akyrtzi]

Is this good to go? 🙏

[rintaro]

Curious why we want to use only ParsedToken from parser? I failed to see a disadvantage using MaterializedToken.

[akyrtzi]

ParsedToken is a much simpler structure than MaterializedToken (essentially just a text buffer + kind) so way easier to experiment with encoding it in alternative storage, like a content addressable storage system.

[rintaro]

Interesting.

I wondered because we consider the internal structure of RawSyntax an implementation detail, and as long as it's wrapped with RawSyntaxData, the smaller footprint (MemoryLayout.size) wouldn't be a benefit of using ParsedToken.
I'm OK with this change, but please just be aware that the structure might change, in fact we have a plan to add some flags (hasError etc.) to ParsedToken in the (near) future.