Move SyntaxClassifier to a separate module

[ahoppen]

The goal of this PR is to move SyntaxClassifier out of the SwiftSyntax module into its own dedicated. I’m not really happy with the PR yet because SyntaxClassifier uses RawSyntax pretty extensively and this requires us to expose more methods from SwiftSyntax as SPI. AFAICT the options here are:

• Leave the PR as-is, accepting that SyntaxClassifier requires the use of SPI
• Rewrite SyntaxClassifier in terms of Syntax nodes? @rintaro I think you tried this and it wasn’t performant enough, right?
• Expose RawSyntax as UnsafeSyntax (or something like that) instead of as SPI
• Leave SyntaxClassifier in the SwiftSyntax module

I thought I set up this PR as a base for discussion.

rdar://98318240

[rintaro]

Rewrite SyntaxClassifier in terms of Syntax nodes? @rintaro I think you tried this and it wasn’t performant enough, right?

I actually didn't try this. Maybe it's interning to try it, but I'm totally OK with using SPI RawSyntax here.

Expose RawSyntax as UnsafeSyntax (or something like that) instead of as SPI
Leave SyntaxClassifier in the SwiftSyntax module

I strongly prefer keeping RawSyntax a SPI. SPI is not a big deal, it's just a user side declaration "yes, I import extra types/functions knowing what I am doing".

But I don't think we want to expose SyntaxData even as SPI. When we want to expose any SyntaxData API, we can do that by implementing Syntax API, and it doesn't cost much.

[rintaro]

I think SyntaxData.contextualClassification doesn't need to be SyntaxData, could you rewrite it to Syntax.contextualClassification ?

[rintaro]

As far as the type itself is marked as @_spi(Name), those members doesn't need to be @_spi(Name). We can just make them public, without @_spi. (I know other members e.g. var kind has it).

[rintaro]

Let's avoid exposing SyntaxData by rewriting SyntaxData.contextualClassification to Syntax.contextualClassification.

[ahoppen]

I updated the PR to not use SyntaxData. The main thing I had to expose were RawSyntaxTokenView and RawSyntaxLayoutView but that doesn’t seem too bad.

There is a small performance penalty associated with the move (I believe because we can’t do whole module optimization anymore), but I think it’s acceptable (measurements from SyntaxClassifierPerformanceTests):

Release

before: average: 0.045, relative standard deviation: 5.368%, values: [0.050073, 0.046481, 0.049106, 0.044456, 0.043300, 0.043836, 0.043944, 0.046100, 0.043229, 0.042674], performanceMetricID:com.apple.XCTPerformanceMetric_WallClockTime, baselineName: "", baselineAverage: , polarity: prefers smaller, maxPercentRegression: 10.000%, maxPercentRelativeStandardDeviation: 10.000%, maxRegression: 0.100, maxStandardDeviation: 0.100
after:  average: 0.056, relative standard deviation: 6.728%, values: [0.062602, 0.057758, 0.059577, 0.053642, 0.053726, 0.053434, 0.057409, 0.057317, 0.050950, 0.050048], performanceMetricID:com.apple.XCTPerformanceMetric_WallClockTime, baselineName: "", baselineAverage: , polarity: prefers smaller, maxPercentRegression: 10.000%, maxPercentRelativeStandardDeviation: 10.000%, maxRegression: 0.100, maxStandardDeviation: 0.100

Debug

before: average: 2.362, relative standard deviation: 0.646%, values: [2.378401, 2.367227, 2.385291, 2.366417, 2.329441, 2.369335, 2.363841, 2.345538, 2.362924, 2.353043]
after:  average: 2.400, relative standard deviation: 1.438%, values: [2.403036, 2.485006, 2.400218, 2.375242, 2.381436, 2.373605, 2.387524, 2.437035, 2.395524, 2.361710]

[ahoppen]

@swift-ci Please test

[ahoppen]

@swift-ci Please test

[rintaro]

LGTM!

[ahoppen]

@swift-ci Please test