Load semantic tokens from a document using a sourcekitd request instead of the 0,0 edit #912
Conversation
…ad of the 0,0 edit This allows us to cancel the semantic tokens request. It is also the last step to allow us to open and edit documents in syntactic-only mode. It also means that we no longer need to send a `WorkspaceSemanticTokensRefreshRequest` to the client after sourcekitd has updated its semantic tokens since with the new design the `DocumentSemanticTokensRangeRequest` will simply return the results once it has the updated semantic token.
|
@swift-ci Please test |
With this change, opening and editing a document no longer causes a non-cancellable AST build. Instead, all semantic information is retrieved via requests that can be cancelled (after we implement cancellation).
ahoppen
force-pushed
the
ahoppen/semantic-tokens-request
branch
from
3a431bd to
ffbf025
Compare
| var tokens: [SyntaxHighlightingToken] = [] | ||
| tokenParser.parseTokens(skTokens, in: snapshot, into: &tokens) |
There was a problem hiding this comment.
I assume there was (maybe still is?) some reason for this, but could parseTokens just return an array instead?
There was a problem hiding this comment.
parseTokens calls recursively into itself for substructures. and it’s easier + more efficient to accumulate tokens in an inout Array. At least, that’s what I think what the reasoning was.
There was a problem hiding this comment.
That could be internal to SyntaxHighlightingToken, I don't see why callers have to care about that.
There was a problem hiding this comment.
Since there’s only one caller, I didn’t care too much. But I made the inout functions private and added an internal entry point that returns the tokens.
|
@swift-ci Please test |
… parameter private This simlifies the callers of `SyntaxHighlightingTokenParser`.
|
@swift-ci Please test |
|
@swift-ci Please test Windows |
This allows us to cancel the semantic tokens request. It is also the last step to allow us to open and edit documents in syntactic-only mode.
It also means that we no longer need to send a
WorkspaceSemanticTokensRefreshRequestto the client after sourcekitd has updated its semantic tokens since with the new design theDocumentSemanticTokensRangeRequestwill simply return the results once it has the updated semantic token.