[Parse/Syntax] Refactoring to decouple the parser from syntax tree creation #21368
Conversation
|
I will review this tomorrow. |
|
I just realized that doing a direct |
akyrtzi
force-pushed
the
syntax-parser-decoupling
branch
from
6e0453b to
150b4cb
Compare
|
@swift-ci Please ASAN test |
|
What's the impact on parse time? |
There's no impact on parse time for the compiler because the libSyntax parsing code path is completely disabled (and I've made no changes to the For libSyntax C++ tree creation itself I didn't bother doing measurements because the architectural change for SwiftSyntax will obviate the need to create the C++ syntax tree for SwiftSyntax and make SwiftSyntax multiple times faster. However there are some performance optimizations to make to speed-up the libSyntax parsing path, which I will push for after we adopt the new model for SwiftSyntax. |
|
@swift-ci Please ASAN test |
We enable it for tests, at least, don't we? |
Good point. I measured and it's ~15% regression in parse time for C++ libSyntax. In absolute terms this is +6.5 ms for a 4,120 LOC file. |
|
That is non-wonderful but probably livable, given that parse time remains a small percentage of overall time for fully-compiled tests. |
akyrtzi
force-pushed
the
syntax-parser-decoupling
branch
from
0fb29ff to
cc04c28
Compare
…eation Instead of creating syntax nodes directly, modify the parser to invoke an abstract interface 'SyntaxParseActions' while it is parsing the source code. This decouples the act of parsing from the act of forming a syntax tree representation. 'SyntaxTreeCreator' is an implementation of SyntaxParseActions that handles the logic of creating a syntax tree. To enforce the layering separation of parsing and syntax tree creation, a static library swiftSyntaxParse is introduced to compose the two. This decoupling is important for introducing a syntax parser library for SwiftSyntax to directly access parsing.
…arser with ParsedSyntaxRecorder::make* Doing a "direct ParsedSyntaxRecorder::record[some syntax]" call from the parser is not a good idea due to possibility of being in a backtracking context when the call is made. Replace them with "ParsedSyntaxRecorder::make[some syntax]" which will implicitly check for backtracking and create a recorded or deferred node accordingly.
* Add and improve documentation comments * Adjust formatting * Rename recordExactRawSyntax -> rec.recordRawSyntax
…account for backticks in an identifier
…proper ranges from the input source
Using dumpTokenKind() function instead of getTokenText().
… null using a DataKind Avoid implicitely assuming 'null' node if its OpaqueSyntaxNode is null, there should be no interpretation of OpaqueSyntaxNode values, a SyntaxParseActions implementation should be able to return null pointers as OpaqueSyntaxNode.
…LLVM_COMMON_DEPENDS Since the generated headers are in 'include/swift/Parse' they are globally accessible so make sure they are created early on. If these get moved inside 'lib/Parse' then we can remove this dependency setting.
akyrtzi
force-pushed
the
syntax-parser-decoupling
branch
from
9aca893 to
c929511
Compare
|
@swift-ci smoke test |
Instead of creating syntax nodes directly, modify the parser to invoke an abstract interface 'SyntaxParseActions' while it is parsing the source code.
This decouples the act of parsing from the act of forming a syntax tree representation.
'SyntaxTreeCreator' is an implementation of SyntaxParseActions that handles the logic of creating a syntax tree.
To enforce the layering separation of parsing and syntax tree creation, a static library swiftSyntaxParse is introduced to compose the two.
This decoupling is important for introducing a syntax parser library for SwiftSyntax to directly access parsing.