Formalize some SourceFile parsing outputs #32161
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
And fix the name of the underlying vector.
Now that SIL files no longer interleave parsing with type-checking, the query doesn't make much sense. Inline it into its only client, `shouldBuildSyntaxTree`.
Sink the `BuildSyntaxTree` and `CollectParsedTokens` bits into `SourceFile::ParsingFlags`, with a static method to get the parsing options from the lang opts. Also add a parsing flag for enabling the interface hash, which can be used instead of calling `enableInterfaceHash`.
The `SaveAndRestore` is unnecessary as `Parser`'s constructor already sets up the interface hash, and the request covers the `FrontendStatsTracer`.
Rename `Bag` to `Tokens`, and query the SourceManager from the ASTContext instead of storing it directly.
Currently when parsing a SourceFile, the parser gets handed pointers so that it can write the interface hash and collected tokens directly into the file. It can also call `setSyntaxRoot` at the end of parsing to set the syntax tree. In preparation for the removal of `performParseOnly`, this commit formalizes these values as outputs of `ParseSourceFileRequest`, ensuring that the file gets parsed when the interface hash, collected tokens, or syntax tree is queried.
This is temporarily swapped in as the token receiver while backtracking, so make sure we don't try to call `finalize` on it.
|
@swift-ci please test |
|
@swift-ci please test source compatibility |
hamishknight
requested review from
rintaro and
CodaFi
and removed request for
rintaro
CodaFi
reviewed
Jun 3, 2020
| /// Retrieve the token receiver from the parser once it has finished parsing. | ||
| std::unique_ptr<ConsumeTokenReceiver> takeTokenReceiver() { | ||
| assert(Tok.is(tok::eof) && "not done parsing yet"); | ||
| auto *receiver = TokReceiver; |
There was a problem hiding this comment.
@rintaro Does it make sense to model TokReceiver as a std::unique_ptr as well? It would make the ownership transfer here a little cleaner to express.
There was a problem hiding this comment.
I was thinking about that too, unfortunately we currently swap in a pointer to a DelayedTokenReceiver on the stack while backtracking.
CodaFi
reviewed
Jun 3, 2020
CodaFi
approved these changes
Jun 3, 2020
A previous commit inadvertently changed the logic such that the member hash of an extension body would be set to a partial interface hash. Luckily this shouldn't have caused any behavioural change as the interface hash itself would have been left unaffected. This commit makes sure we preserve the original behaviour where if we don't have the body tokens hashed separately, we give the body hash a default constructed MD5. Noticed by inspection.
Use `shouldBuildSyntaxTree` instead.
|
@swift-ci please test |
|
Merging so I can make further progress – @rintaro I'm more than happy to address any review feedback in a follow-up. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Currently when parsing a
SourceFile, the parser gets handed pointers so that it can write the interface hash and collected tokens directly into the file. It can also callsetSyntaxRootat the end of parsing to set the syntax tree.In preparation for the removal of
performParseOnly, this PR formalizes these values as outputs ofParseSourceFileRequest, ensuring that the file gets parsed when the interface hash, collected tokens, or syntax tree is queried.In addition, it cleans up
SourceFilea little by sinking a couple of parsing options intoSourceFile::ParsingFlags.