Skip to content

[Parser] Decouple the parser from AST creation (part 2) #26478

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 22 commits into from
Aug 27, 2019
Merged

[Parser] Decouple the parser from AST creation (part 2) #26478

merged 22 commits into from
Aug 27, 2019

Conversation

jansvoboda11
Copy link
Contributor

Instead of creating the AST directly in the parser (and libSyntax or SwiftSyntax via SyntaxParsingContext), make Parser to explicitly create a tree of ParsedSyntaxNodes. Their OpaqueSyntaxNodes can be either libSyntax or SwiftSyntax. If AST is needed, it can be generated from the libSyntax tree.
This PR contains refactored type parsing.

This PR is a part of a Google Summer of Code 2019 project.

@rintaro rintaro self-assigned this Aug 5, 2019
rintaro
rintaro reviewed Aug 6, 2019
View reviewed changes
@rintaro
Copy link
Member

rintaro commented Aug 13, 2019

@swift-ci Please test

@swift-ci
Copy link
Contributor

Build failed
Swift Test Linux Platform
Git Sha - 1ccc21c176c39ea7e02129479dbf88338d3304c1

@swift-ci
Copy link
Contributor

Build failed
Swift Test OS X Platform
Git Sha - 1ccc21c176c39ea7e02129479dbf88338d3304c1

@rintaro
Copy link
Member

rintaro commented Aug 15, 2019

@swift-ci Please test

@swift-ci
Copy link
Contributor

Build failed
Swift Test Linux Platform
Git Sha - 1ccc21c176c39ea7e02129479dbf88338d3304c1

@swift-ci
Copy link
Contributor

Build failed
Swift Test OS X Platform
Git Sha - 1ccc21c176c39ea7e02129479dbf88338d3304c1

@rintaro
Copy link
Member

rintaro commented Aug 16, 2019

@swift-ci Please test

@rintaro
Copy link
Member

rintaro commented Aug 16, 2019 •
edited
Loading

@jansvoboda11 I found that parsing the following source causes memory leak.

var foo: Bar<Baz,

Tested with (in Linux)

$ cd swift
$ cat test.swift
var foo: Bar<Baz,
$ utils/build-script -R --enable-lsan --debug-swift-stdlib --skip-build-benchmarks
$ ../build/Ninja-ReleaseAssert+stdlib-DebugAssert/swift-linux-x86_64/bin/swift -frontend -parse test.swift

Could you look into it? It seems that the parsed GenericArgumentSyntax (Baz,) is just discarded without Release().

@swift-ci
Copy link
Contributor

Build failed
Swift Test Linux Platform
Git Sha - 0cc0f7a

@rintaro
Copy link
Member

rintaro commented Aug 16, 2019

Other leaking cases:

var foo: Bar<
let foo: Bar[{}
let foo: Bar &

@jansvoboda11 jansvoboda11 mentioned this pull request Aug 20, 2019
Closed
@rintaro
Copy link
Member

rintaro commented Aug 23, 2019

@swift-ci Please test

@swift-ci
Copy link
Contributor

Build failed
Swift Test OS X Platform
Git Sha - 0cc0f7a

@swift-ci
Copy link
Contributor

Build failed
Swift Test Linux Platform
Git Sha - a058d4256f5cb571368f031a43e6792b45b861bd

Jan Svoboda and others added 14 commits August 26, 2019 19:10
[Parser] Decouple the parser from AST creation (part 2)
77924c4 
Instead of creating the AST directly in the parser (and libSyntax or
SwiftSyntax via SyntaxParsingContext), make Parser to explicitly create
a tree of ParsedSyntaxNodes. Their OpaqueSyntaxNodes can be either
libSyntax or SwiftSyntax. If AST is needed, it can be generated from the
libSyntax tree.
Use SourceLoc as key in type map
a705127
Update ASTGen documentation.
773cf51
Handle type parsing errors properly
cc1aa01
Consume any token in identifier parsing
0135503
Use correct SourceLoc (before trivia) for Any type parsing
6de2764
Allow nullptr as output of ASTGen
6acfc8b
Fix attribute @ SourceLoc in ASTGen
8b5ca75
Handle failed tuple parsing
9ae61aa
@rintaro
[SyntaxParse] Use ArrayRef to pass list of nodes to ParsedSyntaxResult
52d999d
@rintaro
[SyntaxParse] ParseError is not equivalent to code completion
80e6d27
@rintaro
[SyntaxParse] Support kindof() in isTopNode()
1551db6
@rintaro
[SyntaxParse] Fix memory leak in incomplete composition parsing
c8ac6b6
@rintaro
[SyntaxParse] Fix memory leak
dc0cf99
@rintaro
Copy link
Member

rintaro commented Aug 26, 2019 •
edited
Loading

Linux failure is unrelated
@jansvoboda11 Mind resolving conflicts?

@jansvoboda11
Copy link
Contributor Author

Working on in right now.

@rintaro
Copy link
Member

rintaro commented Aug 26, 2019

@swift-ci Please test

@swift-ci
Copy link
Contributor

Build failed
Swift Test Linux Platform
Git Sha - a058d4256f5cb571368f031a43e6792b45b861bd

@swift-ci
Copy link
Contributor

Build failed
Swift Test OS X Platform
Git Sha - a058d4256f5cb571368f031a43e6792b45b861bd

@rintaro
Copy link
Member

rintaro commented Aug 26, 2019

@swift-ci Please test

@swift-ci
Copy link
Contributor

Build failed
Swift Test Linux Platform
Git Sha - 02a323e

@swift-ci
Copy link
Contributor

Build failed
Swift Test OS X Platform
Git Sha - 02a323e

@rintaro rintaro merged commit e56743e into swiftlang:master Aug 27, 2019
@rintaro rintaro mentioned this pull request Aug 27, 2019
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@rintaro rintaro rintaro left review comments

Assignees

@rintaro rintaro

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@jansvoboda11 @rintaro @swift-ci