Re-enable parse_stdlib tests. #12700
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
For very large source files, the parser's syntax map---which contains a very large number of TrivialLists---was taking an inordinate amount of memory due to the inefficiency of std::deque. Specifically, a std::deque containing just one trivial element would allocate 4k of memory. With the ~120MB SIL output of one of the parse_stdlib tests, these std::deques would add up to > 6GB of memory, most of which is wasted. Replacing the std::deque with a std::vector knocks the memory required for one of the parse_stdlib tests from > 8GB down closer to 2 GB. The parser's syntax map is still large (e.g., a 512MB allocation for the overall vector plus a few hundred MB of raw-syntax data), but not prohibitively so. Part of rdar://problem/34771322.
When printing SIL, we end up printing both @NoEscape and @autoclosure. It's not technically wrong (just redundant), so allow it when parsing the SIL back.
Now that memory usage is (more) under control, re-enable the parse_stdlib tests. Fixes rdar://problem/34771322.
|
@swift-ci please test and merge |
|
Still failing on Linux. Reverting! |
|
Trying to bring this back online for just macOS via #12706 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The
parse_stdlibtests have been disabled for a while due to excessive memory use. At one point, the problem appeared to be theGenericSignatureBuilder. However, now the problem was actually in the parser's syntax map. Address that issue and try to re-enable the tests.For very large source files, the parser's syntax map---which contains a
very large number of
TrivialLists---was taking an inordinate amount ofmemory due to the inefficiency of
std::deque. Specifically, astd::dequecontaining just one trivia element would allocate 4k ofmemory. With the ~120MB SIL output of one of the parse_stdlib tests,
these
std::deques would add up to > 6GB of memory, most of which iswasted.
Replacing the
std::dequewith astd::vectorknocks the memory requiredfor one of the parse_stdlib tests from > 8GB down closer to 2 GB. The
parser's syntax map is still large (e.g., a 512MB allocation for the
overall
std::vectorplus a few hundred MB of raw-syntax data), but notprohibitively so.
Addresses rdar://problem/34771322