Adjust batch partition count calculation #51
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This mirrors the batch partition calculation change in swiftlang/swift#28972. See below for an explanation.
While exploring how to optimize global build times, I noticed that the batch sizing logic wasn't producing multiple batches until a module had 50 files. Based on @graydon's detailed batch calculation comment (which says "cap the batch size ... at ... 25"), and based on the variable names ("
SizeLimit
"), I believe the batch calculation implementation is off.The simple example is a module of 26 files. With the existing batch calculation, the result is a
26/25=1
batch. However this is larger than theDefaultSizeLimit
. This calculation continues to yield1
for modules up to 49 files. Only at modules of 50 files does the batch calculation produce two batches.Here's a table showing the effects of the current partition count calculation:
This table shows that the max batch size is inversely proportional with the number of files. Smaller modules have a higher max batch size, and as a module gets larger, its max batch size trends toward 25. It also shows that after 50, the minimum batch size is consistently 25.
With this proposed change to the batch count calculation, the math is now:
This table shows that the batch size is in fact capped to 25, independent of how many files are in the module. It also show the minimum batch size increases with module size.