Skip to content

[benchmark] bump minimum OS versions #66714

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jun 20, 2023
Merged

Conversation

oxy
Copy link
Contributor

@oxy oxy commented Jun 16, 2023

https://ci.swift.org runs Xcode 14.2, and this PR brings the deployment versions in line with the official minimum supported versions.

@oxy
Copy link
Contributor Author

oxy commented Jun 16, 2023

@swift-ci please benchmark

@oxy oxy requested a review from mikeash June 16, 2023 19:58
@oxy
Copy link
Contributor Author

oxy commented Jun 16, 2023

x86_64 run: it turns out a lot of things were made randomly faster / slower by changing the build target, but its unclear how much this matters -

Performance (x86_64): -O

Regression OLD NEW DELTA RATIO
StringWithCString2 0.0 0.002 +200.0% 0.33x (?)
SIMDReduce.Int8x16.Cast 44.844 70.56 +57.3% 0.64x
CxxStringConversion.cxx.to.swift 142.5 181.0 +27.0% 0.79x (?)
BucketSort 86.958 97.522 +12.1% 0.89x (?)
CharIndexing_ascii_unicodeScalars 4175.385 4602.667 +10.2% 0.91x (?)
CharIndexing_korean_unicodeScalars 4816.0 5277.143 +9.6% 0.91x (?)
CharIndexing_japanese_unicodeScalars 5858.182 6403.333 +9.3% 0.91x (?)
CharIndexing_chinese_unicodeScalars 3623.529 3960.0 +9.3% 0.92x (?)
CharIndexing_russian_unicodeScalars 4133.333 4497.143 +8.8% 0.92x (?)
CharIndexing_punctuated_unicodeScalars 950.0 1032.5 +8.7% 0.92x (?)
CharIndexing_punctuatedJapanese_unicodeScalars 884.762 960.755 +8.6% 0.92x (?)
 
Improvement OLD NEW DELTA RATIO
SIMDReduce.Int32x16.Cast 56.167 17.42 -69.0% 3.22x
SIMDReduce.Int32 59.075 20.027 -66.1% 2.95x
SIMDReduce.Int32x4.Cast 56.185 28.641 -49.0% 1.96x
SIMDReduce.Int32x4.Initializer 56.176 34.133 -39.2% 1.65x
SequenceAlgosContiguousArray 2014.545 1603.571 -20.4% 1.26x (?)
SequenceAlgosArray 2011.667 1605.0 -20.2% 1.25x (?)
Dictionary4 188.125 151.5 -19.5% 1.24x (?)
Dictionary4OfObjects 214.333 182.571 -14.8% 1.17x (?)
CharIndexing_japanese_unicodeScalars_Backwards 7928.0 6854.545 -13.5% 1.16x (?)
CharIndexing_chinese_unicodeScalars_Backwards 4760.0 4145.714 -12.9% 1.15x (?)
CharIndexing_punctuatedJapanese_unicodeScalars_Backwards 1184.211 1032.0 -12.9% 1.15x (?)
CharIndexing_korean_unicodeScalars_Backwards 6250.909 5553.333 -11.2% 1.13x (?)
CharIndexing_punctuated_unicodeScalars_Backwards 1201.6 1073.103 -10.7% 1.12x (?)
CharIndexing_russian_unicodeScalars_Backwards 5136.667 4621.333 -10.0% 1.11x (?)
SIMDReduce.Int8x16.Initializer 57.286 51.875 -9.4% 1.10x (?)
FlattenListLoop 1620.0 1471.0 -9.2% 1.10x (?)
CharIndexing_tweet_unicodeScalars_Backwards 10208.889 9364.444 -8.3% 1.09x (?)
CharIndexing_ascii_unicodeScalars_Backwards 5152.0 4738.462 -8.0% 1.09x (?)

Code size: -O

Performance (x86_64): -Osize

Regression OLD NEW DELTA RATIO
NaiveRRC.append.largeContiguous 0.0 0.419 +41900.0% 0.00x (?)
FlattenListLoop 1412.0 1620.0 +14.7% 0.87x (?)
Set.isStrictSubset.Box0 78.438 86.036 +9.7% 0.91x (?)
CharIndexing_russian_unicodeScalars 4694.545 5080.0 +8.2% 0.92x (?)
 
Improvement OLD NEW DELTA RATIO
SIMDReduce.Int32x16.Cast 56.414 17.904 -68.3% 3.15x
Data.init.Sequence.64kB.Count 86.563 29.5 -65.9% 2.93x
Data.init.Sequence.64kB.Count.I 86.571 29.563 -65.9% 2.93x
Data.append.Sequence.64kB.Count.I 87.0 30.828 -64.6% 2.82x
Data.append.Sequence.64kB.Count 87.0 31.385 -63.9% 2.77x
Data.init.Sequence.809B.Count.I 124.7 53.8 -56.9% 2.32x
Data.init.Sequence.809B.Count 124.643 53.941 -56.7% 2.31x
Data.init.Sequence.2049B.Count.I 144.25 62.455 -56.7% 2.31x
Data.init.Sequence.2047B.Count.I 144.091 62.412 -56.7% 2.31x
Data.init.Sequence.513B.Count.I 130.545 66.7 -48.9% 1.96x
Data.init.Sequence.511B.Count.I 126.909 67.294 -47.0% 1.89x
Data.append.Sequence.809B.Count.I 141.615 76.632 -45.9% 1.85x
Data.append.Sequence.809B.Count 141.636 76.647 -45.9% 1.85x
RemoveWhereMoveInts 13.7 7.688 -43.9% 1.78x
SIMDReduce.Int32x4.Cast 56.621 42.704 -24.6% 1.33x
Array.removeAll.keepingCapacity.Object 6.918 5.296 -23.4% 1.31x (?)
StringWalk 1391.724 1110.909 -20.2% 1.25x
CharIteration_tweet_unicodeScalars_Backwards 9377.778 8155.0 -13.0% 1.15x (?)
CharIteration_ascii_unicodeScalars_Backwards 4741.538 4140.0 -12.7% 1.15x
CharIteration_punctuated_unicodeScalars_Backwards 1103.784 979.487 -11.3% 1.13x (?)
RemoveWhereSwapInts 17.588 15.694 -10.8% 1.12x (?)
ObjectiveCBridgeStubNSDateRefAccess 609.667 544.75 -10.6% 1.12x (?)
CharIteration_punctuatedJapanese_unicodeScalars_Backwards 1112.727 1000.0 -10.1% 1.11x (?)
ArrayInClass 217.857 196.328 -9.9% 1.11x (?)
CharIteration_chinese_unicodeScalars_Backwards 4382.857 3960.0 -9.6% 1.11x (?)
CharIteration_korean_unicodeScalars_Backwards 5860.0 5302.857 -9.5% 1.11x (?)
CharIteration_russian_unicodeScalars_Backwards 4772.308 4334.118 -9.2% 1.10x
CharIteration_japanese_unicodeScalars_Backwards 7328.889 6686.154 -8.8% 1.10x (?)
ObjectiveCBridgeStubFromNSStringRef 121.0 112.0 -7.4% 1.08x (?)
FlattenListFlatMap 4141.0 3850.0 -7.0% 1.08x (?)
CharIndexing_chinese_unicodeScalars_Backwards 7340.0 6831.111 -6.9% 1.07x (?)
DataSubscriptMedium 44.409 41.364 -6.9% 1.07x (?)

Code size: -Osize

Performance (x86_64): -Onone

Regression OLD NEW DELTA RATIO
CxxStringConversion.swift.to.cxx 95.0 103.923 +9.4% 0.91x (?)
 
Improvement OLD NEW DELTA RATIO
ArrayAppendGenericStructs 1783.333 1170.0 -34.4% 1.52x (?)
Array.removeAll.keepingCapacity.Object 8.046 6.31 -21.6% 1.28x (?)

Code size: -swiftlibs

How to read the data The tables contain differences in performance which are larger than 8% and differences in code size which are larger than 1%.

If you see any unexpected regressions, you should consider fixing the
regressions before you merge the PR.

Noise: Sometimes the performance results (not code size!) contain false
alarms. Unexpected regressions which are marked with '(?)' are probably noise.
If you see regressions which you cannot explain you can try to run the
benchmarks again. If regressions still show up, please consult with the
performance team (@eeckstein).

Hardware Overview
  Model Name: Mac mini
  Model Identifier: Macmini8,1
  Processor Name: 6-Core Intel Core i7
  Processor Speed: 3.2 GHz
  Number of Processors: 1
  Total Number of Cores: 6
  L2 Cache (per Core): 256 KB
  L3 Cache: 12 MB
  Memory: 32 GB

@oxy
Copy link
Contributor Author

oxy commented Jun 16, 2023

Condensing the above results: it appears that SIMD (or anything vectorizable) got faster on Intel because of finally gaining SSE4 as a minimum requirement (10.9 supported Merom / 65nm Core 2 Duo (no SSE4), while 10.13 bumped minimums to Penryn / 45nm Core 2 Duo (with SSE4)).

@oxy
Copy link
Contributor Author

oxy commented Jun 16, 2023

@swift-ci please smoke test

@oxy
Copy link
Contributor Author

oxy commented Jun 20, 2023

@swift-ci please merge

@oxy oxy merged commit 9b886e5 into swiftlang:main Jun 20, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants