Skip to content

Enable run-time exclusivity checking in release mode. #20302

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Nov 7, 2018
Merged

Enable run-time exclusivity checking in release mode. #20302

merged 1 commit into from
Nov 7, 2018

Conversation

atrick
Copy link
Contributor

@atrick atrick commented Nov 3, 2018

It's time to change the compiler default for run-time exclusivity
checks. This will expose the feature to more testing and allow the
compiler team to gather performance feedback from users. Until now,
I've been waiting until the most obvious bottlenecks are
resolved--there's no sense having a slew of performance bugs for
issues that are about to be fixed. Recent optimizations landed by Joe
Shajrawi have brought performance where I think it needs to be for
adoption. More optimizations are planned, and some benchmarks should
be further improved, but at this point we're ready to begin receiving
bug reports. That will help prioritize the remaining work for Swift 5.

I'll post to swift-announce shortly.

Fixes rdar://problem/37830912 [SR-7139]: [Exclusivity] Enable -exclusivity-enforcement=checked (dynamic checks) by default in release (-O) mode.

This change could impact Swift programs that previously appeared
well-behaved, but weren't fully tested in debug mode. Now, when running
in release mode, they may trap with the message "error: overlapping
accesses...".

Recent optimizations have brought performance where I think it needs
to be for adoption. More optimizations are planned, and some
benchmarks should be further improved, but at this point we're ready
to begin receiving bug reports. That will help prioritize the
remaining work for Swift 5.

Of the 656 public microbenchmarks in the Swift repository, there are
still several regressions larger than 10%:

TEST                    OLD      NEW      DELTA      RATIO
ClassArrayGetter2       139      1307     +840.3%    **0.11x**
HashTest                631      1233     +95.4%     **0.51x**
NopDeinit               21269    32389    +52.3%     **0.66x**
Hanoi                   1478     2166     +46.5%     **0.68x**
Calculator              127      158      +24.4%     **0.80x**
Dictionary3OfObjects    391      455      +16.4%     **0.86x**
CSVParsingAltIndices2   526      604      +14.8%     **0.87x**
Prims                   549      626      +14.0%     **0.88x**
CSVParsingAlt2          1252     1411     +12.7%     **0.89x**
Dictionary4OfObjects    206      232      +12.6%     **0.89x**
ArrayInClass            46       51       +10.9%     **0.90x**

The common pattern in these benchmarks is to define an array of data
as a class property and to repeatedly access that array through the
class reference. Each of those class property accesses now incurs a
runtime call. Naturally, introducing a runtime call in a loop that
otherwise does almost no work incurs substantial overhead. This is
similar to the issue caused by automatic reference counting. In some
cases, more sophistacated optimization will be able to determine the
same object is repeatedly accessed. Furthermore, the overhead of the
runtime call itself can be improved. But regardless of how well we
optimize, there will always a class of microbenchmarks in which the
runtime check has a noticeable impact.

As a general guideline, avoid performing class property access within
the most performance critical loops, particularly on different objects
in each loop iteration. If that isn't possible, it may help if the
visibility of those class properties is private or internal.

This change could impact Swift programs that previously appeared
well-behaved, but weren't fully tested in debug mode. Now, when running
in release mode, they may trap with the message "error: overlapping
accesses...".

Recent optimizations have brought performance where I think it needs
to be for adoption. More optimizations are planned, and some
benchmarks should be further improved, but at this point we're ready
to begin receiving bug reports. That will help prioritize the
remaining work for Swift 5.

Of the 656 public microbenchmarks in the Swift repository, there are
still several regressions larger than 10%:

TEST                    OLD      NEW      DELTA      RATIO
ClassArrayGetter2       139      1307     +840.3%    **0.11x**
HashTest                631      1233     +95.4%     **0.51x**
NopDeinit               21269    32389    +52.3%     **0.66x**
Hanoi                   1478     2166     +46.5%     **0.68x**
Calculator              127      158      +24.4%     **0.80x**
Dictionary3OfObjects    391      455      +16.4%     **0.86x**
CSVParsingAltIndices2   526      604      +14.8%     **0.87x**
Prims                   549      626      +14.0%     **0.88x**
CSVParsingAlt2          1252     1411     +12.7%     **0.89x**
Dictionary4OfObjects    206      232      +12.6%     **0.89x**
ArrayInClass            46       51       +10.9%     **0.90x**

The common pattern in these benchmarks is to define an array of data
as a class property and to repeatedly access that array through the
class reference. Each of those class property accesses now incurs a
runtime call. Naturally, introducing a runtime call in a loop that
otherwise does almost no work incurs substantial overhead. This is
similar to the issue caused by automatic reference counting. In some
cases, more sophistacated optimization will be able to determine the
same object is repeatedly accessed. Furthermore, the overhead of the
runtime call itself can be improved. But regardless of how well we
optimize, there will always a class of microbenchmarks in which the
runtime check has a noticeable impact.

As a general guideline, avoid performing class property access within
the most performance critical loops, particularly on different objects
in each loop iteration. If that isn't possible, it may help if the
visibility of those class properties is private or internal.
@atrick
Copy link
Contributor Author

atrick commented Nov 3, 2018

@swift-ci test.

@atrick
Copy link
Contributor Author

atrick commented Nov 3, 2018

@swift-ci test source compatibility.

@atrick
Copy link
Contributor Author

atrick commented Nov 3, 2018

@swift-ci benchmark.

@swift-ci
Copy link
Contributor

swift-ci commented Nov 3, 2018

Build comment file:

Performance: -O

TEST OLD NEW DELTA RATIO
Regression
ClassArrayGetter2 106 1642 +1449.0% 0.06x
HashTest 850 1709 +101.1% 0.50x
Hanoi 1981 3204 +61.7% 0.62x
NopDeinit 27010 41598 +54.0% 0.65x
ExclusivityGlobal 5 7 +40.0% 0.71x
PrimsSplit 690 831 +20.4% 0.83x
Prims 687 824 +19.9% 0.83x
CSVParsingAltIndices2 676 783 +15.8% 0.86x
Calculator 188 217 +15.4% 0.87x
CSVParsingAlt2 1561 1797 +15.1% 0.87x
Dictionary4OfObjects 275 313 +13.8% 0.88x
FlattenListLoop 3555 3884 +9.3% 0.92x
Array2D 6202 6735 +8.6% 0.92x
MapReduceAnyCollection 331 357 +7.9% 0.93x
Improvement
SetSymmetricDifferenceBox25 528 457 -13.4% 1.16x
SetSymmetricDifferenceBox0 655 594 -9.3% 1.10x
SetExclusiveOr_OfObjects 6548 5949 -9.1% 1.10x

Code size: -O

TEST OLD NEW DELTA RATIO
Regression
Exclusivity.o 4083 4987 +22.1% 0.82x
main.o 47234 54066 +14.5% 0.87x
DictionaryRemove.o 15746 17642 +12.0% 0.89x
SetTests.o 59613 65621 +10.1% 0.91x
Hash.o 29123 31543 +8.3% 0.92x
DictTest3.o 28932 31228 +7.9% 0.93x
DictTest.o 51659 53848 +4.2% 0.96x
ObjectAllocation.o 4635 4779 +3.1% 0.97x
COWTree.o 14324 14756 +3.0% 0.97x
DictionaryOfAnyHashableStrings.o 10421 10733 +3.0% 0.97x
DictionarySwap.o 28502 29355 +3.0% 0.97x
Prims.o 41873 43001 +2.7% 0.97x
PrimsSplit.o 41925 43053 +2.7% 0.97x
CharacterProperties.o 27545 28217 +2.4% 0.98x
DictTest2.o 20300 20729 +2.1% 0.98x
Array2D.o 4064 4144 +2.0% 0.98x
DictionaryBridge.o 3634 3698 +1.8% 0.98x
RangeIteration.o 1976 2008 +1.6% 0.98x
Combos.o 18237 18493 +1.4% 0.99x
ObserverForwarderStruct.o 3458 3506 +1.4% 0.99x
DriverUtils.o 188783 191255 +1.3% 0.99x
Improvement
ClassArrayGetter.o 6080 5472 -10.0% 1.11x
NopDeinit.o 5907 5443 -7.9% 1.09x
ArrayOfRef.o 12792 11856 -7.3% 1.08x
ArrayOfGenericRef.o 15948 14972 -6.1% 1.07x
ArrayInClass.o 1453 1381 -5.0% 1.05x
RGBHistogram.o 25221 24141 -4.3% 1.04x
ObserverClosure.o 3303 3191 -3.4% 1.04x
ObserverPartiallyAppliedMethod.o 3559 3479 -2.2% 1.02x
Radix2CooleyTukey.o 5127 5022 -2.0% 1.02x
DeadArray.o 1872 1840 -1.7% 1.02x
DictTest4Legacy.o 27864 27496 -1.3% 1.01x
CSVParsing.o 55635 54947 -1.2% 1.01x
DictionaryGroup.o 17391 17210 -1.0% 1.01x

Performance: -Osize

TEST OLD NEW DELTA RATIO
Regression
ClassArrayGetter2 111 1646 +1382.9% 0.07x
HashTest 871 1760 +102.1% 0.49x
Dictionary4OfObjects 297 487 +64.0% 0.61x
NopDeinit 27010 42952 +59.0% 0.63x
Hanoi 2079 3247 +56.2% 0.64x
SetIsSubsetBox25 152 229 +50.7% 0.66x
ExclusivityGlobal 5 7 +40.0% 0.71x
CSVParsingAltIndices2 700 876 +25.1% 0.80x
RGBHistogramOfObjects 16345 20168 +23.4% 0.81x
CSVParsingAlt2 1598 1933 +21.0% 0.83x
DictionarySwapOfObjects 6974 8361 +19.9% 0.83x
ObjectAllocation 132 157 +18.9% 0.84x
PrimsSplit 770 901 +17.0% 0.85x
RangeIterationSigned 153 179 +17.0% 0.85x
Prims 768 893 +16.3% 0.86x
ArrayOfPOD 158 182 +15.2% 0.87x
SetSubtractingBox0 142 161 +13.4% 0.88x
SetIntersectionBox25 256 289 +12.9% 0.89x
SetUnionBox25 342 386 +12.9% 0.89x
DictionarySubscriptDefaultMutationOfObjects 1342 1500 +11.8% 0.89x
SetIntersectionBox0 134 149 +11.2% 0.90x
DictionarySwapAtOfObjects 6423 7064 +10.0% 0.91x
ArrayOfGenericPOD2 130 141 +8.5% 0.92x
SetSubtractingBox25 265 286 +7.9% 0.93x
SetUnion_OfObjects 5239 5654 +7.9% 0.93x
SetUnionBox0 524 565 +7.8% 0.93x

Code size: -Osize

TEST OLD NEW DELTA RATIO
Regression
Exclusivity.o 3815 4751 +24.5% 0.80x
ArrayOfPOD.o 2351 2735 +16.3% 0.86x
main.o 44441 51289 +15.4% 0.87x
DictionaryRemove.o 14479 16199 +11.9% 0.89x
SetTests.o 52485 58677 +11.8% 0.89x
DictionarySwap.o 25771 28291 +9.8% 0.91x
ArrayOfGenericPOD.o 2778 2970 +6.9% 0.94x
DictTest3.o 23094 24526 +6.2% 0.94x
Hash.o 22127 23383 +5.7% 0.95x
DictionarySubscriptDefault.o 26743 28143 +5.2% 0.95x
RGBHistogram.o 23053 24213 +5.0% 0.95x
DictTest4Legacy.o 23033 24089 +4.6% 0.96x
DictTest2.o 16518 17150 +3.8% 0.96x
DictTest.o 48901 50429 +3.1% 0.97x
COWTree.o 13442 13842 +3.0% 0.97x
DictionaryOfAnyHashableStrings.o 9997 10293 +3.0% 0.97x
CharacterProperties.o 25281 25937 +2.6% 0.97x
ObserverForwarderStruct.o 3654 3734 +2.2% 0.98x
ObjectAllocation.o 4505 4601 +2.1% 0.98x
Array2D.o 4227 4307 +1.9% 0.98x
ObserverClosure.o 3437 3501 +1.9% 0.98x
DictionaryGroup.o 15959 16255 +1.9% 0.98x
ObserverUnappliedMethod.o 5371 5467 +1.8% 0.98x
NopDeinit.o 5621 5717 +1.7% 0.98x
ObserverPartiallyAppliedMethod.o 3759 3823 +1.7% 0.98x
LinkedList.o 2124 2156 +1.5% 0.99x
Combos.o 16581 16821 +1.4% 0.99x
Hanoi.o 3594 3642 +1.3% 0.99x
RecursiveOwnedParameter.o 1277 1293 +1.3% 0.99x
DictTest4.o 21247 21511 +1.2% 0.99x
DriverUtils.o 161659 163499 +1.1% 0.99x
Improvement
ClassArrayGetter.o 5881 5353 -9.0% 1.10x
ArrayOfRef.o 13608 12648 -7.1% 1.08x
ArrayOfGenericRef.o 14524 13844 -4.7% 1.05x
Radix2CooleyTukey.o 4836 4724 -2.3% 1.02x
CSVParsing.o 50483 49715 -1.5% 1.02x
ArrayInClass.o 1648 1624 -1.5% 1.01x
How to read the data The tables contain differences in performance which are larger than 8% and differences in code size which are larger than 1%.

If you see any unexpected regressions, you should consider fixing the regressions before you merge the PR.

Noise: Sometimes the performance results (not code size!) contain false alarms. Unexpected regressions which are marked with '(?)' are probably noise. If you see regressions which you cannot explain you can try to run the benchmarks again. If regressions still show up, please consult with the performance team (@eeckstein).

Hardware Overview
  Model Name: Mac Pro
  Model Identifier: MacPro6,1
  Processor Name: 8-Core Intel Xeon E5
  Processor Speed: 3 GHz
  Number of Processors: 1
  Total Number of Cores: 8
  L2 Cache (per Core): 256 KB
  L3 Cache: 25 MB
  Memory: 64 GB

@swift-ci
Copy link
Contributor

swift-ci commented Nov 3, 2018

Build failed
Swift Test Linux Platform
Git Sha - a17dbc7

@atrick
Copy link
Contributor Author

atrick commented Nov 4, 2018

@swift-ci smoke test linux.

@atrick
Copy link
Contributor Author

atrick commented Nov 5, 2018

I'm seeing the same linux crash on master without these changes:

Test project /home/docker_user/src/s/build/buildbot_incremental/foundation-linux-x86_64
    Start 1: TestFoundation
1/1 Test #1: TestFoundation ...................***Exception: Illegal  0.18 sec

0% tests passed, 1 tests failed out of 1

Total Test time (real) =   0.20 sec

The following tests FAILED:
	  1 - TestFoundation (ILLEGAL)
Errors while running CTest
FAILED: CMakeFiles/test.util 

It's not clear how other PR tests have passed.

@atrick
Copy link
Contributor Author

atrick commented Nov 5, 2018

@swift-ci smoke test linux.

@gottesmm
Copy link
Contributor

gottesmm commented Nov 5, 2018

@atrick #20314 should make the errors get printed.

@gottesmm
Copy link
Contributor

gottesmm commented Nov 5, 2018

@swift-ci smoke test linux platform

@gottesmm
Copy link
Contributor

gottesmm commented Nov 5, 2018

@atrick I restarted the smoke test linux job to make sure you got that commit so you can see the error.

@atrick
Copy link
Contributor Author

atrick commented Nov 5, 2018

@swift-ci clean test linux platform.

@swift-ci
Copy link
Contributor

swift-ci commented Nov 6, 2018

Build failed
Swift Test Linux Platform
Git Sha - a17dbc7

@gottesmm
Copy link
Contributor

gottesmm commented Nov 6, 2018

@swift-ci smoke test linux platform

@gottesmm
Copy link
Contributor

gottesmm commented Nov 6, 2018

@swift-ci smoke test Linux platform

@atrick
Copy link
Contributor Author

atrick commented Nov 7, 2018

@swift-ci smoke test.

@atrick atrick merged commit 945ee92 into swiftlang:master Nov 7, 2018
@gottesmm
Copy link
Contributor

gottesmm commented Nov 7, 2018

Congrats @atrick!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants