Skip to content

[AutoDiff] Add differentiation benchmarks #34901

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Dec 2, 2020

Conversation

rxwei
Copy link
Contributor

@rxwei rxwei commented Dec 1, 2020

Revives #31108 by @marcrasi.

@rxwei rxwei changed the title [AutoDiff] Add differentiation benchmark [AutoDiff] Add differentiation benchmarks Dec 1, 2020
@rxwei
Copy link
Contributor Author

rxwei commented Dec 1, 2020

@swift-ci benchmark

@rxwei rxwei requested a review from marcrasi December 1, 2020 01:16
@rxwei rxwei requested review from atrick and eeckstein December 1, 2020 01:26
Copy link
Contributor

@atrick atrick left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@rxwei
Copy link
Contributor Author

rxwei commented Dec 1, 2020

@swift-ci benchmark

1 similar comment
@rxwei
Copy link
Contributor Author

rxwei commented Dec 1, 2020

@swift-ci benchmark

Copy link
Contributor

@eeckstein eeckstein left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm, though it seems that the benchmarks are crashing during run.

@rxwei
Copy link
Contributor Author

rxwei commented Dec 1, 2020

Interesting... I'm unable to reproduce this on macOS.

@rxwei
Copy link
Contributor Author

rxwei commented Dec 2, 2020

@swift-ci benchmark

@rxwei rxwei force-pushed the differentiation-benchmark branch from ef87562 to 3e86dc9 Compare December 2, 2020 11:07
@rxwei
Copy link
Contributor Author

rxwei commented Dec 2, 2020

@swift-ci benchmark

@rxwei
Copy link
Contributor Author

rxwei commented Dec 2, 2020

@edymtt any concerns with 3e86dc9?

@rxwei rxwei requested a review from edymtt December 2, 2020 11:12
@rxwei
Copy link
Contributor Author

rxwei commented Dec 2, 2020

@swift-ci please test

@swift-ci
Copy link
Contributor

swift-ci commented Dec 2, 2020

Performance: -O

Improvement OLD NEW DELTA RATIO
CharacterLiteralsLarge 97 87 -10.3% 1.11x (?)
UTF8Decode_InitFromBytes_ascii_as_ascii 452 407 -10.0% 1.11x (?)
UTF8Decode_InitFromData_ascii_as_ascii 644 581 -9.8% 1.11x (?)
CharacterLiteralsSmall 305 284 -6.9% 1.07x (?)
 
Added MIN MAX MEAN MAX_RSS
DifferentiationArraySum 1799 2178 1927
DifferentiationIdentity 304 306 305
DifferentiationSquare 590 591 591

Code size: -O

Regression OLD NEW DELTA RATIO
TestsUtils.o 28270 28810 +1.9% 0.98x

Performance: -Osize

Regression OLD NEW DELTA RATIO
Breadcrumbs.MutatedIdxToUTF16.ASCII 3 4 +33.3% 0.75x
NSStringConversion.UTF8 652 770 +18.1% 0.85x (?)
Array2D 6272 6752 +7.7% 0.93x (?)
ObjectiveCBridgeFromNSArrayAnyObjectForced 4220 4540 +7.6% 0.93x (?)
 
Improvement OLD NEW DELTA RATIO
ProtocolDispatch 333 307 -7.8% 1.08x (?)
 
Added MIN MAX MEAN MAX_RSS
DifferentiationArraySum 1681 1862 1743
DifferentiationIdentity 311 312 312
DifferentiationSquare 600 600 600

Code size: -Osize

Regression OLD NEW DELTA RATIO
TestsUtils.o 22409 23197 +3.5% 0.97x

Performance: -Onone

Added MIN MAX MEAN MAX_RSS
DifferentiationArraySum 1673 1887 1745
DifferentiationIdentity 380 380 380
DifferentiationSquare 703 704 703

Code size: -swiftlibs

Benchmark Check Report
⚠️ DifferentiationArraySum execution took at least 1653 μs.
Decrease the workload of DifferentiationArraySum by a factor of 2 (10), to be less than 1000 μs.
How to read the data The tables contain differences in performance which are larger than 8% and differences in code size which are larger than 1%.

If you see any unexpected regressions, you should consider fixing the
regressions before you merge the PR.

Noise: Sometimes the performance results (not code size!) contain false
alarms. Unexpected regressions which are marked with '(?)' are probably noise.
If you see regressions which you cannot explain you can try to run the
benchmarks again. If regressions still show up, please consult with the
performance team (@eeckstein).

Hardware Overview
  Model Name: Mac Pro
  Model Identifier: MacPro6,1
  Processor Name: 8-Core Intel Xeon E5
  Processor Speed: 3 GHz
  Number of Processors: 1
  Total Number of Cores: 8
  L2 Cache (per Core): 256 KB
  L3 Cache: 25 MB
  Memory: 64 GB

@rxwei rxwei merged commit b8581cb into swiftlang:main Dec 2, 2020
ainu-bot pushed a commit to google/swift that referenced this pull request Dec 2, 2020
* 'main' of github.com:apple/swift:
  [AutoDiff] Add differentiation benchmarks (swiftlang#34901)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants