Skip to content

[benchmark] Package.swift: Don’t trap on stray files under single-source/ #30601

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Apr 14, 2020

Conversation

lorentey
Copy link
Member

Make benchmark/Package.swift more robust against stray files (such as .DS_store) under benchmark/single-source/ etc.

Currently it traps with an Array index bounds error, leading to a difficult-to-debug build issue.

@lorentey lorentey requested a review from gottesmm March 24, 2020 01:20
@lorentey
Copy link
Member Author

@swift-ci smoke test

@lorentey
Copy link
Member Author

@swift-ci benchmark

@swift-ci
Copy link
Contributor

Performance: -O

Improvement OLD NEW DELTA RATIO
NSStringConversion.MutableCopy.UTF8 845 777 -8.0% 1.09x (?)
NSStringConversion.MutableCopy.Rebridge.UTF8 727 677 -6.9% 1.07x (?)

Code size: -O

Performance: -Osize

Code size: -Osize

Performance: -Onone

Improvement OLD NEW DELTA RATIO
ObjectiveCBridgeStubToNSDate2 1140 890 -21.9% 1.28x (?)
DictionaryOfAnyHashableStrings_lookup 5064 4416 -12.8% 1.15x (?)

Code size: -swiftlibs

How to read the data The tables contain differences in performance which are larger than 8% and differences in code size which are larger than 1%.

If you see any unexpected regressions, you should consider fixing the
regressions before you merge the PR.

Noise: Sometimes the performance results (not code size!) contain false
alarms. Unexpected regressions which are marked with '(?)' are probably noise.
If you see regressions which you cannot explain you can try to run the
benchmarks again. If regressions still show up, please consult with the
performance team (@eeckstein).

Hardware Overview
  Model Name: Mac Pro
  Model Identifier: MacPro6,1
  Processor Name: 12-Core Intel Xeon E5
  Processor Speed: 2.7 GHz
  Number of Processors: 1
  Total Number of Cores: 12
  L2 Cache (per Core): 256 KB
  L3 Cache: 30 MB
  Memory: 64 GB

Copy link
Contributor

@gottesmm gottesmm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@lorentey
Copy link
Member Author

@swift-ci test Linux platform

@swift-ci
Copy link
Contributor

Build failed
Swift Test Linux Platform
Git Sha - 03e4f5f

@lorentey
Copy link
Member Author

lorentey commented Apr 8, 2020

Ah, I missed the failure logs. :-(

@swift-ci please test linux platform

@swift-ci
Copy link
Contributor

swift-ci commented Apr 8, 2020

Build failed
Swift Test Linux Platform
Git Sha - 03e4f5f

@lorentey
Copy link
Member Author

@swift-ci test linux platform

@swift-ci
Copy link
Contributor

Build failed
Swift Test Linux Platform
Git Sha - 03e4f5f

@lorentey
Copy link
Member Author

Ah, I finally caught the build failure in time to look at the logs. They're weird:

18:38:22 
/home/buildnode/jenkins/workspace/swift-PR-Linux/branch-master/swift/benchmark/utils/main.swift:17:8: error: no such module 'Ackermann'
18:38:22 import Ackermann
18:38:22        ^
18:38:22 Traceback (most recent call last):
18:38:22   File "/home/buildnode/jenkins/workspace/swift-PR-Linux/branch-master/swift/benchmark/scripts/build_script_helper.py", line 63, in <module>
18:38:22     main()
18:38:22   File "/home/buildnode/jenkins/workspace/swift-PR-Linux/branch-master/swift/benchmark/scripts/build_script_helper.py", line 57, in main
18:38:22     perform_build(args, swiftbuild_path, "debug", "Benchmark_Onone", "-Onone")
18:38:22   File "/home/buildnode/jenkins/workspace/swift-PR-Linux/branch-master/swift/benchmark/scripts/build_script_helper.py", line 39, in perform_build
18:38:22     shutil.copy(binpath, finalpath)
18:38:22   File "/usr/lib/python2.7/shutil.py", line 119, in copy
18:38:22     copyfile(src, dst)
18:38:22   File "/usr/lib/python2.7/shutil.py", line 82, in copyfile
18:38:22     with open(src, 'rb') as fsrc:
18:38:22 IOError: [Errno 2] No such file or directory: '/home/buildnode/jenkins/workspace/swift-PR-Linux/branch-master/buildbot_linux/benchmarks-linux-x86_64/Benchmark_Onone/debug/SwiftBench'

@lorentey
Copy link
Member Author

🤦‍♂️

@lorentey
Copy link
Member Author

@swift-ci test

@swift-ci

This comment has been minimized.

@swift-ci

This comment has been minimized.

@lorentey lorentey merged commit ee95b8b into swiftlang:master Apr 14, 2020
@lorentey lorentey deleted the fortify-benchmarks branch April 14, 2020 21:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants