[build] ensure benchmark toolchain tests use the just built libraries #38158

edymtt · 2021-06-29T19:24:06Z

The binaries would prefer the dylibs shipped in the OS to the ones just built -- employ DYLD_LIBRARY_PATH to pick up the latter instead.

The main purpose of this change is to unblock the generation of nightly
toolchains.

Addresses rdar://79788142

edymtt · 2021-06-29T19:24:37Z

@swift-ci please smoke test

edymtt · 2021-06-29T19:24:45Z

@swift-ci please build toolchain

shahmishal · 2021-06-29T19:29:09Z

I would recommend reverting my change to skip building benchmark in this PR. Otherwise, build toolchain job will not provide valid testing.

edymtt · 2021-06-29T19:41:38Z

#38159
@swift-ci please smoke test

edymtt · 2021-06-29T19:44:55Z

@swift-ci please smoke test

edymtt · 2021-06-29T19:45:01Z

@swift-ci please build toolchain

swift-ci · 2021-06-29T21:07:10Z

Linux Toolchain (Ubuntu 16.04)
Download Toolchain
Git Sha - 7a12ffe10f4c9b4cfee85d89ea1788e50a8d10d9

Install command
tar zxf swift-PR-38158-615-ubuntu16.04.tar.gz
More info

edymtt · 2021-06-29T22:03:45Z

@swift-ci please smoke test macOS

edymtt · 2021-06-29T22:03:55Z

@swift-ci please test Windows

The binaries would prefer the dylibs shipped in the OS to the ones just built -- employ `DYLD_LIBRARY_PATH` to pick up the latter instead. The main purpose of this change is to unblock the generation of nightly toolchains. Addresses rdar://79788142

…-benchmark-test-79788142" This reverts commit 8822ccc, reversing changes made to d477efb.

…8142" This reverts commit e059d5c, reversing changes made to f1ebf6c.

edymtt · 2021-07-06T14:10:12Z

@swift-ci please smoke test

edymtt · 2021-07-06T14:10:21Z

@swift-ci please build toolchain

swift-ci · 2021-07-06T15:33:10Z

Linux Toolchain (Ubuntu 16.04)
Download Toolchain
Git Sha - db88d43

Install command
tar zxf swift-PR-38158-621-ubuntu16.04.tar.gz
More info

swift-ci · 2021-07-06T19:10:46Z

macOS Toolchain
Download Toolchain
Git Sha - db88d43

Install command
tar -zxf swift-PR-38158-1042-osx.tar.gz --directory ~/

edymtt · 2021-07-06T19:57:22Z

The benchmark tests are being executed without errors

+ env DYLD_LIBRARY_PATH=/Users/buildnode/jenkins/workspace/swift-PR-toolchain-osx/branch-main/buildbot_osx/intermediate-install/macosx-x86_64/Library/Developer/Toolchains/swift-PR-38158-1042.xctoolchain/usr/lib/swift/macosx /Users/buildnode/jenkins/workspace/swift-PR-toolchain-osx/branch-main/buildbot_osx/benchmarks-macosx-x86_64/bin/Benchmark_Onone --num-iters=1 XorLoop
#,TEST,SAMPLES,MIN(μs),MAX(μs),MEAN(μs),SD(μs),MEDIAN(μs)
1023,XorLoop,86,11182,12584,11503,386,11300

Total performance tests executed: 1
+ env DYLD_LIBRARY_PATH=/Users/buildnode/jenkins/workspace/swift-PR-toolchain-osx/branch-main/buildbot_osx/intermediate-install/macosx-x86_64/Library/Developer/Toolchains/swift-PR-38158-1042.xctoolchain/usr/lib/swift/macosx /Users/buildnode/jenkins/workspace/swift-PR-toolchain-osx/branch-main/buildbot_osx/benchmarks-macosx-x86_64/bin/Benchmark_O --num-iters=1 XorLoop
#,TEST,SAMPLES,MIN(μs),MAX(μs),MEAN(μs),SD(μs),MEDIAN(μs)
1023,XorLoop,200,381,451,384,11,381

Total performance tests executed: 1
+ env DYLD_LIBRARY_PATH=/Users/buildnode/jenkins/workspace/swift-PR-toolchain-osx/branch-main/buildbot_osx/intermediate-install/macosx-x86_64/Library/Developer/Toolchains/swift-PR-38158-1042.xctoolchain/usr/lib/swift/macosx /Users/buildnode/jenkins/workspace/swift-PR-toolchain-osx/branch-main/buildbot_osx/benchmarks-macosx-x86_64/bin/Benchmark_Osize --num-iters=1 XorLoop
#,TEST,SAMPLES,MIN(μs),MAX(μs),MEAN(μs),SD(μs),MEDIAN(μs)
1023,XorLoop,200,399,516,415,24,411

Total performance tests executed: 1

gottesmm

LGTM with these small changes.

utils/swift_build_support/swift_build_support/products/benchmarks.py

edymtt · 2021-07-06T22:05:30Z

@swift-ci please python lint

edymtt · 2021-07-06T22:42:27Z

@swift-ci please smoke test

edymtt · 2021-07-06T22:42:36Z

@swift-ci please build toolchain

swift-ci · 2021-07-07T00:18:39Z

Linux Toolchain (Ubuntu 16.04)
Download Toolchain
Git Sha - 0bccd8d

Install command
tar zxf swift-PR-38158-624-ubuntu16.04.tar.gz
More info

swift-ci · 2021-07-07T05:01:50Z

macOS Toolchain
Download Toolchain
Git Sha - 0bccd8d

Install command
tar -zxf swift-PR-38158-1048-osx.tar.gz --directory ~/

edymtt · 2021-07-07T14:06:41Z

Checked that the new run executed the benchmark tests as expected

+ env DYLD_LIBRARY_PATH=/Users/buildnode/jenkins/workspace/swift-PR-toolchain-osx/branch-main/buildbot_osx/intermediate-install/macosx-x86_64/Lib
rary/Developer/Toolchains/swift-PR-38158-1048.xctoolchain/usr/lib/swift/macosx /Users/buildnode/jenkins/workspace/swift-PR-toolchain-osx/branch-m
ain/buildbot_osx/benchmarks-macosx-x86_64/bin/Benchmark_Onone --num-iters=1 XorLoop
#,TEST,SAMPLES,MIN(μs),MAX(μs),MEAN(μs),SD(μs),MEDIAN(μs)
1023,XorLoop,86,11183,12257,11302,210,11227

Total performance tests executed: 1
+ env DYLD_LIBRARY_PATH=/Users/buildnode/jenkins/workspace/swift-PR-toolchain-osx/branch-main/buildbot_osx/intermediate-install/macosx-x86_64/Lib
rary/Developer/Toolchains/swift-PR-38158-1048.xctoolchain/usr/lib/swift/macosx /Users/buildnode/jenkins/workspace/swift-PR-toolchain-osx/branch-m
ain/buildbot_osx/benchmarks-macosx-x86_64/bin/Benchmark_O --num-iters=1 XorLoop
#,TEST,SAMPLES,MIN(μs),MAX(μs),MEAN(μs),SD(μs),MEDIAN(μs)
1023,XorLoop,200,381,912,387,38,383

Total performance tests executed: 1
+ env DYLD_LIBRARY_PATH=/Users/buildnode/jenkins/workspace/swift-PR-toolchain-osx/branch-main/buildbot_osx/intermediate-install/macosx-x86_64/Lib
rary/Developer/Toolchains/swift-PR-38158-1048.xctoolchain/usr/lib/swift/macosx /Users/buildnode/jenkins/workspace/swift-PR-toolchain-osx/branch-m
ain/buildbot_osx/benchmarks-macosx-x86_64/bin/Benchmark_Osize --num-iters=1 XorLoop
#,TEST,SAMPLES,MIN(μs),MAX(μs),MEAN(μs),SD(μs),MEDIAN(μs)
1023,XorLoop,200,399,420,400,3,400

Total performance tests executed: 1

edymtt · 2021-07-07T14:08:43Z

For some reason I don't fully understand, the standalone python lint seems to check more files that the one run as part of macOS tests and detect issues on a file not changed by this PR

15:09:29 + ./utils/python_lint.py
15:09:36 ./test/attr/Inputs/access-note-gen.py:28:10: E127 continuation line over-indented for visual indent
15:09:36 ./test/attr/Inputs/access-note-gen.py:29:10: E127 continuation line over-indented for visual indent

Since this will not affect regular CI runs, I will return to this at a later time.

The binaries would prefer the libraries shipped in the OS to the ones just built -- employ `DYLD_LIBRARY_PATH` to pick up the latter instead. The main purpose of this change is to unblock the generation of nightly toolchains. Addresses rdar://79788142 Cherry picked from commit 3504310, swiftlang#38158

edymtt requested review from gottesmm, drexin, porglezomp, shahmishal and beccadax June 29, 2021 19:24

edymtt mentioned this pull request Jun 29, 2021

Revert "Skip building benchmarks in nightly toolchain preset (79788142)" #38159

Closed

edymtt force-pushed the use_just_build_libraries_for_benchmark_toolchain_tests branch from 7c730a9 to 7a12ffe Compare June 29, 2021 19:44

edymtt added 3 commits July 5, 2021 12:00

Revert "Merge pull request swiftlang#38172 from apple/shahmishal/skip…

f352902

…-benchmark-test-79788142" This reverts commit 8822ccc, reversing changes made to d477efb.

Revert "Merge pull request swiftlang#38155 from apple/shahmishal/r978…

db88d43

…8142" This reverts commit e059d5c, reversing changes made to f1ebf6c.

edymtt force-pushed the use_just_build_libraries_for_benchmark_toolchain_tests branch from 7a12ffe to db88d43 Compare July 5, 2021 19:02

edymtt removed the request for review from beccadax July 5, 2021 19:03

gottesmm approved these changes Jul 6, 2021

View reviewed changes

utils/swift_build_support/swift_build_support/products/benchmarks.py Outdated Show resolved Hide resolved

utils/swift_build_support/swift_build_support/products/benchmarks.py Outdated Show resolved Hide resolved

Address review feedback

0bccd8d

edymtt merged commit 3504310 into swiftlang:main Jul 7, 2021

edymtt mentioned this pull request Jul 7, 2021

[5.5 build] ensure benchmark toolchain tests use the just built dylibs #38286

Merged

edymtt deleted the use_just_build_libraries_for_benchmark_toolchain_tests branch July 8, 2021 14:07

[build] ensure benchmark toolchain tests use the just built libraries #38158

[build] ensure benchmark toolchain tests use the just built libraries #38158

Uh oh!

Conversation

edymtt commented Jun 29, 2021

Uh oh!

edymtt commented Jun 29, 2021

Uh oh!

edymtt commented Jun 29, 2021

Uh oh!

shahmishal commented Jun 29, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

edymtt commented Jun 29, 2021

Uh oh!

edymtt commented Jun 29, 2021

Uh oh!

edymtt commented Jun 29, 2021

Uh oh!

swift-ci commented Jun 29, 2021

Uh oh!

edymtt commented Jun 29, 2021

Uh oh!

edymtt commented Jun 29, 2021

Uh oh!

edymtt commented Jul 6, 2021

Uh oh!

edymtt commented Jul 6, 2021

Uh oh!

swift-ci commented Jul 6, 2021

Uh oh!

swift-ci commented Jul 6, 2021

Uh oh!

edymtt commented Jul 6, 2021

Uh oh!

gottesmm left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

edymtt commented Jul 6, 2021

Uh oh!

edymtt commented Jul 6, 2021

Uh oh!

edymtt commented Jul 6, 2021

Uh oh!

swift-ci commented Jul 7, 2021

Uh oh!

swift-ci commented Jul 7, 2021

Uh oh!

edymtt commented Jul 7, 2021

Uh oh!

edymtt commented Jul 7, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

shahmishal commented Jun 29, 2021 •

edited

Loading

edymtt commented Jul 7, 2021 •

edited

Loading