build-script: support CMake install steps. #25184

adrian-prantl · 2019-06-01T00:18:19Z

Currently, bots that perfrom an installation will build LLDB once using CMake
because that is the default and then again using Xcode because the CMake
install step is not implemented by build-script. This patch adds CMake support
by calling the generic cmake install-lldb step.

rdar://problem/51317901

Currently, bots that perfrom an installation will build LLDB once using CMake because that is the default and then again using Xcode because the CMake install step is not implemented by build-script. This patch adds CMake support by calling the generic cmake install-lldb step. <rdar://problem/51317901>

adrian-prantl · 2019-06-01T00:19:08Z

@swift-ci test package

adrian-prantl · 2019-06-01T00:19:39Z

@swift-ci test

shahmishal · 2019-06-01T01:36:28Z

We should also pull this into swift-5.1-branch

swift-ci · 2019-06-01T01:51:06Z

Build failed
Swift Test Linux Platform
Git Sha - 78218b0

swift-ci · 2019-06-01T02:26:14Z

Build failed
Swift Test OS X Platform
Git Sha - 78218b0

compnerd · 2019-06-01T16:53:04Z

utils/build-script-impl

-                case ${host} in
-                    linux-*)
+                if [[ "$using_xcodebuild" == "TRUE" ]] ; then
+                    case ${host} in


I think that it will be easier to read this as:

if [[ ${host} == macosx-* ]] ; then set_lldb_xcodebuild_options set_lldb_build_mode with_pushd ... continue fi

At that point, you can even collapse the conditions into:

if [[ ${host} == macosx-* && ${using_xcodebuild} == TRUE ]] ; then

weliveindetail · 2019-06-03T10:49:07Z

utils/build-script-impl

-                esac
+                    esac
+                else
+                    INSTALL_TARGETS="install-lldb"


I assume you tested cmake install-lldb? If it works, then let's do this. My upstreaming progress is slow, but I can check this in more detail soon.

weliveindetail · 2019-06-03T14:03:32Z

@swift-ci test

swift-ci · 2019-06-03T15:32:47Z

Build failed
Swift Test Linux Platform
Git Sha - 78218b0

weliveindetail · 2019-06-04T16:05:03Z

/tmp/TemporaryDirectory.ZWG8vT/pkg/Package.swift:3:5: error: type annotation missing in pattern

weliveindetail · 2019-06-04T16:05:13Z

@swift-ci test

swift-ci · 2019-06-04T17:34:24Z

Build failed
Swift Test Linux Platform
Git Sha - 78218b0

dcci · 2019-07-08T23:05:15Z

Since Adrian is out, and this is causing problems on the bots, I'll pick this up (cc: @shahmishal)

dcci · 2019-07-08T23:18:43Z

Of course the log is too old and it's gone. Let's try to run this again and see why it fails.

dcci · 2019-07-08T23:18:51Z

@swift-ci test

swift-ci · 2019-07-09T01:26:47Z

Build failed
Swift Test OS X Platform
Git Sha - 78218b0

swift-ci · 2019-07-09T01:42:45Z

Build failed
Swift Test Linux Platform
Git Sha - 78218b0

dcci · 2019-07-09T02:48:36Z

@swift-ci test

dcci · 2019-07-09T02:48:57Z

Trying again now that Andy merged his PR.

swift-ci · 2019-07-09T04:12:35Z

Build failed
Swift Test Linux Platform
Git Sha - 78218b0

dcci · 2019-07-09T15:07:13Z

21:12:33 Running Swift tests for: check-swift-all-linux-x86_64 check-swift-all-optimize-linux-x86_64
21:12:33 /home/buildnode/jenkins/workspace/swift-PR-Linux@2/branch-master/swift/utils/build-script: fatal error: command terminated with a non-zero exit status 2, aborting
21:12:33 /home/buildnode/jenkins/workspace/swift-PR-Linux@2/branch-master/swift/utils/build-script: fatal error: command terminated with a non-zero exit status 1, aborting

I'll try to reproduce on a linux machine.

dcci · 2019-07-09T16:06:41Z

This is the real issue

06:10:01 2019-07-09 04:10:01.765 SourceKitLSPPackageTests.xctest[2773:38ff9700] manifest parse error(s):
06:10:01 
/home/buildnode/jenkins/workspace/swift-PR-Linux@2/branch-master/tmp/TemporaryDirectory.QLRGsf/pkg/Package.swift:3:5: error: type annotation missing in pattern
06:10:01 let pack
06:10:01     ^

@aciidb0mb3r / @benlangmuir / @akyrtzi

is this something you're aware of?

benlangmuir · 2019-07-09T16:09:55Z

This is not a build failure. It's just output from a test in sourcekit-lsp that gets mistakenly picked up by the regexes in Jenkins. You can see below that the test passed.

benlangmuir · 2019-07-09T16:11:01Z

here's the actual failure, which is much later during package testing:

swift-integration-tests/lit.cfg:205: fatal: repl_swift does not exist!

weliveindetail · 2019-07-09T16:31:18Z

This is not a build failure. It's just output from a test in sourcekit-lsp that gets mistakenly picked up by the regexes in Jenkins. You can see below that the test passed.

Hey Ben, thanks for taking a look.

here's the actual failure, which is much later during package testing:

swift-integration-tests/lit.cfg:205: fatal: repl_swift does not exist!

Interesting. I will have a look.

weliveindetail · 2019-07-10T15:04:01Z

repl_swift gets built but not installed, because it runs install-lldb instead of install:

--- Installing lldb ---
06:07:20 + env DESTDIR=/home/buildnode/jenkins/workspace/swift-PR-Linux@2/branch-master/swift-nightly-install/ /usr/bin/cmake --build /home/buildnode/jenkins/workspace/swift-PR-Linux@2/branch-master/buildbot_linux/lldb-linux-x86_64 -- install-lldb
06:07:20 [1/2][ 50%][0.032s] Python script sym-linking LLDB Python API
06:07:20 [1/2][ 50%][0.032s] cd /home/buildnode/jenkins/workspace/swift-PR-Linux@2/branch-master/buildbot_linux/lldb-linux-x86_64/tools/driver && /usr/bin/cmake -DCMAKE_INSTALL_COMPONENT="lldb" -P /home/buildnode/jenkins/workspace/swift-PR-Linux@2/branch-master/buildbot_linux/lldb-linux-x86_64/cmake_install.cmake
06:07:20 -- Install configuration: "Release"
06:07:20 -- Installing: /home/buildnode/jenkins/workspace/swift-PR-Linux@2/branch-master/swift-nightly-install/usr/bin/lldb
06:07:20

weliveindetail · 2019-07-10T15:13:32Z

@swift-ci test

swift-ci · 2019-07-10T15:15:45Z

Build failed
Swift Test Linux Platform
Git Sha - 78218b0

swift-ci · 2019-07-10T15:17:23Z

Build failed
Swift Test OS X Platform
Git Sha - 78218b0

weliveindetail · 2019-07-11T10:23:17Z

@swift-ci test package

weliveindetail · 2019-07-11T10:39:51Z

With my change the PR tests succeed, but the results are.. strange.

--install-lldb is still passed to build-script, but the log doesn't show any attempt to install lldb (the stage that was missing to install repl_swift before). The only thing that gets installed is libdispatch (on the Linux bot). Is that expected behavior?

The SourceKitLSPPackageTests that printed the pseudo-error type annotation missing in pattern before is not in the logs either. Is that expected behavior?

Apparently, test package doesn't do anything.

@dcci @benlangmuir What do you think?

Well, at least the goal of the PR is accomplished: LLDB doesn't build twice anymore :)

dcci · 2019-07-11T15:04:31Z

utils/build-script-impl

@@ -3555,8 +3555,6 @@ for host in "${ALL_HOSTS[@]}"; do
                            continue
                        ;;
                    esac
-                else
-                    INSTALL_TARGETS="install-lldb"


build-script is fairly complex so I'm probably missing something here, but ... your review just removes an install target? Where do we actually perform the installation for CMake?

IIUC INSTALL_TARGETS collects a list of targets that will be installed (don't ask me where). It's is initialized with install here: https://github.com/apple/swift/blob/2fa436e53e/utils/build-script-impl#L3491 This should be fine for LLDB.

We could go for install-distribution with apple/swift-lldb#1785

benlangmuir · 2019-07-11T15:49:31Z

The SourceKitLSPPackageTests that printed the pseudo-error type annotation missing in pattern before is not in the logs either. Is that expected behavior?

Maybe you were looking only at the tail of the log instead of the full log? I see it in the logs from your PR tests.

weliveindetail · 2019-07-11T16:03:38Z

The SourceKitLSPPackageTests that printed the pseudo-error type annotation missing in pattern before is not in the logs either. Is that expected behavior?

Maybe you were looking only at the tail of the log instead of the full log? I see it in the logs from your PR tests.

Erm no, but I only checked the macOS one. The Linux PR test is actually correct.

benlangmuir · 2019-07-11T16:10:18Z

testUnparsablePackage ran and passed on both macOS and Linux. Maybe the macOS xctest is hiding the output from the passing test while the linux one isn't. Anyway, looks fine.

shahmishal · 2019-07-11T19:37:17Z

@swift-ci build toolchain will build the toolchain.

adrian-prantl requested review from fredriss, weliveindetail, gottesmm and dcci June 1, 2019 00:18

compnerd reviewed Jun 1, 2019

View reviewed changes

weliveindetail approved these changes Jun 3, 2019

View reviewed changes

[build-script] Run default install target before package testing

2fa436e

dcci reviewed Jul 11, 2019

View reviewed changes

weliveindetail merged commit dafd027 into swiftlang:master Jul 11, 2019

build-script: support CMake install steps. #25184

build-script: support CMake install steps. #25184

Uh oh!

Conversation

adrian-prantl commented Jun 1, 2019

Uh oh!

adrian-prantl commented Jun 1, 2019

Uh oh!

adrian-prantl commented Jun 1, 2019

Uh oh!

shahmishal commented Jun 1, 2019

Uh oh!

swift-ci commented Jun 1, 2019

Uh oh!

swift-ci commented Jun 1, 2019

Uh oh!

compnerd Jun 1, 2019

Choose a reason for hiding this comment

Uh oh!

weliveindetail Jun 3, 2019

Choose a reason for hiding this comment

Uh oh!

weliveindetail commented Jun 3, 2019

Uh oh!

swift-ci commented Jun 3, 2019

Uh oh!

weliveindetail commented Jun 4, 2019

Uh oh!

weliveindetail commented Jun 4, 2019

Uh oh!

swift-ci commented Jun 4, 2019

Uh oh!

dcci commented Jul 8, 2019

Uh oh!

dcci commented Jul 8, 2019

Uh oh!

dcci commented Jul 8, 2019

Uh oh!

swift-ci commented Jul 9, 2019

Uh oh!

swift-ci commented Jul 9, 2019

Uh oh!

dcci commented Jul 9, 2019

Uh oh!

dcci commented Jul 9, 2019

Uh oh!

swift-ci commented Jul 9, 2019

Uh oh!

dcci commented Jul 9, 2019

Uh oh!

dcci commented Jul 9, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

benlangmuir commented Jul 9, 2019

Uh oh!

benlangmuir commented Jul 9, 2019

Uh oh!

weliveindetail commented Jul 9, 2019

Uh oh!

weliveindetail commented Jul 10, 2019

Uh oh!

weliveindetail commented Jul 10, 2019

Uh oh!

swift-ci commented Jul 10, 2019

Uh oh!

swift-ci commented Jul 10, 2019

Uh oh!

weliveindetail commented Jul 11, 2019

Uh oh!

weliveindetail commented Jul 11, 2019

Uh oh!

dcci Jul 11, 2019

Choose a reason for hiding this comment

Uh oh!

weliveindetail Jul 11, 2019

Choose a reason for hiding this comment

Uh oh!

weliveindetail Jul 11, 2019

Choose a reason for hiding this comment

Uh oh!

dcci commented Jul 9, 2019 •

edited

Loading