centos8 rpm spec #27
centos8 rpm spec #27
Conversation
motivation: rpm installer for centos8 changes: initial setup: spec, patches, driver script and docker for centos 8
| @@ -0,0 +1,23 @@ | |||
|
|
|||
There was a problem hiding this comment.
Do we need this patch for CentOS 8?
There was a problem hiding this comment.
ideally I would like to get to zero patches. this one specifically does two things in build-presets:
- uses the system ninja instead building it from source. we can also change the spec to download ninja and build it from source if preferable
- disables "test-installable-package", which I took from the fedora spec. not sure why it was needed there. @tachoknight?
There was a problem hiding this comment.
@tomerd I'm trying to get Fedora down to no patches as well. I have this PR which I called buildbot_linux_fedora,no_test, but maybe it would be better to make it more generic to RPM-based distros? The test-installable-package was the source of a lot of errors with Python, I'm trying a build with it enabled to see if it's still an issue.
There was a problem hiding this comment.
ideally I would like to get to zero patches. this one specifically does two things in build-presets:
- uses the system ninja instead building it from source. we can also change the spec to download ninja and build it from source if preferable
I had to do this for CMake under CentOS 8 for awhile; I have an example if you'd like to see how to do it in a spec file here.
- disables "test-installable-package", which I took from the fedora spec. not sure why it was needed there. @tachoknight?
I tested the latest Swift-5.5 snapshot with test-installable-package enabled and it worked, so this is no longer necessary to be removed.
There was a problem hiding this comment.
uses the system ninja instead building it from source. we can also change the spec to download ninja and build it from source if preferable
@shahmishal is there a reason not to use the system ninja when it is available?
There was a problem hiding this comment.
disables "test-installable-package", which I took from the fedora spec. not sure why it was needed there. @tachoknight?
I tested the latest Swift-5.5 snapshot with test-installable-package enabled and it worked, so this is no longer necessary to be removed.
@tachoknight should we update/remove the patch then?
There was a problem hiding this comment.
uses the system ninja instead building it from source. we can also change the spec to download ninja and build it from source if preferable
@shahmishal is there a reason not to use the system ninja when it is available?
I dont see a problem in using the system ninja if it's available.
There was a problem hiding this comment.
disables "test-installable-package", which I took from the fedora spec. not sure why it was needed there. @tachoknight?
I tested the latest Swift-5.5 snapshot with test-installable-package enabled and it worked, so this is no longer necessary to be removed.
@tachoknight should we update/remove the patch then?
I think we should remove the patch, and add support spec file to clone ninja.
|
What I like to do is have the build script write out the spec file, that gives you the ability to set the versions, requirements list, etc. as options to the script, either as arguments or CI environment vars. Below is an example I was working on for packaging under CentOS7 or 8. The swift build was handled by a separate shell script, generating the full tar ball, and then this script was called to write out the spec file and do the rpm build. I was experimenting with package separation, so this version builds Build requirement lists could easily be read in from a separate text file as well. |
I initially had a |
I've been working on a script to do just that; was behind on some of the other packages which I had to track down. One thing to note is that this can't be used as part of the "official" RPM build for the Fedora repo as the spec file already needs to exist in its final form when building the binary RPM. My thought is to use the script to update the spec file and run the build (see the script I currently use to build Swift; the spec file that is contained in the |
how about we do this in stages? first we get manually crafted specs that work across the platforms we want to support: fedora, centos, Amazon Linux, debian and ubuntu. then we optimize further by adding a script that generate the specs for future iterations. this way we can make progress on all the other downstream tooling (CI, signing, distribution) without getting bugged down with the optimization. @shahmishal wdyt? |
| License: ASL 2.0 and Unicode | ||
| URL: https://swift.org |
There was a problem hiding this comment.
| License: ASL 2.0 and Unicode | |
| URL: https://swift.org | |
| License: Apache 2.0 | |
| URL: https://swift.org |
I think we should update the License to Apache 2.0.
There was a problem hiding this comment.
We will have to do the same for Fedora spec file.
|
Going to merge this, and we can make improvements to the scripts before officially releasing the rpm installer. |
|
@tomerd @shahmishal I am trying to understand the logic of merging this pull request when the submitted code is incorrect and there are many other aspects which are still being discussed. Another question is why we are reinventing the wheel? Here is an example of a Jenkinsfile I am using to build developer snapshots deb packages from the official Swift source for the Swift Community Apt Repository // Jenkins Pipeline - swift deb/rpm packager - swift.org amd64
// ubuntu 20.04 main-branch snapshot
pipeline {
agent { label 'swiftaltra' }
environment {
PKG_ITERATION='2021-09-07-a'
PKG_VERSION='5.6'
DIST = 'ubuntu'
VER_NAME = 'focal'
VER_NUM='20.04'
SWIFT_URL="https://swift.org/builds/development/ubuntu2004/swift-DEVELOPMENT-SNAPSHOT-${PKG_ITERATION}/swift-DEVELOPMENT-SNAPSHOT-${PKG_ITERATION}-ubuntu20.04.tar.gz"
}
stages {
stage('Clean Workspace') {
steps {
echo 'Cleaning Workspace'
cleanWs()
script {
currentBuild.displayName = "swift-DEVELOPMENT-SNAPSHOT-${PKG_ITERATION}"
}
}
}
stage('Get Swift Toolchain') {
steps {
echo 'Getting Swift'
sh 'curl -fsSL "$SWIFT_URL" -o swift.tar.gz'
sh 'ls'
}
}
stage('Clean Package') {
steps {
sh 'tar -xzf swift.tar.gz --strip-components=1'
sh 'rm usr/lib/python3/dist-packages/six.py'
}
}
stage('Package DEB') {
steps {
echo "building deb"
sh "rm *.deb || true"
sh "export XZ_DEFAULTS='-9 -T0'; fpm \
-s dir \
-t deb \
-a amd64 \
--name swiftlang \
--version ${PKG_VERSION} \
--iteration ${PKG_ITERATION}-${DIST}-${VER_NAME} \
-d binutils \
-d git \
-d gnupg2 \
-d libc6-dev \
-d libcurl4 \
-d libedit2 \
-d libgcc-9-dev \
-d libpython3.8 \
-d libsqlite3-0 \
-d libstdc++-9-dev \
-d libxml2 \
-d libz3-dev \
-d pkg-config \
-d tzdata \
-d zlib1g-dev \
--replaces clang \
--replaces clang-10 \
--replaces libicu-dev \
--replaces lld \
--replaces lldb \
--replaces llvm \
--maintainer 'Swift Infrastructure <[email protected]>' \
--deb-priority optional \
--url 'https://swift.org' \
--description 'The Swift Programming Language.' \
--license 'Apache License Version 2.0' \
--deb-no-default-config-files \
--deb-compression xz \
usr"
}
}
stage('Upload to Server') {
steps {
echo 'uploading to repository'
sh "scp *.deb [email protected]:/home/futurejones/debs/new-snapshots/${DIST}/${VER_NAME}/"
}
}
}
}This can be done for all the official Swift releases rpm or deb packages. Another huge advantage of this approach is the build time. No need to build the entire swift toolchain which can take hours, a deb or rpm package can be built in minutes. |
This is now resolved with these two PRs: #29 #28
@tomerd @tachoknight Do you see any issue with this? It would help reduce the resource on CI. @futurejones I might have miss understood, but I thought the official spec requires building the package from source.
Thanks for sharing the Jenkinsfile. |
Yes, that is correct. @futurejones, the official Fedora/EPEL RPMS are always built from source on their servers; binary blobs are not allowed to be packaged; the only thing that can be submitted for building is the |
Yes @tachoknight is correct but as far as I was aware the purpose of building the rpm/deb packages was to host them on on an official Swift.org package repository. Not submit them to Ubuntu, Debian, Redhat etc. for inclusion their official repos. For example Swift for Centos8/AArch64 which is available on the Swift-Arm Community repository. |
My goal is to have this repo represent all the files necessary to build the official RPM. It would be a poor user experience that the versions of Swift differ between the self-hosted and repo-based locations. |
|
@tachoknight I am not sure I understand, I was referring the method of building the rpm file not which version of swift is hosted. If you build the rpm from the official swift-5.5-release.tar.gz file or from the source code with a spec file both rpms will install the identical version of swift. By self-hosted I meant hosted officially by Swift.org on their servers. |
|
the motivation behind this work is to make installing swift on linux seamless. this is something we hear about from the community time and again, and where the current tarball distribution falls short given swift's relatively large amount of system dependencies. to address this, we want to transition from tarballs to native installers where possible, with the following phases:
|
|
@tomerd thanks for this information.
All these items should be the same across all the platforms and package systems. |
|
@futurejones totally agree, this is all very much wip and having such doc listing the goals and the open questions / issues would help drive this to completion |
motivation: rpm installer for centos8
changes: initial setup: spec, patches, driver script and docker for centos 8
Open Issues / TODO