OCPBUGS-25019: building rhel8/rhel9/static clis in payload image #704

grokspawn · 2024-02-28T21:31:37Z

This PR generates a new operator-framework-clis image which contains rhel8 & rhel9 FIPS-compliant binaries and an updated static non-FIPS-compliant binary:

-rwxr-xr-x. 1 root root  64M Mar  2 03:54 configmap-server
-rwxr-xr-x. 1 root root  63M Mar  2 03:50 configmap-server-rhel8
-rwxr-xr-x. 1 root root  79M Mar  2 03:55 darwin-amd64-opm
-rwxr-xr-x. 1 root root  41M Mar  2 03:54 initializer
-rwxr-xr-x. 1 root root  41M Mar  2 03:50 initializer-rhel8
-rwxr-xr-x. 1 root root  77M Mar  2 03:54 opm
-rwxr-xr-x. 1 root root  77M Mar  2 03:50 opm-rhel8
-rwxr-xr-x. 1 root root  75M Mar  2 03:57 opm-static
-rwxr-xr-x. 1 root root  41M Mar  2 03:54 registry-server
-rwxr-xr-x. 1 root root  41M Mar  2 03:50 registry-server-rhel8
-rwxr-xr-x. 1 root root 134M Mar  2 03:56 windows-amd64-opm

rhel8 binaries are provided only where the binary is dynamically linked for rhel9 in main/latest OCP, so for e.g. no darwin, windows versions.

Followups in other repos will lead to ART packaging binaries from this image to the mirrors site instead of the image used as the basis for catalog pods.
(and possibly to slimming down the catalog basis image)

grokspawn · 2024-02-28T21:32:04Z

/hold WIP

grokspawn · 2024-02-28T21:41:25Z

currently builds everything, but transplants only the linux opm binary from the rhel8 builder.
When I built the image, I do see both binaries:

-rwxr-xr-x. 1 root root  80551544 Feb 28 21:11 opm
-rwxr-xr-x. 1 root root  80310624 Feb 28 21:07 opm-rhel8

but the ldd output is not what I expected. I thought that the resulting container would be in the context of rhel9 so the rhel8 binary would have some dangling library links, but they all appear to be satisfied:

bash-5.1$ ldd opm 
	linux-vdso.so.1 (0x00007ffe1c9a2000)
	libresolv.so.2 => /lib64/libresolv.so.2 (0x00007f8dea739000)
	libc.so.6 => /lib64/libc.so.6 (0x00007f8dea530000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f8dea752000)
bash-5.1$ ldd opm-rhel8 
	linux-vdso.so.1 (0x00007ffe9ecb3000)
	libresolv.so.2 => /lib64/libresolv.so.2 (0x00007fbc56463000)
	libpthread.so.0 => /lib64/libpthread.so.0 (0x00007fbc5645e000)
	libdl.so.2 => /lib64/libdl.so.2 (0x00007fbc56459000)
	libc.so.6 => /lib64/libc.so.6 (0x00007fbc56250000)
	/lib64/ld-linux-x86-64.so.2 (0x00007fbc5647c000)

These results look strange because both sets are resolved (with different hex values for their requirements) but also because there are more linkages in the rhel8 version.

It's probably a product of the platform version on which they were built, but hopefully we can demystify the issues.

grokspawn · 2024-02-29T02:44:29Z

/retest
PR has no code.

grokspawn · 2024-02-29T14:47:55Z

/hold cancel

grokspawn · 2024-02-29T14:49:07Z

/retest
infra problems in openshift/test/origin

grokspawn · 2024-02-29T18:16:59Z

Failing e2e is waiting for infra PR to merge: openshift/origin#28631

tmshort · 2024-02-29T19:05:16Z

It's probably a product of the platform version on which they were built, but hopefully we can demystify the issues.

Did RHEL9 combine libdl and libpthread into libc?

grokspawn · 2024-02-29T20:31:11Z

/retest

grokspawn · 2024-02-29T20:55:54Z

Did RHEL9 combine libdl and libpthread into libc?
Great question.

Yes, they were:
https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/9/html-single/considerations_in_adopting_rhel_9/index

The libdl library now merged into libc.so.6

In RHEL 8, the libdl library was a distinct library. In RHEL 9, the libdl library has been merged into the core C library libc.so.6. This means that interposing the dlsym function is now much more difficult. Applications that need to control the way how symbol resolution works should switch to the auditor (LD_AUDIT) interfaces.

AND

All threading APIs now merged into libc.so.6

In RHEL 8, the system threading library, libpthread.so, was a distinct library. In RHEL 9, all threading APIs have been merged into the core C library libc.so.6. Moving threading into the core C library enables the library to support threads by default.

joelanford · 2024-02-29T21:30:14Z

If I'm understanding correctly, this will increase the size of all of the catalog images by the same amount as the binary size, which impacts mirroring and image pulling.

Is it possible to build a separate image for the RHEL8 binary?

grokspawn · 2024-02-29T21:48:30Z

/hold
We're having conversations which can change direction of this effort

Signed-off-by: Jordan <[email protected]>

grokspawn · 2024-03-02T06:21:11Z

/retest

grokspawn · 2024-03-02T16:29:12Z

/hold cancel
switched approaches to a new "CLIs" image.

grokspawn · 2024-03-02T17:31:37Z

/retest

grokspawn · 2024-03-04T13:47:20Z

/retest

grokspawn · 2024-03-04T19:12:22Z

/retest

grokspawn · 2024-03-05T15:05:00Z

/retest

openshift-ci-robot · 2024-03-05T17:41:17Z

@grokspawn: This pull request references Jira Issue OCPBUGS-25019, which is valid. The bug has been moved to the POST state.

3 validation(s) were run on this bug

bug is open, matching expected state (open)
bug target version (4.16.0) matches configured target version for branch (4.16.0)
bug is in the state ASSIGNED, which is one of the valid states (NEW, ASSIGNED, POST)

Requesting review from QA contact:
/cc @kuiwang02

The bug has been updated to refer to the pull request using the external bug tracker.

In response to this:

This PR generates a new operator-framework-clis image which contains rhel8 & rhel9 FIPS-compliant binaries and an updated static non-FIPS-compliant binary:
-rwxr-xr-x. 1 root root  64M Mar  2 03:54 configmap-server
-rwxr-xr-x. 1 root root  63M Mar  2 03:50 configmap-server-rhel8
-rwxr-xr-x. 1 root root  79M Mar  2 03:55 darwin-amd64-opm
-rwxr-xr-x. 1 root root  41M Mar  2 03:54 initializer
-rwxr-xr-x. 1 root root  41M Mar  2 03:50 initializer-rhel8
-rwxr-xr-x. 1 root root  77M Mar  2 03:54 opm
-rwxr-xr-x. 1 root root  77M Mar  2 03:50 opm-rhel8
-rwxr-xr-x. 1 root root  75M Mar  2 03:57 opm-static
-rwxr-xr-x. 1 root root  41M Mar  2 03:54 registry-server
-rwxr-xr-x. 1 root root  41M Mar  2 03:50 registry-server-rhel8
-rwxr-xr-x. 1 root root 134M Mar  2 03:56 windows-amd64-opm
rhel8 binaries are provided only where the binary is dynamically linked for rhel9 in main/latest OCP, so for e.g. no darwin, windows versions.

Followups in other repos will lead to ART packaging binaries from this image to the mirrors site instead of the image used as the basis for catalog pods.
(and possibly to slimming down the catalog basis image)

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

everettraven

/lgtm

openshift-ci · 2024-03-05T18:20:16Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: everettraven, grokspawn

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~DOWNSTREAM_OWNERS~~ [grokspawn]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

openshift-ci-robot · 2024-03-05T19:13:14Z

/retest-required

Remaining retests: 0 against base HEAD 78da988 and 2 for PR HEAD a3992fb in total

openshift-ci · 2024-03-05T20:08:28Z

@grokspawn: all tests passed!

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

openshift-ci-robot · 2024-03-05T20:11:35Z

@grokspawn: Jira Issue OCPBUGS-25019: All pull requests linked via external trackers have merged:

openshift/operator-framework-olm#704

Jira Issue OCPBUGS-25019 has been moved to the MODIFIED state.

In response to this:

This PR generates a new operator-framework-clis image which contains rhel8 & rhel9 FIPS-compliant binaries and an updated static non-FIPS-compliant binary:
-rwxr-xr-x. 1 root root  64M Mar  2 03:54 configmap-server
-rwxr-xr-x. 1 root root  63M Mar  2 03:50 configmap-server-rhel8
-rwxr-xr-x. 1 root root  79M Mar  2 03:55 darwin-amd64-opm
-rwxr-xr-x. 1 root root  41M Mar  2 03:54 initializer
-rwxr-xr-x. 1 root root  41M Mar  2 03:50 initializer-rhel8
-rwxr-xr-x. 1 root root  77M Mar  2 03:54 opm
-rwxr-xr-x. 1 root root  77M Mar  2 03:50 opm-rhel8
-rwxr-xr-x. 1 root root  75M Mar  2 03:57 opm-static
-rwxr-xr-x. 1 root root  41M Mar  2 03:54 registry-server
-rwxr-xr-x. 1 root root  41M Mar  2 03:50 registry-server-rhel8
-rwxr-xr-x. 1 root root 134M Mar  2 03:56 windows-amd64-opm
rhel8 binaries are provided only where the binary is dynamically linked for rhel9 in main/latest OCP, so for e.g. no darwin, windows versions.

Followups in other repos will lead to ART packaging binaries from this image to the mirrors site instead of the image used as the basis for catalog pods.
(and possibly to slimming down the catalog basis image)

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

openshift-bot · 2024-03-06T01:36:42Z

[ART PR BUILD NOTIFIER]

This PR has been included in build operator-lifecycle-manager-container-v4.16.0-202403052341.p0.gc84b6ad.assembly.stream.el9 for distgit operator-lifecycle-manager.
All builds following this will include this PR.

openshift-merge-robot · 2024-03-06T14:49:18Z

Fix included in accepted release 4.16.0-0.nightly-2024-03-06-073110

openshift-ci bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Feb 28, 2024

openshift-ci bot requested review from ankitathomas and oceanc80 February 28, 2024 21:32

openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Feb 28, 2024

openshift-ci bot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Feb 29, 2024

openshift-ci bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Feb 29, 2024

build all CLIs in one image

a3992fb

Signed-off-by: Jordan <[email protected]>

grokspawn force-pushed the rhel8-support branch from 736c849 to a3992fb Compare March 2, 2024 04:13

openshift-ci bot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Mar 2, 2024

grokspawn changed the title ~~building rhel8 opm in payload image~~ building rhel8/rhel9/static clis in payload image Mar 4, 2024

grokspawn changed the title ~~building rhel8/rhel9/static clis in payload image~~ OCPBUGS-25019: building rhel8/rhel9/static clis in payload image Mar 5, 2024

openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label Mar 5, 2024

openshift-ci-robot added the jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. label Mar 5, 2024

openshift-ci bot requested a review from kuiwang02 March 5, 2024 17:41

everettraven approved these changes Mar 5, 2024

View reviewed changes

openshift-ci bot assigned everettraven Mar 5, 2024

openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Mar 5, 2024

openshift-merge-bot bot merged commit c84b6ad into openshift:master Mar 5, 2024

grokspawn deleted the rhel8-support branch March 5, 2024 20:20

grokspawn mentioned this pull request Mar 7, 2024

OPRUN-3221: add new tools image for rhel8, rhel9 operator-framework binaries openshift/release#49647

Merged

openshift-ci-robot mentioned this pull request Jul 9, 2024

[release-4.16] OCPBUGS-25019: fix xplat compile for of-tools image; hide utest files from git; use rhel8 art builder image #793

Merged

OCPBUGS-25019: building rhel8/rhel9/static clis in payload image #704

OCPBUGS-25019: building rhel8/rhel9/static clis in payload image #704

Uh oh!

Conversation

grokspawn commented Feb 28, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

grokspawn commented Feb 28, 2024

Uh oh!

grokspawn commented Feb 28, 2024

Uh oh!

grokspawn commented Feb 29, 2024

Uh oh!

grokspawn commented Feb 29, 2024

Uh oh!

grokspawn commented Feb 29, 2024

Uh oh!

grokspawn commented Feb 29, 2024

Uh oh!

tmshort commented Feb 29, 2024

Uh oh!

grokspawn commented Feb 29, 2024

Uh oh!

grokspawn commented Feb 29, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

joelanford commented Feb 29, 2024

Uh oh!

grokspawn commented Feb 29, 2024

Uh oh!

grokspawn commented Mar 2, 2024

Uh oh!

grokspawn commented Mar 2, 2024

Uh oh!

grokspawn commented Mar 2, 2024

Uh oh!

grokspawn commented Mar 4, 2024

Uh oh!

grokspawn commented Mar 4, 2024

Uh oh!

grokspawn commented Mar 5, 2024

Uh oh!

openshift-ci-robot commented Mar 5, 2024

Uh oh!

everettraven left a comment

Choose a reason for hiding this comment

Uh oh!

openshift-ci bot commented Mar 5, 2024

Uh oh!

openshift-ci-robot commented Mar 5, 2024

Uh oh!

openshift-ci bot commented Mar 5, 2024

Uh oh!

openshift-ci-robot commented Mar 5, 2024

Uh oh!

openshift-bot commented Mar 6, 2024

Uh oh!

openshift-merge-robot commented Mar 6, 2024

Uh oh!

Uh oh!

grokspawn commented Feb 28, 2024 •

edited

Loading

grokspawn commented Feb 29, 2024 •

edited

Loading