OCPBUGS-46595: CRD upgrade existing CR validation fix (#3442) #921

grokspawn · 2024-12-18T19:21:06Z

manual fix of failed automation from #917 going to release-4.14.

fixed to pass map[string]interface instead of unstructured.Unstructured

We started seeing some issues with folks who had spurious CRD incompatibility claims when updating operators. It is a failure in OLM code which validates existing CRs against incoming CRDs, recently updated in #3387.

This manifested in InstallPlan .status.Message something like:

retrying execution due to error: error validating existing CRs against new CRD's schema for \"pgadmins.postgres-operator.crunchydata.com\": error validating postgres-operator.crunchydata.com/v1beta1, Kind=PGAdmin \"openshift-operators/example-pgadmin\": updated validation is too restrictive: [].spec.tolerations[0].tolerationSeconds: Invalid value: \"number\": spec.tolerations[0].tolerationSeconds in body must be of type integer: \"number\"

The difference between the predecessor calling convention and the one introduced in #3387 appears to be that one is a pointer and the other is concrete.

old

unstructured.Unstructured{Object:map[string]interface...

new

&unstructured.Unstructured{Object:map[string]interface...

so it would seem that merely type-asserting the value and de-referencing it would yield the appropriate result, but it appears instead that it effectively disables all CR vs CRD reconciliation checks (evidenced by the fact that the unit tests multiply fail).

But k8s already dereferences pointer parameters here during validation. So that isn't it.

And the validate.ValidateCustomResource interface is terrifyingly permissive in allowing customResource as interface{} here. So we cannot derive guidance from it.

Taking a page from k8s' use of the validation API, which uses unstructured.UnstructuredContent() to convert the unstructured.Unstructured into a map[string]interface{} here then we achieve the desired results.

adding tests

Upstream-repository: operator-lifecycle-manager
Upstream-commit: 1cfabfe5a495fe3cb276fce93255cdfed7d60783

grokspawn · 2024-12-18T19:21:27Z

/jira cherry-pick OCPBUGS-46479

openshift-ci-robot · 2024-12-18T19:21:37Z

@grokspawn: Jira Issue OCPBUGS-46479 has been cloned as Jira Issue OCPBUGS-46595. Will retitle bug to link to clone.
/retitle OCPBUGS-46595: CRD upgrade existing CR validation fix (#3442)

In response to this:

/jira cherry-pick OCPBUGS-46479

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

openshift-ci · 2024-12-18T19:21:39Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: grokspawn

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~DOWNSTREAM_OWNERS~~ [grokspawn]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

openshift-ci-robot · 2024-12-18T19:21:50Z

@grokspawn: This pull request references Jira Issue OCPBUGS-46595, which is invalid:

expected dependent Jira Issue OCPBUGS-46479 to be in one of the following states: VERIFIED, RELEASE PENDING, CLOSED (ERRATA), CLOSED (CURRENT RELEASE), CLOSED (DONE), CLOSED (DONE-ERRATA), but it is ON_QA instead

Comment /jira refresh to re-evaluate validity if changes to the Jira bug are made, or edit the title of this pull request to link to a different bug.

The bug has been updated to refer to the pull request using the external bug tracker.

In response to this:

manual fix of failed automation from #917 going to release-4.14.

fixed to pass map[string]interface instead of unstructured.Unstructured

We started seeing some issues with folks who had spurious CRD incompatibility claims when updating operators. It is a failure in OLM code which validates existing CRs against incoming CRDs, recently updated in #3387.

This manifested in InstallPlan .status.Message something like:
retrying execution due to error: error validating existing CRs against new CRD's schema for \"pgadmins.postgres-operator.crunchydata.com\": error validating postgres-operator.crunchydata.com/v1beta1, Kind=PGAdmin \"openshift-operators/example-pgadmin\": updated validation is too restrictive: [].spec.tolerations[0].tolerationSeconds: Invalid value: \"number\": spec.tolerations[0].tolerationSeconds in body must be of type integer: \"number\"
The difference between the predecessor calling convention and the one introduced in #3387 appears to be that one is a pointer and the other is concrete.

old
unstructured.Unstructured{Object:map[string]interface...
new
&unstructured.Unstructured{Object:map[string]interface...
so it would seem that merely type-asserting the value and de-referencing it would yield the appropriate result, but it appears instead that it effectively disables all CR vs CRD reconciliation checks (evidenced by the fact that the unit tests multiply fail).

But k8s already dereferences pointer parameters here during validation. So that isn't it.

And the validate.ValidateCustomResource interface is terrifyingly permissive in allowing customResource as interface{} here. So we cannot derive guidance from it.

Taking a page from k8s' use of the validation API, which uses unstructured.UnstructuredContent() to convert the unstructured.Unstructured into a map[string]interface{} here then we achieve the desired results.

adding tests

Upstream-repository: operator-lifecycle-manager
Upstream-commit: 1cfabfe5a495fe3cb276fce93255cdfed7d60783

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

oceanc80 · 2024-12-18T19:54:24Z

/label backport-risk-assessed

* fixed to pass map[string]interface instead of unstructured.Unstructured We started seeing some issues with folks who had spurious CRD incompatibility claims when updating operators. It is a failure in OLM code which validates existing CRs against incoming CRDs, recently updated in #3387. This manifested in `InstallPlan` `.status.Message` something like: ``` retrying execution due to error: error validating existing CRs against new CRD's schema for \"pgadmins.postgres-operator.crunchydata.com\": error validating postgres-operator.crunchydata.com/v1beta1, Kind=PGAdmin \"openshift-operators/example-pgadmin\": updated validation is too restrictive: [].spec.tolerations[0].tolerationSeconds: Invalid value: \"number\": spec.tolerations[0].tolerationSeconds in body must be of type integer: \"number\" ``` The difference between the predecessor calling convention and the one introduced in #3387 appears to be that one is a pointer and the other is concrete. old ```golang unstructured.Unstructured{Object:map[string]interface... ``` new ```golang &unstructured.Unstructured{Object:map[string]interface... ``` so it would seem that merely type-asserting the value and de-referencing it would yield the appropriate result, but it appears instead that it effectively disables all CR vs CRD reconciliation checks (evidenced by the fact that the unit tests multiply fail). But k8s already dereferences pointer parameters [here](https://github.com/kubernetes/kube-openapi/blob/master/pkg/validation/validate/schema.go#L139-L141) during validation. So that isn't it. And the `validate.ValidateCustomResource` interface is terrifyingly permissive in allowing `customResource` as `interface{}` [here](https://pkg.go.dev/k8s.io/[email protected]/pkg/apiserver/validation#ValidateCustomResource). So we cannot derive guidance from it. Taking a page from k8s' use of the validation API, which uses `unstructured.UnstructuredContent()` to convert the `unstructured.Unstructured` into a `map[string]interface{}` [here](https://github.com/kubernetes/kubernetes/blob/1504f10e7946f95a8b1da35e28e4c7453ff62775/staging/src/k8s.io/apiextensions-apiserver/pkg/registry/customresource/validator.go#L54) then we achieve the desired results. Signed-off-by: Jordan Keister <[email protected]> * adding tests Signed-off-by: Jordan Keister <[email protected]> --------- Signed-off-by: Jordan Keister <[email protected]> Upstream-repository: operator-lifecycle-manager Upstream-commit: 1cfabfe5a495fe3cb276fce93255cdfed7d60783

Xia-Zhao-rh · 2024-12-19T02:16:44Z

/jira refresh

openshift-ci-robot · 2024-12-19T02:16:50Z

@Xia-Zhao-rh: This pull request references Jira Issue OCPBUGS-46595, which is valid. The bug has been moved to the POST state.

7 validation(s) were run on this bug

bug is open, matching expected state (open)
bug target version (4.14.z) matches configured target version for branch (4.14.z)
bug is in the state New, which is one of the valid states (NEW, ASSIGNED, POST)
release note text is set and does not match the template
dependent bug Jira Issue OCPBUGS-46479 is in the state Verified, which is one of the valid states (VERIFIED, RELEASE PENDING, CLOSED (ERRATA), CLOSED (CURRENT RELEASE), CLOSED (DONE), CLOSED (DONE-ERRATA))
dependent Jira Issue OCPBUGS-46479 targets the "4.15.z" version, which is one of the valid target versions: 4.15.0, 4.15.z
bug has dependents

Requesting review from QA contact:
/cc @Xia-Zhao-rh

In response to this:

/jira refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

Xia-Zhao-rh · 2024-12-19T02:17:03Z

/retest

Xia-Zhao-rh · 2024-12-19T03:07:14Z

test PASS. detail https://issues.redhat.com/browse/OCPBUGS-46595
/label qe-approved
/label cherry-pick-approved
/lgtm

openshift-ci-robot · 2024-12-19T03:07:21Z

@grokspawn: This pull request references Jira Issue OCPBUGS-46595, which is valid.

7 validation(s) were run on this bug

bug is open, matching expected state (open)
bug target version (4.14.z) matches configured target version for branch (4.14.z)
bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, POST)
release note text is set and does not match the template
dependent bug Jira Issue OCPBUGS-46479 is in the state Verified, which is one of the valid states (VERIFIED, RELEASE PENDING, CLOSED (ERRATA), CLOSED (CURRENT RELEASE), CLOSED (DONE), CLOSED (DONE-ERRATA))
dependent Jira Issue OCPBUGS-46479 targets the "4.15.z" version, which is one of the valid target versions: 4.15.0, 4.15.z
bug has dependents

Requesting review from QA contact:
/cc @Xia-Zhao-rh

In response to this:

manual fix of failed automation from #917 going to release-4.14.

fixed to pass map[string]interface instead of unstructured.Unstructured

We started seeing some issues with folks who had spurious CRD incompatibility claims when updating operators. It is a failure in OLM code which validates existing CRs against incoming CRDs, recently updated in #3387.

This manifested in InstallPlan .status.Message something like:
retrying execution due to error: error validating existing CRs against new CRD's schema for \"pgadmins.postgres-operator.crunchydata.com\": error validating postgres-operator.crunchydata.com/v1beta1, Kind=PGAdmin \"openshift-operators/example-pgadmin\": updated validation is too restrictive: [].spec.tolerations[0].tolerationSeconds: Invalid value: \"number\": spec.tolerations[0].tolerationSeconds in body must be of type integer: \"number\"
The difference between the predecessor calling convention and the one introduced in #3387 appears to be that one is a pointer and the other is concrete.

old
unstructured.Unstructured{Object:map[string]interface...
new
&unstructured.Unstructured{Object:map[string]interface...
so it would seem that merely type-asserting the value and de-referencing it would yield the appropriate result, but it appears instead that it effectively disables all CR vs CRD reconciliation checks (evidenced by the fact that the unit tests multiply fail).

But k8s already dereferences pointer parameters here during validation. So that isn't it.

And the validate.ValidateCustomResource interface is terrifyingly permissive in allowing customResource as interface{} here. So we cannot derive guidance from it.

Taking a page from k8s' use of the validation API, which uses unstructured.UnstructuredContent() to convert the unstructured.Unstructured into a map[string]interface{} here then we achieve the desired results.

adding tests

Upstream-repository: operator-lifecycle-manager
Upstream-commit: 1cfabfe5a495fe3cb276fce93255cdfed7d60783

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

openshift-ci-robot · 2024-12-19T05:13:36Z

/retest-required

Remaining retests: 0 against base HEAD 82dc3f9 and 2 for PR HEAD 40b1627 in total

openshift-ci-robot · 2024-12-19T12:05:26Z

/retest-required

Remaining retests: 0 against base HEAD 82dc3f9 and 2 for PR HEAD 40b1627 in total

openshift-ci-robot · 2024-12-20T04:27:12Z

/retest-required

Remaining retests: 0 against base HEAD 82dc3f9 and 2 for PR HEAD 40b1627 in total

openshift-ci · 2024-12-20T06:45:33Z

@grokspawn: all tests passed!

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

openshift-ci-robot · 2024-12-20T06:50:31Z

@grokspawn: Jira Issue OCPBUGS-46595: All pull requests linked via external trackers have merged:

openshift/operator-framework-olm#921

Jira Issue OCPBUGS-46595 has been moved to the MODIFIED state.

In response to this:

manual fix of failed automation from #917 going to release-4.14.

fixed to pass map[string]interface instead of unstructured.Unstructured

We started seeing some issues with folks who had spurious CRD incompatibility claims when updating operators. It is a failure in OLM code which validates existing CRs against incoming CRDs, recently updated in #3387.

This manifested in InstallPlan .status.Message something like:
retrying execution due to error: error validating existing CRs against new CRD's schema for \"pgadmins.postgres-operator.crunchydata.com\": error validating postgres-operator.crunchydata.com/v1beta1, Kind=PGAdmin \"openshift-operators/example-pgadmin\": updated validation is too restrictive: [].spec.tolerations[0].tolerationSeconds: Invalid value: \"number\": spec.tolerations[0].tolerationSeconds in body must be of type integer: \"number\"
The difference between the predecessor calling convention and the one introduced in #3387 appears to be that one is a pointer and the other is concrete.

old
unstructured.Unstructured{Object:map[string]interface...
new
&unstructured.Unstructured{Object:map[string]interface...
so it would seem that merely type-asserting the value and de-referencing it would yield the appropriate result, but it appears instead that it effectively disables all CR vs CRD reconciliation checks (evidenced by the fact that the unit tests multiply fail).

But k8s already dereferences pointer parameters here during validation. So that isn't it.

And the validate.ValidateCustomResource interface is terrifyingly permissive in allowing customResource as interface{} here. So we cannot derive guidance from it.

Taking a page from k8s' use of the validation API, which uses unstructured.UnstructuredContent() to convert the unstructured.Unstructured into a map[string]interface{} here then we achieve the desired results.

adding tests

Upstream-repository: operator-lifecycle-manager
Upstream-commit: 1cfabfe5a495fe3cb276fce93255cdfed7d60783

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

openshift-bot · 2024-12-20T08:28:16Z

[ART PR BUILD NOTIFIER]

Distgit: operator-lifecycle-manager
This PR has been included in build operator-lifecycle-manager-container-v4.14.0-202412200705.p0.g3563054.assembly.stream.el8.
All builds following this will include this PR.

openshift-bot · 2024-12-20T08:28:17Z

[ART PR BUILD NOTIFIER]

Distgit: operator-registry
This PR has been included in build operator-registry-container-v4.14.0-202412200705.p0.g3563054.assembly.stream.el8.
All builds following this will include this PR.

openshift-merge-robot · 2024-12-20T16:22:10Z

Fix included in accepted release 4.14.0-0.nightly-2024-12-20-115909

grokspawn · 2024-12-20T21:24:35Z

/cherrypick release-4.13

openshift-cherrypick-robot · 2024-12-20T21:25:24Z

@grokspawn: new pull request created: #924

In response to this:

/cherrypick release-4.13

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

openshift-ci bot requested review from dinhxuanvu and kevinrizza December 18, 2024 19:21

openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Dec 18, 2024

openshift-ci bot changed the title ~~CRD upgrade existing CR validation fix (#3442)~~ OCPBUGS-46595: CRD upgrade existing CR validation fix (#3442) Dec 18, 2024

grokspawn mentioned this pull request Dec 18, 2024

[release-4.15] OCPBUGS-46479: CRD upgrade existing CR validation fix #917

Merged

openshift-ci bot added the backport-risk-assessed Indicates a PR to a release branch has been evaluated and considered safe to accept. label Dec 18, 2024

openshift-ci bot assigned bandrade, emmajiafan, jianzhangbjz, KeenonLee, kuiwang02 and Xia-Zhao-rh Dec 18, 2024

grokspawn force-pushed the manual-crd-upgrade-cr-validation-fix branch from 74e4203 to 40b1627 Compare December 18, 2024 21:15

openshift-ci-robot added jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. and removed jira/invalid-bug Indicates that a referenced Jira bug is invalid for the branch this PR is targeting. labels Dec 19, 2024

openshift-ci bot requested a review from Xia-Zhao-rh December 19, 2024 02:16

openshift-ci bot added the qe-approved Signifies that QE has signed off on this PR label Dec 19, 2024

openshift-ci bot added the cherry-pick-approved Indicates a cherry-pick PR into a release branch has been approved by the release branch manager. label Dec 19, 2024

openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Dec 19, 2024

openshift-merge-bot bot merged commit 3563054 into openshift:release-4.14 Dec 20, 2024
13 checks passed

openshift-cherrypick-robot mentioned this pull request Dec 20, 2024

[release-4.13] OCPBUGS-47507: CRD upgrade existing CR validation fix (#3442) #924

Merged

grokspawn deleted the manual-crd-upgrade-cr-validation-fix branch December 20, 2024 21:29

OCPBUGS-46595: CRD upgrade existing CR validation fix (#3442) #921

OCPBUGS-46595: CRD upgrade existing CR validation fix (#3442) #921

Uh oh!

Conversation

grokspawn commented Dec 18, 2024

Uh oh!

grokspawn commented Dec 18, 2024

Uh oh!

openshift-ci-robot commented Dec 18, 2024

Uh oh!

openshift-ci bot commented Dec 18, 2024

Uh oh!

openshift-ci-robot commented Dec 18, 2024

Uh oh!

oceanc80 commented Dec 18, 2024

Uh oh!

Xia-Zhao-rh commented Dec 19, 2024

Uh oh!

openshift-ci-robot commented Dec 19, 2024

Uh oh!

Xia-Zhao-rh commented Dec 19, 2024

Uh oh!

Xia-Zhao-rh commented Dec 19, 2024

Uh oh!

openshift-ci-robot commented Dec 19, 2024

Uh oh!

openshift-ci-robot commented Dec 19, 2024

Uh oh!

openshift-ci-robot commented Dec 19, 2024

Uh oh!

openshift-ci-robot commented Dec 20, 2024

Uh oh!

openshift-ci bot commented Dec 20, 2024

Uh oh!

Uh oh!

openshift-ci-robot commented Dec 20, 2024

Uh oh!

openshift-bot commented Dec 20, 2024

Uh oh!

openshift-bot commented Dec 20, 2024

Uh oh!

openshift-merge-robot commented Dec 20, 2024

Uh oh!

grokspawn commented Dec 20, 2024

Uh oh!

openshift-cherrypick-robot commented Dec 20, 2024

Uh oh!

Uh oh!