Skip to content

fix CSV unmarshaling error with custom unmarshaler with pretty errors #1138

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

Conversation

grokspawn
Copy link
Contributor

@grokspawn grokspawn commented Aug 22, 2023

Description of the change:

Solves #1039 for bundle property rendering cases.

For bundles with invalid CSVs we would see unhelpful messages like

opm render registry-proxy.engineering.redhat.com/rh-osbs/kmm-kernel-module-management-operator-bundle@sha256:a55eb76e830293c579c4935bf74b9ab5c0ab9f5bcd9a324ae23a797976d86695
2022/11/10 15:01:34 render reference "registry-proxy.engineering.redhat.com/rh-osbs/kmm-kernel-module-management-operator-bundle@sha256:a55eb76e830293c579c4935bf74b9ab5c0ab9f5bcd9a324ae23a797976d86695": unrecognized type: string

This change allows us to catch/format unmarshaling errors of the CSV to provide messages like the following, where there is a marker showing the error as well as more information about the nature of the error (shortened for legibility):

./bin/opm render registry-proxy.engineering.redhat.com/rh-osbs/kmm-kernel-module-management-operator-bundle@sha256:a55eb76e830293c579c4935bf74b9ab5c0ab9f5bcd9a324ae23a797976d86695
2023/08/21 16:21:45 render reference "registry-proxy.engineering.redhat.com/rh-osbs/kmm-kernel-module-management-operator-bundle@sha256:a55eb76e830293c579c4935bf74b9ab5c0ab9f5bcd9a324ae23a797976d86695": json: cannot unmarshal bool into Go struct field ObjectMeta.metadata.annotations of type string at offset 344 (indicated by <==)
{
    "apiVersion": "operators.coreos.com/v1alpha1",
    "kind": "ClusterServiceVersion",
    "metadata": {
        "annotations": {
            "alm-examples": "[\n  {\n    \"apiVersion\": \"kmm.sigs.k8s.io/v1beta1\",\n    \"kind\": \"Module\",\n    \"metadata\": {\n      \"name\": \"module-sample\"\n    },\n    \"spec\": null\n  }\n]",
            "capabilities": "Basic Install",
            "certified": true <== ,
            "containerImage": "registry.redhat.io/kmm/kernel-module-management-operator-rhel8@sha256:fddffaaac1b81ee29bf94af94373c15a7def1b14e34e92ce171aa791022f9ffd",
            "olm.skipRange": "\u003e=0.0.0 \u003c1.0.0",
            "operators.operatorframework.io/builder": "operator-sdk-v1.17.0+git",
            "operators.operatorframework.io/project_layout": "go.kubebuilder.io/v3",
            "repository": "https://github.com/rh-ecosystem-edge/kernel-module-management",
            "support": "Red Hat"
        },
        "name": "kmm-operator.v1.0.0",
        "namespace": "placeholder"
    },

 ---- SNIP ----

Motivation for the change:

Reviewer Checklist

  • Implementation matches the proposed design, or proposal is updated to match implementation
  • Sufficient unit test coverage
  • Sufficient end-to-end test coverage
  • Docs updated or added to /docs
  • Commit messages sensible and descriptive

@openshift-ci openshift-ci bot requested review from gallettilance and njhale August 22, 2023 22:12
@openshift-ci openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Aug 22, 2023
@grokspawn
Copy link
Contributor Author

/hold until I get the utests sorted

@openshift-ci openshift-ci bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Aug 23, 2023
@codecov
Copy link

codecov bot commented Aug 24, 2023

Codecov Report

Merging #1138 (c93d7b8) into master (dba1e10) will increase coverage by 0.01%.
Report is 1 commits behind head on master.
The diff coverage is 58.82%.

❗ Current head c93d7b8 differs from pull request most recent head b7b447f. Consider uploading reports for the commit b7b447f to get more accurate results

@@            Coverage Diff             @@
##           master    #1138      +/-   ##
==========================================
+ Coverage   53.79%   53.81%   +0.01%     
==========================================
  Files         108      108              
  Lines       10166    10195      +29     
==========================================
+ Hits         5469     5486      +17     
- Misses       3736     3741       +5     
- Partials      961      968       +7     
Files Changed Coverage Δ
pkg/registry/csv.go 69.23% <53.33%> (-3.32%) ⬇️
alpha/declcfg/declcfg.go 80.48% <100.00%> (ø)
pkg/prettyunmarshaler/prettyunmarshaler.go 89.74% <100.00%> (ø)

... and 1 file with indirect coverage changes

@grokspawn grokspawn force-pushed the pretty-printer branch 2 times, most recently from 8a13bbb to c93d7b8 Compare August 24, 2023 16:49
@grokspawn
Copy link
Contributor Author

grokspawn commented Aug 24, 2023

I sorted out the sequence, and maybe the multiple 'helper' functions aren't necessary, but it aided my investigation. It turns out that because there is only one named field in the struct, and it is a json.RawMessage we have to be careful to extract it by name.

Now only a single utest is failing: TestExportForBundleWithBadImage, because it attempts a registry pull of image foo and it never comes back from syscall.

I'm investigating, but here's the trace in case something jumps out to someone.

goroutine 69 [syscall, 10 minutes]:
syscall.syscall6(0xc000723728?, 0x104bd5e94?, 0x1081818e0?, 0x10986cf40?, 0xc0007237a8?, 0x104bd60fc?, 0x104b8bfd0?)
	/Users/jordan/.gvm/gos/go1.20.7/src/runtime/sys_darwin.go:45 +0x68
syscall.wait4(0x108149640?, 0xc0007237a8, 0x0?, 0xc000726000)
	/Users/jordan/.gvm/gos/go1.20.7/src/syscall/zsyscall_darwin_arm64.go:43 +0x64
syscall.Wait4(0xc0002ba318?, 0xc000723808, 0xc000121b30?, 0xc000294660?)
	/Users/jordan/.gvm/gos/go1.20.7/src/syscall/syscall_bsd.go:144 +0x48
os.(*Process).wait(0xc000294660)
	/Users/jordan/.gvm/gos/go1.20.7/src/os/exec_unix.go:43 +0xbc
os.(*Process).Wait(...)
	/Users/jordan/.gvm/gos/go1.20.7/src/os/exec.go:132
os/exec.(*Cmd).Wait(0xc000121a20)
	/Users/jordan/.gvm/gos/go1.20.7/src/os/exec/exec.go:890 +0x74
os/exec.(*Cmd).Run(0xc00011e930?)
	/Users/jordan/.gvm/gos/go1.20.7/src/os/exec/exec.go:590 +0x58
os/exec.(*Cmd).CombinedOutput(0xc000121a20)
	/Users/jordan/.gvm/gos/go1.20.7/src/os/exec/exec.go:1005 +0x218
github.com/operator-framework/operator-registry/pkg/containertools.(*ContainerCommandRunner).Pull(0xc00000d560, {0x1069093d1, 0x3})
	/Users/jordan/devel/operator-registry/pkg/containertools/runner.go:90 +0x318
github.com/operator-framework/operator-registry/pkg/image/execregistry.(*Registry).Pull(0xc00000d590, {0x3?, 0xc00070dd10?}, {0x1071d3b68, 0xc0003782b0})
	/Users/jordan/devel/operator-registry/pkg/image/execregistry/registry.go:38 +0x78
github.com/operator-framework/operator-registry/pkg/lib/bundle.(*BundleExporter).Export(0xc000723e60, 0x1, 0x0)
	/Users/jordan/devel/operator-registry/pkg/lib/bundle/exporter.go:67 +0x4c0
github.com/operator-framework/operator-registry/pkg/lib/bundle.TestExportForBundleWithBadImage(0x0?)
	/Users/jordan/devel/operator-registry/pkg/lib/bundle/exporter_test.go:12 +0xb0
testing.tRunner(0xc0001a0680, 0x1071c7878)
	/Users/jordan/.gvm/gos/go1.20.7/src/testing/testing.go:1576 +0x18c
created by testing.(*T).Run
	/Users/jordan/.gvm/gos/go1.20.7/src/testing/testing.go:1629 +0x5e8

@grokspawn
Copy link
Contributor Author

/unhold

Now only a single utest is failing: TestExportForBundleWithBadImage, because it attempts a registry pull of image foo and it never comes back from syscall.

Evidently this issue only exists on mac(??) and it runs cleanly on linux. Going to get this into review while I continue to investigate.

@openshift-ci openshift-ci bot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Aug 24, 2023
@grokspawn
Copy link
Contributor Author

/unapprove
I'd really like to get two sets of eyeballs on this, so I'm removing my (default) approval.

@grokspawn
Copy link
Contributor Author

/remove-approve

@openshift-ci openshift-ci bot removed the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Aug 24, 2023
@grokspawn
Copy link
Contributor Author

Now only a single utest is failing: TestExportForBundleWithBadImage, because it attempts a registry pull of image foo and it never comes back from syscall.

Evidently this issue only exists on mac(??) and it runs cleanly on linux. Going to get this into review while I continue to investigate.

Turns out to be a bad docker install on my mac. :(
Resolved, and all tests pass now.

Copy link
Member

@awgreene awgreene left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/approve

@openshift-ci
Copy link
Contributor

openshift-ci bot commented Aug 25, 2023

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: awgreene, everettraven

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Aug 25, 2023
@awgreene
Copy link
Member

Ah @everettraven already approved
/lgtm

@openshift-ci openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Aug 25, 2023
@openshift-merge-robot openshift-merge-robot merged commit 56771f7 into operator-framework:master Aug 25, 2023
@grokspawn grokspawn deleted the pretty-printer branch August 28, 2023 12:58
grokspawn added a commit to grokspawn/operator-registry that referenced this pull request Sep 5, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. lgtm Indicates that a PR is ready to be merged.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants