Skip to content

OCPBUGS-22217: Fixing flake tests so they don't fail 100% of the time. #595

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Oct 21, 2023

Conversation

tmshort
Copy link
Contributor

@tmshort tmshort commented Oct 20, 2023

E2E tests were updated recently to use a generated namespace with an OperatorGroup created from the beginning of each test. However some of the tests were not updated to reflect this and were still creating and deleting resources in an additional namespace, or began to fail because the test environment required that no OperatorGroups exist from the beginning of the test.

This commit addresses two tests that have been failing at a rate of 100% due to the issues stated above.

Upstream-repository: operator-lifecycle-manager
Upstream-commit: 91b5762a9

E2E tests were updated recently to use a generated namespace with an OperatorGroup created from the beginning of each test. However some of the tests were not updated to reflect this and were still creating and deleting resources in an additional namespace, or began to fail because the test environment required that no OperatorGroups exist from the beginning of the test.

This commit addresses two tests that have been failing at a rate of 100% due to the issues stated above.

Signed-off-by: Daniel Franz <[email protected]>
Upstream-repository: operator-lifecycle-manager
Upstream-commit: 91b5762a9
@openshift-ci-robot openshift-ci-robot added jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. jira/invalid-bug Indicates that a referenced Jira bug is invalid for the branch this PR is targeting. labels Oct 20, 2023
@openshift-ci-robot
Copy link

@tmshort: This pull request references Jira Issue OCPBUGS-22217, which is invalid:

  • expected the bug to target the "4.15.0" version, but no target version was set

Comment /jira refresh to re-evaluate validity if changes to the Jira bug are made, or edit the title of this pull request to link to a different bug.

The bug has been updated to refer to the pull request using the external bug tracker.

In response to this:

E2E tests were updated recently to use a generated namespace with an OperatorGroup created from the beginning of each test. However some of the tests were not updated to reflect this and were still creating and deleting resources in an additional namespace, or began to fail because the test environment required that no OperatorGroups exist from the beginning of the test.

This commit addresses two tests that have been failing at a rate of 100% due to the issues stated above.

Upstream-repository: operator-lifecycle-manager
Upstream-commit: 91b5762a9

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@tmshort
Copy link
Contributor Author

tmshort commented Oct 20, 2023

/jira refresh

@openshift-ci-robot openshift-ci-robot added jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. and removed jira/invalid-bug Indicates that a referenced Jira bug is invalid for the branch this PR is targeting. labels Oct 20, 2023
@openshift-ci-robot
Copy link

@tmshort: This pull request references Jira Issue OCPBUGS-22217, which is valid.

3 validation(s) were run on this bug
  • bug is open, matching expected state (open)
  • bug target version (4.15.0) matches configured target version for branch (4.15.0)
  • bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, POST)

Requesting review from QA contact:
/cc @jianzhangbjz

In response to this:

/jira refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@openshift-ci openshift-ci bot requested a review from jianzhangbjz October 20, 2023 19:21
@tmshort
Copy link
Contributor Author

tmshort commented Oct 20, 2023

/approve

@openshift-ci
Copy link
Contributor

openshift-ci bot commented Oct 20, 2023

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: tmshort

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Oct 20, 2023
@dtfranz
Copy link
Contributor

dtfranz commented Oct 20, 2023

/lgtm

@openshift-ci openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Oct 20, 2023
@tmshort
Copy link
Contributor Author

tmshort commented Oct 20, 2023

"Flake" passed!

@openshift-ci-robot
Copy link

/retest-required

Remaining retests: 0 against base HEAD e15e6c9 and 2 for PR HEAD 455d57f in total

@openshift-ci
Copy link
Contributor

openshift-ci bot commented Oct 21, 2023

@tmshort: all tests passed!

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@openshift-ci openshift-ci bot merged commit ecb5392 into openshift:master Oct 21, 2023
@openshift-ci-robot
Copy link

@tmshort: Jira Issue OCPBUGS-22217: All pull requests linked via external trackers have merged:

Jira Issue OCPBUGS-22217 has been moved to the MODIFIED state.

In response to this:

E2E tests were updated recently to use a generated namespace with an OperatorGroup created from the beginning of each test. However some of the tests were not updated to reflect this and were still creating and deleting resources in an additional namespace, or began to fail because the test environment required that no OperatorGroups exist from the beginning of the test.

This commit addresses two tests that have been failing at a rate of 100% due to the issues stated above.

Upstream-repository: operator-lifecycle-manager
Upstream-commit: 91b5762a9

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@tmshort tmshort deleted the flake branch October 23, 2023 13:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. lgtm Indicates that a PR is ready to be merged.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants