Skip to content

Commit 336fe33

Browse files
perdasilvaperdasilva
committed
(Partially) mitigate flaky tests
Signed-off-by: perdasilva <[email protected]> Signed-off-by: perdasilva <[email protected]>
1 parent 0666fff commit 336fe33

17 files changed

+134
-34
lines changed

‎.github/ISSUE_TEMPLATE/flaky-test.md

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
---
2+
name: Flaky Test Report
3+
about: If an unrelated test is failing for mysterious reasons
4+
title: ''
5+
labels: 'kind/flake'
6+
assignees: ''
7+
8+
---
9+
10+
## Flaky Test Report
11+
12+
<!--
13+
Note: Make sure to first check the prerequisites that can be found in the main README file!
14+
Thanks for filing an issue! Before hitting the button, please provide the following information:
15+
-->
16+
17+
**Failure Log Link**
18+
[Failure Log](put link here)
19+
You can get the link by clicking on the line number in the job logs and copying the url from your browser.
20+
21+
**Relevant Failure Log**
22+
<!--- Paste any relevant failure logs from the job -->
23+

‎.github/PULL_REQUEST_TEMPLATE.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,8 @@ Note: Make sure your branch is rebased to the latest upstream master.
1717
- [ ] Sufficient end-to-end test coverage
1818
- [ ] Docs updated or added to `/doc`
1919
- [ ] Commit messages sensible and descriptive
20+
- [ ] Tests marked as `[FLAKE]` are truly flaky
21+
- [ ] Tests that remove the `[FLAKE]` tag are no longer flaky
2022

2123

2224
<!--

‎.github/workflows/e2e-tests.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,7 @@ jobs:
1919
- uses: actions/setup-go@v2
2020
with:
2121
go-version: '~1.17'
22-
- run: make e2e-local E2E_TEST_CHUNK=${{ matrix.parallel-id }} E2E_TEST_NUM_CHUNKS=${{ strategy.job-total }} E2E_NODES=2 ARTIFACTS_DIR=./artifacts/
22+
- run: make e2e-local E2E_TEST_CHUNK=${{ matrix.parallel-id }} E2E_TEST_NUM_CHUNKS=${{ strategy.job-total }} E2E_NODES=2 ARTIFACTS_DIR=./artifacts/ SKIP='\[FLAKE\]'
2323
- name: Archive Test Artifacts # test results, failed or not, are always uploaded.
2424
if: ${{ always() }}
2525
uses: actions/upload-artifact@v2

‎.github/workflows/flaky-e2e.yml

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
name: flaky-e2e-tests
2+
on:
3+
schedule:
4+
- cron: '30 5,17 * * *' # run this every day at 5:30 and 17:30 UTC (00:30 and 12:30 ET)
5+
push:
6+
branches:
7+
- master
8+
pull_request:
9+
workflow_dispatch:
10+
jobs:
11+
flaky-e2e-tests:
12+
runs-on: ubuntu-latest
13+
steps:
14+
- uses: actions/checkout@v1
15+
- uses: actions/setup-go@v2
16+
with:
17+
go-version: '~1.17'
18+
- run: make e2e-local E2E_NODES=1 TEST='\[FLAKE\]' ARTIFACTS_DIR=./artifacts/
19+
- name: Archive Test Artifacts # test results, failed or not, are always uploaded.
20+
if: ${{ always() }}
21+
uses: actions/upload-artifact@v2
22+
with:
23+
name: e2e-test-output-${{(github.event.pull_request.head.sha||github.sha)}}-${{ github.run_id }}
24+
path: ${{ github.workspace }}/bin/artifacts/*

‎Makefile

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -131,7 +131,7 @@ E2E_TEST_NUM_CHUNKS ?= 4
131131
ifneq (all,$(E2E_TEST_CHUNK))
132132
TEST := $(shell go run ./test/e2e/split/... -chunks $(E2E_TEST_NUM_CHUNKS) -print-chunk $(E2E_TEST_CHUNK) ./test/e2e)
133133
endif
134-
E2E_OPTS ?= $(if $(E2E_SEED),-seed '$(E2E_SEED)') $(if $(TEST),-focus '$(TEST)') -flakeAttempts $(E2E_FLAKE_ATTEMPTS) -nodes $(E2E_NODES) -timeout $(E2E_TIMEOUT) -v -randomizeSuites -race -trace -progress
134+
E2E_OPTS ?= $(if $(E2E_SEED),-seed '$(E2E_SEED)') $(if $(SKIP), -skip '$(SKIP)') $(if $(TEST),-focus '$(TEST)') -flakeAttempts $(E2E_FLAKE_ATTEMPTS) -nodes $(E2E_NODES) -timeout $(E2E_TIMEOUT) -v -randomizeSuites -race -trace -progress
135135
E2E_INSTALL_NS ?= operator-lifecycle-manager
136136
E2E_TEST_NS ?= operators
137137

‎README.md

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -170,6 +170,10 @@ Check out the [contributor documentation][contributor-documentation]. Also, see
170170

171171
See [reporting bugs][bug_guide] for details about reporting any issues.
172172

173+
## Reporting flaky tests
174+
175+
See [reporting flaky tests][flaky_test_guide] for details about reporting flaky tests.
176+
173177
## License
174178

175179
Operator Lifecycle Manager is under Apache 2.0 license. See the [LICENSE][license_file] file for details.
@@ -179,6 +183,7 @@ Operator Lifecycle Manager is under Apache 2.0 license. See the [LICENSE][licens
179183
[proposals_docs]: ./doc/contributors/design-proposals
180184
[license_file]:./LICENSE
181185
[bug_guide]:./doc/dev/reporting_bugs.md
186+
[flaky_test_guide]:./doc/dev/reporting_flakes.md
182187
[git_tool]:https://git-scm.com/downloads
183188
[go_tool]:https://golang.org/dl/
184189
[docker_tool]:https://docs.docker.com/install/

‎doc/dev/reporting_flakes.md

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
# Reporting flakes
2+
3+
If you are struggling to get your PR through because unrelated e2e or unit tests are randomly failing, it's likely
4+
you are being plagued by a flaky test 😱, a test that wasn't constructed as carefully as it should have been as is
5+
failing even when it should be succeeding. When this happens, check our [issues](https://github.com/operator-framework/operator-lifecycle-manager/issues)
6+
to see if it has been filed before. Search also in the `closed issues`. If you find one, re-open it if necessary.
7+
Otherwise, [file](https://github.com/operator-framework/operator-lifecycle-manager/issues/new) a flaky test issue.
8+
9+
Once you have an issue link, you can disable the flaky test by adding the `[FLAKE]` tag to the test name and linking the issue in the code.
10+
11+
Example:
12+
13+
```
14+
// issue: https://github.com/operator-framework/operator-lifecycle-manager/issues/2635
15+
It("[FLAKE] updates multiple intermediates", func() {
16+
...
17+
```
18+
19+
You may be asked by the reviewer to supply evidence that the test is indeed flaky and not an unfortunate side effect of
20+
your contribution. We'll endeavor to make this an easy as process as possible to merge your contribution in as quickly
21+
and safely as possible.

‎test/e2e/catalog_e2e_test.go

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
1-
// +build !bare
1+
//go:build !bare
2+
// +build !bare
23

34
package e2e
45

@@ -870,7 +871,8 @@ var _ = Describe("Catalog represents a store of bundles which OLM can use to ins
870871
Expect(v).Should(Equal(version.OperatorVersion{Version: busyboxVersion}), "latest version of operator not installed: catalog source update failed")
871872
})
872873

873-
It("Dependency has correct replaces field", func() {
874+
// issue: https://github.com/operator-framework/operator-lifecycle-manager/issues/2642
875+
It("[FLAKE] Dependency has correct replaces field", func() {
874876
// Create a CatalogSource that contains the busybox v1 and busybox-dependency v1 images
875877
// Create a Subscription for busybox v1, which has a dependency on busybox-dependency v1.
876878
// Wait for the busybox and busybox2 Subscriptions to succeed

‎test/e2e/crd_e2e_test.go

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,8 @@ import (
2121
var _ = Describe("CRD Versions", func() {
2222
AfterEach(func() { TearDown(testNamespace) }, float64(30))
2323

24-
It("creates v1 CRDs with a v1 schema successfully", func() {
24+
// issue: https://github.com/operator-framework/operator-lifecycle-manager/issues/2640
25+
It("[FLAKE] creates v1 CRDs with a v1 schema successfully", func() {
2526
By("v1 crds with a valid openapiv3 schema should be created successfully by OLM")
2627
c := newKubeClient()
2728
crc := newCRClient()
@@ -94,7 +95,8 @@ var _ = Describe("CRD Versions", func() {
9495
Expect(fetchedInstallPlan.Status.Phase).To(Equal(operatorsv1alpha1.InstallPlanPhaseComplete))
9596
})
9697

97-
It("blocks a CRD upgrade that could cause data loss", func() {
98+
// issue:https://github.com/operator-framework/operator-lifecycle-manager/issues/2638
99+
It("[FLAKE] blocks a CRD upgrade that could cause data loss", func() {
98100
By("checking the storage versions in the existing CRD status and the spec of the new CRD")
99101

100102
c := newKubeClient()

‎test/e2e/csv_e2e_test.go

Lines changed: 18 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -19,8 +19,8 @@ import (
1919
"k8s.io/apimachinery/pkg/api/equality"
2020
k8serrors "k8s.io/apimachinery/pkg/api/errors"
2121
"k8s.io/apimachinery/pkg/api/meta"
22-
"k8s.io/apimachinery/pkg/fields"
2322
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
23+
"k8s.io/apimachinery/pkg/fields"
2424
k8slabels "k8s.io/apimachinery/pkg/labels"
2525
"k8s.io/apimachinery/pkg/runtime"
2626
"k8s.io/apimachinery/pkg/selection"
@@ -64,10 +64,10 @@ var _ = Describe("ClusterServiceVersion", func() {
6464

6565
When("a CustomResourceDefinition was installed alongside a ClusterServiceVersion", func() {
6666
var (
67-
ns corev1.Namespace
68-
crd apiextensionsv1.CustomResourceDefinition
69-
og operatorsv1.OperatorGroup
70-
apiname string
67+
ns corev1.Namespace
68+
crd apiextensionsv1.CustomResourceDefinition
69+
og operatorsv1.OperatorGroup
70+
apiname string
7171
apifullname string
7272
)
7373

@@ -144,7 +144,8 @@ var _ = Describe("ClusterServiceVersion", func() {
144144
}).Should(WithTransform(k8serrors.IsNotFound, BeTrue()))
145145
})
146146

147-
It("can satisfy an associated ClusterServiceVersion's ownership requirement", func() {
147+
// issue: https://github.com/operator-framework/operator-lifecycle-manager/issues/2646
148+
It("[FLAKE] can satisfy an associated ClusterServiceVersion's ownership requirement", func() {
148149
associated := operatorsv1alpha1.ClusterServiceVersion{
149150
ObjectMeta: metav1.ObjectMeta{
150151
Name: "associated-csv",
@@ -261,7 +262,8 @@ var _ = Describe("ClusterServiceVersion", func() {
261262
}).Should(Succeed())
262263
})
263264

264-
It("can satisfy an unassociated ClusterServiceVersion's non-ownership requirement", func() {
265+
// issue:https://github.com/operator-framework/operator-lifecycle-manager/issues/2639
266+
It("[FLAKE] can satisfy an unassociated ClusterServiceVersion's non-ownership requirement", func() {
265267
unassociated := operatorsv1alpha1.ClusterServiceVersion{
266268
ObjectMeta: metav1.ObjectMeta{
267269
Name: "unassociated-csv",
@@ -4206,7 +4208,7 @@ var _ = Describe("ClusterServiceVersion", func() {
42064208

42074209
var _ = Describe("Disabling copied CSVs", func() {
42084210
var (
4209-
ns corev1.Namespace
4211+
ns corev1.Namespace
42104212
csv operatorsv1alpha1.ClusterServiceVersion
42114213
)
42124214

@@ -4255,7 +4257,8 @@ var _ = Describe("Disabling copied CSVs", func() {
42554257
})
42564258

42574259
When("an operator is installed in AllNamespace mode", func() {
4258-
It("should have Copied CSVs in all other namespaces", func() {
4260+
// issue: https://github.com/operator-framework/operator-lifecycle-manager/issues/2643
4261+
It("[FLAKE] should have Copied CSVs in all other namespaces", func() {
42594262
Eventually(func() error {
42604263
requirement, err := k8slabels.NewRequirement(operatorsv1alpha1.CopiedLabelKey, selection.Equals, []string{csv.GetNamespace()})
42614264
if err != nil {
@@ -4335,7 +4338,8 @@ var _ = Describe("Disabling copied CSVs", func() {
43354338
}).Should(Succeed())
43364339
})
43374340

4338-
It("should be reflected in the olmConfig.Status.Condition array that the expected number of copied CSVs exist", func() {
4341+
// issue: https://github.com/operator-framework/operator-lifecycle-manager/issues/2634
4342+
It("[FLAKE] should be reflected in the olmConfig.Status.Condition array that the expected number of copied CSVs exist", func() {
43394343
Eventually(func() error {
43404344
var olmConfig operatorsv1.OLMConfig
43414345
if err := ctx.Ctx().Client().Get(context.TODO(), apitypes.NamespacedName{Name: "cluster"}, &olmConfig); err != nil {
@@ -4391,7 +4395,8 @@ var _ = Describe("Disabling copied CSVs", func() {
43914395
}).Should(Succeed())
43924396
})
43934397

4394-
It("should have copied CSVs in all other Namespaces", func() {
4398+
// issue: https://github.com/operator-framework/operator-lifecycle-manager/issues/2634
4399+
It("[FLAKE] should have copied CSVs in all other Namespaces", func() {
43954400
Eventually(func() error {
43964401
// find copied csvs...
43974402
requirement, err := k8slabels.NewRequirement(operatorsv1alpha1.CopiedLabelKey, selection.Equals, []string{csv.GetNamespace()})
@@ -4420,7 +4425,8 @@ var _ = Describe("Disabling copied CSVs", func() {
44204425
}).Should(Succeed())
44214426
})
44224427

4423-
It("should be reflected in the olmConfig.Status.Condition array that the expected number of copied CSVs exist", func() {
4428+
// issue: https://github.com/operator-framework/operator-lifecycle-manager/issues/2641
4429+
It("[FLAKE] should be reflected in the olmConfig.Status.Condition array that the expected number of copied CSVs exist", func() {
44244430
Eventually(func() error {
44254431
var olmConfig operatorsv1.OLMConfig
44264432
if err := ctx.Ctx().Client().Get(context.TODO(), apitypes.NamespacedName{Name: "cluster"}, &olmConfig); err != nil {

‎test/e2e/gc_e2e_test.go

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -646,7 +646,8 @@ var _ = Describe("Garbage collection for dependent resources", func() {
646646
}).Should(BeNil())
647647
})
648648

649-
It("should have removed the old configmap and put the new configmap in place", func() {
649+
// flake issue: https://github.com/operator-framework/operator-lifecycle-manager/issues/2626
650+
It("[FLAKE] should have removed the old configmap and put the new configmap in place", func() {
650651
Eventually(func() bool {
651652
_, err := kubeClient.GetConfigMap(testNamespace, configmapName)
652653
return k8serrors.IsNotFound(err)

‎test/e2e/installplan_e2e_test.go

Lines changed: 6 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -834,7 +834,7 @@ var _ = Describe("Install Plan", func() {
834834
}()
835835

836836
// Create the catalog source
837-
mainCatalogSourceName := genName("mock-ocs-main-" + strings.ToLower(CurrentGinkgoTestDescription().TestText) + "-")
837+
mainCatalogSourceName := genName("mock-ocs-main-" + strings.ToLower(K8sSafeCurrentTestDescription()) + "-")
838838
_, cleanupCatalogSource := createInternalCatalogSource(c, crc, mainCatalogSourceName, testNamespace, mainManifests, []apiextensions.CustomResourceDefinition{dependentCRD, mainCRD}, []operatorsv1alpha1.ClusterServiceVersion{dependentBetaCSV, dependentStableCSV, mainStableCSV, mainBetaCSV})
839839
defer cleanupCatalogSource()
840840

@@ -3006,7 +3006,8 @@ var _ = Describe("Install Plan", func() {
30063006

30073007
// This It spec verifies that, in cases where there are multiple options to fulfil a dependency
30083008
// across multiple catalogs, we only generate one installplan with one set of resolved resources.
3009-
It("consistent generation", func() {
3009+
//issue: https://github.com/operator-framework/operator-lifecycle-manager/issues/2633
3010+
It("[FLAKE] consistent generation", func() {
30103011

30113012
// Configure catalogs:
30123013
// - one catalog with a package that has a dependency
@@ -3190,15 +3191,16 @@ var _ = Describe("Install Plan", func() {
31903191
Expect(err).NotTo(HaveOccurred())
31913192
})
31923193

3193-
It("should clear clear up the condition in the InstallPlan status that contains an error message when a valid OperatorGroup is created", func() {
3194+
// issue: https://github.com/operator-framework/operator-lifecycle-manager/issues/2636
3195+
It("[FLAKE] should clear up the condition in the InstallPlan status that contains an error message when a valid OperatorGroup is created", func() {
31943196

31953197
// first wait for a condition with a message exists
31963198
cond := operatorsv1alpha1.InstallPlanCondition{Type: operatorsv1alpha1.InstallPlanInstalled, Status: corev1.ConditionFalse, Reason: operatorsv1alpha1.InstallPlanReasonInstallCheckFailed,
31973199
Message: "no operator group found that is managing this namespace"}
31983200

31993201
Eventually(func() bool {
32003202
fetchedInstallPlan, err := fetchInstallPlanWithNamespace(GinkgoT(), crc, installPlanName, ns.GetName(), buildInstallPlanPhaseCheckFunc(operatorsv1alpha1.InstallPlanPhaseInstalling))
3201-
if err != nil || fetchedInstallPlan == nil{
3203+
if err != nil || fetchedInstallPlan == nil {
32023204
return false
32033205
}
32043206
if fetchedInstallPlan.Status.Phase != operatorsv1alpha1.InstallPlanPhaseInstalling {

‎test/e2e/operator_groups_e2e_test.go

Lines changed: 8 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1419,7 +1419,7 @@ var _ = Describe("Operator Group", func() {
14191419
aCSV := newCSV(csvName, opGroupNamespace, "", semver.MustParse("0.0.0"), []apiextensions.CustomResourceDefinition{mainCRD}, nil, &namedStrategy)
14201420

14211421
// Use the It spec name as label after stripping whitespaces
1422-
aCSV.Labels = map[string]string{"label": strings.Replace(CurrentGinkgoTestDescription().TestText, " ", "", -1)}
1422+
aCSV.Labels = map[string]string{"label": K8sSafeCurrentTestDescription()}
14231423
createdCSV, err := crc.OperatorsV1alpha1().ClusterServiceVersions(opGroupNamespace).Create(context.TODO(), &aCSV, metav1.CreateOptions{})
14241424
require.NoError(GinkgoT(), err)
14251425

@@ -1498,7 +1498,7 @@ var _ = Describe("Operator Group", func() {
14981498
})
14991499
require.NoError(GinkgoT(), err)
15001500

1501-
csvList, err := crc.OperatorsV1alpha1().ClusterServiceVersions(corev1.NamespaceAll).List(context.TODO(), metav1.ListOptions{LabelSelector: fmt.Sprintf("label=%s", strings.Replace(CurrentGinkgoTestDescription().TestText, " ", "", -1))})
1501+
csvList, err := crc.OperatorsV1alpha1().ClusterServiceVersions(corev1.NamespaceAll).List(context.TODO(), metav1.ListOptions{LabelSelector: fmt.Sprintf("label=%s", K8sSafeCurrentTestDescription())})
15021502
require.NoError(GinkgoT(), err)
15031503
GinkgoT().Logf("Found CSV count of %v", len(csvList.Items))
15041504
GinkgoT().Logf("Create other namespace %s", otherNamespaceName)
@@ -1795,7 +1795,8 @@ var _ = Describe("Operator Group", func() {
17951795

17961796
// Versions of OLM at 0.14.1 and older had a bug that would place the wrong namespace annotation on copied CSVs,
17971797
// preventing them from being GCd. This ensures that any leftover CSVs in that state are properly cleared up.
1798-
It("cleanup csvs with bad owner operator groups", func() {
1798+
// issue: https://github.com/operator-framework/operator-lifecycle-manager/issues/2644
1799+
It("[FLAKE] cleanup csvs with bad owner operator groups", func() {
17991800

18001801
c := newKubeClient()
18011802
crc := newCRClient()
@@ -1899,7 +1900,7 @@ var _ = Describe("Operator Group", func() {
18991900
aCSV := newCSV(csvName, opGroupNamespace, "", semver.MustParse("0.0.0"), []apiextensions.CustomResourceDefinition{mainCRD}, nil, &namedStrategy)
19001901

19011902
// Use the It spec name as label after stripping whitespaces
1902-
aCSV.Labels = map[string]string{"label": strings.Replace(CurrentGinkgoTestDescription().TestText, " ", "", -1)}
1903+
aCSV.Labels = map[string]string{"label": K8sSafeCurrentTestDescription()}
19031904
createdCSV, err := crc.OperatorsV1alpha1().ClusterServiceVersions(opGroupNamespace).Create(context.TODO(), &aCSV, metav1.CreateOptions{})
19041905
require.NoError(GinkgoT(), err)
19051906

@@ -1978,7 +1979,7 @@ var _ = Describe("Operator Group", func() {
19781979
})
19791980
require.NoError(GinkgoT(), err)
19801981

1981-
csvList, err := crc.OperatorsV1alpha1().ClusterServiceVersions(corev1.NamespaceAll).List(context.TODO(), metav1.ListOptions{LabelSelector: fmt.Sprintf("label=%s", strings.Replace(CurrentGinkgoTestDescription().TestText, " ", "", -1))})
1982+
csvList, err := crc.OperatorsV1alpha1().ClusterServiceVersions(corev1.NamespaceAll).List(context.TODO(), metav1.ListOptions{LabelSelector: fmt.Sprintf("label=%s", K8sSafeCurrentTestDescription())})
19821983
require.NoError(GinkgoT(), err)
19831984
GinkgoT().Logf("Found CSV count of %v", len(csvList.Items))
19841985
GinkgoT().Logf("Create other namespace %s", otherNamespaceName)
@@ -2288,7 +2289,8 @@ var _ = Describe("Operator Group", func() {
22882289
Expect(err).ToNot(HaveOccurred())
22892290
})
22902291

2291-
It("OLM applies labels to Namespaces that are associated with an OperatorGroup", func() {
2292+
// issue: https://github.com/operator-framework/operator-lifecycle-manager/issues/2637
2293+
It("[FLAKE] OLM applies labels to Namespaces that are associated with an OperatorGroup", func() {
22922294
ogLabel, err := getOGLabelKey(operatorGroup)
22932295
Expect(err).ToNot(HaveOccurred())
22942296

‎test/e2e/operator_test.go

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -70,7 +70,8 @@ var _ = Describe("Operator API", func() {
7070
// 14. Ensure the reference to ns-a is eventually removed from o's status.components.refs field
7171
// 15. Delete o
7272
// 16. Ensure o is not re-created
73-
It("should surface components in its status", func() {
73+
// issue: https://github.com/operator-framework/operator-lifecycle-manager/issues/2628
74+
It("[FLAKE] should surface components in its status", func() {
7475
o := &operatorsv1.Operator{}
7576
o.SetName(genName("o-"))
7677

‎test/e2e/subscription_e2e_test.go

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -508,7 +508,8 @@ var _ = Describe("Subscription", func() {
508508
require.Len(GinkgoT(), ips.Items, 2)
509509
})
510510

511-
It("updates multiple intermediates", func() {
511+
// issue: https://github.com/operator-framework/operator-lifecycle-manager/issues/2635
512+
It("[FLAKE] updates multiple intermediates", func() {
512513

513514
crd := newCRD("ins")
514515

@@ -1022,7 +1023,8 @@ var _ = Describe("Subscription", func() {
10221023
// - Delete the referenced InstallPlan
10231024
// - Wait for sub to have status condition SubscriptionInstallPlanMissing true
10241025
// - Ensure original non-InstallPlan status conditions remain after InstallPlan transitions
1025-
It("can reconcile InstallPlan status", func() {
1026+
// issue: https://github.com/operator-framework/operator-lifecycle-manager/issues/2645
1027+
It("[FLAKE] can reconcile InstallPlan status", func() {
10261028
c := newKubeClient()
10271029
crc := newCRClient()
10281030

0 commit comments

Comments
 (0)