ci/tests: add timeout to ansible e2e #2045
Conversation
openshift-ci-robot
added
the
size/XS
There was a problem hiding this comment.
Do we know why these deletions are failing? We don't need to hold this PR up to answer that, but I feel like part of our CI problem is that we're putting band-aids on it here and there and sometimes ignoring or putting off looking into the root causes of the failures and flakes.
ci/tests/e2e-ansible.sh
Outdated
| @@ -44,8 +44,8 @@ remove_operator() { | |||
| kubectl delete --wait=true --ignore-not-found=true -f "$OPERATORDIR/deploy/service_account.yaml" | |||
| kubectl delete --wait=true --ignore-not-found=true -f "$OPERATORDIR/deploy/role.yaml" | |||
| kubectl delete --wait=true --ignore-not-found=true -f "$OPERATORDIR/deploy/role_binding.yaml" | |||
| kubectl delete --wait=true --ignore-not-found=true -f "$OPERATORDIR/deploy/crds/ansible.example.com_memcacheds_crd.yaml" | |||
| kubectl delete --wait=true --ignore-not-found=true -f "$OPERATORDIR/deploy/crds/ansible.example.com_foos_crd.yaml" | |||
| timeout 60s bash -c -- "kubectl delete --wait=true --ignore-not-found=true -f \"$OPERATORDIR/deploy/crds/ansible.example.com_memcacheds_crd.yaml\"" | |||
There was a problem hiding this comment.
I think for these, we can use kubectl's --timeout flag rather than wrapping the bash timeout command.
There was a problem hiding this comment.
I agree, just used bash initially since that's what came to mind. Re: why these are failing, I need to take a deeper dive, just trying to shorten the time to wait to fail right now
There was a problem hiding this comment.
Can you add a TODO comment above these lines that says something about how these delete commands can hang for hours, that we're adding --timeout as a (hopefully temporary) stopgap, and that we need to investigate more. And then create a ticket in our Jira backlog about doing that investigation.
LGTM once we have that stuff in place.
* ci/tests: add timeout to ansible e2e * use kubectl inbuilt timeout * add TODO comment
Description of the change:
Add 60s timeout when deleting CRD's on the Ansible e2e tests
Motivation for the change:
If the deletion fails, it can take up to 4 hours for CI to timeout itself