Add playbooks to allow hotfixing containers

[MoteHue]

No description provided.

[Alex-Welsh]

Added docs

[Alex-Welsh]

For the record, I'm not happy with how regex is handled, and the way it needs to match the entire name not just a substring. I don't know of a better way to do it though so any suggestions are more than welcome

[JohnGarbutt]

So it's a bit late for this comment, but we could generate an image with a thin delta that is just the proposed hot fix, then push that image everywhere? We get to more easily track where it is applied then

[Alex-Welsh]

So it's a bit late for this comment, but we could generate an image with a thin delta that is just the proposed hot fix, then push that image everywhere? We get to more easily track where it is applied then

I'd be interested to hear other opinions but in my mind it might be an "as well as" rather than an "instead of". This playbook should really only be a last resort if we urgently need to get something out.

I can see the two different workflows being:
Major issue discovered -> hotfix with this playbook -> get longer term fix into release train.
Minor issue discovered -> generate new image & diff -> roll out to deployments

[MoteHue]

in my mind it might be an "as well as" rather than an "instead of"

I'd agree with this. This hotfix playbook isn't meant to be the one true solution to urgently resolving a critical issue/vulnerability.

[markgoddard]

So it's a bit late for this comment, but we could generate an image with a thin delta that is just the proposed hot fix, then push that image everywhere? We get to more easily track where it is applied then

It's an interesting alternative, since it reduces the risk of losing the temporary changes in the hotfix. I think the (your) intention here was for this to allow getting fixes out for zero day issues as fast as possible. If we need to build and distribute an image then it's going to take longer, and at some (unknown) point we might as well build images from scratch and do it "properly".

[markgoddard]

Like it! Just one issue with the fail task.