Skip to content

Commit b2c5146

Browse files
pleshakovsjbermankate-osbornADubhlaoich
authored
Add a guide about upgrading applications without downtime (#944)
Problem: As a user of NKG, I want a guide to explain the different strategies I can upgrade my application (NOT NKG) while using NKG specific features (such as traffic splitting) So that I minimize the downtime my application encounters. Solution: - Add a guide that covers NKG specific features that help upgrade applications without downtime - Introduce a new /docs/guide folder for guides. - Add a link to the guides folder from the main README. Testing: none Closes #704 Co-authored-by: Saylor Berman <[email protected]> Co-authored-by: Kate Osborn <[email protected]> Co-authored-by: Alan Dooley <[email protected]> Co-authored-by: Alan Dooley <[email protected]>
1 parent 1864626 commit b2c5146

File tree

2 files changed

+174
-0
lines changed

2 files changed

+174
-0
lines changed

‎README.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,7 @@ Learn about our [design principles](/docs/developer/design-principles.md) and [a
2525
available
2626
on [GitHub Container Registry](https://github.com/nginxinc/nginx-kubernetes-gateway/pkgs/container/nginx-kubernetes-gateway).
2727
4. Deploy various [examples](examples).
28+
5. Read our [guides](/docs/guides).
2829

2930
## NGINX Kubernetes Gateway Releases
3031

‎docs/guides/upgrade-apps-without-downtime.md

Lines changed: 173 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,173 @@
1+
# Using NGINX Kubernetes Gateway to Upgrade Applications without Downtime
2+
3+
This guide explains how to use NGINX Kubernetes Gateway to upgrade applications without downtime.
4+
5+
Multiple upgrade methods are mentioned, assuming existing familiarity: this guide focuses primarily on how to use NGINX
6+
Kubernetes Gateway to accomplish them.
7+
8+
> See the [Architecture document](/docs/architecture.md) to learn more about NGINX Kubernetes Gateway architecture.
9+
10+
## NGINX Kubernetes Gateway Functionality
11+
12+
To understand the upgrade methods, you should be aware of the NGINX features that help prevent application downtime:
13+
graceful configuration reloads and upstream server updates.
14+
15+
### Graceful Configuration Reloads
16+
17+
If a relevant Gateway API or built-in Kubernetes resource is changed, NGINX Kubernetes Gateway will update NGINX by
18+
regenerating the NGINX configuration. NGINX Kubernetes Gateway then sends a reload signal to the master NGINX process to
19+
apply the new configuration.
20+
21+
We call such an operation a reload, during which client requests are not dropped - which defines it as a graceful reload.
22+
23+
This process is further explained in [NGINX's documentation](https://nginx.org/en/docs/control.html?#reconfiguration).
24+
25+
### Upstream Server Updates
26+
27+
Endpoints frequently change during application upgrades: Kubernetes creates Pods for the new version of an application
28+
and removes the old ones, creating and removing the respective Endpoints as well.
29+
30+
NGINX Kubernetes Gateway detects changes to Endpoints by watching their corresponding [EndpointSlices][endpoint-slices].
31+
32+
[endpoint-slices]:https://kubernetes.io/docs/concepts/services-networking/endpoint-slices/
33+
34+
In NGINX configuration, a Service is represented as an [upstream][upstream], and an Endpoint as an
35+
[upstream server][upstream-server].
36+
37+
[upstream]:https://nginx.org/en/docs/http/ngx_http_upstream_module.html#upstream
38+
39+
[upstream-server]:https://nginx.org/en/docs/http/ngx_http_upstream_module.html#server
40+
41+
Two common cases are adding and removing Endpoints:
42+
43+
- If an Endpoint is added, NGINX Kubernetes Gateway adds an upstream server to NGINX that corresponds to the Endpoint,
44+
then reload NGINX. After that, NGINX will start proxying traffic to that Endpoint.
45+
- If an Endpoint is removed, NGINX Kubernetes Gateway removes the corresponding upstream server from NGINX. After
46+
a reload, NGINX will stop proxying traffic to it. However, it will finish proxying any pending requests to that
47+
server before switching to another Endpoint.
48+
49+
As long as you have more than one ready Endpoint, the clients should not experience any downtime during upgrades.
50+
51+
> It is good practice to configure a [Readiness probe][readiness-probe] in the Deployment so that a Pod can advertise
52+
> when it is ready to receive traffic. Note that NGINX Kubernetes Gateway will not add any Endpoint to NGINX that is not
53+
> ready.
54+
55+
[readiness-probe]:https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/
56+
57+
## Before You Begin
58+
59+
For the upgrade methods covered in the next sections, we make the following assumptions:
60+
61+
- You deploy your application as a [Deployment][deployment].
62+
- The Pods of the Deployment belong to a [Service][service] so that Kubernetes creates an [Endpoint][endpoints] for
63+
each Pod.
64+
- You expose the application to the clients via an [HTTPRoute][httproute] resource that references that Service.
65+
66+
[deployment]:https://kubernetes.io/docs/concepts/workloads/controllers/deployment/
67+
68+
[service]:https://kubernetes.io/docs/concepts/services-networking/service/
69+
70+
[httproute]:https://gateway-api.sigs.k8s.io/api-types/httproute/
71+
72+
[endpoints]:https://kubernetes.io/docs/reference/kubernetes-api/service-resources/endpoints-v1/
73+
74+
For example, an application can be exposed using a routing rule like below:
75+
76+
```yaml
77+
- matches:
78+
- path:
79+
type: PathPrefix
80+
value: /
81+
backendRefs:
82+
- name: my-app
83+
port: 80
84+
```
85+
86+
> See the [Cafe example](/examples/cafe-example) for a basic example.
87+
88+
The upgrade methods in the next sections cover:
89+
90+
- Rolling Deployment Upgrades
91+
- Blue-green Deployments
92+
- Canary Releases
93+
94+
## Rolling Deployment Upgrade
95+
96+
To start a [rolling Deployment upgrade][rolling-upgrade], you update the Deployment to use the new version tag of
97+
the application. As a result, Kubernetes terminates the Pods with the old version and create new ones. By default,
98+
Kubernetes also ensures that some number of Pods always stay available during the upgrade.
99+
100+
[rolling-upgrade]:https://kubernetes.io/docs/concepts/workloads/controllers/deployment/#rolling-update-deployment
101+
102+
Such an upgrade will add new upstream servers to NGINX and remove the old ones. As long as the number
103+
of Pods (ready Endpoints) during an upgrade does not reach zero, NGINX will be able to proxy traffic, and thus prevent
104+
any downtime.
105+
106+
This method does not require you to update the HTTPRoute.
107+
108+
## Blue-Green Deployments
109+
110+
With this method, you deploy a new version of the application (blue version) as a separate Deployment,
111+
while the old version (green) keeps running and handling client traffic. Next, you switch the traffic from the
112+
green version to the blue. If the blue works as expected, you terminate the green. Otherwise, you switch the traffic
113+
back to the green.
114+
115+
There are two ways to switch the traffic:
116+
117+
- Update the Service selector to select the Pods of the blue version instead of the green. As a result, NGINX Kubernetes
118+
Gateway removes the green upstream servers from NGINX and add the blue ones. With this approach, it is not
119+
necessary to update the HTTPRoute.
120+
- Create a separate Service for the blue version and update the backend reference in the HTTPRoute to reference this
121+
Service, which leads to the same result as with the previous option.
122+
123+
## Canary Releases
124+
125+
To support canary releases, you can implement an approach with two Deployments behind the same Service (see
126+
[Canary deployment][canary] in the Kubernetes documentation). However, this approach lacks precision for defining the
127+
traffic split between the old and the canary version. You can greatly influence it by controlling the number of Pods
128+
(for example, four Pods of the old version and one Pod of the canary). However, note that NGINX Kubernetes Gateway uses
129+
[`random two least_conn`][random-method] load balancing method, which doesn't guarantee an exact split based on the
130+
number of Pods (80/20 in the given example).
131+
132+
[canary]:https://kubernetes.io/docs/concepts/workloads/controllers/deployment/#canary-deployment
133+
[random-method]:https://nginx.org/en/docs/http/ngx_http_upstream_module.html#random
134+
135+
A more flexible and precise way to implement canary releases is to configure a traffic split in an HTTPRoute. In this
136+
case, you create a separate Deployment for the new version with a separate Service. For example, for the rule below,
137+
NGINX will proxy 95% of the traffic to the old version Endpoints and only 5% to the new ones.
138+
139+
```yaml
140+
- matches:
141+
- path:
142+
type: PathPrefix
143+
value: /
144+
backendRefs:
145+
- name: my-app-old
146+
port: 80
147+
weight: 95
148+
- name: my-app-new
149+
port: 80
150+
weight: 5
151+
```
152+
153+
> There is no stickiness for the requests from the same client. NGINX will independently split each request among
154+
> the backend references.
155+
156+
By updating the rule you can further increase the share of traffic the new version gets and finally completely switch
157+
to the new version:
158+
159+
```yaml
160+
- matches:
161+
- path:
162+
type: PathPrefix
163+
value: /
164+
backendRefs:
165+
- name: my-app-old
166+
port: 80
167+
weight: 0
168+
- name: my-app-new
169+
port: 80
170+
weight: 1
171+
```
172+
173+
See the [Traffic splitting example](/examples/traffic-splitting) from our repository.

0 commit comments

Comments
 (0)