You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/guide/service/annotations.md
+9-5Lines changed: 9 additions & 5 deletions
Original file line number
Diff line number
Diff line change
@@ -80,7 +80,7 @@ Traffic Routing can be controlled with following annotations:
80
80
81
81
- <aname="nlb-target-type">`service.beta.kubernetes.io/aws-load-balancer-nlb-target-type`</a> specifies the target type to configure for NLB. You can choose between
82
82
`instance` and `ip`.
83
-
- `instance` mode will route traffic to all EC2 instances within cluster on the [NodePort](https://kubernetes.io/docs/concepts/services-networking/service/#type-nodeport) opened for your service.
83
+
- `instance` mode will route traffic to all EC2 instances within cluster on the [NodePort](https://kubernetes.io/docs/concepts/services-networking/service/#type-nodeport) opened for your service. The kube-proxy on the individual worker nodes sets up the forwarding of the traffic from the NodePort to the pods behind the service.
84
84
85
85
!!!note ""
86
86
- service must be of type `NodePort` or `LoadBalancer` for `instance` targets
@@ -89,10 +89,14 @@ Traffic Routing can be controlled with following annotations:
89
89
!!!note "default value"
90
90
If you configure `spec.loadBalancerClass`, the controller defaults to `instance` target type
91
91
92
-
- `ip` mode will route traffic directly to the pod IP.
92
+
!!!warning "NodePort allocation"
93
+
k8s version 1.22 and later support disabling NodePort allocation by setting the service field `spec.allocateLoadBalancerNodePorts` to `false`. If the NodePort is not allocated for a service port, the controller will fail to reconcile instance mode NLB.
94
+
95
+
- `ip` mode will route traffic directly to the pod IP. In this mode, AWS NLB sends traffic directly to the Kubernetes pods behind the service, eliminating the need for an extra network hop through the worker nodes in the Kubernetes cluster.
93
96
94
97
!!!note ""
95
-
network plugin must use native AWS VPC networking configuration for pod IP, for example [Amazon VPC CNI plugin](https://github.com/aws/amazon-vpc-cni-k8s).
98
+
- `ip` target mode supports pods running on AWS EC2 instances and AWS Fargate
99
+
- network plugin must use native AWS VPC networking configuration for pod IP, for example [Amazon VPC CNI plugin](https://github.com/aws/amazon-vpc-cni-k8s).
96
100
97
101
!!!example
98
102
```
@@ -282,8 +286,8 @@ Health check on target groups can be configured with following annotations:
282
286
283
287
- <aname="healthcheck-port">`service.beta.kubernetes.io/aws-load-balancer-healthcheck-port`</a> specifies the TCP port to use for target group health check.
284
288
285
-
!!!note ""
286
-
- if you do not specify the health check port, controller uses `traffic-port`as default value
289
+
!!!note "default value"
290
+
- if you do not specify the health check port, the default value will be `spec.healthCheckNodePort` when `externalTrafficPolicy=local` or `traffic-port`otherwise.
AWS Load Balancer Controller supports Network Load Balancer (NLB) with instance or IP targets through Kubernetes service of type `LoadBalancer` with proper annotations.
3
2
4
-
### Instance mode
5
-
Instance target mode supports pods running on AWS EC2 instances. In this mode, AWS NLB sends traffic to the instances and the `kube-proxy` on the individual worker nodes forward it to the pods through one or more worker nodes in the Kubernetes cluster.
6
-
### IP mode
7
-
IP target mode supports pods running on AWS EC2 instances and AWS Fargate. In this mode, the AWS NLB targets traffic directly to the Kubernetes pods behind the service, eliminating the need for an extra network hop through the worker nodes in the Kubernetes cluster.
3
+
AWS Load Balancer Controller supports reconciliation for Kubernetes Services resources of type `LoadBalancer` by Network Load Balancer (NLB) with `instance` or `ip` target type.
8
4
9
-
## Prerequisites
10
-
* AWS LoadBalancer Controller >= v2.2.0
11
-
* Kubernetes >= v1.16 for Service type `NodePort`
12
-
* Kubernetes >= v1.20 or EKS >= 1.16 or the following patch releases for Service type `LoadBalancer`
13
-
- 1.18.18+ for 1.18
14
-
- 1.19.10+ for 1.19
15
-
* Pods have native AWS VPC networking configured, see [Amazon VPC CNI plugin](https://github.com/aws/amazon-vpc-cni-k8s)
16
-
17
-
!!!note "secure by default"
18
-
Starting v2.2.0 release, the AWS Load balancer controller provisions an internal NLB by default. To create an internet-facing load balancer, apply the following annotation to your service:
5
+
!!! info "secure by default"
6
+
Since [:octicons-tag-24: v2.2.0](https://github.com/kubernetes-sigs/aws-load-balancer-controller/releases/tag/v2.2.0) release, the AWS Load balancer controller provisions an `internal` NLB by default.
7
+
8
+
To create an `internet-facing` NLB, following annotation is required on your service:
For backwards compatibility, if this annotation is not specified, existing NLB will continue to use the scheme configured on the AWS resource.
25
-
26
-
## Configuration
27
-
The service resources of type `LoadBalancer` also get reconciled by the kubernetes controller built into the cloudprovider component of the `kube-controller-manager` or the `cloud-controller-manager` aka the `in-tree` controller. The AWS in-tree controller
28
-
ignores those services resources that have the `service.beta.kubernetes.io/aws-load-balancer-type` annotation as `external`. The AWS Load balancer controller support for NLB is based on the in-tree cloud controller ignoring the service resources, so it is very important
29
-
to apply the following annotation on the service resource during creation:
This `external` value to the above annotation causes the in-tree controller to not process the service resource and thus pass it on to the external controller.
14
+
!!! tip ""
15
+
For backwards compatibility, if the [`service.beta.kubernetes.io/aws-load-balancer-scheme`](./annotations.md#lb-scheme) annotation is absent, existing NLB's scheme will remain unchanged.
35
16
36
-
!!!warning "annotation modification"
37
-
Do not modify or add the `service.beta.kubernetes.io/aws-load-balancer-type` annotation on an existing service object. If you need to make changes, for example from classic to NLB or NLB managed
38
-
by the in-tree controller to the one managed by the AWS Load balancer controller, delete the kubernetes service first and then create again with the correct annotation. If you modify the annotation after service creation
39
-
you will end up with leaked AWS load balancer resources.
40
-
41
-
### IP mode
42
-
NLB IP mode is determined based on the `service.beta.kubernetes.io/aws-load-balancer-nlb-target-type` annotation. If the annotation value is `ip`, then NLB will be provisioned in IP mode. Here is the manifest snippet:
* For Kubernetes Services resources of type `LoadBalancer`:
20
+
* Kubernetes >= v1.20 or
21
+
* Kubernetes >= v1.19.10 for 1.19 or
22
+
* Kubernetes >= v1.18.18 for 1.18 or
23
+
* EKS >= v1.16
24
+
* For Kubernetes Services resources of type `NodePort`:
25
+
* Kubernetes >= v1.16
26
+
* For `ip` target type:
27
+
* Pods have native AWS VPC networking configured, see [Amazon VPC CNI plugin](https://github.com/aws/amazon-vpc-cni-k8s)
57
28
58
-
the controller will provision NLB in IP mode. With this, the `service.beta.kubernetes.io/aws-load-balancer-nlb-target-type` annotation gets ignored.
29
+
## Configuration
59
30
60
-
### Instance mode
61
-
Similar to the IP mode, the instance mode is based on the annotation `service.beta.kubernetes.io/aws-load-balancer-nlb-target-type` value `instance`. Here is a sample manifest snippet:
62
-
!!!warning "NodePort allocation"
63
-
k8s version 1.22 and later support disabling NodePort allocation by setting the service field `spec.allocateLoadBalancerNodePorts` to `false`. If the NodePort is not allocated for a service port, the controller will fail to reconcile instance mode NLB.
By default, Kubernetes Service resources of type `LoadBalancer` gets reconciled by the Kubernetes controller built into the CloudProvider component of the kube-controller-manager or the cloud-controller-manager(a.k.a. the in-tree controller).
32
+
33
+
In order to let AWS Load Balancer Controller manage the reconciliation for Kubernetes Services resources of type `LoadBalancer`, you need to offload the reconciliation from in-tree controller to AWS Load Balancer Controller explicitly.
34
+
35
+
36
+
=== "With LoadBalancerClass"
37
+
AWS Load Balancer Controller supports `LoadBalancerClass` feature since [:octicons-tag-24: v2.4.0](https://github.com/kubernetes-sigs/aws-load-balancer-controller/releases/tag/v2.4.0) release for Kubernetes v1.22+ clusters.
38
+
39
+
`LoadBalancerClass` feature provides a CloudProvider agnostic way of offloading the reconciliation for Kubernetes Services resources of type `LoadBalancer` to an external controller.
40
+
41
+
When you specify the `spec.loadBalancerClass` to be `service.k8s.aws/nlb` on a Kubernetes Service resource of type `LoadBalancer`, the AWS Load Balancer Controller takes charge of reconciliation by provision an NLB.
42
+
43
+
!!! warning
44
+
- If you modify a Service resource with matching `spec.loadBalancerClass` by changing its `type` from `LoadBalancer` to anything else, the controller will cleanup provioned NLB for that Service.
45
+
46
+
- If the `spec.loadBalancerClass` is set to a loadBalancerClass that is not recognized by this controller, it will ignore the Service resource regardless of the `service.beta.kubernetes.io/aws-load-balancer-type` annotation.
47
+
48
+
!!! tip
49
+
- By default, the NLB will use `instance` target-type, you can customize it via the [`service.beta.kubernetes.io/aws-load-balancer-nlb-target-type` annotation](./annotations.md#nlb-target-type)
50
+
51
+
- This controller uses `service.k8s.aws/nlb` as the default `LoadBalancerClass`, you can customize it to a different value via the controller flag `--load-balancer-class`.
The AWS in-tree controller supports an AWS specific way of offloading the reconciliation for Kubernetes Services resources of type `LoadBalancer` to an external controller.
93
+
94
+
When you specify the [`service.beta.kubernetes.io/aws-load-balancer-type` annotation](./annotations.md#lb-type) to be `external` on a Kubernetes Service resource of type `LoadBalancer`, the in-tree controller will ignore the Service resource. In addition, If you specify the [`service.beta.kubernetes.io/aws-load-balancer-nlb-target-type` annotation](./annotations.md#nlb-target-type) on the Service resource, the AWS Load Balancer Controller takes charge of reconciliation by provision an NLB.
95
+
96
+
!!! warning
97
+
- It's not recommended to modify or add the `service.beta.kubernetes.io/aws-load-balancer-type` annotation on an existing Service resource. Instead, delete the existing Service resource and recreate a new one if a change is desired.
98
+
99
+
- If you modify this annotation on a existing Service resource, you might end up with leaked AWS Load Balancer resources.
100
+
101
+
!!! note "backwards compatibility for `nlb-ip` type"
102
+
For backwards compatibility, both in-tree and AWS Load Balancer controller supports `nlb-ip` as value of `service.beta.kubernetes.io/aws-load-balancer-type` annotation. The controllers treats it as if you specified both annotation below:
service.beta.kubernetes.io/aws-load-balancer-nlb-target-type: ip
136
+
spec:
137
+
selector:
138
+
app: echoserver
139
+
ports:
140
+
- port: 80
141
+
targetPort: 8080
142
+
protocol: TCP
143
+
type: LoadBalancer
144
+
```
72
145
73
146
## Protocols
74
147
Controller supports both TCP and UDP protocols. Controller also configures TLS termination on NLB if you configure service with certificate annotation.
See [Subnet Discovery](../../deploy/subnet_discovery.md) for details on configuring ELB for public or private placement.
89
162
90
-
91
163
## Security group
92
-
NLB does not currently support managed security groups. For ingress access, the controller adds inbound rules to the node security group for the instance mode, or the ENI security group for the IP mode. In case of multiple
93
-
security groups, the controller expects only one security group tagged with the cluster name as follows:
164
+
AWS currently does not support attaching security groups to NLB. To allow inbound traffic from NLB, the controller automatically adds inbound rules to the worker node security groups by default.
165
+
166
+
!!! tip "disable worker node security group rule management"
167
+
You can disable the worker node security group rule management via [annotation](./annotations.md#manage-backend-sg-rules).
168
+
169
+
### Worker node security groups selection
170
+
The controller automatically selects the worker node security groups that will be modified to allow inbound traffic with following rules:
171
+
172
+
* for `instance` mode, the security group of the each backend worker node's primary ENI will be selected.
173
+
* for `ip` mode, the security group of each backend pod's ENI will be selected.
174
+
175
+
!!! important "multiple security groups on an ENI"
176
+
177
+
if there are multiple security groups attached on an ENI, the controller expects only one security group tagged with following tags:
The AWS Load Balancer Controller supports `LoadBalancerClass` starting v2.4.0 release on k8s 1.22 or later clusters. The LoadBalancerClass provides a cloudprovider agnostic way of offloading the load balancer reconciliation to an external controller. This controller uses the `service.k8s.aws/nlb` as the default class,
103
-
you can configure it to a different value via the controller flag `--load-balancer-class`.
194
+
=== "when Client IP preservation disabled"
104
195
105
-
When you specify the `spec.loadBalancerClass` on a service of type `LoadBalancer` during service creation, this controller creates an internal NLB with instance targets by default. If the LoadBalancerClass is not the configured for this controller, this controller ignores the service resource completely regardless of the annotation
106
-
`service.beta.kubernetes.io/aws-load-balancer-type`. If you modify the service, with `spec.loadBalancerClass`, type from `LoadBalancer` to anything else, the controller will cleanup the NLB.
0 commit comments