Skip to content

Doc updates for NLB instance mode support #2007

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
May 14, 2021
Merged

Doc updates for NLB instance mode support #2007

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 35 additions & 4 deletions docs/guide/service/annotations.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,10 +14,11 @@
| [service.beta.kubernetes.io/load-balancer-source-ranges](#lb-source-ranges) | stringList | | |
| [service.beta.kubernetes.io/aws-load-balancer-type](#lb-type) | string | | |
| [service.beta.kubernetes.io/aws-load-balancer-nlb-target-type](#nlb-target-type) | string | | |
| service.beta.kubernetes.io/aws-load-balancer-name | string | | |
| [service.beta.kubernetes.io/aws-load-balancer-internal](#lb-internal) | boolean | false | |
| [service.beta.kubernetes.io/aws-load-balancer-name](#load-balancer-name) | string | | |
| [service.beta.kubernetes.io/aws-load-balancer-internal](#lb-internal) | boolean | false | deprecated, in favor of [aws-load-balancer-scheme](#lb-scheme)|
| [service.beta.kubernetes.io/aws-load-balancer-scheme](#lb-scheme) | string | internal | |
| [service.beta.kubernetes.io/aws-load-balancer-proxy-protocol](#proxy-protocol-v2) | string | | Set to `"*"` to enable |
| service.beta.kubernetes.io/aws-load-balancer-ip-address-type | string | ipv4 | ipv4 \| dualstack |
| [service.beta.kubernetes.io/aws-load-balancer-ip-address-type](#ip-address-type) | string | ipv4 | ipv4 \| dualstack |
| service.beta.kubernetes.io/aws-load-balancer-access-log-enabled | boolean | false | |
| service.beta.kubernetes.io/aws-load-balancer-access-log-s3-bucket-name | string | | |
| service.beta.kubernetes.io/aws-load-balancer-access-log-s3-bucket-prefix | string | | |
Expand Down Expand Up @@ -45,6 +46,16 @@
## Traffic Routing
Traffic Routing can be controlled with following annotations:

- <a name="load-balancer-name">`service.beta.kubernetes.io/aws-load-balancer-name`</a> specifies the custom name to use for the load balancer.

!!!note "limitations"
- If you modify this annotation after service creation, there is no effect.

!!!example
```
service.beta.kubernetes.io/load-balancer-name: custom-name
```

- <a name="lb-type">`service.beta.kubernetes.io/aws-load-balancer-type`</a> specifies the load balancer type. This controller reconciles those service resources with this annotation set to either `nlb-ip` or `external`.

!!!note ""
Expand All @@ -64,7 +75,7 @@ Traffic Routing can be controlled with following annotations:
- `instance` mode will route traffic to all EC2 instances within cluster on the [NodePort](https://kubernetes.io/docs/concepts/services-networking/service/#nodeport) opened for your service.

!!!note ""
service must be of type "NodePort" or "LoadBalancer" for `instance` targets
service must be of type `NodePort` or `LoadBalancer` for `instance` targets

- `ip` mode will route traffic directly to the pod IP.

Expand Down Expand Up @@ -119,6 +130,16 @@ on the load balancer.
service.beta.kubernetes.io/aws-load-balancer-target-node-labels: label1=value1, label2=value2
```

## Traffic Listening
Traffic Listening can be controlled with following annotations:

- <a name="ip-address-type">`service.beta.kubernetes.io/aws-load-balancer-ip-address-type`</a> specifies the [IP address type](https://docs.aws.amazon.com/elasticloadbalancing/latest/network/network-load-balancers.html#ip-address-type) of NLB.

!!!example
```
service.beta.kubernetes.io/aws-load-balancer-ip-address-type: ipv4
```

## Resource attributes
NLB resource attributes can be controlled via the following annotations:

Expand Down Expand Up @@ -176,8 +197,18 @@ Load balancer access can be controllerd via following annotations:
service.beta.kubernetes.io/load-balancer-source-ranges: 10.0.0.0/24
```

- <a name="lb-scheme">`service.beta.kubernetes.io/aws-load-balancer-scheme`</a> specifies whether the NLB will be internet-facing or internal. Valid values are `internal`, `internet-facing`. If not specified, default is `internal`.

!!!example
```
service.beta.kubernetes.io/aws-load-balancer-scheme: "internet-facing"
```

- <a name="lb-internal">`service.beta.kubernetes.io/aws-load-balancer-internal`</a> specifies whether the NLB will be internet-facing or internal.

!!!note "deprecation note"
This annotation is deprecated starting v2.2.0 release in favor of the new [aws-load-balancer-scheme](#lb-scheme) annotation. It will will be supported, but in case of ties, the aws-load-balancer-scheme gets precedence.

!!!example
```
service.beta.kubernetes.io/aws-load-balancer-internal: "true"
Expand Down
84 changes: 84 additions & 0 deletions docs/guide/service/nlb.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
# Network Load Balancer
AWS Load Balancer Controller supports Network Load Balancer (NLB) with instance or IP targets through Kubernetes service of type `LoadBalancer` with proper annotations.

### Instance mode
Instance target mode supports pods running on AWS EC2 instances. In this mode, AWS NLB sends traffic to the instances and the `kube-proxy` on the individual worker nodes forward it to the pods through one or more worker nodes in the Kubernetes cluster.
### IP mode
IP target mode supports pods running on AWS EC2 instances and AWS Fargate. In this mode, the AWS NLB targets traffic directly to the Kubernetes pods behind the service, eliminating the need for an extra network hop through the worker nodes in the Kubernetes cluster.

## Prerequisites
* AWS LoadBalancer Controller >= v2.2.0
* Kubernetes >= v1.16 for Service type `NodePort`
* Kubernetes >= v1.20 or EKS >= 1.16 or the following patch releases for Service type `LoadBalancer`
- 1.18.18+ for 1.18
- 1.19.10+ for 1.19
* Pods have native AWS VPC networking configured, see [Amazon VPC CNI plugin](https://github.com/aws/amazon-vpc-cni-k8s)

!!!note "secure by default"
Starting v2.2.0 release, the AWS Load balancer controller provisions an internal NLB by default. To create an internet-facing load balancer, apply the following annotation to your service:

```
service.beta.kubernetes.io/aws-load-balancer-scheme: internet-facing
```

For backwards compatibility, if this annotation is not specified, existing NLB will continue to use the scheme configured on the AWS resource.

## Configuration
The service resources of type `LoadBalancer` also get reconciled by the kubernetes controller built into the cloudprovider component of the `kube-controller-manager` or the `cloud-controller-manager` aka the `in-tree` controller. The AWS in-tree controller
ignores those services resources that have the `service.beta.kubernetes.io/aws-load-balancer-type` annotation as `external`. The AWS Load balancer controller support for NLB is based on the in-tree cloud controller ignoring the service resources, so it is very important
to apply the following annotation on the service resource during creation:

```yaml
service.beta.kubernetes.io/aws-load-balancer-type: "external"
```
This `external` value to the above annotation causes the in-tree controller to not process the service resource and thus pass it on to the external controller.

!!!warning "annotation modification"
Do not modify or add the `service.beta.kubernetes.io/aws-load-balancer-type` annotation on an existing service object. If you need to make changes, for example from classic to NLB or NLB managed
by the in-tree controller to the one managed by the AWS Load balancer controller, delete the kubernetes service first and then create again with the correct annotation. If you modify the annotation after service creation
you will end up with leaked AWS load balancer resources.

### IP mode
NLB IP mode is determined based on the `service.beta.kubernetes.io/aws-load-balancer-nlb-target-type` annotation. If the annotation value is `ip`, then NLB will be provisioned in IP mode. Here is the manifest snippet:
```yaml
metadata:
name: my-service
annotations:
service.beta.kubernetes.io/aws-load-balancer-type: "external"
service.beta.kubernetes.io/aws-load-balancer-nlb-target-type: "ip"
```

!!!note "backwards compatibility"
For backwards compatibility, controller still supports the `nlb-ip` as the type annotation. For example, if you specify

```
service.beta.kubernetes.io/aws-load-balancer-type: nlb-ip
```

the controller will provision NLB in IP mode. With this, the `service.beta.kubernetes.io/aws-load-balancer-nlb-target-type` annotation gets ignored.

### Instance mode
Similar to the IP mode, the instance mode is based on the annotation `service.beta.kubernetes.io/aws-load-balancer-type` value `instance`. Here is a sample manifest snippet:

```yaml
metadata:
name: my-service
annotations:
service.beta.kubernetes.io/aws-load-balancer-type: "external"
service.beta.kubernetes.io/aws-load-balancer-nlb-target-type: "instance"
```

## Protocols
Support is available for both TCP and UDP protocols. In case of TCP, NLB in IP mode does not pass the client source IP address to the pods. You can configure [NLB proxy protocol v2](https://docs.aws.amazon.com/elasticloadbalancing/latest/network/load-balancer-target-groups.html#proxy-protocol) via [annotation](https://kubernetes.io/docs/concepts/services-networking/service/#proxy-protocol-support-on-aws) if you need the client source IP address.

to enable proxy protocol v2, apply the following annotation to your service:
```yaml
service.beta.kubernetes.io/aws-load-balancer-proxy-protocol: "*"
```

## Subnet tagging requirements
See [Subnet Discovery](https://kubernetes-sigs.github.io/aws-load-balancer-controller/guide/controller/subnet_discovery/) for details on configuring ELB for public or private placement.


## Security group
NLB does not currently support a managed security group. For ingress access, the controller will resolve the security group for the ENI corresponding to the endpoint pod for IP mode, and the security group of the worker nodes for instance mode. If there is a single security group attached to the the ENI or the instance, it gets used. In case of multiple security groups, the controller expects to find only one security group tagged with the Kubernetes cluster id. Controller will update the ingress rules on the security groups as per the service spec.
41 changes: 0 additions & 41 deletions docs/guide/service/nlb_ip_mode.md
View file
Open in desktop

This file was deleted.

4 changes: 2 additions & 2 deletions mkdocs.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ nav:
- IngressClass: guide/ingress/ingress_class.md
- Certificate Discovery: guide/ingress/cert_discovery.md
- Service:
- NLB-IP mode: guide/service/nlb_ip_mode.md
- NLB: guide/service/nlb.md
- Annotations: guide/service/annotations.md
- TargetGroupBinding:
- TargetGroupBinding: guide/targetgroupbinding/targetgroupbinding.md
Expand Down Expand Up @@ -63,4 +63,4 @@ markdown_extensions:
extra_css:
- https://unpkg.com/material-components-web@latest/dist/material-components-web.min.css
extra_javascript:
- https://unpkg.com/material-components-web@latest/dist/material-components-web.min.js
- https://unpkg.com/material-components-web@latest/dist/material-components-web.min.js