Skip to content

Refactor josh #2

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 54 commits into from
Apr 3, 2017
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
54 commits
Select commit Hold shift + click to select a range
13631ea
On resource record set create, check for existing record based on hos…
joshrosso Mar 23, 2017
a6bab13
Merge refactor changes after fix to r53 deletion logic
joshrosso Mar 23, 2017
fd2c2a6
We should only wait for the TG to come online if we are creating a re…
bigkraig Mar 23, 2017
7244a38
Split up our types into their own files for easier management
bigkraig Mar 23, 2017
15818ba
If a TG fails to modify assume that its state does not change and kee…
bigkraig Mar 23, 2017
112dde4
Merge branch 'refactor' of https://git.tmaws.io/kubernetes/alb-ingres…
joshrosso Mar 23, 2017
c8f5156
Initial route53 validation logic for existing record set
joshrosso Mar 23, 2017
ded9739
Check record set change status
joshrosso Mar 23, 2017
51494fc
WIP, gutting newALBIngressesFromIngress
bigkraig Mar 24, 2017
41c4bd9
Finished reworking newALBIngressFromIngress
bigkraig Mar 24, 2017
368e6a2
Added some comments around rule processing
bigkraig Mar 24, 2017
7695a5d
Fix bug with uninitialized lock
bigkraig Mar 24, 2017
af2a062
Merge refactor; fix conflicts with route53
joshrosso Mar 24, 2017
2f54885
Update ALBIngress type to match newest changes
joshrosso Mar 24, 2017
d2e4198
Added some code for create/modify/delete rules
bigkraig Mar 24, 2017
eaf59be
Make OnUpdate synchronous
joshrosso Mar 24, 2017
ec8ef5c
Delete old resource record, after new record success, when name changes
joshrosso Mar 24, 2017
af8258a
Reintroduce concurrent-capable OnUpdate; but ensure lock can be aquir…
joshrosso Mar 24, 2017
8c6f5fa
On resource record set create, check for existing record based on hos…
joshrosso Mar 23, 2017
d98860e
Initial route53 validation logic for existing record set
joshrosso Mar 23, 2017
6877e5e
Check record set change status
joshrosso Mar 23, 2017
e5034d8
Update ALBIngress type to match newest changes
joshrosso Mar 24, 2017
bd92d55
Make OnUpdate synchronous
joshrosso Mar 24, 2017
c02ed47
Delete old resource record, after new record success, when name changes
joshrosso Mar 24, 2017
9c57390
Reintroduce concurrent-capable OnUpdate; but ensure lock can be aquir…
joshrosso Mar 24, 2017
a14df35
Merge branch 'refactor' of https://git.tmaws.io/kubernetes/alb-ingres…
joshrosso Mar 24, 2017
75ef370
Fixing the tests
bigkraig Mar 24, 2017
e401eaf
Switch to SyncState
bigkraig Mar 24, 2017
a04617e
Fixing annotation error
bigkraig Mar 24, 2017
4060536
sending back new list of loadbalancers incase we delete some
bigkraig Mar 24, 2017
2555775
Started SyncState on TG and added support for changign LB tags
bigkraig Mar 24, 2017
d9572a2
tried to clarify whats happening with r53 in newalbingress and commen…
bigkraig Mar 24, 2017
c2b1be3
Add SyncState for Targets, Listeners, and Route53
joshrosso Mar 25, 2017
7995100
Release lock after completion of NetALBIngressFromIngress; Ensure RRS…
joshrosso Mar 26, 2017
1682f82
Retain deletable ingresses in list for next sync operation
joshrosso Mar 26, 2017
9b714c2
Make TargetGroup deletion more reiliant to in flight Listener Deletions.
joshrosso Mar 26, 2017
3d0af42
Ensure record is only deleted prior to upsert when required (e.g. whe…
joshrosso Mar 26, 2017
bfa4da1
Add more usage docs; example manifests; and aws policy
joshrosso Mar 26, 2017
42e9b12
Comments around ALBIngress management and SyncState
joshrosso Mar 27, 2017
ba501a8
Add logic and logging to handle deletions reappearing in list. Move to
joshrosso Mar 28, 2017
e25739b
Add switch logic to resource record set code
joshrosso Mar 30, 2017
0b3358b
Don't throw errors when batchchange is invalid.
joshrosso Mar 30, 2017
06bd76e
Handle listener failures based on the type of error returned
joshrosso Mar 30, 2017
c947815
Fix deletion logic ensuring deletion don't loop endlessly
joshrosso Mar 31, 2017
a630355
Remove unused glog from loadbalancers
joshrosso Mar 31, 2017
7c3542c
Correct needs modification logic to return true when modification is
joshrosso Mar 31, 2017
96f9cde
Restructure logging, Loadbalancer & Resource Record Set
joshrosso Apr 1, 2017
e216d68
Add listener and target group logging. Propogate albIngress IDs across
joshrosso Apr 1, 2017
62a877d
Remove noise by introducing optional DEBUG logger
joshrosso Apr 2, 2017
8894f8e
Modify Tag logic now handles deletions and upserts
joshrosso Apr 3, 2017
6bf6834
Customized prettify logic to place elements inline
joshrosso Apr 3, 2017
eb629ef
Fix logging of SG struct
joshrosso Apr 3, 2017
4ae3cba
Prevent looping SG modification logic and always log CurrentState
joshrosso Apr 3, 2017
a6a44b9
Handle ALB scheme modifications; correct delete evaluate in r53_resou…
joshrosso Apr 3, 2017
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
59 changes: 56 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,13 +1,66 @@
[![build status](http://git.tmaws.io/kubernetes/alb-ingress/badges/master/build.svg)](http://git.tmaws.io/kubernetes/alb-ingress/commits/master) [![coverage report](http://git.tmaws.io/kubernetes/alb-ingress/badges/master/coverage.svg)](http://git.tmaws.io/kubernetes/alb-ingress/commits/master)


# ALB Ingress Controller

The ALB ingress controller satisfies Kubernetes [ingress resources](https://kubernetes.io/docs/user-guide/ingress) by provisioning an [Application Load Balancer](https://aws.amazon.com/elasticloadbalancing/applicationloadbalancer) and Route 53 DNS record set.
The ALB Ingress Controller satisfies Kubernetes [ingress resources](https://kubernetes.io/docs/user-guide/ingress) by provisioning an [Application Load Balancer](https://aws.amazon.com/elasticloadbalancing/applicationloadbalancer) and Route 53 DNS record set.

## Usage

TODO
This section details deployment of the controller and its behavior regarding ingress resources.

### Deployment and Configuration

The ALB Ingress Controller is a [Kubernetes deployment](https://kubernetes.io/docs/user-guide/deployments). Only a single instance should be run at a time. Any issues, crashes, or other rescheduling needs will be handled by Kubernetes natively. See the [alb-ingress-controller.yaml inside examples](./examples/alb-ingress-controller.yaml) for a sample deployment manifest.

**[TODO]**: Need to validate iam-policy.json mentioned below and see if it can be refined.

In order to perform operations, the controller must be able to resolve an IAM role capable of accessing and provisioning ALB and Route53 resources. There are many ways to achieve this, such as loading `AWS_ACCESS_KEY_ID`/`AWS_ACCESS_SECRET_KEY` as environment variables or using [kube2iam](https://github.com/jtblin/kube2iam). A sample IAM policy with the minimum permissions to run the controller can be found in [examples/alb-iam-policy.json](examples/iam-policy.json).

**[TODO]**: Need to verify ingress.class, mentioned below, works OOTB with this controller. IF not, seems very valuable to implement.

The controller will see ingress events for all namespaces in your cluster. Ingress resources that do not contain [necessary annotations](#annotations) will automatically be ignored. However, you may wish to limit the scope of ingress resources this controller has visibility into. In this case, you can define an `ingress.class` annotation, set the `--watch-namespace=` argument, or both.

Setting the `kubernetes.io/ingress.class: "alb"` annotation allows for classification of ingress resources and is especially helpful when running multiple ingress controllers in the same cluster. See [Using Multiple Ingress Controllers](https://github.com/nginxinc/kubernetes-ingress/tree/master/examples/multiple-ingress-controllers#using-multiple-ingress-controllers) for more details.

Setting the `--watch-namespace` argument constrains the ALB ingress-controller's scope to a **single** namespace. Ingress events outside of the namespace specified here will not be seen by the controller. Currently you cannot specify a watch on multiple namespaces or blacklist specific namespaces. See [this Kubernetes issue](https://github.com/kubernetes/contrib/issues/847) for more details.

Once configured as needed, the controller can be deployed like any Kubernetes deployment.

```bash
$ kubectl apply -f alb-ingress-controller.yaml
```

### Ingress Behavior

Periodically, ingress update events are seen by the controller. The controller retains a list of all ingress resources it knows about, along with the current state of AWS components that satisfy them. When an update event is fired, the controller re-scans the list of ingress resources known to the cluster and determines, by comparing the list to its previously stored one, the ingresses requiring deletion, creation or modification.

An example ingress, from `example/2048/2048-ingress.yaml` is as follows.

```yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: "nginx-ingress"
namespace: "2048-game"
annotations:
alb.ingress.kubernetes.io/scheme: internet-facing
alb.ingress.kubernetes.io/subnets: subnet-0d2bab64,subnet-9c569ce7
alb.ingress.kubernetes.io/security-groups: sg-ccaacaa5
alb.ingress.kubernetes.io/tags: Environment=dev1,ProductCode=PRD999
labels:
app: 2048-nginx-ingress
spec:
rules:
- host: 2048.yourdomain.com
http:
paths:
- path: /
backend:
serviceName: "service-2048"
servicePort: 80
```

The host field specifies the eventual Route 53-managed domain that will route to this service. The service, service-2048, must be of type NodePort (see [examples/echoservice/echoserver-service.yaml](examples/echoservice/echoserver-service.yaml)) in order for the provisioned ALB to route to it. If no NodePort exists, the controller will not attempt to provision resources in AWS. For details on purpose of annotations seen above, see [Annotations](#annotations).

## Annotations

Expand Down
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
76 changes: 76 additions & 0 deletions examples/alb-ingress-controller.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
# Application Load Balancer (ALB) Ingress Controller Deploymnet Manifest.
# This manifest details sensible defaults for deploying an ALB Ingress Controller.
# Github: https://github.com/coreos-inc/alb-ingress-controller
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
labels:
app: alb-ingress-controller
name: alb-ingress-controller
# Namespace the ALB Ingress Controller should run in. Does not impact which
# namespaces it's able to resolve ingress resource for. For limiting ingress
# namespace scope, see --watch-namespace.
namespace: kube-system
spec:
replicas: 1
selector:
matchLabels:
app: alb-ingress-controller
strategy:
rollingUpdate:
maxSurge: 1
maxUnavailable: 1
type: RollingUpdate
template:
metadata:
creationTimestamp: null
labels:
app: alb-ingress-controller
spec:
containers:
- args:
- /server
# Ingress controllers must have a default backend deployment where
# all unknown locations can be routed to. Often this is a 404 page. The
# default backend is not particularly helpful to the ALB Ingress Controller
# but is still required. The default backend and its respective service
# must be running Kubernetes for this controller to start.
- --default-backend-service=kube-system/default-http-backend
# Limit the namespace where this ALB Ingress Controller deployment will
# resolve ingress resources. If left commented, all namespaces are used.
#- --watch-namespace=your-k8s-namespace
env:
# AWS region this ingress controller will operate in.
# List of regions:
# http://docs.aws.amazon.com/general/latest/gr/rande.html#vpc_region
- name: AWS_REGION
value: us-west-1
# Name of your cluster. Used when naming resources created
# by the ALB Ingress Controller, providing distinction between
# clusters.
- name: CLUSTER_NAME
value: my-k8s-cluster
# Enables logging on all outbound requests sent to the AWS API.
# If logging is desired, set to true.
- name: AWS_DEBUG
value: "false"
- name: POD_NAME
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: metadata.name
- name: POD_NAMESPACE
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: metadata.namespace
# Repository location of the ALB Ingress Controller.
image: quay.io/joshrosso/alb-ingress:0.6
imagePullPolicy: Always
name: server
resources: {}
terminationMessagePath: /dev/termination-log
dnsPolicy: ClusterFirst
restartPolicy: Always
securityContext: {}
terminationGracePeriodSeconds: 30
Original file line number Diff line number Diff line change
Expand Up @@ -5,12 +5,12 @@ metadata:
namespace: echoserver
annotations:
alb.ingress.kubernetes.io/scheme: internal
alb.ingress.kubernetes.io/subnets: subnet-a4f0098e,subnet-457ed533,subnet-95c904cd
alb.ingress.kubernetes.io/security-groups: sg-723a380a,sg-a6181ede,sg-a5181edd
alb.ingress.kubernetes.io/subnets: subnet-0d2bab64,subnet-9c569ce7
alb.ingress.kubernetes.io/security-groups: sg-ccaacaa5
alb.ingress.kubernetes.io/tags: Environment=dev1,ProductCode=PRD999,InventoryCode=echo-app
spec:
rules:
- host: echoserver.nonprod-tmaws.io
- host: aaaaaa.josh-test-dns.com
http:
paths:
- path: /
Expand Down
54 changes: 54 additions & 0 deletions examples/iam-policy.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"elasticloadbalancing:AddTags",
"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer",
"elasticloadbalancing:AttachLoadBalancerToSubnets",
"elasticloadbalancing:ConfigureHealthCheck",
"elasticloadbalancing:CreateListener",
"elasticloadbalancing:CreateLoadBalancer",
"elasticloadbalancing:CreateLoadBalancerListeners",
"elasticloadbalancing:CreateRule",
"elasticloadbalancing:CreateTargetGroup",
"elasticloadbalancing:DeleteListener",
"elasticloadbalancing:DeleteLoadBalancer",
"elasticloadbalancing:DeleteLoadBalancerListeners",
"elasticloadbalancing:DeleteRule",
"elasticloadbalancing:DeleteTargetGroup",
"elasticloadbalancing:DescribeListeners",
"elasticloadbalancing:DescribeLoadBalancers",
"elasticloadbalancing:DescribeRules",
"elasticloadbalancing:DescribeTags",
"elasticloadbalancing:DescribeTargetGroupAttributes",
"elasticloadbalancing:DescribeTargetGroups",
"elasticloadbalancing:DescribeTargetHealth",
"elasticloadbalancing:ModifyListener",
"elasticloadbalancing:ModifyLoadBalancerAttributes",
"elasticloadbalancing:ModifyRule",
"elasticloadbalancing:ModifyTargetGroup",
"elasticloadbalancing:ModifyTargetGroupAttributes",
"elasticloadbalancing:RegisterTargets",
"elasticloadbalancing:RegisterInstancesWithLoadBalancer",
"elasticloadbalancing:RemoveTags",
"elasticloadbalancing:SetLoadBalancerListenerSSLCertificate",
"elasticloadbalancing:SetSecurityGroups",
"elasticloadbalancing:SetSubnets"
],
"Resource": "*"
},
{
"Effect": "Allow",
"Action": [
"route53:ChangeResourceRecordSets",
"route53:GetChange",
"route53:GetHostedZone",
"route53:ListHostedZones",
"route53:ListResourceRecordSets"
],
"Resource": "*"
}
]
}
2 changes: 0 additions & 2 deletions main.go
Original file line number Diff line number Diff line change
Expand Up @@ -24,12 +24,10 @@ func main() {
glog.Exit("A CLUSTER_NAME environment variable must be defined")
}

noop, _ := strconv.ParseBool(os.Getenv("NOOP"))
awsDebug, _ := strconv.ParseBool(os.Getenv("AWS_DEBUG"))

config := &controller.Config{
ClusterName: clusterName,
Noop: noop,
AWSDebug: awsDebug,
}

Expand Down
Binary file removed manifests/2048/.2048-ingress.yaml.swp
Binary file not shown.
51 changes: 0 additions & 51 deletions manifests/alb-ingress.yaml

This file was deleted.

51 changes: 0 additions & 51 deletions manifests/default-backend.yaml

This file was deleted.

20 changes: 0 additions & 20 deletions manifests/gopherserv-svc.yaml

This file was deleted.

Loading