Skip to content

Commit d5ceafb

Browse files
jmymycmcga1125bharathkkb
authored
feat: GKE autopilot support (#1148)
* adding auto-pilot support * fixes * add kitchen tests * Update main.tf * fix: add back in * Update examples/simple_autopilot_private/README.md Co-authored-by: Bharath KKB <[email protected]> * Update examples/simple_autopilot_private/README.md Co-authored-by: Bharath KKB <[email protected]> * Update modules/beta-autopilot-public-cluster/versions.tf Co-authored-by: Bharath KKB <[email protected]> * Update examples/simple_autopilot_private/main.tf Co-authored-by: Bharath KKB <[email protected]> * update dates and remove Vars * fixes * i hate symlinks * add vars and outputs * docs generation * add random string to subnet names * Update main.tf * adding auto-pilot support * fixes * add kitchen tests * Update main.tf * fix: add back in * Update examples/simple_autopilot_private/README.md Co-authored-by: Bharath KKB <[email protected]> * Update examples/simple_autopilot_private/README.md Co-authored-by: Bharath KKB <[email protected]> * Update modules/beta-autopilot-public-cluster/versions.tf Co-authored-by: Bharath KKB <[email protected]> * Update examples/simple_autopilot_private/main.tf Co-authored-by: Bharath KKB <[email protected]> * update dates and remove Vars * fixes * i hate symlinks * add vars and outputs * docs generation * add random string to subnet names * Update main.tf * remove random name Co-authored-by: Corey McGalliard <[email protected]> Co-authored-by: Bharath KKB <[email protected]>
1 parent 7976d17 commit d5ceafb

File tree

167 files changed

+5083
-272
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

167 files changed

+5083
-272
lines changed

‎.kitchen.yml

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -244,3 +244,17 @@ suites:
244244
controls:
245245
- gcloud
246246
- kubectl
247+
- name: "simple_autopilot_private"
248+
driver:
249+
root_module_directory: test/fixtures/simple_autopilot_private
250+
verifier:
251+
systems:
252+
- name: simple_autopilot_private
253+
backend: local
254+
- name: "simple_autopilot_public"
255+
driver:
256+
root_module_directory: test/fixtures/simple_autopilot_public
257+
verifier:
258+
systems:
259+
- name: simple_autopilot_public
260+
backend: local

‎README.md

Lines changed: 1 addition & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -43,8 +43,8 @@ module "gke" {
4343
ip_range_pods = "us-central1-01-gke-01-pods"
4444
ip_range_services = "us-central1-01-gke-01-services"
4545
http_load_balancing = false
46-
horizontal_pod_autoscaling = true
4746
network_policy = false
47+
horizontal_pod_autoscaling = true
4848
4949
node_pools = [
5050
{
@@ -253,8 +253,6 @@ The node_pools variable takes the following parameters:
253253
| tags | The list of instance tags applied to all nodes | | Required |
254254
| value | The value for the taint | | Required |
255255
| version | The Kubernetes version for the nodes in this pool. Should only be set if auto_upgrade is false | " " | Optional |
256-
257-
258256
## Requirements
259257

260258
Before this module can be used on a project, you must ensure that the following pre-requisites are fulfilled:

‎autogen/main/README.md

Lines changed: 14 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -72,20 +72,26 @@ module "gke" {
7272
subnetwork = "us-central1-01"
7373
ip_range_pods = "us-central1-01-gke-01-pods"
7474
ip_range_services = "us-central1-01-gke-01-services"
75+
{% if autopilot_cluster != true %}
7576
http_load_balancing = false
76-
horizontal_pod_autoscaling = true
7777
network_policy = false
78+
{% endif %}
79+
horizontal_pod_autoscaling = true
7880
{% if private_cluster %}
7981
enable_private_endpoint = true
8082
enable_private_nodes = true
8183
master_ipv4_cidr_block = "10.0.0.0/28"
8284
{% endif %}
83-
{% if beta_cluster %}
84-
istio = true
85-
cloudrun = true
86-
dns_cache = false
85+
{% if beta_cluster and autopilot_cluster != true %}
86+
istio = true
87+
cloudrun = true
88+
dns_cache = false
89+
{% endif %}
90+
{% if autopilot_cluster %}
91+
enable_autopilot = true
8792
{% endif %}
8893
94+
{% if autopilot_cluster != true %}
8995
node_pools = [
9096
{
9197
name = "default-node-pool"
@@ -152,6 +158,7 @@ module "gke" {
152158
"default-node-pool",
153159
]
154160
}
161+
{% endif %}
155162
}
156163
```
157164

@@ -166,6 +173,7 @@ Then perform the following commands on the root folder:
166173
<!-- BEGINNING OF PRE-COMMIT-TERRAFORM DOCS HOOK -->
167174
<!-- END OF PRE-COMMIT-TERRAFORM DOCS HOOK -->
168175

176+
{% if autopilot_cluster != true %}
169177
## node_pools variable
170178
The node_pools variable takes the following parameters:
171179

@@ -220,8 +228,7 @@ The node_pools variable takes the following parameters:
220228
| tags | The list of instance tags applied to all nodes | | Required |
221229
| value | The value for the taint | | Required |
222230
| version | The Kubernetes version for the nodes in this pool. Should only be set if auto_upgrade is false | " " | Optional |
223-
224-
231+
{% endif %}
225232
## Requirements
226233

227234
Before this module can be used on a project, you must ensure that the following pre-requisites are fulfilled:

‎autogen/main/cluster.tf.tmpl

Lines changed: 36 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
/**
2-
* Copyright 2018 Google LLC
2+
* Copyright 2022 Google LLC
33
*
44
* Licensed under the Apache License, Version 2.0 (the "License");
55
* you may not use this file except in compliance with the License.
@@ -35,7 +35,7 @@ resource "google_container_cluster" "primary" {
3535
node_locations = local.node_locations
3636
cluster_ipv4_cidr = var.cluster_ipv4_cidr
3737
network = "projects/${local.network_project_id}/global/networks/${var.network}"
38-
38+
{% if autopilot_cluster != true %}
3939
dynamic "network_policy" {
4040
for_each = local.cluster_network_policy
4141

@@ -44,6 +44,7 @@ resource "google_container_cluster" "primary" {
4444
provider = network_policy.value.provider
4545
}
4646
}
47+
{% endif %}
4748

4849
dynamic "release_channel" {
4950
for_each = local.release_channel
@@ -64,13 +65,13 @@ resource "google_container_cluster" "primary" {
6465
subnetwork = "projects/${local.network_project_id}/regions/${local.region}/subnetworks/${var.subnetwork}"
6566

6667
{% if beta_cluster %}
67-
default_snat_status{
68+
default_snat_status {
6869
disabled = var.disable_default_snat
6970
}
7071
{% endif %}
7172
min_master_version = var.release_channel != null ? null : local.master_version
7273

73-
{% if beta_cluster %}
74+
{% if beta_cluster and autopilot_cluster != true %}
7475
dynamic "cluster_telemetry" {
7576
for_each = local.cluster_telemetry_type_is_set ? [1] : []
7677
content {
@@ -98,7 +99,7 @@ resource "google_container_cluster" "primary" {
9899
logging_service = var.logging_service
99100
monitoring_service = var.monitoring_service
100101
{% endif %}
101-
102+
{% if autopilot_cluster != true %}
102103
cluster_autoscaling {
103104
enabled = var.cluster_autoscaling.enabled
104105
dynamic "auto_provisioning_defaults" {
@@ -107,14 +108,14 @@ resource "google_container_cluster" "primary" {
107108
content {
108109
service_account = local.service_account
109110
oauth_scopes = local.node_pools_oauth_scopes["all"]
110-
{% if beta_cluster %}
111+
{% if beta_cluster %}
111112
min_cpu_platform = lookup(var.node_pools[0], "min_cpu_platform", "")
112-
{% endif %}
113+
{% endif %}
113114
}
114115
}
115-
{% if beta_cluster %}
116+
{% if beta_cluster %}
116117
autoscaling_profile = var.cluster_autoscaling.autoscaling_profile != null ? var.cluster_autoscaling.autoscaling_profile : "BALANCED"
117-
{% endif %}
118+
{% endif %}
118119
dynamic "resource_limits" {
119120
for_each = local.autoscaling_resource_limits
120121
content {
@@ -124,16 +125,15 @@ resource "google_container_cluster" "primary" {
124125
}
125126
}
126127
}
127-
128+
{% endif %}
128129
vertical_pod_autoscaling {
129130
enabled = var.enable_vertical_pod_autoscaling
130131
}
131-
132+
{% if autopilot_cluster != true %}
132133
default_max_pods_per_node = var.default_max_pods_per_node
133-
134134
enable_shielded_nodes = var.enable_shielded_nodes
135135
enable_binary_authorization = var.enable_binary_authorization
136-
{% if beta_cluster %}
136+
{% if beta_cluster %}
137137
enable_intranode_visibility = var.enable_intranode_visibility
138138
enable_kubernetes_alpha = var.enable_kubernetes_alpha
139139
enable_tpu = var.enable_tpu
@@ -153,6 +153,10 @@ resource "google_container_cluster" "primary" {
153153
}
154154

155155
enable_l4_ilb_subsetting = var.enable_l4_ilb_subsetting
156+
{% endif %}
157+
{% endif %}
158+
{% if autopilot_cluster %}
159+
enable_autopilot = true
156160
{% endif %}
157161
dynamic "master_authorized_networks_config" {
158162
for_each = local.master_authorized_networks_config
@@ -181,14 +185,15 @@ resource "google_container_cluster" "primary" {
181185
horizontal_pod_autoscaling {
182186
disabled = !var.horizontal_pod_autoscaling
183187
}
184-
188+
{% if autopilot_cluster != true %}
185189
network_policy_config {
186190
disabled = !var.network_policy
187191
}
188-
{% if beta_cluster %}
192+
{% endif %}
193+
{% if beta_cluster and autopilot_cluster != true %}
189194

190195
istio_config {
191-
disabled = ! var.istio
196+
disabled = !var.istio
192197
auth = var.istio_auth
193198
}
194199

@@ -258,24 +263,25 @@ resource "google_container_cluster" "primary" {
258263
end_time = maintenance_exclusion.value.end_time
259264
}
260265
}
261-
262266
{% else %}
263267
daily_maintenance_window {
264268
start_time = var.maintenance_start_time
265269
}
266270
{% endif %}
267271
}
268272

273+
{% if autopilot_cluster != true %}
269274
lifecycle {
270275
ignore_changes = [node_pool, initial_node_count, resource_labels["asmv"], resource_labels["mesh_id"]]
271276
}
277+
{% endif %}
272278

273279
timeouts {
274280
create = "45m"
275281
update = "45m"
276282
delete = "45m"
277283
}
278-
284+
{% if autopilot_cluster != true %}
279285
node_pool {
280286
name = "default-pool"
281287
initial_node_count = var.initial_node_count
@@ -321,6 +327,7 @@ resource "google_container_cluster" "primary" {
321327
}
322328
}
323329
}
330+
{% endif %}
324331

325332
dynamic "resource_usage_export_config" {
326333
for_each = var.resource_usage_export_dataset_id != "" ? [{
@@ -362,6 +369,7 @@ resource "google_container_cluster" "primary" {
362369
}
363370
{% endif %}
364371

372+
{% if autopilot_cluster != true %}
365373
remove_default_node_pool = var.remove_default_node_pool
366374

367375
dynamic "database_encryption" {
@@ -380,27 +388,30 @@ resource "google_container_cluster" "primary" {
380388
workload_pool = workload_identity_config.value.workload_pool
381389
}
382390
}
391+
{% endif %}
383392

393+
{% if autopilot_cluster != true %}
384394
dynamic "authenticator_groups_config" {
385395
for_each = local.cluster_authenticator_security_group
386396
content {
387397
security_group = authenticator_groups_config.value.security_group
388398
}
389399
}
390-
391-
{% if beta_cluster %}
400+
{% endif %}
401+
{% if beta_cluster %}
392402
notification_config {
393403
pubsub {
394404
enabled = var.notification_config_topic != "" ? true : false
395-
topic = var.notification_config_topic
405+
topic = var.notification_config_topic
396406
}
397407
}
398-
{% endif %}
408+
{% endif %}
399409
}
400-
410+
{% if autopilot_cluster != true %}
401411
/******************************************
402412
Create Container Cluster node pools
403413
*****************************************/
414+
{% endif %}
404415
{% if update_variant %}
405416
locals {
406417
force_node_pool_recreation_resources = [
@@ -491,6 +502,7 @@ resource "random_id" "name" {
491502
}
492503

493504
{% endif %}
505+
{% if autopilot_cluster != true %}
494506
resource "google_container_node_pool" "pools" {
495507
{% if beta_cluster %}
496508
provider = google-beta
@@ -698,3 +710,4 @@ resource "google_container_node_pool" "pools" {
698710
delete = "45m"
699711
}
700712
}
713+
{% endif %}

‎autogen/main/dns.tf.tmpl

Lines changed: 9 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
/**
2-
* Copyright 2018 Google LLC
2+
* Copyright 2022 Google LLC
33
*
44
* Licensed under the Apache License, Version 2.0 (the "License");
55
* you may not use this file except in compliance with the License.
@@ -35,7 +35,9 @@ module "gcloud_delete_default_kube_dns_configmap" {
3535

3636
module_depends_on = concat(
3737
[google_container_cluster.primary.master_version],
38+
{% if autopilot_cluster != true %}
3839
[for pool in google_container_node_pool.pools : pool.name]
40+
{% endif %}
3941
)
4042
}
4143

@@ -63,7 +65,9 @@ EOF
6365
depends_on = [
6466
module.gcloud_delete_default_kube_dns_configmap.wait,
6567
google_container_cluster.primary,
68+
{% if autopilot_cluster != true %}
6669
google_container_node_pool.pools,
70+
{% endif %}
6771
]
6872
}
6973

@@ -89,7 +93,9 @@ EOF
8993
depends_on = [
9094
module.gcloud_delete_default_kube_dns_configmap.wait,
9195
google_container_cluster.primary,
96+
{% if autopilot_cluster != true %}
9297
google_container_node_pool.pools,
98+
{% endif %}
9399
]
94100
}
95101

@@ -118,6 +124,8 @@ EOF
118124
depends_on = [
119125
module.gcloud_delete_default_kube_dns_configmap.wait,
120126
google_container_cluster.primary,
127+
{% if autopilot_cluster != true %}
121128
google_container_node_pool.pools,
129+
{% endif %}
122130
]
123131
}

‎autogen/main/firewall.tf.tmpl

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
/**
2-
* Copyright 2018 Google LLC
2+
* Copyright 2022 Google LLC
33
*
44
* Licensed under the Apache License, Version 2.0 (the "License");
55
* you may not use this file except in compliance with the License.

0 commit comments

Comments
 (0)