Skip to content

Commit ae17a3f

Browse files
marko7460marko7460
committed
Moving autoscaling from beta to GA
1 parent 79e5e14 commit ae17a3f

File tree

25 files changed

+197
-34
lines changed

25 files changed

+197
-34
lines changed

‎README.md

Lines changed: 20 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -39,6 +39,7 @@ module "gke" {
3939
{
4040
name = "default-node-pool"
4141
machine_type = "e2-medium"
42+
node_locations = "us-central1-b,us-central1-c"
4243
min_count = 1
4344
max_count = 100
4445
local_ssd_count = 0
@@ -77,6 +78,18 @@ module "gke" {
7778
}
7879
}
7980
81+
node_pools_taints = {
82+
all = []
83+
84+
default-node-pool = [
85+
{
86+
key = "default-node-pool"
87+
value = true
88+
effect = "PREFER_NO_SCHEDULE"
89+
},
90+
]
91+
}
92+
8093
node_pools_tags = {
8194
all = []
8295
@@ -103,6 +116,7 @@ Then perform the following commands on the root folder:
103116
| add\_cluster\_firewall\_rules | Create additional firewall rules | bool | `"false"` | no |
104117
| basic\_auth\_password | The password to be used with Basic Authentication. | string | `""` | no |
105118
| basic\_auth\_username | The username to be used with Basic Authentication. An empty value will disable Basic Authentication, which is the recommended configuration. | string | `""` | no |
119+
| cluster\_autoscaling | Cluster autoscaling configuration. See [more details](https://cloud.google.com/kubernetes-engine/docs/reference/rest/v1beta1/projects.locations.clusters#clusterautoscaling) | object | `<map>` | no |
106120
| cluster\_ipv4\_cidr | The IP address range of the kubernetes pods in this cluster. Default is an automatically assigned CIDR. | string | `"null"` | no |
107121
| cluster\_resource\_labels | The GCE resource labels (a map of key/value pairs) to be applied to the cluster | map(string) | `<map>` | no |
108122
| configure\_ip\_masq | Enables the installation of ip masquerading, which is usually no longer required when using aliasied IP addresses. IP masquerading uses a kubectl call, so when you have a private cluster, you will need access to the API server. | string | `"false"` | no |
@@ -201,17 +215,22 @@ The node_pools variable takes the following parameters:
201215
| auto_upgrade | Whether the nodes will be automatically upgraded | true (if cluster is regional) | Optional |
202216
| disk_size_gb | Size of the disk attached to each node, specified in GB. The smallest allowed disk size is 10GB | 100 | Optional |
203217
| disk_type | Type of the disk attached to each node (e.g. 'pd-standard' or 'pd-ssd') | pd-standard | Optional |
218+
| effect | Effect for the taint | | Required |
204219
| image_type | The image type to use for this node. Note that changing the image type will delete and recreate all nodes in the node pool | COS | Optional |
205220
| initial_node_count | The initial number of nodes for the pool. In regional or multi-zonal clusters, this is the number of nodes per zone. Changing this will force recreation of the resource. Defaults to the value of min_count | " " | Optional |
221+
| key | The key required for the taint | | Required |
206222
| local_ssd_count | The amount of local SSD disks that will be attached to each cluster node | 0 | Optional |
207223
| machine_type | The name of a Google Compute Engine machine type | e2-medium | Optional |
208224
| max_count | Maximum number of nodes in the NodePool. Must be >= min_count | 100 | Optional |
209225
| min_count | Minimum number of nodes in the NodePool. Must be >=0 and <= max_count. Should be used when autoscaling is true | 1 | Optional |
210226
| name | The name of the node pool | | Required |
211227
| node_count | The number of nodes in the nodepool when autoscaling is false. Otherwise defaults to 1. Only valid for non-autoscaling clusers | | Required |
228+
| node_locations | The list of zones in which the cluster's nodes are located. Nodes must be in the region of their regional cluster or in the same region as their cluster's zone for zonal clusters. Defaults to cluster level node locations if nothing is specified | " " | Optional |
229+
| node_metadata | Options to expose the node metadata to the workload running on the node | | Optional |
212230
| preemptible | A boolean that represents whether or not the underlying node VMs are preemptible | false | Optional |
213231
| service_account | The service account to be used by the Node VMs | " " | Optional |
214232
| tags | The list of instance tags applied to all nodes | | Required |
233+
| value | The value for the taint | | Required |
215234
| version | The Kubernetes version for the nodes in this pool. Should only be set if auto_upgrade is false | " " | Optional |
216235

217236

@@ -231,7 +250,7 @@ The [project factory](https://github.com/terraform-google-modules/terraform-goog
231250
- [kubectl](https://github.com/kubernetes/kubernetes/releases) 1.9.x
232251
#### Terraform and Plugins
233252
- [Terraform](https://www.terraform.io/downloads.html) 0.12
234-
- [Terraform Provider for GCP][terraform-provider-google] v2.9
253+
- [Terraform Provider for GCP][terraform-provider-google] v3.41
235254

236255
### Configure a Service Account
237256
In order to execute this module you must have a Service Account with the

‎autogen/main/README.md

Lines changed: 2 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -79,9 +79,7 @@ module "gke" {
7979
{
8080
name = "default-node-pool"
8181
machine_type = "e2-medium"
82-
{% if beta_cluster %}
8382
node_locations = "us-central1-b,us-central1-c"
84-
{% endif %}
8583
min_count = 1
8684
max_count = 100
8785
local_ssd_count = 0
@@ -119,7 +117,6 @@ module "gke" {
119117
node-pool-metadata-custom-value = "my-node-pool"
120118
}
121119
}
122-
{% if beta_cluster %}
123120
124121
node_pools_taints = {
125122
all = []
@@ -132,7 +129,6 @@ module "gke" {
132129
},
133130
]
134131
}
135-
{% endif %}
136132
137133
node_pools_tags = {
138134
all = []
@@ -169,14 +165,10 @@ The node_pools variable takes the following parameters:
169165
| auto_upgrade | Whether the nodes will be automatically upgraded | true (if cluster is regional) | Optional |
170166
| disk_size_gb | Size of the disk attached to each node, specified in GB. The smallest allowed disk size is 10GB | 100 | Optional |
171167
| disk_type | Type of the disk attached to each node (e.g. 'pd-standard' or 'pd-ssd') | pd-standard | Optional |
172-
{% if beta_cluster %}
173168
| effect | Effect for the taint | | Required |
174-
{% endif %}
175169
| image_type | The image type to use for this node. Note that changing the image type will delete and recreate all nodes in the node pool | COS | Optional |
176170
| initial_node_count | The initial number of nodes for the pool. In regional or multi-zonal clusters, this is the number of nodes per zone. Changing this will force recreation of the resource. Defaults to the value of min_count | " " | Optional |
177-
{% if beta_cluster %}
178171
| key | The key required for the taint | | Required |
179-
{% endif %}
180172
| local_ssd_count | The amount of local SSD disks that will be attached to each cluster node | 0 | Optional |
181173
| machine_type | The name of a Google Compute Engine machine type | e2-medium | Optional |
182174
| max_count | Maximum number of nodes in the NodePool. Must be >= min_count | 100 | Optional |
@@ -188,19 +180,15 @@ The node_pools variable takes the following parameters:
188180
| min_count | Minimum number of nodes in the NodePool. Must be >=0 and <= max_count. Should be used when autoscaling is true | 1 | Optional |
189181
| name | The name of the node pool | | Required |
190182
| node_count | The number of nodes in the nodepool when autoscaling is false. Otherwise defaults to 1. Only valid for non-autoscaling clusers | | Required |
191-
{% if beta_cluster %}
192183
| node_locations | The list of zones in which the cluster's nodes are located. Nodes must be in the region of their regional cluster or in the same region as their cluster's zone for zonal clusters. Defaults to cluster level node locations if nothing is specified | " " | Optional |
193184
| node_metadata | Options to expose the node metadata to the workload running on the node | | Optional |
194-
{% endif %}
195185
| preemptible | A boolean that represents whether or not the underlying node VMs are preemptible | false | Optional |
196186
{% if beta_cluster %}
197187
| sandbox_type | Sandbox to use for pods in the node pool | | Required |
198188
{% endif %}
199189
| service_account | The service account to be used by the Node VMs | " " | Optional |
200190
| tags | The list of instance tags applied to all nodes | | Required |
201-
{% if beta_cluster %}
202191
| value | The value for the taint | | Required |
203-
{% endif %}
204192
| version | The Kubernetes version for the nodes in this pool. Should only be set if auto_upgrade is false | " " | Optional |
205193

206194

@@ -221,9 +209,9 @@ The [project factory](https://github.com/terraform-google-modules/terraform-goog
221209
#### Terraform and Plugins
222210
- [Terraform](https://www.terraform.io/downloads.html) 0.12
223211
{% if beta_cluster %}
224-
- [Terraform Provider for GCP Beta][terraform-provider-google-beta] v2.9
212+
- [Terraform Provider for GCP Beta][terraform-provider-google-beta] v3.41
225213
{% else %}
226-
- [Terraform Provider for GCP][terraform-provider-google] v2.9
214+
- [Terraform Provider for GCP][terraform-provider-google] v3.41
227215
{% endif %}
228216

229217
### Configure a Service Account

‎autogen/main/cluster.tf.tmpl

Lines changed: 2 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -65,10 +65,11 @@ resource "google_container_cluster" "primary" {
6565
logging_service = var.logging_service
6666
monitoring_service = var.monitoring_service
6767

68-
{% if beta_cluster %}
6968
cluster_autoscaling {
7069
enabled = var.cluster_autoscaling.enabled
70+
{% if beta_cluster %}
7171
autoscaling_profile = var.cluster_autoscaling.autoscaling_profile != null ? var.cluster_autoscaling.autoscaling_profile : "BALANCED"
72+
{% endif %}
7273
dynamic "resource_limits" {
7374
for_each = local.autoscalling_resource_limits
7475
content {
@@ -78,7 +79,6 @@ resource "google_container_cluster" "primary" {
7879
}
7980
}
8081
}
81-
{% endif %}
8282

8383
default_max_pods_per_node = var.default_max_pods_per_node
8484

@@ -389,10 +389,8 @@ resource "google_container_node_pool" "pools" {
389389
{% endif %}
390390
project = var.project_id
391391
location = local.location
392-
{% if beta_cluster %}
393392
// use node_locations if provided, defaults to cluster level node_locations if not specified
394393
node_locations = lookup(each.value, "node_locations", "") != "" ? split(",", each.value["node_locations"]) : null
395-
{% endif %}
396394

397395
cluster = google_container_cluster.primary.name
398396

‎autogen/main/main.tf.tmpl

Lines changed: 0 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -51,7 +51,6 @@ locals {
5151
node_pools = zipmap(local.node_pool_names, tolist(toset(var.node_pools)))
5252

5353
release_channel = var.release_channel != null ? [{ channel : var.release_channel }] : []
54-
{% if beta_cluster %}
5554

5655
autoscalling_resource_limits = var.cluster_autoscaling.enabled ? [{
5756
resource_type = "cpu"
@@ -63,8 +62,6 @@ locals {
6362
maximum = var.cluster_autoscaling.max_memory_gb
6463
}] : []
6564

66-
{% endif %}
67-
6865

6966
custom_kube_dns_config = length(keys(var.stub_domains)) > 0
7067
upstream_nameservers_config = length(var.upstream_nameservers) > 0

‎autogen/main/variables.tf.tmpl

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -207,27 +207,31 @@ variable "enable_kubernetes_alpha" {
207207
description = "Whether to enable Kubernetes Alpha features for this cluster. Note that when this option is enabled, the cluster cannot be upgraded and will be automatically deleted after 30 days."
208208
default = false
209209
}
210+
{% endif %}
210211

211212
variable "cluster_autoscaling" {
212213
type = object({
213214
enabled = bool
215+
{% if beta_cluster %}
214216
autoscaling_profile = string
217+
{% endif %}
215218
min_cpu_cores = number
216219
max_cpu_cores = number
217220
min_memory_gb = number
218221
max_memory_gb = number
219222
})
220223
default = {
221224
enabled = false
225+
{% if beta_cluster %}
222226
autoscaling_profile = "BALANCED"
227+
{% endif %}
223228
max_cpu_cores = 0
224229
min_cpu_cores = 0
225230
max_memory_gb = 0
226231
min_memory_gb = 0
227232
}
228233
description = "Cluster autoscaling configuration. See [more details](https://cloud.google.com/kubernetes-engine/docs/reference/rest/v1beta1/projects.locations.clusters#clusterautoscaling)"
229234
}
230-
{% endif %}
231235

232236
variable "node_pools_taints" {
233237
type = map(list(object({ key = string, value = string, effect = string })))

‎cluster.tf

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -56,6 +56,17 @@ resource "google_container_cluster" "primary" {
5656
logging_service = var.logging_service
5757
monitoring_service = var.monitoring_service
5858

59+
cluster_autoscaling {
60+
enabled = var.cluster_autoscaling.enabled
61+
dynamic "resource_limits" {
62+
for_each = local.autoscalling_resource_limits
63+
content {
64+
resource_type = lookup(resource_limits.value, "resource_type")
65+
minimum = lookup(resource_limits.value, "minimum")
66+
maximum = lookup(resource_limits.value, "maximum")
67+
}
68+
}
69+
}
5970

6071
default_max_pods_per_node = var.default_max_pods_per_node
6172

@@ -181,6 +192,8 @@ resource "google_container_node_pool" "pools" {
181192
name = each.key
182193
project = var.project_id
183194
location = local.location
195+
// use node_locations if provided, defaults to cluster level node_locations if not specified
196+
node_locations = lookup(each.value, "node_locations", "") != "" ? split(",", each.value["node_locations"]) : null
184197

185198
cluster = google_container_cluster.primary.name
186199

‎examples/node_pool/main.tf

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -24,7 +24,7 @@ provider "google-beta" {
2424
}
2525

2626
module "gke" {
27-
source = "../.."
27+
source = "../../modules/beta-public-cluster/"
2828
project_id = var.project_id
2929
name = "${local.cluster_type}-cluster${var.cluster_name_suffix}"
3030
region = var.region

‎main.tf

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -48,6 +48,16 @@ locals {
4848

4949
release_channel = var.release_channel != null ? [{ channel : var.release_channel }] : []
5050

51+
autoscalling_resource_limits = var.cluster_autoscaling.enabled ? [{
52+
resource_type = "cpu"
53+
minimum = var.cluster_autoscaling.min_cpu_cores
54+
maximum = var.cluster_autoscaling.max_cpu_cores
55+
}, {
56+
resource_type = "memory"
57+
minimum = var.cluster_autoscaling.min_memory_gb
58+
maximum = var.cluster_autoscaling.max_memory_gb
59+
}] : []
60+
5161

5262
custom_kube_dns_config = length(keys(var.stub_domains)) > 0
5363
upstream_nameservers_config = length(var.upstream_nameservers) > 0

‎modules/beta-private-cluster-update-variant/README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -315,7 +315,7 @@ The [project factory](https://github.com/terraform-google-modules/terraform-goog
315315
- [kubectl](https://github.com/kubernetes/kubernetes/releases) 1.9.x
316316
#### Terraform and Plugins
317317
- [Terraform](https://www.terraform.io/downloads.html) 0.12
318-
- [Terraform Provider for GCP Beta][terraform-provider-google-beta] v2.9
318+
- [Terraform Provider for GCP Beta][terraform-provider-google-beta] v3.41
319319

320320
### Configure a Service Account
321321
In order to execute this module you must have a Service Account with the

‎modules/beta-private-cluster-update-variant/main.tf

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -59,7 +59,6 @@ locals {
5959
}] : []
6060

6161

62-
6362
custom_kube_dns_config = length(keys(var.stub_domains)) > 0
6463
upstream_nameservers_config = length(var.upstream_nameservers) > 0
6564
network_project_id = var.network_project_id != "" ? var.network_project_id : var.project_id

‎modules/beta-private-cluster/README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -293,7 +293,7 @@ The [project factory](https://github.com/terraform-google-modules/terraform-goog
293293
- [kubectl](https://github.com/kubernetes/kubernetes/releases) 1.9.x
294294
#### Terraform and Plugins
295295
- [Terraform](https://www.terraform.io/downloads.html) 0.12
296-
- [Terraform Provider for GCP Beta][terraform-provider-google-beta] v2.9
296+
- [Terraform Provider for GCP Beta][terraform-provider-google-beta] v3.41
297297

298298
### Configure a Service Account
299299
In order to execute this module you must have a Service Account with the

‎modules/beta-private-cluster/main.tf

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -59,7 +59,6 @@ locals {
5959
}] : []
6060

6161

62-
6362
custom_kube_dns_config = length(keys(var.stub_domains)) > 0
6463
upstream_nameservers_config = length(var.upstream_nameservers) > 0
6564
network_project_id = var.network_project_id != "" ? var.network_project_id : var.project_id

‎modules/beta-public-cluster-update-variant/README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -302,7 +302,7 @@ The [project factory](https://github.com/terraform-google-modules/terraform-goog
302302
- [kubectl](https://github.com/kubernetes/kubernetes/releases) 1.9.x
303303
#### Terraform and Plugins
304304
- [Terraform](https://www.terraform.io/downloads.html) 0.12
305-
- [Terraform Provider for GCP Beta][terraform-provider-google-beta] v2.9
305+
- [Terraform Provider for GCP Beta][terraform-provider-google-beta] v3.41
306306

307307
### Configure a Service Account
308308
In order to execute this module you must have a Service Account with the

‎modules/beta-public-cluster-update-variant/main.tf

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -59,7 +59,6 @@ locals {
5959
}] : []
6060

6161

62-
6362
custom_kube_dns_config = length(keys(var.stub_domains)) > 0
6463
upstream_nameservers_config = length(var.upstream_nameservers) > 0
6564
network_project_id = var.network_project_id != "" ? var.network_project_id : var.project_id

‎modules/beta-public-cluster/README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -280,7 +280,7 @@ The [project factory](https://github.com/terraform-google-modules/terraform-goog
280280
- [kubectl](https://github.com/kubernetes/kubernetes/releases) 1.9.x
281281
#### Terraform and Plugins
282282
- [Terraform](https://www.terraform.io/downloads.html) 0.12
283-
- [Terraform Provider for GCP Beta][terraform-provider-google-beta] v2.9
283+
- [Terraform Provider for GCP Beta][terraform-provider-google-beta] v3.41
284284

285285
### Configure a Service Account
286286
In order to execute this module you must have a Service Account with the

‎modules/beta-public-cluster/main.tf

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -59,7 +59,6 @@ locals {
5959
}] : []
6060

6161

62-
6362
custom_kube_dns_config = length(keys(var.stub_domains)) > 0
6463
upstream_nameservers_config = length(var.upstream_nameservers) > 0
6564
network_project_id = var.network_project_id != "" ? var.network_project_id : var.project_id

0 commit comments

Comments
 (0)