Skip to content

Commit 6868fd3

Browse files
splichysplichy
authored
Merge branch 'master' into feat/firewall_disco_pod_ranges
2 parents d47c2dc + 8507e09 commit 6868fd3

File tree

60 files changed

+338
-125
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

60 files changed

+338
-125
lines changed

‎CHANGELOG.md

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,40 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
66

77
Extending the adopted spec, each change should have a link to its corresponding pull request appended.
88

9+
## [24.0.0](https://github.com/terraform-google-modules/terraform-google-kubernetes-engine/compare/v23.3.0...v24.0.0) (2022-11-21)
10+
11+
12+
### ⚠ BREAKING CHANGES
13+
14+
* cost_management_config is out of beta now (#1470)
15+
* update variant - recreate node pools on max_pods_per_node or pod_range change (#1464)
16+
* expose global master access in GA modules (#1421)
17+
* min tpb bump for location_policy
18+
* min TPG bump for location_policy (#1453)
19+
* add service_external_ips option (#1441)
20+
* Adding Support for Cost Allocation Feature in Beta (#1413)
21+
* add boot_disk_kms_key variable for node pools to GA modules (#1371)
22+
23+
### Features
24+
25+
* add boot_disk_kms_key variable for node pools to GA modules ([#1371](https://github.com/terraform-google-modules/terraform-google-kubernetes-engine/issues/1371)) ([d9a44c6](https://github.com/terraform-google-modules/terraform-google-kubernetes-engine/commit/d9a44c60198e2bea72aa1f36c5dbe34e59416dbf))
26+
* add location_policy and fix permadiff ([#1452](https://github.com/terraform-google-modules/terraform-google-kubernetes-engine/issues/1452)) ([aecccf0](https://github.com/terraform-google-modules/terraform-google-kubernetes-engine/commit/aecccf0bb8ca950fab5598ce8ec4b91f45dcb4a9))
27+
* add nodepool autoscaling vars avail in GKE 1.24.1 ([#1415](https://github.com/terraform-google-modules/terraform-google-kubernetes-engine/issues/1415)) ([f57f3ce](https://github.com/terraform-google-modules/terraform-google-kubernetes-engine/commit/f57f3ce58de14076a03182aa3b37aae58beac29a))
28+
* add service_external_ips option ([#1441](https://github.com/terraform-google-modules/terraform-google-kubernetes-engine/issues/1441)) ([e9de006](https://github.com/terraform-google-modules/terraform-google-kubernetes-engine/commit/e9de006f535e67a311a01e60a554c636f127fafa))
29+
* Add support for https_proxy parameter for the config_sync.git block ([#1457](https://github.com/terraform-google-modules/terraform-google-kubernetes-engine/issues/1457)) ([43bbd3c](https://github.com/terraform-google-modules/terraform-google-kubernetes-engine/commit/43bbd3c7ac48560e76a6ad2448d8e1901f9d4e4a))
30+
* Adding Support for Cost Allocation Feature in Beta ([#1413](https://github.com/terraform-google-modules/terraform-google-kubernetes-engine/issues/1413)) ([ba3dcd0](https://github.com/terraform-google-modules/terraform-google-kubernetes-engine/commit/ba3dcd0b617ff82367c5fbaffa5dc76e6f9f2cb1))
31+
* cost_management_config is out of beta now ([#1470](https://github.com/terraform-google-modules/terraform-google-kubernetes-engine/issues/1470)) ([10ea608](https://github.com/terraform-google-modules/terraform-google-kubernetes-engine/commit/10ea6081c532aa0bcd5fdd8addbb15fedfe18ee0))
32+
* expose global master access in GA modules ([#1421](https://github.com/terraform-google-modules/terraform-google-kubernetes-engine/issues/1421)) ([4278f2c](https://github.com/terraform-google-modules/terraform-google-kubernetes-engine/commit/4278f2cd2dfc81ae71230162d53ec30401a5e54f))
33+
* Make creation of istio-system namespace optional ([#1439](https://github.com/terraform-google-modules/terraform-google-kubernetes-engine/issues/1439)) ([335c62a](https://github.com/terraform-google-modules/terraform-google-kubernetes-engine/commit/335c62a546f9b35b6825783e004c46f3d5f2440b))
34+
* update variant - recreate node pools on max_pods_per_node or pod_range change ([#1464](https://github.com/terraform-google-modules/terraform-google-kubernetes-engine/issues/1464)) ([b006593](https://github.com/terraform-google-modules/terraform-google-kubernetes-engine/commit/b006593cf9d81ca018468ad440c70509fdcef082))
35+
36+
37+
### Bug Fixes
38+
39+
* location-policy permadrifting [#1445](https://github.com/terraform-google-modules/terraform-google-kubernetes-engine/issues/1445) ([aecccf0](https://github.com/terraform-google-modules/terraform-google-kubernetes-engine/commit/aecccf0bb8ca950fab5598ce8ec4b91f45dcb4a9))
40+
* min tpb bump for location_policy ([0ddd297](https://github.com/terraform-google-modules/terraform-google-kubernetes-engine/commit/0ddd297a1d57cd4e58849e780d592147eac24321))
41+
* min TPG bump for location_policy ([#1453](https://github.com/terraform-google-modules/terraform-google-kubernetes-engine/issues/1453)) ([0ddd297](https://github.com/terraform-google-modules/terraform-google-kubernetes-engine/commit/0ddd297a1d57cd4e58849e780d592147eac24321))
42+
943
## [23.3.0](https://github.com/terraform-google-modules/terraform-google-kubernetes-engine/compare/v23.2.0...v23.3.0) (2022-10-28)
1044

1145

‎README.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -147,6 +147,7 @@ Then perform the following commands on the root folder:
147147
| disable\_legacy\_metadata\_endpoints | Disable the /0.1/ and /v1beta1/ metadata server endpoints on the node. Changing this value will cause all node pools to be recreated. | `bool` | `true` | no |
148148
| dns\_cache | The status of the NodeLocal DNSCache addon. | `bool` | `false` | no |
149149
| enable\_binary\_authorization | Enable BinAuthZ Admission controller | `bool` | `false` | no |
150+
| enable\_cost\_allocation | Enables Cost Allocation Feature and the cluster name and namespace of your GKE workloads appear in the labels field of the billing export to BigQuery | `bool` | `false` | no |
150151
| enable\_network\_egress\_export | Whether to enable network egress metering for this cluster. If enabled, a daemonset will be created in the cluster to meter network egress traffic. | `bool` | `false` | no |
151152
| enable\_resource\_consumption\_export | Whether to enable resource consumption metering on this cluster. When enabled, a table will be created in the resource export BigQuery dataset to store resource consumption data. The resulting table can be joined with the resource usage table or with BigQuery billing export. | `bool` | `true` | no |
152153
| enable\_shielded\_nodes | Enable Shielded Nodes features on all nodes in this cluster | `bool` | `true` | no |

‎autogen/main/cluster.tf.tmpl

Lines changed: 10 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -53,13 +53,13 @@ resource "google_container_cluster" "primary" {
5353
channel = release_channel.value.channel
5454
}
5555
}
56-
{% if beta_cluster %}
5756
dynamic "cost_management_config" {
5857
for_each = var.enable_cost_allocation ? [1] : []
5958
content {
6059
enabled = var.enable_cost_allocation
6160
}
6261
}
62+
{% if beta_cluster %}
6363
dynamic "confidential_nodes" {
6464
for_each = local.confidential_node_config
6565
content {
@@ -434,14 +434,12 @@ resource "google_container_cluster" "primary" {
434434
enable_private_endpoint = private_cluster_config.value.enable_private_endpoint
435435
enable_private_nodes = private_cluster_config.value.enable_private_nodes
436436
master_ipv4_cidr_block = private_cluster_config.value.master_ipv4_cidr_block
437-
{% if beta_cluster %}
438437
dynamic "master_global_access_config" {
439438
for_each = var.master_global_access_enabled ? [var.master_global_access_enabled] : []
440439
content {
441440
enabled = master_global_access_config.value
442441
}
443442
}
444-
{% endif %}
445443
}
446444
}
447445
{% endif %}
@@ -499,7 +497,9 @@ locals {
499497
"enable_integrity_monitoring",
500498
"local_ssd_count",
501499
"machine_type",
500+
"max_pods_per_node",
502501
"min_cpu_platform",
502+
"pod_range",
503503
"preemptible",
504504
"spot",
505505
"service_account",
@@ -510,8 +510,9 @@ locals {
510510
}
511511

512512
# This keepers list is based on the terraform google provider schemaNodeConfig
513-
# resources where "ForceNew" is "true". schemaNodeConfig can be found in node_config.go at
514-
# https://github.com/terraform-providers/terraform-provider-google/blob/master/google/node_config.go#L22
513+
# resources where "ForceNew" is "true". schemaNodeConfig can be found in resource_container_node_pool.go at
514+
# https://github.com/hashicorp/terraform-provider-google/blob/main/google/resource_container_node_pool.go and node_config.go at
515+
# https://github.com/terraform-providers/terraform-provider-google/blob/main/google/node_config.go
515516
resource "random_id" "name" {
516517
for_each = merge(local.node_pools, local.windows_node_pools)
517518
byte_length = 2
@@ -597,7 +598,7 @@ resource "google_container_node_pool" "windows_pools" {
597598
for_each = local.node_pools
598599
{% else %}
599600
for_each = local.windows_node_pools
600-
{% endif %}
601+
{% endif %}
601602
{% if update_variant %}
602603
name = { for k, v in random_id.name : k => v.hex }[each.key]
603604
{% else %}
@@ -632,6 +633,8 @@ resource "google_container_node_pool" "windows_pools" {
632633
min_node_count = lookup(autoscaling.value, "min_count", 1)
633634
max_node_count = lookup(autoscaling.value, "max_count", 100)
634635
location_policy = lookup(autoscaling.value, "location_policy", null)
636+
total_min_node_count = lookup(autoscaling.value, "total_min_count", null)
637+
total_max_node_count = lookup(autoscaling.value, "total_max_count", null)
635638
}
636639
}
637640

@@ -642,7 +645,7 @@ resource "google_container_node_pool" "windows_pools" {
642645
type = lookup(placement_policy.value, "placement_policy", null)
643646
}
644647
}
645-
648+
646649
dynamic "network_config" {
647650
for_each = length(lookup(each.value, "pod_range", "")) > 0 ? [each.value] : []
648651
content {

‎autogen/main/variables.tf.tmpl

Lines changed: 1 addition & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -196,13 +196,11 @@ variable "node_pools_linux_node_configs_sysctls" {
196196
{% endif %}
197197
{% endif %}
198198

199-
{% if beta_cluster %}
200199
variable "enable_cost_allocation" {
201200
type = bool
202201
description = "Enables Cost Allocation Feature and the cluster name and namespace of your GKE workloads appear in the labels field of the billing export to BigQuery"
203202
default = false
204203
}
205-
{% endif %}
206204
variable "resource_usage_export_dataset_id" {
207205
type = string
208206
description = "The ID of a BigQuery Dataset for using BigQuery as the destination of resource usage export."
@@ -410,16 +408,13 @@ variable "master_ipv4_cidr_block" {
410408
description = "(Beta) The IP range in CIDR notation to use for the hosted master network"
411409
default = "10.0.0.0/28"
412410
}
413-
{% if beta_cluster %}
414411

415412
variable "master_global_access_enabled" {
416413
type = bool
417-
description = "(Beta) Whether the cluster master is accessible globally (from any region) or only within the same region as the private endpoint."
418-
414+
description = "Whether the cluster master is accessible globally (from any region) or only within the same region as the private endpoint."
419415
default = true
420416
}
421417
{% endif %}
422-
{% endif %}
423418

424419
variable "dns_cache" {
425420
type = bool

‎autogen/main/versions.tf.tmpl

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -24,15 +24,15 @@ terraform {
2424
required_providers {
2525
google-beta = {
2626
source = "hashicorp/google-beta"
27-
version = ">= 4.36.0, < 5.0"
27+
version = ">= 4.42.0, < 5.0"
2828
}
2929
kubernetes = {
3030
source = "hashicorp/kubernetes"
3131
version = "~> 2.10"
3232
}
3333
}
3434
provider_meta "google-beta" {
35-
module_name = "blueprints/terraform/terraform-google-kubernetes-engine{% if module_registry_name %}:{{ module_registry_name }}{% endif %}/v23.3.0"
35+
module_name = "blueprints/terraform/terraform-google-kubernetes-engine{% if module_registry_name %}:{{ module_registry_name }}{% endif %}/v24.0.0"
3636
}
3737
{% else %}
3838
required_providers {
@@ -46,7 +46,7 @@ terraform {
4646
}
4747
}
4848
provider_meta "google" {
49-
module_name = "blueprints/terraform/terraform-google-kubernetes-engine{% if module_registry_name %}:{{ module_registry_name }}{% endif %}/v23.3.0"
49+
module_name = "blueprints/terraform/terraform-google-kubernetes-engine{% if module_registry_name %}:{{ module_registry_name }}{% endif %}/v24.0.0"
5050
}
5151
{% endif %}
5252
}

‎autogen/safer-cluster/main.tf.tmpl

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -161,6 +161,9 @@ module "gke" {
161161
// We suggest to define policies about which images can run on a cluster.
162162
enable_binary_authorization = true
163163

164+
// Enable cost allocation support
165+
enable_cost_allocation = var.enable_cost_allocation
166+
164167
// Use of PodSecurityPolicy admission controller
165168
// https://cloud.google.com/kubernetes-engine/docs/how-to/pod-security-policies
166169
enable_pod_security_policy = var.enable_pod_security_policy

‎autogen/safer-cluster/variables.tf.tmpl

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -356,6 +356,12 @@ variable "enable_resource_consumption_export" {
356356
default = true
357357
}
358358

359+
variable "enable_cost_allocation" {
360+
type = bool
361+
description = "Enables Cost Allocation Feature and the cluster name and namespace of your GKE workloads appear in the labels field of the billing export to BigQuery"
362+
default = false
363+
}
364+
359365
variable "sandbox_enabled" {
360366
type = bool
361367
description = "(Beta) Enable GKE Sandbox (Do not forget to set `image_type` = `COS_CONTAINERD` to use it)."

‎autogen/safer-cluster/versions.tf.tmpl

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,6 @@ terraform {
2323
required_version = ">=0.13"
2424

2525
provider_meta "google-beta" {
26-
module_name = "blueprints/terraform/terraform-google-kubernetes-engine{% if module_registry_name %}:{{ module_registry_name }}{% endif %}/v23.3.0"
26+
module_name = "blueprints/terraform/terraform-google-kubernetes-engine{% if module_registry_name %}:{{ module_registry_name }}{% endif %}/v24.0.0"
2727
}
2828
}

‎cluster.tf

Lines changed: 16 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -47,6 +47,12 @@ resource "google_container_cluster" "primary" {
4747
channel = release_channel.value.channel
4848
}
4949
}
50+
dynamic "cost_management_config" {
51+
for_each = var.enable_cost_allocation ? [1] : []
52+
content {
53+
enabled = var.enable_cost_allocation
54+
}
55+
}
5056

5157
subnetwork = "projects/${local.network_project_id}/regions/${local.region}/subnetworks/${var.subnetwork}"
5258

@@ -303,7 +309,6 @@ resource "google_container_cluster" "primary" {
303309
resource "google_container_node_pool" "pools" {
304310
provider = google
305311
for_each = local.node_pools
306-
307312
name = each.key
308313
project = var.project_id
309314
location = local.location
@@ -331,9 +336,11 @@ resource "google_container_node_pool" "pools" {
331336
dynamic "autoscaling" {
332337
for_each = lookup(each.value, "autoscaling", true) ? [each.value] : []
333338
content {
334-
min_node_count = lookup(autoscaling.value, "min_count", 1)
335-
max_node_count = lookup(autoscaling.value, "max_count", 100)
336-
location_policy = lookup(autoscaling.value, "location_policy", null)
339+
min_node_count = lookup(autoscaling.value, "min_count", 1)
340+
max_node_count = lookup(autoscaling.value, "max_count", 100)
341+
location_policy = lookup(autoscaling.value, "location_policy", null)
342+
total_min_node_count = lookup(autoscaling.value, "total_min_count", null)
343+
total_max_node_count = lookup(autoscaling.value, "total_max_count", null)
337344
}
338345
}
339346

@@ -456,7 +463,6 @@ resource "google_container_node_pool" "pools" {
456463
resource "google_container_node_pool" "windows_pools" {
457464
provider = google
458465
for_each = local.windows_node_pools
459-
460466
name = each.key
461467
project = var.project_id
462468
location = local.location
@@ -484,9 +490,11 @@ resource "google_container_node_pool" "windows_pools" {
484490
dynamic "autoscaling" {
485491
for_each = lookup(each.value, "autoscaling", true) ? [each.value] : []
486492
content {
487-
min_node_count = lookup(autoscaling.value, "min_count", 1)
488-
max_node_count = lookup(autoscaling.value, "max_count", 100)
489-
location_policy = lookup(autoscaling.value, "location_policy", null)
493+
min_node_count = lookup(autoscaling.value, "min_count", 1)
494+
max_node_count = lookup(autoscaling.value, "max_count", 100)
495+
location_policy = lookup(autoscaling.value, "location_policy", null)
496+
total_min_node_count = lookup(autoscaling.value, "total_min_count", null)
497+
total_max_node_count = lookup(autoscaling.value, "total_max_count", null)
490498
}
491499
}
492500

‎docs/upgrading_to_v24.0.md

Lines changed: 62 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,62 @@
1+
# Upgrading to v24.0
2+
The v24.0 release of *kubernetes-engine* is a backwards incompatible
3+
release.
4+
5+
### master_global_access_enabled in GA private-cluster module
6+
7+
`master_global_access` is now supported in GA private-cluster module and defaults to true. To opt out, set `master_global_access_enabled` to `false`.
8+
9+
```diff
10+
module "gke" {
11+
- source = "terraform-google-modules/kubernetes-engine/google//modules/private-cluster"
12+
- version = "~> 23.0"
13+
+ source = "terraform-google-modules/kubernetes-engine/google//modules/private-cluster"
14+
+ version = "~> 24.0"
15+
...
16+
+ master_global_access_enabled = false
17+
}
18+
```
19+
20+
### Update variant random ID keepers updated
21+
22+
The v24.0 release updates the keepers for the update variant modules. This will force a recreation of the nodepools.
23+
24+
To avoid this, it is possible to edit the remote state of the `random_id` resource to add the new attributes.
25+
26+
1. Perform a `terraform plan` as normal, identifying the `random_id` resources changing and the new `max_pods_per_node` and `pod_range` attributes
27+
```tf
28+
~ keepers = { # forces replacement
29+
+ "max_pods_per_node" = ""
30+
+ "pod_range" = ""
31+
# (19 unchanged elements hidden)
32+
}
33+
# (2 unchanged attributes hidden)
34+
}
35+
```
36+
2. Pull the remote state locally: `terraform state pull > default.tfstate`
37+
1. Back up the original remote state: `cp default.tfstate original.tfstate`
38+
1. Edit the `random_id` resources to add in the new `max_pods_per_node` and `pod_range` attributes from the `terraform plan` step
39+
```diff
40+
"attributes": {
41+
"b64_std": "pool-02-vb4=",
42+
"b64_url": "pool-02-vb4",
43+
"byte_length": 2,
44+
"dec": "pool-02-48574",
45+
"hex": "pool-02-bdbe",
46+
"id": "vb4",
47+
"keepers": {
48+
...
49+
"taints": "",
50+
+ "max_pods_per_node": "",
51+
+ "pod_range": ""
52+
},
53+
"prefix": "pool-02-"
54+
}
55+
```
56+
5. Bump the serial number at the top
57+
1. Push the modified state to the remote `terraform state push default.tfstate`
58+
1. Confirm the `random_id` resource no longer changes (or the corresponding `nodepool`) in a `terraform plan`
59+
60+
### Minimum Google Provider versions
61+
62+
Minimum Google Provider versions have been updated to `4.42.0`.

‎examples/simple_regional/main.tf

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -38,6 +38,7 @@ module "gke" {
3838
ip_range_services = var.ip_range_services
3939
create_service_account = false
4040
service_account = var.compute_engine_service_account
41+
enable_cost_allocation = true
4142
enable_binary_authorization = var.enable_binary_authorization
4243
skip_provisioners = var.skip_provisioners
4344
}

‎modules/acm/versions.tf

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -19,11 +19,11 @@ terraform {
1919
required_version = ">= 0.13.0"
2020

2121
provider_meta "google" {
22-
module_name = "blueprints/terraform/terraform-google-kubernetes-engine:acm/v23.3.0"
22+
module_name = "blueprints/terraform/terraform-google-kubernetes-engine:acm/v24.0.0"
2323
}
2424

2525
provider_meta "google-beta" {
26-
module_name = "blueprints/terraform/terraform-google-kubernetes-engine:acm/v23.3.0"
26+
module_name = "blueprints/terraform/terraform-google-kubernetes-engine:acm/v24.0.0"
2727
}
2828

2929
required_providers {

‎modules/asm/versions.tf

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -26,10 +26,10 @@ terraform {
2626
}
2727

2828
provider_meta "google" {
29-
module_name = "blueprints/terraform/terraform-google-kubernetes-engine:asm/v23.3.0"
29+
module_name = "blueprints/terraform/terraform-google-kubernetes-engine:asm/v24.0.0"
3030
}
3131

3232
provider_meta "google-beta" {
33-
module_name = "blueprints/terraform/terraform-google-kubernetes-engine:asm/v23.3.0"
33+
module_name = "blueprints/terraform/terraform-google-kubernetes-engine:asm/v24.0.0"
3434
}
3535
}

‎modules/auth/versions.tf

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,6 @@ terraform {
1919
required_version = ">= 0.13.0"
2020

2121
provider_meta "google" {
22-
module_name = "blueprints/terraform/terraform-google-kubernetes-engine:auth/v23.3.0"
22+
module_name = "blueprints/terraform/terraform-google-kubernetes-engine:auth/v24.0.0"
2323
}
2424
}

0 commit comments

Comments
 (0)