Skip to content

Commit 8786814

Browse files
bberriotbberriot
committed
rebase
2 parents 5da662a + ac062f8 commit 8786814

File tree

47 files changed

+571
-116
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

47 files changed

+571
-116
lines changed

‎.github/workflows/stale.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,7 @@ jobs:
2121
stale:
2222
runs-on: ubuntu-latest
2323
steps:
24-
- uses: actions/stale@v6
24+
- uses: actions/stale@v7
2525
with:
2626
repo-token: ${{ secrets.GITHUB_TOKEN }}
2727
stale-issue-message: 'This issue is stale because it has been open 60 days with no activity. Remove stale label or comment or this will be closed in 7 days'

‎README.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -156,6 +156,7 @@ Then perform the following commands on the root folder:
156156
| filestore\_csi\_driver | The status of the Filestore CSI driver addon, which allows the usage of filestore instance as volumes | `bool` | `false` | no |
157157
| firewall\_inbound\_ports | List of TCP ports for admission/webhook controllers. Either flag `add_master_webhook_firewall_rules` or `add_cluster_firewall_rules` (also adds egress rules) must be set to `true` for inbound-ports firewall rules to be applied. | `list(string)` | <pre>[<br> "8443",<br> "9443",<br> "15017"<br>]</pre> | no |
158158
| firewall\_priority | Priority rule for firewall rules | `number` | `1000` | no |
159+
| gce\_pd\_csi\_driver | Whether this cluster should enable the Google Compute Engine Persistent Disk Container Storage Interface (CSI) Driver. | `bool` | `true` | no |
159160
| grant\_registry\_access | Grants created cluster-specific service account storage.objectViewer and artifactregistry.reader roles. | `bool` | `false` | no |
160161
| horizontal\_pod\_autoscaling | Enable horizontal pod autoscaling addon | `bool` | `true` | no |
161162
| http\_load\_balancing | Enable httpload balancer addon | `bool` | `true` | no |
@@ -282,6 +283,7 @@ The node_pools variable takes the following parameters:
282283
| tags | The list of instance tags applied to all nodes | | Required |
283284
| value | The value for the taint | | Required |
284285
| version | The Kubernetes version for the nodes in this pool. Should only be set if auto_upgrade is false | " " | Optional |
286+
| location_policy | [Location policy](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/container_node_pool#location_policy) specifies the algorithm used when scaling-up the node pool. Location policy is supported only in 1.24.1+ clusters. | " " | Optional |
285287

286288
## windows_node_pools variable
287289
The windows_node_pools variable takes the same parameters as [node_pools](#node\_pools-variable) but is reserved for provisioning Windows based node pools only. This variable is introduced to satisfy a [specific requirement](https://cloud.google.com/kubernetes-engine/docs/how-to/creating-a-cluster-windows#create_a_cluster_and_node_pools) for the presence of at least one linux based node pool in the cluster before a windows based node pool can be created.

‎autogen/main/README.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -232,6 +232,7 @@ The node_pools variable takes the following parameters:
232232
| tags | The list of instance tags applied to all nodes | | Required |
233233
| value | The value for the taint | | Required |
234234
| version | The Kubernetes version for the nodes in this pool. Should only be set if auto_upgrade is false | " " | Optional |
235+
| location_policy | [Location policy](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/container_node_pool#location_policy) specifies the algorithm used when scaling-up the node pool. Location policy is supported only in 1.24.1+ clusters. | " " | Optional |
235236

236237
## windows_node_pools variable
237238
The windows_node_pools variable takes the same parameters as [node_pools](#node\_pools-variable) but is reserved for provisioning Windows based node pools only. This variable is introduced to satisfy a [specific requirement](https://cloud.google.com/kubernetes-engine/docs/how-to/creating-a-cluster-windows#create_a_cluster_and_node_pools) for the presence of at least one linux based node pool in the cluster before a windows based node pool can be created.

‎autogen/main/cluster.tf.tmpl

Lines changed: 19 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -136,6 +136,17 @@ resource "google_container_cluster" "primary" {
136136
}
137137
}
138138
{% endif %}
139+
{% if autopilot_cluster == true %}
140+
cluster_autoscaling {
141+
dynamic "auto_provisioning_defaults" {
142+
for_each = var.create_service_account ? [1] : []
143+
144+
content {
145+
service_account = local.service_account
146+
}
147+
}
148+
}
149+
{% endif %}
139150
vertical_pod_autoscaling {
140151
enabled = var.enable_vertical_pod_autoscaling
141152
}
@@ -210,7 +221,6 @@ resource "google_container_cluster" "primary" {
210221
disabled = !var.horizontal_pod_autoscaling
211222
}
212223

213-
214224
{% if autopilot_cluster != true %}
215225
network_policy_config {
216226
disabled = !var.network_policy
@@ -223,6 +233,14 @@ resource "google_container_cluster" "primary" {
223233
gcp_filestore_csi_driver_config {
224234
enabled = var.filestore_csi_driver
225235
}
236+
237+
dynamic "gce_persistent_disk_csi_driver_config" {
238+
for_each = local.cluster_gce_pd_csi_config
239+
240+
content {
241+
enabled = gce_persistent_disk_csi_driver_config.value.enabled
242+
}
243+
}
226244
{% endif %}
227245
{% if beta_cluster and autopilot_cluster != true %}
228246

@@ -239,14 +257,6 @@ resource "google_container_cluster" "primary" {
239257
}
240258
}
241259

242-
dynamic "gce_persistent_disk_csi_driver_config" {
243-
for_each = local.cluster_gce_pd_csi_config
244-
245-
content {
246-
enabled = gce_persistent_disk_csi_driver_config.value.enabled
247-
}
248-
}
249-
250260
kalm_config {
251261
enabled = var.kalm_config
252262
}

‎autogen/main/main.tf.tmpl

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -95,6 +95,7 @@ locals {
9595
enabled = false
9696
provider = null
9797
}]
98+
cluster_gce_pd_csi_config = var.gce_pd_csi_driver ? [{ enabled = true }] : [{ enabled = false }]
9899
{% endif %}
99100
{% if beta_cluster and autopilot_cluster != true %}
100101
cluster_cloudrun_config_load_balancer_config = (var.cloudrun && var.cloudrun_load_balancer_type != "") ? {
@@ -109,7 +110,6 @@ locals {
109110
)
110111
] : []
111112
cluster_cloudrun_enabled = var.cloudrun
112-
cluster_gce_pd_csi_config = var.gce_pd_csi_driver ? [{ enabled = true }] : [{ enabled = false }]
113113
gke_backup_agent_config = var.gke_backup_agent_config ? [{ enabled = true }] : [{ enabled = false }]
114114
{% endif %}
115115
logmon_config_is_set = length(var.logging_enabled_components) > 0 || length(var.monitoring_enabled_components) > 0 || var.monitoring_enable_managed_prometheus

‎autogen/main/variables.tf.tmpl

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -603,6 +603,12 @@ variable "cluster_dns_domain" {
603603
default = ""
604604
}
605605

606+
variable "gce_pd_csi_driver" {
607+
type = bool
608+
description = "Whether this cluster should enable the Google Compute Engine Persistent Disk Container Storage Interface (CSI) Driver."
609+
default = true
610+
}
611+
606612
{% endif %}
607613
variable "timeouts" {
608614
type = map(string)
@@ -712,11 +718,5 @@ variable "enable_identity_service" {
712718
description = "Enable the Identity Service component, which allows customers to use external identity providers with the K8S API."
713719
default = false
714720
}
715-
716-
variable "gce_pd_csi_driver" {
717-
type = bool
718-
description = "(Beta) Whether this cluster should enable the Google Compute Engine Persistent Disk Container Storage Interface (CSI) Driver."
719-
default = false
720-
}
721721
{% endif %}
722722
{% endif %}

‎build/int.cloudbuild.yaml

Lines changed: 20 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -401,9 +401,26 @@ steps:
401401
- verify private-zonal-with-networking
402402
name: 'gcr.io/cloud-foundation-cicd/$_DOCKER_IMAGE_DEVELOPER_TOOLS:$_DOCKER_TAG_VERSION_DEVELOPER_TOOLS'
403403
args: ['/bin/bash', '-c', 'cft test run TestPrivateZonalWithNetworking --stage teardown --verbose --test-dir test/integration']
404-
405-
406-
404+
- id: init simple-autopilot-private-non-default-sa
405+
waitFor:
406+
- prepare
407+
name: 'gcr.io/cloud-foundation-cicd/$_DOCKER_IMAGE_DEVELOPER_TOOLS:$_DOCKER_TAG_VERSION_DEVELOPER_TOOLS'
408+
args: ['/bin/bash', '-c', 'cft test run TestSimpleAutopilotPrivateNonDefaultSA --stage init --verbose']
409+
- id: apply simple-autopilot-private-non-default-sa
410+
waitFor:
411+
- init simple-autopilot-private-non-default-sa
412+
name: 'gcr.io/cloud-foundation-cicd/$_DOCKER_IMAGE_DEVELOPER_TOOLS:$_DOCKER_TAG_VERSION_DEVELOPER_TOOLS'
413+
args: ['/bin/bash', '-c', 'cft test run TestSimpleAutopilotPrivateNonDefaultSA --stage apply --verbose']
414+
- id: verify simple-autopilot-private-non-default-sa
415+
waitFor:
416+
- apply simple-autopilot-private-non-default-sa
417+
name: 'gcr.io/cloud-foundation-cicd/$_DOCKER_IMAGE_DEVELOPER_TOOLS:$_DOCKER_TAG_VERSION_DEVELOPER_TOOLS'
418+
args: ['/bin/bash', '-c', 'cft test run TestSimpleAutopilotPrivateNonDefaultSA --stage verify --verbose']
419+
- id: teardown simple-autopilot-private-non-default-sa
420+
waitFor:
421+
- verify simple-autopilot-private-non-default-sa
422+
name: 'gcr.io/cloud-foundation-cicd/$_DOCKER_IMAGE_DEVELOPER_TOOLS:$_DOCKER_TAG_VERSION_DEVELOPER_TOOLS'
423+
args: ['/bin/bash', '-c', 'cft test run TestSimpleAutopilotPrivateNonDefaultSA --stage teardown --verbose']
407424
tags:
408425
- 'ci'
409426
- 'integration'

‎cluster.tf

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -154,7 +154,6 @@ resource "google_container_cluster" "primary" {
154154
disabled = !var.horizontal_pod_autoscaling
155155
}
156156

157-
158157
network_policy_config {
159158
disabled = !var.network_policy
160159
}
@@ -166,6 +165,14 @@ resource "google_container_cluster" "primary" {
166165
gcp_filestore_csi_driver_config {
167166
enabled = var.filestore_csi_driver
168167
}
168+
169+
dynamic "gce_persistent_disk_csi_driver_config" {
170+
for_each = local.cluster_gce_pd_csi_config
171+
172+
content {
173+
enabled = gce_persistent_disk_csi_driver_config.value.enabled
174+
}
175+
}
169176
}
170177

171178
datapath_provider = var.datapath_provider

‎docs/upgrading_to_v25.0.md

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
1+
# Upgrading to v25.0
2+
The v25.0 release of *kubernetes-engine* is a backwards incompatible
3+
release.
4+
5+
### gce_pd_csi_driver is GA and enabled by default
6+
7+
`gce_pd_csi_driver` is now supported in GA modules and defaults to true. To opt out, set `gce_pd_csi_driver` to `false`.
8+
9+
```diff
10+
module "gke" {
11+
- source = "terraform-google-modules/kubernetes-engine"
12+
- version = "~> 24.0"
13+
+ source = "terraform-google-modules/kubernetes-engine"
14+
+ version = "~> 25.0"
15+
...
16+
+ gce_pd_csi_driver = false
17+
}
18+
```
19+
20+
### Use the created service account when creating autopilot clusters
21+
22+
When `create_service_account` is `true` pass the created service account to the `cluster_autoscaling` -> `auto_provisioning_defaults` block
23+
for the `beta-autopilot-private-cluster` / `beta-autopilot-public-cluster` modules.
24+
25+
This will mean that the `Nodes` will use the created service account, where previously the default service account was erronously used instead.
26+
27+
To opt out, set `create_service_account` to `false`
28+
29+
```diff
30+
module "gke" {
31+
- source = "terraform-google-modules/kubernetes-engine"
32+
- version = "~> 24.0"
33+
+ source = "terraform-google-modules/kubernetes-engine"
34+
+ version = "~> 25.0"
35+
...
36+
+ create_service_account = false
37+
}
38+
```
39+
40+
### Minimum Google Provider versions
41+
42+
Minimum Google Provider versions have been updated to `4.44.0`.

‎examples/simple_autopilot_private_non_default_sa/README.md

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
# Simple Regional Autopilot Cluster
2+
3+
This example illustrates how to create a simple autopilot cluster with beta features and
4+
not using the default service account.
5+
6+
<!-- BEGINNING OF PRE-COMMIT-TERRAFORM DOCS HOOK -->
7+
## Inputs
8+
9+
| Name | Description | Type | Default | Required |
10+
|------|-------------|------|---------|:--------:|
11+
| project\_id | The project ID to host the cluster in | `any` | n/a | yes |
12+
13+
## Outputs
14+
15+
| Name | Description |
16+
|------|-------------|
17+
| cluster\_name | Cluster name |
18+
| kubernetes\_endpoint | The cluster endpoint |
19+
| location | n/a |
20+
| master\_kubernetes\_version | Kubernetes version of the master |
21+
| network\_name | The name of the VPC being created |
22+
| region | The region in which the cluster resides |
23+
| service\_account | The service account to default running nodes as if not overridden in `node_pools`. |
24+
| subnet\_names | The names of the subnet being created |
25+
| zones | List of zones in which the cluster resides |
26+
27+
<!-- END OF PRE-COMMIT-TERRAFORM DOCS HOOK -->
28+
29+
To provision this example, run the following from within this directory:
30+
- `terraform init` to get the plugins
31+
- `terraform plan` to see the infrastructure plan
32+
- `terraform apply` to apply the infrastructure build
33+
- `terraform destroy` to destroy the built infrastructure

‎examples/simple_autopilot_private_non_default_sa/main.tf

Lines changed: 59 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,59 @@
1+
/**
2+
* Copyright 2022 Google LLC
3+
*
4+
* Licensed under the Apache License, Version 2.0 (the "License");
5+
* you may not use this file except in compliance with the License.
6+
* You may obtain a copy of the License at
7+
*
8+
* http://www.apache.org/licenses/LICENSE-2.0
9+
*
10+
* Unless required by applicable law or agreed to in writing, software
11+
* distributed under the License is distributed on an "AS IS" BASIS,
12+
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
* See the License for the specific language governing permissions and
14+
* limitations under the License.
15+
*/
16+
17+
locals {
18+
cluster_type = "simple-ap-private-non-default-sa"
19+
network_name = "${local.cluster_type}-network"
20+
subnet_name = "${local.cluster_type}-subnet"
21+
master_auth_subnetwork = "${local.cluster_type}-master-subnet"
22+
pods_range_name = "ip-range-pods-${local.cluster_type}"
23+
svc_range_name = "ip-range-svc-${local.cluster_type}"
24+
subnet_names = [for subnet_self_link in module.gcp-network.subnets_self_links : split("/", subnet_self_link)[length(split("/", subnet_self_link)) - 1]]
25+
}
26+
27+
28+
data "google_client_config" "default" {}
29+
30+
provider "kubernetes" {
31+
host = "https://${module.gke.endpoint}"
32+
token = data.google_client_config.default.access_token
33+
cluster_ca_certificate = base64decode(module.gke.ca_certificate)
34+
}
35+
36+
module "gke" {
37+
source = "../../modules/beta-autopilot-private-cluster/"
38+
project_id = var.project_id
39+
name = "${local.cluster_type}-cluster"
40+
regional = true
41+
region = "us-central1"
42+
network = module.gcp-network.network_name
43+
subnetwork = local.subnet_names[index(module.gcp-network.subnets_names, local.subnet_name)]
44+
ip_range_pods = local.pods_range_name
45+
ip_range_services = local.svc_range_name
46+
release_channel = "REGULAR"
47+
enable_vertical_pod_autoscaling = true
48+
enable_private_endpoint = true
49+
enable_private_nodes = true
50+
master_ipv4_cidr_block = "172.16.0.0/28"
51+
datapath_provider = "ADVANCED_DATAPATH"
52+
53+
master_authorized_networks = [
54+
{
55+
cidr_block = "10.60.0.0/17"
56+
display_name = "VPC"
57+
},
58+
]
59+
}

‎examples/simple_autopilot_private_non_default_sa/network.tf

Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,50 @@
1+
/**
2+
* Copyright 2022 Google LLC
3+
*
4+
* Licensed under the Apache License, Version 2.0 (the "License");
5+
* you may not use this file except in compliance with the License.
6+
* You may obtain a copy of the License at
7+
*
8+
* http://www.apache.org/licenses/LICENSE-2.0
9+
*
10+
* Unless required by applicable law or agreed to in writing, software
11+
* distributed under the License is distributed on an "AS IS" BASIS,
12+
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
* See the License for the specific language governing permissions and
14+
* limitations under the License.
15+
*/
16+
17+
module "gcp-network" {
18+
source = "terraform-google-modules/network/google"
19+
version = ">= 4.0.1"
20+
21+
project_id = var.project_id
22+
network_name = local.network_name
23+
24+
subnets = [
25+
{
26+
subnet_name = local.subnet_name
27+
subnet_ip = "10.0.0.0/17"
28+
subnet_region = "us-central1"
29+
subnet_private_access = true
30+
},
31+
{
32+
subnet_name = local.master_auth_subnetwork
33+
subnet_ip = "10.60.0.0/17"
34+
subnet_region = "us-central1"
35+
},
36+
]
37+
38+
secondary_ranges = {
39+
(local.subnet_name) = [
40+
{
41+
range_name = local.pods_range_name
42+
ip_cidr_range = "192.168.0.0/18"
43+
},
44+
{
45+
range_name = local.svc_range_name
46+
ip_cidr_range = "192.168.64.0/18"
47+
},
48+
]
49+
}
50+
}

0 commit comments

Comments
 (0)