Skip to content

feat:add support for provisioning windows node pools #1402

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 13 additions & 0 deletions .kitchen.yml
Original file line number Diff line number Diff line change
Expand Up @@ -201,6 +201,19 @@ suites:
backend: gcp
controls:
- gcp
- name: "simple_windows_node_pool"
driver:
root_module_directory: test/fixtures/simple_windows_node_pool
verifier:
systems:
- name: gcloud
backend: local
controls:
- gcloud
- name: gcp
backend: gcp
controls:
- gcp
- name: "deploy_service"
driver:
root_module_directory: test/fixtures/deploy_service
Expand Down
9 changes: 9 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -200,6 +200,7 @@ Then perform the following commands on the root folder:
| subnetwork | The subnetwork to host the cluster in (required) | `string` | n/a | yes |
| timeouts | Timeout for cluster operations. | `map(string)` | `{}` | no |
| upstream\_nameservers | If specified, the values replace the nameservers taken by default from the node’s /etc/resolv.conf | `list(string)` | `[]` | no |
| windows\_node\_pools | List of maps containing Windows node pools | `list(map(string))` | `[]` | no |
| zones | The zones to host the cluster in (optional if regional cluster / required if zonal) | `list(string)` | `[]` | no |

## Outputs
Expand Down Expand Up @@ -232,6 +233,9 @@ Then perform the following commands on the root folder:
<!-- END OF PRE-COMMIT-TERRAFORM DOCS HOOK -->

## node_pools variable

> Use this variable for provisioning linux based node pools. For Windows based node pools use [windows_node_pools](#windows\_node\_pools-variable)

The node_pools variable takes the following parameters:

| Name | Description | Default | Requirement |
Expand Down Expand Up @@ -270,6 +274,11 @@ The node_pools variable takes the following parameters:
| tags | The list of instance tags applied to all nodes | | Required |
| value | The value for the taint | | Required |
| version | The Kubernetes version for the nodes in this pool. Should only be set if auto_upgrade is false | " " | Optional |

## windows_node_pools variable
The windows_node_pools variable takes the same parameters as [node_pools](#node\_pools-variable) but is reserved for provisioning Windows based node pools only. This variable is introduced to satisfy a [specific requirement](https://cloud.google.com/kubernetes-engine/docs/how-to/creating-a-cluster-windows#create_a_cluster_and_node_pools) for the presence of at least one linux based node pool in the cluster before a windows based node pool can be created.


## Requirements

Before this module can be used on a project, you must ensure that the following pre-requisites are fulfilled:
Expand Down
8 changes: 8 additions & 0 deletions autogen/main/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -174,6 +174,9 @@ Then perform the following commands on the root folder:

{% if autopilot_cluster != true %}
## node_pools variable

> Use this variable for provisioning linux based node pools. For Windows based node pools use [windows_node_pools](#windows\_node\_pools-variable)

The node_pools variable takes the following parameters:

| Name | Description | Default | Requirement |
Expand Down Expand Up @@ -228,7 +231,12 @@ The node_pools variable takes the following parameters:
| tags | The list of instance tags applied to all nodes | | Required |
| value | The value for the taint | | Required |
| version | The Kubernetes version for the nodes in this pool. Should only be set if auto_upgrade is false | " " | Optional |

## windows_node_pools variable
The windows_node_pools variable takes the same parameters as [node_pools](#node\_pools-variable) but is reserved for provisioning Windows based node pools only. This variable is introduced to satisfy a [specific requirement](https://cloud.google.com/kubernetes-engine/docs/how-to/creating-a-cluster-windows#create_a_cluster_and_node_pools) for the presence of at least one linux based node pool in the cluster before a windows based node pool can be created.

{% endif %}

## Requirements

Before this module can be used on a project, you must ensure that the following pre-requisites are fulfilled:
Expand Down
18 changes: 17 additions & 1 deletion autogen/main/cluster.tf.tmpl
Original file line number Diff line number Diff line change
Expand Up @@ -500,7 +500,7 @@ locals {
# resources where "ForceNew" is "true". schemaNodeConfig can be found in node_config.go at
# https://github.com/terraform-providers/terraform-provider-google/blob/master/google/node_config.go#L22
resource "random_id" "name" {
for_each = local.node_pools
for_each = merge(local.node_pools, local.windows_node_pools)
byte_length = 2
prefix = format("%s-", lookup(each.value, "name"))
keepers = merge(
Expand Down Expand Up @@ -569,13 +569,22 @@ resource "random_id" "name" {

{% endif %}
{% if autopilot_cluster != true %}
{% for i in range(2) %}
{% if i == 0 %}
resource "google_container_node_pool" "pools" {
{% else %}
resource "google_container_node_pool" "windows_pools" {
{% endif %}
{% if beta_cluster %}
provider = google-beta
{% else %}
provider = google
{% endif %}
{% if i == 0 %}
for_each = local.node_pools
{% else %}
for_each = local.windows_node_pools
{% endif %}
{% if update_variant %}
name = { for k, v in random_id.name : k => v.hex }[each.key]
{% else %}
Expand Down Expand Up @@ -756,6 +765,7 @@ resource "google_container_node_pool" "pools" {
}
}

{% if i == 0 %}
dynamic "linux_node_config" {
for_each = length(merge(
local.node_pools_linux_node_configs_sysctls["all"],
Expand All @@ -770,6 +780,7 @@ resource "google_container_node_pool" "pools" {
}
}
{% endif %}
{% endif %}

shielded_instance_config {
enable_secure_boot = lookup(each.value, "enable_secure_boot", false)
Expand All @@ -790,5 +801,10 @@ resource "google_container_node_pool" "pools" {
update = lookup(var.timeouts, "update", "45m")
delete = lookup(var.timeouts, "delete", "45m")
}

{% if i == 1 %}
depends_on = [google_container_node_pool.pools[0]]
{% endif %}
}
{% endfor %}
{% endif %}
13 changes: 11 additions & 2 deletions autogen/main/main.tf.tmpl
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,8 @@ locals {
// Build a map of maps of node pools from a list of objects
node_pool_names = [for np in toset(var.node_pools) : np.name]
node_pools = zipmap(local.node_pool_names, tolist(toset(var.node_pools)))
windows_node_pool_names = [for np in toset(var.windows_node_pools) : np.name]
windows_node_pools = zipmap(local.windows_node_pool_names, tolist(toset(var.windows_node_pools)))
{% endif %}

release_channel = var.release_channel != null ? [{ channel : var.release_channel }] : []
Expand Down Expand Up @@ -164,8 +166,15 @@ locals {
}]

{% if autopilot_cluster != true %}
cluster_output_node_pools_names = concat([for np in google_container_node_pool.pools : np.name], [""])
cluster_output_node_pools_versions = { for np in google_container_node_pool.pools : np.name => np.version }
cluster_output_node_pools_names = concat(
[for np in google_container_node_pool.pools : np.name], [""],
[for np in google_container_node_pool.windows_pools : np.name], [""]
)

cluster_output_node_pools_versions = merge(
{ for np in google_container_node_pool.pools : np.name => np.version },
{ for np in google_container_node_pool.windows_pools : np.name => np.version },
)
{% endif %}

cluster_master_auth_list_layer1 = local.cluster_output_master_auth
Expand Down
6 changes: 6 additions & 0 deletions autogen/main/variables.tf.tmpl
Original file line number Diff line number Diff line change
Expand Up @@ -148,6 +148,12 @@ variable "node_pools" {
]
}

variable "windows_node_pools" {
type = list(map(string))
description = "List of maps containing Windows node pools"
default = []
}

variable "node_pools_labels" {
type = map(map(string))
description = "Map of maps containing node labels by node-pool name"
Expand Down
20 changes: 20 additions & 0 deletions autogen/main/variables_defaults.tf.tmpl
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,10 @@ locals {
[for node_pool in var.node_pools : node_pool["name"]],
[for node_pool in var.node_pools : {}]
),
zipmap(
[for node_pool in var.windows_node_pools : node_pool["name"]],
[for node_pool in var.windows_node_pools : {}]
),
var.node_pools_labels
)

Expand All @@ -38,6 +42,10 @@ locals {
[for node_pool in var.node_pools : node_pool["name"]],
[for node_pool in var.node_pools : {}]
),
zipmap(
[for node_pool in var.windows_node_pools : node_pool["name"]],
[for node_pool in var.windows_node_pools : {}]
),
var.node_pools_metadata
)

Expand All @@ -48,6 +56,10 @@ locals {
[for node_pool in var.node_pools : node_pool["name"]],
[for node_pool in var.node_pools : []]
),
zipmap(
[for node_pool in var.windows_node_pools : node_pool["name"]],
[for node_pool in var.windows_node_pools : []]
),
var.node_pools_taints
)

Expand All @@ -58,6 +70,10 @@ locals {
[for node_pool in var.node_pools : node_pool["name"]],
[for node_pool in var.node_pools : []]
),
zipmap(
[for node_pool in var.windows_node_pools : node_pool["name"]],
[for node_pool in var.windows_node_pools : []]
),
var.node_pools_tags
)

Expand All @@ -68,6 +84,10 @@ locals {
[for node_pool in var.node_pools : node_pool["name"]],
[for node_pool in var.node_pools : []]
),
zipmap(
[for node_pool in var.windows_node_pools : node_pool["name"]],
[for node_pool in var.windows_node_pools : []]
),
var.node_pools_oauth_scopes
)
{% if beta_cluster %}
Expand Down
1 change: 1 addition & 0 deletions autogen/safer-cluster/main.tf.tmpl
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,7 @@ module "gke" {
initial_node_count = (var.initial_node_count == 0) ? 1 : var.initial_node_count

node_pools = var.node_pools
windows_node_pools = var.windows_node_pools
node_pools_labels = var.node_pools_labels
node_pools_metadata = var.node_pools_metadata
node_pools_taints = var.node_pools_taints
Expand Down
6 changes: 6 additions & 0 deletions autogen/safer-cluster/variables.tf.tmpl
Original file line number Diff line number Diff line change
Expand Up @@ -152,6 +152,12 @@ variable "node_pools" {
]
}

variable "windows_node_pools" {
type = list(map(string))
description = "List of maps containing node pools"
default = []
}

variable "node_pools_labels" {
type = map(map(string))
description = "Map of maps containing node labels by node-pool name"
Expand Down
15 changes: 15 additions & 0 deletions build/int.cloudbuild.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -236,6 +236,21 @@ steps:
- verify beta-cluster-local
name: 'gcr.io/cloud-foundation-cicd/$_DOCKER_IMAGE_DEVELOPER_TOOLS:$_DOCKER_TAG_VERSION_DEVELOPER_TOOLS'
args: ['/bin/bash', '-c', 'source /usr/local/bin/task_helper_functions.sh && kitchen_do destroy beta-cluster-local']
- id: converge simple-windows-node-pool-local
waitFor:
- create all
name: 'gcr.io/cloud-foundation-cicd/$_DOCKER_IMAGE_DEVELOPER_TOOLS:$_DOCKER_TAG_VERSION_DEVELOPER_TOOLS'
args: ['/bin/bash', '-c', 'source /usr/local/bin/task_helper_functions.sh && kitchen_do converge simple-windows-node-pool-local']
- id: verify simple-windows-node-pool-local
waitFor:
- converge simple-windows-node-pool-local
name: 'gcr.io/cloud-foundation-cicd/$_DOCKER_IMAGE_DEVELOPER_TOOLS:$_DOCKER_TAG_VERSION_DEVELOPER_TOOLS'
args: ['/bin/bash', '-c', 'source /usr/local/bin/task_helper_functions.sh && kitchen_do verify simple-windows-node-pool-local']
- id: destroy simple-windows-node-pool-local
waitFor:
- verify simple-windows-node-pool-local
name: 'gcr.io/cloud-foundation-cicd/$_DOCKER_IMAGE_DEVELOPER_TOOLS:$_DOCKER_TAG_VERSION_DEVELOPER_TOOLS'
args: ['/bin/bash', '-c', 'source /usr/local/bin/task_helper_functions.sh && kitchen_do destroy simple-windows-node-pool-local']
- id: converge deploy-service-local
waitFor:
- create all
Expand Down
Loading