Skip to content

Commit bc0769d

Browse files
mmontanmmontan
committed
Add a parameter 'registry_project_id'
The PR allows configuring the project holding the GCR registry when used in connection with 'create_service_account'=true and grant_registry_access=true. Holding the GCR is a project with other resources increases the risk of exposing sensitive data to the service account running the nodes, as the required permissions of role roles/storage.objectViewer provide access to all storage objects in the project.
1 parent 81eb717 commit bc0769d

File tree

33 files changed

+534
-14
lines changed

33 files changed

+534
-14
lines changed

‎.kitchen.yml

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -29,6 +29,7 @@ platforms:
2929
- name: local
3030

3131
suites:
32+
<<<<<<< HEAD
3233
# Disabled due to issue #274
3334
# (https://github.com/terraform-google-modules/terraform-google-kubernetes-engine/issues/274)
3435
# - name: "deploy_service"
@@ -38,6 +39,22 @@ suites:
3839
# systems:
3940
# - name: deploy_service
4041
# backend: local
42+
=======
43+
- name: "create_service_account"
44+
driver:
45+
root_module_directory: test/fixtures/create_service_account
46+
verifier:
47+
systems:
48+
- name: create_service_account
49+
backend: local
50+
- name: "deploy_service"
51+
driver:
52+
root_module_directory: test/fixtures/deploy_service
53+
verifier:
54+
systems:
55+
- name: deploy_service
56+
backend: local
57+
>>>>>>> Add a parameter 'registry_project_id'
4158
- name: "disable_client_cert"
4259
driver:
4360
root_module_directory: test/fixtures/disable_client_cert

‎README.md

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -134,7 +134,7 @@ Then perform the following commands on the root folder:
134134
| kubernetes\_version | The Kubernetes version of the masters. If set to 'latest' it will pull latest available version in the selected region. | string | `"latest"` | no |
135135
| logging\_service | The logging service that the cluster should write logs to. Available options include logging.googleapis.com, logging.googleapis.com/kubernetes (beta), and none | string | `"logging.googleapis.com"` | no |
136136
| maintenance\_start\_time | Time window specified for daily maintenance operations in RFC3339 format | string | `"05:00"` | no |
137-
| master\_authorized\_networks\_config | The desired configuration options for master authorized networks. The object format is {cidr_blocks = list(object({cidr_block = string, display_name = string}))}. Omit the nested cidr_blocks attribute to disallow external access (except the cluster node IPs, which GKE automatically whitelists). | object | `<list>` | no |
137+
| master\_authorized\_networks\_config | The desired configuration options for master authorized networks. The object format is {cidr_blocks = list(object({cidr_block = string, display_name = string}))}. Omit the nested cidr_blocks attribute to disallow external (except the cluster node IPs, which GKE automatically whitelists). | object | `<list>` | no |
138138
| monitoring\_service | The monitoring service that the cluster should write metrics to. Automatically send metrics from pods in the cluster to the Google Cloud Monitoring API. VM metrics will be collected by Google Compute Engine regardless of this setting Available options include monitoring.googleapis.com, monitoring.googleapis.com/kubernetes (beta) and none | string | `"monitoring.googleapis.com"` | no |
139139
| name | The name of the cluster (required) | string | n/a | yes |
140140
| network | The VPC network to host the cluster in (required) | string | n/a | yes |
@@ -151,6 +151,7 @@ Then perform the following commands on the root folder:
151151
| project\_id | The project ID to host the cluster in (required) | string | n/a | yes |
152152
| region | The region to host the cluster in (required) | string | n/a | yes |
153153
| regional | Whether is a regional cluster (zonal cluster if set false. WARNING: changing this after cluster creation is destructive!) | bool | `"true"` | no |
154+
| registry\_project\_id | Project holding the Google Container Registry. If empty, we use the cluster project. If grant_registry_access is true, storage.objectViewer role is assigned on this project. | string | `""` | no |
154155
| remove\_default\_node\_pool | Remove default node pool while setting up the cluster | bool | `"false"` | no |
155156
| service\_account | The service account to run nodes as if not overridden in `node_pools`. The create_service_account variable default value (true) will cause a cluster-specific service account to be created. | string | `""` | no |
156157
| stub\_domains | Map of stub domains and their resolvers to forward DNS queries for a certain domain to an external DNS server | map(list(string)) | `<map>` | no |
@@ -212,6 +213,9 @@ following project roles:
212213
- roles/iam.serviceAccountUser
213214
- roles/resourcemanager.projectIamAdmin (only required if `service_account` is set to `create`)
214215

216+
Additionally, if `service_account` is set to `create` and `grant_registry_access` is requested, the service account requires the following role on the `registry_project_id` project:
217+
- roles/resourcemanager.projectIamAdmin
218+
215219
### Enable APIs
216220
In order to operate with the Service Account you must activate the following APIs on the project where the Service Account was created:
217221

‎autogen/README.md

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -269,6 +269,9 @@ following project roles:
269269
- roles/iam.serviceAccountUser
270270
- roles/resourcemanager.projectIamAdmin (only required if `service_account` is set to `create`)
271271

272+
Additionally, if `service_account` is set to `create` and `grant_registry_access` is requested, the service account requires the following role on the `registry_project_id` project:
273+
- roles/resourcemanager.projectIamAdmin
274+
272275
### Enable APIs
273276
In order to operate with the Service Account you must activate the following APIs on the project where the Service Account was created:
274277

‎autogen/sa.tf

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -64,7 +64,7 @@ resource "google_project_iam_member" "cluster_service_account-monitoring_viewer"
6464

6565
resource "google_project_iam_member" "cluster_service_account-gcr" {
6666
count = var.create_service_account && var.grant_registry_access ? 1 : 0
67-
project = var.project_id
67+
project = var.registry_project_id == "" ? var.project_id : var.registry_project_id
6868
role = "roles/storage.objectViewer"
6969
member = "serviceAccount:${google_service_account.cluster_service_account[0].email}"
7070
}

‎autogen/variables.tf

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -79,7 +79,7 @@ variable "node_version" {
7979

8080
variable "master_authorized_networks_config" {
8181
type = list(object({cidr_blocks = list(object({cidr_block = string, display_name = string}))}))
82-
description = "The desired configuration options for master authorized networks. The object format is {cidr_blocks = list(object({cidr_block = string, display_name = string}))}. Omit the nested cidr_blocks attribute to disallow external access (except the cluster node IPs, which GKE automatically whitelists)."
82+
description = "The desired configuration options for master authorized networks. The object format is {cidr_blocks = list(object({cidr_block = string, display_name = string}))}. Omit the nested cidr_blocks attribute to disallow external (except the cluster node IPs, which GKE automatically whitelists)."
8383
default = []
8484
}
8585

@@ -269,6 +269,12 @@ variable "grant_registry_access" {
269269
default = false
270270
}
271271

272+
variable "registry_project_id" {
273+
type = string
274+
description = "Project holding the Google Container Registry. If empty, we use the cluster project. If grant_registry_access is true, storage.objectViewer role is assigned on this project."
275+
default = ""
276+
}
277+
272278
variable "service_account" {
273279
type = string
274280
description = "The service account to run nodes as if not overridden in `node_pools`. The create_service_account variable default value (true) will cause a cluster-specific service account to be created."

‎examples/create_service_account/README.md

Lines changed: 45 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,45 @@
1+
# Simple Regional Cluster
2+
3+
This example illustrates how to create a simple private cluster.
4+
5+
<!-- BEGINNING OF PRE-COMMIT-TERRAFORM DOCS HOOK -->
6+
## Inputs
7+
8+
| Name | Description | Type | Default | Required |
9+
|------|-------------|:----:|:-----:|:-----:|
10+
| cluster\_name\_suffix | A suffix to append to the default cluster name | string | `""` | no |
11+
| ip\_range\_pods | The secondary ip range to use for pods | string | n/a | yes |
12+
| ip\_range\_services | The secondary ip range to use for pods | string | n/a | yes |
13+
| network | The VPC network to host the cluster in | string | n/a | yes |
14+
| project\_id | The project ID to host the cluster in | string | n/a | yes |
15+
| region | The region to host the cluster in | string | n/a | yes |
16+
| registry\_project\_id | Project name for the GCR registry | string | n/a | yes |
17+
| subnetwork | The subnetwork to host the cluster in | string | n/a | yes |
18+
| zones | The zone to host the cluster in (required if is a zonal cluster) | list(string) | n/a | yes |
19+
20+
## Outputs
21+
22+
| Name | Description |
23+
|------|-------------|
24+
| ca\_certificate | |
25+
| client\_token | |
26+
| cluster\_name | Cluster name |
27+
| ip\_range\_pods | The secondary IP range used for pods |
28+
| ip\_range\_services | The secondary IP range used for services |
29+
| kubernetes\_endpoint | |
30+
| location | |
31+
| master\_kubernetes\_version | The master Kubernetes version |
32+
| network | |
33+
| project\_id | |
34+
| region | |
35+
| service\_account | The service account to default running nodes as if not overridden in `node_pools`. |
36+
| subnetwork | |
37+
| zones | List of zones in which the cluster resides |
38+
39+
<!-- END OF PRE-COMMIT-TERRAFORM DOCS HOOK -->
40+
41+
To provision this example, run the following from within this directory:
42+
- `terraform init` to get the plugins
43+
- `terraform plan` to see the infrastructure plan
44+
- `terraform apply` to apply the infrastructure build
45+
- `terraform destroy` to destroy the built infrastructure

‎examples/create_service_account/main.tf

Lines changed: 63 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,63 @@
1+
/**
2+
* Copyright 2018 Google LLC
3+
*
4+
* Licensed under the Apache License, Version 2.0 (the "License");
5+
* you may not use this file except in compliance with the License.
6+
* You may obtain a copy of the License at
7+
*
8+
* http://www.apache.org/licenses/LICENSE-2.0
9+
*
10+
* Unless required by applicable law or agreed to in writing, software
11+
* distributed under the License is distributed on an "AS IS" BASIS,
12+
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
* See the License for the specific language governing permissions and
14+
* limitations under the License.
15+
*/
16+
17+
locals {
18+
cluster_type = "create-service-account"
19+
}
20+
21+
provider "google" {
22+
version = "~> 2.12.0"
23+
region = var.region
24+
}
25+
26+
data "google_compute_subnetwork" "subnetwork" {
27+
name = var.subnetwork
28+
project = var.project_id
29+
region = var.region
30+
}
31+
32+
module "gke" {
33+
source = "../../modules/private-cluster/"
34+
project_id = var.project_id
35+
name = "${local.cluster_type}-cluster${var.cluster_name_suffix}"
36+
regional = false
37+
region = var.region
38+
zones = var.zones
39+
network = var.network
40+
subnetwork = var.subnetwork
41+
ip_range_pods = var.ip_range_pods
42+
ip_range_services = var.ip_range_services
43+
create_service_account = true
44+
grant_registry_access = true
45+
registry_project_id = var.registry_project_id
46+
enable_private_endpoint = true
47+
enable_private_nodes = true
48+
master_ipv4_cidr_block = "172.16.0.0/28"
49+
50+
master_authorized_networks_config = [
51+
{
52+
cidr_blocks = [
53+
{
54+
cidr_block = data.google_compute_subnetwork.subnetwork.ip_cidr_range
55+
display_name = "VPC"
56+
},
57+
]
58+
},
59+
]
60+
}
61+
62+
data "google_client_config" "default" {
63+
}

‎examples/create_service_account/outputs.tf

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
1+
/**
2+
* Copyright 2018 Google LLC
3+
*
4+
* Licensed under the Apache License, Version 2.0 (the "License");
5+
* you may not use this file except in compliance with the License.
6+
* You may obtain a copy of the License at
7+
*
8+
* http://www.apache.org/licenses/LICENSE-2.0
9+
*
10+
* Unless required by applicable law or agreed to in writing, software
11+
* distributed under the License is distributed on an "AS IS" BASIS,
12+
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
* See the License for the specific language governing permissions and
14+
* limitations under the License.
15+
*/
16+
17+
output "kubernetes_endpoint" {
18+
sensitive = true
19+
value = module.gke.endpoint
20+
}
21+
22+
output "client_token" {
23+
sensitive = true
24+
value = base64encode(data.google_client_config.default.access_token)
25+
}
26+
27+
output "ca_certificate" {
28+
value = module.gke.ca_certificate
29+
}
30+
31+
output "service_account" {
32+
description = "The service account to default running nodes as if not overridden in `node_pools`."
33+
value = module.gke.service_account
34+
}
35+

‎examples/create_service_account/test_outputs.tf

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
../../test/fixtures/all_examples/test_outputs.tf

‎examples/create_service_account/variables.tf

Lines changed: 54 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,54 @@
1+
/**
2+
* Copyright 2018 Google LLC
3+
*
4+
* Licensed under the Apache License, Version 2.0 (the "License");
5+
* you may not use this file except in compliance with the License.
6+
* You may obtain a copy of the License at
7+
*
8+
* http://www.apache.org/licenses/LICENSE-2.0
9+
*
10+
* Unless required by applicable law or agreed to in writing, software
11+
* distributed under the License is distributed on an "AS IS" BASIS,
12+
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
* See the License for the specific language governing permissions and
14+
* limitations under the License.
15+
*/
16+
17+
variable "project_id" {
18+
description = "The project ID to host the cluster in"
19+
}
20+
21+
variable "cluster_name_suffix" {
22+
description = "A suffix to append to the default cluster name"
23+
default = ""
24+
}
25+
26+
variable "region" {
27+
description = "The region to host the cluster in"
28+
}
29+
30+
variable "zones" {
31+
type = list(string)
32+
description = "The zone to host the cluster in (required if is a zonal cluster)"
33+
}
34+
35+
variable "network" {
36+
description = "The VPC network to host the cluster in"
37+
}
38+
39+
variable "subnetwork" {
40+
description = "The subnetwork to host the cluster in"
41+
}
42+
43+
variable "ip_range_pods" {
44+
description = "The secondary ip range to use for pods"
45+
}
46+
47+
variable "ip_range_services" {
48+
description = "The secondary ip range to use for pods"
49+
}
50+
51+
variable "registry_project_id" {
52+
description = "Project name for the GCR registry"
53+
}
54+

‎modules/beta-private-cluster/README.md

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -169,7 +169,7 @@ In either case, upgrading to module version `v1.0.0` will trigger a recreation o
169169
| kubernetes\_version | The Kubernetes version of the masters. If set to 'latest' it will pull latest available version in the selected region. | string | `"latest"` | no |
170170
| logging\_service | The logging service that the cluster should write logs to. Available options include logging.googleapis.com, logging.googleapis.com/kubernetes (beta), and none | string | `"logging.googleapis.com"` | no |
171171
| maintenance\_start\_time | Time window specified for daily maintenance operations in RFC3339 format | string | `"05:00"` | no |
172-
| master\_authorized\_networks\_config | The desired configuration options for master authorized networks. The object format is {cidr_blocks = list(object({cidr_block = string, display_name = string}))}. Omit the nested cidr_blocks attribute to disallow external access (except the cluster node IPs, which GKE automatically whitelists). | object | `<list>` | no |
172+
| master\_authorized\_networks\_config | The desired configuration options for master authorized networks. The object format is {cidr_blocks = list(object({cidr_block = string, display_name = string}))}. Omit the nested cidr_blocks attribute to disallow external (except the cluster node IPs, which GKE automatically whitelists). | object | `<list>` | no |
173173
| master\_ipv4\_cidr\_block | (Beta) The IP range in CIDR notation to use for the hosted master network | string | `"10.0.0.0/28"` | no |
174174
| monitoring\_service | The monitoring service that the cluster should write metrics to. Automatically send metrics from pods in the cluster to the Google Cloud Monitoring API. VM metrics will be collected by Google Compute Engine regardless of this setting Available options include monitoring.googleapis.com, monitoring.googleapis.com/kubernetes (beta) and none | string | `"monitoring.googleapis.com"` | no |
175175
| name | The name of the cluster (required) | string | n/a | yes |
@@ -190,6 +190,7 @@ In either case, upgrading to module version `v1.0.0` will trigger a recreation o
190190
| project\_id | The project ID to host the cluster in (required) | string | n/a | yes |
191191
| region | The region to host the cluster in (required) | string | n/a | yes |
192192
| regional | Whether is a regional cluster (zonal cluster if set false. WARNING: changing this after cluster creation is destructive!) | bool | `"true"` | no |
193+
| registry\_project\_id | Project holding the Google Container Registry. If empty, we use the cluster project. If grant_registry_access is true, storage.objectViewer role is assigned on this project. | string | `""` | no |
193194
| remove\_default\_node\_pool | Remove default node pool while setting up the cluster | bool | `"false"` | no |
194195
| resource\_usage\_export\_dataset\_id | The dataset id for which network egress metering for this cluster will be enabled. If enabled, a daemonset will be created in the cluster to meter network egress traffic. | string | `""` | no |
195196
| sandbox\_enabled | (Beta) Enable GKE Sandbox (Do not forget to set `image_type` = `COS_CONTAINERD` and `node_version` = `1.12.7-gke.17` or later to use it). | bool | `"false"` | no |
@@ -258,6 +259,9 @@ following project roles:
258259
- roles/iam.serviceAccountUser
259260
- roles/resourcemanager.projectIamAdmin (only required if `service_account` is set to `create`)
260261

262+
Additionally, if `service_account` is set to `create` and `grant_registry_access` is requested, the service account requires the following role on the `registry_project_id` project:
263+
- roles/resourcemanager.projectIamAdmin
264+
261265
### Enable APIs
262266
In order to operate with the Service Account you must activate the following APIs on the project where the Service Account was created:
263267

‎modules/beta-private-cluster/sa.tf

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -64,7 +64,7 @@ resource "google_project_iam_member" "cluster_service_account-monitoring_viewer"
6464

6565
resource "google_project_iam_member" "cluster_service_account-gcr" {
6666
count = var.create_service_account && var.grant_registry_access ? 1 : 0
67-
project = var.project_id
67+
project = var.registry_project_id == "" ? var.project_id : var.registry_project_id
6868
role = "roles/storage.objectViewer"
6969
member = "serviceAccount:${google_service_account.cluster_service_account[0].email}"
7070
}

‎modules/beta-private-cluster/variables.tf

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -79,7 +79,7 @@ variable "node_version" {
7979

8080
variable "master_authorized_networks_config" {
8181
type = list(object({ cidr_blocks = list(object({ cidr_block = string, display_name = string })) }))
82-
description = "The desired configuration options for master authorized networks. The object format is {cidr_blocks = list(object({cidr_block = string, display_name = string}))}. Omit the nested cidr_blocks attribute to disallow external access (except the cluster node IPs, which GKE automatically whitelists)."
82+
description = "The desired configuration options for master authorized networks. The object format is {cidr_blocks = list(object({cidr_block = string, display_name = string}))}. Omit the nested cidr_blocks attribute to disallow external (except the cluster node IPs, which GKE automatically whitelists)."
8383
default = []
8484
}
8585

@@ -267,6 +267,12 @@ variable "grant_registry_access" {
267267
default = false
268268
}
269269

270+
variable "registry_project_id" {
271+
type = string
272+
description = "Project holding the Google Container Registry. If empty, we use the cluster project. If grant_registry_access is true, storage.objectViewer role is assigned on this project."
273+
default = ""
274+
}
275+
270276
variable "service_account" {
271277
type = string
272278
description = "The service account to run nodes as if not overridden in `node_pools`. The create_service_account variable default value (true) will cause a cluster-specific service account to be created."

0 commit comments

Comments
 (0)