Skip to content

Commit 52b2a14

Browse files
authored
BREAKING CHANGE: update module to allow for control over individual cluster instances and latest features (#243)
1 parent 6711deb commit 52b2a14

35 files changed

+2182
-1006
lines changed

.github/workflows/pre-commit.yml

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -46,7 +46,6 @@ jobs:
4646
uses: clowdhaus/terraform-composite-actions/[email protected]
4747
with:
4848
terraform-version: ${{ steps.minMax.outputs.minVersion }}
49-
terraform-docs-version: ${{ env.TERRAFORM_DOCS_VERSION }}
5049
args: 'terraform_validate --color=always --show-diff-on-failure --files ${{ matrix.directory }}/*'
5150

5251
- name: Pre-commit Terraform ${{ steps.minMax.outputs.minVersion }}
@@ -55,7 +54,6 @@ jobs:
5554
uses: clowdhaus/terraform-composite-actions/[email protected]
5655
with:
5756
terraform-version: ${{ steps.minMax.outputs.minVersion }}
58-
terraform-docs-version: ${{ env.TERRAFORM_DOCS_VERSION }}
5957
args: 'terraform_validate --color=always --show-diff-on-failure --files $(ls *.tf)'
6058

6159
preCommitMaxVersion:

README.md

Lines changed: 255 additions & 99 deletions
Large diffs are not rendered by default.

UPGRADE-6.0.md

Lines changed: 262 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,262 @@
1+
# Upgrade from v5.x to v6.x
2+
3+
If you have any questions regarding this upgrade process, please consult the `examples` directory:
4+
5+
- [Autoscaling](examples/autoscaling): A PostgreSQL cluster with enhanced monitoring and autoscaling enabled
6+
- [Global Cluster](examples/global_cluster): A PostgreSQL global cluster with clusters provisioned in two different region
7+
- [MySQL](examples/mysql): A simple MySQL cluster
8+
- [PostgreSQL](examples/postgresql): A simple PostgreSQL cluster
9+
- [S3 Import](examples/s3_import): A MySQL cluster created from a Percona Xtrabackup stored in S3
10+
- [Serverless](examples/serverless): Serverless PostgreSQL and MySQL clusters
11+
12+
If you find a bug, please open an issue with supporting configuration to reproduce.
13+
14+
## Changes
15+
16+
- Added `create_db_subnet_group` variable to control whether a DB subnet group is created or not; previously if `db_subnet_group_name` was specified then an existing subnet group was used. Now the combination of `create_db_subnet_group` and `db_subnet_group_name` determine the name of the subnet group AND whether to create anew or use existing
17+
- Minimum version of AWS provider increased to v3.63.0 to support features added
18+
- Added `random_password_length` to give users control over the length of the random password that can be created; defaults to prior existing value of `10`
19+
- Local variable check added on `aws_rds_cluster` for attributes `database_name`, `master_username` and `master_pass` to support secondary clusters (global cluster, replication cluster, etc.)
20+
- Conditional creation check for `is_serverless` added to `aws_rds_cluster_instance`, `aws_appautoscaling_target`, and `aws_appautoscaling_policy` since these are not applicable for serverless clusters
21+
- Added `availability_zone`, `copy_tags_to_snapshot` attributes to `aws_rds_cluster_instance` which are now accessible via the `instances` map
22+
- Removed `engine_version` from `aws_rds_cluster_instance` lifecycle ignore block now that this has been [patched in upstream AWS provider](https://github.com/hashicorp/terraform-provider-aws/pull/17111)
23+
- New resource `aws_rds_cluster_endpoint` added to allow for creation of custom, additional endpoints
24+
- New resource `aws_rds_cluster_role_association` added to allow for association IAM roles with the cluster
25+
- Where applicable, variable default values have been set to `null` to use upstream AWS provider default values
26+
- Update variable descriptions from upstream provider docs
27+
28+
## List of backwards incompatible changes
29+
30+
- Coalesce of previous, default IAM enhanced monitoring role name has been removed (this was marked as a `TODO` to remove at next breaking change)
31+
- `Name` tag removed from DB subnet group and enhanced monitoring role; the name is now set by the provider resource and this is no longer necessary
32+
- `iam_roles` removed from `aws_rds_cluster` resource with the addition of `aws_rds_cluster_role_association`; per the docs this will cause conflicts and only one should be used and the role association resource contains more functionality/features
33+
- `replication_source_identifier` added to `aws_rds_cluster` ignore lifecycle block to support [secondary, replication clusters per the docs](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/rds_cluster#replication_source_identifier)
34+
- `global_cluster_identifier` added to `aws_rds_clsuter` ignore lifecycle block to support [global clusters per the docs](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/rds_global_cluster#new-global-cluster-from-existing-db-cluster)
35+
- `replica_count`replaced with `instances`; instances are now controlled by a map of maps, allowing users to create a homogenous cluster or a heterogenous cluster with fine grain control over the instances provisioned through the use of the new map attributes. This means the `aws_rds_cluster_instance` no longer uses `count` and now uses `for_each` for better stability and isolated change control.
36+
- `iam_roles` variable has been re-purposed from a list of IAM roles to associate with the cluster via the `aws_rds_cluster` resource and instead is now a map of maps to associate roles via the new `aws_rds_cluster_role_association` resource
37+
- `aws_appautoscaling_target` resource identifier changed from `read_replica_count` to `this` to follow conventions. This change requires either re-creation or a Terraform state rename to avoid disruption (see below)
38+
39+
### Variable and output changes
40+
41+
1. Removed variables:
42+
43+
- `replica_count`replaced with `instances`; instances are now controlled by a map of maps, allowing users to create a homogenous cluster or a heterogenous cluster with fine grain control over the instances provisioned through the use of the new map attributes
44+
- `instances_parameters` - no longer required now that individual instance attributes are defined within the map of maps passed to `instances`
45+
- `instance_type_replica` - no longer required now that individual instance attributes are defined within the map of maps passed to `instances`
46+
47+
2. Renamed variables:
48+
49+
- `password` -> `master_password` to match resource's convention
50+
- `username` -> `master_username` to match resource's convention'
51+
- `replica_scale_enabled` -> `autoscaling_enabled`
52+
- `replica_scale_min` -> `autoscaling_min_capacity`
53+
- `replica_scale_max` -> `autoscaling_max_capacity`
54+
- `replica_scale_cpu` -> `autoscaling_target_cpu`
55+
- `replica_scale_connections` -> `autoscaling_target_connections`
56+
57+
3. Added variables:
58+
59+
- `create_db_subnet_group` to control whether a DB subnet group is created or not; previously controlled by whether `db_subnet_group_name` was specified or not
60+
- `random_password_length` to give users control over the length of the random password that can be created
61+
- `enable_global_write_forwarding` to support the same attribute on `aws_rds_cluster`
62+
- `performance_insights_retention_period` to support the same attribute on `aws_rds_cluster`
63+
- `db_cluster_db_instance_parameter_group_name` to support the `db_instance_parameter_group_name` attribute on `aws_rds_cluster`
64+
- `cluster_timeouts` to support `create`, `update`, and `delete` timeouts on `aws_rds_cluster`
65+
- `instances_use_identifier_prefix` to enable the option of using a prefix name strategy on cluster instances created
66+
- `instance_timeouts` to support `create`, `update`, and `delete` timeouts on `aws_rds_cluster_instance`
67+
- `endpoints` to support new resources `aws_rds_cluster_endpoint` resource(s)
68+
69+
4. Removed outputs:
70+
71+
- `rds_cluster_instance_*` outputs have been removed in favor of one output `cluster_instances` which contains a map of all instances created and their attributes
72+
73+
5. Renamed outputs:
74+
75+
- Outputs that started with `rds_cluster_*` have been updated to start with `cluster_*` (dropping the preceding `rds_`)
76+
77+
6. Added outputs:
78+
79+
- `db_subnet_group_name`
80+
- `cluster_members`
81+
- `cluster_engine_version_actual`
82+
- `additional_cluster_endpoints`
83+
- `cluster_role_associations`
84+
85+
## Upgrade Migrations
86+
87+
### Before 5.x Example
88+
89+
```hcl
90+
module "cluster_before" {
91+
source = "terraform-aws-modules/rds-aurora/aws"
92+
version = "~> 5.0"
93+
94+
name = "before-5.x"
95+
engine = "aurora-postgresql"
96+
engine_version = "11.9"
97+
instance_type = "db.r5.large"
98+
- replica_count = 3
99+
100+
- instances_parameters = [
101+
# List index should be equal to `replica_count`
102+
# Omitted keys replaced by module defaults
103+
{
104+
instance_type = "db.r5.2xlarge"
105+
publicly_accessible = true
106+
},
107+
{
108+
instance_type = "db.r5.2xlarge"
109+
},
110+
{
111+
instance_name = "reporting"
112+
instance_type = "db.r5.large"
113+
instance_promotion_tier = 15
114+
}
115+
]
116+
117+
- autoscaling_enabled = true
118+
- autoscaling_min_capacity = 1
119+
- autoscaling_max_capacity = 5
120+
121+
vpc_id = "vpc-12345678"
122+
subnets = ["subnet-12345678", "subnet-87654321"]
123+
124+
allowed_security_groups = ["sg-12345678"]
125+
allowed_cidr_blocks = ["10.20.0.0/20"]
126+
127+
storage_encrypted = true
128+
apply_immediately = true
129+
monitoring_interval = 10
130+
131+
db_parameter_group_name = "default"
132+
db_cluster_parameter_group_name = "default"
133+
134+
enabled_cloudwatch_logs_exports = ["postgresql"]
135+
136+
tags = {
137+
Environment = "dev"
138+
Terraform = "true"
139+
}
140+
}
141+
```
142+
143+
### After 6.x Example
144+
145+
```hcl
146+
module "cluster_after" {
147+
source = "terraform-aws-modules/rds-aurora/aws"
148+
version = "~> 6.0"
149+
150+
name = "after-6.x"
151+
engine = "aurora-postgresql"
152+
engine_version = "11.9"
153+
instance_type = "db.r5.large"
154+
155+
+ instances = {
156+
1 = {
157+
instance_type = "db.r5.2xlarge"
158+
publicly_accessible = true
159+
}
160+
2 = {
161+
instance_type = "db.r5.2xlarge"
162+
}
163+
3 = {
164+
identifier = "reporting"
165+
instance_type = "db.r5.large"
166+
instance_promotion_tier = 15
167+
}
168+
]
169+
170+
+ autoscaling_enabled = true
171+
+ autoscaling_min_capacity = 1
172+
+ autoscaling_max_capacity = 5
173+
174+
vpc_id = "vpc-12345678"
175+
subnets = ["subnet-12345678", "subnet-87654321"]
176+
177+
allowed_security_groups = ["sg-12345678"]
178+
allowed_cidr_blocks = ["10.20.0.0/20"]
179+
180+
storage_encrypted = true
181+
apply_immediately = true
182+
monitoring_interval = 10
183+
184+
db_parameter_group_name = "default"
185+
db_cluster_parameter_group_name = "default"
186+
187+
enabled_cloudwatch_logs_exports = ["postgresql"]
188+
189+
tags = {
190+
Environment = "dev"
191+
Terraform = "true"
192+
}
193+
}
194+
```
195+
196+
### State Changes
197+
198+
To migrate from the `v5.x` version to `v6.x` version example shown above, the following state move commands can be performed to maintain the current resources without modification:
199+
200+
```bash
201+
terraform state mv 'module.cluster_before.aws_rds_cluster_instance.this[0]' 'module.cluster_after.aws_rds_cluster_instance.this["1"]'
202+
# Note: this move will need to be made for each instance in the cluster, where `<index>` is the instance creation index/position and is mapped to its new `<key>` specified
203+
# in the `var.instances` map. See next line for rough pattern to follow for all instances in your cluster
204+
# terraform state mv 'module.cluster_before.aws_rds_cluster_instance.this[<index>]' 'module.cluster_after.aws_rds_cluster_instance.this["<key>"]'
205+
206+
terraform state mv 'module.cluster_before.aws_appautoscaling_policy.autoscaling_read_replica_count[0]' 'module.cluster_after.aws_appautoscaling_policy.this[0]'
207+
terraform state mv 'module.cluster_before.aws_appautoscaling_target.read_replica_count[0]' 'module.cluster_after.aws_appautoscaling_target.this[0]'
208+
```
209+
210+
For example, if you previously had a configuration such as (truncated for brevity):
211+
212+
```hcl
213+
module "aurora" {
214+
source = "terraform-aws-modules/rds-aurora/aws"
215+
version = "~> 5.x"
216+
217+
instance_type = "db.r5.large"
218+
instance_type_replica = "db.t3.medium"
219+
replica_count = 2
220+
replica_scale_enabled = true
221+
replica_scale_min = 2
222+
replica_scale_max = 5
223+
```
224+
225+
After updating the configuration to the latest 6.x changes:
226+
227+
```hcl
228+
module "aurora" {
229+
source = "terraform-aws-modules/rds-aurora/aws"
230+
version = "~> 6.x"
231+
232+
instance_class = "db.r5.large"
233+
instances = {
234+
1 = {
235+
promotion_tier = 1
236+
}
237+
2 = {
238+
instance_class = "db.t3.medium"
239+
promotion_tier = 2
240+
}
241+
242+
autoscaling_enabled = true
243+
autoscaling_min_capacity = 2
244+
autoscaling_max_capacity = 5
245+
```
246+
247+
The associated Terraform state move commands would be:
248+
249+
```bash
250+
terraform state mv 'module.aurora.aws_rds_cluster_instance.this[0]' 'module.aurora.aws_rds_cluster_instance.this["1"]'
251+
terraform state mv 'module.aurora.aws_rds_cluster_instance.this[1]' 'module.aurora.aws_rds_cluster_instance.this["2"]'
252+
253+
terraform state mv 'module.aurora.aws_appautoscaling_policy.autoscaling_read_replica_count[0]' 'module.aurora.aws_appautoscaling_policy.this[0]'
254+
terraform state mv 'module.aurora.aws_appautoscaling_target.read_replica_count[0]' 'module.aurora.aws_appautoscaling_target.this[0]'
255+
```
256+
257+
### Configuration Changes
258+
259+
To avoid re-creation of the security group created by the module, you can add the following attribute and value:
260+
```hcl
261+
security_group_description = "Managed by Terraform"
262+
```

examples/autoscaling/README.md

Lines changed: 18 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -20,13 +20,13 @@ Note that this example may create resources which cost money. Run `terraform des
2020
| Name | Version |
2121
|------|---------|
2222
| <a name="requirement_terraform"></a> [terraform](#requirement\_terraform) | >= 0.13 |
23-
| <a name="requirement_aws"></a> [aws](#requirement\_aws) | >= 3.8 |
23+
| <a name="requirement_aws"></a> [aws](#requirement\_aws) | >= 3.63 |
2424

2525
## Providers
2626

2727
| Name | Version |
2828
|------|---------|
29-
| <a name="provider_aws"></a> [aws](#provider\_aws) | >= 3.8 |
29+
| <a name="provider_aws"></a> [aws](#provider\_aws) | >= 3.63 |
3030

3131
## Modules
3232

@@ -51,18 +51,24 @@ No inputs.
5151

5252
| Name | Description |
5353
|------|-------------|
54+
| <a name="output_additional_cluster_endpoints"></a> [additional\_cluster\_endpoints](#output\_additional\_cluster\_endpoints) | A map of additional cluster endpoints and their attributes |
55+
| <a name="output_cluster_arn"></a> [cluster\_arn](#output\_cluster\_arn) | Amazon Resource Name (ARN) of cluster |
56+
| <a name="output_cluster_database_name"></a> [cluster\_database\_name](#output\_cluster\_database\_name) | Name for an automatically created database on cluster creation |
57+
| <a name="output_cluster_endpoint"></a> [cluster\_endpoint](#output\_cluster\_endpoint) | Writer endpoint for the cluster |
58+
| <a name="output_cluster_engine_version_actual"></a> [cluster\_engine\_version\_actual](#output\_cluster\_engine\_version\_actual) | The running version of the cluster database |
59+
| <a name="output_cluster_hosted_zone_id"></a> [cluster\_hosted\_zone\_id](#output\_cluster\_hosted\_zone\_id) | The Route53 Hosted Zone ID of the endpoint |
60+
| <a name="output_cluster_id"></a> [cluster\_id](#output\_cluster\_id) | The RDS Cluster Identifier |
61+
| <a name="output_cluster_instances"></a> [cluster\_instances](#output\_cluster\_instances) | A map of cluster instances and their attributes |
62+
| <a name="output_cluster_master_password"></a> [cluster\_master\_password](#output\_cluster\_master\_password) | The database master password |
63+
| <a name="output_cluster_master_username"></a> [cluster\_master\_username](#output\_cluster\_master\_username) | The database master username |
64+
| <a name="output_cluster_members"></a> [cluster\_members](#output\_cluster\_members) | List of RDS Instances that are a part of this cluster |
65+
| <a name="output_cluster_port"></a> [cluster\_port](#output\_cluster\_port) | The database port |
66+
| <a name="output_cluster_reader_endpoint"></a> [cluster\_reader\_endpoint](#output\_cluster\_reader\_endpoint) | A read-only endpoint for the cluster, automatically load-balanced across replicas |
67+
| <a name="output_cluster_resource_id"></a> [cluster\_resource\_id](#output\_cluster\_resource\_id) | The RDS Cluster Resource ID |
68+
| <a name="output_cluster_role_associations"></a> [cluster\_role\_associations](#output\_cluster\_role\_associations) | A map of IAM roles associated with the cluster and their attributes |
69+
| <a name="output_db_subnet_group_name"></a> [db\_subnet\_group\_name](#output\_db\_subnet\_group\_name) | The db subnet group name |
5470
| <a name="output_enhanced_monitoring_iam_role_arn"></a> [enhanced\_monitoring\_iam\_role\_arn](#output\_enhanced\_monitoring\_iam\_role\_arn) | The Amazon Resource Name (ARN) specifying the enhanced monitoring role |
5571
| <a name="output_enhanced_monitoring_iam_role_name"></a> [enhanced\_monitoring\_iam\_role\_name](#output\_enhanced\_monitoring\_iam\_role\_name) | The name of the enhanced monitoring role |
5672
| <a name="output_enhanced_monitoring_iam_role_unique_id"></a> [enhanced\_monitoring\_iam\_role\_unique\_id](#output\_enhanced\_monitoring\_iam\_role\_unique\_id) | Stable and unique string identifying the enhanced monitoring role |
57-
| <a name="output_rds_cluster_database_name"></a> [rds\_cluster\_database\_name](#output\_rds\_cluster\_database\_name) | Name for an automatically created database on cluster creation |
58-
| <a name="output_rds_cluster_endpoint"></a> [rds\_cluster\_endpoint](#output\_rds\_cluster\_endpoint) | The cluster endpoint |
59-
| <a name="output_rds_cluster_id"></a> [rds\_cluster\_id](#output\_rds\_cluster\_id) | The ID of the cluster |
60-
| <a name="output_rds_cluster_instance_endpoints"></a> [rds\_cluster\_instance\_endpoints](#output\_rds\_cluster\_instance\_endpoints) | A list of all cluster instance endpoints |
61-
| <a name="output_rds_cluster_instance_ids"></a> [rds\_cluster\_instance\_ids](#output\_rds\_cluster\_instance\_ids) | A list of all cluster instance ids |
62-
| <a name="output_rds_cluster_master_password"></a> [rds\_cluster\_master\_password](#output\_rds\_cluster\_master\_password) | The master password |
63-
| <a name="output_rds_cluster_master_username"></a> [rds\_cluster\_master\_username](#output\_rds\_cluster\_master\_username) | The master username |
64-
| <a name="output_rds_cluster_port"></a> [rds\_cluster\_port](#output\_rds\_cluster\_port) | The port |
65-
| <a name="output_rds_cluster_reader_endpoint"></a> [rds\_cluster\_reader\_endpoint](#output\_rds\_cluster\_reader\_endpoint) | The cluster reader endpoint |
66-
| <a name="output_rds_cluster_resource_id"></a> [rds\_cluster\_resource\_id](#output\_rds\_cluster\_resource\_id) | The Resource ID of the cluster |
6773
| <a name="output_security_group_id"></a> [security\_group\_id](#output\_security\_group\_id) | The security group ID of the cluster |
6874
<!-- END OF PRE-COMMIT-TERRAFORM DOCS HOOK -->

examples/autoscaling/main.tf

Lines changed: 15 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@ provider "aws" {
33
}
44

55
locals {
6-
name = "advanced"
6+
name = "example-${replace(basename(path.cwd), "_", "-")}"
77
region = "eu-west-1"
88
tags = {
99
Owner = "user"
@@ -37,24 +37,24 @@ module "vpc" {
3737
module "aurora" {
3838
source = "../../"
3939

40-
name = local.name
41-
engine = "aurora-postgresql"
42-
engine_version = "11.9"
43-
instance_type = "db.r5.large"
44-
instance_type_replica = "db.t3.large"
40+
name = local.name
41+
engine = "aurora-postgresql"
42+
engine_version = "11.12"
43+
instance_class = "db.r6g.large"
44+
instances = { 1 = {} }
4545

46-
vpc_id = module.vpc.vpc_id
47-
db_subnet_group_name = module.vpc.database_subnet_group_name
48-
create_security_group = true
49-
allowed_cidr_blocks = module.vpc.private_subnets_cidr_blocks
46+
vpc_id = module.vpc.vpc_id
47+
db_subnet_group_name = module.vpc.database_subnet_group_name
48+
create_db_subnet_group = false
49+
create_security_group = true
50+
allowed_cidr_blocks = module.vpc.private_subnets_cidr_blocks
5051

51-
replica_count = 1
52-
replica_scale_enabled = true
53-
replica_scale_min = 1
54-
replica_scale_max = 5
52+
autoscaling_enabled = true
53+
autoscaling_min_capacity = 1
54+
autoscaling_max_capacity = 5
5555

5656
monitoring_interval = 60
57-
iam_role_name = "${local.name}-enhanced-monitoring"
57+
iam_role_name = "${local.name}-monitor"
5858
iam_role_use_name_prefix = true
5959
iam_role_description = "${local.name} RDS enhanced monitoring IAM role"
6060
iam_role_path = "/autoscaling/"

0 commit comments

Comments
 (0)