feat: Add variable to allow userdata to be passed as string to aws launch template

README.md

@@ -477,6 +477,7 @@ In case the setup does not work as intended follow the trace of events:
 | <a name="input_userdata_post_install"></a> [userdata\_post\_install](#input\_userdata\_post\_install) | Script to be ran after the GitHub Actions runner is installed on the EC2 instances | `string` | `""` | no |
 | <a name="input_userdata_pre_install"></a> [userdata\_pre\_install](#input\_userdata\_pre\_install) | Script to be ran before the GitHub Actions runner is installed on the EC2 instances | `string` | `""` | no |
 | <a name="input_userdata_template"></a> [userdata\_template](#input\_userdata\_template) | Alternative user-data template, replacing the default template. By providing your own user\_data you have to take care of installing all required software, including the action runner. Variables userdata\_pre/post\_install are ignored. | `string` | `null` | no |
+| <a name="input_userdata_override"></a> [userdata\_override](#input\_userdata\_override) | Alternative to userdata_template to pass userdata as string. Note: Usage of this variable overrides all templates in place by this module. You are responsible for the installation and configuration of all software, including the action runner. Use with caution. Variables userdata_pre/post_install and userdata_template are ignored. | `string` | `null` | no |
 | <a name="input_vpc_id"></a> [vpc\_id](#input\_vpc\_id) | The VPC for security groups of the action runners. | `string` | n/a | yes |
 | <a name="input_webhook_lambda_s3_key"></a> [webhook\_lambda\_s3\_key](#input\_webhook\_lambda\_s3\_key) | S3 key for webhook lambda function. Required if using S3 bucket to specify lambdas. | `any` | `null` | no |
 | <a name="input_webhook_lambda_s3_object_version"></a> [webhook\_lambda\_s3\_object\_version](#input\_webhook\_lambda\_s3\_object\_version) | S3 object version for webhook lambda function. Useful if S3 versioning is enabled on source bucket. | `any` | `null` | no |

examples/userdata/README.md

@@ -0,0 +1,33 @@
+# Action runners deployment userdata example
+
+The modules in the subdirectories of this example showcase how you can pass userdata to the AWS instances.
+
+The github-runner module uses [AWS Launch Templates](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-launch-templates.html)
+from which the runners are spawned. The launch template is passed the configuration script with the [user_data](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/launch_template#user_data)
+argument.
+
+If you need a lot of software installed on the runner, the pre-built AMI could be the best direction as it provides a 
+performance boost since the runner software does not need to be installed on every deployment. 
+[Refer to the packer scripts](https://github.com/philips-labs/terraform-aws-github-runner/tree/develop/images) 
+for more information.
+
+There are multiple ways to pass  userdata to the runners, enumerated below:
+
+- [userdata_pre_install](https://github.com/philips-labs/terraform-aws-github-runner#input_userdata_post_install) & [userdata_post_install](https://github.com/philips-labs/terraform-aws-github-runner#input_userdata_post_install) (recommended)
+- [userdata_template](https://github.com/philips-labs/terraform-aws-github-runner#input_userdata_template) (intermediate)
+- [userdata_override](https://github.com/philips-labs/terraform-aws-github-runner#input_userdata_override) (advanced)
+
+## Pre & Post Install (Recommended)
+This is the recommended way to pass additional scripts to the runners using userdata.
+`userdata_pre_install` is run prior to the installation and setup of the runner software, and `userdata_post_install` 
+is run afterwards. 
+
+## Template (Intermediate)
+`userdata_template` can be used for intermediate use cases. It accepts *the path to a script* which will then be loaded
+with [templatefile](https://www.terraform.io/language/functions/templatefile). This allows the user to reference this
+module's template variables such as `${install_runner}` and `${start_runner}`
+
+## Override (Advanced)
+`userdata_override` can be used for advanced use cases. It accepts the userdata as a whole as string. Only use this option
+if you know what you are doing. All template related variables in the module will be overridden, and you are fully
+responsible for the setup and configuration of the runner.

examples/userdata/userdata_override/README.md

@@ -0,0 +1,28 @@
+# Action runners deployment userdata_override example
+
+**NOTE:** This is an advanced use case, all templates defined in the module
+are overridden using this method. Use with caution.
+
+This module shows how to create GitHub action runners while passing the userdata
+as a string.
+
+## Usages
+
+Steps for the full setup, such as creating a GitHub app can be found in the root module's [README](../../../README.md). First download the Lambda releases from GitHub. Alternatively you can build the lambdas locally with Node or Docker, there is a simple build script in `<root>/.ci/build.sh`. In the `main.tf` you can simply remove the location of the lambda zip files, the default location will work in this case.
+
+> Ensure you have set the version in `lambdas-download/main.tf` for running the example. The version needs to be set to a GitHub release version, see https://github.com/philips-labs/terraform-aws-github-runner/releases
+
+
+```bash
+cd lambdas-download
+terraform init
+terraform apply
+cd ..
+```
+
+Before running Terraform, ensure the GitHub app is configured.
+
+```bash
+terraform init
+terraform apply
+```

examples/userdata/userdata_override/data.tf

@@ -0,0 +1,15 @@
+data "aws_iam_policy_document" "runner" {
+  statement {
+    effect = "Allow"
+
+    actions = [
+      "s3:List*",
+      "s3:Get*",
+    ]
+
+    resources = [
+      module.runners.binaries_syncer.bucket.arn,
+      "${module.runners.binaries_syncer.bucket.arn}/*"
+    ]
+  }
+}

examples/userdata/userdata_override/lambdas-download/main.tf

@@ -0,0 +1,25 @@
+locals {
+  version = "v1.2.0"
+}
+
+module "lambdas" {
+  source = "../../../../modules/download-lambda"
+  lambdas = [
+    {
+      name = "webhook"
+      tag  = local.version
+    },
+    {
+      name = "runners"
+      tag  = local.version
+    },
+    {
+      name = "runner-binaries-syncer"
+      tag  = local.version
+    }
+  ]
+}
+
+output "files" {
+  value = module.lambdas.files
+}

examples/userdata/userdata_override/main.tf

@@ -0,0 +1,149 @@
+locals {
+  environment = "ubuntu"
+  aws_region  = "us-east-1"
+  # aws_region  = "eu-west-1"
+
+  userdata = templatefile("${path.module}/scripts/user-data.sh", {
+    install_runner = templatefile("${path.module}/scripts/install-runner.sh", {
+      S3_LOCATION_RUNNER_DISTRIBUTION = "${module.runners.binaries_syncer.bucket.id}/actions-runner-linux.tar.gz",
+      RUNNER_ARCHITECTURE             = "x64",
+      ssm_key_cloudwatch_agent_config = aws_ssm_parameter.cloudwatch_agent_config_runner.name
+    }),
+    start_runner = file("${path.module}/scripts/start-runner.sh")
+  })
+
+  logfiles = [
+    {
+      "log_group_name" : "syslog",
+      "prefix_log_group" : true,
+      "file_path" : "/var/log/syslog",
+      "log_stream_name" : "{instance_id}"
+    },
+    {
+      "log_group_name" : "user_data",
+      "prefix_log_group" : true,
+      "file_path" : "/var/log/user-data.log",
+      "log_stream_name" : "{instance_id}/user_data"
+    },
+    {
+      "log_group_name" : "runner",
+      "prefix_log_group" : true,
+      "file_path" : "/opt/actions-runner/_diag/Runner_**.log",
+      "log_stream_name" : "{instance_id}/runner"
+    }
+  ]
+  loggroups_names     = distinct([for l in local.logfiles : l.log_group_name])
+  log_ssm_config_name = "${local.environment}-cloudwatch_agent_config_runner"
+
+  key = filebase64("private-key.pem")
+}
+
+resource "random_id" "random" {
+  byte_length = 20
+}
+
+data "aws_caller_identity" "current" {}
+
+module "runners" {
+  source = "../../../"
+
+  aws_region = local.aws_region
+  vpc_id     = module.vpc.vpc_id
+  subnet_ids = module.vpc.private_subnets
+
+  environment = local.environment
+  tags = {
+    Project = "ProjectX"
+  }
+
+  github_app = {
+    key_base64     = local.key
+    id             = var.github_app_id
+    webhook_secret = random_id.random.hex
+  }
+  webhook_lambda_zip                = "./lambdas-download/webhook.zip"
+  runner_binaries_syncer_lambda_zip = "./lambdas-download/runner-binaries-syncer.zip"
+  runners_lambda_zip                = "./lambdas-download/runners.zip"
+  # webhook_lambda_zip                = "lambdas-download/webhook.zip"
+  # runner_binaries_syncer_lambda_zip = "lambdas-download/runner-binaries-syncer.zip"
+  # runners_lambda_zip                = "lambdas-download/runners.zip"
+
+  enable_organization_runners = false
+  runner_extra_labels         = "ubuntu,example"
+
+  # enable access to the runners via SSM
+  enable_ssm_on_runners = true
+
+  runner_run_as = "ubuntu"
+
+  # AMI selection and userdata
+  #
+  # configure your pre-built AMI + userdata
+  userdata_override = local.userdata
+  ami_owners        = ["099720109477"] # Canonical's Amazon account ID
+
+  ami_filter = {
+    name = ["ubuntu/images/hvm-ssd/ubuntu-focal-20.04-amd64-server-*"]
+  }
+
+  instance_types = [
+    "t3.small"
+  ]
+
+  runner_iam_role_managed_policy_arns = [
+    aws_iam_policy.runner.arn
+  ]
+
+  # Custom build AMI, no custom userdata needed.
+  # option 2: Build custom AMI see ../../../images/ubuntu-focal
+  #           disable lines above (option 1) and enable the ones below
+  # ami_filter = { name = ["github-runner-ubuntu-focal-amd64-*"] }
+  # ami_owners = [data.aws_caller_identity.current.account_id]
+
+
+  block_device_mappings = [{
+    # Set the block device name for Ubuntu root device
+    device_name           = "/dev/sda1"
+    delete_on_termination = true
+    volume_type           = "gp3"
+    volume_size           = 30
+    encrypted             = true
+    iops                  = null
+  }]
+
+  // Since the logging SSM parameters are not outputs of the module, they need to be setup outside of the module
+  enable_cloudwatch_agent = false
+  runner_log_files        = []
+
+  # Uncomment to enable ephemeral runners
+  # delay_webhook_event      = 0
+  # enable_ephemeral_runners = true
+  # enabled_userdata         = false
+
+  # Uncommet idle config to have idle runners from 9 to 5 in time zone Amsterdam
+  # idle_config = [{
+  #   cron      = "* * 9-17 * * *"
+  #   timeZone  = "Europe/Amsterdam"
+  #   idleCount = 1
+  # }]
+
+}
+
+resource "aws_ssm_parameter" "cloudwatch_agent_config_runner" {
+  name = "${local.environment}-cloudwatch_agent_config_runner"
+  type = "String"
+  value = templatefile("${path.module}/scripts/cloudwatch_config.json", {
+    logfiles = jsonencode(local.logfiles)
+  })
+}
+
+resource "aws_cloudwatch_log_group" "gh_runners" {
+  count             = length(local.loggroups_names)
+  name              = local.loggroups_names[count.index]
+  retention_in_days = 14
+}
+
+resource "aws_iam_policy" "runner" {
+  name   = "github-runner-${local.environment}-access"
+  policy = data.aws_iam_policy_document.runner.json
+}

examples/userdata/userdata_override/outputs.tf

@@ -0,0 +1,15 @@
+output "runners" {
+  value = {
+    lambda_syncer_name = module.runners.binaries_syncer.lambda.function_name
+  }
+}
+
+output "webhook_endpoint" {
+  value = module.runners.webhook.endpoint
+}
+
+output "webhook_secret" {
+  sensitive = true
+  value     = random_id.random.hex
+}
+

examples/userdata/userdata_override/providers.tf

@@ -0,0 +1,11 @@
+provider "aws" {
+  region = local.aws_region
+
+  // If you use roles with specific permissions please add your role
+  // assume_role {
+  //   role_arn = "arn:aws:iam::123456789012:role/MyAdminRole"
+  // }
+}
+
+provider "random" {
+}