Skip to content

Commit 6ea4117

Browse files
markgoddardmarkgoddard
committed
Add Terraform and Ansible to deploy GitHub runner VMs
1 parent 1770918 commit 6ea4117

File tree

13 files changed

+363
-0
lines changed

13 files changed

+363
-0
lines changed

‎terraform/github-runners/.terraform.lock.hcl

Lines changed: 46 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎terraform/github-runners/README.rst

Lines changed: 148 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,148 @@
1+
========================
2+
Terraform GitHub runners
3+
========================
4+
5+
This Terraform configuration deploys a GitHub Actions runner VMs on an
6+
OpenStack cloud for the stackhpc-release-train repository.
7+
8+
Usage
9+
=====
10+
11+
These instructions show how to use this Terraform configuration manually. They
12+
assume you are running an Ubuntu host that will be used to run Terraform. The
13+
machine should have network access to the VM that will be created by this
14+
configuration.
15+
16+
Install Terraform:
17+
18+
.. code-block:: console
19+
20+
wget -qO - terraform.gpg https://apt.releases.hashicorp.com/gpg | sudo gpg --dearmor -o /usr/share/keyrings/terraform-archive-keyring.gpg
21+
sudo echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/terraform-archive-keyring.gpg] https://apt.releases.hashicorp.com $(lsb_release -cs) main" | sudo tee /etc/apt/sources.list.d/terraform.list
22+
sudo apt update
23+
sudo apt install terraform
24+
25+
Clone and initialise the repo:
26+
27+
.. code-block:: console
28+
29+
git clone https://github.com/stackhpc/stackhpc-release-train
30+
cd stackhpc-release-train
31+
32+
Change to the terraform/github-runners directory:
33+
34+
.. code-block:: console
35+
36+
cd terraform/github-runners
37+
38+
Initialise Terraform:
39+
40+
.. code-block:: console
41+
42+
terraform init
43+
44+
Create an OpenStack clouds.yaml file with your credentials to access an
45+
OpenStack cloud. Alternatively, download one from Horizon. The credentials
46+
should be scoped to the stackhpc-release project.
47+
48+
.. code-block:: console
49+
50+
cat << EOF > clouds.yaml
51+
---
52+
clouds:
53+
sms-lab:
54+
auth:
55+
auth_url: https://api.sms-lab.cloud:5000
56+
username: <username>
57+
project_name: <project>
58+
domain_name: default
59+
interface: public
60+
EOF
61+
62+
Export environment variables to use the correct cloud and provide a password:
63+
64+
.. code-block:: console
65+
66+
export OS_CLOUD=sms-lab
67+
read -p OS_PASSWORD -s OS_PASSWORD
68+
export OS_PASSWORD
69+
70+
Verify that the Terraform variables in terraform.tfvars are correct.
71+
72+
Generate a plan:
73+
74+
.. code-block:: console
75+
76+
terraform plan
77+
78+
Apply the changes:
79+
80+
.. code-block:: console
81+
82+
terraform apply -auto-approve
83+
84+
Create a virtualenv:
85+
86+
.. code-block:: console
87+
88+
python3 -m venv venv
89+
90+
Activate the virtualenv:
91+
92+
.. code-block:: console
93+
94+
source venv/bin/activate
95+
96+
Install Python dependencies:
97+
98+
.. code-block:: console
99+
100+
pip install -r ansible/requirements.txt
101+
102+
Install Ansible Galaxy dependencies:
103+
104+
.. code-block:: console
105+
106+
ansible-galaxy collection install -r ansible/requirements.yml
107+
ansible-galaxy role install -r ansible/requirements.yml
108+
109+
Create a GitHub PAT token (classic) with repo:all scope. Export an environment
110+
variable with the token.
111+
112+
.. code-block:: console
113+
114+
read -p PERSONAL_ACCESS_TOKEN -s PERSONAL_ACCESS_TOKEN
115+
export PERSONAL_ACCESS_TOKEN
116+
117+
Deploy runners:
118+
119+
.. code-block:: console
120+
121+
ansible-playbook ansible/site.yml -i ansible/inventory.yml
122+
123+
To remove runners:
124+
125+
.. code-block:: console
126+
127+
ansible-playbook ansible/site.yml -i ansible/inventory.yml -e runner_state=absent
128+
129+
Troubleshooting
130+
===============
131+
132+
Install service fails
133+
---------------------
134+
135+
If you see the following::
136+
137+
TASK [monolithprojects.github_actions_runner : Install service] ********************************************************************************************************************************************
138+
fatal: [10.205.0.50]: FAILED! => changed=true
139+
cmd: ./svc.sh install ubuntu
140+
msg: '[Errno 2] No such file or directory: b''./svc.sh'''
141+
rc: 2
142+
stderr: ''
143+
stderr_lines: <omitted>
144+
stdout: ''
145+
stdout_lines: <omitted>
146+
147+
It might mean the runner is already registered, possibly from a previous VM.
148+
Remove the runner using Ansible or the GitHub settings.

‎terraform/github-runners/ansible.cfg

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
[defaults]
2+
stdout_callback = community.general.yaml
3+
host_key_checking = False
4+
pipelining = True
5+
deprecation_warnings=False

‎terraform/github-runners/ansible/group_vars/all

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
---
2+
runner_user: "{{ ansible_facts.user_id }}"
3+
github_account: stackhpc
4+
github_repo: stackhpc-release-train
5+
runner_labels:
6+
- stackhpc-release-train

‎terraform/github-runners/ansible/inventory.yml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
plugin: cloud.terraform.terraform_provider

‎terraform/github-runners/ansible/requirements.txt

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
ansible-core==2.15.*

‎terraform/github-runners/ansible/requirements.yml

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
---
2+
collections:
3+
- name: cloud.terraform
4+
- name: community.general
5+
roles:
6+
- name: monolithprojects.github_actions_runner

‎terraform/github-runners/ansible/site.yml

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
---
2+
- name: Deploy GitHub runners
3+
hosts: runners
4+
gather_facts: no
5+
tasks:
6+
- name: Wait for connection
7+
ansible.builtin.wait_for_connection:
8+
9+
- name: Gather facts
10+
ansible.builtin.setup:
11+
12+
- import_role:
13+
name: monolithprojects.github_actions_runner
14+
become: true
15+
16+
- name: Ensure runner service is running
17+
ansible.builtin.service:
18+
name: actions.runner.stackhpc-stackhpc-release-train.{{ ansible_facts.hostname }}.service
19+
state: started
20+
enabled: true
21+
become: true
22+
when: runner_state | default('started') == 'started'

‎terraform/github-runners/outputs.tf

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
output "access_ip_v4" {
2+
value = openstack_compute_instance_v2.runners.*.access_ip_v4
3+
}
4+
5+
output "access_cidr" {
6+
value = data.openstack_networking_subnet_v2.network.cidr
7+
}
8+
9+
output "access_gw" {
10+
value = data.openstack_networking_subnet_v2.network.gateway_ip
11+
}
12+
13+
resource "ansible_host" "runners" {
14+
for_each = { for host in openstack_compute_instance_v2.runners : host.name => host.access_ip_v4 }
15+
name = each.value
16+
groups = ["runners"]
17+
}
18+
19+
resource "ansible_group" "runners" {
20+
name = "runners"
21+
variables = {
22+
ansible_user = var.ssh_username
23+
}
24+
}

‎terraform/github-runners/provider.tf

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
#provider "openstack" {
2+
# use environment variables
3+
#}
4+
5+
terraform {
6+
required_version = ">= 0.14"
7+
backend "local" {
8+
}
9+
required_providers {
10+
ansible = {
11+
source = "ansible/ansible"
12+
version = "1.1.0"
13+
}
14+
openstack = {
15+
source = "terraform-provider-openstack/openstack"
16+
}
17+
}
18+
}

‎terraform/github-runners/templates/userdata.cfg.tpl

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
#cloud-config
2+
# Configure SSH keys here, to avoid creating an ephemeral keypair.
3+
# This means only the instance needs to be cleaned up if the destroy fails.
4+
ssh_authorized_keys:
5+
- ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIJXNGOmsnRcZdmjMA5f3fgq8l8+QMjBywJQzuvxlhslx mark@mark-xps15
6+
- ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAICY4CwVM472QS5TYhEwAHge4vbkuBtpVUCC8cyIolYR5 [email protected]
7+
- ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIHK1Kf1hRYskYfYtXnqnyzD6vFbpKVyUtcxcn2pYL2y+ [email protected]
8+
- ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIC/F0fT8MuXUBEDeCbi9XoaefpZuWf35NOHHECN8VyIq [email protected]
9+
- ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIATVu5COwRn9FirPl2GhQIY/RmZkNGM+CcXOhT7WujBf [email protected]
10+
- ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIJg3W6wKjLrKObikQz84K3oZJjUDpp2c6k62ZE4x1fev [email protected]
11+
- ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAID1c17mnzF71ElO+xbJKwxc2bgoR8Yb+DhcENrWfYt0d dawud
12+
- ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDeIsZEbrsSuxBcr8qI4jiZxXavEv+Crh+QwDNcezF6xgrTkIzCodYuJ6CNLhbee1A6IBIB5csmJWZ6BE8BD1RqxPH1AnCVIoQD+b606HCWYQ8UAhD7E485fE3DxDffAgsSsaK5LBatdIBu3epOrJiKGIMuJmGSAv3Hp86BkQ2y9bQXMp1jhn+cQ1WoZm9AK77viEM4X2sVmWMipUu3RV1tlI/W9mBO6GCi2N/+kDtzpZzjQEevr/xgu4zir99z2s/xDIMN2IssQm3nD4pMTrj7lkpKehtdYkHOFSW4qvhwgRuyONtoWAJjt7krOURYI4W6Inxva4yv/tqTyCFEIV4jfFGuoN7N9uaA7HtaEtSWtxtGXPkITutE3u4vsvKhPHOwjhHIL9cgy/EquP2Jm/gN8UG+iogSsLqbfJVbLdpO3X1i6m3dIsJ5WYKhPGBXRfTXRJQ0H/tbDTDr0zt7xqQDZREExirtyJ6auMYOXOrE60wMkH2hEPMeretx+4oZ1KE= will@juno
13+
- ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpMZpCHFX5yp7+5mrm4rW0x7dIQSS+49i5nVe5HrXxWSCvz8RVtEstFBN8OuOnUgZnDfpxrcxhoiCtv5WWEJQOzeUNlcjzMjTW7MdLqpVciLQLad7gUl1kepv6P0iyKrb0ze5xA0nfKVKInQmU3jSklE+aghHUXGre/pwAaqKJGsMLXLfW4uqjdG2RSAL/Bpg9126Ly0u/r1zhaNefkpIVqAYGqS7aXis6FTic/WfK4PbImqMuGjqiHNbNLT2pyzoBTRt3WDE0cRI6ryEYKKtHPtQ15H/J7kj18vek4ca7wA7nK+0UtrtSWnE2U4l/tybJ8AQQOKJ2ocjAyvDOrfumSRd+KS/ljxrOFSmyV4Zap6WJrUgzicQKOpZ0L9f3JxwTc/NlXk/BTA+COaoM3NwjYu+eHuJudJSiA0EHTQ3LdLhZiyFTo+DX+J6sZzr0keGWd0gRXQ8A0vKi1tHqerR7Reh1NGobaNzScO/1nyDAZJLt9BNI1OFMJEuZD83CTSU= [email protected]
14+
15+
# Not including these until claimed.
16+
# - ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDNG48SwtW6B9Exs08fuSyAq+KTQ7xMJaypDXiku8vIuP2Q7le6GTwmlULX12djtTvTewRzV6AsqkCyGq+2+Yo8EaKZbU08pKLuzlTH++89vKYaq+FxbbpgrDlvnT1rA5Af4AGrNcUGj3f3ovdc4XC9HL2MvVcb3lKZelCLae7HNeEES4oMVLd+GytkQPruj4SDqLLedGkJRmDN7Z88zjjhWDrVNFeXvcuJOXwlRXvuw8gR4NQgoPpET91Do8uMj5YpvH19yc9U3XOZx9wj1+sh2XJwQF6/anOpNxoFmICuSJLFc/Ulrmk8ogq3GRKHX+dT6eTIZrkX0IkSNfDnaOi20e1YP5yN0d8NffVrZHRzUwQz5LJB8fXyQgd1JIDcMyIluCjz3ND90oOoGMIDABZmz5+8eqECEh7Du7b1/XFSpPoTRWJ4YrpmSbCuYBpLKe/vNJAqUQgxwpgYuFUEJiBfWrE3B4w7FAUcc1l4/A78OrYt1umnK3OM41OZobChavE=
17+
# - ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIO4AyLmXAw4uceOvUte8H0tKEif7q63KIncL39I0QkpC
18+
# - ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOUaGDMtMzd93zbAhvZO0bDBqdna1QgHRQgDImKuPTXs

‎terraform/github-runners/terraform.tfvars

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
vm_count = 2
2+
ssh_username = "ubuntu"
3+
vm_image = "Ubuntu-20.04"
4+
vm_flavor = "general.v1.tiny"
5+
vm_network = "stackhpc-release"
6+
vm_subnet = "stackhpc-release-subnet"

‎terraform/github-runners/vm.tf

Lines changed: 62 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,62 @@
1+
variable "vm_count" {
2+
type = number
3+
}
4+
5+
variable "ssh_username" {
6+
type = string
7+
}
8+
9+
variable "vm_name" {
10+
type = string
11+
default = "srt-runner"
12+
}
13+
14+
variable "vm_image" {
15+
type = string
16+
}
17+
18+
variable "vm_flavor" {
19+
type = string
20+
}
21+
22+
variable "vm_network" {
23+
type = string
24+
}
25+
26+
variable "vm_subnet" {
27+
type = string
28+
}
29+
30+
locals {
31+
image_is_uuid = length(regexall("^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$", var.vm_image)) > 0
32+
}
33+
34+
data "openstack_images_image_v2" "image" {
35+
name = var.vm_image
36+
most_recent = true
37+
count = local.image_is_uuid ? 0 : 1
38+
}
39+
40+
data "openstack_networking_subnet_v2" "network" {
41+
name = var.vm_subnet
42+
}
43+
44+
resource "openstack_compute_instance_v2" "runners" {
45+
count = var.vm_count
46+
name = format("%s%s", var.vm_name, count.index)
47+
flavor_name = var.vm_flavor
48+
config_drive = true
49+
user_data = file("templates/userdata.cfg.tpl")
50+
network {
51+
name = var.vm_network
52+
}
53+
54+
block_device {
55+
uuid = local.image_is_uuid ? var.vm_image: data.openstack_images_image_v2.image[0].id
56+
source_type = "image"
57+
volume_size = 20
58+
boot_index = 0
59+
destination_type = "volume"
60+
delete_on_termination = true
61+
}
62+
}

0 commit comments

Comments
 (0)