Skip to content

Commit 0d6485c

Browse files
sjpbjovial
sjpb
and
jovial
authored
Add support for configuring CA certificates (#574)
* Add cacert role Useful if you have to trust a local CA. * complete cacerts, with compute-init support * enable cacerts for stackhpc env * update compute-init docs for slurm.yml features * refactor cacerts compute-init to use role directly * add roles/ to compute-init directories * add missing cacerts defaults * Add cacert role Useful if you have to trust a local CA. * complete cacerts, with compute-init support * enable cacerts for stackhpc env * update compute-init docs for slurm.yml features * refactor cacerts compute-init to use role directly * add roles/ to compute-init directories * add missing cacerts defaults * swap cacerts to use export task file * fixup merge --------- Co-authored-by: Will Szumski <[email protected]>
1 parent 8b1c94c commit 0d6485c

File tree

16 files changed

+115
-24
lines changed

16 files changed

+115
-24
lines changed

‎ansible/.gitignore

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -32,6 +32,8 @@ roles/*
3232
!roles/mysql/**
3333
!roles/systemd/
3434
!roles/systemd/**
35+
!roles/cacerts/
36+
!roles/cacerts/**
3537
!roles/cuda/
3638
!roles/cuda/**
3739
!roles/freeipa/

‎ansible/bootstrap.yml

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -130,6 +130,14 @@
130130
- appliances_mode == 'configure'
131131
- not (dnf_repos_allow_insecure_creds | default(false)) # useful for development
132132

133+
- hosts: cacerts:!builder
134+
tags: cacerts
135+
gather_facts: false
136+
tasks:
137+
- name: Install custom cacerts
138+
import_role:
139+
name: cacerts
140+
133141
- hosts: squid
134142
tags: squid
135143
gather_facts: yes

‎ansible/roles/cacerts/defaults/main.yml

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
#cacerts_dest_dir: /etc/pki/ca-trust/source/anchors/
2+
cacerts_cert_dir: "{{ appliances_environment_root }}/cacerts"
3+
cacerts_update: true

‎ansible/roles/cacerts/tasks/configure.yml

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
---
2+
3+
- name: Copy all certificates
4+
copy:
5+
src: "{{ item }}"
6+
dest: /etc/pki/ca-trust/source/anchors/
7+
owner: root
8+
group: root
9+
mode: 0644
10+
with_fileglob:
11+
- "{{ cacerts_cert_dir }}/*"
12+
become: true
13+
14+
- name: Update trust store
15+
command: update-ca-trust extract
16+
become: true

‎ansible/roles/cacerts/tasks/export.yml

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
- name: Copy cacerts from deploy host to /exports/cluster/cacerts/
2+
copy:
3+
src: "{{ item }}"
4+
dest: /exports/cluster/cacerts/
5+
owner: root
6+
group: root
7+
mode: 0644
8+
with_fileglob:
9+
- "{{ cacerts_cert_dir }}/*"
10+
delegate_to: "{{ groups['control'] | first }}"
11+
run_once: true

‎ansible/roles/cacerts/tasks/main.yml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
- import_tasks: configure.yml

‎ansible/roles/compute_init/README.md

Lines changed: 19 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -48,6 +48,7 @@ it also requires an image build with the role name added to the
4848
| bootstrap.yml | selinux | None required - use image build | Maybe [1] |
4949
| bootstrap.yml | sshd | Fully supported | No |
5050
| bootstrap.yml | dnf_repos | None at present [2] | - |
51+
| bootstrap.yml | cacerts | Supported [3] | - |
5152
| bootstrap.yml | squid | Not relevant for compute nodes | n/a |
5253
| bootstrap.yml | tuned | Fully supported | No |
5354
| bootstrap.yml | freeipa_server | Not relevant for compute nodes | n/a |
@@ -61,25 +62,25 @@ it also requires an image build with the role name added to the
6162
| bootstrap.yml | ansible_init (install) | Not relevant during boot | n/a |
6263
| bootstrap.yml | k3s (install) | Not relevant during boot | n/a |
6364
| hooks/post-bootstrap.yml | ? | None at present | n/a |
64-
| iam.yml | freeipa_client | None at present [3] | Yes |
65+
| iam.yml | freeipa_client | None at present [4] | Yes |
6566
| iam.yml | freeipa_server | Not relevant for compute nodes | n/a |
6667
| iam.yml | sssd | Fully supported | No |
6768
| filesystems.yml | block_devices | None required - role deprecated | n/a |
6869
| filesystems.yml | nfs | All client functionality | No |
69-
| filesystems.yml | manila | All functionality | No [4] |
70+
| filesystems.yml | manila | All functionality | No [5] |
7071
| filesystems.yml | lustre | None at present | Yes |
71-
| extras.yml | basic_users | All functionality [5] | No |
72-
| extras.yml | eessi | All functionality [6] | No |
73-
| extras.yml | cuda | None required - use image build | Yes [7] |
72+
| extras.yml | basic_users | All functionality [6] | No |
73+
| extras.yml | eessi | All functionality [7] | No |
74+
| extras.yml | cuda | None required - use image build | Yes [8] |
7475
| extras.yml | persist_hostkeys | Not relevant for compute nodes | n/a |
7576
| extras.yml | compute_init (export) | Not relevant for compute nodes | n/a |
7677
| extras.yml | k9s (install) | Not relevant during boot | n/a |
77-
| extras.yml | extra_packages | None at present [8] | - |
78+
| extras.yml | extra_packages | None at present [9] | - |
7879
| slurm.yml | mysql | Not relevant for compute nodes | n/a |
7980
| slurm.yml | rebuild | Not relevant for compute nodes | n/a |
80-
| slurm.yml | openhpc [9] | All slurmd functionality | No |
81-
| slurm.yml | (set memory limits) | None at present | - |
82-
| slurm.yml | (block ssh) | None at present | - |
81+
| slurm.yml | openhpc [10] | All slurmd functionality | No |
82+
| slurm.yml | (set memory limits) | Fully supported | No |
83+
| slurm.yml | (block ssh) | Fully supported | No |
8384
| portal.yml | (openondemand server) | Not relevant for compute nodes | n/a |
8485
| portal.yml | (openondemand vnc desktop) | None required - use image build | No |
8586
| portal.yml | (openondemand jupyter server) | None required - use image build | No |
@@ -92,16 +93,17 @@ it also requires an image build with the role name added to the
9293
Notes:
9394
1. `selinux` is set to disabled in StackHPC images.
9495
2. Requirement for this functionality is TBD.
95-
3. FreeIPA client functionality would be better provided using a client fork
96+
3. `cacerts_cert_dir` must be the same on all nodes.
97+
4. FreeIPA client functionality would be better provided using a client fork
9698
which uses pkinit keys rather than OTP to reenrol nodes.
97-
4. Assuming default Ceph client version.
98-
5. Assumes home directory already exists on shared storage.
99-
6. Assumes `cvmfs_config` is the same on control node and all compute nodes.
100-
7. If `cuda` role was run during build, the nvidia-persistenced is enabled
99+
5. Assuming default Ceph client version.
100+
6. Assumes home directory already exists on shared storage.
101+
7. Assumes `cvmfs_config` is the same on control node and all compute nodes.
102+
8. If `cuda` role was run during build, the nvidia-persistenced is enabled
101103
and will start during boot.
102-
8. Would require `dnf_repos`.
103-
9. `openhpc` does not need to be added to `compute_init_enable`, this is
104-
automatically enabled by adding `compute`.
104+
9. Would require `dnf_repos`.
105+
10. `openhpc` does not need to be added to `compute_init_enable`, this is
106+
automatically enabled by adding `compute`.
105107

106108
## Approach
107109
This works as follows:

‎ansible/roles/compute_init/files/compute-init.yml

Lines changed: 13 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,7 @@
99
enable_compute: "{{ os_metadata.meta.compute | default(false) | bool }}"
1010
enable_resolv_conf: "{{ os_metadata.meta.resolv_conf | default(false) | bool }}"
1111
enable_etc_hosts: "{{ os_metadata.meta.etc_hosts | default(false) | bool }}"
12+
enable_cacerts: "{{ os_metadata.meta.cacerts | default(false) | bool }}"
1213
enable_sssd: "{{ os_metadata.meta.sssd | default(false) | bool }}"
1314
enable_sshd: "{{ os_metadata.meta.sshd | default(false) | bool }}"
1415
enable_tuned: "{{ os_metadata.meta.tuned | default(false) | bool }}"
@@ -134,13 +135,12 @@
134135
mode: 0644
135136
when: enable_etc_hosts
136137

137-
- name: Configure sssd
138+
- name: Configure cacerts
138139
ansible.builtin.include_role:
139-
name: sssd
140-
tasks_from: configure.yml
140+
name: cacerts
141141
vars:
142-
sssd_conf_src: "/mnt/cluster/hostconfig/{{ ansible_hostname }}/sssd.conf"
143-
when: enable_sssd
142+
cacerts_cert_dir: "/mnt/cluster/cacerts"
143+
when: enable_cacerts
144144

145145
- name: Configure sshd
146146
ansible.builtin.include_role:
@@ -153,6 +153,14 @@
153153
include_tasks: tasks/tuned.yml
154154
when: enable_tuned
155155

156+
- name: Configure sssd
157+
ansible.builtin.include_role:
158+
name: sssd
159+
tasks_from: configure.yml
160+
vars:
161+
sssd_conf_src: "/mnt/cluster/hostconfig/{{ ansible_hostname }}/sssd.conf"
162+
when: enable_sssd
163+
156164
# NFS client mount
157165
- name: If nfs-clients is present
158166
include_tasks: tasks/nfs-clients.yml

‎ansible/roles/compute_init/tasks/export.yml

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -72,6 +72,12 @@
7272
run_once: true
7373
delegate_to: "{{ groups['control'] | first }}"
7474

75+
- name: Export cacerts
76+
ansible.builtin.include_role:
77+
name: cacerts
78+
tasks_from: export.yml
79+
when: "'cacerts' in group_names"
80+
7581
- name: Create hostconfig directory
7682
file:
7783
path: "/exports/cluster/hostconfig/{{ inventory_hostname }}/"

‎ansible/roles/compute_init/tasks/install.yml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -33,6 +33,8 @@
3333
dest: files/NetworkManager-dns-none.conf
3434
- src: ../../basic_users/filter_plugins/filter_keys.py
3535
dest: filter_plugins/filter_keys.py
36+
- src: ../../cacerts
37+
dest: roles/
3638
- src: ../../sssd
3739
dest: roles/
3840
- src: ../../sshd

‎environments/.stackhpc/cacerts/myCA.pem

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
-----BEGIN CERTIFICATE-----
2+
MIIDgzCCAmugAwIBAgIUd5qnvmXczLvacv3Mu2hzwJlmimMwDQYJKoZIhvcNAQEL
3+
BQAwUTELMAkGA1UEBhMCWFgxFTATBgNVBAcMDERlZmF1bHQgQ2l0eTEcMBoGA1UE
4+
CgwTRGVmYXVsdCBDb21wYW55IEx0ZDENMAsGA1UEAwwEdGVzdDAeFw0yNTAyMTIx
5+
NjIxNTlaFw0zMDAyMTExNjIxNTlaMFExCzAJBgNVBAYTAlhYMRUwEwYDVQQHDAxE
6+
ZWZhdWx0IENpdHkxHDAaBgNVBAoME0RlZmF1bHQgQ29tcGFueSBMdGQxDTALBgNV
7+
BAMMBHRlc3QwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQDExC9wqRyG
8+
vQ5FYGb48iDfq8er4WvWO94F/q746mCHvVJn7GTu3AMavIXCYqH9WnXY0lzey7xU
9+
/40/F/xihQfGYFrY+8ssYrT8Z+H3fSuwmq6XqsHcCupBQHKTTjZWaVMODxF4Eq5F
10+
Vyk4/AJpoOFLrzjUA9Sw74HKBH+r3N74x+3fFzElFGfjtFXPlgnYi9T9dXEEoNc7
11+
Udulcr6MrL+l6ITr0Grti4FP0qOari9a4XqC7G2Jtga1PF/GaMlyrmQphnhpS7ph
12+
n1dr6hYWmHZ1r1vcNBxBl71CoOVoLwk9v2x0jOsbYpzAp5CJEl/6whwo/Pn2JzIV
13+
xbCuVg9znbHpAgMBAAGjUzBRMB0GA1UdDgQWBBSEbb8xKKL1NwsRfzeZ7Shyq9xq
14+
QTAfBgNVHSMEGDAWgBSEbb8xKKL1NwsRfzeZ7Shyq9xqQTAPBgNVHRMBAf8EBTAD
15+
AQH/MA0GCSqGSIb3DQEBCwUAA4IBAQB2z7YMpZKAPY19EWaTV80Gwks56hBClcfR
16+
6Y6d/7+ltML5pRHCFB2fF850Rj5vmnflSwrSWDcDbRktEfha3OIhHWtY8TzF7Zkx
17+
dIMyN8JaqjmJ488WGhcuqQDIK5sREg/JfECVeBId5mF390TKszlM9FNQL1NOC0D+
18+
I/+BeWHYAu4dGWQR6xbC6SYUMbhTQrQSgJFckq5i2fQPcNK8Xlnzc+oxjJuqgsfB
19+
P1oLnrb2OVHEpjuxdK1UYds3z/6ilKwZQvx6uuv0baSbTsQT9TXKpbAZCynOQnGS
20+
3rzTeOTapwsj1yVlAuo7koxbjFFaz6b1nGC5Ap/rGeVdIT7ZVKF/
21+
-----END CERTIFICATE-----

‎environments/.stackhpc/hooks/pre.yml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,8 @@
44
tasks:
55
- name: Output OS version
66
command: cat /etc/redhat-release
7+
changed_when: false
8+
79
- name: Write CI-generated inventory and secrets for debugging
810
ansible.builtin.copy:
911
dest: /etc/ci-config/

‎environments/.stackhpc/inventory/extra_groups

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -37,4 +37,7 @@ builder
3737
builder
3838

3939
[rebuild:children]
40-
control
40+
control
41+
42+
[cacerts:children]
43+
cluster

‎environments/.stackhpc/tofu/main.tf

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -80,7 +80,7 @@ module "cluster" {
8080
standard: { # NB: can't call this default!
8181
nodes: ["compute-0", "compute-1"]
8282
flavor: var.other_node_flavor
83-
compute_init_enable: ["compute", "etc_hosts", "nfs", "basic_users", "eessi", "tuned"]
83+
compute_init_enable: ["compute", "etc_hosts", "nfs", "basic_users", "eessi", "tuned", "cacerts"]
8484
ignore_image_changes: true
8585
}
8686
# Example of how to add another partition:

‎environments/common/inventory/groups

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -165,3 +165,6 @@ extra_packages
165165

166166
[pulp]
167167
# Add builder to this group to enable automatically syncing of pulp during image build
168+
169+
[cacerts]
170+
# Hosts to configure CA certificates and trusts on

‎environments/common/layouts/everything

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -111,3 +111,6 @@ control
111111
[extra_packages:children]
112112
# Hosts to install specified additional packages on
113113
builder
114+
115+
[cacerts]
116+
# Hosts to configure CA certificates and trusts on

0 commit comments

Comments
 (0)