Deploy rebuild machinery (#44)

* deploy rebuild machinery

* add rebuild group

* fix groups

* mock openstack command

* add debug flags temporarily

* export new PATH

* remove rebuild group from everything layout - configure in generated inventory instead

Author: sjpb

README.md

@@ -121,7 +121,7 @@ Although most of the inventory uses the group convention described above there a
     - An inventory group `<cluster_name>_<partition_name>` defining the hosts it contains - these must be homogenous w.r.t CPU and memory.
     - An entry in the `openhpc_slurm_partitions` mapping in `environments/<environment>/inventory/group_vars/openhpc/overrides.yml`.
     See the [openhpc role documentation](https://github.com/stackhpc/ansible-role-openhpc#slurmconf) for more options.
-
+- On an OpenStack cloud, rebuilding/reimaging compute nodes from Slurm can be enabled by defining a `rebuild` group containing the relevant compute hosts (e.g. in the generated `hosts` file).
 
 ## Adding new functionality
 TODO: this is just rough notes:

ansible/slurm.yml

@@ -18,6 +18,15 @@
     - import_role:
         name: stackhpc.openhpc
 
+- name: Setup slurm-driven reimage
+  hosts: rebuild
+  become: yes
+  tags:
+    - rebuild
+  tasks:
+    - import_role:
+        name: stackhpc.slurm_openstack_tools.rebuild
+
 - name: Set locked memory limits on user-facing nodes
   hosts:
     - compute

ansible/validate.yml

@@ -27,4 +27,14 @@
     - import_role:
         name: opendistro
         tasks_from: validate.yml
-      tags: validate
+      tags: validate
+
+- name: Validate rebuild configuration
+  hosts: rebuild
+  gather_facts: false
+  tags: rebuild
+  tasks:
+    - import_role:
+        name: stackhpc.slurm_openstack_tools.rebuild
+        tasks_from: validate.yml
+      tags: validate

dev/vagrant-example-configure.sh

@@ -20,4 +20,4 @@ ansible-playbook ansible/adhoc/generate-passwords.yml
 
 echo "Running site.yml"
 
-ansible-playbook ansible/site.yml
+ansible-playbook -vvvv ansible/site.yml -e "openhpc_rebuild_clouds=/tmp/vagrant-example/openstack"

environments/common/inventory/groups

@@ -66,3 +66,6 @@ cluster # TODO: FIXME in branch
 [selinux:children]
 # Define selinux status for these
 cluster
+
+[rebuild]
+# nodes in 'compute' group which can be rebuilt from slurm (on an OpenStack cloud)

environments/common/layouts/everything

@@ -35,4 +35,4 @@ control
 control
 
 [filebeat:children]
-slurm_stats
+slurm_stats