Merge branch 'stackhpc/yoga' into image_cloud_init_c9s

Author: mnasiadka

doc/source/configuration/monitoring.rst

@@ -136,3 +136,39 @@ mgrs group and list them as the endpoints for prometheus. Additionally,
 depending on your configuration, you may need set the
 ``kolla_enable_prometheus_ceph_mgr_exporter`` variable to ``true`` in order to
 enable the ceph mgr exporter.
+
+OpenStack Capacity
+==================
+
+OpenStack Capacity allows you to see how much space you have avaliable
+in your cloud. StackHPC Kayobe Config includes this exporter by default
+and it's necessary that some variables are set to allow deployment.
+
+To successfully deploy OpenStack Capacity, you are required to specify
+the OpenStack application credentials in ``kayobe/secrets.yml`` as:
+
+.. code-block:: yaml
+
+    secrets_os_exporter_auth_url: <some_auth_url>
+    secrets_os_exporter_credential_id: <some_credential_id>
+    secrets_os_exporter_credential_secret: <some_credential_secret>
+
+After defining your credentials, You may deploy OpenStack Capacity
+using the ``ansible/deploy-os-capacity-exporter.yml`` Ansible playbook
+via Kayobe.
+
+.. code-block:: console
+
+    kayobe playbook run ansible/deploy-os-capacity-exporter.yml
+
+It is required that you re-configure the Prometheus, Grafana and HAProxy
+services following deployment, to do this run the following Kayobe command.
+
+.. code-block:: console
+
+    kayobe overcloud service reconfigure -kt grafana,prometheus,haproxy
+
+If you notice ``HaproxyServerDown`` or ``HaproxyBackendDown`` prometheus
+alerts after deployment it's likely the os_exporter secrets have not been
+set correctly, double check you have entered the correct authentication
+information appropiate to your cloud and re-deploy.

doc/source/configuration/release-train.rst

@@ -192,6 +192,16 @@ promoted to production:
 
    kayobe playbook run $KAYOBE_CONFIG_PATH/ansible/pulp-repo-promote-production.yml
 
+Synchronising all Kolla container images can take a long time. A limited list
+of images can be synchronised using the ``stackhpc_pulp_images_kolla_filter``
+variable, which accepts a whitespace-separated list of regular expressions
+matching Kolla image names. Usage is similar to ``kolla-build`` CLI arguments.
+For example:
+
+.. code-block:: console
+
+   kayobe playbook run $KAYOBE_CONFIG_PATH/ansible/pulp-container-sync.yml -e stackhpc_pulp_images_kolla_filter='"^glance nova-compute$"'
+
 Initial seed deployment
 -----------------------
 

doc/source/configuration/wazuh.rst

@@ -17,8 +17,8 @@ The short version
 #. Deploy the Wazuh agents: ``kayobe playbook run $KAYOBE_CONFIG_PATH/ansible/wazuh-agent.yml``
 
 
-Wazuh Manager
-=============
+Wazuh Manager Host
+==================
 
 Provision using infra-vms
 -------------------------
@@ -57,7 +57,9 @@ Define VM sizing in ``etc/kayobe/inventory/group_vars/wazuh-manager/infra-vms``:
   infra_vm_data_capacity: "200G"
 
 
-Optional: define LVM volumes ``etc/kayobe/inventory/group_vars/wazuh-manager/lvm``:
+Optional: define LVM volumes in ``etc/kayobe/inventory/group_vars/wazuh-manager/lvm``.
+``/var/ossec`` often requires greater storage space, and ``/var/lib/wazuh-indexer``
+may be beneficial too.
 
 .. code-block:: console
 
@@ -73,7 +75,7 @@ Optional: define LVM volumes ``etc/kayobe/inventory/group_vars/wazuh-manager/lvm
           size: "100%VG"
           filesystem: "ext4"
           mount: true
-          mntp: “/var/lib/elasticsearch”
+          mntp: "/var/ossec"
           create: true
 
 
@@ -249,7 +251,7 @@ It will be used by wazuh secrets playbook to generate wazuh secrets vault file.
 .. code-block:: console
 
   kayobe playbook run $KAYOBE_CONFIG_PATH/ansible/wazuh-secrets.yml
-  ansible-vault encrypt --vault-password-file ~/vault.pass $KAYOBE_CONFIG_PATH/inventory/group_vars/wazuh/wazuh-manager/wazuh-secrets
+  ansible-vault encrypt --vault-password-file ~/vault.pass $KAYOBE_CONFIG_PATH/wazuh-secrets.yml
 
 
 TLS (optional)
@@ -288,6 +290,21 @@ Example OpenSSL rune to convert to PKCS#8:
 
 TODO: document how to use a local certificate. Do we need to override all certificates?
 
+Custom SCA Policies (optional)
+------------------------------
+
+Wazuh ships with a large selection of Security Configuration Assessment
+rulesets. However, you may find you want to add more. This can be achieved via
+`custom policies <https://documentation.wazuh.com/current/user-manual/capabilities/sec-config-assessment/how-to-configure.html>`_.
+
+SKC supports this automatically, just add the policy file from this PR to
+``{{ kayobe_env_config_path }}/wazuh/custom_sca_policies``.
+
+Currently, Wazuh does not ship with a CIS benchmark for Rocky 9. You can find
+the in-development policy here: https://github.com/wazuh/wazuh/pull/17810 To
+include this in your deployment, simply copy it to
+``{{ kayobe_env_config_path }}/wazuh/custom_sca_policies/cis_rocky_linux_9.yml``.
+
 Deploy
 ------
 
@@ -303,7 +320,7 @@ Encrypt the keys (and remember to commit to git):
 ``ansible-vault encrypt --vault-password-file ~/vault.pass $KAYOBE_CONFIG_PATH/ansible/wazuh/certificates/certs/*.key``
 
 Verification
-==============
+------------
 
 The Wazuh portal should be accessible on port 443 of the Wazuh
 manager’s IPs (using HTTPS, with the root CA cert in ``etc/kayobe/ansible/wazuh/certificates/wazuh-certificates/root-ca.pem``).
@@ -315,11 +332,9 @@ Troubleshooting
 
 Logs are in ``/var/log/wazuh-indexer/wazuh.log``. There are also logs in the journal.
 
-============
 Wazuh agents
 ============
 
-
 Wazuh agent playbook is located in ``etc/kayobe/ansible/wazuh-agent.yml``.
 
 Wazuh agent variables file is located in ``etc/kayobe/inventory/group_vars/wazuh-agent/wazuh-agent``.
@@ -333,13 +348,13 @@ Deploy the Wazuh agents:
 ``kayobe playbook run $KAYOBE_CONFIG_PATH/ansible/wazuh-agent.yml``
 
 Verification
-=============
+------------
 
 The Wazuh agents should register with the Wazuh manager. This can be verified via the agents page in Wazuh Portal.
 Check CIS benchmark output in agent section.
 
-Additional resources:
-=====================
+Additional resources
+--------------------
 
 For times when you need to upgrade wazuh with elasticsearch to version with opensearch or you just need to deinstall all wazuh components:
 Wazuh purge script: https://github.com/stackhpc/wazuh-server-purge

etc/kayobe/ansible/deploy-os-capacity-exporter.yml

@@ -0,0 +1,29 @@
+---
+- hosts: monitoring
+  gather_facts: false
+
+  tasks:
+    - name: Create os-capacity directory
+      ansible.builtin.file:
+        path: /opt/kayobe/os-capacity/
+        state: directory
+
+    - name: Template clouds.yml
+      ansible.builtin.template:
+        src: templates/os_capacity-clouds.yml.j2
+        dest: /opt/kayobe/os-capacity/clouds.yaml
+
+    - name: Ensure os_capacity container is running
+      docker_container:
+        name: os_capacity
+        image: ghcr.io/stackhpc/os-capacity:master
+        env:
+          OS_CLOUD: openstack
+          OS_CLIENT_CONFIG_FILE: /etc/openstack/clouds.yaml
+        mounts:
+          - type: bind
+            source: /opt/kayobe/os-capacity/
+            target: /etc/openstack/
+        network_mode: host
+        restart_policy: unless-stopped
+      become: true

etc/kayobe/ansible/ovn-fix-chassis-priorities.yml

@@ -0,0 +1,69 @@
+---
+# Sometimes, typically after restarting OVN services, the priorities of entries
+# in the ha_chassis and gateway_chassis tables in the OVN northbound database
+# can become misaligned. This results in broken routing for external (bare
+# metal/SR-IOV) ports.
+
+# This playbook can be used to fix the issue by realigning the priorities of
+# the table entries. It does so by assigning the highest priority to the
+# "first" (sorted alphabetically) OVN NB DB host. This results in all gateways
+# being scheduled to a single host, but is less complicated than trying to
+# balance them (and it's also not clear to me how to map between individual
+# ha_chassis and gateway_chassis entries).
+
+# The playbook can be run as follows:
+# kayobe playbook run $KAYOBE_CONFIG_PATH/ansible/ovn-fix-chassis-priorities.yml
+
+# If the 'controllers' group does not align with the group used to deploy the
+# OVN NB DB, this can be overridden by passing the following:
+# '-e ovn_nb_db_group=some_other_group'
+
+- name: Find OVN DB DB Leader
+  hosts: "{{ ovn_nb_db_group | default('controllers') }}"
+  tasks:
+    - name: Find the OVN NB DB leader
+      command: docker exec -it ovn_nb_db ovn-nbctl get-connection
+      changed_when: false
+      failed_when: false
+      register: ovn_check_result
+      check_mode: no
+
+    - name: Group hosts by leader/follower role
+      group_by:
+        key: "ovn_nb_{{ 'leader' if ovn_check_result.rc == 0 else 'follower' }}"
+      changed_when: false
+
+    - name: Assert one leader exists
+      assert:
+        that:
+          - groups['ovn_nb_leader'] | default([]) | length == 1
+
+- name: Fix OVN chassis priorities
+  hosts: ovn_nb_leader
+  vars:
+    ovn_nb_db_group: controllers
+    ovn_nb_db_hosts_sorted: "{{ query('inventory_hostnames', ovn_nb_db_group) | sort | list }}"
+    ha_chassis_max_priority: 32767
+    gateway_chassis_max_priority: "{{ ovn_nb_db_hosts_sorted | length }}"
+  tasks:
+    - name: Fix ha_chassis priorities
+      command: >-
+        docker exec -it ovn_nb_db
+        bash -c '
+        ovn-nbctl find ha_chassis chassis_name={{ item }} |
+        awk '\''$1 == "_uuid" { print $3 }'\'' |
+        while read uuid; do ovn-nbctl set ha_chassis $uuid priority={{ priority }}; done'
+      loop: "{{ ovn_nb_db_hosts_sorted }}"
+      vars:
+        priority: "{{ ha_chassis_max_priority | int - ovn_nb_db_hosts_sorted.index(item) }}"
+
+    - name: Fix gateway_chassis priorities
+      command: >-
+        docker exec -it ovn_nb_db
+        bash -c '
+        ovn-nbctl find gateway_chassis chassis_name={{ item }} |
+        awk '\''$1 == "_uuid" { print $3 }'\'' |
+        while read uuid; do ovn-nbctl set gateway_chassis $uuid priority={{ priority }}; done'
+      loop: "{{ ovn_nb_db_hosts_sorted }}"
+      vars:
+        priority: "{{ gateway_chassis_max_priority | int - ovn_nb_db_hosts_sorted.index(item) }}"

etc/kayobe/ansible/templates/os_capacity-clouds.yml.j2

@@ -0,0 +1,10 @@
+clouds:
+  openstack:
+    auth:
+      auth_url: "{{ secrets_os_exporter_auth_url }}"
+      application_credential_id: "{{ secrets_os_exporter_credential_id }}"
+      application_credential_secret: "{{ secrets_os_exporter_credential_secret }}"
+    region_name: "RegionOne"
+    interface: "internal"
+    identity_api_version: 3
+    auth_type: "v3applicationcredential"

etc/kayobe/ansible/wazuh-manager.yml

@@ -17,6 +17,63 @@
     - role: "{{ playbook_dir }}/roles/wazuh-ansible/wazuh-ansible/roles/wazuh/ansible-filebeat-oss"
     - role: "{{ playbook_dir }}/roles/wazuh-ansible/wazuh-ansible/roles/wazuh/wazuh-dashboard"
   post_tasks:
+    - block:
+        - name: Check if custom SCA policies directory exists
+          stat:
+            path: "{{ local_custom_sca_policies_path }}"
+          register: custom_sca_policies_folder
+          delegate_to: localhost
+          become: no
+
+        - name: Gather list of custom SCA policies
+          find:
+            paths: "{{ local_custom_sca_policies_path }}"
+            patterns: '*.yml'
+          delegate_to: localhost
+          register: custom_sca_policies
+          when: custom_sca_policies_folder.stat.exists
+
+        - name: Allow Wazuh agents to execute commands in SCA policies sent from the Wazuh manager
+          blockinfile:
+            path: "/var/ossec/etc/local_internal_options.conf"
+            state: present
+            owner: wazuh
+            group: wazuh
+            block: |
+              sca.remote_commands=1
+          when: custom_sca_policies.files | length > 0
+
+        - name: Copy custom SCA policy files to Wazuh manager
+          copy:
+            # Note the trailing slash to copy directory contents
+            src: "{{ local_custom_sca_policies_path }}/"
+            dest: "/var/ossec/etc/shared/default/"
+            owner: wazuh
+            group: wazuh
+          when: custom_sca_policies.files | length > 0
+
+        - name: Add custom policy definition(s) to the shared Agent config
+          blockinfile:
+            path: "/var/ossec/etc/shared/default/agent.conf"
+            state: present
+            owner: wazuh
+            group: wazuh
+            marker: "{mark} ANSIBLE MANAGED BLOCK Custom SCA Policies"
+            insertafter: "<!-- Shared agent configuration here -->"
+            block: |
+              {% filter indent(width=2, first=true) %}
+              <sca>
+                <policies>
+              {% for item in custom_sca_policies.files %}
+                  <policy>etc/shared/{{ item.path | basename }}</policy>
+              {% endfor %}
+                </policies>
+              </sca>
+              {% endfilter %}
+          when: custom_sca_policies.files | length > 0
+      notify:
+        - Restart wazuh
+
     - name: Set http/s_proxy vars in ossec-init.conf for vulnerability detector
       blockinfile:
         path: "/var/ossec/etc/ossec.conf"

etc/kayobe/dnf.yml

@@ -112,19 +112,19 @@ dnf_custom_repos_rocky:
   appstream:
     baseurl: "{{ stackhpc_repo_rocky_appstream_url }}"
     description: "Rocky Linux $releasever - AppStream"
-    file: Rocky-AppStream
+    file: "{{ 'Rocky-AppStream' if os_release == '8' else 'rocky' }}"
     gpgkey: file:///etc/pki/rpm-gpg/RPM-GPG-KEY-rockyofficial
     gpgcheck: yes
   baseos:
     baseurl: "{{ stackhpc_repo_rocky_baseos_url }}"
     description: "Rocky Linux $releasever - BaseOS"
-    file: Rocky-BaseOS
+    file: "{{ 'Rocky-BaseOS' if os_release == '8' else 'rocky' }}"
     gpgkey: file:///etc/pki/rpm-gpg/RPM-GPG-KEY-rockyofficial
     gpgcheck: yes
   extras:
     baseurl: "{{ stackhpc_repo_rocky_extras_url }}"
     description: "Rocky Linux $releasever - Extras"
-    file: Rocky-Extras
+    file: "{{ 'Rocky-Extras' if os_release == '8' else 'rocky-extras' }}"
     gpgkey: file:///etc/pki/rpm-gpg/RPM-GPG-KEY-rockyofficial
     gpgcheck: yes
 

etc/kayobe/environments/ci-aio/automated-setup.sh

@@ -6,7 +6,7 @@ cat << EOF | sudo tee -a /etc/hosts
 10.205.3.187 pulp-server pulp-server.internal.sms-cloud
 EOF
 
-if [ sudo vgdisplay | grep -q lvm2 ]; then
+if sudo vgdisplay | grep -q lvm2; then
    sudo lvextend -L 4G /dev/rootvg/lv_home -r || true
    sudo lvextend -L 4G /dev/rootvg/lv_tmp -r || true
 fi

etc/kayobe/inventory/group_vars/wazuh-manager/wazuh-manager

@@ -24,6 +24,9 @@ local_certs_path: "{{ playbook_dir }}/wazuh/certificates"
 # Ansible control host custom certificates directory
 local_custom_certs_path: "{{ playbook_dir }}/wazuh/custom_certificates"
 
+# Ansible custom SCA policies directory
+local_custom_sca_policies_path: "{{ kayobe_env_config_path }}/wazuh/custom_sca_policies"
+
 # Indexer variables
 indexer_node_name: "{{ inventory_hostname }}"