stackhpc · markgoddard · Jul 5, 2024 · Jun 28, 2024 · Jul 1, 2024 · Jul 1, 2024
@@ -81,9 +81,9 @@ jobs:
       KAYOBE_VAULT_PASSWORD: ${{ secrets.KAYOBE_VAULT_PASSWORD }}
       KAYOBE_IMAGE: ${{ inputs.kayobe_image }}
       # NOTE(upgrade): Reference the PREVIOUS release here.
-      PREVIOUS_KAYOBE_IMAGE: ghcr.io/stackhpc/stackhpc-kayobe-config:stackhpc-zed
+      PREVIOUS_KAYOBE_IMAGE: ghcr.io/stackhpc/stackhpc-kayobe-config:stackhpc-2023.1
       # NOTE(upgrade): Reference the PREVIOUS release branch here.
-      PREVIOUS_BRANCH: stackhpc/zed
+      PREVIOUS_BRANCH: stackhpc/2023.1
     steps:
       - name: Install Package
         uses: ConorMacBride/install-package@main

@@ -8,7 +8,7 @@ on:
   push:
     branches:
       # NOTE(upgrade): Reference only the current release branch here.
-      - stackhpc/2023.1
+      - stackhpc/2024.1
 
   workflow_call:
     inputs:

@@ -4,7 +4,7 @@ on:
   push:
     branches:
       # NOTE(upgrade): Reference only the current release branch here.
-      - stackhpc/2023.1
+      - stackhpc/2024.1
 jobs:
   promote:
     name: Trigger Pulp promotion workflows

@@ -13,7 +13,7 @@ build:
     python: "3.7"
   jobs:
     post_checkout:
-      - git remote set-branches origin master stackhpc/2023.1 stackhpc/zed stackhpc/yoga stackhpc/xena stackhpc/wallaby
+      - git remote set-branches origin master stackhpc/2024.1 stackhpc/2023.1 stackhpc/zed stackhpc/yoga stackhpc/xena stackhpc/wallaby
       - git fetch --unshallow
 
 # Build documentation in the doc/source/ directory with Sphinx

@@ -29,8 +29,8 @@
 # -- StackHPC Kayobe configuration --------------------------------------
 # Variables to override
 
-current_series = "2023.1"
-previous_series = "zed"
+current_series = "2024.1"
+previous_series = "2023.1"
 branch = f"stackhpc/{current_series}"
 
 # Substitutions loader

@@ -9,7 +9,7 @@ This environment creates a Universe-from-nothing_-style deployment of Kayobe con
 .. warning::
 
     This guide was written for the Yoga release and has not been validated for
-    Antelope. Proceed with caution.
+    Caracal. Proceed with caution.
 
 Prerequisites
 =============

@@ -13,7 +13,7 @@ make changes to the setup process.
 .. warning::
 
     This guide was written for the Yoga release and has not been validated for
-    Antelope. Proceed with caution.
+    Caracal. Proceed with caution.
 
 Prerequisites
 =============
@@ -30,7 +30,7 @@ Download the setup script:
 
 .. parsed-literal::
 
-   wget https://raw.githubusercontent.com/stackhpc/stackhpc-kayobe-config/stackhpc/2023.1/etc/kayobe/environments/ci-aio/automated-setup.sh
+   wget https://raw.githubusercontent.com/stackhpc/stackhpc-kayobe-config/stackhpc/2024.1/etc/kayobe/environments/ci-aio/automated-setup.sh
 
 Change the permissions on the script:
 

@@ -9,7 +9,7 @@ service, and pushed there once built.
 .. warning::
 
     This guide was written for the Yoga release and has not been validated for
-    Antelope. Proceed with caution.
+    Caracal. Proceed with caution.
 
 
 Prerequisites

@@ -5,7 +5,7 @@ Multinode Test Environment
 .. warning::
 
     This guide was written for the Yoga release and has not been validated for
-    Antelope. Proceed with caution.
+    Caracal. Proceed with caution.
 
 
 Set up hosts

@@ -7,13 +7,13 @@ This section describes the Release Train process of creating new package reposit
 Preparations
 ============
 
-1. Before building images, you should check for any outstanding PRs into the earliest supported release. Below are the links for the 2023.1 (Antelope) branches.
+1. Before building images, you should check for any outstanding PRs into the earliest supported release. Below are the links for the 2024.1 (Caracal) branches.
 
- kayobe-config: https://github.com/stackhpc/stackhpc-kayobe-config/pulls?q=is%3Apr+is%3Aopen+base%3Astackhpc%2F2023.1
+ kayobe-config: https://github.com/stackhpc/stackhpc-kayobe-config/pulls?q=is%3Apr+is%3Aopen+base%3Astackhpc%2F2024.1
 
- kolla: https://github.com/stackhpc/kolla/pulls?q=is%3Apr+is%3Aopen+base%3Astackhpc%2F2023.1
+ kolla: https://github.com/stackhpc/kolla/pulls?q=is%3Apr+is%3Aopen+base%3Astackhpc%2F2024.1
 
- kolla-ansible: https://github.com/stackhpc/kolla-ansible/pulls?q=is%3Apr+is%3Aopen+base%3Astackhpc%2F2023.1
+ kolla-ansible: https://github.com/stackhpc/kolla-ansible/pulls?q=is%3Apr+is%3Aopen+base%3Astackhpc%2F2024.1
 
  You should also check any referenced source trees in etc/kayobe/kolla.yml.
 
@@ -152,7 +152,7 @@ Upgrading OpenStack to the next release in a multinode environment
 .. warning::
 
     This guide was written for the Wallaby release and has not been validated
-    for Antelope. Proceed with caution.
+    for Caracal. Proceed with caution.
 
 As this is not a full production system, only a reduced number of steps need to be followed to upgrade to a new release. Below describes these steps, with ``stackhpc/wallaby`` as the starting branch:
 

@@ -35,193 +35,70 @@ Notable changes in the |current_release| Release
 There are many changes in the OpenStack |current_release| release described in
 the release notes for each project. Here are some notable ones.
 
-Systemd container management
-----------------------------
-
-Containers deployed by Kolla Ansible are now managed by Systemd. Containers log
-to journald and have a unit file in ``/etc/systemd/system`` named
-``kolla-<container name>-container.service``. Manual control of containers
-should be performed using ``systemd start|stop|restart`` etc. rather than using
-the Docker CLI.
-
-Secure RBAC
------------
-
-Secure Role Based Access Control (RBAC) is an ongoing effort in OpenStack, and
-new policies have been evolving alongside the deprecated legacy policies.
-Several projects have changed the default value of the ``[oslo_policy]
-enforce_new_defaults`` configuration option to ``True``, meaning that the
-deprecated legacy policies are no longer applied. This results in more strict
-policies that may affect existing API users. The following projects have made
-this change:
-
-* Glance
-* Nova
-
-Some things to watch out for:
-
-* Policies may require the ``member`` role rather than the deprecated
-  ``_member_`` and ``Member`` roles.
-* Application credentials may need to be regenerated to grant any roles
-  required by the secure RBAC policies.
-* Application credentials generated before the existence of any implicit roles
-  will not be granted those roles. This may include the ``reader`` role, which
-  is referenced in some of the new secure RBAC policies. This issue has been
-  seen in app creds generated in the Yoga release. See `Keystone bug 2030061
-  <https://bugs.launchpad.net/keystone/+bug/2030061>`_.
-
-  While the Keystone docs suggest that the ``member`` role should imply the
-  ``reader`` role, it has been seen at a customer that newly-generated app
-  creds in the Antelope release may need both the ``member`` and ``reader``
-  role specified.
-
-  Here are some SQL scripts you can call to first see if any app creds are
-  affected, and then add the reader role where needed. It is recommended to
-  `backup the database
-  <https://docs.openstack.org/kayobe/latest/administration/overcloud.html#performing-database-backups>`__
-  before running these.
-
-  .. code-block:: sql
-
-     docker exec -it mariadb bash
-     mysql -u root -p  keystone
-     # Enter the database password when prompted.
-
-     SELECT application_credential.internal_id, role.id AS reader_role_id
-     FROM application_credential, role
-     WHERE role.name = 'reader'
-     AND NOT EXISTS (
-         SELECT 1
-         FROM application_credential_role
-         WHERE application_credential_role.application_credential_id = application_credential.internal_id
-         AND application_credential_role.role_id = role.id
-     );
-
-     INSERT INTO application_credential_role (application_credential_id, role_id)
-     SELECT application_credential.internal_id, role.id
-     FROM application_credential, role
-     WHERE role.name = 'reader'
-     AND NOT EXISTS (
-         SELECT 1
-         FROM application_credential_role
-         WHERE application_credential_role.application_credential_id = application_credential.internal_id
-         AND application_credential_role.role_id = role.id
-     );
-
-* If you have overwritten ``[auth] tempest_roles`` in your Tempest config, such
-  as to add the ``creator`` role for Barbican, you will need to also add the
-  ``member role``. eg:
-
-  .. code-block:: ini
-
-     [auth]
-     tempest_roles = creator,member
-* To check trusts for the _member_ role, you will need to list the role
-  assignments in the database, as only the trustor and trustee users can show
-  trust details from the CLI:
-
-  .. code-block:: console
-
-     openstack trust list
-     docker exec -it mariadb bash
-     mysql -u root -p  keystone
-     # Enter the database password when prompted.
-     SELECT * FROM trust_role WHERE trust_id = '<trust-id>' AND role_id = '<_member_-role-id>';
-* Policies may require the ``reader`` role rather than the non-standardised
-  ``observer`` role. The following error was observed in Horizon: ``Policy doesn’t allow os_compute_api:os-simple-tenant-usage:show to be performed``,
-  when the user only had the observer role in the project. It is best to keep the observer role until all projects have the ``enforce_new_defaults``
-  config option set. A one liner is shown below (or update your projects config):
-
-  .. code-block:: console
-
-     openstack role assignment list --effective --role observer -f value -c User -c Project | while read line; do echo $line | xargs bash -c 'openstack role add --user $1 --project $2 reader' _; done
-
-OVN enabled by default
-----------------------
-
-OVN is now enabled by default in StackHPC Kayobe Configuration.  This change
-was made to align with our standard deployment configuration.
-
-There is currently not a tested migration path from OVS to OVN on a running
-system. If you are using a Neutron plugin other than ML2/OVN, set
-``kolla_enable_ovn`` to ``false`` in ``etc/kayobe/kolla.yml``.
-
-For new deployments using OVN, see
-:kolla-ansible-doc:`reference/networking/neutron.html#ovn-ml2-ovn`.
-
-Kolla config merging
---------------------
-
-The Antelope release introduces Kolla config merging between Kayobe
-environments and base configurations. Before Antelope, any configuration under
-``$KAYOBE_CONFIG_PATH/kolla/config`` would be ignored when any Kayobe
-environment was activated.
-
-In Antelope, the Kolla configuration from the base will be merged with the
-environment. This can result in significant changes to the Kolla config. Take
-extra care when creating the Antelope branch of the kayobe-config and always
-check the config diff.
+Heat disabled by default
+------------------------
+
+The Heat OpenStack service is no longer enabled by default.
+
+This behavior can be overridden manually:
+
+.. code-block:: yaml
+   :caption: ``kolla.yml``
+
+   kolla_enable_heat: true
+
+Wherever possible, Magnum deployments should be migrated to the CAPI Helm
+driver. Instructions for enabling the driver can be found `here
+<../configuration/magnum-capi.rst>`_. Enable the driver, recreate any clusters
+using Heat, and disable the service.
+
+TODO: guide for disabling Heat
+
+Grafana Volume
+--------------
+The Grafana container volume is no longer used. If you wish to automatically
+remove the old volume, set ``grafana_remove_old_volume`` to ``true`` in
+``kolla/globals.yml``. Note that doing this will lose any plugins installed via
+the CLI directly and not through Kolla. If you have previously installed
+Grafana plugins via the Grafana UI or CLI, you must change to installing them
+at image build time. The Grafana volume, which contains existing custom
+plugins, will be automatically removed in the next release.
+
+Prometheus HAproxy Exporter
+---------------------------
+Due to the change from using the ``prometheus-haproxy-exporter`` to using the
+native support for Prometheus which is now built into HAProxy, metric names may
+have been replaced and/or removed, and in some cases the metric names may have
+remained the same but the labels may have changed. Alerts and dashboards may
+also need to be updated to use the new metrics. Please review any configuration
+that references the old metrics as this is not a backwards compatible change.
+
+Horizon configuration
+---------------------
+The Horizon role has been reworked to the preferred ``local_settings.d``
+configuration model. Files ``local_settings`` and ``custom_local_settings``
+have been renamed to ``_9998-kolla-settings.py`` and
+``_9999-custom-settings.py`` respectively. Users who use Horizon's custom
+configuration must change the names of those files in
+``etc/kolla/config/horizon`` as well.
 
 Known issues
 ============
 
-* Rebuilds of servers with volumes are broken if there are any Nova compute
-  services running an older release, including any that are down. Old compute
-  services should be removed using ``openstack compute service delete``, then
-  remaining compute services restarted. See `LP#2040264
-  <https://bugs.launchpad.net/nova/+bug/2040264>`__.
-
-* The OVN sync repair tool removes metadata ports, breaking OVN load balancers.
-  See `LP#2038091 <https://bugs.launchpad.net/neutron/+bug/2038091>`__.
-
-* When you try to generate config before the 2023.1 upgrade (i.e. using 2023.1
-  Kolla-Ansible but still running Zed kolla-toolbox), it will fail on Octavia.
-  This patch is needed to fix this:
-  https://review.opendev.org/c/openstack/kolla-ansible/+/905500
-
-* If you run ``kayobe overcloud service upgrade`` twice, it will cause shard
-  allocation to be disabled in OpenSearch. See `LP#2049512
-  <https://bugs.launchpad.net/kolla-ansible/+bug/2049512>`__ for details.
-
-  You can check if this is affecting your system with the following command. If
-  ``transient.cluster.routing.allocation.enable=none`` is present, shard
-  allocation is disabled.
-
-  .. code-block:: console
-
-     curl http://<controller-ip>:9200/_cluster/settings
-
-  For now, the easiest way to fix this is to turn allocation back on:
-
-  .. code-block:: console
-
-     curl -X PUT http://<controller-ip>:9200/_cluster/settings -H 'Content-Type:application/json' -d '{"transient":{"cluster":{"routing":{"allocation":{"enable":"all"}}}}}'
-
-* Docker log-opts are currently not configured in Antelope. You will see these
-  being removed when running a host configure in check+diff mode. See bug for
-  details (fix released):
-  https://bugs.launchpad.net/ansible-collection-kolla/+bug/2040105
-
-* /etc/hosts are not templated correctly when running a host configure with
-  ``--limit``. To work around this, run your host configures with
-  ``--skip-tags etc-hosts``. If you do need to change ``/etc/hosts``, for
-  example with any newly-added hosts, run a full host configure afterward with
-  ``--tags etc-hosts``. See bug for details (fix released):
-  https://bugs.launchpad.net/kayobe/+bug/2051714
+* None!
 
 Security baseline
 =================
 
-As part of the Zed and Antelope releases we are looking to improve the security
+As part of the Caracal release we are looking to improve the security
 baseline of StackHPC OpenStack deployments. If any of the following have not
-been done, they should ideally be completed before the upgrade begins,
-otherwise afterwards.
+been done, they should be completed before the upgrade begins.
 
 .. TODO: Add these when docs exist
 
    * Enable `host firewalling <TODO>`_
-   * Enable `Center for Internet Security (CIS) compliance <TODO>`_
 
+* Enable `Center for Internet Security (CIS) compliance <../configuration/security-hardening.rst>`_
 * Enable TLS on the :kayobe-doc:`public API network
   <configuration/reference/kolla-ansible.html#tls-encryption-of-apis>`
 * Enable TLS on the `internal API network <../configuration/vault.html>`_

@@ -1,6 +1,6 @@
 ====================================
-2023.1 Antelope Series Release Notes
+2024.1 Caracal Series Release Notes
 ====================================
 
 .. release-notes::
-   :branch: stackhpc/2023.1
+   :branch: stackhpc/2024.1
@@ -16,7 +16,7 @@ when used with Kayobe's :kayobe-doc:`multiple environments
 <multiple-environments>` feature.
 
 This configuration should be consumed using the `StackHPC Kayobe fork
-<https://github.com/stackhpc/kayobe/tree/stackhpc/2023.1>`__, which includes
+<https://github.com/stackhpc/kayobe/tree/stackhpc/2024.1>`__, which includes
 backported support for Ansible collections.
 
 New deployments

@@ -10,8 +10,8 @@
 set -eu
 
 BASE_PATH=~
-KAYOBE_BRANCH=stackhpc/2023.1
-KAYOBE_CONFIG_BRANCH=stackhpc/2023.1
+KAYOBE_BRANCH=stackhpc/2024.1
+KAYOBE_CONFIG_BRANCH=stackhpc/2024.1
 KAYOBE_ENVIRONMENT=aufn-ceph
 
 PELICAN_HOST="10.0.0.34 pelican pelican.service.compute.sms-lab.cloud"