Secret rotation docs post-review changes

Author: Alex-Welsh

doc/source/operations/secret-rotation.rst

@@ -16,7 +16,7 @@ be automatically regenerated with a ``kayobe overcloud service deploy``.
 Some secrets require manual input from the operator to change.
 
 Following this process, there may be a few seconds of network downtime for
-running VMs when Neutron is reconfigured.
+running VMs when Neutron is reconfigured when using ML2/OVS.
 
 There will be API downtime for all services. The main reason for the outage is
 that RabbitMQ must be completely stopped to change the secrets it uses. The
@@ -77,8 +77,7 @@ Full method
    ``kolla_docker`` dict in ``ansible/roles/nova/tasks/bootstrap_service.yml`` See
    `here
    <https://github.com/stackhpc/kolla-ansible/pull/496/commits/9da473a63414493517da668075b8c958fec56e96>`__
-   for an example. (If you are using the latest ``stackhpc/yoga`` branch of
-   Kolla-Ansible this should already be set)
+   for an example.
 
    .. code::
 
@@ -144,9 +143,20 @@ Full method
     ``kayobe-config/etc/kayobe/ansible/`` if not, merge the latest
     ``stackhpc-kayobe-config``
 
-    .. code:: bash
+    1. Run the playbook to generate a new keypair and add it to the authorised
+       keys of your hosts.
+
+       .. code:: bash
+
+          kayobe playbook run $KAYOBE_CONFIG_PATH/ansible/rekey-hosts.yml
+
+    2. Ensure you can SSH to other nodes using the new keypair
+
+    3. Re-run the playbook with arguments to remove the old keypair.
+
+       .. code:: bash
 
-       kayobe playbook run $KAYOBE_CONFIG_PATH/ansible/rekey-hosts.yml
+          kayobe playbook run $KAYOBE_CONFIG_PATH/ansible/rekey-hosts.yml -t remove-key -e rekey_remove_existing_key=true
 
 10. Update the Pulp password
 
@@ -165,8 +175,7 @@ Full method
           kayobe seed service deploy -t seed-deploy-containers -kt none
 
        (note you may need to skip docker registry login since the password will
-       now be ‘incorrect’ e.g. ``-e``
-       ``deploy_containers_registry_attempt_login``)
+       now be ‘incorrect’ e.g. ``-e deploy_containers_registry_attempt_login=false``)
 
 11. Rotate ``horizon_secret_key``
 
@@ -197,20 +206,20 @@ Full method
 
           pwgen -s 40 1
 
-    2. Exec into the Grafana container on a controller
+    2. Update the value of ``grafana_admin_password`` in ``passwords.yml``
+
+    3. Exec into the Grafana container on a controller
 
        .. code:: bash
 
           sudo docker exec -it grafana bash
 
-    3. Run the password reset command, then enter the new password
+    4. Run the password reset command, then enter the new password
 
        .. code:: bash
 
           grafana-cli admin reset-admin-password --password-from-stdin
 
-    4. Update the value of ``grafana_admin_password`` in ``passwords.yml``
-
 13. Update the MariaDB database password
 
     1. Generate a new secret:
@@ -219,52 +228,51 @@ Full method
 
           pwgen -s 40 1
 
-    2. Exec into the MariaDB container on a controller
+    2. Update ``database_password`` in ``passwords.yml`` with your new
+       password. Make a note of the old password.
+
+    3. Exec into the MariaDB container on a controller
 
        .. code:: bash
 
           sudo docker exec -it mariadb bash
 
-    3. Log in to the database. You will be prompted for the password. Use the
-       existing value of ``database_password``
+    4. Log in to the database. You will be prompted for the password. Use the
+       old value of ``database_password``
 
        .. code:: bash
 
           mysql -uroot -p
 
-    4. Check the current state of the ``root`` user
+    5. Check the current state of the ``root`` user
 
        .. code:: bash
 
           SELECT Host,User,Password FROM mysql.user WHERE User='root';
 
-    5. Update the password for the ``root`` user
+    6. Update the password for the ``root`` user
 
        .. code:: bash
 
           SET PASSWORD FOR 'root'@'%' = PASSWORD('newpassword');
 
-    6. Check that the password hash has changed in the user list
+    7. Check that the password hash has changed in the user list
 
        .. code:: bash
 
           SELECT Host,User,Password FROM mysql.user WHERE User='root';
 
-    7. If there are any remaining root users with the old password e.g.
+    8. If there are any remaining root users with the old password e.g.
        ``root@localhost``, change the password for them too
 
-    8. Update ``database_password`` in ``passwords.yml`` with your new
-       password
-
-
 .. _nova-change:
 
 14. Update the Nova Database password
+
       .. warning::
 
          From this point onward, service may be disrupted
 
-
     #. Create a new ``nova_database_password`` and store it in
        ``passwords.yml``
 
@@ -296,51 +304,30 @@ Full method
        ``00000000-0000-0000-0000-000000000000``, change the above command
        accordingly)
 
-
 15.  Re-encrypt your ``passwords.yml`` file
 
+16. Stop all OpenStack services
+
+    .. code:: bash
+
+       kayobe playbook run $KAYOBE_CONFIG_PATH/ansible/stop-openstack-services.yml
 
 .. _k-a-change:
 
-16. Delete the service users in Keystone. The exact users will depend on the
+17. Delete the service users in Keystone. The exact users will depend on the
     deployment. Multinode example:
 
       .. note::
 
          Alternatively, cherry-pick
-         `this patch <https://review.opendev.org/c/openstack/kolla-ansible/+/903178>`__
+         `this patch<https://review.opendev.org/c/openstack/kolla-ansible/+/903178>`__
 
 
     .. code:: bash
 
       openstack user delete glance cinder placement nova neutron heat magnum magnum_trustee_domain_admin barbican designate
 
-17. Stop services using RabbitMQ
-
-    .. code:: bash
-
-       kayobe playbook run $KAYOBE_CONFIG_PATH/ansible/stop-openstack-services.yml
-
-18. Nuke RabbitMQ
-
-    .. code:: bash
-
-       kayobe overcloud host command run -l controllers --become --command "docker stop rabbitmq && docker rm rabbitmq && docker volume rm rabbitmq"
-
-19. Reconfigure Overcloud services to apply changes
-
-
-      .. warning::
-
-         VMs should continue running, but connections to them will briefly be
-         disrupted when Neutron is redeployed
-
-   .. code:: bash
-
-      kayobe overcloud service deploy
-
-
-20. Flush the Memcached data on all controllers (any old data will now be
+18. Flush the Memcached data on all controllers (any old data will now be
     inaccessible)
 
     #. Install Telnet (on one of the controllers)
@@ -367,6 +354,23 @@ Full method
           flush_all
           quit
 
+19. Nuke RabbitMQ
+
+    .. code:: bash
+
+       kayobe overcloud host command run -l controllers --become --command "docker stop rabbitmq && docker rm rabbitmq && docker volume rm rabbitmq"
+
+20. Reconfigure Overcloud services to apply changes
+
+      .. warning::
+
+         VMs should continue running, but connections to them will briefly be
+         disrupted when Neutron is redeployed when using ML2/OVS
+
+   .. code:: bash
+
+      kayobe overcloud service deploy
+
 21. Manually update ``heat_domain_admin_password``
 
     #. TODO: Instructions
@@ -381,7 +385,7 @@ Full method
        if individual user accounts are used
 
     2. Any existing ``openrc`` files generated by Kolla Ansible will need to be
-       re-generated or edited to use the new Kolla admin password
+       re-generated or edited to use the new Keystone admin password
 
 24. Create a PR to merge the new secrets into your main Kayobe configuration
     branch
@@ -505,6 +509,7 @@ Full password list
    docker_registry_password
    secrets_pulp_password
    redis_master_password
+   haproxy_password
    keystone_ssh_key
       private_key
       public_key

tox.ini

@@ -14,7 +14,7 @@ commands =
   yamllint etc/kayobe
   reno lint
   # secret-rotation must be skipped because it includes purposeful whitespace
-  doc8 README.rst doc/source --ignore D001 --ignore-path doc/source/operations/secret-rotation.rst 
+  doc8 README.rst doc/source --ignore D001 --ignore-path-errors doc/source/operations/secret-rotation.rst;D002
 # StackHPC Kayobe configuration release notes:
 [testenv:releasenotes]
 allowlist_externals = rm