Skip to content

Support RabbitMQ minor version upgrades and Quorum migration #72

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 3 commits into from
Closed

Support RabbitMQ minor version upgrades and Quorum migration #72

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
51e604d
Add rabbit migration command to multinode.sh
Alex-Welsh Aug 22, 2024
1000d46
Merge remote-tracking branch 'origin/main' into rabbit-migration
Alex-Welsh Aug 28, 2024
947c5de
Do not skip rabbitmq releases during upgrade
Alex-Welsh Aug 28, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions ansible/deploy-openstack-config.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -89,6 +89,18 @@
vars:
kayobe_environment_path: "{{ src_directory }}/kayobe-config/etc/kayobe/environments/{{ kayobe_config_environment }}"

- name: Revert RabbitMQ changes to avoid git conflicts (upgrade)
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This stuff gets a bit fiddly, and brittle when considering support for multiple releases. Longer term it probably needs a different solution, like writing the config overrides to a specific file that can be deleted on upgrade.

ansible.builtin.shell:
cmd: >-
if grep "om_enable_rabbitmq_high_availability: false" {{ kayobe_environment_path }}/kolla/globals.yml ; then
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this do the right thing for:

  • yoga -> zed
  • zed -> antelope
  • antelope -> caracal (hopefully yes)
  • caracal -> epoxy

(bearing in mind SKC may change over time...)

sed -i -e 's/om_enable_rabbitmq_high_availability: false/om_enable_rabbitmq_high_availability: true/' \
-e 's/om_enable_rabbitmq_quorum_queues: true/om_enable_rabbitmq_quorum_queues: false/' \
{{ kayobe_environment_path }}/kolla/globals.yml
fi
when: upgrade | bool
vars:
kayobe_environment_path: "{{ src_directory }}/kayobe-config/etc/kayobe/environments/{{ kayobe_config_environment }}"

- name: Stash Kayobe Config changes (upgrade)
ansible.builtin.command:
cmd: git stash
Expand Down
24 changes: 23 additions & 1 deletion ansible/files/multinode.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -336,6 +336,27 @@ function upgrade_overcloud() {
kayobe overcloud service upgrade
}

function rabbit_migration() {
# Ensure RabbitMQ is upgraded to 3.13
if kayobe overcloud host command run -l controllers -b --command "docker exec rabbitmq rabbitmqctl --version | grep 3.11" --show-output; then
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: It shouldn't be an issue here, but you need to be careful with . in grep

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not an issue right now, but RabbitMQ versions are split into three: e.g. 3.13.7. there is a chance they'll go all the way up to 3.13.11, which would also get caught by this grep

kayobe kolla ansible run "rabbitmq-upgrade 3.12"
fi
if kayobe overcloud host command run -l controllers -b --command "docker exec rabbitmq rabbitmqctl --version | grep 3.12" --show-output; then
kayobe kolla ansible run "rabbitmq-upgrade 3.13"
fi

# Set quorum flag and execute RabbitMQ queue migration script.
if [[ -f $KAYOBE_CONFIG_PATH/../../tools/rabbitmq-quorum-migration.sh ]]; then
sed -i -e 's/om_enable_rabbitmq_high_availability: true/om_enable_rabbitmq_high_availability: false/' \
-e 's/om_enable_rabbitmq_quorum_queues: false/om_enable_rabbitmq_quorum_queues: true/' \
$KAYOBE_CONFIG_PATH/environments/$KAYOBE_ENVIRONMENT/kolla/globals.yml
$KAYOBE_CONFIG_PATH/../../tools/rabbitmq-quorum-migration.sh
else
echo "Migration script not found. Check Openstack Release"
return 1
fi
}

function usage() {
set +x

Expand All @@ -351,6 +372,7 @@ function usage() {
echo " build_kayobe_image"
echo " run_tempest"
echo " upgrade_overcloud"
echo " rabbit_migration"
}

function main() {
Expand All @@ -373,7 +395,7 @@ function main() {
$cmd
;;
# Standard commands.
(build_kayobe_image|deploy_full|deploy_seed|deploy_overcloud|deploy_wazuh|create_resources|run_tempest|upgrade_overcloud)
(build_kayobe_image|deploy_full|deploy_seed|deploy_overcloud|deploy_wazuh|create_resources|run_tempest|upgrade_overcloud|rabbit_migration)
setup
$cmd
report_success
Expand Down