Skip to content

Commit 564bfbb

Browse files
markgoddardmarkgoddard
authored
Merge pull request #785 from stackhpc/2023.1-zed-merge
2023.1: zed merge
2 parents e4599b9 + c426d5d commit 564bfbb

17 files changed

+168
-32
lines changed

‎.github/workflows/stackhpc-promote.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@ name: Promote package repositories
33
on:
44
push:
55
branches:
6-
# NOTE(mgoddard): Reference only the current release branch here.
6+
# NOTE(upgrade): Reference only the current release branch here.
77
- stackhpc/2023.1
88
jobs:
99
promote:

‎doc/source/operations/octavia.rst

Lines changed: 52 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -38,3 +38,55 @@ when building new images.
3838

3939
To rollback an image update, simply delete the old image. The next newest image with
4040
a tag matching ``amp_image_tag`` will be selected.
41+
42+
Manually deleting broken load balancers
43+
=======================================
44+
45+
Sometimes, a load balancer will get stuck in a broken state of ``PENDING_CREATE`` or ``PENDING_UPDATE``.
46+
When in this state, the load balancer cannot be deleted; you will see the error ``Invalid state PENDING_CREATE of loadbalancer resource``.
47+
To delete a load balancer in this state, you will need to manually update its provisioning status in the database.
48+
49+
Find the database password:
50+
51+
.. code-block:: console
52+
53+
ansible-vault view --vault-password-file <path-to-vault-pw> $KOLLA_CONFIG_PATH/passwords.yml
54+
55+
# Search for database_password with:
56+
/^database
57+
58+
Access the database from a controller:
59+
60+
.. code-block:: console
61+
62+
docker exec -it mariadb bash
63+
mysql -u root -p octavia
64+
# Enter the database password when promted.
65+
66+
List the load balancers to find the ID of the broken one(s):
67+
68+
.. code-block:: console
69+
70+
SELECT * FROM load_balancer;
71+
72+
Set the provisioning status to ERROR for any broken load balancer:
73+
74+
.. code-block:: console
75+
76+
UPDATE load_balancer SET provisioning_status='ERROR' WHERE id='<id>';
77+
78+
Delete the load balancer from the OpenStack CLI, cascading if any stray
79+
Amphorae are hanging around:
80+
81+
.. code-block:: console
82+
83+
openstack loadbalancer delete <id> --cascade
84+
85+
86+
Sometimes, Amphora may also fail to delete if they are stuck in state
87+
``BOOTING``. These can be resolved entirely from the OpenStack CLI:
88+
89+
.. code-block:: console
90+
91+
openstack loadbalancer amphora configure <amphora-id>
92+
openstack loadbalancer amphora delete <amphora-id>

‎doc/source/operations/upgrading.rst

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -71,6 +71,31 @@ to::
7171

7272
ark.stackhpc.com/stackhpc/etcd:zed-rocky-9-20230821T155947
7373

74+
OVN enabled by default
75+
----------------------
76+
77+
OVN is now enabled by default in StackHPC Kayobe Configuration. This change
78+
was made to align with our standard deployment configuration.
79+
80+
There is currently not a tested migration path from OVS to OVN on a running
81+
system. If you are using a Neutron plugin other than ML2/OVN, set
82+
``kolla_enable_ovn`` to ``false`` in ``etc/kayobe/kolla.yml``.
83+
84+
For new deployments using OVN, see
85+
:kolla-ansible-doc:`reference/networking/neutron.html#ovn-ml2-ovn`.
86+
87+
Known issues
88+
============
89+
90+
* Rebuilds of servers with volumes are broken if there are any Nova compute
91+
services running an older release, including any that are down. Old compute
92+
services should be removed using ``openstack compute service delete``, then
93+
remaining compute services restarted. See `LP#2040264
94+
<https://bugs.launchpad.net/nova/+bug/2040264>`__.
95+
96+
* The OVN sync repair tool removes metadata ports, breaking OVN load balancers.
97+
See `LP#2038091 <https://bugs.launchpad.net/neutron/+bug/2038091>`__.
98+
7499
Security baseline
75100
=================
76101

‎etc/kayobe/ansible/growroot.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -77,7 +77,7 @@
7777
vars:
7878
pv: "{{ pvs.stdout | from_json }}"
7979
disk_tmp: "{{ pv.report[0].pv[0].pv_name[:-1] }}"
80-
disk: "{{ disk_tmp[:-1] if disk_tmp[-1] == 'p' else disk_tmp }}"
80+
disk: "{{ disk_tmp[:-1] if disk_tmp[-1] == 'p' and disk_tmp[:4] == 'nvme' else disk_tmp }}"
8181
part_num: "{{ pv.report[0].pv[0].pv_name[-1] }}"
8282
become: true
8383
failed_when: "growpart.rc != 0 and 'NOCHANGE' not in growpart.stdout"

‎etc/kayobe/cephadm.yml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,7 @@ cephadm_ceph_release: "quincy"
1212
cephadm_image: "{{ stackhpc_docker_registry if stackhpc_sync_ceph_images | bool else 'quay.io' }}/ceph/ceph:{{ cephadm_image_tag }}"
1313

1414
# Ceph container image tag.
15-
cephadm_image_tag: "v17.2.6"
15+
cephadm_image_tag: "v17.2.7"
1616

1717
# Ceph custom repo workaround for Ubuntu Jammy as there are no official ceph repos for jammy.
1818
cephadm_custom_repos: "{{ ansible_facts['distribution_release'] == 'jammy' }}"
@@ -92,7 +92,7 @@ cephadm_commands_post: "{{ cephadm_commands_post_default + cephadm_commands_post
9292
cephadm_commands_pre_default: []
9393
cephadm_commands_pre_extra: []
9494

95-
cephadm_commands_post_default: "{{ ['mgr module enable prometheus'] if kolla_enable_prometheus_ceph_mgr_exporter | bool else [] }}"
95+
cephadm_commands_post_default: "{{ ['mgr module enable prometheus'] if kolla_enable_prometheus_ceph_mgr_exporter | default(False) | bool else [] }}"
9696
cephadm_commands_post_extra: []
9797

9898
###############################################################################

‎etc/kayobe/inventory/group_vars/all/vault

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -10,13 +10,13 @@ seed_hashicorp_registry_password: "{{ stackhpc_docker_registry_password if stack
1010
seed_consul_docker_image: "{{ stackhpc_docker_registry ~ '/' if stackhpc_sync_hashicorp_images | bool else '' }}hashicorp/consul"
1111

1212
# Seed Consul container image tag.
13-
seed_consul_docker_tag: "1.16.1"
13+
seed_consul_docker_tag: "1.16.3"
1414

1515
# Seed Vault container image.
1616
seed_vault_docker_image: "{{ stackhpc_docker_registry ~ '/' if stackhpc_sync_hashicorp_images | bool else '' }}hashicorp/vault"
1717

1818
# Seed Vault container image tag.
19-
seed_vault_docker_tag: "1.14.1"
19+
seed_vault_docker_tag: "1.14.6"
2020

2121
# Seed Vault PKI Role name
2222
seed_vault_pki_role_name: "ServerCert"

‎etc/kayobe/kolla.yml

Lines changed: 23 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -106,12 +106,9 @@ kolla_tag: "{{ openstack_release }}-{{ kolla_base_distro }}-{{ kolla_base_distro
106106
# type: git
107107
# location: https://github.com/openstack/ironic
108108
# reference: master
109-
# NOTE (Alex-Welsh): The reference for many of these entries could be
110-
# 'stackhpc/{{ openstack_release }}' which would keep the branch up to date
111-
# for the current release. This is nice in theory but in practice, the stackhpc
112-
# forks change with every release and the elements in this list change with
113-
# them. Explicitly using /2021.3 makes it more intuitive to find and edit these
114-
# entries.
109+
# NOTE(upgrade): These sources should be checked with each release. StackHPC
110+
# branches are only required when we have custom backports. For a new release,
111+
# we may have caught up with upstream.
115112
kolla_sources:
116113
bifrost-base-additions-stackhpc-inspector-plugins:
117114
# Install our custom inspector plugins.
@@ -121,11 +118,11 @@ kolla_sources:
121118
cloudkitty-base:
122119
type: git
123120
location: https://github.com/stackhpc/cloudkitty.git
124-
reference: stackhpc/2023.1
121+
reference: stackhpc/{{ openstack_release }}
125122
horizon-plugin-cloudkitty-dashboard:
126123
type: git
127124
location: https://github.com/stackhpc/cloudkitty-dashboard.git
128-
reference: stackhpc/2023.1
125+
reference: stackhpc/{{ openstack_release }}
129126
ironic-inspector-additions-stackhpc-inspector-plugins:
130127
# Install our custom inspector plugins.
131128
type: git
@@ -135,11 +132,15 @@ kolla_sources:
135132
magnum-base:
136133
type: git
137134
location: https://github.com/stackhpc/magnum.git
138-
reference: stackhpc/2023.1
135+
reference: stackhpc/{{ openstack_release }}
136+
neutron-base:
137+
type: git
138+
location: https://github.com/stackhpc/neutron.git
139+
reference: stackhpc/{{ openstack_release }}
139140
neutron-base-plugin-networking-generic-switch:
140141
type: git
141142
location: https://github.com/stackhpc/networking-generic-switch.git
142-
reference: stackhpc/2023.1
143+
reference: stackhpc/{{ openstack_release }}
143144

144145
###############################################################################
145146
# Kolla image build configuration.
@@ -284,18 +285,22 @@ kolla_build_blocks:
284285
{% if stackhpc_kolla_clean_up_repo_mirrors | bool %}
285286
{% if kolla_base_distro == 'rocky' %}
286287
RUN \
287-
tar -xzf /etc/yum.repos.d.backup/repos.tar.gz -C /etc/yum.repos.d && \
288-
if grep -r '{{ stackhpc_repo_mirror_url }}' /etc/yum.repos.d; then \
289-
echo "Found repository mirror in Yum repositories"; \
290-
exit 1; \
288+
if [ -f /etc/yum.repos.d.backup/repos.tar.gz ]; then \
289+
tar -xzf /etc/yum.repos.d.backup/repos.tar.gz -C /etc/yum.repos.d && \
290+
rm -rf /etc/yum.repos.d.backup/; \
291291
fi && \
292-
rm -rf /etc/yum.repos.d.backup/
292+
if grep -r '{{ stackhpc_repo_mirror_url }}' /etc/yum.repos.d; then \
293+
echo "Found repository mirror in Yum repositories"; \
294+
exit 1; \
295+
fi
293296
{% else %}
294297
RUN \
295-
mv /etc/apt/sources.list.backup /etc/apt/sources.list && \
298+
if [ -f /etc/apt/sources.list.backup ]; then \
299+
mv /etc/apt/sources.list.backup /etc/apt/sources.list; \
300+
fi && \
296301
if grep -r '{{ stackhpc_repo_mirror_url }}' /etc/apt/sources.list; then \
297-
echo "Found repository mirror in APT repositories"; \
298-
exit 1; \
302+
echo "Found repository mirror in APT repositories"; \
303+
exit 1; \
299304
fi
300305
{% endif %}
301306
{% endif %}

‎etc/kayobe/kolla/globals.yml

Lines changed: 12 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -12,9 +12,17 @@ kayobe_image_tags:
1212
bifrost:
1313
rocky: 2023.1-rocky-9-20231013T151957
1414
ubuntu: 2023.1-ubuntu-jammy-20231013T151957
15+
cloudkitty:
16+
rocky: 2023.1-rocky-9-20231115T110235
17+
ubuntu: 2023.1-ubuntu-jammy-20231115T110235
18+
neutron:
19+
rocky: 2023.1-rocky-9-20231115T110249
20+
ubuntu: 2023.1-ubuntu-jammy-20231115T110249
1521

1622
openstack_tag: "{% raw %}{{ kayobe_image_tags['openstack'][kolla_base_distro] }}{% endraw %}"
1723
bifrost_tag: "{% raw %}{{ kayobe_image_tags['bifrost'][kolla_base_distro] }}{% endraw %}"
24+
cloudkitty_tag: "{% raw %}{{ kayobe_image_tags['cloudkitty'][kolla_base_distro] }}{% endraw %}"
25+
neutron_tag: "{% raw %}{{ kayobe_image_tags['neutron'][kolla_base_distro] }}{% endraw %}"
1826

1927
om_enable_rabbitmq_high_availability: true
2028

@@ -40,6 +48,7 @@ prometheus_ceph_mgr_exporter_endpoints:
4048
# Use inventory hostnames as labels
4149
prometheus_instance_label: "{% raw %}{{ ansible_facts.hostname }}{% endraw %}"
4250

43-
#############################################################################
44-
45-
neutron_ovn_distributed_fip: true
51+
# Make openstack-exporter use Nova API version 2.1 to keep metrics the same as
52+
# in Yoga. This is required to include a valid value for the flavor_id label on
53+
# openstack_nova_server_status metrics.
54+
prometheus_openstack_exporter_compute_api_version: "2.1"

‎releasenotes/notes/bump-ceph-images-f9d4721aa1d37729.yaml

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
---
2+
upgrade:
3+
- |
4+
Updates default Ceph images to v17.2.7 for Quincy.

‎releasenotes/notes/bump-hashicorp-images-7ef66e1328983bbe.yaml

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
---
2+
upgrade:
3+
- |
4+
Updates Consul to 1.16.3 and Vault to 1.14.6.

‎releasenotes/notes/cloudkitty-fix-es-to-os-migration-b0efd2626f59d977.yaml

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
---
2+
fixes:
3+
- |
4+
Fixes the bulk API of CloudKitty so that it now supports the migration
5+
from Elasticsearch to OpenSearch.

‎releasenotes/notes/enable_ovn-b84974f398242dcb.yaml

Lines changed: 7 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,10 @@
11
---
22
upgrade:
33
- |
4-
Enabled ML2/OVN by default as checks preventing accidental migration
5-
from ML2/OVS were added in kolla-ansible. If you are using a Neutron
6-
plugin other than ML2/OVN, set `kolla_enable_ovn` to `false`.
7-
OVN distributed FIP is also enabled, to disable it set
8-
`neutron_ovn_distributed_fip` to `false` in `etc/kayobe/kolla/globals.yml`.
4+
Enabled ML2/OVN by default. Checks preventing accidental migration
5+
from ML2/OVS were added in Kolla Ansible. If you are using a Neutron
6+
plugin other than ML2/OVN, set ``kolla_enable_ovn`` to ``false``.
7+
8+
OVN distributed FIP is disabled, to enable it set
9+
``neutron_ovn_distributed_fip`` to ``true`` in
10+
``etc/kayobe/kolla/globals.yml``.

‎releasenotes/notes/fix-growroot-playbook-6a8ee02d7c0fbcb3.yaml

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
---
2+
fixes:
3+
- |
4+
Fixes an issue with the growroot playbook where disks such as 'sdp' would
5+
become 'sd' due to the removal of the trailing 'p' when dealing with nvme
6+
devices.

‎releasenotes/notes/fix-lb-fip-issue-when-neutron-restarts-6aafd03d115a1ee4.yaml

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
---
2+
fixes:
3+
- |
4+
Fixes Neutron so that load balancer FIPs are not broken on Neutron restart.
5+
See `Neutron bug report
6+
<https://bugs.launchpad.net/neutron/+bug/2042938>`__.

‎releasenotes/notes/fix-regression-plugging-802-3ad-port-group-cd48d2e6f43b363f.yaml

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
---
2+
fixes:
3+
- |
4+
Fixes issue where Netmiko devices were sending no commands to the switch
5+
since plug_bond_to_network is overridden in
6+
networking_generic_switch/devices/netmiko_devices/init.py and
7+
PLUG_BOND_TO_NETWORK to set to None.
8+
See `NGS bug report
9+
<https://bugs.launchpad.net/networking-generic-switch/+bug/2041516>`__.

‎releasenotes/notes/neutron-containers-built-from-fork-e9f3c287b694fa8b.yaml

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
---
2+
features:
3+
- |
4+
Neutron containers are now built from our StackHPC fork.

‎releasenotes/notes/openstack-exporter-nova-api-fa5c2a9663bc97e2.yaml

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
---
2+
fixes:
3+
- |
4+
Restores valid value for the ``flavor_id`` label on
5+
``openstack_nova_server_status`` Prometheus metrics.

0 commit comments

Comments
 (0)