Skip to content

Commit 57e54bb

Browse files
markgoddardmarkgoddard
committed
Promote multiple tags simultaneously based on Kayobe config
This modifies the container image promotion process to be similar to package repo promotion. The tags for each image are stored in files in StackHPC Kayobe configuration (one per base distro), and these are queried to determine which tag to promote for each image. This avoids the need to specify tags when promoting images, and will allow for automatic tag promotion after merges to StackHPC Kayobe Configuration.
1 parent 512b123 commit 57e54bb

File tree

5 files changed

+212
-3
lines changed

5 files changed

+212
-3
lines changed

‎.github/workflows/container-promote.yml

Lines changed: 60 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,60 @@
1+
---
2+
name: Promote container repositories
3+
on:
4+
workflow_dispatch:
5+
inputs:
6+
filter:
7+
description: Space-separated list of regular expressions matching images to promote
8+
type: string
9+
required: false
10+
default: ""
11+
kayobe_config_branch:
12+
required: false
13+
description: Branch of StackHPC Kayobe configuration to use
14+
default: stackhpc/yoga
15+
check-mode:
16+
description: Check mode
17+
type: boolean
18+
required: false
19+
default: false
20+
21+
env:
22+
ANSIBLE_FORCE_COLOR: True
23+
ANSIBLE_VAULT_PASSWORD_FILE: ${{ github.workspace }}/vault-pass
24+
jobs:
25+
container-promote:
26+
name: Promote container repositories
27+
runs-on: ubuntu-latest
28+
steps:
29+
- name: Checkout
30+
uses: actions/checkout@v3
31+
32+
- name: Setup Release Train & dependencies
33+
uses: ./.github/actions/setup
34+
with:
35+
vault-password: ${{ secrets.ANSIBLE_VAULT_PASSWORD }}
36+
vault-password-file: ${{ env.ANSIBLE_VAULT_PASSWORD_FILE }}
37+
38+
- name: Clone StackHPC Kayobe configuration repository
39+
uses: actions/checkout@v3
40+
with:
41+
repository: stackhpc/stackhpc-kayobe-config
42+
ref: refs/heads/${{ github.event.inputs.kayobe_config_branch }}
43+
path: stackhpc-kayobe-config
44+
45+
- name: Promote images from stackhpc-dev to stackhpc namespace in Ark
46+
run: |
47+
args=""
48+
if [[ $CHECK_MODE = true ]]; then
49+
args="$args --check --diff"
50+
fi
51+
source venv/bin/activate &&
52+
ansible-playbook -i ansible/inventory \
53+
ansible/dev-pulp-container-tag-query-kayobe.yml \
54+
ansible/dev-pulp-container-promote.yml \
55+
-e kolla_container_image_filter="'$FILTER'" \
56+
-e kayobe_config_repo_path=./stackhpc-kayobe-config/ \
57+
$args
58+
env:
59+
FILTER: ${{ github.event.inputs.filter }}
60+
CHECK_MODE: ${{ inputs.check-mode }}

‎ansible/dev-pulp-container-promote.yml

Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,50 @@
1+
---
2+
# This playbook promotes images in the development namespace (stackhpc-dev) to
3+
# the release namespace (stackhpc). This makes them available to clients.
4+
# Images with a tag defined by dev_pulp_repository_container_promotion_tags are
5+
# promoted. dev_pulp_repository_container_promotion_tags is a dict with the
6+
# following format:
7+
#
8+
# dev_pulp_repository_container_promotion_tags:
9+
# <image name>:
10+
# - <image tag 1>
11+
# - <image tag 2>
12+
13+
- name: Promote dev Pulp containers
14+
hosts: localhost
15+
gather_facts: false
16+
tasks:
17+
- name: Fail if container images to promote are not defined
18+
fail:
19+
msg: >
20+
The container images to promote must be specified via
21+
'dev_pulp_repository_container_promotion_tags'
22+
when: dev_pulp_repository_container_promotion_tags is not defined
23+
24+
- name: Display which tags are being promoted
25+
debug:
26+
var: "dev_pulp_repository_container_promotion_tags"
27+
28+
# Copy tags from stackhpc-dev to stackhpc repositories.
29+
- import_role:
30+
name: stackhpc.pulp.pulp_container_content
31+
vars:
32+
pulp_url: "{{ dev_pulp_url }}"
33+
pulp_username: "{{ dev_pulp_username }}"
34+
pulp_password: "{{ dev_pulp_password }}"
35+
pulp_container_content: >-
36+
{%- set contents = [] -%}
37+
{%- for image, tags in dev_pulp_repository_container_promotion_tags.items() -%}
38+
{%- set src_image_repo = "stackhpc-dev/" ~ image -%}
39+
{%- set dest_image_repo = "stackhpc/" ~ image -%}
40+
{%- set content = {
41+
"allow_missing": False,
42+
"src_repo": src_image_repo,
43+
"src_is_push": true,
44+
"repository": dest_image_repo,
45+
"tags": tags,
46+
} -%}
47+
{%- set _ = contents.append(content) -%}
48+
{%- endfor -%}
49+
{{ contents }}
50+
pulp_container_content_wait: false

‎ansible/dev-pulp-container-tag-query-kayobe.yml

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,32 @@
1+
---
2+
# This playbook queries the Pulp container image tags defined in a Kayobe
3+
# configuration repository (in etc/kayobe/kolla-image-tags.yml). It then sets
4+
# the 'dev_pulp_repository_container_promotion_tags' tag map variable based
5+
# upon those tags, which defines the set of container image tags that
6+
# will be promoted when the dev-pulp-container-promote.yml playbook is run.
7+
8+
- import_playbook: kayobe-container-tag-query.yml
9+
10+
- name: Set dev Pulp container image tags to promote from Kayobe config tags
11+
hosts: localhost
12+
gather_facts: true
13+
tasks:
14+
- name: Set a fact about container image tags to promote
15+
set_fact:
16+
dev_pulp_repository_container_promotion_tags: >-
17+
{{ kayobe_kolla_image_tags |
18+
dict2items |
19+
selectattr('key', 'in', kolla_container_images_filtered) |
20+
items2dict }}
21+
22+
- name: Display container promotion tags facts
23+
debug:
24+
var: dev_pulp_repository_container_promotion_tags
25+
26+
- name: Assert that tags variable is populated
27+
assert:
28+
that:
29+
- dev_pulp_repository_container_promotion_tags | length > 0
30+
msg: >-
31+
Distribution promotion tag variable
32+
'dev_pulp_repository_container_promotion_tags' is empty

‎ansible/kayobe-container-tag-query.yml

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
1+
---
2+
# This playbook queries the Pulp container image tags defined in a Kayobe
3+
# configuration repository (in etc/kayobe/kolla-image-tags.yml). It then sets
4+
# the 'kayobe_pulp_container_tags' tag map variable based upon those tags.
5+
#
6+
# The kayobe-config repository path should be specified via
7+
# kayobe_config_repo_path.
8+
9+
- name: Query container image tags for Kayobe
10+
hosts: localhost
11+
gather_facts: True
12+
vars:
13+
kayobe_config_repo_path: ""
14+
tasks:
15+
- name: Fail if Kayobe config repo path is not specified
16+
fail:
17+
msg: >
18+
Kayobe config git repository path must be specified via 'kayobe_config_repo_path'.
19+
when: not kayobe_config_repo_path
20+
21+
- name: Fail if Kayobe config repo path is not a directory
22+
fail:
23+
msg: >
24+
Kayobe config git repository path {{ kayobe_config_repo_path }} is not a directory.
25+
when: kayobe_config_repo_path is not directory
26+
27+
- name: List Kayobe Kolla image tags
28+
command:
29+
cmd: >-
30+
{{ kayobe_config_repo_path | realpath }}/tools/kolla-images.py list-tags
31+
chdir: "{{ kayobe_config_repo_path | realpath }}"
32+
register: list_tags
33+
check_mode: false
34+
changed_when: false
35+
36+
- name: Set a fact about Kayobe Kolla image tags
37+
set_fact:
38+
kayobe_kolla_image_tags: "{{ list_tags.stdout | from_yaml }}"
39+
40+
- name: Display Kayobe Kolla image tags
41+
debug:
42+
var: kayobe_kolla_image_tags

‎docs/usage/content-workflows.md

Lines changed: 28 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -237,17 +237,42 @@ Alternatively, to update the tag for a specific container, update `etc/kayobe/ko
237237
skydive_analyzer_tag: wallaby-20220811T091848
238238
```
239239

240-
## Promoting container images
240+
## Promoting container images (Zed release onwards)
241241

242242
!!! note
243243

244244
This should only be performed when container images are ready for release.
245245

246-
The [Promote container repositories](https://github.com/stackhpc/stackhpc-release-train/actions/workflows/container-promote-old.yml) workflow runs on demand.
246+
The [Promote container repositories](https://github.com/stackhpc/stackhpc-release-train/actions/workflows/container-promote.yml) workflow is triggered automatically when a change is merged to stackhpc-kayobe-config.
247+
It may also be run on demand.
248+
249+
It runs the following playbooks:
250+
251+
* `dev-pulp-container-tag-query-kayobe.yml`: Query the Pulp container image tags defined in a Kayobe configuration repository and set the tag map variable `dev_pulp_repository_container_promotion_tags` based upon those tags. A path to a Kayobe configuration repository must be specified via `kayobe_config_repo_path`.
252+
* `dev-pulp-container-promote.yml`: Promote a set of container images from `stackhpc-dev` to `stackhpc` namespace. The tags to be promoted are defined via `dev_pulp_repository_container_promotion_tags`.
253+
254+
Use GitHub Actions to run this workflow, or to run it manually:
255+
256+
```
257+
ansible-playbook -i ansible/inventory \
258+
ansible/dev-pulp-container-tag-query-kayobe.yml \
259+
ansible/dev-pulp-container-promote.yml \
260+
-e kayobe_config_repo_path=../stackhpc-kayobe-config/
261+
```
262+
263+
In this example, the Pulp container image tags defined in the `etc/kayobe/kolla-image-tags.yml` file in `../stackhpc-kayobe-config` repository (relative to the current working directory) will be promoted to releases.
264+
265+
## Promoting container images (Yoga release and earlier)
266+
267+
!!! note
268+
269+
This should only be performed when container images are ready for release.
270+
271+
The [Promote container repositories (old)](https://github.com/stackhpc/stackhpc-release-train/actions/workflows/container-promote-old.yml) workflow runs on demand.
247272
It should be run when container images need to be released, typically after a change to [update container image tags](#updating-container-image-tags-in-kayobe-configuration) has been approved.
248273
It runs the following playbook:
249274

250-
* `dev-pulp-container-promote.yml`: Promote a set of container images from `stackhpc-dev` to `stackhpc` namespace. The tag to be promoted is defined via `dev_pulp_repository_container_promotion_tag` which should be specified as an extra variable (`-e`).
275+
* `dev-pulp-container-promote-old.yml`: Promote a set of container images from `stackhpc-dev` to `stackhpc` namespace. The tag to be promoted is defined via `dev_pulp_repository_container_promotion_tag` which should be specified as an extra variable (`-e`).
251276

252277
Use GitHub Actions to run this workflow, or to run it manually:
253278

0 commit comments

Comments
 (0)