Skip to content

Commit 03c8577

Browse files
committed
update image build docs
1 parent 5a00228 commit 03c8577

File tree

1 file changed

+18
-39
lines changed

1 file changed

+18
-39
lines changed

docs/image-build.md

Lines changed: 18 additions & 39 deletions
Original file line numberDiff line numberDiff line change
@@ -24,65 +24,44 @@ The steps for building site-specific fat images or extending an existing fat ima
2424
```hcl
2525
flavor = "general.v1.small" # VM flavor to use for builder VMs
2626
networks = ["26023e3d-bc8e-459c-8def-dbd47ab01756"] # List of network UUIDs to attach the VM to
27+
source_image_name = "Rocky-9-GenericCloud-Base-9.4" # Name of image to create VM with, i.e. starting image
28+
inventory_groups = "control,login,compute" # Comma-separated list of inventory groups to add build VM to, in addition to "builder" group
29+
2730
```
2831
Note that:
2932
- The network used for the Packer VM must provide outbound internet access but does not need to provide access to resources which the final cluster nodes require (e.g. Slurm control node, network filesystem servers etc.).
3033
- For additional options such as non-default private key locations or jumphost configuration see the variable descriptions in `./openstack.pkr.hcl`.
31-
- For an example of configuration for extending an existing fat image see below.
34+
- The `control,login,compute` inventory groups mean that the resultant image contains packages for all nodes in the cluster - this produces
35+
a site-specific fat image.
3236
3337
3. Activate the venv and the relevant environment.
3438
3539
4. Build images using the relevant variable definition file, e.g.:
3640
3741
cd packer/
38-
PACKER_LOG=1 /usr/bin/packer build -only=openstack.openhpc --on-error=ask -var-file=$PKR_VAR_environment_root/builder.pkrvars.hcl openstack.pkr.hcl
39-
40-
Note that the `-only` flag here restricts Packer to a single specific "build" definition (in Packer terminology). Options here are:
41-
- `-only=openstack.openhpc`: Build a fat image including Mellanox OFED
42-
- `-only=openstack.openhpc-cuda`: Build a fat image including Mellanox OFED, Nvidia drivers and CUDA
43-
- `-only=openstack.openhpc-extra`: Build an image which *extends* an existing fat image
44-
45-
5. The built image will be automatically uploaded to OpenStack with a name prefixed `openhpc-` and including a timestamp and a shortened git hash.
46-
47-
# Defining an "extra" image build
48-
49-
An "extra" image build starts with an existing fat image (e.g. one provided by StackHPC) rather than a RockyLinux GenericCloud image, and only runs a specific subset of the
50-
Ansible in the appliance. This allows adding additional functionality into site-specific images, without modifying the existing functionality in the base fat image. This is the recommended way to build site-specific images.
51-
52-
To configure an "extra" image build, prepare a Packer variable definition file as described above but also including:
53-
54-
- `extra_build_image_name`: A string to add into the final image name.
55-
- `source_image` or `source_image_name`: The UUID or name of the fat image to start from (which must already be present in OpenStack).
56-
- `extra_build_groups`: A list of Ansible inventory groups to put the build VM into, in addition to the `builder` group. This defines the roles/functionality
57-
which are added to the image.
58-
- `extra_build_volume_size`: A number giving the size in GB of the volume for the build VM's root disk and therefore the resulting image size.
59-
Note this assumes the default of `use_blockstorage_volume = true`.
60-
61-
E.g. to add the lustre client to an RockyLinux 9 image:
62-
63-
# environments/site/lustre.pkvars.hcl
42+
PACKER_LOG=1 /usr/bin/packer build -on-error=ask -var-file=$PKR_VAR_environment_root/builder.pkrvars.hcl openstack.pkr.hcl
6443
65-
extra_build_image_name = "lustre" # output image name will be like "openhpc-lustre-RL9-$timestamp-$commit"
66-
source_image_name = "openhpc-ofed-RL9-240906-1041-32568dbb" # e.g. current StackHPC RL9 image
67-
extra_build_groups = ["lustre"] # only run lustre role during this extra build
68-
extra_build_volume_size = 15 # default non-CUDA build image size has enough free space
44+
**NB:** If the build fails while creating the volume, check if the source image has the `signature_verified` property:
6945
70-
# ... define flavor, network, etc as normal
46+
openstack image show $SOURCE_IMAGE
7147
48+
If it does, remove this property:
7249
73-
Then, reference this build and variables file in the Packer build command:
50+
openstack image unset --property signature_verified $SOURCE_IMAGE
7451
75-
PACKER_LOG=1 /usr/bin/packer build -only=openstack.openhpc-extra --on-error=ask -var-file=environments/site/lustre.pkvars.hcl openstack.pkr.hcl
52+
then delete the failed volume, select cancelling the build when Packer queries, and then retry. This is [Openstack bug 1823445](https://bugs.launchpad.net/cinder/+bug/1823445).
7653
77-
**NB:** If the build fails while creating the volume, check if the source image has the `signature_verified` property:
7854
79-
openstack image show $SOURCE_IMAGE
55+
5. The built image will be automatically uploaded to OpenStack with a name prefixed `openhpc` and including a timestamp and a shortened git hash.
8056
81-
If it does, remove this property:
57+
# Extending an existing image
8258
83-
openstack image unset --property signature_verified $SOURCE_IMAGE
59+
Extending an existing images uses the same process as described above is followed, but the Packer variable definition file should:
60+
- Set `source_image_name` to be an existing fat image (e.g. one provided by StackHPC) rather than a RockyLinux GenericCloud image
61+
- Set `inventory_groups` should only include the additional functionality (= role name), e.g. `lustre`
62+
- Probably set the variable `image_name`, e.g. to `openhpc-lustre` to distinguish it from an existing fat image
8463
85-
then delete the failed volume, select cancelling the build when Packer queries, and then retry. This is [Openstack bug 1823445](https://bugs.launchpad.net/cinder/+bug/1823445).
64+
Setting the inventory groups in this way allows adding additional functionality into images from StackHPC, without modifying the existing packages in the image (which have been tested in CI). This is the recommended way to modify images to add site-specific functionality.
8665
8766
# Build Process
8867

0 commit comments

Comments
 (0)