elemental

Image Customization

This section provides an overview of how the elemental3 command-line interface enables users to customize and extend an image that is based on a specific set of components defined in a release manifest.

For general information about the customization process, refer to the Customization Process section.

For information on booting images in air-gapped environments, refer to the Air-gapped images section.

For information on how to boot a customized image, refer to the Booting a customized image section.

For real-life examples, refer to the deployment scenario examples in the Cookbook documentation.

Customization Process

The customization process is executed through the elemental3 customize command.

As part of the command, users are expected to provide:

  1. Specifications for the image - these are defined as options to the command itself. Available specification options can be viewed by calling the command’s help message: elemental3 customize -h.
  2. Information about the desired state of the image - this is done through the definition of a configuration directory that the user creates beforehand. For more information on the directory itself, refer to the Configuration Directory Guide.

Please familiarize yourself with both points before attempting to customize your first image.

For further understanding of the customization process, you can also refer to the following sections:

Limitations

Currently, the image customization process has the following limitations:

  1. Supports customizing images only for x86_64 platforms.

Elemental is in active development, and these limitations will be addressed as part of the product roadmap.

Overview

NOTE: The user is able to customize Linux-only images by excluding Kubernetes resources and deployments from the configuration directory (regardless of whether this is under kubernetes/manifests, kubernetes/cluster.yaml or release.yaml). This is currently an implicit process, but it is possible that an explicit option for it (e.g. a flag) is added at a later stage.

This section provides a high-level overview of the steps that Elemental’s tooling goes through in order to produce a customized and extended image.

Steps:

  1. Parse the user provided image specifications.
  2. Parse the configuration directory that the user has defined in the image specification.
  3. Parse the solution release manifest that the user has defined as a release reference in the release.yaml file of the configuration directory.
  4. Pull and parse the core platform release manifest that the aforementioned solution manifest extends.
  5. Begin the customization process:
    1. Unpack the pre-built installer ISO that is defined in the core platform release manifest.
    2. Prepare for Kubernetes cluster creation and resource deployment:
      1. Prepare Helm charts and Kubernetes manifests
      2. Download RKE2 artifacts (tarball, checksums) and install script.
      3. For air-gapped images, embed the Helm charts, the RKE2 container images, and the container images used by the workloads.
    3. Prepare any other overlays or firstboot configurations based on what was defined in the configuration directory.
    4. Produce a description of the desired installation state and merge it with the installer ISO description.
    5. Produce the final desired image type.
  6. Mark customization as completed.

image

Execution

Starting the customization process can be done either by directly working with the elemental3 binary, or by using the elemental3 container image. Below you can find the minimum set of options for running both use cases.

Binary

sudo elemental3 customize --type <raw/qcow2/iso> --config-dir <path>

IMPORTANT: The above process is long running, as it involves pulling multiple component images over the network. Downloaded artifacts are cached under <config-dir>/cache and are reused for future runs.

Unless configured otherwise, after execution, the resulting ready-to-boot image will reside in the configuration directory path and use the image-<timestamp>.<image-type> naming format.

NOTE: You can specify another path for the output using the --output (-o) option, however, be mindful if running Elemental 3 from a container, as it would require including the mounted configuration directory as a prefix (e.g. –output /config/).

Container image

NOTE: This section assumes you have pulled the elemental3 container image and referenced it in the ELEMENTAL_IMAGE variable.

Run the customization process:

IMPORTANT: The above process is long running, as it involves pulling multiple component images over the network. For a quicker execution of subsequent runs, mount a directory from the host at /cache:

  podman run -it -v <PATH_TO_CONFIG_DIR>:/config -v <PATH_TO_CACHE_DIR>:/cache $ELEMENTAL_IMAGE customize --type <raw/iso>

Unless configured otherwise, the above process will produce a customized RAW, qcow2 or ISO image under the specified <PATH_TO_CONFIG_DIR> directory.

qcow2 images

The qcow2 image type produces the same disk layout as the raw type, but stored in a sparse qcow2 file. The virtual disk size is set by raw.diskSize in install.yaml, while the file only holds the data actually written to the image. The operating system partitions are created and grown to fill the virtual disk on first boot, so the image file stays at its minimum size.

Per-node overlays

Multi-node deployments boot every node from the same image (each node picks its configuration at boot, e.g. network settings matched by MAC address). Instead of copying the image once per node, use --overlay <name> to create a thin qcow2 overlay per node on top of the customized image:

podman run -it -v <PATH_TO_CONFIG_DIR>:/config $ELEMENTAL_IMAGE customize --type qcow2 \
    --output /config/base.qcow2 --overlay node1,node2,node3

This creates base.qcow2 plus node1.qcow2, node2.qcow2 and node3.qcow2 next to it. Each overlay starts empty and only stores the data written by its node, so the nodes diverge over time while sharing the unmodified data of the base image. The flag can be repeated or given a comma separated list, and .qcow2 is appended to names that do not already have it.

The base image path is recorded in each overlay relative to the overlay itself, so keep the base image in the same directory when moving or copying the images. Do not modify or boot the base image directly once overlays depend on it.

Cache configurations

During the customization process, artifacts downloaded during customization can be optionally cached to speed up future builds.

The cache is controlled by the following elemental3 customize options:

Default run, cached artifacts will reside under <PATH_TO_CONFIG_DIR>/cache on the host machine:

podman run -it \
  -v <PATH_TO_CONFIG_DIR>:/config \
  $ELEMENTAL_IMAGE \
  customize --type <raw/iso>

Offline run, only previously cached artifacts are used and the process fails if any of them is missing. Adding --network none ensures nothing is downloaded during customization:

podman run -it --network none \
  -v <PATH_TO_CONFIG_DIR>:/config \
  -v <PATH_TO_CACHE_DIR>:/cache \
  $ELEMENTAL_IMAGE \
  customize --type <raw/iso> \
  --cache offline

Refresh run, previously cached artifacts are discarded and downloaded again:

podman run -it \
  -v <PATH_TO_CONFIG_DIR>:/config \
  -v <PATH_TO_CACHE_DIR>:/cache \
  $ELEMENTAL_IMAGE \
  customize --type <raw/iso> \
  --cache refresh

No cache run, artifacts will not be cached and previously cached artifacts will be ignored:

podman run -it \
  -v <PATH_TO_CONFIG_DIR>:/config \
  $ELEMENTAL_IMAGE \
  customize --type <raw/iso> \
  --cache off

NOTE: A container image with the latest tag, is served from the cache until the cache is refreshed.

For locally built release manifests not served by a registry, the easiest way to seed them into the cache is:

podman push \
  localhost/release-manifest:dev \
  oci:<PATH_TO_CACHE_DIR>/oci:localhost/release-manifest:dev

Air-gapped images

Passing the --airgap option to elemental3 customize produces an image that creates its Kubernetes cluster and deploys its workloads without any network access by embedding all of the necessary artifacts into the image at customization time.

This includes:

At boot time an embedded registry is started by the elemental-registry.service that uses Hauler to serve the container images for the Kubernetes cluster.

Customizing an air-gapped image:

podman run -it \
  -v <PATH_TO_CONFIG_DIR>:/config \
  -v <PATH_TO_CACHE_DIR>:/cache \
  $ELEMENTAL_IMAGE \
  customize --type <raw/iso> \
  --airgap

Booting a customized image

NOTE: The below RAM and vCPU resources are just reference values, feel free to tweak them based on what your environment needs.

The customized image can be booted as any other regular image. Below you can find an example of how this can be done by using Libvirt to setup a virtual machine from a customized image that has a static network configured for machines with the FE:C4:05:42:8B:01 MAC address.