The cluster-api-provider-incus book

LXC will spin up a plain Ubuntu instance with haproxy installed.

The controller will automatically update the list of backends on the haproxy configuration as control plane nodes are added or removed from the cluster.

No other configuration is required for “lxc” mode. The load balancer instance can be configured through the .instanceSpec field.

The load balancer container is a single point of failure to access the workload cluster control plane. Therefore, it should only be used for development or evaluation clusters.

OCI will spin up an OCI instance running the kindest/haproxy image.

The controller will automatically update the list of backends on the haproxy configuration as control plane nodes are added or removed from the cluster.

No other configuration is required for “oci” mode. The load balancer instance can be configured through the .instanceSpec field.

The load balancer container is a single point of failure to access the workload cluster control plane. Therefore, it should only be used for development or evaluation clusters.

Requires server extensions: “instance_oci”

OVN will create a network load balancer.

The controller will automatically update the list of backends for the network load balancer as control plane nodes are added or removed from the cluster.

The cluster administrator is responsible to ensure that the OVN network is configured properly and that the LXCMachineTemplate objects have appropriate profiles to use the OVN network.

When using the “ovn” mode, the load balancer address must be set in .spec.controlPlaneEndpoint.host on the LXCCluster object.

Requires server extensions: “network_load_balancer”, “network_load_balancer_health_checks”

External will not create a load balancer. It must be used alongside something like kube-vip, otherwise the cluster will fail to provision.

When using the “external” mode, the load balancer address must be set in .spec.controlPlaneEndpoint.host on the LXCCluster object.

ControlPlaneEndpoint represents the endpoint to communicate with the control plane.

SecretRef references a secret with credentials to access the LXC (e.g. Incus, LXD) server.

LoadBalancer is configuration for provisioning the load balancer of the cluster.

Unprivileged will launch unprivileged LXC containers for the cluster machines.

Known limitations apply for unprivileged LXC containers (e.g. cannot use NFS volumes).

Do not apply the default kubeadm profile on container instances.

In this case, the cluster administrator is responsible to create the profile manually and set the .spec.template.spec.profiles field of all LXCMachineTemplate objects.

For more details on the default kubeadm profile that is applied, see https://lxc.github.io/cluster-api-provider-incus/reference/profile/kubeadm.html

Ready denotes that the LXC cluster (infrastructure) is ready.

Conditions defines current service state of the LXCCluster.

V1Beta2 groups all status fields that will be added in LXCCluster’s status with the v1beta2 version.

Standard object’s metadata. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#metadata

conditions represents the observations of a LXCCluster’s current state. Known condition types are Ready, LoadBalancerAvailable, Deleting, Paused.

InstanceSpec can be used to adjust the load balancer instance configuration.

Flavor is configuration for the instance size (e.g. t3.micro, or c2-m4).

Examples:

• t3.micro – match specs of an EC2 t3.micro instance
• c2-m4 – 2 cores, 4 GB RAM

Profiles is a list of profiles to attach to the instance.

Image to use for provisioning the load balancer machine. If not set, a default image based on the load balancer type will be used.

• “oci”: ghcr.io/neoaggelos/cluster-api-provider-lxc/haproxy:v0.0.1
• “lxc”: haproxy from the default simplestreams server

Target where the load balancer machine should be provisioned, when infrastructure is a production cluster.

Can be one of:

• name: where name is the name of a cluster member.
• @name: where name is the name of a cluster group.

Target is ignored when infrastructure is single-node (e.g. for development purposes).

For more information on cluster groups, you can refer to https://linuxcontainers.org/incus/docs/main/explanation/clustering/#cluster-groups

NetworkName is the name of the network to create the load balancer.

Name is the image name or alias.

Note that Incus and Canonical LXD use incompatible image servers for Ubuntu images. To address this issue, setting image name to ubuntu:VERSION is a shortcut for:

• Incus: “images:ubuntu/VERSION/cloud” (from https://images.linuxcontainers.org)
• LXD: “ubuntu:VERSION” (from https://cloud-images.ubuntu.com/releases)

Fingerprint is the image fingerprint.

Server is the remote server, e.g. “https://images.linuxcontainers.org”

Protocol is the protocol to use for fetching the image, e.g. “simplestreams”.

ProviderID is the container name in ProviderID format (lxc:///).

InstanceType is “container” or “virtual-machine”. Empty defaults to “container”.

Flavor is configuration for the instance size (e.g. t3.micro, or c2-m4).

Examples:

• t3.micro – match specs of an EC2 t3.micro instance
• c2-m4 – 2 cores, 4 GB RAM

Profiles is a list of profiles to attach to the instance.

Devices allows overriding the configuration of the instance disk or network.

Device configuration must be formatted using the syntax “,=”.

For example, to specify a different network for an instance, you can use:

  # override device "eth0", to be of type "nic" and use network "my-network"
devices:
- eth0,type=nic,network=my-network

Config allows overriding instance configuration keys.

Note that the provider will always set the following configuration keys:

• “cloud-init.user-data”: cloud-init config data
• “user.cluster-name”: name of owning cluster
• “user.cluster-namespace”: namespace of owning cluster
• “user.cluster-role”: instance role (e.g. control-plane, worker)
• “user.machine-name”: name of machine (should match instance hostname)

See https://linuxcontainers.org/incus/docs/main/reference/instance_options/#instance-options for details.

Image to use for provisioning the machine. If not set, a kubeadm image from the default upstream simplestreams source will be used, based on the version of the machine.

Note that the default source does not support images for all Kubernetes versions, refer to the documentation for more details on which versions are supported and how to build a base image for any version.

Target where the machine should be provisioned, when infrastructure is a production cluster.

Can be one of:

• name: where name is the name of a cluster member.
• @name: where name is the name of a cluster group.

Target is ignored when infrastructure is single-node (e.g. for development purposes).

For more information on cluster groups, you can refer to https://linuxcontainers.org/incus/docs/main/explanation/clustering/#cluster-groups

Ready denotes that the LXC machine is ready.

LoadBalancerConfigured will be set to true once for each control plane node, after the load balancer instance is reconfigured.

Addresses is the list of addresses of the LXC machine.

Conditions defines current service state of the LXCMachine.

V1Beta2 groups all status fields that will be added in LXCMachine’s status with the v1beta2 version.

Standard object’s metadata. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#metadata

Spec is the specification of the desired behavior of the machine.

  # override device "eth0", to be of type "nic" and use network "my-network"
devices:
- eth0,type=nic,network=my-network

conditions represents the observations of a LXCMachine’s current state. Known condition types are Ready, InstanceProvisioned, Deleting, Paused.

Name is the name of the secret to use. The secret must already exist in the same namespace as the parent object.