Skip to content

Platform Notes⚓︎

The operator installs the same way on every conformant Kubernetes distribution. What differs is how the prerequisites are met: Multus, the CNI plugin binaries, cert-manager, storage, node tuning and the monitoring stack. This page records the platform-specific steps.

MicroK8s⚓︎

  • Multus comes from the community add-ons. The add-on installs the Multus DaemonSet but not the reference CNI plugins the attachments need; confirm macvlan and static are present and copy a containernetworking plugins release into the directory if they are not.

    sudo microk8s enable community
sudo microk8s enable multus
ls /var/snap/microk8s/current/opt/cni/bin | grep -E 'macvlan|static'

  • cert-manager: sudo microk8s enable cert-manager.

  • Storage: hostpath-storage works on a single node only. Multi-node clusters need a storage class that follows pods between nodes; in the absence of a built-in or otherwise preferred option, Longhorn is the suggested choice.

  • Node tuning: the Site's nodeConfig DaemonSet applies sysctls and loads the sctp module. If you manage nodes yourself, raise the open-file limit for containerd as well:

    echo 'ulimit -n 1048576 || true' | sudo tee -a /var/snap/microk8s/current/args/containerd-env

  • NodePort exposure of N2 or S1 needs the API server's --service-node-port-range widened to include 38412 and 36412; Multus-attached endpoints need no such change.

  • Firewall: with ufw active, allow the MicroK8s control ports between nodes (25000, 16443, 10250, 10255, 12379, 10257, 10259, 19001) or pods cannot reach the API server.

OpenShift⚓︎

  • The operator detects OpenShift; install with the standard command. The chart never bundles cert-manager, and MetalLB and the Prometheus stack are off by default, as OpenShift needs. Enable the operator's ServiceMonitor and switch off any bundled operator the cluster already provides:

    helm install xcore expeto/xfabric -n xcore --create-namespace -f values.yaml \
    --set metrics.serviceMonitor.enabled=true \
    --set global.imageCredentials.username='<username>' \
    --set global.imageCredentials.password='<password>' \
    --set agent.apiToken='<site API token>'

  • cert-manager: install the cert-manager Operator from OperatorHub. The chart's install-time guard checks that the CustomResourceDefinition exists.

  • Multus is part of the OpenShift networking stack; declare attachments as usual. The parent interface for macvlan must exist on the worker nodes' host network.
  • MetalLB: install the MetalLB Operator from OperatorHub, then --set bgp.enabled=true.
  • Monitoring: enable user-workload monitoring (enableUserWorkload: true in the cluster-monitoring-config ConfigMap); its Prometheus discovers the operator's ServiceMonitor. Never bundle kube-prometheus-stack on OpenShift; the cluster version operator owns those CustomResourceDefinitions and the install loses the ownership conflict.
  • Node tuning is rendered as a MachineConfig for the pool named in spec.nodeConfig.workerRoleName. The operator references an existing MachineConfigPool; it does not create one.
  • SCTP is enabled through the same MachineConfig; without nodeConfig, load the module with your own MachineConfig before deploying an AMF or MME.
  • Use oc where this guide says kubectl, and a project instead of a namespace.

Tanzu Kubernetes Grid⚓︎

TKG ships Antrea but not Multus, cert-manager or the data-backend operators. Install the prerequisites once per cluster:

helm repo add jetstack https://charts.jetstack.io
helm install cert-manager jetstack/cert-manager -n cert-manager --create-namespace --set installCRDs=true

kubectl apply -f https://raw.githubusercontent.com/k8snetworkplumbingwg/multus-cni/master/deployments/multus-daemonset.yml

kubectl apply \
    -f https://raw.githubusercontent.com/k8snetworkplumbingwg/whereabouts/master/doc/crds/daemonset-install.yaml \
    -f https://raw.githubusercontent.com/k8snetworkplumbingwg/whereabouts/master/doc/crds/whereabouts.cni.cncf.io_ippools.yaml \
    -f https://raw.githubusercontent.com/k8snetworkplumbingwg/whereabouts/master/doc/crds/whereabouts.cni.cncf.io_overlappingrangeipreservations.yaml

The chart's bundled MariaDB and Dragonfly operators cover the rest.

  • Parent NIC: worker VMs usually have a single vNIC. Either add a second vSphere port group to the worker template through the ClusterClass, or reuse the pod-network NIC when the LAN carries the extra subnets. Confirm the interface name before writing the attachments:

    kubectl debug node/<worker> -it --image=busybox -- ip -br link

  • Pod Security Admission: TKG enforces baseline on workload namespaces. The operator's pods are restricted-grade and need no exemption; the user-plane pods run privileged and need a namespace labelled for the privileged profile.

  • Storage: the vSphere CSI default class is used unless spec.storage.className or spec.ignite.storage.className says otherwise.
  • NSX-T Container Plugin clusters bundle Multus and expose port groups as VIFs; skip the Multus step and reference the VIF as the macvlan parent.

Amazon EKS⚓︎

  • Multus: install the AWS-maintained Multus release for your cluster version. Each attachment maps to an additional Elastic Network Interface on the node, so pick instance types with enough interfaces (two are reserved for the cluster itself) and trunking support.
  • Source/destination check must be disabled on every instance that runs a user plane, or the VPC drops the forwarded UE traffic.
  • Load balancing: MetalLB is not used on EKS. Serve N2 or S1 on a node port (ingress[].nodePort in the Site), or attach the radios' segment directly through Multus.
  • Autoscaling: Karpenter provisions nodes with the right interface capacity; size its requirements for the user-plane instance types.
  • Storage: the EBS CSI driver's gp3 class as default.

Kind and other development clusters⚓︎

Kind ships neither Multus nor the reference plugins. Install the Multus thick DaemonSet and copy the containernetworking plugins into /opt/cni/bin on the node, then use the node's eth0 as the macvlan parent. Shrink Ignite (spec.ignite.replicas: 1, smaller resources and jvm) and set spec.mariaDb.antiAffinity: false on single-node clusters.