Skip to content

Site Resource Reference⚓︎

The Site custom resource (sites.expeto.io, API version expeto.io/v1) is the deployment contract of an xCore: one document that declares every network attachment, mobility and subscriber function, user plane and data backend of the core. The operator reconciles the core from it and reports the outcome in its status. This page lists the fields of the Site in xCore 4.0.6. For how the sections fit together, and for complete examples, see Define and apply a Site.

The Site resource is changing

Today you write the Site yourself, from the design agreed with Expeto Support, and change it with kubectl. Expeto intends the Expeto Agent to write the Site instead, from the configuration held in xControl (the Site, its Systems, radios and subscriber settings). As that work lands, fields move from this resource into xControl, the schema changes between releases, and editing the Site directly may no longer be possible: the Agent will own it, and changes will be made in xControl. Keep your Site under version control, review every change with Expeto Support, and treat this page as a description of the current release rather than a stable interface.

Conventions⚓︎

  • Required fields are marked in the tables. Everything else is optional and takes the default shown, or is absent when no default is shown.
  • On change says what happens when the field is edited on a running Site. Live changes are applied by the running pods. Restart changes roll the affected pods and are rejected while spec.maintenanceMode is false. Fixed fields cannot be changed once the object they shape exists. See Change a Site.
  • The API server checks types, enumerations, patterns and ranges when the Site is applied. The operator then checks cross-references (attachment names, peer user planes, service ranges, identities) and reports a violation as Ready=False with a reason; see Troubleshooting.
  • Fields the operator reads but this page does not list are reserved for Expeto Support or for test tooling. Leave them unset.

Top level⚓︎

Field Type Default On change Description
spec.network object see below Network attachments, service address ranges and BGP.
spec.topology object, required see below The control plane and the user planes.
spec.elm object, required Restart Service map, tenant identity and Ignite credentials of the network functions.
spec.ignite object see below The Apache Ignite cluster that holds session state.
spec.dragonfly object Restart The Dragonfly cache for identifier allocation and timers.
spec.mariaDb object see below The MariaDB database of the SMF and the subscriber store.
spec.storage object size: 1Gi Fixed The Site's data volume: size and className.
spec.nodeConfig object Restart Node tuning applied to the worker nodes.
spec.maintenanceMode boolean true Live false makes the admission webhook reject every Restart edit until it is set back to true.
spec.namePrefix string "" Fixed Prefix for every resource the operator creates. Set it at installation or not at all.
spec.pullPolicy Always, IfNotPresent, Never Always Live Image pull policy of every network-function pod. Use IfNotPresent with release images.
spec.nfcm.enabled boolean true Restart Deploys NFCM, the monitoring and statistics function.
spec.udsf.enabled boolean true Restart Deploys the UDSF with its identifier allocator and timer services.
spec.description string Live Free text.

Network⚓︎

spec.network.attachments[]⚓︎

Each entry becomes one NetworkAttachmentDefinition in the Site's namespace. Topology interfaces refer to attachments by name.

Field Type Default Description
name string, required Name of the attachment and of the NetworkAttachmentDefinition.
plugins list of CNI plugin objects, required The CNI plugin chain, rendered verbatim: macvlan or ipvlan on a VLAN interface, SR-IOV, a route-override chain, with static, host-local or whereabouts address management. Use ipam.type: static when the topology pins every address.
cniVersion string 0.3.1 CNI specification version stamped into the configuration.

A change to an attachment restarts the pods that use it.

spec.network.serviceRanges[]⚓︎

Address ranges MetalLB allocates to LoadBalancer Services, for endpoints realized as an ingress address rather than a pod on an attachment. Changes are live.

Field Type Default Description
name string, required Unique name; ingress entries refer to it.
addresses list, required CIDRs (192.0.2.0/28) or inclusive ranges (192.0.2.50-192.0.2.62).
advertise[] list of 1 to 8, required How the range is announced. Each entry is either BGP (peering, a peering of the metallb router; optional aggregationLength, aggregationLengthV6, localPref, communities, prefixes) or layer 2 (l2: {}, optionally limited by l2.interfaces and l2.nodeSelectors).
autoAssign boolean true Lets MetalLB assign addresses from the range automatically.
avoidBuggyIps boolean false Excludes .0 and .255 addresses.

spec.network.bgp⚓︎

BGP routers with named peerings. Each router runs in one of two places, set by its impl:

  • metallb, the default: FRR-K8s on the nodes, configured through MetalLB. It needs MetalLB with FRR-K8s and the chart's bgp.enabled=true, and announces service ranges and UE pools. A Site has at most one metallb router.
  • gobgp: a GoBGP sidecar in the dataplane pod of each user plane whose UE pools name one of the router's peerings. It needs neither MetalLB nor bgp.enabled, and announces those UE pools from the user plane itself. Agree its use with Expeto Support; BGP lists its constraints.

Changes are live, with one exception: giving a user plane its first gobgp pool, or removing its last, restarts that user plane's dataplane pod, even while spec.maintenanceMode is false.

Field Type Default Description
routers[] list, required The Site's BGP routers: at most one with impl: metallb, and any number with impl: gobgp.
routers[].name string, required Router name.
routers[].asn integer, required Local AS number.
routers[].impl metallb or gobgp metallb Where the router runs; see above.
routers[].routerId string FRR's choice (metallb); the user plane's N6 transit address (gobgp) BGP router identifier.
routers[].peerings[] list, required Named groups of BGP sessions. Peering names are unique across the Site; service ranges and UE pools refer to them.
routers[].peerings[].attachment string The attachment whose segment the sessions ride. For a gobgp router, the first peering's attachment must be the name of an n6 entry with a transitIpRange on every user plane that uses the router.
routers[].peerings[].peers[] list, required name, address (IPv4 or IPv6), peerAsn (required); port (179), holdTime (45s), keepaliveTime (15s), gracefulRestart (true), ebgpMultiHop, sourceAddress (required when the session rides an attachment), receiveMode (all or filtered with receivePrefixes), and bfd: {} to enable Bidirectional Forwarding Detection. A gobgp router uses only address (IPv4), peerAsn, holdTime, keepaliveTime and gracefulRestart, and ignores the others.
nodeSelector label selector all nodes Nodes that run the BGP sessions.
rawConfig string FRR configuration appended verbatim, for options the Site does not model. Agree it with Expeto Support.

spec.network.podCidr⚓︎

The cluster's pod network CIDR, as a string. Needed only when a user plane source-NATs GTP-U out of its N3 interface, so that GTP-U to peers inside the cluster is not diverted. Live.

Topology⚓︎

Field Type Default On change Description
spec.topology.controlPlane object see below The PLMN, the mobility and subscriber functions and the SMF's interfaces.
spec.topology.userPlanes[] list of up to 32 see below One entry per UPF.
spec.topology.customerId integer ≥ 1 1 Restart The numeric ID of the Customer in xControl that owns the Site.
spec.topology.ipProtocolMode string Restart IP stack of the user planes. Set ipv4.

spec.topology.controlPlane⚓︎

Field Type Default On change Description
plmnId 5 or 6 digits Restart MCC followed by a 2- or 3-digit MNC (001001, 00101). It is the PLMN the AMF and MME serve and the home PLMN the HSS, UDM and UDR key subscribers under. Required when amf, mme or subscriberManagement.enabled is set. A Site with mme needs a 3-digit MNC.
dataNetworks list of up to 16 names [internet] Restart DNNs (5G) or APNs (4G) the core serves. The AMF takes the first as its default; the MME gets a P-GW association for each.
slices[] list of up to 8 AMF built-in: SST 1, SD 000001 Restart S-NSSAIs the AMF advertises on every TAC: sst (0 to 255, required) and sd (6 hex digits). The listed slices replace the built-in one; list {sst: 1} as well when Mobile Assets are provisioned without an SD.
subscriberManagement object Restart The subscriber store; see below.
amf object Restart The AMF; see below.
mme object Restart, except timers The MME; see below.
n16[] list of up to 8 endpoints the SMF's Service address Restart GTP-C endpoints of the SMF towards an upstream or roaming control plane, such as the xRouter of a public core. See Endpoints; canonicalIp selects the advertised address when an entry has several.
usageReporting object Restart intervalMinutes (1 to 1440, default 59): how long after scheduling a session's usage query the SMF sends it. thresholdMegabytes (default 10): the volume after which a UPF reports usage on its own; applies to sessions set up after the change.
overrideApn string Restart The SMF replaces the APN or DNN the UE requests with this one.

amf and amfN11Host are mutually exclusive. amfN11Host names an AMF the Site does not host and exists for test setups only; connecting an external AMF is not a supported integration.

subscriberManagement⚓︎

Field Type Default Description
enabled boolean, required Deploys the subscriber store.
mode 5G, 4G, Dual 5G 5G deploys the UDR, UDM and AUSF; 4G the HSS; Dual all four, with the HSS and UDM cancelling each other's registrations when a UE changes radio technology. amf needs 5G or Dual; mme needs 4G or Dual.
globalTitles list of up to 16 digit strings a placeholder SS7 Global Title prefixes of the HSS home PLMN; the first is the HLR Global Title.

The Agent provisions Mobile Assets into whichever stores the mode deploys. The keys those stores need, transport keys and the SUCI private key, are loaded once per Site; see Load the subscriber-store keys.

amf⚓︎

Field Type Default Description
n2[] exactly one endpoint, required The gNodeB-facing NGAP endpoint. See Endpoints.
trackingAreas list of up to 64 integers, 1 to 16777215 [1] TACs the AMF serves, rendered as 24-bit TACs.
nasSecurity integrity, ciphering: lists of up to 4 of NULL, SNOW3G, AES, ZUC integrity NIA0, NIA1; ciphering NEA0, NEA1 NAS algorithm preference, most preferred first. AES renders as NIA2/NEA2, SNOW3G as NIA1/NEA1, ZUC as NIA3/NEA3, NULL as NIA0/NEA0. The defaults put the null algorithms first; set both lists on a production Site, AES first.
identity region (0 to 255), set (0 to 1023), pointer (0 to 63) 1, 1, 1 The AMF Identifier within the PLMN.

mme⚓︎

Field Type Default Description
s1[] exactly one endpoint, required The eNodeB-facing S1AP endpoint, served by the S1 transport controller (STC). See Endpoints.
trackingAreas list of up to 64 integers, 1 to 65535 [1] TACs the MME serves, rendered as 16-bit TACs.
nasSecurity as for amf integrity EIA2, EIA1; ciphering EEA0 only NAS algorithm preference, rendered as EIA and EEA names. With ciphering unset the MME keeps its built-in default, null ciphering only; set it on a production Site, AES first.
identity mmegi (0 to 65535), mmec (0 to 255) 2, 1 MME group ID and MME code. In a combined core it must not map (TS 23.003) onto the AMF identity.
timers object the MME's defaults NAS and mobility timers in milliseconds; see below. Live: applied without a restart.
sgwList[], pgwList[] up to 16 of name, address the Site's SMF S-GW and P-GW nodes the MME selects from.
realm domain name epc.mnc<MNC>.mcc<MCC>.3gppnetwork.org Diameter origin realm.
instanceId integer ≥ 1 1 Key prefix of the MME in its session store.

mme.timers fields, with their ranges and the defaults used when unset:

Field Range Default Timer
t3402Ms 6000 to 1440000 720000 Attach and tracking area update retry back-off given to the UE.
t3412Ms 6000 to 6960000 3240000 Periodic tracking area update.
t3413Ms 1000 to 60000 3000 Paging retransmission.
t3422Ms 1000 to 60000 6000 DETACH REQUEST retransmission.
t3423Ms 6000 to 6960000 3240000 Deactivate ISR timer given to the UE.
t3442Ms 1000 to 60000 6000 SERVICE REJECT back-off given to the UE.
t3450Ms 1000 to 60000 6000 ATTACH ACCEPT, TRACKING AREA UPDATE ACCEPT and GUTI REALLOCATION retransmission.
t3460Ms 1000 to 60000 6000 AUTHENTICATION REQUEST and SECURITY MODE COMMAND retransmission.
t3470Ms 1000 to 60000 6000 IDENTITY REQUEST retransmission.
t3485Ms 1000 to 60000 8000 ACTIVATE EPS BEARER CONTEXT REQUEST retransmission.
t3486Ms 1000 to 60000 8000 MODIFY EPS BEARER CONTEXT REQUEST retransmission.
t3489Ms 1000 to 60000 4000 ESM INFORMATION REQUEST retransmission.
t3495Ms 1000 to 60000 8000 DEACTIVATE EPS BEARER CONTEXT REQUEST retransmission.
mobileReachableMs 6000 to 7200000 3480000, or T3412 plus 4 minutes when only t3412Ms is set Mobile reachable timer. Must be longer than the effective T3412.
implicitDetachMs 6000 to 3600000 3480000 Implicit detach timer.

A timer removed from the Site reverts to its default.

Endpoints⚓︎

amf.n2[], mme.s1[], controlPlane.n16[] and userPlanes[].n3[] share one model. Each entry is realized in exactly one of two ways:

  • On an attachment: name names an entry of spec.network.attachments and transitIpRange pins the pod's address as a host address with prefix length (10.0.2.10/24). gatewayIp optionally names the next hop on that segment.
  • As an ingress address: ingress[] holds a LoadBalancer address, either range (a serviceRanges name) with ip pinned inside it, or, on amf.n2 and mme.s1 only, a nodePort served on every node. The 3GPP ports (38412 for NGAP, 36412 for S1AP) are outside the API server's default node-port range, so a node port needs that range widened. An N3 ingress address needs replicas of at most 1 on its user plane.

amf.n2 and mme.s1 take one entry, and that entry at most one ingress address.

spec.topology.userPlanes[]⚓︎

Each entry is one UPF: a controller and one or more dataplane pods.

Field Type Default On change Description
name string of up to 50 characters, required Fixed Unique within the Site; names every resource of the user plane.
mode 4G, 5G, Dual, required Restart The radio generation it serves. With the interfaces it declares, the mode decides its role; see the table below.
systemId integer ≥ 1 Restart The numeric ID of the xControl System whose Mobile Assets the user plane anchors. Required on every anchor (a user plane with n6), unique within the Site, and refused on a front (a user plane without n6).
numberOfSessions integer 3000 Fixed Size of the session table. A full table rejects new sessions.
replicas integer 1 Restart Active dataplane pods.
n3[] up to 8 endpoints Restart, except routes, mtu Radio-facing GTP-U interface (N3, S1-U). See Endpoints. Also takes routes[] (destination, gateway inside the transit subnet), mtu (≥ 576) and canonicalIp.
n6[] up to 8 Restart, except routes, mtu, dnsServers, pool reachability Data-network interface (N6, SGi); see below.
n9[] up to 8 Restart Interface to other user planes (N9, S5-U): name and transitIpRange on an attachment, or both omitted for the pod network; peerUpfs (required) names the user planes at the far end.
udpMem, tcpMem integer ≥ 4096 32000000, 16000000 Restart Kernel socket buffer sizes in the dataplane pods.

The role of a user plane follows from its mode and interfaces:

Interfaces Mode 5G Mode 4G Mode Dual
N3 and N9, no N6 (a front) intermediate UPF SGW-U SGW-U and intermediate UPF
N9 and N6 (an anchor) session anchor behind an intermediate UPF PGW-U PGW-U and 5G session anchor, behind a front
N3 and N6 (an anchor) session anchor (PSA) none: 4G sessions are refused 5G session anchor; 4G sessions are refused

The SMF never uses one user plane as both the SGW-U and the PGW-U of a 4G session. A user plane in mode 4G or Dual with both N3 and N6 is selected for both roles, so every 4G session through it is refused. Serve 4G radios from a front with N3 and N9, chained to anchors with N9 and N6.

n6[]⚓︎

Field Type Default Description
name string The attachment. Omit it, together with transitIpRange and gatewayIp, to egress on the pod network.
dataNetworkName string, required The DNN or APN this interface serves.
transitIpRange host address with prefix Required with name.
gatewayIp address Next hop on the data network.
ueIpPools[] list of up to 16, required cidr (required), allocationType (dynamic or static, default dynamic) and reachability, one of sourceNat: {} (the default), static: {} (the network team routes the pool to the N6 address, and the Site status lists the routes) or bgp: {peering: <name>}. All pools of one user plane use the same reachability. A pool on a peering of a gobgp router is announced by the user plane's own GoBGP sidecar; see spec.network.bgp.
dnsServers list of addresses DNS servers handed to Mobile Assets. Live.
routes[], mtu As for n3. Live.

Service map and Ignite credentials: spec.elm⚓︎

Field Type Description
operatorIdentifier tenantId, serviceId, both required The tenant and service the network functions report and store state under. Pick two stable identifiers; no user plane name may derive the same service ID.
services[] list of componentName with value[] of serviceName and serviceDomain, required How the network functions find each other: the Services the operator creates in the namespace. The operator passes the list to every network function as written and adds nothing to it, so copy it from the examples. A Site with an AMF also lists the identifier allocator, SMF, AUSF and UDM entries the AMF resolves by network-function type.
dbInfo dbType: ignite, hostname, username, password or passwordSecret Ignite credentials. Reference a Secret with passwordSecret: {name, key} rather than writing the password into the Site. The operator rewrites hostname to its Ignite proxy.
logging, http objects Log rotation and HTTP client timeouts of the network functions. Leave the defaults.

Every spec.elm change is Restart.

Data backends⚓︎

spec.ignite⚓︎

Field Type Default Description
enabled boolean, required Deploys Ignite.
replicas integer, required 3 Ignite nodes.
storageEngine aipersist, aimem aipersist aipersist keeps the data on each node's volume and survives restarts; aimem keeps it in memory only.
storage size, className size: 25Gi Per-node work volume. Fixed once created.
resources Kubernetes resource requirements 11 GiB memory request and limit Ignite container resources.
jvm minHeap, maxHeap 2g, 4g JVM heap.
dataRegion maxSize, initSize maxSize: 1Gi Page cache (aipersist) or in-memory region (aimem).
inMemoryWorkDir boolean false Keeps Ignite's work directory in memory. Volatile: a restarted node loses its cluster state.

Every spec.ignite change is Restart. The operator refuses a memory limit that cannot hold the heap plus the data region.

spec.mariaDb⚓︎

Field Type Default On change Description
enabled boolean, required true Restart MariaDB is deployed whenever the SMF, the MME or the subscriber store needs it, whatever this says.
replicas 1, or an odd number ≥ 3 1 Fixed between 1 and Galera 1 runs a standalone server; 3 or more runs a Galera cluster. Moving between the two shapes is refused; growing or shrinking between odd sizes is supported.
antiAffinity boolean true Fixed One Galera replica per node. Set false only on test clusters.

spec.dragonfly⚓︎

Field Type Default Description
enabled boolean, required Deploys the cache.
snapshot enabled, cron, storage disabled; */5 * * * *; size: 1Gi Persists the cache to a volume on the schedule and at shutdown, so identifier allocations survive a restart.

Node tuning: spec.nodeConfig⚓︎

Applies host tuning to the worker nodes, through a privileged DaemonSet or, on OpenShift, a MachineConfig. Leave it out when the nodes are prepared another way, such as by the netcfg operator the chart installs. Every change is Restart and can interrupt the data plane: the DaemonSet re-runs on every node, and a MachineConfig drains and reboots the pool.

Field Description
backend auto (default: MachineConfig on OpenShift, DaemonSet elsewhere), daemonset or machineconfig.
sysctls[] Kernel parameters, by dotted name.
extraKernelModules[] Modules to load in addition to sctp and tun.
extraRoutes[], extraIPTables[], extraIpAddresses[] Routes, iptables rules and interface addresses to install before the dataplane starts.
txInterfaces[], txQueueLength Transmit queue length of the named interfaces.
workerRoleName OpenShift MachineConfigPool the MachineConfig targets; worker when that pool has machines.

Status⚓︎

Field Description
status.conditions[] Ready and the subsystem conditions GatewaysReady, ControlPlaneReady, PeeringReady and InfrastructureReady. A False condition names its cause in reason; see Troubleshooting. The child resources carry NetworkConfigured and DatabaseReady.
status.lastReconcileTime Time of the last reconcile pass. It advances on every pass; a stale value means the operator is not running.
status.lastReconcileError The error of the last failed pass, verbatim.
status.requiredUpstreamRoutes[] For pools with reachability.static: the routes the network team must install, with the next hop and a ready-to-use command.