Define and Apply a Site⚓︎

A Site is the single document that describes an xCore: its network attachments, the mobility and subscriber functions it runs, its user planes with their address pools, and its data backends. This page explains how the document is put together so that you can read the Site agreed for your deployment, gives four complete examples of the deployment shapes, and shows how to apply one. The Site resource reference lists every field.

Who writes the Site⚓︎

The Site for your deployment is written together with Expeto Support during the design phase, from your network and identity plan, and applied during installation. Today you hold the document: keep it under version control, apply it with kubectl, and review every later change with Expeto Support before you apply it.

That is going to change. Expeto intends the Expeto Agent to write the Site from configuration held in xControl, so that the Site, its Systems and its radios are configured in one place and the core follows. As that work lands, fields move from the Site into xControl, the schema changes from one release to the next, and the Site may stop being a document you edit directly: the Agent will own it. A release that changes the schema ships with the steps to migrate an existing Site; see Upgrade, change and uninstall.

The examples on this page illustrate the deployment shapes for the design conversation. They are starting points for that conversation, not templates to adapt on your own.

Anatomy of a Site⚓︎

apiVersion: expeto.io/v1
kind: Site
metadata:
  name: acmecorp-campus           # one Site per namespace
spec:
  network:                         # attachments, service ranges, BGP
  topology:
    controlPlane: {}               # PLMN, AMF, MME, subscriber store, SMF interfaces
    userPlanes: []                 # one entry per UPF
  elm: {}                          # service map, tenant identity, Ignite credentials
  ignite: {}                       # session-state cluster
  dragonfly: {}                    # cache
  mariaDb: {}                      # relational store
  nodeConfig: {}                   # optional node tuning

Network attachments⚓︎

spec.network.attachments declares one NetworkAttachmentDefinition per segment the core touches. The plugins list is a CNI plugin chain rendered as-is, so use whatever the nodes support; the common shape is macvlan on a VLAN interface with static addressing, where every pod's address is pinned from the topology:

network:
  attachments:
    - name: n3-radio
      plugins:
        - type: macvlan
          master: ens192            # node interface carrying the segment
          mtu: 1500
          ipam:
            type: static

Attachment names are how the topology refers to segments; a topology entry that names an undeclared attachment leaves the Site Ready=False with reason TopologyMissingNADs.

Add serviceRanges when an endpoint must be a load-balanced address rather than a pod on a segment (radios that cannot be adjacent to a Multus attachment, cloud networks), and bgp when pools or service addresses are advertised to a peer router. Both are described in Architecture; a BGP router with impl: metallb, the default, needs the chart installed with bgp.enabled=true.

Control plane⚓︎

spec.topology.controlPlane declares the identity of the network and the functions that serve it:

plmnId, the PLMN as MCC and MNC in one string. It is required whenever a mobility function or the subscriber store is declared, and the MNC length is part of the identity. A Site with an MME needs a 3-digit MNC.
subscriberManagement, the subscriber store: mode 5G deploys the UDR, UDM and AUSF, 4G the HSS, Dual all four.
amf, the AMF with its N2 endpoint and tracking areas, and mme, the MME with its S1 endpoint, tracking areas and timers. Each needs the subscriber store it authenticates through: 5G or Dual for the AMF, 4G or Dual for the MME.
dataNetworks, the DNNs or APNs the core serves, and slices, the S-NSSAIs the AMF advertises.
n16, the SMF's GTP-C endpoint towards an upstream control plane, such as the xRouter of a public core.

Declaring both amf and mme gives a combined core. The two identities must not map onto each other. There is no N26 interface between them; see Architecture for how a combined core interworks.

User planes⚓︎

Each entry in spec.topology.userPlanes is one UPF, and its mode and interfaces define its role; see Architecture. An anchor, a user plane with n6, carries the systemId that ties it to a System in xControl and holds the UE address pools of that System. A front, without n6, serves every System and carries no systemId. 4G radios always connect to a front (the SGW-U), which forwards over N9 to the anchors (the PGW-Us).

Each pool under ueIpPools is a CIDR and, optionally, how it reaches the outside:

ueIpPools:
  - cidr: 10.100.0.0/16                 # source NAT on the UPF (default)
  - cidr: 10.101.0.0/16
    reachability:
      static: {}                        # the network team routes the pool to the N6 address
  - cidr: 10.102.0.0/16
    reachability:
      bgp:
        peering: southbound             # advertised over a declared peering

All pools on one user plane must use the same mode.

Service map and Ignite credentials⚓︎

spec.elm is required. operatorIdentifier names the tenant and service the network functions report under; pick two stable identifiers. services is the map the network functions use to locate each other; the entries are the Services the operator creates in the namespace, so copy the block from the examples unchanged. Sites with an AMF carry four additional entries (the identifier allocator, the SMF, the AUSF and the UDM) that the AMF resolves by network-function type. dbInfo holds the Ignite credentials; reference the password from a Secret rather than writing it into the Site.

Data backends and node tuning⚓︎

ignite, mariaDb and dragonfly size the data backends. The defaults suit a production cluster: three Ignite replicas with 11 GiB of memory and a 25 GiB volume each, and a standalone MariaDB. mariaDb.replicas: 3 runs a Galera cluster and is fixed at creation; see MariaDB topology. nodeConfig applies node tuning through the operator; leave it out when the netcfg operator the chart installs, or the platform team, prepares the nodes.

Change control⚓︎

spec.maintenanceMode (default true) allows every edit. Set it to false to have the admission webhook reject edits that would restart pods until a change window; see Change a Site.

Examples⚓︎

Each example is a complete Site for one deployment shape, with placeholder addressing (10.0.x.0/24 segments, 10.100.0.0/16 and 10.101.0.0/16 pools, ens192 as the parent interface, PLMN 001001). Pick transit segments and pools that do not overlap the cluster's pod and Service networks; the operator refuses a Site whose ranges do.

Public core⚓︎

Terminates the tunnels an xRouter forwards over IPX and breaks subscriber traffic out to two data networks. The xRouter's user plane sends GTP-U to the SGW-U's N3 address and its control plane reaches the SMF over the signalling attachment. No radio-facing functions. Download.

# Public xCore: terminates the tunnels an xRouter forwards over IPX and
# breaks subscriber traffic out to two data networks. No radio-facing
# functions; the SMF acts as SGW-C and PGW-C.
# Illustrative deployment shape with placeholder addressing. The Site for a real
# deployment is agreed with Expeto Support during design; do not adapt this on your own.
apiVersion: expeto.io/v1
kind: Site
metadata:
  name: acmecorp-public
spec:
  network:
    attachments:
      - name: n3-xrouter              # GTP-U from the xRouter user plane
        plugins:
          - type: macvlan
            master: ens192
            mtu: 1500
            ipam:
              type: static
      - name: signalling              # GTP-C from the xRouter control plane
        plugins:
          - type: macvlan
            master: ens192
            mtu: 1500
            ipam:
              type: static
      - name: n9-transit              # SGW-U to PGW-U
        plugins:
          - type: macvlan
            master: ens192
            mtu: 1500
            ipam:
              type: static
      - name: n6-internet
        plugins:
          - type: macvlan
            master: ens192
            mtu: 1500
            ipam:
              type: static
      - name: n6-corporate
        plugins:
          - type: macvlan
            master: ens192
            mtu: 1500
            ipam:
              type: static
  topology:
    ipProtocolMode: ipv4
    customerId: 2851                  # the Customer's numeric ID in xControl
    controlPlane:
      n16:
        - name: signalling
          transitIpRange: 10.0.16.10/24
          gatewayIp: 10.0.16.1
    userPlanes:
      # SGW-U: receives GTP-U from the xRouter and forwards to the anchors.
      # It serves every System, so it has no systemId.
      - name: sgw-u
        mode: "4G"
        numberOfSessions: 50000
        n3:
          - name: n3-xrouter
            transitIpRange: 10.0.3.10/24
            gatewayIp: 10.0.3.1
        n9:
          - name: n9-transit
            transitIpRange: 10.0.9.10/24
            peerUpfs: [pgw-u-internet, pgw-u-corporate]
      # PGW-U for the "internet" System: source NAT on the UPF.
      - name: pgw-u-internet
        mode: "4G"
        systemId: 2                   # System ID in xControl
        numberOfSessions: 50000
        n9:
          - name: n9-transit
            transitIpRange: 10.0.9.20/24
            peerUpfs: [sgw-u]
        n6:
          - name: n6-internet
            dataNetworkName: internet
            transitIpRange: 10.0.6.10/24
            gatewayIp: 10.0.6.1
            ueIpPools:
              - cidr: 10.100.0.0/16
            dnsServers: [10.0.6.53]
      # PGW-U for the "corporate" System: the corporate router carries a
      # static route for the pool; the Site status lists it.
      - name: pgw-u-corporate
        mode: "4G"
        systemId: 3
        numberOfSessions: 50000
        n9:
          - name: n9-transit
            transitIpRange: 10.0.9.30/24
            peerUpfs: [sgw-u]
        n6:
          - name: n6-corporate
            dataNetworkName: corporate
            transitIpRange: 10.0.7.10/24
            gatewayIp: 10.0.7.1
            ueIpPools:
              - cidr: 10.101.0.0/16
                reachability:
                  static: {}
            dnsServers: [10.0.7.53]
  elm:
    operatorIdentifier:
      serviceId: slice1
      tenantId: acmecorp
    services:
      - componentName: NFCM
        value:
          - serviceName: MONITORING
            serviceDomain: nfcm-monitoring:8080
          - serviceName: STATISTICS
            serviceDomain: nfcm-statistics:8080
      - componentName: TIMER
        value:
          - serviceName: TIMER
            serviceDomain: udsf-timer:8080
      - componentName: UDSF
        value:
          - serviceName: UDSF
            serviceDomain: udsf:8080
      - componentName: UPF
        value:
          - serviceName: UPFIDALLOCATOR
            serviceDomain: udsf-idallocator:8080
    dbInfo:
      dbType: ignite
      hostname: ignite-proxy
      username: ignite
      passwordSecret:
        name: ignite-credentials
        key: password
  ignite:
    enabled: true
    replicas: 3
  dragonfly:
    enabled: true
  mariaDb:
    enabled: true
    replicas: 3

Private 5G core⚓︎

An AMF, the 5G subscriber store and one session-anchor UPF serving a campus of gNodeBs. Download.

# Private 5G core: AMF, 5G subscriber store and one session-anchor UPF
# serving a campus of gNodeBs.
# Illustrative deployment shape with placeholder addressing. The Site for a real
# deployment is agreed with Expeto Support during design; do not adapt this on your own.
apiVersion: expeto.io/v1
kind: Site
metadata:
  name: acmecorp-campus
spec:
  network:
    attachments:
      - name: n2-radio                # gNodeB control plane (NGAP over SCTP)
        plugins:
          - type: macvlan
            master: ens192
            mtu: 1500
            ipam:
              type: static
      - name: n3-radio                # gNodeB user plane (GTP-U)
        plugins:
          - type: macvlan
            master: ens192
            mtu: 1500
            ipam:
              type: static
      - name: n6-enterprise           # data network
        plugins:
          - type: macvlan
            master: ens192
            mtu: 1500
            ipam:
              type: static
  topology:
    ipProtocolMode: ipv4
    customerId: 2851
    controlPlane:
      plmnId: "001001"                # MCC 001, MNC 001; must match SIMs and radios
      dataNetworks: [enterprise]
      subscriberManagement:
        enabled: true
        mode: "5G"                    # UDR, UDM and AUSF on the Site
      amf:
        n2:
          - name: n2-radio
            transitIpRange: 10.0.2.10/24
            gatewayIp: 10.0.2.1
        trackingAreas: [1]
    userPlanes:
      # One session-anchor UPF: N3 from the radios, N6 to the enterprise.
      - name: psa-upf
        mode: "5G"
        systemId: 2                   # System ID in xControl
        numberOfSessions: 10000
        n3:
          - name: n3-radio
            transitIpRange: 10.0.3.10/24
        n6:
          - name: n6-enterprise
            dataNetworkName: enterprise
            transitIpRange: 10.0.6.10/24
            gatewayIp: 10.0.6.1
            ueIpPools:
              - cidr: 10.100.0.0/16
            dnsServers: [10.0.6.53]
  elm:
    operatorIdentifier:
      serviceId: slice1
      tenantId: acmecorp
    services:
      - componentName: NFCM
        value:
          - serviceName: MONITORING
            serviceDomain: nfcm-monitoring:8080
          - serviceName: STATISTICS
            serviceDomain: nfcm-statistics:8080
      - componentName: TIMER
        value:
          - serviceName: TIMER
            serviceDomain: udsf-timer:8080
      - componentName: UDSF
        value:
          - serviceName: UDSF
            serviceDomain: udsf:8080
      - componentName: UPF
        value:
          - serviceName: UPFIDALLOCATOR
            serviceDomain: udsf-idallocator:8080
      # Peers the AMF resolves by network-function type.
      - componentName: IDALLOCATOR
        value:
          - serviceName: IDALLOCATOR
            serviceDomain: udsf-idallocator:8080
      - componentName: SMF
        value:
          - serviceName: PDUSESSION
            serviceDomain: smf:8080
      - componentName: AUSF
        value:
          - serviceName: UEAUTHENTICATION
            serviceDomain: sdm-ausf:8080
      - componentName: UDM
        value:
          - serviceName: SDM
            serviceDomain: sdm-udm:8080
          - serviceName: UECM
            serviceDomain: sdm-udm:8080
    dbInfo:
      dbType: ignite
      hostname: ignite-proxy
      username: ignite
      passwordSecret:
        name: ignite-credentials
        key: password
  ignite:
    enabled: true
    replicas: 3
  dragonfly:
    enabled: true
  mariaDb:
    enabled: true
    replicas: 3

Private 4G core⚓︎

An MME with its S1 transport controller, the HSS, and an SGW-U in front of a PGW-U, serving eNodeBs. Download.

# Private 4G core: MME with its S1 transport controller, HSS, and an
# SGW-U / PGW-U pair serving a site of eNodeBs.
# Illustrative deployment shape with placeholder addressing. The Site for a real
# deployment is agreed with Expeto Support during design; do not adapt this on your own.
apiVersion: expeto.io/v1
kind: Site
metadata:
  name: acmecorp-plant
spec:
  network:
    attachments:
      - name: s1-radio                # eNodeB control plane (S1AP over SCTP)
        plugins:
          - type: macvlan
            master: ens192
            mtu: 1500
            ipam:
              type: static
      - name: s1u-radio               # eNodeB user plane (GTP-U)
        plugins:
          - type: macvlan
            master: ens192
            mtu: 1500
            ipam:
              type: static
      - name: s5u-transit             # SGW-U to PGW-U
        plugins:
          - type: macvlan
            master: ens192
            mtu: 1500
            ipam:
              type: static
      - name: sgi-enterprise          # data network
        plugins:
          - type: macvlan
            master: ens192
            mtu: 1500
            ipam:
              type: static
  topology:
    ipProtocolMode: ipv4
    customerId: 2851
    controlPlane:
      plmnId: "001001"
      dataNetworks: [enterprise]      # the APN; also the MME's P-GW association
      subscriberManagement:
        enabled: true
        mode: "4G"                    # HSS on the Site
      mme:
        s1:
          - name: s1-radio
            transitIpRange: 10.0.2.11/24
            gatewayIp: 10.0.2.1
        identity:
          mmegi: 2
          mmec: 1
        trackingAreas: [1]
        nasSecurity:
          integrity: [AES, SNOW3G]    # EIA2 preferred, EIA1 fallback
    userPlanes:
      # SGW-U: S1-U from the eNodeBs, S5-U to the PGW-U. It serves every
      # System, so it has no systemId.
      - name: sgw-u
        mode: "4G"
        numberOfSessions: 10000
        n3:
          - name: s1u-radio
            transitIpRange: 10.0.3.20/24
        n9:
          - name: s5u-transit
            transitIpRange: 10.0.9.10/24
            peerUpfs: [pgw-u]
      # PGW-U: anchors the UE addresses and egresses on SGi.
      - name: pgw-u
        mode: "4G"
        systemId: 3                   # System ID in xControl
        numberOfSessions: 10000
        n9:
          - name: s5u-transit
            transitIpRange: 10.0.9.20/24
            peerUpfs: [sgw-u]
        n6:
          - name: sgi-enterprise
            dataNetworkName: enterprise
            transitIpRange: 10.0.6.20/24
            gatewayIp: 10.0.6.1
            ueIpPools:
              - cidr: 10.101.0.0/16
            dnsServers: [10.0.6.53]
  elm:
    operatorIdentifier:
      serviceId: slice1
      tenantId: acmecorp
    services:
      - componentName: NFCM
        value:
          - serviceName: MONITORING
            serviceDomain: nfcm-monitoring:8080
          - serviceName: STATISTICS
            serviceDomain: nfcm-statistics:8080
      - componentName: TIMER
        value:
          - serviceName: TIMER
            serviceDomain: udsf-timer:8080
      - componentName: UDSF
        value:
          - serviceName: UDSF
            serviceDomain: udsf:8080
      - componentName: UPF
        value:
          - serviceName: UPFIDALLOCATOR
            serviceDomain: udsf-idallocator:8080
    dbInfo:
      dbType: ignite
      hostname: ignite-proxy
      username: ignite
      passwordSecret:
        name: ignite-credentials
        key: password
  ignite:
    enabled: true
    replicas: 3
  dragonfly:
    enabled: true
  mariaDb:
    enabled: true
    replicas: 3

Combined 4G and 5G core⚓︎

The AMF and the MME side by side, the subscriber store in dual mode, one front user plane for both radio types and one anchor per System that serves both. Because every anchor is in mode Dual, a UE that moves between 4G and 5G keeps its address. This is the shape the combined core installation walks through. Download.

# Combined 4G and 5G private core: the AMF and the MME side by side, the
# subscriber store in dual mode, one front user plane for both radio types
# and one anchor per System that serves both. Every anchor is mode Dual, so
# the core interworks without N26: a UE that changes radio keeps its
# address. There is no N26 interface between the AMF and the MME.
# Illustrative deployment shape with placeholder addressing. The Site for a real
# deployment is agreed with Expeto Support during design; do not adapt this on your own.
apiVersion: expeto.io/v1
kind: Site
metadata:
  name: acmecorp-hq
spec:
  network:
    attachments:
      - name: n2-radio                # gNodeB control plane (NGAP over SCTP)
        plugins:
          - type: macvlan
            master: ens192
            mtu: 1500
            ipam:
              type: static
      - name: s1-radio                # eNodeB control plane (S1AP over SCTP)
        plugins:
          - type: macvlan
            master: ens192
            mtu: 1500
            ipam:
              type: static
      - name: n3-radio                # GTP-U from both radio types
        plugins:
          - type: macvlan
            master: ens192
            mtu: 1500
            ipam:
              type: static
      - name: n9-transit              # front to anchors
        plugins:
          - type: macvlan
            master: ens192
            mtu: 1500
            ipam:
              type: static
      - name: n6-office               # data network of the "office" System
        plugins:
          - type: macvlan
            master: ens192
            mtu: 1500
            ipam:
              type: static
      - name: n6-plant                # data network of the "plant" System
        plugins:
          - type: macvlan
            master: ens192
            mtu: 1500
            ipam:
              type: static
  topology:
    ipProtocolMode: ipv4
    customerId: 2851                  # the Customer's numeric ID in xControl
    controlPlane:
      plmnId: "001001"                # 3-digit MNC: the MME needs one for 4G attach
      dataNetworks: [enterprise]
      subscriberManagement:
        enabled: true
        mode: "Dual"                  # UDR, UDM, AUSF and HSS
      amf:
        n2:
          - name: n2-radio
            transitIpRange: 10.0.2.10/24
            gatewayIp: 10.0.2.1
        identity:
          region: 1
          set: 1
          pointer: 1
        trackingAreas: [1]
      mme:
        s1:
          - name: s1-radio
            transitIpRange: 10.0.4.10/24
            gatewayIp: 10.0.4.1
        identity:
          mmegi: 2
          mmec: 1
        trackingAreas: [1]
        nasSecurity:
          integrity: [AES, SNOW3G]    # EIA2 preferred, EIA1 fallback
    userPlanes:
      # Front for both radio types: the SGW-U for 4G and the intermediate
      # UPF for 5G. It serves every System, so it has no systemId.
      - name: upf-front
        mode: "Dual"
        numberOfSessions: 20000
        n3:
          - name: n3-radio
            transitIpRange: 10.0.3.10/24
        n9:
          - name: n9-transit
            transitIpRange: 10.0.9.10/24
            peerUpfs: [upf-office, upf-plant]
      # Anchor of the "office" System: PGW-U for 4G and session anchor for
      # 5G, one address pool for both.
      - name: upf-office
        mode: "Dual"
        systemId: 2                   # System ID in xControl
        numberOfSessions: 10000
        n9:
          - name: n9-transit
            transitIpRange: 10.0.9.20/24
            peerUpfs: [upf-front]
        n6:
          - name: n6-office
            dataNetworkName: enterprise
            transitIpRange: 10.0.6.10/24
            gatewayIp: 10.0.6.1
            ueIpPools:
              - cidr: 10.100.0.0/16
            dnsServers: [10.0.6.53]
      # Anchor of the "plant" System. The plant network routes its pool to
      # the N6 address; the Site status lists the route it needs.
      - name: upf-plant
        mode: "Dual"
        systemId: 3
        numberOfSessions: 10000
        n9:
          - name: n9-transit
            transitIpRange: 10.0.9.30/24
            peerUpfs: [upf-front]
        n6:
          - name: n6-plant
            dataNetworkName: enterprise
            transitIpRange: 10.0.7.10/24
            gatewayIp: 10.0.7.1
            ueIpPools:
              - cidr: 10.101.0.0/16
                reachability:
                  static: {}
            dnsServers: [10.0.7.53]
  elm:
    operatorIdentifier:
      serviceId: slice1
      tenantId: acmecorp
    services:
      - componentName: NFCM
        value:
          - serviceName: MONITORING
            serviceDomain: nfcm-monitoring:8080
          - serviceName: STATISTICS
            serviceDomain: nfcm-statistics:8080
      - componentName: TIMER
        value:
          - serviceName: TIMER
            serviceDomain: udsf-timer:8080
      - componentName: UDSF
        value:
          - serviceName: UDSF
            serviceDomain: udsf:8080
      - componentName: UPF
        value:
          - serviceName: UPFIDALLOCATOR
            serviceDomain: udsf-idallocator:8080
      # Peers the AMF resolves by network-function type.
      - componentName: IDALLOCATOR
        value:
          - serviceName: IDALLOCATOR
            serviceDomain: udsf-idallocator:8080
      - componentName: SMF
        value:
          - serviceName: PDUSESSION
            serviceDomain: smf:8080
      - componentName: AUSF
        value:
          - serviceName: UEAUTHENTICATION
            serviceDomain: sdm-ausf:8080
      - componentName: UDM
        value:
          - serviceName: SDM
            serviceDomain: sdm-udm:8080
          - serviceName: UECM
            serviceDomain: sdm-udm:8080
    dbInfo:
      dbType: ignite
      hostname: ignite-proxy
      username: ignite
      passwordSecret:
        name: ignite-credentials
        key: password
  ignite:
    enabled: true
    replicas: 3
  dragonfly:
    enabled: true
  mariaDb:
    enabled: true
    replicas: 3

Apply the Site⚓︎

Create the Ignite credential the Site references, then apply the Site agreed with Expeto Support:

kubectl -n xcore create secret generic ignite-credentials --from-literal=password='<password>'
kubectl -n xcore apply -f site.yaml
kubectl -n xcore get sites.expeto.io -w

The API server validates the document against the CustomResourceDefinition schema and rejects malformed addresses, unknown enum values and missing required fields immediately. The operator then validates the cross-references (attachment names, peer user planes, service ranges, identities) and reports any problem as a condition on the Site within seconds. Bring-up of a healthy Site takes a few minutes: the Ignite cluster initialises first, then the databases, then the network functions, and Ready turns True when the Agent, every user plane and every control-plane function report ready.

kubectl -n xcore wait --for=condition=Ready --timeout=15m sites.expeto.io/acmecorp-campus

Continue with Verify the installation.