Install a Combined Core⚓︎

A combined core is one xCore that serves 4G and 5G private radios at the same time: the AMF and the MME side by side, a subscriber store holding both the 4G HSS and the 5G UDR, UDM and AUSF, and user planes that anchor both radio technologies. This page takes you through the whole installation, from an empty cluster to a 4G and a 5G device attached, including loading the keys the subscriber store needs. Each step links to the page that covers it in depth.

What you build⚓︎

The core in this walk-through is the combined core example of the Site definition page:

Every anchor is in mode Dual, so the core interworks without N26 (TS 23.501 §5.17.2.3): when a device moves between 4G and 5G, its session moves with it and keeps its address, instead of the device attaching afresh. There is no N26 interface between the AMF and the MME.

The installation involves these roles; one person may hold several:

• the platform team, who owns the Kubernetes cluster;
• the network team, who owns the radio, transit and data-network segments and the addressing;
• an xControl administrator with permission to create Sites, Systems and Keysets;
• two key custodians, who hold the SIM batch's transport key and the home network's SUCI private key and enter them into the core.

Before you start⚓︎

Have the following ready. The planning and prerequisites page explains each item.

• A Kubernetes cluster that meets the cluster requirements, with SCTP on the nodes that will run the AMF and the MME.
• An xControl installation of release 3.3.22 or later, reachable from the cluster over HTTPS, and presenting a certificate issued by a public certificate authority.
• The registry credentials from Expeto Support.
• The network plan: segments and addresses for N2, S1-MME, N3 and S1-U, N9 and each data network, and the UE address pool of each System.
• The identity plan: the PLMN, with a 3-digit MNC, the tracking area codes, and the AMF and MME identities.
• The SIMs: a batch whose IMSIs belong to the PLMN, and the SIM vendor's output file (.out) for the batch, with each SIM's K, and usually its OPc, encrypted under the batch's transport key.
• For 5G, the SIMs personalized for SUCI concealment with the home network's public key, its key identifier and protection scheme Profile B; see Generate a SUCI key pair.
• The radios: eNodeBs and gNodeBs configured with the PLMN, the tracking area codes, and the core's S1-MME and N2 addresses.
• The key material, held by the key custodians and never sent by email or chat: the transport key of each SIM batch, and the SUCI private key of each key identifier. To generate a SUCI key pair, see Generate a SUCI key pair.

Step 1: prepare the cluster⚓︎

Install the cluster prerequisites the platform team owns: Multus CNI with the reference CNI plugins, cert-manager, a default StorageClass and, for BGP or load-balanced radio endpoints, MetalLB with FRR-K8s. The platform notes give the commands for MicroK8s, OpenShift, Tanzu and EKS. The xCore chart installs the netcfg node-configuration operator itself, unless another Expeto product on the cluster already runs it; see Install the operator.

Step 2: prepare xControl⚓︎

Create the objects the core registers under and provisions from. Note each numeric identifier as you go; the Site definition and the chart values use them.

1. Customer. Use the Customer that owns the private network, or create it. Note its numeric ID: it becomes customerId in the Site.
2. Site. Create the Site under the Customer, with Site type xCore, enable API tokens on it and create a token, as described in Connect the first Site. Note the Site's numeric ID and keep the token secret: they become agent.connections.siteId and agent.apiToken.
3. Systems. Create one System per anchor user plane, under the Customer, on the Site, with the anchor's UE address pool as its network. Note each System's numeric ID: it becomes the anchor's systemId.
4. APN and System Profile. Make sure the APN the Mobile Assets use exists and matches a data network of the Site (enterprise in the example), and that a System Profile uses it.
5. Keyset. Create a Keyset for the SIM batch. Set K4 Field Type to what the SIM vendor encrypted under the transport key: K Only, or K And OPC when the OPc is encrypted too.
6. Keyset Mapping. Open the Site, click Edit and select the Keyset Mappings tab. Switch Enable on, click Add Keyset Mapping, select the Keyset and enter an HSS Identifier: a whole number, 1 for the first batch. The HSS Identifier is the name the core's subscriber store will know the batch's transport key by in step 6. Identifier 0 is reserved for Mobile Assets whose K is not encrypted; do not map a Keyset to it.

Step 3: install the operator⚓︎

Install the xCore chart into its own namespace with the Site ID, xControl's address and the API token, as described in Install the operator:

helm install xcore expeto/xfabric -n xcore --create-namespace -f values.yaml \
    --set global.imageCredentials.username='<username>' \
    --set global.imageCredentials.password='<password>' \
    --set agent.apiToken='<site API token>'
kubectl -n xcore rollout status deployment/xcore-xfabric
helm test xcore -n xcore --filter name=xcore-xfabric-test-preflight --logs

The preflight test must report every required prerequisite as satisfied before you continue.

Step 4: apply the Site⚓︎

Write the Site from the combined core example with your addresses, PLMN, identities, customerId and systemId values, agree it with Expeto Support, then apply it:

kubectl -n xcore create secret generic ignite-credentials --from-literal=password='<password>'
kubectl -n xcore apply -f site.yaml
kubectl -n xcore wait --for=condition=Ready --timeout=15m sites.expeto.io/acmecorp-hq

Ready turns True once the Agent, every user plane and every control-plane function (the SMF, the AMF, the MME and the subscriber store) report ready. If it does not, kubectl describe site names the cause; see Troubleshooting.

Step 5: verify the installation⚓︎

Work through Verify the installation: the Site's conditions, the workloads, the preflight check, and the Site showing Connected in xControl. A combined core runs, in addition to the shared functions, the AMF's three mm-* Deployments, the mme-mme and mme-stc Deployments with the mme-cache store, and the sdm-hss, sdm-udr, sdm-udm and sdm-ausf subscriber store with its sdm-cache.

Step 6: load the subscriber-store keys⚓︎

The subscriber store needs two kinds of key that xControl never holds, and they are loaded on the Site itself, once per Site, through the Expeto CLI. Treat the loading as a small key ceremony.

Transport key. The SIM vendor encrypts each SIM's K, and usually its OPc, under the batch's transport key before delivering the batch. xControl stores the encrypted values as delivered, and the Agent provisions them into the HSS and the UDR with the HSS Identifier of the Keyset Mapping. The HSS and the UDR decrypt them with the transport key stored under that identifier. A combined core needs the transport key in both: the HSS authenticates the SIMs on 4G and the UDR on 5G. The operator variant (OP) is not loaded: Mobile Assets carry their OPc.

SUCI private key. A 5G device conceals its identity in a Subscription Concealed Identifier (SUCI), encrypted with the home network's public key. The UDM de-conceals it with the matching private key, which the UDR stores per PLMN under the key identifier the SIMs carry. Without it, a 5G device whose SIM conceals its identity cannot register.

Prepare the ceremony⚓︎

• Two people take part: a key custodian who enters each key, and a second person who witnesses and checks every step.
• Use a workstation the platform team approves, with kubectl access to the core's namespace. Do not share or record the screen.
• Bring the keys on the medium your security policy prescribes, as hexadecimal strings: 32 digits for an AES-128 transport key, 64 digits for a SUCI private key.
• Have the HSS Identifier of each Keyset Mapping and the key identifier of each SUCI key at hand.

• For each SUCI key, the custodian computes its fingerprint on the offline machine before the ceremony, without displaying the key, and brings the fingerprint. The fingerprint is taken over the key exactly as it is entered, in lowercase hexadecimal, which is how this pipeline prints it:

  openssl pkey -in hn-key-1.pem -noout -text \
      | awk '/^priv:/{f=1;next} /^pub:/{f=0} f' | tr -d ' :\n' | sha1sum
  

The CLI never takes a key on its command line. It prompts for every key and masks the input, so a key does not reach the shell history or the screen. The confirmation it prints after an add, and the detail view of a single keyset (read keyset name=<n>, read udr-keyset name=<n>), can show the stored values; verify with the list views shown below instead, and clear the terminal when you finish.

Load the transport key into the HSS⚓︎

Open the CLI in the Agent pod:

kubectl -n xcore exec -it xcore-xfabric-agent-0 -c agent-cli -- cli

Move to the 4G folder and list the keysets the HSS holds. The record 0, without a transport key, is created by the Agent for Mobile Assets without a Keyset.

expeto_shell>cd /lte
expeto_shell>lte>read keyset

Add the transport key under the HSS Identifier, here 1. The CLI prompts for the transport key and for an OP; enter the transport key, and press Enter at the OP prompt to leave it empty:

expeto_shell>lte>add keyset name=1 encryption=AES_128
please input transport key: ********************************
please input OP:

encryption names the algorithm the transport key is used with. AES_128, the default, matches a 32-digit key. For a SIM batch encrypted with another algorithm, ask Expeto Support.

Load the transport key into the UDR⚓︎

Move to the 5G folder and add the same transport key under the same identifier, again leaving the OP empty:

expeto_shell>lte>cd /nr
expeto_shell>nr>add udr-keyset name=1 encryption=AES_128
please input transport key: ********************************
please input OP:

Load the SUCI private key into the UDR⚓︎

Add the private key under the key identifier the SIMs carry, here 1. Without plmn, the key is stored for every home PLMN the store serves, which on a Site is its plmnId; plmn=<MCC and MNC> stores it for one PLMN only.

expeto_shell>nr>add suci pkId=1
SUCI key ('openssl ec -in <pem file> -text -noout' and then take 'priv:' hex value without colon and new-lines): ****************************************************************

Verify and close⚓︎

List what each store holds:

expeto_shell>nr>read udr-keyset
expeto_shell>nr>read suci
expeto_shell>nr>cd /lte
expeto_shell>lte>read keyset

• read udr-keyset lists each keyset with Transport Key Yes and OP No.
• read suci lists each key identifier with its PLMN and the SHA-1 fingerprint of the private key. The witness compares the fingerprint with the one the custodian brought.
• read keyset lists the HSS keysets by name: 0 and each HSS Identifier you added.

Type exit, then clear the terminal and its scrollback.

To replace a key later, for example when a new SIM batch arrives, use update keyset, update udr-keyset or update suci with the same arguments; delete keyset name=<n>, delete udr-keyset name=<n> and delete suci pkId=<n> remove one. Every Site of the Customer that serves the batch needs the same transport key under the HSS Identifier its own Keyset Mapping uses.

Generate a SUCI key pair⚓︎

The home network key pair is generated once, on an offline machine, by the key custodians. This guide uses protection scheme Profile B of TS 33.501 (NIST P-256), the scheme the CLI's add suci is written for; ask Expeto Support before personalizing SIMs with Profile A.

openssl genpkey -algorithm EC -pkeyopt ec_paramgen_curve:P-256 -out hn-key-1.pem
openssl ec -in hn-key-1.pem -pubout -outform DER -conv_form compressed 2>/dev/null \
    | tail -c 33 | xxd -p -c 66

The second command prints the public key in its compressed form, 33 bytes: give it to the SIM vendor together with the key identifier (1 here, 1 to 255) and the protection scheme. The private key stays with the key custodians; the hexadecimal string the CLI asks for is the priv: value of openssl ec -in hn-key-1.pem -noout -text, without colons and line breaks (the pipeline in Prepare the ceremony produces it).

Step 7: provision the Mobile Assets⚓︎

Import the SIM batch into xControl with a bulk import of the vendor's .out file: select the Customer, one of the Systems, the System Profile and the Keyset you mapped in step 2. Mobile Assets added one at a time on the Mobile Assets page take the encrypted K and OPc as delivered, with the same Keyset.

The Agent provisions each Mobile Asset into the HSS and the UDR. Check one in the CLI:

expeto_shell>cd /lte
expeto_shell>lte>read hss <imsi>
expeto_shell>lte>cd /nr
expeto_shell>nr>read udr <imsi>

A Mobile Asset that does not appear in either store is reported by the Agent's synchronisation check; in the CLI, cd /debug/ops and read sync-sub-error list the failures.

Step 8: connect the radios and attach⚓︎

Point the eNodeBs at the MME's S1-MME address (SCTP port 36412) and the gNodeBs at the AMF's N2 address (SCTP port 38412). Both send GTP-U to the front user plane's N3 address. In the CLI, cd /lte and read enodeb list the connected eNodeBs, and cd /nr and read ran the gNodeBs.

Then attach one device on each radio technology:

1. Attach a device on 4G. In the CLI, cd /lte and read mme -f imsi=<imsi> show its attach, and cd /nr and read smf -f imsi=<imsi> its data session with the address from the System's pool. The Mobile Asset shows the session in xControl.
2. Register a device on 5G. cd /nr, read amf <imsi> shows its registration and read smf -f imsi=<imsi> its PDU session.
3. Move the 4G device to 5G coverage. It registers on 5G and keeps its address.

A device rejected at authentication usually points at the keys: a transport key that differs from the SIM vendor's, a Keyset Mapping whose HSS Identifier differs from the keyset name you loaded, or, on 5G only, a missing SUCI key or a key identifier that differs from the SIMs'.

Step 9: hand over⚓︎

Run the installation and upgrade verification set before the core is handed over, and keep the Site definition and the chart values under version control for the upgrades to come.