From a402ae6d8d0748d1fec855ae335eeb25e5652e6c Mon Sep 17 00:00:00 2001 From: "Arrobo, Gabriel" Date: Fri, 30 Oct 2026 22:04:23 -0700 Subject: [PATCH 1/2] Bump SD-Core and BESS-UPF Helm chart versions to 6.1.0 Signed-off-by: Arrobo, Gabriel --- onramp/blueprints.rst | 2 +- onramp/devel.rst | 6 ++-- onramp/gnb.rst | 2 +- onramp/network.rst | 2 +- onramp/start.rst | 80 ++++++++++++++++++++++++------------------- 5 files changed, 50 insertions(+), 42 deletions(-) diff --git a/onramp/blueprints.rst b/onramp/blueprints.rst index c22a90a..2f436fa 100644 --- a/onramp/blueprints.rst +++ b/onramp/blueprints.rst @@ -116,7 +116,7 @@ of ``vars/main.yml``: helm: local_charts: false chart_ref: oci://ghcr.io/omec-project/bess-upf - chart_version: 6.0.0 + chart_version: 6.1.0 values_file: "deps/5gc/roles/upf/templates/upf-5g-values.yaml" default_upf: ip: diff --git a/onramp/devel.rst b/onramp/devel.rst index 7eb35ab..5f79d46 100644 --- a/onramp/devel.rst +++ b/onramp/devel.rst @@ -41,7 +41,7 @@ block of the ``core`` section of ``vars/main.yml`` to replace: helm: local_charts: false chart_ref: oci://ghcr.io/omec-project/sd-core - chart_version: 6.0.0 + chart_version: 6.1.0 with @@ -50,10 +50,10 @@ with helm: local_charts: true chart_ref: "/home/ubuntu/aether/sdcore-helm-charts/sdcore-helm-charts" - chart_version: 6.0.0 + chart_version: 6.1.0 Note that variable ``core.helm.local_charts`` is a boolean, and in -this example, the configured chart version stays ``6.0.0``. When +this example, the configured chart version stays ``6.1.0``. When ``local_charts`` is set to ``true``, ``chart_version`` is ignored by the playbook and the local chart contents are used directly. diff --git a/onramp/gnb.rst b/onramp/gnb.rst index b09c6ce..2a49a55 100644 --- a/onramp/gnb.rst +++ b/onramp/gnb.rst @@ -79,7 +79,7 @@ using. helm: local_charts: false # set chart_ref to local path name if true chart_ref: oci://ghcr.io/omec-project/sd-core - chart_version: 6.0.0 + chart_version: 6.1.0 upf: access_subnet: "192.168.252.1/24" # access subnet & gateway core_subnet: "192.168.250.1/24" # core subnet & gateway diff --git a/onramp/network.rst b/onramp/network.rst index 74ff97c..df70d78 100644 --- a/onramp/network.rst +++ b/onramp/network.rst @@ -221,7 +221,7 @@ sections, but for a summary, see the :doc:`Quick Reference `. helm: local_charts: false # set chart_ref to local path name if true chart_ref: oci://ghcr.io/omec-project/sd-core - chart_version: 6.0.0 + chart_version: 6.1.0 upf: access_subnet: "192.168.252.1/24" # access subnet & gateway core_subnet: "192.168.250.1/24" # core subnet & gateway diff --git a/onramp/start.rst b/onramp/start.rst index ff2ba12..a03ad72 100644 --- a/onramp/start.rst +++ b/onramp/start.rst @@ -325,29 +325,29 @@ the following: .. code-block:: $ kubectl get pods --all-namespaces - NAMESPACE NAME READY STATUS RESTARTS AGE - kube-system cloud-controller-manager-node1 1/1 Running 0 74m - kube-system etcd-node1 1/1 Running 0 75m - kube-system helm-install-rke2-canal-kg4x2 0/1 Completed 0 74m - kube-system helm-install-rke2-coredns-rcdwb 0/1 Completed 0 74m - kube-system helm-install-rke2-metrics-server-n2cxf 0/1 Completed 0 74m - kube-system helm-install-rke2-multus-zfdhg 0/1 Completed 0 74m - kube-system helm-install-rke2-runtimeclasses-nbcnc 0/1 Completed 0 74m - kube-system helm-install-rke2-snapshot-controller-crd-jts65 0/1 Completed 0 74m - kube-system helm-install-rke2-snapshot-controller-mcfm8 0/1 Completed 0 74m - kube-system helm-install-rke2-traefik-crd-vlbkm 0/1 Completed 0 74m - kube-system helm-install-rke2-traefik-hqzkl 0/1 Completed 0 74m - kube-system kube-apiserver-node1 1/1 Running 0 75m - kube-system kube-controller-manager-node1 1/1 Running 0 75m - kube-system kube-proxy-node1 1/1 Running 0 74m - kube-system kube-scheduler-node1 1/1 Running 0 75m - kube-system rke2-canal-rdk8p 2/2 Running 0 74m - kube-system rke2-coredns-rke2-coredns-6b85489767-5294x 1/1 Running 0 74m - kube-system rke2-coredns-rke2-coredns-autoscaler-8fcd75dfd-7p44m 1/1 Running 0 74m - kube-system rke2-metrics-server-9d9d8b8f-jtjnd 1/1 Running 0 72m - kube-system rke2-multus-cbk7n 1/1 Running 0 72m - kube-system rke2-snapshot-controller-5d85999559-cb682 1/1 Running 0 72m - kube-system rke2-traefik-jpkrj 1/1 Running 0 72m + NAMESPACE NAME READY STATUS RESTARTS AGE + kube-system cloud-controller-manager-node1 1/1 Running 0 4d6h + kube-system etcd-node1 1/1 Running 0 4d6h + kube-system helm-install-rke2-canal-tswmm 0/1 Completed 0 4d6h + kube-system helm-install-rke2-coredns-qksdc 0/1 Completed 0 4d6h + kube-system helm-install-rke2-metrics-server-vlvth 0/1 Completed 0 4d6h + kube-system helm-install-rke2-multus-htszr 0/1 Completed 0 4d6h + kube-system helm-install-rke2-runtimeclasses-mqg2c 0/1 Completed 0 4d6h + kube-system helm-install-rke2-snapshot-controller-crd-9xftw 0/1 Completed 0 4d6h + kube-system helm-install-rke2-snapshot-controller-dxn6s 0/1 Completed 1 (4d6h ago) 4d6h + kube-system helm-install-rke2-traefik-crd-4nljp 0/1 Completed 0 4d6h + kube-system helm-install-rke2-traefik-t6q65 0/1 Completed 2 (4d6h ago) 4d6h + kube-system kube-apiserver-node1 1/1 Running 0 4d6h + kube-system kube-controller-manager-node1 1/1 Running 0 4d6h + kube-system kube-proxy-node1 1/1 Running 0 4d6h + kube-system kube-scheduler-node1 1/1 Running 0 4d6h + kube-system rke2-canal-nkcv7 2/2 Running 0 4d6h + kube-system rke2-coredns-rke2-coredns-6b85489767-v8jfs 1/1 Running 0 4d6h + kube-system rke2-coredns-rke2-coredns-autoscaler-8fcd75dfd-tzltc 1/1 Running 0 4d6h + kube-system rke2-metrics-server-9d9d8b8f-vg7lw 1/1 Running 0 4d6h + kube-system rke2-multus-x6xk4 1/1 Running 0 4d6h + kube-system rke2-snapshot-controller-5d85999559-kc4ld 1/1 Running 0 4d6h + kube-system rke2-traefik-4td4d 1/1 Running 0 4d6h If you are interested in seeing the details about how Kubernetes is customized for Aether, look at @@ -386,19 +386,27 @@ versions installed. .. code-block:: $ kubectl get pods -n aether-5gc - NAME READY STATUS RESTARTS AGE - amf-5887bbf6c5-pc9g2 1/1 Running 0 6m13s - ausf-6dbb7655c7-42z7m 1/1 Running 0 6m13s - metricfunc-b9f8c667b-r2x9g 1/1 Running 0 6m13s - nrf-54bf88c78c-kcm7t 1/1 Running 0 6m13s - nssf-5b85b8978d-d29jm 1/1 Running 0 6m13s - pcf-758d7cfb48-dwz9x 1/1 Running 0 6m13s - simapp-6cccd6f787-jnxc7 1/1 Running 0 6m13s - smf-7f89c6d849-wzqvx 1/1 Running 0 6m13s - udm-768b9987b4-9qz4p 1/1 Running 0 6m13s - udr-8566897d45-kv6zd 1/1 Running 0 6m13s - upf-0 5/5 Running 0 6m13s - webui-5894ffd49d-gg2jh 1/1 Running 0 6m13s + NAME READY STATUS RESTARTS AGE + af-794f6bc999-8z9td 1/1 Running 0 7h43m + amf-5d84ccb587-x8b2r 1/1 Running 0 7h43m + ausf-57c665bd98-r9t9x 1/1 Running 0 7h43m + kafka-dual-0 1/1 Running 0 7h43m + kafka-entity-operator-76764c7dfc-mptj8 2/2 Running 0 7h43m + metricfunc-6bdb886f6d-jmn8s 1/1 Running 0 7h43m + mongodb-0 2/2 Running 0 7h43m + mongodb-1 2/2 Running 0 7h43m + mongodb-arb-0 2/2 Running 0 7h43m + mongodb-kubernetes-operator-7989c65684-bvlxw 1/1 Running 0 7h44m + nrf-5b76f8cbdc-fb8ws 1/1 Running 0 7h43m + nssf-749598c758-lwkzm 1/1 Running 0 7h43m + pcf-6c77f966d5-j86p9 1/1 Running 0 7h43m + simapp-6549f79d79-rh6dq 1/1 Running 0 7h43m + smf-5f7bcc456b-95567 1/1 Running 0 7h43m + strimzi-cluster-operator-7c667c9ccb-7xtcn 1/1 Running 0 7h44m + udm-5bc879dc6c-4dqhf 1/1 Running 0 7h43m + udr-59f87695b4-sbs4s 1/1 Running 0 7h43m + upf-0 5/5 Running 0 7h43m + webui-76c549747b-6xv2r 1/1 Running 0 7h43m If you see problematic pods that are not getting into the ``Running`` state, a reset usually corrects the problem. Type: From de36cf1b9f73d888d251d546488b5e8af42de3cb Mon Sep 17 00:00:00 2001 From: "Arrobo, Gabriel" Date: Fri, 30 Oct 2026 22:12:57 -0700 Subject: [PATCH 2/2] Align UPF networks with latest OnRamp changes Signed-off-by: Arrobo, Gabriel --- onramp/blueprints.rst | 133 ++++++++++++++++++++++++------------------ onramp/gnb.rst | 29 +++++---- onramp/network.rst | 125 ++++++++++++++++++++++++++++++++++----- onramp/ref.rst | 65 ++++++++++++--------- onramp/start.rst | 10 ++-- 5 files changed, 246 insertions(+), 116 deletions(-) diff --git a/onramp/blueprints.rst b/onramp/blueprints.rst index 2f436fa..e8f6e0c 100644 --- a/onramp/blueprints.rst +++ b/onramp/blueprints.rst @@ -111,41 +111,42 @@ of ``vars/main.yml``: .. code-block:: upf: - access_subnet: "192.168.252.1/24" # access subnet & gateway - core_subnet: "192.168.250.1/24" # core subnet & gateway helm: local_charts: false chart_ref: oci://ghcr.io/omec-project/bess-upf chart_version: 6.1.0 values_file: "deps/5gc/roles/upf/templates/upf-5g-values.yaml" - default_upf: - ip: - access: "192.168.252.3" - core: "192.168.250.3" - ue_ip_pool: "192.168.100.0/24" - additional_upfs: + instances: + default: + ue_ip_pool: "192.168.100.0/24" "1": - ip: - access: "192.168.252.6" - core: "192.168.250.6" + access: { ip: "192.168.252.6" } + core: { ip: "192.168.250.6" } ue_ip_pool: "172.248.0.0/16" - # "2": - # ip: - # access: "192.168.252.7" - # core: "192.168.250.7" - # ue_ip_pool: "172.247.0.0/16" - -As shown above, one additional UPF is enabled (beyond ``upf-0`` that -already came up as part of SD-Core), with the spec for yet another UPF -commented out. In this example configuration, each UPF is assigned a -subnet on the ``access`` and ``core`` bridges, along with the IP -address pool for UEs that the UPF serves. To launch this second UPF, + +Each entry in ``instances`` is a UPF. The ``default`` instance is the +built-in UPF that comes up as part of SD-Core, and every other +instance (``"1"`` above) is deployed by ``5gc-upf-install`` as its own +release, in namespace ``aether-upf-``. In this example +configuration, each UPF is assigned an address on the ``access`` and +``core`` networks (which default to the same subnets used by the +``default`` UPF), along with the IP address pool for UEs that the UPF +serves. Additional instances can be added by following the same +pattern, each with its own addresses and ``ue_ip_pool``; a UPF can +also be attached to different networks, as described in +:doc:`Network Details `. To launch this second UPF, type: .. code-block:: $ make 5gc-upf-install +(Alternatively, ``make aether-add-upfs`` first runs the router step, +which builds any network the new instance needs, and then installs the +UPF. Each additional UPF also gets a systemd unit, +``aether-upf--routes.service``, carrying its UE pool route and +NAT.) + At this point the new UPF(s) will be running in their own namespaces (you can verify this using ``kubectl get pods --all-namespaces``), but no traffic will be directed to them until UEs are assigned to their IP @@ -472,9 +473,9 @@ Returning to the OnRamp blueprint, it includes the following: core steps needed for the DPDK blueprint. * When ``core.upf.mode`` is set to ``dpdk``, the 5GC core and - additional UPF install roles validate that the selected - ``core.data_iface`` has at least two VFs configured before deploying - the UPF workload. + additional UPF install roles validate that the interface the UPF's + ``access`` and ``core`` networks sit on has at least two VFs bound to + the ``vfio-pci`` driver before deploying the UPF workload. * Integration tests require SR-IOV capable servers, and so have not been automated as part of the integration test suite. @@ -492,12 +493,13 @@ section: .. code-block:: upf: - access_subnet: "192.168.252.1/24" # access subnet & gateway - core_subnet: "192.168.250.1/24" # core subnet & gateway mode: dpdk # Options: af_packet or dpdk # If mode set to 'dpdk': - # - make sure at least two VF devices are created out of 'data_iface' - # and these devices are attached to vfio-pci driver. + # - make sure at least two VF devices are created out of the interface + # the access and core networks sit on, bound to the vfio-pci driver. + # - the SR-IOV pools default to intel.com/intel_sriov_vfio_access and + # intel.com/intel_sriov_vfio_core; override them with + # core.networks.access.resource_name / core.networks.core.resource_name. Note the VF device requirement in the ``upf`` block comments. The ``core`` block continues to point at the standard SD-Core values file, @@ -518,8 +520,18 @@ Make targets: The ``5gc-install`` target runs the router step before deploying the core. When ``core.upf.mode`` is set to ``dpdk``, OnRamp renders the UPF -for SR-IOV/DPDK and checks that ``core.data_iface`` has at least two -configured VFs before deploying the workload. +for SR-IOV/DPDK and checks that the interface the UPF's networks sit +on has at least two VFs bound to ``vfio-pci`` before deploying the +workload. + +Two details of the SR-IOV resource pools are worth knowing. If the +``access`` and ``core`` networks' ``resource_name`` values are +different (the default), the chart pins the access and core sides to +``vf 0`` and ``vf 1`` specifically, so those two VFs must be bound to +``vfio-pci`` and have non-zero MAC addresses. If the two values are the +same, both sides share one pool exposing every ``vfio-pci``-bound VF on +the interface, so *all* of those VFs must have non-zero MAC addresses. +In ``dpdk`` mode both sides of a UPF must be on the same interface. OAI 5G RAN @@ -638,8 +650,8 @@ parameters`` section need to be modified to work with the Aether Core: The ``core`` section of ``vars/main.yml`` is similar to that used in other blueprints, with two variable settings of note. First, -``ran_subnet`` is set to ``"172.20.0.0/16"`` and not the empty string -(``""``). As a general rule, ``core.ran_subnet`` is set to the empty +``networks.access.ran_subnet`` is set to ``"172.20.0.0/16"`` and not the empty string +(``""``). As a general rule, ``core.networks.access.ran_subnet`` is set to the empty string whenever a physical gNB is on the same L2 network as the Core, but in the case of an OAI-based gNB, the RAN stack runs in a Macvlan-connected Docker container, and so the variable is set to @@ -790,8 +802,8 @@ as many servers under ``srsran.servers`` section. The ``core`` section of ``vars/main.yml`` is similar to that used in other blueprints, with two variable settings of note. First, -set ``ran_subnet`` to proper ran subnet as per your setup. -As a general rule, ``core.ran_subnet`` is set to the empty(``""``) +set ``networks.access.ran_subnet`` to proper ran subnet as per your setup. +As a general rule, ``core.networks.access.ran_subnet`` is set to the empty(``""``) string whenever a physical gNB is on the same L2 network as the Core. Second, variable ``values_file`` is set to @@ -935,8 +947,8 @@ many servers under ``ocudu.servers`` section. The ``core`` section of ``vars/main.yml`` is similar to that used in other blueprints, with two variable settings of note. First, -set ``ran_subnet`` to the correct RAN subnet for your deployment. -As a general rule, ``core.ran_subnet`` is set to the empty (``""``) +set ``networks.access.ran_subnet`` to the correct RAN subnet for your deployment. +As a general rule, ``core.networks.access.ran_subnet`` is set to the empty (``""``) string whenever a physical gNB is on the same L2 network as the Core. Second, variable ``values_file`` is set to @@ -965,9 +977,9 @@ To deploy the OCUDU blueprint in simulation mode, run the following: When ``ocudu.simulation`` is set to ``true``, target ``ocudu-uesim-start`` starts the simulated srsRAN UE, installs its rendered ``ue_conf`` file, configures a default route inside the UE -network namespace based on ``core.upf.default_upf.ue_ip_pool``, and -then runs a ping test toward the Core subnet derived from -``core.upf.core_subnet``. In other words, this step does more than +network namespace based on +``core.upf.instances.default.ue_ip_pool``, and then runs a ping test +toward the Core subnet derived from ``core.networks.core.subnet``. In other words, this step does more than launch the UE; it also performs a basic end-to-end connectivity check. To deploy the OCUDU blueprint with a USRP and physical UE, set @@ -1102,7 +1114,7 @@ configuration file that includes parameters such as PLMN, TAC, etc. The ``core`` section of ``vars/main.yml`` is similar to that used in other blueprints, with two variable settings of note. First, -set ``ran_subnet`` to proper ran subnet as per your setup. +set ``networks.access.ran_subnet`` to proper ran subnet as per your setup. Second, variable ``values_file`` is set to ``"deps/5gc/roles/core/templates/sdcore-5g-values.yaml"`` by default, @@ -1138,25 +1150,32 @@ how to override this setting. It is not technically a self-contained blueprint, but rather, a configuration option that can be applied to any of the blueprints defined in this section. -The override requires setting variable ``core.upf.multihop_gnb`` to -``true``. This causes OnRamp to configure the UPF's N3 interface from -the same subnet as ``core.data_iface``. For example, suppose -``core.data_iface`` corresponds to subnet 10.21.61.0/24 and the gNB is -on subnet 10.202.1.0/24. Configure the parameters as follows: +The override replaces the ``access`` network with one on the server's +own subnet whose gateway is an external router, rather than this +server. This is done by setting ``core.networks.access.gateway`` to +the router's address (the default, ``host``, makes the server the +gateway on an isolated subnet). The UPF's N3 interface is then +configured from the same subnet as the server's interface, which is +the one carrying the default route unless ``core.networks.access.iface`` +says otherwise. For example, suppose the server's interface +corresponds to subnet 10.21.61.0/24, whose router is 10.21.61.1, and +the gNB is on subnet 10.202.1.0/24. Configure the parameters as +follows: .. code-block:: - data_iface: ens18 - ran_subnet: "10.202.1.0/24" - upf: - access_subnet: "10.21.61.1/24" # access subnet & gateway - core_subnet: "192.168.250.1/24" # core subnet & gateway - multihop_gnb: true # N3 directly reachable via data_iface - default_upf: - ip: - access: "10.21.61.12" # same subnet as data_iface when multihop_gnb=true - core: "192.168.250.3" - ue_ip_pool: "192.168.100.0/24" + core: + networks: + access: + subnet: "10.21.61.0/24" + gateway: "10.21.61.1" # the router's address; OnRamp configures nothing on the host + ran_subnet: "10.202.1.0/24" # the gNB network + upf: + instances: + default: + access: + ip: "10.21.61.12" # same subnet as the server's interface + ue_ip_pool: "192.168.100.0/24" To connect multiple gNBs on different subnets, you must also modify the specified values file (e.g., diff --git a/onramp/gnb.rst b/onramp/gnb.rst index 2a49a55..7336f7e 100644 --- a/onramp/gnb.rst +++ b/onramp/gnb.rst @@ -62,33 +62,32 @@ Modify Configuration ~~~~~~~~~~~~~~~~~~~~~~~~ Modify the ``core`` section of ``vars/main.yml`` to match the -following, substituting your local details for ``ens18`` and -``10.76.28.113``. Of particular note, setting ``ran_subnet`` to the -empty string indicates that the gNB is connected to the same physical -L2 network as your Aether cluster, and the new ``values_file`` is -tailored for a physical RAN rather than the emulated RAN we've been -using. +following, substituting your server's IP address for +``10.76.28.113``. Of particular note, setting +``networks.access.ran_subnet`` to the empty string indicates that the +gNB is connected to the same physical L2 network as your Aether +cluster, and the new ``values_file`` is tailored for a physical RAN +rather than the emulated RAN we've been using. The Core finds its +network interface from the server's default route, so there is no +interface name to substitute in the ``core`` section. .. code-block:: core: standalone: true # set to false to manage device groups/slices manually via the SD-Core webui - data_iface: ens18 values_file: "deps/5gc/roles/core/templates/sdcore-5g-values.yaml" - ran_subnet: "" # set to empty string to get subnet from 'data_iface' + networks: + access: + ran_subnet: "" # set to empty string to get subnet from the access interface helm: local_charts: false # set chart_ref to local path name if true chart_ref: oci://ghcr.io/omec-project/sd-core chart_version: 6.1.0 upf: - access_subnet: "192.168.252.1/24" # access subnet & gateway - core_subnet: "192.168.250.1/24" # core subnet & gateway mode: af_packet # Options: af_packet or dpdk - default_upf: - ip: - access: "192.168.252.3" - core: "192.168.250.3" - ue_ip_pool: "192.168.100.0/24" + instances: + default: + ue_ip_pool: "192.168.100.0/24" amf: ip: "10.76.28.113" diff --git a/onramp/network.rst b/onramp/network.rst index df70d78..a678821 100644 --- a/onramp/network.rst +++ b/onramp/network.rst @@ -206,31 +206,29 @@ with the container manged by Docker, independent of Kubernetes). Finally, all of the configurable parameters used throughout this section are defined in the ``core`` and ``gnbsim`` sections of the ``vars/main.yml`` file. Note that an empty value for -``core.ran_subnet`` implies the physical L2 network is used to connect -RAN elements to the core, as is typically the case when connecting -physical gNBs. Many of the other variables are explained in subsequent -sections, but for a summary, see the :doc:`Quick Reference `. +``core.networks.access.ran_subnet`` implies the physical L2 network is +used to connect RAN elements to the core, as is typically the case +when connecting physical gNBs. Many of the other variables are +explained in subsequent sections, but for a summary, see the +:doc:`Quick Reference `. .. code-block:: core: standalone: true # set to false to manage device groups/slices manually via the SD-Core webui - data_iface: ens18 values_file: "deps/5gc/roles/core/templates/sdcore-5g-values.yaml" - ran_subnet: "172.20.0.0/16" # set to empty string to get subnet from 'data_iface' + networks: + access: + ran_subnet: "172.20.0.0/16" # gNB network; "" derives it from the access interface helm: local_charts: false # set chart_ref to local path name if true chart_ref: oci://ghcr.io/omec-project/sd-core chart_version: 6.1.0 upf: - access_subnet: "192.168.252.1/24" # access subnet & gateway - core_subnet: "192.168.250.1/24" # core subnet & gateway mode: af_packet # Options: af_packet or dpdk - default_upf: - ip: - access: "192.168.252.3" - core: "192.168.250.3" - ue_ip_pool: "192.168.100.0/24" + instances: + default: + ue_ip_pool: "192.168.100.0/24" amf: ip: "10.76.28.113" @@ -242,6 +240,107 @@ sections, but for a summary, see the :doc:`Quick Reference `. iface: gnbaccess subnet_prefix: "172.20" +The ``core`` section no longer names a data interface: the Core uses +the interface carrying the server's default route. Only +``gnbsim.router.data_iface`` needs to be set to your server's +interface. Everything else about the UPF's networking is optional, +as described next. + +UPF Networking +~~~~~~~~~~~~~~~~~ + +The UPF has two sides: *access* (N3, towards the gNBs) and *core* (N6, +towards the data network). Each side attaches to a *network*, which +is described by ``core.networks``. The list of UPFs, and the network +each of their sides uses, is described by ``core.upf.instances``. +Every field has a default, so with neither key set you get the +deployment OnRamp has always shipped: one UPF, both sides attached as +Macvlan children of the interface carrying the server's default route, +and the server acting as the gateway on ``192.168.252.0/24`` (access) +and ``192.168.250.0/24`` (core). The install prints the resolved UPF +networking so you can see what the defaults decided before anything is +deployed. + +Each entry in ``core.networks`` (keyed by name) has the following +fields, all optional: + +.. list-table:: + :widths: 15 30 55 + :header-rows: 1 + + * - Field + - Default + - Description + * - ``iface`` + - Interface carrying the default route + - Host interface the network is reached through. + * - ``vlan`` + - None + - 802.1Q tag. The UPF attaches to ``.``, which + ``5gc-router-install`` creates on the host. + * - ``subnet`` + - ``192.168.252.0/24`` for ``access``, ``192.168.250.0/24`` for + ``core`` + - The network in CIDR form. Required for any other network. + * - ``gateway`` + - ``host`` + - ``host`` makes the server the gateway, at the subnet's first + address: it creates a Macvlan interface named after the network, + forwards between it and ``iface``, routes each UPF's UE pool to + that UPF, and masquerades the pools on the way out. An IP + address names an external router instead, in which case OnRamp + configures nothing on the host for this network. + * - ``attach`` + - ``macvlan`` + - How the UPF pod attaches in ``af_packet`` mode. ``macvlan`` + shares the interface; ``host-device`` moves it into the pod, + which requires an external gateway and makes the interface + absent from the host while the UPF runs. + * - ``resource_name`` + - ``intel.com/intel_sriov_vfio_access`` / ``..._core`` + - SR-IOV resource pool in ``dpdk`` mode. Required for networks + other than ``access`` and ``core``. + * - ``ran_subnet`` + - Derived from ``iface``'s own subnet + - On an ``access`` network, the gNB network the UPF routes + downstream traffic to. Set it explicitly when the gNBs are not on + that interface's L2 network. + +Each entry in ``core.upf.instances`` (keyed by name) has ``access`` and +``core`` (each ``{network, ip}``, defaulting to the network of the same +name and the third address of its subnet), ``ue_ip_pool`` (required +except for ``default``, which uses ``192.168.100.0/24``), and ``mode`` +(defaults to ``core.upf.mode``). The instance named ``default`` is the +built-in UPF deployed with the Core and always exists. Every other +instance is deployed as its own release by ``5gc-upf-install``; see +the :doc:`Multiple UPFs ` blueprint. + +To put one side of the UPF on a different interface, name only what +differs. For example, the following moves the core side to a second NIC +whose router is not this server: + +.. code-block:: + + core: + networks: + core: + iface: eno2 + subnet: 10.60.0.0/24 + gateway: 10.60.0.1 + +The install refuses configurations that cannot work rather than +deploying them. Examples include a gateway outside its subnet, +``host-device`` with the server as gateway, moving the interface that +carries the server's default route into a pod, a network used by both a +``dpdk`` and an ``af_packet`` UPF, two UPFs sharing an address, +overlapping UE pools, and unknown (or removed) keys. In ``dpdk`` mode +both sides of a UPF must be on the same interface, and ``attach`` does +not apply. + +Uninstalling the router step restores the host to the state it was in +before the install (forwarding, GRO, and ``sysctl`` settings, plus any +VLAN interfaces OnRamp created). + Packet Traces ~~~~~~~~~~~~~~~~~ diff --git a/onramp/ref.rst b/onramp/ref.rst index 96f4102..5f9e120 100644 --- a/onramp/ref.rst +++ b/onramp/ref.rst @@ -82,28 +82,34 @@ the list is not comprehensive. * - Variable - Default - Description - * - `core.ran_subnet` + * - `core.networks.access.ran_subnet` - `172.20.0.0/16` - - Overlay subnet connecting Core to RAN when gNBs run in a container; set to empty string ("") when gNBs are directly connected via `core.data_iface`. + - Overlay subnet connecting Core to RAN when gNBs run in a container; set to empty string ("") to derive it from the access interface when gNBs are directly connected to the server's physical network. + * - `core.networks.` + - `access`, `core` (optional) + - Networks the UPF attaches to; fields are `iface`, `vlan`, `subnet`, `gateway`, `attach`, `resource_name` and `ran_subnet`. All have defaults, so omitting this key gives the single-interface deployment on the default-route interface. See :doc:`Network Details `. + * - `core.networks..resource_name` + - `intel.com/intel_sriov_vfio_access` (`access`), `intel.com/intel_sriov_vfio_core` (`core`) + - SR-IOV resource pool name in `dpdk` mode. + * - `core.upf.instances.` + - `default` (optional) + - UPFs, each with `access` and `core` (`{network, ip}`), `ue_ip_pool` and `mode`. `default` is the built-in UPF; every other instance is deployed by `5gc-upf-install`. * - `core.standalone` - `true` - Core to run standalone with simapp-managed subscribers; set to `false` to manage device groups/slices via SD-Core webui. - * - `core.data_iface` - - `ens18` - - Network interface used by UPF; same as `gnbsim.data_iface` when co-located on a single server. * - `core.amf.ip` - `"10.76.28.113"` - - IP address of AMF; edit to match IP address assigned to `core.data_iface`. + - IP address of AMF; edit to match IP address assigned to the server's network interface. + * - `core.amf.external_traffic_policy` + - unset + - Optional `externalTrafficPolicy` of the AMF N2 Service; `Cluster` or `Local`. * - `core.upf.mode` - `af_packet` - - UPF datapath mode; set to `dpdk` to enable the integrated SR-IOV/DPDK rendering in the core and additional-UPF templates. - * - `core.upf.multihop_gnb` - - `false` - - Override default N3 interface; set to `true` when external gNB is multiple hops away. - * - `gnbsim.data_iface` + - UPF datapath mode for instances that do not set their own; set to `dpdk` to enable the integrated SR-IOV/DPDK rendering in the core and additional-UPF templates. + * - `gnbsim.router.data_iface` - `ens18` - - Network interface used by gNBsim; same as `core.data_iface` when co-located on a single server. + - Network interface used by gNBsim; set to the server's interface (the Core derives its own interface from the default route). * - `oai.simulation` - `true` - Run UE in simulation mode; set to `false` to connect real UEs. @@ -424,7 +430,7 @@ Network Subnets OnRamp configures a set of subnets in support of a given Aether deployment. The following subnets are defined in ``vars/main.yml``. -With the exception of ``core.ran_subnet``, these variables typically +With the exception of ``core.networks.access.ran_subnet``, these variables typically do not need to be modified for an initial deployment of a blueprint. .. list-table:: @@ -435,25 +441,30 @@ do not need to be modified for an initial deployment of a blueprint. - Ansible Variable - Description * - `172.20.0.0/16` - - ``aether.ran_subnet`` + - ``core.networks.access.ran_subnet`` - Assigned to container-based gNBs connecting to the Core via an overlay subnet. Other gNB implementations connect to the Core - over the subnet assigned to the server's physical interface (as - defined by ``core.data_iface``). - * - `192.168.250.1/24` - - ``core.upf.core_subnet`` - - Gateway/interface CIDR assigned to the `core` bridge that connects UPF(s) to the Internet. - * - `192.168.252.1/24` - - ``core.upf.access_subnet`` - - Gateway/interface CIDR assigned to the `access` bridge that connects UPF(s) to the RAN. + over the subnet assigned to the server's physical interface + (the interface carrying the default route, unless overridden by + ``core.networks.access.iface``). + * - `192.168.250.0/24` + - ``core.networks.core.subnet`` + - Subnet of the `core` network that connects UPF(s) to the + Internet. By default the server is the gateway, at the first + address (`192.168.250.1`). + * - `192.168.252.0/24` + - ``core.networks.access.subnet`` + - Subnet of the `access` network that connects UPF(s) to the RAN. + By default the server is the gateway, at the first address + (`192.168.252.1`). * - `192.168.100.0/24` - - ``core.default_upf.ue_ip_pool`` + - ``core.upf.instances.default.ue_ip_pool`` - Assigned (by the Core) to UEs connecting to Aether. When - multiple UPFs are deployed—in addition to - ``core.default_upf``\ —each is assigned its own ``ue_ip_pool`` - subnet. + multiple UPFs are deployed—in addition to the ``default`` + instance—each is assigned its own ``ue_ip_pool`` subnet. * - `10.76.28.0/24` - N/A - Used throughout OnRamp documentation as an exemplar for the local subnet on which Aether severs and radios are deployed. - Corresponds to the network interface defined by variable ``core.data_iface``. + Corresponds to the server's network interface (the one carrying + the default route). diff --git a/onramp/start.rst b/onramp/start.rst index a03ad72..bbbf92f 100644 --- a/onramp/start.rst +++ b/onramp/start.rst @@ -194,15 +194,17 @@ adjust the location and filename of your private key) node1 ansible_host=10.76.28.113 ansible_user=aether ansible_ssh_private_key_file=~/.ssh/id_rsa -The second set of parameters is in ``vars/main.yml``, where the **two** lines -currently reading +The second set of parameters is in ``vars/main.yml``, where the line +(in the ``gnbsim`` section) currently reading .. code-block:: data_iface: ens18 -need to be edited to replace ``ens18`` with the device interface for -your server, and the line specifying the IP address of the Core's AMF +needs to be edited to replace ``ens18`` with the device interface for +your server. (The Core finds its own interface from the server's +default route, so there is no corresponding line in the ``core`` +section.) The line specifying the IP address of the Core's AMF needs to be edited to reflect your server's IP address: .. code-block::