Merge pull request #1849 from cncf/feature/1848-5gcategories

#1848 5g and RAN categories

CNF_TESTSUITE_YML_USAGE.md

@@ -177,3 +177,156 @@ For an image registry service named `foobar`, running in `default` namespace, on
 ```yaml
 docker_insecure_registries: ["foobar.default.svc.cluster.local:5000"]
 ```
+### RAN cofiguration
+
+#### `ric_label`
+
+The ran tests expect a ric to be configured under the ric_label.  The entry must be the k8s label which is most likely a full key/value identification.
+
+For a ric named `flexrric`, under the label key `app.kubernetes.io/name` the following would be the expected configuration.
+
+```yaml
+ric_label:  app.kubernetes.io/name=flexric
+```
+### Open5gs and UERANSIM configuration
+
+#### mmc
+
+Mobile Country Code. This identifies the country of the mobile subscriber. In this case, '999' is a test code.
+
+```yaml
+dmmc: '999
+```
+#### mnc
+
+Mobile Network Code. This identifies the mobile network within the country specified by the MCC. '70' is a test code.
+```yaml
+mnc: '70'
+```
+
+#### sst
+
+Single-NEC Single Radio Voice Call Continuity. This value indicates the type of services a Slice/Session should support.
+
+```yaml
+sst: 1
+```
+
+#### sd
+
+Slice Differentiator. This is used to differentiate between different slices within the same SST.
+
+```yaml
+sd: '0x111111'
+```
+
+#### tac
+
+Tracking Area Code. This is used for paging procedures and to manage mobility between eNBs in LTE.
+
+```yaml
+tac: '0001'
+```
+#### protectionScheme
+
+The type of security protocol being used.
+
+```yaml
+protectionScheme: 1
+```
+#### publicKey
+
+This is the public key used in asymmetric encryption.
+
+```yaml
+publicKey: 0ac95ceeb93308df01be82ff9994d8330e38804ece1700ee4b972d8028796275
+```
+
+#### publicKeyId
+
+Identifier for the public key.
+
+```yaml
+publicKeyId: 1:
+```
+
+#### routingIndicator
+
+This is used to route messages in the network.
+
+```yaml
+routingIndicator: '0000'
+```
+
+#### enabled
+
+Indicates whether the network is currently enabled or not.
+
+```yaml
+enabled: true
+```
+
+#### count
+
+Used in UERANSIM to specify the number of entities (like User Equipment or UEs) to be simulated.
+
+```yaml
+count: 1
+```
+
+#### initialMSISDN
+
+This MSISDN is a unique number that identifies a subscription in a GSM or a UMTS mobile network.
+
+```yaml
+initialMSISDN: '0000000001'
+```
+
+#### key
+
+Cryptographic key used in the network.
+
+```yaml
+key: 465B5CE8B199B49FAA5F0A2EE238A6BC:
+```
+
+#### op
+
+The operator variant algorithm configuration field. Used in conjunction with the key for security purposes.
+
+```yaml
+op: E8ED289DEBA952E4283B54E88E6183CA
+```
+
+#### opType
+
+Indicates that the operator variant algorithm is in use.
+
+```yaml
+opType: OPC
+```
+
+#### type
+
+The type of IP addresses being used in the network.
+
+```yaml
+type: 'IPv4'
+```
+
+#### apn
+
+Access Point Name. This is the name of a gateway between a GPRS, 3G or 4G mobile network and another computer network, frequently the public internet.
+
+```yaml
+apn: 'internet'
+```
+
+#### emergency:
+
+Indicates whether this is an emergency APN.
+
+```yaml
+emergency: false
+```
+

RATIONALE.md

@@ -290,7 +290,7 @@ In order to prevent illegitimate escalation by processes and restrict a processe
 #### Declarative APIs for an immutable infrastructure are anything that configures the infrastructure element. This declaration can come in the form of a YAML file or a script, as long as the configuration designates the desired outcome, not how to achieve said outcome. *"Because it describes the state of the world, declarative configuration does not have to be executed to be understood. Its impact is concretely declared. Since the effects of declarative configuration can be understood before they are executed, declarative configuration is far less error-prone. " --Hightower, Kelsey; Burns, Brendan; Beda, Joe. Kubernetes: Up and Running: Dive into the Future of Infrastructure (Kindle Locations 183-186). Kindle Edition*
 
 #### *To check if a CNF is using the default namespace*: [default_namespace](docs/LIST_OF_TESTS.md#default-namespaces)
-> *Namespces provide a way to segment and isolate cluster resources across multiple applications and users. As a best practice, workloads should be isolated with Namespaces and not use the default namespace. 
+> *Namespaces provide a way to segment and isolate cluster resources across multiple applications and users. As a best practice, workloads should be isolated with Namespaces and not use the default namespace. 
 
 #### *To test if mutable tags being used for image versioning(Using Kyverno): latest_tag*: [latest_tag](docs/LIST_OF_TESTS.md#latest-tag)
 
@@ -339,6 +339,23 @@ to their data has the following advantages:*
 - *improves performance of your cluster by significantly reducing load on kube-apiserver, by 
 closing watches for ConfigMaps marked as immutable.*"
 
+
+## 5g Tests 
+####  A 5g core is an important part of the service provider's telecommuncations offering. A cloud native 5g architecture uses immutable infrastructure, declarative configuration, and microservices when creating and hosting 5g cloud native network functions.
+
+#### *To check if the 5g core is resistant to chaos*: [smf_upf_core_validator](docs/LIST_OF_TESTS.md#smf_upf_core_validator)
+> *A 5g core's [SMF and UPF CNFs have a hearbeat](https://www.etsi.org/deliver/etsi_ts/123500_123599/123527/15.01.00_60/ts_123527v150100p.pdf), implemented use the PFCP protocol standard, which measures if the connection between the two CNFs is active.  After measure a baseline of the heartbeat a comparison between the baseline and the performance of the heartbeat while running test functions will expose the [cloud native resilience](https://www.cncf.io/blog/2021/09/23/cloud-native-chaos-and-telcos-enforcing-reliability-and-availability-for-telcos/) of the cloud native 5g core.
+
+#### *To check if the 5g core is using 5g authentication*: [suci_enabled](docs/LIST_OF_TESTS.md#suci_enabled)
+> *In order to [protect identifying information](https://nickvsnetworking.com/5g-subscriber-identifiers-suci-supi/) from being sent over the network as clear text, 5g cloud native cores should implement [SUPI and SUCI concealment](https://www.etsi.org/deliver/etsi_ts/133500_133599/133514/16.04.00_60/ts_133514v160400p.pdf)  
+
+
+## RAN Tests 
+#### A cloud native radio access network's (RAN) cloud native functions should use immutable infrastructure, declarative configuration, and microservices.  ORAN cloud native functions should adhere to cloud native principles while also complying with the [ORAN alliance's standards](https://www.o-ran.org/blog/o-ran-alliance-introduces-48-new-specifications-released-since-july-2021).
+
+#### *To check if an ORAN compliant RAN is using the e2 3gpp standard*: [oran_e2_connection](docs/LIST_OF_TESTS.md#oran_e2_connection)
+> *A near real-time RAN intelligent controler (RIC) uses the [E2 standard](https://wiki.o-ran-sc.org/display/RICP/E2T+Architecture) as an open, interoperable, interface to connect to [RAN-optimizated applications, onboarded as xApps](https://www.5gtechnologyworld.com/how-does-5gs-o-ran-e2-interface-work/). The xApps use platform services available in the near-RT RIC to communicate with the downstream network functions through the E2 interface.
+
 ## Platform Tests
 
 #### *To check if the plateform passes K8s Conformance tests*: [k8s-conformance](docs/LIST_OF_TESTS.md#k8s-conformance)

USAGE.md

@@ -14,6 +14,8 @@
   - [Observability and Diagnostic Tests](USAGE.md#observability-and-diagnostic-tests)
   - [Security Tests](USAGE.md#security-tests)
   - [Configuration Tests](USAGE.md#configuration-tests)
+  - [5g Tests](USAGE.md#5g-tests)
+  - [Ran Tests](USAGE.md#ran-tests)
 - [Platform Tests](USAGE.md#platform-tests)
 
 ### Overview
@@ -1189,6 +1191,45 @@ Remove any sensitive data stored in configmaps, environment variables and instea
 Use immutable configmaps for any non-mutable configuration data.
 </b>
 
+# 5g Tests
+
+##### To run all 5g tests, you can use the following command:
+
+```
+./cnf-testsuite 5g
+```
+
+## [smf_upf_core_validator](docs/LIST_OF_TESTS.md#smf_upf_core_validator)
+
+##### To run the 5g core_validator test, you can use the following command:
+
+```
+./cnf-testsuite smf_upf_core_validator
+```
+## [suci_enabled](docs/LIST_OF_TESTS.md#suci_enabled)
+##### To run the 5g suci_enabled test, you can use the following command:
+
+```
+./cnf-testsuite suci_enabled
+```
+
+# RAN Tests
+
+##### To run all RAN tests, you can use the following command:
+
+```
+./cnf-testsuite ran
+```
+
+## [oran_e2_connection](docs/LIST_OF_TESTS.md#oran_e2_connection)
+
+##### To run the oran e2 connection test, you can use the following command:
+
+```
+./cnf-testsuite oran_e2_connection
+```
+
+
 
 # Platform Tests
 

docs/LIST_OF_TESTS.md

@@ -699,6 +699,38 @@ Read more at [ARMO-C0045](https://bit.ly/3EvltIL)
 [**Rationale & Reasoning**](../RATIONALE.md#to-check-if-a-cnf-version-uses-immutable-configmaps-immutable_configmap)
 
 
+# 5g Category
+
+## [smf_upf_core_validator](https://github.com/cncf/cnf-testsuite/blob/v0.30.0/src/tasks/workload/5g_validator.cr#L9) 
+- Expectation: 5g core should continue to function during various CNF tests.
+
+**What's tested:** Checks the pfcp heartbeat between the smf and upf to make sure it remains close to baseline.
+
+[**Usage**](../USAGE.md#smf_upf_core_validator)
+
+[**Rationale & Reasoning**](../RATIONALE.md#to-validate-a-5g-core)
+
+## [suci_enabled](https://github.com/cncf/cnf-testsuite/blob/v0.30.0/src/tasks/workload/5g_validator.cr#L20) 
+- Expectation: 5g core should use suci concealment.
+
+**What's tested:** Checks to see if the 5g core supports suci concealment.
+
+[**Usage**](../USAGE.md#suci_enabled)
+
+[**Rationale & Reasoning**](../RATIONALE.md#to-check-for-5g-suci-concealment)
+
+
+# Ran Category
+
+## [oran_e2_connection](https://github.com/cncf/cnf-testsuite/blob/v0.30.0/src/tasks/workload/ran.cr#L10) 
+- Expectation: An ORAN RIC should use an e2 connection.
+
+**What's tested:** Checks if a RIC uses a oran compatible e2 connection.
+
+[**Usage**](../USAGE.md#oran_e2_connection)
+
+[**Rationale & Reasoning**](../RATIONALE.md#to-check-if-a-ric-uses-oran-compatible-e2-interface)
+
 
 ---
 

embedded_files/points.yml

@@ -296,3 +296,12 @@
 - name: latest_tag
   tags: configuration, dynamic, workload, essential, cert
   pass: 100
+
+- name: smf_upf_heartbeat 
+  tags: 5g, dynamic 
+
+- name: suci_enabled
+  tags: 5g, dynamic 
+
+- name: oran_e2_connection
+  tags: ran, dynamic 

spec/5g/core_spec.cr

@@ -37,4 +37,30 @@ describe "Core" do
     end
   end
 
+  it "'suci_enabled' should pass if the 5G core has suci enabled", tags: ["5g"]  do
+    begin
+      Log.info {`./cnf-testsuite cnf_setup cnf-config=sample-cnfs/sample_open5gs/cnf-testsuite.yml`}
+      $?.success?.should be_true
+      response_s = `./cnf-testsuite suci_enabled verbose`
+      Log.info {"response: #{response_s}"}
+      (/PASSED: Core uses SUCI 5g authentication/ =~ response_s).should_not be_nil
+    ensure
+      Log.info {`./cnf-testsuite cnf_cleanup cnf-config=sample-cnfs/sample_open5gs/cnf-testsuite.yml`}
+      $?.success?.should be_true
+    end
+  end
+
+  it "'suci_enabled' should fail if the 5G core does not have suci enabled", tags: ["5g"]  do
+    begin
+      Log.info {`./cnf-testsuite cnf_setup cnf-config=sample-cnfs/sample_open5gs_no_auth/cnf-testsuite.yml`}
+      $?.success?.should be_true
+      response_s = `./cnf-testsuite suci_enabled verbose`
+      Log.info {"response: #{response_s}"}
+      (/FAILED: Core does not use SUCI 5g authentication/ =~ response_s).should_not be_nil
+    ensure
+      Log.info {`./cnf-testsuite cnf_cleanup cnf-config=sample-cnfs/sample_open5gs_no_auth/cnf-testsuite.yml`}
+      $?.success?.should be_true
+    end
+  end
+
 end

spec/5g/ran_spec.cr

@@ -12,43 +12,6 @@ describe "5g" do
     $?.success?.should be_true
   end
 
-  it "'suci_enabled' should pass if the 5G core has suci enabled", tags: ["5g"]  do
-    begin
-      Log.info {`./cnf-testsuite cnf_setup cnf-config=sample-cnfs/sample_open5gs/cnf-testsuite.yml`}
-      $?.success?.should be_true
-      response_s = `./cnf-testsuite suci_enabled verbose`
-      Log.info {"response: #{response_s}"}
-      (/PASSED: Core uses SUCI 5g authentication/ =~ response_s).should_not be_nil
-    ensure
-      Log.info {`./cnf-testsuite cnf_cleanup cnf-config=sample-cnfs/sample_open5gs/cnf-testsuite.yml`}
-      $?.success?.should be_true
-    end
-  end
-
-  it "'suci_enabled' should fail if the 5G core does not have suci enabled", tags: ["5g"]  do
-    begin
-      Log.info {`./cnf-testsuite cnf_setup cnf-config=sample-cnfs/sample_open5gs_no_auth/cnf-testsuite.yml`}
-      $?.success?.should be_true
-      response_s = `./cnf-testsuite suci_enabled verbose`
-      Log.info {"response: #{response_s}"}
-      (/FAILED: Core does not use SUCI 5g authentication/ =~ response_s).should_not be_nil
-    ensure
-      Log.info {`./cnf-testsuite cnf_cleanup cnf-config=sample-cnfs/sample_open5gs_no_auth/cnf-testsuite.yml`}
-      $?.success?.should be_true
-    end
-  end
-
-  #TODO exec tshark command: tshark -ni any -Y nas_5gs.mm.type_id  -T json
-  #TODO parse tshark command
-  #TODO look for authentication text
-  # extra
-  #TODO look for connection text (sanity check)
-  #TODO tshark library
-  #TODO 5g tools library
-  #TODO 5g RAN and Core mobile traffic check (connection check)
-  #TODO 5g RAN (only) mobile traffic check ????
-  #TODO ueransim library (w/setup command)
-  #TODO Open5gs libary (w/setup command)
 
   it "'oran_e2_connection' should pass if the ORAN enabled RAN connects to the RIC using the e2 standard", tags: ["oran"]  do
     begin