Reference Architecture 23.02

New Components/Features: - Media Analytics Libraries Intel Deep Learning Streamer (Intel DL Streamer), GStreamer, OpenVINO toolkit OpenCL software, Level zero GPU, DPC++, and VAAPI from the IntelPU toolkit - FlexRAN software running as a Docker container (now available without NDA) - Rook/Ceph as a storage-related component - Rocky Linux 9.1 as base operating system (with some limitations mentioned below) - Non-root user deployment of Virtual Machine Reference System Architecture (VMRA) - Custom cluster naming in VMRA - Support for using Amazon Web Services (AWS) and Azure 'Cloud' CLIs as an alternative to Terraform - Azure Kubernetes Service (AKS) support for static CPU Management Policy and IntelPU Control Plane Plugin for Kubernetes - Intel Software Guard Extensions (Intel SGX) on AKS Updates/Changes: - Software versions upgraded for the majority of RA components (See User Guide for complete BOM and versions) Notable updates: Kubernetes to v1.26.1 MinIO to v4.5.8 DPDK to v22.11.1 Service Mesh to v1.17.1 VPP to v2302 KMRA to v2.3 - Eliminated the BMRA for Object Storage Setup deployment model. The storage-related features (MinIO, LPVSP, and Rook/Ceph) are now provided as optional components in select configuration profiles - Support of geo-specific mirrors for Kubespray (for example, in the People.s Republic of China) - Supported Kubernetes versions updated for AKS and Amazon EKS - Ubuntu images updated for AKS and Amazon EKS - Ability to deploy more RA software components on Azure and AWS Elasticsearch Kibana New Hardware (Platforms/CPUs/GPUs/Accelerators): - N/A Removed Support: - full_nfv profile - Ubuntu 20.04 as base operating system - Rocky Linux 9.0 as base operating system Known Limitations/Restrictions: - When using the Cilium CNI, secondary interfaces are not supported - Intelynamic Load Balancer (Intel) is not fully supported on Rocky Linux 9.1 - FlexRAN container support is limited to FlexRAN v22.07, Ubuntu 22.04 base operating system, and only on 3rd Gen Intel Xeon Scalable processors - Media Analytics is supported only with Docker runtime - MinIO is supported only with CRI-O runtime - VMRA cluster expansion with additional VM nodes might fail - Trusted Certificate Attestation (TCA) is not fully functional in VMRA Co-authored-by: Ali Shah, Syed Faraz <[email protected]> Co-authored-by: Benedikt, Jan <[email protected]> Co-authored-by: Fiala, Jiri <[email protected]> Co-authored-by: Gherghe, Calin <[email protected]> Co-authored-by: Jiang, Renzhi <[email protected]> Co-authored-by: Kasanic, Denis <[email protected]> Co-authored-by: Liu, Mark <[email protected]> Co-authored-by: MacGillivray, Mac <[email protected]> Co-authored-by: Mlynek, Krystian <[email protected]> Co-authored-by: Musial, Michal <[email protected]> Co-authored-by: Park, Seungweon <[email protected]> Co-authored-by: Pedersen, Michael <[email protected]> Co-authored-by: Prokes, Jiri <[email protected]> Co-authored-by: Vrana, Roman <[email protected]> Co-authored-by: Zenov, Mykyta <[email protected]>
intel · Mar 31, 2023 · e7d7466 · e7d7466
1 parent 486c979
commit e7d7466
Show file tree

Hide file tree

Showing 687 changed files with 10,951 additions and 5,650 deletions.
diff --git a/Makefile b/Makefile
@@ -1,6 +1,7 @@
 CEK_DIRECTORIES_WITH_SHELL_FILES ?= roles/ examples/ playbooks/infra/ playbooks/intel/
 ARCH ?= 'icx'
 NIC ?= 'cvl'
+MIRRORS ?= false
 PLAYBOOKS_DIRS = playbooks playbooks/infra playbooks/intel
 PLAYBOOK_NAMES = access basic full_nfv on_prem regional_dc remote_fp storage build_your_own
 
@@ -40,7 +41,8 @@ k8s-profile: clean-playbooks
 	--mode k8s \
 	-p $(PROFILE) \
 	-a $(ARCH) \
-	-n ${NIC}
+	-n ${NIC} \
+	-m ${MIRRORS}
 
 vm-profile: clean-playbooks
 	python3 generate/render.py \
@@ -53,7 +55,8 @@ vm-profile: clean-playbooks
 	--mode vm \
 	-p $(PROFILE) \
 	-a $(ARCH) \
-	-n ${NIC}
+	-n ${NIC} \
+	-m ${MIRRORS}
 
 cloud-profile: clean-playbooks
 	python3 generate/render.py \
@@ -65,7 +68,8 @@ cloud-profile: clean-playbooks
 	--mode cloud \
 	-p $(PROFILE) \
 	-a $(ARCH) \
-	-n ${NIC}
+	-n ${NIC} \
+	-m ${MIRRORS}
 
 clean: clean-playbooks clean-project-root-dir
 

diff --git a/Pipfile b/Pipfile
@@ -6,19 +6,20 @@ name = "pypi"
 [packages]
 ansible = "~=5.7.1"
 "ansible-core" = "~=2.12"
-cryptography = "~=3.3"
-jinja2 = "~=2.11"
+cryptography = "~=39.0"
+jinja2 = "~=3.1"
 netaddr = "~=0.7.19"
 pbr = "~=5.4"
 jmespath = "~=0.9.5"
-"ruamel.yaml" = "~=0.16.10"
+"ruamel.yaml" = "~=0.17.21"
 "ruamel.yaml.clib" = "~=0.2.6"
-MarkupSafe = "~=1.1"
+MarkupSafe = "~=2.1"
 ipaddr = "*"
 
 [dev-packages]
-ansible-lint = "~=5.4.0"
+ansible-lint = "~=6.12.1"
 pylint = "~=2.15.4"
+bandit = "~=1.7.4"
 
 [requires]
 python_version = "3"
diff --git a/README.md b/README.md
@@ -6,6 +6,8 @@ The software provided here is for reference only and not intended for production
 
 ## Quickstart guide
 
+**_NOTE:_** Instruction provided bellow are prepared for deployment done under root user by default. If you want to do deployment under non-root user then read [this](docs/rootless_deployment.md) file first and then continue with following steps under that non-root user.
+
 1. Initialize git submodules to download Kubespray code.
 
     ```bash
@@ -32,7 +34,7 @@ The software provided here is for reference only and not intended for production
         export PROFILE=regional_dc
         ```
 
-    - For **Kubernetes Remote Forwarding Platform Infrastructure** deployment:
+    - For **Kubernetes Remote Central Office-Forwarding Configuration** deployment:
 
         ```bash
         export PROFILE=remote_fp
@@ -44,27 +46,15 @@ The software provided here is for reference only and not intended for production
         export PROFILE=on_prem
         ```
 
-    - For **Kubernetes Full NFV Infrastructure** deployment:
-
-        ```bash
-        export PROFILE=full_nfv
-        ```
-
-    - For **Kubernetes Storage Infrastructure** deployment:
-
-        ```bash
-        export PROFILE=storage
-        ```
-
     - For **Kubernetes Build-Your-Own Infrastructure** deployment:
 
         ```bash
         export PROFILE=build_your_own
         ```
 
-3. Install dependencies using a) or b)
+3. Install dependencies using one of the following methods
 
-    a) Non-invasive virtual environment method
+    a) Non-invasive virtual environment using pipenv
 
     ```bash
     pip3 install pipenv
@@ -73,11 +63,21 @@ The software provided here is for reference only and not intended for production
     pipenv shell
     ```
 
-    b) System wide installation method
+    b) Non-invasive virtual environment using venv
 
-   ```bash
-   pip3 install -r requirements.txt
-   ```
+    ```bash
+    python3 -m venv venv
+    # Then to activate new virtual environment
+    source venv/bin/activate
+    # Install dependencies in venv
+    pip3 install -r requirements.txt
+    ```
+
+    c) System wide environment (not recommended)
+
+    ```bash
+    pip3 install -r requirements.txt
+    ```
 
 4. Generate example host_vars, group_vars and inventory files for Intel Container Experience Kits profiles.
 
@@ -139,7 +139,9 @@ The software provided here is for reference only and not intended for production
     - update details relevant for vm_host (e.g.: datalane_interfaces, ...)
     - update VMs definition in host_vars/host-for-vms-1.yml - use that template for the first vm_host
     - update VMs definition in host_vars/host-for-vms-2.yml - use that template for the second and all other vm_hosts
-    - update/create host_vars for all defined VMs (e.g.: host_vars/vm-ctrl-1.yml and host_vars/vm-work-1.yml)
+    - update/create host_vars for all defined VMs (e.g.: host_vars/vm-ctrl-1.cluster1.local.yml and host_vars/vm-work-1.cluster1.local.yml)
+      In case that vm_cluster_name is not defined or is empty, short host_vars file names should be used for VMs
+      (e.g.: host_vars/vm-ctrl-1.yml and host_vars/vm-work-1.yml)
       Needed details are at least dataplane_interfaces
       For more details see [VM case configuration guide](docs/vm_config_guide.md)
 
@@ -175,6 +177,7 @@ Refer to the documentation linked below to see configuration details for selecte
 - [VM case configuration guide](docs/vm_config_guide.md)
 - [VM multinode setup guide](docs/vm_multinode_setup_guide.md)
 - [VM cluster expansion guide](docs/vm_cluster_expansion_guide.md)
+- [Non-root deployment guide](docs/rootless_deployment.md)
 ## Prerequisites and Requirements
 
 - Required packages on the target servers: **Python3**.
@@ -200,3 +203,38 @@ Contributors, beside basic set of packages, should also install developer packag
 ```bash
 pipenv install --dev
 ```
+
+or
+
+```bash
+pip install -r ci-requirements.txt
+```
+
+### Run lint checks locally
+
+Several lint checks are configured for the repository. All of them can be run on local environment using prepared bash scripts or by leveraging pre-commit hooks.
+
+Prerequisite packages:
+
+- developer python packages (ci-requirements.txt/Pipfile)
+- shellcheck
+- pre-commit python package
+
+Required checks in CI:
+
+- ansible-lint
+- bandit
+- pylint
+- shellcheck
+
+Check can be run by following command:
+
+```bash
+./scrits/run_<linter_name>.sh
+```
+
+or alternatively:
+
+```bash
+pre-commit run <linter_name> --all-files
+```
diff --git a/SECURITY.md b/SECURITY.md
@@ -0,0 +1,5 @@
+# Security Policy
+Intel is committed to rapidly addressing security vulnerabilities affecting our customers and providing clear guidance on the solution, impact, severity and mitigation. 
+
+## Reporting a Vulnerability
+Please report any security vulnerabilities in this project [utilizing the guidelines here](https://www.intel.com/content/www/us/en/security-center/vulnerability-handling-guidelines.html).
diff --git a/action_plugins/cpupin.py b/action_plugins/cpupin.py
@@ -1,6 +1,6 @@
 #!/usr/bin/python
 #
-#   Copyright (c) 2020-2022 Intel Corporation.
+#   Copyright (c) 2020-2023 Intel Corporation.
 #
 #   Licensed under the Apache License, Version 2.0 (the "License");
 #   you may not use this file except in compliance with the License.
@@ -485,7 +485,7 @@ def _allocate_cpus(self, task_vars):
 
         # Select random NUMA
         if not self.numa:
-            self.numa = random.choice(self.numa_nodes)
+            self.numa = random.choice(self.numa_nodes) # nosec B311 # pseudo random is not used for security purposes
 
         if not self.cpus:
             self.cpu_list = self._select_cpus(task_vars['numa_nodes_cpus'], self.number, self.numa)

diff --git a/action_plugins/validate_cpusets.py b/action_plugins/validate_cpusets.py
@@ -1,5 +1,5 @@
 #
-#   Copyright (c) 2020-2022 Intel Corporation.
+#   Copyright (c) 2020-2023 Intel Corporation.
 #
 #   Licensed under the Apache License, Version 2.0 (the "License");
 #   you may not use this file except in compliance with the License.

diff --git a/action_plugins/validate_isolcpus.py b/action_plugins/validate_isolcpus.py
@@ -1,5 +1,5 @@
 #
-#   Copyright (c) 2020-2022 Intel Corporation.
+#   Copyright (c) 2020-2023 Intel Corporation.
 #
 #   Licensed under the Apache License, Version 2.0 (the "License");
 #   you may not use this file except in compliance with the License.

diff --git a/ansible.cfg b/ansible.cfg
@@ -16,3 +16,6 @@ fact_caching_timeout = 7200
 
 action_plugins = ./action_plugins:~/.ansible/plugins/action:/usr/share/ansible/plugins/action
 library = ./library
+
+log_path = ./.ansible_last_run.log
+display_args_to_stdout = False