Merge branch 'IBM:main' into main

docs/run-the-playbooks-for-disconnected.md

@@ -31,7 +31,7 @@ Disconnected playbook are mentioned below. Please refer the **4 Run the Playbook
 * A mirror host. This can be any host that can access the internet (mainly the registry being mirrored from) as well as the registry being mirrored to.
   This registries being mirrored from would typically be the Red Hat registries (registry.redhat.io, quay.io etc)
 * The file server, configured mentioned below.
-* Appropriately updated variables in your `all.yaml`. Refer the variables documentation.
+* Appropriately updated variables in your `disconnected.yaml`. Refer the variables documentation.
 
 ### File Server
 

docs/set-variables-group-vars.md

@@ -209,7 +209,31 @@
 **rhcos_live_initrd** | CoreOS initramfs to be used for the bootstrap, control and compute nodes. | rhcos-4.12.3-s390x-live-initramfs.s390x.img
 **rhcos_live_rootfs** | CoreOS rootfs to be used for the bootstrap, control and compute nodes. | rhcos-4.12.3-s390x-live-rootfs.s390x.img
 
-## 16 - (Optional) Disconnected cluster setup
+## 16 - (Optional) Create compute node in a day-2 operation
+
+**Variable Name** | **Description** | **Example**
+:--- | :--- | :---
+**day2_compute_node.vm_name** | Name of the compute node VM.  | compute-4
+**day2_compute_node.vm_hostname** | Hostnames for compute node. | compute-4
+**day2_compute_node.vm_vm_ip** | IPv4 address of the compute node. | 192.168.10.99
+**day2_compute_node.vm_vm_ipv6** | IPv6 address of the compute node. | fd00::99
+**day2_compute_node.vm_mac** | MAC address of the compute node if use_dhcp variable is 'True'. | 52:54:00:18:1A:2B
+**day2_compute_node.vm_interface** | The network interface used for given IP addresses of the compute node. | enc1
+**day2_compute_node.hostname** | The hostname of the KVM host | kvm-host-01
+**day2_compute_node.host_user** | KVM host user which is used to create the VM | root
+**day2_compute_node.host_arch** | KVM host architecture.  | s390x
+
+## 17 - (Optional) Agent Based Installer
+
+**Variable Name** | **Description** | **Example**
+:--- | :--- | :---
+**abi.flag** | This is the flag, Will be used to identify during execution. Few checks in the playbook will be depend on this (default value will be False)  | True
+**abi.ansible_workdir** | This will be work directory name, it will keep required data that need to be present during or after execution | ansible_workdir
+**abi.ocp_installer_version** | Version will contain value of openshift-installer binary version user desired to be used | '4.15.0-rc.8'
+**abi.ocp_installer_url** | This is the base url of openshift installer binary it will remain same as static value, User Do not need to give value until user wants to change the mirror | 'https://mirror.openshift.com/pub/openshift-v4/s390x/clients/ocp/'
+
+
+## Disconnected cluster setup (Optional)
 **Variable Name** | **Description** | **Example**
 :--- | :--- | :---
 **disconnected.enabled** | True or False, to enable disconnected mode | False
@@ -241,30 +265,6 @@
 **disconnected.mirroring.oc_mirror.image_set.storageConfig.registry.skipTLS** | True of False same purpose served as in standard image set i.e. skip the tls for the registry   during mirroring.| false
 **disconnected.mirrroing.oc_mirror.image_set.mirror** | YAML containing a list of what needs to be mirrored. See the oc mirror image set documentation. | see oc-mirror [image set](https://docs.openshift.com/container-platform/latest/installing/disconnected_install/installing-mirroring-disconnected.html#oc-mirror-creating-image-set-config_installing-mirroring-disconnected)   documentation
 
-## 17 - (Optional) Create compute node in a day-2 operation
-
-**Variable Name** | **Description** | **Example**
-:--- | :--- | :---
-**day2_compute_node.vm_name** | Name of the compute node VM.  | compute-4
-**day2_compute_node.vm_hostname** | Hostnames for compute node. | compute-4
-**day2_compute_node.vm_vm_ip** | IPv4 address of the compute node. | 192.168.10.99
-**day2_compute_node.vm_vm_ipv6** | IPv6 address of the compute node. | fd00::99
-**day2_compute_node.vm_mac** | MAC address of the compute node if use_dhcp variable is 'True'. | 52:54:00:18:1A:2B
-**day2_compute_node.vm_interface** | The network interface used for given IP addresses of the compute node. | enc1
-**day2_compute_node.hostname** | The hostname of the KVM host | kvm-host-01
-**day2_compute_node.host_user** | KVM host user which is used to create the VM | root
-**day2_compute_node.host_arch** | KVM host architecture.  | s390x
-
-## 18 - (Optional) Agent Based Installer
-
-**Variable Name** | **Description** | **Example**
-:--- | :--- | :---
-**abi.flag** | This is the flag, Will be used to identify during execution. Few checks in the playbook will be depend on this (default value will be False)  | True
-**abi.ansible_workdir** | This will be work directory name, it will keep required data that need to be present during or after execution | ansible_workdir
-**abi.ocp_installer_version** | Version will contain value of openshift-installer binary version user desired to be used | '4.15.0-rc.8'
-**abi.ocp_installer_url** | This is the base url of openshift installer binary it will remain same as static value, User Do not need to give value until user wants to change the mirror | 'https://mirror.openshift.com/pub/openshift-v4/s390x/clients/ocp/'
-
-
 ## Hosted Control Plane ( Optional )
 **Variable Name** | **Description** | **Example**
 :--- | :--- | :---
@@ -340,4 +340,4 @@
 **hcp.data_plane.zvm.nodes.interface.options** | Configurations options  | layer2=1
 **hcp.data_plane.zvm.interface.ip** | IP addresses for to be used for zVM nodes | 192.168.10.1
 **hcp.data_plane.zvm.nodes.dasd.disk_id** | Disk id for dasd disk to be used for zVM node | 4404 
-**hcp.data_plane.zvm.nodes.lun** | Disk details of fcp disk to be used for zVM node | 4404
+**hcp.data_plane.zvm.nodes.lun** | Disk details of fcp disk to be used for zVM node | 4404

inventories/default/group_vars/.gitignore

@@ -1,4 +1,5 @@
 /*
 !.gitignore
 !all.yaml.template
-!hcp.yaml.template
+!hcp.yaml.template
+

inventories/default/group_vars/all.yaml.template

@@ -258,80 +258,8 @@ rhcos_live_kernel: "rhcos-4.12.3-s390x-live-kernel-s390x"
 rhcos_live_initrd: "rhcos-4.12.3-s390x-live-initramfs.s390x.img"
 rhcos_live_rootfs: "rhcos-4.12.3-s390x-live-rootfs.s390x.img"
 
-# Section 16 - (Optional) Setup disconnected clusters
-# Warning: currently, the oc-mirror plugin is officially downloadable to amd64 only.
-disconnected:
-  enabled: False
-  registry:
-    url: 'registry url'
-    pull_secret: '' # this is similar to env.redhat.pull_secret but it will only contain secrets to be applied to the cluster in disconnected mode.
-    mirror_pull_secret: '' # this should contain a pull secret that contains the combination of env.redhat.pull_secret and pull secret  with push access to mirror registry for mirroring
-    ca_trusted: False
-    ca_cert: |
-      -----BEGIN CERTIFICATE-----
-      if ca_trusted is False, then this ca will be added to mirror host anchors as well as to the install config of cluster
-      -----END CERTIFICATE-----
-  mirroring:
-    host: # this is the host that can access the internet as well as the registry
-      name: hosname
-      ip: x.x.x.x
-      user: mirroruser # with become access
-      pass: mirrorpassword
-    file_server: # in disconnected mode, the client binaries and rhcos will be put on env.file_server and then downloaded to the final destination from there. For now, its only oc-mirror. Rest of artifacts will be downloaded from urls
-      clients_dir: 'clients'
-      oc_mirror_tgz: 'oc-mirror.tar.gz' # name of oc-mirror plugin binary in clients_dir. should be a tar.gz file. You must place this in your ftp server after downloading it yourself from https://console.redhat.com/openshift/downloads for amd64 (or building it yourself if your mirror host is s390x)
-    client_download: # this will download oc binary to the mirror host for use on the mirror host for mirroring
-      ocp_download_url: "https://mirror.openshift.com/pub/openshift-v4/multi/clients/ocp/4.13.1/amd64/"
-      ocp_client_tgz: 'openshift-client-linux.tar.gz' # name of the oc binary. Should be a tar.gz file
-    legacy:
-      platform: False # if true then platform is mirrored in the old way and install config will be patched with the imagecontentsourcepolicy
-      ocp_quay_release_image_tag: '4.13.1-s390x'
-      ocp_org: 'ocp4'
-      ocp_repo: 'openshift4'
-      ocp_tag: 'v4.13.1' #  platform images will be pushed to {tegistry_url}/{ocp_org}/{ocp_repo}:{ocp_tag}
-    oc_mirror:
-      oc_mirror_args:
-        continue_on_error: False
-        source_skip_tls: False
-      post_mirror:
-        mapping:
-          replace:
-            enabled: False
-            list:
-            - regexp: what
-              replace: with
-      release_image_tag: '4.13.1-multi'
-      image_set: # this field is a standard image set from oc-mirror documentation. The only exception is the storageConfig which is altered to allow substitution of disconnected.registry.url
-        storageConfig:
-          registry:
-            enabled: True # use registry storage backend. Currently only method supproted
-            imageURL: # the final value will be {imageURL: disconnected.registry.url/org/repo}
-              org: mirror
-              repo: oc-mirror-metadata
-            skipTLS: false # standard field form oc-mirror schema
-        mirror: # this field is also atandard from the oc-mirror schema. It will be substituted as is into the final image set.
-          platform:
-            architectures:
-            - multi # note: while image tags such as `multi-s390x` are also available on quay, we cannot mirror these, so list can only contain pure architecture names such as `s390x` or the multiarch `multi`
-            channels:
-            - name: stable-4.13
-              full: false
-              minVersion: 4.13.1
-              maxVersion: 4.13.1
-          operators:
-            - catalog: registry.redhat.io/redhat/redhat-operator-index:v4.13
-              full: false
-              packages:
-              - name: serverless-operator
-                channels:
-                - name: stable
-                  #minVersion: '2.4.1-0'
-                  #maxVersion: '2.4.1-0'
-          additionalImages:
-          - name: registry.redhat.io/ubi8/ubi:latest
-          helm: {}
 
-# Section 17 - (Optional) Create additional compute node in a day-2 operation
+# Section 16 - (Optional) Create additional compute node in a day-2 operation
 
 day2_compute_node:
   vm_name:
@@ -345,9 +273,10 @@ day2_compute_node:
   host_arch:
 
 
-# Section 18 - Agent Based Installer ( Optional )
+# Section 17 - Agent Based Installer ( Optional )
 abi:
-  flag: Flase
+  flag: False
   ansible_workdir: 'ansible_workdir'
   ocp_installer_version: '4.15.0-rc.8'
   ocp_installer_url: 'https://mirror.openshift.com/pub/openshift-v4/s390x/clients/ocp/'
+  boot_method: <pxe|iso>

inventories/default/group_vars/disconnected.yaml

@@ -0,0 +1,71 @@
+# Warning: currently, the oc-mirror plugin is officially downloadable to amd64 only.
+disconnected:
+  enabled: False
+  registry:
+    url: 'registry url'
+    pull_secret: '' # this is similar to env.redhat.pull_secret but it will only contain secrets to be applied to the cluster in disconnected mode.
+    mirror_pull_secret: '' # this should contain a pull secret that contains the combination of env.redhat.pull_secret and pull secret  with push access to mirror registry for mirroring
+    ca_trusted: False
+    ca_cert: |
+      -----BEGIN CERTIFICATE-----
+      if ca_trusted is False, then this ca will be added to mirror host anchors as well as to the install config of cluster
+      -----END CERTIFICATE-----
+  mirroring:
+    host: # this is the host that can access the internet as well as the registry
+      name: hosname
+      ip: x.x.x.x
+      user: mirroruser # with become access
+      pass: mirrorpassword
+    file_server: # in disconnected mode, the client binaries and rhcos will be put on env.file_server and then downloaded to the final destination from there. For now, its only oc-mirror. Rest of artifacts will be downloaded from urls
+      clients_dir: 'clients'
+      oc_mirror_tgz: 'oc-mirror.tar.gz' # name of oc-mirror plugin binary in clients_dir. should be a tar.gz file. You must place this in your ftp server after downloading it yourself from https://console.redhat.com/openshift/downloads for amd64 (or building it yourself if your mirror host is s390x)
+    client_download: # this will download oc binary to the mirror host for use on the mirror host for mirroring
+      ocp_download_url: "https://mirror.openshift.com/pub/openshift-v4/multi/clients/ocp/4.13.1/amd64/"
+      ocp_client_tgz: 'openshift-client-linux.tar.gz' # name of the oc binary. Should be a tar.gz file
+    legacy:
+      platform: False # if true then platform is mirrored in the old way and install config will be patched with the imagecontentsourcepolicy
+      ocp_quay_release_image_tag: '4.13.1-s390x'
+      ocp_org: 'ocp4'
+      ocp_repo: 'openshift4'
+      ocp_tag: 'v4.13.1' #  platform images will be pushed to {tegistry_url}/{ocp_org}/{ocp_repo}:{ocp_tag}
+    oc_mirror:
+      oc_mirror_args:
+        continue_on_error: False
+        source_skip_tls: False
+      post_mirror:
+        mapping:
+          replace:
+            enabled: False
+            list:
+            - regexp: what
+              replace: with
+      release_image_tag: '4.13.1-multi'
+      image_set: # this field is a standard image set from oc-mirror documentation. The only exception is the storageConfig which is altered to allow substitution of disconnected.registry.url
+        storageConfig:
+          registry:
+            enabled: True # use registry storage backend. Currently only method supproted
+            imageURL: # the final value will be {imageURL: disconnected.registry.url/org/repo}
+              org: mirror
+              repo: oc-mirror-metadata
+           skipTLS: false # standard field form oc-mirror schema
+        mirror: # this field is also atandard from the oc-mirror schema. It will be substituted as is into the final image set.
+          platform:
+            architectures:
+            - multi # note: while image tags such as `multi-s390x` are also available on quay, we cannot mirror these, so list can only contain pure architecture names such as `s390x` or the multiarch `multi`
+            channels:
+            - name: stable-4.13
+              full: false
+              minVersion: 4.13.1
+              maxVersion: 4.13.1
+          operators:
+            - catalog: registry.redhat.io/redhat/redhat-operator-index:v4.13
+              full: false
+              packages:
+              - name: serverless-operator
+                channels:
+                - name: stable
+                  #minVersion: '2.4.1-0'
+                  #maxVersion: '2.4.1-0'
+          additionalImages:
+          - name: registry.redhat.io/ubi8/ubi:latest
+          helm: {}

playbooks/0_setup.yaml

@@ -5,6 +5,8 @@
   connection: local
   become: false
   gather_facts: true
+  vars_files:
+    - "{{ inventory_dir }}/group_vars/disconnected.yaml" 
   roles:
     - set_inventory
 
@@ -64,7 +66,7 @@
   become: false
   gather_facts: true
   vars_files:
-    - "{{ inventory_dir }}/group_vars/all.yaml"
+    - "{{ inventory_dir }}/group_vars/all.yaml" 
   vars:
     packages: "{{ env.pkgs.controller }}"
     ssh_target: [ "{{ env.file_server.ip }}", "{{ env.file_server.user }}", "{{ env.file_server.pass }}", "{{ path_to_key_pair }}" ]

playbooks/6_create_nodes.yaml

@@ -8,13 +8,13 @@
     - name: Loop over node types and include role
       include_role:
         name: check_for_lpar_nodes
-      loop: 
+      loop:
         - bootstrap
         - compute
         - control
       loop_control:
         loop_var: node_type
-  
+
 # Prepare and then create the temporary bootstrap node and the control nodes
 - name: 6 create nodes - prepare KVM guests
   hosts: kvm_host
@@ -44,7 +44,7 @@
         node_type: "bootstrap"
         node_name: "{{ item }}"
         ignition: "bootstrap"
-      include_tasks: 
+      include_tasks:
         file: ../roles/boot_LPAR/tasks/main.yaml
       loop: "{{ q('list',env.cluster.nodes[node_type].vm_name) | flatten }}"
       when: item in hosts_with_host_vars
@@ -54,7 +54,7 @@
         node_type: "control"
         node_name: "{{ item }}"
         ignition: "master"
-      include_tasks: 
+      include_tasks:
         file: ../roles/boot_LPAR/tasks/main.yaml
       loop: "{{ q('list',env.cluster.nodes[node_type].vm_name) | flatten }}"
       when: item in hosts_with_host_vars
@@ -68,27 +68,16 @@
     - common
     - create_control_nodes
 
-- name: 6 create nodes - wait for bootstrap to connect control plane (for non-root user)
-  hosts: bastion
-  become: true
-  environment:
-    KUBECONFIG: "/home/{{ env.bastion.access.user }}/.kube/config"
-  gather_facts: true
-  vars_files:
-    - "{{ inventory_dir }}/group_vars/all.yaml"
-  roles:
-    - {role: wait_for_bootstrap, when: env.bastion.access.user != "root"}
-
-- name: 6 create nodes - wait for bootstrap to connect to control plane (for root user)
+- name: 6 create nodes - wait for bootstrap to connect control plane
   hosts: bastion
   become: true
   environment:
-    KUBECONFIG: "/{{ env.bastion.access.user }}/.kube/config"
+    KUBECONFIG: "{{ '/home/' if (env.bastion.access.user != 'root') else '/'}}{{ env.bastion.access.user }}/.kube/config"
   gather_facts: true
   vars_files:
     - "{{ inventory_dir }}/group_vars/all.yaml"
   roles:
-    - {role: wait_for_bootstrap, when: env.bastion.access.user == "root"}
+    - wait_for_bootstrap
 
 - name: 6 create nodes - once bootstrapping is complete, tear down bootstrap.
   hosts: kvm_host[0]
@@ -111,7 +100,7 @@
         name: "{{ env.cluster.nodes.bootstrap.vm_name }}"
         command: undefine
       ignore_errors: true
-      
+
 - name: 6 create nodes - once bootstrapping is complete, create compute nodes.
   hosts: kvm_host
   tags: create_compute_nodes
@@ -131,7 +120,7 @@
         node_type: "compute"
         node_name: "{{ item }}"
         ignition: "worker"
-      include_tasks: 
+      include_tasks:
         file: ../roles/boot_LPAR/tasks/main.yaml
       loop: "{{ q('list',env.cluster.nodes[node_type].vm_name) | flatten }}"
       when: item in hosts_with_host_vars

playbooks/destroy_abi_cluster.yaml

@@ -7,3 +7,4 @@
   roles:
     - delete_nodes
     - destroy_bastion
+    - cleanup_kvm

playbooks/disconnected_apply_operator_manifests.yaml

@@ -7,6 +7,7 @@
   gather_facts: true
   vars_files:
     - "{{ inventory_dir }}/group_vars/all.yaml"
+    - "{{ inventory_dir }}/group_vars/disconnected.yaml"
   tasks:
     - name: apply operator manifests when cluster is disconnected
       ansible.builtin.include_role:

playbooks/disconnected_mirror_artifacts.yaml

@@ -4,6 +4,7 @@
   gather_facts: false
   vars_files:
     - "{{ inventory_dir }}/group_vars/all.yaml"
+    - "{{ inventory_dir }}/group_vars/disconnected.yaml"
   tasks:
     - name: mirror ocp artifacts when disconnected install
       ansible.builtin.include_role: