Address review comments

.github/workflows/scan-images.yaml

@@ -15,22 +15,8 @@ jobs:
           - releases/1.8/stable/kubeflow
           - releases/1.9/stable
           - releases/latest/edge
-    runs-on: ubuntu-24.04
+    runs-on: [self-hosted, linux, X64, jammy, large]
     steps:
-      # Ideally we'd use self-hosted runners, but this effort is still not stable
-      # This action will remove unused software (dotnet, haskell, android libs, codeql,
-      # and docker images) from the GH runner, which will liberate around 60 GB of storage
-      # distributed in 40GB for root and around 20 for a mnt point.
-      - name: Maximise GH runner space
-        uses: easimon/maximize-build-space@v7
-        with:
-          root-reserve-mb: 29696
-          remove-dotnet: 'true'
-          remove-haskell: 'true'
-          remove-android: 'true'
-          remove-codeql: 'true'
-          remove-docker-images: 'true'
-
       - name: Checkout
         uses: actions/checkout@v3
         with:
@@ -42,11 +28,6 @@ jobs:
           sudo snap install yq
           echo "date=$(date '+%Y-%m-%d-%H-%M-%S')" >> $GITHUB_OUTPUT
 
-      - name: Set up Python ${{ matrix.python-version }}
-        uses: actions/setup-python@v5
-        with:
-          python-version: 3.12
-
       - name: Checkout kubeflow-ci
         uses: actions/checkout@v3
         with:
@@ -63,8 +44,8 @@ jobs:
           RELEASE=${BUNDLE_SPLIT[1]}
           RISK=${BUNDLE_SPLIT[2]}
 
-          pip3 install -r scripts/airgapped/requirements.txt
-          python3 scripts/airgapped/get-all-images.py ${{ matrix.bundle }}/bundle.yaml > image_list.txt
+          pip3 install -r scripts/requirements.txt
+          python3 scripts/get-all-images.py ${{ matrix.bundle }}/bundle.yaml > image_list.txt
           echo "Image list:"
           cat ./image_list.txt
           echo "release_risk=${RELEASE}-${RISK}" >> $GITHUB_OUTPUT

scripts/README.md

@@ -0,0 +1,26 @@
+# Utility Script
+
+This directory contains helper scripts for Charmed Kubeflow, during CI and not only.
+
+## Gather images used by a bundle
+
+You can get a list of all the OCI images used by the bundle by running the following command:
+```bash
+pip3 install -r scritps/requirements.txt
+
+python3 scripts/get-all-images.py \
+    --append-images tests/airgapped/ckf-1.8-testing-images.txt \
+    releases/1.8/stable/kubeflow/bundle.yaml \
+    > images-all.txt
+```
+
+The script will gather the images in the following way:
+1. For each `application` in the provided `bundle.yaml` file:
+2. detect if it's owned by us or another team (by looking at the `_github_dependency_repo_name` and such metadata)
+3. clone its repo, by looking at `_github_repo_name` and such metadata
+4. If owned by another team: only parse it's `metadata.yaml` and look for `oci-resources`
+5. If owned by us: run the `tools/get-images.sh` script the repo **must** have
+6. If a repo does not have `tools/get-images.sh` (i.e. kubeflow-roles) then the script should skip the repo
+7. If the `get-images.sh` script either fails (return code non zero) or has error logs then the script should **fail**
+8. Aggregate the outputs of all `get-images.sh` scripts to one output
+9. If user passed an argument `--append-images` then the script will amend a list of images we need for airgap testing

scripts/airgapped/README.md

@@ -7,12 +7,13 @@ to create airgap artifacts or via our testing scripts.
 We'll document some use-case scenarios here for the different scripts.
 
 ## Prerequisites
+NOTE: All the commands are expected to be run from the root directory of the repo
 
 To use the scripts in this directory you'll need to install a couple of Python
 and Ubuntu packages on the host machine, driving the test (not the LXC machine
 that will contain the airgapped environment).
 ```
-pip3 install -r requirements.txt
+pip3 install -r scripts/airgapped/requirements.txt
 sudo apt install pigz
 sudo snap install docker
 sudo snap install yq
@@ -32,9 +33,10 @@ This script makes the following assumptions:
    the images for that repo
 
 ```bash
-python3 scripts/airgapped/get-all-images.py \
+python3 scripts/get-all-images.py \
+    --append-images=tests/airgapped/ckf-1.8-testing-images.txt \
     releases/1.8/stable/kubeflow/bundle.yaml \
-    --airgap-testing > images.txt
+    > images.txt
 ```
 
 ## Pull images to docker cache

scripts/airgapped/requirements.txt

@@ -1,5 +1,3 @@
 docker
-#FIXME: remove requests pin when https://github.com/docker/docker-py/issues/3256 is solved
-requests<2.32.0
+requests
 PyYAML
-gitpython