UI visual regression testing to cover UI widgets visibility

.github/workflows/ui_notebooks_test.yaml

@@ -0,0 +1,110 @@
+name: UI notebooks tests
+
+on: [pull_request]
+
+concurrency:
+  group: ${{ github.head_ref }}-${{ github.workflow }}
+  cancel-in-progress: true
+
+env:
+  CODEFLARE_OPERATOR_IMG: "quay.io/project-codeflare/codeflare-operator:dev"
+
+jobs:
+  verify-0_basic_ray:
+    runs-on: ubuntu-20.04-4core
+
+    steps:
+      - name: Checkout code
+        uses: actions/checkout@v4
+        with:
+          submodules: recursive
+
+      - name: Checkout common repo code
+        uses: actions/checkout@v4
+        with:
+          repository: "project-codeflare/codeflare-common"
+          ref: "main"
+          path: "common"
+
+      - name: Checkout CodeFlare operator repository
+        uses: actions/checkout@v4
+        with:
+          repository: project-codeflare/codeflare-operator
+          path: codeflare-operator
+
+      - name: Set Go
+        uses: actions/setup-go@v5
+        with:
+          go-version-file: "./codeflare-operator/go.mod"
+          cache-dependency-path: "./codeflare-operator/go.sum"
+
+      - name: Set up gotestfmt
+        uses: gotesttools/gotestfmt-action@v2
+        with:
+          token: ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Set up specific Python version
+        uses: actions/setup-python@v5
+        with:
+          python-version: "3.9"
+          cache: "pip" # caching pip dependencies
+
+      - name: Setup and start KinD cluster
+        uses: ./common/github-actions/kind
+
+      - name: Deploy CodeFlare stack
+        id: deploy
+        run: |
+          cd codeflare-operator
+          echo Setting up CodeFlare stack
+          make setup-e2e
+          echo Deploying CodeFlare operator
+          make deploy -e IMG="${CODEFLARE_OPERATOR_IMG}" -e ENV="e2e"
+          kubectl wait --timeout=120s --for=condition=Available=true deployment -n openshift-operators codeflare-operator-manager
+          cd ..
+
+      - name: Setup Guided notebooks execution
+        run: |
+          echo "Installing papermill and dependencies..."
+          pip install poetry ipython ipykernel
+          poetry config virtualenvs.create false
+          echo "Installing SDK..."
+          poetry install --with test,docs
+
+      - name: Install Yarn dependencies
+        run: |
+          poetry run yarn install
+          poetry run yarn playwright install chromium
+        working-directory: ui-tests
+
+      - name: Fix widget_notebook_example.ipynb notebook for test
+        run: |
+          # Remove login/logout cells, as KinD doesn't support authentication using token
+          jq -r 'del(.cells[] | select(.source[] | contains("Create authentication object for user permissions")))' widget_notebook_example.ipynb > 0_basic_ray.ipynb.tmp && mv 0_basic_ray.ipynb.tmp 0_basic_ray.ipynb
+          jq -r 'del(.cells[] | select(.source[] | contains("auth.logout()")))' widget_notebook_example.ipynb > widget_notebook_example.ipynb.tmp && mv 0_basic_ray.ipynb.tmp 0_basic_ray.ipynb
+          # Set explicit namespace as SDK need it (currently) to resolve local queues
+          sed -i "s/head_memory_limits=2,/head_memory_limits=2, namespace='default',/" widget_notebook_example.ipynb
+        working-directory: demo-notebooks/guided-demos
+
+      - name: Run UI notebook tests
+        run: |
+          set -euo pipefail
+
+          poetry run yarn test
+        working-directory: ui-tests
+
+      - name: Upload Playwright Test assets
+        if: always()
+        uses: actions/upload-artifact@v4
+        with:
+          name: ipywidgets-test-assets
+          path: |
+            ui-tests/test-results
+
+      - name: Upload Playwright Test report
+        if: always()
+        uses: actions/upload-artifact@v4
+        with:
+          name: ipywidgets-test-report
+          path: |
+            ui-tests/playwright-report

.gitignore

@@ -8,3 +8,7 @@ Pipfile.lock
 build/
 tls-cluster-namespace
 quicktest.yaml
+node_modules
+.DS_Store
+ui-tests/playwright-report
+ui-tests/test-results

demo-notebooks/guided-demos/widget_notebook_example.ipynb

@@ -0,0 +1,212 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "8d4a42f6",
+   "metadata": {},
+   "source": [
+    "In this notebook, we will go through the basics of using the SDK to:\n",
+    " - Spin up a Ray cluster with our desired resources\n",
+    " - View the status and specs of our Ray cluster\n",
+    " - Take down the Ray cluster when finished"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "b55bc3ea-4ce3-49bf-bb1f-e209de8ca47a",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Import pieces from codeflare-sdk\n",
+    "from codeflare_sdk import Cluster, ClusterConfiguration, TokenAuthentication"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "614daa0c",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Create authentication object for user permissions\n",
+    "# IF unused, SDK will automatically check for default kubeconfig, then in-cluster config\n",
+    "# KubeConfigFileAuthentication can also be used to specify kubeconfig path manually\n",
+    "auth = TokenAuthentication(\n",
+    "    token = \"XXXXX\",\n",
+    "    server = \"XXXXX\",\n",
+    "    skip_tls=False\n",
+    ")\n",
+    "auth.login()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "bc27f84c",
+   "metadata": {},
+   "source": [
+    "Here, we want to define our cluster by specifying the resources we require for our batch workload. Below, we define our cluster object (which generates a corresponding RayCluster).\n",
+    "\n",
+    "NOTE: 'quay.io/modh/ray:2.35.0-py39-cu121' is the default image used by the CodeFlare SDK for creating a RayCluster resource. \n",
+    "If you have your own Ray image which suits your purposes, specify it in image field to override the default image."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "0f4bc870-091f-4e11-9642-cba145710159",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Create and configure our cluster object\n",
+    "# The SDK will try to find the name of your default local queue based on the annotation \"kueue.x-k8s.io/default-queue\": \"true\" unless you specify the local queue manually below\n",
+    "cluster = Cluster(ClusterConfiguration(\n",
+    "    name='raytest', \n",
+    "    head_cpu_requests='500m',\n",
+    "    head_cpu_limits='500m',\n",
+    "    head_memory_requests=2,\n",
+    "    head_memory_limits=2,\n",
+    "    head_extended_resource_requests={'nvidia.com/gpu':0}, # For GPU enabled workloads set the head_extended_resource_requests and worker_extended_resource_requests\n",
+    "    worker_extended_resource_requests={'nvidia.com/gpu':0},\n",
+    "    num_workers=2,\n",
+    "    worker_cpu_requests='250m',\n",
+    "    worker_cpu_limits=1,\n",
+    "    worker_memory_requests=4,\n",
+    "    worker_memory_limits=4,\n",
+    "    # image=\"\", # Optional Field \n",
+    "    write_to_file=False, # When enabled Ray Cluster yaml files are written to /HOME/.codeflare/resources \n",
+    "    # local_queue=\"local-queue-name\" # Specify the local queue manually\n",
+    "))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "12eef53c",
+   "metadata": {},
+   "source": [
+    "Next, we want to bring our cluster up, so we call the `up()` function below to submit our Ray Cluster onto the queue, and begin the process of obtaining our resource cluster."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "f0884bbc-c224-4ca0-98a0-02dfa09c2200",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Bring up the cluster\n",
+    "cluster.up()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "657ebdfb",
+   "metadata": {},
+   "source": [
+    "Now, we want to check on the status of our resource cluster, and wait until it is finally ready for use."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "3c1b4311-2e61-44c9-8225-87c2db11363d",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "cluster.status()"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "a99d5aff",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "cluster.wait_ready()"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "df71c1ed",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "cluster.status()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "b3a55fe4",
+   "metadata": {},
+   "source": [
+    "Let's quickly verify that the specs of the cluster are as expected."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "7fd45bc5-03c0-4ae5-9ec5-dd1c30f1a084",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "cluster.details()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "5af8cd32",
+   "metadata": {},
+   "source": [
+    "Finally, we bring our resource cluster down and release/terminate the associated resources, bringing everything back to the way it was before our cluster was brought up."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "5f36db0f-31f6-4373-9503-dc3c1c4c3f57",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "cluster.down()"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "0d41b90e",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "auth.logout()"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.9.19"
+  },
+  "vscode": {
+   "interpreter": {
+    "hash": "f9f85f796d01129d0dd105a088854619f454435301f6ffec2fea96ecbd9be4ac"
+   }
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}