Fastapi mangum extras (#234)

* add pip-compile for more reliable installation Signed-off-by: Niels Bantilan <[email protected]> * fix unit and integration tests Signed-off-by: Niels Bantilan <[email protected]> * mangum extra Signed-off-by: Niels Bantilan <[email protected]> * move uvicorn to fastapi extra Signed-off-by: Niels Bantilan <[email protected]> * update ci Signed-off-by: Niels Bantilan <[email protected]> * update deps Signed-off-by: Niels Bantilan <[email protected]> * update pre-commit Signed-off-by: Niels Bantilan <[email protected]> * update notebooks Signed-off-by: Niels Bantilan <[email protected]> * update requirements-dev with mangum Signed-off-by: Niels Bantilan <[email protected]> * update Signed-off-by: Niels Bantilan <[email protected]> * pin mangum Signed-off-by: Niels Bantilan <[email protected]> * install transformers[torch] Signed-off-by: Niels Bantilan <[email protected]> * lint Signed-off-by: Niels Bantilan <[email protected]> * fix bentoml for py3.11 Signed-off-by: Niels Bantilan <[email protected]> * add py11 ci dockerfile Signed-off-by: Niels Bantilan <[email protected]> * use buildx in CI Signed-off-by: Niels Bantilan <[email protected]> * try compressing docker image Signed-off-by: Niels Bantilan <[email protected]> * testing integration tests Signed-off-by: Niels Bantilan <[email protected]> * delete local image for sandbox docker build Signed-off-by: Niels Bantilan <[email protected]> * disable some tests in ci Signed-off-by: Niels Bantilan <[email protected]> * update Signed-off-by: Niels Bantilan <[email protected]> --------- Signed-off-by: Niels Bantilan <[email protected]>
unionai-oss · Aug 1, 2023 · 4fea727 · 4fea727
1 parent 8e1f890
commit 4fea727
Show file tree

Hide file tree

Showing 28 changed files with 193 additions and 149 deletions.
diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml
@@ -18,7 +18,7 @@ jobs:
       fail-fast: false
       matrix:
         test-dir: ["unit", "integration"]
-        python-version: ["3.7", "3.8", "3.9", "3.10"]
+        python-version: ["3.8", "3.9", "3.10", "3.11"]
 
     env:
       UNIONML_CI: "1"
@@ -27,6 +27,8 @@ jobs:
 
     steps:
     - uses: actions/checkout@v2
+    - name: Set up Docker Buildx
+      uses: docker/setup-buildx-action@v2
     - name: Set up Python ${{ matrix.python-version }}
       uses: actions/setup-python@v2
       with:
@@ -42,7 +44,8 @@ jobs:
       run: |
         python -m pip install --upgrade pip
         pip install awscli
-        pip install -r requirements.txt -r requirements-dev.txt -r requirements-docs.txt
+        pip install typing_extensions  # required for mangum install
+        pip install -r requirements-docs.txt
         pip install --no-deps -e .
     - name: Pip info
       run: python -m pip list
@@ -55,14 +58,11 @@ jobs:
     - name: Setup Flyte Sandbox
       if: ${{ matrix.test-dir == 'integration' }}
       run: |
-        flytectl sandbox start --source .
+        flytectl demo start
         flytectl config init
     - name: Pytest
       run: |
         pytest tests/${{ matrix.test-dir }}
     - name: Build Docs
-      # build error AttributeError: 'document' object has no attribute 'findall'
-      # in the furo theme, see: https://github.com/unionai-oss/unionml/runs/6786848582?check_suite_focus=true#step:8:2531
-      # excluding docs build step from 3.7 for now.
-      if: ${{ matrix.python-version != '3.7' && matrix.test-dir == 'unit' }}
+      if: ${{ matrix.test-dir == 'unit' }}
       run: make docs
diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml
@@ -8,12 +8,12 @@ repos:
   hooks:
     - id: black
 - repo: https://github.com/PyCQA/isort
-  rev:  5.10.1
+  rev:  5.12.0
   hooks:
     - id: isort
       args: ["--profile", "black"]
 - repo: https://github.com/pre-commit/pre-commit-hooks
-  rev: v4.1.0
+  rev: v4.2.0
   hooks:
     - id: check-yaml
       args: [ "--unsafe" ]
@@ -33,6 +33,10 @@ repos:
       pass_filenames: false
       additional_dependencies:
       - types-requests
+- repo: https://github.com/conorfalvey/check_pdb_hook
+  rev: 0.0.9
+  hooks:
+    - id: check_pdb_hook
 - repo: local
   hooks:
   - id: convert-myst-to-ipynb

diff --git a/Makefile b/Makefile
@@ -9,3 +9,6 @@ docs:
 
 quick-docs:
 	NB_EXECUTION_MODE=off $(MAKE) docs
+
+setup:
+	pip install -r requirements-dev.txt
diff --git a/ci/py37/Dockerfile → ci/py311/Dockerfile b/ci/py37/Dockerfile → ci/py311/Dockerfile
@@ -1,4 +1,4 @@
-FROM python:3.7-slim-buster
+FROM python:3.11-slim-buster
 
 WORKDIR /root
 ENV VENV /opt/venv

diff --git a/docs/notebooks/mnist.ipynb b/docs/notebooks/mnist.ipynb
@@ -2,15 +2,15 @@
  "cells": [
   {
    "cell_type": "markdown",
-   "id": "a75f89386fbeda94d3e4eccd0a4b76d0",
+   "id": "4798d95a2aa741e49bf5ad316d92320a",
    "metadata": {},
    "source": [
     "# MNIST: Digits Classification"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "6bd730481e96f527bf560252946afa83",
+   "id": "9b87507ae43f2a53ade1732aef459954",
    "metadata": {
     "tags": [
      "add-colab-badge"
@@ -22,7 +22,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "50bb5dc426b848b29f9a1f1090844c5c",
+   "id": "8a98b0c432c8aa1addbd7c375393c1a5",
    "metadata": {},
    "source": [
     "The MNIST dataset is considered to be the \"hello world\" dataset of machine\n",
@@ -42,7 +42,7 @@
   {
    "cell_type": "code",
    "execution_count": null,
-   "id": "1076f494dde02aee04abe186ba5be442",
+   "id": "6218ce653dbc9305d4c6a323308a6de4",
    "metadata": {
     "tags": [
      "remove-cell"
@@ -56,7 +56,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "b4ab3e8841ab4e74a6994979ab243687",
+   "id": "2a5fe5a56a3579bd6eac791774214d37",
    "metadata": {
     "tags": [
      "remove-cell"
@@ -70,7 +70,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "f60a216a0502d01dee018b7732de6234",
+   "id": "2e3848974c88806207b69f4b2db8dab9",
    "metadata": {},
    "source": [
     "## Setup and importing libraries\n",
@@ -81,7 +81,7 @@
   {
    "cell_type": "code",
    "execution_count": null,
-   "id": "f0550d7dae055e028bba12c377655b7f",
+   "id": "7107ee82571c64ff67175678f96f7286",
    "metadata": {},
    "outputs": [],
    "source": [
@@ -103,7 +103,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "8ff99219432894cd9a5dda340d1e1b39",
+   "id": "c5c1e2d27147ca3be50d5a50dc6fd692",
    "metadata": {},
    "source": [
     "Let's break down the code cell above.\n",
@@ -129,7 +129,7 @@
   {
    "cell_type": "code",
    "execution_count": null,
-   "id": "8ae4871489db9871cb19d7c08d1b486b",
+   "id": "5942db20bd08b3dc154be6981b65f80b",
    "metadata": {},
    "outputs": [],
    "source": [
@@ -142,7 +142,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "93a2b5553bc5acb95c7557d12f172326",
+   "id": "94b609f8782897a85c3a178bfc94bcb4",
    "metadata": {},
    "source": [
     "We do this so we don't have to re-download the dataset it every time we need to\n",
@@ -156,7 +156,7 @@
   {
    "cell_type": "code",
    "execution_count": null,
-   "id": "f14d6452e97542d3e1e7dc0f309f37dd",
+   "id": "6fc2ab05dacbcf86e4a5b510c3c91348",
    "metadata": {},
    "outputs": [],
    "source": [
@@ -211,7 +211,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "25b60d28342dcfe96537f8b7bb11977a",
+   "id": "a846e3f8c24799b0c13bdcaa43b27771",
    "metadata": {},
    "source": [
     "The `Dataset` and `Model` objects expose function decorators where we define\n",
@@ -236,7 +236,7 @@
   {
    "cell_type": "code",
    "execution_count": null,
-   "id": "c1e477a01ea71f3b2d11b38518893367",
+   "id": "b16f523c696bf4020c865c84311876cc",
    "metadata": {},
    "outputs": [],
    "source": [
@@ -252,7 +252,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "7759d72e372c309faac690fe33db2928",
+   "id": "181110f25a9900a68af89efafe88194f",
    "metadata": {},
    "source": [
     "Note that we pass a dictionary of `hyperparameters` when we invoke\n",
@@ -273,7 +273,7 @@
   {
    "cell_type": "code",
    "execution_count": null,
-   "id": "dd2db9775cf4746d791db8f49d7eb48d",
+   "id": "3ac4c807a1c06e7c190bf05e557982af",
    "metadata": {},
    "outputs": [],
    "source": [
@@ -291,7 +291,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "14c9b2d756929c981cd0f157e6f1b51f",
+   "id": "ca1faf97556cdf19364962acfc0a826d",
    "metadata": {},
    "source": [
     "We also need to take care to handle the `None` case when we press the `clear`\n",
@@ -301,7 +301,7 @@
   {
    "cell_type": "code",
    "execution_count": null,
-   "id": "3b9eda693333f0352081b4c2ab2237de",
+   "id": "875ff8a49e9ce31289f264d95df720b1",
    "metadata": {
     "tags": [
      "remove-output"
@@ -322,7 +322,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "e5a0d6d4fef4e87e8af91825cdc4724e",
+   "id": "025b1a6a6396ab9ecb09b68a35b4d1de",
    "metadata": {},
    "source": [
     "You might notice that the model may not perform as well as you might expect...\n",