Copy_of_compute_KNN_features.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "anaconda-cloud": {},
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.6.0"
    },
    "colab": {
      "name": "Copy of compute_KNN_features.ipynb",
      "provenance": [],
      "collapsed_sections": []
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "b1L5Qv6OEKXd"
      },
      "source": [
        "Version 1.1.1"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "A5qpGXgBEKXe"
      },
      "source": [
        "# The task"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "N1T3NIxYEKXe"
      },
      "source": [
        "In this assignment you will need to implement features, based on nearest neighbours. \n",
        "\n",
        "KNN classifier (regressor) is a very powerful model, when the features are homogeneous and it is a very common practice to use KNN as first level model. In this homework we will extend KNN model and compute more features, based on nearest neighbors and their distances. \n",
        "\n",
        "You will need to implement a number of features, that were one of the key features, that leaded the instructors to prizes in [Otto](https://www.kaggle.com/c/otto-group-product-classification-challenge) and [Springleaf](https://www.kaggle.com/c/springleaf-marketing-response) competitions. Of course, the list of features you will need to implement can be extended, in fact in competitions the list was at least 3 times larger. So when solving a real competition do not hesitate to make up your own features.   \n",
        "\n",
        "You can optionally implement multicore feature computation. Nearest neighbours are hard to compute so it is preferable to have a parallel version of the algorithm. In fact, it is really a cool skill to know how to use `multiprocessing`, `joblib` and etc. In this homework you will have a chance to see the benefits of parallel algorithm. "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "54F0m1AzEKXf"
      },
      "source": [
        "# Check your versions"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xkRNsPJxEKXg"
      },
      "source": [
        "Some functions we use here are not present in old versions of the libraries, so make sure you have up-to-date software. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "WeTn_wufEMQe",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "1dad511a-7d29-4f71-c1ff-9a62f2940550"
      },
      "source": [
        "!git clone https://github.com/hse-aml/competitive-data-science"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "fatal: destination path 'competitive-data-science' already exists and is not an empty directory.\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "N8-CXLalEKXg",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 235
        },
        "outputId": "ce489dd1-34c6-4936-cffe-2b39480ce2f7"
      },
      "source": [
        "import numpy as np\n",
        "import pandas as pd \n",
        "import sklearn\n",
        "import scipy.sparse \n",
        "!pip install scikit-learn==0.19.0\n",
        "!pip install scipy==0.19.1\n",
        "!pip install numpy==1.13.1\n",
        "!pip install pandas==0.20.3\n",
        "for p in [np, pd, sklearn, scipy]:\n",
        "    print (p.__name__, p.__version__)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Requirement already satisfied: scikit-learn==0.19.0 in /usr/local/lib/python3.6/dist-packages (0.19.0)\n",
            "Requirement already satisfied: scipy==0.19.1 in /usr/local/lib/python3.6/dist-packages (0.19.1)\n",
            "Requirement already satisfied: numpy>=1.8.2 in /usr/local/lib/python3.6/dist-packages (from scipy==0.19.1) (1.13.1)\n",
            "Requirement already satisfied: numpy==1.13.1 in /usr/local/lib/python3.6/dist-packages (1.13.1)\n",
            "Requirement already satisfied: pandas==0.20.3 in /usr/local/lib/python3.6/dist-packages (0.20.3)\n",
            "Requirement already satisfied: python-dateutil>=2 in /usr/local/lib/python3.6/dist-packages (from pandas==0.20.3) (2.8.1)\n",
            "Requirement already satisfied: numpy>=1.7.0 in /usr/local/lib/python3.6/dist-packages (from pandas==0.20.3) (1.13.1)\n",
            "Requirement already satisfied: pytz>=2011k in /usr/local/lib/python3.6/dist-packages (from pandas==0.20.3) (2018.9)\n",
            "Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.6/dist-packages (from python-dateutil>=2->pandas==0.20.3) (1.15.0)\n",
            "numpy 1.13.1\n",
            "pandas 0.20.3\n",
            "sklearn 0.19.0\n",
            "scipy 0.19.1\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "dGiwN4xNK93A"
      },
      "source": [
        ""
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "yqT67dNXEKXk"
      },
      "source": [
        "The versions should be not less than:\n",
        "\n",
        "    numpy 1.13.1\n",
        "    pandas 0.20.3\n",
        "    sklearn 0.19.0\n",
        "    scipy 0.19.1\n",
        "   \n",
        "**IMPORTANT!** The results with `scipy=1.0.0` will be different! Make sure you use _exactly_ version `0.19.1`."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "OXONMXtCEKXl"
      },
      "source": [
        "# Load data"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "K5RkxnIUEKXl"
      },
      "source": [
        "Learn features and labels. These features are actually OOF predictions of linear models."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Ck2xmLQQEKXm"
      },
      "source": [
        "train_path = 'competitive-data-science/readonly/KNN_features_data/X.npz'\n",
        "train_labels = 'competitive-data-science/readonly/KNN_features_data/Y.npy'\n",
        "\n",
        "test_path = 'competitive-data-science/readonly/KNN_features_data/X_test.npz'\n",
        "test_labels = 'competitive-data-science/readonly/KNN_features_data/Y_test.npy'\n",
        "\n",
        "# Train data\n",
        "X = scipy.sparse.load_npz(train_path)\n",
        "Y = np.load(train_labels)\n",
        "\n",
        "# Test data\n",
        "X_test = scipy.sparse.load_npz(test_path)\n",
        "Y_test = np.load(test_labels)\n",
        "\n",
        "# Out-of-fold features we loaded above were generated with n_splits=4 and skf seed 123\n",
        "# So it is better to use seed 123 for generating KNN features as well \n",
        "skf_seed = 123\n",
        "n_splits = 4"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "hVyeD7iaseq9",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 67
        },
        "outputId": "c40affa9-e0ee-46d8-85ae-0ec010159725"
      },
      "source": [
        "np.bincount(Y)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([5413, 5440,  563,  511, 5375,  212,  551,  554,  552,  544,  547,\n",
              "        528,  543, 5377, 2665, 2699,  545,  570,  575, 2745,   12, 5367,\n",
              "       5361,  515,  556,  355,  527,  546,  526])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 48
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "51qUHnp2se8S"
      },
      "source": [
        "from sklearn.base import BaseEstimator, ClassifierMixin\n",
        "from sklearn.neighbors import NearestNeighbors\n",
        "from multiprocessing import Pool\n",
        "import numpy as np"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "YUoajqmwsfNK"
      },
      "source": [
        ""
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "dtln3t_3sfdu"
      },
      "source": [
        ""
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Rv_lbujBEKXp"
      },
      "source": [
        "Below you need to implement features, based on nearest neighbors."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "hBg3lkXKEKXq"
      },
      "source": [
        "\n",
        "\n",
        "\n",
        "class NearestNeighborsFeats(BaseEstimator, ClassifierMixin):\n",
        "    '''\n",
        "        This class should implement KNN features extraction \n",
        "    '''\n",
        "    def __init__(self, n_jobs, k_list, metric, n_classes=None, n_neighbors=None, eps=1e-6):\n",
        "        self.n_jobs = n_jobs\n",
        "        self.k_list = k_list\n",
        "        self.metric = metric\n",
        "        \n",
        "        if n_neighbors is None:\n",
        "            self.n_neighbors = max(k_list) \n",
        "        else:\n",
        "            self.n_neighbors = n_neighbors\n",
        "            \n",
        "        self.eps = eps        \n",
        "        self.n_classes_ = n_classes\n",
        "    \n",
        "    def fit(self, X, y):\n",
        "        '''\n",
        "            Set's up the train set and self.NN object\n",
        "        '''\n",
        "        # Create a NearestNeighbors (NN) object. We will use it in `predict` function \n",
        "        self.NN = NearestNeighbors(n_neighbors=max(self.k_list), \n",
        "                                      metric=self.metric, \n",
        "                                      n_jobs=1, \n",
        "                                      algorithm='brute' if self.metric=='cosine' else 'auto')\n",
        "        self.NN.fit(X)\n",
        "        \n",
        "        # Store labels \n",
        "        self.y_train = y\n",
        "        \n",
        "        # Save how many classes we have\n",
        "        self.n_classes = np.unique(y).shape[0] if self.n_classes_ is None else self.n_classes_\n",
        "        \n",
        "        \n",
        "    def predict(self, X):       \n",
        "        '''\n",
        "            Produces KNN features for every object of a dataset X\n",
        "        '''\n",
        "        if self.n_jobs == 1:\n",
        "            test_feats = []\n",
        "            for i in range(X.shape[0]):\n",
        "                test_feats.append(self.get_features_for_one(X[i:i+1]))\n",
        "        else:\n",
        "          pool = Pool(self.n_jobs)\n",
        "          gen = (X[i:i+1] for i in range(X.shape[0]))\n",
        "          test_feats = pool.map(self.get_features_for_one, gen)\n",
        "          pool.close()\n",
        "          pool.join()\n",
        "\n",
        "           # Comment out this line once you implement the code\n",
        "           # assert False, 'You need to implement it for n_jobs > 1'\n",
        "\n",
        "        return np.vstack(test_feats)\n",
        "        \n",
        "        \n",
        "    def get_features_for_one(self, x):\n",
        "        '''\n",
        "            Computes KNN features for a single object `x`\n",
        "        '''\n",
        "\n",
        "        NN_output = self.NN.kneighbors(x)\n",
        "        \n",
        "        # Vector of size `n_neighbors`\n",
        "        # Stores indices of the neighbors\n",
        "        neighs = NN_output[1][0]\n",
        "        \n",
        "        # Vector of size `n_neighbors`\n",
        "        # Stores distances to corresponding neighbors\n",
        "        neighs_dist = NN_output[0][0] \n",
        "\n",
        "        # Vector of size `n_neighbors`\n",
        "        # Stores labels of corresponding neighbors\n",
        "        neighs_y = self.y_train[neighs] \n",
        "        \n",
        "        ## ========================================== ##\n",
        "        ##              YOUR CODE BELOW\n",
        "        ## ========================================== ##\n",
        "        \n",
        "        # We will accumulate the computed features here\n",
        "        # Eventually it will be a list of lists or np.arrays\n",
        "        # and we will use np.hstack to concatenate those\n",
        "        return_list = [] \n",
        "\n",
        "\n",
        "        ''' \n",
        "            1. Fraction of objects of every class.\n",
        "               It is basically a KNNСlassifiers predictions.\n",
        "\n",
        "               Take a look at `np.bincount` function, it can be very helpful\n",
        "               Note that the values should sum up to one\n",
        "        '''\n",
        "        for k in self.k_list:\n",
        "            freq_count = np.bincount(neighs_y[0:k], minlength=self.n_classes)\n",
        "            feats = freq_count/(freq_count.sum())\n",
        "            assert len(feats) == self.n_classes\n",
        "            return_list += [feats]\n",
        "        \n",
        "        '''\n",
        "            2. Same label streak: the largest number N, \n",
        "               such that N nearest neighbors have the same label.\n",
        "               \n",
        "               What can help you: `np.where`\n",
        "        '''\n",
        "        t = neighs_y[0]\n",
        "        feats = 0\n",
        "        if (np.where(neighs_y[0:]!=t)[0].shape[0]!=0): \n",
        "          feats = np.where(neighs_y[0:]!=t)[0][0]\n",
        "        else:\n",
        "          feats = neighs_y.shape[0]\n",
        "        return_list += [feats]\n",
        "        \n",
        "        '''\n",
        "            3. Minimum distance to objects of each class\n",
        "               Find the first instance of a class and take its distance as features.\n",
        "               \n",
        "               If there are no neighboring objects of some classes, \n",
        "               Then set distance to that class to be 999.\n",
        "\n",
        "               `np.where` might be helpful\n",
        "        '''\n",
        "        feats = []\n",
        "        #### to do some stuff\n",
        "        for i in range(self.n_classes):\n",
        "          if (np.where(neighs_y[0:]==i)[0].shape[0])!=0:\n",
        "            t = np.where(neighs_y[0:]==i)[0][0]\n",
        "            feats.append(neighs_dist[t])\n",
        "          else:\n",
        "            feats.append(999)\n",
        "        assert len(feats) == self.n_classes\n",
        "        return_list += [feats]\n",
        "        \n",
        "        '''\n",
        "            4. Minimum *normalized* distance to objects of each class\n",
        "               As 3. but we normalize (divide) the distances\n",
        "               by the distance to the closest neighbor.\n",
        "               \n",
        "               If there are no neighboring objects of some classes, \n",
        "               Then set distance to that class to be 999.\n",
        "               \n",
        "               Do not forget to add self.eps to denominator.\n",
        "        '''\n",
        "        feats = []\n",
        "        ls_d = neighs_dist[0]\n",
        "        for c in range(self.n_classes):\n",
        "          if (np.where(neighs_y[0:]==c)[0].shape[0])!=0:\n",
        "            t = np.where(neighs_y[0:]==c)[0][0]\n",
        "            feats.append(neighs_dist[t]/(ls_d+self.eps))\n",
        "          else:\n",
        "            feats.append(999)\n",
        "        assert len(feats) == self.n_classes\n",
        "        return_list += [feats]\n",
        "        \n",
        "        '''\n",
        "            5. \n",
        "               5.1 Distance to Kth neighbor\n",
        "                   Think of this as of quantiles of a distribution\n",
        "               5.2 Distance to Kth neighbor normalized by \n",
        "                   distance to the first neighbor\n",
        "               \n",
        "               feat_51, feat_52 are answers to 5.1. and 5.2.\n",
        "               should be scalars\n",
        "               \n",
        "               Do not forget to add self.eps to denominator.\n",
        "        '''\n",
        "        for k in self.k_list:\n",
        "            feat_51 = neighs_dist[k-1] \n",
        "            feat_52 = (neighs_dist[k-1])/(ls_d + self.eps)\n",
        "            return_list += [[feat_51, feat_52]]\n",
        "        '''\n",
        "            6. Mean distance to neighbors of each class for each K from `k_list` \n",
        "                   For each class select the neighbors of that class among K nearest neighbors \n",
        "                   and compute the average distance to those objects\n",
        "                   \n",
        "                   If there are no objects of a certain class among K neighbors, set mean distance to 999\n",
        "                   \n",
        "               You can use `np.bincount` with appropriate weights\n",
        "               Don't forget, that if you divide by something, \n",
        "               You need to add `self.eps` to denominator.\n",
        "\n",
        "\n",
        "               feats = np.zeros((self.n_classes,1))\n",
        "        for k in self.k_list:\n",
        "          for c in range(self.n_classes):\n",
        "            if ((neighs_y==c).sum()!=0):\n",
        "              feats[c] = neighs_dist[neighs_y==c].sum()/freq_count[c]\n",
        "            else:\n",
        "              feats[c] = 999\n",
        "        '''\n",
        "        \n",
        "        for k in self.k_list:\n",
        "          feats = [0 for i in range(self.n_classes)]\n",
        "          for c in range(self.n_classes):\n",
        "            if (neighs_y[0:k]==c).sum()!=0 :\n",
        "              feats[c] = (np.where(neighs_y[0:k]==c,1,0)*neighs_dist[0:k])[0:k].sum()/(np.bincount(neighs_y[0:k], minlength=self.n_classes)[c] + self.eps)\n",
        "            else:\n",
        "              feats[c] = 999\n",
        "          assert len(feats) == self.n_classes\n",
        "          return_list += [feats]\n",
        "        \n",
        "        \n",
        "        # merge\n",
        "        knn_feats = np.hstack(return_list)\n",
        "        \n",
        "        assert knn_feats.shape == (239,) or knn_feats.shape == (239, 1)\n",
        "        return knn_feats"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "tTXaeF5kEKXt"
      },
      "source": [
        "## Sanity check"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "DWKLtgQGEKXu"
      },
      "source": [
        "To make sure you've implemented everything correctly we provide you the correct features for the first 50 objects."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "gy539BeHEKXu",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "204260a6-769a-4fe0-fcac-31c146748c31"
      },
      "source": [
        "# a list of K in KNN, starts with one \n",
        "k_list = [3, 8, 32]\n",
        "\n",
        "# Load correct features\n",
        "true_knn_feats_first50 = np.load('competitive-data-science/readonly/KNN_features_data/knn_feats_test_first50.npy')\n",
        "\n",
        "# Create instance of our KNN feature extractor\n",
        "NNF = NearestNeighborsFeats(n_jobs=1, k_list=k_list, metric='minkowski')\n",
        "\n",
        "# Fit on train set\n",
        "NNF.fit(X, Y)\n",
        "\n",
        "# Get features for test\n",
        "test_knn_feats = NNF.predict(X_test[:50])\n",
        "\n",
        "# This should be zero\n",
        "print ('Deviation from ground truth features: %f' % np.abs(test_knn_feats - true_knn_feats_first50).sum())\n",
        "\n",
        "deviation =np.abs(test_knn_feats - true_knn_feats_first50).sum(0)\n",
        "for m in np.where(deviation > 1e-3)[0]: \n",
        "    p = np.where(np.array([87, 88, 117, 146, 152, 239]) > m)[0][0]\n",
        "    print ('There is a problem in feature %d, which is a part of section %d.' % (m, p + 1))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Deviation from ground truth features: 0.000000\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "schyDsT1EKXx"
      },
      "source": [
        "Now implement parallel computations and compute features for the train and test sets. "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "aHvgZtgvEKXy"
      },
      "source": [
        "## Get features for test"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "s2jywLM2EKXy"
      },
      "source": [
        "Now compute features for the whole test set."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "9CmIofxaEKXz",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 50
        },
        "outputId": "11d8963a-e4c9-409c-b9a0-ae97d0dc8807"
      },
      "source": [
        "for metric in ['minkowski', 'cosine']:\n",
        "    print (metric)\n",
        "    \n",
        "    # Create instance of our KNN feature extractor\n",
        "    NNF = NearestNeighborsFeats(n_jobs=4, k_list=k_list, metric=metric)\n",
        "    \n",
        "    # Fit on train set\n",
        "    NNF.fit(X, Y)\n",
        "\n",
        "    # Get features for test\n",
        "    test_knn_feats = NNF.predict(X_test)\n",
        "    \n",
        "    # Dump the features to disk\n",
        "    np.save('competitive-data-science/knn_feats_%s_test.npy' % metric , test_knn_feats)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "minkowski\n",
            "cosine\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "XoRGjkHvEKX3"
      },
      "source": [
        "## Get features for train"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "g8_fuTudEKX3"
      },
      "source": [
        "Compute features for train, using out-of-fold strategy."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "x54AkAToEKX4",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 50
        },
        "outputId": "3ec00ea1-0d4c-405f-c0c7-301a349f54c8"
      },
      "source": [
        "# Differently from other homework we will not implement OOF predictions ourselves\n",
        "# but use sklearn's `cross_val_predict`\n",
        "from sklearn.model_selection import cross_val_predict\n",
        "from sklearn.model_selection import StratifiedKFold\n",
        "\n",
        "# We will use two metrics for KNN\n",
        "for metric in ['minkowski', 'cosine']:\n",
        "    print (metric)\n",
        "    \n",
        "    # Set up splitting scheme, use StratifiedKFold\n",
        "    # use skf_seed and n_splits defined above with shuffle=True\n",
        "    skf = StratifiedKFold(shuffle=True,n_splits=4, random_state=skf_seed)\n",
        "    \n",
        "    # Create instance of our KNN feature extractor\n",
        "    # n_jobs can be larger than the number of cores\n",
        "    NNF = NearestNeighborsFeats(n_jobs=4, k_list=k_list, metric=metric)\n",
        "    \n",
        "    # Get KNN features using OOF use cross_val_predict with right parameters\n",
        "    preds = cross_val_predict(NNF, X, Y, cv=skf, n_jobs=1)\n",
        "    \n",
        "    # Save the features\n",
        "    np.save('competitive-data-science/knn_feats_%s_train.npy' % metric, preds)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "minkowski\n",
            "cosine\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Aa6bowttEKX7"
      },
      "source": [
        "# Submit"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8t6-GUqyEKX7"
      },
      "source": [
        "If you made the above cells work, just run the following cell to produce a number to submit."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "_OcK5GiDEKX7",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "b287c8b1-a227-480a-e261-ea85834901b6"
      },
      "source": [
        "s = 0\n",
        "for metric in ['minkowski', 'cosine']:\n",
        "    knn_feats_train = np.load('competitive-data-science/knn_feats_%s_train.npy' % metric)\n",
        "    knn_feats_test = np.load('competitive-data-science/knn_feats_%s_test.npy' % metric)\n",
        "    \n",
        "    s += knn_feats_train.mean() + knn_feats_test.mean()\n",
        "    \n",
        "answer = np.floor(s)\n",
        "print (answer)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "3838.0\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "4ZJwIDZdjzyI"
      },
      "source": [
        "import requests\n",
        "import json\n",
        "import numpy as np\n",
        "from collections import OrderedDict\n",
        "\n",
        "def array_to_hash(x):\n",
        "    x_tupled = None\n",
        "    if type(x) == list:\n",
        "        x_tupled = tuple(x)\n",
        "    elif type(x) == np.ndarray:\n",
        "        x_tupled = tuple(list(x.flatten()))\n",
        "    elif type(x) == tuple:\n",
        "        x_tupled = x\n",
        "    else:\n",
        "        raise RuntimeError('unexpected type of input: {}'.format(type(x)))\n",
        "    return hash(tuple(map(float, x_tupled)))\n",
        "\n",
        "def almostEqual(x, y):\n",
        "    return abs(x - y) < 1e-3\n",
        "\n",
        "\n",
        "class Grader(object):\n",
        "    def __init__(self):\n",
        "        self.submission_page = 'https://hub.coursera-apps.org/api/onDemandProgrammingScriptSubmissions.v1'\n",
        "        self.assignment_key = 'r2N4iqFlEeeRFQqEddeEzg'\n",
        "        self.parts = OrderedDict([\n",
        "                    ('1O8kU', 'statistic')])\n",
        "        self.answers = {key: None for key in self.parts}\n",
        "\n",
        "    @staticmethod\n",
        "    def ravel_output(output):\n",
        "        '''\n",
        "           If student accedentally submitted np.array with one\n",
        "           element instead of number, this function will submit\n",
        "           this number instead\n",
        "        '''\n",
        "        if isinstance(output, np.ndarray) and output.size == 1:\n",
        "            output = output.item(0)\n",
        "        return output\n",
        "\n",
        "    def submit(self, email, token):\n",
        "        submission = {\n",
        "                    \"assignmentKey\": self.assignment_key, \n",
        "                    \"submitterEmail\": email, \n",
        "                    \"secret\": token, \n",
        "                    \"parts\": {}\n",
        "                  }\n",
        "        for part, output in self.answers.items():\n",
        "            if output is not None:\n",
        "                submission[\"parts\"][part] = {\"output\": output}\n",
        "            else:\n",
        "                submission[\"parts\"][part] = dict()\n",
        "        request = requests.post(self.submission_page, data=json.dumps(submission))\n",
        "        response = request.json()\n",
        "        if request.status_code == 201:\n",
        "            print('Submitted to Coursera platform. See results on assignment page!')\n",
        "        elif u'details' in response and u'learnerMessage' in response[u'details']:\n",
        "            print(response[u'details'][u'learnerMessage'])\n",
        "        else:\n",
        "            print(\"Unknown response from Coursera: {}\".format(request.status_code))\n",
        "            print(response)\n",
        "\n",
        "    def status(self):\n",
        "        print(\"You want to submit these numbers:\")\n",
        "        for part_id, part_name in self.parts.items():\n",
        "            answer = self.answers[part_id]\n",
        "            if answer is None:\n",
        "                answer = '-'*10\n",
        "            print(\"Task {}: {}\".format(part_name, answer))\n",
        "               \n",
        "    def submit_part(self, part, output):\n",
        "        self.answers[part] = output\n",
        "        print(\"Current answer for task {} is: {}\".format(self.parts[part], output))\n",
        "\n",
        "    def submit_tag(self, tag, output):\n",
        "        part_id = [k for k, v in self.parts.items() if v == tag]\n",
        "        if len(part_id)!=1:\n",
        "            raise RuntimeError('cannot match tag with part_id: found {} matches'.format(len(part_id)))\n",
        "        part_id = part_id[0]\n",
        "        self.submit_part(part_id, str(self.ravel_output(output)))"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dNky4FDREKX-"
      },
      "source": [
        "Submit!"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "a2kTUpJwEKX_",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 84
        },
        "outputId": "708ddca5-af4b-4c34-d418-2e2bfb530f11"
      },
      "source": [
        "grader = Grader()\n",
        "\n",
        "grader.submit_tag('statistic', answer)\n",
        "\n",
        "STUDENT_EMAIL = \"surya19@iiserb.ac.in\"\n",
        "STUDENT_TOKEN = \"xHywfxY4ORNN5Ofu\"\n",
        "grader.status()\n",
        "\n",
        "grader.submit(STUDENT_EMAIL, STUDENT_TOKEN)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Current answer for task statistic is: 3838.0\n",
            "You want to submit these numbers:\n",
            "Task statistic: 3838.0\n",
            "Submitted to Coursera platform. See results on assignment page!\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "zUQBUQ5cjP1r"
      },
      "source": [
        ""
      ],
      "execution_count": null,
      "outputs": []
    }
  ]
}