Spam_Classifier.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h1>Spam Classifier for SMS messages</h1>\n",
    "<br />\n",
    "This notebook shows how to implement a basic spam classifier for SMS messages using Keras and Tensorflow.\n",
    "The idea is to use the SMS spam collection dataset available at https://archive.ics.uci.edu/ml/datasets/sms+spam+collection to train a neural network by leveraging on the built-in container for TensorFlow in Amazon SageMaker."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's get started by getting the Amazon SageMaker session and the current execution role, using the Amazon SageMaker high-level SDK for Python."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "arn:aws:iam::825935527263:role/service-role/AmazonSageMaker-ExecutionRole-20180311T170786\n"
     ]
    }
   ],
   "source": [
    "import os\n",
    "import sagemaker\n",
    "from sagemaker import get_execution_role\n",
    "\n",
    "sagemaker_session = sagemaker.Session()\n",
    "\n",
    "role = get_execution_role()\n",
    "print(role)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We now download the spam collection dataset, unzip it and read the first 10 rows."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 117,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current\n",
      "                                 Dload  Upload   Total   Spent    Left  Speed\n",
      "100  198k  100  198k    0     0  64290      0  0:00:03  0:00:03 --:--:-- 64290\n",
      "Archive:  smsspamcollection.zip\n",
      "  inflating: SMSSpamCollection       \n",
      "  inflating: readme                  \n",
      "ham\tGo until jurong point, crazy.. Available only in bugis n great world la e buffet... Cine there got amore wat...\n",
      "ham\tOk lar... Joking wif u oni...\n",
      "spam\tFree entry in 2 a wkly comp to win FA Cup final tkts 21st May 2005. Text FA to 87121 to receive entry question(std txt rate)T&C's apply 08452810075over18's\n",
      "ham\tU dun say so early hor... U c already then say...\n",
      "ham\tNah I don't think he goes to usf, he lives around here though\n",
      "spam\tFreeMsg Hey there darling it's been 3 week's now and no word back! I'd like some fun you up for it still? Tb ok! XxX std chgs to send, £1.50 to rcv\n",
      "ham\tEven my brother is not like to speak with me. They treat me like aids patent.\n",
      "ham\tAs per your request 'Melle Melle (Oru Minnaminunginte Nurungu Vettam)' has been set as your callertune for all Callers. Press *9 to copy your friends Callertune\n",
      "spam\tWINNER!! As a valued network customer you have been selected to receivea £900 prize reward! To claim call 09061701461. Claim code KL341. Valid 12 hours only.\n",
      "spam\tHad your mobile 11 months or more? U R entitled to Update to the latest colour mobiles with camera for Free! Call The Mobile Update Co FREE on 08002986030\n"
     ]
    }
   ],
   "source": [
    "!curl https://archive.ics.uci.edu/ml/machine-learning-databases/00228/smsspamcollection.zip -o smsspamcollection.zip\n",
    "!unzip -o smsspamcollection.zip\n",
    "!head -10 SMSSpamCollection"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We now load the dataset into a Pandas dataframe and execute some data preparation.\n",
    "More specifically we have to:\n",
    "<ul>\n",
    "    <li>replace the target column values (ham/spam) with numeric values (0/1)</li>\n",
    "    <li>tokenize the sms messages and encode based on word counts</li>\n",
    "    <li>split into train and test sets</li>\n",
    "    <li>upload to a S3 bucket for training</li>\n",
    "</ul>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>spam</th>\n",
       "      <th>0</th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "      <th>3</th>\n",
       "      <th>4</th>\n",
       "      <th>5</th>\n",
       "      <th>6</th>\n",
       "      <th>7</th>\n",
       "      <th>8</th>\n",
       "      <th>...</th>\n",
       "      <th>9003</th>\n",
       "      <th>9004</th>\n",
       "      <th>9005</th>\n",
       "      <th>9006</th>\n",
       "      <th>9007</th>\n",
       "      <th>9008</th>\n",
       "      <th>9009</th>\n",
       "      <th>9010</th>\n",
       "      <th>9011</th>\n",
       "      <th>9012</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>1</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>1</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>1</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>1</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>1</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>1</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>1</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27</th>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28</th>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29</th>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5542</th>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5543</th>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5544</th>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5545</th>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5546</th>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5547</th>\n",
       "      <td>1</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5548</th>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5549</th>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5550</th>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5551</th>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5552</th>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5553</th>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5554</th>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5555</th>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5556</th>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5557</th>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5558</th>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5559</th>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5560</th>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5561</th>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5562</th>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5563</th>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5564</th>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5565</th>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5566</th>\n",
       "      <td>1</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5567</th>\n",
       "      <td>1</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5568</th>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5569</th>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5570</th>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5571</th>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5572 rows × 9014 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "      spam    0    1    2    3    4    5    6    7    8  ...   9003  9004  \\\n",
       "0        0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  ...    0.0   0.0   \n",
       "1        0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  ...    0.0   0.0   \n",
       "2        1  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  ...    0.0   0.0   \n",
       "3        0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  ...    0.0   0.0   \n",
       "4        0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  ...    1.0   0.0   \n",
       "5        1  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  ...    0.0   0.0   \n",
       "6        0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  ...    0.0   0.0   \n",
       "7        0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  ...    0.0   0.0   \n",
       "8        1  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  ...    0.0   0.0   \n",
       "9        1  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  ...    0.0   0.0   \n",
       "10       0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  ...    0.0   0.0   \n",
       "11       1  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  ...    0.0   0.0   \n",
       "12       1  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  ...    0.0   0.0   \n",
       "13       0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  ...    0.0   0.0   \n",
       "14       0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  ...    0.0   0.0   \n",
       "15       1  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  ...    1.0   0.0   \n",
       "16       0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  ...    1.0   0.0   \n",
       "17       0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  ...    0.0   0.0   \n",
       "18       0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  ...    0.0   0.0   \n",
       "19       1  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  ...    0.0   0.0   \n",
       "20       0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  ...    0.0   0.0   \n",
       "21       0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  ...    0.0   0.0   \n",
       "22       0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  ...    0.0   0.0   \n",
       "23       0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  ...    0.0   0.0   \n",
       "24       0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  ...    0.0   0.0   \n",
       "25       0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  ...    0.0   0.0   \n",
       "26       0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  ...    0.0   0.0   \n",
       "27       0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  ...    0.0   0.0   \n",
       "28       0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  ...    0.0   0.0   \n",
       "29       0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  ...    0.0   0.0   \n",
       "...    ...  ...  ...  ...  ...  ...  ...  ...  ...  ...  ...    ...   ...   \n",
       "5542     0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  ...    0.0   0.0   \n",
       "5543     0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  ...    0.0   0.0   \n",
       "5544     0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  ...    0.0   0.0   \n",
       "5545     0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  ...    0.0   0.0   \n",
       "5546     0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  ...    0.0   0.0   \n",
       "5547     1  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  ...    0.0   0.0   \n",
       "5548     0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  ...    0.0   0.0   \n",
       "5549     0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  ...    0.0   0.0   \n",
       "5550     0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  ...    1.0   0.0   \n",
       "5551     0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  ...    0.0   0.0   \n",
       "5552     0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  ...    0.0   0.0   \n",
       "5553     0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  ...    0.0   0.0   \n",
       "5554     0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  ...    0.0   0.0   \n",
       "5555     0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  ...    0.0   0.0   \n",
       "5556     0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  ...    0.0   0.0   \n",
       "5557     0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  ...    0.0   0.0   \n",
       "5558     0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  ...    0.0   0.0   \n",
       "5559     0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  ...    1.0   0.0   \n",
       "5560     0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  ...    0.0   0.0   \n",
       "5561     0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  ...    0.0   0.0   \n",
       "5562     0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  ...    0.0   0.0   \n",
       "5563     0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  ...    0.0   0.0   \n",
       "5564     0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  ...    0.0   0.0   \n",
       "5565     0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  ...    0.0   0.0   \n",
       "5566     1  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  ...    0.0   0.0   \n",
       "5567     1  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  ...    0.0   0.0   \n",
       "5568     0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  ...    0.0   0.0   \n",
       "5569     0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  ...    0.0   0.0   \n",
       "5570     0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  ...    0.0   0.0   \n",
       "5571     0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  ...    0.0   0.0   \n",
       "\n",
       "      9005  9006  9007  9008  9009  9010  9011  9012  \n",
       "0      0.0   0.0   0.0   0.0   0.0   0.0   0.0   0.0  \n",
       "1      0.0   0.0   0.0   0.0   0.0   0.0   0.0   0.0  \n",
       "2      0.0   0.0   0.0   0.0   0.0   0.0   0.0   0.0  \n",
       "3      0.0   0.0   0.0   0.0   0.0   0.0   0.0   0.0  \n",
       "4      0.0   0.0   0.0   0.0   0.0   0.0   0.0   0.0  \n",
       "5      0.0   0.0   0.0   0.0   0.0   0.0   0.0   0.0  \n",
       "6      0.0   0.0   0.0   0.0   0.0   0.0   0.0   0.0  \n",
       "7      0.0   0.0   0.0   0.0   0.0   0.0   0.0   0.0  \n",
       "8      0.0   0.0   0.0   0.0   0.0   0.0   0.0   0.0  \n",
       "9      0.0   0.0   0.0   0.0   0.0   0.0   0.0   0.0  \n",
       "10     0.0   0.0   0.0   0.0   0.0   0.0   0.0   0.0  \n",
       "11     0.0   0.0   0.0   0.0   0.0   0.0   0.0   0.0  \n",
       "12     0.0   0.0   0.0   0.0   0.0   0.0   0.0   0.0  \n",
       "13     0.0   0.0   0.0   0.0   0.0   0.0   0.0   0.0  \n",
       "14     0.0   0.0   0.0   0.0   0.0   0.0   0.0   0.0  \n",
       "15     0.0   0.0   0.0   0.0   0.0   0.0   0.0   0.0  \n",
       "16     0.0   0.0   0.0   0.0   0.0   0.0   0.0   0.0  \n",
       "17     0.0   0.0   0.0   0.0   0.0   0.0   0.0   0.0  \n",
       "18     0.0   0.0   0.0   0.0   0.0   0.0   0.0   0.0  \n",
       "19     0.0   0.0   0.0   0.0   0.0   0.0   0.0   0.0  \n",
       "20     0.0   0.0   0.0   0.0   0.0   0.0   0.0   0.0  \n",
       "21     0.0   0.0   0.0   0.0   0.0   0.0   0.0   0.0  \n",
       "22     0.0   0.0   0.0   0.0   0.0   0.0   0.0   0.0  \n",
       "23     0.0   0.0   0.0   0.0   0.0   0.0   0.0   0.0  \n",
       "24     0.0   0.0   0.0   0.0   0.0   0.0   0.0   0.0  \n",
       "25     0.0   0.0   0.0   0.0   0.0   0.0   0.0   0.0  \n",
       "26     0.0   0.0   0.0   0.0   0.0   0.0   0.0   0.0  \n",
       "27     0.0   0.0   0.0   0.0   0.0   0.0   0.0   0.0  \n",
       "28     0.0   0.0   0.0   0.0   0.0   0.0   0.0   0.0  \n",
       "29     0.0   0.0   0.0   0.0   0.0   0.0   0.0   0.0  \n",
       "...    ...   ...   ...   ...   ...   ...   ...   ...  \n",
       "5542   0.0   0.0   0.0   0.0   0.0   0.0   0.0   0.0  \n",
       "5543   0.0   0.0   0.0   0.0   0.0   0.0   0.0   0.0  \n",
       "5544   0.0   0.0   0.0   0.0   0.0   0.0   0.0   0.0  \n",
       "5545   0.0   0.0   0.0   0.0   0.0   0.0   0.0   0.0  \n",
       "5546   0.0   0.0   0.0   0.0   0.0   0.0   0.0   0.0  \n",
       "5547   0.0   0.0   0.0   0.0   0.0   0.0   0.0   0.0  \n",
       "5548   0.0   0.0   0.0   0.0   0.0   0.0   0.0   0.0  \n",
       "5549   0.0   0.0   0.0   0.0   0.0   0.0   0.0   0.0  \n",
       "5550   0.0   0.0   0.0   0.0   0.0   0.0   0.0   0.0  \n",
       "5551   0.0   0.0   0.0   0.0   0.0   0.0   0.0   0.0  \n",
       "5552   0.0   0.0   0.0   0.0   0.0   0.0   0.0   0.0  \n",
       "5553   0.0   0.0   0.0   0.0   0.0   0.0   0.0   0.0  \n",
       "5554   0.0   0.0   0.0   0.0   0.0   0.0   0.0   0.0  \n",
       "5555   0.0   0.0   0.0   0.0   0.0   0.0   0.0   0.0  \n",
       "5556   0.0   0.0   0.0   0.0   0.0   0.0   0.0   0.0  \n",
       "5557   0.0   0.0   0.0   0.0   0.0   0.0   0.0   0.0  \n",
       "5558   0.0   0.0   0.0   0.0   0.0   0.0   0.0   0.0  \n",
       "5559   0.0   0.0   0.0   0.0   0.0   0.0   0.0   0.0  \n",
       "5560   0.0   0.0   0.0   0.0   0.0   0.0   0.0   0.0  \n",
       "5561   0.0   0.0   0.0   0.0   0.0   0.0   0.0   0.0  \n",
       "5562   0.0   0.0   0.0   0.0   0.0   0.0   0.0   0.0  \n",
       "5563   0.0   0.0   0.0   0.0   0.0   0.0   0.0   0.0  \n",
       "5564   0.0   0.0   0.0   0.0   0.0   0.0   0.0   0.0  \n",
       "5565   0.0   0.0   0.0   0.0   0.0   0.0   0.0   0.0  \n",
       "5566   0.0   0.0   0.0   0.0   0.0   0.0   0.0   0.0  \n",
       "5567   0.0   0.0   0.0   0.0   0.0   0.0   0.0   0.0  \n",
       "5568   0.0   0.0   0.0   0.0   0.0   0.0   0.0   0.0  \n",
       "5569   0.0   0.0   0.0   0.0   0.0   0.0   0.0   0.0  \n",
       "5570   0.0   0.0   0.0   0.0   0.0   0.0   0.0   0.0  \n",
       "5571   0.0   0.0   0.0   0.0   0.0   0.0   0.0   0.0  \n",
       "\n",
       "[5572 rows x 9014 columns]"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas\n",
    "import pickle\n",
    "import numpy\n",
    "from keras.preprocessing.text import Tokenizer\n",
    "from keras.preprocessing.text import one_hot\n",
    "from keras.preprocessing.text import text_to_word_sequence\n",
    "\n",
    "df = pandas.read_csv('SMSSpamCollection', sep='\\t', header=None)\n",
    "df[df.columns[0]] = df[df.columns[0]].map({'ham': 0, 'spam': 1})\n",
    "\n",
    "targets = df[df.columns[0]].values\n",
    "messages = df[df.columns[1]].values\n",
    "\n",
    "def vectorize_sequences(sequences, dimension):\n",
    "    results = numpy.zeros((len(sequences), dimension))\n",
    "    for i, sequence in enumerate(sequences):\n",
    "       results[i, sequence] = 1. \n",
    "    return results\n",
    "\n",
    "def one_hot_encode(messages, dimension):\n",
    "    data = []\n",
    "    for msg in messages:\n",
    "        temp = one_hot(msg, vocabulary_lenght)\n",
    "        data.append(temp)\n",
    "    return data\n",
    "\n",
    "# one hot encoding for each SMS message\n",
    "vocabulary_lenght = 9013\n",
    "one_hot_data = one_hot_encode(messages, vocabulary_lenght)\n",
    "encoded_messages = vectorize_sequences(one_hot_data, vocabulary_lenght)\n",
    "\n",
    "# This code is an alternative way to encode text messages using Keras Tokenizer.\n",
    "# The difference is that we will need to save the Tokenizer in order to re-use vocabulary when doing inferences.\n",
    "# t = Tokenizer()\n",
    "# t.fit_on_texts(messages)\n",
    "# dump the tokenizer for later usage when doing inference\n",
    "# pickle.dump(t, open(\"tokenizer.p\", \"wb\"))\n",
    "# messages = df[df.columns[1]]\n",
    "# encoded_messages = t.texts_to_matrix(messages, mode='count')\n",
    "\n",
    "df2 = pandas.DataFrame(encoded_messages)\n",
    "df2.insert(0, 'spam', targets)\n",
    "\n",
    "# split into training and test sets\n",
    "train_set = df2[:4000] \n",
    "test_set = df2[4000:]\n",
    "\n",
    "train_set.to_csv('sms_train_set.gz', header=False, index=False, compression='gzip')\n",
    "test_set.to_csv('sms_test_set.gz', header=False, index=False, compression='gzip')\n",
    "\n",
    "df2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We have to upload the two files back to Amazon S3 in order to be accessed by the Amazon SageMaker training cluster."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "import boto3\n",
    "\n",
    "bucket_name = 'immersionday-sagemaker-test'\n",
    "\n",
    "s3 = boto3.resource('s3')\n",
    "target_bucket = s3.Bucket(bucket_name)\n",
    "\n",
    "with open('sms_train_set.gz', 'rb') as data:\n",
    "    target_bucket.upload_fileobj(data, 'messageantispam/sms_train_set.gz')\n",
    "    \n",
    "with open('sms_test_set.gz', 'rb') as data:\n",
    "    target_bucket.upload_fileobj(data, 'messageantispam/sms_test_set.gz')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h2>Training the model with TensorFlow</h2>\n",
    "\n",
    "We are now ready to run the training using the Amazon SageMaker TensorFlow built-in container. First let's have a look at the script defining our neural network."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 120,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "import numpy as np\r\n",
      "import os\r\n",
      "import tensorflow as tf\r\n",
      "import pandas\r\n",
      "import json\r\n",
      "from tensorflow.python.estimator.export.export import build_raw_serving_input_receiver_fn\r\n",
      "from tensorflow.python.estimator.export.export_output import PredictOutput\r\n",
      "\r\n",
      "INPUT_TENSOR_NAME = 'inputs'\r\n",
      "SIGNATURE_NAME = \"serving_default\"\r\n",
      "LEARNING_RATE = 0.01\r\n",
      "\r\n",
      "def model_fn(features, labels, mode, params):\r\n",
      "\r\n",
      "    first_hidden_layer = tf.keras.layers.Dense(16, activation='relu', name='first-layer')(features[INPUT_TENSOR_NAME])\r\n",
      "    second_hidden_layer = tf.keras.layers.Dense(16, activation='relu')(first_hidden_layer)\r\n",
      "    output_layer = tf.keras.layers.Dense(1, activation='sigmoid')(second_hidden_layer)\r\n",
      "    predictions = output_layer\r\n",
      "\r\n",
      "    # Provide an estimator spec for `ModeKeys.PREDICT`.\r\n",
      "    if mode == tf.estimator.ModeKeys.PREDICT:\r\n",
      "        return tf.estimator.EstimatorSpec(\r\n",
      "            mode=mode,\r\n",
      "            predictions={\"spam\": predictions},\r\n",
      "            export_outputs={SIGNATURE_NAME: PredictOutput({\"spam\": predictions})})\r\n",
      "\r\n",
      "    # 2. Define the loss function for training/evaluation using Tensorflow.\r\n",
      "    loss = tf.losses.log_loss(labels, predictions)\r\n",
      "\r\n",
      "    # 3. Define the training operation/optimizer using Tensorflow operation/optimizer.\r\n",
      "    train_op = tf.contrib.layers.optimize_loss(\r\n",
      "        loss=loss,\r\n",
      "        global_step=tf.contrib.framework.get_global_step(),\r\n",
      "        learning_rate=LEARNING_RATE,\r\n",
      "        optimizer=\"SGD\")\r\n",
      "\r\n",
      "    # 4. Generate predictions as Tensorflow tensors.\r\n",
      "    predictions_dict = {\"spam\": predictions}\r\n",
      "\r\n",
      "    eval_metric_ops = {\r\n",
      "        \"accuracy\": tf.metrics.accuracy(\r\n",
      "            labels, predictions)\r\n",
      "    }\r\n",
      "\r\n",
      "    # Provide an estimator spec for `ModeKeys.EVAL` and `ModeKeys.TRAIN` modes.\r\n",
      "    return tf.estimator.EstimatorSpec(\r\n",
      "        mode=mode,\r\n",
      "        loss=loss,\r\n",
      "        train_op=train_op,\r\n",
      "        eval_metric_ops=eval_metric_ops)\r\n",
      "\r\n",
      "#def keras_model_fn(hyperparameters):\r\n",
      "#    model = tf.keras.models.Sequential()\r\n",
      "#    model.add(tf.keras.layers.Dense(16, activation='relu',input_shape=(9013,), name='inputs'))\r\n",
      "#    model.add(tf.keras.layers.Dense(16, activation='relu'))\r\n",
      "#    model.add(tf.keras.layers.Dense(1, activation='sigmoid'))\r\n",
      "\r\n",
      "#    model.compile(loss='binary_crossentropy',\r\n",
      "#                  optimizer='rmsprop',\r\n",
      "#                  metrics=['accuracy'])\r\n",
      "#    return model\r\n",
      "\r\n",
      "def train_input_fn(training_dir, params):\r\n",
      "    return _input_fn(training_dir, 'sms_train_set.gz')\r\n",
      "\r\n",
      "\r\n",
      "def eval_input_fn(training_dir, params):\r\n",
      "    return _input_fn(training_dir, 'sms_test_set.gz')\r\n",
      "\r\n",
      "\r\n",
      "def _input_fn(training_dir, training_filename):\r\n",
      "    df = pandas.read_csv(os.path.join(training_dir, training_filename))\r\n",
      "    \r\n",
      "    X = df[df.columns[1:]].values.astype(dtype=np.float32)\r\n",
      "    # Reshaping labels as expected by TF\r\n",
      "    y = df[df.columns[0]].values.reshape((-1, 1)).astype(dtype=np.float32)\r\n",
      "    \r\n",
      "    return tf.estimator.inputs.numpy_input_fn(\r\n",
      "        x={INPUT_TENSOR_NAME: X},\r\n",
      "        y=y,\r\n",
      "        num_epochs=None,\r\n",
      "        shuffle=True)()\r\n",
      "\r\n",
      "def serving_input_fn(params):\r\n",
      "    tensor = tf.placeholder(tf.float32, shape=[1, 9013])\r\n",
      "    return build_raw_serving_input_receiver_fn({INPUT_TENSOR_NAME: tensor})()\r\n"
     ]
    }
   ],
   "source": [
    "!cat 'SMS_keras_script.py'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We are now ready to run the training using the TensorFlow estimator object of the SageMaker Python SDK."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 122,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:sagemaker:Creating training-job with name: message-anti-spam-tf-2018-06-19-08-30-30-896\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      ".................\n",
      "\u001b[31m2018-06-19 08:33:16,590 INFO - root - running container entrypoint\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:16,590 INFO - root - starting train task\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:16,596 INFO - container_support.training - Training starting\u001b[0m\n",
      "\u001b[31m/usr/local/lib/python2.7/dist-packages/h5py/__init__.py:36: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.\n",
      "  from ._conv import register_converters as _register_converters\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:18,555 INFO - botocore.vendored.requests.packages.urllib3.connectionpool - Starting new HTTP connection (1): 169.254.170.2\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:18,791 INFO - botocore.vendored.requests.packages.urllib3.connectionpool - Starting new HTTPS connection (1): sagemaker-eu-west-1-825935527263.s3.amazonaws.com\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:18,838 INFO - botocore.vendored.requests.packages.urllib3.connectionpool - Starting new HTTPS connection (2): sagemaker-eu-west-1-825935527263.s3.amazonaws.com\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:18,854 INFO - botocore.vendored.requests.packages.urllib3.connectionpool - Starting new HTTPS connection (1): sagemaker-eu-west-1-825935527263.s3.eu-west-1.amazonaws.com\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:18,927 INFO - botocore.vendored.requests.packages.urllib3.connectionpool - Starting new HTTPS connection (2): sagemaker-eu-west-1-825935527263.s3.eu-west-1.amazonaws.com\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:19,016 INFO - botocore.vendored.requests.packages.urllib3.connectionpool - Starting new HTTPS connection (1): s3.eu-west-1.amazonaws.com\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:19,063 INFO - tf_container - ----------------------TF_CONFIG--------------------------\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:19,063 INFO - tf_container - {\"environment\": \"cloud\", \"cluster\": {\"master\": [\"algo-1:2222\"]}, \"task\": {\"index\": 0, \"type\": \"master\"}}\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:19,063 INFO - tf_container - ---------------------------------------------------------\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:19,063 INFO - tf_container - creating RunConfig:\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:19,064 INFO - tf_container - {'save_checkpoints_secs': 300}\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:19,064 INFO - tensorflow - TF_CONFIG environment variable: {u'environment': u'cloud', u'cluster': {u'master': [u'algo-1:2222']}, u'task': {u'index': 0, u'type': u'master'}}\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:19,064 INFO - tf_container - creating an estimator from the user-provided model_fn\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:19,064 INFO - tensorflow - Using config: {'_save_checkpoints_secs': 300, '_session_config': None, '_keep_checkpoint_max': 5, '_tf_random_seed': None, '_task_type': u'master', '_global_id_in_cluster': 0, '_is_chief': True, '_cluster_spec': <tensorflow.python.training.server_lib.ClusterSpec object at 0x7f44ba7a4910>, '_model_dir': u's3://immersionday-sagemaker-test/messageantispam/output/message-anti-spam-tf-2018-06-19-08-30-30-896/checkpoints', '_num_worker_replicas': 1, '_task_id': 0, '_log_step_count_steps': 100, '_master': '', '_save_checkpoints_steps': None, '_keep_checkpoint_every_n_hours': 10000, '_evaluation_master': '', '_service': None, '_save_summary_steps': 100, '_num_ps_replicas': 0}\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:19,065 INFO - tensorflow - Skip starting Tensorflow server as there is only one node in the cluster.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:19.075254: I tensorflow/core/platform/s3/aws_logging.cc:54] Initializing config loader against fileName /root//.aws/config and using profilePrefix = 1\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:19.075990: I tensorflow/core/platform/s3/aws_logging.cc:54] Initializing config loader against fileName /root//.aws/credentials and using profilePrefix = 0\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:19.076009: I tensorflow/core/platform/s3/aws_logging.cc:54] Setting provider to read credentials from /root//.aws/credentials for credentials file and /root//.aws/config for the config file , for use with profile default\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:19.076028: I tensorflow/core/platform/s3/aws_logging.cc:54] Creating HttpClient with max connections2 and scheme http\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:19.076052: I tensorflow/core/platform/s3/aws_logging.cc:54] Initializing CurlHandleContainer with size 2\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:19.076076: I tensorflow/core/platform/s3/aws_logging.cc:54] Creating TaskRole with default ECSCredentialsClient and refresh rate 900000\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:19.076141: I tensorflow/core/platform/s3/aws_logging.cc:54] Unable to open config file /root//.aws/credentials for reading.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:19.076154: I tensorflow/core/platform/s3/aws_logging.cc:54] Failed to reload configuration.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:19.076166: I tensorflow/core/platform/s3/aws_logging.cc:54] Unable to open config file /root//.aws/config for reading.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:19.076177: I tensorflow/core/platform/s3/aws_logging.cc:54] Failed to reload configuration.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:19.076189: I tensorflow/core/platform/s3/aws_logging.cc:54] Credentials have expired or will expire, attempting to repull from ECS IAM Service.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:19.076256: I tensorflow/core/platform/s3/aws_logging.cc:54] Pool grown by 2\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:19.076273: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:19.079429: I tensorflow/core/platform/s3/aws_logging.cc:54] Initializing CurlHandleContainer with size 25\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:19.081335: I tensorflow/core/platform/s3/aws_logging.cc:54] Pool grown by 2\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:19.081358: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:19.138716: E tensorflow/core/platform/s3/aws_logging.cc:60] No response body. Response code: 404\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:19.138769: W tensorflow/core/platform/s3/aws_logging.cc:57] If the signature check failed. This could be because of a time skew. Attempting to adjust the signer.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:19.139598: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:27,360 INFO - tensorflow - Calling model_fn.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:28,343 WARNING - tensorflow - From /opt/ml/code/SMS_keras_script.py:33: get_global_step (from tensorflow.contrib.framework.python.ops.variables) is deprecated and will be removed in a future version.\u001b[0m\n",
      "\u001b[31mInstructions for updating:\u001b[0m\n",
      "\u001b[31mPlease switch to tf.train.get_global_step\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:28,513 INFO - tensorflow - Done calling model_fn.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:28,514 INFO - tensorflow - Create CheckpointSaverHook.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:28.514829: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:28.571236: E tensorflow/core/platform/s3/aws_logging.cc:60] No response body. Response code: 404\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:28.571268: W tensorflow/core/platform/s3/aws_logging.cc:57] If the signature check failed. This could be because of a time skew. Attempting to adjust the signer.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:28.571440: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:28.581193: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:28.587846: E tensorflow/core/platform/s3/aws_logging.cc:60] No response body. Response code: 404\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:28.587878: W tensorflow/core/platform/s3/aws_logging.cc:57] If the signature check failed. This could be because of a time skew. Attempting to adjust the signer.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:28.588023: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:28.595586: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:28.601701: E tensorflow/core/platform/s3/aws_logging.cc:60] No response body. Response code: 404\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:28.601727: W tensorflow/core/platform/s3/aws_logging.cc:57] If the signature check failed. This could be because of a time skew. Attempting to adjust the signer.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:28.601878: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:28.610987: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:28.617515: E tensorflow/core/platform/s3/aws_logging.cc:60] No response body. Response code: 404\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:28.617562: W tensorflow/core/platform/s3/aws_logging.cc:57] If the signature check failed. This could be because of a time skew. Attempting to adjust the signer.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:28.617731: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:28.627507: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:28.633912: E tensorflow/core/platform/s3/aws_logging.cc:60] No response body. Response code: 404\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:28.633959: W tensorflow/core/platform/s3/aws_logging.cc:57] If the signature check failed. This could be because of a time skew. Attempting to adjust the signer.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:28.634116: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:28.717027: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:28.725478: I tensorflow/core/platform/s3/aws_logging.cc:54] Deleting file: /tmp/s3_filesystem_XXXXXX20180619T0833281529397208716\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:28.725775: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:28.734027: I tensorflow/core/platform/s3/aws_logging.cc:54] Deleting file: /tmp/s3_filesystem_XXXXXX20180619T0833281529397208725\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:28.734243: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:28.745273: I tensorflow/core/platform/s3/aws_logging.cc:54] Deleting file: /tmp/s3_filesystem_XXXXXX20180619T0833281529397208734\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:28.777303: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:28.786483: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:28.836641: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:28.845672: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:28.854797: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:28.872698: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:28.904671: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:28.912104: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:28,988 INFO - tensorflow - Graph was finalized.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:28.992087: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:28.997933: E tensorflow/core/platform/s3/aws_logging.cc:60] No response body. Response code: 404\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:28.997967: W tensorflow/core/platform/s3/aws_logging.cc:57] If the signature check failed. This could be because of a time skew. Attempting to adjust the signer.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:28.998102: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:29,087 INFO - tensorflow - Running local_init_op.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:29,092 INFO - tensorflow - Done running local_init_op.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:29.120276: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:29.124872: E tensorflow/core/platform/s3/aws_logging.cc:60] No response body. Response code: 404\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:29.124902: W tensorflow/core/platform/s3/aws_logging.cc:57] If the signature check failed. This could be because of a time skew. Attempting to adjust the signer.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:29.125059: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:29.237264: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:29.555392: I tensorflow/core/platform/s3/aws_logging.cc:54] Deleting file: /tmp/s3_filesystem_XXXXXX20180619T0833291529397209236\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:29.556329: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:29.570291: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:29.628423: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:29,907 INFO - tensorflow - Saving checkpoints for 1 into s3://immersionday-sagemaker-test/messageantispam/output/message-anti-spam-tf-2018-06-19-08-30-30-896/checkpoints/model.ckpt.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:29.919382: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:29.928013: I tensorflow/core/platform/s3/aws_logging.cc:54] Deleting file: /tmp/s3_filesystem_XXXXXX20180619T0833291529397209919\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:29.933559: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:29.955347: I tensorflow/core/platform/s3/aws_logging.cc:54] Deleting file: /tmp/s3_filesystem_XXXXXX20180619T0833291529397209928\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:29.956857: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:29.969013: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:30.065074: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:30.076237: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:30.091540: I tensorflow/core/platform/s3/aws_logging.cc:54] Deleting file: /tmp/s3_filesystem_XXXXXX20180619T0833301529397210075\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:30.091733: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:30.102094: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:30.129312: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:30.138979: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:30.147651: I tensorflow/core/platform/s3/aws_logging.cc:54] Deleting file: /tmp/s3_filesystem_XXXXXX20180619T0833301529397210138\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:30.147833: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:30.155414: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:30.166214: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:30.174917: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:30.184347: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:30.193877: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:30.204195: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:30.309278: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:30.317490: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:30.331324: I tensorflow/core/platform/s3/aws_logging.cc:54] Deleting file: /tmp/s3_filesystem_XXXXXX20180619T0833301529397210317\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:30.331507: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:30.342118: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:30.356881: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:30.367602: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:30.407896: I tensorflow/core/platform/s3/aws_logging.cc:54] Deleting file: /tmp/s3_filesystem_XXXXXX20180619T0833301529397210367\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:30.408153: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:30.417767: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:30.546589: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:30.577149: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:30.582373: E tensorflow/core/platform/s3/aws_logging.cc:60] No response body. Response code: 404\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:30.582401: W tensorflow/core/platform/s3/aws_logging.cc:57] If the signature check failed. This could be because of a time skew. Attempting to adjust the signer.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:30.582550: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:30.598907: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:30.635629: I tensorflow/core/platform/s3/aws_logging.cc:54] Deleting file: /tmp/s3_filesystem_XXXXXX20180619T0833301529397210598\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:30.635957: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:30.647430: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:30.711711: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:30.725724: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:30.733749: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:30.743914: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:30.755346: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:30.764099: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:30.776390: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:30.787807: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:30.796152: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:30.803854: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:30.811331: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[31m2018-06-19 08:33:34,034 INFO - tensorflow - Calling model_fn.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:34,191 INFO - tensorflow - Done calling model_fn.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:34,211 INFO - tensorflow - Starting evaluation at 2018-06-19-08:33:34\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:34,270 INFO - tensorflow - Graph was finalized.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:34,270 INFO - tensorflow - Restoring parameters from s3://immersionday-sagemaker-test/messageantispam/output/message-anti-spam-tf-2018-06-19-08-30-30-896/checkpoints/model.ckpt-1\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:34.293037: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:34.307348: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:34.316734: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:34.327142: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:34.338615: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:34.349047: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:34.357729: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:34.370853: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:34.383414: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:34.399910: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:34.417633: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:34,458 INFO - tensorflow - Running local_init_op.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:34,465 INFO - tensorflow - Done running local_init_op.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:34,811 INFO - tensorflow - Evaluation [10/100]\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:35,080 INFO - tensorflow - Evaluation [20/100]\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:35,374 INFO - tensorflow - Evaluation [30/100]\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:35,623 INFO - tensorflow - Evaluation [40/100]\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:35,919 INFO - tensorflow - Evaluation [50/100]\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:36,163 INFO - tensorflow - Evaluation [60/100]\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:36,579 INFO - tensorflow - Evaluation [70/100]\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:36,866 INFO - tensorflow - Evaluation [80/100]\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:37,108 INFO - tensorflow - Evaluation [90/100]\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:37,435 INFO - tensorflow - Evaluation [100/100]\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:37,513 INFO - tensorflow - Finished evaluation at 2018-06-19-08:33:37\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:37,514 INFO - tensorflow - Saving dict for global step 1: accuracy = 0.0, global_step = 1, loss = 0.71055955\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:37.514624: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:37.521730: E tensorflow/core/platform/s3/aws_logging.cc:60] No response body. Response code: 404\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:37.521813: W tensorflow/core/platform/s3/aws_logging.cc:57] If the signature check failed. This could be because of a time skew. Attempting to adjust the signer.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:37.522030: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:37.531713: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:37.537366: E tensorflow/core/platform/s3/aws_logging.cc:60] No response body. Response code: 404\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:37.537531: W tensorflow/core/platform/s3/aws_logging.cc:57] If the signature check failed. This could be because of a time skew. Attempting to adjust the signer.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:37.537850: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:37.546330: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:37.551256: E tensorflow/core/platform/s3/aws_logging.cc:60] No response body. Response code: 404\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:37.551421: W tensorflow/core/platform/s3/aws_logging.cc:57] If the signature check failed. This could be because of a time skew. Attempting to adjust the signer.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:37.551744: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:37.563109: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:37.580052: I tensorflow/core/platform/s3/aws_logging.cc:54] Deleting file: /tmp/s3_filesystem_XXXXXX20180619T0833371529397217562\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:37.723502: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:37.733595: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:37.743381: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:37.752253: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:37.763791: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:38.437427: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:38.467817: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:38.476941: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:38.508884: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:38.554075: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:38.591235: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:38.599679: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:38,729 INFO - tensorflow - Calling model_fn.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:38,761 INFO - tensorflow - Done calling model_fn.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:38,761 INFO - tensorflow - Signatures INCLUDED in export for Classify: None\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:38,761 INFO - tensorflow - Signatures INCLUDED in export for Regress: None\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:38,761 INFO - tensorflow - Signatures INCLUDED in export for Predict: ['serving_default']\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:38.761976: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:38.780239: E tensorflow/core/platform/s3/aws_logging.cc:60] No response body. Response code: 404\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:38.780272: W tensorflow/core/platform/s3/aws_logging.cc:57] If the signature check failed. This could be because of a time skew. Attempting to adjust the signer.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:38.780417: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:38,814 INFO - tensorflow - Restoring parameters from s3://immersionday-sagemaker-test/messageantispam/output/message-anti-spam-tf-2018-06-19-08-30-30-896/checkpoints/model.ckpt-1\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:38.824007: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:38.911237: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:38.920273: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:38.930273: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:38.940158: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:38.951488: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:38.958331: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:38.966835: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:38.975820: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:38.986632: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:38.995435: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:39.016310: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:39.023442: E tensorflow/core/platform/s3/aws_logging.cc:60] No response body. Response code: 404\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:39.023475: W tensorflow/core/platform/s3/aws_logging.cc:57] If the signature check failed. This could be because of a time skew. Attempting to adjust the signer.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:39.023616: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:39.031940: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:39.037719: E tensorflow/core/platform/s3/aws_logging.cc:60] No response body. Response code: 404\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:39.037783: W tensorflow/core/platform/s3/aws_logging.cc:57] If the signature check failed. This could be because of a time skew. Attempting to adjust the signer.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:39.038087: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:39.048462: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:39.054314: E tensorflow/core/platform/s3/aws_logging.cc:60] No response body. Response code: 404\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:39.054375: W tensorflow/core/platform/s3/aws_logging.cc:57] If the signature check failed. This could be because of a time skew. Attempting to adjust the signer.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:39.054643: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:39.064200: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:39.071838: E tensorflow/core/platform/s3/aws_logging.cc:60] No response body. Response code: 404\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:39.071864: W tensorflow/core/platform/s3/aws_logging.cc:57] If the signature check failed. This could be because of a time skew. Attempting to adjust the signer.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:39.072010: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:39.081724: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:39.087972: E tensorflow/core/platform/s3/aws_logging.cc:60] No response body. Response code: 404\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:39.088045: W tensorflow/core/platform/s3/aws_logging.cc:57] If the signature check failed. This could be because of a time skew. Attempting to adjust the signer.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:39.088227: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:39.099544: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:39.107515: I tensorflow/core/platform/s3/aws_logging.cc:54] Deleting file: /tmp/s3_filesystem_XXXXXX20180619T0833391529397219099\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:39.107717: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:39.115931: I tensorflow/core/platform/s3/aws_logging.cc:54] Deleting file: /tmp/s3_filesystem_XXXXXX20180619T0833391529397219107\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:39.116123: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:39.149013: I tensorflow/core/platform/s3/aws_logging.cc:54] Deleting file: /tmp/s3_filesystem_XXXXXX20180619T0833391529397219115\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:39,149 INFO - tensorflow - Assets added to graph.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:39,149 INFO - tensorflow - No assets to write.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:39.149656: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:39.155007: E tensorflow/core/platform/s3/aws_logging.cc:60] No response body. Response code: 404\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:39.155040: W tensorflow/core/platform/s3/aws_logging.cc:57] If the signature check failed. This could be because of a time skew. Attempting to adjust the signer.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:39.155195: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:39.164655: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:39.169924: E tensorflow/core/platform/s3/aws_logging.cc:60] No response body. Response code: 404\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:39.169994: W tensorflow/core/platform/s3/aws_logging.cc:57] If the signature check failed. This could be because of a time skew. Attempting to adjust the signer.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:39.170152: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:39.178095: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:39.183264: E tensorflow/core/platform/s3/aws_logging.cc:60] No response body. Response code: 404\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:39.183291: W tensorflow/core/platform/s3/aws_logging.cc:57] If the signature check failed. This could be because of a time skew. Attempting to adjust the signer.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:39.183441: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:39.193506: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:39.201822: I tensorflow/core/platform/s3/aws_logging.cc:54] Deleting file: /tmp/s3_filesystem_XXXXXX20180619T0833391529397219193\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:39.220739: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:39.229098: I tensorflow/core/platform/s3/aws_logging.cc:54] Deleting file: /tmp/s3_filesystem_XXXXXX20180619T0833391529397219220\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:39.233583: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:39.333757: I tensorflow/core/platform/s3/aws_logging.cc:54] Deleting file: /tmp/s3_filesystem_XXXXXX20180619T0833391529397219229\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:39.334117: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:39.345214: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:39.409633: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:39.418437: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:39.431194: I tensorflow/core/platform/s3/aws_logging.cc:54] Deleting file: /tmp/s3_filesystem_XXXXXX20180619T0833391529397219418\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:39.431399: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:39.442062: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:39.463725: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:39.472719: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:39.480692: I tensorflow/core/platform/s3/aws_logging.cc:54] Deleting file: /tmp/s3_filesystem_XXXXXX20180619T0833391529397219472\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:39.480886: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:39.487290: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:39.497976: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:39.518932: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:39.533820: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:39.553835: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:39.569137: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:39.816289: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:39.823916: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:39.837922: I tensorflow/core/platform/s3/aws_logging.cc:54] Deleting file: /tmp/s3_filesystem_XXXXXX20180619T0833391529397219823\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:39.838125: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:39.846621: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:39.858952: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:39.879241: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:39.884355: E tensorflow/core/platform/s3/aws_logging.cc:60] No response body. Response code: 404\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:39.884386: W tensorflow/core/platform/s3/aws_logging.cc:57] If the signature check failed. This could be because of a time skew. Attempting to adjust the signer.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:39.884529: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:39.897006: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:40.065788: I tensorflow/core/platform/s3/aws_logging.cc:54] Deleting file: /tmp/s3_filesystem_XXXXXX20180619T0833391529397219896\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:40,065 INFO - tensorflow - SavedModel written to: s3://immersionday-sagemaker-test/messageantispam/output/message-anti-spam-tf-2018-06-19-08-30-30-896/checkpoints/export/Servo/temp-1529397218/saved_model.pb\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:40.066368: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:40.074713: E tensorflow/core/platform/s3/aws_logging.cc:60] No response body. Response code: 404\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:40.074746: W tensorflow/core/platform/s3/aws_logging.cc:57] If the signature check failed. This could be because of a time skew. Attempting to adjust the signer.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:40.074902: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:40.086192: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:40.098783: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:40.117140: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:40.126903: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:40.163229: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:40.171714: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:40.186867: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:40.194678: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:40.268486: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:40.276381: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:40.409676: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:40.419610: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:40.429528: E tensorflow/core/platform/s3/aws_logging.cc:60] No response body. Response code: 404\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:40.429569: W tensorflow/core/platform/s3/aws_logging.cc:57] If the signature check failed. This could be because of a time skew. Attempting to adjust the signer.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:40.429738: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:40.443625: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:40.449975: E tensorflow/core/platform/s3/aws_logging.cc:60] No response body. Response code: 404\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:40.450001: W tensorflow/core/platform/s3/aws_logging.cc:57] If the signature check failed. This could be because of a time skew. Attempting to adjust the signer.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:40.450151: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:40.460852: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:40,474 INFO - tensorflow - loss = 0.71554136, step = 1\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[31m2018-06-19 08:33:46,508 INFO - tensorflow - global_step/sec: 16.57\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:46,509 INFO - tensorflow - loss = 0.5651593, step = 101 (6.036 sec)\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:51,680 INFO - tensorflow - global_step/sec: 19.3356\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:51,681 INFO - tensorflow - loss = 0.47762156, step = 201 (5.172 sec)\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:56,385 INFO - tensorflow - global_step/sec: 21.2533\u001b[0m\n",
      "\u001b[31m2018-06-19 08:33:56,386 INFO - tensorflow - loss = 0.43916783, step = 301 (4.705 sec)\u001b[0m\n",
      "\u001b[31m2018-06-19 08:34:02,303 INFO - tensorflow - global_step/sec: 16.8987\u001b[0m\n",
      "\u001b[31m2018-06-19 08:34:02,304 INFO - tensorflow - loss = 0.4276031, step = 401 (5.918 sec)\u001b[0m\n",
      "\u001b[31m2018-06-19 08:34:07,248 INFO - tensorflow - global_step/sec: 20.2209\u001b[0m\n",
      "\u001b[31m2018-06-19 08:34:07,250 INFO - tensorflow - loss = 0.38467485, step = 501 (4.946 sec)\u001b[0m\n",
      "\u001b[31m2018-06-19 08:34:12,629 INFO - tensorflow - global_step/sec: 18.5843\u001b[0m\n",
      "\u001b[31m2018-06-19 08:34:12,630 INFO - tensorflow - loss = 0.39950448, step = 601 (5.381 sec)\u001b[0m\n",
      "\u001b[31m2018-06-19 08:34:17,743 INFO - tensorflow - global_step/sec: 19.5529\u001b[0m\n",
      "\u001b[31m2018-06-19 08:34:17,744 INFO - tensorflow - loss = 0.32356095, step = 701 (5.114 sec)\u001b[0m\n",
      "\u001b[31m2018-06-19 08:34:23,005 INFO - tensorflow - global_step/sec: 19.0077\u001b[0m\n",
      "\u001b[31m2018-06-19 08:34:23,006 INFO - tensorflow - loss = 0.2890182, step = 801 (5.261 sec)\u001b[0m\n",
      "\u001b[31m2018-06-19 08:34:28,401 INFO - tensorflow - global_step/sec: 18.5319\u001b[0m\n",
      "\u001b[31m2018-06-19 08:34:28,401 INFO - tensorflow - loss = 0.23074369, step = 901 (5.396 sec)\u001b[0m\n",
      "\u001b[31m2018-06-19 08:34:33,853 INFO - tensorflow - global_step/sec: 18.3415\u001b[0m\n",
      "\u001b[31m2018-06-19 08:34:33,854 INFO - tensorflow - loss = 0.2510291, step = 1001 (5.452 sec)\u001b[0m\n",
      "\u001b[31m2018-06-19 08:34:39,986 INFO - tensorflow - global_step/sec: 16.3045\u001b[0m\n",
      "\u001b[31m2018-06-19 08:34:39,987 INFO - tensorflow - loss = 0.19686358, step = 1101 (6.133 sec)\u001b[0m\n",
      "\u001b[31m2018-06-19 08:34:45,266 INFO - tensorflow - global_step/sec: 18.9409\u001b[0m\n",
      "\u001b[31m2018-06-19 08:34:45,267 INFO - tensorflow - loss = 0.1732278, step = 1201 (5.280 sec)\u001b[0m\n",
      "\u001b[31m2018-06-19 08:34:49,909 INFO - tensorflow - global_step/sec: 21.5361\u001b[0m\n",
      "\u001b[31m2018-06-19 08:34:49,910 INFO - tensorflow - loss = 0.110844694, step = 1301 (4.643 sec)\u001b[0m\n",
      "\u001b[31m2018-06-19 08:34:55,589 INFO - tensorflow - global_step/sec: 17.6058\u001b[0m\n",
      "\u001b[31m2018-06-19 08:34:55,590 INFO - tensorflow - loss = 0.09636633, step = 1401 (5.680 sec)\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:01,382 INFO - tensorflow - global_step/sec: 17.261\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:01,383 INFO - tensorflow - loss = 0.12609942, step = 1501 (5.793 sec)\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:07,449 INFO - tensorflow - global_step/sec: 16.4836\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:07,450 INFO - tensorflow - loss = 0.096275344, step = 1601 (6.067 sec)\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:13,257 INFO - tensorflow - global_step/sec: 17.2184\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:13,258 INFO - tensorflow - loss = 0.07229985, step = 1701 (5.808 sec)\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:19,105 INFO - tensorflow - global_step/sec: 17.0981\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:19,106 INFO - tensorflow - loss = 0.10629001, step = 1801 (5.849 sec)\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:24,553 INFO - tensorflow - global_step/sec: 18.3568\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:24,554 INFO - tensorflow - loss = 0.061263397, step = 1901 (5.447 sec)\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:29,620 INFO - tensorflow - Saving checkpoints for 2000 into s3://immersionday-sagemaker-test/messageantispam/output/message-anti-spam-tf-2018-06-19-08-30-30-896/checkpoints/model.ckpt.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:29.621520: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:29.689167: I tensorflow/core/platform/s3/aws_logging.cc:54] Deleting file: /tmp/s3_filesystem_XXXXXX20180619T0835291529397329621\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:29.693969: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:29.752129: I tensorflow/core/platform/s3/aws_logging.cc:54] Deleting file: /tmp/s3_filesystem_XXXXXX20180619T0835291529397329689\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:29.752520: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:29.835325: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:29.899175: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:29.908290: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:29.924732: I tensorflow/core/platform/s3/aws_logging.cc:54] Deleting file: /tmp/s3_filesystem_XXXXXX20180619T0835291529397329908\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:29.924915: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:29.936312: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:29.961529: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:29.972632: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:29.982948: I tensorflow/core/platform/s3/aws_logging.cc:54] Deleting file: /tmp/s3_filesystem_XXXXXX20180619T0835291529397329972\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:29.983148: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:29.993165: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:30.004887: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:30.017064: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:30.077230: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:30.087488: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:30.099730: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:30.184056: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:30.192890: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:30.208313: I tensorflow/core/platform/s3/aws_logging.cc:54] Deleting file: /tmp/s3_filesystem_XXXXXX20180619T0835301529397330192\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:30.208500: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:30.218943: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:30.232649: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:30.243404: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:30.260697: I tensorflow/core/platform/s3/aws_logging.cc:54] Deleting file: /tmp/s3_filesystem_XXXXXX20180619T0835301529397330243\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:30.260941: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:30.275441: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:30.401463: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:30.430818: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:30.438878: E tensorflow/core/platform/s3/aws_logging.cc:60] No response body. Response code: 404\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:30.438911: W tensorflow/core/platform/s3/aws_logging.cc:57] If the signature check failed. This could be because of a time skew. Attempting to adjust the signer.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:30.439067: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:30.456306: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:30.477217: I tensorflow/core/platform/s3/aws_logging.cc:54] Deleting file: /tmp/s3_filesystem_XXXXXX20180619T0835301529397330456\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:30.477446: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:30.491031: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:30.521146: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:30,533 INFO - tensorflow - Skip the current checkpoint eval due to throttle secs (600 secs).\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:30.534825: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:30.581286: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:30.634669: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:30.642937: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:30,664 INFO - tensorflow - Loss for final step: 0.083949186.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:30,665 INFO - tensorflow - Training has already ended. But the last eval is skipped due to eval throttle_secs. Now evaluating the final checkpoint.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:30.665326: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:30.674461: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:30.685228: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:30.692889: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:30.702098: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:30.713011: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:30.726724: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:30.733896: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:30.742686: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:30.749518: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[31m2018-06-19 08:35:33,994 INFO - tensorflow - Calling model_fn.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:34,145 INFO - tensorflow - Done calling model_fn.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:34,165 INFO - tensorflow - Starting evaluation at 2018-06-19-08:35:34\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:34,222 INFO - tensorflow - Graph was finalized.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:34,222 INFO - tensorflow - Restoring parameters from s3://immersionday-sagemaker-test/messageantispam/output/message-anti-spam-tf-2018-06-19-08-30-30-896/checkpoints/model.ckpt-2000\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:34.246654: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:34.262708: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:34.270963: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:34.280114: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:34.287472: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:34.300498: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:34.307882: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:34.320729: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:34.330562: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:34.340797: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:34.351431: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:34,389 INFO - tensorflow - Running local_init_op.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:34,396 INFO - tensorflow - Done running local_init_op.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:34,786 INFO - tensorflow - Evaluation [10/100]\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:35,060 INFO - tensorflow - Evaluation [20/100]\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:35,345 INFO - tensorflow - Evaluation [30/100]\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:35,649 INFO - tensorflow - Evaluation [40/100]\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:35,929 INFO - tensorflow - Evaluation [50/100]\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:36,188 INFO - tensorflow - Evaluation [60/100]\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:36,441 INFO - tensorflow - Evaluation [70/100]\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:36,762 INFO - tensorflow - Evaluation [80/100]\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:37,007 INFO - tensorflow - Evaluation [90/100]\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:37,291 INFO - tensorflow - Evaluation [100/100]\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:37,340 INFO - tensorflow - Finished evaluation at 2018-06-19-08:35:37\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:37,340 INFO - tensorflow - Saving dict for global step 2000: accuracy = 0.0, global_step = 2000, loss = 0.08554166\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:37.341426: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:37.411193: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:37.466946: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:37.476599: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:37,491 INFO - tensorflow - Calling model_fn.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:37,525 INFO - tensorflow - Done calling model_fn.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:37,525 INFO - tensorflow - Signatures INCLUDED in export for Classify: None\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:37,525 INFO - tensorflow - Signatures INCLUDED in export for Regress: None\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:37,525 INFO - tensorflow - Signatures INCLUDED in export for Predict: ['serving_default']\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:37.525952: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:37.533943: E tensorflow/core/platform/s3/aws_logging.cc:60] No response body. Response code: 404\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:37.533974: W tensorflow/core/platform/s3/aws_logging.cc:57] If the signature check failed. This could be because of a time skew. Attempting to adjust the signer.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:37.534130: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:37,558 INFO - tensorflow - Restoring parameters from s3://immersionday-sagemaker-test/messageantispam/output/message-anti-spam-tf-2018-06-19-08-30-30-896/checkpoints/model.ckpt-2000\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:37.569111: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:37.584956: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:37.592310: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:37.602236: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:37.610383: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:37.620403: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:37.628427: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:37.641260: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:37.651246: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:37.660632: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:37.670386: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:37.689692: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:37.697349: E tensorflow/core/platform/s3/aws_logging.cc:60] No response body. Response code: 404\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:37.697381: W tensorflow/core/platform/s3/aws_logging.cc:57] If the signature check failed. This could be because of a time skew. Attempting to adjust the signer.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:37.697536: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:37.708311: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:37.713564: E tensorflow/core/platform/s3/aws_logging.cc:60] No response body. Response code: 404\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:37.713590: W tensorflow/core/platform/s3/aws_logging.cc:57] If the signature check failed. This could be because of a time skew. Attempting to adjust the signer.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:37.713740: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:37.723663: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:37.731116: E tensorflow/core/platform/s3/aws_logging.cc:60] No response body. Response code: 404\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:37.731165: W tensorflow/core/platform/s3/aws_logging.cc:57] If the signature check failed. This could be because of a time skew. Attempting to adjust the signer.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:37.731291: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:37.745564: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:37.755433: I tensorflow/core/platform/s3/aws_logging.cc:54] Deleting file: /tmp/s3_filesystem_XXXXXX20180619T0835371529397337745\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:37,755 INFO - tensorflow - Assets added to graph.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:37,755 INFO - tensorflow - No assets to write.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:37.756004: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:37.764461: E tensorflow/core/platform/s3/aws_logging.cc:60] No response body. Response code: 404\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:37.764488: W tensorflow/core/platform/s3/aws_logging.cc:57] If the signature check failed. This could be because of a time skew. Attempting to adjust the signer.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:37.764645: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:37.776877: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:37.782815: E tensorflow/core/platform/s3/aws_logging.cc:60] No response body. Response code: 404\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:37.782846: W tensorflow/core/platform/s3/aws_logging.cc:57] If the signature check failed. This could be because of a time skew. Attempting to adjust the signer.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:37.783002: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:37.793314: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:37.798109: E tensorflow/core/platform/s3/aws_logging.cc:60] No response body. Response code: 404\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:37.798135: W tensorflow/core/platform/s3/aws_logging.cc:57] If the signature check failed. This could be because of a time skew. Attempting to adjust the signer.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:37.798281: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:37.843441: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:37.851441: I tensorflow/core/platform/s3/aws_logging.cc:54] Deleting file: /tmp/s3_filesystem_XXXXXX20180619T0835371529397337843\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:37.870966: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:37.912502: I tensorflow/core/platform/s3/aws_logging.cc:54] Deleting file: /tmp/s3_filesystem_XXXXXX20180619T0835371529397337870\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:37.914367: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:37.993274: I tensorflow/core/platform/s3/aws_logging.cc:54] Deleting file: /tmp/s3_filesystem_XXXXXX20180619T0835371529397337912\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:37.993611: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:38.009166: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:38.078925: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:38.089795: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:38.105824: I tensorflow/core/platform/s3/aws_logging.cc:54] Deleting file: /tmp/s3_filesystem_XXXXXX20180619T0835381529397338089\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:38.106012: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:38.120001: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:38.159780: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:38.169750: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:38.177898: I tensorflow/core/platform/s3/aws_logging.cc:54] Deleting file: /tmp/s3_filesystem_XXXXXX20180619T0835381529397338169\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:38.178095: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:38.186271: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:38.197954: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:38.252341: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:38.261976: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:38.271829: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:38.357844: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:38.420256: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:38.429826: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:38.444676: I tensorflow/core/platform/s3/aws_logging.cc:54] Deleting file: /tmp/s3_filesystem_XXXXXX20180619T0835381529397338429\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:38.444910: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:38.454675: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:38.465660: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:38.485402: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:38.490778: E tensorflow/core/platform/s3/aws_logging.cc:60] No response body. Response code: 404\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:38.490810: W tensorflow/core/platform/s3/aws_logging.cc:57] If the signature check failed. This could be because of a time skew. Attempting to adjust the signer.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:38.490964: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[31m2018-06-19 08:35:38.504406: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:38.540655: I tensorflow/core/platform/s3/aws_logging.cc:54] Deleting file: /tmp/s3_filesystem_XXXXXX20180619T0835381529397338504\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:38,540 INFO - tensorflow - SavedModel written to: s3://immersionday-sagemaker-test/messageantispam/output/message-anti-spam-tf-2018-06-19-08-30-30-896/checkpoints/export/Servo/temp-1529397337/saved_model.pb\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:38.541236: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:38.545904: E tensorflow/core/platform/s3/aws_logging.cc:60] No response body. Response code: 404\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:38.545937: W tensorflow/core/platform/s3/aws_logging.cc:57] If the signature check failed. This could be because of a time skew. Attempting to adjust the signer.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:38.546111: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:38.556146: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:38.568172: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:38.586139: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:38.594988: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:38.620976: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:38.629232: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:38.646313: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:38.654527: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:38.720148: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:38.728701: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:38.752174: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:38.762133: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:38.767884: E tensorflow/core/platform/s3/aws_logging.cc:60] No response body. Response code: 404\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:38.767945: W tensorflow/core/platform/s3/aws_logging.cc:57] If the signature check failed. This could be because of a time skew. Attempting to adjust the signer.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:38.768199: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:38.783853: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:38.789251: E tensorflow/core/platform/s3/aws_logging.cc:60] No response body. Response code: 404\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:38.789291: W tensorflow/core/platform/s3/aws_logging.cc:57] If the signature check failed. This could be because of a time skew. Attempting to adjust the signer.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:38.789441: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:38.799901: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:38,813 INFO - botocore.vendored.requests.packages.urllib3.connectionpool - Starting new HTTPS connection (1): immersionday-sagemaker-test.s3.amazonaws.com\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:38,941 INFO - botocore.vendored.requests.packages.urllib3.connectionpool - Starting new HTTPS connection (1): immersionday-sagemaker-test.s3.eu-west-1.amazonaws.com\u001b[0m\n",
      "\u001b[31m2018-06-19 08:35:39,087 INFO - tf_container - Downloaded saved model at /opt/ml/model/export/Servo/1529397218/saved_model.pb\u001b[0m\n",
      "===== Job Complete =====\n",
      "Billable seconds: 226\n"
     ]
    }
   ],
   "source": [
    "from sagemaker.tensorflow import TensorFlow\n",
    "\n",
    "output_bucket_path = 's3://immersionday-sagemaker-test/messageantispam/output/'\n",
    "\n",
    "estimator = TensorFlow(entry_point='SMS_keras_script.py',\n",
    "                               role=role,\n",
    "                               training_steps=5000,                                  \n",
    "                               evaluation_steps=100,\n",
    "                               hyperparameters={'learning_rate': 0.01},\n",
    "                               train_instance_count=1,\n",
    "                               train_instance_type='ml.c4.xlarge',\n",
    "                               output_path=output_bucket_path,\n",
    "                               base_job_name='message-anti-spam-tf')\n",
    "\n",
    "estimator.fit('s3://'+ bucket_name +'/messageantispam/')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h2>Deploying the model</h2>\n",
    "\n",
    "Let's deploy the trained model to an endpoint and execute inferences."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 123,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:sagemaker:Creating model with name: message-anti-spam-tf-2018-06-19-08-30-30-896\n",
      "INFO:sagemaker:Creating endpoint with name message-anti-spam-tf-endpoint\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "--------------------------------------------------------------!"
     ]
    }
   ],
   "source": [
    "tf_predictor = estimator.deploy(initial_instance_count=1,\n",
    "                                instance_type='ml.m5.xlarge',\n",
    "                                endpoint_name='message-anti-spam-tf-endpoint')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 124,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{u'outputs': {u'spam': {u'dtype': u'DT_FLOAT', u'floatVal': [0.5087814331054688], u'tensorShape': {u'dim': [{u'size': u'1'}, {u'size': u'1'}]}}}}\n"
     ]
    }
   ],
   "source": [
    "import tensorflow as tf\n",
    "import numpy\n",
    "from sagemaker.tensorflow.model import TensorFlowPredictor\n",
    "\n",
    "# Uncomment the following line to connect to an existing endpoint.\n",
    "# tf_predictor = TensorFlowPredictor('message-anti-spam-tf-endpoint')\n",
    "\n",
    "test_messages = ['Free prizes for you! Download ringtones on http://sws44.to Answer YES']\n",
    "one_hot_test_messages = one_hot_encode(test_messages, vocabulary_lenght)\n",
    "encoded_test_messages = vectorize_sequences(one_hot_test_messages, vocabulary_lenght)\n",
    "\n",
    "tensor_proto = tf.make_tensor_proto(values=encoded_test_messages, shape=[1, vocabulary_lenght], dtype=tf.float32)\n",
    "result = tf_predictor.predict(tensor_proto)\n",
    "\n",
    "print(result)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's do an example of loading a model from the saved model artifacts."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 101,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.6947067\n"
     ]
    }
   ],
   "source": [
    "import tensorflow as tf\n",
    "\n",
    "path_to_model = './Model/'\n",
    "\n",
    "test_messages = ['Free prizes for you! Download ringtones on http://sws44.to Answer YES']\n",
    "one_hot_test_messages = one_hot_encode(test_messages, vocabulary_lenght)\n",
    "encoded_test_messages = vectorize_sequences(one_hot_test_messages, vocabulary_lenght)\n",
    "\n",
    "with tf.Session(graph=tf.Graph()) as sess:\n",
    "   tf.saved_model.loader.load(\n",
    "       sess,\n",
    "       [tf.saved_model.tag_constants.SERVING],\n",
    "       path_to_model)\n",
    "\n",
    "   sigmoid_tensor = sess.graph.get_tensor_by_name('dense_1/Sigmoid:0')\n",
    "   predictions = sess.run(sigmoid_tensor, {'Placeholder_1:0': encoded_test_messages})\n",
    "\n",
    "   print(predictions[0][0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h2>Training the model with MXNet</h2>\n",
    "\n",
    "We are now ready to run the training using the Amazon SageMaker MXNet built-in container. First let's have a look at the script defining our neural network."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "from __future__ import print_function\r\n",
      "\r\n",
      "import logging\r\n",
      "import mxnet as mx\r\n",
      "from mxnet import gluon, autograd\r\n",
      "from mxnet.gluon import nn\r\n",
      "import numpy as np\r\n",
      "import json\r\n",
      "import time\r\n",
      "\r\n",
      "import pip\r\n",
      "\r\n",
      "try:\r\n",
      "    from pip import main as pipmain\r\n",
      "except:\r\n",
      "    from pip._internal import main as pipmain\r\n",
      "\r\n",
      "pipmain(['install', 'pandas'])\r\n",
      "import pandas\r\n",
      "\r\n",
      "logging.basicConfig(level=logging.DEBUG)\r\n",
      "\r\n",
      "# ------------------------------------------------------------ #\r\n",
      "# Training methods                                             #\r\n",
      "# ------------------------------------------------------------ #\r\n",
      "\r\n",
      "\r\n",
      "def train(hyperparameters, input_data_config, channel_input_dirs, output_data_dir,\r\n",
      "          num_gpus, num_cpus, hosts, current_host, **kwargs):\r\n",
      "    # SageMaker passes num_cpus, num_gpus and other args we can use to tailor training to\r\n",
      "    # the current container environment, but here we just use simple cpu context.\r\n",
      "    ctx = mx.cpu()\r\n",
      "\r\n",
      "    # retrieve the hyperparameters we set in notebook (with some defaults)\r\n",
      "    batch_size = hyperparameters.get('batch_size', 200)\r\n",
      "    epochs = hyperparameters.get('epochs', 10)\r\n",
      "    learning_rate = hyperparameters.get('learning_rate', 0.01)\r\n",
      "    momentum = hyperparameters.get('momentum', 0.9)\r\n",
      "    log_interval = hyperparameters.get('log_interval', 200)\r\n",
      "\r\n",
      "    train_data_path = channel_input_dirs['train']\r\n",
      "    val_data_path = channel_input_dirs['val']\r\n",
      "    train_data = get_train_data(train_data_path, batch_size)\r\n",
      "    val_data = get_val_data(val_data_path, batch_size)\r\n",
      "\r\n",
      "    # define the network\r\n",
      "    net = define_network()\r\n",
      "\r\n",
      "    # Collect all parameters from net and its children, then initialize them.\r\n",
      "    net.initialize(mx.init.Normal(sigma=1.), ctx=ctx)\r\n",
      "    \r\n",
      "    # Trainer is for updating parameters with gradient.\r\n",
      "    if len(hosts) == 1:\r\n",
      "        kvstore = 'device' if num_gpus > 0 else 'local'\r\n",
      "    else:\r\n",
      "        kvstore = 'dist_device_sync' if num_gpus > 0 else 'dist_sync'\r\n",
      "\r\n",
      "    trainer = gluon.Trainer(net.collect_params(), 'sgd',\r\n",
      "                            {'learning_rate': learning_rate, 'momentum': momentum},\r\n",
      "                            kvstore=kvstore)\r\n",
      "    \r\n",
      "    metric = mx.metric.Accuracy()\r\n",
      "    loss = gluon.loss.SoftmaxCrossEntropyLoss()\r\n",
      "\r\n",
      "    for epoch in range(epochs):\r\n",
      "        \r\n",
      "        # reset data iterator and metric at begining of epoch.\r\n",
      "        metric.reset()\r\n",
      "        btic = time.time()\r\n",
      "        for i, (data, label) in enumerate(train_data):\r\n",
      "            # Copy data to ctx if necessary\r\n",
      "            data = data.as_in_context(ctx)\r\n",
      "            label = label.as_in_context(ctx)\r\n",
      "            \r\n",
      "            # Start recording computation graph with record() section.\r\n",
      "            # Recorded graphs can then be differentiated with backward.\r\n",
      "            with autograd.record():\r\n",
      "                output = net(data)\r\n",
      "                L = loss(output, label)\r\n",
      "                L.backward()\r\n",
      "\r\n",
      "            # take a gradient step with batch_size equal to data.shape[0]\r\n",
      "            trainer.step(data.shape[0])\r\n",
      "\r\n",
      "            # update metric at last.\r\n",
      "            metric.update([label], [output])\r\n",
      "\r\n",
      "            if i % log_interval == 0 and i > 0:\r\n",
      "                name, acc = metric.get()\r\n",
      "                print('[Epoch %d Batch %d] Training: %s=%f, %f samples/s' %\r\n",
      "                      (epoch, i, name, acc, batch_size / (time.time() - btic)))\r\n",
      "\r\n",
      "            btic = time.time()\r\n",
      "\r\n",
      "        name, acc = metric.get()\r\n",
      "        print('[Epoch %d] Training: %s=%f' % (epoch, name, acc))\r\n",
      "\r\n",
      "        name, val_acc = test(ctx, net, val_data)\r\n",
      "        print('[Epoch %d] Validation: %s=%f' % (epoch, name, val_acc))\r\n",
      "\r\n",
      "    return net\r\n",
      "\r\n",
      "\r\n",
      "def save(net, model_dir):\r\n",
      "    # save the model\r\n",
      "    y = net(mx.sym.var('data'))\r\n",
      "    y.save('%s/model.json' % model_dir)\r\n",
      "    net.collect_params().save('%s/model.params' % model_dir)\r\n",
      "\r\n",
      "\r\n",
      "def define_network():\r\n",
      "    net = nn.Sequential()\r\n",
      "    with net.name_scope():\r\n",
      "        net.add(nn.Dense(16, activation='relu'))\r\n",
      "        net.add(nn.Dense(16, activation='relu'))\r\n",
      "        net.add(nn.Dense(2))\r\n",
      "        #net.add(nn.Dense(1))\r\n",
      "    return net\r\n",
      "\r\n",
      "\r\n",
      "def get_train_data(data_path, batch_size):\r\n",
      "    print('Train data path: ' + data_path)\r\n",
      "    df = pandas.read_csv(data_path + '/sms_train_set.gz')\r\n",
      "    features = df[df.columns[1:]].values.astype(dtype=np.float32)\r\n",
      "    labels = df[df.columns[0]].values.reshape((-1, 1)).astype(dtype=np.float32)\r\n",
      "    \r\n",
      "    return gluon.data.DataLoader(gluon.data.ArrayDataset(features, labels), batch_size=batch_size, shuffle=True)\r\n",
      "\r\n",
      "\r\n",
      "def get_val_data(data_path, batch_size):\r\n",
      "    print('Validation data path: ' + data_path)\r\n",
      "    df = pandas.read_csv(data_path + '/sms_test_set.gz')\r\n",
      "    features = df[df.columns[1:]].values.astype(dtype=np.float32)\r\n",
      "    labels = df[df.columns[0]].values.reshape((-1, 1)).astype(dtype=np.float32)\r\n",
      "    \r\n",
      "    return gluon.data.DataLoader(gluon.data.ArrayDataset(features, labels), batch_size=batch_size, shuffle=False)\r\n",
      "\r\n",
      "def test(ctx, net, val_data):\r\n",
      "    metric = mx.metric.Accuracy()\r\n",
      "    for data, label in val_data:\r\n",
      "        data = data.as_in_context(ctx)\r\n",
      "        label = label.as_in_context(ctx)\r\n",
      "        output = net(data)\r\n",
      "        metric.update([label], [output])\r\n",
      "    return metric.get()\r\n",
      "\r\n",
      "\r\n",
      "# ------------------------------------------------------------ #\r\n",
      "# Hosting methods                                              #\r\n",
      "# ------------------------------------------------------------ #\r\n",
      "\r\n",
      "\r\n",
      "def model_fn(model_dir):\r\n",
      "    \"\"\"\r\n",
      "    Load the gluon model. Called once when hosting service starts.\r\n",
      "\r\n",
      "    :param: model_dir The directory where model files are stored.\r\n",
      "    :return: a model (in this case a Gluon network)\r\n",
      "    \"\"\"\r\n",
      "    symbol = mx.sym.load('%s/model.json' % model_dir)\r\n",
      "    outputs = mx.symbol.softmax(data=symbol, name='softmax_label')\r\n",
      "    inputs = mx.sym.var('data')\r\n",
      "    param_dict = gluon.ParameterDict('model_')\r\n",
      "    net = gluon.SymbolBlock(outputs, inputs, param_dict)\r\n",
      "    net.load_params('%s/model.params' % model_dir, ctx=mx.cpu())\r\n",
      "    return net\r\n",
      "\r\n",
      "def transform_fn(net, data, input_content_type, output_content_type):\r\n",
      "    \"\"\"\r\n",
      "    Transform a request using the Gluon model. Called once per request.\r\n",
      "\r\n",
      "    :param net: The Gluon model.\r\n",
      "    :param data: The request payload.\r\n",
      "    :param input_content_type: The request content type.\r\n",
      "    :param output_content_type: The (desired) response content type.\r\n",
      "    :return: response payload and content type.\r\n",
      "    \"\"\"\r\n",
      "    # we can use content types to vary input/output handling, but\r\n",
      "    # here we just assume json for both\r\n",
      "    \r\n",
      "    try:\r\n",
      "        parsed = json.loads(data)\r\n",
      "        nda = mx.nd.array(parsed)\r\n",
      "        output = net(nda)\r\n",
      "        #prediction = mx.nd.argmax(output, axis=1)\r\n",
      "        response_body = json.dumps(output.asnumpy().tolist())\r\n",
      "        return response_body, output_content_type\r\n",
      "    except Exception as ex:\r\n",
      "        response_body = '{error: }' + str(ex)\r\n",
      "        return response_body, output_content_type\r\n",
      "    "
     ]
    }
   ],
   "source": [
    "!cat 'SMS_MXNet_script.py'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We are now ready to run the training using the MXNet estimator object of the SageMaker Python SDK."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:sagemaker:Creating training-job with name: message-anti-spam-mxnet-2018-06-19-20-47-20-349\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "................\n",
      "\u001b[31m2018-06-19 20:49:52,326 INFO - root - running container entrypoint\u001b[0m\n",
      "\u001b[31m2018-06-19 20:49:52,326 INFO - root - starting train task\u001b[0m\n",
      "\u001b[31m2018-06-19 20:49:52,332 INFO - container_support.training - Training starting\u001b[0m\n",
      "\u001b[31m2018-06-19 20:49:54,445 INFO - mxnet_container.train - MXNetTrainingEnvironment: {'enable_cloudwatch_metrics': False, 'available_gpus': 0, 'channels': {u'train': {u'TrainingInputMode': u'File', u'RecordWrapperType': u'None', u'S3DistributionType': u'FullyReplicated'}, u'val': {u'TrainingInputMode': u'File', u'RecordWrapperType': u'None', u'S3DistributionType': u'FullyReplicated'}}, '_ps_verbose': 0, 'resource_config': {u'current_host': u'algo-1', u'network_interface_name': u'ethwe', u'hosts': [u'algo-1']}, 'user_script_name': u'SMS_MXNet_script.py', 'input_config_dir': '/opt/ml/input/config', 'channel_dirs': {u'train': u'/opt/ml/input/data/train', u'val': u'/opt/ml/input/data/val'}, 'code_dir': '/opt/ml/code', 'output_data_dir': '/opt/ml/output/data/', 'output_dir': '/opt/ml/output', 'model_dir': '/opt/ml/model', 'hyperparameters': {u'sagemaker_program': u'SMS_MXNet_script.py', u'learning_rate': 0.01, u'batch_size': 200, u'epochs': 20, u'log_interval': 200, u'sagemaker_region': u'eu-west-1', u'sagemaker_enable_cloudwatch_metrics': False, u'sagemaker_job_name': u'message-anti-spam-mxnet-2018-06-19-20-47-20-349', u'sagemaker_container_log_level': 20, u'sagemaker_submit_directory': u's3://sagemaker-eu-west-1-825935527263/message-anti-spam-mxnet-2018-06-19-20-47-20-349/source/sourcedir.tar.gz'}, 'hosts': [u'algo-1'], 'job_name': 'message-anti-spam-mxnet-2018-06-19-20-47-20-349', '_ps_port': 8000, 'user_script_archive': u's3://sagemaker-eu-west-1-825935527263/message-anti-spam-mxnet-2018-06-19-20-47-20-349/source/sourcedir.tar.gz', '_scheduler_host': u'algo-1', 'sagemaker_region': u'eu-west-1', '_scheduler_ip': '10.32.0.4', 'input_dir': '/opt/ml/input', 'user_requirements_file': None, 'current_host': u'algo-1', 'container_log_level': 20, 'available_cpus': 4, 'base_dir': '/opt/ml'}\u001b[0m\n",
      "\u001b[31mDownloading s3://sagemaker-eu-west-1-825935527263/message-anti-spam-mxnet-2018-06-19-20-47-20-349/source/sourcedir.tar.gz to /tmp/script.tar.gz\u001b[0m\n",
      "\u001b[31m2018-06-19 20:49:54,557 INFO - botocore.vendored.requests.packages.urllib3.connectionpool - Starting new HTTP connection (1): 169.254.170.2\u001b[0m\n",
      "\u001b[31m2018-06-19 20:49:54,676 INFO - botocore.vendored.requests.packages.urllib3.connectionpool - Starting new HTTPS connection (1): sagemaker-eu-west-1-825935527263.s3.amazonaws.com\u001b[0m\n",
      "\u001b[31m2018-06-19 20:49:54,722 INFO - botocore.vendored.requests.packages.urllib3.connectionpool - Starting new HTTPS connection (2): sagemaker-eu-west-1-825935527263.s3.amazonaws.com\u001b[0m\n",
      "\u001b[31m2018-06-19 20:49:54,735 INFO - botocore.vendored.requests.packages.urllib3.connectionpool - Starting new HTTPS connection (1): sagemaker-eu-west-1-825935527263.s3.eu-west-1.amazonaws.com\u001b[0m\n",
      "\u001b[31m2018-06-19 20:49:54,865 INFO - botocore.vendored.requests.packages.urllib3.connectionpool - Starting new HTTPS connection (2): sagemaker-eu-west-1-825935527263.s3.eu-west-1.amazonaws.com\u001b[0m\n",
      "\u001b[31m2018-06-19 20:49:54,996 INFO - mxnet_container.train - Starting distributed training task\u001b[0m\n",
      "\u001b[31mCollecting pandas\n",
      "  Downloading https://files.pythonhosted.org/packages/9d/5c/052a3aef6b5e337eca210ca668b78707c4fca762dcc3a0b7f9f3420f1e97/pandas-0.23.1-cp27-cp27mu-manylinux1_x86_64.whl (11.8MB)\u001b[0m\n",
      "\u001b[31mRequirement already satisfied: numpy>=1.9.0 in /usr/local/lib/python2.7/dist-packages (from pandas) (1.13.3)\u001b[0m\n",
      "\u001b[31mRequirement already satisfied: python-dateutil>=2.5.0 in /usr/local/lib/python2.7/dist-packages (from pandas) (2.7.3)\u001b[0m\n",
      "\u001b[31mCollecting pytz>=2011k (from pandas)\u001b[0m\n",
      "\u001b[31m  Downloading https://files.pythonhosted.org/packages/dc/83/15f7833b70d3e067ca91467ca245bae0f6fe56ddc7451aa0dc5606b120f2/pytz-2018.4-py2.py3-none-any.whl (510kB)\u001b[0m\n",
      "\u001b[31mRequirement already satisfied: six>=1.5 in /usr/local/lib/python2.7/dist-packages (from python-dateutil>=2.5.0->pandas) (1.11.0)\u001b[0m\n",
      "\u001b[31mInstalling collected packages: pytz, pandas\u001b[0m\n",
      "\u001b[31mSuccessfully installed pandas-0.23.1 pytz-2018.4\u001b[0m\n",
      "\u001b[31mTrain data path: /opt/ml/input/data/train\u001b[0m\n",
      "\u001b[31mValidation data path: /opt/ml/input/data/val\u001b[0m\n",
      "\u001b[31m[Epoch 0] Training: accuracy=0.753188\u001b[0m\n",
      "\u001b[31m[Epoch 0] Validation: accuracy=0.797581\u001b[0m\n",
      "\u001b[31m[Epoch 1] Training: accuracy=0.797949\u001b[0m\n",
      "\u001b[31m[Epoch 1] Validation: accuracy=0.751750\u001b[0m\n",
      "\u001b[31m[Epoch 2] Training: accuracy=0.829207\u001b[0m\n",
      "\u001b[31m[Epoch 2] Validation: accuracy=0.861871\u001b[0m\n",
      "\u001b[31m[Epoch 3] Training: accuracy=0.855464\u001b[0m\n",
      "\u001b[31m[Epoch 3] Validation: accuracy=0.869510\u001b[0m\n",
      "\u001b[31m[Epoch 4] Training: accuracy=0.867217\u001b[0m\n",
      "\u001b[31m[Epoch 4] Validation: accuracy=0.873329\u001b[0m\n",
      "\u001b[31m[Epoch 5] Training: accuracy=0.870718\u001b[0m\n",
      "\u001b[31m[Epoch 5] Validation: accuracy=0.856143\u001b[0m\n",
      "\u001b[31m[Epoch 6] Training: accuracy=0.867967\u001b[0m\n",
      "\u001b[31m[Epoch 6] Validation: accuracy=0.866327\u001b[0m\n",
      "\u001b[31m[Epoch 7] Training: accuracy=0.873218\u001b[0m\n",
      "\u001b[31m[Epoch 7] Validation: accuracy=0.863781\u001b[0m\n",
      "\u001b[31m[Epoch 8] Training: accuracy=0.873968\u001b[0m\n",
      "\u001b[31m[Epoch 8] Validation: accuracy=0.869510\u001b[0m\n",
      "\u001b[31m[Epoch 9] Training: accuracy=0.867467\u001b[0m\n",
      "\u001b[31m[Epoch 9] Validation: accuracy=0.879058\u001b[0m\n",
      "\u001b[31m[Epoch 10] Training: accuracy=0.872718\u001b[0m\n",
      "\u001b[31m[Epoch 10] Validation: accuracy=0.872693\u001b[0m\n",
      "\u001b[31m[Epoch 11] Training: accuracy=0.877719\u001b[0m\n",
      "\u001b[31m[Epoch 11] Validation: accuracy=0.880968\u001b[0m\n",
      "\u001b[31m[Epoch 12] Training: accuracy=0.876469\u001b[0m\n",
      "\u001b[31m[Epoch 12] Validation: accuracy=0.879058\u001b[0m\n",
      "\u001b[31m[Epoch 13] Training: accuracy=0.872718\u001b[0m\n",
      "\u001b[31m[Epoch 13] Validation: accuracy=0.879694\u001b[0m\n",
      "\u001b[31m[Epoch 14] Training: accuracy=0.875219\u001b[0m\n",
      "\u001b[31m[Epoch 14] Validation: accuracy=0.868873\u001b[0m\n",
      "\u001b[31m[Epoch 15] Training: accuracy=0.874469\u001b[0m\n",
      "\u001b[31m[Epoch 15] Validation: accuracy=0.867600\u001b[0m\n",
      "\u001b[31m[Epoch 16] Training: accuracy=0.877719\u001b[0m\n",
      "\u001b[31m[Epoch 16] Validation: accuracy=0.880968\u001b[0m\n",
      "\u001b[31m[Epoch 17] Training: accuracy=0.878970\u001b[0m\n",
      "\u001b[31m[Epoch 17] Validation: accuracy=0.876512\u001b[0m\n",
      "\u001b[31m[Epoch 18] Training: accuracy=0.889472\u001b[0m\n",
      "\u001b[31m[Epoch 18] Validation: accuracy=0.884150\u001b[0m\n",
      "\u001b[31m[Epoch 19] Training: accuracy=0.887472\u001b[0m\n",
      "\u001b[31m[Epoch 19] Validation: accuracy=0.882877\u001b[0m\n",
      "===== Job Complete =====\n",
      "Billable seconds: 101\n"
     ]
    }
   ],
   "source": [
    "from sagemaker.mxnet import MXNet\n",
    "\n",
    "m = MXNet('SMS_MXNet_script.py', \n",
    "          role=role,\n",
    "          train_instance_count=1, \n",
    "          train_instance_type='ml.c4.xlarge',\n",
    "          base_job_name='message-anti-spam-mxnet',\n",
    "          hyperparameters={'batch_size': 200, \n",
    "                         'epochs': 20, \n",
    "                         'learning_rate': 0.01, \n",
    "                         'log_interval': 200})\n",
    "\n",
    "inputs = {'train': 's3://'+ bucket_name +'/messageantispam/sms_train_set.gz',\n",
    " 'val': 's3://' + bucket_name + '/messageantispam/sms_test_set.gz'}\n",
    "\n",
    "m.fit(inputs)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h2>Deploying the model</h2>\n",
    "\n",
    "Let's deploy the trained model to an endpoint and execute inferences."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:sagemaker:Creating model with name: message-anti-spam-mxnet-2018-06-19-20-47-20-349\n",
      "INFO:sagemaker:Creating endpoint with name message-anti-spam-mxnet-endpoint-3\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "---------------------------------------------------!"
     ]
    }
   ],
   "source": [
    "mxnet_pred = m.deploy(initial_instance_count=1,\n",
    "                      instance_type='ml.m5.xlarge',\n",
    "                      endpoint_name='message-anti-spam-mxnet-endpoint')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[0.2672363221645355, 0.7327636480331421]]\n",
      "[1]\n"
     ]
    }
   ],
   "source": [
    "from sagemaker.mxnet.model import MXNetPredictor\n",
    "\n",
    "# Uncomment the following line to connect to an existing endpoint.\n",
    "# mxnet_pred = MXNetPredictor('message-anti-spam-mxnet-endpoint')\n",
    "\n",
    "test_messages = [\"Free entry in 2 a wkly comp to win FA Cup final tkts 21st May 2005. Text FA to 87121 to receive entry question(std txt rate)T&C's apply 08452810075over18's\"]\n",
    "one_hot_test_messages = one_hot_encode(test_messages, vocabulary_lenght)\n",
    "encoded_test_messages = vectorize_sequences(one_hot_test_messages, vocabulary_lenght)\n",
    "\n",
    "result = mxnet_pred.predict(encoded_test_messages)\n",
    "value = numpy.argmax(result, axis=1)\n",
    "\n",
    "print(result)\n",
    "print(value)\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "conda_tensorflow_p27",
   "language": "python",
   "name": "conda_tensorflow_p27"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.14"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}