lab2.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h3>Ćwiczenie 2. Brakujące wartości</h3>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h5>Pobranie zbioru danych dotyczących rejsu Titanic ze strony: https://www.openml.org/d/40945</h5>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h5>Dokonanie wstępnej analizy pierwszych 20 wierszy</h5>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>pclass</th>\n",
       "      <th>survived</th>\n",
       "      <th>name</th>\n",
       "      <th>sex</th>\n",
       "      <th>age</th>\n",
       "      <th>sibsp</th>\n",
       "      <th>parch</th>\n",
       "      <th>ticket</th>\n",
       "      <th>fare</th>\n",
       "      <th>cabin</th>\n",
       "      <th>embarked</th>\n",
       "      <th>boat</th>\n",
       "      <th>body</th>\n",
       "      <th>home.dest</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Allen, Miss. Elisabeth Walton</td>\n",
       "      <td>female</td>\n",
       "      <td>29</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>24160</td>\n",
       "      <td>211.3375</td>\n",
       "      <td>B5</td>\n",
       "      <td>S</td>\n",
       "      <td>2</td>\n",
       "      <td>?</td>\n",
       "      <td>St Louis, MO</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Allison, Master. Hudson Trevor</td>\n",
       "      <td>male</td>\n",
       "      <td>0.9167</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>113781</td>\n",
       "      <td>151.55</td>\n",
       "      <td>C22 C26</td>\n",
       "      <td>S</td>\n",
       "      <td>11</td>\n",
       "      <td>?</td>\n",
       "      <td>Montreal, PQ / Chesterville, ON</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>Allison, Miss. Helen Loraine</td>\n",
       "      <td>female</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>113781</td>\n",
       "      <td>151.55</td>\n",
       "      <td>C22 C26</td>\n",
       "      <td>S</td>\n",
       "      <td>?</td>\n",
       "      <td>?</td>\n",
       "      <td>Montreal, PQ / Chesterville, ON</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>Allison, Mr. Hudson Joshua Creighton</td>\n",
       "      <td>male</td>\n",
       "      <td>30</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>113781</td>\n",
       "      <td>151.55</td>\n",
       "      <td>C22 C26</td>\n",
       "      <td>S</td>\n",
       "      <td>?</td>\n",
       "      <td>135</td>\n",
       "      <td>Montreal, PQ / Chesterville, ON</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>Allison, Mrs. Hudson J C (Bessie Waldo Daniels)</td>\n",
       "      <td>female</td>\n",
       "      <td>25</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>113781</td>\n",
       "      <td>151.55</td>\n",
       "      <td>C22 C26</td>\n",
       "      <td>S</td>\n",
       "      <td>?</td>\n",
       "      <td>?</td>\n",
       "      <td>Montreal, PQ / Chesterville, ON</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Anderson, Mr. Harry</td>\n",
       "      <td>male</td>\n",
       "      <td>48</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>19952</td>\n",
       "      <td>26.55</td>\n",
       "      <td>E12</td>\n",
       "      <td>S</td>\n",
       "      <td>3</td>\n",
       "      <td>?</td>\n",
       "      <td>New York, NY</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Andrews, Miss. Kornelia Theodosia</td>\n",
       "      <td>female</td>\n",
       "      <td>63</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>13502</td>\n",
       "      <td>77.9583</td>\n",
       "      <td>D7</td>\n",
       "      <td>S</td>\n",
       "      <td>10</td>\n",
       "      <td>?</td>\n",
       "      <td>Hudson, NY</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>Andrews, Mr. Thomas Jr</td>\n",
       "      <td>male</td>\n",
       "      <td>39</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>112050</td>\n",
       "      <td>0</td>\n",
       "      <td>A36</td>\n",
       "      <td>S</td>\n",
       "      <td>?</td>\n",
       "      <td>?</td>\n",
       "      <td>Belfast, NI</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Appleton, Mrs. Edward Dale (Charlotte Lamson)</td>\n",
       "      <td>female</td>\n",
       "      <td>53</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>11769</td>\n",
       "      <td>51.4792</td>\n",
       "      <td>C101</td>\n",
       "      <td>S</td>\n",
       "      <td>D</td>\n",
       "      <td>?</td>\n",
       "      <td>Bayside, Queens, NY</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>Artagaveytia, Mr. Ramon</td>\n",
       "      <td>male</td>\n",
       "      <td>71</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>PC 17609</td>\n",
       "      <td>49.5042</td>\n",
       "      <td>?</td>\n",
       "      <td>C</td>\n",
       "      <td>?</td>\n",
       "      <td>22</td>\n",
       "      <td>Montevideo, Uruguay</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>Astor, Col. John Jacob</td>\n",
       "      <td>male</td>\n",
       "      <td>47</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>PC 17757</td>\n",
       "      <td>227.525</td>\n",
       "      <td>C62 C64</td>\n",
       "      <td>C</td>\n",
       "      <td>?</td>\n",
       "      <td>124</td>\n",
       "      <td>New York, NY</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Astor, Mrs. John Jacob (Madeleine Talmadge Force)</td>\n",
       "      <td>female</td>\n",
       "      <td>18</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>PC 17757</td>\n",
       "      <td>227.525</td>\n",
       "      <td>C62 C64</td>\n",
       "      <td>C</td>\n",
       "      <td>4</td>\n",
       "      <td>?</td>\n",
       "      <td>New York, NY</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Aubart, Mme. Leontine Pauline</td>\n",
       "      <td>female</td>\n",
       "      <td>24</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>PC 17477</td>\n",
       "      <td>69.3</td>\n",
       "      <td>B35</td>\n",
       "      <td>C</td>\n",
       "      <td>9</td>\n",
       "      <td>?</td>\n",
       "      <td>Paris, France</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Barber, Miss. Ellen 'Nellie'</td>\n",
       "      <td>female</td>\n",
       "      <td>26</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>19877</td>\n",
       "      <td>78.85</td>\n",
       "      <td>?</td>\n",
       "      <td>S</td>\n",
       "      <td>6</td>\n",
       "      <td>?</td>\n",
       "      <td>?</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Barkworth, Mr. Algernon Henry Wilson</td>\n",
       "      <td>male</td>\n",
       "      <td>80</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>27042</td>\n",
       "      <td>30</td>\n",
       "      <td>A23</td>\n",
       "      <td>S</td>\n",
       "      <td>B</td>\n",
       "      <td>?</td>\n",
       "      <td>Hessle, Yorks</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>Baumann, Mr. John D</td>\n",
       "      <td>male</td>\n",
       "      <td>?</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>PC 17318</td>\n",
       "      <td>25.925</td>\n",
       "      <td>?</td>\n",
       "      <td>S</td>\n",
       "      <td>?</td>\n",
       "      <td>?</td>\n",
       "      <td>New York, NY</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>Baxter, Mr. Quigg Edmond</td>\n",
       "      <td>male</td>\n",
       "      <td>24</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>PC 17558</td>\n",
       "      <td>247.5208</td>\n",
       "      <td>B58 B60</td>\n",
       "      <td>C</td>\n",
       "      <td>?</td>\n",
       "      <td>?</td>\n",
       "      <td>Montreal, PQ</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Baxter, Mrs. James (Helene DeLaudeniere Chaput)</td>\n",
       "      <td>female</td>\n",
       "      <td>50</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>PC 17558</td>\n",
       "      <td>247.5208</td>\n",
       "      <td>B58 B60</td>\n",
       "      <td>C</td>\n",
       "      <td>6</td>\n",
       "      <td>?</td>\n",
       "      <td>Montreal, PQ</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Bazzani, Miss. Albina</td>\n",
       "      <td>female</td>\n",
       "      <td>32</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>11813</td>\n",
       "      <td>76.2917</td>\n",
       "      <td>D15</td>\n",
       "      <td>C</td>\n",
       "      <td>8</td>\n",
       "      <td>?</td>\n",
       "      <td>?</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>Beattie, Mr. Thomson</td>\n",
       "      <td>male</td>\n",
       "      <td>36</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>13050</td>\n",
       "      <td>75.2417</td>\n",
       "      <td>C6</td>\n",
       "      <td>C</td>\n",
       "      <td>A</td>\n",
       "      <td>?</td>\n",
       "      <td>Winnipeg, MN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    pclass  survived                                               name  \\\n",
       "0        1         1                      Allen, Miss. Elisabeth Walton   \n",
       "1        1         1                     Allison, Master. Hudson Trevor   \n",
       "2        1         0                       Allison, Miss. Helen Loraine   \n",
       "3        1         0               Allison, Mr. Hudson Joshua Creighton   \n",
       "4        1         0    Allison, Mrs. Hudson J C (Bessie Waldo Daniels)   \n",
       "5        1         1                                Anderson, Mr. Harry   \n",
       "6        1         1                  Andrews, Miss. Kornelia Theodosia   \n",
       "7        1         0                             Andrews, Mr. Thomas Jr   \n",
       "8        1         1      Appleton, Mrs. Edward Dale (Charlotte Lamson)   \n",
       "9        1         0                            Artagaveytia, Mr. Ramon   \n",
       "10       1         0                             Astor, Col. John Jacob   \n",
       "11       1         1  Astor, Mrs. John Jacob (Madeleine Talmadge Force)   \n",
       "12       1         1                      Aubart, Mme. Leontine Pauline   \n",
       "13       1         1                       Barber, Miss. Ellen 'Nellie'   \n",
       "14       1         1               Barkworth, Mr. Algernon Henry Wilson   \n",
       "15       1         0                                Baumann, Mr. John D   \n",
       "16       1         0                           Baxter, Mr. Quigg Edmond   \n",
       "17       1         1    Baxter, Mrs. James (Helene DeLaudeniere Chaput)   \n",
       "18       1         1                              Bazzani, Miss. Albina   \n",
       "19       1         0                               Beattie, Mr. Thomson   \n",
       "\n",
       "       sex     age  sibsp  parch    ticket      fare    cabin embarked boat  \\\n",
       "0   female      29      0      0     24160  211.3375       B5        S    2   \n",
       "1     male  0.9167      1      2    113781    151.55  C22 C26        S   11   \n",
       "2   female       2      1      2    113781    151.55  C22 C26        S    ?   \n",
       "3     male      30      1      2    113781    151.55  C22 C26        S    ?   \n",
       "4   female      25      1      2    113781    151.55  C22 C26        S    ?   \n",
       "5     male      48      0      0     19952     26.55      E12        S    3   \n",
       "6   female      63      1      0     13502   77.9583       D7        S   10   \n",
       "7     male      39      0      0    112050         0      A36        S    ?   \n",
       "8   female      53      2      0     11769   51.4792     C101        S    D   \n",
       "9     male      71      0      0  PC 17609   49.5042        ?        C    ?   \n",
       "10    male      47      1      0  PC 17757   227.525  C62 C64        C    ?   \n",
       "11  female      18      1      0  PC 17757   227.525  C62 C64        C    4   \n",
       "12  female      24      0      0  PC 17477      69.3      B35        C    9   \n",
       "13  female      26      0      0     19877     78.85        ?        S    6   \n",
       "14    male      80      0      0     27042        30      A23        S    B   \n",
       "15    male       ?      0      0  PC 17318    25.925        ?        S    ?   \n",
       "16    male      24      0      1  PC 17558  247.5208  B58 B60        C    ?   \n",
       "17  female      50      0      1  PC 17558  247.5208  B58 B60        C    6   \n",
       "18  female      32      0      0     11813   76.2917      D15        C    8   \n",
       "19    male      36      0      0     13050   75.2417       C6        C    A   \n",
       "\n",
       "   body                        home.dest  \n",
       "0     ?                     St Louis, MO  \n",
       "1     ?  Montreal, PQ / Chesterville, ON  \n",
       "2     ?  Montreal, PQ / Chesterville, ON  \n",
       "3   135  Montreal, PQ / Chesterville, ON  \n",
       "4     ?  Montreal, PQ / Chesterville, ON  \n",
       "5     ?                     New York, NY  \n",
       "6     ?                       Hudson, NY  \n",
       "7     ?                      Belfast, NI  \n",
       "8     ?              Bayside, Queens, NY  \n",
       "9    22              Montevideo, Uruguay  \n",
       "10  124                     New York, NY  \n",
       "11    ?                     New York, NY  \n",
       "12    ?                    Paris, France  \n",
       "13    ?                                ?  \n",
       "14    ?                    Hessle, Yorks  \n",
       "15    ?                     New York, NY  \n",
       "16    ?                     Montreal, PQ  \n",
       "17    ?                     Montreal, PQ  \n",
       "18    ?                                ?  \n",
       "19    ?                     Winnipeg, MN  "
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.read_csv('Titanic.csv')\n",
    "df.head(20)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Dane składają się z 14 kolumn, między innymi takie jak: age, sex, name, survived. Wśród wartości są dane ilościowe oraz jakościowe. Widoczne braki w danych w postaci znaku '?'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h5>Uzycie funkcji: pd.isnull().sum() oraz pd.isnull().mean w celu sprawdzenia brakujących danych.</h5>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Zbiór danych nalezy zmodyfikować poprzez zmianę znaku '?' reprezentującego braki w danych na wartość na.nan, aby zostały one wykryte przez funkcję .isnull()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>pclass</th>\n",
       "      <th>survived</th>\n",
       "      <th>name</th>\n",
       "      <th>sex</th>\n",
       "      <th>age</th>\n",
       "      <th>sibsp</th>\n",
       "      <th>parch</th>\n",
       "      <th>ticket</th>\n",
       "      <th>fare</th>\n",
       "      <th>cabin</th>\n",
       "      <th>embarked</th>\n",
       "      <th>boat</th>\n",
       "      <th>body</th>\n",
       "      <th>home.dest</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Allen, Miss. Elisabeth Walton</td>\n",
       "      <td>female</td>\n",
       "      <td>29</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>24160</td>\n",
       "      <td>211.3375</td>\n",
       "      <td>B5</td>\n",
       "      <td>S</td>\n",
       "      <td>2</td>\n",
       "      <td>NaN</td>\n",
       "      <td>St Louis, MO</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Allison, Master. Hudson Trevor</td>\n",
       "      <td>male</td>\n",
       "      <td>0.9167</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>113781</td>\n",
       "      <td>151.55</td>\n",
       "      <td>C22 C26</td>\n",
       "      <td>S</td>\n",
       "      <td>11</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Montreal, PQ / Chesterville, ON</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>Allison, Miss. Helen Loraine</td>\n",
       "      <td>female</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>113781</td>\n",
       "      <td>151.55</td>\n",
       "      <td>C22 C26</td>\n",
       "      <td>S</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Montreal, PQ / Chesterville, ON</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>Allison, Mr. Hudson Joshua Creighton</td>\n",
       "      <td>male</td>\n",
       "      <td>30</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>113781</td>\n",
       "      <td>151.55</td>\n",
       "      <td>C22 C26</td>\n",
       "      <td>S</td>\n",
       "      <td>NaN</td>\n",
       "      <td>135</td>\n",
       "      <td>Montreal, PQ / Chesterville, ON</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>Allison, Mrs. Hudson J C (Bessie Waldo Daniels)</td>\n",
       "      <td>female</td>\n",
       "      <td>25</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>113781</td>\n",
       "      <td>151.55</td>\n",
       "      <td>C22 C26</td>\n",
       "      <td>S</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Montreal, PQ / Chesterville, ON</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Anderson, Mr. Harry</td>\n",
       "      <td>male</td>\n",
       "      <td>48</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>19952</td>\n",
       "      <td>26.55</td>\n",
       "      <td>E12</td>\n",
       "      <td>S</td>\n",
       "      <td>3</td>\n",
       "      <td>NaN</td>\n",
       "      <td>New York, NY</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Andrews, Miss. Kornelia Theodosia</td>\n",
       "      <td>female</td>\n",
       "      <td>63</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>13502</td>\n",
       "      <td>77.9583</td>\n",
       "      <td>D7</td>\n",
       "      <td>S</td>\n",
       "      <td>10</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Hudson, NY</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>Andrews, Mr. Thomas Jr</td>\n",
       "      <td>male</td>\n",
       "      <td>39</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>112050</td>\n",
       "      <td>0</td>\n",
       "      <td>A36</td>\n",
       "      <td>S</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Belfast, NI</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Appleton, Mrs. Edward Dale (Charlotte Lamson)</td>\n",
       "      <td>female</td>\n",
       "      <td>53</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>11769</td>\n",
       "      <td>51.4792</td>\n",
       "      <td>C101</td>\n",
       "      <td>S</td>\n",
       "      <td>D</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Bayside, Queens, NY</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>Artagaveytia, Mr. Ramon</td>\n",
       "      <td>male</td>\n",
       "      <td>71</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>PC 17609</td>\n",
       "      <td>49.5042</td>\n",
       "      <td>NaN</td>\n",
       "      <td>C</td>\n",
       "      <td>NaN</td>\n",
       "      <td>22</td>\n",
       "      <td>Montevideo, Uruguay</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   pclass  survived                                             name     sex  \\\n",
       "0       1         1                    Allen, Miss. Elisabeth Walton  female   \n",
       "1       1         1                   Allison, Master. Hudson Trevor    male   \n",
       "2       1         0                     Allison, Miss. Helen Loraine  female   \n",
       "3       1         0             Allison, Mr. Hudson Joshua Creighton    male   \n",
       "4       1         0  Allison, Mrs. Hudson J C (Bessie Waldo Daniels)  female   \n",
       "5       1         1                              Anderson, Mr. Harry    male   \n",
       "6       1         1                Andrews, Miss. Kornelia Theodosia  female   \n",
       "7       1         0                           Andrews, Mr. Thomas Jr    male   \n",
       "8       1         1    Appleton, Mrs. Edward Dale (Charlotte Lamson)  female   \n",
       "9       1         0                          Artagaveytia, Mr. Ramon    male   \n",
       "\n",
       "      age  sibsp  parch    ticket      fare    cabin embarked boat body  \\\n",
       "0      29      0      0     24160  211.3375       B5        S    2  NaN   \n",
       "1  0.9167      1      2    113781    151.55  C22 C26        S   11  NaN   \n",
       "2       2      1      2    113781    151.55  C22 C26        S  NaN  NaN   \n",
       "3      30      1      2    113781    151.55  C22 C26        S  NaN  135   \n",
       "4      25      1      2    113781    151.55  C22 C26        S  NaN  NaN   \n",
       "5      48      0      0     19952     26.55      E12        S    3  NaN   \n",
       "6      63      1      0     13502   77.9583       D7        S   10  NaN   \n",
       "7      39      0      0    112050         0      A36        S  NaN  NaN   \n",
       "8      53      2      0     11769   51.4792     C101        S    D  NaN   \n",
       "9      71      0      0  PC 17609   49.5042      NaN        C  NaN   22   \n",
       "\n",
       "                         home.dest  \n",
       "0                     St Louis, MO  \n",
       "1  Montreal, PQ / Chesterville, ON  \n",
       "2  Montreal, PQ / Chesterville, ON  \n",
       "3  Montreal, PQ / Chesterville, ON  \n",
       "4  Montreal, PQ / Chesterville, ON  \n",
       "5                     New York, NY  \n",
       "6                       Hudson, NY  \n",
       "7                      Belfast, NI  \n",
       "8              Bayside, Queens, NY  \n",
       "9              Montevideo, Uruguay  "
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = df.iloc[:,:].replace('?', np.nan)\n",
    "df.head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Obliczneie ilości nan dla poszczególnych kolumn"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "pclass          0\n",
       "survived        0\n",
       "name            0\n",
       "sex             0\n",
       "age           263\n",
       "sibsp           0\n",
       "parch           0\n",
       "ticket          0\n",
       "fare            1\n",
       "cabin        1014\n",
       "embarked        2\n",
       "boat          823\n",
       "body         1188\n",
       "home.dest     564\n",
       "dtype: int64"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nan_sum = df.isnull().sum()\n",
    "nan_sum\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Oblicznie procentowej ilości nan w konkretnych kolumnach dla wszystkich danych w kolumnie"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "pclass        0.000000\n",
       "survived      0.000000\n",
       "name          0.000000\n",
       "sex           0.000000\n",
       "age          20.091673\n",
       "sibsp         0.000000\n",
       "parch         0.000000\n",
       "ticket        0.000000\n",
       "fare          0.076394\n",
       "cabin        77.463713\n",
       "embarked      0.152788\n",
       "boat         62.872422\n",
       "body         90.756303\n",
       "home.dest    43.086325\n",
       "dtype: float64"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nan_sum/df.shape[0]*100 #df.shape[0] - liczba wierszy"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Policzenie średniej ilości nan-ów w kazdej kolumnie - wynikiem będzie stosunek ilości nan-ów do wierszy w kolumnie z występującymi danymi"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "pclass       0.000000\n",
       "survived     0.000000\n",
       "name         0.000000\n",
       "sex          0.000000\n",
       "age          0.200917\n",
       "sibsp        0.000000\n",
       "parch        0.000000\n",
       "ticket       0.000000\n",
       "fare         0.000764\n",
       "cabin        0.774637\n",
       "embarked     0.001528\n",
       "boat         0.628724\n",
       "body         0.907563\n",
       "home.dest    0.430863\n",
       "dtype: float64"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.isnull().mean()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Obliczenie jaki procent w dataframe stanowią braki danych"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "21.035687002073555"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nan_sum.sum()/df.size*100"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h5>Określenie typu braków danych: całkowicie przypadkowe (MCAR), przypadkowe (MAR), nieprzypadkowe (MNAR).</h5>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>pclass</th>\n",
       "      <th>survived</th>\n",
       "      <th>name</th>\n",
       "      <th>sex</th>\n",
       "      <th>age</th>\n",
       "      <th>sibsp</th>\n",
       "      <th>parch</th>\n",
       "      <th>ticket</th>\n",
       "      <th>fare</th>\n",
       "      <th>cabin</th>\n",
       "      <th>embarked</th>\n",
       "      <th>boat</th>\n",
       "      <th>body</th>\n",
       "      <th>home.dest</th>\n",
       "      <th>bodyNull</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Allen, Miss. Elisabeth Walton</td>\n",
       "      <td>female</td>\n",
       "      <td>29</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>24160</td>\n",
       "      <td>211.3375</td>\n",
       "      <td>B5</td>\n",
       "      <td>S</td>\n",
       "      <td>2</td>\n",
       "      <td>NaN</td>\n",
       "      <td>St Louis, MO</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Allison, Master. Hudson Trevor</td>\n",
       "      <td>male</td>\n",
       "      <td>0.9167</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>113781</td>\n",
       "      <td>151.55</td>\n",
       "      <td>C22 C26</td>\n",
       "      <td>S</td>\n",
       "      <td>11</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Montreal, PQ / Chesterville, ON</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>Allison, Miss. Helen Loraine</td>\n",
       "      <td>female</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>113781</td>\n",
       "      <td>151.55</td>\n",
       "      <td>C22 C26</td>\n",
       "      <td>S</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Montreal, PQ / Chesterville, ON</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>Allison, Mr. Hudson Joshua Creighton</td>\n",
       "      <td>male</td>\n",
       "      <td>30</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>113781</td>\n",
       "      <td>151.55</td>\n",
       "      <td>C22 C26</td>\n",
       "      <td>S</td>\n",
       "      <td>NaN</td>\n",
       "      <td>135</td>\n",
       "      <td>Montreal, PQ / Chesterville, ON</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>Allison, Mrs. Hudson J C (Bessie Waldo Daniels)</td>\n",
       "      <td>female</td>\n",
       "      <td>25</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>113781</td>\n",
       "      <td>151.55</td>\n",
       "      <td>C22 C26</td>\n",
       "      <td>S</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Montreal, PQ / Chesterville, ON</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1304</th>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>Zabour, Miss. Hileni</td>\n",
       "      <td>female</td>\n",
       "      <td>14.5</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>2665</td>\n",
       "      <td>14.4542</td>\n",
       "      <td>NaN</td>\n",
       "      <td>C</td>\n",
       "      <td>NaN</td>\n",
       "      <td>328</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1305</th>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>Zabour, Miss. Thamine</td>\n",
       "      <td>female</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>2665</td>\n",
       "      <td>14.4542</td>\n",
       "      <td>NaN</td>\n",
       "      <td>C</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1306</th>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>Zakarian, Mr. Mapriededer</td>\n",
       "      <td>male</td>\n",
       "      <td>26.5</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2656</td>\n",
       "      <td>7.225</td>\n",
       "      <td>NaN</td>\n",
       "      <td>C</td>\n",
       "      <td>NaN</td>\n",
       "      <td>304</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1307</th>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>Zakarian, Mr. Ortin</td>\n",
       "      <td>male</td>\n",
       "      <td>27</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2670</td>\n",
       "      <td>7.225</td>\n",
       "      <td>NaN</td>\n",
       "      <td>C</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1308</th>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>Zimmerman, Mr. Leo</td>\n",
       "      <td>male</td>\n",
       "      <td>29</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>315082</td>\n",
       "      <td>7.875</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>1309 rows × 15 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "      pclass  survived                                             name  \\\n",
       "0          1         1                    Allen, Miss. Elisabeth Walton   \n",
       "1          1         1                   Allison, Master. Hudson Trevor   \n",
       "2          1         0                     Allison, Miss. Helen Loraine   \n",
       "3          1         0             Allison, Mr. Hudson Joshua Creighton   \n",
       "4          1         0  Allison, Mrs. Hudson J C (Bessie Waldo Daniels)   \n",
       "...      ...       ...                                              ...   \n",
       "1304       3         0                             Zabour, Miss. Hileni   \n",
       "1305       3         0                            Zabour, Miss. Thamine   \n",
       "1306       3         0                        Zakarian, Mr. Mapriededer   \n",
       "1307       3         0                              Zakarian, Mr. Ortin   \n",
       "1308       3         0                               Zimmerman, Mr. Leo   \n",
       "\n",
       "         sex     age  sibsp  parch  ticket      fare    cabin embarked boat  \\\n",
       "0     female      29      0      0   24160  211.3375       B5        S    2   \n",
       "1       male  0.9167      1      2  113781    151.55  C22 C26        S   11   \n",
       "2     female       2      1      2  113781    151.55  C22 C26        S  NaN   \n",
       "3       male      30      1      2  113781    151.55  C22 C26        S  NaN   \n",
       "4     female      25      1      2  113781    151.55  C22 C26        S  NaN   \n",
       "...      ...     ...    ...    ...     ...       ...      ...      ...  ...   \n",
       "1304  female    14.5      1      0    2665   14.4542      NaN        C  NaN   \n",
       "1305  female     NaN      1      0    2665   14.4542      NaN        C  NaN   \n",
       "1306    male    26.5      0      0    2656     7.225      NaN        C  NaN   \n",
       "1307    male      27      0      0    2670     7.225      NaN        C  NaN   \n",
       "1308    male      29      0      0  315082     7.875      NaN        S  NaN   \n",
       "\n",
       "     body                        home.dest  bodyNull  \n",
       "0     NaN                     St Louis, MO         1  \n",
       "1     NaN  Montreal, PQ / Chesterville, ON         1  \n",
       "2     NaN  Montreal, PQ / Chesterville, ON         1  \n",
       "3     135  Montreal, PQ / Chesterville, ON         0  \n",
       "4     NaN  Montreal, PQ / Chesterville, ON         1  \n",
       "...   ...                              ...       ...  \n",
       "1304  328                              NaN         0  \n",
       "1305  NaN                              NaN         1  \n",
       "1306  304                              NaN         0  \n",
       "1307  NaN                              NaN         1  \n",
       "1308  NaN                              NaN         1  \n",
       "\n",
       "[1309 rows x 15 columns]"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['bodyNull'] = np.where(df['body'].isnull(), 1, 0)\n",
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "survived\n",
       "0    0.850433\n",
       "1    1.000000\n",
       "Name: bodyNull, dtype: float64"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.groupby(['survived'])['bodyNull'].mean()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "W szczegółowym opisie danych wyczytałem, ze body to Body Identification Number, co oznacza, ze ta kolumna powinna tyczyć się jedynie osób, które nie przezyly,\n",
    "co obrazują powysze wyniki: \n",
    "- 1.000 - 100% brak danych (100% nan-ów) dla pasazerow, ktorzy przezyli (brak koniecznosci identyfikacji ciala)\n",
    "- 0.850433 - 85% brak danych dla ludzi, ktorzy nie przezyli (ciała większości osób nie zostały znalezione)\n",
    ">Powyzsze wyniki pozwalają wysunąć wnioski, ze mamy do czynienia z MNAR - braki w danych nie są przypadkowe."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Powyzsze dwie linie kodu zapisane w jednej:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "survived\n",
       "0    0.850433\n",
       "1    1.000000\n",
       "Name: body, dtype: float64"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['body'].isna().groupby(df['survived']).mean()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Obliczając procentowy brak danych dla całego zbioru danych, wyszedł on na poziomie 21%, co uniemozliwia nam usunięcie wszystkich wierszy z tego typu brakami.\n",
    "Jak napisałem powyzej, w zbiorze mamy do czynienia z nieprzypadkowymi brakami danych, co równierz nalezy uwzględnić w analizie. W celu  usunięcia braków, nalzałoby się skupić na poszczególnych kolumnach i dla kazdej indywidualnie decydować czy mozemy usunać w niej wiersze z NaN"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h5>Sprawdzenie liczebności poszczególnych etykiet dla danych zmiennych\n",
    "jakościowych</h5>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>pclass</th>\n",
       "      <th>survived</th>\n",
       "      <th>name</th>\n",
       "      <th>sex</th>\n",
       "      <th>age</th>\n",
       "      <th>sibsp</th>\n",
       "      <th>parch</th>\n",
       "      <th>ticket</th>\n",
       "      <th>fare</th>\n",
       "      <th>cabin</th>\n",
       "      <th>embarked</th>\n",
       "      <th>boat</th>\n",
       "      <th>body</th>\n",
       "      <th>home.dest</th>\n",
       "      <th>bodyNull</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Allen, Miss. Elisabeth Walton</td>\n",
       "      <td>female</td>\n",
       "      <td>29</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>24160</td>\n",
       "      <td>211.3375</td>\n",
       "      <td>B5</td>\n",
       "      <td>S</td>\n",
       "      <td>2</td>\n",
       "      <td>NaN</td>\n",
       "      <td>St Louis, MO</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Allison, Master. Hudson Trevor</td>\n",
       "      <td>male</td>\n",
       "      <td>0.9167</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>113781</td>\n",
       "      <td>151.55</td>\n",
       "      <td>C22 C26</td>\n",
       "      <td>S</td>\n",
       "      <td>11</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Montreal, PQ / Chesterville, ON</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>Allison, Miss. Helen Loraine</td>\n",
       "      <td>female</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>113781</td>\n",
       "      <td>151.55</td>\n",
       "      <td>C22 C26</td>\n",
       "      <td>S</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Montreal, PQ / Chesterville, ON</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>Allison, Mr. Hudson Joshua Creighton</td>\n",
       "      <td>male</td>\n",
       "      <td>30</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>113781</td>\n",
       "      <td>151.55</td>\n",
       "      <td>C22 C26</td>\n",
       "      <td>S</td>\n",
       "      <td>NaN</td>\n",
       "      <td>135</td>\n",
       "      <td>Montreal, PQ / Chesterville, ON</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>Allison, Mrs. Hudson J C (Bessie Waldo Daniels)</td>\n",
       "      <td>female</td>\n",
       "      <td>25</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>113781</td>\n",
       "      <td>151.55</td>\n",
       "      <td>C22 C26</td>\n",
       "      <td>S</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Montreal, PQ / Chesterville, ON</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   pclass  survived                                             name     sex  \\\n",
       "0       1         1                    Allen, Miss. Elisabeth Walton  female   \n",
       "1       1         1                   Allison, Master. Hudson Trevor    male   \n",
       "2       1         0                     Allison, Miss. Helen Loraine  female   \n",
       "3       1         0             Allison, Mr. Hudson Joshua Creighton    male   \n",
       "4       1         0  Allison, Mrs. Hudson J C (Bessie Waldo Daniels)  female   \n",
       "\n",
       "      age  sibsp  parch  ticket      fare    cabin embarked boat body  \\\n",
       "0      29      0      0   24160  211.3375       B5        S    2  NaN   \n",
       "1  0.9167      1      2  113781    151.55  C22 C26        S   11  NaN   \n",
       "2       2      1      2  113781    151.55  C22 C26        S  NaN  NaN   \n",
       "3      30      1      2  113781    151.55  C22 C26        S  NaN  135   \n",
       "4      25      1      2  113781    151.55  C22 C26        S  NaN  NaN   \n",
       "\n",
       "                         home.dest  bodyNull  \n",
       "0                     St Louis, MO         1  \n",
       "1  Montreal, PQ / Chesterville, ON         1  \n",
       "2  Montreal, PQ / Chesterville, ON         1  \n",
       "3  Montreal, PQ / Chesterville, ON         0  \n",
       "4  Montreal, PQ / Chesterville, ON         1  "
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Liczba etykiet zmiennej pclass: 3\n",
      "Liczba etykiet zmiennej survived: 2\n",
      "Liczba etykiet zmiennej name: 1307\n",
      "Liczba etykiet zmiennej sex: 2\n",
      "Liczba etykiet zmiennej age: 99\n",
      "Liczba etykiet zmiennej sibsp: 7\n",
      "Liczba etykiet zmiennej parch: 8\n",
      "Liczba etykiet zmiennej ticket: 929\n",
      "Liczba etykiet zmiennej fare: 282\n",
      "Liczba etykiet zmiennej cabin: 187\n",
      "Liczba etykiet zmiennej embarked: 4\n",
      "Liczba etykiet zmiennej boat: 28\n",
      "Liczba etykiet zmiennej body: 122\n",
      "Liczba etykiet zmiennej home.dest: 370\n",
      "Liczba etykiet zmiennej bodyNull: 2\n"
     ]
    }
   ],
   "source": [
    "for i in df.columns:\n",
    "    print('Liczba etykiet zmiennej ' + i + ': {}'.format(len(df[i].unique())))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h5> Wyświetlenie z użyciem funkcji print liczby wszystkich pasażerów.</h5>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Liczba wszystkich pasazerow:  1307\n"
     ]
    }
   ],
   "source": [
    "print('Liczba wszystkich pasazerow: ', len(df.name.unique()))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h5>Wykorzystanie podobny sposób jak w punkcie 1.</h5>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Liczba wszystkich pasazerow: 1307\n"
     ]
    }
   ],
   "source": [
    "print('Liczba wszystkich pasazerow: {}'.format(len(df['name'].unique())))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h5>Komentarz wyników otrzymanych w punkcie 1 i 2 oraz podział zmiennych ze\n",
    "względu na dużą i małą moc zbioru (kardynalność).</h5>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Na podstawie podpunktu 1 i 2 zauwazyłem, ze dane dziela sie na etykiety do wartości 30 oraz powyzej 99, dlatego tez według takiego kryterium podziele je na zbiory"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "card_1=[] #duza moc zbioru\n",
    "card_2=[] #mala moc zbioru\n",
    "for i in df.columns:\n",
    "    np.where(len(df[i].unique()) >= 30, card_1.append(df[i]), card_2.append(df[i]))\n",
    "# print(card_1)[1,:]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h5>Sprawdzeniue ilości unikalnych etykiet dla zmiennej mówiącej o kabinie danego\n",
    "pasażera. </h5>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Liczba unikalnych etykiet: 187\n",
      "Typ zmiennej cabin_label: <class 'numpy.ndarray'>\n"
     ]
    }
   ],
   "source": [
    "cabin_label = df.cabin.unique()\n",
    "print(\"Liczba unikalnych etykiet: \" + str(len(cabin_label)))\n",
    "print(\"Typ zmiennej cabin_label: \" + str(type(cabin_label)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Pandas DataFrame jest przeznaczone do przechowywania danych tabelarycznych, natomiast NumPy array słuzy do przechowywania danych numerycznych."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h5>Redukcja cech dla zmiennej opisującej kabiny poprzez zastąpienie\n",
    "obecnych etykiet w formacie LL11 do etykiet zawierających tylko\n",
    "pierwszą literę.</h5>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>cabin</th>\n",
       "      <th>CabinReduced</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>B5</td>\n",
       "      <td>B</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>C22 C26</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>C22 C26</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>C22 C26</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>C22 C26</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>E12</td>\n",
       "      <td>E</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>D7</td>\n",
       "      <td>D</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>A36</td>\n",
       "      <td>A</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>C101</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>C62 C64</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>C62 C64</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>B35</td>\n",
       "      <td>B</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>A23</td>\n",
       "      <td>A</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>B58 B60</td>\n",
       "      <td>B</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>B58 B60</td>\n",
       "      <td>B</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>D15</td>\n",
       "      <td>D</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>C6</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      cabin CabinReduced\n",
       "0        B5            B\n",
       "1   C22 C26            C\n",
       "2   C22 C26            C\n",
       "3   C22 C26            C\n",
       "4   C22 C26            C\n",
       "5       E12            E\n",
       "6        D7            D\n",
       "7       A36            A\n",
       "8      C101            C\n",
       "9       NaN          NaN\n",
       "10  C62 C64            C\n",
       "11  C62 C64            C\n",
       "12      B35            B\n",
       "13      NaN          NaN\n",
       "14      A23            A\n",
       "15      NaN          NaN\n",
       "16  B58 B60            B\n",
       "17  B58 B60            B\n",
       "18      D15            D\n",
       "19       C6            C"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['CabinReduced'] = np.where(df['cabin'].isnull(), np.nan, df['cabin'].astype(str).str[0])\n",
    "df[['cabin', 'CabinReduced']].head(20)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h5>Wyświetlenie (jak w punkcie 1) liczby etykiet dla zmiennych z punktu wyzej.</h5>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Liczba etykiet zmiennej CabinReduced: 9\n"
     ]
    }
   ],
   "source": [
    "print('Liczba etykiet zmiennej CabinReduced: {}'.format(len(df.CabinReduced.unique())))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "95.18716577540107"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "100 - len(df['CabinReduced'].unique())/len(df['cabin'].unique())*100\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Zredukowano kardynalność zbioru zmiennej opisującej kabiny o 95.18716577540107%"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Taka redukcja znacznie ułatwi i przyspieszy nam analizę zbioru danych, co prawda kosztem utraty szczegółowości danych, ale mimo wszystko przy zachowaniu kluczowej inforamcji - litery mówiącej o danej klasie. Mowa w tym przypadku o redukcji az o\n",
    "95.18716577540107% w stosunku do zbioru pierwotnego. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "df.to_csv('Titanic_modified.csv', index=False) #index=False jeśli nie chchemy zapisywać indeksów do pliku"
   ]
  }
 ],
 "metadata": {
  "interpreter": {
   "hash": "f959eabf291552a887313e5fc997545559a716633a8ac30b7d916bcbbbbab4d7"
  },
  "kernelspec": {
   "display_name": "Python 3.6.10 ('DataScience')",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.7"
  },
  "orig_nbformat": 4
 },
 "nbformat": 4,
 "nbformat_minor": 2
}