07.1-pandas.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Pandas\n",
    "\n",
    "* Pandas es un paquete de Python que implementa ciertos tipos de datos, construidos sobre `numpy`\n",
    "* Los tipos básicos de Pandas son\n",
    "    * Series: 1 dimensión\n",
    "        * Un array 1D con etiquetas en sus ejes.\n",
    "    * DataFrames: 2 dimensiones\n",
    "        * Datos en formato tabular, con etiquetas en sus ejes.\n",
    "    * Un DataFrame son datos tabulares (tablas)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy\n",
    "import pandas"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Series de datos\n",
    "\n",
    "* Array 1D con etiquetas en sus ejes.\n",
    "* De forma general se crean de la forma\n",
    "```python\n",
    "pandas.Series(data, index=index)\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Ejemplo\n",
    "\n",
    "s = pandas.Series(range(0, 100, 10))\n",
    "s"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "s.values  # Array 1D"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "type(s.values)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "s.index  # Etiquetas. Puesto que no hemos indicado nada, el índice es numérico"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Indexing, slicing\n",
    "s[5]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "s[5:7]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* La particularidad de las `Series`, es que sus ejes están etiquetados, luego podemos indicar el índice que queramos."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "pandas.Series([1, 2, 3, 4, 5],\n",
    "              index=[\"uno\", \"dos\", \"tres\", \"cuatro\", \"cinco\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Lo que nos permite crear `Series` a través de diccionarios"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "edad = {\n",
    "    \"maria\": 25,\n",
    "    \"pedro\": 76,\n",
    "    \"arturo\": 80,\n",
    "    \"rosa\": 19,\n",
    "}\n",
    "\n",
    "pandas.Series(edad)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## DataFrame\n",
    "\n",
    "* Un DataFrame es una representación de datos, de forma tabular, con etiquetas en los índices.\n",
    "* Se pueden crear de varias maneras"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# A través de una Serie\n",
    "\n",
    "edad = {\n",
    "    \"maria\": 25,\n",
    "    \"pedro\": 76,\n",
    "    \"arturo\": 80,\n",
    "    \"rosa\": 19,\n",
    "}\n",
    "\n",
    "edad = pandas.Series(edad)\n",
    "\n",
    "personas = pandas.DataFrame(edad, columns=[\"edad\"])\n",
    "personas"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# A través de varias Series\n",
    "\n",
    "edad = {\n",
    "    \"maria\": 25,\n",
    "    \"pedro\": 76,\n",
    "    \"arturo\": 80,\n",
    "    \"rosa\": 19,\n",
    "}\n",
    "\n",
    "edad = pandas.Series(edad)\n",
    "\n",
    "altura = {\n",
    "    \"maria\": 1.75,\n",
    "    \"pedro\": 1.90,\n",
    "    \"arturo\": 1.68,\n",
    "    \"rosa\": 1.65,\n",
    "}\n",
    "\n",
    "altura = pandas.Series(altura)\n",
    "\n",
    "personas = pandas.DataFrame({\"edad\": edad, \"altura\": altura})\n",
    "personas"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# A través de un diccionario\n",
    "\n",
    "datos = {\n",
    "    \"altura\": [1.75, 1.90, 1.68, 1.65],\n",
    "    \"edad\": [25, 76, 80, 19],\n",
    "}\n",
    "\n",
    "nombres = [\"maria\", \"pedro\", \"arturo\", \"rosa\"]\n",
    "\n",
    "personas = pandas.DataFrame(datos, index=nombres)\n",
    "personas"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# A través de un diccionario\n",
    "\n",
    "datos = {\n",
    "    \"altura\": [1.75, 1.90, 1.68, 1.65],\n",
    "    \"edad\": [25, 76, 80, 19],\n",
    "    \"nombres\": [\"maria\", \"pedro\", \"arturo\", \"rosa\"],\n",
    "}\n",
    "\n",
    "\n",
    "personas = pandas.DataFrame(datos)\n",
    "print(personas)\n",
    "personas = personas.set_index(\"nombres\")\n",
    "personas"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# A través de una lista de diccionarios\n",
    "\n",
    "datos = [\n",
    "    {'altura': 1.75, 'edad': 25},\n",
    "    {'altura': 1.9, 'edad': 76},\n",
    "    {'altura': 1.68, 'edad': 80},\n",
    "    {'altura': 1.65, 'edad': 19}\n",
    "]\n",
    " \n",
    "personas = pandas.DataFrame(datos, index=[\"maria\", \"pedro\", \"arturo\", \"rosa\"])\n",
    "personas "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Pandas gestiona que algún dato no exista, rellenando con NaN (not a number)\n",
    "\n",
    "edad = {\n",
    "    \"maria\": 25,\n",
    "    \"pedro\": 76,\n",
    "    \"arturo\": 80,\n",
    "    \"rosa\": 19,\n",
    "    \"sebastian\": 17\n",
    "}\n",
    "\n",
    "edad = pandas.Series(edad)\n",
    "\n",
    "altura = {\n",
    "    \"maria\": 1.75,\n",
    "    \"pedro\": 1.90,\n",
    "    \"arturo\": 1.68,\n",
    "    \"rosa\": 1.65,\n",
    "}\n",
    "\n",
    "altura = pandas.Series(altura)\n",
    "\n",
    "personas = pandas.DataFrame({\"edad\": edad, \"altura\": altura})\n",
    "personas"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "# A través de un fichero\n",
    "\n",
    "data = pandas.read_csv(\"data/populations.txt\")\n",
    "data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# A través de un fichero (esta vez si)\n",
    "\n",
    "data = pandas.read_csv(\"data/populations.txt\", sep=\"\\t\")\n",
    "data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# A través de un fichero (todavía mejor)\n",
    "\n",
    "data = pandas.read_csv(\"data/populations.txt\", sep=\"\\t\", index_col=0)\n",
    "data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Indexing y selección de datos\n",
    "\n",
    "### Series\n",
    "\n",
    "* Una `Series` en `pandas` tiene similitudes con un `array` de `numpy` y un diccionario de Python, con lo que podemos realizar operaciones similares."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "s = pandas.Series([1, 5, 2, 3, 4, 5],\n",
    "                  index=[\"a\", \"b\", \"c\", \"d\", \"e\", \"f\"])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "s[\"a\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "\"a\" in s"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "s.keys()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "s.values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Podemos modificar\n",
    "\n",
    "s[\"a\"] = 10"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Añadir\n",
    "\n",
    "s[\"z\"] = 20\n",
    "s"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "s[\"a\": \"d\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Al igual que en numpy, podemos indexar a partir de otro array/lista\n",
    "\n",
    "s[[\"a\", \"d\"]]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Y también a través de un array de booleanos\n",
    "\n",
    "s[s > 3]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Peligro: las series tienen índices implícitos (Python, orden de los elementos) y explícitos (proporcionados por nosotros.\n",
    "* Si ambos índices son enteros, puede ser una fuente de confusión.\n",
    "* EJemplo"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "s = pandas.Series([\"a\", \"b\", \"c\", \"d\", \"e\", \"f\"],\n",
    "                  index=[1, 3, 5, 7, 9, 11])\n",
    "s"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "s[3]  # 'b', índice explícito cuando se accede a un elemento"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "s[3:5]  # ['d', 'e'], índice implícito de Python"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Para solucionar estas ambiguedades, Pandas ofrece dos interfaces para los índices explícitos e implícitos: `.loc` y `.iloc`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "s.loc[3]  # Explícito"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "s.loc[3:5]  # Explícito"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "s.iloc[3]  # Implícito"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "s.iloc[3:5]  # Implícito"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### DataFrame\n",
    "\n",
    "* Para simplificar, podemos considerar un DataFrame como un diccionario de Series (columnas)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Pandas gestiona que algún dato no exista, rellenando con NaN (not a number)\n",
    "\n",
    "edad = {\n",
    "    \"maria\": 25,\n",
    "    \"pedro\": 76,\n",
    "    \"arturo\": 80,\n",
    "    \"rosa\": 19,\n",
    "    \"marta\": 33,\n",
    "    \"juan\": 77,\n",
    "    \"sebastian\": 17\n",
    "}\n",
    "\n",
    "edad = pandas.Series(edad)\n",
    "\n",
    "altura = {\n",
    "    \"maria\": 1.75,\n",
    "    \"pedro\": 1.90,\n",
    "    \"arturo\": 1.68,\n",
    "    \"rosa\": 1.65,\n",
    "    \"marta\": 1.70,\n",
    "    \"juan\": 1.84,\n",
    "}\n",
    "\n",
    "altura = pandas.Series(altura)\n",
    "\n",
    "peso = {\n",
    "    \"maria\": 62,\n",
    "    \"pedro\": 122,\n",
    "    \"arturo\": 75,\n",
    "    \"rosa\": 64,\n",
    "    \"marta\": 55,\n",
    "    \"juan\": 70,\n",
    "}\n",
    "\n",
    "peso = pandas.Series(peso)\n",
    "\n",
    "personas = pandas.DataFrame({\"edad\": edad, \"altura\": altura, \"peso\": peso})\n",
    "personas"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Puedo acceder a las columnas directamente\n",
    "\n",
    "personas[\"edad\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# O bien \n",
    "\n",
    "personas.edad"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Tambien podemos añadir y realizar operaciones element-wise\n",
    "\n",
    "personas[\"imc\"] = personas[\"peso\"] / (personas[\"altura\"]**2)\n",
    "personas"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* *Pero*, la estructura de un DataFrame son Series, que en esencia son arrays.\n",
    "* Se puede acceer al array a través del atributo `.values`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "personas.values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Accedo a la fila 0\n",
    "\n",
    "personas.values[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Accedo a la columna 0\n",
    "\n",
    "personas.values[:, 0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Un DataFrame también tiene índices explícitos e implícitos, a los que se accede también a través de `.iloc` y `.loc`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "personas.iloc[1:4]  # Indice implícito"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "personas.iloc[1:4, 2:4]  # Indice implícito"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "personas.loc[\"maria\": \"rosa\"]  # Indice explícito"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "personas.loc[\"maria\": \"rosa\", \"peso\": \"imc\"]  # Indice explícito"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Esto nos permite utilizar todas las formas de indexing de `numpy`!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "personas.loc[personas[\"imc\"] > 25]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Plotting"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import matplotlib\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "personas[[\"imc\", \"peso\"]].plot(kind=\"bar\", figsize=(15,10))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data.plot(figsize=(15,10))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}