F0C-09-2DPlots.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# FLIP (00): Data Science \n",
    "**(Module 02: Data Visualization)**\n",
    "\n",
    "---\n",
    "- Materials in this module include resources collected from various open-source online repositories.\n",
    "- You are free to use,but NOT allowed to change and distribute this package.\n",
    "\n",
    "Prepared by and for \n",
    "**Student Members** |\n",
    "2006-2018 [TULIP Lab](http://www.tulip.org.au), Australia\n",
    "\n",
    "---\n",
    "\n",
    "# Session J - 2D Plots\n",
    "\n",
    "\n",
    "In our last lesson, we took a somewhat \"in depth\" look at the `plot` command to get a better feel for how matplotlib's plotting functions work. In that lesson, we saw just how flexible the typical matplotlib plotting function can be. In this lesson, we'll expand our view a bit, and take a look at a few of the more common plotting functions that matplotlib provides. While we won't cover all of the plotting functions in the library, we'll hopefully cover just enough to inspire you to continue exploring everything that matplotlib has to offer. \n",
    "\n",
    "Let's go ahead and get started with the obligatory set of calls to set up our Jupyter notebook's environment."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "\n",
    "from IPython.display import set_matplotlib_formats\n",
    "set_matplotlib_formats('retina')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Getting the Data\n",
    "\n",
    "In our last lesson, we created a small bit of fake data to play around with. While this can be useful when learning about a new function, it's not particularly interesting or satisfying. In this lesson, we'll explore a few more of the plotting functions that matplotlib provides, but this time we'll use some real world data to explore those functions. The data set that we're going to use is the (in)famous [Iris data set][1] introduced by the English statistician [Ronald Fisher][2] in his 1936 paper on [Linear Discriminant Analysis][3]. \n",
    "\n",
    "[1]: https://en.wikipedia.org/wiki/Iris_flower_data_set\n",
    "[2]: https://en.wikipedia.org/wiki/Ronald_Fisher\n",
    "[3]: https://en.wikipedia.org/wiki/Linear_discriminant_analysis\n",
    "\n",
    "The Iris data set has been used in a multitude of papers and tutorials and comes with many data visualization and statistical packages. Since we're not currently using any of those packages though, we'll have to get the data ourselves once again from the [UCI Machine Learning Repository][4]. The code below is nearly identical to the code used in our earlier EDA lesson, but to give you a quick refresher, all it does is download the data from the repository and load it into a variable called `data`. And, just like in that last lesson, I've once again included the data file for your use if you can't download it directly from the UCI Repository. You can find the file at `./data/iris_data.csv`.\n",
    "\n",
    "[4]: https://archive.ics.uci.edu/ml/datasets/Iris"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "!pip install urllib2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import csv\n",
    "import urllib2\n",
    "from collections import namedtuple\n",
    "\n",
    "# Create a namedtuple constructor for each record in the Iris data\n",
    "#    1. sepal length in cm\n",
    "#    2. sepal width in cm\n",
    "#    3. petal length in cm\n",
    "#    4. petal width in cm\n",
    "#    5. class: \n",
    "#       -- Iris-setosa\n",
    "#       -- Iris-versicolour\n",
    "#       -- Iris-virginica\n",
    "fields = ('sepal_length', \n",
    "          'sepal_width', \n",
    "          'petal_length', \n",
    "          'petal_width', \n",
    "          'species')\n",
    "IrisRecord = namedtuple('IrisRecord', fields)\n",
    "\n",
    "# Download and read in the data from the UCI Machine Learning Repository\n",
    "response = urllib2.urlopen('https://archive.ics.uci.edu/ml/machine-learning-databases/iris/iris.data')\n",
    "iris_data_csv = response.read().strip()\n",
    "\n",
    "# Convert each record into a format that's easier to work with (i.e.,\n",
    "# wrap each record in the namedtuple that we created).\n",
    "data = []\n",
    "for row in csv.reader(iris_data_csv.splitlines()):\n",
    "    data.append(IrisRecord(\n",
    "        sepal_length          = float(row[0]),\n",
    "        sepal_width           = float(row[1]),\n",
    "        petal_length          = float(row[2]),\n",
    "        petal_width           = float(row[3]),\n",
    "        species               = row[4]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## The Boxplot\n",
    "\n",
    "We're going to begin our exploration of our data with the [boxplot][1], one of the most useful visualization techniques for quickly getting an overview of the \"spread\" of numerical features within a data set. A boxplot splits the data from each feature into quartiles. The \"box\" in a boxplot, shows what is known as the [interquartile range (IQR)][2], which sounds complex, but is really just a fancy way of saying the middle 50%. Inside the box, you'll typically find a line that represents the median value of that feature. Above and below the box, are \"whiskers\" (or, pairs of lines that extend outward from the box) that show the boundaries of the upper and lower 25%. Finally, outliers are typically visualized by a set of points that lie just beyond the whiskers and represent each individual outlier in the graph.\n",
    "\n",
    "[1]: https://en.wikipedia.org/wiki/Box_plot\n",
    "[2]: https://en.wikipedia.org/wiki/Interquartile_range\n",
    "\n",
    "The `pyplot` interface provides a function called, appropriately, `boxplot` that takes a 2-dimensional array of numerical data and draws one boxplot for each column in the array. In the following code, we will use the `numpy.column_stack` function to stitch together data into a 2D array. Then we'll call the `boxplot` function with the newly formatted data. In the call, we will also pass in a dictionary with formatting information using the `flierprops` parameter---incidentally, \"fliers\" is the term matplotlib uses for outliers in the graph. This will give you a taste of just how easy it is to fine tune the look of your boxplot. Finally, we'll change the `x-axis` ticks to the names of each feature that we are showing in the plot."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Convert the Iris data into a 2-dimensional matrix \n",
    "# where each column is one feature in the data set.\n",
    "x = np.column_stack([\n",
    "    [r.sepal_length for r in data],\n",
    "    [r.sepal_width for r in data],\n",
    "    [r.petal_length for r in data],\n",
    "    [r.petal_width for r in data],\n",
    "])\n",
    "\n",
    "# Plot the data. We are changing the look of the outliers (aka, fliers),\n",
    "# one of the many configurable bits of the boxplot.\n",
    "plt.boxplot(x, flierprops={'marker':'o',                    # Mark outliers with a circle\n",
    "                           'markerfacecolor':'none',        # Empty circles (i.e., no color) \n",
    "                           'color': 'none',                 # Remove the outlier lines\n",
    "                           'markersize':3})                 # Reduce the size of the ourlier markers\n",
    "\n",
    "# Add the x-axis labels offset by 1 so they line up with\n",
    "# each of the box plots.\n",
    "names = ['Sepal Length', 'Sepal Width', 'Petal Length', 'Petal Width']\n",
    "plt.xticks(range(1, len(names)+1), names);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Creating a Color Map\n",
    "\n",
    "In the next few plots we'll be color coding the data according to its species, and though we don't really care what color each species ends up as, we do care that the color used is consistent throughout the rest of the plots. The next little bit of code uses the default list of colors that matplotlib typically cycles through, and the uber-useful [`defaultdict`][1] data structure to create a simple color map that we can use throughout the rest of this lesson.\n",
    "\n",
    "[1]: https://docs.python.org/2/library/collections.html#collections.defaultdict"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from itertools import cycle\n",
    "from collections import defaultdict\n",
    "\n",
    "# Create a color map of species names to colors\n",
    "color_cycler = cycle(plt.rcParams['axes.prop_cycle'])\n",
    "cmap = defaultdict(lambda: next(color_cycler)['color'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## The Scatterplot\n",
    "\n",
    "A really great tool for visualizing the relationships between features in our data is the [scatterplot][1]. A scatterplot graphs data using [cartesian coordinates][2] (i.e., `x` and `y` values), where each axis in the graph represents a different feature in the data. Plotting your data on a scatterplot allows you to quickly see relationships between features in your data set, and also visually pick out clusters as we'll see in our examples below. \n",
    "\n",
    "[1]: https://en.wikipedia.org/wiki/Scatter_plot\n",
    "[2]: https://en.wikipedia.org/wiki/Cartesian_coordinate_system\n",
    "\n",
    "To see an example of matplotlib's `scatter` function in action, we'll be plotting the sepal length and width against one another for our first example, and the petal length and width against each other for the second. The `scatter` function expects two lists: the first contains the `x-axis` values, and the second the `y-axis` values. So, the first thing we need to do is create a list containing the sepal lengths and another containing the widths. We also want to draw each point in the graph in a color specific to its species, so we'll also create a third list containing the color for each of the `x` and `y` pairs. Lines 4-8 below create each of these lists by simply iterating through each observation in the data and appending the length, width, and color (according to the species) to the appropriate list.\n",
    "\n",
    "When we call the `scatter` function we'll set the `color` parameter to `'none'` and the `edgecolors` parameter to our list of colors. Doing so will make each point in the graph an empty circle outlined by the color in the `edgecolors` list. We'll also reduce the size of each circle's outline using the `linewidths` parameter to make the plot a little more aesthetically pleasing. Next, we'll set the `x` and `y` labels to make the graph a bit easier to understand. Finally, we'll create a legend so we can see which species each data point belongs to in our graph. \n",
    "\n",
    "### Legends and \"Proxy Artists\"\n",
    "\n",
    "Calling the `legend` function without passing in any parameters will create a legend that contains an entry for every observation in our data that has a label attached to it. We don't currently have any of our data points labeled, and doing so would be a bit tough without calling the `scatter` function three different times---one for each species in our data set. To create our legend then, we will need to create a set of what matplotlib calls [\"proxy artists\"][3] that are data points that are used only for the creation of a legend and do not actually exist in our graph. Lines 27-31 create a list of \"proxy artists\". Then, on line 33 we pass that list into the `legend` function to create a legend with an entry for each species in our data set.\n",
    "\n",
    "[3]: http://matplotlib.org/users/legend_guide.html#creating-artists-specifically-for-adding-to-the-legend-aka-proxy-artists"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Extract the x and y values (i.e., sepal length and width)\n",
    "# from the data set. Also, create a list of corresponding\n",
    "# colors based on the species of each record.\n",
    "sepal_length, sepal_width, colors = [], [], []\n",
    "for r in data:\n",
    "    sepal_length.append(r.sepal_length)\n",
    "    sepal_width.append(r.sepal_width)\n",
    "    colors.append(cmap[r.species])\n",
    "    \n",
    "# Plot the data on a scatter plot. Each point will have an outline \n",
    "# color according to their species, but we'll leave the centers \n",
    "# colorless (for aesthetic purposes only). Finally, we'll reduce\n",
    "# the line width of each data point (again for aesthetics only).\n",
    "plt.scatter(sepal_length, sepal_width, edgecolors=colors, c='none', linewidths=0.5)\n",
    "\n",
    "# Add x and y labels\n",
    "plt.xlabel('Sepal Length')\n",
    "plt.ylabel('Sepal Width')\n",
    "\n",
    "# Create a few \"proxy artists\" for creating the legend. Calling the \n",
    "# legend() function without passing in any parameters will create a\n",
    "# legend for any labeled data in the graph, so we could do something\n",
    "# funky where we add a label for one line per species, but that \n",
    "# doesn't seem as simple and clean as just creating a few lines that\n",
    "# we use just to create the legend. Plus, this is an officially \n",
    "# sanctioned way to create a legend (just see the legend docs).\n",
    "legend_handles = [\n",
    "    plt.scatter([], [], marker='o', label='setosa', edgecolors=cmap['Iris-setosa'], c='none', linewidths=0.5),\n",
    "    plt.scatter([], [], marker='o', label='versicolor', edgecolors=cmap['Iris-versicolor'], c='none', linewidths=0.5),\n",
    "    plt.scatter([], [], marker='o', label='virginica', edgecolors=cmap['Iris-virginica'], c='none', linewidths=0.5)\n",
    "]\n",
    "\n",
    "plt.legend(handles=legend_handles, scatterpoints=1, ncol=3, loc='upper center');"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the scatterplot above, we can see a somewhat positive relationship between the sepal length and the width, especially within the Iris-setosa species. We can also see a definite division between the setosa species and the other two species. Assuming we wanted to create a model for classifying each flower according to their species, we could use the sepal dimensions, but that would really only give us a good model for classifying the setosa species versus everything else. So, perhaps we should keep exploring.\n",
    "\n",
    "In the next example, we'll create another scatterplot, this time, however, we're going to plot the petal length against its width. We'll also move the legend around a bit so as to not obscure any of our data, but other than that, the next example is nearly identical to the previous one."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Extract the x and y values (i.e., petal length and width)\n",
    "# from the data set. Also, create a list of corresponding\n",
    "# colors based on the species of each record.\n",
    "petal_length, petal_width, colors = [], [], []\n",
    "for r in data:\n",
    "    petal_length.append(r.petal_length)\n",
    "    petal_width.append(r.petal_width)\n",
    "    colors.append(cmap[r.species])\n",
    "    \n",
    "# Plot the data on a scatter plot. Each point will have an outline \n",
    "# color according to their species, but we'll leave the centers \n",
    "# colorless (for aesthetic purposes only). Finally, we'll reduce\n",
    "# the line width of each data point (again for aesthetics only).\n",
    "plt.scatter(petal_length, petal_width, edgecolors=colors, c='none', linewidths=0.5)\n",
    "\n",
    "# Add x and y labels\n",
    "plt.xlabel('Petal Length')\n",
    "plt.ylabel('Petal Width')\n",
    "\n",
    "# Create a few \"proxy artists\" for creating the legend. Calling the \n",
    "# legend() function without passing in any parameters will create a\n",
    "# legend for any labeled data in the graph, so we could do something\n",
    "# funky where we add a label for one line per species, but that \n",
    "# doesn't seem as simple and clean as just creating a few lines that\n",
    "# we use just to create the legend. Plus, this is an officially \n",
    "# sanctioned way to create a legend (just see the legend docs).\n",
    "legend_handles = [\n",
    "    plt.scatter([], [], marker='o', label='setosa', edgecolors=cmap['Iris-setosa'], c='none', linewidths=0.5),\n",
    "    plt.scatter([], [], marker='o', label='versicolor', edgecolors=cmap['Iris-versicolor'], c='none', linewidths=0.5),\n",
    "    plt.scatter([], [], marker='o', label='virginica', edgecolors=cmap['Iris-virginica'], c='none', linewidths=0.5)\n",
    "]\n",
    "\n",
    "plt.legend(handles=legend_handles, scatterpoints=1, loc='upper left');"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the plot above we now see a definitively positive correlation between the petal width and length. In all species, as the length of the petal increases, so too does its width. Also, we can see a better division between the species this time. Setosa, as before, is linearly separable from the other two species, but you now no longer see as much overlap with the versicolor and virginica species as we did when looking at the scatterplot for the sepal dimensions. So, this looks like a better set of features to use if we wanted to create a model for classifying our data.\n",
    "\n",
    "## The Histogram\n",
    "\n",
    "Next up is the [histogram][1], a wonderful tool for visualizing the distribution of numerical data. In the next example we'll use matplotlib's `hist` function to analyze the distribution of one of the features in the Iris data set. For those unfamiliar, a histogram \"bins\" its given data, i.e., it will divide the data up into several intervals, and then count the number of observations in each bin. The `hist` function also takes a parameter called `histtype` that allows us to describe the type of histogram we want to create. Since we want to see how the data for each species is distributed, we'll choose the `'barstacked'` histogram and pass in a list of colors: one color per species. \n",
    "\n",
    "The code below creates three lists to hold the petal length for each species of flower. then we create the corresponding list of colors. Finally, we pass everything into the `hist` function along with the `histtype` parameter and the number of bins we want to divide our data into. Incidentally, the default number of bins in a histogram is 10, but you can always increase the number of bins for a more granular view of your data. Doing so can sometimes surface  interesting insights, so it's generally a good idea to play around with the number of bins a bit to see what effect it has on the distribution of your data.\n",
    "\n",
    "[1]: https://en.wikipedia.org/wiki/Histogram"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Split the data set up into one list per species\n",
    "setosa = [r.petal_length for r in data if r.species == 'Iris-setosa']\n",
    "versicolor = [r.petal_length for r in data if r.species == 'Iris-versicolor']\n",
    "virginica = [r.petal_length for r in data if r.species == 'Iris-virginica']\n",
    "\n",
    "# Create a list of colors (one per species)\n",
    "colors = [cmap['Iris-setosa'], cmap['Iris-versicolor'], cmap['Iris-virginica']]\n",
    "\n",
    "# Plot a stacked histogram\n",
    "plt.hist([setosa, versicolor, virginica], color=colors, histtype='barstacked', bins=20);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the histogram above, we can again see the division we saw earlier between the setosa species and everything else. One thing that's interesting is that the other species look to be somewhat normally distributed around a length of 5, but with the versicolor species falling mainly left of the center, and the virginica species falling to its right.\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "## Parallel Coordinates\n",
    "\n",
    "Matplotlib provides quite a few plotting functions right out of the box, but from time-to-time you're bound to run into one that's missing. One useful visualization technique that is currently absent from the matplotlib library is the parallel coordinates technique. Lucky for us, it's relatively easy to create our own parallel coordinates plotting function, and in this lesson, we'll see how to use what we've learned in the past couple of lessons to create a custom visualization function.\n",
    "\n",
    "\n",
    "\n",
    "The [parallel coordinates][1] technique is a useful tool for understanding multi-dimensional numerical data, and it can be used to easily visualize clusters in these high-dimensional data sets. This technique displays each observation in a data set as a separate line in the graph where each point in the line is the value for a specific feature in the data set. The visualization can also have variable `y-axis` scales for each feature, but this isn't necessarily a requirement, and so we'll be leaving that part out of our plotting function, to make things a bit easier on us.\n",
    "\n",
    "So, now that we have a decent understanding of what we'll be creating, let's go ahead and get our environment setup so we can dive right into the creation of our custom plotting function.\n",
    "\n",
    "[1]: https://eagereyes.org/techniques/parallel-coordinates\n",
    "\n",
    "### Getting the Data\n",
    "\n",
    "In this lesson, as in the last, we'll be using the [Iris data set][1] to test out our plotting function. So, the code below is exactly the same as what we used in the previous lesson to download and load in the data from the [UCI Machine Learning Repository][2].\n",
    "\n",
    "[1]: https://en.wikipedia.org/wiki/Iris_flower_data_set\n",
    "[2]: https://archive.ics.uci.edu/ml/datasets/Iris"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import csv\n",
    "import urllib2\n",
    "from collections import namedtuple\n",
    "\n",
    "# Create a namedtuple constructor for each record in the Iris data\n",
    "#    1. sepal length in cm\n",
    "#    2. sepal width in cm\n",
    "#    3. petal length in cm\n",
    "#    4. petal width in cm\n",
    "#    5. class: \n",
    "#       -- Iris-setosa\n",
    "#       -- Iris-versicolour\n",
    "#       -- Iris-virginica\n",
    "fields = ('sepal_length', \n",
    "          'sepal_width', \n",
    "          'petal_length', \n",
    "          'petal_width', \n",
    "          'species')\n",
    "IrisRecord = namedtuple('IrisRecord', fields)\n",
    "\n",
    "# Download and read in the data from the UCI Machine Learning Repository\n",
    "response = urllib2.urlopen('https://archive.ics.uci.edu/ml/machine-learning-databases/iris/iris.data')\n",
    "iris_data_csv = response.read().strip()\n",
    "\n",
    "# Convert each record into a format that's easier to work with (i.e.,\n",
    "# wrap each record in the namedtuple that we created).\n",
    "data = []\n",
    "for row in csv.reader(iris_data_csv.splitlines()):\n",
    "    data.append(IrisRecord(\n",
    "        sepal_length          = float(row[0]),\n",
    "        sepal_width           = float(row[1]),\n",
    "        petal_length          = float(row[2]),\n",
    "        petal_width           = float(row[3]),\n",
    "        species               = row[4]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Creating a Parallel Coordinates Function\n",
    "\n",
    "And, now that we have our data loaded in, we're ready to create our plotting function.\n",
    "\n",
    "The first two lines in the code below import the `cycle` function from the `itertools` module and the `defaultdict` function from the `collections` module just like we did in our last lesson so we can create a simple color map that we'll use to color code the different classes in our data. The first few lines of our plotting function (lines 8 and 9) use the imported functions to create that color map. The first line calls the `cycle` function on the default list of colors. This will create an iterator that just repeats once the underlying list has been exhausted. On the next line we create the color map, which is just a dictionary that will grab the next color from our color iterator whenever a key is passed in that doesn't already exist in the color map.\n",
    "\n",
    "Notice that our `parallelcoords` function takes two parameters: the first one is the 2-dimensional matrix of data that we want to visualize, and the second is a boolean that tells us whether or not the matrix contains a column for a target variable. \n",
    "\n",
    "> On a side note, a target variable is a variable that contains some type of classification of an observation, and an observation is just another name for a record in a data set. It's called a target variable because it's usually the value that we're trying to predict in a machine learning model. So, for example, in our data, the target variable would be the species of the flower, and so we could create a model (e.g., a decision tree) that would take a bit of data for a flower (like the petal length and width) and then try to determine its species based on that data, i.e., predict its target variable.\n",
    "\n",
    "Ok, that aside, let's get back to our plotting function's interface. \n",
    "\n",
    "On lines 15-21 we see the meat of the plotting function. In this bit of code we loop through each record in our data set and plot each one as a separate line in the graph. But, before doing so, we check if there's a target variable in the data set. If one does exist, we assume it's the last variable in the record, and we plot everything but that last column. Then, we use the last column to pick the color for the current line from our color map. If there is no target variable in the data set, we simply plot all of the data for the current record and let matploltib pick out the color.\n",
    "\n",
    "Everything after line 21 is just cleaning up the graphic to make it a bit easier to read. We start with properly labeling the `x-axis`. To do this, we'll need to get a list of the feature names from the `_fields` attribute of the `namedtuple`s in our data set. Lines\n",
    "\n",
    "If the user has passed in a list of `namedtuples`, we will pull a list of feature names from the `_fields` attribute and use those names to label the `x-axis` ticks. Line 29 tries to get that list and if the data doesn't contain nicely labeled `namedtuple`s, we simply go with a range of numbers for the `x-axis` labels, which is what line 31 is doing. Finally, line 32 takes the feature names list that we pulled from the data set and updates the `x-axis` tick labels with that list.\n",
    "\n",
    "With the long feature names on the `x-axis`, the graph can look a bit cluttered, so we hide the `y-axis` origin label on line 36 to make it look a bit cleaner.\n",
    "\n",
    "Line 39 draws the grid lines for `x-axis`.\n",
    "\n",
    "And, finally, lines 43-55 create a legend. Now, this bit is optional based on whether or not a target variable exists in the data. If a target variable does exist, the first thing we do is pull a list of the unique values from the target variable, which we do on line 45. Then, on line 54, we create a proxy artist (see the previous lesson for an explanation of proxy artists) for each of the classes in the list and give each one a color from the color map we created at the beginning of the function. Lastly, we call the `legend` function and pass in the list of proxy artists to create our legend, and we're done."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from itertools import cycle\n",
    "from collections import defaultdict\n",
    "\n",
    "def parallelcoords(m, target=True):\n",
    "    \"\"\"Plots parallel coordinates from a 2D matrix of numerical data\n",
    "    \"\"\" \n",
    "    # Create a color map of species names to colors\n",
    "    color_cycler = cycle(plt.rcParams['axes.prop_cycle'])\n",
    "    cmap = defaultdict(lambda: next(color_cycler)['color'])\n",
    "        \n",
    "    # Plot the data, one line for each record in the data set.\n",
    "    # Remember calling plot continuously works because pyplot\n",
    "    # keeps track of the current figure and adds a new line\n",
    "    # to the current figure every time you call plot.\n",
    "    for record in m:\n",
    "        # If the data contains a target variable, plot only the features\n",
    "        # and use the target variable to determine the color of the line\n",
    "        if target:\n",
    "            plt.plot(record[:-1], c=cmap[record[-1]])\n",
    "        else:\n",
    "            plt.plot(record)\n",
    "\n",
    "    # Label the x-axis with the name of each feature in the data set.\n",
    "    try:\n",
    "        # If each record is a namedtuple, get the list of fields;\n",
    "        # we'll use those for the x- and y-axis labels of the \n",
    "        # scatterplot matrix. If target is True, don't get the\n",
    "        # last field name.\n",
    "        features = m[0]._fields[:-1] if target else m[0]._fields \n",
    "    except:\n",
    "        features = range(len(m[0]) - 1) if target else range(len(m[0]))\n",
    "    plt.xticks(range(len(features)), features)\n",
    "\n",
    "    # Grab the y-axis from the current Axes object so we can hide the \n",
    "    # origin label since it looks a little crowded otherwise.\n",
    "    plt.gca().yaxis.get_major_ticks()[0].set_visible(False)\n",
    "\n",
    "    # Show the grid lines for the x-axis\n",
    "    plt.grid(axis='x')\n",
    "\n",
    "    # If we have a target variable in the data, create a legend\n",
    "    # with an entry for each class in the target variable.\n",
    "    if target:\n",
    "        # Get the list of classes for the target variable\n",
    "        classes = list(set(r[-1] for r in m))\n",
    "\n",
    "        # Create a few \"proxy artists\" for creating the legend. Calling the \n",
    "        # legend() function without passing in any parameters will create a\n",
    "        # legend for any labeled data in the graph, so we could do something\n",
    "        # funky where we add a label for one line per species, but that \n",
    "        # doesn't seem as simple and clean as just creating a few lines that\n",
    "        # we use just to create the legend. Plus, this is an officially \n",
    "        # sanctioned way to create a legend (just see the legend docs).\n",
    "        legend_handles = [plt.Line2D([], [], color=cmap[cls], label=cls) for cls in classes]\n",
    "        plt.legend(handles=legend_handles);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's go ahead and give our `parallelcoords` function a try with the Iris data we read in from UCI Machine Learning repository. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "parallelcoords(data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Using the parallel coordinates technique it's really easy to see that there are at least two definite clusters: one for the setosa species and another for everything else. However, unlike the techniques we saw in previous lessons, e.g., the scatterplot, we can actually see where the divisions between the clusters are greatest on more than just one or two variables. Looking at our graph above, we can see that a good division just does not exist between any of the species when looking at sepal dimensions, but the petal dimensions look like a really good predictor of species. In fact, we can see that there's even a pretty nice separation between the versicolor and virginica species when considering the petal length and width. So, this visualization technique is really useful when trying to determine which features will be most effective when differentiating between different classes in your data. \n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "## Conclusion\n",
    "\n",
    "And that will bring us to the end of this lesson. Though we didn't have time to cover all of the different visualization types, we did go over quite a few, and we discussed how each can be useful when doing a bit of exploratory analysis on a data set. Hopefully, this short introduction has piqued your interest enough to get you exploring all of the [visualization functions][1] that the matplotlib provides.\n",
    "\n",
    "[1]: http://matplotlib.org/api/pyplot_summary.html"
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python [default]",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}