1_Data_Exploration.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Plagiarism Text Data\n",
    "\n",
    "In this project, you will be tasked with building a plagiarism detector that examines a text file and performs binary classification; labeling that file as either plagiarized or not, depending on how similar the text file is when compared to a provided source text. \n",
    "\n",
    "The first step in working with any dataset is loading the data in and noting what information is included in the dataset. This is an important step in eventually working with this data, and knowing what kinds of features you have to work with as you transform and group the data!\n",
    "\n",
    "So, this notebook is all about exploring the data and noting patterns about the features you are given and the distribution of data. \n",
    "\n",
    "> There are not any exercises or questions in this notebook, it is only meant for exploration. This notebook will note be required in your final project submission.\n",
    "\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Read in the Data\n",
    "\n",
    "The cell below will download the necessary data and extract the files into the folder `data/`.\n",
    "\n",
    "This data is a slightly modified version of a dataset created by Paul Clough (Information Studies) and Mark Stevenson (Computer Science), at the University of Sheffield. You can read all about the data collection and corpus, at [their university webpage](https://ir.shef.ac.uk/cloughie/resources/plagiarism_corpus.html). \n",
    "\n",
    "> **Citation for data**: Clough, P. and Stevenson, M. Developing A Corpus of Plagiarised Short Answers, Language Resources and Evaluation: Special Issue on Plagiarism and Authorship Analysis, In Press. [Download]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "jupyter": {
     "outputs_hidden": true
    }
   },
   "outputs": [],
   "source": [
    "!wget https://s3.amazonaws.com/video.udacity-data.com/topher/2019/January/5c4147f9_data/data.zip\n",
    "!unzip data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "jupyter": {
     "outputs_hidden": true
    }
   },
   "outputs": [],
   "source": [
    "# import libraries\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import os"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This plagiarism dataset is made of multiple text files; each of these files has characteristics that are is summarized in a `.csv` file named `file_information.csv`, which we can read in using `pandas`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "jupyter": {
     "outputs_hidden": true
    }
   },
   "outputs": [],
   "source": [
    "csv_file = 'data/file_information.csv'\n",
    "plagiarism_df = pd.read_csv(csv_file)\n",
    "\n",
    "# print out the first few rows of data info\n",
    "plagiarism_df.head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Types of Plagiarism\n",
    "\n",
    "Each text file is associated with one **Task** (task A-E) and one **Category** of plagiarism, which you can see in the above DataFrame.\n",
    "\n",
    "###  Five task types, A-E\n",
    "\n",
    "Each text file contains an answer to one short question; these questions are labeled as tasks A-E.\n",
    "* Each task, A-E, is about a topic that might be included in the Computer Science curriculum that was created by the authors of this dataset. \n",
    "    * For example, Task A asks the question: \"What is inheritance in object oriented programming?\"\n",
    "\n",
    "### Four categories of plagiarism \n",
    "\n",
    "Each text file has an associated plagiarism label/category:\n",
    "\n",
    "1. `cut`: An answer is plagiarized; it is copy-pasted directly from the relevant Wikipedia source text.\n",
    "2. `light`: An answer is plagiarized; it is based on the Wikipedia source text and includes some copying and paraphrasing.\n",
    "3. `heavy`: An answer is plagiarized; it is based on the Wikipedia source text but expressed using different words and structure. Since this doesn't copy directly from a source text, this will likely be the most challenging kind of plagiarism to detect.\n",
    "4. `non`: An answer is not plagiarized; the Wikipedia source text is not used to create this answer.\n",
    "5. `orig`: This is a specific category for the original, Wikipedia source text. We will use these files only for comparison purposes.\n",
    "\n",
    "> So, out of the submitted files, the only category that does not contain any plagiarism is `non`.\n",
    "\n",
    "In the next cell, print out some statistics about the data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "jupyter": {
     "outputs_hidden": true
    }
   },
   "outputs": [],
   "source": [
    "# print out some stats about the data\n",
    "print('Number of files: ', plagiarism_df.shape[0])  # .shape[0] gives the rows \n",
    "# .unique() gives unique items in a specified column\n",
    "print('Number of unique tasks/question types (A-E): ', (len(plagiarism_df['Task'].unique())))\n",
    "print('Unique plagiarism categories: ', (plagiarism_df['Category'].unique()))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You should see the number of text files in the dataset as well as some characteristics about the `Task` and `Category` columns. **Note that the file count of 100 *includes* the 5 _original_ wikipedia files for tasks A-E.** If you take a look at the files in the `data` directory, you'll notice that the original, source texts start with the filename `orig_` as opposed to `g` for \"group.\" \n",
    "\n",
    "> So, in total there are 100 files, 95 of which are answers (submitted by people) and 5 of which are the original, Wikipedia source texts.\n",
    "\n",
    "Your end goal will be to use this information to classify any given answer text into one of two categories, plagiarized or not-plagiarized."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Distribution of Data\n",
    "\n",
    "Next, let's look at the distribution of data. In this course, we've talked about traits like class imbalance that can inform how you develop an algorithm. So, here, we'll ask: **How evenly is our data distributed among different tasks and plagiarism levels?**\n",
    "\n",
    "Below, you should notice two things:\n",
    "* Our dataset is quite small, especially with respect to examples of varying plagiarism levels.\n",
    "* The data is distributed fairly evenly across task and plagiarism types."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "jupyter": {
     "outputs_hidden": true
    }
   },
   "outputs": [],
   "source": [
    "# Show counts by different tasks and amounts of plagiarism\n",
    "\n",
    "# group and count by task\n",
    "counts_per_task=plagiarism_df.groupby(['Task']).size().reset_index(name=\"Counts\")\n",
    "print(\"\\nTask:\")\n",
    "display(counts_per_task)\n",
    "\n",
    "# group by plagiarism level\n",
    "counts_per_category=plagiarism_df.groupby(['Category']).size().reset_index(name=\"Counts\")\n",
    "print(\"\\nPlagiarism Levels:\")\n",
    "display(counts_per_category)\n",
    "\n",
    "# group by task AND plagiarism level\n",
    "counts_task_and_plagiarism=plagiarism_df.groupby(['Task', 'Category']).size().reset_index(name=\"Counts\")\n",
    "print(\"\\nTask & Plagiarism Level Combos :\")\n",
    "display(counts_task_and_plagiarism)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It may also be helpful to look at this last DataFrame, graphically.\n",
    "\n",
    "Below, you can see that the counts follow a pattern broken down by task. Each task has one source text (original) and the highest number on `non` plagiarized cases."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "jupyter": {
     "outputs_hidden": true
    }
   },
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "% matplotlib inline\n",
    "\n",
    "# counts\n",
    "group = ['Task', 'Category']\n",
    "counts = plagiarism_df.groupby(group).size().reset_index(name=\"Counts\")\n",
    "\n",
    "plt.figure(figsize=(8,5))\n",
    "plt.bar(range(len(counts)), counts['Counts'], color = 'blue')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Up Next\n",
    "\n",
    "This notebook is just about data loading and exploration, and you do not need to include it in your final project submission. \n",
    "\n",
    "In the next few notebooks, you'll use this data to train a complete plagiarism classifier. You'll be tasked with extracting meaningful features from the text data, reading in answers to different tasks and comparing them to the original Wikipedia source text. You'll engineer similarity features that will help identify cases of plagiarism. Then, you'll use these features to train and deploy a classification model in a SageMaker notebook instance. "
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "conda_amazonei_mxnet_p36",
   "language": "python",
   "name": "conda_amazonei_mxnet_p36"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}