course_notes.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "e31ca78f-e0bb-442d-80d4-0c5629a37671",
   "metadata": {},
   "source": [
    "# Statistics for life science"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f24e053f-490d-46be-863c-a5ad2460f3c3",
   "metadata": {},
   "source": [
    "Based on the courses. Made by me."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6562c20b-c07b-41c2-9e52-4d2603c0b97a",
   "metadata": {},
   "source": [
    "## Introduction"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bfc4a652-880e-442b-8c2e-3d2a758a708f",
   "metadata": {},
   "source": [
    "Statistics is the study of the collection, summarizing, analyzing, interpreting, and presenting of data."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "09cf6f76-782c-42c4-8405-fd1f185f304f",
   "metadata": {},
   "source": [
    "## Lecture 1"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "19c45f3f-8cea-4d8a-946d-e88e637caa84",
   "metadata": {},
   "source": [
    "### Population and sample"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "27156c4c-0488-4691-8be3-e3a85a7ab5ff",
   "metadata": {},
   "source": [
    "**Population** = all things of interest in our study / a group of subjects that have something in common\n",
    "\n",
    "**Sample** = subset of those things\n",
    "\n",
    "Sample statistics are *estimators* of population parameters.\n",
    "\n",
    "Sampling brings *uncertainty* - by chance the properties of the sample are not the same as in the population.\n",
    "\n",
    "Statistical tests are used to quantify the *uncertainty* of the estimated parameters, i.e determine the likely magnitude of departure from truth.\n",
    "\n",
    "We use statistics to determine if the observed difference is a true difference (**significant difference**) or just due to chance. We can never be 100% sure that the difference is true."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "abbec8db-559f-4e18-8c65-75422d947d07",
   "metadata": {},
   "source": [
    "**Some symbols**\n",
    "\n",
    "|Sample statistic|Population parameter|Represents|\n",
    "|----------------|--------------------|----------|\n",
    "|$\\bar{x}$ (x bar)|$\\mu$ (lowercase mu)|Mean|\n",
    "|p|$\\pi$ (lowercase pi)|Proportion|\n",
    "|SD or s|$\\sigma$ (lowercase sigma)|Standard deviation|\n",
    "|$s^2$|$\\sigma^2$|Variance|"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6b42366f-5d17-4a0f-9284-c7470cfa7e1f",
   "metadata": {},
   "source": [
    "**Parameter** - a quantity describing a population\n",
    "\n",
    "**Statistic** - a representative quantity calculated from the sample\n",
    "\n",
    "A sample should be:\n",
    "\n",
    "- random (every member of a population must have the same chance of being selected for the sample)\n",
    "\n",
    "- large enough (**n** = sample size)\n",
    "\n",
    "- representative = must reflect the entire population"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "52ec067b",
   "metadata": {},
   "source": [
    "### Scales of measurement\n",
    "\n",
    "Qualitative - categorical\n",
    "\n",
    "Quantitative - numerical\n",
    "\n",
    "**Discrete** variables (only whole numbers, no values can exist between values) \n",
    "\n",
    "**Continuous** variables (floating points, an infinite number of values exist between each value)\n",
    "\n",
    "Categorical data: \n",
    "\n",
    "**Nominal** (not ordered)  \n",
    "\n",
    "**Ordnial** (can be ordered).\n",
    "\n",
    "Numerical data: \n",
    "\n",
    "**Inteval** (zero has no real meaning, zero does not indicate total absence of the attribute) \n",
    "\n",
    "**Ratio** (has a true zero, meaning total absence. One can say: this has the double value of that)\n",
    "\n",
    "A continous variable can be categorized into an ordinal scale: low/medium/high\n",
    "\n",
    "A *variable* is something that can be measured or counted.\n",
    "\n",
    "A *parameter* is a value that represents a characteristic of a population: ex mean or proportion."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "827f652e",
   "metadata": {},
   "source": [
    "### Types of statistics\n",
    "\n",
    "**Inference** - inferential statistics - draw inferences about population from sample.\n",
    "\n",
    "**Description** - descriptive statistics - summarizing and describing the data.\n",
    "\n",
    "Descriptive statistics: central tendency (mean, median, mode) - spread of data (range, IQR, SD, variance)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2a199046-0f97-4751-b9b1-db8e3ba9fa60",
   "metadata": {},
   "source": [
    "### Central tendency"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9604cbee-e7a3-4cf6-9f75-4281fb1b57e4",
   "metadata": {},
   "source": [
    "Measures of central tendency: the central position of the data."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1ca1032d-dbe2-47ae-a83e-2f6ee6cb3aee",
   "metadata": {
    "tags": []
   },
   "source": [
    "#### Mean\n",
    "\n",
    "Aka *average* or *arithmetic mean*.\n",
    "\n",
    "$\\bar{x}=\\frac{\\text{Sum of all sample values}}{\\text{Size of sample}}=\\frac{\\sum_{i=1}^{n}x_i}{n}$\n",
    "\n",
    "Used when the data is *symmetric*. Symmetric distribution of data means the mean and the median are very close. A set of data is symmetric if the mean, median, and mode all occur at the same number. When graphed, the two sides of the graph will be almost mirror images of one another.\n",
    "\n",
    "<img src=\"symmetric_and_skewed_data.png\" alt=\"pic failed\" width=\"500\" height=\"300\">\n",
    "\n",
    "The sum symbol (uppercase sigma) $\\sum_{i=1}^{n}$ means: sum every number in the set, starting from i=1 and go all the way to 1=n, in other words, to the last number in the set."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "187bf786-c7a3-4083-8f83-c32275a73055",
   "metadata": {
    "vscode": {
     "languageId": "r"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "5.14285714285714"
      ],
      "text/latex": [
       "5.14285714285714"
      ],
      "text/markdown": [
       "5.14285714285714"
      ],
      "text/plain": [
       "[1] 5.142857"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Code example of mean.\n",
    "\n",
    "example_dataset<- c(4,6,8,7,5,3,3)\n",
    "n<- length(example_dataset)\n",
    "\n",
    "dataset_mean<- sum(example_dataset)/n\n",
    "\n",
    "dataset_mean"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "94599981-2d5d-4778-b0db-50959a2963b7",
   "metadata": {},
   "source": [
    "#### Median\n",
    "\n",
    "The *middle value* of the dataset. Used when distribution is *skewed*, meaning the mean and the median are far apart. \n",
    "\n",
    "*Right skewed*: The mean is greater than the median. The mean overestimates the most common values in a positively skewed distribution. The mean is pulled to the right by extreme high values.\n",
    "\n",
    "*Left skewed*: The mean is less than the median. The mean underestimates the most common values in a negatively skewed distribution. The mean is pulled to the left by extreme low values\n",
    "\n",
    "<img src=\"right_left_skew.png\" alt=\"pic failed\" width=\"500\" height=\"300\">\n",
    "\n",
    "Algorithm for median:\n",
    "1. sort the data\n",
    "2. find the middle value\n",
    "3. if dataset is odd, median is the middle number\n",
    "4. if dataset is even, median is the mean of the two middle nrs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "fb74f39d-acbd-460a-bc8a-01eaade6e004",
   "metadata": {
    "vscode": {
     "languageId": "r"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "4.5"
      ],
      "text/latex": [
       "4.5"
      ],
      "text/markdown": [
       "4.5"
      ],
      "text/plain": [
       "[1] 4.5"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Code example of median.\n",
    "\n",
    "# Modify this dataset to try even and odd datasets.\n",
    "example_dataset<- c(4,6,8,5,3,3)\n",
    "n<- length(example_dataset)\n",
    "\n",
    "sorted_dataset<- sort(example_dataset)\n",
    "\n",
    "# Finding the median.\n",
    "if (n %% 2 == 1) {\n",
    "  # If n is not evenly divisible by 2, meaning it is odd.\n",
    "  # The middle number is selected.\n",
    "  median <- sorted_dataset[n %/% 2 + 1]\n",
    "  # %/% is integer division or floor division,\n",
    "  # the number is divided and rounded down to the closest int\n",
    "} else {\n",
    "  # If n is evenly divisible by 2, meaning it is even.\n",
    "  # The mean of the two middle values is selected.\n",
    "  median <- mean(sorted_dataset[c(n %/% 2, n %/% 2 + 1)])\n",
    "}\n",
    "  \n",
    "median"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "59285f02-aba5-478c-badc-12f668e08b99",
   "metadata": {},
   "source": [
    "#### Mode\n",
    "\n",
    "The value that appears most often in the dataset. \n",
    "\n",
    "There can be several modes, if many values appear equally often. There is no mode when all values appear the same number of times in a data set.\n",
    "\n",
    "Mainly used with *categorical* data. Example: favourite color."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "ed286b6d-a71c-492f-9db3-53780f0e06d7",
   "metadata": {
    "vscode": {
     "languageId": "r"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "example_dataset\n",
       " 1  2  3  6  7 14 \n",
       " 1  1  1  4  2  1 "
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Example of mode.\n",
    "\n",
    "example_dataset<- c(1,6,2,3,6,6,7,6,7,14)\n",
    "\n",
    "# Create a frequency table.\n",
    "freq_table <- table(example_dataset)\n",
    "freq_table\n",
    "# Upper row: the numbers found in the dataset.\n",
    "# Lower row: how often they appear in the dataset."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e0023293-05b4-45c5-a426-c2d3e283920d",
   "metadata": {},
   "source": [
    "### The spread of the data\n",
    "\n",
    "*Median* is used with *IQR*.\n",
    "\n",
    "*Mean* is used with *SD*."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "12116aeb-f053-40ca-abfb-4ce3e5d2c8a3",
   "metadata": {},
   "source": [
    "#### Range\n",
    "\n",
    "The difference between the largest and smallest values in a dataset. \n",
    "\n",
    "$\\text{Range}=\\text{value}_{max}-\\text{value}_{min}$\n",
    "\n",
    "It is a measure of dispersion or spread, providing an indication of how much the data values vary.\n",
    "\n",
    "<img src=\"range.png\" alt=\"pic failed\" width=\"200\" height=\"200\">"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e9517830-d683-4813-9194-5758176c50e3",
   "metadata": {},
   "source": [
    "#### Interquartile range - IQR\n",
    "\n",
    "*Quartiles* are **values** that divide ordered numbers into quarters.\n",
    "\n",
    "<img src=\"IQR.png\" alt=\"pic failed\" width=\"250\" height=\"250\">\n",
    "\n",
    "Q2=median of the data\n",
    "\n",
    "Q1=median of the lower half of the data\n",
    "\n",
    "Q3=median of the upper half of the data\n",
    "\n",
    "IQR = the difference between the third quartile and the first quartile.\n",
    "\n",
    "$\\text{IQR}=\\text{Q3}-\\text{Q1}$\n",
    "\n",
    "IQR = the *range of the middle 50%* of the data.\n",
    "\n",
    "Can also be written as an interval: [3.5\t8.5] –> 50% of the data points are located within this interval.\n",
    "\n",
    "There are several different methods to compute the quartiles. Therefore, different software tools may report different values.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c3139765-e6ce-464b-ad76-e1450beb5738",
   "metadata": {},
   "source": [
    "#### Box plot\n",
    "\n",
    "Aka **box and whisker** plot.\n",
    "\n",
    "Graphical representation of the *median, quartiles and range*. May also show *outliers*.\n",
    "\n",
    "<img src=\"box_plot.png\" alt=\"pic failed\" width=\"300\" height=\"300\">\n",
    "\n",
    "The upper whisker shows the **max value** of the data (excluding outliers?)\n",
    "\n",
    "The lower whisker shows the **min value** of the data (excluding outliers?)\n",
    "\n",
    "It follows that the **range** is between the upper and the lower whiskers.\n",
    "\n",
    "The middle bar in the box shows the **median**, the upper box line the **Q3** and the lower **Q1**. \n",
    "\n",
    "The box itself shows the **IQR**, the range of the central 50% of the data."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fbe05b27-857f-4a1a-8bc2-45cb2bcd4866",
   "metadata": {},
   "source": [
    "**Quantiles and percentiles**\n",
    "\n",
    "Q3 = 0.75 quantile = 75th percentile = 75% of data lies below this\n",
    "\n",
    "Q2 = 0.5 quantile = 50th percentile = 50% of data lies below this\n",
    "\n",
    "Q1 = 0.25 quantile = 25th percentile = 25% of data lies below this\n",
    "\n",
    "You can also say for example the 0.875 quantile.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "01466e37-0495-4b5e-88f2-253545150143",
   "metadata": {},
   "source": [
    "#### Standard deviation\n",
    "\n",
    "And **variance**.\n",
    "\n",
    "Lets remember the symbols:\n",
    "\n",
    "|Sample statistic|Population parameter|Represents|\n",
    "|----------------|--------------------|----------|\n",
    "|$\\bar{x}$ (x bar)|$\\mu$ (lowercase mu)|Mean|\n",
    "|SD or s|$\\sigma$ (lowercase sigma)|Standard deviation|\n",
    "|$s^2$|$\\sigma^2$|Variance|\n",
    "\n",
    "The standard deviation is the square root of the sum of all the squared differences between each datapoint in the set and the sample mean divided by sample size minus one:\n",
    "\n",
    "$\\text{SD} = \\sqrt{\\frac{\\sum_{i=1}^{n}(x_i - \\bar{x})^2}{n-1}}$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "597a14b1-8163-4d65-b6ed-daeefbb646a4",
   "metadata": {
    "vscode": {
     "languageId": "r"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "1.95180014589707"
      ],
      "text/latex": [
       "1.95180014589707"
      ],
      "text/markdown": [
       "1.95180014589707"
      ],
      "text/plain": [
       "[1] 1.9518"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Code example of standard deviation.\n",
    "\n",
    "# The data points here are x_1, x_2, x_3 and so on.\n",
    "example_dataset<- c(4,6,8,7,5,3,3)\n",
    "n<- length(example_dataset)\n",
    "dataset_mean<- mean(example_dataset)\n",
    "\n",
    "# First, all the differences between the data points \n",
    "# and the mean are calculated. (yay vectorization)\n",
    "differences<- example_dataset - dataset_mean\n",
    "\n",
    "# Then they are all squared. Explanation why below in text.\n",
    "squared_differences<- differences^2\n",
    "\n",
    "# Next, they are all added together.\n",
    "summed_squared_diffs<- sum(squared_differences)\n",
    "\n",
    "# Then divided by n-1. Explanation why below in text.\n",
    "result<- summed_squared_diffs/(n-1)\n",
    "\n",
    "# Finally, the square root is taken. Explanation why below in text.\n",
    "SD<- sqrt(result)\n",
    "SD"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "08ce3bae",
   "metadata": {},
   "source": [
    "##### Some extra material"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "772264b8-18e5-4eb2-b7bf-9222e0725479",
   "metadata": {},
   "source": [
    "**n-1 and degrees of freedom**\n",
    "\n",
    "The subtraction of 1 from the sample size (n-1) is known as Bessel's correction and is used when estimating the SD from a sample rather than the entire population.\n",
    "\n",
    "When calculating the SD from a sample, the goal is to *estimate* the population standard deviation based on a smaller subset of data. Since the sample is a smaller representation of the population, using the sample SD formula with n would tend to underestimate the true population standard deviation.\n",
    "\n",
    "Statisticians and researchers have extensively studied and validated Bessel's correction through mathematical analysis, simulation studies, and empirical observations. They have shown that using 'n-1' in the denominator provides a more accurate and unbiased estimate of the population standard deviation, particularly when working with small samples.\n",
    "\n",
    "n-1 = the degrees of freedom of the sample.\n",
    "\n",
    "The subtraction of 1 accounts for the fact that when estimating the SD from a sample, one degree of freedom is lost due to the calculation of the sample mean.\n",
    "\n",
    "Degrees of freedom refer to the number of values in a sample that are free to vary or be chosen independently. It represents the number of observations in the sample that are available to provide information or contribute to the estimation of parameters.\n",
    "\n",
    "When calculating the standard deviation from a sample, one of the steps involves calculating the sample mean, which represents the average value of the sample. The calculation of the sample mean involves summing up all the data points and dividing by the sample size.\n",
    "\n",
    "However, the calculation of the sample mean introduces a constraint or restriction on the data points. Once the sample mean is calculated, the values in the sample are no longer entirely free to vary independently. The sample mean imposes a restriction on the data because it is determined by the values in the sample.\n",
    "\n",
    "Due to this restriction, one degree of freedom is lost in the sample.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b782a48d-8186-4f9d-9dce-3b1e2139545a",
   "metadata": {},
   "source": [
    "**Further down the rabbit hole**\n",
    "\n",
    "When calculating the sample mean, you sum up all the data points in the sample and divide by the sample size. \n",
    "\n",
    "$\\bar{x}=\\frac{\\sum_{i=1}^{n}x_i}{n}$\n",
    "\n",
    "The resulting sample mean represents a central tendency of the data. It serves as an estimate of the population mean based on the sample.\n",
    "\n",
    "The sample mean imposes a constraint because it depends on the individual values within the sample. Once you calculate the sample mean, the values in the sample are no longer entirely free to vary independently because they must satisfy the constraint imposed by the calculated mean.\n",
    "\n",
    "Suppose you have a sample of three numbers: \n",
    "\n",
    "5, 8, 10 \n",
    "\n",
    "The mean of this sample is \n",
    "\n",
    "$\\frac{{(5 + 8 + 10)}}{{3}} = 7.67$\n",
    "\n",
    "If you change any of the values in the sample, the mean will also change accordingly. For example, if you change the last value from 10 to 12, the mean becomes \n",
    "\n",
    "$\\frac{{(5 + 8 + 12)}}{{3}} = 8.33$\n",
    "\n",
    "The sample mean imposes a restriction on the possible values within the sample because it is influenced by and dependent on the individual values.\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ce024d4b-e778-44b9-bd10-8791bcf8cd55",
   "metadata": {},
   "source": [
    "**But why is it -1 specifically?**\n",
    "\n",
    "Each data point in your set can vary freely, except for the last one, which is determined by the calculated mean. So you need to adjust for this loss of one degree of freedom.\n",
    "\n",
    "Example from before:\n",
    "\n",
    "The mean of a sample:\n",
    "\n",
    "$\\frac{{(5 + 8 + 10)}}{{3}} = 7.67$\n",
    "\n",
    "Two of the nrs are free to vary, but then the last number needs to be whatever is required to create the same mean.\n",
    "\n",
    "Example:\n",
    "\n",
    "$\\frac{{(2 + 19 + x)}}{{3}} = 7.67$\n",
    "\n",
    "solve for x:\n",
    "\n",
    "$x=7.67\\cdot3-21$\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "8b2c79e4-4f82-47b2-be9e-a7e90d6d4c3e",
   "metadata": {
    "vscode": {
     "languageId": "r"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "2.01"
      ],
      "text/latex": [
       "2.01"
      ],
      "text/markdown": [
       "2.01"
      ],
      "text/plain": [
       "[1] 2.01"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "x=7.67*3-21\n",
    "x"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cb662452-3353-4e37-9b9f-faecef2aa167",
   "metadata": {},
   "source": [
    "X must be this value. It is not free to vary. That is why the degrees of freedom are n-1."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "70c1cbde-911b-4d5d-822a-cbde6d44a4b0",
   "metadata": {},
   "source": [
    "There is some confusion here remaining, regarding negative numbers. For example:\n",
    "\n",
    "$\\frac{{(200 + 190 + x)}}{{3}} = 7.67$\n",
    "\n",
    "solve for x:\n",
    "\n",
    "$x=7.67\\cdot3-390$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "e5534b52-47e8-435c-9472-eb013ddfef14",
   "metadata": {
    "vscode": {
     "languageId": "r"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "-366.99"
      ],
      "text/latex": [
       "-366.99"
      ],
      "text/markdown": [
       "-366.99"
      ],
      "text/plain": [
       "[1] -366.99"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "x=7.67*3-390\n",
    "x"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3f39d2ef-0c76-40cb-a2ef-d1162d9b3136",
   "metadata": {},
   "source": [
    "How can there be a negative number? If we are measuring for example length?"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9d9f3eb6-cb81-49d2-87b6-596c4cf3c328",
   "metadata": {},
   "source": [
    "**Why are the differences squared?**\n",
    "\n",
    "To eliminate the cancellation of positive and negative deviations: When calculating the deviation of each data point from the mean, some deviations will be positive and some will be negative. If you were to simply sum up the deviations without squaring them, the positive and negative deviations would cancel each other out. This would result in a net deviation of zero, which does not reflect the actual spread or variability in the data. By squaring the differences, all deviations become positive values, preserving the magnitude of the deviations and ensuring that they contribute to the measure of variability.\n",
    "\n",
    "To emphasize larger deviations: Squaring the differences magnifies the impact of larger deviations from the mean. This is important because larger deviations are generally considered to have a greater influence on the overall variability of the dataset. By squaring the differences, you give more weight to these larger deviations, providing a more meaningful representation of the data spread and allowing the SD to be more sensitive to extreme values."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "57a7edca-fe74-4048-9c6d-fad6e52c4e8c",
   "metadata": {},
   "source": [
    "**Why is the square root taken?** \n",
    "\n",
    "The main reason for taking the square root is to ensure that the standard deviation is expressed in the **original units of the data**, making it more interpretable and comparable to the original data values. Without taking the square root, the result would be in squared units, which may not be as meaningful or easily interpretable.\n",
    "\n",
    "**And why is the square root taken after division by n-1 and not right after squaring?**\n",
    "\n",
    "We might have to get back to this later."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6e8cc2b6-5981-4e34-970e-fb8988e3696f",
   "metadata": {},
   "source": [
    "#### Showing standard deviation\n",
    "\n",
    "Usually presented with the mean like this: **4±2**\n",
    "\n",
    "->One standard deviation away from the mean in both directions: [2 6]\n",
    "\n",
    "Usually represented graphically in *bar charts*.\n",
    "\n",
    "<img src=\"SD.png\" alt=\"pic failed\" width=\"300\" height=\"300\">\n",
    "\n",
    "- the height of the bar = the mean\n",
    "\n",
    "- the “error bar” = one standard deviation (in each direction)\n",
    "\n",
    "About 68% of the data points can be found within ±1 SD. Denoted by the error bars.\n",
    "\n",
    "About 95% of the data points can be found within ±2 SD (1.96 to be precise).\n",
    "\n",
    "- given normal distribution\n",
    "\n",
    "**The SD = the average distance of a datapoint to the mean in a given set.**\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c72f675c-8b76-423c-9af3-dec9284ab4e5",
   "metadata": {},
   "source": [
    "The sample **variance** is related to the standard deviation: \n",
    "\n",
    "$\\text{var}=(\\text{SD})^2$\n",
    "\n",
    "- no simple interpretation. Mainly used in theoretical statistics\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f4a69bcf-123b-40e7-bc79-a8bdeb18be3b",
   "metadata": {},
   "source": [
    "*Median* is used with *IQR*.\n",
    "\n",
    "*Mean* is used with *SD*."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "43fc5d26-0f9d-4240-a198-309f4ff953af",
   "metadata": {},
   "source": [
    "### The normal distributiuon"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c016649b-66a2-4604-91f8-dd647cd42857",
   "metadata": {},
   "source": [
    "aka *Gaussian distribution* or *Z-distribution* – the most common distribution used in statistics.\n",
    "\n",
    "Is probable when several factors each contribute a little to the value of a variable.\n",
    "\n",
    "Has been found to be true for a huge number of variables.\n",
    "\n",
    "Used for **parametric** tests. (parametric tests are developed assuming normal dist)\n",
    "\n",
    "Examples of other distributions: poisson distribution, uniform distribution, bernoulli distribution: [more](https://datasciencedojo.com/blog/types-of-statistical-distributions-in-ml/)\n",
    "\n",
    "**Frequency distributions** describes the nr of times each value or variable appears in a sample.\n",
    "\n",
    "**Probability distribution**: the distribution of a variable in the whole population. (A probability distribution is a mathematical function or model that describes the likelihood of occurrence of each possible outcome in a random experiment or process. It assigns probabilities to each outcome, and these probabilities must sum up to 1.)\n",
    "\n",
    "**Histograms** show distribution of data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "faa73451-3222-4e55-884b-39f2d5173350",
   "metadata": {
    "vscode": {
     "languageId": "r"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA0gAAANICAIAAAByhViMAAAACXBIWXMAABJ0AAASdAHeZh94AAAgAElEQVR4nOzdfXyU1YE3/CuEag2+NjZkMJAGFWFR4ktthdQUym0Jle0LENc+bDH3VlpIdRf9ILqI3ba6pltp7V3fWrTaIFKssbas3U0A99PuQm1kxVYhakNEKxpAE8pbCISQ5488N+sDKSZMZiZz8v3+NXO4zswvrUd+nmvmJKOjoyMCACD9DUh1AAAAeodiBwAQCMUOACAQih0AQCAUOwCAQCh2AACBUOwAAAKh2AEABEKxAwAIhGKXHsaOHZvxfw0YMGDYsGETJ06sqanpztwXXnihpqbmj3/84/G99Z/+9Kfa2trjmwt9SvLX0e7du//+7//+7LPPPuWUUz760Y9WVlb6ZT+ku+Svo23btpWVlQ0ePPiUU0659NJLq6qqjit4v9FBOrjsssuiKMrIyBg4cGBmZmbn/3cZGRn33Xff+879/Oc/H0XRLbfcchzv29LSMmrUqIEDBx7HXOhrkr+OJk+eHEXRiSeeWFBQ0Pl23/ve9443PvQJSV5HBw4c+Ku/+qsoigYNGpSTk9P5ditWrIjjJwicHbt08tWvfrWtra2tre2VV1755Cc/2dHR8fWvf33Pnj2JeK+1a9c++OCDY8eOffnllxPx+pAqSVtHGzZs+Pd///coip599tnXXnvtG9/4RhRF3/rWt3r9jSD5kraOnnnmmbq6uiFDhrzxxhtvv/32Zz7zmSiKfvKTn/T6GwVDsUs/GRkZ55133hNPPHHCCSc0NTUtXbo0iqJXX311+vTpQ4YMOemkk84999yvf/3rBw8ejKJo7Nixv/jFL6Io+va3v3355Zcf48ojTJky5Stf+cof/vCH5P5wkCRJWEed/1E0fPjwiy66KIqiq6++OoqiP//5z2+99VZyf1ZIlCSso3379p1//vnTp0/Pzs7OzMwcM2ZMFEVd/rXF/yeV24V0W+fW9+zZs987OH78+CiKrr/++r179+bn50dRlJWVdc4552RkZERR9K1vfaujo+ORRx4ZNWpUFEWXX355ZWXlMa48wq9//evVq1fffffdURS5FUsYkryOXn/99V/84hdr1qzpfPriiy92/lt3x44dyfl5IRGS//dRp/b29mefffYjH/lIRkbGE088kfCfM23ZsUtjZ511VhRFDQ0Nzz33XFNT01lnnbVt27b6+vr58+dHUfTrX/86iqKysrLzzjsviqKioqKZM2ce48ojfPKTn5w4ceKll16axB8IUiBx6yg/P/9zn/tcUVFRFEXt7e0LFy6MouiKK644/fTTk/kDQhIk9O+jTjfccMPYsWNff/31L37xi9OnT0/Kj5WWBqY6AL1g/Pjxu3fv3rZt2+rVqzds2NC5Gd7S0hLPldDfJG4d7d69+0tf+tKKFSuGDBny4x//OHE/AqRc4tbR+PHjm5qali9fvmzZsoKCgjvuuCNxP0VaU+zS2Ntvvx1F0dlnn33w4MGysrJly5Z1dHQMHjz4zDPP/Esf4un+ldBPJHodbdq06bOf/ezLL788ZsyYp59+eujQoYn6SSB1kvD30Re+8IUvfOELkyZNmjlz5gMPPPCtb31rwAB3Hbvgf5R09e677/72t7+NomjUqFFLly597LHHcnNz169fv3Xr1r/5m7/5S7O6fyX0B4leR+vXr//4xz/+8ssvT58+/be//a1WR5ASuo6eeOKJL3zhC9/85jc7n44dOzaKoubmZrsSf4lil5Zee+21q6++ev/+/dnZ2X/7t39bX18fRVEsFrvooosOHDjw85///Ogpnfvb3bkS+olEr6O2trbp06c3NzdfeeWVP/vZzwYNGpTInwZSIwl/H/3iF7/49re//fvf/76lpeX73/9+FEUnn3xy56f6OJpbsenkwQcf/MlPfnLo0KEDBw50jnzzm9/sPNE+iqL169fn5eUdPHhw586dURTt3r2785ozzzwziqIlS5Z88IMf7Pw201+6EvqDpK2jmpqazZs3R1G0cuXKrKysw+ObNm3ydxLpLmnr6HOf+9zo0aM3btx40UUXnXjiifv374+i6J/+6Z/ch/2LUvmVXLqtcwF0ysjIyMvL+9SnPlVdXX34goULF+bk5AwZMuRrX/vaz372s84r//M//7Ojo2Pjxo1jxoz54Ac/OG3atGNfebQ1a9ZEjjshFEleRxUVFV3+W3fLli1J/sGhFyX/76OtW7f+7//9v4cNG3byySdfcskllZWVhw4dSuaPnF4yOvziQgCAINjJBAAIhGIHABAIxQ4AIBCKHQBAIBQ7AIBAKHYAAIFQ7AAAAqHYAQAEQrEDAAiEYgcAEAjFDgAgEIodAEAgFDsAgEAodgAAgVDsAAACodgBAARCsQMACIRiBwAQCMUOACAQih0AQCAUOwCAQCh2AACBUOwAAAKh2AEABEKxAwAIhGIHABAIxQ4AIBCKHQBAIBQ7AIBAKHYAAIFQ7AAAAqHYAQAEYmDS3qm1tbWqqmrDhg1bt25ta2uLxWKFhYXTpk3LyspKWgYAgIBldHR0JOFt1q1bN3ny5MGDB48bNy47OzuKoubm5tra2rfeemvlypUXX3xxEjIAAIQtScXu0ksvnTFjxty5c48Yr6ysvP/++2tra5OQAQAgbEkqdqeffvrGjRvPOuusI8ZbWlqGDh3a1NSUhAz93Gc+85nf/e533b/+0ksvrampSVweeK+vf/3r995773FMPPHEE2tra4cNG9brkaCfaGtrGzly5I4dO45j7uTJkx977LFej0Q8kvQZu+Li4vnz59999905OTmHB5uamm677baioqLkZOjn/vjHP46Y+LmzL728Oxdvfv63r9auTnQkOOy111778JgPjysb16NZbfvaHit/7J133lHs4Li1tra+9tpr3/7KdcOHHLn5cmwr1v7nq/X1CUrFcUtSsVu8eHFZWVksFhs+fHh2dnZGRkZzc3NDQ8OECROWL1+enAzk/dWFF1zx2e5c2dba8rZiR3Jl52df+LkLezRl/579j5XbLYBe8L8++rFLRozq0ZTXtza++vy7CcrDcUtSscvNza2urt68eXNdXV1jY2MURTk5OYWFhfn5+ckJAAAQvKQed7J27dr3Hneyc+dOx50AAPSWJB1QvG7dury8vIqKiqamptzc3KFDh+7atWvRokXDhg1bv359cjIAAIQtSTt25eXlCxcu7PK4kzlz5rzvcSdr167dt2/fEYMdHR179+79/Oc/35tB08eBAwfWrl3b3t7ezeuP/h8QAAhMkopdfX19aWnp0eOlpaU33njjsec2NjZeccUVXfaSjIyMQ4cO9U7EdPPUU09dffXVqU4BAPQhaXDcSSwWa2lpOXp89erVkydP7uWg6aOtre20wUNu+fc/dPP6Wz8WS2ge6LRr167ubyQfduDAgeikRMQB6F8cdwL0mhdffPHCCy88vmPPx84c2+t5APobx50AvWbXrl0dHR23rrt1QGbPvph1z5R7EhQJoF9J3nEny5cv/+UvfxmLxa666qrLLrusc/Cdd94pLy9/4oknkhYDSLTs/OwBA3tW7DI/kJmgMAD9SpKOO7nrrrvmzJmTl5e3c+fOT3/6008//XTneEtLS1VVVXIyAACELUk7dvfee29VVdXEiROjKLrqqqumT5/+wgsvnHPOOcl5dwCA/iBJO3Y7d+4cOXJk5+NJkybNmjWrvLz8+D5hDQBAl5JU7C699NI777zz4MGDnU/vuOOON998c/78+YdHAACIU5KK3QMPPLBixYrs7OxHHnkkiqKsrKwVK1Y89dRTH//4x5MTAAAgeEn6jN0555zT0NBQW1ubm5vbOXLuuefW1dU9/fTTL774YnIyAACELXnHnZxwwgmXX375ESNTp06dOnVq0jIAAAQsSbdiAQBINMUOACAQih0AQCAUOwCAQCh2AACBUOwAAAKh2AEABEKxAwAIhGIHABAIxQ4AIBCKHQBAIBQ7AIBAKHYAAIFQ7AAAAqHYAQAEQrEDAAiEYgcAEAjFDgAgEIodAEAgFDsAgEAodgAAgVDsAAACodgBAARCsQMACIRiBwAQCMUOACAQih0AQCAUOwCAQCh2AACBUOwAAAKh2AEABEKxAwAIhGIHABAIxQ4AIBCKHQBAIBQ7AIBAKHYAAIFQ7AAAAqHYAQAEQrEDAAiEYgcAEAjFDgAgEIodAEAgFDsAgEAodgAAgVDsAAACodgBAARCsQMACIRiBwAQCMUOACAQih0AQCAUOwCAQCh2AACBUOwAAAIxMNUBACCpWltbq6qqNmzYsHXr1ra2tlgsVlhYOG3atKysrFRHg3jZsQOgH1m3bl1eXl5FRUVTU1Nubu7QoUN37dq1aNGiYcOGrV+/PtXpIF527ADoR8rLyxcuXDh37twjxisrK+fMmVNbW5uSVNBbFDtIJ24hQZzq6+tLS0uPHi8tLb3xxhuTnwd6l1uxkDbcQoL4FRcXz58/f/v27e8dbGpqmjdvXlFRUapSQW+xYwdpwy0kiN/ixYvLyspisdjw4cOzs7MzMjKam5sbGhomTJiwfPnyVKeDeCl2kDbcQoL45ebmVldXb968ua6urrGxMYqinJycwsLC/Pz8VEeDXuBWLKQNt5AgfjU1NXv37i0oKLjyyisLCgrWrl17//33f+c733nppZdSHQ16gWIHaWPx4sVNTU2xWOzcc8+97LLLxo4de9555w0ePLi+vv6RRx5JdTpIDyUlJVu2bImi6OGHH54yZUpmZubEiRPb29uLioqqq6tTnQ7i5VYspA23kKAXVVRULFmy5PDHG0pKShYsWFBSUpLaVBAnxQ7SSWtr69q1a9973MnOnTsddwLHobGxcfz48Yefjhs3btOmTe87JT8/v62t7eg/yszMPHjwYO8mhOOg2EHaWLdu3eTJkwcPHjxu3Ljc3NwoipqbmxctWnTDDTesXLny4osvTnVASA+rV68+ePBgcXHxk08+OXv27M7BpUuXjho16tgTY7HY888/f+DAgSPGn3vuueuvvz4hWaGHFDtIG/Ecd9LW1vbDH/6wpaXliPEDBw40NTV9//vf7+Ws0FfNnDlz2bJlt99++7Zt21auXDlr1qzMzMzOwZqamvedfsEFFxw9uGPHjoyMjASEhR5T7CBtxHPcyc6dOx9//PHW1tYjxnfv3r1p0ybFjv6jsrKy88HevXsbGhoyMzOjKJo0adLNN988evTolEaDXqDYQdroPO7k7rvvzsnJOTzY1NR02223ve9xJ2eeeeaaNWuOHl+9evXkyZN7OSj0bcuXL//lL38Zi8WuuuqqzpEZM2a88847paWlTzzxRGqzQZwcdwJpw3EnEL+77rprzpw5eXl5O3fu/PSnP/300093jre0tFRVVaU2G8TPjh2kDcedQPzuvffeqqqqiRMnRlF01VVXTZ8+/YUXXjjnnHNSnQt6h2IH6eS9t5Auu+yyzsF33nmnvLzcLSTojp07d44cObLz8aRJk2bNmlVeXt6dr01AWnArFtKGW0gQv0svvfTOO+88fObcHXfc8eabb86fP98pdIRBsYO00XkL6a677vrxj3/8xBNPfPGLX3zf81SBIzzwwAMrVqzIzs7u/GRqVlbWihUrnnrqqY9//OOpjga9wK1YSBtuIUH8zjnnnIaGhtra2s5TvqMoOvfcc+vq6p5++ukXX3wxtdkgfnbsIG24hQS94oQTTrj88svPPffc945MnTr1G9/4RupCQe9Q7CBtuIUEwLG5FQtpwy0kAI5NsYN00nkL6YiRqVOnTp06NVWRAOg73IoFAAiEYgcAEAjFDgAgEIodAEAgFDsAgEAodgAAgVDsAAACodgBAARCsQMACIRiBwAQCMUOACAQih0AQCAUOwCAQCh2AACBUOwAAAKh2AEABEKxAwAIhGIHABAIxQ4AIBCKHQBAIBQ7IF21t7VHUfTRj340o+emTJmS6vgAvW9gqgMAHKeD+w9GUVS6qPTDZ3+4RxN//4vfb6/bnphQAKmk2AHpbdjFw4ZeOLRHU7a8uKWxrjFBeQBSyK1YAIBAKHYAAIFQ7AAAAqHYAQAEQrEDAAiEYgcAEAjFDgAgEIodANBjz7/68rp1647j975kZGSsWLEi1fGD5YBiAKDH9uzb91cfKViy4Js9nfj5W+e98847iYhEpNgBAMdn0AdPumTEqJ7OOvEDJyQiDJ3cigUACIRiBwAQCMUOACAQih0AQCAUOwCAQCh2AACBUOwAAAKh2AEABEKxAwAIhGIHABAIxQ4AIBCKHQBAIBQ7AIBAKHYAAIFQ7AAAAqHYAQAEQrEDAAiEYgcAEAjFDgAgEIodAEAgFDsAgEAodgAAgVDsAAACodgBAARCsQMACIRiBwAQCMUOACAQih0AQCAUOwCAQCh2AACBUOwAAAKh2AEABEKxAwAIxMCkvVNra2tVVdWGDRu2bt3a1tYWi8UKCwunTZuWlZWVtAwAAAFL0o7dunXr8vLyKioqmpqacnNzhw4dumvXrkWLFg0bNmz9+vXJyQAAELYk7diVl5cvXLhw7ty5R4xXVlbOmTOntrY2OTEAAAKWpB27+vr60tLSo8dLS0s3bdqUnAwAAGFLUrErLi6eP3/+9u3b3zvY1NQ0b968oqKi5GQAAAhbkm7FLl68uKysLBaLDR8+PDs7OyMjo7m5uaGhYcKECcuXL09OBgCAsCWp2OXm5lZXV2/evLmurq6xsTGKopycnMLCwvz8/OQEAAAIXpJuxdbU1Ozdu7egoODKK68sKChYu3bt/fff/53vfOell15KTgAAgOAlqdiVlJRs2bIliqKHH354ypQpmZmZEydObG9vLyoqqq6uTk4GAICwJe+A4k4VFRVLliw5/A3ZkpKSBQsWlJSUHHvWli1bDhw4cMRg5y1dAAA6JbvYNTY2jh8//vDTcePGve9xJ42Njfn5+YcOHUpsMgCANJe8Yrd69eqDBw8WFxc/+eSTs2fP7hxcunTpqFGjjj0xFou99dZbLS0tR4yvXbv27/7u7xKSFQAgDSWp2M2cOXPZsmW33377tm3bVq5cOWvWrMzMzM7Bmpqa952em5t79OBrr72WgKQAAOkqScWusrKy88HevXsbGhoyMzOjKJo0adLNN988evTo5GQAgIA1NjZu3Lixp7OOviFGWkv2Z+wGDRo0ZsyYzsczZsxI8rsDQKj+8R//8fA2Cv1WsosdAJAIBw8enDXl84vn3dqjWY1N7w6ZNjlBkUi+JBW7FStWrFq16i/96T333JOcGAAAAUvSAcUjR45sbm6+995733777YNHSU4GAICwJWnHbsSIEY8++uiaNWsWLFhwySWXJOdNAQD6lSTt2EVRNGDAgOuuu+6MM85I2jsCAPQrSf3yxE033ZTMtwMA6FeSt2MHAEBCKXYAAIFwjh1d2PH2n7Zu3frRj360m9dnZmbed9993b8eAEgExY4u7NzemHHCBz9c9JluXv/rR36wceNGxQ4AUkuxo2sfPPnUT5b9fTcv/t0TP0lkFgCgW3zGDgAgEIodAEAg3IoFoH9pbW2tqqrasGHD1q1b29raYrFYYWHhtGnTsrKyUh0N4mXHDoB+ZN26dXl5eRUVFU1NTbm5uUOHDt21a9eiRYuGDRu2fv36VKeDeNmxA6AfKS8vX7hw4dy5c48Yr6ysnDNnTm1tbUpSQW+xYwdAP1JfX19aWnr0eGlp6aZNm5KfB3qXYgdAP1JcXDx//vzt27e/d7CpqWnevHlFRUWpSgW9xa1YAPqRxYsXl5WVxWKx4cOHZ2dnZ2RkNDc3NzQ0TJgwYfny5alOB/FS7ADoR3Jzc6urqzdv3lxXV9fY2BhFUU5OTmFhYX5+fqqjQS9Q7ADoX1pbW9euXfve40527tzpuBPC4DN2APQjjjshbHbsAOhH4jnuZO/evbfeeuu+ffuOGH/rrbcOHTrUy0HhuCh2APQjxzju5MYbbzz23Pb29h07dhxd7Hbv3t1r+SA+ih0A/UjncSd33313Tk7O4cGmpqbbbrvtfY87OfXUUysrK48eX7169eTJk3s5KBwXxQ6AfsRxJ4RNsQOgH3HcCWFT7ADoX5YvX/7LX/4yFotdddVVl112WefgO++8U15e/sQTT6Q2G8TJcScA9CN33XXXnDlz8vLydu7c+elPf/rpp5/uHG9paamqqkptNoifHTsA+pF77723qqpq4sSJURRdddVV06dPf+GFF84555xU54LeYccOgH5k586dI0eO7Hw8adKkWbNmlZeXd3R0pDYV9BbFDoB+5NJLL73zzjsPHjzY+fSOO+54880358+ff3gE0ppiB0A/8sADD6xYsSI7O/uRRx6JoigrK2vFihVPPfXUxz/+8VRHg17gM3YA9CPnnHNOQ0NDbW1tbm5u58i5555bV1f39NNPv/jii6nNBvFT7ADoX0444YTLL7/8iJGpU6dOnTo1VZGgt7gVCwAQCMUOACAQih0AQCAUOwCAQCh2AACBUOwAAAKh2AEABEKxAwAIhGIHABAIv3kC0klra2tVVdWGDRu2bt3a1tYWi8UKCwunTZuWlZWV6mgApJ4dO0gb69aty8vLq6ioaGpqys3NHTp06K5duxYtWjRs2LD169enOh0AqWfHDtJGeXn5woUL586de8R4ZWXlnDlzamtrU5IKgL7Djh2kjfr6+tLS0qPHS0tLN23alPw8APQ1ih2kjeLi4vnz52/fvv29g01NTfPmzSsqKkpVKgD6DrdiIW0sXry4rKwsFosNHz48Ozs7IyOjubm5oaFhwoQJy5cvT3U6AFJPsYO0kZubW11dvXnz5rq6usbGxiiKcnJyCgsL8/PzUx0NgD7BrVhIGzU1NXv37i0oKLjyyisLCgrWrl17//33f+c733nppZdSHQ2APkGxg7RRUlKyZcuWKIoefvjhKVOmZGZmTpw4sb29vaioqLq6OtXpAEg9t2Ih/VRUVCxZsuTwN2RLSkoWLFhQUlJy7FktLS379+8/YnDPnj0JiQhAKih2kH4aGxvHjx9/+Om4cePe97iTxsbGYcOGHTx4MLHJAEgpxQ7SyerVqw8ePFhcXPzkk0/Onj27c3Dp0qWjRo069sRYLPbCCy8cvWP33HPPXX/99QnJCkDSKXaQNmbOnLls2bLbb79927ZtK1eunDVrVmZmZudgTU3N+04///zzjx7csWNHRkZGAsICkAKKHaSNysrKzgd79+5taGjIzMyMomjSpEk333zz6NGjUxoNgD5BsYP0M2jQoDFjxnQ+njFjRmrDANB3OO4EACAQduwgbaxYsWLVqlV/6U/vueeeZIYBoA9S7PqQF1988dVXX+3mxc8991xCw9AHjRw58vHHH1+2bNnUqVNzcnJSHQeAPkex60OuvvrqTZtfH3jCid25uK21ZdAZZyY6En3KiBEjHn300TVr1ixYsOCSSy5JdRwA+hzFrg9pb2+fMu+Oj02d2Z2Lf3HnTa/818pER6KvGTBgwHXXXXfGGWekOggAfZFiB2nmpptuSnUEAPoo34oFAAiEYgcAEAjFDgAgEIodAEAgFDsAgEAodgAAgVDsAAACodgBAARCsQMACIRiBwAQCMUOACAQih0AQCAUOwCAQCh2AACBUOwAAAKh2AEABEKxAwAIhGIHABAIxQ4AIBCKHQBAIBQ7AIBAKHYAAIFQ7AAAAqHYAQAEQrEDAAiEYgcAEAjFDgAgEIodAEAgFDsAgEAodgAAgVDsAAACodgBAARCsQMACIRiBwAQCMUOACAQih0AQCAUOwCAQCh2AACBUOwAAAKh2AEABEKxAwAIhGIHABAIxQ4AIBCKHQBAIBQ7AIBAKHYAAIFQ7AAAAqHYAQAEQrEDAAiEYgcAEAjFDgAgEIodAEAgFDsAgEAodgAAgVDsAAACodgBAARiYKoDACRbe1v7rl27Vq9e3dOJAwcOHDdu3AknnJCIVADxU+yAfueN59949dVXr7jiiuOYu2TJki996Uu9HgmgVyh2QL9zqP3QsIuG3fjMjT2deMcldxw4cCARkQB6hc/YAQAEQrEDAAiEYgcAEAjFDgAgEIodAEAguih2b7zxRvJzQGCsI4ifdQQ91UWxO/fccz/5yU8++OCDO3bsSH4gCIN1BPGzjqCnuih2b7/99he/+MWlS5cOGTJk2rRpTz311P79+5OfDNKadQTxS9A6am1tXbp06S233FJWVjZjxox58+Y9+uijLS0t8b8ypFwXxe7MM8+cPXv2b37zm02bNo0bN2727Nm5ublf+cpX1qxZk/x8kKasI4hfItbRunXr8vLyKioqmpqacnNzhw4dumvXrkWLFg0bNmz9+vW9GB5SouvfPHHo0KFnn3325z//+c9//vNDhw5Nnz795JNP/tznPnfNNdd873vfS3JESFPWEcSv19dReXn5woUL586de8R4ZWXlnDlzamtreyM1pEwXO3azZ88eMmTI1KlT9+7d++CDDzY2Nj744IPf+973fvOb31RWViY/IqQj6wjil4h1VF9fX1paevR4aWnppk2b4ssLqdfFjl1mZuZPf/rT4uLizMzM944PHz78iSeeSFYwSG/WEcQvEeuouLh4/vz5d999d05OzuHBpqam2267raioKK640Ad0Uezuu+++jo6O3bt3n3rqqW+++WZ2dnZWVlYURVlZWZ/61KeSnhDSknUE8UvEOlq8eHFZWVksFhs+fHh2dnZGRkZzc3NDQ8OECROWL1/eq/EhBbq4Ffv888/n5eU9/vjjURT96Ec/ys/P/+1vf5v0YJDerCOIXyLWUW5ubnV19aZNm77//e9fe+21X/7yl++6666GhoZVq1ZlZ2f3RmpIpS527K6//vqvfvWrf/d3fxdF0be+9a2srKx/+Id/WLduXdKzQRqzjiB+iVhHNTU1n/jEJwoKCgoKCp555pmlS5c2NjaeffbZs2fPvuCCC3opOKRMFzt2GzZsmDVrVucHGgYMGDBr1qyXX3456cEgvVlHEL9ErKOSkil6KUgAACAASURBVJItW7ZEUfTwww9PmTIlMzNz4sSJ7e3tRUVF1dXVvRAaUqqLHbtRo0atWrVq5syZnU9XrVo1cuTI5KaCtGcdQfwSuo4qKiqWLFly+BuyJSUlCxYsKCkpOfasLVu2HDhw4IjBxsbG3koFceqi2H3/+98vKSn56U9/es4557z++uu//vWv//Vf/zX5ySCtWUcQv4Suo8bGxvHjxx9+Om7cuPc97qSxsTE/P//QoUO9lQF6XRe3YseOHfvHP/6xpKTkAx/4wPjx4+vq6t77jz7QHdYRxC9B62j16tUbN24sLi5+8sknDw8uXbp01KhRx54Yi8V27NjRfJSnnnpq4MCuD/yHJOviH8S2trZf//rXAwcOPPvss6MoWrFiRRRFX/va15IdDdKZdQTxS8Q6mjlz5rJly26//fZt27atXLmy8zN8nYM1NTXvO/3UU089evDkk0+OJxL0oi6K3TXXXPP0009/6lOf+sAHPnB40F9I0CPWEcQvEevo8K+s2Lt3b0NDQ+c3MyZNmnTzzTePHj06nleGvqCLYrdixYpnn33Wt74hHtYRxC+h62jQoEFjxozpfDxjxoxEvAUkXxefscvKyho6dGjyo0BIrCOIn3UEPdXFjt03v/nNWbNmffvb3z7rrLMyMjI6B0888cTkBoP0Zh1B/BKxjlasWLFq1aq/9Kf33HNPPC8OKddFsVuwYMGuXbuqqqreO9jR0ZGsSBAC6wjil4h1NHLkyMcff3zZsmVTp07NycmJLyD0OV0Uu/c9yAd4X9YRxC8R62jEiBGPPvromjVrFixYcMkll/T660NqdfEZu+zs7Jdeeun666+fPXv27t27/+u//svvRYaeso4gfglaRwMGDLjuuuvOOOOM+F8K+pouit3jjz9+9dVXFxQUrFmzJiMj49prr73//vuTnwzSmnUE8UvcOrrpppuGDx/eKy8FfUoXxe4HP/jBsmXL/vmf/zmKovz8/Kqqqu9+97vxv1Nra+vSpUtvueWWsrKyGTNmzJs379FHH21paYn/laEPStA6gn7FOoKe6qLYbdy48fDRPlEUnX/++du3b4/zbdatW5eXl1dRUdHU1JSbmzt06NBdu3YtWrRo2LBh69evj/PFoQ9KxDqC/sY6gp7q4ssTH/vYx+6///5bb7218+ljjz120UUXxfk25eXlCxcunDt37hHjlZWVc+bMqa2tjfP1oa9JxDqC/sY6gp7qotjdd999V1xxxUMPPbRjx47Ro0dv3769O78+79jq6+tLS0uPHi8tLb3xxhvjfHHogxKxjqC/sY6gp7oodueee+4rr7xSXV392muvxWKxyZMnn3766XG+TXFx8fz58+++++73HhrU1NR02223FRUVxfni0AclYh1Bf2MdQU91Uewee+yxzgeDBw8+dOjQr371qyju36O3ePHisrKyWCw2fPjw7OzsjIyM5ubmhoaGCRMmLF++PJ5Xhr4pEesI+hvrCHqqi2L3k5/8pPNBR0fH1q1bN27cWFJSEudCys3Nra6u3rx5c11dXWNjYxRFOTk5hYWF+fn58bws9FmJWEfQ31hH0FNdFLsjfonej370o0ceeST+d2ptbV27du2GDRu2bt3a1tYWi8V27tw5bdq0rKys+F8c+poErSPoV6wj6Kkujjs5QllZ2fPPPx/n2zjuhH6uV9YR9HPWEbyvLnbs3qujo2P58uXx/96VOI87aWlp2b9//xGDe/bsiTMVJEdvrSPoz6wj6I4uit0pp5xy+HF7e/u+ffsWLVoU59vEc9xJY2PjsGHDDh48GGcGSKZErCPob6wj6Kkuit1///d/v/fphz70oQ9/+MNxvk08x53EYrEXXnjh6B2755577vrrr48zGCRIItYR9DfWEfTUsb4Ve7Szzz772muvPY63ifO4k/PPP//owR07dmRkZBxHGEiCRKwj6G+sI+ipLordn//85x/+8Ifjx48fMWJEQ0PDf/zHf5SVlZ122mlRFJ100knH9zb987iTV1555etf/3r3r29sbByduDQkVyLWEfQ31hH0VBfF7k9/+tMPf/jDr371q51PH3rooZqamocffjj+NysoKCgoKIj/ddLFCy+88Mtf/dvYq77czev37N2b0DwkU+LWEfQf1hH0VBfFbs2aNe/d/Z46deq8efOSlygsJ2adXPL3t3Xz4jVLH0hoGJLJOoL4WUfQU10Uu/z8/H/7t3+75pprOp/+6le/GjJkSJxvs2LFiiPOmXyve+65J87Xh74mEesI+hvrCHqqi2L3L//yL1OnTv3Vr341YsSIV1999Ze//OWjjz4a59uMHDny8ccfX7Zs2dSpU9/7xVgIVSLWEfQ31hH0VBfFbvLkyS+++OKSJUveeOONgoKC3/3udxdffHGcbzNixIhHH310zZo1CxYsuOSSS+J8Nej7ErGOoL+xjqCnuv6VYm+99VZDQ8O+ffvKy8v/9Kc/9c47DRhw3XXXOTSc/iMR6wj6G+sIeqSLYvf4449fffXVBQUFa9asycjIuPbaa++///5eebObbrpp+PDhvfJS0Mclbh1B/2EdQU91Uex+8IMfLFu27J//+Z+jKMrPz6+qqvrud7+b9GCQ3qwjiJ91BD3VRbHbuHHjmDFjDj89//zzt2/fnsRIEALrCOJnHUFPdVHsPvaxj91///3t7e2dTx977LGLLroouakg7VlHED/rCHqqi2/F3nfffVdcccVDDz20Y8eO0aNHb9++vaamJvnJIK1ZRxA/6wh66shid+DAge3bt7/yyivV1dWvvfZaLBabPHny6aefnpJwkKasI4ifdQTH4chi94EPfOCLX/zi888///nPfz4lgSAA1hHEzzqC43DkZ+wyMjJ++MMf3nzzzQ0NDQcOHGj/v1ISDtKUdQTxs47gOHTxGbtrrrlmz549jzzyyHsHOzo6khUJQmAdQfysI+ipLordhg0bkp8DAmMdQfysI+ip/yl2X/rSl+69997TTjtt8ODBKQwEac06gvhZR3Dc/uczdkuXLm1tbe18nJGRsW3bthRFgjRmHUH8rCM4bl0cUAwAQDpS7AAAAqHYAQAE4v/3rdjKysrTTjvt6MdRFH31q19Nai5IW9YRxM86guPzP8Xuwgsv/OlPf3r0404WEnSHdQTxs47guP1PsXvhhRdSmAPCYB1B/KwjOG4+YwcAEIgufvME0Ge1trZWVVVt2LBh69atbW1tsVissLBw2rRpWVlZqY4GQOrZsYO0sW7dury8vIqKiqamptzc3KFDh+7atWvRokXDhg1bv359qtMBkHp27CBtlJeXL1y4cO7cuUeMV1ZWzpkzp7a2NiWpAOg77NhB2qivry8tLT16vLS0dNOmTcnPA0Bfo9hB2iguLp4/f/727dvfO9jU1DRv3ryioqJUpQKg73ArFtLG4sWLy8rKYrHY8OHDs7OzMzIympubGxoaJkyYsHz58lSnAyD1FDtIG7m5udXV1Zs3b66rq2tsbIyiKCcnp7CwMD8/P9XRAOgTFDtIJ62trWvXrn3vcSc7d+5MxHEnDz300Lp163o6a+vWrb0bA4AeUewgbaxbt27y5MmDBw8eN25cbm5uFEXNzc2LFi264YYbVq5cefHFF/fie/2f//N//jzgz2cWnNmjWe++8W4vZgCgpxQ7SBtJPu7k4zM+fvmsy3s0Zc2P11T9oap3YwDQfYodpI1jHHdy4403Hnvu7t27/+Ef/mHPnj1HjG/btu3QoUO9FhGAlHLcCaSNeI47ycjIOPnkk884yimnnJLIyAAklR07SBvxHHdy8skn/+AHPzh6fPXq1TU1NYnJC0CyKXaQNhx3AsCxKXaQZgoKCgoKClKdAoC+yGfsAAACYccO0saKFStWrVr1l/70nnvuSWYYAPogO3aQNkaOHNnc3Hzvvfe+/fbbB4+S6nQApJ4dO0gbI0aMePTRR9esWbNgwYJLLrkk1XEA6HPs2EE6GTBgwHXXXXfGGWekOggAfZEdO0gzN910U6ojANBH2bEDAAiEYgcAEAjFDgAgEIodAEAgFDsAgEAodgAAgXDcCb3gUHv766+//vzzz3fz+pycnKFDhyY0EgD0Q4odvWBP8/ZvfOMb3/jGN7p5/ahRo+rq6hKZCAD6I8WO3tARlX7znov/+uruXPvfv3js94/5dfUA0Pt8xg4AIBCKHQBAIBQ7AIBAKHYAAIFQ7AAAAuFbsQBA8mzb0fyP//iPd955Z08n5uXl/eY3v0lEpJAodgBA8uxvOzD14glF5xf2aNbLb2y+75dVCYoUEsUOAEiq4jEXffnKz/Voyqr/rlXsusNn7AAAAqHYAQAEQrEDAAiEYgcAEAjFDgAgEIodAEAgFDsAgEAodgAAgVDsAAACodgBAARCsQMACIRiBwAQCMUOACAQih0AQCAUOwCAQCh2AACBUOwAAAKh2AEABEKxAwAIhGIHABCIgakOAABJ1draWlVVtWHDhq1bt7a1tcViscLCwmnTpmVlZaU6GsTLjh0A/ci6devy8vIqKiqamppyc3OHDh26a9euRYsWDRs2bP369alOB/GyYwdAP1JeXr5w4cK5c+ceMV5ZWTlnzpza2tqUpILeYscOgH6kvr6+tLT06PHS0tJNmzYlPw/0LsUOgH6kuLh4/vz527dvf+9gU1PTvHnzioqKUpUKeotbsQD0I4sXLy4rK4vFYsOHD8/Ozs7IyGhubm5oaJgwYcLy5ctTnQ7ipdgB0I/k5uZWV1dv3ry5rq6usbExiqKcnJzCwsL8/PxUR4NeoNgB0L+0trauXbv2vced7Ny503EnhMFn7ADoRxx3Qtjs2AHQjzjuhLApdgD0I8c47uTGG2889tw///nP11xzTUtLyxHjzc3Nhw4d6rWIEAfFDoB+pPO4k7vvvjsnJ+fwYFNT02233fa+x52ceOKJF1544f79+48Yf/3113//+9/3flboOcUOgH4knuNOTjrppG9+85tHj69evfrJJ59MTF7oGcUOgH7EcSeETbEDoN8pKCgoKChIdQrofY47AQAIhB07APqRFStWrFq16i/96T333JPMMNDr7NgB0I+MHDmyubn53nvvffvttw8eJdXpIF527ADoR0aMGPHoo4+uWbNmwYIFl1xySarjQC+zYwdA/zJgwIDrrrvujDPOSHUQ6H127ADod2666aZUR4CEsGMHABAIxQ4AIBCKHQBAIBQ7AIBAKHYAAIFQ7AAAAqHYAQAEQrEDAAiEYgfQXTsbd1577bUZPVdQUJDq7EC/4DdPAHTXofZDE742YdT/GtWjWW/+4c3qf65OUCSA91LsAHpg8HmDR3xyRI+mdHR0JCgMwBHcigUACIRiBwAQCMUOACAQih0AQCB8eQIA+pB33333hhtu2L9/f08n1tbWThpzcSIikUYUOwDoQ1555ZWlS5dee+XnBwzI6NHEHe++m6BIpBHFDgD6nAduvGVgZmaPpqz679oEhSGN+IwdAEAgFDsAgEAodgAAgVDsAAACodgBAARCsQMACIRiBwAQCMUOACAQih0AQCAUOwCAQCh2AACBUOwAAAKh2AEABEKxAwAIhGIHABAIxQ4AIBCKHQBAIBQ7AIBAKHYAAIFQ7AAAAqHYAQAEQrEDAAiEYgcAEAjFDgAgEIodAEAgFDsAgEAodgAAgVDsAAACodgBAARCsQMACIRiBwAQCMUOACAQih0AQCAUOwCAQCh2AACBGJi0d2ptba2qqtqwYcPWrVvb2tpisVhhYeG0adOysrKSlgEAIGBJ2rFbt25dXl5eRUVFU1NTbm7u0KFDd+3atWjRomHDhq1fvz45GQAAwpakHbvy8vKFCxfOnTv3iPHKyso5c+bU1tYmJwYAQMCStGNXX19fWlp69HhpaemmTZuSkwEAIGxJKnbFxcXz58/fvn37ewebmprmzZtXVFSUnAwAAGFL0q3YxYsXl5WVxWKx4cOHZ2dnZ2RkNDc3NzQ0TJgwYfny5cnJAAAQtiQVu9zc3Orq6s2bN9fV1TU2NkZRlJOTU1hYmJ+f/75zt27devHFF7e2th4x3tbW1t7enpC4AABpKEnFrqam5hOf+ERBQUFBQcEzzzyzdOnSxsbGs88+e/bs2RdccMGx5w4ePPjBBx9saWk5Yvyll1668847ExYZACDNJKnYlZSUvPLKK+edd97DDz/8ta99bcaMGRMnTmxoaCgqKvrZz35WUlJyjLkZGRlXXnnl0eNnnHFGRUVFwiIDAKSZ5B1Q3KmiomLJkiWHvyFbUlKyYMGCYxc7AAC6I9m/UqyxsXH8+PGHn44bN85xJwAAvSJ5xW716tUbN24sLi5+8sknDw8uXbp01KhRScsAABCwJN2KnTlz5rJly26//fZt27atXLly1qxZmZmZnYM1NTXJyQAAELYkFbvKysrOB3v37m1oaMjMzIyiaNKkSTfffPPo0aOTkwEAIGzJ/vLEoEGDxowZ0/l4xowZSX53AICAJfvLEwAAJEiyd+zS3fTp0//jP/6jmxcfOHBgwEknJzQPAMBhil3P1NXVnTP+ynMvm9Cdi/97xWONr25MdCQAgE6KXY8NGTnmgis+250rN69/VrEDAJJGsSPZWv7c3NTUdMstt3R/yte+9rWhQ4cmLlIaaW1traqq2rBhw9atW9va2mKxWGFh4bRp07KyslIdDYDUU+xItu2v17fsP/Bv61/u5vWv/OfK888//2//9m8TmiotrFu3bvLkyYMHDx43blxubm4URc3NzYsWLbrhhhtWrlx58cUXpzogACmm2JECJ3/ow//Pv/y4mxd/e3JhQsOkkfLy8oULF86dO/eI8crKyjlz5tTW1qYkFQB9h+NOIG3U19eXlpYePV5aWup3LgMQKXaQRoqLi+fPn799+/b3DjY1Nc2bN6+oqChVqQDoO9yKhbSxePHisrKyWCw2fPjw7OzsjIyM5ubmhoaGCRMmLF++PNXpAEg9xQ7SRm5ubnV19ebNm+vq6hobG6MoysnJKSwszM/P78703/3ud3v27DlicP369R0dHb2fFYBUUOwgbdTU1HziE58oKCgoKCh45plnli5d2tjYePbZZ8+ePfuCCy449tzGxsZPfvKTBw4cSE5UAFLCZ+wgbZSUlGzZsiWKoocffnjKlCmZmZkTJ05sb28vKiqqrq4+9txYLLZ///6Oo6xatWrgQP+BBxAI/0KH9FNRUbFkyZLD35AtKSlZsGBBSUlJalMBkHJ27CD9NDY2jh8//vDTcePGOe4EgEixg/SyevXqjRs3FhcXP/nkk4cHly5dOmrUqBSmAqCPcCsW0sbMmTOXLVt2++23b9u2beXKlbNmzcrMzOwcrKmpSXU6AFJPsYO0UVlZ2flg7969DQ0NmZmZURRNmjTp5ptvHj16dEqjAdAnKHaQfgYNGjRmzJjOxzNmzEhtGAD6Dp+xAwAIhGIHABAIxQ4AIBCKHQBAIBQ7AIBAKHYAAIFQ7AAAAqHYAQAEQrEDAAiEYgcAEAjFDgAgEIodAEAgFDsAgEAodgAAgRiY6gAAkFStra1VVVUbNmzYunVrW1tbLBYrLCycNm1aVlZWqqNBvOzYAdCPrFu3Li8vr6KioqmpKTc3d+jQobt27Vq0aNGwYcPWr1+f6nQQLzt2APQj5eXlCxcunDt37hHjlZWVc+bMqa2tTUkq6C127ADoR+rr60tLS48eLy0t3bRpU/LzQO+yYweQWHua9rS3t1911VXHMfdLX/rSX//1X/d6pP6suLh4/vz5d999d05OzuHBpqam2267raioKIXBoFcodgCJtePNHVFG1DykuacTX3z6xVgsptj1rsWLF5eVlcViseHDh2dnZ2dkZDQ3Nzc0NEyYMGH58uWpTgfxUuwAEi4jM+Ov/6nH/eyd195JRJh+Ljc3t7q6evPmzXV1dY2NjVEU5eTkFBYW5ufnd2f6mjVrWltbjxhcv359R0dH72eFnlPsAOhHampqPvGJTxQUFBQUFDzzzDNLly5tbGw8++yzZ8+efcEFFxx7bmNj46RJk1paWo7+o4yMjMTkhZ7x5QkA+pGSkpItW7ZEUfTwww9PmTIlMzNz4sSJ7e3tRUVF1dXVx54bi8X27t3bcZRVq1ZlZmYmJT68Dzt2APRHFRUVS5YsOfwN2ZKSkgULFpSUlKQ2FcTJjh0A/VFjY+P48eMPPx03bpzjTgiAYgdA/7J69eqNGzcWFxc/+eSThweXLl06atSoFKaCXuFWLAD9yMyZM5ctW3b77bdv27Zt5cqVs2bNyszM7BysqalJdTqIl2IHQD9SWVnZ+WDv3r0NDQ2dX3qYNGnSzTffPHr06JRGg16g2AHQHw0aNGjMmDGdj2fMmJHaMNBbfMYOACAQih0AQCAUOwCAQCh2AACBUOwAAAKh2AEABEKxAwAIhGIHABAIxQ4AIBCKHQBAIBQ7AIBAKHYAAIFQ7AAAAjEw1QEAAN7Ha41vtbW1fehDHzqOuQsXLrzxxht7PVLfpNgBAH3djt27BmZm/ugf5vd04qLHl77xxhuJiNQ3KXYAQBoYMGBA6fj/1dNZy/9jZSLC9Fk+YwcAEAjFDgAgEIodAEAgFDsAgEAodgAAgVDsAAACodgBAARCsQMACIRiBwAQCMUOACAQih0AQCAUOwCAQCh2AACBUOwAAAKh2AEABEKxAwAIhGIHABAIxQ4AIBCKHQBAIBQ7AIBADEx1AHgf7W0HnnvuuRNPPLGb13/kIx+59NJLExoJAPomxY6+rmXnjkce++ljP1/RnYv37909NDfn5ZdfTnQqAOiDFDvSwGdu/NZFnyntzpXP/XzJxp/9KNF5AKBv8hk7AIBAKHYAAIFQ7AAAAqHYAQAEQrEDAAiEYgcAEAjFDgAgEIodAEAgFDsAgEAodgAAgVDsAAACodgBAARCsQMACIRiBwAQCMUOACAQih0AQCAUOwCAQCh2AACBUOwAAAKh2AEABEKxAwAIhGIHABAIxQ4AIBCKHQBAIAamOkCKvfHGG9/+9rcPHTrUzesbGxtHJDQQAMDx6u/Frra29seP/OSiK6/q5vU7d+1KaB4AgOPW34tdFEUnDjr5Cwu/282Ln//X5QkNAwBw3HzGDgAgEIodAEAgFDsAgED4jB0AJERra+u+fft6Omv37t2JCEM/odgBQEJ86lOfevbZZ1Odgv5FsQOAhNi9e/c3yr7ypU9/pkezlq7693965EcJikTwFDsASJTs004bPuSsHk0587TTExSG/sCXJwAAAqHYAQAEQrEDAAiEYgcAEAjFDgAgEIodAEAgFDsAgEA4xw6gjzrQcqC+vv6JJ57o6cSTTz65pKQkIyMjEamAvkyxA+ijGl9u3NS06TfP/qZHszo6Ovbt3Pfqq6+OGDEiQcGAPkuxA+irOqJxZeOmVkzt0aQ97+xZeN7CgwcPJigU0Jf5jB0AQCAUOwCAQCh2AACBUOwAAAKh2AEABMK3YgGAYB1sb9+5c+drr73W04knnnjiWWedlYhICaXYAQDB+n39q39a+5+VlZU9nZiRkdHQ0FBQUJCIVImj2AEAwTp46NA1JVO+PvPaHs1q2rXzY7Ov2bdvX4JSJY5iBwCE7LRBJw8f0rObqiefdFKCwiSaL08AAAQiwB27n/70p3/4wx+6efErr7yS0DAkWcufm999991bbrmlm9cPGDDguuuuGzJkSEJTAUByBFjs7rzzzm17Wk8b3K1N13ff2JToPCTT9s31u1v2Pfnr33Xz+tdfeHbMmDFXX311QlMBQHIEWOyiKBr7N18e+zfd+pjk099d+Id/fzLReUiijtNyYl9+oKqbV9/56fM7OjoSGggAksZn7AAAAqHYAQAEQrEDAAiEYgcAEAjFDgAgEGF+Kxag32o/2B5F0Y033njqqaf2dO6ECRPmzJmTgFBAkih2AEFp3dkaRdG2D27bnbW7RxPffPHNbdu2KXaQ1hQ7gACV3FKSe15uj6bUfKdm72/3JigPpJd9+/dHUTR27NjMzMyezp06depDDz2UgFDdotgBAPz/7G3dF0XRN7705bwP5/RoYtVvnqmvr09MqG5R7AAAujDp0rF/9ZGCHk15+Y3Xtza8nKA83aHY0a91HDq0efPm559//ojx0aNHf/CDH0xJJAA4bskrdq2trVVVVRs2bNi6dWtbW1ssFissLJw2bVpWVlbSMsARWnY233rrrbfeeusR47994aWxF56fkkjHZh1B/Kwj+qAtW7Zs27bt6PGebjQkqditW7du8uTJgwcPHjduXG5ubhRFzc3NixYtuuGGG1auXHnxxRcnJwYcoaOjY+ptd58/ccoR42efOywleY7NOoL4WUf0TZdffvnrr79+9Hjd+hdGXXRh918nScWuvLx84cKFc+fOPWK8srJyzpw5tbW1yYkBRzvhpKyTTj39iMHj+BpUElhHED/riL5p//79D95067TiTx0xfuo55/bodTI6Ojp6L9VfdPrpp2/cuPGss846YrylpWXo0KFNTU3HmPvOO+989rOfbWtrO2J89+7d9fX1hw4dOmJ8zJgxf9refPKHzuxOsJ3b3m7ZuSM2YnR3Lo6i6O1XXjxt8FmDzsjuzsV/3vZW666duef+VQ9ePDdv0Okf6taLb93Sumd37jmjEvPib7bu2dP9F3/rlRfPiOVlndatF9/R+OaBlj2Dz+7Jiw8ZmnXqGd168bffPLCvJy/+8otnnNXFi6/4edXFI8/p5oskTZLX0ZvNb57y4VN6lHDPu3t2bNmRV5iXkZHRo4mNdY0nnnLih4Z26x+hw9rb2t/e+PbgEYNPyDqhRxPf/X/bu/egqOr+geMHdr3uri6siiuLXDYQ0EAfRIPSpgnTURu1ksmxKcvpajXZVOM0WI5NN8oups7omFaYl0Efr9mAadZMUwniFaSQi3IRUIi4xq5wfn/s8+zDD1Zg2XP2cny//vKc2fP5fM+X74fvh71ZcqPjZkdQmb519wAAEnVJREFUVJBTVwmCUHGuIsAUoDFonLqqsbax8VqjKd7kbLqq/Kph+mEBwf1a/HY3/7l5rfDa2Oixg4YOcurCxurGyRMmnzx50qmrfI6b66ihpnZM/37x/i9Lw19Xa6v/FRXtJzhXRxdKL48YrgkNMjp1lfWm9XzJ5ZjQ8OFDnHtj8eWq8ps3O6LHhzl1lSAIeUWF48eMHTWy+1/Uvav+60bVjRv/iox2Nt35kqIA3YiQ0c7V+z+W9vyykolhEUMHD3HqwtLqSqsoRkVFOXWVIAjnz583jRodqBvZ7Xzm/n+Hxzpx1256xm7mzJlvvPHGp59+OmbM/z42XFdXt3r16rvvvrv3a/V6/bPPPmuxWLqdt1gsVVVVPR+/fv36P//8s58Da21tLS0tnTixv41dYWFhcHCwTtev3a61tbWsrCw2tr+NnVPBW1parly54lRwk8mk1Wr7Gfzq1asxMf1tjy5dujR+/HiNpl+7XXNzc0VFRXR0f5dpQUFBaGhoP4M3NTVVVlY6FTwsLKznG2sixvbrDwM389o6srNarfn5+ZMnO/GqgU1FRYVKpTIanduQBEE4c+ZMXFycs8+wNjQ0XL9+PTLSub+DBUG4dOlSSEhIP+vIzmKxXLp0KT4+3tl05eXlgwcPDgpybkMSRfHs2bPx8fH+/k7/p5F33nmns5f4HOqoJ+qopwHXUV1d3d9//x0REeHUVcKt96OAcc59IaWbnrGrrq5etmzZsWPHIiIiDAaDn59ffX19cXHxfffdt3v3boOhX0+AAbc56ghwHXUEZXNTY2dTWlpaUFBw7do1QRDGjBkTHx8fGhrqtuyAMlBHgOuoIyiVWxs7AAAAyMfpd2AAAADAO9HYAQAAKASNHQAAgELQ2AEAACgEjR0AAIBC0NgBAAAoBI0dAACAQtDYAQAAKASNHQAAgELQ2AEAACgEjR0AAIBC0NgBAAAoBI0dAACAQtDYAQAAKASNHQAAgELQ2AEAACgEjR0AAIBC0NgBAAAoBI0dAACAQtDYAQAAKASNHQAAgELQ2AEAACiE2tMDkF5cXFx7e7tOp5Mj+NWrVwMCAmQKfuXKFYPBoNVqfS54WVnZ6NGjNRqNLwb//PPPly5dKkdwnyZrHfVUW1vr5+c3evRo96RrbGxsbGw0mUzuSdfe3l5RUWE2m92TrrOzs7i4ODw8XK1202946uhW3FxHvWhtba2trQ0LC/P0QARB5v3IKc3NzXV1daGhoZ4eiCBIV0cKbOyam5unT59+3333yRF8zZo1U6dOvffee+UI/tZbbyUmJs6YMUOO4GlpadOnT7/77rvlCP7mm2/eddddSUlJcgRftWpVcnLy9OnT5Qj+xhtvtLW1yRHZ18laRz1t27Zt8ODBjz32mHvSZWdnnz59+plnnnFPuoKCgo0bN7otXVNT02uvvbZo0SKj0eiejNTRrbi5jnqRk5OTmZnptkXYO1n3I6f88ssvR48e9ZJpkayORMWZNGnSF198IVPwmJiYTZs2yRQ8Kipq8+bNMgU3m81bt26VKXhoaOhXX30lU3CTyZSRkSFTcKPRuHPnTpmC+zRZ66inpUuXPv30025Ll56enpiY6LZ0WVlZgwcPdlu6mpoaQRDy8/PdlpE6uhU311Evdu7caTQaPT2K/5B1P3LKl19+GRER4elR/IdUdcR77AAAABSCxg4AAEAhaOwAAAAUgsYOAABAIWjsAAAAFEKBjZ2/v79KpfLF4CqViuBKCu7TZF3qDtP5+7vv15FKpXJzOndOpi2XmzNSRw65uY564VU/I+8ZjPeMRJBuMH6iKLoexaucOXMmPDxcr9fLETwvL89sNo8cOVKO4KdPn46MjBwxYoQcwXNzc6OiomQKnpOTEx0dLdOXcJ46dSo2Nlamr7I8derUxIkTZfr2Y58max31VFJSolKp3PY1ofX19eXl5fHx8e5JZ7FYcnJy3PmtXSdPnpw5c6bbmlfq6FbcXEe9aGlpyc/PnzZtmqcHIggy70dOaWpq+uOPP6ZOnerpgQiCdHWkwMYOAADg9qTAl2IBAABuTzR2AAAACkFjBwAAoBA0dgAAAApBYwcAAKAQNHYAAAAKQWMHAACgEDR2AAAACqGExq64uPiBBx4ICAgICwv7+OOPbSdramoWLlyo1+unTZuWk5Mz4ODHjx+fMmWKRqOJjY3du3evtMHtqqqqqqqqpA3++OOP+3WRlZUlYfCOjo5XX33VZDKFhIRs3rxZwpFv3rzZ7/+LjIyUcOR5eXkzZszQarUxMTF79uyRcOS+q8/bl3Z++ozmsKLlS2fXtQzlS+ewdmTN6HDNu+jPP/88dOjQwMZzm/CqeXC4Hbhft2XjwSnqNhJPzY9c3Yvo49rb200m0/PPP19RUZGdna3X63ft2iWK4uzZs5csWVJeXr5u3TqDwdDW1jaA4HV1dXq9fv369ZWVlZ988smgQYOKi4ulCm7X3NwcGRn5wQcf2A6lCp6cnLxhw4bC/2pubpYw+EsvvTRjxoyioqLMzEy1Wp2bmytV8L/++quwi9TU1FWrVkkV3Gq1Go3Gl19+uaysbNu2bSqV6sKFC1IF91193r6089N7tFtVtEzp7LqVoXzpHNaOfBlvteZd0dnZuXTp0hdffHEA47l9eNU8ONwO3KznsvHUFPUciUfmR77uxecbu59//nnEiBFWq9V2+Nprrz366KMlJSX+/v7V1dW2k7GxsRkZGQMIfuTIkejoaPthcHDwnj17pAput3z58kGDBtl2FAmDBwUFXbx4sesZqYK3tLRotdqCggLb4dq1aw8cOCD5tIiieP78+aioqNbWVqmCFxcXC4JQVVVlO4yMjNy2bZscI/chfd6+tPPTZzSHFS1fOruuZShfOoe1I2tGh2velYzp6em2/9LXYWN3m1eTnbfNQ8/twM16LhtPTZHDBeyR+ZGve/H5l2LHjh372WefqdVq22F9fb3RaCwoKIiIiAgKCrKdTE5Ovnjx4gCC33vvvSdOnLD9u7KysqGhYdKkSVIFt9m/f39BQcHs2bNth1IFb2lpqampef3113U63YQJE7Zv3y5h8F9//XXkyJExMTG2w9WrVy9YsEDaabFZsWJFenr6sGHDpAoeFhYWHh6+adOmv//++9ChQxUVFUlJSXKM3If0efvSzk+f0RxWtHzpbLqVoXzpHNaOrBkdrnlXMj788MP79u176KGHBjae24RXzYPD7cDNei4bT01Rz5F4an7k6158vrGLjIx88sknbf/OysrKzMxcvHhxdXW1wWCwP8ZgMFRXVw8guFarNRqN9fX1iYmJ0dHRb7/9dmxsrFTBBUGoqqpauXJlRkaG/UcrVfDi4mKVSvXggw8WFxe/8847zz///PHjx6UKXlNTM2rUqBUrVhgMhuDg4LVr14qiKOG02GRnZzc1Ndm2PamC+/v779q1691339Xr9QsWLHjnnXeio6MlH7lv6fP2pZ2fPqM5rGj50gmOylC+dA5rR9aMDte8KxkjIiISEhLGjBkzsPHcJrxqHhxuB24eQ89l46kp6jkST82PfN2Lq7/IvERbW1taWtqWLVt27dqVlJR08eJFPz+/rg+wWq0DDq7T6d5///1jx46tW7fu/vvv7+zslCS4KIrLli1LS0szm832k1IFj4uLa29vV6lUgiCkpqZmZWXt2LEjOTlZkuANDQ3nzp1btGhRaWlpUVHRnDlzQkJCpBq53dq1a1etWmX7t1TBr127tmjRoq1bt6ampubl5S1ZsiQ+Pl7ykfuWPm9f2vnpZ7RuFS1fOodlKF86h7Vj/+UuR0aHaz4lJWXAGV0cz23Cq+bB4XZw//33e2o8Nt4zRZ6dHzm6F59/xk4QhKKiooSEhAsXLuTl5c2fP18QhKCgoPr6evsDBvxqTkNDQ0NDw6BBg1JSUj788MOEhITdu3dLFXzjxo3+/v6PPfZYe3t7R0eH1Wq1WCxSBRcEwbZMbaKjo69duyZV8MDAwHHjxr399tsjRoxISEhYvHjxwYMHJRy5IAiFhYUXLlywv0olVfDvvvvObDY/9dRTWq125syZTz755Ndffy3tyH1On7cv7fz0J1rPipYvncMylC+dw9oZcLr+ZHS45l3J6OJ4bhPeNg89twMPDsbGq6bIU/MjU/fi842dxWKZO3fuvHnzsrKybN+LIQhCXFxccXGxfXZ+//33uLi4AQT//PPPly1bZj8MDg5ua2uTKvjvv/+elZU1bNiwoUOHHj58ePXq1RMnTpQq+LfffvvEE0/YD0tKSsxms1TBo6Kibt682dnZaTvU6/XDhw+XKrhNRkbGokWLhg4dajuUKni3PVsURavVKu3IfU6fty/t/PQZzWFFy5fOYRnKl85h7Qw4XX8yOlzzrmR0cTy3Ca+aB4fbgacGY+c9U+Sp+ZGxe5Hm0x2es2/fPoPBcPny5bL/qq2tFUUxJSXlhRdeaG1t/eabbwIDAwf26eUzZ85oNJoDBw40NjZmZ2frdLoffvhBquBdLVy40P5xPEmCX7x4UaVSrVu3rqam5uDBgzqd7ty5cxKOPCkp6fXXX6+rqzt58mRgYODhw4clDC6K4pQpU7Zv3971jCTBS0tLtVrtli1bGhoaTpw4YTAY9u3bJ+3IfZHD2z9w4MDZs2d7eYBM6W5V0fLdnV3XMpQvncPakS/jrda8i5577rmuHyqUb7X4Lu+Zh1ttB+7Xbdl4cIq6jsRT8yNf9+Lzjd2aNWu6taqPPPKIKIrXr1+fP39+QEBAYmKiK98UdfDgwcTERNsXFH/99de2k1IFt+u6o0gV/Icffpg2bZpWq508efLRo0elDV5dXT1v3ryRI0dGRUVt3bpV2uD19fX+/v6FhYVdT0oV/Keffrrrrrs0Gs2ECRM2b94sbXAf5fD277jjjrS0tF4eIFO6W1W0TOm6kqSx6zOdw9qRNaPDNe+ibju0fKvFd3nVPDjcDtyv27Lx4BR1G4lH5ke+7sVPdO0zWQAAAPASPv8eOwAAANjQ2AEAACgEjR0AAIBC0NgBAAAoBI0dAACAQtDYAQAAKASNHQAAgELQ2AEAACgEjR0AAIBC0NgBAAAoBI0dAACAQtDYAQAAKASNHQAAgELQ2AEAACgEjR0AAIBC0NgBAAAoBI0dAACAQtDYAQAAKASNHQAAgELQ2AEAACgEjR0AAIBC0Nj5sM8++8xsNnc9Y7VaAwIC9u7d2/PBlZWVfn5+7hoa4DOoI8B11JH3oLHzYampqWVlZadPn7afOXnypMVimTt3rgdHBfgW6ghwHXXkPWjsfNi4ceNmzpyZmZlpP3PgwIF58+YNHz7cg6MCfAt1BLiOOvIeNHa+bcmSJfZCEkXx4MGDixcvFgQhKysrISFBo9GYTKb333+/6yV//PGHTqezH86aNevbb78VBKGoqGj27Nl6vf6ee+7Zs2ePG28C8DDqCHAddeQtRPiyGzduqNXq3NxcURR/++234cOHt7S0NDU1DRs2bMOGDTU1Nd9//71arS4qKqqoqLD9uAsLC7VarT1CSkrKjh07/vnnn9DQ0DVr1tTW1h47diwoKOjHH3/01E0BbkYdAa6jjrwEz9j5NoPBMGvWLNsfSfbnvYcMGXLmzJkVK1aMGjUqKChoyJAhdXV1vcf5/vvvOzs709LSRo8enZKSsnz58i1btrjlDgDPo44A11FHXoLGzufZn/3ev3+/7XlvtVp95MiRmJiYKVOmvPfee2q1upfLRVEUBKGkpKS2ttZkMhmNRqPRuG3bNqvV6p7xA96AOgJcRx15g96mGD5h4cKFzzzzzI4dO8rLy22fPzp69OhHH3106tSp8ePHC4IQEhLS7ZKOjg5RFG2fNq+urhYEwWg0ms3m/Px82wOuXLliKzDgNkEdAa6jjrwBz9j5PJ1ON2/evFdeeWXu3LkajUYQhIaGBrVarVar29raPvnkk4qKipaWFvvjAwMD29raMjIyLBbLli1bbMUzZ86c2tra9evX19XV/fzzzwkJCWfPnvXYLQFuRx0BrqOOvILH3t0H6die+t6zZ4/tsL29fenSpTqdzmw2f/zxxytXrgwMDCwqKrL/uNPT00eNGqVSqaZNmzZnzpwdO3aIopibm3vPPfdoNJrw8PANGzZ47GYAD6GOANdRRx7nJ/IMJwAAgCLwUiwAAIBC0NgBAAAoBI0dAACAQtDYAQAAKASNHQAAgELQ2AEAACgEjR0AAIBC0NgBAAAoBI0dAACAQtDYAQAAKASNHQAAgELQ2AEAACgEjR0AAIBC0NgBAAAoBI0dAACAQtDYAQAAKASNHQAAgELQ2AEAACjE/wFZ3dTimM3j3wAAAABJRU5ErkJggg==",
      "text/plain": [
       "Plot with title “Data 3”"
      ]
     },
     "metadata": {
      "image/png": {
       "height": 420,
       "width": 420
      }
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Some examples of histograms.\n",
    "\n",
    "# Set a seed for reproducibility\n",
    "set.seed(123)\n",
    "\n",
    "# Generate random data for three histograms\n",
    "data1 <- rnorm(1000, mean = 50, sd = 10)\n",
    "data2 <- rbeta(1000, shape1 = 2, shape2 = 5)\n",
    "data3 <- rpois(1000, lambda = 8)\n",
    "\n",
    "# Create a new plot window with larger width for the x-axis\n",
    "par(mfrow = c(1, 3), mar = c(5, 4, 2, 1))\n",
    "\n",
    "# Plot the first histogram\n",
    "hist(data1, main = \"Data 1\", xlab = \"Value\", ylab = \"Frequency\", col = \"skyblue\", xlim = c(20, 80))\n",
    "\n",
    "# Plot the second histogram\n",
    "hist(data2, main = \"Data 2\", xlab = \"Value\", ylab = \"Frequency\", col = \"lightgreen\", xlim = c(0, 1))\n",
    "\n",
    "# Plot the third histogram\n",
    "hist(data3, main = \"Data 3\", xlab = \"Value\", ylab = \"Frequency\", col = \"pink\", xlim = c(0, 20))\n",
    "\n",
    "# Reset the plot window settings\n",
    "par(mfrow = c(1, 1), mar = c(5, 4, 4, 2) + 0.1)\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "767af5c0-fdf8-4fab-9aeb-d3af257e2051",
   "metadata": {},
   "source": [
    "The normal distribution is **symmetric**, has a **mean** and a **standard deviation**.\n",
    "\n",
    "- The **mean μ** denotes where the center of the curve is.\n",
    " \n",
    "- The **standard deviation σ** determines the width of the curve\n",
    " \n",
    "For a normal  distribution curve the **area** under the curve is always 1.\n",
    "\n",
    "Since the normal distribution is a probability distribution, the total area under the curve represents the total probability, and it must sum up to 1. This property holds true for all probability distributions, not just the standard normal distribution. For any probability distribution, the total area under its curve must be equal to 1, as it represents the probability of all possible outcomes in the distribution."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "34fd583d-db28-42cb-99c9-d4ab493d58f3",
   "metadata": {},
   "source": [
    "<img src=\"bell_curve.png\" alt=\"pic failed\" width=\"600\" height=\"300\">\n",
    "\n",
    "($\\mu$ should be 175 here)\n",
    "\n",
    "The following notation can be used: P(180<X<190) = 0.25\n",
    "\n",
    "- what is the probability of observing a value between 180-190 in this normal distribution."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "20bee16a-c3c0-4db7-b513-158f0ea76987",
   "metadata": {},
   "source": [
    "<img src=\"bell_curve_2.png\" alt=\"pic failed\" width=\"400\" height=\"300\">\n",
    "\n",
    "+/- 1 SD covers ~68% of the area."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "74767eb7-3ba1-4c62-8a19-ec6ddde31863",
   "metadata": {},
   "source": [
    "This is the normal curve function:\n",
    "\n",
    "$f(x) = \\frac{1}{\\sigma\\sqrt{2\\pi}} e^{-\\frac{1}{2}\\left(\\frac{x-\\mu}{\\sigma}\\right)^2}$\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9040ea64",
   "metadata": {},
   "source": [
    "A higher SD means the data points are spread out over a wider range of values.\n",
    "\n",
    "<img src=\"different_sds.png\" alt=\"pic failed\" width=\"400\" height=\"300\">"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4801a41e-633a-488f-b375-6d7acb98ff85",
   "metadata": {},
   "source": [
    "### The standard normal distribution\n",
    "<img src=\"stndr_norm_dist.png\" alt=\"pic failed\" width=\"450\" height=\"300\">\n",
    "\n",
    "A standardized normal dist where the mean=0 and the standard deviation = 1.\n",
    "\n",
    "95% of data lies between ±1.96 standard deviations -> 5% of data lies outside of the ±1.96 marks. "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9b2017c2-e486-4c62-bb41-6f98e6ae4e0b",
   "metadata": {},
   "source": [
    "### Z-score\n",
    "\n",
    "We can use a relationship between any specific normal distribution (**x**) and the standard normal distribution (**Z**).\n",
    "\n",
    "*Standardize* numbers from x like so:\n",
    "\n",
    "$Z=\\frac{(x-\\mu)}{\\sigma}$\n",
    "\n",
    "This gives the corresponding value on Z, or the **Z-score**.\n",
    "\n",
    "For example, the x value 180 from our previous normal distribution:\n",
    "\n",
    "$Z=\\frac{(180-175)}{10}$\n",
    "\n",
    "The corresponding value for 180 on x is 0.5 on Z. \n",
    "\n",
    "The Z-score is 0.5 \n",
    "\n",
    "0.5 standard deviations on Z. \n",
    "\n",
    "The probability of finding a person shorter than 180 (x<180) is the same as the probability that the Z-score is smaller than 0.5 in the standard normal distribution (assuming randomness). \n",
    "\n",
    "-> P(x < 180) = P(Z < 0.5)\n",
    "\n",
    "The formula also works in reverse: \n",
    "\n",
    "$x=\\sigma Z+\\mu$\n",
    "\n",
    "This lets us find a specific value on any normal distribution corresponding to a certain Z-score.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5f03a868",
   "metadata": {},
   "source": [
    "## Lecture 2\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b9c3c6b3-6dea-4b40-971b-ded48cd4503b",
   "metadata": {},
   "source": [
    "### The central limit theorem\n",
    "\n",
    "Sample *means* based on large samples will be approximately normally distributed, even though the distribution we sample from is not normal. So if we take many samples and calcualte their means, the means will be normally distributed.\n",
    "\n",
    "The **mean values** will be normally distributed even though the original distribution is non-normal.\n",
    "\n",
    "<img src=\"central_limit_theorem.png\" alt=\"pic failed\" width=\"550\" height=\"400\">\n",
    "\n",
    "If we increase the sample size (n), we approach normal distribution.\n",
    "\n",
    "The more the original distribution deviates from normal, the larger sample size we need to approach normality.\n",
    "\n",
    "A sample size of n=30 is usually large enough to assume that the sample means are normally distributed for most underlying distributions. **When n ≥ 30, the central limit theorem applies**. The sampling distribution will approximately follow a normal distribution.\n",
    "\n",
    "It is important to know that the mean values are normally distributed because:\n",
    "\n",
    "Most (parametric) statistical tests assume normal distribution. Ex P-value, confidence interval.\n",
    "\n",
    "(A **t-statistic** is the difference between two means divided by the standard error of that difference (SE_diff). The SE_diff tells you how confident you can be about the difference between two means.)\n",
    "\n",
    "$t=\\frac{(\\bar{x}_1-\\bar{x}_2)}{\\text{SE}_\\text{diff}}$\n",
    "\n",
    "$\\text{SE}_\\text{diff}=\\sqrt{\\frac{s²_1}{n_1}+\\frac{s²_2}{n_2}}=\\frac{SD_1}{\\sqrt{n_1}}+\\frac{SD_2}{\\sqrt{n_2}}= SE_1+SE_2$\n",
    "\n",
    "(This is actually the formula for a t-test)\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e1af98bb",
   "metadata": {},
   "source": [
    "If n<30 we can plot the data to determine if it follows a normal distribution.\n",
    "\n",
    "If it doesnt we can try to **transform the data** so it becomes normally distributed (ex log-transform) or use a **non-parametric test**."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7509d291",
   "metadata": {},
   "source": [
    "### Standard error of the mean (SE / SEM)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "80ddcaa6",
   "metadata": {},
   "source": [
    " **Standard errors and confidence intervals measure how precise our population estimations inferred from the sample are.** "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6ae97f80",
   "metadata": {},
   "source": [
    "The standard error of the mean is the estimated **standard deviation of the sample means**.\n",
    "\n",
    "Tells us the *expected spread* of the sample means (if we were to draw multiple samples).\n",
    "\n",
    "We expect that 68% of the sample means can be found within +/- SEM\n",
    "\n",
    "The SEM is a measure of the expected spread of the sample means around the population mean $\\mu$."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "787dd242",
   "metadata": {},
   "source": [
    "$SE=\\frac{SD}{\\sqrt{n}}$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "72ce59f0",
   "metadata": {},
   "source": [
    "SD = standard deviation of a samlpe\n",
    "\n",
    "The SE decreases as n increases. - larger sample, less error"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6e77183c",
   "metadata": {},
   "source": [
    "When we take a sample from a population, the sample mean (average of the sample) may not perfectly match the population mean (average of the entire population). The standard error provides us with an estimate of how much the sample mean is likely to vary from the true population mean.\n",
    "\n",
    "In other words, the SEM quantifies the amount of sampling variability or random error that is expected in the sample mean. It tells us how much we can expect the sample mean to vary if we were to take multiple random samples from the same population and compute their means.\n",
    "\n",
    "A standard error is measure of the precision of the estimated population parameter.\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "066388bd",
   "metadata": {},
   "source": [
    "\n",
    "<img src=\"SD_vs_SE.png\" alt=\"pic failed\" width=\"450\" height=\"300\">"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7c1e378c",
   "metadata": {},
   "source": [
    "= 68% confidence interval\n",
    "\n",
    "expressed as a range [low high]\n",
    "\n",
    "low = mean - SE\n",
    "\n",
    "high = mean +SE\n",
    "\n",
    "You need to state whether error bars signify SD or SE."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "46cb1132",
   "metadata": {},
   "source": [
    "### Confidence intervals"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ab554fe4",
   "metadata": {},
   "source": [
    "**The standard in statistics is that we want to be 95% certain.**\n",
    "\n",
    "Confidence interval = the mean +/- the SEM\n",
    "\n",
    "$CI=\\bar{x}\\pm \\text{SE}(\\bar{x})$\n",
    "\n",
    "upper limit = mean + SE\n",
    "\n",
    "lower limit = mean - SE\n",
    "\n",
    "Confidence interval: [lower_limit   upper_limit]\n",
    "\n",
    "So confidence intervals are the range in which the true mean lies."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ea166f7f",
   "metadata": {},
   "source": [
    "The standard error tells us how certain (in percent) we are that the true mean lies within the range.\n",
    "\n",
    "In statistics, we want to be 95% certain.\n",
    "\n",
    "To get from 68% confidence interval to 95% we multiply the SE by 1.96 (because we are using the normal dist)\n",
    "\n",
    "We can only use this (multipying with 1.96) if n > 100. If n is smaller, we need a **t-distribution**.\n",
    "\n",
    "**Or is it if n>30??**\n",
    "\n",
    "If we repeat a study 100 times, drawing a different sample every time, we would expect 95% of the confidence intervals to cover the true population mean $\\mu$.\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fe527ee9",
   "metadata": {},
   "source": [
    "<img src=\"95_CIs.png\" alt=\"pic failed\" width=\"300\" height=\"300\">"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "70428045",
   "metadata": {},
   "source": [
    "<img src=\"calculate_95CI.png\" alt=\"pic failed\" width=\"500\" height=\"250\">"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "de9ee0f8",
   "metadata": {},
   "source": [
    "The standard normal distribution aka the Z-distribution is appropriate when the data approximates a normal distribution, i.e. when n>30.\n",
    "\n",
    "The area in the tails is called alpha. For a standard of 95% certainty, we set $\\alpha= 0.0025+0.0025=0.05$.\n",
    "\n",
    "Certain **Z critical values** are used for various confidence levels: ex 90, 95, 99."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ca4b628a",
   "metadata": {},
   "source": [
    "### T-distribution"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "66ef679d",
   "metadata": {},
   "source": [
    "<img src=\"t_dist.png\" alt=\"pic failed\" width=\"500\" height=\"250\">"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ce2af60e",
   "metadata": {},
   "source": [
    "The t-distribution curve will change shape depending on the *degrees of freedom*.\n",
    "\n",
    "As the degrees of freedom increase, the t-distribution approaches the normal distribution.\n",
    "\n",
    "At *df=100 (n=101)*, the t-distribution is virtually the same as the normal distribution."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cce8f511",
   "metadata": {},
   "source": [
    "**Sort of extra material**\n",
    "\n",
    "Confidence intervals:\n",
    "\n",
    "$\\bar{x} \\pm Z_{\\alpha/2}\\cdot\\frac{\\sigma}{\\sqrt{n}}$ OR $\\bar{x} \\pm Z_{\\alpha/2}\\cdot\\text{SE}$\n",
    "\n",
    "This formula uses the symbols for t-distribution stuff. The Z means we should use a Z-score from the standard normal distribution for a certain value of alpha.\n",
    "\n",
    "If we want for ex a 95% confidence interval, we use α = 1 - 0.95 = 0.05\n",
    "\n",
    "Divided by 2, this gives us 0.025, which corresponds to one of the tails in the standard normal distribution. We use the Z-score for that tail (for example Z = 1.96)."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "81e4722f",
   "metadata": {},
   "source": [
    "If our sample size is small (n<30) and we need to estimate the standard deviation:\n",
    "\n",
    "$\\bar{x} \\pm t_{n-1, \\alpha/2}\\cdot\\frac{\\text{SD}}{\\sqrt{n}}$\n",
    "\n",
    "t tells us that we should use a t-score from a t-distribution with n-1 df, for the same given value of alpha"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "269c3c40",
   "metadata": {},
   "source": [
    "So, we determine our alpha: $\\alpha = \\text{something}$\n",
    "\n",
    "Then we get the critical value with which to multiply the SE to get our CIs: $Z_{\\alpha/2}$ or $t_{n-1, \\alpha/2}$, depending on sample size.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "30098650",
   "metadata": {},
   "source": [
    "You use a t-table to determine which t-distribution to use. This is built in to most software."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1ccc01d8",
   "metadata": {},
   "source": [
    "### Degrees of freedom"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5831f3c2",
   "metadata": {},
   "source": [
    "The number of values in the final calculation that are free to vary.\n",
    "\n",
    "A simplified interpretation: **df=n-1** when we estimate one parameter in our calculations.\n",
    "\n",
    "Generally: df = nr of observations we use to estimate a parameter - nr of estimated intermediate parameters.\n",
    "\n",
    "When you compare two groups or estimate two parameters, the df is total n (n1 + n2) minus 2."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0597c1eb",
   "metadata": {},
   "source": [
    "### SE and CI for proportions"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c7589aa7",
   "metadata": {},
   "source": [
    "$SE(p)=\\sqrt{\\frac{p(1-p)}{n}}$\n",
    "\n",
    "$CI=p\\pm Z_{1-\\alpha/2}\\cdot SE(p)$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a165a14a",
   "metadata": {},
   "source": [
    "Example. \n",
    "\n",
    "We want to estimate the proportion of smokers in sweden. We set $\\alpha = 0.05$ for 95% certainty. \n",
    "\n",
    "n=368, smokers=79\n",
    "\n",
    "proportion of smokers = $p = \\frac{79}{368} = 0.215$\n",
    "\n",
    "$SE(p)=\\sqrt{\\frac{p(1-p)}{n}} = \\sqrt{\\frac{0.215(1-0.215)}{368}} = 0.0214$\n",
    "\n",
    "$CI=p\\pm Z_{1-\\alpha/2}\\cdot SE(p) = 0.215 \\pm 1.96 \\cdot 0.0214$ = [0.173 0.257] \n",
    "\n",
    "We are 95% certain that the true proportion of smokers in sweden lies between 17.3% and 25.7%. Our best estimate is 21.4%."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c6629c67",
   "metadata": {},
   "source": [
    "### Testing hypotheses\n",
    "\n",
    "Hypothesis = statement that can be tested true or false\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "efaa01b5",
   "metadata": {},
   "source": [
    "**Step 1**: State the null hypothesis ($H_0$) and the alternative hypothesis ($H_1$).\n",
    "\n",
    "The $H_0$ typically assumes that there is no effect, no difference, no relationsip. It assumes that any observed difference is due to chance or sample variability. \n",
    "\n",
    "The $H_1$ states the opposite. That there is an effect, difference or relationship. This is the hypothesis you are trying to prove.\n",
    "\n",
    "The goal is to evaluate whether the data provides enough info to reject the $H_0$ in favour of $H_1$. You want to determine whether the observed results are so unlikely to occur by chance that you can conclude significance."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e302ce9a",
   "metadata": {},
   "source": [
    "**Step 2**: Choose a critical significance level ($\\alpha$) for the test.\n",
    "\n",
    "How sure you want to be. Ex for 95% certainty, $\\alpha$ = 1-0.95 = 0.05\n",
    "\n",
    "$\\alpha$ is the probability of making a **Type 1 error** (rejecting a true $H_0$) aka *false positive*.\n",
    "\n",
    "Ex there is a 5% chance that we rejected a true $H_0$, meaning that we found results that actually aren't there."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5085de0d",
   "metadata": {},
   "source": [
    "**Step 3**: Decide appropriate test depending on distribution, variable scale and statistical assumptions"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5dd90077",
   "metadata": {},
   "source": [
    "**Step 4**: get your p-value\n",
    "\n",
    "The p-value indicates the probability of obtaining the observed results (or something even more extreme) by chance (i.e. if $H_0$ is true). "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1127518b",
   "metadata": {},
   "source": [
    "**Step 5**: Make a decision on which hypothesis to reject based on the p-value.\n",
    "\n",
    "If $\\text{p-value} \\leq \\alpha$ -> reject $H_0$ in favour of $H_1$.\n",
    "\n",
    "If $\\text{p-value} \\geq \\alpha$ -> $H_0$ cannot be rejected."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9edc39f2",
   "metadata": {},
   "source": [
    "Two-sided hypotheses vs one-sided.\n",
    "\n",
    "**Two-sided**: \n",
    "\n",
    "$H_0$: $\\mu_1=\\mu_2$\n",
    "\n",
    "$H_1$: $\\mu_1 \\neq \\mu_2$\n",
    "\n",
    "Asks if there is a change in either direction. The results will show which one it is.\n",
    "\n",
    "**One-sided**:\n",
    "\n",
    "$H_0$: $\\mu_1 < \\mu_2$\n",
    "\n",
    "$H_1$: $\\mu_1>\\mu_2$\n",
    "\n",
    "Asks if the first mean is greater thant the second."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d9cd9fae",
   "metadata": {},
   "source": [
    "## Lecture 3"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "664b810b",
   "metadata": {},
   "source": [
    "### The one-sample t-test and p-values"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b2e8a616",
   "metadata": {},
   "source": [
    "A t-test is a statistical test where the test statistic follows a t-distribution. \n",
    "\n",
    "A one-samlpe t-test checks if a mean value from from a sample is *significantly different* from a hypothesized or known value.\n",
    "\n",
    "formula: $t=\\frac{\\bar{x}-\\mu_0}{\\text{SE}}$\n",
    "\n",
    "$\\bar{x}$ = estimated mean from your sample\n",
    "\n",
    "SE = standard error of the mean of your sample\n",
    "\n",
    "$\\mu_0$ = the hypothesized value (reference value)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "03debcdf",
   "metadata": {},
   "source": [
    "The t-statistic is used on the t distribution corresponding to the appropriate degrees of freedom.\n",
    "\n",
    " - meaning once we have our mean and have calculated our t-statistic\n",
    " - lets say its 2.8\n",
    " - we use $\\pm$ 2.8 to delimit the alpha on our corresponding t-distribution (mean is 0)\n",
    " - then we calculate the **p value** (the percent chance that a random datapoint would be in the alpha) with software\n",
    "\n",
    "The area under the t-distribution curve that is our alpha when we plugged in our t-statistic is the p-value. (the sum of the tail areas)\n",
    "\n",
    "The p-value is always between 0 and 1.\n",
    "\n",
    "If we assume there is no effect, the p-value can be interpreted as the probability that the observed effect (or something more extreme) could be due to chance.\n",
    "\n",
    "If **p-value > 0.05** = weak evidence of effect. There is more than a 5% chance that observed results are due to chance.\n",
    "\n",
    "**p-value between 0.01 - 0.05** = moderate evidence of effect\n",
    "\n",
    "**p-value < 0.01** = strong evidence of effect\n",
    "\n",
    "There is a **significant effect** only if p-value > 0.05\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b0d95e8a",
   "metadata": {},
   "source": [
    "### Critical value"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "db9a5957",
   "metadata": {},
   "source": [
    "The p-value can be compared to the $\\alpha$-value in order to decide whether or not to reject H0.\n",
    "- alternatively, a critical value can be used, and compared with the calculated t-value\n",
    "- The critical value defines where on you distribution the aplha-value is covered"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e7d57e28",
   "metadata": {},
   "source": [
    "### Independent two-sample t-test = unpaired t-test"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dd7350cc",
   "metadata": {},
   "source": [
    "Compares the means of *two populations*.\n",
    "\n",
    "Since we cant measure the whole population, we take a sample and do a t-test.\n",
    "\n",
    "(If we happen to know the population variance, the corresponding Z test is used)\n",
    "\n",
    "In an unpaired t-test the populations should be **independent**.\n",
    "\n",
    "$t=\\frac{\\bar{x}_1-\\bar{x}_2}{\\sqrt{\\frac{s_1²}{n_1}+\\frac{s_2²}{n_2}}}$ (This is Welch's t-test. Used with unequal variances (heteroscedasticity))\n",
    "\n",
    "This is another way of doing it, Student's t-test, when you assume equal variance (homoscedasticity): $t=\\frac{\\bar{x}_1-\\bar{x}_2}{(\\frac{\\sqrt{(s_1²+s_2²)}}{\\sqrt{\\frac{n_1+n_2}{2}}})}$ This uses the pooled SE of the groups\n",
    "\n",
    "\n",
    "$\\bar{x}$ =sample means\n",
    "\n",
    "s = standard deviations\n",
    "\n",
    "$s²$ = variance\n",
    "\n",
    "n = sample sizes\n",
    "\n",
    "This equation only works if the two groups have **the same sample size**.\n",
    "\n",
    "(If sample sizes differ, we put more weight on the larger one)\n",
    "\n",
    "We use a t-distribution of **df=n-2** because we have estimated two parameters.\n",
    "\n",
    "The p-value is the area of the alpha tails of that t-dist.\n",
    "\n",
    "T-test results are usually reported with a 95% confidence inteval.\n",
    "\n",
    "- calculated like this: $(\\bar{x}_1 - \\bar{x}_2) \\pm t_{n1+n2-2, \\alpha/2}SE$\n",
    "- meaning the difference of the means times a t-value depending on the df (n1+n2-2) and the alpha, times the SE\n",
    "- the SE here is actually $SE_{diff}$. Look at the formula above under Central Limit Theorem heading. \n",
    "\n",
    "If the H0 value is not within the confidence interval, we can reject the H0.\n",
    "- H0: $\\mu_1=\\mu_2$ -> H0: $\\mu_1-\\mu_2=0$\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cac35066",
   "metadata": {},
   "source": [
    "Even if the difference between the means is the same in two tests, the pooled SE or SE_diff will determine the precision of the test. If the combined SE is lower, the t-value will be higher, and the p-value lower.\n",
    "\n",
    "<img src=\"t_test_precision.png\" alt=\"pic failed\" width=\"500\" height=\"250\">"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "64024878",
   "metadata": {},
   "source": [
    "The unpaired t-test relies on **normality** and **equal variance**.\n",
    "\n",
    "Normality can be tested with **shapiro-wilks test**.\n",
    "\n",
    "You can also plot it to see.\n",
    "\n",
    "If n > 30, normality can be assumed (central limit theorem)\n",
    "\n",
    "If n is small, but we know that the underlying values are normal, we can assume normality.\n",
    "\n",
    "**Levene's test** can compare variances (H0: no diff in var)\n",
    "\n",
    "If populations dont have equal var, we can use **Welch's t-test**."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "02c0424b",
   "metadata": {},
   "source": [
    "<img src=\"t_tests_in_r.png\" alt=\"pic failed\" width=\"500\" height=\"250\">"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a2cedfdf",
   "metadata": {},
   "source": [
    "### Parametric and non-parametric tests\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ff123221",
   "metadata": {},
   "source": [
    "If you cannot calculate **parameters** (such as means and SD), you cannot use **parametric tests**."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1e560871",
   "metadata": {},
   "source": [
    "### Paired t-test = dependent sample t-test"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "75686379",
   "metadata": {},
   "source": [
    "Used to determine if the **mean difference** from pairs of observations is different from zero.\n",
    "\n",
    "For ex, compares observations on paired individuals: \n",
    "\n",
    "- Before and after studies:\n",
    "\n",
    "<img src=\"paired_t_test.png\" alt=\"pic failed\" width=\"500\" height=\"250\">\n",
    "\n",
    "- Matched pair design (red:women, blue:men):\n",
    "- we pair individuals with similar charachteristics\n",
    "- we pair men with men of similar attribute and women with women of similar attributes\n",
    "- then for ex one person in each pair is drug test subject and the other control\n",
    "\n",
    "<img src=\"matched_pair_design.png\" alt=\"pic failed\" width=\"500\" height=\"250\">\n",
    "\n",
    "- Paired study design:\n",
    "\n",
    "<img src=\"paired_study.png\" alt=\"pic failed\" width=\"500\" height=\"250\">\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bd39036a",
   "metadata": {},
   "source": [
    "Here's how its done:\n",
    "- Calculate the mean and the sd of the **differences**\n",
    "- H0: $\\mu_D=0$ -> the mean difference between the paired values is zero\n",
    "- H1: $\\mu_D\\neq0$\n",
    "- the paired t-test checks if the observed stuff is just due to chance\n",
    "- formula: $t=\\frac{\\bar{x}_D-\\mu_D}{SE}$ -> the mean of the differences minus the hypothesized value, in this case 0 -> $t=\\frac{\\bar{x}_D}{SE}$\n",
    "- the t-statistic of a paired t-test is the mean of the differences divided by the SE\n",
    "- a p-value is obtained\n",
    "- CI: $\\bar{x}_D \\pm t_{n-1, \\alpha/2}SE$\n",
    "- if the H0 value (0) is included in the interval, we cannot reject H0\n",
    "\n",
    "Assumptions for paired t-test: \n",
    "- the differences need to be normally distributed (when n is small)\n",
    "\n",
    "The paired t-test is just a one-samlpe t-test based on the differences between pairs."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f3106e2a",
   "metadata": {},
   "source": [
    "## Testing proportion differences"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2e3b8d3d",
   "metadata": {},
   "source": [
    "Example hypothesis:\n",
    "\n",
    "$H_0: \\pi_1=\\pi_2$ in other words $\\pi_1-\\pi_2=0$\n",
    "\n",
    "$H_1: \\pi_1 \\neq \\pi_2$ in other words $\\pi_1-\\pi_2\\neq0$\n",
    "\n",
    "Population proportion: $\\pi$\n",
    "\n",
    "Sample proportion: p"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dbe2fac4",
   "metadata": {},
   "source": [
    "### Z-test\n",
    "\n",
    "Differences in proportions"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2fff3572",
   "metadata": {},
   "source": [
    "$Z=\\frac{p_1-p_2}{\\sqrt{p(1-p)(1/n_1+1/n_2)}}$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1adf3409",
   "metadata": {},
   "source": [
    "$p=\\frac{n_1p_1+n_2p_2}{n_1+n_2}$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4fbac41a",
   "metadata": {},
   "source": [
    "Testing proportion differences require a large sample size, so we assume normal dist (Z-test)\n",
    "\n",
    "More assumptions: $n_1p_1>5$, $n_2p_2>5$, $n_1(1-p_1)>5$, $n_2(1-p_2)>5$\n",
    "\n",
    "Ex. if $p_1=0.1$ then $n_1$ needs to be at least 51. 0.1*51=5.1\n",
    "\n",
    "and independent observations\n",
    "\n",
    "There is a good example in the Diana pdf pg 24"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "033fef1a",
   "metadata": {},
   "source": [
    "We then use software to calculate the p-value from the Z-score\n",
    "\n",
    "We can also compare to critical values in our Z-dist. If the Z-score is greater than or smaller than the alpha, we can reject the H0"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4dd1e71a",
   "metadata": {},
   "source": [
    "#### Confidence interval"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "43e13fb7",
   "metadata": {},
   "source": [
    "$(p_1-p_2) \\pm z_{\\alpha/2}SE$\n",
    "\n",
    "$SE=\\sqrt{\\frac{p_1(1-p_1)}{n_1}+\\frac{p_2(1-p_2)}{n_2}}$\n",
    "\n",
    "If the interval includes 0 (as in $H0: \\pi_1-\\pi_2=0$), we cannot reject the H0"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "766ec333",
   "metadata": {},
   "source": [
    "## Chi-square test"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "13b2b062",
   "metadata": {},
   "source": [
    "A Chi-square $(\\chi²)$ test the hypothesis that proportions are the same in different groups.\n",
    "\n",
    "Chi-square can only be used to test actual numbers (**frequencies**). Not percentages, proportions, means etc\n",
    "\n",
    "**Frequency/absoulte frequency**: number of events observed\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4f36f1e0",
   "metadata": {},
   "source": [
    "We set up a contingency table\n",
    "\n",
    "||Condition 1|Condition 2|Totals|\n",
    "|---------------|------------|------------|------------|\n",
    "|Samlpe group 1|A|C|R1|\n",
    "|Samlpe group 2|B|D|R2|\n",
    "|Totals|C1|C2|N|\n",
    "\n",
    "H0: There is no difference in condition proportion between the sample groups.\n",
    "\n",
    "H1: There is a difference "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c3d54c25",
   "metadata": {},
   "source": [
    "Assumption: all *expected* cell frequencies must be $\\geq5$\n",
    "\n",
    "We calculate the $(\\chi²)$ value using a chi-square distribution ($df=(r-1)(c-1)$) and the p-value from that.\n",
    "\n",
    "A chi-square distribution is a probability distribution. Arrow points at chi-square value\n",
    "\n",
    "<img src=\"chi_2_dist.png\" alt=\"pic failed\" width=\"500\" height=\"250\">\n",
    "\n",
    "If we are old-school we can use a chi-sqare table. If our observed chi-square value is higher than the corresponding table value, we can reject H0"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c889b004",
   "metadata": {},
   "source": [
    "## Type 1 and type 2 errors"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dd0761d2",
   "metadata": {},
   "source": [
    "The H0 always states that there is no difference, or no relationship between variables.\n",
    "\n",
    "In the population, H0 is either true or false, but with a sample: No test is ever 100% certain.\n",
    "\n",
    "The outcome of a statistical test is a decision to reject or retain the H0.\n",
    "\n",
    "A test is always based on probabilities. An incorrect conclusion is always a risk.\n",
    "\n",
    "Two types of errors are possible in hypothesis testing."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "475a8bbd",
   "metadata": {},
   "source": [
    "### Type 1 error"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "61b82e86",
   "metadata": {},
   "source": [
    "When we reject a true H0. A false positive.\n",
    "\n",
    "The probability to reject a true H0 is the **type 1 error rate** = $\\alpha$ = significance level\n",
    "\n",
    "Ex $\\alpha =0.05$ means there is a 5% chance for a type 1 error: rejecting a true H0\n",
    "\n",
    "If the H0 is actually false, there is zero risk to make a type 1 error."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2fe9b043",
   "metadata": {},
   "source": [
    "### Type 2 error"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "369e96ae",
   "metadata": {},
   "source": [
    "When we retain a false H0. A false negative.\n",
    "\n",
    "$\\beta$ represents the **type 2 error rate** = the probability of a type 2 error.\n",
    "\n",
    "The value of $\\beta$ depends on several factors, for ex the sample size.\n",
    "\n",
    "A common reason for a type 2 error is that the sample size is too small.\n",
    "\n",
    "As n increases, $\\beta$ decreases.\n",
    "\n",
    "Again, if the H0 is actually true, there is zero risk for a type 2 error"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "56e1fd4a",
   "metadata": {},
   "source": [
    "## Statistical power and sample size"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e3409519",
   "metadata": {},
   "source": [
    "The **power** of a statistical test is $1-\\beta$ = the probability of NOT committing a type 2 error = the probability that the test will reject a false H0 = that the test will give the correct answer with a false H0\n",
    "\n",
    "**Power** = probability of rejecting a false H0 = $1-\\beta$\n",
    "\n",
    "The power can  be said to be the ability to find a result when one actually exists.\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "833cb18c",
   "metadata": {},
   "source": [
    "The power for comparing two independent groups depends on:\n",
    "- sample sizes\n",
    "- anticipated standardized difference beteen the groups\n",
    "- the significance level $\\alpha$\n",
    "\n",
    "We usually define the power before the study by adjusting sample size"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bdf4b051",
   "metadata": {},
   "source": [
    "Sample size n also depends on study design type and which test is used.\n",
    "\n",
    "Here we will only look at calculating n for comparing means of two independent groups\n",
    "\n",
    "$m=\\frac{2(z_{1-\\alpha/2}+z_{1-\\beta})²}{\\Delta²}$\n",
    "\n",
    "m = number of samples per group\n",
    "\n",
    "$\\alpha$ = significance level = type 1 error rate\n",
    "\n",
    "$\\beta$ = type 2 error rate\n",
    "\n",
    "$\\Delta$ = anticipated standardized difference between the two groups (Cohen's delta)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fcfca4d2",
   "metadata": {},
   "source": [
    "If we reduce the $\\alpha$, the critical value $z_{1-\\alpha/2}$ will increase and the sample size will increase"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5f541ed2",
   "metadata": {},
   "source": [
    "The power $1-\\beta$ is usually set to 0.8\n",
    "\n",
    "This means the $\\beta$ = 0.2 and we have a 20% risk of committing a type 2 error\n",
    "\n",
    "Increasing the power will increase the critical value $z_{1-\\beta}$ which will increase the sample size"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "55aa0045",
   "metadata": {},
   "source": [
    "By using the standard values for $\\alpha = 0.05$ and $1-\\beta=0.8$ we can simplify the formula:\n",
    "\n",
    "$m=\\frac{16}{\\Delta²}$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "775ad956",
   "metadata": {},
   "source": [
    "Delta is the standardized effect size:\n",
    "\n",
    "$\\Delta=\\frac{\\delta}{\\sigma}$\n",
    "\n",
    "$\\delta$ = difference between the two groups\n",
    "\n",
    "$\\sigma$ = SD\n",
    "\n",
    "So before we can calculate the sample size we need to have a sense of the SD and expected difference of the two groups\n",
    "\n",
    "The SD of what exactly? Seems it is the known SD of the type of data (for example systolic blood pressure)\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "199510df",
   "metadata": {},
   "source": [
    "What to do if we dont know the effect size?\n",
    "\n",
    "We approximate:\n",
    "- $\\Delta_{Cohen}\\leq0.2$ is considered a small effect\n",
    "- $\\Delta_{Cohen}\\approx0.5$ is considered a moderate effect\n",
    "- $\\Delta_{Cohen}\\leq0.8$ is considered a large effect\n",
    "\n",
    "The greater the expected effect, the smaller the sample size needs to be\n",
    "\n",
    "Most clinical studies are expected to produce a moderate effect (meaning the change in mean is half the size of the SD: $\\frac{\\delta}{\\sigma}$, so $\\Delta_{Cohen}=0.5$ can be used -> sample size $\\geq$ 64"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "172d2bec",
   "metadata": {},
   "source": [
    "HOWEVER, when we work with lab animals, the genetic, age and environment factors will be similar, so the variation between animals will be lower - a lower SD\n",
    "\n",
    "Then, more realistic deltas would be:\n",
    "- $\\Delta_{Cohen}\\leq0.5$ is considered a small effect\n",
    "- $\\Delta_{Cohen}\\approx1$ is considered a moderate effect\n",
    "- $\\Delta_{Cohen}\\leq1.5$ is considered a large effect\n",
    "\n",
    "If we assume a moderate effect this will give us a sample size of 16\n",
    "\n",
    "If we are expecting a large effect the sample size will be 7 (7.11)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "659bb856",
   "metadata": {},
   "source": [
    "In most lab experiments, as opposed to clinical studies, cost and time will usually be the limiting factors for sample size.\n",
    "\n",
    "Lab experiments are usually not repeated more than 10 times\n",
    "\n",
    "In some cases with low sample size, the type 2 error rate (risk for false negative) becomes very large"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "220f113e",
   "metadata": {},
   "source": [
    "For an example, go to www.biomath.info/power/\n",
    "- select \"two groups\"\n",
    "- select \"t-test on groups means\"\n",
    "- press \"Go\"\n",
    "- find sample size: enter diff between means: 4, and SD: 8\n",
    "- press \"result\"\n",
    "\n",
    "You should obtain sample size 64"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "87a4cc8d",
   "metadata": {},
   "source": [
    "## Lecture 4 - ANOVA"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5321f4d1",
   "metadata": {},
   "source": [
    "Analysis of variance.\n",
    "\n",
    "**One-way** means it involves only **one factor**.\n",
    "\n",
    "ANOVA can compare the means of **3 or more** groups.\n",
    "\n",
    "(If you use ANOVA to compare the means of two groups, you will get the same results as with a t-test)\n",
    "\n",
    "The *independent variable* (the factor) should have a categorical scale."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e4824062",
   "metadata": {},
   "source": [
    "The **f-statistic** computed by ANOVA is the square of the t-statistic: $F=t²$\n",
    "\n",
    "<img src=\"f_dist.png\" alt=\"pic failed\" width=\"500\" height=\"250\">"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "76944abf",
   "metadata": {},
   "source": [
    "ANOVA can only test a **two-sided** hypothesis, since all values in the dist are positive.\n",
    "\n",
    "T-tests can do both one- and two-sided\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ca3ef8d7",
   "metadata": {},
   "source": [
    "If we use t-tests to compare three or more groups, the type 1 error rate increases (risk of false positive = rejecting a true H0)\n",
    "\n",
    "For every test there is a 5% chance, the chances compound with multiple tests\n",
    "\n",
    "Risk of type 1 error = $1-(1-\\alpha)²$\n",
    "- k = nr of comparisons\n",
    "- $\\alpha$ = significance level\n",
    "\n",
    "ANOVA compares all means in one go and retains 5% type 1 error risk\n",
    "\n",
    "It is also a lot easier to run and interpret one test"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1713cf4c",
   "metadata": {},
   "source": [
    "### How ANOVA works"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d56ef4ba",
   "metadata": {},
   "source": [
    "H0: the population means are equal for all groups\n",
    "\n",
    "H1: not all popolation means are equal\n",
    "\n",
    "Example:\n",
    "- H0: μ1 = μ2 = μ3\n",
    "- H1: at least one mean differ from the others"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "facffbe0",
   "metadata": {},
   "source": [
    "Assumptions for ANOVA:\n",
    "- dependent variable normally dist within each group: the observations of each group should be normal\n",
    "- homogeneity of variance: the variance of each group should be the same (test by Levene's test)\n",
    "- alternatively, the SD of all groups are approx equal. *Rule of thumb*: ratio of largest SD to smallest SD should not be >2\n",
    "- independent observations"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "89dd3d97",
   "metadata": {},
   "source": [
    "### The F-ratio\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9eac9411",
   "metadata": {},
   "source": [
    "Anova calculates the f-ratio/f-statistic: $F=\\frac{\\text{variation between groups}}{\\text{variation within groups}}$\n",
    "\n",
    "Algorithm:\n",
    "- calculate the means within the groups\n",
    "- calculate the **grand mean** = the mean of *all* observations\n",
    "- estimate the **variation between groups** (distance between grand mean and groups means)\n",
    "- calculate the pooled **variation within the groups** = the distance between data points and their respecitve mean\n",
    "\n",
    "\n",
    "<img src=\"f_ratio.png\" alt=\"pic failed\" width=\"500\" height=\"250\">"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6584f880",
   "metadata": {},
   "source": [
    "We will reject H0 of anova only if variation between groups $\\geq$ variation within groups.\n",
    "- if F $\\approx$ 1, we dont reject H0\n",
    "\n",
    "If we reject the H0, it means there is a difference between at least one of the means.\n",
    "- we can do a post-hoc test to see which one\n",
    "- Ex by doing 3 separate t-tests\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bf3a7263",
   "metadata": {},
   "source": [
    "### One-way ANOVA calculations"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1ba59fd6",
   "metadata": {},
   "source": [
    "The sum of squared distances:\n",
    "\n",
    "The sum of squares within the groups = **Error sum of squares SSE**\n",
    "- summing the squares of the differences in all groups to their respective means\n",
    "- $SSE=\\sum_{i=1}^k \\sum_{j=1}^{n_i} (y_{ij}-\\bar{y}_i)²$\n",
    "- $y_{ij}$ = the data points j in each group i\n",
    "- $\\bar{y}_i$ = the means of each group\n",
    "- k= number of groups\n",
    "- $n_i$ = the number of observations in the groups\n",
    "- $\\sum\\sum$ means double summation aka nested summation: used when there are two indeces\n",
    "\n",
    "The sum of squares between the groups = **SSB**\n",
    "- the sum of (group sample size multiplied by squared difference between group mean and grand mean)\n",
    "- $SSB=\\sum_{i=1}^k n_i(\\bar{y}_i-\\bar{y})²$\n",
    "- $n_i$ = number of observations in groups\n",
    "- $\\bar{y}_i$ = group means\n",
    "- $\\bar{y}$ = grand mean\n",
    "- k = number of groups\n",
    "\n",
    "The total sum of squares: **SST** = SSB + SSE\n",
    "- SST is also the sum of the sqared differences between each data point and the grand mean\n",
    "- $SST=\\sum_{i=1}^N (y_i-\\bar{y})²$\n",
    "- $y_i$ = data points\n",
    "- $\\bar{y}$ = grand mean\n",
    "- N = total number of data points $\\sum_{i=1}^kn$, k = number of groups\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7920043c",
   "metadata": {},
   "source": [
    "The *df between groups* is the number of groups minus one or k-1\n",
    "\n",
    "The *df within groups* is the total number of data points minus the number of groups or N-k\n",
    "\n",
    "The *df for the SST* is the total number of data points minus one or N-1"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "06976e7e",
   "metadata": {},
   "source": [
    "**Mean square** = the sum of squares (SS) divided by corresponding df\n",
    "\n",
    "MSB = mean square between groups\n",
    "\n",
    "MSE = mean square of error (within groups)\n",
    "\n",
    "F-ratio = $\\frac{MSB}{MSE} = \\frac{\\text{variation between groups}}{\\text{variation within groups}}$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6e2cd6d1",
   "metadata": {},
   "source": [
    "The f-distribution to be used corresponds to the degrees of freedom of the MSB and MSE.\n",
    "\n",
    "Use the critical value of this f-distribution to find the p value."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3a3b4449",
   "metadata": {},
   "source": [
    "ANOVA table\n",
    "\n",
    "<img src=\"anova_table.png\" alt=\"pic failed\" width=\"500\" height=\"250\">"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "909dcbf3",
   "metadata": {},
   "source": [
    "## POST-HOC tests / multiple comparison tests"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ad47cd1b",
   "metadata": {},
   "source": [
    "aka **a posterior** tests\n",
    "\n",
    "**Fisher’s Least Significant Difference Test (LSD)**\n",
    "- the simplest test for multiple comparisons\n",
    "- Basically a set of pairwise t-tests.\n",
    "- uses the pooled variance from all groups (t-test uses pooled var from only two groups)\n",
    "- uses df N-k (t-test uses $n_1+n_2-2$), k= total nr of groups \n",
    "- does not correct for multiple comparisons but can be used for comparing three groups only\n",
    "- needs something like bonferroni afterwards (with 3 groups or above 3 groups?)\n",
    "\n",
    "When we have more than 3 groups, we should use a post-hoc test that corrects for multiple comparisons.\n",
    "- Ex **Bonferroni** method or \n",
    "- **Tukey’s test**: compares the mean of each group with the means of the other groups\n",
    "    - probably the most common\n",
    "    - based on a studentized range distribution (q)\n",
    "    - $q_s=\\frac{Y_a-Y_b}{SE}$\n",
    "    - $Q_s$ = value to be compared to the critical q-value\n",
    "    - $Y-a$ = the larger of the two means compared\n",
    "    - $Y-b$ = the smaller of the two means compared\n",
    "    - SE = standard error of the data in question\n",
    "- **Dunnett´s test**: compares the mean of each group with the mean of a control group.\n",
    "- **Holm´s test**: compares all pairs of means. Powerful, but cannot compute a set of CIs.\n",
    "\n",
    "Post-hoc tests have been created to deal with the problems when we make several comparisons.\n",
    "\n",
    "They are commonly applied after a significant ANOVA result.\n",
    "\n",
    "If we dont reject the H0, there is no difference and thus no reason to do a post-hoc test.\n",
    "\n",
    "Most post-hoc tests keep the **family-wise error rate (FWER)** at 0.05 even though we make several comparisons.\n",
    "\n",
    "**Bonferroni method** is the simplest method to adjust for multiple comparisons.\n",
    "- Multiply the p-values by the number of test we do\n",
    "- bonferroni should only be used when you make few comparisons\n",
    "\n",
    "A posteriori tests are generally **less powerful** than ANOVA.\n",
    "- Sometimes an ANOVA may report a significant result, especially one that is just significant (e.g. p = 0.04), but subsequent post-hoc testing cannot distinguish between any treatments/samples.\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "11ca3429",
   "metadata": {},
   "source": [
    "### Reporting ANOVA results\n",
    "\n",
    "for example:\n",
    "\n",
    "<img src=\"anova_results.png\" alt=\"pic failed\" width=\"500\" height=\"250\">"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8d7b2d63",
   "metadata": {},
   "source": [
    "## Repeated measures ANOVA"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a74b8e3a",
   "metadata": {},
   "source": [
    "Can be seen as a version of the paired t-test when we have more than 2 measurements.\n",
    "\n",
    "Ex. before treatment, after two weeks, after four weeks\n",
    "\n",
    "In repeated measures anova:\n",
    "\n",
    "$F=\\frac{\\text{variance between groups}}{\\text{variance in error}}$\n",
    "\n",
    "The error variability is the variation within the groups, when the variation between subjects have been removed.\n",
    "\n",
    "So we are removing the variation between the test subjects and only keeping the variation each subject has in themselves\n",
    "\n",
    "We are removing the differences between subjects.\n",
    "\n",
    "If we remove the difference between the subjects in a group from the sum of squares within the group, we’re left with only the variance from measurement to measurment of the same individual.\n",
    "\n",
    "One-way ANOVA: total variation = between group variation + within group variation\n",
    "\n",
    "Repeated-measures ANOVA: total variation = between group variation + (within group variation - between subject variation)\n",
    "\n",
    "The df of the variance in error is the df of the total within group variance minus the df for the subjects."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4249e508",
   "metadata": {},
   "source": [
    "calculate the between groups variation = (sum of squares of time points)\n",
    "- df = k-1, k= nr of groups\n",
    "- $SSB=\\sum_{i=1}^k n_i(\\bar{y}_i-\\bar{y})²$\n",
    "- $n_i$ = number of observations in groups\n",
    "- $\\bar{y}_i$ = group means\n",
    "- $\\bar{y}$ = grand mean\n",
    "- k = number of groups\n",
    "- this is the same as in one-way anova\n",
    "    \n",
    "calculate the total sum of squares\n",
    "- df N-1, N = total nr of measurements\n",
    "- $SST=\\sum_{i=1}^N (y_i-\\bar{y})²$\n",
    "- $y_i$ = data points\n",
    "- $\\bar{y}$ = grand mean\n",
    "- N = total number of data points $\\sum_{i=1}^kn$, k = number of groups\n",
    "- this is the same as in one-way anova\n",
    "\n",
    "calculate the sum of squares of the subjects (variation between individuals)\n",
    "- df = n-1\n",
    "- $SS_{\\text{subject}}= n\\cdot \\sum_{i=1}^k (\\bar{X}_i-\\bar{\\bar{X}})²$\n",
    "- $SS_{\\text{subject}}$ = sum of squares for the subject variation (subject effect)\n",
    "- n = number of subjects\n",
    "- k = number of conditions (ex time points)\n",
    "- $\\bar{X}_i$ = the mean of the measurements for subject i across all time points\n",
    "- $\\bar{\\bar{X}}$ = the grand mean = the mean of all measurements across all subjects and timepoints\n",
    "\n",
    "then calculate the remainder \n",
    "- df = (k-1)(n-1)\n",
    "- remainder = SST - SSB - $SS_{\\text{subject}}$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5da3debf",
   "metadata": {},
   "source": [
    "The f-ratio to use = $\\frac{\\text{mean square}_{\\text{group}}}{\\text{mean square}_{\\text{remainder}}}$\n",
    "\n",
    "mean square = sum of squares divided by df\n",
    "\n",
    "\n",
    "<img src=\"repeated_measures_table.png\" alt=\"pic failed\" width=\"500\" height=\"250\">"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "23e111e1",
   "metadata": {},
   "source": [
    "H0: $\\mu_1=\\mu_2=\\mu_3=...=\\mu_k$\n",
    "\n",
    "H1: not all means are equal"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "94c510ec",
   "metadata": {},
   "source": [
    "Assumptions for repeated measures anova:\n",
    "- **Sphericity**: the variances of the differences between the groups should be equal.\n",
    "- Can be tested with **Mauchly’s test of sphericity**\n",
    "    - H0: sphericity (all equal)\n",
    "\t- H1: no sphericity"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "30a7fa5b",
   "metadata": {},
   "source": [
    "### Post hoc testing"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fbbec34d",
   "metadata": {},
   "source": [
    "A post-hoc analysis following a repeated-measures ANOVA works just as a set of paired t-tests. The p-\n",
    "values can for example be corrected with Bonferroni.\n",
    "\n",
    "<img src=\"report_repeated_anova.png\" alt=\"pic failed\" width=\"500\" height=\"250\">"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4fa08431",
   "metadata": {},
   "source": [
    "## Lecture 5 - non-parametric tests\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ae812a4a",
   "metadata": {},
   "source": [
    "Non-parametric tests are used with non-normal or skewed data.\n",
    "\n",
    "They make inferences about the population medians from the samples.\n",
    "\n",
    "Non-parametric tests do not require any specific distribution.\n",
    "\n",
    "Parametric tests cannot be used on nominal or ordinal data - non-parametric can.\n",
    "\n",
    "Non-parametric test have less power - always use parametric tests if possible.\n",
    "\n",
    "Non-parametric tests are robust against outliers.\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1b19481c",
   "metadata": {},
   "source": [
    "### Non-parametric confidence interval\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b9d8ece1",
   "metadata": {},
   "source": [
    "To calculate a 95% non-parametric confidence interval based on the median, we just\n",
    "sort the values and pick the values that correspond to the i1st and i2nd values as our min\n",
    "and max limits of the confidence interval:\n",
    "\n",
    "Sort the values.\n",
    "\n",
    "Then calculate which indexes (which number) will be the confidence intervals.\n",
    "\n",
    "$i1=\\frac{n}{2}-\\frac{1.96\\sqrt{n}}{2}$\n",
    "\n",
    "$i2=1+\\frac{n}{2}+\\frac{1.96\\sqrt{n}}{2}$\n",
    "\n",
    "Lets say we end up with i1=6 and i2=15 (rounded to nearest digit):\n",
    "\n",
    "We then pick index numbers [6] and [15] from our dataset, for example [6]=0.09 and [15]=2.0\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "690de586",
   "metadata": {},
   "source": [
    "### Mann-whitney U test"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bec354f5",
   "metadata": {},
   "source": [
    "aka Wilcoxon Mann-Whitney test, aka Wilcoxon rank sum test.\n",
    "\n",
    "Corresponding non-parametric test to the **independent t-test**\n",
    "\n",
    "In contrast to the t-test, can also be used on ordinal data.\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1579eb41",
   "metadata": {},
   "source": [
    "**Requirements**:\n",
    "- at least ordinal scale\n",
    "- two independent samples\n",
    "\n",
    "**Hypothesis**:\n",
    "- H0: $P(A>B)=0.5$ = the distribution of the data in the two groups is equal\n",
    "- H1: $P(A>B)\\neq0.5$ = the distribution of the data in the two groups is not equal\n",
    "\n",
    "<img src=\"manu_hypotheses.png\" alt=\"pic failed\" width=\"500\" height=\"250\">\n",
    "\n",
    "**Method**:\n",
    "- Rank both samples, and sum the ranks in the smallest sample. \n",
    "- This rank sum is then compared with values in a table."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4aeb3f55",
   "metadata": {},
   "source": [
    "\n",
    "**Example**:\n",
    "- Control group = [118 120 121 122 123]\n",
    "- Treatment group = [110 112 114 116 118]\n",
    "\n",
    "- 𝐻0 : there will be no difference between the drug and the control group\n",
    "- 𝐻1 : there will be a difference between the drug and the control group\n",
    "\n",
    "The critical significance level is chosen to be 0.05.\n",
    "\n",
    "**First, we rank all the data**:\n",
    "\n",
    "|Control group:|120|118|121|122|123|\n",
    "|---------------|------------|------------|------------|---------------|-----------|\n",
    "|Rank:|7|5.5|8|9|10|\n",
    "|Treatment group:|110|112|114|116|118|\n",
    "|Rank:|1|2|3|4|5.5|\n",
    "\n",
    "**Then we sum the ranks of the smallest sample (by n value)** (in this case both samples are the same size so it does not matter which group we select). E.g. sum the ranks of the treatment group:\n",
    "- W = 1 + 2 + 3 + 4 + 5.5= 14.5\n",
    "\n",
    "The sum of ranks of the treatment group is 14.5, which is **outside the critical values** in the Mann-Whitney table **[19 36]**. \n",
    "- Thus, we **reject the null hypothesis**.\n",
    "\n",
    "The sum of ranks of the control group is 40, which is also outside the critical interval [19 36].\n",
    "\n",
    "With help of statistical software, the p-value was calculated to 0.016.\n",
    " \n",
    "The p-value (0.016) is less than our significance level (α=0.05). Hence, we reject the null hypothesis.\n",
    "\n",
    "There is a significant difference.\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5a4fdf30",
   "metadata": {},
   "source": [
    "For larger sample sizes (n>10), we can use the following expression that is approximately normally distributed for moderate to large sample sizes:\n",
    "\n",
    "$Z=\\frac{W-n_1(n_1+n_2+1)/2}{\\sqrt{n_1n_2(n_1+n_2+1)/12}}$\n",
    "\n",
    "W = sum of ranks for the smaller sample (smaller n)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0092e4b7",
   "metadata": {},
   "source": [
    "The p-values for non-parametric tests are usually higher than for parametric tests if the data is normally distributed."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8665bba9",
   "metadata": {},
   "source": [
    "We can also use the Mann-Whitney U test if we want to compare two independent groups when the data has been measured on an **ordinal scale** (t-test require at least interval scale).\n",
    "\n",
    "Example: \n",
    "- 1 = no pain \n",
    "- 2 = mild pain\n",
    "- 3 = moderate pain\n",
    "- 4 = severe pain \n",
    "- 5 = very severe pain\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4bc4572d",
   "metadata": {},
   "source": [
    "### Wilcoxon signed-rank test"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2b4e624f",
   "metadata": {},
   "source": [
    "Wilcoxon signed-rank test can be used as an **alternative to the paired t-test**, when the difference between the pairs cannot be assumed to be normally distributed.\n",
    "\n",
    "**Requirements**:\n",
    "- Two dependent samples\n",
    "- Data on at least ordinal scale\n",
    "- Difference between pairs is **symmetrically distributed** around the median (if not, use Sign test)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "316e9b78",
   "metadata": {},
   "source": [
    "**Hypothesis**:\n",
    "- H0: η=0, the median of the differences between the paired values is equal to zero\n",
    "- H1: η ≠ 0, the median of the differences between the paired values is not equal to zero"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1377e543",
   "metadata": {},
   "source": [
    "<img src=\"wilcox_table.png\" alt=\"pic failed\" width=\"500\" height=\"250\">"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0befe2d8",
   "metadata": {},
   "source": [
    "Sum of the ranks with a\n",
    "- positive difference W+ = 3 + 1 + 6 + 5 + 3 = 18\n",
    "- negative difference W- = 3\n",
    "- The **smallest value of W+ and W-** is called the T score, i.e. T = 3.\n",
    "- If T score < critical value, the null hypothesis is rejected."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ee2399b1",
   "metadata": {},
   "source": [
    "If n>10, the Wilcoxon signed ranks statistics approximately follow the Z distribution:\n",
    "\n",
    "$Z=\\frac{T-\\frac{n(n+1)}{4}}{\\sqrt{\\frac{n(n+1)(2n+1)}{24}}}$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cb17b123",
   "metadata": {},
   "source": [
    "### Sign test"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d7c2757d",
   "metadata": {},
   "source": [
    "The sign test is similar to “Wilcoxon signed-rank test”, i.e. is used on paired samples. \n",
    "\n",
    "Use when the distribution of differences between pairs is **neither normal nor symmetrical**.\n",
    "\n",
    "**Requirements**:\n",
    "- Two dependent samples\n",
    "- Data on at least ordinal scale\n",
    "\n",
    "**Hypothesis**:\n",
    "- H0: the number of positive differences is equal to the number of negative differences\n",
    "- H1: the number of positive differences is not equal to the number of negative differences"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e26a2199",
   "metadata": {},
   "source": [
    "**Example**:\n",
    "\n",
    "<img src=\"sign_table.png\" alt=\"pic failed\" width=\"500\" height=\"250\">\n",
    "\n",
    "Sum of\n",
    "- positive signs W+ = 6\n",
    "- negative signs W- = 1\n",
    "\n",
    "The test statistic for the Sign test is the smallest number of signs of either the positive or negative signs.\n",
    "\n",
    "T= 1 (we had only one difference with a negative sign).\n",
    "\n",
    "If T score is greater than critical value (for example 0), we cannot reject the null hypothesis. \n",
    "\n",
    "Using statistical software, the p-value is calculated."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "999b4111",
   "metadata": {},
   "source": [
    "The p-value is the probability that the observed difference (or one more extreme) is due to chance, if the null hypothesis is true.\n",
    "\n",
    "**Example**: A t-test results in a p-value of 0.03\n",
    "- There is a 3% risk that the observed difference (or greater difference) is due to chance if the population means are in fact identical."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f2124f60",
   "metadata": {},
   "source": [
    "<img src=\"p_table.png\" alt=\"pic failed\" width=\"500\" height=\"250\">\n",
    "\n",
    "Important! Although your test result in a p-value of 0.07 and you do NOT reject the null hypothesis, it does not mean that there is no difference. Your sample size might simply have been to small (you might have committed a type II error)."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "83e2a78c",
   "metadata": {},
   "source": [
    "#### STATISTICAL SIGNIFICANCE ≠ SCIENTIFIC IMPORTANCE\n",
    "\n",
    "If a difference is not statistically significant, it could still be real and intriguing enough to be worthy of further investigation.\n",
    "\n",
    "”Not significant” does not imply that there is no effect. It means that we have failed to demonstrate the existence of one.\n",
    "\n",
    "If a difference is statistically significant, then it may be real but not large enough to be interesting.\n",
    "\n",
    "Do not overemphasize statistical significance – look at the size of the effects!"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1eecc339",
   "metadata": {},
   "source": [
    "### Outliers"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "17e774d0",
   "metadata": {},
   "source": [
    "In statistics, an outlier is an observation point that is distant from other observations.\n",
    "\n",
    "An outlier may be due to **natural variation** or it may indicate **an error**. \n",
    "- An outlier due to an error is usually excluded from the data set.\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "efa1040d",
   "metadata": {},
   "source": [
    "There are many different methods to detect outliers. \n",
    "\n",
    "One method that is used in box plots to mark an outlier is if the data point is:\n",
    "\n",
    "$x>Q3+1.5\\cdot\\text{IQR}$ or $x<Q1-1.5\\cdot\\text{IQR}$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f8f4d573",
   "metadata": {},
   "source": [
    "An outlier can simply be due to the variability:\n",
    "\n",
    "By chance our study with 30 randomly selected men happened to include a long man (although this is very unlikely)\n",
    "\n",
    "If the outlier test identifies one or more values as outlier, you must decide whether to exclude or keep it. The proper action depends on what causes the outliers. \n",
    "\n",
    "Broadly, there are three causes for outliers:\n",
    "- Data entry or measurement **error** → correct the error if possible, otherwise remove it because you know it’s incorrect.\n",
    "- Sampling **problems** and unusual conditions → the collected items falls outside the population you are studying, you can legitimately remove it.\n",
    "- **Natural variations** → extreme values can occur as a natural part of the population you are studying, nothing is wrong, keep it.\n",
    "\n",
    "When you decide to remove outliers, document the excluded data points and explain your reasoning."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6760ec01",
   "metadata": {},
   "source": [
    "### Robustness"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2ccad65d",
   "metadata": {},
   "source": [
    "Another alternative to deal with outliers is to use **non-parametric tests** as these tests are robust against extreme values."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a63e0754",
   "metadata": {},
   "source": [
    "Example: If one patient in our previous example had a blood pressure of 180 (the true value is 155), it still get rank 20 (it is still the highest measurement) and will have no effect on the results:\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "93c679eb",
   "metadata": {},
   "source": [
    "### Transformation of data"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dbaaae62",
   "metadata": {},
   "source": [
    "It is quite common that biological data have a log-normal distribution\n",
    "\n",
    "<img src=\"log_dist.png\" alt=\"pic failed\" width=\"500\" height=\"250\">\n",
    "\n",
    "**Examples**:\n",
    "- Concentration of antibodies in 30 individuals\n",
    "- Most individuals will have a low IgG concentration but a few will have an ongoing infection and will thus have high level of IgG concentration.\n",
    "- Expression of 30 different genes in a cell\n",
    "- Most genes will be turned of in a cell, whereas some genes are turned on."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d8d349a9",
   "metadata": {},
   "source": [
    "If the data has a log-normal distribution, we should not use parametric test. However, we can try to transform the data to become normally distributed.\n",
    "\n",
    "If the transformed data show normal distribution, we can use the transformed values to compute e.g. a t-test.\n",
    "\n",
    "For the descriptive statistics, you should always use the raw, untransformed data."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2f37432b",
   "metadata": {},
   "source": [
    "## Lecture 6 - correlation and regression"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9c437dd2",
   "metadata": {},
   "source": [
    "### Correlation"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bb398f66",
   "metadata": {},
   "source": [
    "Correlation is a measure that expresses the extent to which two variables are **linearly related**, meaning that the values of both variables change together in a consistent way.\n",
    "\n",
    "Positive correlation: When the value of one variable increases then the value of the other variable increases\n",
    "\n",
    "Negative correlation: When the value of one variable increases then the value of the other variable decreases\n",
    "\n",
    "NOTE! Correlation between variables does not imply causality."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9043c5a9",
   "metadata": {},
   "source": [
    "The **population correlation coefficient ρ (rho)** is a measure of the linear relationship between two variables.\n",
    "\n",
    "The **sample correlation coefficient r (Pearson product-moment correlation coefficient)** is an estimate of ρ and is used to measure the linear relationship between two variables in the sample observations.\n",
    "\n",
    "$r=\\frac{\\sum_{i=1}^n\\left(x_i-\\bar{x}\\right)\\left(y_i-\\bar{y}\\right)}{\\sqrt{\\sum_{i=1}^n\\left(x_i-\\bar{x}\\right)^2 \\sum_{i=1}^n\\left(y_i-\\bar{y}\\right)^2}}$\n",
    "\n",
    "The **correlation coefficient** ranges between -1 and 1.\n",
    "- The closer to 1, the stronger the positive linear relationship\n",
    "- The closer to -1, the stronger the negative linear relationship\n",
    "- The closer to 0, the weaker the linear relationship\n",
    "\n",
    "If r is near one (plus or minus), it means that our observations lie close to the regression line.\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3cde9603",
   "metadata": {},
   "source": [
    "Significance level: α = 0.05\n",
    "\n",
    "- H0: ρ=0, no correlation between the variables\n",
    "- H1: ρ≠0\n",
    "\n",
    "$t_{o b s}=\\frac{r}{\\sqrt{\\left(1-r^2\\right) /(n-2)}}$\n",
    "\n",
    "If you use a t-table, the degree of freedom is df = n – 2.\n",
    "\n",
    "As usual, if this p-value is less than the significance level, then we can reject the null hypothesis. We then conclude that the correlation is significant."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0af24937",
   "metadata": {},
   "source": [
    "**Assumptions for Pearson’s correlation – parametric test**\n",
    "- Both variables are continuous (ratio or interval scale)\n",
    "- Variables are normally distributed\n",
    "- Linear relationship between the variables\n",
    "\n",
    "**Assumptions for Spearman’s rank correlation – non-parametric test**\n",
    "- Both variables are on at least ordinal scale\n",
    "- Monotonic relationship, i.e., if and only if the variable is increasing or decreasing, so does the value of the other variable\n",
    "- Meaning it does not need to be linear"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3ae4976f",
   "metadata": {},
   "source": [
    "### Spearmans rho correlation coefficient"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "eb0e68c3",
   "metadata": {},
   "source": [
    "\n",
    "Similar to other non-parametric tests, Spearman’s test is based on ranks.\n",
    "\n",
    "<img src=\"spearman_positive.png\" alt=\"pic failed\" width=\"500\" height=\"250\">"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e2d14f3f",
   "metadata": {},
   "source": [
    "$r_s=1-\\frac{6 \\cdot \\sum_{i=1}^n d_i^2}{n\\left(n^2-1\\right)}$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2919e198",
   "metadata": {},
   "source": [
    "### Regression"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4d7ec38b",
   "metadata": {},
   "source": [
    "A relationship between two variables can be **further examined** by regression analysis.\n",
    "\n",
    "**Regression is a general term for models where the value of a dependent variable is explained by one or more explanatory variables.**\n",
    "\n",
    "The most common form of regression is linear regression where the relationship between the variables is assumed to be linear.\n",
    "\n",
    "**Simple linear regression**: one explanatory variable\n",
    "\n",
    "**Multiple linear regression**: more than one explanatory variable"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "172ce158",
   "metadata": {},
   "source": [
    "### Simple linear regression"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5e9b6cc5",
   "metadata": {},
   "source": [
    "The aim of linear regression is to **fit a line to the data points**.\n",
    "\n",
    "Simple linear regression analysis describes the linear relationship between a dependent and an independent variable.\n",
    "\n",
    "To analyse a relationship between two continuous variables, we should always start with a scatter plot (to visually estimate the relation)\n",
    "\n",
    "Simple linear regression expresses the relationship as a **linear equation**: $y=\\beta_0+\\beta_1x$\n",
    "\n",
    "β0 = the intercept of the y-axis\n",
    "\n",
    "β1 = the slope of the line (regression coefficient)\n",
    "\n",
    "<img src=\"linear_equations.png\" alt=\"pic failed\" width=\"800\" height=\"150\">"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6c4e2032",
   "metadata": {},
   "source": [
    "y is the dependent variable (also called outcome or response variable)\n",
    "\n",
    "x is the independent variable (also called explanatory variable, predictor or covariate)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6029d6a2",
   "metadata": {},
   "source": [
    "The Greek letters are used to show that β0 and β1 are population parameters. For samples, the equation is usually expressed as\n",
    "\n",
    "$y=b_0+b_1x$\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5f4c875b",
   "metadata": {},
   "source": [
    "Regression finds the line that best fits the observations.\n",
    "\n",
    "Regression analysis, using **ordinary least squares (OLS)**, finds the line that results in the lowest sum of squared errors: \n",
    "- SSE or $\\sum e²$\n",
    "\n",
    "The distance between the observations and the estimated line are called residuals ($e_i$):\n",
    "- $e = y_i − \\hat{y}_i$\n",
    "\n",
    "The distance is measured only on the y-axis"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "26a7be40",
   "metadata": {},
   "source": [
    "We calculate these things with software\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1baf09e9",
   "metadata": {},
   "source": [
    "Simple linear regression is mainly used for two purposes:\n",
    "- Fit a model, which can be used to **predict the value of y**\n",
    "- E.g. if we know the body height, we can predict the weight.\n",
    "- Or to make a standard curve to get concentration from absorbance in spectrophotometry \n",
    "\n",
    "- **Hypothesis testing**, to make inference about the slope.\n",
    "- E.g. is there a significant positive linear relationship between height and weight (is the weight associated to the length)? This is similar to the Person’s correlation.\n",
    "- H0: no relationship exist (β1=0) = Slope is equal to zero.\n",
    "- H1: there is a relationship (β1≠0)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f4ed4f4d",
   "metadata": {},
   "source": [
    "SPSS/R will generate the **R2 value**, which is a measure (between zero and one) of how well the model explains the data.\n",
    "\n",
    "R2 value of 0.5 = 50% of the variance in the observed values of the dependent variable is explained by the model\n",
    "\n",
    "- $R²=1-\\frac{\\text{SSE}}{\\text{SST}}$\n",
    "- $\\text{SSE}=\\sum (y_i-\\hat{y})²$, $\\hat{y}=$ regression line\n",
    "- $\\text{SST}=\\sum (y_i-\\bar{y})²$, $\\bar{y}$= horizontal line at the mean of all y values\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "501a5951",
   "metadata": {},
   "source": [
    "**Assumptions**:\n",
    "- **Linearity**: Linear relationship between dependent and independent variables\n",
    "- **Homoscedasticity**: Constant variance of the residuals (differences)\n",
    "- **Normality**: Residuals (differences) are normally distributed\n",
    "- **Independence**: Residuals are independent (the value of one residual should not provide information or influence the value of another residual) Such errors usually occur in time series regression models. E.g. data may be collected during winter and summer times.\n",
    "\n",
    "If not linear: Fix by using a non-linear function (non-linear regression) or non-linear transformation.\n",
    "\n",
    "If not homoscedastic: Can be fixed by transformation of the data\n",
    "\n",
    "If not normal: check outliers and/or transform the data.\n",
    "\n",
    "If not independent: Fix by e.g. seasonally adjust the data or include a dummy variable and use multiple regression.\n",
    "\n",
    "SPSS/R have tools to identify outliers. Follow the general steps for removing an outlier."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2da92548",
   "metadata": {},
   "source": [
    "**Similarities** between **Pearson’s correlation and regression**:\n",
    "- Both methods will result in the same p-value\n",
    "- $r^2=R^2$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "43fef615",
   "metadata": {},
   "source": [
    "**Differences** between **Pearson’s correlation and regression**:\n",
    "- The aim is not to fit a model with correlation, just to analyse linear relationship between two variables.\n",
    "- The variables do not have to be defined as dependent and independent in correlation analysis. Hence, it does not matter which variable you put on the y-axis.\n",
    "- Inference about the correlation, requires that both x and y are measured, and both are sampled from Gaussian distributions, whereas inference about the regression line requires that the residuals follow a Gaussian distribution.\n",
    "- Hypothesis testing\n",
    "    - Correlation: H0: ρ=0 (no linear relationship between variables)\n",
    "    - Regression: H0: β1=0 (the slope of the regression line is equal to zero )"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "26cd5089",
   "metadata": {},
   "source": [
    "**Example**: \n",
    "\n",
    "Use correlation if you are only interested if cholesterol correlate with exercise. If you also want to estimate how much the exercise reduce cholesterol (e.g. 10 hours of exercise per week reduce the cholesterol level with 0.064 mmol/L), use regression."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "348a7d9c",
   "metadata": {},
   "source": [
    "### Multiple linear regression"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3e392c34",
   "metadata": {},
   "source": [
    "Most dependent variables are influenced by more than one factor.\n",
    "\n",
    "When we have **more than one covariate** (more than one explanatory variable) on the right hand side of the model, we are dealing with multiple (linear) regression. Note that we may also include categorical variables such as smokers (YES/NO).\n",
    "\n",
    "<img src=\"multiple_linear_equation.png\" alt=\"pic failed\" width=\"800\" height=\"150\">"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6be07a27",
   "metadata": {},
   "source": [
    "For a multiple linear relationship, **ANOVA** tests the hypothesis:\n",
    "- H0: $\\beta_1 = \\beta_2 = \\beta_3 = \\beta_4 =0$\n",
    "- H1: at least one $\\beta \\neq$ 0\n",
    "\n",
    "By computing the F-statistic:\n",
    "- $F=\\frac{(\\text{RSSE}-\\text{SSE}/q)}{\\text{SSE}/(N-k)}$\n",
    "- df=q, N-k\n",
    "- RSSE = Squared sum of residuals for the **simple model** ($y=\\beta_0$) (all coefficients =0)\n",
    "- SSE = Squared sum of residuals for the proposed model (the multiple linear equation)\n",
    "- q = Difference in number of coefficients in the simple vs. proposed model\n",
    "- N = number of observations\n",
    "- k = Number of coefficients in the proposed model including the intercept\n",
    "\n",
    "A significant p-value doesn’t mean that all explanatory variables have a significant influence on y —only that **at least one** does."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2a8c064a",
   "metadata": {},
   "source": [
    "If the F-test tells us that we can reject H0, at least one of the coefficients are ≠ 0. We can now look at each coefficient\n",
    "\n",
    "<img src=\"multiple_table.png\" alt=\"pic failed\" width=\"800\" height=\"150\">\n",
    "\n",
    "Since the variable “Diet” does not seem to influence cholesterol levels, we might simply remove “Diet” from the model.\n",
    "\n",
    "Note that a **bigger model will fit the data better than a simpler one**. We can run an F-test to see if the bigger model is statistically significant better than the smaller one. A model including less variables is much easier to interpret.\n",
    "\n",
    "**Interpretaions**: The coefficient value for “Exercise” is: -0.05. This means that the cholesterol level is reduced with 0.05 mmol/L per unit exercise. If this unit is hours exercise per week, we can predict that one hour of exercise per week reduce the cholesterol level with 0.05 mmol/L on average.\n",
    "\n",
    "Smoker was a categorical variable (YES/NO). How do we interpret b3, since smoking have no unit?\n",
    "- If b3 = 0.1, it means that smokers have a mean value of cholesterol 0.1 mmol/L higher than non-smokers.\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "R",
   "language": "R",
   "name": "ir"
  },
  "language_info": {
   "codemirror_mode": "r",
   "file_extension": ".r",
   "mimetype": "text/x-r-source",
   "name": "R",
   "pygments_lexer": "r",
   "version": "4.1.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}