Appendix B Linear Algebra.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Add lib input sys.path\n",
    "import os\n",
    "import sys\n",
    "import time\n",
    "\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import sklearn\n",
    "import matplotlib.pyplot as plt\n",
    "from scipy.optimize import minimize\n",
    "import math\n",
    "from sklearn.preprocessing import normalize\n",
    "from functools import partial\n",
    "import h5py\n",
    "from scipy.spatial import distance\n",
    "\n",
    "nb_dir = os.path.split(os.getcwd())[0]\n",
    "if nb_dir not in sys.path:\n",
    "    sys.path.append(nb_dir)\n",
    "\n",
    "from matplotlib.colors import ListedColormap\n",
    "import libs.linear_models as lm\n",
    "import libs.data_util as data\n",
    "import libs.nn as nn\n",
    "import libs.plot as myplot\n",
    "from sklearn import decomposition\n",
    "from sklearn.model_selection import train_test_split\n",
    "from scipy.optimize import minimize_scalar\n",
    "\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Exercise B.1\n",
    "\n",
    "* (a) \n",
    "  * There is only one vector in the first set, it is independent.\n",
    "  * The vector $\\begin{bmatrix}1\\\\2\\end{bmatrix}$ and $\\begin{bmatrix}2\\\\4\\end{bmatrix}$ are dependent.\n",
    "  * The vector $\\begin{bmatrix}1\\\\2\\end{bmatrix}$ and $\\begin{bmatrix}1\\\\1\\end{bmatrix}$ are independent.\n",
    "  * since $-2\\begin{bmatrix}1\\\\2\\end{bmatrix} + 5 \\begin{bmatrix}1\\\\1\\end{bmatrix} = \\begin{bmatrix}3\\\\1\\end{bmatrix}$, the vectors are dependent.\n",
    "\n",
    "* (b) Both the vectors in set 3 and set 4 can span $R^2$.\n",
    "\n",
    "* (c) If $v_1,\\dots, v_d$ are independent, suppose $u=\\sum^d_{i=1}\\alpha_i v_i$, if $\\alpha_i$ are not unique, i.e. there exists $\\beta \\ne \\alpha$ such that $u=\\sum^d_{i=1}\\beta_iv_i$. Then we have $\\sum^d_{i=1}(\\alpha_i - \\beta_i)v_i = 0$, since $\\beta \\ne \\alpha$, we have a non-trivial linear combination of $v_i$ that equals to 0. This contradicts with the property that $v_i$ are independent. So $\\alpha$ is uniuqe.\n",
    "\n",
    "Suppose $u=\\sum^d_{i=1}\\alpha_i v_i$ holds for unique $\\alpha_i$, if $v_i$ are not independent, then we have non-trivial solution to $\\sum^d_{i=1}\\beta_i v_i = 0$, we can write, e.g. $v_d = \\sum^{d-1}_{i=1}\\gamma_i v_i$, take this into $u$, we have $u=\\sum^d_{i=1} \\delta_i v_i$ where $\\delta_d = 0$. This contradicts with the assumption that $\\alpha$ is uniuqe. So $v_i$ must be independent.\n",
    "\n",
    "* (d) The vectors in set 3 are a basis for $R^2$.\n",
    "* (e) Consider $\\sum^m_{i=1}\\alpha_iv_i = 0$, if one of the $v_i$ is zero, we can set the corresponding $\\alpha_i \\ne 0$ and all other $\\alpha$ to be zeros, so we have non-trivial solution $\\alpha$, which says the vectors are dependent.\n",
    "\n",
    "* (f) Since $v_1,\\dots,v_m$ are in space of $R^d$, let the basis of $R^d$ be $e_1,\\dots,e_d$. We have $v_i = \\sum^d_{k=1}a_{ik}e_k$. From problem (e), if $v_1,\\dots,v_m$ are independent, they can'be be zero. So their coordinates $a_{ik}$ on the basis $e_k$ won't be all zero. Pick the first non-zero coordinate, e.g. $a_{11}$, we have $e_1 = \\frac{1}{a_{11}}(v_1 -\\sum^d_{i=2}a_{ik}e_k)$, so we see that $v_1, e_2, \\dots, e_d$ can span $R^d$. Similarly, we can replace $e_2$ with $v_2$, thus have $v_1, v_2, e_3, \\dots, e_d$ span the space $R^d$. Continue doing this until we have $v_1,\\dots, v_d$. If we add one more vector $v_{d+1}$ into the list of vectors, we know it can be represented by a linear combination of $v_1, \\dots, v_d$ since by construction they space the $R^d$. This contradicts with the assumption that $v_1, \\dots, v_m$ are independent. So we must have $m\\le d$\n",
    "\n",
    "See the [$n+1$ vectors in  $R^n$  cannot be linearly independent](https://math.stackexchange.com/questions/473853/n1-vectors-in-mathbbrn-cannot-be-linearly-independent) on math.stackexchange.com.\n",
    "\n",
    "* (g) We start from $v_1$ and construct a set of vectors that are perpendicular to each other. Let $u_1 = \\frac{v_1}{|v_1|}$. Let $u_2 = v_2 - v^T_2u_1u_1$. We see that $u_2$ is perpendicular to $u_1$ and both have unit length. Next, construct $u_3$ such that it's perpendicular to both $u_1,u_2$. Contintue in such pattern until we have $u_m$. As $u_i$ are constructed from $v_i$, we see that $u_i$ should span $R^d$ as well. If $m < d$, we can construct $u_{m+1}$ which is perpendicular to $u_1,\\dots, u_m$, this will contradict the assumption that $v_1, \\dots, v_m$ span $R^d$. So we must have $m \\ge d$.\n",
    "\n",
    "* (h) A basis spans $R^d$, from problem (g), we see that its cardinality $m \\ge d$. Also the vectors in basis are independent, from problem (f), we have $m \\le d$, combine both conditions, we have $m=d$.\n",
    "\n",
    "* (i) If $v_1,v_2$ are orthogonal, we have $v_1^Tv_2 = 0$, for any $\\alpha_1, \\alpha_2$, and \n",
    "$\\alpha_1v_1 + \\alpha_2v_2$, multiply the expression by $v_1^T$, we have $\\alpha_1 v^T_1v_1 + \\alpha_2 v^T_1v_2 = \\alpha_1 |v_1|^2$, for this equal to zero, we would have $\\alpha_1 = 0$ unless $v_1=0$, which is not true by assumption. Similarly, we can show that $\\alpha_2 = 0$ as well. So $v_1,v_2$ are independent. \n",
    "\n",
    "If $v_1, v_2$ are independent, then we have $v^T_1(v_2 - \\lambda v_1) = v^T_1(v_2 - \\frac{v^T_1v_2}{v^T_1v_1}v_1) = v^T_1v_2 - \\frac{v^T_1v^T_1v_2v_1}{v^T_1v_1} = v^T_1v_2 - v^T_1v_2\\frac{v^T_1v_1}{v^T_1v_1} = v^T_1v_2 - v^T_1v_2 = 0$\n",
    "\n",
    "So $v_1$ and $v_2 - \\lambda v_1$ are orthogonal.\n",
    "\n",
    "If a vector $u$ is spanned by $v_1,v_2$, we have $u=a_1v_1 + a_2v_2 = (a_1+\\lambda a_2)v_1 + a_2 (v_2 - \\lambda v_1)  = b_1 v_1 + b_2 (v_2 - \\lambda v_1)$\n",
    "\n",
    "So they have the same span. \n",
    "\n",
    "The above derivation doesn't assume $v_1, v_2$ are dependent, as long as $v_1, v_2 \\ne 0$, we still have $v_1, v_2-\\lambda v_1$ orthogonal to each other and have the same span as $v_1, v_2$. \n",
    "\n",
    "* (j) Follow the construction of problem (i), we can transform any set of independent vectors to pairwise orthogonal vectors with the same span. \n",
    "\n",
    "* (k) We have $\\hat{v}_1 = \\frac{v_1}{|v_1|}$, $\\hat{v}_2 = v_2 - \\lambda_{21}v_1$, $\\hat{v}_3 = v_3 - \\lambda_{31}v_1 - \\lambda{32}v_2$, etc. \n",
    "\n",
    "* (l) Let $u=\\sum a_i v_i$, multiply both sides by $v^T_j$, we have $uv^T_j = \\sum a_i v^T_j v_i = a_j$, so $a_i = v^T_i u$.\n",
    "\n",
    "* (m) For any set of $d$ linearly independent vectors, we can construct an orthonormal basis by problem (k), this basis has a dimension of $d$, it's thus a basis for $R^d$.\n",
    "\n",
    "\n",
    "#### Exercise B.2\n",
    "\n",
    "* (a) \n",
    "\n",
    "$AB = \\begin{bmatrix}3 & 8 \\\\3 & 7 \\\\ 6 & 0 \\end{bmatrix}$\n",
    "\n",
    "$Ax = \\begin{bmatrix}5 \\\\4 \\\\ 9 \\end{bmatrix}$\n",
    "\n",
    "$Bx$: doesn't exist\n",
    "\n",
    "$BA$: doesn't exist\n",
    "\n",
    "$B^TA^T = \\begin{bmatrix}3 & 3 & 6\\\\8 & 7 & 0 \\end{bmatrix}$\n",
    "\n",
    "$x^TAx = 40$\n",
    "\n",
    "$B^TAB = \\begin{bmatrix}18 & 15\\\\15 & 37 \\end{bmatrix}$\n",
    "\n",
    "$A^{-1} = \\begin{bmatrix}-\\frac{1}{3} & \\frac{2}{3} & 0\\\\\\frac{2}{3} & -\\frac{1}{3} & 0 \\\\ 0 & 0 & \\frac{1}{3} \\end{bmatrix}$\n",
    "\n",
    "* (b) $\\sum\\sum x_ix_jA_{ij} = \\sum x_i (1A_{i1}+2A_{i2}+3A_{i3}) = 1(1A_{11}+2A_{12}+3A_{13}) + 2(1A_{21}+2A_{22}+3A_{23}) + 3(1A_{31}+2A_{32}+3A_{33}) = 1(1+4)+2(2+2)+3(9) = 5 + 8 + 27 = 40$\n",
    "\n",
    "Which is equal to $x^TAx$ above.\n",
    "\n",
    "* (c) Solve for eigenvalues, we have $\\lambda_1=\\lambda_2 = 3, \\lambda_3 = -1$ and $v_1 =  \\begin{bmatrix}b \\\\ b \\\\ c \\end{bmatrix}$, $v_2 =  \\begin{bmatrix}-b \\\\ -b \\\\ \\frac{2b^2}{c} \\end{bmatrix}$, $v_3 = \\begin{bmatrix}a \\\\ -a \\\\ 0 \\end{bmatrix}$\n",
    "\n",
    "where $a,b,c$ are any numbers.\n",
    "\n",
    "* (d) There are 3 linearly independent rows in $B$. There are 3 linearly independent columns in $B$\n",
    "\n",
    "* (e) As $C$ is the basis matrix, we have for each \n",
    "$a_i = \\sum^r_{k=1}r_{ik}c_k = \\sum^r_{k=1} a_ic^T_k c_k = C\\begin{bmatrix}a_ic^T_1 \\\\ a_ic^T_2 \\\\ \\dots \\\\ a_ic^T_r\\end{bmatrix}$\n",
    "\n",
    "so we can construct $R=\\begin{bmatrix}a_1c^T_1 & \\dots & a_dc^T_1\\\\ \\dots & \\dots & \\dots\\\\ a_1c^T_r & \\dots & a_dc^T_r\\end{bmatrix}$ and $A=CR$.\n",
    "\n",
    "  * (i) The column rank of $A$ is $r$ since the columns are spanned by basis $c_1, \\dots, c_r$\n",
    "  * (ii) The dimension of $R$ is $r\\times d$\n",
    "  * (iii) Let $R_{1,},\\dots, R_{r,}$ be the rows of matrix $R$, as $A=CR$, we have $A=c_1R_{1,} + \\dots + c_rR_{r,}$, clearly, the $i$-th row of $A$, $A_{i,} = c_{1i}R_{1,}+\\dots + c_{ri}R_{r,}$. It's a linear combination of rows in $R$.\n",
    "  \n",
    "  * (iv) Since rows in $A$ is a linear combination of rows in $R$, hence the dimension of the subspace spanned by the rows of $A$ is at most $r$, i.e. $\\text{column-rank}(A) \\ge \\text{row-rank}(A)$\n",
    "  * (v) Similarly apply (iii)-(iv) on $A^T = R^TC^T$, The row-rank of $A^T$ (i.e. the column rank of $A$) is less than or equal to the column rank of $A^T$, which is the row rank of $A$, i.e.  $\\text{column-rank}(A) \\le \\text{row-rank}(A)$\n",
    "  \n",
    "  Combine (iv) and (v) we have $\\text{column-rank}(A) = \\text{row-rank}(A)$\n",
    "  \n",
    "See [Proofs that column rank = row rank on wikipedia](https://en.wikipedia.org/wiki/Rank_(linear_algebra))  \n",
    "\n",
    "#### Exercise B.3\n",
    "\n",
    "* $AA^{+}A = U\\Gamma V^T V \\Gamma^{-1} U^TU\\Gamma V^T = U\\Gamma\\Gamma^{-1}\\Gamma V^T = U\\Gamma V^T = A$\n",
    "\n",
    "* $A^{+}AA^{+} = V \\Gamma^{-1} U^T  U\\Gamma V^T V \\Gamma^{-1} U^T = V\\Gamma^{-1} U^T =A^{+}$\n",
    "\n",
    "* $(AA^{+})^T = (A^{+})^TA^T = (V \\Gamma^{-1} U^T)^T V\\Gamma U^T = U\\Gamma^{-1}V^TV\\Gamma U^T = UU^T = U\\Gamma V^TV \\Gamma^{-1} U^T = AA^{+}$\n",
    "\n",
    "* $(A^{+}A)^T = A^T(A^{+})^T = V\\Gamma U^T (V \\Gamma^{-1} U^T)^T = V\\Gamma U^T U\\Gamma^{-1}V^T = VV^T = V\\Gamma^{-1} U^TU\\Gamma V^T = A^{+}A$\n",
    "\n",
    "* $(A^T)^{+} = (V\\Gamma U^T)^{+} = U\\Gamma^{-1} V^T$ by definition,  and $(A^{+})^T = U\\Gamma^{-1}V^T$ so we have $(A^T)^{+} =(A^{+})^T$\n",
    "\n",
    "#### Exercise B.4\n",
    "\n",
    "* (a) Since for $|A|$, every summand has exactly one term from each column, for $I$, only the diagonal entries are nonzero, so we have $|I| = \\epsilon_{1,\\dots ,k}I_{11}\\dots I_{kk} = 1$\n",
    "\n",
    "Similarly we have $|D| = \\prod^N_{i=1}D_{ii}$\n",
    "\n",
    "* (b) For a given order of $i_1,\\dots, i_N$ (from $1,\\dots, N$), we have $|A| = \\sum_{j_1,\\dots,j_N} \\epsilon_{j_1,\\dots,j_N}A^T_{i_1j_1}\\dots A^T_{i_Nj_N}$. There are $N!$ such orders of $1,\\dots, N$, so we have $|A| =  \\frac{1}{N!}\\sum_{i_1,\\dots,i_N}\\sum_{j_1,\\dots,j_N}\\epsilon_{i_1,\\dots,i_N}\\epsilon_{j_1,\\dots,j_N}A_{i_1j_1}\\dots A_{i_Nj_N}$\n",
    "\n",
    "* (c) $|A^T| = \\frac{1}{N!}\\sum_{i_1,\\dots,i_N}\\sum_{j_1,\\dots,j_N} \\epsilon_{i_1,\\dots,i_N}\\epsilon_{j_1,\\dots,j_N}A^T_{i_1j_1}\\dots A^T_{i_Nj_N} = \\frac{1}{N!}\\sum_{i_1,\\dots,i_N}\\sum_{j_1,\\dots,j_N} \\epsilon_{i_1,\\dots,i_N}\\epsilon_{j_1,\\dots,j_N}A_{j_1i_1}\\dots A_{j_Ni_N} = \\frac{1}{N!}\\sum_{j_1,\\dots,j_N}\\sum_{i_1,\\dots,i_N}\\epsilon_{j_1,\\dots,j_N}\\epsilon_{i_1,\\dots,i_N}A_{j_1i_1}\\dots A_{j_Ni_N} = |A|$\n",
    "\n",
    "* (d) We have $1 = |I| = |O^TO| = |O||O^T| = |O|^2$, so $|O| = \\pm 1$\n",
    "\n",
    "$|OAO^T| = |O||A||O^T| = |O|^2|A| = |A|$\n",
    "\n",
    "* (e) $|A^{-1}A| = |A^{-1}||A| = |I_n| = 1$, so we have $|A^{-1}=\\frac{1}{|A|}$\n",
    "\n",
    "* (f) If $A$ is symmetric, we can write $A=U\\Lambda U^T$, thus $|A|=|U||\\Lambda||U^T| = |U|^2 |\\Lambda| = |\\Lambda| = \\prod^n_{i=1}\\lambda_i$\n",
    "\n",
    "#### Exercise B.5\n",
    "\n",
    "Consider $M=\\begin{bmatrix}I_n & A \\\\ -B & I_d\\end{bmatrix}$, according to the formula of block matrix, we have\n",
    "\n",
    "$|M| = |M_{22}||F_1| = |M_{11}||F_2|$, whereas\n",
    "\n",
    "$|M_{22}| = |M_{11}|= 1$, so we have $|F_1| = |M_{11}-M_{12}M^{-1}_{22}M_{21}| =  |I_n + AI_dB| = |I_n + AB|$ and $|F_2| = |M_{22} - M_{21}M^{-1}_{11}M_{12}| = |I_d + BI_nA| = |I_d + BA|$\n",
    "\n",
    "so we have $|I_n + AB| = |I_d + BA|$\n",
    "\n",
    "This is the Sylvester's determinant theorem.\n",
    "\n",
    "We have $A+XBY^T = A(I + A^{-1}XBY^T)$, take determinant on both sides and apply Sylvester's theorem above, we have\n",
    "\n",
    "$|A+XBY^T| = |A||I + A^{-1}XBY^T| = |A| |I + BY^TA^{-1}X| = |A||B(B^{-1}+Y^TA^{-1}X| = |A||B||B^{-1}+Y^TA^{-1}X|$\n",
    "\n",
    "#### Exercise B.6\n",
    "\n",
    "$\\|A\\|_2 = \\max_{\\|x\\|=1}\\|Ax\\|$, consider instead $\\|Ax\\|^2 = (Ax)^T(Ax) = x^TA^TAx = x^TV\\Gamma U^T U \\Gamma V^T x = x^T V \\Gamma^2 V^T x = x^TSx $ \n",
    "\n",
    "where $S=V \\Gamma^2 V^T$, it's clear that the eigenvalues of $S$ are the diagonal entries in $\\Gamma^2$, i.e. $\\lambda_i = \\gamma_i^2$.  Let $e_i$ be the eigenvectors of $S$, then we have $x=\\sum a_i e_i$, so we have $Sx = \\sum a_i \\lambda_i e_i = \\sum \\lambda_i a_i e_i$\n",
    "\n",
    "thus $x^TSx = \\sum a_i e^T_i \\sum \\lambda_j a_j e_j = \\sum \\lambda_i a_i^2 $ as $e_i$ are orthonormal. \n",
    "Since we have $x^Tx = \\sum a_i^2 = 1$, so $x^TSx = \\sum \\lambda_i a_i^2  \\le \\sum \\lambda_1 a_i^2 = \\lambda_1$, where $\\lambda_1 $ is the largest eigenvalue of $S$, thus the maximum value of $\\|A\\|_2$ is $\\sqrt{\\lambda_1} = \\gamma_1$ the largest singular value. This is achieved when $x$ is the eigenvector corresponding to the $\\lambda_1$.\n",
    "\n",
    "See [the proof at math.stackexchange.com](https://math.stackexchange.com/questions/586663/why-does-the-spectral-norm-equal-the-largest-singular-value).\n",
    "\n",
    "$\\|A\\|_F = trace(AA^T) = trace(U\\Gamma V^T V\\Gamma U^T) = trace(U\\Gamma^2 U^T) = trace(\\Gamma^2 U^TU) = trace(\\Gamma^2) = \\sum^{\\rho}_{i=1}\\gamma_i^2$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Exercise B.7\n",
    "\n",
    "We express $x$ in terms of $z$, we have $x =U\\Lambda^{\\frac{1}{2}}z + \\mu$, so $x-\\mu = U\\Lambda^{\\frac{1}{2}}z$\n",
    "\n",
    "We have \n",
    "\n",
    "\\begin{align*}\n",
    "P(x) &= \\frac{1}{(2\\pi)^{\\frac{d}{2}}|\\Sigma|^{\\frac{1}{2}}}e^{-\\frac{1}{2}(x-\\mu)^T\\Sigma^{-1}(x-\\mu)} \\\\\n",
    "&= \\frac{1}{(2\\pi)^{\\frac{d}{2}}|\\Sigma|^{\\frac{1}{2}}}e^{-\\frac{1}{2}(U\\Lambda^{\\frac{1}{2}}z)^T\\Sigma^{-1}(U\\Lambda^{\\frac{1}{2}}z)} \\\\\n",
    "&= \\frac{1}{(2\\pi)^{\\frac{d}{2}}|\\Sigma|^{\\frac{1}{2}}}e^{-\\frac{1}{2}z^T\\Lambda^{\\frac{1}{2}}U^T\\Sigma^{-1}U\\Lambda^{\\frac{1}{2}}z}\\\\\n",
    "&= \\frac{1}{(2\\pi)^{\\frac{d}{2}}|\\Sigma|^{\\frac{1}{2}}}e^{-\\frac{1}{2}z^T\\Lambda^{\\frac{1}{2}}U^T(U\\Lambda U^T)^{-1}U\\Lambda^{\\frac{1}{2}}z}\\\\\n",
    "&= \\frac{1}{(2\\pi)^{\\frac{d}{2}}|\\Sigma|^{\\frac{1}{2}}}e^{-\\frac{1}{2}z^T\\Lambda^{\\frac{1}{2}}U^TU^{-T}\\Lambda^{-1} U^{-1}U\\Lambda^{\\frac{1}{2}}z}\\\\\n",
    "&= \\frac{1}{(2\\pi)^{\\frac{d}{2}}|\\Sigma|^{\\frac{1}{2}}}e^{-\\frac{1}{2}z^Tz}\\\\\n",
    "\\end{align*}\n",
    "\n",
    "Also from exercise B.4 (d), we have $|\\Sigma| = |U\\Lambda U^T| = |\\Lambda|$, so $|\\Sigma|^{-\\frac{1}{2}} = |\\Lambda|^{-\\frac{1}{2}}$\n",
    "\n",
    "\\begin{align*}\n",
    "E[x] &= \\int dx xP(x) \\\\\n",
    "&= \\int \\frac{1}{|J|}dz (U\\Lambda^{\\frac{1}{2}}z+\\mu) \\frac{1}{(2\\pi)^{\\frac{d}{2}}|\\Sigma|^{\\frac{1}{2}}}e^{-\\frac{1}{2}z^Tz}\\\\\n",
    "&= \\int dz (U\\Lambda^{\\frac{1}{2}}z+\\mu) \\frac{1}{(2\\pi)^{\\frac{d}{2}}}e^{-\\frac{1}{2}z^Tz}\\\\\n",
    "\\end{align*}\n",
    "\n",
    "We also have $xx^T = (U\\Lambda^{\\frac{1}{2}}z + \\mu)(U\\Lambda^{\\frac{1}{2}}z + \\mu)^T = (U\\Lambda^{\\frac{1}{2}}z + \\mu)(z^T\\Lambda^{\\frac{1}{2}}U^T + \\mu^T) = U\\Lambda^{\\frac{1}{2}}zz^T\\Lambda^{\\frac{1}{2}}U^T + U\\Lambda^{\\frac{1}{2}}z\\mu^t +\\mu z^T\\Lambda^{\\frac{1}{2}}U^T + \\mu\\mu^T$\n",
    "\n",
    "take this into $E[xx^T]$ we obtain the corresponding formula."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}