hw3_session3_decision_trees_solution.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<center>\n",
    "<img src=\"../../img/ods_stickers.jpg\">\n",
    "## Открытый курс по машинному обучению. Сессия № 3\n",
    "Авторы материала: Юрий Кашницкий (@yorko) и Максим Уваров (@maxis42). Материал распространяется на условиях лицензии [Creative Commons CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/). Можно использовать в любых целях (редактировать, поправлять и брать за основу), кроме коммерческих, но с обязательным упоминанием автора материала."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# <center>Домашнее задание № 3. Решение \n",
    "## <center> Деревья решений для классификации и регрессии"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**В этом задании мы разберемся с тем, как работает дерево решений в задаче регрессии, а также построим (и настроим) классифицирующие деревья решений в задаче прогнозирования сердечно-сосудистых заболеваний. \n",
    "Заполните код в клетках (где написано \"Ваш код здесь\") и ответьте на вопросы в [веб-форме](https://goo.gl/sF6P4Q).**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "from matplotlib import pyplot as plt\n",
    "%matplotlib inline\n",
    "from ipywidgets import Image\n",
    "from io import StringIO\n",
    "import pydotplus #pip install pydotplus\n",
    "from sklearn.model_selection import train_test_split, GridSearchCV\n",
    "from sklearn.metrics import accuracy_score\n",
    "from sklearn.tree import DecisionTreeClassifier, export_graphviz"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. Простой пример восстановления регрессии с помощью дерева решений"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Рассмотрим следующую одномерную задачу восстановления регрессии. Неформально, надо построить функцию $a(x)$, приближающую искомую зависимость $y = f(x)$ в терминах среднеквадратичной ошибки: $min \\sum_i {(a(x_i) - f(x_i))}^2$. Подробно мы рассмотрим эту задачу в следующий раз ([4-я статья курса](https://habrahabr.ru/company/ods/blog/323890/)), а пока поговорим о том, как решать эту задачу с помощью дерева решений. Предварительно прочитайте небольшой раздел [\"Дерево решений в задаче регрессии\"](https://habrahabr.ru/company/ods/blog/322534/#derevo-resheniy-v-zadache-regressii) 3-ей статьи курса. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYYAAAEKCAYAAAAW8vJGAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAEqtJREFUeJzt3X2QXXddx/H3xzTF5UECJliSNqaMEEVQg2unUh8KrYapDK2Pwx8o9Sk+IjhMsLEzMj7MgMTx+WkyBQfHjqglhIpgaC3o6Ewr26ZladNIqS10U+yiEwRZaRq+/rF34/6W3WSX7L3n7t73a2an555zes/nnt3s557fOXtPqgpJkuZ8WdcBJEnDxWKQJDUsBklSw2KQJDUsBklSw2KQJDUsBklSw2KQJDUsBklS47yuA3wpNm/eXDt27Og6hiStKXfeeeenqmrL2dZbk8WwY8cOJiYmuo4hSWtKkoeXs55DSZKkhsUgSWpYDJKkhsUgSWpYDJKkxlBclZTkF4GfAAqYBH60qv6321SSNBwOHZli/+FjHD8xw9ZNY+zdvZNrdm3r2/Y6P2JIsg34BWC8ql4AbABe2W0qSRoOh45Mse/gJFMnZihg6sQM+w5OcujIVN+22Xkx9JwHjCU5D3gycLzjPJI0FPYfPsbMyVPNvJmTp9h/+Fjfttl5MVTVFPBbwMeBR4FPV9X7F66XZE+SiSQT09PTg44pSZ04fmJmRfNXQ+fFkOQZwNXAxcBW4ClJXrVwvao6UFXjVTW+ZctZ/6JbktaFrZvGVjR/NXReDMCVwL9X1XRVnQQOAi/uOJMkDYW9u3cytnFDM29s4wb27t7Zt20Ow1VJHwcuTfJkYAa4AvCDkCQJTl99NMirkjovhqq6I8lNwF3AE8AR4EC3qSRpeFyza1tfi2ChzosBoKreCLyx6xySpOE4xyBJGiIWgySpYTFIkhoWgySpYTFIkhoWgySpYTFIkhoWgySpYTFIkhoWgySpYTFIkhoWgySpYTFIkhoWgySpYTFIkhpDUQxJNiW5Kcn9SY4m+dauM0nSqBqKG/UAvwf8fVX9QJLzgSd3HUiSRlXnxZDk6cB3ANcCVNXjwONdZpKkUTYMQ0kXA9PAnyU5kuSGJE/pOpQkjaphKIbzgBcBf1JVu4D/Aa5buFKSPUkmkkxMT08POqMkjYxhKIZHgEeq6o7e45uYLYpGVR2oqvGqGt+yZctAA0rSKOm8GKrqk8AnkuzszboCuK/DSJI00jo/+dzzGuDG3hVJDwI/2nEeSRpZQ1EMVXU3MN51DknSEAwlSZKGi8UgSWpYDJKkhsUgSWpYDJKkhsUgSWpYDJKkhsUgSWpYDJKkhsUgSWpYDJKkhsUgSWpYDJKkhsUgSWpYDJKkhsUgSWoMTTEk2ZDkSJL3dJ1FkkbZ0BQD8FrgaNchJGnUDUUxJLkQ+B7ghq6zSNKoG4piAH4XeAPwha6DSNKo67wYkrwceKyq7jzLenuSTCSZmJ6eHlA6SRo9nRcDcBnwiiQPAe8AXprkLxauVFUHqmq8qsa3bNky6IySNDI6L4aq2ldVF1bVDuCVwG1V9aqOY0nSyOq8GCRJw+W8rgPMV1UfBD7YcQxJGmkeMUiSGhaDJKlhMUiSGhaDJKlhMUiSGhaDJKlhMUiSGhaDJKlhMUiSGhaDJKlhMUiSGhaDJKkxVB+iJ0mr5dCRKfYfPsbxEzNs3TTG3t07uWbXtq5jrQkWg6R159CRKfYdnGTm5CkApk7MsO/gJIDlsAwOJUlad/YfPna6FObMnDzF/sPHOkq0tlgMktad4ydmVjRfrc6LIclFST6Q5L4k9yZ5bdeZJK1tWzeNrWi+Wp0XA/AE8Pqqej5wKfBzSZ7fcSZJa9je3TsZ27ihmTe2cQN7d+/sKNHa0vnJ56p6FHi0N/2ZJEeBbcB9nQaTtGbNnWD2qqQvTaqq6wynJdkB/BPwgqr67wXL9gB7ALZv3/7NDz/88MDzSdJaluTOqho/23rDMJQEQJKnAu8EXrewFACq6kBVjVfV+JYtWwYfUJJGxFAUQ5KNzJbCjVV1sOs8kjTKOi+GJAHeChytqt/uOo8kjbrOiwG4DPhh4KVJ7u59XdV1KEkaVcNwVdI/A+k6hyRp1jAcMUiShojFIElqWAySpIbFIElqWAySpIbFIElqWAySpIbFIElqWAySpIbFIElqWAySpIbFIElqdP4hepKGy6EjU94Sc8RZDJJOO3Rkin0HJ5k5eQqAqRMz7Ds4CWA5jBCHkiSdtv/wsdOlMGfm5Cn2Hz7WUSJ1YSiKIcnLkhxL8kCS67rOI42q4ydmVjRf61PnQ0lJNgB/BHwX8AjwoSQ3V9V93SaTlm+9jMtv3TTG1CIlsHXTWAdp1JVhOGK4BHigqh6sqseBdwBXd5xJWra5cfmpEzMU/z8uf+jIVNfRVmzv7p2MbdzQzBvbuIG9u3d2lEhdOGsxJLklyTf2McM24BPzHj/SmyetCetpXP6aXdt40/e9kG2bxgiwbdMYb/q+F67Jox996ZYzlPRLwO8meQj45ap6tL+RFpdkD7AHYPv27V1EkBa13sblr9m1zSIYcWc9Yqiqu6rqJcB7gL9P8sYkqzngOAVcNO/xhb15C3McqKrxqhrfsmXLKm5eXTp0ZIrL3nwbF1/3d1z25tvW5PDLUuPvjstrrVrWOYYkAY4BfwK8Bvhokh9epQwfAp6b5OIk5wOvBG5epefWEFsvY/OOy2u9Wc45hn9h9h387zA79n8tcDlwSZID5xqgqp4Afh44DBwF/rqq7j3X59XwWy9j847La71ZzjmGPcB9VVUL5r8mydHVCFFV7wXeuxrPpbVjPY3NOy6v9WQ55xjuXaQU5nzPKufRCHFsXhpO5/R3DFX14GoF0ehxbF4aTp3/5bNG19zQy3r4i2FpPbEY1CnH5qXhMwwfiSFJGiIWgySpYTFIkhoWgySpYTFIkhoWgySpYTFIkhoWgySpYTFIkhoWgySpYTFIkhoWgySp0WkxJNmf5P4kH07yriSbuswjSer+iOEW4AVV9Q3AvwH7Os4jSSOv02Koqvf37vkMcDtwYZd5JEndHzHM92PA+7oOIUmjru836klyK3DBIouur6p399a5HngCuPEMz7MH2AOwffv2PiSVJMEAiqGqrjzT8iTXAi8HrqiqOsPzHAAOAIyPjy+53ig4dGTK22FK6ptOb+2Z5GXAG4DvrKrPdZllrTh0ZIp9ByeZOXkKgKkTM+w7OAlgOUhaFV2fY/hD4GnALUnuTvKnHecZevsPHztdCnNmTp5i/+FjHSWStN50esRQVV/T5fbXouMnZlY0X5JWqusjBq3Q1k1jK5ovSStlMawxe3fvZGzjhmbe2MYN7N29s6NEktabToeStHJzJ5i9KklSv1gMa9A1u7ZZBJL6xqEkSVLDYpAkNSwGSVLDYpAkNSwGSVLDYpAkNSwGSVLDYpAkNSwGSVLDYpAkNSwGSVLDYpAkNYaiGJK8Pkkl2dx1FkkadZ0XQ5KLgO8GPt51FknSEBQD8DvAG4DqOogkqeNiSHI1MFVV9yxj3T1JJpJMTE9PDyCdJI2mvt+oJ8mtwAWLLLoe+GVmh5HOqqoOAAcAxsfHPbqQpD7pezFU1ZWLzU/yQuBi4J4kABcCdyW5pKo+2e9ckqTFdXZrz6qaBJ419zjJQ8B4VX2qq0ySpOE4+SxJGiKdHTEsVFU7us4gSfKIQZK0gMUgSWpYDJKkhsUgSWpYDJKkhsUgSWpYDJKkhsUgSWpYDJKkhsUgSWpYDJKkhsUgSWpYDJKkhsUgSWpYDJKkRufFkOQ1Se5Pcm+St3SdR5JGXac36knyEuBq4Bur6vNJnnW2/0eS1F9dHzH8DPDmqvo8QFU91nEeSRp5XRfD84BvT3JHkn9M8i0d55Gkkdf3oaQktwIXLLLo+t72nwlcCnwL8NdJnlNVtcjz7AH2AGzfvr1/gSVpxPW9GKrqyqWWJfkZ4GCvCP41yReAzcD0Is9zADgAMD4+/kXFIUlaHV0PJR0CXgKQ5HnA+cCnOk0kSSOu06uSgLcBb0vyEeBx4NWLDSNJkgan02KoqseBV3WZQZLU6nooSZI0ZCwGSVLDYpAkNSwGSVLDYpAkNSwGSVLDYpAkNSwGSVLDYpAkNSwGSVLDYpAkNSwGSVLDYpAkNSwGSVLDYpAkNSwGSVKj02JI8k1Jbk9yd5KJJJd0mUeS1P2tPd8C/GpVvS/JVb3Hl/drY4eOTLH/8DGOn5hh66Yx9u7eyTW7tvVrc5K0JnVdDAV8RW/66cDxfm3o0JEp9h2cZObkKQCmTsyw7+AkgOUgSfN0fY7hdcD+JJ8AfgvY168N7T987HQpzJk5eYr9h4/1a5OStCb1/Yghya3ABYssuh64AvjFqnpnkh8C3gpcucTz7AH2AGzfvn3FOY6fmFnRfEkaVX0vhqpa9Bc9QJI/B17be/g3wA1neJ4DwAGA8fHxWmmOrZvGmFqkBLZuGlvpU0nSutb1UNJx4Dt70y8FPtqvDe3dvZOxjRuaeWMbN7B3985+bVKS1qSuTz7/JPB7Sc4D/pfeUFE/zJ1g9qokSTqzVK14VKZz4+PjNTEx0XUMSVpTktxZVeNnW6/roSRJ0pCxGCRJDYtBktSwGCRJDYtBktRYk1clJZkGHj6Hp9gMfGqV4qwmcy3fMGYCc62UuVbmXHN9dVVtOdtKa7IYzlWSieVcsjVo5lq+YcwE5lopc63MoHI5lCRJalgMkqTGqBbDga4DLMFcyzeMmcBcK2WulRlIrpE8xyBJWtqoHjFIkpYwEsWQZH+S+5N8OMm7kmxaYr2XJTmW5IEk1w0g1w8muTfJF5IseaVBkoeSTCa5O0nfPz1wBbkGtr+SPDPJLUk+2vvvM5ZY71RvP92d5OY+5jnja0/ypCR/1Vt+R5Id/cqywlzXJpmet49+YgCZ3pbksSQfWWJ5kvx+L/OHk7yo35mWmevyJJ+et69+ZUC5LkrygST39f4dvnaRdfq7z6pq3X8B3w2c15v+TeA3F1lnA/Ax4DnA+cA9wPP7nOvrgJ3AB4HxM6z3ELB5gPvrrLkGvb+AtwDX9aavW+x72Fv22QHsn7O+duBngT/tTb8S+KshyXUt8IeD+lnqbfM7gBcBH1li+VXA+4AAlwJ3DEmuy4H3DHJf9bb7bOBFvemnAf+2yPexr/tsJI4Yqur9VfVE7+HtwIWLrHYJ8EBVPVhVjwPvAK7uc66jVTV0N51eZq5B76+rgbf3pt8OXNPHbZ3Ncl77/Lw3AVckyRDkGriq+ifgv86wytXAn9es24FNSZ49BLk6UVWPVtVdvenPAEeBhTeO6es+G4liWODHmG3ahbYBn5j3+BG++JvRlQLen+TO3r2vh8Gg99dXVdWjvelPAl+1xHpfnmQiye1J+lUey3ntp9fpvSn5NPCVfcqzklwA398bfrgpyUV9zrQcw/xv71uT3JPkfUm+ftAb7w1B7gLuWLCor/us6zu4rZoktwIXLLLo+qp6d2+d64EngBuHKdcyfFtVTSV5FnBLkvt773a6zrWqzpRp/oOqqiRLXU731b199RzgtiSTVfWx1c66hv0t8JdV9fkkP8XsUc1LO840rO5i9ufps0muAg4Bzx3UxpM8FXgn8Lqq+u9BbRfWUTFU1ZVnWp7kWuDlwBXVG6RbYAqY/+7pwt68vuZa5nNM9f77WJJ3MTtkcE7FsAq5Vn1/nSlTkv9I8uyqerR3yPzYEs8xt68eTPJBZt9trXYxLOe1z63zSGZvXft04D9XOceKc1XV/Aw3MHvupmt9+bd3rub/Mq6q9yb54ySbq6rvn6GUZCOzpXBjVR1cZJW+7rORGEpK8jLgDcArqupzS6z2IeC5SS5Ocj6zJwz7dlXLciV5SpKnzU0zeyJ90asoBmzQ++tm4NW96VcDX3RUk+QZSZ7Um94MXAbc14csy3nt8/P+AHDbEm9IBpprwTj0K5gdv+7azcCP9K60uRT49Lxhw84kuWDuvFCSS5j9fdnvcqe3zbcCR6vqt5dYrb/7bNBn3Lv4Ah5gdjzu7t7X3NUiW4H3zlvvKmavAPgYs0Mq/c71vcyODX4e+A/g8MJczF5hck/v695hyTXo/cXs+Pw/AB8FbgWe2Zs/DtzQm34xMNnbV5PAj/cxzxe9duDXmH3zAfDlwN/0fvb+FXhOv79vy8z1pt7P0T3AB4CvHUCmvwQeBU72fq5+HPhp4Kd7ywP8US/zJGe4Qm/AuX5+3r66HXjxgHJ9G7PnFT8873fWVYPcZ/7lsySpMRJDSZKk5bMYJEkNi0GS1LAYJEkNi0GS1LAYJEkNi0GS1LAYpFXQ+/z87+pN/0aSP+g6k/SlWjeflSR17I3Ar/U+6HAXsx83Ia1J/uWztEqS/CPwVODymv0cfWlNcihJWgVJXsjsnbcetxS01lkM0jnqfWLpjczeVeuzvU/zldYsi0E6B0meDBwEXl9VR4FfZ/Z8g7RmeY5BktTwiEGS1LAYJEkNi0GS1LAYJEkNi0GS1LAYJEkNi0GS1LAYJEmN/wOg1rDmxPLdDAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x114774ba8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "X = np.linspace(-2, 2, 7)\n",
    "y = X ** 3\n",
    "\n",
    "plt.scatter(X, y)\n",
    "plt.xlabel(r'$x$')\n",
    "plt.ylabel(r'$y$');"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Проделаем несколько шагов в построении дерева решений. Исходя из соображений симметрии, выберем пороги для разбиения равными соответственно 0, 1.5 и -1.5. Напомним, что в случае задачи восстановления регрессии листовая вершина выдает среднее значение ответа по всем объектам обучающей выборки, попавшим в эту вершину."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Итак, начнём. Дерево глубины 0 состоит из одного корня, который содержит всю обучающую выборку. Как будут выглядеть предсказания данного дерева для $x \\in [-2, 2]$? Постройте соответствующий график. Тут без `sklearn` – разбираемся просто с ручкой, бумажкой и Python, если надо."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXYAAAD8CAYAAABjAo9vAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAEfpJREFUeJzt3XtwpXV9x/H3t9kF46VGuqu4C3Fhqmmtti49UrzUC1CXoY5sre3wh6146VZbLTjOOmyZqdP+4yWO1Y62TgZ1dMp4w3WlDjRAkdp2BjSQhQhLFCm3LEqoE9Qa2Yvf/pETmg25nTzPOSf57fs1k9nnPOe3v9/3PEk+efJ7fidPZCaSpHL8UrcLkCTVy2CXpMIY7JJUGINdkgpjsEtSYQx2SSqMwS5JhTHYJakwBrskFWZDNwbdtGlTbtu2rRtDS9K6dcsttzySmZuXa9eVYN+2bRsjIyPdGFqS1q2IuG8l7ZyKkaTCGOySVBiDXZIKY7BLUmEMdkkqTC2rYiLi3cDbgATGgDdn5s/r6FuS1rt9oxMMDo9zcGqaLX297N4xwM7tW9s2XuUz9ojYCvwV0MjMFwA9wIVV+5WkEuwbnWDP3jEmpqZJYGJqmj17x9g3OtG2MeuaitkA9EbEBuDJwMGa+pWkdW1weJzpw0eP2Td9+CiDw+NtG7NysGfmBPBh4H7gIeDRzLx2fruI2BURIxExMjk5WXVYSVoXDk5Nt7S/DnVMxTwDuAA4DdgCPCUi3ji/XWYOZWYjMxubNy/7jlhJKsKWvt6W9tehjqmYc4H/zszJzDwM7AVeWkO/krTu7d4xQO/GnmP29W7sYfeOgbaNWceqmPuBsyLiycA0cA7gH4KRJHh89UsnV8VUDvbMvDkirgRuBY4Ao8BQ1X4lqRQ7t29ta5DPV8s69sx8H/C+OvqSJFXjO08lqTAGuyQVxmCXpMIY7JJUGINdkgpjsEtSYQx2SSqMwS5JhTHYJakwBrskFcZgl6TCGOySVBiDXZIKY7BLUmEMdkkqTC3BHhF9EXFlRNwVEQci4iV19CtJal0tN9oAPgb8a2a+ISJOAJ5cU7+SpBZVDvaIeDrwCuAigMw8BByq2q8kaXXqmIo5DZgEPhMRoxFxeUQ8pYZ+JUmrUEewbwDOAP4pM7cD/wtcOr9RROyKiJGIGJmcnKxhWEnSQuoI9geBBzPz5ubjK5kJ+mNk5lBmNjKzsXnz5hqGlSQtpHKwZ+YPgAciYqC56xzgzqr9SpJWp65VMe8CrmiuiLkHeHNN/UqSWlRLsGfmfqBRR1+SpGp856kkFcZgl6TCGOySVBiDXZIKY7BLUmEMdkkqjMEuSYUx2CWpMAa7JBXGYJekwhjsklQYg12SCmOwS1JhDHZJKozBLkmFMdglqTC1BXtE9ETEaER8va4+JUmtq/OM/WLgQI39SZJWoZZgj4hTgN8HLq+jP0nS6tV1xv5R4L3AL2rqT5K0SpWDPSJeCzycmbcs025XRIxExMjk5GTVYSVJi6jjjP1lwOsi4l7gC8DZEfHP8xtl5lBmNjKzsXnz5hqGlSQtpHKwZ+aezDwlM7cBFwI3ZOYbK1cmSVoV17FLUmE21NlZZt4I3Fhnn5Kk1njGLkmFMdglqTAGuyQVxmCXpMIY7JJUGINdkgpjsEtSYQx2SSqMwS5JhTHYJakwBrskFcZgl6TC1PpHwCSpLvtGJxgcHufg1DRb+nrZvWOAndu3drusdcFgl7Tm7BudYM/eMaYPHwVgYmqaPXvHAAz3FXAqRtKaMzg8/nioz5o+fJTB4fEuVbS+GOyS1pyDU9Mt7dex6riZ9akR8Y2IuDMi7oiIi+soTNLxa0tfb0v7daw6ztiPAO/JzOcDZwF/GRHPr6FfScep3TsG6N3Yc8y+3o097N4x0KWK1pfKF08z8yHgoeb2TyLiALAVuLNq35KOT7MXSF0VszqRmfV1FrEN+Cbwgsz88bzndgG7APr7+3/7vvvuq21cSToeRMQtmdlYrl1tF08j4qnAV4BL5oc6QGYOZWYjMxubN2+ua1hJ0jy1BHtEbGQm1K/IzL119ClJWp06VsUE8CngQGZ+pHpJkqQq6jhjfxnwJ8DZEbG/+XF+Df1KklahjlUx/wlEDbVIkmrgO08lqTAGuyQVxmCXpMIY7JJUGINdkgpjsEtSYQx2SSqMwS5JhTHYJakwBrskFcZgl6TCGOySVJjKfwRM0tqyb3TCW8od5wx2qSD7RifYs3eM6cNHAZiYmmbP3jEAw/044lSMVJDB4fHHQ33W9OGjDA6Pd6kidUNdt8Y7LyLGI+LuiLi0jj4lte7g1HRL+1WmylMxEdEDfAL4PeBB4NsRcVVm3lm1b6lTSpmX3tLXy8QCIb6lr7cL1ahb6jhjPxO4OzPvycxDwBeAC2roV+qI2Xnpialpkv+fl943OtHt0lq2e8cAvRt7jtnXu7GH3TsGulSRuqGOi6dbgQfmPH4Q+J0a+l3YJZfA/v1t617Hn/77p/jMkaNP2H/CFT3Q39eFilZvJ/Dynz7G/T+a5tCRo5ywoYf+k3rZ9B8ndrs0AbzoRfDRj7Z9mI6tiomIXcAugP7+/k4NKy3r0AKhvtT+tW7TU09k01MN8uNZHcE+AZw65/EpzX3HyMwhYAig0WjkqkfrwE87rVwJc9Pv+cANC85Lb+3r5b8uPbsLFUnV1DHH/m3guRFxWkScAFwIXFVDv1rjSpmbdl5apakc7Jl5BHgnMAwcAL6UmXdU7VdrXylrpndu38r7X/9Ctvb1Esycqb//9S9cd795SLNqmWPPzKuBq+voS+tHSWumd27fapCrGL7zVKu22Npo10xL3WWwa9Wcm5bWJv8ImFZtdupiva+KkUpjsKsS56altcepGEkqjMEuSYUx2CWpMAa7JBXGYJekwhjsklQYg12SCmOwS1JhDHZJKozBLkmFMdglqTAGuyQVplKwR8RgRNwVEbdHxFcjYn3d0l2SClT1jP064AWZ+ZvAd4E91UuSJFVRKdgz89rmPU8BbgJOqV6SJKmKOufY3wJcU2N/kqRVWPZGGxFxPXDyAk9dlplfa7a5DDgCXLFEP7uAXQD9/f2rKlaStLxlgz0zz13q+Yi4CHgtcE5m5hL9DAFDAI1GY9F2x4N9oxPeTk5S21S6NV5EnAe8F3hlZv6snpLKtm90gj17x5g+fBSAialp9uwdAzDcJdWi6hz7x4GnAddFxP6I+GQNNRVtcHj88VCfNX34KIPD412qSFJpKp2xZ+av1lXI8eLg1HRL+yWpVb7ztMO29PW2tF+SWmWwd9juHQP0buw5Zl/vxh527xjoUkWSSlNpKkatm71A6qoYSe1isHfBzu1bDXJJbeNUjCQVxmCXpMIY7JJUGINdkgpjsEtSYQx2SSqMwS5JhTHYJakwBrskFcZgl6TCGOySVBiDXZIKU0uwR8R7IiIjYlMd/UmSVq9ysEfEqcBrgPurlyNJqqqOM/a/Z+aG1llDX5KkiioFe0RcAExk5m0raLsrIkYiYmRycrLKsJKkJSx7o42IuB44eYGnLgP+mplpmGVl5hAwBNBoNDy7l6Q2WTbYM/PchfZHxAuB04DbIgLgFODWiDgzM39Qa5WSpBVb9a3xMnMMeObs44i4F2hk5iM11CVJWiXXsUtSYWq7mXVmbqurL0nS6nnGLkmFMdglqTAGuyQVxmCXpMIY7JJUGINdkgpjsEtSYQx2SSqMwS5JhTHYJakwBrskFcZgl6TCGOySVBiDXZIKY7BLUmEqB3tEvCsi7oqIOyLiQ3UUJUlavUo32oiIVwMXAL+VmY9FxDOX+z+SpPaqesb+DuADmfkYQGY+XL0kSVIVVYP9ecDvRsTNEfHvEfHiOoqSJK3eslMxEXE9cPICT13W/P8nAWcBLwa+FBGnZ2Yu0M8uYBdAf39/lZolSUtYNtgz89zFnouIdwB7m0H+rYj4BbAJmFygnyFgCKDRaDwh+CVJ9ag6FbMPeDVARDwPOAF4pGpRkqTVq7QqBvg08OmI+A5wCHjTQtMwkqTOqRTsmXkIeGNNtUiSauA7TyWpMAa7JBXGYJekwhjsklQYg12SCmOwS1JhDHZJKozBLkmFMdglqTAGuyQVxmCXpMIY7JJUGINdkgpjsEtSYQx2SSqMwS5JhakU7BHxooi4KSL2R8RIRJxZV2GSpNWpemu8DwF/m5nXRMT5zcevqlzVIvaNTjA4PM7BqWm29PWye8cAO7dvbddwkrQuVQ32BH65uf104GDF/ha1b3SCPXvHmD58FICJqWn27B0DMNwlaY6qc+yXAIMR8QDwYWBP9ZIWNjg8/nioz5o+fJTB4fF2DSlJ69KyZ+wRcT1w8gJPXQacA7w7M78SEX8MfAo4d5F+dgG7APr7+1su9ODUdEv7Jel4tWywZ+aCQQ0QEZ8DLm4+/DJw+RL9DAFDAI1GI1srE7b09TKxQIhv6etttStJKlrVqZiDwCub22cD36vY36J27xigd2PPMft6N/awe8dAu4aUpHWp6sXTPwM+FhEbgJ/TnGpph9kLpK6KkaSlRWbLsyKVNRqNHBkZ6fi4krSeRcQtmdlYrp3vPJWkwhjsklQYg12SCmOwS1JhDHZJKkxXVsVExCRwX4UuNgGP1FROnaxr5dZiTWBdrbKu1lSt6zmZuXm5Rl0J9qoiYmQlS346zbpWbi3WBNbVKutqTafqcipGkgpjsEtSYdZrsA91u4BFWNfKrcWawLpaZV2t6Uhd63KOXZK0uPV6xi5JWsS6CPaIGIyIuyLi9oj4akT0LdLuvIgYj4i7I+LSDtT1RxFxR0T8IiIWvdIdEfdGxNjsTb/XUF0dO14RcVJEXBcR32v++4xF2h1tHqf9EXFVG+tZ8rVHxIkR8cXm8zdHxLZ21dJiXRdFxOScY/S2DtT06Yh4OCK+s8jzERH/0Kz59og4o901rbCuV0XEo3OO1d90qK5TI+IbEXFn8/vw4gXatPeYZeaa/wBeA2xobn8Q+OACbXqA7wOnAycAtwHPb3Ndvw4MADcCjSXa3Qts6uDxWrauTh8vZm50fmlz+9KFPofN537ageOz7GsH/gL4ZHP7QuCLa6Sui4CPd+prqTnmK4AzgO8s8vz5wDVAAGcBN6+Rul4FfL2Tx6o57rOBM5rbTwO+u8Dnsa3HbF2csWfmtZl5pPnwJuCUBZqdCdydmfdk5iHgC8AFba7rQGauuZuurrCuTh+vC4DPNrc/C+xs41jLWclrn1vvlcA5ERFroK6Oy8xvAj9aoskFwOdyxk1AX0Q8ew3U1RWZ+VBm3trc/glwAJh/44i2HrN1EezzvIWZn3TzbQUemPP4QZ54MLslgWsj4pbmvV/Xgk4fr2dl5kPN7R8Az1qk3ZMiYiQiboqIdoX/Sl77422aJxWPAr/SpnpaqQvgD5u/vl8ZEae2uaaVWMvfey+JiNsi4pqI+I1OD96cwtsO3DzvqbYes6p3UKrNUjfNzsyvNdtcBhwBrlhLda3AyzNzIiKeCVwXEXc1zza6XVetlrnx+eMyMyNiseVYz2keq9OBGyJiLDO/X3et69i/AJ/PzMci4s+Z+a3i7C7XtFbdyszX008j4nxgH/DcTg0eEU8FvgJckpk/7tS4sIaCPZe4aTbMXDQCXguck81JqnkmgLlnL6c097W1rhX2MdH89+GI+Cozv3JXCvYa6qr9eC1VU0T8MCKenZkPNX/lfHiRPmaP1T0RcSMzZzt1B/tKXvtsmwdj5taPTwf+p+Y6Wq4rM+fWcDkz1y66rS3fe1XNDdPMvDoi/jEiNmVm2/+GTERsZCbUr8jMvQs0aesxWxdTMRFxHvBe4HWZ+bNFmn0beG5EnBYRJzBzwattqypWKiKeEhFPm91m5kLwglfxO6zTx+sq4E3N7TcBT/itIiKeEREnNrc3AS8D7mxDLSt57XPrfQNwwyInFB2ta9487OuYmb/ttquAP22u9DgLeHTOtFvXRMTJs9dFIuJMZvKu3T+caY75KeBAZn5kkWbtPWadvmK8mg/gbmbmo/Y3P2ZXK2wBrp7T7nxmrkB/n5kpiXbX9QfMzI09BvwQGJ5fFzMrHG5rftyxVurq9PFiZn7634DvAdcDJzX3N4DLm9svBcaax2oMeGsb63nCawf+jpmTB4AnAV9ufu19Czi93Z+3Fdb1/ubX0W3AN4Bf60BNnwceAg43v67eCrwdeHvz+QA+0ax5jCVWiHW4rnfOOVY3AS/tUF0vZ+a62u1zMuv8Th4z33kqSYVZF1MxkqSVM9glqTAGuyQVxmCXpMIY7JJUGINdkgpjsEtSYQx2SSrM/wEQqBI8czcrOgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x114fc3470>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "xx = np.linspace(-2, 2, 100)\n",
    "predictions = [np.mean(y) for x in xx]\n",
    "\n",
    "X = np.linspace(-2, 2, 7)\n",
    "y = X ** 3\n",
    "\n",
    "plt.scatter(X, y);\n",
    "plt.plot(xx, predictions, c='red');"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Произведем первое разбиение выборки по предикату $[x < 0]$. Получим дерево глубины 1 с двумя листьями. Постройте аналогичный график предсказаний для этого дерева."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXYAAAD8CAYAAABjAo9vAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAEzxJREFUeJzt3X+QXWV9x/H3181GV0BXTRQTCIEpxlJtG2Zl8EfREsZQ6khqbaUztuKPpmpFdJw4pMzUaf+xGseWjrZOBml1ij8xRupAA1Sx0z+ILCyw/IoiRWEDsrSz+GtNNuHbP+69cRN2sz/OuffuffJ+zezsueeePM/3nmw+efY5z70nMhNJUjme0e0CJEn1MtglqTAGuyQVxmCXpMIY7JJUGINdkgpjsEtSYQx2SSqMwS5JhVnWjU5XrFiRa9eu7UbXktSzbrvtticyc+Vcx3Ul2NeuXcvw8HA3upaknhURP5zPcU7FSFJhDHZJKozBLkmFMdglqTAGuyQVppZVMRHxQeBdQAKjwNsz85d1tC1JvW7nyBjbdu1h78QkqwYH2LJxHZvWr25bf5VH7BGxGng/MJSZLwP6gIuqtitJJdg5MsbWHaOMTUySwNjEJFt3jLJzZKxtfdY1FbMMGIiIZcCzgb01tStJPW3brj1MTh08bN/k1EG27drTtj4rB3tmjgGfAH4EPAo8mZk3HHlcRGyOiOGIGB4fH6/arST1hL0TkwvaX4c6pmKeB1wInAqsAo6LiLceeVxmbs/MocwcWrlyznfESlIRVg0OLGh/HeqYijkP+J/MHM/MKWAH8Koa2pWknrdl4zoG+vsO2zfQ38eWjeva1mcdq2J+BJwdEc8GJoENgB8EI0lwaPVLJ1fFVA72zNwdEdcAtwMHgBFge9V2JakUm9avbmuQH6mWdeyZ+RHgI3W0JUmqxneeSlJhDHZJKozBLkmFMdglqTAGuyQVxmCXpMIY7JJUGINdkgpjsEtSYQx2SSqMwS5JhTHYJakwBrskFcZgl6TCGOySVJhagj0iBiPimoi4PyLui4hX1tGuJGnharnRBnAF8B+Z+eaIWA48u6Z2JUkLVDnYI+K5wDnAxQCZuR/YX7VdSdLi1DEVcyowDvxLRIxExJURcVwN7UqSFqGOYF8GnAn8c2auB34OXHbkQRGxOSKGI2J4fHy8hm4lSTOpI9gfAR7JzN3Nx9fQCPrDZOb2zBzKzKGVK1fW0K0kaSaVgz0zHwMejoh1zV0bgHurtitJWpy6VsVcAlzdXBHzIPD2mtqVJC1QLcGemXcAQ3W0JUmqxneeSlJhDHZJKozBLkmFMdglqTAGuyQVxmCXpMIY7JJUGINdkgpjsEtSYQx2SSqMwS5JhTHYJakwBrskFcZgl6TCGOySVBiDXZIKU1uwR0RfRIxExDfralOStHB1jtgvBe6rsT1J0iLUEuwRcRLw+8CVdbQnSVq8ukbs/wB8GHiqpvYkSYtUOdgj4g3A45l52xzHbY6I4YgYHh8fr9qtJGkWdYzYXw28MSIeAr4EnBsR/3bkQZm5PTOHMnNo5cqVNXQrSZpJ5WDPzK2ZeVJmrgUuAr6VmW+tXJkkaVFcxy5JhVlWZ2OZeTNwc51tSpIWxhG7JBXGYJekwhjsklQYg12SCmOwS1JhDHZJKozBLkmFMdglqTAGuyQVxmCXpMLU+pECUs+79Vb4whe6XYVKdfbZ8Ja3tL0bg12a7oorGsF+wgndrkQleuopg13quH374KUvhXvv7XYlx7ydI2Ns27WHvROTrBocYMvGdWxav7rbZfUEg12abmoK+vu7XcUxb+fIGFt3jDI5dRCAsYlJtu4YBTDc58GLp9J0BvuSsG3XnkOh3jI5dZBtu/Z0qaLeYrBL0xnsS8LeickF7dfh6riZ9ckR8e2IuDci7omIS+soTOoKg31JWDU4sKD9OlwdI/YDwIcy8wzgbOAvI+KMGtqVOs9gXxK2bFzHQH/fYfsG+vvYsnFdlyrqLZUvnmbmo8Cjze2fRsR9wGrAZQXqPVNTcPzx3a7imNe6QOqqmMWpdVVMRKwF1gO7Z3huM7AZYM2aNXV2K9XHEfuSsWn9aoN8kWq7eBoRxwNfAz6QmT858vnM3J6ZQ5k5tHLlyrq6leo1NQXLl3e7CqmSWoI9IvpphPrVmbmjjjalrti/3xG7el4dq2IC+CxwX2Z+snpJUhc5FaMC1DFifzXwp8C5EXFH8+uCGtqVOs9gVwHqWBXz30DUUIvUfQa7CuA7T6XpDHYVwGCXpjPYVQCDXZrOYFcBDHZpOoNdBTDYpZZMOHjQYFfPM9illqmpxneDXT3OYJdaDHYVwmCXWvbvb3w32NXjDHappTVi90PA1OMMdqnFqRgVwmCXWgx2FcJgl1oMdhWi1jsoST2tkGDfOTLmLeWOcQa71FJAsO8cGWPrjlEmpw4CMDYxydYdowCG+zHEqRippYBg37Zrz6FQb5mcOsi2XXu6VJG6oa5b450fEXsi4oGIuKyONqWOKyDY905MLmi/ylTHrfH6gE8DvwecAfxJRJxRtV2pk3aOjPHef90NwPuvuZudI2NdrmhxVg0OLGi/ylTHiP0s4IHMfDAz9wNfAi6soV2pI1rz0hNP/hyAxyYPsnXHaE+G+5aN6xjo7zts30B/H1s2rutSReqGOoJ9NfDwtMePNPdJPaE1L7384AEADjyjr2fnpTetX81H3/RyVg8OEMDqwQE++qaXe+H0GNOxVTERsRnYDLBmzZpOdSvNqTX/vOypxkXHqWcsO2x/r9m0frVBfoyrI9jHgJOnPT6pue8wmbkd2A4wNDSUNfSrJaCENdOrBgcYm5hk2VPNEXtf36H9Ui+qYyrmVuD0iDg1IpYDFwHX1tCulrjW3PTYxCTJr9ZM99rcdGteur85FbP/Gf3OS6unVR6xZ+aBiHgfsAvoA67KzHsqV6Yl72hrpntp1N6q9a493wHgBYPHcYnz0uphtcyxZ+Z1wHV1tKXeUdKa6U3rV7Pp/NPhq/CVS86BUwx19S7feapFK27NdAFvUJLAYFcFxa2ZNthVCD8ETIvWmoPu9VUxhxjsKoTBrkqKWjNtsKsQTsVILQa7CmGwSy379ze+G+zqcQa71DI1BX19ENHtSqRKDHapZWrK0bqKYLBLLQa7CmGwSy0GuwphsEstU1OwfHm3q5AqM9ilFkfsKoTBLrUY7CqEwS61GOwqhMEutRjsKoTBLrUY7CpEpWCPiG0RcX9E3BURX4+IwboKkzpu/36DXUWoOmK/EXhZZv4m8D1ga/WSpC5xxK5CVAr2zLwhMw80H94CnFS9JKlLDHYVos459ncA19fYntRZBrsKMeeNNiLiJuDEGZ66PDO/0TzmcuAAcPVR2tkMbAZYs2bNooqV2spgVyHmDPbMPO9oz0fExcAbgA2ZmUdpZzuwHWBoaGjW444FO0fGyrmdXEkMdhWi0q3xIuJ84MPAazPzF/WUVLadI2Ns3THK5NRBAMYmJtm6YxTAcO82PytGhag6x/4p4ATgxoi4IyI+U0NNRdu2a8+hUG+ZnDrItl17ulSRDnHErkJUGrFn5q/VVcixYu/E5IL2q4MMdhXCd5522KrBgQXtVwcZ7CqEwd5hWzauY6C/77B9A/19bNm4rksV6RCDXYWoNBWjhWtdIHVVzBLkRwqoEAZ7F2xav9ogX4ocsasQTsVILQa7CmGwSwCZcOCAwa4iGOwSNEIdDHYVwWCXoDENAwa7imCwS2CwqygGuwQGu4pisEvwq2D3Q8BUAINdAkfsKorBLoHBrqL03jtPN2yAJ57odhUqzb59je8GuwrQe8F+yinwnOd0uwqV6Mwz4Zxzul2FVFnvBftVV3W7Akla0mqZY4+ID0VERsSKOtqTJC1e5WCPiJOB1wM/ql6OJKmqOkbsf0/jhtZZQ1uSpIoqBXtEXAiMZead8zh2c0QMR8Tw+Ph4lW4lSUcx58XTiLgJOHGGpy4H/orGNMycMnM7sB1gaGjI0b0ktcmcwZ6Z5820PyJeDpwK3BkRACcBt0fEWZn5WK1VSpLmbdHLHTNzFHhh63FEPAQMZabvHpKkLvIjBSSpMLW9QSkz19bVliRp8RyxS1JhDHZJKozBLkmFMdglqTAGuyQVxmCXpMIY7JJUGINdkgpjsEtSYQx2SSqMwS5JhTHYJakwBrskFcZgl6TCGOySVJjKwR4Rl0TE/RFxT0R8vI6iJEmLV+lGGxHxu8CFwG9l5r6IeOFcf0aS1F5VR+zvAf4uM/cBZObj1UuSJFVRNdhfAvxOROyOiO9ExCvqKEqStHhzTsVExE3AiTM8dXnzzz8fOBt4BfCViDgtM3OGdjYDmwHWrFlTpWZJ0lHMGeyZed5sz0XEe4AdzSD/bkQ8BawAxmdoZzuwHWBoaOhpwS9JqkfVqZidwO8CRMRLgOXAE1WLkiQtXqVVMcBVwFURcTewH3jbTNMwkqTOqRTsmbkfeGtNtUiSauA7TyWpMAa7JBXGYJekwhjsklQYg12SCmOwS1JhDHZJKozBLkmFMdglqTAGuyQVxmCXpMIY7JJUGINdkgpjsEtSYQx2SSqMwS5JhakU7BHx2xFxS0TcERHDEXFWXYVJkhan6q3xPg78TWZeHxEXNB+/rnJVs9g5Msa2XXvYOzHJqsEBtmxcx6b1q9vVnST1pKrBnsBzmtvPBfZWbG9WO0fG2LpjlMmpgwCMTUyydccogOEuSdNUnWP/ALAtIh4GPgFsrV7SzLbt2nMo1Fsmpw6ybdeednUpST1pzhF7RNwEnDjDU5cDG4APZubXIuKPgc8C583SzmZgM8CaNWsWXOjeickF7ZekY9WcwZ6ZMwY1QER8Hri0+fCrwJVHaWc7sB1gaGgoF1YmrBocYGyGEF81OLDQpiSpaFWnYvYCr21unwt8v2J7s9qycR0D/X2H7Rvo72PLxnXt6lKSelLVi6d/DlwREcuAX9KcammH1gVSV8VI0tFF5oJnRSobGhrK4eHhjvcrSb0sIm7LzKG5jvOdp5JUGINdkgpjsEtSYQx2SSqMwS5JhenKqpiIGAd+WKGJFcATNZVTJ+uav6VYE1jXQlnXwlSt65TMXDnXQV0J9qoiYng+S346zbrmbynWBNa1UNa1MJ2qy6kYSSqMwS5JhenVYN/e7QJmYV3ztxRrAutaKOtamI7U1ZNz7JKk2fXqiF2SNIueCPaI2BYR90fEXRHx9YgYnOW48yNiT0Q8EBGXdaCuP4qIeyLiqYiY9Up3RDwUEaOtm34vobo6dr4i4vkRcWNEfL/5/XmzHHeweZ7uiIhr21jPUV97RDwzIr7cfH53RKxtVy0LrOviiBifdo7e1YGaroqIxyPi7lmej4j4x2bNd0XEme2uaZ51vS4inpx2rv66Q3WdHBHfjoh7m/8OL53hmPaes8xc8l/A64Flze2PAR+b4Zg+4AfAacBy4E7gjDbX9evAOuBmYOgoxz0ErOjg+Zqzrk6fLxo3Or+suX3ZTH+Hzed+1oHzM+drB94LfKa5fRHw5SVS18XApzr1s9Ts8xzgTODuWZ6/ALgeCOBsYPcSqet1wDc7ea6a/b4YOLO5fQLwvRn+Htt6znpixJ6ZN2TmgebDW4CTZjjsLOCBzHwwM/cDXwIubHNd92Xmkrvp6jzr6vT5uhD4XHP7c8CmNvY1l/m89un1XgNsiIhYAnV1XGb+F/B/RznkQuDz2XALMBgRL14CdXVFZj6ambc3t38K3AcceeOItp6zngj2I7yDxv90R1oNPDzt8SM8/WR2SwI3RMRtzXu/LgWdPl8vysxHm9uPAS+a5bhnRcRwRNwSEe0K//m89kPHNAcVTwIvaFM9C6kL4A+bv75fExEnt7mm+VjK//ZeGRF3RsT1EfEbne68OYW3Hth9xFNtPWdV76BUm6PdNDszv9E85nLgAHD1UqprHl6TmWMR8ULgxoi4vzna6HZdtZrjxueHZGZGxGzLsU5pnqvTgG9FxGhm/qDuWnvYvwNfzMx9EfEXNH6rOLfLNS1Vt9P4efpZRFwA7ARO71TnEXE88DXgA5n5k071C0so2PMoN82GxkUj4A3AhmxOUh1hDJg+ejmpua+tdc2zjbHm98cj4us0fuWuFOw11FX7+TpaTRHx44h4cWY+2vyV8/FZ2midqwcj4mYao526g30+r711zCPRuPXjc4H/rbmOBdeVmdNruJLGtYtua8u/vaqmh2lmXhcR/xQRKzKz7Z8hExH9NEL96szcMcMhbT1nPTEVExHnAx8G3piZv5jlsFuB0yPi1IhYTuOCV9tWVcxXRBwXESe0tmlcCJ7xKn6Hdfp8XQu8rbn9NuBpv1VExPMi4pnN7RXAq4F721DLfF779HrfDHxrlgFFR+s6Yh72jTTmb7vtWuDPmis9zgaenDbt1jURcWLrukhEnEUj79r9nzPNPj8L3JeZn5zlsPaes05fMV7MF/AAjfmoO5pfrdUKq4Drph13AY0r0D+gMSXR7rr+gMbc2D7gx8CuI+uiscLhzubXPUulrk6fLxrz0/8JfB+4CXh+c/8QcGVz+1XAaPNcjQLvbGM9T3vtwN/SGDwAPAv4avNn77vAae3+e5tnXR9t/hzdCXwbeGkHavoi8Cgw1fy5eifwbuDdzecD+HSz5lGOskKsw3W9b9q5ugV4VYfqeg2N62p3TcusCzp5znznqSQVpiemYiRJ82ewS1JhDHZJKozBLkmFMdglqTAGuyQVxmCXpMIY7JJUmP8H7ACerYIcgLYAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1159636a0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "xx = np.linspace(-2, 2, 200)\n",
    "predictions = [np.mean(y[X < 0]) if x < 0 else np.mean(y[X >= 0])\n",
    "              for x in xx]\n",
    "\n",
    "X = np.linspace(-2, 2, 7)\n",
    "y = X ** 3\n",
    "\n",
    "plt.scatter(X, y);\n",
    "plt.plot(xx, predictions, c='red');"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "В алгоритме построения дерева решений признак и значение порога, по которым происходит разбиение выборки, выбираются исходя из некоторого критерия. Для регрессии обычно используется дисперсионный критерий:\n",
    "$$Q(X, j, t) = D(X) - \\dfrac{|X_l|}{|X|} D(X_l) - \\dfrac{|X_r|}{|X|} D(X_r),$$\n",
    "где $X$ – выборка, находящаяся в текущей вершине, $X_l$ и $X_r$ – разбиение выборки $X$ на две части по предикату $[x_j < t]$ (то есть по $j$-ому признаку и порогу $t$), $|X|$, $|X_l|$, $|X_r|$ - размеры соответствующих выборок, а $D(X)$ – дисперсия ответов на выборке $X$:\n",
    "$$D(X) = \\dfrac{1}{|X|} \\sum_{x_j \\in X}(y_j – \\dfrac{1}{|X|}\\sum_{x_i \\in X}y_i)^2,$$\n",
    "где $y_i = y(x_i)$ – ответ на объекте $x_i$. При каждом разбиении вершины выбираются признак $j$ и значение порога $t$, максимизирующие значение функционала $Q(X, j, t)$.\n",
    "\n",
    "В нашем случае признак всего один, поэтому $Q$ зависит только от значения порога $t$ (и ответов выборки в данной вершине). \n",
    "\n",
    "Постройте график функции $Q(X, t)$ в корне в зависимости от значения порога $t$ на отрезке $[-1.9, 1.9]$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "def regression_var_criterion(X, y, t):\n",
    "    X_left, X_right = X[X < t], X[X >= t]\n",
    "    y_left, y_right = y[X < t], y[X >= t]\n",
    "    return np.var(y) - X_left.shape[0] / X.shape[0] * np.var(y_left) - X_right.shape[0] / X.shape[0] * np.var(y_right)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAEKCAYAAADjDHn2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzt3X20XHV97/H3Z+YE8AFMAiFGIEQUkaJC4RSQJ7kgqBQFH1tKa/SCyAVbXPe6ary6sEv/KEjrspWqpUABQS5WRfCZmAr4AGiCAQIoASQQCCRKeEbgzHzvH3tPMpzMOTOcnJn5zfw+r7XOOnv27Jn9zd4n853v/j1sRQRmZmaTqfQ7ADMzS5+ThZmZteVkYWZmbTlZmJlZW04WZmbWlpOFmZm15WRhZmZtOVmYmVlbThZmZtbWSLfeWNL5wNHA2oh4XbluNnAZsAC4B3hfRKxv8drPAX9OkcwWA6dFm6Hm2223XSxYsGAa/wVmZsNv2bJlv4+IOe2261qyAC4AzgYualq3CFgSEWdIWlQ+/njziyQdABwIvKFc9TPgTcDVk+1swYIFLF26dFoCNzPLhaRVnWzXtctQEXEt8PC41ccAF5bLFwLHtnopsBWwBbAlMAN4qEthmplZB3rdZjE3ItaUyw8Cc8dvEBHXAT8B1pQ/P4qI21u9maSTJC2VtHTdunXditnMLHt9a+Au2yA2aYeQ9Gpgd2BHYAfgMEkHT/Ae50TEaESMzpnT9pKbmZlNUa+TxUOS5gGUv9e22OadwPUR8UREPAH8AHhjD2M0M7Nxep0srgQWlssLgStabHMv8CZJI5JmUDRut7wMZWZmvdG1ZCHpUuA6YDdJqyWdAJwBHCFpJfDm8jGSRiWdW770G8BdwC3ATcBNEfGdbsVpZmbtda3rbEQcN8FTh7fYdilwYrlcAz7crbjMzOyF6+Y4C+uyK5bfz11rn+h3GNNq/1dtywGv2q7fYVgHLr5+FWsf+2O/w5hWR+7xcl63w8v6HUaSnCwG1GN/fI6PXracCJD6Hc30iICr71jHlR85qN+hWBtrH/8jn/r2CmC4/v5+dc96Lj1p/36HkiQniwG1/N5HiICLT9iPg3Ydjm/iJ120lHsffqrfYVgHnh2rA/C597yB943u1Odopsc/XHkrl/3qPsZqdUaqnjZvPB+RAXXjveuRYM+dhqdkrlZErT7pFGCWiMZ5qg5LWQH86fyZPP1cjd88+Hi/Q0mSk8WAWrZqPbvN3Zqtt5rR71CmjZPF4Bgrz9NIdXiSxT47zwKKL2K2KSeLAVSrB8vvfWTDH/ewGKlow4eQpW1DZVEZnmSxw8wXMXebLVm2ysmiFSeLAbRy7eM8/swYe88frmRRrVRcWQyIxnkaGaJkIYm9589yspiAk8UAunHVIwBDWVk4WQyGjZXFcH2E7LPzLFavf3rougRPh+E605lYtmo9275kC3be9sX9DmVaVau+DDUoxoawsgDY2+0WE3KyGEA33ruevXeehYaoJwo0Kot6v8OwDjTO0zC1WQDs8Ypt2GKk4ktRLThZDJiHn3yW3/3+yaFrr4Dig8eVxWAYqw1nZbHlSJXX7/Aybrz3kX6HkhwniwFzY/mNZ9jaK8BtFoNkGHtDNeyz8yxuWf0oz4zV+h1KUpwsBsyye9czUhFv2HF4BuM1VCsVVxYDYhjHWTTsPX8Wz9bqrLj/sX6HkhQniwGzbNV69njFNmw1o9rvUKadK4vBMay9oQD23nkmsLGKt0L2c0Otf/JZjv7iz/odRsfWPPo0Cw9Y0O8wuqIxgjsihq7xftgMa28ogO233oqdZr+IL/z4Di74xT39Dqcje7xiG855/2hX95F9shipiv132bbfYXSsWoHj99u532F0ReODp1aPoby8MUyGtTdUwyfetjtLbm911+c09aIbffbJYuutZvDP79uz32EYxTgLKL61jgzfVbahMsyVBcBRr5/HUa+f1+8wkjJ8FxxtYDVXFpa2Ye4NZa05WVgyGo2l7hGVvo3jLPwRkgufaUuGK4vBsaGycNtSNpwsLBmNSxpjnvIjecPeZmGbcrKwZLiyGBzD3hvKNtW1ZCHpfElrJa1oWjdb0mJJK8vfLeeskDRf0lWSbpd0m6QF3YrT0rGhsqg5WaTOlUV+ullZXAC8ddy6RcCSiNgVWFI+buUi4KyI2B3YFxicDs82ZY2xFa4s0ufeUPnpWrKIiGuBh8etPga4sFy+EDh2/Osk/QkwEhGLy/d5IiKe6laclg73hhocGysLX8nORa/P9NyIWFMuPwjMbbHNa4BHJH1L0q8lnSWp5RAtSSdJWipp6bp167oVs/WI2ywGhyuL/PTta0FEBNDqU2EEOBj4GPBnwC7AByZ4j3MiYjQiRufMmdOtUK1H3BtqcAzr/SxsYr1OFg9JmgdQ/m7VFrEaWB4Rd0fEGPBtYO8exmh94spicNTqdSSoOFlko9fJ4kpgYbm8ELiixTa/AmZKapQKhwG39SA267ONlYWTRerG6uGqIjPd7Dp7KXAdsJuk1ZJOAM4AjpC0Enhz+RhJo5LOBYiIGsUlqCWSbgEE/Ee34rR0NBpLXVmkr1YPt1dkpmuzzkbEcRM8dXiLbZcCJzY9Xgy8oUuhWaI8zmJwFJWFe0LlxGfbkuFxFoPDlUV+nCwsGe4NNTjG6nW3WWTGycKS4d5Qg8OVRX6cLCwZ7g01OMZq7g2VGycLS4Z7Qw2OWj18L4vMOFlYMlxZDA73hsqPz7YlY2ObhRu4U+c2i/w4WVgyPM5icLg3VH6cLCwZHmcxOFxZ5MfJwpLhNovB4bmh8uNkYclwb6jB4coiP04WlgxXFoOjGGfhj4+c+GxbMtwbanC4ssiPk4Ulw5XF4Bir1zd0SLA8OFlYMjZUFu46mzxXFvlxsrBkuLIYHO4NlR8nC0uGJKoVuTfUAHBlkR8nC0tKtSJXFgPAc0Plx2fbkjJSkXtDDQBXFvlxsrCkuLIYDJ4bKj9OFpaUEbdZDIRazZVFbkbabSBpS+DdwILm7SPiM90Ly3JVrVRcWQyAsXp4nEVmOqksrgCOAcaAJ5t+JiXpfElrJa1oWjdb0mJJK8vfsyZ5/TaSVks6u4MYbUiMVORxFgPAbRb5aVtZADtGxFun8N4XAGcDFzWtWwQsiYgzJC0qH398gtd/Frh2Cvu1AeY2i8Hg3lD56eRs/0LS61/oG0fEtcDD41YfA1xYLl8IHNvqtZL2AeYCV73Q/dpgG6m6N9QgcGWRn06SxUHAMkm/lXSzpFsk3TzF/c2NiDXl8oMUCeF5JFWAfwY+1u7NJJ0kaamkpevWrZtiSJaSqlxZDIKxet3JIjOdXIZ6Wzd2HBEhqdWnwinA9yNitTT5H2NEnAOcAzA6OupPmCHgEdyDwZVFftomi4hYJWlP4OBy1U8j4qYp7u8hSfMiYo2kecDaFtu8EThY0inAS4EtJD0REYumuE8bIE4Wg6HmuaGy0/YylKTTgEuA7cufiyX97RT3dyWwsFxeSNHT6nki4viImB8RCyguRV3kRJGPos3CySJl9XpQD1xZZKaTNosTgP0i4vSIOB3YH/hQuxdJuhS4Dtit7AJ7AnAGcISklcCby8dIGpV07lT/ETY8PM4ifbUozo8ri7x00mYhoNb0uFaum1REHDfBU4e32HYpcGKL9RdQdMG1THgEd/oa56fqrrNZ6SRZ/Cdwg6TLy8fHAud1LyTLWTHOwl1nU9ao/FxZ5KWTBu7PS7qaogstwAcj4tddjcqyNVIRz9WcLFLWGGHvNou8TJgsJG0TEY9Jmg3cU/40npsdEeMH3JlttmpFPP2cL0OlrFH5eW6ovExWWXwNOBpYBjT/71X5eJcuxmWZcptF+ja2WThZ5GTCZBERR6sYFfemiLi3hzFZxqqVCmOeSDBpbrPI06TdGSIigO/1KBYzVxYDwL2h8tTJ2b5R0p91PRIzoFp1b6jUubLIUyddZ/cDjpe0iuI+FqIoOt7Q1cgsS64s0teYFdhtFnnpJFm8petRmJV8P4v0ubLIU9vLUBGxCtgJOKxcfqqT15lNhSuL9I15nEWWOplI8NMUd7P7RLlqBnBxN4OyfHluqPQ1krnHWeSlkwrhncA7KO+7HREPAFt3MyjLlyuL9I25N1SWOjnbz5ZdaANA0ku6G5LlrFoRY57uI2k1t1lkqZNk8XVJ/w7MlPQh4MeApxO3rnBlkb4x94bKUicTCf6TpCOAx4DdgNMjYnHXI7MsFeMsnCxS5soiT22ThaQzI+LjwOIW68ymlSuL9I15bqgsdXIZ6ogW69423YGYwcbeUBFOGKlqTFE+4gburEw2Rfn/Ak4BdpF0c9NTWwM/73ZglqfGpY16gHtmpsmVRZ7aTVH+A+AfgUVN6x/3vSysWxofQGP1OtVKtc/RWCseZ5GnyZJFRMQ9kk4d/4RvfmTd0qgs3G6RLveGytMLuflR81+Gb35kXbGxsnCySJV7Q+XJNz+ypGyoLHwDpGS5zSJPXbv5kaTzJa2VtKJp3WxJiyWtLH/PavG6vSRdJ+lWSTdL+oup7N8GU7Va/Em6skjXxsrCvaFy0s2bH10AvHXcukXAkojYFVjC8xvOG54C3h8Re5Sv/4KkmVPYvw0gt1mkz5VFnjpJFvsB10m6q/ymf8u4rrQtRcS1wPhG8GOAC8vlC4FjW7zujohYWS4/AKwF5nQQpw2B5t5QlqZaOXeX2yzy0uubH82NiDXl8oPA3Mk2lrQvsAVw1wTPnwScBDB//vxpDNP6xZVF+jZUFu46m5VOKot5wMMRsaq8+dF64OWbu+PmmWxbkTQP+CrwwYho+TUzIs6JiNGIGJ0zx8XHMHBvqPS5N1SeOkkWXwaeaHr8RLluKh4qk0AjGaxttZGkbSga1j8ZEddPcV82gBqNpq4s0uU2izx1kiwUTRP1lN/yO7l81cqVwMJyeSFwxSY7k7YALgcuiohvTHE/NqA2VBbuOpss94bKUydn+25JfydpRvlzGnB3uxdJuhS4DthN0mpJJwBnAEdIWgm8uXyMpFFJjXtkvA84BPiApOXlz15T+LfZAHKbRfoalYULi7x0UiGcDPwr8CmKNoYllI3Kk4mI4yZ46vAW2y4FTiyXL8b3+M5Wo9HUvaHSVavXGamIYsyu5aKTmx+tBf6yB7GYubIYAGP1cHtFhnzR0ZLi3lDpq9XCPaEy5GRhSXFvqPS5ssiTk4UlxZVF+mr1YKTqj47cdHIP7i2BdwMLmrePiM90LyzL1cY2Czdwp8qVRZ466Q11BfAoxX0tnuluOJY7j7NIX6M3lOWlk2SxY0SMnz3WrCsat+p0m0W6XFnkqZMLj7+Q9PquR2LGxstQbrNIV63u3lA56qSyOIhiNPXvKC5DiWIewDd0NTLLUtW9oZLnyiJPnSSLt3U9CrOSK4v0FeMs3BsqN23PeDkt+Uzg7eXPzHKd2bSrujdU8lxZ5KltsignDrwE2L78uVjS33Y7MMuTK4v01er1DR0RLB+dXIY6AdgvIp4EkHQmxWyyX+xmYJanqueGSp4rizx1dD8LoNb0uFauM5t2jWvhHmeRLveGylMnlcV/AjdIurx8fCxwXvdCspxVPc4iea4s8tTJFOWfl3Q1RRdaKO6J/euuRmXZqsptFqmr1YMZM9wbKjcTJgtJ20TEY5JmA/eUP43nZkfEw90Pz3Lj3lDpKyoLJ4vcTFZZfA04mmJOqOaveSof79LFuCxT7g2VPs8NlacJk0VEHF3+fmXvwrHcVSpCgrqTRbJqddxmkaFOxlkcKOkl5fJfS/q8pPndD81yNVKRK4uEubLIUycXHr8MPCVpT+D/AHcBX+1qVJa1akXuDZUw94bKUyfJYiwiAjgGODsi/g3YurthWc5GKhVXFgnzOIs8dZIsHpf0CeCvge9JqgAz2r1I0vmS1kpa0bRutqTFklaWv2dN8NqF5TYrJS3s9B9jw8GVRdrGau4NlaNOzvhfUExNfkJEPAjsCJzVwesuAMbfNGkRsCQidgWWlI+fp+yq+2lgP2Bf4NMTJRUbTkWbhbvOpsqVRZ46qiyAf4mIn0p6DbAXcGm7F0XEtcD4sRjHABeWyxdSjAYf7y3A4oh4OCLWA4vZNOnYEHNlkbaxemwYaW/56CRZXAtsKWkH4CrgbyiqhqmYGxFryuUHgbktttkBuK/p8epy3SYknSRpqaSl69atm2JIlpqRijw3VMLcGypPHU0kGBFPAe8CvhQR7wVet7k7LhvNN+sTISLOiYjRiBidM2fO5oZkiahWXVmkzL2h8tRRspD0RuB44Hsv4HWtPCRpXvmm84C1Lba5H9ip6fGO5TrLhHtDpc1tFnnq5EP/o8AngMsj4lZJuwA/meL+rgQavZsWAle02OZHwJGSZpUN20eW6ywTbrNIm+eGylMns85eA1wj6cXl47uBv2v3OkmXAocC20laTdHD6Qzg65JOAFYB7yu3HQVOjogTI+JhSZ8FflW+1Wc8aWFe3Bsqba4s8tQ2WZSXoM4DXgrML0dyfzgiTpnsdRFx3ARPHd5i26XAiU2PzwfObxebDSdXFumKCGpus8hSJ7XkFyi6s/4BICJuAg7pZlCWN88Nla5GEndlkZ+OLjxGxH3jVtVabmg2DVxZpKuRxD3OIj+d3Fb1PkkHACFpBnAacHt3w7KcjVQqHmeRKFcW+eqksjgZOJViYNz9FCO4T+1mUJY3Vxbp2lBZuDdUdiatLCRVgb+JiON7FI8ZI1XxzJivdKbIlUW+Jv16EBE14K96FIsZ4MoiZY0uze4NlZ9O2ix+Juls4DLgycbKiLixa1FZ1twbKl2uLPLVSbLYq/z9maZ1ARw2/eGYubJIWaPjgSuL/HQygvt/9CIQswbPDZWuDZWFu85mp5MR3P+7xepHgWURsXz6Q7LcubJIl3tD5auTMz5K0X12h/LnwxQ3I/oPSX/fxdgsU54bKl1us8hXJ20WOwJ7R8QTAJI+TTFV+SHAMuBz3QvPclStiJoH5SXJvaHy1UllsT3FPbgbnqO4493T49abTYuRqntDpcqVRb46qSwuAW6Q1Lj3xNuBr0l6CXBb1yKzbLnNIl0b2yycLHLTSW+oz0r6AXBguerkckpxKO6eZzat3BsqXRsrCzdw56bTM74V8FhE/AuwStIruxiTZc6VRbo8ziJfbZNF2aD9cYpbqwLMAC7uZlCWN/eGSpfHWeSrk8rincA7KKf6iIgHgK27GZTlzZVFutwbKl+dJItnIyIopvigbNg26xrPDZUu94bKVyfJ4uuS/h2YKelDwI+Bc7sbluWsWqkQAXUnjOS4N1S+OukN9U+SjgAeA3YDTo+IxV2PzLLVuB4+Vg+28IdSUtwbKl+djLOgTA6LASRVJB0fEZd0NTLLVuNbq9st0uPKIl8Tfj2QtI2kT0g6W9KRKnwEuBt43+bsVNJpklZIulXSR1s8/zJJ35F0U7nNBzdnfzZYGtfD3SMqPbXynLjNIj+TVRZfBdYD1wEnAv8XEHDs5sw2K+l1wIeAfYFngR9K+m5E3Nm02anAbRHxdklzgN9KuiQinp3qfm1wuLJIl8dZ5GuyZLFLRLweQNK5wBpgfkT8cTP3uTtwQ0Q8Vb73NcC7eP6EhAFsLUnAS4GHgbHN3K8NiI2VhZNFajzOIl+TtVI911go78W9ehoSBcAK4GBJ20p6MXAUsNO4bc6mSCoPALcAp0XEJtckJJ0kaamkpevWrZuG0CwFjXsluLJIj9ss8jVZZbGnpMfKZQEvKh8LiIjYZio7jIjbJZ0JXEUx0G85UBu32VvK9YcBrwIWS/ppRDw27r3OAc4BGB0d9SfLkHBlkS73hsrXhGc8IqoRsU35s3VEjDQtTylRNL33eRGxT0QcQtEucse4TT4IfCsKdwK/A167Ofu0wbGhzcL3tEiOK4t89eXrgaTty9/zKdorvjZuk3uBw8tt5lKM77i7lzFa/2wcZ+HeUKlxb6h8dTTOogu+KWlbinaRUyPiEUknA0TEV4DPAhdIuoXistfHI+L3fYrVesy9odLlyiJffUkWEXFwi3VfaVp+ADiyp0FZMtxmka7GpUFXFvlxK5Ulx72h0uXKIl9OFpYcVxbpqtWDakUUQ6AsJ04WlpyNbRZu4E7NWJksLD9OFpacDZWFu84mp1avu70iU04Wlhz3hkqXK4t8OVlYcprvZ2FpqdXDlUWmnCwsORW5skiVK4t8OVlYchrzDrmySE+t5mSRKycLS47bLNJVi/AkgpnyWbfkNNosnCzSU/NlqGw5WVhyqr6tarLG3MCdLScLS86IL0Mlq1avu7LIlJOFJafq6T6SNeYG7mw5WVhyRjyRYLJq9fD9tzPlZGHJcWWRrmKchT82cuSzbsnZ0GZRcwN3ajyCO19OFpacqqf7SNaYG7iz5WRhyXFvqHS5ssiXk4Ulx20W6fLcUPlysrDkuDdUulxZ5MvJwpLT+CxyZZGeYpyFPzZy5LNuyZHESEW+rWqCXFnkqy/JQtJpklZIulXSRyfY5lBJy8ttrul1jNZf1YpcWSRorF7f0FvN8jLS6x1Keh3wIWBf4Fngh5K+GxF3Nm0zE/gS8NaIuFfS9r2O0/prpCJqvgd3clxZ5KsflcXuwA0R8VREjAHXAO8at81fAd+KiHsBImJtj2O0PnNlkSb3hspXP5LFCuBgSdtKejFwFLDTuG1eA8ySdLWkZZLe3/Mora9GqhX3hkqQK4t89fwyVETcLulM4CrgSWA5UGsR1z7A4cCLgOskXR8RdzRvJOkk4CSA+fPndzt06yFXFmny3FD56stZj4jzImKfiDgEWA/cMW6T1cCPIuLJiPg9cC2wZ4v3OSciRiNidM6cOd0P3HrGvaHS5MoiX/3qDbV9+Xs+RXvF18ZtcgVwkKSR8lLVfsDtvY3S+smVRZrGap4bKlc9vwxV+qakbYHngFMj4hFJJwNExFfKS1U/BG4G6sC5EbGiT7FaHxSVhZNFalxZ5KsvySIiDm6x7ivjHp8FnNWzoCwprizSNFYPj7PIlFuqLEkjlYrHWSTIlUW+nCwsSa4s0hMR7g2VMZ91S9JI1b2hUtPI3a4s8uRkYUlyZZGesTJ5uzdUnpwsLEnuDZWexvlwZZEnJwtLkiuL9DTOhyuLPDlZWJJGKp4bKjWN3mmuLPLkZGFJcmWRng2VRdUfGznyWbckeW6o9LjNIm9OFpakakWMeVBeUtwbKm9OFpakYpyFk0VKXFnkzcnCklR1A3dy3Bsqb/2addZsUiMVcd/6pzji89f0OxQrPVvzZaicOVlYkt6zz448Mzb+BorWb/vMn8W+r5zd7zCsD5wsLEkHvno7Dnz1dv0Ow8xKbrMwM7O2nCzMzKwtJwszM2vLycLMzNpysjAzs7acLMzMrC0nCzMza8vJwszM2lLEcMy/I2kdsGqSTbYDft+jcKbC8W0ex7d5HN/mGeT4do6IOe3eYGiSRTuSlkbEaL/jmIjj2zyOb/M4vs2TQ3y+DGVmZm05WZiZWVs5JYtz+h1AG45v8zi+zeP4Ns/Qx5dNm4WZmU1dTpWFmZlN0dAmC0lnSfqNpJslXS5p5gTbvVXSbyXdKWlRD+N7r6RbJdUlTdhLQdI9km6RtFzS0gTj69fxmy1psaSV5e9ZE2xXK4/dcklX9iCuSY+HpC0lXVY+f4OkBd2O6QXG9wFJ65qO2Yk9jO18SWslrZjgeUn61zL2myXt3avYOozvUEmPNh2703sc306SfiLptvL/7mkttpn6MYyIofwBjgRGyuUzgTNbbFMF7gJ2AbYAbgL+pEfx7Q7sBlwNjE6y3T3Adn04fm3j6/Px+xywqFxe1Or8ls890cNj1vZ4AKcAXymX/xK4LLH4PgCc3eu/t3LfhwB7AysmeP4o4AeAgP2BGxKL71Dgu/04duX+5wF7l8tbA3e0OL9TPoZDW1lExFURMVY+vB7YscVm+wJ3RsTdEfEs8P+AY3oU3+0R8dte7GsqOoyvb8ev3M+F5fKFwLE92u9kOjkezXF/AzhcUq9uat3P89VWRFwLPDzJJscAF0XhemCmpHm9ia6j+PoqItZExI3l8uPA7cAO4zab8jEc2mQxzv+kyKbj7QDc1/R4NZse3H4L4CpJyySd1O9gxunn8ZsbEWvK5QeBuRNst5WkpZKul9TthNLJ8diwTfll5lFg2y7Htcm+SxOdr3eXlyi+IWmn3oTWkUH4//pGSTdJ+oGkPfoVRHl580+BG8Y9NeVjOND34Jb0Y+DlLZ76ZERcUW7zSWAMuKSXsZX7bhtfBw6KiPslbQ8slvSb8htOKvF1zWTxNT+IiJA0Ube+ncvjtwvw35JuiYi7pjvWIfId4NKIeEbShymqoMP6HNOguJHi7+0JSUcB3wZ27XUQkl4KfBP4aEQ8Nl3vO9DJIiLePNnzkj4AHA0cHuUFu3HuB5q/Oe1YrutJfB2+x/3l77WSLqe4lDAtyWIa4uvb8ZP0kKR5EbGmLKPXTvAejeN3t6SrKb5tdStZdHI8GtusljQCvAz4Q5fiGa9tfBHRHMu5FG1Dqejq39vmav5gjojvS/qSpO0iomdzRkmaQZEoLomIb7XYZMrHcGgvQ0l6K/D3wDsi4qkJNvsVsKukV0ragqLBses9Zjol6SWStm4sUzTat+yJ0Sf9PH5XAgvL5YXAJpWQpFmStiyXtwMOBG7rYkydHI/muN8D/PcEX2T6Et+469fvoLjunYorgfeXPXr2Bx5tuhTZd5Je3mh/krQvxedrr74IUO77POD2iPj8BJtN/Rj2q+W+2z/AnRTX5paXP40eKK8Avt+03VEUvQbuorj80qv43klxvfAZ4CHgR+Pjo+i1clP5c2tq8fX5+G0LLAFWAj8GZpfrR4Fzy+UDgFvK43cLcEIP4trkeACfofjSArAV8F/l3+cvgV16dcw6jO8fy7+1m4CfAK/tYWyXAmuA58q/vROAk4GTy+cF/FsZ+y1M0ouwT/F9pOnYXQ8c0OP4DqJo47y56XPvqOk6hh7BbWZmbQ3tZSgzM5s+ThZmZtaWk4WZmbXlZGFmZm05WZiZWVtOFmaApJmSTimXD5X03S7s4wOSzn6Br7mnHCMyfv0/SPrY9EVnNjknC7PCTIoZYTsmqdqlWMxAFmXHAAACB0lEQVSS42RhVjgDeJWk5cBZwEvLifR+I+mSppG590g6U9KNwHslvUrSD8uJHn8q6bXldu+VtKKcVK55epZXlNuvlLRhKg1Jx6m4b8kKSWe2ClDSJyXdIelnFNPHm/XMQM8NZTaNFgGvi4i9JB1KMX3IHsADwM8ppgr5WbntHyJibwBJSyhGx66UtB/wJYqJ904H3hLFJIbNN97ai2J+qmeA30r6IlCjuOfKPsB6ilmGj42IbzdeJGkfiuk59qL4f3sjsGz6D4NZa04WZq39MiJWA5TVxgI2JovLyvUvpZhS5L+abkmxZfn758AFkr4ONE/otiQiHi1ffxuwM8XUJVdHxLpy/SUUN9r5dtPrDgYuj3KeM/Xgrn9mzZwszFp7pmm5xvP/rzxZ/q4Aj0TEXuNfHBEnl5XGnwPLysqg3fuaJcttFmaFxyluRdmxKKak/p2k98KG+xvvWS6/KiJuiIjTgXU8f1ro8X4JvEnSdmWj+XHANeO2uRY4VtKLypmI3/5CYjXbXP5WY0ZxHwdJP5e0AniaYqbdThwPfFnSp4AZFLcqvQk4S9KuFLN8LinXbVKBlPteI2kRxSyvAr4X424+FRE3SrqsfJ+1FNONm/WMZ501M7O2fBnKzMzacrIwM7O2nCzMzKwtJwszM2vLycLMzNpysjAzs7acLMzMrC0nCzMza+v/A+kDKZI/dagTAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1150265f8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "thresholds = np.linspace(-1.9, 1.9, 100)\n",
    "crit_by_thres = [regression_var_criterion(X, y, thres) for thres in thresholds]\n",
    "\n",
    "plt.plot(thresholds, crit_by_thres)\n",
    "plt.xlabel('threshold')\n",
    "plt.ylabel('Regression criterion');"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**<font color='red'>Вопрос 1.</font> Оптимально ли с точки зрения дисперсионного критерия выбранное нами значение порога $t = 0$?**\n",
    "- Да\n",
    "- Нет\n",
    "\n",
    "**<font color='red'>Ответ:</font> Нет.**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Теперь произведем разбиение в каждой из листовых вершин. В левой (соответствующей ветви $x < 0$) – по предикату $[x < -1.5]$, а в правой (соответствующей ветви $x \\geqslant 0$) – по предикату $[x < 1.5]$. Получится дерево глубины 2 с 7 вершинами и 4 листьями. Постройте график предсказаний этого дерева для $x \\in [-2, 2]$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXYAAAD8CAYAAABjAo9vAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAFSdJREFUeJzt3X2QXXddx/H3l23abh/oAgltk3RJGUu0WjHM0ikUFdpKsDw0Ijg4IlTAKKICw4Rp7AyMOspDGBQHkIlQwaGCWsKCWEipUB2daeuWtF36ECnIQzeFpmiaPizN3eTrH/fesEl3s3v3nHvvuWffr5mdvXvv2d/57snms9/93nP3RGYiSaqPJ/S7AElSuQx2SaoZg12SasZgl6SaMdglqWYMdkmqGYNdkmrGYJekmjHYJalmjuvHTleuXJnr1q3rx64laWDdcsstD2TmqoW260uwr1u3jomJiX7sWpIGVkR8ZzHbOYqRpJox2CWpZgx2SaoZg12SasZgl6SaKeWsmIh4K/AGIIFJ4Lcy80dlrC1Jg2581xTbdu5mz75pVo8Ms2XjejZtWNO1/RXu2CNiDfCHwFhm/gwwBLyq6LqSVAfju6bYumOSqX3TJDC1b5qtOyYZ3zXVtX2WdR77ccBwRDSAk4A9Ja0rSZ179FH40pdgZqbflXDL+Ne56JEDAHx35AwmzzyH6cZBtu3c3bWuvXCwZ+ZURLwP+C4wDVyXmdcdvV1EbAY2A4yOjhbdrSTN7+Mfhze9qd9VAPCns27//TM3MnnmOQDs2TfdtX0WDvaIeBJwGXA2sA/4p4h4dWZ+cvZ2mbkd2A4wNjbmFbQldc9DDzXfT0zA8HBfS/mNv7mRHzz0GAD7Tzj58P2rR7pXVxmjmEuA/8nMvQARsQN4LvDJY36WJHVLewRz3nlw/PF9LeWVrzmNrTsmmW4cPHzf8Iohtmxc37V9lhHs3wUuiIiTaI5iLgb8QzCS+qcd7Mf15c9hHaE9R+/lWTFlzNhviohrgK8BM8AuWiMXSeqLRgOe8ITmWwVs2rCmq0F+tFJ+nGXmO4F3lrGWJBU2MwMrVvS7ir6pxo8zSSpTo1GJMUy/GOyS6mdmxmCXpFppNBzFSFKt2LFLUs3YsUtSzdixS1LNeLqjJNWMpztKUs3YsUtSzdixS1LN2LFLUs3YsUtSzXi6oyTVjKMYSaoZRzHFRcRIRFwTEXdHxF0R8Zwy1pWkJVnmHXtZP9I+AHwpM18REccDJ5W0riR1bpl37IW/8og4DfgF4HKAzDwAHCi6riQt2TLv2MsYxZwN7AX+NiJ2RcRHI+LkEtaVpKVZ5h17GcF+HPAs4K8zcwPwCHDF0RtFxOaImIiIib1795awW0mahx17YfcC92bmTa2Pr6EZ9EfIzO2ZOZaZY6tWrSpht5I0Dzv2YjLz+8D3ImJ9666LgTuLritJS7bMX6BU1lf+B8DVrTNivgX8VknrSlLnlvkoppRgz8xbgbEy1pKkwhzFSFLNLPOO3WCXVD927JJUI5lw8KAduyTVxsxM870duyTVRDvY7dglqSYajeZ7O3ZJqgk7doNdUs04YzfYJdWMoxiDXVLNOIox2CXVjB27wS6pZuzYDXZJNWPHbrBLqhk7doNdUs14uqPBLqlm2qMYO/biImIoInZFxBfKWlOSOmbHXmrH/mbgrhLXk6TO+eRpOcEeEWuBFwMfLWM9SVoynzwtrWP/S+DtwKGS1pOkpbFjLx7sEfES4P7MvGWB7TZHxERETOzdu7fobiVpbnbspXTsFwIvi4hvA58GLoqITx69UWZuz8yxzBxbtWpVCbuVpDnYsRcP9szcmplrM3Md8CrgK5n56sKVSdJS2LF7HrukmvF0R0r9yjPzBuCGMteUpI74AiU7dkk1Y8dusEuqGTt2g11SzdixG+ySasbTHQ12STXj6Y4Gu6SacRRjsEuqmfYoZmiov3X0kcEuqV5mZprdekS/K+kbg11SvTQay3q+Dga7pLppd+zLmMEuqV7s2A12STVjx17uHwGTpLKM75pi287d7Nk3zeqRYbZsXM+mDWsW/sRGw2DvdwGSdLTxXVNs3THJdOMgAFP7ptm6YxJg4XCfmXEU0+8CJOlo23buPhzqbdONg2zbuXvhT3YUY7BLqp49+6Y7uv8IPnlaysWsz4qIr0bEnRFxR0S8uYzCJC1fq0eGO7r/CHbspXTsM8DbMvNc4ALgTRFxbgnrSlqmtmxcz/CKI/8kwPCKIbZsXL/wJ9uxF3/yNDPvA+5r3X4oIu4C1gB3Fl1b0vLUfoJ0SWfF2LGXe1ZMRKwDNgA3zfHYZmAzwOjoaJm7lVRDmzasWVyQH82OvbwnTyPiFOAzwFsyc//Rj2fm9swcy8yxVatWlbVbSTqSHXs5wR4RK2iG+tWZuaOMNSVpSTyPvZSzYgL4GHBXZr6/eEmSVICvPC2lY78Q+E3gooi4tfV2aQnrSlLnHMWUclbMfwDL9y/aS6oWnzz1laeSasaO3WCXVDN27Aa7pJqxYzfYJdWMHbvBLqlm7NgNdkk14wuUDHZJNeMLlLw0nnSEG26Aq67qdxUq4uGHDfZ+FyBVyoc/DOPjsHZtvyvRUq1bB897Xr+r6CuDXZpt/37YsAFuetxfnpYGhjN2abb9++G00/pdhVSIwS7Ntn8/PPGJ/a5CKsRRjDTbgw8OfLCP75pa2iXlVBsGuzTbgHfs47um2LpjkunGQQCm9k2zdcckgOG+jDiKkdoOHYKHHhroGfu2nbsPh3rbdOMg23bu7lNF6oeyLo33oojYHRH3RMQVZawp9dwjj0DmQHfse/ZNd3S/6qmMS+MNAR8Cfhk4F/j1iDi36LpSL43vmmLTn18LwLv/cw/ju6b6XNHSrB4Z7uh+1VMZHfv5wD2Z+a3MPAB8GrishHWlnmjPpR/e+78A3HvwOLbumBzIcN+ycT3DK4aOuG94xRBbNq7vU0XqhzKCfQ3wvVkf39u6TxoI7bn0qY89CsDDx580sHPpTRvW8K6Xn8eakWECWDMyzLtefp5PnC4zPTsrJiI2A5sBRkdHe7VbaUHt+fMpB5rBvv+Ek4+4f9Bs2rDGIF/mygj2KeCsWR+vbd13hMzcDmwHGBsbyxL2qwqowznTq0eGmdo3zSntjv2E4cP3S4OojFHMfwHnRMTZEXE88Crg8yWsq4prz6an9k2T/Pic6UGbTbfn0odHMSec5FxaA61wsGfmDPD7wE7gLuAfM/OOouuq+upyznR7Lr12qAHAqaue4lxaA62UGXtmXgtcW8ZaGhx1Omd604Y18OzT4V9g5zteDENDC3+SVFG+8lRLVrtzpvfvh1NOMdQ18Ax2LVntzpmuwR8Ak8A/AqYC2jPoQT8r5rAB/wNgUpvBrkJqdc60F9lQTTiKkdrs2FUTBrvU5oxdNWGwS2127KoJg11qc8aumjDYJfjx1ZPs2FUDBrsEtbh6ktTm6Y5luf9+uOeeflehpXrggeZ7g101YLCX5aUvhZtv7ncVKur00/tdgVSYwV6WH/4QXvACuMJreQ+sE0+ECy/sdxVSYQZ7WWZmYHQUXvjCflciaZnzydOyNBqwYkW/q5Akg700jQYc5y9AkvqvULBHxLaIuDsibo+Iz0bESFmFDRw7dkkVUbRj/zLwM5n5s8B/A1uLlzSgZmYMdkmVUCjYM/O61jVPAW4E1hYvaUDZsUuqiDJn7K8DvljieoPFGbukilgwiSLieuCMOR66MjM/19rmSmAGuPoY62wGNgOMjo4uqdjKOnSo+WbHLqkCFgz2zLzkWI9HxOXAS4CLMzOPsc52YDvA2NjYvNsNpJnWNGqRwT6+a6o+l5OTVDmFZgcR8SLg7cAvZuaj5ZQ0gBqN5vtFBPv4rim27phkunEQgKl902zdMQlguEsqRdEZ+weBU4EvR8StEfGREmoaPO1gX8SMfdvO3YdDvW26cZBtO3d3ozJJy1Chjj0zf6KsQgZaBx37nn3THd0vSZ3yladl6GDGvnpkuKP7JalTBnsZOujYt2xcz/CKoSPuG14xxJaN67tRmaRlyBOvy9DBjL39BKlnxUjqFoO9DB107NAMd4NcUrc4iilDh+exS1I3Gexl6GAUI0ndZrCXocNRjCR1k8FeBoNdUoUY7GVwxi6pQgz2Mjhjl1QhBnsZHMVIqhCDvQwGu6QKMdjL4IxdUoUY7GVwxi6pQgz2MjiKkVQhBnsZDHZJFVJKsEfE2yIiI2JlGesNHGfskiqkcLBHxFnAC4HvFi9nQDljl1QhZXTsf0HzgtZZwlqDyVGMpAopFOwRcRkwlZm3LWLbzRExERETe/fuLbLb6jHYJVXIgrODiLgeOGOOh64E/ojmGGZBmbkd2A4wNjZWr+7eGbukClkw2DPzkrnuj4jzgLOB2yICYC3wtYg4PzO/X2qVVeeMXVKFLDmJMnMSeGr744j4NjCWmQ+UUNdgaQf70NCxt5OkHvA89jI0Gs0xTPM3F0nqq9JmB5m5rqy1Bs7MjGMYSZVhx16GdscuSRVgsJfBYJdUIQZ7GQx2SRVisJfBGbukCjHYy2DHLqlCDPYyGOySKsRgL4PBLqlCDPYyOGOXVCEGexns2CVViMFeBoNdUoUY7GUw2CVViMFeBmfskirEYC+DHbukCjHYy2CwS6oQg70MBrukCikc7BHxBxFxd0TcERHvLaOogeOMXVKFFEqjiHgBcBnwzMx8LCKeutDn1JIdu6QKKdqxvxF4d2Y+BpCZ9xcvaQAZ7JIqpGiwPwP4+Yi4KSL+LSKeXUZRA6fRcBQjqTIWTKOIuB44Y46Hrmx9/pOBC4BnA/8YEU/PzJxjnc3AZoDR0dEiNVfPzIwdu6TKWDDYM/OS+R6LiDcCO1pBfnNEHAJWAnvnWGc7sB1gbGzsccE/0BzFSKqQoqOYceAFABHxDOB44IGiRQ0cg11ShRQdDF8FXBURXwcOAK+dawxTe87YJVVIoTTKzAPAq0uqZXA5Y5dUIb7ytKhDh5pvBrukijDYi2o0mu8NdkkVYbAX1Q52Z+ySKsJgL2pmpvnejl1SRRjsRTmKkVQxBntRBrukijHYi3LGLqliDPainLFLqhiDvShHMZIqxmAvymCXVDEGe1HO2CVVjMFelDN2SRVjsBflKEZSxRjsRTmKkVQxBntRduySKsZgL8oZu6SKKRTsEfFzEXFjRNwaERMRcX5ZhQ0MO3ZJFVN0MPxe4I8z84sRcWnr4+cXrmoe47um2LZzN3v2TbN6ZJgtG9ezacOabu1ucZyxS6qYommUwBNbt08D9hRcb17ju6bYumOS6QPN0cfU/z3K1s/cDpn9DXc7dkkVUzTY3wLsjIj30RzrPLd4SXPbtnM3042D3Pyh1/DUR/7vxw/8Wbf22KETT+x3BZIELCLYI+J64Iw5HroSuBh4a2Z+JiJ+DfgYcMk862wGNgOMjo52XOiefdMAbD//Vzj5wI+OeOytv/SMjtcr1apVcPbZ/a1BkloiM5f+yREPAiOZmRERwIOZ+cSFPm9sbCwnJiY62teF7/4KU61wn23NyDD/ecVFHa0lSYMoIm7JzLGFtit6uuMe4Bdbty8CvlFwvXlt2bie4RVDR9w3vGKILRvXd2uXkjSQis7Yfxv4QEQcB/yI1qilG9pPkFburBhJqphCo5ilWsooRpKWu16NYiRJFWOwS1LNGOySVDMGuyTVjMEuSTXTl7NiImIv8J0CS6wEHiipnDJZ1+JVsSawrk5ZV2eK1vW0zFy10EZ9CfaiImJiMaf89Jp1LV4VawLr6pR1daZXdTmKkaSaMdglqWYGNdi397uAeVjX4lWxJrCuTllXZ3pS10DO2CVJ8xvUjl2SNI+BCPaI2BYRd0fE7RHx2YgYmWe7F0XE7oi4JyKu6EFdr4yIOyLiUETM+0x3RHw7IibbF/2uUF09O14R8eSI+HJEfKP1/knzbHewdZxujYjPd7GeY37tEXFCRPxD6/GbImJdt2rpsK7LI2LvrGP0hh7UdFVE3B8RX5/n8YiIv2rVfHtEPKvbNS2yrudHxIOzjtU7elTXWRHx1Yi4s/X/8M1zbNPdY5aZlX8DXggc17r9HuA9c2wzBHwTeDpwPHAbcG6X6/opYD1wAzB2jO2+Dazs4fFasK5eHy+aFzq/onX7irn+DVuPPdyD47Pg1w78HvCR1u1XAf9QkbouBz7Yq++l1j5/AXgW8PV5Hr8U+CIQwAXATRWp6/nAF3p5rFr7PRN4Vuv2qcB/z/Hv2NVjNhAde2Zel5kzrQ9vBNbOsdn5wD2Z+a3MPAB8Grisy3XdlZm7u7mPpVhkXb0+XpcBn2jd/gSwqYv7WshivvbZ9V4DXNy6Sli/6+q5zPx34H+PscllwN9l043ASEScWYG6+iIz78vMr7VuPwTcBRx94YiuHrOBCPajvI7mT7qjrQG+N+vje3n8weyXBK6LiFta136tgl4fr9Mz877W7e8Dp8+z3YkRMRERN0ZEt8J/MV/74W1aTcWDwFO6VE8ndQH8auvX92si4qwu17QYVf6/95yIuC0ivhgRP93rnbdGeBuAm456qKvHrOgVlEpzrItmZ+bnWttcCcwAV1eprkV4XmZORcRTgS9HxN2tbqPfdZVqgQufH5aZGRHznY71tNaxejrwlYiYzMxvll3rAPtn4FOZ+VhE/A7N3yq86O/cvkbz++nhiLgUGAfO6dXOI+IU4DPAWzJzf6/2CxUK9sy85FiPR8TlwEuAi7M1pDrKFDC7e1nbuq+rdS1yjanW+/sj4rM0f+UuFOwl1FX68TpWTRHxg4g4MzPva/3Kef88a7SP1bci4gaa3U7Zwb6Yr729zb3RvPTjacAPS66j47oyc3YNH6X53EW/deX/XlGzwzQzr42ID0fEyszs+t+QiYgVNEP96szcMccmXT1mAzGKiYgXAW8HXpaZj86z2X8B50TE2RFxPM0nvLp2VsViRcTJEXFq+zbNJ4LnfBa/x3p9vD4PvLZ1+7XA436riIgnRcQJrdsrgQuBO7tQy2K+9tn1vgL4yjwNRU/rOmoO+zKa89t++zzwmtaZHhcAD84au/VNRJzRfl4kIs6nmXfd/uFMa58fA+7KzPfPs1l3j1mvnzFeyhtwD8151K2tt/bZCquBa2dtdynNZ6C/SXMk0e26foXmbOwx4AfAzqPronmGw22ttzuqUlevjxfN+fS/At8Argee3Lp/DPho6/ZzgcnWsZoEXt/Feh73tQN/QrN5ADgR+KfW997NwNO7/e+2yLre1fo+ug34KvCTPajpU8B9QKP1ffV64HeB3209HsCHWjVPcowzxHpc1+/POlY3As/tUV3Po/m82u2zMuvSXh4zX3kqSTUzEKMYSdLiGeySVDMGuyTVjMEuSTVjsEtSzRjsklQzBrsk1YzBLkk18//cm7Ezz9cANQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x115a02b38>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "xx = np.linspace(-2, 2, 200)\n",
    "\n",
    "def prediction(x, X, y):\n",
    "    if x >= 1.5:\n",
    "        return np.mean(y[X >= 1.5])\n",
    "    elif x < 1.5 and x >= 0:\n",
    "        return np.mean(y[(X >= 0) & (X < 1.5)])\n",
    "    elif x >= -1.5 and x < 0:\n",
    "        return np.mean(y[(X < 0) & (X >= -1.5)])\n",
    "    else:\n",
    "        return np.mean(y[X < -1.5])\n",
    "    \n",
    "    \n",
    "predictions = [prediction(x, X, y) for x in xx]\n",
    "\n",
    "X = np.linspace(-2, 2, 7)\n",
    "y = X ** 3\n",
    "\n",
    "plt.scatter(X, y);\n",
    "plt.plot(xx, predictions, c='red');"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**<font color='red'>Вопрос 2.</font> Из скольки отрезков состоит график (необходимо считать как горизонтальные, так и вертикальные прямые), изображающий предсказания построенного дерева на отрезке [-2, 2]?**\n",
    "- 5\n",
    "- 6\n",
    "- 7\n",
    "- 8\n",
    "\n",
    "**<font color='red'>Ответ:</font> 3 вариант, из семи**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. Построение дерева решений для прогноза сердечно-сосудистых заболеваний\n",
    "Считаем в DataFrame знакомый нам набор данных по сердечно-сосудистым заболеваниям. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = pd.read_csv('../../data/mlbootcamp5_train.csv', index_col='id', sep=';')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>age</th>\n",
       "      <th>gender</th>\n",
       "      <th>height</th>\n",
       "      <th>weight</th>\n",
       "      <th>ap_hi</th>\n",
       "      <th>ap_lo</th>\n",
       "      <th>cholesterol</th>\n",
       "      <th>gluc</th>\n",
       "      <th>smoke</th>\n",
       "      <th>alco</th>\n",
       "      <th>active</th>\n",
       "      <th>cardio</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>id</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>18393</td>\n",
       "      <td>2</td>\n",
       "      <td>168</td>\n",
       "      <td>62.0</td>\n",
       "      <td>110</td>\n",
       "      <td>80</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>20228</td>\n",
       "      <td>1</td>\n",
       "      <td>156</td>\n",
       "      <td>85.0</td>\n",
       "      <td>140</td>\n",
       "      <td>90</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>18857</td>\n",
       "      <td>1</td>\n",
       "      <td>165</td>\n",
       "      <td>64.0</td>\n",
       "      <td>130</td>\n",
       "      <td>70</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>17623</td>\n",
       "      <td>2</td>\n",
       "      <td>169</td>\n",
       "      <td>82.0</td>\n",
       "      <td>150</td>\n",
       "      <td>100</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>17474</td>\n",
       "      <td>1</td>\n",
       "      <td>156</td>\n",
       "      <td>56.0</td>\n",
       "      <td>100</td>\n",
       "      <td>60</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      age  gender  height  weight  ap_hi  ap_lo  cholesterol  gluc  smoke  \\\n",
       "id                                                                          \n",
       "0   18393       2     168    62.0    110     80            1     1      0   \n",
       "1   20228       1     156    85.0    140     90            3     1      0   \n",
       "2   18857       1     165    64.0    130     70            3     1      0   \n",
       "3   17623       2     169    82.0    150    100            1     1      0   \n",
       "4   17474       1     156    56.0    100     60            1     1      0   \n",
       "\n",
       "    alco  active  cardio  \n",
       "id                        \n",
       "0      0       1       0  \n",
       "1      0       1       1  \n",
       "2      0       0       1  \n",
       "3      0       1       1  \n",
       "4      0       0       0  "
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Сделайте небольшие преобразования признаков: постройте признак \"возраст в годах\" (полных лет), а также постройте по 3 бинарных признака на основе `cholesterol` и `gluc`, где они, соответственно, равны 1, 2 или 3. Эта техника называется dummy-кодированием или One Hot Encoding (OHE), удобней всего в данном случае использовать `pandas.get_dummmies`. Исходные признаки `cholesterol` и `gluc` после кодирования использовать не нужно."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "df['age_years'] = (df.age / 365.25).astype('int')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "train_df = pd.get_dummies(df, columns=['cholesterol', \n",
    "                                       'gluc']).drop(['age','cardio'],\n",
    "                                                     axis=1)\n",
    "target = df.cardio"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>gender</th>\n",
       "      <th>height</th>\n",
       "      <th>weight</th>\n",
       "      <th>ap_hi</th>\n",
       "      <th>ap_lo</th>\n",
       "      <th>smoke</th>\n",
       "      <th>alco</th>\n",
       "      <th>active</th>\n",
       "      <th>age_years</th>\n",
       "      <th>cholesterol_1</th>\n",
       "      <th>cholesterol_2</th>\n",
       "      <th>cholesterol_3</th>\n",
       "      <th>gluc_1</th>\n",
       "      <th>gluc_2</th>\n",
       "      <th>gluc_3</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>id</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2</td>\n",
       "      <td>168</td>\n",
       "      <td>62.0</td>\n",
       "      <td>110</td>\n",
       "      <td>80</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>50</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>156</td>\n",
       "      <td>85.0</td>\n",
       "      <td>140</td>\n",
       "      <td>90</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>55</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>165</td>\n",
       "      <td>64.0</td>\n",
       "      <td>130</td>\n",
       "      <td>70</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>51</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2</td>\n",
       "      <td>169</td>\n",
       "      <td>82.0</td>\n",
       "      <td>150</td>\n",
       "      <td>100</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>48</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1</td>\n",
       "      <td>156</td>\n",
       "      <td>56.0</td>\n",
       "      <td>100</td>\n",
       "      <td>60</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>47</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    gender  height  weight  ap_hi  ap_lo  smoke  alco  active  age_years  \\\n",
       "id                                                                         \n",
       "0        2     168    62.0    110     80      0     0       1         50   \n",
       "1        1     156    85.0    140     90      0     0       1         55   \n",
       "2        1     165    64.0    130     70      0     0       0         51   \n",
       "3        2     169    82.0    150    100      0     0       1         48   \n",
       "4        1     156    56.0    100     60      0     0       0         47   \n",
       "\n",
       "    cholesterol_1  cholesterol_2  cholesterol_3  gluc_1  gluc_2  gluc_3  \n",
       "id                                                                       \n",
       "0               1              0              0       1       0       0  \n",
       "1               0              0              1       1       0       0  \n",
       "2               0              0              1       1       0       0  \n",
       "3               1              0              0       1       0       0  \n",
       "4               1              0              0       1       0       0  "
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Разбейте выборку на обучающую и отложенную (holdout) части в пропорции 7/3. Для этого используйте метод `sklearn.model_selection.train_test_split`, зафиксируйте у него `random_state`=17."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_train, X_valid, y_train, y_valid = train_test_split(train_df.values, target.values,\n",
    "                                                     test_size=.3, random_state=17)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Обучите на выборке `(X_train, y_train)` дерево решений с ограничением на максимальную глубину в 3. Зафиксируйте у дерева `random_state=17`. Визуализируйте дерево с помошью `sklearn.tree.export_graphviz`, `dot` и `pydot`. Пример дан в [статье](https://habrahabr.ru/company/ods/blog/322534/) под спойлером \"Код для отрисовки дерева\". Названия файлов писать без кавычек, для того чтобы работало в jupyter notebook. Обратите внимание, что команды в Jupyter notebook, начинающиеся с восклицательного знака – это терминальные команды (который мы обычно запускаем в терминале/командной строке)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "tree = DecisionTreeClassifier(max_depth=3, random_state=17).fit(X_train, \n",
    "                                                                y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "a8f1e4b126824f16b82a4b87a5e71f2b"
      }
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Widget Javascript not detected.  It may not be installed or enabled properly.\n"
     ]
    }
   ],
   "source": [
    "dot_data = StringIO()\n",
    "export_graphviz(tree, feature_names=train_df.columns, \n",
    "                out_file=dot_data, filled=True)\n",
    "graph = pydotplus.graph_from_dot_data(dot_data.getvalue())  \n",
    "Image(value=graph.create_png())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**<font color='red'>Вопрос 3.</font> Какие 3 признака задействованы в построении дерева решений?**\n",
    "- weight, height, gluc=3\n",
    "- smoke, age, gluc=3\n",
    "- age, weight, chol=3\n",
    "- age, ap_hi, chol=3\n",
    "\n",
    "**<font color='red'>Ответ:</font> 4 вариант**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Сделайте с помощью обученного дерева прогноз для отложенной выборки `(X_valid, y_valid)`. Посчитайте долю верных ответов (accuracy)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "tree_pred_valid = tree.predict(X_valid)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.7212857142857143"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tree_acc_valid = accuracy_score(y_valid, tree_pred_valid)\n",
    "tree_acc_valid"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Теперь на кросс-валидации по выборке  `(X_train, y_train)`  настройте глубину дерева, чтобы повысить качество модели. Используйте GridSearchCV, 5-кратную кросс-валидацию. Зафиксируйте у дерева `random_state`=17. Перебирайте параметр `max_depth` от 2 до 10. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 2.79 s, sys: 23.9 ms, total: 2.81 s\n",
      "Wall time: 2.82 s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "tree_params = {'max_depth': list(range(2, 11))}\n",
    "\n",
    "tree_grid = GridSearchCV(DecisionTreeClassifier(random_state=17), \n",
    "                         tree_params, \n",
    "                         cv=5, scoring='accuracy') \n",
    "\n",
    "tree_grid.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Нарисуйте график того, как меняется средняя доля верных ответов на кросс-валидации в зависимости от значения `max_depth`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZgAAAEKCAYAAAAvlUMdAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzt3Xl81OW1x/HPIRD2nbAG2RVFASUiICJqqdiqeBUVvG6tFm1Frfbaatvrem/r1rpUr/teFS11QUSpdSdBVtnXBFRWScK+BLKc+8f8BoeQZRIyTJL5vl+veWV+z2+ZM9Tm5Hme3+855u6IiIhUtTrxDkBERGonJRgREYkJJRgREYkJJRgREYkJJRgREYkJJRgREYkJJRgREYkJJRgREYkJJRgREYmJuvEOIJ7atGnjXbt2jXcYIiI1ypw5c3LcPaW84xI6wXTt2pXZs2fHOwwRkRrFzL6N5jgNkYmISEzENMGY2UgzW25mmWZ2awn7HzKzecFrhZltDdq7mNncoH2xmV0btDcys/fNbFnQfm/Eta40s+yI610dy+8mIiJli9kQmZklAY8DI4C1wCwzm+TuS8LHuPtNEcdfDxwfbG4ABrv7XjNrAiwys0nAVuBBd//UzJKBj83sLHf/IDjvDXcfH6vvJCIi0YtlD2YgkOnuq9x9HzABGFXG8WOB1wHcfZ+77w3a64fjdPfd7v5p+BhgLpAao/hFROQQxDLBdALWRGyvDdoOYmZdgG7AJxFtnc1sQXCN+9x9fbFzWgDnAB9HNF9gZgvMbKKZda6aryEiIpVRXSb5xwAT3b0w3ODua9y9L9ATuMLM2oX3mVldQr2dR919VdD8HtA1OOcj4KWSPsjMxpnZbDObnZ2dHaOvIyIisUww64DIXkRq0FaSMQTDY8UFPZdFwCkRzU8DK9394YjjciOG1Z4FBpRyvafdPc3d01JSyr2NW0REKimWCWYW0MvMugUT8mOAScUPMrPeQEtgekRbqpk1DN63BIYCy4Pt/wGaA78udp0OEZvnAkur9NuIHCbLN+7gpYxvWLd1T7xDETkkMbuLzN0LzGw8MBVIAp5398Vmdjcw293DyWYMMMHdPeL0o4G/mJkDRujOsYVmlgr8AVgGzDUzgMfc/VngBjM7FygANgNXxuq7iVS1Vdk7mbxgA+/NX8/KTTsBeGDqcm77SW8uGXgEwX/rIjWKHfh7PbGkpaW5nuSXeFm7ZTeTF2xg8oL1LFq3HYCBXVtxTr8O9Ovcgns/WEZGVi5DerTmvgv60rlVozhHLBJiZnPcPa3c45RglGDk8Nm0PY/3F4Z6KnO/2wpAv84tOKdvB37atwMdmjfcf6y78/rMNfxpylKK3Ln1rN5celIX6tRRb0biSwkmCkowcjhs3rWPDxaFksqM1Ztxh97tm3JOv46c07cjR7Quu2eybusebntrIV+syOakbq2474K+dG3T+DBFL3IwJZgoKMFIrGzbk8+/Fm9k8oINTMvMobDI6Z7SmHP6duScfh3o2bZpha7n7vxj9lrueX8J+YVF3HJmb64c0pUk9WYkDpRgoqAEI1Vp974C/r10E+/NX8/ny7PZV1hEasuGnNOvI2f37cAxHZod8mT9xm15/P7thXyybBMDurTk/tF96ZHSpIq+gUh0lGCioAQjhyovv5DPlmczecF6Pl66iT35hbRrVp+fHhfqqfTv3KLK7wBzd97+eh13vbeEvPxCbh5xJFef0l29GTlsok0wCV0PRqQy8guLmJaZw3vz1/PR4u/ZsbeAVo2TuWBAJ87p25ETu7aK6US8mXH+CakM7dmGP76ziD9/sIwpizby4Oi+9GpXsaE3kVhSD0Y9GIlCYZEzY1Uu7y1YzweLNrJ1dz5NG9RlZJ/2nNOvI0N6tKZu0uFfecndeW/BBu54dxG79hZy4496cc2w7nGJRRKHejAih6ioyJn73RYmL9jA+ws3kL1jL42SkxhxTDvO6duRU45sQ/26SXGN0cw4N0hwd7y7mAemLueDRRt4YHQ/ju7QLK6xiagHox6MRHB3Fq3bznsL1jN5/nrWb8sjuW4dTj+qLef068jpvdvSMDm+SaUsUxZu4L/fWcT2vHzGn9aLX53Wg3rqzUgVUw9GpAKWb9zBe/PXM3nBer7J3U3dOsawI1O4ZeRR/OjodjRtUC/eIUblJ8d1YFD31tw5aTEP/XsFHy7eyAOj+3Jsp+bxDk0SkHow6sEkFHdny+581mzezdote1jx/Q4+WLSBFd/vpI7BkB5tOKdfB87s054WjZLjHe4h+dfijfzhnUVs2bWPXw7vwfjTe8Z9SE9qB92mHAUlmNpp25581m7ZzZrNe1i7JZRIwgll7Zbd7NpXeMDxJ3ZtyTn9OnLWsR1IaVo/TlHHxtbd+7h78hLemruOI9s14cEL+9E3tUW8w5IaTgkmCkowNdOuvQURSWM3a4LEEU4o2/MKDji+Sf26pLZsSOdWjUI/WzY6YLumDH8dik+Wfc/v31rEph15jBvWg1//qBcN6qk3I5WjBBMFJZjqKS+/cH9vY82WPawNeh9rgt7I5l37Dji+Qb06BySN4gmkecN6Wu6eUM/uT+8v5Y3Za+iR0pj7R/djQJeW8Q5LaiAlmCgowcTHvoIi1m/dE5E0fuh9rNmyh+wdew84PjmpDqktG9KplF5I68bJSiAV8MWKbG57ayHrt+3h6qHd+M2Pj1JvRipECSYKSjCHx/fb83j43yvI2rSLNVt2s3F7HpH/2SXVMTq2aEDnlgf3Pjq3akRKk/paor6K7cjL594PlvHqjO/o1qYx94/uy4ldW8U7LKkhlGCioAQTe9/m7uLS52aQvWMvfVNblDgH0r5ZAz15HicZmTn89p8LWLd1D1cM7spvRx5Fo2Q9vSBl03MwEndLN2zn8udnUlBYxBvjBtOvs+5eqm6G9GzD1F8P4/4Pl/Fixjd8smwT913Ql8E9Wsc7NKkF9GejxMScbzdz8VPTSTLjzWuUXKqzxvXrcteoY3lj3CDMYOwzX/HHdxayc29B+SeLlEE9GKlyny3fxLV/n0OH5g155aqBpLZULfma4KTurfnwxmE8+K/lPJ++mk+XZXPfBX0Z2qtNvEMjL7+Q7B17yd21j5wde8nZGX7tI2fnXrbnFTB6QCrn9usY71AlguZgNAdTpd6bv56b35xHr7ZNeennA2vdg4uJYs63m7ll4gJWZe9i7MDO3PaTo2lWhc8LuTs79haQGySIH5LGvgOSR27ws7TeVNP6dWnTtD6FRc53m3dzUVoqd57bR/NIMVYtJvnNbCTwCJAEPOvu9xbb/xBwWrDZCGjr7i3MrAvwNqEhvHrA39z9yeCcAcCLQENgCnCju7uZtQLeALoC3wAXufuWsuJTgqlar874lj++s4gTu7Ti2SvTqvQXkhx+efmFPPTvFTzzxSraNWvAn88/juFHtS31+KIiZ+ue/P0JI3vn3h8SSLGEkb1zL/sKig66hhm0bJRMmybJtGlSn9ZN6u9/n9KkPq2D922a1qd14+T9t1fnFxbx0EcreOLzLHqkNOHxS07gqPaqjRMrcU8wZpYErABGAGuBWcBYd19SyvHXA8e7+8/NLDmIba+ZNQEWAUPcfb2ZzQRuAGYQSjCPuvsHZnY/sNnd7zWzW4GW7v67smJUgqka7s7/fZbFA1OXc3rvtjx+yQnVesVhqZivv9vCbycuYOWmnZx/QieOateUnCB5ZEf0Ojbv2kdh0cG/T+rWsf2JIZwwUprUD7aDhNGkPm2aJtOqUfIh3VH45cpsbnpjHjvyCrjjnD6MHdhZz0jFQHVIMIOBO939zGD7NgB3/3Mpx2cAd7j7R8XaWwNfA4MABz51997BvrHAcHe/xsyWB+83mFkH4DN3P6qsGJVgDp2786cpS3nmy9WM6t+RBy/sp+Xha6G9BYU8+vFKnvx8FYVFToN6dfYnjJSIJBGZMFKaJtO6cX2aN6x3WJ9j2rQjj9+8OZ8vV+bw0+M68OcLjlNvuopVh9uUOwFrIrbXAieVdGAwJNYN+CSirTPwPtATuCXovaQF14m8ZqfgfTt33xC83wi0K+WzxgHjAI444ogKfiWJVFBYxO/fXsibs9dyxeAu3HFOHz0QWUvVr5vELWf2ZtywHiTVMRonJ1XbnkHbpg146WcDefKLLP7yrxUsWLeVv409gf66k/Gwqy5/ao4BJrr7/mVu3X2Nu/cllGCuMLMSE0ZJPNQtK7Fr5u5Pu3uau6elpKQcatwJKy+/kOtem8ubs9dywxm9uPNcJZdE0LxhPZrUr1ttk0tYnTrGr4b35M1rBlFUBKOfyODpL7IoKmEIT2InlglmHdA5Yjs1aCvJGOD1kna4+3pCczCnBOenlnLN74OhMYKfmyoduZRp594Cfv7iLKYu/p7bzz6Gm0ccWe1/4UhiGtClFe/fMJQzjm7Ln6Ys4+cvzSJ3597yT5QqEcsEMwvoZWbdgkn7McCk4geZWW+gJTA9oi3VzBoG71sCQ4HlwRDYdjMbZKHfaJcD7wanTQKuCN5fEdEuVWjLrn385zNfMWP1Zv56UT9+PrRbvEMSKVOLRsk8eekA7h7Vh4zMXM565EsysnLiHVZCiFmCcfcCYDwwFVgKvOnui83sbjM7N+LQMcAEP/Bug6OBGWY2H/gceNDdFwb7fgU8C2QCWcAHQfu9wAgzWwn8KNiWKrRh2x4ufGo6Szfu4KlLB3D+CanlnyRSDZgZlw/uytvXDaFJ/br857Mz+OtHKygoPPhWaak6etBSd5FFZVX2Ti57bibb9uTz7BVpDOqutaqkZtq1t4D/fncRb81dx8BurXhkTH86NG8Y77BqlGjvIqsuk/xSjS1at40Ln5xOXn4hE8YNUnKRGq1x/br89aL+/OXCfixat42fPPIlHy/9Pt5h1UpKMFKmmas3M/bpr6hftw5vXjuYYzs1j3dIIlXiggGpvHf9UNo3b8hVL83m7veWsLegsPwTJWpKMFKqT5Z9z2XPzaBts/pM/OUQeqQ0iXdIIlWqR0oT3v7VEK4Y3IXn01cz+onpfJOzK95h1RpKMFKid75ex7iX53Bku6a8ec1gOrbQGLXUTg3qJXHXqGN58tIBfJu7i7P/No1355X2RIVUhBKMHOSljG/49RvzSOvaktd+cRKtm2hFZKn9Rh7bnik3nsJR7Zty44R5/G7iAnbvU02cQ6EEI/u5O4/8eyV3TFrMj45ux4s/G0hTreEkCSS1ZSMmjBvEdaf14M05azj3sXSWb9wR77BqLCUYAUJLrd/13hIe+vcKLjghlScvPWH/UugiiaReUh1uObM3L/98IFt353PuY9N4bcZ3JPIjHZWlBCPkFxbxX/+Yz4sZ3/Dzk7vxwOi+h7RkukhtcEqvFD648RQGdmvF799eyPjXvmZ7Xn68w6pR9FskweXlF/LLv8/hra/X8ZsRR/LfZx+tRStFAilN6/PSzwby25FH8eHijfz00S+Zt2ZrvMOqMZRgEtiOvHyueH4mHy/bxD2j+nD9Gb20aKVIMT+szDx4/8rMz3yxSiszR0EJJkHl7tzL2Ge+Ys63W3j44v5cNrhrvEMSqdYGdGnJlBtO4Yyj2/K/U5ZqZeYoKMEkoHVbQ4tWZm7ayTOXpzGqf6fyTxIRmjeqx5OXDuCeUX3IyNLKzOVRgkkwmZt2MvqJDLJ37OWVq07itN5t4x2SSI1iZlw2uCtv/2oITRpoZeayKMEkkIVrt3HRU9PJL3TeGDeYE7u2indIIjVWn47NeW/8UM4/PpVHP17JJc/OYMO2PfEOq1pRgkkQ07NyGfvMVzRKTmLitYM5pmOzeIckUuM1rl+Xv1zUj79epJWZS6IEkwA+WvI9V7wwkw7NGzDx2iF0bdM43iGJ1Crnn5DK5OuH0iFiZeZ9BRoyU4Kp5f45Zy3X/n0OR3doxpvXDKZ98wbxDkmkVuqe0oS3IlZmvuCJjIRfmVkJphZ7ftpqfvOP+Qzq3orXrj6Jlo2T4x2SSK0WXpn5qcsG8N3m3Zz9t2m8PP0b8hP0BgAlmFrI3fnrv5Zz9+QljOzTnuevPJHG9evGOyyRhHFmn9DKzH1Tm3P7u4sZ8dfPmbxgfcI9nKkEU8sUFTl3TFrMo59kcnFaZx675Hjq19WilSKHW6cWDXn16pN44coTaVAvifGvfc15/5dORmbiPDcT0wRjZiPNbLmZZZrZrSXsf8jM5gWvFWa2NWjvb2bTzWyxmS0ws4sjzvky4pz1ZvZO0D7czLZF7Ls9lt+tOiooLOKmN+fx8vRvuWZYd+694DgtWikSR2bGab3b8v4Np/CXC/uRu3Mflzw7g8ufn8mS9dvjHV7MWayWoDazJGAFMAJYC8wCxrr7klKOvx443t1/bmZHAu7uK82sIzAHONrdtxY755/Au+7+spkNB/7L3c+ONsa0tDSfPXt2Zb5etfT+gg1c99pc/uvHRzL+9F7xDkdEisnLL+SV6d/y2KeZbM/L57z+nbh5xJF0btUo3qFViJnNcfe08o4r98/bIFFUxkAg091Xufs+YAIwqozjxwKvA7j7CndfGbxfD2wCUorF1Qw4HXinkvHVOtMys2naoC7Xntoj3qGISAka1EviF8O688VvT+PaU3swZeEGzvjL59z93hI279oX7/CqXDTjJyvN7AEzO6aC1+4ErInYXhu0HcTMugDdgE9K2DcQSAayiu06D/jY3SP7mYPNbL6ZfWBmfSoYb403LTOHQd1ba1hMpJpr3rAevxvZm89uGc5/HN+JFzNWc+r9n/L4p5m1qkxzNL+J+hEa6nrWzL4ys3FB76EqjQEmunthZKOZdQBeAX7m7sXv89vf4wnMBbq4ez/gb5TSswnin21ms7Ozs6vsC8Tbd7m7WbN5D0N7tol3KCISpQ7NG3Lf6L5M/fUwBvVozQNTlzP8gc94bcZ3tWJts3ITjLvvcPdn3H0I8DvgDmCDmb1kZj3LOHUd0DliOzVoK8kYDkwW4SGw94E/uPtXxfa1ITQE935EnNvdfWfwfgpQLziu+Pd52t3T3D0tJSWl+O4aKz1Y0fXknq3jHImIVFSvdk155vI0Jl47mM6tGvH7txfy44e/4MNFG2t0qeao5mDM7Fwzext4GPgL0B14D5hSxqmzgF5m1s3MkgklkUklXL830BKYHtGWDLwNvOzuE0u49mhgsrvnRZzT3oJqWcGwWh0gt7zvV1ukZ+bQtml9eqQ0iXcoIlJJaV1bMfHawTxzeRp1zLj273M4/4kMZq7eHO/QKiWap+9WAp8CD7h7RkT7RDMbVtpJ7l5gZuOBqUAS8Ly7Lzazu4HZ7h5ONmOACX5gmr4IGAa0NrMrg7Yr3X1exDn3FvvI0cAvzawA2AOM8Zqc+iugqMiZnpXLqUemqCKlSA1nZow4ph2nHZXCP+eu5a8freCip6ZzRu+2/HZkb45q3zTeIUat3NuUzaxJeOiptqkttykvWb+dnzz6JQ9e2I/RA1LjHY6IVKE9+wp5IWM1T3yWxa69BVxwQio3jTiSji0axi2mKrtNGXjczFpEXLilmT1/SNFJlcrQ/ItIrdUwOYlfDe/JF7ecxlVDu/HuvPUMf/Az/jxlKdt258c7vDJFk2D6Rj7g6O5bgONjF5JU1LTMHLqnNKZD8/j9RSMisdWycTJ/+OkxfPJfp3J23w48/eUqTrn/E576PIu8/MLyLxAH0SSYOmbWMrxhZq2Ibu5GDoN9BUXMXL2Zk3vo9mSRRJDashF/vag/U244hQFdWvLnD5Zx2oOf8ebsNRRWs8U0o0kwfwGmm9k9ZvY/QAZwf2zDkmjNX7uV3fsKOVnPv4gklKM7NOOFnw3k9V8Mom2zBvx24gLOeuQL/r3k+2pza3M0z8G8DFwAfA9sBM5391diHZhEZ9rKHOoYDO6u+ReRRDS4R2ve+dUQnvjPE8gvdK5+eTYXP/UVc77dEu/QohvqCm4vzgYaAJjZEe7+XUwjk6hkZOVwXKfmNG9UL96hiEicmBlnHdeBHx3TjjdmreHhf6/kgicyOLNPO245szc928bn+bhoHrQ818xWAquBz4FvgA9iHJdEYdfeAr7+bitDNDwmIkC9pDpcOqgLn98ynN+MOJL0zFzOfPgLbntrId9vzyv/AlUsmjmYe4BBwAp37wacAXxV9ilyOMxcvZmCItcEv4gcoHH9ulx/Ri8+v2U4lw3qwsQ5azj1gU95YOoytucdvlubo0kw+e6eS+husjru/ilQ7gM2EnvpmTkk161DWteW5R8sIgmndZP63HluHz6+eThn9mnP459mcer9n/LctNXsLYj9rc3RJJitZtYE+AJ41cweAXbFNiyJxrTMHNK6tKRBPZVEFpHSHdG6EY+MOZ7J1w/l2E7NuWfyEu6ZXGLtxyoVzST/KEJre90E/CfQHLg7lkFJ+XJ27mXZxh3ccuZR8Q5FRGqIYzs155WrTmLayhxSW8b+wewyE0xQzXKyu58GFAEvxTwiiUpGVmihaD3/IiIVNbTX4fm9UeYQWVAArMjMmh+WaCRqGZk5NG1Ql+M66X8aEameohki2wksNLOPiJh7cfcbYhaVlCs9K4fB3VuTVEfL84tI9RRNgnkreEk1ES6PfPXQ7vEORUSkVOUmGHfXvEs180N5ZM2/iEj1VW6CMbPVwEErp7m7/nyOk2mZObRrVp8eKY3jHYqISKmiGSKLfKiyAXAh0Co24Uh5wuWRh6s8sohUc9Gsppwb8Vrn7g8DPz0MsUkJlm3cweZd+zQ8JiLVXjRDZCdEbNYh1KNRwbE4Sc/U/IuI1AzRJIq/RLwvILSq8kWxCUfKk54VKo/cvnmDeIciIlKmaO4iO62yFzezkcAjQBLwrLvfW2z/Q0D4+o2Atu7ewsz6A08AzYBC4H/d/Y3gnBeBU4FtwXlXuvs8C01IPAL8BNgdtM+tbOzV0b6CImas2syFaanxDkVEpFzR1IP5k5m1iNhuGZROLu+8JOBx4CzgGGCsmR0TeYy73+Tu/d29P/A3fnjeZjdwubv3AUYCD0fGANwSPs/d5wVtZwG9gtc4QgmqVpm3Zit78gsZouX5RaQGiGY15bPcfWt4w923EOollGcgkOnuq9x9HzCB0MKZpRkLvB58xgp3Xxm8Xw9sAlLK+bxRwMse8hXQwsw6RBFnjZGeqfLIIlJzRJNgksysfnjDzBoC9cs4PqwTsCZie23QdhAz6wJ0Az4pYd9AIBnIimj+XzNbYGYPRcQW9efVVOmZKo8sIjVHNAnmVeBjM7vKzK4CPqLqV1UeA0wMFtfcL+iBvAL8zN2LgubbgN7AiYSex/ldRT7IzMaZ2Wwzm52dnX3okR8mO/cWMG+NyiOLSM0RzXMw9wH/AxwdvO5x9/ujuPY6oHPEdmrQVpIxBMNjYWbWDHgf+EMw5BWOZ0MwDLYXeIHQUFzUn+fuT7t7mrunpaSUN+pWfcxcnUtBkTNUCUZEaohonoPpBnzm7h8G2w3NrKu7f1POqbOAXsH56wglkUtKuH5voCUwPaItGXib0JzKxGLHd3D3DcFdY+cBi4Jdk4DxZjYBOAnY5u4byvt+NUV6Zi7JdeswoIvKI4tIzRDNENk/CBUbCysM2srk7gXAeGAqsBR4090Xm9ndZnZuxKFjgAnuHrne2UXAMOBKM5sXvPoH+141s4XAQqANod4VwBRgFZAJPAP8KorvVmOkZ+ZwYleVRxaRmiOaBy3rBneBAeDu+4IeRrncfQqhX/yRbbcX276zhPP+Dvy9lGueXkq7A9dFE1dNo/LIIlITRdODyY7scZjZKCAndiFJcSqPLCI1UTQ9mGsJDUs9BhihW4Evj2lUcoD0lSqPLCI1TzRLxWQBg8ysSbC9M+ZRyQFUHllEaqKoVkU2s58CfYAG4Rok7n53DOOSwHe5u1m7ZQ/jhqm+m4jULNGsRfYkcDFwPaEhsguBLjGOSwLTguX5tf6YiNQ00UzyD3H3y4Et7n4XMBg4MrZhSVh6lsoji0jNFE2C2RP83G1mHYF8oFYtIlldFRU5GZk5nNyzjcoji0iNE80czORgqfwHgLmAE3qQUWJs6cbtbNmdz8kaHhORGiiau8juCd7+08wmAw3cfVtZ50jVyMjU8y8iUnNFdRdZWLDA5N4YxSLFTMvMoYfKI4tIDRXNHIzEwb6CImau3qzei4jUWEow1dTX321hT36hEoyI1FilJhgzW2JmfzSzHoczIAlJz8qljsEglUcWkRqqrB7MWKAx8C8zm2lmNwW3KcthkJGZw3GpLWjeUOWRRaRmKjXBuPt8d7/N3XsANwBHAF+Z2adm9ovDFmECCpdHPrmHei8iUnNFNQfj7l+5+02EVlFuATwW06gSXLg8suZfRKQmi6Zk8omEhssuAFYDTxFFRUupvGkrc6mv8sgiUsOVmmDM7E+EFrncDEwATnb3tYcrsESWkZVDmsoji0gNV1YPJg8Y6e4rD1cwAtk7QuWRfztS5ZFFpGYraw5mNTCoeKOZXWZml8QupMSWkRVanl/rj4lITVdWghkPvF1C+1vAb6K5uJmNNLPlZpZpZreWsP8hM5sXvFaY2dagvb+ZTTezxWa2wMwujjjn1eCai8zseTOrF7QPN7NtEde7PZoYq5uMzFyaNajLsSqPLCI1XFlDZPVKKo/s7rvCv9TLYmZJwOPACGAtMMvMJrn7kohr3RRx/PXA8cHmbuByd18ZPHszx8ymuvtW4FXg0uC414CrgSeC7S/d/ezyYquu3J1pmTkM7qHyyCJS85XVg2loZgdVuTKzpkByFNceCGS6+yp330foRoFRZRw/FngdwN1XhOd+3H09sAlICbaneACYCaRGEUuN8N3m3azbuke3J4tIrVBWgnkOmGhm+8sjm1lXQoniuSiu3QlYE7G9Nmg7SPAZ3YBPStg3kFBCyyrWXg+4DPgwonmwmc03sw/MrE8UMVYr4fLISjAiUhuUOkTm7g+a2U7gCzNrEjTvBO519ydKO6+SxgAT3b0wstHMOgCvAFe4e1Gxc/4P+MLdvwy25wJd3H2nmf0EeAfoVfyDzGwcMA7giCOOqNpvcYgyMnNp36wB3duoPLKI1HxlPsnv7k+6exegK9DV3btUILmsAzpHbKcGbSUZQzA8FmZmzYBLQuIHAAATEklEQVT3gT+4+1fF9t1BaMjs5ohYt4fnjNx9ClDPzA7qCrj70+6e5u5pKSkpUX6V2CsqcjKychjSs7XKI4tIrRDtUjE73H1HBa89C+hlZt3MLJlQEplU/CAz6w20BKZHtCUTuoPtZXefWOz4q4EzgbGRvRoza2/Bb+ZgWK0OkFvBmONmyYZQeeShGh4TkVoiZvVg3L2A0K3OU4GlwJvuvtjM7jazcyMOHQNMCCbtwy4ChgFXRtx23D/Y9yTQDphe7Hbk0cAiM5sPPAqMKXbNam3/8y9KMCJSS1gN+h1c5dLS0nz27NnxDgOAy5+fyfqte/j3zafGOxQRkTKZ2Rx3TyvvuHIXuwwuNoTQPMz+49395UpHJwfYV1DErNWbuSit1txxLSIS1WrKrwA9gHlA+C4vB5Rgqki4PPIQDY+JSC0STQ8mDTimJs1n1DTpmTkqjywitU40k/yLgPaxDiSRpWflqjyyiNQ60fRg2gBLzGwmsDfc6O7nln6KRGtHXj7z1mzl2lO7xzsUEZEqFU2CuTPWQSSymas3U1jkWp5fRGqdchOMu39+OAJJVOmZofLIJ6g8sojUMuXOwZjZIDObZWY7zWyfmRWa2fbDEVwiSM/M4cSurVQeWURqnWgm+R8jtJT+SqAhoforj8cyqESxaUcey7/fwZCeuntMRGqfaNciywSS3L3Q3V8ARsY2rMQwPSu0VJrmX0SkNopmkn93sPjkPDO7H9hADNcwSyTpmTkqjywitVY0ieKy4LjxwC5CS/BfEMugEoG7k56Zq/LIIlJrRXMX2bdm1hDo4O53HYaYEsK3uaHyyHr+RURqq2juIjuH0DpkHwbb/c3soLouUjHpwfL8Wn9MRGqraIbI7gQGAlsB3H0e0C2GMSWE9MwclUcWkVotmgST7+7birVp4ctDUFTkTM/K5eSebVQeWURqrWjuIltsZpcASWbWC7gByIhtWLVbuDzyyXr+RURqsWh6MNcDfQgtdPk6sB34dSyDqu3SM1UeWURqv2juItsN/CF4SRVIz8qlZ9smtGvWIN6hiIjETKkJprw7xbRcf+XsLShk5upcLk7rHO9QRERiqqwezGBgDaFhsRmAZqOrwNffbSUvv0jDYyJS65U1B9Me+D1wLPAIMALIcffPo13C38xGmtlyM8s0s1tL2P+Qmc0LXivMbGvQ3t/MppvZYjNbYGYXR5zTzcxmBNd8I1jGBjOrH2xnBvu7RvuPcDhlBOWRT1J5ZBGp5UpNMMHClh+6+xXAICAT+MzMxkdzYTNLIrTq8lnAMcBYMzum2Gfc5O793b0/8DfgrWDXbuByd+9DaGHNh82sRbDvPuAhd+8JbAGuCtqvArYE7Q8Fx1U70zJz6KvyyCKSAMq8iyzoFZwP/B24DngUeDvKaw8EMt19lbvvAyYAo8o4fiyh4TjcfYW7rwzerwc2ASkWemjkdGBicM5LwHnB+1HBNsH+M6yaPWSyIy+f+Wu36fZkEUkIZU3yv0xoeGwKcJe7L6rgtTsRmsMJWwucVMpndSG0OsAnJewbCCQDWUBrYKu7F0Rcs1Pxz3P3AjPbFhyfU8G4Y2bGKpVHFpHEUVYP5lKgF3AjkGFm24PXjhhUtBwDTHT3wshGM+sAvAL8zN2LquKDzGycmc02s9nZ2dlVccmopWflqDyyiCSMsuZg6rh70+DVLOLV1N2bRXHtdYSW9g9LDdpKMoZgeCzMzJoB7wN/cPevguZcoIWZhXtekdfc/3nB/ubB8cW/19PunubuaSkpKVF8jaqTkZmr8sgikjBiWThsFtAruOsrmVASOejZGjPrDbQEpke0JROa63nZ3cPzLbi7A58Co4OmK4B3g/eTgm2C/Z8Ex1cL4fLIuj1ZRBJFzBJMME8yHpgKLAXedPfFZna3mUU+pDkGmFAsGVwEDAOujLiNuX+w73fAzWaWSWiO5bmg/TmgddB+M3DQbdHxtL88sib4RSRBRLPYZaW5+xRCNwlEtt1ebPvOEs77O6E710q65ipCd6gVb88DLjyEcGNq2spQeeQ+HVUeWUQSQyyHyCTg7mRk5TKkRxuVRxaRhKEEcxiEyyNreExEEokSzGEwTcvzi0gCUoI5DDKycujQvAHdVB5ZRBKIEkyMFRX9MP9SzVauERGJKSWYGFuyYTtbd+cztJfmX0QksSjBxFi4PPIQrT8mIglGCSbGpmXm0EvlkUUkASnBxNDegkJmfbNZd4+JSEJSgomhud+GyiMP6aH5FxFJPEowMZSRFSqPPEgJRkQSkBJMDKUH5ZGbNVB5ZBFJPEowMRIujzxU8y8ikqCUYGIkXB55iNYfE5EEpQQTI9Myg/LIR6g8sogkJiWYGMnIymFgN5VHFpHEpQQTA5t25LHi+516el9EEpoSTAxkZIbKI2uCX0QSmRJMDKRn5tC8YT2O6dgs3qGIiMSNEkwVc3fSM3MY3L21yiOLSEJTgqli3+TuZv22PE7upeExEUlsMU0wZjbSzJabWaaZ3VrC/ofMbF7wWmFmWyP2fWhmW81scrFzvow4Z72ZvRO0DzezbRH7bo/ldytNeHn+k7U8jIgkuLqxurCZJQGPAyOAtcAsM5vk7kvCx7j7TRHHXw8cH3GJB4BGwDWR13X3UyLO+SfwbsTuL9397Kr8HhWVnplDR5VHFhGJaQ9mIJDp7qvcfR8wARhVxvFjgdfDG+7+MbCjtIPNrBlwOvBO1YR76AqLnOmrchnSU+WRRURimWA6AWsittcGbQcxsy5AN+CTClz/POBjd98e0TbYzOab2Qdm1qeUzxpnZrPNbHZ2dnYFPq58S9aHyiOfrOVhRESqzST/GGCiuxdW4JwDejzAXKCLu/cD/kYpPRt3f9rd09w9LSUlpdIBlyQ9Kzz/ogl+EZFYJph1QOeI7dSgrSRjODBZlMnM2hAagns/3Obu2919Z/B+ClAvOO6wSQ/KI7dVeWQRkZgmmFlALzPrZmbJhJLIpOIHmVlvoCUwvQLXHg1Mdve8iOu0t2Diw8wGEvpuuYcQf4WoPLKIyIFidheZuxeY2XhgKpAEPO/ui83sbmC2u4eTzRhggrt75Plm9iXQG2hiZmuBq9x9asQ59xb7yNHAL82sANgDjCl+zVgKl0dWghERCYlZgoH9Q1VTirXdXmz7zlLOPaWk9mDf8BLaHgMeq0ycVSE9M1Qe+aTureIVgohItVJdJvlrvPSsHPp1VnlkEZEwJZgqsD0vnwVrt+nuMRGRCEowVUDlkUVEDqYEUwXSM3NoUE/lkUVEIinBVIH0zBxO7KryyCIikZRgDtGm7Xms3LRTtyeLiBSjBHOIMrJCz3Jqgl9E5EBKMIdomsoji4iUSAnmELg7GZk5DOmh8sgiIsUpwRyC1Tm7WL8tjyGafxEROYgSzCFID+ZfhirBiIgcRAnmEGQE5ZG7tm4U71BERKodJZhKKixyMrJUHllEpDRKMJW0ZP12tu3J1/CYiEgplGAqaVpmqDzykB5af0xEpCRKMJWUkZXDke1UHllEpDRKMJWQlx8qjzxET++LiJRKCaYS5n63ReWRRUTKoQRTCfWS6nDaUSkqjywiUoa68Q6gJjqxayte+NnAeIchIlKtxbQHY2YjzWy5mWWa2a0l7H/IzOYFrxVmtjVi34dmttXMJhc750UzWx1xXv+g3czs0eCzFpjZCbH8biIiUraY9WDMLAl4HBgBrAVmmdkkd18SPsbdb4o4/nrg+IhLPAA0Aq4p4fK3uPvEYm1nAb2C10nAE8FPERGJg1j2YAYCme6+yt33AROAUWUcPxZ4Pbzh7h8DOyrweaOAlz3kK6CFmXWoRNwiIlIFYplgOgFrIrbXBm0HMbMuQDfgkyiv/b/BMNhDZla/op8nIiKxV13uIhsDTHT3wiiOvQ3oDZwItAJ+V5EPMrNxZjbbzGZnZ2dXPFIREYlKLBPMOqBzxHZq0FaSMUQMj5XF3TcEw2B7gRcIDcVF/Xnu/rS7p7l7WkpKSjQfKSIilRDLBDML6GVm3cwsmVASmVT8IDPrDbQEpkdz0fC8ioWWMD4PWBTsmgRcHtxNNgjY5u4bDv1riIhIZcTsLjJ3LzCz8cBUIAl43t0Xm9ndwGx3DyebMcAEd/fI883sS0JDYU3MbC1wlbtPBV41sxTAgHnAtcEpU4CfAJnAbuBnsfpuIiJSPiv2ez2hmFk28G0lT28D5FRhOFWlusYF1Tc2xVUxiqtiamNcXdy93DmGhE4wh8LMZrt7WrzjKK66xgXVNzbFVTGKq2ISOa7qcheZiIjUMkowIiISE0owlfd0vAMoRXWNC6pvbIqrYhRXxSRsXJqDERGRmFAPRkREYkIJpoLMrLOZfWpmS8xssZndGO+YAMysgZnNNLP5QVx3xTumSGaWZGZfFy+/EE9m9o2ZLQzKPsyOdzxhZtbCzCaa2TIzW2pmg6tBTEdFlMiYZ2bbzezX8Y4LwMxuCv6bX2Rmr5tZg3jHBGBmNwYxLY73v5WZPW9mm8xsUURbKzP7yMxWBj9bVvXnKsFUXAHwG3c/BhgEXGdmx8Q5JoC9wOnu3g/oD4wMVjSoLm4ElsY7iBKc5u79q9ltpI8AH7p7b6Af1eDfzd2XB/9O/YEBhB5mfjvOYWFmnYAbgDR3P5bQQ91j4hsVmNmxwC8ILWXVDzjbzHrGMaQXgZHF2m4FPnb3XsDHwXaVUoKpoGAttLnB+x2E/s8f91Wbg/XZdgab9YJXtZhgM7NU4KfAs/GOpbozs+bAMOA5AHff5+5byz7rsDsDyHL3yj6kXNXqAg3NrC6hGlLr4xwPwNHADHff7e4FwOfA+fEKxt2/ADYXax4FvBS8f4nQ0ltVSgnmEJhZV0JF0mbEN5KQYBhqHrAJ+Mjdq0VcwMPAb4GieAdSjAP/MrM5ZjYu3sEEugHZwAvBkOKzZtY43kEVE/XitLHm7uuAB4HvgA2E1iD8V3yjAkJrJJ5iZq3NrBGhZaw6l3PO4dYuYr3GjUC7qv4AJZhKMrMmwD+BX7v79njHA+DuhcEQRiowMOimx5WZnQ1scvc58Y6lBEPd/QRC1VCvM7Nh8Q6I0F/jJwBPuPvxwC5iMHRRWcHCtecC/4h3LADBvMEoQom5I9DYzC6Nb1Tg7kuB+4B/AR8SWjcxmnIkcRGsBVnlIx5KMJVgZvUIJZdX3f2teMdTXDCk8ikHj7nGw8nAuWb2DaGqpqeb2d/jG1JI8Ncv7r6J0HzCwLLPOCzWAmsjep8TCSWc6uIsYK67fx/vQAI/Ala7e7a75wNvAUPiHBMA7v6cuw9w92HAFmBFvGMq5vuI1ek7EBr5qFJKMBUUlAl4Dljq7n+NdzxhZpZiZi2C9w2BEcCy+EYF7n6bu6e6e1dCQyufuHvc/8I0s8Zm1jT8HvgxP5R+iBt33wisMbOjgqYzgCVxDKm4A0qbVwPfAYPMrFHw/80zqAY3RQCYWdvg5xGE5l9ei29EB5kEXBG8vwJ4t6o/IGbL9ddiJwOXAQuD+Q6A37v7lDjGBNABeMnMkgj94fCmu1ebW4KroXbA26HfSdQFXnP3D+Mb0n7XEypLkQysopqUnggS8QjgmnjHEubuM8xsIjCX0B2eX1N9npz/p5m1BvKB6+J5s4aZvQ4MB9oE5U/uAO4F3jSzqwitKn9RlX+unuQXEZFY0BCZiIjEhBKMiIjEhBKMiIjEhBKMiIjEhBKMiIjEhBKMSAWZmUc+LGpmdc0sOxYrRZvZZ2ZWqYU4zey8yIVYD+VaIpWhBCNScbuAY4MHWiH0bMi6OMZTmvOA6rDStyQoJRiRyplCaIVoKPZ0u5kNNLPpwWKVGeGn8oO6Jc8H748LaoU0iryomTU0swlBHZi3gYYR+34cXHeumf0jWA8vXNfm/qC2zUwz62lmQwitGfZAUL+lR3CZC4NjVpjZKTH6txEBlGBEKmsCMCYobtWXA1fUXgacEixWeTvwp6D9EaCnmf0H8AJwjbvvLnbdXwK73f1oQk9bDwAwszbAH4EfBQt0zgZujjhvm7sfBzwGPOzuGYSWArklqOOSFRxX190HAr8Ori8SM1oqRqQS3H1BUK5hLKHeTKTmhJbt6UVohdp6wTlFZnYlsAB4yt3TS7j0MODRiM9YELQPIjTclR4sb5MMTI847/WInw+VEXp4cdY5QNeyvqPIoVKCEam8SYRqkQwHWke03wN86u7/ESShzyL29QJ2ElpaviKMUI2fsaXs91LeF7c3+FmI/v8vMaYhMpHKex64y90XFmtvzg+T/leGG4NqlY8S6qW0NrPRJVzzC+CS4PhjCQ2/AXwFnBwuuxusBn1kxHkXR/wM92x2AE0r/rVEqoYSjEgluftad3+0hF33A382s685sJfwEPC4u68ArgLuDS/pHuEJoImZLQXuJjSUhbtnE0pWrwfDZtOB3hHntQzabwRuCtomALcENxv0QOQw02rKIjVcUMwtzd1z4h2LSCT1YEREJCbUgxERkZhQD0ZERGJCCUZERGJCCUZERGJCCUZERGJCCUZERGJCCUZERGLi/wGEYicoT2TsGgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x115ce6a20>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(tree_params['max_depth'], \n",
    "         tree_grid.cv_results_['mean_test_score'])\n",
    "plt.xlabel('Max depth')\n",
    "plt.ylabel('Mean CV accuracy');"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Напечатайте лучшее значение `max_depth`, то есть такое, при котором среднее значение метрики качества на кросс-валидации максимально. Посчитайте также, какова теперь доля верных ответов на отложенной выборке. Все это можно сделать с помощью обученного экземпляра класса `GridSearchCV`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Best params: {'max_depth': 6}\n",
      "Best cross validaton score 0.732204081632653\n"
     ]
    }
   ],
   "source": [
    "print(\"Best params:\", tree_grid.best_params_)\n",
    "print(\"Best cross validaton score\", tree_grid.best_score_)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.7256666666666667"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tuned_tree_acc_valid = accuracy_score(y_valid, \n",
    "                                      tree_grid.predict(X_valid))\n",
    "tuned_tree_acc_valid"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.006073809995378543"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tuned_tree_acc_valid / tree_acc_valid - 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**<font color='red'>Вопрос 4.</font> Имеется ли на кривой валидации по максимальной глубине дерева пик `accuracy`, если перебирать `max_depth` от 2 до 10? Повысила ли настройка глубины дерева качество классификации (accuracy) более чем на 1% на отложенной выборке (надо посмотреть на выражение (acc2 - acc1) / acc1 * 100%, где acc1 и acc2 – доли верных ответов на отложенной выборке до и после настройки max_depth соответственно)?**\n",
    "- да, да\n",
    "- да, нет\n",
    "- нет, да\n",
    "- нет, нет\n",
    "\n",
    "**<font color='red'>Ответ:</font> 2 вариант**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Обратимся опять (как и в 1 домашке) к картинке, демонстрирующей шкалу SCORE для расчёта риска смерти от сердечно-сосудистого заболевания в ближайшие 10 лет.\n",
    "<img src='../../img/SCORE2007.png' width=70%>\n",
    "\n",
    "Создайте бинарные признаки, примерно соответствующие этой картинке:\n",
    "- $age \\in [45,50), \\ldots age \\in [60,65) $ (4 признака)\n",
    "- верхнее артериальное давление: $ap\\_hi \\in [120,140), ap\\_hi \\in [140,160), ap\\_hi \\in [160,180),$ (3 признака)\n",
    "\n",
    "Если значение возраста или артериального давления не попадает ни в один из интервалов, то все бинарные признаки будут равны нулю. Далее будем строить дерево решений с этим признаками, а также с признаками ``smoke``, ``cholesterol``  и ``gender``. Из признака ``cholesterol`` надо сделать 3 бинарных, соотв-х уникальным значениям признака ( ``cholesterol``=1,  ``cholesterol``=2 и  ``cholesterol``=3), эта техника называется dummy-кодированием или One Hot Encoding (OHE). Признак ``gender`` надо перекодировать: значения 1 и 2 отобразить на 0 и 1. Признак лучше переименовать в ``male`` (0 - женщина, 1 - мужчина). В общем случае кодирование значений делает ``sklearn.preprocessing.LabelEncoder``, но в данном случае легко обойтись и без него.\n",
    "\n",
    "Итак, дерево решений строится на 12 бинарных признаках (исходные признаки не берем).\n",
    "\n",
    "Постройте дерево решений с ограничением на максимальную глубину = 3 и обучите его на всей исходной обучающей выборке. Используйте `DecisionTreeClassifier`, на всякий случай зафикисровав `random_state=17`, остальные аргументы (помимо `max_depth` и `random_state`) оставьте по умолчанию. \n",
    "\n",
    "**<font color='red'>Вопрос 5.</font> Какой бинарный признак из 12 перечисленных оказался самым важным для обнаружения ССЗ, то есть поместился в вершину построенного дерева решений?**\n",
    "- Верхнее артериальное давление от 160 до 180 (мм рт.ст.)\n",
    "- Пол мужской / женский\n",
    "- Верхнее артериальное давление от 140 до 160 (мм рт.ст.)\n",
    "- Возраст от 50 до 55 (лет)\n",
    "- Курит / не курит\n",
    "- Возраст от 60 до 65 (лет)\n",
    "\n",
    "**<font color='red'>Ответ:</font> 3 вариант**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "sub_df = pd.DataFrame(df.smoke.copy())\n",
    "sub_df['male']  = df.gender - 1\n",
    "\n",
    "sub_df['age_45_50'] = ((df.age_years >= 45) \n",
    "                       & (df.age_years < 50) ).astype('int')\n",
    "sub_df['age_50_55'] = ((df.age_years >= 50) \n",
    "                       & (df.age_years < 55) ).astype('int')\n",
    "sub_df['age_55_60'] = ((df.age_years >= 55) \n",
    "                       & (df.age_years < 60) ).astype('int')\n",
    "sub_df['age_60_65'] = ((df.age_years >= 60) \n",
    "                       & (df.age_years < 65) ).astype('int')\n",
    "\n",
    "sub_df['ap_hi_120_140'] = ((df.ap_hi >= 120) \n",
    "                           & (df.ap_hi < 140)).astype('int')\n",
    "sub_df['ap_hi_140_160'] = ((df.ap_hi >= 140) \n",
    "                           & (df.ap_hi < 160)).astype('int')\n",
    "sub_df['ap_hi_160_180'] = ((df.ap_hi >= 160) \n",
    "                           & (df.ap_hi < 180)).astype('int')\n",
    "\n",
    "sub_df['chol=1'] = (df.cholesterol == 1).astype('int')\n",
    "sub_df['chol=2'] = (df.cholesterol == 2).astype('int')\n",
    "sub_df['chol=3'] = (df.cholesterol == 3).astype('int')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>smoke</th>\n",
       "      <th>male</th>\n",
       "      <th>age_45_50</th>\n",
       "      <th>age_50_55</th>\n",
       "      <th>age_55_60</th>\n",
       "      <th>age_60_65</th>\n",
       "      <th>ap_hi_120_140</th>\n",
       "      <th>ap_hi_140_160</th>\n",
       "      <th>ap_hi_160_180</th>\n",
       "      <th>chol=1</th>\n",
       "      <th>chol=2</th>\n",
       "      <th>chol=3</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>id</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    smoke  male  age_45_50  age_50_55  age_55_60  age_60_65  ap_hi_120_140  \\\n",
       "id                                                                           \n",
       "0       0     1          0          1          0          0              0   \n",
       "1       0     0          0          0          1          0              0   \n",
       "2       0     0          0          1          0          0              1   \n",
       "3       0     1          1          0          0          0              0   \n",
       "4       0     0          1          0          0          0              0   \n",
       "\n",
       "    ap_hi_140_160  ap_hi_160_180  chol=1  chol=2  chol=3  \n",
       "id                                                        \n",
       "0               0              0       1       0       0  \n",
       "1               1              0       0       0       1  \n",
       "2               0              0       0       0       1  \n",
       "3               1              0       1       0       0  \n",
       "4               0              0       1       0       0  "
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sub_df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [],
   "source": [
    "tree = DecisionTreeClassifier(max_depth=3, \n",
    "                              random_state=17).fit(sub_df, target)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Widget Javascript not detected.  It may not be installed or enabled properly.\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "02625368ced443e28bcecb24bb7b9be3"
      }
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "dot_data = StringIO()\n",
    "export_graphviz(tree, feature_names=sub_df.columns, \n",
    "                out_file=dot_data, filled=True)\n",
    "graph = pydotplus.graph_from_dot_data(dot_data.getvalue())  \n",
    "Image(value=graph.create_png())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Хотя отрисовку деревьев в `sklearn` можно улучшать, но видно, что в первую очередь в дереве проверяется, попадает ли верхнее артериальное давление в диапазон от 140 до 160 мм рт.ст."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.1"
  },
  "name": "lesson4_part2_Decision_trees.ipynb"
 },
 "nbformat": 4,
 "nbformat_minor": 1
}