teach_Model_Tuning（Binary）.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "9631465a",
   "metadata": {},
   "source": [
    "# 分类模型建模调参"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "18439093",
   "metadata": {},
   "source": [
    "## 【任务描述】   \n",
    "使用sklearn进行分类模型的建模和调参：  \n",
    "1. 掌握调用sklearn库中的预置模型建模的方法；\n",
    "2. 掌握常用的分类模型调参技巧；\n",
    "3. 能够对分类模型进行评估和优化。\n",
    "    \n",
    "## 【任务准备】    \n",
    "学员已完成分类数据特征工程实践课程，已保存完成特征工程的数据。        "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6b1d0806",
   "metadata": {},
   "source": [
    "## 【任务实施】    \n",
    "### 步骤 1: 启动Jupyter Notebook    \n",
    "+ 在搜索栏输入\"cmd\"命令，启动命令提示符窗口。    \n",
    "+ 输入\"jupyter notebook\"命令，并按回车键启动Jupyter Notebook。   \n",
    "    \n",
    "### 步骤 2: 创建新的Notebook    \n",
    "+ 在Jupyter的Web界面中，点击右上角的 \"New\" 按钮。    \n",
    "+ 选择 \"Python 3\"内核来创建一个新的Python 3 Notebook。    \n",
    "    \n",
    "### 步骤 3: 导入必要的库\n",
    "+ 使用numpy和pandas库来进行数据处理，使用sklearn、lightgbm和xgboost库来进行数据的建模预测和评估，使用matplotlib和seaborn来进行数据的可视化。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "0770b822",
   "metadata": {},
   "outputs": [],
   "source": [
    "# 数据处理库\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "import scipy.stats as st\n",
    "\n",
    "# 模型预测库\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "from sklearn.ensemble import RandomForestClassifier\n",
    "\n",
    "# 参数搜索和评价库\n",
    "from sklearn.model_selection import cross_val_score,train_test_split\n",
    "from sklearn.metrics import confusion_matrix, classification_report, roc_curve\n",
    "\n",
    "# 定义字体，在图表中正常显示汉字\n",
    "plt.rcParams['font.sans-serif'] = ['SimHei']\n",
    "# 在图表中正常显示负号\n",
    "plt.rcParams['axes.unicode_minus'] = False\n",
    "\n",
    "#导入warnings包，利用过滤器来实现忽略警告语句。\n",
    "import warnings\n",
    "warnings.filterwarnings('ignore')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2eca7221",
   "metadata": {},
   "source": [
    "### 步骤 4: 导入数据集"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "676df48a",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>是否幸存</th>\n",
       "      <th>仓位等级</th>\n",
       "      <th>兄弟姐妹个数</th>\n",
       "      <th>父母子女个数</th>\n",
       "      <th>票价</th>\n",
       "      <th>登船港口</th>\n",
       "      <th>年龄段</th>\n",
       "      <th>性别分类_female</th>\n",
       "      <th>性别分类_male</th>\n",
       "      <th>性别分类_unknown</th>\n",
       "      <th>客舱_labelEncode</th>\n",
       "      <th>船票信息_labelEncode</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>7.2500</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>135</td>\n",
       "      <td>409</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>71.2833</td>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>74</td>\n",
       "      <td>472</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>7.9250</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>135</td>\n",
       "      <td>533</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>53.1000</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>50</td>\n",
       "      <td>41</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>8.0500</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>135</td>\n",
       "      <td>374</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>710</th>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>5</td>\n",
       "      <td>29.1250</td>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>135</td>\n",
       "      <td>378</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>711</th>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>13.0000</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>135</td>\n",
       "      <td>84</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>712</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>30.0000</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>27</td>\n",
       "      <td>13</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>713</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>30.0000</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>54</td>\n",
       "      <td>9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>714</th>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>7.7500</td>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>135</td>\n",
       "      <td>372</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>715 rows × 12 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "     是否幸存  仓位等级  兄弟姐妹个数  父母子女个数       票价  登船港口  年龄段  性别分类_female  性别分类_male  \\\n",
       "0       0     3       1       0   7.2500     1    2            0          1   \n",
       "1       1     1       1       0  71.2833     2    3            1          0   \n",
       "2       1     3       0       0   7.9250     1    2            1          0   \n",
       "3       1     1       1       0  53.1000     1    3            1          0   \n",
       "4       0     3       0       0   8.0500     1    3            0          1   \n",
       "..    ...   ...     ...     ...      ...   ...  ...          ...        ...   \n",
       "710     0     3       0       5  29.1250     3    3            1          0   \n",
       "711     0     2       0       0  13.0000     1    2            0          1   \n",
       "712     1     1       0       0  30.0000     1    2            1          0   \n",
       "713     1     1       0       0  30.0000     2    2            0          1   \n",
       "714     0     3       0       0   7.7500     3    3            0          1   \n",
       "\n",
       "     性别分类_unknown  客舱_labelEncode  船票信息_labelEncode  \n",
       "0               0             135               409  \n",
       "1               0              74               472  \n",
       "2               0             135               533  \n",
       "3               0              50                41  \n",
       "4               0             135               374  \n",
       "..            ...             ...               ...  \n",
       "710             0             135               378  \n",
       "711             0             135                84  \n",
       "712             0              27                13  \n",
       "713             0              54                 9  \n",
       "714             0             135               372  \n",
       "\n",
       "[715 rows x 12 columns]"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data_FE = pd.read_csv('data_FE.csv')\n",
    "data_FE"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fc7bcf2d",
   "metadata": {},
   "source": [
    "### 步骤 5: 切割训练集和测试集    \n",
    "* 按比例切割训练集和测试集(一般测试集的比例有30%、25%、20%、15%和10%)\n",
    "* 按目标变量分层进行等比切割\n",
    "* 设置随机种子以便结果能复现"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "aeda386f",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "((536, 11), (179, 11))"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X = data_FE.iloc[:,1:]\n",
    "y = data_FE.iloc[:,:1]\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, stratify=y, random_state=42)\n",
    "# 查看数据形状\n",
    "X_train.shape, X_test.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8e58c4a2",
   "metadata": {},
   "source": [
    "### 步骤 6: 模型创建     \n",
    "6.1 创建基于线性模型的分类模型（逻辑回归）"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "11dc33c1",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training set score: 0.80\n",
      "Testing set score: 0.76\n"
     ]
    }
   ],
   "source": [
    "# 默认参数逻辑回归模型\n",
    "lr = LogisticRegression()\n",
    "lr.fit(X_train, y_train)\n",
    "# 查看训练集和测试集score值\n",
    "print(\"Training set score: {:.2f}\".format(lr.score(X_train, y_train)))\n",
    "print(\"Testing set score: {:.2f}\".format(lr.score(X_test, y_test)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "e41bcd7b",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training set score: 0.81\n",
      "Testing set score: 0.77\n"
     ]
    }
   ],
   "source": [
    "# 调整参数后的逻辑回归模型\n",
    "lr2 = LogisticRegression(C=100)\n",
    "lr2.fit(X_train, y_train)\n",
    "print(\"Training set score: {:.2f}\".format(lr2.score(X_train, y_train)))\n",
    "print(\"Testing set score: {:.2f}\".format(lr2.score(X_test, y_test)))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5b54daaf",
   "metadata": {},
   "source": [
    "6.2 创建基于树的分类模型（随机森林）"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "674c5459",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training set score: 1.00\n",
      "Testing set score: 0.77\n"
     ]
    }
   ],
   "source": [
    "rfc = RandomForestClassifier()\n",
    "rfc.fit(X_train, y_train)\n",
    "print(\"Training set score: {:.2f}\".format(rfc.score(X_train, y_train)))\n",
    "print(\"Testing set score: {:.2f}\".format(rfc.score(X_test, y_test)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "23a2e752",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training set score: 0.87\n",
      "Testing set score: 0.80\n"
     ]
    }
   ],
   "source": [
    "# 调整参数后的随机森林分类模型\n",
    "rfc2 = RandomForestClassifier(n_estimators=100, max_depth=5)\n",
    "rfc2.fit(X_train, y_train)\n",
    "print(\"Training set score: {:.2f}\".format(rfc2.score(X_train, y_train)))\n",
    "print(\"Testing set score: {:.2f}\".format(rfc2.score(X_test, y_test)))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "faaaf029",
   "metadata": {},
   "source": [
    "### 步骤 7: 输出模型预测结果    \n",
    "* 输出模型预测分类标签\n",
    "* 输出不通分类标签的预测概率"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "8403ff29",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0, 0, 1, 0, 0, 1, 0, 0, 0, 0, 1, 1, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0,\n",
       "       0, 0, 1, 1, 1, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 1, 1, 0, 0, 1, 1, 0,\n",
       "       0, 0, 1, 0, 1, 0, 0, 1, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 0, 0, 1, 0,\n",
       "       1, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 0, 0, 1, 1, 0, 1, 0, 1, 0, 0, 0,\n",
       "       1, 0, 1, 0, 0, 1, 1, 0, 0, 0, 1, 1, 0, 1, 1, 0, 0, 1, 0, 0, 0, 0,\n",
       "       1, 0, 0, 0, 0, 1, 1, 0, 0, 1, 0, 1, 0, 0, 0, 0, 1, 1, 1, 0, 1, 0,\n",
       "       1, 0, 1, 1, 1, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0,\n",
       "       1, 1, 1, 0, 0, 1, 0, 1, 0, 1, 1, 1, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0,\n",
       "       0, 0, 0, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 0, 0, 1, 1, 1, 0,\n",
       "       0, 0, 0, 1, 1, 0, 0, 1, 0, 0, 1, 0, 0, 1, 0, 1, 0, 0, 0, 0, 1, 1,\n",
       "       0, 1, 1, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1,\n",
       "       0, 0, 0, 0, 0, 0, 1, 0, 1, 0, 1, 1, 0, 1, 0, 0, 0, 0, 0, 0, 1, 1,\n",
       "       0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 1, 1,\n",
       "       0, 1, 0, 1, 0, 0, 0, 0, 0, 1, 0, 1, 1, 0, 1, 1, 0, 1, 0, 1, 0, 0,\n",
       "       1, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 1, 0, 0, 1, 0, 0, 0, 0, 1, 1, 1,\n",
       "       1, 0, 0, 0, 1, 1, 0, 1, 1, 1, 0, 1, 0, 0, 0, 0, 1, 1, 0, 1, 1, 0,\n",
       "       0, 1, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 1, 0, 1, 1, 0, 0, 0, 1, 0, 0,\n",
       "       0, 1, 1, 1, 0, 1, 0, 1, 1, 0, 0, 1, 1, 0, 0, 1, 0, 1, 0, 0, 0, 0,\n",
       "       0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1, 1, 0, 1, 0, 1, 1,\n",
       "       0, 1, 1, 0, 0, 1, 0, 0, 1, 1, 1, 0, 1, 0, 0, 0, 1, 1, 0, 0, 1, 1,\n",
       "       0, 1, 0, 1, 1, 0, 0, 0, 0, 1, 0, 1, 1, 0, 0, 0, 0, 1, 0, 0, 1, 0,\n",
       "       1, 0, 0, 1, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 1, 0, 1, 1,\n",
       "       0, 0, 0, 1, 0, 1, 0, 0, 0, 0, 1, 0, 1, 0, 0, 1, 1, 0, 0, 1, 0, 0,\n",
       "       0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 1, 1, 1,\n",
       "       1, 0, 1, 0, 0, 0, 1, 1], dtype=int64)"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 预测标签\n",
    "pred = lr.predict(X_train)\n",
    "pred"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "2108b1b9",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0.78673133, 0.21326867],\n",
       "       [0.89755559, 0.10244441],\n",
       "       [0.37928636, 0.62071364],\n",
       "       ...,\n",
       "       [0.6032195 , 0.3967805 ],\n",
       "       [0.43883354, 0.56116646],\n",
       "       [0.07981926, 0.92018074]])"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 预测标签概率\n",
    "pred_proba = lr.predict_proba(X_train)\n",
    "pred_proba"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a99ab837",
   "metadata": {},
   "source": [
    "### 步骤 8: 模型评估    \n",
    "* 模型评估是为了知道模型的泛化能力。\n",
    "* 交叉验证（cross-validation）是一种评估泛化性能的统计学方法，它比单次划分训练集和测试集的方法更加稳定、全面。\n",
    "* 在交叉验证中，数据被多次划分，并且需要训练多个模型。\n",
    "* 最常用的交叉验证是 k 折交叉验证（k-fold cross-validation），其中 k 是由用户指定的数字，通常取 5 或 10。\n",
    "* 准确率（precision）度量的是被预测为正例的样本中有多少是真正的正例\n",
    "* 召回率（recall）度量的是正类样本中有多少被预测为正类\n",
    "* f-分数是准确率与召回率的调和平均"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "d9305c59",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0.72222222, 0.77777778, 0.75925926, 0.81481481, 0.68518519,\n",
       "       0.81481481, 0.86792453, 0.83018868, 0.88679245, 0.83018868])"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "lr = LogisticRegression(C=100)\n",
    "scores = cross_val_score(lr, X_train, y_train, cv=10)\n",
    "# k折交叉验证分数\n",
    "scores"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "c7ca3ea2",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Average cross-validation score: 0.80\n"
     ]
    }
   ],
   "source": [
    "# 平均交叉验证分数\n",
    "print(\"Average cross-validation score: {:.2f}\".format(scores.mean()))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5c54bb75",
   "metadata": {},
   "source": [
    "### 步骤 9: 混淆矩阵      \n",
    "* 计算二分类问题的混淆矩阵\n",
    "* 计算精确率、召回率以及f-分数"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "d2945d19",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[278,  41],\n",
       "       [ 62, 155]], dtype=int64)"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 训练模型\n",
    "lr = LogisticRegression(C=100)\n",
    "lr.fit(X_train, y_train)\n",
    "# 模型预测结果\n",
    "pred = lr.predict(X_train)\n",
    "# 混淆矩阵\n",
    "confusion_matrix(y_train, pred)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "a2c0ccee",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.82      0.87      0.84       319\n",
      "           1       0.79      0.71      0.75       217\n",
      "\n",
      "    accuracy                           0.81       536\n",
      "   macro avg       0.80      0.79      0.80       536\n",
      "weighted avg       0.81      0.81      0.81       536\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# 精确率、召回率以及f1-score\n",
    "print(classification_report(y_train, pred))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "664d3c5c",
   "metadata": {},
   "source": [
    "### 步骤 10: 绘制ROC曲线    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "114a65b1",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.legend.Legend at 0x20bbcd397c0>"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjUAAAGsCAYAAAAllFaOAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAA9hAAAPYQGoP6dpAAA4nElEQVR4nO3deXhUZZr+8bvIHswKYQ8QBNPK2mKaJS2ooNDsg9gqCMrgsLmQEdPCiDa2C4zaAXXEMc0SQIRRu1VaGmwcIIjSLYFOJAZQkACyS4eqYJIihPf3B0P9CNkqRSWVnHw/11XXlaqct86TQ6BuznnO+9qMMUYAAAD1XCNfFwAAAOANhBoAAGAJhBoAAGAJhBoAAGAJhBoAAGAJhBoAAGAJhBoAAGAJ/r4uoLZcvHhRx44dU1hYmGw2m6/LAQAAbjDGKD8/X61atVKjRpWfi2kwoebYsWOKjY31dRkAAMADR44cUZs2bSrdpsGEmrCwMEmXDkp4eLiPqwEAAO5wOByKjY11fY5XpsGEmsuXnMLDwwk1AADUM+60jtAoDAAALIFQAwAALIFQAwAALIFQAwAALIFQAwAALIFQAwAALIFQAwAALIFQAwAALIFQAwAALIFQAwAALMGnoebMmTOKi4tTbm6uW9unp6frxhtvVNOmTZWSklKzxQEAgHrFZ6Hmxx9/1LBhw9wONKdPn9aIESN0//33a/v27Vq1apU2b95cs0UCAIB6w2cLWt53332677779Le//c2t7VetWqWWLVvqmWeekc1m07PPPqslS5bo9ttvr+FKAQCom4wxKiwu8XUZLiEBfm4tPFlTfBZqUlNT1aFDByUlJbm1fVZWlu644w7XwfrFL36h2bNnV7i90+mU0+l0PXc4HNdULwAAdYkxRmP+e7t2HsrzdSkuOb8bpNBAn0UL311+6tChQ7W2dzgciouLcz0PDw/X0aNHK9x+3rx5ioiIcD1iY2M9rhUAgLqmsLikTgWausB3caqa/P39FRQU5HoeHBysgoKCCrefPXu2nnjiCddzh8NBsAEAWFLGnIEKDfTzdRkKCfBtDfUm1ERHR+v06dOu5/n5+QoMDKxw+6CgoFIhCAAAqwoN9PPpZZ+6ot4cgYSEBK1evdr1PDMzU61bt/ZhRQCAuqiuNc/WlILz1v8Zq6vOhRqHw6GQkBAFBASUen3EiBF65JFHtHnzZt1666169dVXNWjQIB9VCQCoi+pi8yxqT52bUbhbt25at25dmdebNm2q3//+9xo0aJBatmyp7OxszZkzxwcVAgDqqobYPHtLuyif97LUFT4/U2OMKfW8ssn4pk+frrvuukt79uxR//79FR4eXsPVAQDqq7rSPFvTfD03TF3i81BTXR07dlTHjh19XQYAoI64sofmyj4TmmcbHv60AQD1Fj00uFKd66kBAMBdFfXQ0GfSMHGmBgBgCVf20NBn0jARagAAlkAPDfjTBwDUG1dPrMcEdLgSoQYAUC/QFIyq0CgMAKgXKptYj8ZgSJypAQDUQ1dPrEdjMCRCDQCgDqlsMUom1kNV+I0AANQJ9MzgWtFTAwCoE9xdjJL+GVSEMzUAgDqnssUo6Z9BRQg1AIA6h54ZeILfGACAz1S0wjbgCUINAMAnaAyGt9EoDADwCVbYhrdxpgYA4HOssA1vINQAAHyOxmB4A79BAOCmyma7RfXRGAxvI9QAgBtoagXqPhqFAcAN7s52i+qjMRjewpkaAKimyma7RfXRGAxvIdQAQDmu7p9hhWig7uNvJQBchf4ZoH6ipwYArlJZ/wz9H0DdxZkaAKjE1f0z9H8AdRehBgAqQf8MUH/wNxUAxGrRgBUQagA0eDQGA9ZAozCABo/VogFr4EwNAFyB1aKB+otQA6BBqqiHhsZgoP7iby6ABoceGsCa6KkB0ODQQwNYE2dqADRo9NAA1kGoAdCg0UMDWAeXnwAAgCUQagAAgCUQagAAgCUQagAAgCUQagAAgCUQagAAgCUQagAAgCUQagAAgCUQagAAgCUwjSaABqGiVbkBWAehBoDlsSo30DBw+QmA5bEqN9AwcKYGQIPCqtyAdRFqADQorMoNWBd/swF4zZXNuHUJjcFAw0CoAeAVNOMC8DUahQF4RUXNuHUJjcGAtXGmBoDXXdmMW5fQGAxYG6EGaKC83f9yZd8KzbgAfIF/dYAGiP4XAFbks56a7OxsJSQkKCoqSsnJyTLGVDnmlVdeUfPmzRUeHq67775bZ86cqYVKAeupyf4X+lYA+IpPztQ4nU4NHz5cgwYN0po1a/T4448rLS1NEydOrHDM1q1btXz5cm3dulV+fn6aMWOGZs6cqbS0tNorHLAgb/e/0LcCwFd8EmrWr18vu92ulJQUhYaG6qWXXtIjjzxSaaj56quvNGTIEMXHx0uS7r//fi1atKjC7Z1Op5xOp+u5w+Hw3g8AWAj9LwCswif/kmVlZal3794KDQ2VJHXr1k05OTmVjunSpYseffRRTZkyRWFhYVqyZInuvPPOCrefN2+ennvuOa/WDWurqxPH1QQmowNgRT4JNQ6HQ3Fxca7nNptNfn5+ysvLU1RUVLljBg8erE6dOqljx46SpISEBM2aNavCfcyePVtPPPFEqX3GxsZ66SeA1dA4CwD1n08ahf39/RUUFFTqteDgYBUUFFQ45r333tOhQ4e0d+9enTlzRl26dNEDDzxQ4fZBQUEKDw8v9QAqUh8mjqsJNPUCsBKfnKmJjo5WdnZ2qdfy8/MVGBhY4ZjVq1dr2rRprp6ahQsXKiIiQmfPnlVkZGRNlosGpq5OHFcTaOoFYCU+CTUJCQlavHix63lubq6cTqeio6MrHHPhwgWdPHnS9fz48eOSpJISegPgXTTOAkD95JN/ufv16ye73a4VK1ZowoQJmj9/vgYOHCg/Pz85HA6FhIQoICCg1JjExESlpKSoTZs2CgkJ0cKFC9WnTx81adLEFz8CAACoY3wSavz9/ZWamqqxY8cqOTlZJSUlSk9Pl3TpTqiFCxdq1KhRpcYkJSXp2LFjev755/Xjjz+qT58+WrJkiQ+qBwAAdZHNuDOVbw05evSoMjIy1LdvX8XExNTovhwOhyIiImS322kaRhkF5y/opmc/lSTl/G4Ql58AoI6ozue3T//lbt26tVq3bu3LEgAAgEX4bO0nAAAAbyLUAAAASyDUAAAASyDUAAAASyDUAAAAS+C+VTQYla3CzarVAFD/EWrQILAKNwBYH5ef0CC4uwo3q1YDQP3FmRo0OJWtws2q1QBQfxFqUCdV1v/iiSt7ZliFGwCsiX/ZUefQ/wIA8AQ9Nahz3O1/8QQ9MwBgXZypQZ1WWf+LJ+iZAQDrItSgTqP/BQDgLi4/AQAASyDUAAAASyDUAAAASyDUAAAASyDUAAAASyDUAAAASyDUAAAASyDUAAAASyDUAAAAS2CqVvjc1StyX7miNgAA7iLUwKdYkRsA4C1cfoJPVbYiNytqAwCqgzM1qDOuXpGbFbUBANVBqEGdwYrcAIBrwScIat2VjcE0BQMAvIVQg1pFYzAAoKbQKIxaVVFjME3BAIBrxZka+MyVjcE0BQMArhWhBj5DYzAAwJu4/AQAACyBUAMAACyBUAMAACyBhga4JSsrS2+//bYyMzOVn5+vsLAw9ejRQ1OmTFH37t19XR4AAIQaVG7Hjh1KSkrSl19+WeZ727dv11tvvaU+ffvqP195VT1vSajy/ZhsDwBQUwg1qNC6des0ZswYFRUVlXo9ODi41Gvbv/xS/fvfpqajZiv0+qqDDQAANYGeGpRrx44dpQJNfHy8Fi1aJLvdrsLCQp09e1YLX39D/tFtJEnmwnn9+NE8OY9/69b7M9keAMDbOFODciUlJbkCza9//WutXLlSgYGBru9HRETo36ZMVcqhNvrxk9+rYN82mQvnFbf/A332h/Qq35/J9gAA3saZGpSRmZnp6qGJj48vE2iuZPMPUNPhM3XDDfGSLl2K+m7PNwoN9K/0QaABAHgboQZlpKamur6eMWNGqUBjjFHB+Qv/97jU9GvzC9C0Rx4pdzwAALWFy08oIzMz0/X12LFjXV9XtsL2vfeP1b/PeLzMeAAAagtnalBGfn6+pEt3OUVERLher2yF7RZNoxUUFFRqPAAAtYkzNSgjLCxMklRUVCS73V4q2Fx29QrbDodDTqez1HgAAGoTZ2pQRo8ePVxfv/vuu+Vuc3mF7ctNv6tWrSp3PAAAtYVQgzImT57s+vq1117T+fPnK93e6XTq9ddfL3c8AAC1hVCDMnr06KG+fftKkvbt26fx48dXGGycTqfGjx+vffv2SZISExNZCwoA4BOEGpRr4cKFCg4OliS999576tatm1L/+y1ddP4kSbLb7Vq0aJG6d++u999/X5IUEhKiBQsW+KxmAEDDZjPGGF8XURscDociIiJkt9sVHh7u63LqhYrWfpJfgFRSXOqlkJAQvf/++xo6dGgtVggAsLrqfH5zpgalXDm53u13DtJf/3eT+vzfpSiXqwJNYmKi0tPTCTQAAJ/ilm64VDi53q3/oZbx3ys/81MVn/peF88XqmfHlrr55z/X5MmT6aEBANQJhBq4VDS5niQFNuugJndNk3Rpsr33p/Zh/SYAQJ1CqEG5rpxc72qssA0AqIuqFWoKCwu1YcMGZWVl6dSpU7ruuuvUrl07DRkyRHFxcdXacXZ2tiZOnKj9+/fr4Ycf1ssvv+z2B+V9992nmJgYvfHGG9XaJ9x3eXI9AADqC7cahYuKivTcc8+pe/fu2rx5szp16qRRo0apV69eKigo0IMPPqjBgwdrz549bu3U6XRq+PDh6tmzpzIyMpSTk6O0tDS3xn766afatGmTnn/+ebe2BwAADUOV/xX//vvvNWbMGN19993avXu3a9HCKyUnJ2vLli269957NXPmTD344IOVvuf69etlt9uVkpKi0NBQvfTSS3rkkUc0ceLESscVFhZq+vTpmj9/viIjIyvd1ul0utYiki7dEgYAAKyryjM127dv14IFC/T000+XG2guu+2225Senq5vv/1WVU19k5WVpd69eys0NFSS1K1bN+Xk5FRZ7PPPP6/CwkL5+/tr06ZNle5n3rx5ioiIcD1iY2OrfH8AAFB/VRlqxo0bp/79+7v1ZlFRUXrxxRer7I1xOBylenBsNpv8/PyUl1f+nTeSdPjwYaWkpKhjx446fPiwkpOTNXr06AqDzezZs2W3212PI0eOuPUzAACA+sknnaD+/v5lzvoEBweroKBAUVFR5Y5JS0tT8+bNtXHjRgUFBWnGjBlq166dNm7cqLvuuqvM9kFBQZWeWWqIjDEqLC6p8PsF5yv+HgAAdZ1PQk10dLSys7NLvZafn6/AwMAKx/zwww8aMGCAK6iEhYWpU6dOOnjwYI3WahUVTqwHAIBFuB1q4uLi3Lrl+vvvv69ym4SEBC1evNj1PDc3V06nU9HR0RWOiY2NLdV3c/HiRf3www9q165dlftD5RPrXe2WdlEKCSh/jhoAAOoqt0ONu7dcu6Nfv36y2+1asWKFJkyYoPnz52vgwIHy8/OTw+FQSEiIAgICSo359a9/rZ49e+qPf/yjevXqpTfeeENOp1OJiYleq6uhqGxiPYnJ9QAA9ZPbocbdZmG3durvr9TUVI0dO1bJyckqKSlRenq6pEt3Qi1cuFCjRo0qNSY+Pl7/8z//ozlz5mjv3r26/vrr9fHHHyssLMxrdTUUTKwHALAin32yjRo1St99950yMjLUt29fxcTESLp0KaoiQ4cOZSXoariyMZgmYACA1fn0v+utW7dW69atfVmCZdEYDABoaNxaJgH1T0WNwTQBAwCsisaKBuDKxmCagAEAVuWTW7pRu2gMBgA0BD65pRsAAMDbfHJLNwAAgLfRKAwAACzBq6Hm/Pnz3nw7AAAAt3nUPXr8+HG98MIL+vbbb1VScmlSN2OM9u7dq+PHj3u1QAAAAHd4dKbmgQce0MmTJxUSEqKQkBDdc8892rdvn6ZNm+bt+gAAANziUaj56quv9Oabb+rJJ5+U3W7XtGnTtGTJEm3YsMHb9QEAALjFo1DTqlUrffbZZ0pISNA333yjwsJCdenSRbt37/Z2fQAAAG7xqKdm3rx5GjdunO666y6NGjVKXbt2lSQlJiZ6tThU7MrFKsvDApYAgIbGo1AzevRoHTt2TGFhYUpNTdWqVav0008/acKECd6uD+VgsUoAAMryeO78qKgo19cPPfSQN2qBmyparLI8LGAJAGgoPAo1R48e1WOPPaaHH35YQ4YM0c0336wWLVpo6dKlatGihbdrRCWuXKyyPCxgCQBoKDxqFJ48ebKCg4PVo0cPSdKaNWsUExOjqVOnerM2uOHyYpUVPQg0AICGwqMzNZ9//rn27t2rVq1aSZJuuOEGvfjii+rSpYtXiwMAAHCXR2dq4uLi9L//+7+lXtu0aZPatWvnlaIAAACqy6MzNSkpKRo5cqTWrFmj9u3b6+DBg/r888/18ccfe7s+AAAAt3h0pmbAgAHKzs7WL3/5S0lSv379lJ2drTvuuMOrxQEAALjL41u627dvr9mzZ+v8+fPy9/f4bQAAALzCozM1+fn5mjx5spo3b67Q0FBlZ2erTZs22rlzp7frAwAAcItHoWbixIn64YcftGLFCjVu3FgRERF67LHH9Mgjj3i7PgAAALd4dN3os88+c52dadSokWw2m8aPH6+XXnrJ2/UBAAC4xaMzNT/72c+0fPlySZLNZpPNZtP27dvVuXNnrxYHAADgLo/O1LzxxhsaMmSIFi1apPz8fN177706dOiQ1q5d6+36AAAA3OJRqElISND+/fv1ySef6OjRo2rTpo2GDh2qiIgIb9cHAADgFo/vxY6IiNC4ceO8WQsAAIDHPOqpWbRokY4dO+btWgAAADzmUah5/fXX9fXXX3u7FgAAAI95FGqeeeYZvfDCCzp37py36wEAAPCIRz01+/fv18WLF9WpUydNmDBBjRs3dn3v2Wef9VpxAAAA7vIo1OTm5io+Pl7x8fE6deqU63Wbzea1wgAAAKrDo1CzbNkyb9cBAABwTarsqfn6669LnY2pymeffXZNBQEAAHiiylBz6NAh3X777fryyy8r3e7s2bO69957tWbNGhljvFYgAACAO6q8/DR8+HB16NBB06dPV9OmTfXAAw+oV69eatasmfLz8/X999/r448/1vvvv6+ZM2fq4Ycfro26AQAASnGrp6Zz585KT0/Xpk2b9Kc//UkpKSk6ffq0wsLC1K5dOw0aNEjbt29XZGRkDZcLAABQvmo1Ct9xxx264447aqoWAAAAj3m89hNqlzFGhcUlkqSC8yU+rgYAgLqHUFMPGGM05r+3a+ehPF+XAgBAneXRMgmoXYXFJeUGmlvaRSkkwM8HFQEAUPdwpqaeyZgzUKGBl4JMSIAfszgDAPB/CDX1TGign0ID+WMDAOBqXH4CAACWQKgBAACWQKgBAACWQKgBAACWUK1Qc+7cOe3atUunT58u872LFy9q7dq1XivMyowxKjh/oRoPJtsDAKAqbt9Gs3HjRo0ePVrnz5+XzWZTamqqJkyYoMOHD2vx4sVasmSJSkpKNGLEiJqst95jIj0AAGqG22dqZs+erZkzZ8rpdGrVqlWaOXOmhg4dqg4dOmjLli169dVXdeTIkZqs1RIqmkjPHUy2BwBAxdw+U/PNN9+4Li/dfffdmjBhglq3bq2srCx17ty5xgq0sisn0nMHk+0BAFAxt0PN+fPnFRYW5noeFBSkOXPmqG3btjVSWEPARHoAAHiP25+oxhj98pe/lJ/fpTMLDodDQ4YMUWBgYKntdu3a5d0KAQAA3OB2qFm2bFlN1gEAAHBN3A41Dz74oE6cOKH09HQVFxcrMTFRcXFxHu84OztbEydO1P79+/Xwww/r5ZdfdrtfpLi4WDfffLPeeOMN3XbbbR7XAAAArMPtu5/++te/6oYbbtBLL72klJQUde3aVStXrvRop06nU8OHD1fPnj2VkZGhnJwcpaWluT3+5ZdfVnZ2tkf7BgAA1uR2qElOTlZKSoqysrK0a9cuffjhh5oxY4ZHO12/fr3sdrtSUlJ0/fXX66WXXtKSJUvcGvvdd9/p1VdfVfv27T3ad22ofHI9JtIDAKAmuH35ac+ePRo+fLjr+Z133qnCwkIdP35cLVu2rNZOs7Ky1Lt3b4WGhkqSunXrppycHLfGTpkyRbNmzdL69esr3c7pdMrpdLqeOxyOatXoKSbXAwDAN9w+U1NSUuIKIZeFhITowoUL1d6pw+Eo1Y9js9nk5+envLzKg8CyZctkt9s1c+bMKvcxb948RUREuB6xsbHVrtMT7k6ux0R6AAB4V7Vu6W7btm2pZt6zZ8+qa9euatTo/2ejf/7zn1Xv1N9fQUFBpV4LDg5WQUGBoqKiyh1z+vRpzZ49Wxs2bJC/f9Vlz549W0888YTrucPhqLVgc1llk+sxkR4AAN7ldqjZvHmz13YaHR1dptE3Pz+/zJw3V0pKStKkSZPUo0cPt/YRFBRUJjjVNibXAwCg9rj9iWuz2fTLX/6y1FkZTyUkJGjx4sWu57m5uXI6nYqOjq5wzLvvvquwsDC9+eabki6tGD5s2DDNmTNHs2bNuuaaAABA/eZ2qLn99tuVl5en8PDwa95pv379ZLfbtWLFCk2YMEHz58/XwIED5efnJ4fDoZCQEAUEBJQac/DgwVLP77vvPiUlJWnw4MHXXA8AAKj/qtVT460eEH9/f6Wmpmrs2LFKTk5WSUmJ0tPTJV26E2rhwoUaNWpUqTFX38IdHBysFi1aKDIy0is1AQCA+q1aDR+VBYjLoaekxL15WEaNGqXvvvtOGRkZ6tu3r2JiYiRduhTlji1btri1HQAAaBiqFWrS09NLrdR9rVq3bq3WrVt77f0AAEDDVa1Q061bN6/01AAAAHib27cyLVu2rMzkewAAAHVFtVbpBgAAqKuufdIZAACAOoBQAwAALIFQAwAALIFQAwAALIFQAwAALIFQAwAALIFQAwAALIFQAwAALIFQAwAALIFQAwAALKFaC1qifMYYFRaXSJIKzpf4uBoAABomQs01MsZozH9v185Deb4uBQCABo3LT9eosLik3EBzS7sohQT4+aAiAAAaJs7UeFHGnIEKDbwUZEIC/GSz2XxcEQAADQehxotCA/0UGsghBQDAF7j8BAAALIFQAwAALIFQAwAALIFQAwAALIFQAwAALIFQAwAALIFQAwAALIFQAwAALIFQAwAALIFQAwAALIFQAwAALIFQAwAALIFQAwAALIFQAwAALIFQAwAALIFQAwAALIFQAwAALIFQAwAALIFQAwAALIFQAwAALIFQAwAALIFQAwAALIFQAwAALIFQAwAALIFQAwAALIFQAwAALIFQAwAALIFQAwAALIFQAwAALIFQAwAALIFQAwAALIFQAwAALIFQAwAALIFQAwAALIFQAwAALIFQAwAALIFQAwAALMFnoSY7O1sJCQmKiopScnKyjDFVjklNTVXLli0VEBCgu+66S8ePH6+FSgEAQH3gk1DjdDo1fPhw9ezZUxkZGcrJyVFaWlqlY7Zt26ZnnnlGK1eu1MGDB1VUVKQnn3yydgoGAAB1nk9Czfr162W325WSkqLrr79eL730kpYsWVLpmH379umtt97SwIED1aZNG02cOFEZGRm1VDEAAKjr/H2x06ysLPXu3VuhoaGSpG7duiknJ6fSMZMmTSr1fN++ferYsWOF2zudTjmdTtdzh8NxDRUDAIC6zidnahwOh+Li4lzPbTab/Pz8lJeX59b4M2fO6O2339b06dMr3GbevHmKiIhwPWJjY6+5bgAAUHf5JNT4+/srKCio1GvBwcEqKChwa/z06dPVt29fDR06tMJtZs+eLbvd7nocOXLkmmoGAAB1m08uP0VHRys7O7vUa/n5+QoMDKxy7NKlS7V161ZlZmZWul1QUFCZ4AQAAKzLJ2dqEhIS9Le//c31PDc3V06nU9HR0ZWO++qrr5SUlKQ1a9aoefPmNV0mAACoR3wSavr16ye73a4VK1ZIkubPn6+BAwfKz89PDodDxcXFZcacPHlSw4cP11NPPaWePXvq3LlzOnfuXG2XDgAA6iif9dSkpqZq6tSpat68uT744APNnz9f0qU7odatW1dmzOrVq3Xq1CnNmTNHYWFhrgcAAIAk2Yw7U/nWkKNHjyojI0N9+/ZVTExMje7L4XAoIiJCdrtd4eHhXnvfgvMXdNOzn0qScn43SKGBPmlTAgDAkqrz+e3TT+DWrVurdevWviwBAABYBAtaAgAASyDUAAAASyDUAAAASyDUAAAASyDUAAAASyDUAAAASyDUAAAASyDUAAAASyDUAAAASyDUAAAASyDUAAAASyDUAAAASyDUAAAASyDUAAAASyDUAAAASyDUAAAASyDUAAAASyDUAAAASyDUAAAASyDUAAAASyDUAAAASyDUAAAASyDUAAAASyDUAAAASyDUAAAASyDUAAAAS/D3dQEAgPqtpKRExcXFvi4D9VRAQID8/Py88l6EGgCAR4wxOnHihM6ePevrUlDPRUZGqkWLFrLZbNf0PoQaAIBHLgeaZs2aKTQ09Jo/kNDwGGNUUFCgU6dOSZJatmx5Te9HqAEAVFtJSYkr0DRp0sTX5aAeCwkJkSSdOnVKzZo1u6ZLUTQKAwCq7XIPTWhoqI8rgRVc/j261t4sQg0AwGNccoI3eOv3iFADAAAsgVADAGgw0tLSZLPZZLPZFBwcrJ49e+qvf/2r6/vbtm1T165dFRwcrNtvv12HDh1yfe/IkSMaMGCAGjdurP79++vAgQNu7dPTcag+Qg0AoEHp0qWL8vLytH//fg0bNkyjR4+Ww+HQ2bNnNWLECP3Lv/yLcnJyFB0drXHjxkm6dJfOqFGj1LZtW+3evVuJiYkaM2aMjDGV7svTcfCMzTSQI+twOBQRESG73a7w8HCvvW/B+Qu66dlPJUk5vxuk0EBuKANgfUVFRTp48KDi4uIUHBws6dIHeGFxiU/qCQnwc6svIy0tTQsXLlRmZqakSzUHBQXpyy+/1J49e/TMM8/o4MGDstlsOnz4sNq1a6fc3Fx9//33Gj16tI4fP67g4GAVFRUpIiJCu3btUufOnSvc3+bNmz0a19CU9/t0WXU+v/kEBgB4RWFxies/ebXN0/9U/ulPf5IkxcbG6r333lO3bt1c4aht27YKCwtTTk6Odu7cqZ49e7o+cIODg5WUlKRGjSq/4PHFF19UOC4tLU1paWnasmWLJCk3N1dxcXGuszi33XabHnroIf3zn//Ua6+9pjfeeEMjRozQiy++qL1792rlypWSpN27d+u2227TyZMn5e/vrw0bNig5OVlHjhzRmDFj9OabbyooKKjax6Y+ItQAABqU3bt3KzIyUufPn1dISIjefvttNW/eXHl5eYqIiCi1bUREhPLy8nT06FE1b9681Pf+8z//s8p9VTbu73//e5Xj3377bTVp0kR/+MMfdPPNN0uSxowZo1tvvVUXL15Uo0aNtGHDBo0YMUL+/v46cOCARo4cqbfeekv9+/fXmDFj9Morr2jOnDlV7ssKCDUAAK8ICfBTzu8G+Wzf7oqPj9df/vIXvfDCCzp69KgmTpxY4baXz5oUFxd7NCmcp+MuO3funLZu3aqAgADXa/Hx8WrWrJl27typhIQEbdiwQf/+7/8uSVq9erV+/vOf61//9V8lSVOnTtWSJUsINQAAVIfNZqsXfYWBgYFq3769kpKS1L17d+Xm5qp9+/Zq0qSJsrOzS21rt9sVHR2tyMhI7du3r9T3WrVqpSVLluhXv/pVhfuqbNzVCgoKyrw2derUUoHmsjFjxmj9+vW66aab9PXXX+vOO++UdOnM0K5duxQZGSlJunDhgq677roK67Ma7n4CADRIXbp0Ud++fbV48WJJUrdu3ZSVleU6O3Pw4EGdO3dOXbp00c9//nPt2rVLFy9elCT99NNPOnXqlNq2bVvpPiobZ7PZVFLy/xurMzIyyoxv3Lhxue979913a/369dq0aZPuvPNOV89MmzZtNGLECGVmZiozM1NZWVnauHFjNY9M/UWoAQA0WNOmTdPSpUt14cIFDRs2TIWFhXr66ad14MABJSUlqX///mrTpo1Gjhwpm82mmTNnKjc3V8nJyerUqZNuuummSt+/snFt2rTRN998o7y8PJ08eVKvvvqq23V37dpVdrtd77zzju6++27X6/fff78+//xzfffdd5Kk1157rdLLa1ZDqAEANFhjxozRhQsX9Mknnyg8PFxr167Vn//8Z910001yOByuO4yuu+46ffrpp/rqq6900003KScnR3/+85+rvI28snG33367Bg8erK5du2r48OF64YUXqlX7qFGj9Oc//7nU5a8OHTpo+fLleuKJJ9S5c2dlZ2dr9erV1T8w9RTz1Fwj5qkB0BBVNq8IUF3emqeGMzUAAMASCDUAAMASCDUAAMASCDUAAMAS6GoFAPhMVlaW3n77bWVmZio/P19hYWHq0aOHpkyZou7du/u6PNQzhBoAQK3bsWOHkpKS9OWXX5b53vbt2/XWW28pMTFRCxYsUEJCgg8qRH3E5ScAQK1at26d+vXrVybQXH0r7xdffKF+/fpp3bp1tVke6jFCDQCg1uzYsUNjxoxRUVGRpEuLMy5atEh2u12FhYU6e/asFi1apPj4eEmX5i8ZM2aMduzY4cuyUU8QagAAtSYpKckVaH7961/r66+/1rRp01yTqkVERGjatGnKysrSPffcI+lSsLm8CnVN2rJli9q3b1/j+7ls7ty5euihhzwen5aWpttuu+2at7ESQg0AoFZkZma6LjnFx8dr5cqVCgwMLHfboKAgvfPOO64zNl988YWysrK8Vsttt92mtLQ0r70f6gZCDQCgVqSmprq+njFjRoWB5rLAwEA9/vjj5Y4HyuOzUJOdna2EhARFRUUpOTlZ7ixBlZ6erhtvvFFNmzZVSkpKLVQJAPCWzMxM19djx451a8y4cePKHe+pqVOnymazKT09XRMnTpTNZtPUqVNLbbN27Vq1a9dOUVFRev31112vP/TQQ5o7d67rDNJ//dd/ub63Y8cO9erVSxERERo9erTsdrvre++8847at2+vxo0b61e/+pXOnDlTan+/+93vFBkZqXbt2unzzz93vf7BBx8oPj5eTZs21aOPPuq6bFeV559/XjExMerYsaN27drl1pgNGzbIZrOVesydO9f1/RUrVqhTp05q2rSp/uM//qPUZ7bNZtM333yjKVOmKDo6utTP/uabb6p9+/Zq1aqV5s6dq4sXL7pVj8eMDxQVFZn27dubKVOmmP3795shQ4aYpUuXVjrm1KlTJjw83Dz33HPm22+/NTfffLPZtGmT2/u02+1GkrHb7ddafik/OYtNu6c+Me2e+sT85Cz26nsDQF1VWFhocnJyTGFhodtjunTpYiSZ4ODgau0rKCjISDJdu3atbpllFBQUmLy8PJOYmGjefPNNk5eXZwoKCowxxmzevNlcd911pk+fPmb37t3mtddeM4GBga7vP/jgg6Z3796mT58+5pNPPjFHjhwxxhiTl5dnmjRpYp577jlz6NAhM2jQIDNp0iRjjDH5+fnG39/frF692uTm5pphw4aZWbNmGWOM+e1vf2tiYmLM+PHjzYEDB8z9999vbr31VmOMMTt27DChoaHmww8/NHv27DG9evUyM2bMKPWzLFu2zPTv37/Uax9//LGJiooy6enp5osvvjDR0dFltilPcXGxycvLM3l5eWbbtm0mPDzc7Ny50xhjzNatW01gYKD55JNPzO7du02bNm3MypUrXWMlmd69e5snn3zSbNmyxRQXX/os/OCDD0xMTIzZvHmz2blzp+nYsaNZsGBBufuv7PepOp/fPgk1H374oYmKijI//fSTMcaYzMxMk5iYWOmYBQsWmPj4eHPx4kVjjDEfffSRGTdunNv7JNQAgPd4Emr69OljJBlJ5uzZs26NOXv2rGtM3759PS23jP79+5tly5aVem3z5s1GksnMzDTGGON0Oo0kk5uba4y5FGqaNWtWpvaVK1eaFi1auD6fNmzYYGJiYowxl0JUSEiISUtLMwUFBebixYumpKTEGHMp1LRo0cJ1DD/99FPTvn17Y4wxkydPNv/2b//m2scXX3xhQkJCXPswpvxQ89BDD5lHH33U9fw3v/mNW6Hmsvz8fPOzn/3MLFq0yPXapEmTzL333ut6PmvWLHPPPfe4nksykydPLvNed911l3nxxRddz1etWmXi4+PL3a+3Qo1PLj9lZWWpd+/eCg0NlSR169ZNOTk5VY654447ZLPZJEm/+MUvKj2t5nQ65XA4Sj0AAL7To0cP19fvvvuuW2NWrVpV7viaEhUV5ZrJ+HLPj7niUsuECRMUERFRaszRo0d1+vRpRUVFKTIyUvfcc49Onz6toqIihYSE6P3331dqaqpiYmI0ePBgff/9966xffr0cc3PExgY6NrXkSNH1KFDB9d2HTp0UGFhoX788cdK6z9+/LhiY2NLjauOKVOmqEePHpo2bVqpn++jjz5SZGSkIiMj9dprr+nw4cOlxl3Z+3RZeT/DkSNHqlVPdfkk1DgcDsXFxbme22w2+fn5KS8vz+0x4eHhOnr0aIXbz5s3TxEREa7HlX/IAIDaN3nyZNfXr732ms6fP1/p9k6ns1RPy5Xjr1WjRo3K7eW8fGt5RRo3blzmtTZt2uiWW25RZmamMjMzlZWVpX/84x8KCAjQmTNnFBUVpS+++EInT55Us2bNSt2eXtH+2rZtWyr8HDhwQKGhoWratGml9TVr1kzHjh1zPb86fFTmrbfeUkZGRpmG7DZt2mjq1Kmlfr7ly5eX2qa841Lez9C2bVu36/GET0KNv7+/goKCSr0WHBysgoICt8dUtf3s2bNlt9tdj5pKhyEBfsr53SDl/G6QQgL8amQfAGAFPXr0UN++fSVJ+/bt0/jx4ysMNk6nU+PHj9e+ffskSYmJiV5dC6pjx4767LPPdPz4cX322WcqKSnx+L2GDh2qQ4cO6auvvpKfn5/WrFmjwYMHyxijH3/8UQMGDNCGDRvkcDjUqFEjt5plH374Ya1atUofffSR9u3bp5kzZ2ry5MmuqxUVGTlypFatWqUvv/xSf//73/WHP/zBrZ9h165deuqpp7R48WKVlJTo7NmzOnfunKRLZ6c+/vhjnThxQhcuXNDTTz+tp59+usr3nDx5shYuXKj09HT94x//0Ny5c8s0ZXubT9Z+io6OVnZ2dqnX8vPzK729Lzo6WqdPn3Z7+6CgoDLBqSbYbDaFBrKEFgC4Y+HCherXr5+Kior03nvvKSsrS48//rjGjRuniIgI2e12rVq1Sq+//ror0ISEhGjBggVereOZZ57R/fffr7i4OMXGxlbZAlGZyMhIrV27Vo8++qiys7PVuXNnrV27Vv7+/oqPj9fvf/97TZs2TSdOnFD37t21ZMmSKt/zlltu0fLly/XUU0/pzJkzuvfeezVv3rwqx40ePVpff/21Ro4cqSZNmmjkyJH67rvvqhy3du1a5efnq1+/fq7XEhMTtW3bNt16662aO3euxo8frxMnTmjAgAFu3V4/evRoHTt2TBMmTND58+c1ZcoUPfbYY1WOuxY2U975txq2adMmTZkyxXWgc3NzdeONN+rcuXPy8yv/bMfSpUu1evVqbdy4UdKlmR8nT56sb7/91q19OhwO11+Yqk4vAgAqV1RUpIMHDyouLq7Mmk1VWbduXamlEi4LCgqS0+ks9drlnpShQ4dec82ouyr7farO57dPLj/169dPdrtdK1askCTNnz9fAwcOlJ+fnxwOh4qLi8uMGTFihLZt26bNmzfrwoULevXVVzVo0KDaLh0AcI2GDh2qrVu3KjExsdTrVweaxMREpaenE2jgNp9cN/H391dqaqrGjh2r5ORklZSUKD09XdKlO6EWLlyoUaNGlRrTtGlT/f73v9egQYMUERGhxo0bu3UKDwBQ9yQkJGjbtm3KyspSamqqMjMzlZ+fr7CwMPXo0UOTJ0/2ag8NGgafXH667OjRo8rIyFDfvn0VExPj1pj9+/drz5496t+/f7UuI3H5CQC851ouPwFX89blJ592uLZu3VqtW7eu1piOHTuqY8eONVQRAKA6fPj/YliIt36PWNASAFBtAQEBklTp1BqAuy7/Hl3+vfIU9yIDAKrNz89PkZGROnXqlCQpNDS0yjlUgKsZY1RQUKBTp04pMjKywjug3UWoAQB4pEWLFpLkCjaApyIjI12/T9eCUAMA8IjNZlPLli3VrFmzcqfiANwREBBwzWdoLiPUAACuiZ+fn9c+lIBrQaMwAACwBEINAACwBEINAACwhAbTU3N5Yh+Hw+HjSgAAgLsuf267M0Ffgwk1+fn5kqTY2FgfVwIAAKorPz9fERERlW7j07WfatPFixd17NgxhYWFeX2CKIfDodjYWB05coR1pWoQx7l2cJxrB8e5dnCca0dNHmdjjPLz89WqVSs1alR510yDOVPTqFEjtWnTpkb3ER4ezl+aWsBxrh0c59rBca4dHOfaUVPHuaozNJfRKAwAACyBUAMAACyBUOMFQUFB+u1vf6ugoCBfl2JpHOfawXGuHRzn2sFxrh115Tg3mEZhAABgbZypAQAAlkCoAQAAlkCoAQAAlkCoAQAAlkCocUN2drYSEhIUFRWl5ORkt9afSE9P14033qimTZsqJSWlFqqs/zw5zqmpqWrZsqUCAgJ011136fjx47VQaf3myXG+rLi4WF27dtWWLVtqrkCLuJbjfN999+mxxx6rweqsw5Pj/Morr6h58+YKDw/X3XffrTNnztRCpfXfmTNnFBcXp9zcXLe298XnIKGmCk6nU8OHD1fPnj2VkZGhnJwcpaWlVTrm9OnTGjFihO6//35t375dq1at0ubNm2un4HrKk+O8bds2PfPMM1q5cqUOHjyooqIiPfnkk7VTcD3lyXG+0ssvv6zs7OyaK9AiruU4f/rpp9q0aZOef/75mi3SAjw5zlu3btXy5cu1detW7dq1S0VFRZo5c2btFFyP/fjjjxo2bJjbgcZnn4MGlfrwww9NVFSU+emnn4wxxmRmZprExMRKxyxYsMDEx8ebixcvGmOM+eijj8y4ceNqvNb6zJPjvHjxYvPHP/7R9Xzp0qXmhhtuqNE66ztPjvNl3377rYmMjDTt27c3mzdvrsEq6z9Pj3NBQYHp0KGDWbJkSU2XaAmeHOdXXnnFJCcnu56vXLnS9OnTp0brtIIBAwaYhQsXGknm4MGDVW7vq89BztRUISsrS71791ZoaKgkqVu3bsrJyalyzB133OFaOPMXv/iFdu3aVeO11meeHOdJkyZp9OjRruf79u1Tx44da7TO+s6T43zZlClTNGvWLLVr164mS7QET4/z888/r8LCQvn7+2vTpk3VumTVEHlynLt06aI//elPOnDggE6dOqUlS5bozjvvrI1y67XU1FTNmDHD7e199TlIqKmCw+FQXFyc67nNZpOfn5/y8vLcHhMeHq6jR4/WaJ31nSfH+UpnzpzR22+/renTp9dUiZbg6XFetmyZ7HY7p+nd5MlxPnz4sFJSUtSxY0cdPnxYycnJGj16NMGmEp4c58GDB6tTp07q2LGjmjdvrp9++kmzZs2qjXLrtQ4dOlRre199DhJqquDv719m2ufg4GAVFBS4Paaq7eHZcb7S9OnT1bdvXw0dOrQmyrMMT47z6dOnNXv2bC1ZskT+/v41XaIleHKc09LS1Lx5c23cuFFz5szRli1blJ6ero0bN9Z0ufWWJ8f5vffe06FDh7R3716dOXNGXbp00QMPPFDTpTY4vvoc5F+oKkRHR5dpjMzPz1dgYGClY06fPu329vDsOF+2dOlSbd26VZmZmTVUnXV4cpyTkpI0adIk9ejRo4arsw5PjvMPP/ygAQMGuD4IwsLC1KlTJx08eLBGa63PPDnOq1ev1rRp0xQfHy9JWrhwoSIiInT27FlFRkbWZLkNiq8+BzlTU4WEhAT97W9/cz3Pzc2V0+lUdHS022MyMzPVunXrGq2zvvPkOEvSV199paSkJK1Zs0bNmzev6TLrPU+O87vvvqs33nhDkZGRioyM1LZt2zRs2DDNnz+/Nkqulzw5zrGxsSosLHQ9v3jxon744Qd6mCrhyXG+cOGCTp486Xp+eRqIkpKSmiu0AfLZ52CNtyLXc8XFxSYmJsYsX77cGGPMlClTzLBhw4wxxtjtdnP+/PkyY06fPm2Cg4PNpk2bTHFxsRk6dKh59NFHa7Xu+saT43zixAnTrFkz88ILL5j8/HzXAxXz5DgfPHiw1KNXr15m9erVJi8vrzZLr1c8Oc579+41jRs3Nh988IE5cuSI+c1vfmOaNGliHA5HrdZen3hynOfNm2diYmLMW2+9ZdLS0kyPHj24+6kadNXdT3Xtc5BQ44YPP/zQhISEmGbNmpkmTZqY7OxsY4wx7dq1Mx9++GG5Y958800TEBBgmjZtatq1a2dOnDhRixXXT9U9zgsWLDCSyjxQOU9+n6/Uv39/bul2gyfH+ZNPPjE9evQwwcHBpnPnzmbbtm21WHH9VN3jXFhYaB577DHTqlUrExgYaPr372/2799fy1XXX1eHmrr2OWj7vyJRhaNHjyojI0N9+/ZVTEyMW2P279+vPXv2qH///goPD6/hCq3Bk+OM6uM41w6Oc+3gONddtf05SKgBAACWQKMwAACwBEINAACwBEINAACwBEINAACwBEINAACwBEINgDotLS1NNputzGPx4sWur4OCgtSrVy9lZGRIkubOnev6XuPGjXXrrbe6vgfAugg1AOq8Ll26KC8vr9TjytcPHDigIUOGaOTIka5F84YMGaK8vDzt3r1bN954o0aPHu3LHwFALSDUAKjz/Pz8XGtPXX74+/u7Xm/Tpo1++9vfKj8/X1lZWZKkgIAARUZGqkOHDnr22Wd15MgR/fjjjz7+SQDUJEINAMvw8/NTcXFxqdeMMXrvvfcUFRXFKsyAxfn7ugAAqMru3btLBZLPP/+81PcvXryolStXSpK6d++uTZs2ad26dYqMjFRRUZGaNm2qd955R/7+/JMHWBl/wwHUefHx8frLX/7iet6qVSvt3LnTFXYKCgrUpEkTrVq1ShEREZKk22+/XampqZo6dari4uI0ZMgQX5UPoJYQagDUeYGBgWrfvn2Z1y+HneDgYLVo0aLU90JDQ9W+fXs9+uijGjt2rF599VU1bty4lioG4Av01ACoty6HnasDzZWGDBmi6OhorV69uhYrA+ALhBoAltaoUSNNnjxZqampvi4FQA0j1ACwvEmTJikrK0uZmZm+LgVADbIZY4yviwAAALhWnKkBAACWQKgBAACWQKgBAACWQKgBAACWQKgBAACWQKgBAACWQKgBAACWQKgBAACWQKgBAACWQKgBAACW8P8A77OyexdTnogAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "fpr, tpr, thresholds = roc_curve(y_test, lr.decision_function(X_test))\n",
    "plt.plot(fpr, tpr, label=\"ROC Curve\")\n",
    "plt.xlabel(\"FPR\")\n",
    "plt.ylabel(\"TPR (recall)\")\n",
    "# 找到最接近于0的阈值\n",
    "close_zero = np.argmin(np.abs(thresholds))\n",
    "plt.plot(fpr[close_zero], tpr[close_zero], 'o', markersize=10, label=\"threshold zero\", fillstyle=\"none\", c='k', mew=2)\n",
    "plt.legend(loc=4)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2e464caf",
   "metadata": {},
   "source": [
    "## 【任务总结】    \n",
    "做数据分析的目的，就是运用数据、结合业务来得到某些我们想要知道的结果。    \n",
    "分析的第一步就是建模，搭建一个预测模型或者其他模型。    \n",
    "从模型得到预测结果之后，还要要分析这个模型是不是足够可靠，这样就需要评估这个模型。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3496beb1",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}