idc-model-breast-cancer-analyzer-web-app.ipynb

{"cells":[{"metadata":{"_uuid":"343521fceb9f6402d1ee2e0d63c4b270b4b7cf34"},"cell_type":"markdown","source":"### Glossary of Terms\n\nI didn't know much about the medical aspects of breast cancer before starting this project. My guess is that you're also not an expert. Therefore, I'd like to start by explaining some terms to help us understand this topic.\n\n**Histopathology**<br>\nThis inolves examining glass tissue slides under a microscope to see if disease is present. In this case that would be examining tissue samples from lymp nodes in order to detect breast cancer.\n\n**Whole Slide Image (WSI)**<br> \nA digitized high resolution image of a glass slide taken with a scanner. The images can be several gigabytes in size. Therefore, to allow them to be used in machine learning these digital images are cut up into Patches.\n\n**Patch** <br>\nA patch is a small, usually rectangular, piece of an image. For example, a 50x50 patch is a square patch containing 2500 pixels, taken from a larger image of size say 1000x1000 pixels.\n\n**Lymph Node**<br>\nThis is a small bean shaped structure that's part of the body's immune system. Lymph nodes filter substances that travel through the lymphatic fluid. They contain lymphocytes (white blood cells) that help the body fight infection and disease.\n\n\n**Sentinel Lymph Node**<br>\nA blue dye and/or radioactive tracer is injected near the tumor. The first lymph node reached by this injected substance is called the sentinel lymph node. The images that we will be using are all of tissue samples taken from sentinel lymph nodes.\n\n**Invasive Ductal Carcinoma**<br> \nThis is the most common subtype of all breast cancers. Almost 80% of diagnosed breast cancers are of this subtype.\n\n**Metastasis**<br>\nThe spread of cancer cells to new areas of the body, often via the lymph system or bloodstream.\n\n**Metastatic Cancer**<br> \nA metastatic cancer or metastatic tumor is one that has spread from the site where it started into different area/s of the body.\n\n***\n\n"},{"metadata":{"_uuid":"aebaeac0cf0ab1d64b8208b0f986cf7da584fced"},"cell_type":"markdown","source":"### Reference Papers\n\nThese are the papers that are associated with the two datasets that we will be using:\n\n> Automatic detection of invasive ductal carcinoma in whole slide images with Convolutional Neural Networks<br>\n> [Link](https://www.researchgate.net/publication/263052166_Automatic_detection_of_invasive_ductal_carcinoma_in_whole_slide_images_with_Convolutional_Neural_Networks)\n> \n> \n> 1399 H&E-stained sentinel lymph node sections of breast cancer patients: the CAMELYON dataset<br>\n> [Link](https://academic.oup.com/gigascience/article/7/6/giy065/5026175)\n\n\n***"},{"metadata":{"_uuid":"cde03c3446dae4350be3ee13a161fc79e3f0914c"},"cell_type":"markdown","source":"### What problem needs to be solved?\n\nIn order to detect cancer,  the sentinel lymph node needs to be surgically removed, hispathologically processed and then examined by a pathologist The pathologist needs to visually scan large regions where there is no cancer in order to ultimately find malignant areas. The process is tedious and time consuming and small malignant areas could be missed.\n\n***\n\n"},{"metadata":{"_uuid":"f567ba3b2395bee330bd48cab503a5114df333e6"},"cell_type":"markdown","source":"### What is the potential of computational pathology?\n\nThis is what two experts have to say:\n\nThis is a 1:21 video featuring Prof Jeroen van der Laak, a co-author of the CAMELYON paper.<br>\nhttps://www.youtube.com/watch?v=MCRtJDDrdcA\n\n<iframe width=\"560\" height=\"315\" src=\"https://www.youtube.com/embed/MCRtJDDrdcA?rel=0\" frameborder=\"0\" allow=\"accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture\" allowfullscreen></iframe>\n\nThis 1:46 video features Ian Ellis, Professor of Cancer Pathology at Nottingham University<br>\nhttps://www.youtube.com/watch?v=gcFnyuc7Bew\n\n<iframe width=\"560\" height=\"315\" src=\"https://www.youtube.com/embed/gcFnyuc7Bew?rel=0\" frameborder=\"0\" allow=\"accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture\" allowfullscreen></iframe>\n\n***"},{"metadata":{"trusted":true,"_uuid":"748b1ca0e007abe9e7cd378bb7aac9b9cd09a7b8"},"cell_type":"code","source":"","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"a03659c71e42d9babf0ff71b6f6ee769f3632216"},"cell_type":"markdown","source":"\n### Project Objective\n\nThe goal of this project is to create a web based tool that can batch analyze histopathology image patches and predict if breast cancer is present. The web app will be able to detect two forms of breast cancer:<br>\n\n- Invasive Ductal Carcinoma (IDC)\n- Metastatic Cancer\n\nWe will create two CNN models, an IDC_model and a Metastatic_model. These models will be loaded into a Tensorflowjs web app. The user will be able to select which cancer he or she wants to detect, submit many images at the same time, and get an instant prediction indicating whether or not breast cancer is present in those images.\n\nThis app will speed up a pathologist's workflow and provide diagnosis support.\n\nTo ensure privacy and patient confidentiality, all images will be processed locally and never uploaded to an external server.\n\nWe will create the models in two kernels. In part 1 we will build the IDC_model and in part 2 the Metastatic_model. In part 2 I will also cover some lessons learned while building the app.\n\n***\n\nAll the html, css, and javascript code used to build the web app is available on Github. The technology that enables this app to work is new. Therefore, I recommend using the latest version of the Chrome browser. When using Safari for example, you may see a message indicating that the model is loading but the app may actually be frozen.\n\nLive web app:<br>\nhttp://histo.test.woza.work/<br>\nGithub:<br>\nhttps://github.com/vbookshelf/Cancer-Tissue-Analyzer"},{"metadata":{"trusted":true,"_uuid":"2553824f255919ff018ab28c0fd1c34662a1a14a"},"cell_type":"code","source":"# End of Intro\n#=========================================================================================#\n# Start of Part 1","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"9a87801b30172e0b5819a1dbced38e881c7264bd"},"cell_type":"markdown","source":"### Part 1 - IDC Model"},{"metadata":{"_uuid":"fb2bd8f0254cc961a61af1233e3ab167a8594fa8"},"cell_type":"markdown","source":"In this section we will create the IDC_model. This model will predict whether or not Invasive Ductal Carcinoma is present. \n\n**Dataset**\n\nWe will use the 'Breast Histopathology Images' dataset. This dataset consists of 277,524 image patches of size 50x50 (198,738 IDC negative and 78,786 IDC positive). The images are in png format.\n\n**Results**\n\nOur cnn model will achieve an accuracy and F1 score that is greater than 0.85. This is based on a classification threshold of 0.5. It should be noted that the creators of the [paper](https://www.researchgate.net/publication/263052166_Automatic_detection_of_invasive_ductal_carcinoma_in_whole_slide_images_with_Convolutional_Neural_Networks) realting to this dataset used a threshold of 0.29.\n"},{"metadata":{"_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","trusted":true},"cell_type":"code","source":"from numpy.random import seed\nseed(101)\nfrom tensorflow import set_random_seed\nset_random_seed(101)\n\nimport pandas as pd\nimport numpy as np\n\nimport tensorflow\n\nfrom tensorflow.keras.models import Sequential\nfrom tensorflow.keras.layers import Dense, Dropout, Conv2D, MaxPooling2D, Flatten\nfrom tensorflow.keras.optimizers import Adam\nfrom tensorflow.keras.metrics import categorical_crossentropy\nfrom tensorflow.keras.preprocessing.image import ImageDataGenerator\nfrom tensorflow.keras.models import Model\nfrom tensorflow.keras.callbacks import EarlyStopping, ReduceLROnPlateau, ModelCheckpoint\n\nimport os\nimport cv2\n\nimport imageio\nimport skimage\nimport skimage.io\nimport skimage.transform\n\nfrom sklearn.utils import shuffle\nfrom sklearn.metrics import confusion_matrix\nfrom sklearn.model_selection import train_test_split\nimport itertools\nimport shutil\nimport matplotlib.pyplot as plt\n%matplotlib inline\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"d52f02b7103dac5130ccc5cfd2222a9f8c7a1b6b"},"cell_type":"code","source":"# Number of samples we want in each class.Total images used = SAMPLE_SIZE X 2\n# The minority class is class 1 with 78786 samples.\n\nSAMPLE_SIZE = 78786\n\nIMAGE_SIZE = 50","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"d52d2ffda062945825b2458824cae26019059f57"},"cell_type":"markdown","source":"### What files are available?\n\nThe images are grouped into 279 folders by patient_id. Each patient folder has two sub-folders that groups together images with the same class --> 0 or 1. There are a lot of folders to work with."},{"metadata":{"trusted":true,"_uuid":"fe3a2b7c84dfa44b46d6ec069c09c5524657e802","_kg_hide-output":true},"cell_type":"code","source":"os.listdir('../input/IDC_regular_ps50_idx5')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"7db37c5f6d92953e21eaa37ce246df05dbeed0c1"},"cell_type":"code","source":"# Check the number of patient folders.\n\npatients = os.listdir('../input/IDC_regular_ps50_idx5')\n\nlen(patients)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"fd97c892d31c99910e1a069cfd6e4b838e270995"},"cell_type":"markdown","source":"### Copy all images into one directory\nThis will make it easier to work with this data."},{"metadata":{"trusted":true,"_uuid":"a01865c676af839aa5037278d74222c738e30179"},"cell_type":"code","source":"# Create a new directory to store all available images\nall_images_dir = 'all_images_dir'\nos.mkdir(all_images_dir)\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"95b68d456cacf8f212832809b8a18d9a0bc7101c"},"cell_type":"code","source":"# check that the new diectory has been created\n!ls","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"d82447a2f4d1d0e862ffcb3ec48e764c159dbc55"},"cell_type":"code","source":"# This code copies all images from their seperate folders into the same \n# folder called all_images_dir.\n\n# Create a list with all the patient id numbers.\n# Each patient id folder has 2 sub folders --> folder 0 and folder 1\n\n# Example:\n    # '10285'\n        # '0'\n        # '1'\n\n# create a list of all patient id's\npatient_list = os.listdir('../input/IDC_regular_ps50_idx5')\n\nfor patient in patient_list:\n    \n    path_0 = '../input/IDC_regular_ps50_idx5/' + str(patient) + '/0'\n    path_1 = '../input/IDC_regular_ps50_idx5/' + str(patient) + '/1'\n\n\n    # create a list of all files in folder 0\n    file_list_0 = os.listdir(path_0)\n    # create a list of list all file in folder 1\n    file_list_1 = os.listdir(path_1)\n\n    # move the 0 images to all_images_dir\n    for fname in file_list_0:\n\n        # source path to image\n        src = os.path.join(path_0, fname)\n        # destination path to image\n        dst = os.path.join(all_images_dir, fname)\n        # copy the image from the source to the destination\n        shutil.copyfile(src, dst)\n\n\n    # move the 1 images to all_images_dir\n    for fname in file_list_1:\n\n        # source path to image\n        src = os.path.join(path_1, fname)\n        # destination path to image\n        dst = os.path.join(all_images_dir, fname)\n        # copy the image from the source to the destination\n        shutil.copyfile(src, dst)\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"4f25b3700f97cbd5f55e77ffdcdde2dc813c95f5"},"cell_type":"code","source":"# check how many images are in all_images_dir\n# should be 277,524\n\n# size: 2.5GB\n\nlen(os.listdir('all_images_dir'))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"1b14eb7d4263990c0a9dcba0aa01ef56224accb0"},"cell_type":"markdown","source":"### Create a dataframe containing all the information"},{"metadata":{"trusted":true,"_uuid":"2450c640a1e91bb7ddc8541f0f15f8c328a1bcf2"},"cell_type":"code","source":"image_list = os.listdir('all_images_dir')\n\ndf_data = pd.DataFrame(image_list, columns=['image_id'])\n\ndf_data.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"4d20cd5b2b51a96e5b3e0a3252c826b93cbd25ef"},"cell_type":"code","source":"# Define Helper Functions\n\n# Each file name has this format:\n# '14211_idx5_x2401_y1301_class1.png'\n\ndef extract_patient_id(x):\n    # split into a list\n    a = x.split('_')\n    # the id is the first index in the list\n    patient_id = a[0]\n    \n    return patient_id\n\ndef extract_target(x):\n    # split into a list\n    a = x.split('_')\n    # the target is part of the string in index 4\n    b = a[4]\n    # the ytarget i.e. 1 or 2 is the 5th index of the string --> class1\n    target = b[5]\n    \n    return target\n\n# extract the patient id\n\n# create a new column called 'patient_id'\ndf_data['patient_id'] = df_data['image_id'].apply(extract_patient_id)\n# create a new column called 'target'\ndf_data['target'] = df_data['image_id'].apply(extract_target)\n\ndf_data.head(10)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"297e38d8fccb01cebc5d31dfe50d8f9de86a8351"},"cell_type":"code","source":"df_data.shape","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"6dda72185d30b21abb8989038124c488f3a6dc90"},"cell_type":"markdown","source":"### Display a random sample of train images by class"},{"metadata":{"_kg_hide-input":true,"trusted":true,"_uuid":"a2def6b0762c4aba4b474ac386bb6cfaf3649311"},"cell_type":"code","source":"# source: https://www.kaggle.com/gpreda/honey-bee-subspecies-classification\n\ndef draw_category_images(col_name,figure_cols, df, IMAGE_PATH):\n    \n    \"\"\"\n    Give a column in a dataframe,\n    this function takes a sample of each class and displays that\n    sample on one row. The sample size is the same as figure_cols which\n    is the number of columns in the figure.\n    Because this function takes a random sample, each time the function is run it\n    displays different images.\n    \"\"\"\n    \n\n    categories = (df.groupby([col_name])[col_name].nunique()).index\n    f, ax = plt.subplots(nrows=len(categories),ncols=figure_cols, \n                         figsize=(4*figure_cols,4*len(categories))) # adjust size here\n    # draw a number of images for each location\n    for i, cat in enumerate(categories):\n        sample = df[df[col_name]==cat].sample(figure_cols) # figure_cols is also the sample size\n        for j in range(0,figure_cols):\n            file=IMAGE_PATH + sample.iloc[j]['image_id']\n            im=cv2.imread(file)\n            ax[i, j].imshow(im, resample=True, cmap='gray')\n            ax[i, j].set_title(cat, fontsize=16)  \n    plt.tight_layout()\n    plt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"2c71f69b0edbb1999851ed12f9e8bdae7ef939a3"},"cell_type":"code","source":"IMAGE_PATH = 'all_images_dir/'\n\ndraw_category_images('target',4, df_data, IMAGE_PATH)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"a16506d07e5f102b1fdb172fa0ba5f5202a5d533"},"cell_type":"markdown","source":"### Balance the class distribution"},{"metadata":{"trusted":true,"_uuid":"f15793d9606cde40aabf53dfdc1816824f72b592"},"cell_type":"code","source":"# What is the class distribution?\n\ndf_data['target'].value_counts()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"dbf91ea28135821c4aab3acb628ac8292df69d94"},"cell_type":"code","source":"# take a sample of the majority class 0 (total = 198738)\ndf_0 = df_data[df_data['target'] == '0'].sample(SAMPLE_SIZE, random_state=101)\n# take a sample of class 1 (total = 78786)\ndf_1 = df_data[df_data['target'] == '1'].sample(SAMPLE_SIZE, random_state=101)\n\n# concat the two dataframes\ndf_data = pd.concat([df_0, df_1], axis=0).reset_index(drop=True)\n\n# Check the new class distribution\ndf_data['target'].value_counts()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"a6e14c3bdb999ab0c3acda58c689f73e90476381"},"cell_type":"markdown","source":"### Create the train and  val sets\n"},{"metadata":{"trusted":true,"_uuid":"08fe5aec1a86b64e914edc75e9b0c785dd9e00b4"},"cell_type":"code","source":"# train_test_split\n\n# stratify=y creates a balanced validation set.\ny = df_data['target']\n\ndf_train, df_val = train_test_split(df_data, test_size=0.10, random_state=101, stratify=y)\n\nprint(df_train.shape)\nprint(df_val.shape)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"c9fca5b1044b35b0ba6e4103e135dc4e4f69bf1f"},"cell_type":"code","source":"df_train['target'].value_counts()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"6bb24212bd0a4637c6a02c0187ed474c30745990"},"cell_type":"code","source":"df_val['target'].value_counts()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"5c0eb1110f124d841159a4312f52f1e9466c2b14"},"cell_type":"markdown","source":"### Create a Directory Structure"},{"metadata":{"trusted":true,"_uuid":"c78886470314cdc844487570f672a9afae61587d"},"cell_type":"code","source":"# Create a new directory\nbase_dir = 'base_dir'\nos.mkdir(base_dir)\n\n\n#[CREATE FOLDERS INSIDE THE BASE DIRECTORY]\n\n# now we create 2 folders inside 'base_dir':\n\n# train_dir\n    # a_no_idc\n    # b_has_idc\n\n# val_dir\n    # a_no_idc\n    # b_has_idc\n\n\n\n# create a path to 'base_dir' to which we will join the names of the new folders\n# train_dir\ntrain_dir = os.path.join(base_dir, 'train_dir')\nos.mkdir(train_dir)\n\n# val_dir\nval_dir = os.path.join(base_dir, 'val_dir')\nos.mkdir(val_dir)\n\n\n# [CREATE FOLDERS INSIDE THE TRAIN AND VALIDATION FOLDERS]\n# Inside each folder we create seperate folders for each class\n\n# create new folders inside train_dir\na_no_idc = os.path.join(train_dir, 'a_no_idc')\nos.mkdir(a_no_idc)\nb_has_idc = os.path.join(train_dir, 'b_has_idc')\nos.mkdir(b_has_idc)\n\n\n# create new folders inside val_dir\na_no_idc = os.path.join(val_dir, 'a_no_idc')\nos.mkdir(a_no_idc)\nb_has_idc = os.path.join(val_dir, 'b_has_idc')\nos.mkdir(b_has_idc)\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"7250bdad70673cf1a4ce5f0a53911ff03f029527"},"cell_type":"code","source":"# check that the folders have been created\nos.listdir('base_dir/train_dir')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"868757fc1d72fc1c54adfac6185e0354f7a4e50c"},"cell_type":"markdown","source":"### Transfer the images into the folders¶"},{"metadata":{"trusted":true,"_uuid":"7f8edcb3172f2583c165b1ab18dedaba8261b9ea"},"cell_type":"code","source":"# Set the id as the index in df_data\ndf_data.set_index('image_id', inplace=True)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"de23e2288f785db71eacbb55b8a284e5902ec04e"},"cell_type":"code","source":"# Get a list of train and val images\ntrain_list = list(df_train['image_id'])\nval_list = list(df_val['image_id'])\n\n\n\n# Transfer the train images\n\nfor image in train_list:\n    \n    # the id in the csv file does not have the .tif extension therefore we add it here\n    fname = image\n    # get the label for a certain image\n    target = df_data.loc[image,'target']\n    \n    # these must match the folder names\n    if target == '0':\n        label = 'a_no_idc'\n    if target == '1':\n        label = 'b_has_idc'\n    \n    # source path to image\n    src = os.path.join(all_images_dir, fname)\n    # destination path to image\n    dst = os.path.join(train_dir, label, fname)\n    # move the image from the source to the destination\n    shutil.move(src, dst)\n    \n\n# Transfer the val images\n\nfor image in val_list:\n    \n    # the id in the csv file does not have the .tif extension therefore we add it here\n    fname = image\n    # get the label for a certain image\n    target = df_data.loc[image,'target']\n    \n    # these must match the folder names\n    if target == '0':\n        label = 'a_no_idc'\n    if target == '1':\n        label = 'b_has_idc'\n    \n\n    # source path to image\n    src = os.path.join(all_images_dir, fname)\n    # destination path to image\n    dst = os.path.join(val_dir, label, fname)\n    # move the image from the source to the destination\n    shutil.move(src, dst)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"1e895302c8c74f7e5e0d9b52e8d564144f082186"},"cell_type":"code","source":"# check how many train images we have in each folder\n\nprint(len(os.listdir('base_dir/train_dir/a_no_idc')))\nprint(len(os.listdir('base_dir/train_dir/b_has_idc')))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"b1e7e50cef260226eed29bb11742c08bbac1287c"},"cell_type":"code","source":"# check how many val images we have in each folder\n\nprint(len(os.listdir('base_dir/val_dir/a_no_idc')))\nprint(len(os.listdir('base_dir/val_dir/b_has_idc')))\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"722fee40665f455fa038ae4af7412dd1fce8849f"},"cell_type":"code","source":"# End of Data Preparation\n### ================================================================================== ###\n# Start of Model Building","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"3c3b5228e6ada5ee1edecc63407664eacf5731f1"},"cell_type":"markdown","source":"### Set Up the Generators"},{"metadata":{"trusted":true,"_uuid":"90c1df028e4c15202d6743da05c0711e97bdbcec"},"cell_type":"code","source":"train_path = 'base_dir/train_dir'\nvalid_path = 'base_dir/val_dir'\n\n\nnum_train_samples = len(df_train)\nnum_val_samples = len(df_val)\ntrain_batch_size = 10\nval_batch_size = 10\n\n\ntrain_steps = np.ceil(num_train_samples / train_batch_size)\nval_steps = np.ceil(num_val_samples / val_batch_size)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"df818b545e53289b585189cf6d1d82ace86c8181"},"cell_type":"code","source":"datagen = ImageDataGenerator(rescale=1.0/255)\n\ntrain_gen = datagen.flow_from_directory(train_path,\n                                        target_size=(IMAGE_SIZE,IMAGE_SIZE),\n                                        batch_size=train_batch_size,\n                                        class_mode='categorical')\n\nval_gen = datagen.flow_from_directory(valid_path,\n                                        target_size=(IMAGE_SIZE,IMAGE_SIZE),\n                                        batch_size=val_batch_size,\n                                        class_mode='categorical')\n\n# Note: shuffle=False causes the test dataset to not be shuffled\ntest_gen = datagen.flow_from_directory(valid_path,\n                                        target_size=(IMAGE_SIZE,IMAGE_SIZE),\n                                        batch_size=1,\n                                        class_mode='categorical',\n                                        shuffle=False)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"404bcc4854384fa18209eb5ee105446d9b481d75"},"cell_type":"markdown","source":"### Create the Model Architecture"},{"metadata":{"trusted":true,"_uuid":"60eae5dac2d5d90dd5a03d0871f3d77650bf8b09","_kg_hide-output":true},"cell_type":"code","source":"# Source: https://www.kaggle.com/fmarazzi/baseline-keras-cnn-roc-fast-5min-0-8253-lb\n\nkernel_size = (3,3)\npool_size= (2,2)\nfirst_filters = 32\nsecond_filters = 64\nthird_filters = 128\n\ndropout_conv = 0.3\ndropout_dense = 0.3\n\n\nmodel = Sequential()\nmodel.add(Conv2D(first_filters, kernel_size, activation = 'relu', \n                 input_shape = (IMAGE_SIZE, IMAGE_SIZE, 3)))\nmodel.add(Conv2D(first_filters, kernel_size, activation = 'relu'))\nmodel.add(Conv2D(first_filters, kernel_size, activation = 'relu'))\nmodel.add(MaxPooling2D(pool_size = pool_size)) \nmodel.add(Dropout(dropout_conv))\n\nmodel.add(Conv2D(second_filters, kernel_size, activation ='relu'))\nmodel.add(Conv2D(second_filters, kernel_size, activation ='relu'))\nmodel.add(Conv2D(second_filters, kernel_size, activation ='relu'))\nmodel.add(MaxPooling2D(pool_size = pool_size))\nmodel.add(Dropout(dropout_conv))\n\nmodel.add(Conv2D(third_filters, kernel_size, activation ='relu'))\nmodel.add(Conv2D(third_filters, kernel_size, activation ='relu'))\nmodel.add(Conv2D(third_filters, kernel_size, activation ='relu'))\nmodel.add(MaxPooling2D(pool_size = pool_size))\nmodel.add(Dropout(dropout_conv))\n\nmodel.add(Flatten())\nmodel.add(Dense(256, activation = \"relu\"))\nmodel.add(Dropout(dropout_dense))\nmodel.add(Dense(2, activation = \"softmax\"))\n\nmodel.summary()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"ce211aa4b707778b73ac60fe42f13b9eb685a7f5"},"cell_type":"markdown","source":"### Train the Model"},{"metadata":{"trusted":true,"_uuid":"02fb3d732dff9af1596b8f56c5ff3dc0bcd101cd"},"cell_type":"code","source":"model.compile(Adam(lr=0.0001), loss='binary_crossentropy', \n              metrics=['accuracy'])\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"a27114bb8b6838a675b77abb0ee65a9f72c6e162"},"cell_type":"code","source":"filepath = \"model.h5\"\ncheckpoint = ModelCheckpoint(filepath, monitor='val_acc', verbose=1, \n                             save_best_only=True, mode='max')\n\nreduce_lr = ReduceLROnPlateau(monitor='val_acc', factor=0.5, patience=3, \n                                   verbose=1, mode='max', min_lr=0.00001)\n                              \n                              \ncallbacks_list = [checkpoint, reduce_lr]\n\nhistory = model.fit_generator(train_gen, steps_per_epoch=train_steps, \n                    validation_data=val_gen,\n                    validation_steps=val_steps,\n                    epochs=60, verbose=1,\n                   callbacks=callbacks_list)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"78c171dc5909de81001c2374cbfa30f1f9158147"},"cell_type":"markdown","source":"### Evaluate the model using the val set"},{"metadata":{"trusted":true,"_uuid":"219b0e9391c490824191c99b295ae47f7bee564c"},"cell_type":"code","source":"# get the metric names so we can use evaulate_generator\nmodel.metrics_names","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"132099a89f8459035bfd45d7b27ac4a0da47b0ae"},"cell_type":"code","source":"# Here the best epoch will be used.\n\nmodel.load_weights('model.h5')\n\nval_loss, val_acc = \\\nmodel.evaluate_generator(test_gen, \n                        steps=len(df_val))\n\nprint('val_loss:', val_loss)\nprint('val_acc:', val_acc)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"00e74d6b276e8b1dbc5d2b9abe4efb5bed1eea61"},"cell_type":"markdown","source":"### Plot the Training Curves"},{"metadata":{"_kg_hide-input":true,"trusted":true,"_uuid":"f4aa7183e51d9d4598a590e94c7882291eebb514"},"cell_type":"code","source":"# display the loss and accuracy curves\n\nimport matplotlib.pyplot as plt\n\nacc = history.history['acc']\nval_acc = history.history['val_acc']\nloss = history.history['loss']\nval_loss = history.history['val_loss']\n\nepochs = range(1, len(acc) + 1)\n\nplt.plot(epochs, loss, 'bo', label='Training loss')\nplt.plot(epochs, val_loss, 'b', label='Validation loss')\nplt.title('Training and validation loss')\nplt.legend()\nplt.figure()\n\nplt.plot(epochs, acc, 'bo', label='Training acc')\nplt.plot(epochs, val_acc, 'b', label='Validation acc')\nplt.title('Training and validation accuracy')\nplt.legend()\nplt.figure()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"ca9a7b03dc412c38bac81aba9d8bb0b6cd80f61b"},"cell_type":"markdown","source":"### Make a prediction on the val set\nWe need these predictions to calculate the AUC score, print the Confusion Matrix and calculate the F1 score."},{"metadata":{"trusted":true,"_uuid":"24f90dc4b0e618267800b43f0a9e5f736469865f"},"cell_type":"code","source":"# make a prediction\npredictions = model.predict_generator(test_gen, steps=len(df_val), verbose=1)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"55b4269bf393e9699320a0ccb07de3765a9e0548"},"cell_type":"code","source":"predictions.shape","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"15972146171e8a5b5588b60e83bece03c8ae217d"},"cell_type":"code","source":"# This is how to check what index keras has internally assigned to each class. \ntest_gen.class_indices","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"9b7097ace33c1b81a33526aa581b1a2c16ba9670"},"cell_type":"code","source":"# Put the predictions into a dataframe.\n# The columns need to be oredered to match the output of the previous cell\n\ndf_preds = pd.DataFrame(predictions, columns=['no_idc', 'has_idc'])\n\ndf_preds.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"d73625503d727c2e4b18ed7e4d036f65183b391f"},"cell_type":"code","source":"# Get the true labels\ny_true = test_gen.classes\n\n# Get the predicted labels as probabilities\ny_pred = df_preds['has_idc']","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"1bbd432a630e69684a34aa181786a6d8d4f811bd"},"cell_type":"markdown","source":"### What is the AUC Score?"},{"metadata":{"trusted":true,"_uuid":"24b04893c841ed8992560c855756ada0ecb87b7b"},"cell_type":"code","source":"from sklearn.metrics import roc_auc_score\n\nroc_auc_score(y_true, y_pred)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"28ea2d21008495b0fe0d70887a047042ac06b53d"},"cell_type":"markdown","source":"### Create a Confusion Matrix"},{"metadata":{"_kg_hide-input":true,"trusted":true,"_uuid":"af3db45056935a3431d50e836a3fa452acf51df4"},"cell_type":"code","source":"# Source: Scikit Learn website\n# http://scikit-learn.org/stable/auto_examples/\n# model_selection/plot_confusion_matrix.html#sphx-glr-auto-examples-model-\n# selection-plot-confusion-matrix-py\n\n\ndef plot_confusion_matrix(cm, classes,\n                          normalize=False,\n                          title='Confusion matrix',\n                          cmap=plt.cm.Blues):\n    \"\"\"\n    This function prints and plots the confusion matrix.\n    Normalization can be applied by setting `normalize=True`.\n    \"\"\"\n    if normalize:\n        cm = cm.astype('float') / cm.sum(axis=1)[:, np.newaxis]\n        print(\"Normalized confusion matrix\")\n    else:\n        print('Confusion matrix, without normalization')\n\n    print(cm)\n\n    plt.imshow(cm, interpolation='nearest', cmap=cmap)\n    plt.title(title)\n    plt.colorbar()\n    tick_marks = np.arange(len(classes))\n    plt.xticks(tick_marks, classes, rotation=45)\n    plt.yticks(tick_marks, classes)\n\n    fmt = '.2f' if normalize else 'd'\n    thresh = cm.max() / 2.\n    for i, j in itertools.product(range(cm.shape[0]), range(cm.shape[1])):\n        plt.text(j, i, format(cm[i, j], fmt),\n                 horizontalalignment=\"center\",\n                 color=\"white\" if cm[i, j] > thresh else \"black\")\n\n    plt.ylabel('True label')\n    plt.xlabel('Predicted label')\n    plt.tight_layout()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"643d17f82b0ac82b18e54369180b2e89e42c760c"},"cell_type":"code","source":"# Get the labels of the test images.\n\ntest_labels = test_gen.classes","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"1ec35b798d8cd23eb32b851ebe9085b7310462e3"},"cell_type":"code","source":"test_labels.shape","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"375f94a739f7f87eba5785676ecf9a6364bf5390"},"cell_type":"code","source":"# argmax returns the index of the max value in a row\ncm = confusion_matrix(test_labels, predictions.argmax(axis=1))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"a05cf1c00f0e17e62a05a752912a4a6cea992e1c"},"cell_type":"code","source":"# Print the label associated with each class\ntest_gen.class_indices","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"4695cb8ee3e8b7397050af6ad98e0993a0e62e66"},"cell_type":"code","source":"# Define the labels of the class indices. These need to match the \n# order shown above.\ncm_plot_labels = ['no_idc', 'has_idc']\n\nplot_confusion_matrix(cm, cm_plot_labels, title='Confusion Matrix')\n","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"8a9c45ee48964b8083d3a6bf06be027339ddc185"},"cell_type":"markdown","source":"### Create a Classification Report"},{"metadata":{"trusted":true,"_uuid":"c074dc70027c89f85afbca9246624f12101e1a01"},"cell_type":"code","source":"from sklearn.metrics import classification_report\n\n# Generate a classification report\n\n# For this to work we need y_pred as binary labels not as probabilities\ny_pred_binary = predictions.argmax(axis=1)\n\nreport = classification_report(y_true, y_pred_binary, target_names=cm_plot_labels)\n\nprint(report)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"5b740b6dff66339d660a8ee7f39818f6c045a7ff"},"cell_type":"markdown","source":"**Recall **= Given a class, will the classifier be able to detect it?<br>\n**Precision** = Given a class prediction from a classifier, how likely is it to be correct?<br>\n**F1 Score** = The harmonic mean of the recall and precision. Essentially, it punishes extreme values.\n"},{"metadata":{"_uuid":"5959471e2771fa9cc9a5ed3b212a8da26bf46136"},"cell_type":"markdown","source":"### Convert the model to from Keras to Tensorflowjs\nThis conversion needs to be done so that the model can be loaded into the web app."},{"metadata":{"_kg_hide-output":true,"trusted":true,"_uuid":"007e141c772fe63f0383435642d8c3c4a7605260"},"cell_type":"code","source":"!pip install tensorflowjs","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"fbcbb9ad6050553920edf2277fc08dcc89d04704"},"cell_type":"code","source":"# Use the command line conversion tool to convert the model\n\n!tensorflowjs_converter --input_format keras model.h5 tfjs_model_1/model","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"c00f9303fe526a2c14d7d237b8824c45f982b659"},"cell_type":"code","source":"","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"6543d0726d3ce0cbcae293c5403bcc25257dd527"},"cell_type":"code","source":"# Delete all_images_dir and base_dir directory to prevent a Kaggle error.\n# Kaggle allows a max of 500 files to be saved.\n\nshutil.rmtree('all_images_dir')\nshutil.rmtree('base_dir')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"03dc8b3365c6ebff6ddf247d8bb4b856c9a67e2c"},"cell_type":"markdown","source":"### Conclusion"},{"metadata":{"trusted":true,"_uuid":"9622115537ac68ec79b921def82a49a5e8734cf6"},"cell_type":"markdown","source":"In part 2 we will use the same workflow to build a model to detect metastatic cancer.\n\nMany thanks to Paul Mooney for making the Breast Histopathology Images dataset available on Kaggle. \n\nSee you in part 2."},{"metadata":{"trusted":true,"_uuid":"1b93bb31b4d4ae642acdaa46ff9f6eb40637b847"},"cell_type":"code","source":"","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.6.6","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat":4,"nbformat_minor":1}