cross_entropy_loss_function.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "cross_entropy_loss_function.ipynb",
      "provenance": [],
      "collapsed_sections": [],
      "authorship_tag": "ABX9TyMlTvPcrkRiwa8YjJ7Zr2hY",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/albertofernandezvillan/ml-dl-cv-notebooks/blob/main/cross_entropy_loss_function.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "nWdGwnqpabZB"
      },
      "source": [
        "# Cross-Entropy Loss Function\n",
        "\n",
        "Also called **logarithmic loss**, **log loss** or **logistic loss**. Each predicted class probability is compared to the actual class desired output 0 or 1 and a score/loss is calculated that penalizes the probability based on how far it is from the actual expected value. The penalty is logarithmic in nature yielding a large score for large differences close to 1 and small score for small differences tending to 0.\n",
        "\n",
        "Cross-entropy loss is used when adjusting model weights during training. The aim is to minimize the loss, i.e, the smaller the loss the better the model. A perfect model has a cross-entropy loss of 0.\n",
        "\n",
        "`binary_crossentropy`: Used as a loss function for binary classification model. The `binary_crossentropy` function computes the cross-entropy loss between true labels and predicted labels.\n",
        "\n",
        "`categorical_crossentropy`: Used as a loss function for multi-class classification model where there are two or more output labels. The output label is assigned one-hot category encoding value in form of 0s and 1.\n",
        "\n",
        "`sparse_categorical_crossentropy`: Used as a loss function for multi-class classification model where the output label is assigned integer value (0, 1, 2, 3…). This loss function is mathematically same as the `categorical_crossentropy`. It just has a different interface.\n",
        "\n",
        "\n",
        "* https://vitalflux.com/cross-entropy-loss-explained-with-python-examples/\n",
        "* https://towardsdatascience.com/cross-entropy-for-classification-d98e7f974451\n",
        "* https://towardsdatascience.com/cross-entropy-loss-function-f38c4ec8643e\n",
        "* https://ml-cheatsheet.readthedocs.io/en/latest/loss_functions.html"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "YV1L4tZXNoFZ",
        "outputId": "aa730e05-6cb5-43ee-bdbf-a6fbb869a479"
      },
      "source": [
        "# https://machinelearningmastery.com/cross-entropy-for-machine-learning/\n",
        "\n",
        "from numpy import asarray\n",
        "from keras import backend\n",
        "from keras.losses import binary_crossentropy\n",
        "\n",
        "# prepare classification data\n",
        "p = asarray([1, 0, 0, 0])\n",
        "q = asarray([0.775, 0.116, 0.039, 0.070])\n",
        "\n",
        "# convert to keras variables\n",
        "y_true = backend.variable(p)\n",
        "y_pred = backend.variable(q)\n",
        "\n",
        "# calculate the average cross-entropy\n",
        "mean_ce = backend.eval(binary_crossentropy(y_true, y_pred))\n",
        "print('Average Cross Entropy: %.3f nats' % mean_ce)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Average Cross Entropy: 0.123 nats\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "O-rnVoGGVszq",
        "outputId": "d4a8d946-67be-489c-bfca-43d560538885"
      },
      "source": [
        "# https://keras.io/api/losses/probabilistic_losses/#binarycrossentropy-class\n",
        "\n",
        "# Example 1: (batch_size = 1, number of samples = 4)\n",
        "# y_true (true label): This is either 0 or 1.\n",
        "# y_pred (predicted value): This is the model's prediction\n",
        "# which either represents a logit, (i.e, value in [-inf, inf] when from_logits=True) \n",
        "# or a probability (i.e, value in [0., 1.] when from_logits=False).\n",
        "\n",
        "import tensorflow as tf\n",
        "\n",
        "y_true = [1, 0, 0, 0]\n",
        "y_pred = [0.775, 0.116, 0.039, 0.070]\n",
        "\n",
        "bce = tf.keras.losses.BinaryCrossentropy(from_logits=False)\n",
        "res_1 = bce(y_true, y_pred).numpy()\n",
        "print(res_1)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "0.12263538\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "w6Y8Qkg5WqFc",
        "outputId": "c0c93c3b-84fc-4bbb-afb9-7046accbfdee"
      },
      "source": [
        "import tensorflow as tf\n",
        "\n",
        "y_true = [1, 1, 1, 1]\n",
        "y_pred = [0.775, 0.116, 0.039, 0.070]\n",
        "\n",
        "bce = tf.keras.losses.BinaryCrossentropy(from_logits=False)\n",
        "res_2 = bce(y_true, y_pred).numpy()\n",
        "print(res_2)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "2.0781264\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "68YU0k5zZy0z",
        "outputId": "52c9362c-da2e-4715-9dc9-40a84a501bd1"
      },
      "source": [
        "print((res_1 + res_2)/2)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "1.1003808975219727\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "LC9Vk1uMWvUW",
        "outputId": "6fa19fd6-91b9-49cc-cdc7-df6d46e7243d"
      },
      "source": [
        "import tensorflow as tf\n",
        "\n",
        "# Example 2: (batch_size = 2, number of samples = 4)  \n",
        "y_true = [[1, 1], [0, 1], [0, 1], [0, 1]]\n",
        "y_pred = [[0.775, 0.775], [0.116, 0.116], [0.039, 0.039], [0.070, 0.070]]\n",
        "\n",
        "bce = tf.keras.losses.BinaryCrossentropy(from_logits=False)\n",
        "bce(y_true, y_pred).numpy()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "1.1003809"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 39
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "oPDGDWKMopQa"
      },
      "source": [
        "<img src=\"https://miro.medium.com/max/3648/1*wz59MnXkXdWxrq_fkA2wew.png\">"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Uh62znnET7fz"
      },
      "source": [
        "# https://stats.stackexchange.com/questions/303229/why-does-keras-binary-crossentropy-loss-function-return-wrong-values/303254\n",
        "\n",
        "import keras.backend as K\n",
        "import numpy as np\n",
        "import math\n",
        "\n",
        "def my_binary_crossentropy(y_true, y_pred):\n",
        "    result = []\n",
        "    for i in range(len(y_pred)):\n",
        "        y_pred[i] = [max(min(x, 1 - K.epsilon()), K.epsilon()) for x in y_pred[i]]\n",
        "        result.append(-np.mean([y_true[i][j] * math.log(y_pred[i][j]) + (1 - y_true[i][j]) * math.log(1 - y_pred[i][j]) for j in range(len(y_pred[i]))]))\n",
        "    return np.mean(result)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "PBaay9bnT_2W",
        "outputId": "76c98f4b-e246-4a3c-9997-63b89b46d931"
      },
      "source": [
        "y_true = np.array([[1, 0, 0, 0]])\n",
        "y_pred = np.array([[0.775, 0.116, 0.039, 0.070]])\n",
        "\n",
        "print(my_binary_crossentropy(y_true, y_pred))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "0.12263550720499095\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "by20YGgKdbxX",
        "outputId": "12ad8852-c332-44d8-b9b9-46ee58d64c72"
      },
      "source": [
        "y_true = np.array([[1, 1], [0, 1], [0, 1], [0, 1]])\n",
        "y_pred = np.array([[0.775, 0.775], [0.116, 0.116], [0.039, 0.039], [0.070, 0.070]])\n",
        "\n",
        "my_binary_crossentropy(y_true, y_pred)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "1.1003816295137243"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 48
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Gs3zpwyohD_Q"
      },
      "source": [
        "y_true = np.ones(100)\n",
        "y_pred = np.arange(0, 10, 0.1)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "oFQZRwhqh3Xh",
        "outputId": "6a093f4a-f0f7-4a84-a39a-f121f3376e96"
      },
      "source": [
        "bce = tf.keras.losses.BinaryCrossentropy(from_logits=False)\n",
        "bce(y_true, y_pred).numpy()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0.2334638386964798"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 59
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "aEpPC1wviQBi"
      },
      "source": [
        "# https://ml-cheatsheet.readthedocs.io/en/latest/loss_functions.html\n",
        "\n",
        "values = []\n",
        "y_true = [1]\n",
        "\n",
        "bce = tf.keras.losses.BinaryCrossentropy(from_logits=False)\n",
        "for y_pred in np.arange(0, 1, 0.001):\n",
        "  res = bce(y_true, [y_pred]).numpy()\n",
        "  values.append(res)  "
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 295
        },
        "id": "lhknUx6Xjl6g",
        "outputId": "a0daf906-c789-4a97-b241-06926753495b"
      },
      "source": [
        "import matplotlib.pyplot as plt\n",
        "\n",
        "plt.plot(np.arange(0, 1, 0.001), values)\n",
        "plt.ylabel('log loss')\n",
        "plt.xlabel('predicted probability')\n",
        "plt.title('log loss when true label = 1')\n",
        "plt.show()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAEWCAYAAABhffzLAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deZxcZZ3v8c+39zXdWToLhCQEQoAJghAQ3BBFL4OMjIoOjBuKwwwz4x11rr5UvDMuo8OM4zqOo1EREEUFNy6KyLAFEJAkbGGHsGQj6exLd3rL7/5xTndXeklXOl1V3VXf9+tVrzpbned3qpPf89RzznmOIgIzMysdZYUOwMzM8suJ38ysxDjxm5mVGCd+M7MS48RvZlZinPjNzEqME38JkvS8pDNzsN/bJX1wrPd7MCRdIelfCh1HLkj6jKSrs9x21N9DMX+HpcqJ32wMSJonKSRVFDqWiUDSIkk3SdokyTcT5ZkTv1meuFLYRxfwM+CiQgdSipz4S5ykaklfk7QufX1NUnXG+o9LWp+u+2Daqj0yi/2WSfq0pBckbZR0laSmdF2NpKslbZa0TdL9kmak6y6UtErSTknPSXrXEPuukdQuaVo6f6mkbkmT0vnPS/paxkcmS/pNus/7JB2Rsa+jJd0saYukJyW9M2PdFZL+a7jPDrA0fd8maZek09JjuVvSVyVtBj4zsHtm4C8FSU2Svp9+52sl/Yuk8pG+7/Sz10p6SdJ2SUsl/cmATaalx7pT0h2S5mbzPeRCRDwZEd8HHs1lOTY0J367FDgVOAE4HjgF+DSApLOAjwJnAkcCrzuA/V6Yvs4A5gMNwDfTde8DmoDDgKnA3wDtkuqBbwB/GhGNwCuBBwfuOCL2APcDp6eLTgdeAF6VMX9HxkfOBz4LTAaeAb6QHl89cDPwY2B6ut23JB070meH8Nr0vTkiGiLinnT+FcAqYMZ+PpvpCqCb5Pt+OfAmINvzJjcCC9JjWQH8aMD6dwGfB6aRfK8/gqy/hyFJenVaeQ/3enWWsVseOfHbu4DPRcTGiGglSXLvSde9E/hBRDwaEW3AZw5wv1+JiFURsQv4JHB+2rLtIkn4R0ZET0Qsj4gd6ef2Aosk1UbE+ogYrkV4B3B6ur+XkVQYp0uqAU6mvwUO8MuI+GNEdJMkuxPS5ecAz0fEDyKiOyIeAH4OvCOLz2ZrXUT8Z7r/9v1tmP7qORv4cETsjoiNwFdJEvGIIuLyiNgZER0kf6vje39lpX4TEUvT9ZcCp0k6jOy+h+HKvCsimvfzuiub2C2/nPjtEJLWcq8X0mW961ZnrMucHs1+K0havj8EbgJ+knYh/bukyojYDfwFyS+A9WkXy9HD7P8Okl8gJwKPkLRYTyf59fJMRGzO2PaljOk2kl8fAHOBV2S2UEkqrJlZfDZbB/KdzQUqSY69N57vkLTC90tSuaTLJD0raQfwfLpq2lCxpJXxFpK/UzbfgxURJ35bR/Ifv9ecdBnAemB2xrrDDnK/3cCGiOiKiM9GxLEk3TnnAO8FiIibIuKNwCzgCeC7w+z/D8BC4K3AHRHxWFrG2ezbzbM/q9PPZrZQGyLikgM4zl7DXZkycPluoC5jPjO5rgY6gGkZ8UyKiIF99UP5S+Bckm65JmBeulwZ2/T9/SQ1AFNI/k6j/h4kvSY9pzHc6zVZxG555sRv1wCfltSSniz9J6D35OPPgPdLOkZSHfB/D3C/H5F0eJpkvgj8NCK6JZ0h6bj0pOUOkq6fvZJmSDo37XPuAHaRdP0MknY9LQf+jv5E/weSXwvZJv4bgKMkvUdSZfo6WdIxB3CcvVrTWOePsN2DwGslzUm7YT7ZuyIi1gO/B74saZKSE+RHSDp9uJ1laCT5zjaTVCxfHGKbs9M++SqSvv57I2I1B/E9RMSdaSUx3OvOoT6nRA1Qlc7XKOOiAsstJ377F2AZ8DBJl8mKdBkRcSNJ3/ltJCc2700/05HFfi8n6dJZCjwH7AE+lK6bCVxHkvQfJ0nUPyT59/hRklboFpKum/21Ou8g6Rr5Y8Z8I/v27w8rInaSnDw9Py3zJeDfgANOQGlF9AXg7rS75NRhtrsZ+CnJ972cJOlmei9JMnwM2EryPc3KIoSrSLrT1qafvXeIbX4M/DPJd3sS8O40pjH7Hg7AXKCd/qt62oEnc1ieZZAfxGLZSluAK4Hq9GSnmU1AbvHbfkl6q5Jr/SeTtAL/n5O+2cTmxG8j+WtgI/As0MP+u17MbAJwV4+ZWYlxi9/MrMTkbNAoSZeTXJ+9MSIWZSz/EMkleD0kdxJ+fKR9TZs2LebNm5erUM3MitLy5cs3RUTLwOW5HC3wCpKxWa7qXSDpDJKbTI6PiA5JI96RCDBv3jyWLVuWkyDNzIqVpBeGWp6zrp6IWEpyvXCmS4DL0rFCSMciMTOzPMp3H/9RwGuUDG97h6STh9tQ0sWSlkla1tramscQzcyKW74TfwXJ+CCnAh8DfiZJQ20YEUsiYnFELG5pGdRFZWZmo5TvxL8G+EUk/kgytsm0ET5jZmZjKN+J/1ckD+ZA0lEkY5JsynMMZmYlLZeXc15DMl76NElrSAaHuhy4XNJKoBN4X/gOMjOzvMpZ4o+IC4ZZ9e5clWlmZiMr6jt3b3l8A9+6/ZlCh2FmNq4UdeK//clWvrt0VaHDMDMbV4o68Q99oaiZWWkr6sQPwz8I1cysVBV14hfga4bMzPZV3InffT1mZoMUdeIH8G0CZmb7Kv7EX+gAzMzGmaJO/BLO/GZmAxR34sd9/GZmAxV14gc3+M3MBirqxC/55K6Z2UDFnfgLHYCZ2ThU1Ikf3NVjZjZQUSf+pKun0FGYmY0vRZ74RbjNb2a2j5wlfkmXS9qYPm1r4Lp/lBSScvq8Xffxm5kNlssW/xXAWQMXSjoMeBPwYg7L7uOuHjOzfeUs8UfEUmDLEKu+CnycfJx3lU/umpkNlNc+fknnAmsj4qG8lOfOHjOzQXL2sPWBJNUBnyLp5slm+4uBiwHmzJkz+oLd5Dcz20c+W/xHAIcDD0l6HpgNrJA0c6iNI2JJRCyOiMUtLS2jKlDCV/WYmQ2QtxZ/RDwCTO+dT5P/4ojYlKsy3dFjZjZYLi/nvAa4B1goaY2ki3JV1v74qh4zs33lrMUfEReMsH5ersruJV/VY2Y2SHHfuYs8OqeZ2QDFnfjdyW9mNkhRJ35wV4+Z2UBFnfiFT+6amQ1U1InffT1mZoMVd+I3M7NBijrx97b3fWWPmVm/4k78aeZ33jcz61fcid+DNpiZDVLUib+XG/xmZv2KOvH3d/U49ZuZ9SruxF/oAMzMxqGiTvy93N43M+tX1InfV/WYmQ1W5Ik/yfx+CpeZWb+iTvxmZjZYSSR+d/WYmfXL5aMXL5e0UdLKjGVfkvSEpIcl/VJSc67KT8rL5d7NzCamXLb4rwDOGrDsZmBRRLwMeAr4ZA7L9527ZmZDyFnij4ilwJYBy34fEd3p7L3A7FyVv2+5+SjFzGxiKGQf/weAG4dbKeliScskLWttbR1VAX2Xc/qqHjOzPgVJ/JIuBbqBHw23TUQsiYjFEbG4paVldOX07WtUHzczK0oV+S5Q0oXAOcAbIseD6PjkrpnZYHlN/JLOAj4OnB4Rbfkq1w1+M7N+ubyc8xrgHmChpDWSLgK+CTQCN0t6UNK3c1U+9F/V49E5zcz65azFHxEXDLH4+7kqbyju6jEzG6w07twtdABmZuNIaSR+Z34zsz5Fnfjlvh4zs0GKOvH3cYvfzKxPUSf+vhu4nPnNzPoUd+L3E7jMzAYp7sRf6ADMzMahok78vdzgNzPrV9SJv++Zu+7rMTPrU+SJv9ARmJmNP0Wd+Hu5vW9m1q+oE7/H4zczG6yoE39vX4+v4zcz61fUid9d/GZmgxV14u/jBr+ZWZ+iTvz9D1s3M7NexZ343dljZjZILh+9eLmkjZJWZiybIulmSU+n75NzVX4mX9VjZtYvly3+K4CzBiz7BHBLRCwAbknnc6a/q8eZ38ysV84Sf0QsBbYMWHwucGU6fSXw57kqH3wdv5nZUPLdxz8jItan0y8BM4bbUNLFkpZJWtba2jqqwjxkg5nZYAU7uRvJyGnDtsUjYklELI6IxS0tLQdX1kF92sysuOQ78W+QNAsgfd+Yy8J6r+rx6JxmZv3ynfivB96XTr8P+HVOS3NXj5nZILm8nPMa4B5goaQ1ki4CLgPeKOlp4Mx0Pufc4Dcz61eRqx1HxAXDrHpDrsocyA1+M7PBivvO3b4ncBU4EDOzcaSoE7+ZmQ1W1Im/7wYuX9BpZtanuBN/75ANzvtmZn1KIvGbmVm/ok78vdzgNzPrd0CJX9JkSS/LVTBjzXfumpkNNmLil3S7pEmSpgArgO9K+kruQzt47uoxMxssmxZ/U0TsAN4GXBURryC563bCcHvfzKxfNom/Ih1Q7Z3ADTmOJyfc02Nm1i+bxP854CbgmYi4X9J84OnchjU2pP4r+c3MLDHiWD0RcS1wbcb8KuDtuQxqrLiL38xssGxO7v57enK3UtItklolvTsfwY0Vd/WYmfXLpqvnTenJ3XOA54EjgY/lMqix0v+wdTMz65XVyd30/c3AtRGxPYfxjCm5s8fMbJBsxuO/QdITQDtwiaQWYE9uwxpb7uoxM+s3Yos/Ij4BvBJYHBFdwG7g3IMpVNJHJD0qaaWkayTVHMz+hi8neffonGZm/bI5uVsJvBv4qaTrgIuAzaMtUNKhwP8mqUgWAeXA+aPd337LSt/d4jcz65dNV89/A5XAt9L596TLPniQ5dZK6gLqgHUHsa9hecgGM7PBskn8J0fE8Rnzt0p6aLQFRsRaSf8BvEhy3uD3EfH7gdtJuhi4GGDOnDmjLS4t86A+bmZWVLK5qqdH0hG9M+mduz2jLVDSZJJzBIcDhwD1Q90XEBFLImJxRCxuaWkZbWnJvtzHb2bWJ5sW/8eA2yStIsmkc4H3H0SZZwLPRUQrgKRfkJw8vvog9jkkd/WYmQ2WzZANt0haACxMFz0ZER0HUeaLwKmS6ki6et4ALDuI/Y3IXT1mZv2GTfyS3jbMqiMlERG/GE2BEXFfenXQCqAbeABYMpp9jcQNfjOzwfbX4v+z/awLYFSJHyAi/hn459F+Plu9o3O6xW9m1m/YxB8RB9OPPy64xW9mNliJPGzdTX4zs15Fnfj7hmxw3jcz61MSid/MzPqNeDnnMFf3bAceiYiNYx/S2HOD38ysXzY3cF0EnAbcls6/DlgOHC7pcxHxwxzFdtB6x+MP9/WYmfXJJvFXAMdExAYASTOAq4BXAEuBcZv48RO4zMwGyaaP/7DepJ/amC7bAnTlJiwzM8uVbFr8t0u6Abg2nT8vXVYPbMtZZGOgPD27u3ev2/xmZr2ySfx/B7wNeHU6fyXw80g6zs/IVWBjoboi+UHT0b23wJGYmY0f2QzSFpLuAjpJusv/GBPkbGl1ZTkAe7pGPYq0mVnRyebRi+8E/kjSxfNO4D5J5+U6sLFQU+kWv5nZQNl09VxK8hSujQCSWoD/Aa7LZWBjoabCLX4zs4GyuaqnbMCNWpuz/FzB1fR19bjFb2bWK5sW/+8k3QRck87/BfDb3IU0dnpP7rrFb2bWL5uTux+T9HbgVemiJRHxy9yGNTb6WvzdTvxmZr2yafETET8Hfp7jWMZc3+Wc7uoxM+uzv0cv7mTo0Q5EcpXnpNEWKqkZ+B6wKC3jAxFxz2j3N5yyMlFTWUZbZ/dY79rMbMLa3xO4GnNY7teB30XEeZKqgLpcFdRUW8n2do8sYWbWK6uunrEkqQl4LXAhQER0ktwclhPNtVVO/GZmGQpxWebhQCvwA0kPSPpeOu7PPiRdLGmZpGWtra2jLqyptpJtbU78Zma9CpH4K4ATgf+OiJcDu4FPDNwoIpZExOKIWNzS0jLqwprq3NVjZpapEIl/DbAmIu5L568jqQhyotl9/GZm+8h74o+Il4DVkhami94APJar8nxy18xsX3k/uZv6EPCj9IqeVcD7c1VQc10lbZ09dHbvpapiQow0YWaWUwVJ/BHxILA4H2U11VYCsL29i5bG6nwUaWY2rhV9E3hKfZLsW3d2FDgSM7PxoegT/yHNNQCs395e4EjMzMaHok/8hzbXArBumxO/mRmUQOKf1lBNZblYu21PoUMxMxsXij7xl5WJWU21bvGbmaWKPvEDzJ5cy4tb2godhpnZuFASif/I6Q08u3EXEUONMm1mVlpKIvEvmN7Azo5uXtrhfn4zs5JI/EdOTx4t8NSGXQWOxMys8Eoi8R8zK0n8K9duL3AkZmaFVxKJv7muivkt9ax4YWuhQzEzK7iSSPwAJ82ZzIoXt/oEr5mVvJJJ/CfOnczWti6e27S70KGYmRVUyST+Uw6fAsDdz24ucCRmZoVVMol//rR65k6t49bHNxQ6FDOzgiqZxC+J1x89nT88u5n2zp5Ch2NmVjAlk/gB3njMDDq69/I/bvWbWQkrWOKXVC7pAUk35KvMU+dP5ZCmGq5dviZfRZqZjTuFbPH/A/B4PgssKxNvP2k2dz7d6gezmFnJKkjilzQbeDPwvXyXfd5Js4mAH9/3Yr6LNjMbFwrV4v8a8HFg73AbSLpY0jJJy1pbW8es4LlT6/lffzKDK//wPDv3dI3Zfs3MJoq8J35J5wAbI2L5/raLiCURsTgiFre0tIxpDH/7uiPZsaebq+91q9/MSk8hWvyvAt4i6XngJ8DrJV2dzwCOP6yZ1y1s4Vu3P8PmXR35LNrMrODynvgj4pMRMTsi5gHnA7dGxLvzHcen33ws7Z09/Mfvn8x30WZmBVVS1/FnOnJ6A+975Tx+cv9qlr+wpdDhmJnlTUETf0TcHhHnFKr8D5+5gEOba/nITx9iV0d3ocIwM8urkm3xAzTWVPLVvziBNVvb+PQvH/GQzWZWEko68QOcPG8KHz7zKH714DqWLF1V6HDMzHKuotABjAcfev2RPLlhJ5f97gnmTq3nrEUzCx2SmVnOlHyLH5KRO//jvOM5fnYzH7pmBbd4EDczK2JO/KnaqnKu/MApHDNrEpdcvYJbn3DyN7Pi5MSfoam2kqs+cAoLZzbyV1ct93g+ZlaUnPgHaK6r4pqLT+W1C6bxqV8+whd+8xhdPcMOKWRmNuE48Q+hobqC7753Me89bS7fvfM5Llhyr4dxNrOi4cQ/jIryMj537iK+fv4JPL5+B2d//U5ueHidr/U3swnPiX8E555wKNd/6NUcNqWOv//xA/z1D5ezcceeQodlZjZqTvxZOKKlgV9c8ko++adHc8dTrbzhy3fwnTuepaPbD203s4nHiT9LFeVl/PXpR/C7D7+Wkw+fwr/e+ARv/MpSbnxkvbt/zGxCceI/QIdPq+fyC0/myg+cQnVFGZf8aAVnf+MufrdyPXv3ugIws/HPiX+UTj+qhRv/4TV8+R3Hs6erh7+5egVnf+NOfv3gWl/+aWbjmiZCN8XixYtj2bJlhQ5jWN09e7nh4fX8561P82zrbmZMquY9p87lglPmMLWhutDhmVmJkrQ8IhYPWu7EP3b27g3ueKqVy+9+jjuf3kRVRRlnL5rJOxYfxmnzp1JWpkKHaGYlZLjEn/fROSUdBlwFzAACWBIRX893HLlQVibOOHo6Zxw9nWc27uTKP7zArx5cy68eXMehzbW8/cRDOe+kw5gzta7QoZpZCct7i1/SLGBWRKyQ1AgsB/48Ih4b7jMTpcU/lD1dPdz06Etct3wNdz2ziQg4fnYTZx83i7OPm8VhU1wJmFlujNuuHkm/Br4ZETcPt81ETvyZ1m1r59cPruPGlet5eM12AI47NKkE3njsdI5oaUByd5CZjY1xmfglzQOWAosiYseAdRcDFwPMmTPnpBdeeCHv8eXS6i1t/PaR9fx25Us8tHobALMn13LGwumccXQLp82fRm1VeYGjNLOJbNwlfkkNwB3AFyLiF/vbtlha/MNZt62d257cyG1PtHL3M5to7+qhqqKMU+dP5ZVHTOW0+VNZdGgT5T45bGYHYFwlfkmVwA3ATRHxlZG2L/bEn6mju4c/PreF255o5Y6nNvJs624AGqsrOOXwKZx2xFROnT+VY2dN8lVCZrZf4+mqHgHfBx7PJumXmuqKcl6zoIXXLGgBjmXjjj3c+9wW7nl2M/eu2swtT2wEoLGmghMOa+bEOZM5ce5kTjismabaysIGb2YTQiGu6nk1cCfwCNB7i+unIuK3w32mlFr8I3lp+x7uWbWJ+5/fyooXtvLUhp30jhSxYHoDJ86ZzMvnNPOy2c0smNFAZblvzjYrVeOqq+dAOfEPb1dHNw+t3saKF7ay4sWtPLB6G9vaugCoqijj6JmN/MkhTSw6dBKLDmli4cxGaip90tisFDjxl4iI4LlNu3lk7XYeXbeDlWu3s3Ltdnbs6QagvEwsmN7AsYdM4uiZjRw1I3nNaqrxpaRmRcaJv4RFBGu2tvPouu2sXLuDleuSSqF1Z0ffNo3VFRyVVgQLZzRw1MxGFs5o9FhDZhOYE78NsnV3J09t2MlTG3fx1Es7eXLDTp58aSfb27v6tplcV8n8lgYOn1bP/JZ65k+rZ35LA3Om1LnLyGycc+K3rEQErTs7+iqBZ1t3s6p1F89t2s3GjF8IZYJDJ9cyf1pSKRzRUs/cqfXMmVLHIc21VFX4pLJZoY2byzltfJPE9Ek1TJ9Uk15S2m/nni6e39TGqk27WNW6m1WbdvPcpl0se34Luzv7H0NZJpjVVMucKXUcNqX3vY456WtKfZXPJ5gVkBO/Za2xppLjZjdx3OymfZZHBBt2dPDilra+1+r0/bYnW/c5lwBQV1XOnCl1zJ5cx+zJtRzSXMOsploOaa7l0OZaWhqrfZeyWQ458dtBk8TMphpmNtVwyuFTBq1v7+xhzda2QRXD6i1t3LdqMzs7uvfZvqIs2d8hTUmlcEhzbfrqn26srvCvBrNRcuK3nKutKmfBjEYWzGgccv2OPV2s37aHddvaWbutnXXb2lm/fQ9rt7Wz7IWtvPTweroHPM+4rqqcmZNqmD6pmhmTapgxqYbpjf3Tvet8AtpsMCd+K7hJNZVMmlnJwplDVww9e5MTzuu2J5XC2q3tbNjRwYade9i4Yw8PvLiNl3bsobN78LOOm2ormZFWDtMba5jZlEy3NFQzrbGaaQ3VTGuoosG/IKyEOPHbuFde1t+VdOKcyUNuExHsaO/mpR172JC+Nu7s6JvesKODZzduYuPOjkG/HgCqK8r6KoHkvZppjRnTDdW0pPNNtZWuJGxCc+K3oiCJprpKmuqG/+UAyXORN+/upHVnB5t2Zb462bSzg9ZdHazbvoeH125ny+5OeoaoJCrLxdT6/ophSl0Vk+urmJK+Jtf1TlcypT6pKHyy2sYTJ34rKWVloqWxmpbGke9I3rs32NrWmVQKaQWRVBidbM6oMJ7esIutbZ20ZVzSmkmC5trKpHJIK4mp9VX7zE+pr2RyXRVT66uZXF/prifLKSd+s2GUlYmpDdVMbahmIcP/iui1p6uHrW2dbN7Vyda2Trbs7mTr7k62tHUl7+lr9ZY2Hlq9ja1tnXT1DH0DZXmZaK6tpKk2+RXTXFtJc11VMl9bSXNd8krmq5L5dF2FR2S1ETjxm42RmspyZjXVMqupNqvtI4JdHd19FUJSWXSxZXcH29u72NbWxfb25LVpVyfPtO5ie1tX34B7w2morhixcmisqWRSbQWTaipprKlgUm3yXl3hq6BKgRO/WYFIorEmScJzp9Zn/bmevcGO9i62pZXCtrbOvoqit7LY1t7J9rZkmw07dqXLh/+F0au6oqyvEphUU7nvdFpBTKqp6Ks4Gmsq0+2S6fqqcndRTQBO/GYTTHmZmJyeIzgQEUFbZw/b27vYuaebHXu62JExvXNPNzvau5LlvdPtXazZ2saO9m527umiY4hLZjOViUG/JhqqK2moLqchnW6sqaC+qpyGmuRcRkN1Rbquf7qustyPFs0hJ36zEiGJ+uoK6qtH/9++o7sno4JIKoPeSmHHPtP969Zta2dXR3ffa6j7LQbHCvVV/RVBfXUFjWnFUF9dkVYoyXRDzfDr6qvLqa30r5CBCpL4JZ0FfB0oB74XEZcVIg4zOzDVFeVUN5Qz7SCe09DR3cPujh527enOqBC62NW3rCt970mXp9N7uti4c88+nxviattBJKirLKeuOvmlUVeVVAj11RXUV1VQV5VM7/NeVUFddfmg9Q3VFdRVT/xfJIV42Ho58F/AG4E1wP2Sro+Ix/Idi5nlX3VFOdUV5Uw5wK6qgSKC9q6epBLorQz2qUy6aevsoa2jm92dPbR1drO7o/996+5O1mxt71u/u6N7yJv7hlNbWU59dVKRDFtpZFQevdvVVpWn8+XUVlb0T6fb5OOej0K0+E8BnomIVQCSfgKcCzjxm1nWJKXJtILpI19tm5XO7r3s7uhmd2dSaezuGPDe2U1bR8+w63d1dLNxR8c+60c6LzJQVUVZUhlUJpXBF996HK+YP3VsDjBViMR/KLA6Y34N8IqBG0m6GLgYYM6cOfmJzMxKWlVFGVUVB37ifH+6e/ayu7OH9vRXR1tnD+1dPcl7Ot/Wt76Htq5u2juT7rD2rm4aayrHLJZe4/bkbkQsAZZA8gSuAodjZjYqFeVlNNWW0VQ79gl8tApxi99a4LCM+dnpMjMzy4NCJP77gQWSDpdUBZwPXF+AOMzMSlLeu3oiolvS3wM3kVzOeXlEPJrvOMzMSlVB+vgj4rfAbwtRtplZqfMwfmZmJcaJ38ysxDjxm5mVGCd+M7MSo4jxf2+UpFbghVF+fBqwaQzDmQh8zKXBx1waDuaY50ZEy8CFEyLxHwxJyyJicaHjyCcfc2nwMZeGXByzu3rMzEqME7+ZWYkphcS/pNABFICPuTT4mEvDmB9z0ffxm5nZvkqhxW9mZhmc+M3MSkzRJH5JZ0l6UtIzkj4xxPpqST9N198naV7+oxxbWRzzRyU9JulhSbdImluIOMfSSMecsd3bJYWkCX/pXzbHLOmd6d/6UUk/zneMYy2Lf9tzJN0m6YH03/fZhYhzrEi6XNJGSSuHWS9J30i/j4clnXhQBRp7O88AAAcNSURBVEbEhH+RDO/8LDAfqAIeAo4dsM3fAt9Op88HflrouPNwzGcAden0JaVwzOl2jcBS4F5gcaHjzsPfeQHwADA5nZ9e6LjzcMxLgEvS6WOB5wsd90Ee82uBE4GVw6w/G7gREHAqcN/BlFcsLf6+B7hHRCfQ+wD3TOcCV6bT1wFvkJT7x9nnzojHHBG3RURbOnsvydPOJrJs/s4Anwf+DdiTz+ByJJtj/ivgvyJiK0BEbMxzjGMtm2MOYFI63QSsy2N8Yy4ilgJb9rPJucBVkbgXaJY0a7TlFUviH+oB7ocOt01EdAPbgbF9dH1+ZXPMmS4iaTFMZCMec/oT+LCI+E0+A8uhbP7ORwFHSbpb0r2SzspbdLmRzTF/Bni3pDUkz/b4UH5CK5gD/f++X+P2Yes2diS9G1gMnF7oWHJJUhnwFeDCAoeSbxUk3T2vI/lVt1TScRGxraBR5dYFwBUR8WVJpwE/lLQoIvYWOrCJoFha/Nk8wL1vG0kVJD8PN+clutzI6qH1ks4ELgXeEhEdeYotV0Y65kZgEXC7pOdJ+kKvn+AneLP5O68Bro+Iroh4DniKpCKYqLI55ouAnwFExD1ADclgZsUqq//v2SqWxJ/NA9yvB96XTp8H3BrpWZMJasRjlvRy4DskSX+i9/vCCMccEdsjYlpEzIuIeSTnNd4SEcsKE+6YyObf9q9IWvtImkbS9bMqn0GOsWyO+UXgDQCSjiFJ/K15jTK/rgfem17dcyqwPSLWj3ZnRdHVE8M8wF3S54BlEXE98H2Sn4PPkJxEOb9wER+8LI/5S0ADcG16HvvFiHhLwYI+SFkec1HJ8phvAt4k6TGgB/hYREzYX7NZHvM/At+V9BGSE70XTuSGnKRrSCrvael5i38GKgEi4tsk5zHOBp4B2oD3H1R5E/i7MjOzUSiWrh4zM8uSE7+ZWYlx4jczKzFO/GZmJcaJ38ysxDjx24Qn6XWSbkin3zLCqJ3Nkv52FGV8RtL/OZg4x2K/kubtZwTH70k6Np1+Pr2mH0l/yPjsX45F3DaxOfHbuCWp/EA/ExHXR8Rl+9mkmWSk1rxJ7xTPuYj4YEQ8NsTyV6aT8wAnfnPit/xLW55PSPqRpMclXSepLl33vKR/k7QCeIekN0m6R9IKSddKaki3OyvdxwrgbRn7vlDSN9PpGZJ+Kemh9PVK4DLgCEkPSvpSut3HJN2fjnP+2Yx9XSrpKUl3AQuHOZYrJH1b0rJ023My4rhe0q3ALZKmSPpVWsa9kl6WsZvj02N8WtJfpZ9vUPIMhRWSHpGUOTplxTDf3e1DDU8haVc6eRnwmvTYPyJpqaQTMra7S9Lx2fwNbWJz4rdCWQh8KyKOAXawbyt8c0ScCPwP8GngzHR+GfBRSTXAd4E/A04CZg5TxjeAOyLieJKxzh8FPgE8GxEnRMTHJL2JZFybU4ATgJMkvVbSSSR3d59Acsfkyfs5lnnp598MfDuNj7TM8yLidOCzwAMR8TLgU8BVGZ9/GfB64DTgnyQdQjKk9FvT4z4D+LLUN4z4/r67/fkEcGd67F8luZv9QgBJRwE1EfFQlvuyCcyJ3wpldUTcnU5fDbw6Y91P0/dTSR6ycbekB0nGWpoLHA08FxFPp7fpXz1MGa8H/hsgInoiYvsQ27wpfT0ArEj3vQB4DfDLiGiLiB0MHism088iYm9EPE0yRs7R6fKbI6J3jPVXAz9MY7kVmCqpdzz5X0dEe0RsAm4jqUQEfFHSwyQV4KHAjHT7/X13B+Ja4BxJlcAHgCtGuR+bYIpirB6bkAaOFZI5vzt9F0nyvCBzw8zuiTEg4F8j4jsDyvjwAexjuGPZPXDDA/j8u4AW4KSI6FIy2mjNfrY/YBHRJulmkod8vJPk15OVALf4rVDmKBlHHZITjncNsc29wKskHQkgqT7tkngCmCfpiHS7C4b4LMAtJI+cRFK5pCZgJ8nwzb1uAj6Qce7gUEnTSR7d+OeSaiU1knQrDecdksrSeOYDTw6xzZ0kyRxJrwM2pb8kAM6VVCNpKslAXfeTDBu+MU36Z5D80umVzXc3lIHHDvA9ki6x+3uf4GXFz4nfCuVJ4O8kPQ5MJu2SyRQRrSR90NekXR73AEdHxB7gYuA36cnd4Yac/gfgDEmPAMtJntu6maTraKWkL0XE74EfA/ek210HNEbECpIup4dInlx2/36O5UXgj+l2f5PGN9BnSM4fPExykvV9GeseJuniuRf4fESsA34ELE5jei9JZddrxO9uGA8DPemJ7o8ARMRykvMEP8hyH1YEPDqn5Z2kecANEbGowKEcNElXkBzLdYWOZTTSE8m3k1SofnpViXCL36xESXovcB9wqZN+aXGL38ysxLjFb2ZWYpz4zcxKjBO/mVmJceI3MysxTvxmZiXm/wPCHhdgy2KK/QAAAABJRU5ErkJggg==\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "aZfs93Qvj4hK"
      },
      "source": [
        "values = []\n",
        "y_true = [0]\n",
        "\n",
        "bce = tf.keras.losses.BinaryCrossentropy(from_logits=False)\n",
        "for y_pred in np.arange(0, 1, 0.001):\n",
        "  res = bce(y_true, [y_pred]).numpy()\n",
        "  values.append(res)  "
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 279
        },
        "id": "HsCRJ4L8j6-H",
        "outputId": "f8f8950e-69fa-42b7-9b7e-679f9d965969"
      },
      "source": [
        "import matplotlib.pyplot as plt\n",
        "\n",
        "plt.plot(np.arange(0, 1, 0.001), values)\n",
        "plt.ylabel('log loss')\n",
        "plt.xlabel('predicted probability')\n",
        "'log loss when true label = 0'\n",
        "plt.show()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXgAAAEGCAYAAABvtY4XAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deXhd1Xnv8e+rwRqs0ZZs2bJl2cYjNsZGmDEkQCBACDSBJJCShmZwmrZpk/ZyS5LeZrptk+Y2bfIkDXVJQoAMjUlIHTIACVMgGPCAjbENngd5kGRrnqXz3j/OtiIbyz62tc/R2fp9nkfPmfbZ692S/dPS2uusbe6OiIhET0aqCxARkXAo4EVEIkoBLyISUQp4EZGIUsCLiERUVqoLGKysrMyrq6tTXYaISNpYs2ZNg7uXn+i1ERXw1dXVrF69OtVliIikDTPbPdRrGqIREYkoBbyISESFFvBmNsfMXh701WJmnwirPREROVZoY/Du/hpwPoCZZQK1wMNhtSciIsdK1hDN1cB2dx/yZICIiAyvZAX8bcAPk9SWiIiQhIA3szHATcCKIV5fZmarzWx1fX192OWIiIwayejBXw+sdfdDJ3rR3Ze7e42715SXn3CuvohIZD2+6RD3PL09lH0nI+BvR8MzIiIn9MSWQ3z72Z2h7DvUgDezscA1wE/DbEdEJF31x5ysDAtl36EuVeDu7cD4MNsQEUlnfTEnM6SA1ydZRURSKMwevAJeRCSF1IMXEYmo/n4nKyOcKFbAi4ikkHrwIiIR1R+LkZWpgBcRiRz14EVEIkqzaEREIko9eBGRiIr34DWLRkQkctSDFxGJqP5YTGPwIiJR1NfvZCjgRUSiR7NoREQiql9j8CIi0dTv6sGLiERSX7+TqWmSIiLRozF4EZGI6u3XYmMiIpHU0xdjTJaGaEREIqenP8aYzDQMeDMrMbOHzGyLmW02s0vCbE9EJJ24ezzgQ+rBZ4Wy1z/4GvBrd7/VzMYA+SG3JyKSNvpijjuh9eBDC3gzKwauAO4EcPceoCes9kRE0k1vfwyA7DQcg58O1APfNbN1ZnavmY09fiMzW2Zmq81sdX19fYjliIiMLD198YBPxzH4LGAJ8C13Xwy0A3cfv5G7L3f3GnevKS8vD7EcEZGRZSDg07AHvw/Y5+4vBI8fIh74IiICdKdrwLv7QWCvmc0Jnroa2BRWeyIi6eboGHzanWQNfBz4fjCDZgfwpyG3JyKSNnr6w+3Bhxrw7v4yUBNmGyIi6SqdT7KKiMhJdPXGAz43OzOU/SvgRURSpKu3H4DcbPXgRUQipXMg4NWDFxGJlC4FvIhINB0N+LwxCngRkUgZOMmabh90EhGRk+tUD15EJJqODtHkZCngRUQipaOnn9zsDDJ10W0RkWhp6+6jICe8BQUU8CIiKdLWpYAXEYmk9u4+xirgRUSip1UBLyISTe3dfRQq4EVEokdDNCIiEdXW3UdBrgJeRCRyNE1SRCSC+vpjdPXGFPAiIlHT3h1fpkBj8CIiEdPa3QsQ6iyaUC+6bWa7gFagH+hzd12AW0QEaOqIB3xxfnZobYQa8IEr3b0hCe2IiKSNxo4eAErzx4TWhoZoRERSoDHowZeG2IMPO+AdeMzM1pjZshNtYGbLzGy1ma2ur68PuRwRkZGhKejBhzlEE3bAX+7uS4Drgb8wsyuO38Ddl7t7jbvXlJeXh1yOiMjI0Nge78GX5KXpEI271wa3dcDDwNIw2xMRSReNHT0U5GQxJqTrsUKIAW9mY82s8Oh94FpgY1jtiYikk6aOHkpCHJ6BcGfRTAQeNrOj7fzA3X8dYnsiImmjsaM31Bk0EGLAu/sOYFFY+xcRSWcNbd1MKMwJtQ1NkxQRSYH61m4mFOaG2oYCXkQkyfpjTkNbN+XqwYuIRMuR9h5iDhOKFPAiIpFS19oFQHmBAl5EJFLqW7sB9eBFRCLnYHO8B6+TrCIiEVPb1EmGQUWxAl5EJFJqGzupKMolOzPcCFbAi4gk2b6mTqaU5ofejgJeRCTJahs7qSzNC70dBbyISBL19cc42NJFZYkCXkQkUg62dNEfc/XgRUSiZl9jJ4B68CIiUbOzoR2A6WVjQ29LAS8ikkQ76tvIycpQD15EJGp21LczvWwsGRkWelsKeBGRJNpe38bM8oKktKWAFxFJku6+fvY2djKjPPzxd1DAi4gkze7DHfTHPDoBb2aZZrbOzB4Juy0RkZFs84EWAOZWFCWlvWT04P8a2JyEdkRERrRNB1oYk5kxMsfgzazUzM47je2nAG8H7j3dwkREombT/hZmTSxgTFZyRsdP2YqZPWVmRWY2DlgL/JeZfTXB/f878L+B2En2v8zMVpvZ6vr6+gR3KyKSfjYfaGXepOQMz0BiPfhid28B3gXc7+4XAW891ZvM7Eagzt3XnGw7d1/u7jXuXlNeXp5Q0SIi6eZQSxcNbd3MH2EBn2Vmk4D3AKdzovQy4CYz2wX8CLjKzB48/RJFRNLfuj2NAJxfVZK0NhMJ+C8AjwLb3P0lM5sBbD3Vm9z9U+4+xd2rgduAJ9z9jrOqVkQkTa3d08SYzAzOnZy8HnzWqTZw9xXAikGPdwC3hFmUiEjUrN3dyILKInKyMpPWZiInWf8lOMmabWa/NbN6Mzutnri7P+XuN555mSIi6aunL8aG2maWVJUmtd1EhmiuDU6y3gjsAs4B7gqzKBGRKHl1fzM9fTGWTBt5AX90GOftwAp3bw6xHhGRyFm14wgANdXJDfhTjsEDj5jZFqAT+JiZlQNd4ZYlIhIdz21rYG5FIRMKc5Pa7il78O5+N3ApUOPuvUA7cHPYhYmIREFXbz8v7jrCZeeUJb3tU/bgzSwbuAO4wswAngbuCbkuEZFIWLO7kZ6+GJedMz7pbScyRPMtIBv4j+Dx+4PnPhxWUSIiUfHM6/VkZRhLp4/MgL/Q3RcNevyEma0PqyARkahwdx7bdIhLZo6nICeRuB1eicyi6TezmUcfBJ9k7Q+vJBGRaNhW18bOhnauPbciJe0n8ivlLuBJM9sBGDAN+NNQqxIRiYDHNh0C4Jp5E1PSfiJLFfzWzGYBc4KnXnP37nDLEhFJf4+9epBFU0uoKE7u9Mijhgx4M3vXEC+dY2a4+09DqklEJO3tPdLB+n3N3PW2OafeOCQn68G/4ySvOaCAFxEZwsPragG4+fzJKathyIB3d42zi4icAXfn4XW1XDxjHFNK81NWR3IuDCgiMoqs29vEzoZ23rVkSkrrUMCLiAyzFav3kpudwfULUjM98igFvIjIMGru6OVn6/Zz86JKCnOzU1pLImvRnGg2TTPwirvXDX9JIiLp68er99LZ288HLq1OdSkJfdDpQ8AlwJPB47cAa4DpZvYFd38gpNpERNJKf8x5YNVullaPY34Sr706lEQv+DHP3W9x91uA+cSnSV4E/F2YxYmIpJMnt9Sx50jHiOi9Q2IBP9XdDw16XBc8dwToDacsEZH04u5848ltTCnN49pzU7M0wfESGaJ5ysweAVYEj28NnhsLNA31JjPLBZ4BcoJ2HnL3z55lvSIiI9Lvtjbw8t4m/umdC8nOHBnzVxIJ+L8A3gVcHjz+HvATd3fgypO8rxu4yt3bgouGPGtmv3L3VWdVsYjICOPufP23W5lUnMstF1SmupwBiSw25mb2LNBDfOz9xSDcT/k+oC14mB18nfJ9IiLp5tltDaze3cjnbzqXnKzMVJcz4JR/R5jZe4AXiQ/NvAd4wcxuTWTnZpZpZi8TH7d/3N1fOME2y8xstZmtrq+vP73qRURSrD/m/OMvNjN1XB63LZ2a6nKOkcgQzWeIX9WpDsDMyoHfAA+d6o3u3g+cb2YlwMNmtsDdNx63zXJgOUBNTY16+CKSVh5as5ctB1v5xvsWj6jeOyQ2iybjuA80HU7wfQPcvYn4PPrrTud9IiIjWXt3H//vsddZUlXC2xdOSnU5b5BID/7XZvYo8MPg8XuBX57qTUFPv9fdm8wsD7gG+PIZVyoiMsJ8/Ymt1Ld2c88dF2BmqS7nDRI5yXqXmd0CXBY8tdzdH05g35OA75lZJvEe/4/d/ZEzL1VEZOR4dX8z9/5uJ++tmcoF00pTXc4JJXSZb3f/CfCT09mxu28AFp9JUSIiI1l/zPnUT1+hND+bT90wN9XlDOlkl+xr5cTTGo34LMjUL7QgIpIC33l2Jxv2NfO1286nJH9MqssZ0smu6FSYzEJERNLBpv0tfOXR13jrvInctCh1l+NLxMj4PK2ISBro6u3nr3+0juL8bL58y8IReWJ1sITG4EVEBP7xF5vZWtfG/R9cyviCnFSXc0rqwYuIJOBn62p5YNVuPnz5dK6YXZ7qchKigBcROYWNtc3c/dMNLJ0+jr+7fuTOmjmeAl5E5CSOtPfwZw+uoTR/DP/xx0tGzFLAidAYvIjIEDp7+vnw916irrWbFR+9hLI0GHcfLH1+FYmIJFF/zPmrH61j3d4mvn7bYhZNLUl1SadNAS8ichx357MrN/L4pkN87h3nct2CilSXdEYU8CIig7g7//yrLTy4ag8fffOMEXMB7TOhgBcRCbg7X/71ayx/Zgfvv3gad1+XPjNmTkQBLyJCPNz/9bHXuefp7bzvoio+f9O5I/6TqqeiWTQiMurFYs4XHtnEfb/fxW0XTuX/3ryAjIz0DndQwIvIKNfTF+NvV6zn5+v386HLp/OZG+ZFItxBAS8io1hbdx9/9sAant3WwN3Xz+WjV8xI+2GZwRTwIjIq7T3SwUfuX83Wuja+cut5vLtmaqpLGnYKeBEZdVbtOMzHHlxDf8z57p0Xps3iYadLAS8io4a78+ALe/j8yleZNj6fez9wIdPLxqa6rNAo4EVkVGjt6uXTD2/k5+v3c+Wccr52+2KKcrNTXVaoQgt4M5sK3A9MJH5t1+Xu/rWw2hMRGcrG2mb+8gdr2XOkg7veNoePvXlmZGbKnEyYPfg+4G/dfa2ZFQJrzOxxd98UYpsiIgNiMee+3+/iS7/ewrj8Mfxo2SUsnT4u1WUlTWgB7+4HgAPB/VYz2wxUAgp4EQndnsMd3PXQel7YeYSr507gK+9exLixY1JdVlIlZQzezKqBxcALJ3htGbAMoKqqKhnliEiEuTvff2EP//TLzWSa8S+3nse7L5gSqfntiQo94M2sAPgJ8Al3bzn+dXdfDiwHqKmp8bDrEZHo2l7fxv/52UZ+v/0wb5pVxpduOY/KkrxUl5UyoQa8mWUTD/fvu/tPw2xLREavrt5+vvnkNv7z6R3kZGfwj+9cwPuWVo3KXvtgYc6iMeDbwGZ3/2pY7YjI6Pbkljr+YeVG9h7p5J2LK/nUDXOZUJib6rJGhDB78JcB7wdeMbOXg+c+7e6/DLFNERklth5q5Z9/tYUnttQxs3wsP/jIRVw6syzVZY0oYc6ieRYY3X8ficiwq2/t5t9/8zo/emkv+dmZ3H39XD542XTGZOnyFsfTJ1lFJC109PTx3ed28a2nttPV28/7L57GX109a9RNfTwdCngRGdE6e/p5cNVu7nl6O4fbe7h2/kTuvn4uM8oLUl3aiKeAF5ERqau3n++/sIdvPbWdhrZu3jSrjE+8dTYXTCtNdWlpQwEvIiNKe3cfP3xxD8uf2UFdazeXzhzPt+5YwoXVo2eJgeGigBeREaGutYvv/X4XDzy/m5auPi6aPo6v376Yi2eMT3VpaUsBLyIptb2+jXt/t4OfrK2ltz/GdedWsOyKGSyu0lDM2VLAi0jSxWLOM1vruf/53Tz5Wh3ZmRncesEUPvKmGZG+AEeyKeBFJGmaO3pZsWYvD67aza7DHZQV5PDxK8/h/ZdUU16Yk+ryIkcBLyKh21jbzAPP7+Z/1tfS1RujZlopn7xmNtcvmKQPKIVIAS8ioWhs7+FnL9fy49X72HyghdzsDN65uJI7Lp7GuZOLU13eqKCAF5Fh0x+Mra9YvZffbKqjpz/GwspivnDzudy8qJLi/GhfA3WkUcCLyFlxd7YcbGXl+v08vLaWgy1dlOZnc8fF03h3zRTmTSpKdYmjlgJeRM7IroZ2fr5+PyvX72drXRuZGcabZ5fzuZvmc9XciRpbHwEU8CKSsEMtXfx8/X5+vn4/6/c1A7C0ehxf/KMF3LCggvEFmgkzkijgReSk9h7p4NFXD/LoqwdZvbsRd1hQWcSnb5jLjedNZvIoviTeSKeAF5FjuDuvHWrl1xsP8uirh9h8IH4p5XmTivjE1bN5x6JJWskxTSjgRYS+/hjr9jbx+KZDPPrqQXYf7sAMaqaV8vdvn8e18yuoGp+f6jLlNCngRUaphrZunn6tnidfq+OZ1+tp6eojO9O4dGYZH71iJtfMn6hPl6Y5BbzIKBGLORtqm3lySx1PvVbHhtpm3KG8MIe3nVvBlXMncPmsMopyNVc9KkILeDP7DnAjUOfuC8JqR0SGdqC5k+e2Hea5bQ0883o9h9t7MIPFU0v4m7fO5sq5E5g/qYiMDF0+OYrC7MHfB3wDuD/ENkRkkOaOXp7fEQ/057Y3sKO+HYDxY8dw+awyrpo7gTfNKtd1TEeJ0ALe3Z8xs+qw9i8i8cvard3dyLPbGnhuWwOv1DYTc8gfk8lF08fxvqVVXHZOGXMmFqqXPgqlfAzezJYBywCqqqpSXI3IyNba1cua3Y28uPMIL+48woZ9zfT0x8jMMBZPLeHjV83isnPKOH9qiT5JKqkPeHdfDiwHqKmp8RSXIzKiHG7r5qVd8UB/adcRXt0f76FnZhgLK4u587JqLpo+jqXTx1Gok6NynJQHvIjEuTs7G9pZt6eJNXsaeWnnEbbWtQGQk5XB4qoS/vKqWSytHsfiqhLG5ui/r5yc/oWIpEhrVy/r9zazbk8ja/c0sm5vE00dvQAU5mRxQXUp71xSyUXTx7GgspicrMwUVyzpJsxpkj8E3gKUmdk+4LPu/u2w2hMZyWIxZ0dDG2t3N7FubyNrdzfxel0rHgxKzppQwNvmV7C4qoQl00qZWV5Apk6KylkKcxbN7WHtW2Qkc3f2HOlgw75mXqlt5pV9zWysbaa1uw+AotwsFleVcv3CCpZUlbJoagnFeRo/l+GnIRqRs+Du7GvsZMO+ZjbUNrExCPSWrniYj8nMYN6kQm46fzKLppawpKqUGWVjNWVRkkIBL5KgWMzZdbidzQdaeXV/0DuvbR4YN8/ONOZWFHHjosksrCxmYWUxsycWarqipIwCXuQE2rv72HKwhU0HWtl8oIXNB1rYcqCVzt5+ALIyjDkVhVx3bgULpxRzXmUJsysKdCJURhQFvIxq7k5tUyebBwX55gMt7D7SMXACtCg3i3mTinjvhVOZP6mI+ZOLOGdCAbnZCnMZ2RTwMmo0tHXz+qFWth5qG7jdcrBlYLwcoHp8PvMmFfGuJfGLRc+fXMTk4lzMNGYu6UcBL5HT2N7D64daeb2uja2HWuP3D7VxpL1nYJui3CxmTyzkxkWT40E+qYi5FYX68JBEiv41S1pyd46097Cjof2YXvnrh9poaOse2K4gJ4tZEwu4Zt5EZk0sYPbEQuZUFDKhMEe9cok8BbyMaD19MfYc6WB7fRs76tuD2zZ2NLQPzF6B+OqJsyYWcuWccmZPLBwI80kaXpFRTAEvI8KR9p6B8N5e3z5wu+dIB/2xP6xBV16Yw8zysdywcBIzywuYUT6Wc8oLqCzJ09xykeMo4CVpmjt72XO4g12H29l9uJ2dDR3sbHhjb3xMVgbTx49lbkUhb184iRnlY5lZXsD08rG6nJzIaVDAy7Bxd5o6eoMA73jD7eCTnAAVRblMG5/PDQsnMaNsLDMnFDCzrIDK0jytwyIyDBTwclrcnfq27qAn3sHuw+1/uG1oP2bKoRlMLs5j2vh8rltQQfX4fKaNH0v1+LFUjcsnb4zmkYuESQEvb9Dc0cvexg72HukIbjsHHu9r7KS7LzawbYbBlNJ8po3P5+bzK5k2Pp/q8WOpLstnSmm+PgwkkkIK+FGos6effY2Dwvu4IG8d1AsHKMzNYmppPudMKODKOROYOi6fqiDIK0vytNaKyAilgI+gtu4+9jd1UtvUGb9t7GRvY2fQA++goe3YsfDc7AymlOYztTSPmupSppbmM3VcXvBcPsX5OrEpko4U8GmmP+bUtXaxv6mTfY2d7G+K3x8c6C3H9cCzMozJJXlMKc3j6rkTmTouj6nj4kMoU8flUV6gD/2IRJECfoQZ6H03/iGw419d1DZ1crCl65h54QDFedkDAX7R9HFMLskb+KosyaO8MEezUkRGIQV8krg7jR29HGju5FBLFweauzjU3MXBo/eD2+PHv7MyjEkluUwuPj68c6ksyWNSSR4FWj9FRE5AyTAM+vpj1LV2c7Cli4PNwdfx91u66Bk0+wTiM1DKC3OoKM5jetlYLpkx/pje95TSPMoK1PsWkTOjgD8Jd6e5s5e61m7qWrqpa40H9aHmY3vdDW3dHDdqwpisDCYV5zKxKJfFVSVUFOVSUZz7h9viXMoLcsjK1AwUEQlHqAFvZtcBXwMygXvd/UthtpeoWMw53N5DXWsXda3d1AfhXdfazaGWroFAr2/rfkOvG+JLzcZDOo85FYVUFOdRUZQ7EOiTinMpyc/WiUsRSanQAt7MMoFvAtcA+4CXzGylu28Kq83e/hgNbUd720FoB/frBwV4Q1vPG05UQvxk5YTCHCYU5bB0+jgmFOZQXpjDxKLc4Pn4rdYMF5F0EGZSLQW2ufsOADP7EXAzMKwB7+684xvPcrC5i8PtPQOXWTvKDMaPHUN5YTyc50wsZEJRDhMKcwfCfEJhLuWFOfrUpYhESpgBXwnsHfR4H3DR8RuZ2TJgGUBVVdVpN2JmzJpQyMLKkmMC++j9soIcsjXOLSKjUMrHGtx9ObAcoKam5o3jJgn4t/eeP6w1iYhEQZhd21pg6qDHU4LnREQkCcIM+JeAWWY23czGALcBK0NsT0REBgltiMbd+8zsL4FHiU+T/I67vxpWeyIicqxQx+Dd/ZfAL8NsQ0RETkzTS0REIkoBLyISUQp4EZGIUsCLiESU+fGf7U8hM6sHdp/h28uAhmEsJx3omEcHHXP0nc3xTnP38hO9MKIC/myY2Wp3r0l1HcmkYx4ddMzRF9bxaohGRCSiFPAiIhEVpYBfnuoCUkDHPDromKMvlOONzBi8iIgcK0o9eBERGUQBLyISUWkX8GZ2nZm9ZmbbzOzuE7yeY2b/Hbz+gplVJ7/K4ZPA8f6NmW0ysw1m9lszm5aKOofTqY550Ha3mJmbWdpPp0vkmM3sPcHP+lUz+0GyaxxuCfzbrjKzJ81sXfDv+4ZU1DmczOw7ZlZnZhuHeN3M7OvB92SDmS05qwbdPW2+iC87vB2YAYwB1gPzj9vmz4F7gvu3Af+d6rpDPt4rgfzg/sfS+XgTPeZgu0LgGWAVUJPqupPwc54FrANKg8cTUl13Eo55OfCx4P58YFeq6x6G474CWAJsHOL1G4BfAQZcDLxwNu2lWw9+4ELe7t4DHL2Q92A3A98L7j8EXG1mlsQah9Mpj9fdn3T3juDhKuJXzkpnifyMAb4IfBnoSmZxIUnkmD8CfNPdGwHcvS7JNQ63RI7ZgaLgfjGwP4n1hcLdnwGOnGSTm4H7PW4VUGJmk860vXQL+BNdyLtyqG3cvQ9oBsYnpbrhl8jxDvYh4r/909kpjzn4s3Wqu/8imYWFKJGf82xgtpk9Z2arzOy6pFUXjkSO+XPAHWa2j/h1JT6enNJS6nT/z59Uyi+6LcPDzO4AaoA3p7qWMJlZBvBV4M4Ul5JsWcSHad5C/K+0Z8xsobs3pbSqcN0O3Ofu/2pmlwAPmNkCd4+lurB0kW49+EQu5D2wjZllEf/T7nBSqht+CV243MzeCnwGuMndu5NUW1hOdcyFwALgKTPbRXyccmWan2hN5Oe8D1jp7r3uvhN4nXjgp6tEjvlDwI8B3P15IJf4olxRltD/+USlW8AnciHvlcAHgvu3Ak94cPYiDZ3yeM1sMfCfxMM93cdl4RTH7O7N7l7m7tXuXk38vMNN7r46NeUOi0T+Xf+MeO8dMysjPmSzI5lFDrNEjnkPcDWAmc0jHvD1Sa0y+VYCfxLMprkYaHb3A2e6s7QaovEhLuRtZl8AVrv7SuDbxP+U20b8ZMZtqav47CR4vF8BCoAVwbnkPe5+U8qKPksJHnOkJHjMjwLXmtkmoB+4y93T9S/TRI/5b4H/MrNPEj/hemcad9YAMLMfEv9FXRacW/gskA3g7vcQP9dwA7AN6AD+9KzaS/Pvl4iIDCHdhmhERCRBCngRkYhSwIuIRJQCXkQkohTwIiIRpYCXtGBmbzGzR4L7N51ilckSM/vzM2jjc2b2v86mzuHYr5lVn2S1wXvNbH5wf1cwJx4z+/2g975vOOqW9KeAl5Qys8zTfY+7r3T3L51kkxLiq4omTfCp6dC5+4fdfdMJnr80uFsNKOAFUMBLSIKe5BYz+76ZbTazh8wsP3htl5l92czWAu82s2vN7HkzW2tmK8ysINjuumAfa4F3Ddr3nWb2jeD+RDN72MzWB1+XAl8CZprZy2b2lWC7u8zspWCN7c8P2tdnzOx1M3sWmDPEsdxnZveY2epg2xsH1bHSzJ4Afmtm48zsZ0Ebq8zsvEG7WRQc41Yz+0jw/gKLr+G/1sxeMbPBqylmDfG9e+pEyzKYWVtw90vAm4Jj/6SZPWNm5w/a7lkzW5TIz1DSnwJewjQH+A93nwe0cGyv+rC7LwF+A/w98Nbg8Wrgb8wsF/gv4B3ABUDFEG18HXja3RcRX2f7VeBuYLu7n+/ud5nZtcTXbVkKnA9cYGZXmNkFxD/pfD7xTw9eeJJjqQ7e/3bgnqA+gjZvdfc3A58H1rn7ecCngfsHvf884CrgEuAfzGwy8aWO3xkc95XAv5oNLG19su/dydwN/C449n8j/snuOwHMbDaQ6+7rE9yXpDkFvIRpr7s/F9x/ELh80Gv/HdxeTPxiDs+Z2cvE1xGaBswFdrr71uDj6Q8O0cZVwLcA3L3f3ZtPsM21wdc6YG2w71nAm4CH3b3D3Vt441oog/3Y3WPuvpX4GjBzg+cfd/ej63tfDgNUoJMAAAIYSURBVDwQ1PIEMN7Mjq5n/j/u3unuDcCTxH9ZGPBPZraB+C+6SmBisP3JvnenYwVwo5llAx8E7jvD/UgaSqu1aCTtHL8OxuDH7cGtEQ/J2wdvOHhYYRgY8M/u/p/HtfGJ09jHUMfSfvyGp/H+PwbKgQvcvdfiq2PmnmT70+buHWb2OPELSbyH+F9DMkqoBy9hqrL4Ot4QP/H37Am2WQVcZmbnAJjZ2GAoYQtQbWYzg+1uP8F7AX5L/FKFmFmmmRUDrcSXFT7qUeCDg8b2K81sAvFL/v2RmeWZWSHx4aChvNvMMoJ6ZgCvnWCb3xEPbczsLUBD8JcBwM1mlmtm44kvNvUS8aWs64Jwv5L4Xy5HJfK9O5Hjjx3gXuJDWS8dvSKUjA4KeAnTa8BfmNlmoJRgKGUwd68nPkb8w2Co4nlgrrt3AcuAXwQnWYdaCvmvgSvN7BVgDfHreh4mPuSz0cy+4u6PAT8Ang+2ewgodPe1xIeK1hO/EtZLJzmWPcCLwXZ/FtR3vM8RH9/fQPxk5wcGvbaB+NDMKuCL7r4f+D5QE9T0J8R/qR11yu/dEDYA/cEJ508CuPsa4uP4301wHxIRWk1SQmFm1cAj7r4gxaWcNTO7j/ixPJTqWs5EcEL3KeK/OHU1pFFEPXiRCDOzPwFeAD6jcB991IMXEYko9eBFRCJKAS8iElEKeBGRiFLAi4hElAJeRCSi/j90hOqtmc0bNwAAAABJRU5ErkJggg==\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "wdC4aMGgSfb7"
      },
      "source": [
        "`CategoricalCrossentropy` class and `SparseCategoricalCrossentropy` class:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "y5gH8lRxQUZ_",
        "outputId": "b5ba45f7-d78a-4b49-bb76-42d7bca26384"
      },
      "source": [
        "import tensorflow as tf\n",
        "import numpy as np\n",
        "\n",
        "# CategoricalCrossentropy\n",
        "# Computes the crossentropy loss between the labels and predictions.\n",
        "# Use this crossentropy loss function when there are two or more label classes. \n",
        "# We expect labels to be provided in a one_hot representation. \n",
        "y_true = [[0, 0, 1]]\n",
        "y_pred = [[0.05, 0.95, 0]]\n",
        "\n",
        "cce = tf.keras.losses.CategoricalCrossentropy()\n",
        "print(cce(y_true, y_pred).numpy())\n",
        "\n",
        "# SparseCategoricalCrossentropy\n",
        "# Computes the crossentropy loss between the labels and predictions.\n",
        "# Use this crossentropy loss function when there are two or more label classes. \n",
        "# We expect labels to be provided as integers.\n",
        "y_true = [[2]]\n",
        "y_pred = [[0.05, 0.95, 0]]\n",
        "\n",
        "scce = tf.keras.losses.SparseCategoricalCrossentropy()\n",
        "print(scce(y_true, y_pred).numpy())"
      ],
      "execution_count": 13,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "16.118095\n",
            "16.118097\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Hj5v3FWwUP17"
      },
      "source": [
        "# Python & NumPy utilities\n",
        "\n",
        "`tf.keras.utils.to_categorical(y, num_classes=None, dtype=\"float32\")`: Converts a class vector (integers) to binary class matrix. E.g. for use with categorical_crossentropy."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "A8IBJupEUK0w",
        "outputId": "0bae5e03-618f-4b13-c5e9-a3bc06edc405"
      },
      "source": [
        "a = tf.keras.utils.to_categorical([0, 1, 2, 3], num_classes=4)\n",
        "a = tf.constant(a, shape=[4, 4])\n",
        "\n",
        "b = tf.constant([.9, .04, .03, .03,\n",
        "                 .3, .45, .15, .13,\n",
        "                 .04, .01, .94, .05,\n",
        "                 .12, .21, .5, .17],\n",
        "                shape=[4, 4])\n",
        "\n",
        "cce = tf.keras.losses.CategoricalCrossentropy(reduction=tf.keras.losses.Reduction.NONE)\n",
        "print(cce(a, b).numpy())\n",
        "print(tf.keras.backend.categorical_crossentropy(a, b).numpy())\n",
        "print(tf.keras.losses.categorical_crossentropy(a, b).numpy())"
      ],
      "execution_count": 32,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "tf.Tensor(\n",
            "[[1. 0. 0. 0.]\n",
            " [0. 1. 0. 0.]\n",
            " [0. 0. 1. 0.]\n",
            " [0. 0. 0. 1.]], shape=(4, 4), dtype=float32)\n",
            "[0.10536048 0.8280665  0.10109606 1.7719568 ]\n",
            "[0.10536048 0.8280665  0.10109606 1.7719568 ]\n",
            "[0.10536048 0.8280665  0.10109606 1.7719568 ]\n"
          ],
          "name": "stdout"
        }
      ]
    }
  ]
}