src/demo.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "source": [
    "## Demo: On Using Quasirandom Sequences in Machine Learning for Model Weight Initialization"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "source": [
    "The following demo shows training, evaluation, and analysis of a Multi-Layer Perceptron (MLP) on the MNIST dataset, a Convolutional Neural Network (CNN) on the CIFAR-10 dataset, and an LSTM model on the IMDB dataset. The demo is based on the \"On Using Quasirandom Sequences in Machine Learning for Model Weight Initialization\" paper. The demo is divided into the following sections:\n",
    "\n",
    "1. MNIST dataset and MLP model\n",
    "2. CIFAR-10 dataset and CNN model\n",
    "3. IMDB dataset and LSTM model\n",
    "4. Analysis: Quasi-random vs. pseudo-random initialization\n",
    "\n",
    "The goal of this demo is to serve as an entry point for users and to facilitate reproducibility.\n",
    "\n",
    "Other setups can also be explored. Here are some examples of parameter values (not all combinations will work):\n",
    "* `dataset_id` is either  `mnist`, `cifar10`, or `imdb_reviews`;\n",
    "* `kernel_initializer` is either `glorot-normal`, `glorot-uniform`, `he-normal`, `he-uniform`, `lecun-normal`, `lecun-uniform`, `orthogonal`, `random-normal`, `random-uniform`, or `truncated-normal`;\n",
    "* `initializer_type` is either `quasi-random` or `pseudo-random`;\n",
    "* `model_id` is either `baseline-ann`, `baseline-cnn`, `baseline-lstm`, or `baseline-transformer`;\n",
    "* `optimizer` is etier `adam` or `sgd` (the deafult one is `adam`)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3' \n",
    "import tensorflow as tf\n",
    "tf.get_logger().setLevel('FATAL')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "from train_and_eval import run_experiment # See src/train_and_eval.py pydoc for more info\n",
    "from utils import plot_demo\n",
    "EPOCHS = 10\n",
    "SEQUENCE_SCHEME=\"auto\"\n",
    "AUTO_MIN_SEED_VALUE=1\n",
    "AUTO_MAX_SEED_VALUE=10\n",
    "AUTO_EPOCH_COUNT=1\n",
    "AUTO_SEEDS_COUNT=5\n",
    "NUM_REPS=2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### MNIST MLP -- Image Classification"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:datapipeline:Loading MNIST dataset with flatten=True and one_hot=True\n",
      "INFO:datapipeline:Converting labels to one-hot encoding\n",
      "INFO:datapipeline:Flattening images\n",
      "INFO:datapipeline:MNIST Dataset loaded\n",
      "INFO:datapipeline:x_train shape (60000, 784)\n",
      "INFO:datapipeline:y_train shape (60000, 10)\n",
      "INFO:datapipeline:x_test shape (10000, 784)\n",
      "INFO:datapipeline:y_test shape (10000, 10)\n",
      "INFO:utils:Splitting Data into Train and Validation\n",
      "INFO:utils:Training observation count is set to 60000\n",
      "INFO:utils:x_train shape before splitting (60000, 784)\n",
      "INFO:utils:y_train shape before splitting (60000, 10)\n",
      "INFO:utils:Training observation count equals the length of x_train. No validation set will be created.\n",
      "INFO:utils:x_train shape after splitting (60000, 784)\n",
      "INFO:utils:y_train shape after splitting (60000, 10)\n",
      "INFO:utils:x_val shape after splitting (0, 784)\n",
      "INFO:utils:y_val shape after splitting (0, 10)\n",
      "INFO:train_and_eval:Choosing the best seed from this set: [1, 3, 6, 7, 10]\n",
      "INFO:models:Initialized ModelBuilder with model_id=baseline-ann, input_shape=[784], output_shape=10, output_activation=softmax, model_config={'initializer_type': 'quasi-random', 'kernel_initializer': 'lecun-uniform', 'bias_initializer': 'zeros', 'recurrent_initializer': 'orthogonal', 'max_features': None, 'units': 32, 'embedding_dim': None}\n",
      "INFO:trainer:Initializing lecun-uniform from quasi-random, seed=None\n",
      "INFO:trainer:Initializing zeros from quasi-random, seed=None\n",
      "INFO:custom_initializers:Setting seed value for LecunUniformQR to 1.\n",
      "INFO:custom_initializers:Setting seed value for LecunUniformQR to 2.\n",
      "INFO:custom_initializers:Setting seed value for LecunUniformQR to 3.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m938/938\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 2ms/step - categorical_accuracy: 0.3661 - loss: 159.6252"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 1: train accuracy = -1, val accuracy = -1, test accuracy = 0.6355999708175659, time per epoch = 2.94 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m938/938\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m4s\u001b[0m 3ms/step - categorical_accuracy: 0.3663 - loss: 159.5036\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:models:Initialized ModelBuilder with model_id=baseline-ann, input_shape=[784], output_shape=10, output_activation=softmax, model_config={'initializer_type': 'quasi-random', 'kernel_initializer': 'lecun-uniform', 'bias_initializer': 'zeros', 'recurrent_initializer': 'orthogonal', 'max_features': None, 'units': 32, 'embedding_dim': None}\n",
      "INFO:trainer:Initializing lecun-uniform from quasi-random, seed=None\n",
      "INFO:trainer:Initializing zeros from quasi-random, seed=None\n",
      "INFO:custom_initializers:Setting seed value for LecunUniformQR to 3.\n",
      "INFO:custom_initializers:Setting seed value for LecunUniformQR to 4.\n",
      "INFO:custom_initializers:Setting seed value for LecunUniformQR to 5.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m938/938\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 2ms/step - categorical_accuracy: 0.4586 - loss: 2.9722"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 1: train accuracy = -1, val accuracy = -1, test accuracy = 0.7742000222206116, time per epoch = 2.88 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m938/938\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m3s\u001b[0m 3ms/step - categorical_accuracy: 0.4588 - loss: 2.9707\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:models:Initialized ModelBuilder with model_id=baseline-ann, input_shape=[784], output_shape=10, output_activation=softmax, model_config={'initializer_type': 'quasi-random', 'kernel_initializer': 'lecun-uniform', 'bias_initializer': 'zeros', 'recurrent_initializer': 'orthogonal', 'max_features': None, 'units': 32, 'embedding_dim': None}\n",
      "INFO:trainer:Initializing lecun-uniform from quasi-random, seed=None\n",
      "INFO:trainer:Initializing zeros from quasi-random, seed=None\n",
      "INFO:custom_initializers:Setting seed value for LecunUniformQR to 6.\n",
      "INFO:custom_initializers:Setting seed value for LecunUniformQR to 7.\n",
      "INFO:custom_initializers:Setting seed value for LecunUniformQR to 8.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m938/938\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 2ms/step - categorical_accuracy: 0.5305 - loss: 2.0230"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 1: train accuracy = -1, val accuracy = -1, test accuracy = 0.8489999771118164, time per epoch = 2.64 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m938/938\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m3s\u001b[0m 2ms/step - categorical_accuracy: 0.5307 - loss: 2.0220\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:models:Initialized ModelBuilder with model_id=baseline-ann, input_shape=[784], output_shape=10, output_activation=softmax, model_config={'initializer_type': 'quasi-random', 'kernel_initializer': 'lecun-uniform', 'bias_initializer': 'zeros', 'recurrent_initializer': 'orthogonal', 'max_features': None, 'units': 32, 'embedding_dim': None}\n",
      "INFO:trainer:Initializing lecun-uniform from quasi-random, seed=None\n",
      "INFO:trainer:Initializing zeros from quasi-random, seed=None\n",
      "INFO:custom_initializers:Setting seed value for LecunUniformQR to 7.\n",
      "INFO:custom_initializers:Setting seed value for LecunUniformQR to 8.\n",
      "INFO:custom_initializers:Setting seed value for LecunUniformQR to 9.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m938/938\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 2ms/step - categorical_accuracy: 0.1489 - loss: 5.9471"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 1: train accuracy = -1, val accuracy = -1, test accuracy = 0.3666999936103821, time per epoch = 2.65 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m938/938\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m3s\u001b[0m 2ms/step - categorical_accuracy: 0.1490 - loss: 5.9440\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:models:Initialized ModelBuilder with model_id=baseline-ann, input_shape=[784], output_shape=10, output_activation=softmax, model_config={'initializer_type': 'quasi-random', 'kernel_initializer': 'lecun-uniform', 'bias_initializer': 'zeros', 'recurrent_initializer': 'orthogonal', 'max_features': None, 'units': 32, 'embedding_dim': None}\n",
      "INFO:trainer:Initializing lecun-uniform from quasi-random, seed=None\n",
      "INFO:trainer:Initializing zeros from quasi-random, seed=None\n",
      "INFO:custom_initializers:Setting seed value for LecunUniformQR to 10.\n",
      "INFO:custom_initializers:Setting seed value for LecunUniformQR to 11.\n",
      "INFO:custom_initializers:Setting seed value for LecunUniformQR to 12.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m938/938\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 2ms/step - categorical_accuracy: 0.3692 - loss: 4.5230"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 1: train accuracy = -1, val accuracy = -1, test accuracy = 0.7355999946594238, time per epoch = 2.69 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m938/938\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m3s\u001b[0m 2ms/step - categorical_accuracy: 0.3694 - loss: 4.5205\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:train_and_eval:Resetting the state. Starting QRNG dimension is now set to 6.\n",
      "INFO:train_and_eval:Running experiment #1\n",
      "INFO:models:Initialized ModelBuilder with model_id=baseline-ann, input_shape=[784], output_shape=10, output_activation=softmax, model_config={'initializer_type': 'quasi-random', 'kernel_initializer': 'lecun-uniform', 'bias_initializer': 'zeros', 'recurrent_initializer': 'orthogonal', 'max_features': None, 'units': 32, 'embedding_dim': None}\n",
      "INFO:trainer:Initializing lecun-uniform from quasi-random, seed=None\n",
      "INFO:trainer:Initializing zeros from quasi-random, seed=None\n",
      "INFO:custom_initializers:Setting seed value for LecunUniformQR to 6.\n",
      "INFO:custom_initializers:Setting seed value for LecunUniformQR to 7.\n",
      "INFO:custom_initializers:Setting seed value for LecunUniformQR to 8.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"font-weight: bold\">Model: \"functional_1\"</span>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[1mModel: \"functional_1\"\u001b[0m\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓\n",
       "┃<span style=\"font-weight: bold\"> Layer (type)                    </span>┃<span style=\"font-weight: bold\"> Output Shape           </span>┃<span style=\"font-weight: bold\">       Param # </span>┃\n",
       "┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩\n",
       "│ input_layer (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">InputLayer</span>)        │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">784</span>)            │             <span style=\"color: #00af00; text-decoration-color: #00af00\">0</span> │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ dense_1 (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">Dense</span>)                 │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">32</span>)             │        <span style=\"color: #00af00; text-decoration-color: #00af00\">25,120</span> │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ dense_2 (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">Dense</span>)                 │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">32</span>)             │         <span style=\"color: #00af00; text-decoration-color: #00af00\">1,056</span> │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ predictions (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">Dense</span>)             │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">10</span>)             │           <span style=\"color: #00af00; text-decoration-color: #00af00\">330</span> │\n",
       "└─────────────────────────────────┴────────────────────────┴───────────────┘\n",
       "</pre>\n"
      ],
      "text/plain": [
       "┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓\n",
       "┃\u001b[1m \u001b[0m\u001b[1mLayer (type)                   \u001b[0m\u001b[1m \u001b[0m┃\u001b[1m \u001b[0m\u001b[1mOutput Shape          \u001b[0m\u001b[1m \u001b[0m┃\u001b[1m \u001b[0m\u001b[1m      Param #\u001b[0m\u001b[1m \u001b[0m┃\n",
       "┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩\n",
       "│ input_layer (\u001b[38;5;33mInputLayer\u001b[0m)        │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m784\u001b[0m)            │             \u001b[38;5;34m0\u001b[0m │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ dense_1 (\u001b[38;5;33mDense\u001b[0m)                 │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m32\u001b[0m)             │        \u001b[38;5;34m25,120\u001b[0m │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ dense_2 (\u001b[38;5;33mDense\u001b[0m)                 │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m32\u001b[0m)             │         \u001b[38;5;34m1,056\u001b[0m │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ predictions (\u001b[38;5;33mDense\u001b[0m)             │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m10\u001b[0m)             │           \u001b[38;5;34m330\u001b[0m │\n",
       "└─────────────────────────────────┴────────────────────────┴───────────────┘\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"font-weight: bold\"> Total params: </span><span style=\"color: #00af00; text-decoration-color: #00af00\">26,506</span> (103.54 KB)\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[1m Total params: \u001b[0m\u001b[38;5;34m26,506\u001b[0m (103.54 KB)\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"font-weight: bold\"> Trainable params: </span><span style=\"color: #00af00; text-decoration-color: #00af00\">26,506</span> (103.54 KB)\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[1m Trainable params: \u001b[0m\u001b[38;5;34m26,506\u001b[0m (103.54 KB)\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"font-weight: bold\"> Non-trainable params: </span><span style=\"color: #00af00; text-decoration-color: #00af00\">0</span> (0.00 B)\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[1m Non-trainable params: \u001b[0m\u001b[38;5;34m0\u001b[0m (0.00 B)\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/10\n",
      "\u001b[1m938/938\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 2ms/step - categorical_accuracy: 0.5244 - loss: 2.0534"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 1: train accuracy = -1, val accuracy = -1, test accuracy = 0.8503999710083008, time per epoch = 2.69 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m938/938\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m3s\u001b[0m 2ms/step - categorical_accuracy: 0.5246 - loss: 2.0525\n",
      "Epoch 2/10\n",
      "\u001b[1m906/938\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m━\u001b[0m \u001b[1m0s\u001b[0m 1ms/step - categorical_accuracy: 0.8651 - loss: 0.4735"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 2: train accuracy = -1, val accuracy = -1, test accuracy = 0.8973000049591064, time per epoch = 1.38 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m938/938\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2s\u001b[0m 2ms/step - categorical_accuracy: 0.8655 - loss: 0.4721\n",
      "Epoch 3/10\n",
      "\u001b[1m905/938\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m━\u001b[0m \u001b[1m0s\u001b[0m 1ms/step - categorical_accuracy: 0.9001 - loss: 0.3507"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 3: train accuracy = -1, val accuracy = -1, test accuracy = 0.9061999917030334, time per epoch = 1.37 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m938/938\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2s\u001b[0m 2ms/step - categorical_accuracy: 0.9001 - loss: 0.3505\n",
      "Epoch 4/10\n",
      "\u001b[1m909/938\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m━\u001b[0m \u001b[1m0s\u001b[0m 2ms/step - categorical_accuracy: 0.9149 - loss: 0.3008"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 4: train accuracy = -1, val accuracy = -1, test accuracy = 0.9164000153541565, time per epoch = 1.43 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m938/938\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2s\u001b[0m 2ms/step - categorical_accuracy: 0.9149 - loss: 0.3007\n",
      "Epoch 5/10\n",
      "\u001b[1m935/938\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m━\u001b[0m \u001b[1m0s\u001b[0m 2ms/step - categorical_accuracy: 0.9245 - loss: 0.2628"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 5: train accuracy = -1, val accuracy = -1, test accuracy = 0.9172000288963318, time per epoch = 1.49 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m938/938\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2s\u001b[0m 2ms/step - categorical_accuracy: 0.9245 - loss: 0.2628\n",
      "Epoch 6/10\n",
      "\u001b[1m934/938\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m━\u001b[0m \u001b[1m0s\u001b[0m 2ms/step - categorical_accuracy: 0.9326 - loss: 0.2354"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 6: train accuracy = -1, val accuracy = -1, test accuracy = 0.9265999794006348, time per epoch = 1.44 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m938/938\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2s\u001b[0m 2ms/step - categorical_accuracy: 0.9326 - loss: 0.2354\n",
      "Epoch 7/10\n",
      "\u001b[1m929/938\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m━\u001b[0m \u001b[1m0s\u001b[0m 2ms/step - categorical_accuracy: 0.9368 - loss: 0.2133"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 7: train accuracy = -1, val accuracy = -1, test accuracy = 0.9293000102043152, time per epoch = 1.50 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m938/938\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2s\u001b[0m 2ms/step - categorical_accuracy: 0.9368 - loss: 0.2133\n",
      "Epoch 8/10\n",
      "\u001b[1m922/938\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m━\u001b[0m \u001b[1m0s\u001b[0m 2ms/step - categorical_accuracy: 0.9422 - loss: 0.1952"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 8: train accuracy = -1, val accuracy = -1, test accuracy = 0.9315000176429749, time per epoch = 1.51 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m938/938\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2s\u001b[0m 2ms/step - categorical_accuracy: 0.9422 - loss: 0.1952\n",
      "Epoch 9/10\n",
      "\u001b[1m916/938\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m━\u001b[0m \u001b[1m0s\u001b[0m 2ms/step - categorical_accuracy: 0.9463 - loss: 0.1811"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 9: train accuracy = -1, val accuracy = -1, test accuracy = 0.9334999918937683, time per epoch = 1.52 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m938/938\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2s\u001b[0m 2ms/step - categorical_accuracy: 0.9463 - loss: 0.1810\n",
      "Epoch 10/10\n",
      "\u001b[1m914/938\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m━\u001b[0m \u001b[1m0s\u001b[0m 2ms/step - categorical_accuracy: 0.9493 - loss: 0.1713"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 10: train accuracy = -1, val accuracy = -1, test accuracy = 0.9368000030517578, time per epoch = 1.62 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m938/938\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2s\u001b[0m 2ms/step - categorical_accuracy: 0.9493 - loss: 0.1712\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:datapipeline:Loading MNIST dataset with flatten=True and one_hot=True\n",
      "INFO:datapipeline:Converting labels to one-hot encoding\n",
      "INFO:datapipeline:Flattening images\n",
      "INFO:datapipeline:MNIST Dataset loaded\n",
      "INFO:datapipeline:x_train shape (60000, 784)\n",
      "INFO:datapipeline:y_train shape (60000, 10)\n",
      "INFO:datapipeline:x_test shape (10000, 784)\n",
      "INFO:datapipeline:y_test shape (10000, 10)\n",
      "INFO:utils:Splitting Data into Train and Validation\n",
      "INFO:utils:Training observation count is set to 60000\n",
      "INFO:utils:x_train shape before splitting (60000, 784)\n",
      "INFO:utils:y_train shape before splitting (60000, 10)\n",
      "INFO:utils:Training observation count equals the length of x_train. No validation set will be created.\n",
      "INFO:utils:x_train shape after splitting (60000, 784)\n",
      "INFO:utils:y_train shape after splitting (60000, 10)\n",
      "INFO:utils:x_val shape after splitting (0, 784)\n",
      "INFO:utils:y_val shape after splitting (0, 10)\n",
      "INFO:train_and_eval:Choosing the best seed from this set: [1, 3, 4, 5, 6]\n",
      "INFO:models:Initialized ModelBuilder with model_id=baseline-ann, input_shape=[784], output_shape=10, output_activation=softmax, model_config={'initializer_type': 'quasi-random', 'kernel_initializer': 'lecun-uniform', 'bias_initializer': 'zeros', 'recurrent_initializer': 'orthogonal', 'max_features': None, 'units': 32, 'embedding_dim': None}\n",
      "INFO:trainer:Initializing lecun-uniform from quasi-random, seed=None\n",
      "INFO:trainer:Initializing zeros from quasi-random, seed=None\n",
      "INFO:custom_initializers:Setting seed value for LecunUniformQR to 1.\n",
      "INFO:custom_initializers:Setting seed value for LecunUniformQR to 2.\n",
      "INFO:custom_initializers:Setting seed value for LecunUniformQR to 3.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m938/938\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 2ms/step - categorical_accuracy: 0.3670 - loss: 159.7609"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 1: train accuracy = -1, val accuracy = -1, test accuracy = 0.6686000227928162, time per epoch = 2.95 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m938/938\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m4s\u001b[0m 3ms/step - categorical_accuracy: 0.3671 - loss: 159.6391\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:models:Initialized ModelBuilder with model_id=baseline-ann, input_shape=[784], output_shape=10, output_activation=softmax, model_config={'initializer_type': 'quasi-random', 'kernel_initializer': 'lecun-uniform', 'bias_initializer': 'zeros', 'recurrent_initializer': 'orthogonal', 'max_features': None, 'units': 32, 'embedding_dim': None}\n",
      "INFO:trainer:Initializing lecun-uniform from quasi-random, seed=None\n",
      "INFO:trainer:Initializing zeros from quasi-random, seed=None\n",
      "INFO:custom_initializers:Setting seed value for LecunUniformQR to 3.\n",
      "INFO:custom_initializers:Setting seed value for LecunUniformQR to 4.\n",
      "INFO:custom_initializers:Setting seed value for LecunUniformQR to 5.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m938/938\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 2ms/step - categorical_accuracy: 0.4559 - loss: 3.0272"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 1: train accuracy = -1, val accuracy = -1, test accuracy = 0.7724999785423279, time per epoch = 2.95 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m938/938\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m4s\u001b[0m 3ms/step - categorical_accuracy: 0.4560 - loss: 3.0258\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:models:Initialized ModelBuilder with model_id=baseline-ann, input_shape=[784], output_shape=10, output_activation=softmax, model_config={'initializer_type': 'quasi-random', 'kernel_initializer': 'lecun-uniform', 'bias_initializer': 'zeros', 'recurrent_initializer': 'orthogonal', 'max_features': None, 'units': 32, 'embedding_dim': None}\n",
      "INFO:trainer:Initializing lecun-uniform from quasi-random, seed=None\n",
      "INFO:trainer:Initializing zeros from quasi-random, seed=None\n",
      "INFO:custom_initializers:Setting seed value for LecunUniformQR to 4.\n",
      "INFO:custom_initializers:Setting seed value for LecunUniformQR to 5.\n",
      "INFO:custom_initializers:Setting seed value for LecunUniformQR to 6.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m938/938\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 2ms/step - categorical_accuracy: 0.3493 - loss: 3.8615"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 1: train accuracy = -1, val accuracy = -1, test accuracy = 0.7049999833106995, time per epoch = 2.93 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m938/938\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m4s\u001b[0m 3ms/step - categorical_accuracy: 0.3495 - loss: 3.8596\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:models:Initialized ModelBuilder with model_id=baseline-ann, input_shape=[784], output_shape=10, output_activation=softmax, model_config={'initializer_type': 'quasi-random', 'kernel_initializer': 'lecun-uniform', 'bias_initializer': 'zeros', 'recurrent_initializer': 'orthogonal', 'max_features': None, 'units': 32, 'embedding_dim': None}\n",
      "INFO:trainer:Initializing lecun-uniform from quasi-random, seed=None\n",
      "INFO:trainer:Initializing zeros from quasi-random, seed=None\n",
      "INFO:custom_initializers:Setting seed value for LecunUniformQR to 5.\n",
      "INFO:custom_initializers:Setting seed value for LecunUniformQR to 6.\n",
      "INFO:custom_initializers:Setting seed value for LecunUniformQR to 7.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m938/938\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 2ms/step - categorical_accuracy: 0.3601 - loss: 6.9961"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 1: train accuracy = -1, val accuracy = -1, test accuracy = 0.7387999892234802, time per epoch = 2.97 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m938/938\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m4s\u001b[0m 3ms/step - categorical_accuracy: 0.3603 - loss: 6.9918\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:models:Initialized ModelBuilder with model_id=baseline-ann, input_shape=[784], output_shape=10, output_activation=softmax, model_config={'initializer_type': 'quasi-random', 'kernel_initializer': 'lecun-uniform', 'bias_initializer': 'zeros', 'recurrent_initializer': 'orthogonal', 'max_features': None, 'units': 32, 'embedding_dim': None}\n",
      "INFO:trainer:Initializing lecun-uniform from quasi-random, seed=None\n",
      "INFO:trainer:Initializing zeros from quasi-random, seed=None\n",
      "INFO:custom_initializers:Setting seed value for LecunUniformQR to 6.\n",
      "INFO:custom_initializers:Setting seed value for LecunUniformQR to 7.\n",
      "INFO:custom_initializers:Setting seed value for LecunUniformQR to 8.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m938/938\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 2ms/step - categorical_accuracy: 0.5205 - loss: 2.0558"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 1: train accuracy = -1, val accuracy = -1, test accuracy = 0.839900016784668, time per epoch = 2.96 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m938/938\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m4s\u001b[0m 3ms/step - categorical_accuracy: 0.5207 - loss: 2.0548\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:train_and_eval:Resetting the state. Starting QRNG dimension is now set to 6.\n",
      "INFO:train_and_eval:Running experiment #1\n",
      "INFO:models:Initialized ModelBuilder with model_id=baseline-ann, input_shape=[784], output_shape=10, output_activation=softmax, model_config={'initializer_type': 'quasi-random', 'kernel_initializer': 'lecun-uniform', 'bias_initializer': 'zeros', 'recurrent_initializer': 'orthogonal', 'max_features': None, 'units': 32, 'embedding_dim': None}\n",
      "INFO:trainer:Initializing lecun-uniform from quasi-random, seed=None\n",
      "INFO:trainer:Initializing zeros from quasi-random, seed=None\n",
      "INFO:custom_initializers:Setting seed value for LecunUniformQR to 6.\n",
      "INFO:custom_initializers:Setting seed value for LecunUniformQR to 7.\n",
      "INFO:custom_initializers:Setting seed value for LecunUniformQR to 8.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"font-weight: bold\">Model: \"functional_1\"</span>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[1mModel: \"functional_1\"\u001b[0m\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓\n",
       "┃<span style=\"font-weight: bold\"> Layer (type)                    </span>┃<span style=\"font-weight: bold\"> Output Shape           </span>┃<span style=\"font-weight: bold\">       Param # </span>┃\n",
       "┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩\n",
       "│ input_layer (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">InputLayer</span>)        │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">784</span>)            │             <span style=\"color: #00af00; text-decoration-color: #00af00\">0</span> │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ dense_1 (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">Dense</span>)                 │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">32</span>)             │        <span style=\"color: #00af00; text-decoration-color: #00af00\">25,120</span> │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ dense_2 (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">Dense</span>)                 │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">32</span>)             │         <span style=\"color: #00af00; text-decoration-color: #00af00\">1,056</span> │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ predictions (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">Dense</span>)             │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">10</span>)             │           <span style=\"color: #00af00; text-decoration-color: #00af00\">330</span> │\n",
       "└─────────────────────────────────┴────────────────────────┴───────────────┘\n",
       "</pre>\n"
      ],
      "text/plain": [
       "┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓\n",
       "┃\u001b[1m \u001b[0m\u001b[1mLayer (type)                   \u001b[0m\u001b[1m \u001b[0m┃\u001b[1m \u001b[0m\u001b[1mOutput Shape          \u001b[0m\u001b[1m \u001b[0m┃\u001b[1m \u001b[0m\u001b[1m      Param #\u001b[0m\u001b[1m \u001b[0m┃\n",
       "┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩\n",
       "│ input_layer (\u001b[38;5;33mInputLayer\u001b[0m)        │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m784\u001b[0m)            │             \u001b[38;5;34m0\u001b[0m │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ dense_1 (\u001b[38;5;33mDense\u001b[0m)                 │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m32\u001b[0m)             │        \u001b[38;5;34m25,120\u001b[0m │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ dense_2 (\u001b[38;5;33mDense\u001b[0m)                 │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m32\u001b[0m)             │         \u001b[38;5;34m1,056\u001b[0m │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ predictions (\u001b[38;5;33mDense\u001b[0m)             │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m10\u001b[0m)             │           \u001b[38;5;34m330\u001b[0m │\n",
       "└─────────────────────────────────┴────────────────────────┴───────────────┘\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"font-weight: bold\"> Total params: </span><span style=\"color: #00af00; text-decoration-color: #00af00\">26,506</span> (103.54 KB)\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[1m Total params: \u001b[0m\u001b[38;5;34m26,506\u001b[0m (103.54 KB)\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"font-weight: bold\"> Trainable params: </span><span style=\"color: #00af00; text-decoration-color: #00af00\">26,506</span> (103.54 KB)\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[1m Trainable params: \u001b[0m\u001b[38;5;34m26,506\u001b[0m (103.54 KB)\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"font-weight: bold\"> Non-trainable params: </span><span style=\"color: #00af00; text-decoration-color: #00af00\">0</span> (0.00 B)\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[1m Non-trainable params: \u001b[0m\u001b[38;5;34m0\u001b[0m (0.00 B)\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/10\n",
      "\u001b[1m938/938\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 2ms/step - categorical_accuracy: 0.5287 - loss: 2.0603"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 1: train accuracy = -1, val accuracy = -1, test accuracy = 0.8482000231742859, time per epoch = 2.88 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m938/938\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m3s\u001b[0m 3ms/step - categorical_accuracy: 0.5289 - loss: 2.0593\n",
      "Epoch 2/10\n",
      "\u001b[1m931/938\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m━\u001b[0m \u001b[1m0s\u001b[0m 2ms/step - categorical_accuracy: 0.8648 - loss: 0.4823"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 2: train accuracy = -1, val accuracy = -1, test accuracy = 0.8873999714851379, time per epoch = 1.50 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m938/938\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2s\u001b[0m 2ms/step - categorical_accuracy: 0.8649 - loss: 0.4820\n",
      "Epoch 3/10\n",
      "\u001b[1m929/938\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m━\u001b[0m \u001b[1m0s\u001b[0m 2ms/step - categorical_accuracy: 0.8992 - loss: 0.3562"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 3: train accuracy = -1, val accuracy = -1, test accuracy = 0.9070000052452087, time per epoch = 1.50 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m938/938\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2s\u001b[0m 2ms/step - categorical_accuracy: 0.8993 - loss: 0.3561\n",
      "Epoch 4/10\n",
      "\u001b[1m925/938\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m━\u001b[0m \u001b[1m0s\u001b[0m 2ms/step - categorical_accuracy: 0.9149 - loss: 0.3059"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 4: train accuracy = -1, val accuracy = -1, test accuracy = 0.9168999791145325, time per epoch = 1.50 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m938/938\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2s\u001b[0m 2ms/step - categorical_accuracy: 0.9149 - loss: 0.3057\n",
      "Epoch 5/10\n",
      "\u001b[1m910/938\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m━\u001b[0m \u001b[1m0s\u001b[0m 2ms/step - categorical_accuracy: 0.9237 - loss: 0.2691"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 5: train accuracy = -1, val accuracy = -1, test accuracy = 0.9199000000953674, time per epoch = 1.52 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m938/938\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2s\u001b[0m 2ms/step - categorical_accuracy: 0.9238 - loss: 0.2689\n",
      "Epoch 6/10\n",
      "\u001b[1m930/938\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m━\u001b[0m \u001b[1m0s\u001b[0m 2ms/step - categorical_accuracy: 0.9303 - loss: 0.2421"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 6: train accuracy = -1, val accuracy = -1, test accuracy = 0.9204000234603882, time per epoch = 1.49 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m938/938\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2s\u001b[0m 2ms/step - categorical_accuracy: 0.9304 - loss: 0.2420\n",
      "Epoch 7/10\n",
      "\u001b[1m928/938\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m━\u001b[0m \u001b[1m0s\u001b[0m 2ms/step - categorical_accuracy: 0.9365 - loss: 0.2183"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 7: train accuracy = -1, val accuracy = -1, test accuracy = 0.9240999817848206, time per epoch = 1.51 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m938/938\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2s\u001b[0m 2ms/step - categorical_accuracy: 0.9365 - loss: 0.2182\n",
      "Epoch 8/10\n",
      "\u001b[1m923/938\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m━\u001b[0m \u001b[1m0s\u001b[0m 2ms/step - categorical_accuracy: 0.9421 - loss: 0.2022"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 8: train accuracy = -1, val accuracy = -1, test accuracy = 0.9236999750137329, time per epoch = 1.51 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m938/938\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2s\u001b[0m 2ms/step - categorical_accuracy: 0.9422 - loss: 0.2021\n",
      "Epoch 9/10\n",
      "\u001b[1m913/938\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m━\u001b[0m \u001b[1m0s\u001b[0m 2ms/step - categorical_accuracy: 0.9456 - loss: 0.1862"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 9: train accuracy = -1, val accuracy = -1, test accuracy = 0.9340999722480774, time per epoch = 1.52 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m938/938\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2s\u001b[0m 2ms/step - categorical_accuracy: 0.9456 - loss: 0.1861\n",
      "Epoch 10/10\n",
      "\u001b[1m918/938\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m━\u001b[0m \u001b[1m0s\u001b[0m 2ms/step - categorical_accuracy: 0.9493 - loss: 0.1728"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 10: train accuracy = -1, val accuracy = -1, test accuracy = 0.939300000667572, time per epoch = 1.51 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m938/938\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2s\u001b[0m 2ms/step - categorical_accuracy: 0.9493 - loss: 0.1727\n"
     ]
    }
   ],
   "source": [
    "mlp_quasi_stats = [run_experiment(\n",
    "    experiment_id=1,\n",
    "    sequence_scheme=SEQUENCE_SCHEME,\n",
    "    units=32,\n",
    "    dataset_id=\"mnist\",\n",
    "    kernel_initializer=\"lecun-uniform\",\n",
    "    initializer_type=\"quasi-random\",\n",
    "    model_id=\"baseline-ann\",\n",
    "    epochs=EPOCHS,\n",
    "    auto_epoch_count=AUTO_EPOCH_COUNT,\n",
    "    auto_min_seed_value=AUTO_MIN_SEED_VALUE,\n",
    "    auto_max_seed_value=AUTO_MAX_SEED_VALUE,\n",
    "    auto_seeds_count=AUTO_SEEDS_COUNT,\n",
    ")[0] for _ in range(NUM_REPS)]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:datapipeline:Loading MNIST dataset with flatten=True and one_hot=True\n",
      "INFO:datapipeline:Converting labels to one-hot encoding\n",
      "INFO:datapipeline:Flattening images\n",
      "INFO:datapipeline:MNIST Dataset loaded\n",
      "INFO:datapipeline:x_train shape (60000, 784)\n",
      "INFO:datapipeline:y_train shape (60000, 10)\n",
      "INFO:datapipeline:x_test shape (10000, 784)\n",
      "INFO:datapipeline:y_test shape (10000, 10)\n",
      "INFO:utils:Splitting Data into Train and Validation\n",
      "INFO:utils:Training observation count is set to 60000\n",
      "INFO:utils:x_train shape before splitting (60000, 784)\n",
      "INFO:utils:y_train shape before splitting (60000, 10)\n",
      "INFO:utils:Training observation count equals the length of x_train. No validation set will be created.\n",
      "INFO:utils:x_train shape after splitting (60000, 784)\n",
      "INFO:utils:y_train shape after splitting (60000, 10)\n",
      "INFO:utils:x_val shape after splitting (0, 784)\n",
      "INFO:utils:y_val shape after splitting (0, 10)\n",
      "INFO:train_and_eval:Seeds for this experiment will be selected at random. Setting sequence_scheme to random.\n",
      "INFO:train_and_eval:Running experiment #1\n",
      "INFO:models:Initialized ModelBuilder with model_id=baseline-ann, input_shape=[784], output_shape=10, output_activation=softmax, model_config={'initializer_type': 'pseudo-random', 'kernel_initializer': 'lecun-uniform', 'bias_initializer': 'zeros', 'recurrent_initializer': 'orthogonal', 'max_features': None, 'units': 64, 'embedding_dim': None}\n",
      "INFO:trainer:Initializing lecun-uniform from pseudo-random, seed=None\n",
      "INFO:trainer:Initializing zeros from pseudo-random, seed=None\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"font-weight: bold\">Model: \"functional_3\"</span>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[1mModel: \"functional_3\"\u001b[0m\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓\n",
       "┃<span style=\"font-weight: bold\"> Layer (type)                    </span>┃<span style=\"font-weight: bold\"> Output Shape           </span>┃<span style=\"font-weight: bold\">       Param # </span>┃\n",
       "┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩\n",
       "│ input_layer (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">InputLayer</span>)        │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">784</span>)            │             <span style=\"color: #00af00; text-decoration-color: #00af00\">0</span> │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ dense_1 (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">Dense</span>)                 │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">64</span>)             │        <span style=\"color: #00af00; text-decoration-color: #00af00\">50,240</span> │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ dense_2 (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">Dense</span>)                 │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">64</span>)             │         <span style=\"color: #00af00; text-decoration-color: #00af00\">4,160</span> │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ predictions (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">Dense</span>)             │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">10</span>)             │           <span style=\"color: #00af00; text-decoration-color: #00af00\">650</span> │\n",
       "└─────────────────────────────────┴────────────────────────┴───────────────┘\n",
       "</pre>\n"
      ],
      "text/plain": [
       "┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓\n",
       "┃\u001b[1m \u001b[0m\u001b[1mLayer (type)                   \u001b[0m\u001b[1m \u001b[0m┃\u001b[1m \u001b[0m\u001b[1mOutput Shape          \u001b[0m\u001b[1m \u001b[0m┃\u001b[1m \u001b[0m\u001b[1m      Param #\u001b[0m\u001b[1m \u001b[0m┃\n",
       "┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩\n",
       "│ input_layer (\u001b[38;5;33mInputLayer\u001b[0m)        │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m784\u001b[0m)            │             \u001b[38;5;34m0\u001b[0m │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ dense_1 (\u001b[38;5;33mDense\u001b[0m)                 │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m64\u001b[0m)             │        \u001b[38;5;34m50,240\u001b[0m │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ dense_2 (\u001b[38;5;33mDense\u001b[0m)                 │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m64\u001b[0m)             │         \u001b[38;5;34m4,160\u001b[0m │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ predictions (\u001b[38;5;33mDense\u001b[0m)             │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m10\u001b[0m)             │           \u001b[38;5;34m650\u001b[0m │\n",
       "└─────────────────────────────────┴────────────────────────┴───────────────┘\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"font-weight: bold\"> Total params: </span><span style=\"color: #00af00; text-decoration-color: #00af00\">55,050</span> (215.04 KB)\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[1m Total params: \u001b[0m\u001b[38;5;34m55,050\u001b[0m (215.04 KB)\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"font-weight: bold\"> Trainable params: </span><span style=\"color: #00af00; text-decoration-color: #00af00\">55,050</span> (215.04 KB)\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[1m Trainable params: \u001b[0m\u001b[38;5;34m55,050\u001b[0m (215.04 KB)\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"font-weight: bold\"> Non-trainable params: </span><span style=\"color: #00af00; text-decoration-color: #00af00\">0</span> (0.00 B)\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[1m Non-trainable params: \u001b[0m\u001b[38;5;34m0\u001b[0m (0.00 B)\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/10\n",
      "\u001b[1m938/938\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 2ms/step - categorical_accuracy: 0.5503 - loss: 9.9638"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 1: train accuracy = -1, val accuracy = -1, test accuracy = 0.8030999898910522, time per epoch = 2.87 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m938/938\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m3s\u001b[0m 3ms/step - categorical_accuracy: 0.5504 - loss: 9.9577\n",
      "Epoch 2/10\n",
      "\u001b[1m911/938\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m━\u001b[0m \u001b[1m0s\u001b[0m 2ms/step - categorical_accuracy: 0.8207 - loss: 1.0386"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 2: train accuracy = -1, val accuracy = -1, test accuracy = 0.8568000197410583, time per epoch = 1.53 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m938/938\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2s\u001b[0m 2ms/step - categorical_accuracy: 0.8210 - loss: 1.0345\n",
      "Epoch 3/10\n",
      "\u001b[1m929/938\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m━\u001b[0m \u001b[1m0s\u001b[0m 2ms/step - categorical_accuracy: 0.8687 - loss: 0.6326"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 3: train accuracy = -1, val accuracy = -1, test accuracy = 0.8865000009536743, time per epoch = 1.55 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m938/938\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2s\u001b[0m 2ms/step - categorical_accuracy: 0.8687 - loss: 0.6320\n",
      "Epoch 4/10\n",
      "\u001b[1m924/938\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m━\u001b[0m \u001b[1m0s\u001b[0m 2ms/step - categorical_accuracy: 0.8974 - loss: 0.4568"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 4: train accuracy = -1, val accuracy = -1, test accuracy = 0.900600016117096, time per epoch = 1.51 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m938/938\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2s\u001b[0m 2ms/step - categorical_accuracy: 0.8975 - loss: 0.4562\n",
      "Epoch 5/10\n",
      "\u001b[1m932/938\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m━\u001b[0m \u001b[1m0s\u001b[0m 2ms/step - categorical_accuracy: 0.9153 - loss: 0.3558"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 5: train accuracy = -1, val accuracy = -1, test accuracy = 0.9081000089645386, time per epoch = 1.54 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m938/938\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2s\u001b[0m 2ms/step - categorical_accuracy: 0.9153 - loss: 0.3556\n",
      "Epoch 6/10\n",
      "\u001b[1m935/938\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m━\u001b[0m \u001b[1m0s\u001b[0m 2ms/step - categorical_accuracy: 0.9278 - loss: 0.2873"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 6: train accuracy = -1, val accuracy = -1, test accuracy = 0.9169999957084656, time per epoch = 1.54 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m938/938\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2s\u001b[0m 2ms/step - categorical_accuracy: 0.9278 - loss: 0.2872\n",
      "Epoch 7/10\n",
      "\u001b[1m912/938\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m━\u001b[0m \u001b[1m0s\u001b[0m 2ms/step - categorical_accuracy: 0.9384 - loss: 0.2422"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 7: train accuracy = -1, val accuracy = -1, test accuracy = 0.9229999780654907, time per epoch = 1.52 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m938/938\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2s\u001b[0m 2ms/step - categorical_accuracy: 0.9384 - loss: 0.2419\n",
      "Epoch 8/10\n",
      "\u001b[1m926/938\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m━\u001b[0m \u001b[1m0s\u001b[0m 2ms/step - categorical_accuracy: 0.9454 - loss: 0.2036"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 8: train accuracy = -1, val accuracy = -1, test accuracy = 0.926800012588501, time per epoch = 1.50 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m938/938\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2s\u001b[0m 2ms/step - categorical_accuracy: 0.9454 - loss: 0.2035\n",
      "Epoch 9/10\n",
      "\u001b[1m930/938\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m━\u001b[0m \u001b[1m0s\u001b[0m 2ms/step - categorical_accuracy: 0.9510 - loss: 0.1775"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 9: train accuracy = -1, val accuracy = -1, test accuracy = 0.9316999912261963, time per epoch = 1.49 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m938/938\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2s\u001b[0m 2ms/step - categorical_accuracy: 0.9511 - loss: 0.1774\n",
      "Epoch 10/10\n",
      "\u001b[1m919/938\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m━\u001b[0m \u001b[1m0s\u001b[0m 2ms/step - categorical_accuracy: 0.9570 - loss: 0.1556"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 10: train accuracy = -1, val accuracy = -1, test accuracy = 0.9337000250816345, time per epoch = 1.51 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m938/938\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2s\u001b[0m 2ms/step - categorical_accuracy: 0.9570 - loss: 0.1555\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:datapipeline:Loading MNIST dataset with flatten=True and one_hot=True\n",
      "INFO:datapipeline:Converting labels to one-hot encoding\n",
      "INFO:datapipeline:Flattening images\n",
      "INFO:datapipeline:MNIST Dataset loaded\n",
      "INFO:datapipeline:x_train shape (60000, 784)\n",
      "INFO:datapipeline:y_train shape (60000, 10)\n",
      "INFO:datapipeline:x_test shape (10000, 784)\n",
      "INFO:datapipeline:y_test shape (10000, 10)\n",
      "INFO:utils:Splitting Data into Train and Validation\n",
      "INFO:utils:Training observation count is set to 60000\n",
      "INFO:utils:x_train shape before splitting (60000, 784)\n",
      "INFO:utils:y_train shape before splitting (60000, 10)\n",
      "INFO:utils:Training observation count equals the length of x_train. No validation set will be created.\n",
      "INFO:utils:x_train shape after splitting (60000, 784)\n",
      "INFO:utils:y_train shape after splitting (60000, 10)\n",
      "INFO:utils:x_val shape after splitting (0, 784)\n",
      "INFO:utils:y_val shape after splitting (0, 10)\n",
      "INFO:train_and_eval:Seeds for this experiment will be selected at random. Setting sequence_scheme to random.\n",
      "INFO:train_and_eval:Running experiment #1\n",
      "INFO:models:Initialized ModelBuilder with model_id=baseline-ann, input_shape=[784], output_shape=10, output_activation=softmax, model_config={'initializer_type': 'pseudo-random', 'kernel_initializer': 'lecun-uniform', 'bias_initializer': 'zeros', 'recurrent_initializer': 'orthogonal', 'max_features': None, 'units': 64, 'embedding_dim': None}\n",
      "INFO:trainer:Initializing lecun-uniform from pseudo-random, seed=None\n",
      "INFO:trainer:Initializing zeros from pseudo-random, seed=None\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"font-weight: bold\">Model: \"functional_5\"</span>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[1mModel: \"functional_5\"\u001b[0m\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓\n",
       "┃<span style=\"font-weight: bold\"> Layer (type)                    </span>┃<span style=\"font-weight: bold\"> Output Shape           </span>┃<span style=\"font-weight: bold\">       Param # </span>┃\n",
       "┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩\n",
       "│ input_layer (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">InputLayer</span>)        │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">784</span>)            │             <span style=\"color: #00af00; text-decoration-color: #00af00\">0</span> │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ dense_1 (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">Dense</span>)                 │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">64</span>)             │        <span style=\"color: #00af00; text-decoration-color: #00af00\">50,240</span> │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ dense_2 (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">Dense</span>)                 │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">64</span>)             │         <span style=\"color: #00af00; text-decoration-color: #00af00\">4,160</span> │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ predictions (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">Dense</span>)             │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">10</span>)             │           <span style=\"color: #00af00; text-decoration-color: #00af00\">650</span> │\n",
       "└─────────────────────────────────┴────────────────────────┴───────────────┘\n",
       "</pre>\n"
      ],
      "text/plain": [
       "┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓\n",
       "┃\u001b[1m \u001b[0m\u001b[1mLayer (type)                   \u001b[0m\u001b[1m \u001b[0m┃\u001b[1m \u001b[0m\u001b[1mOutput Shape          \u001b[0m\u001b[1m \u001b[0m┃\u001b[1m \u001b[0m\u001b[1m      Param #\u001b[0m\u001b[1m \u001b[0m┃\n",
       "┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩\n",
       "│ input_layer (\u001b[38;5;33mInputLayer\u001b[0m)        │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m784\u001b[0m)            │             \u001b[38;5;34m0\u001b[0m │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ dense_1 (\u001b[38;5;33mDense\u001b[0m)                 │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m64\u001b[0m)             │        \u001b[38;5;34m50,240\u001b[0m │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ dense_2 (\u001b[38;5;33mDense\u001b[0m)                 │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m64\u001b[0m)             │         \u001b[38;5;34m4,160\u001b[0m │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ predictions (\u001b[38;5;33mDense\u001b[0m)             │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m10\u001b[0m)             │           \u001b[38;5;34m650\u001b[0m │\n",
       "└─────────────────────────────────┴────────────────────────┴───────────────┘\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"font-weight: bold\"> Total params: </span><span style=\"color: #00af00; text-decoration-color: #00af00\">55,050</span> (215.04 KB)\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[1m Total params: \u001b[0m\u001b[38;5;34m55,050\u001b[0m (215.04 KB)\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"font-weight: bold\"> Trainable params: </span><span style=\"color: #00af00; text-decoration-color: #00af00\">55,050</span> (215.04 KB)\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[1m Trainable params: \u001b[0m\u001b[38;5;34m55,050\u001b[0m (215.04 KB)\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"font-weight: bold\"> Non-trainable params: </span><span style=\"color: #00af00; text-decoration-color: #00af00\">0</span> (0.00 B)\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[1m Non-trainable params: \u001b[0m\u001b[38;5;34m0\u001b[0m (0.00 B)\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/10\n",
      "\u001b[1m938/938\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 2ms/step - categorical_accuracy: 0.5728 - loss: 10.9389"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 1: train accuracy = -1, val accuracy = -1, test accuracy = 0.8392000198364258, time per epoch = 2.93 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m938/938\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m4s\u001b[0m 3ms/step - categorical_accuracy: 0.5729 - loss: 10.9326\n",
      "Epoch 2/10\n",
      "\u001b[1m920/938\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m━\u001b[0m \u001b[1m0s\u001b[0m 2ms/step - categorical_accuracy: 0.8568 - loss: 1.4420"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 2: train accuracy = -1, val accuracy = -1, test accuracy = 0.8783000111579895, time per epoch = 1.51 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m938/938\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2s\u001b[0m 2ms/step - categorical_accuracy: 0.8570 - loss: 1.4379\n",
      "Epoch 3/10\n",
      "\u001b[1m931/938\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m━\u001b[0m \u001b[1m0s\u001b[0m 2ms/step - categorical_accuracy: 0.8890 - loss: 0.8454"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 3: train accuracy = -1, val accuracy = -1, test accuracy = 0.8986999988555908, time per epoch = 1.49 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m938/938\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2s\u001b[0m 2ms/step - categorical_accuracy: 0.8891 - loss: 0.8447\n",
      "Epoch 4/10\n",
      "\u001b[1m922/938\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m━\u001b[0m \u001b[1m0s\u001b[0m 2ms/step - categorical_accuracy: 0.9092 - loss: 0.5857"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 4: train accuracy = -1, val accuracy = -1, test accuracy = 0.9110999703407288, time per epoch = 1.50 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m938/938\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2s\u001b[0m 2ms/step - categorical_accuracy: 0.9093 - loss: 0.5848\n",
      "Epoch 5/10\n",
      "\u001b[1m918/938\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m━\u001b[0m \u001b[1m0s\u001b[0m 2ms/step - categorical_accuracy: 0.9232 - loss: 0.4307"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 5: train accuracy = -1, val accuracy = -1, test accuracy = 0.9157999753952026, time per epoch = 1.51 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m938/938\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2s\u001b[0m 2ms/step - categorical_accuracy: 0.9232 - loss: 0.4301\n",
      "Epoch 6/10\n",
      "\u001b[1m919/938\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m━\u001b[0m \u001b[1m0s\u001b[0m 2ms/step - categorical_accuracy: 0.9336 - loss: 0.3386"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 6: train accuracy = -1, val accuracy = -1, test accuracy = 0.9197999835014343, time per epoch = 1.51 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m938/938\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2s\u001b[0m 2ms/step - categorical_accuracy: 0.9337 - loss: 0.3381\n",
      "Epoch 7/10\n",
      "\u001b[1m937/938\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m━\u001b[0m \u001b[1m0s\u001b[0m 2ms/step - categorical_accuracy: 0.9435 - loss: 0.2665"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 7: train accuracy = -1, val accuracy = -1, test accuracy = 0.9273999929428101, time per epoch = 1.54 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m938/938\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2s\u001b[0m 2ms/step - categorical_accuracy: 0.9435 - loss: 0.2664\n",
      "Epoch 8/10\n",
      "\u001b[1m922/938\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m━\u001b[0m \u001b[1m0s\u001b[0m 2ms/step - categorical_accuracy: 0.9507 - loss: 0.2159"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 8: train accuracy = -1, val accuracy = -1, test accuracy = 0.9294999837875366, time per epoch = 1.51 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m938/938\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2s\u001b[0m 2ms/step - categorical_accuracy: 0.9507 - loss: 0.2158\n",
      "Epoch 9/10\n",
      "\u001b[1m922/938\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m━\u001b[0m \u001b[1m0s\u001b[0m 2ms/step - categorical_accuracy: 0.9558 - loss: 0.1801"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 9: train accuracy = -1, val accuracy = -1, test accuracy = 0.9312999844551086, time per epoch = 1.50 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m938/938\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2s\u001b[0m 2ms/step - categorical_accuracy: 0.9559 - loss: 0.1799\n",
      "Epoch 10/10\n",
      "\u001b[1m921/938\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m━\u001b[0m \u001b[1m0s\u001b[0m 2ms/step - categorical_accuracy: 0.9630 - loss: 0.1516"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 10: train accuracy = -1, val accuracy = -1, test accuracy = 0.934499979019165, time per epoch = 1.51 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m938/938\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2s\u001b[0m 2ms/step - categorical_accuracy: 0.9630 - loss: 0.1515\n"
     ]
    }
   ],
   "source": [
    "mlp_pseudo_stats = [run_experiment(\n",
    "    experiment_id=1,\n",
    "    sequence_scheme=SEQUENCE_SCHEME,\n",
    "    units=64,\n",
    "    dataset_id=\"mnist\",\n",
    "    kernel_initializer=\"lecun-uniform\",\n",
    "    initializer_type=\"pseudo-random\",\n",
    "    model_id=\"baseline-ann\",\n",
    "    epochs=EPOCHS,\n",
    "    auto_epoch_count=AUTO_EPOCH_COUNT,\n",
    "    auto_min_seed_value=AUTO_MIN_SEED_VALUE,\n",
    "    auto_max_seed_value=AUTO_MAX_SEED_VALUE,\n",
    "    auto_seeds_count=AUTO_SEEDS_COUNT,\n",
    ")[0] for _ in range(NUM_REPS)]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "source": [
    "### CIFAR-10 CNN -- Image Classification"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:datapipeline:Loading CIFAR10 dataset with flatten=False and one_hot=True\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading data from https://www.cs.toronto.edu/~kriz/cifar-10-python.tar.gz\n",
      "\u001b[1m170498071/170498071\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m4s\u001b[0m 0us/step\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:datapipeline:Converting labels to one-hot encoding\n",
      "INFO:datapipeline:CIFAR10 Dataset loaded\n",
      "INFO:datapipeline:x_train shape (50000, 32, 32, 3)\n",
      "INFO:datapipeline:y_train shape (50000, 10)\n",
      "INFO:datapipeline:x_test shape (10000, 32, 32, 3)\n",
      "INFO:datapipeline:y_test shape (10000, 10)\n",
      "INFO:utils:Splitting Data into Train and Validation\n",
      "INFO:utils:Training observation count is set to 50000\n",
      "INFO:utils:x_train shape before splitting (50000, 32, 32, 3)\n",
      "INFO:utils:y_train shape before splitting (50000, 10)\n",
      "INFO:utils:Training observation count equals the length of x_train. No validation set will be created.\n",
      "INFO:utils:x_train shape after splitting (50000, 32, 32, 3)\n",
      "INFO:utils:y_train shape after splitting (50000, 10)\n",
      "INFO:utils:x_val shape after splitting (0, 32, 32, 3)\n",
      "INFO:utils:y_val shape after splitting (0, 10)\n",
      "INFO:train_and_eval:Choosing the best seed from this set: [1, 3, 7, 8, 9]\n",
      "INFO:models:Initialized ModelBuilder with model_id=baseline-cnn, input_shape=[32, 32, 3], output_shape=10, output_activation=softmax, model_config={'initializer_type': 'quasi-random', 'kernel_initializer': 'lecun-normal', 'bias_initializer': 'zeros', 'recurrent_initializer': 'orthogonal', 'max_features': None, 'units': 32, 'embedding_dim': None}\n",
      "INFO:trainer:Initializing lecun-normal from quasi-random, seed=None\n",
      "INFO:trainer:Initializing zeros from quasi-random, seed=None\n",
      "INFO:custom_initializers:Setting seed value for LecunNormalQR to 1.\n",
      "INFO:custom_initializers:Setting seed value for LecunNormalQR to 2.\n",
      "INFO:custom_initializers:Setting seed value for LecunNormalQR to 3.\n",
      "INFO:custom_initializers:Setting seed value for LecunNormalQR to 4.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m782/782\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 4ms/step - categorical_accuracy: 0.1028 - loss: 12.1827"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 1: train accuracy = -1, val accuracy = -1, test accuracy = 0.10010000318288803, time per epoch = 5.90 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m782/782\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m7s\u001b[0m 6ms/step - categorical_accuracy: 0.1028 - loss: 12.1727\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:models:Initialized ModelBuilder with model_id=baseline-cnn, input_shape=[32, 32, 3], output_shape=10, output_activation=softmax, model_config={'initializer_type': 'quasi-random', 'kernel_initializer': 'lecun-normal', 'bias_initializer': 'zeros', 'recurrent_initializer': 'orthogonal', 'max_features': None, 'units': 32, 'embedding_dim': None}\n",
      "INFO:trainer:Initializing lecun-normal from quasi-random, seed=None\n",
      "INFO:trainer:Initializing zeros from quasi-random, seed=None\n",
      "INFO:custom_initializers:Setting seed value for LecunNormalQR to 3.\n",
      "INFO:custom_initializers:Setting seed value for LecunNormalQR to 4.\n",
      "INFO:custom_initializers:Setting seed value for LecunNormalQR to 5.\n",
      "INFO:custom_initializers:Setting seed value for LecunNormalQR to 6.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m782/782\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 4ms/step - categorical_accuracy: 0.2296 - loss: 2.1595"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 1: train accuracy = -1, val accuracy = -1, test accuracy = 0.40369999408721924, time per epoch = 4.60 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m782/782\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m5s\u001b[0m 5ms/step - categorical_accuracy: 0.2297 - loss: 2.1591\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:models:Initialized ModelBuilder with model_id=baseline-cnn, input_shape=[32, 32, 3], output_shape=10, output_activation=softmax, model_config={'initializer_type': 'quasi-random', 'kernel_initializer': 'lecun-normal', 'bias_initializer': 'zeros', 'recurrent_initializer': 'orthogonal', 'max_features': None, 'units': 32, 'embedding_dim': None}\n",
      "INFO:trainer:Initializing lecun-normal from quasi-random, seed=None\n",
      "INFO:trainer:Initializing zeros from quasi-random, seed=None\n",
      "INFO:custom_initializers:Setting seed value for LecunNormalQR to 7.\n",
      "INFO:custom_initializers:Setting seed value for LecunNormalQR to 8.\n",
      "INFO:custom_initializers:Setting seed value for LecunNormalQR to 9.\n",
      "INFO:custom_initializers:Setting seed value for LecunNormalQR to 10.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m782/782\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 4ms/step - categorical_accuracy: 0.2410 - loss: 2.1669"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 1: train accuracy = -1, val accuracy = -1, test accuracy = 0.39800000190734863, time per epoch = 4.93 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m782/782\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m6s\u001b[0m 5ms/step - categorical_accuracy: 0.2411 - loss: 2.1666\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:models:Initialized ModelBuilder with model_id=baseline-cnn, input_shape=[32, 32, 3], output_shape=10, output_activation=softmax, model_config={'initializer_type': 'quasi-random', 'kernel_initializer': 'lecun-normal', 'bias_initializer': 'zeros', 'recurrent_initializer': 'orthogonal', 'max_features': None, 'units': 32, 'embedding_dim': None}\n",
      "INFO:trainer:Initializing lecun-normal from quasi-random, seed=None\n",
      "INFO:trainer:Initializing zeros from quasi-random, seed=None\n",
      "INFO:custom_initializers:Setting seed value for LecunNormalQR to 8.\n",
      "INFO:custom_initializers:Setting seed value for LecunNormalQR to 9.\n",
      "INFO:custom_initializers:Setting seed value for LecunNormalQR to 10.\n",
      "INFO:custom_initializers:Setting seed value for LecunNormalQR to 11.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m782/782\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 4ms/step - categorical_accuracy: 0.2042 - loss: 2.6556"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 1: train accuracy = -1, val accuracy = -1, test accuracy = 0.37279999256134033, time per epoch = 4.58 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m782/782\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m5s\u001b[0m 5ms/step - categorical_accuracy: 0.2043 - loss: 2.6548\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:models:Initialized ModelBuilder with model_id=baseline-cnn, input_shape=[32, 32, 3], output_shape=10, output_activation=softmax, model_config={'initializer_type': 'quasi-random', 'kernel_initializer': 'lecun-normal', 'bias_initializer': 'zeros', 'recurrent_initializer': 'orthogonal', 'max_features': None, 'units': 32, 'embedding_dim': None}\n",
      "INFO:trainer:Initializing lecun-normal from quasi-random, seed=None\n",
      "INFO:trainer:Initializing zeros from quasi-random, seed=None\n",
      "INFO:custom_initializers:Setting seed value for LecunNormalQR to 9.\n",
      "INFO:custom_initializers:Setting seed value for LecunNormalQR to 10.\n",
      "INFO:custom_initializers:Setting seed value for LecunNormalQR to 11.\n",
      "INFO:custom_initializers:Setting seed value for LecunNormalQR to 12.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m782/782\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 4ms/step - categorical_accuracy: 0.2021 - loss: 2.2574"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 1: train accuracy = -1, val accuracy = -1, test accuracy = 0.37959998846054077, time per epoch = 4.57 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m782/782\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m5s\u001b[0m 5ms/step - categorical_accuracy: 0.2022 - loss: 2.2571\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:train_and_eval:Resetting the state. Starting QRNG dimension is now set to 3.\n",
      "INFO:train_and_eval:Running experiment #1\n",
      "INFO:models:Initialized ModelBuilder with model_id=baseline-cnn, input_shape=[32, 32, 3], output_shape=10, output_activation=softmax, model_config={'initializer_type': 'quasi-random', 'kernel_initializer': 'lecun-normal', 'bias_initializer': 'zeros', 'recurrent_initializer': 'orthogonal', 'max_features': None, 'units': 32, 'embedding_dim': None}\n",
      "INFO:trainer:Initializing lecun-normal from quasi-random, seed=None\n",
      "INFO:trainer:Initializing zeros from quasi-random, seed=None\n",
      "INFO:custom_initializers:Setting seed value for LecunNormalQR to 3.\n",
      "INFO:custom_initializers:Setting seed value for LecunNormalQR to 4.\n",
      "INFO:custom_initializers:Setting seed value for LecunNormalQR to 5.\n",
      "INFO:custom_initializers:Setting seed value for LecunNormalQR to 6.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"font-weight: bold\">Model: \"functional_1\"</span>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[1mModel: \"functional_1\"\u001b[0m\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓\n",
       "┃<span style=\"font-weight: bold\"> Layer (type)                    </span>┃<span style=\"font-weight: bold\"> Output Shape           </span>┃<span style=\"font-weight: bold\">       Param # </span>┃\n",
       "┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩\n",
       "│ input_layer (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">InputLayer</span>)        │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">32</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">32</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">3</span>)      │             <span style=\"color: #00af00; text-decoration-color: #00af00\">0</span> │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ conv2d (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">Conv2D</span>)                 │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">30</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">30</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">32</span>)     │           <span style=\"color: #00af00; text-decoration-color: #00af00\">896</span> │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ max_pooling2d (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">MaxPooling2D</span>)    │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">15</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">15</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">32</span>)     │             <span style=\"color: #00af00; text-decoration-color: #00af00\">0</span> │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ conv2d_1 (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">Conv2D</span>)               │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">13</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">13</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">64</span>)     │        <span style=\"color: #00af00; text-decoration-color: #00af00\">18,496</span> │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ max_pooling2d_1 (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">MaxPooling2D</span>)  │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">6</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">6</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">64</span>)       │             <span style=\"color: #00af00; text-decoration-color: #00af00\">0</span> │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ conv2d_2 (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">Conv2D</span>)               │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">4</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">4</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">64</span>)       │        <span style=\"color: #00af00; text-decoration-color: #00af00\">36,928</span> │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ flatten (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">Flatten</span>)               │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">1024</span>)           │             <span style=\"color: #00af00; text-decoration-color: #00af00\">0</span> │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ predictions (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">Dense</span>)             │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">10</span>)             │        <span style=\"color: #00af00; text-decoration-color: #00af00\">10,250</span> │\n",
       "└─────────────────────────────────┴────────────────────────┴───────────────┘\n",
       "</pre>\n"
      ],
      "text/plain": [
       "┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓\n",
       "┃\u001b[1m \u001b[0m\u001b[1mLayer (type)                   \u001b[0m\u001b[1m \u001b[0m┃\u001b[1m \u001b[0m\u001b[1mOutput Shape          \u001b[0m\u001b[1m \u001b[0m┃\u001b[1m \u001b[0m\u001b[1m      Param #\u001b[0m\u001b[1m \u001b[0m┃\n",
       "┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩\n",
       "│ input_layer (\u001b[38;5;33mInputLayer\u001b[0m)        │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m32\u001b[0m, \u001b[38;5;34m32\u001b[0m, \u001b[38;5;34m3\u001b[0m)      │             \u001b[38;5;34m0\u001b[0m │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ conv2d (\u001b[38;5;33mConv2D\u001b[0m)                 │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m30\u001b[0m, \u001b[38;5;34m30\u001b[0m, \u001b[38;5;34m32\u001b[0m)     │           \u001b[38;5;34m896\u001b[0m │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ max_pooling2d (\u001b[38;5;33mMaxPooling2D\u001b[0m)    │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m15\u001b[0m, \u001b[38;5;34m15\u001b[0m, \u001b[38;5;34m32\u001b[0m)     │             \u001b[38;5;34m0\u001b[0m │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ conv2d_1 (\u001b[38;5;33mConv2D\u001b[0m)               │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m13\u001b[0m, \u001b[38;5;34m13\u001b[0m, \u001b[38;5;34m64\u001b[0m)     │        \u001b[38;5;34m18,496\u001b[0m │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ max_pooling2d_1 (\u001b[38;5;33mMaxPooling2D\u001b[0m)  │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m6\u001b[0m, \u001b[38;5;34m6\u001b[0m, \u001b[38;5;34m64\u001b[0m)       │             \u001b[38;5;34m0\u001b[0m │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ conv2d_2 (\u001b[38;5;33mConv2D\u001b[0m)               │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m4\u001b[0m, \u001b[38;5;34m4\u001b[0m, \u001b[38;5;34m64\u001b[0m)       │        \u001b[38;5;34m36,928\u001b[0m │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ flatten (\u001b[38;5;33mFlatten\u001b[0m)               │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m1024\u001b[0m)           │             \u001b[38;5;34m0\u001b[0m │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ predictions (\u001b[38;5;33mDense\u001b[0m)             │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m10\u001b[0m)             │        \u001b[38;5;34m10,250\u001b[0m │\n",
       "└─────────────────────────────────┴────────────────────────┴───────────────┘\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"font-weight: bold\"> Total params: </span><span style=\"color: #00af00; text-decoration-color: #00af00\">66,570</span> (260.04 KB)\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[1m Total params: \u001b[0m\u001b[38;5;34m66,570\u001b[0m (260.04 KB)\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"font-weight: bold\"> Trainable params: </span><span style=\"color: #00af00; text-decoration-color: #00af00\">66,570</span> (260.04 KB)\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[1m Trainable params: \u001b[0m\u001b[38;5;34m66,570\u001b[0m (260.04 KB)\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"font-weight: bold\"> Non-trainable params: </span><span style=\"color: #00af00; text-decoration-color: #00af00\">0</span> (0.00 B)\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[1m Non-trainable params: \u001b[0m\u001b[38;5;34m0\u001b[0m (0.00 B)\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/10\n",
      "\u001b[1m782/782\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 4ms/step - categorical_accuracy: 0.2447 - loss: 2.1290"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 1: train accuracy = -1, val accuracy = -1, test accuracy = 0.414900004863739, time per epoch = 4.59 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m782/782\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m5s\u001b[0m 5ms/step - categorical_accuracy: 0.2448 - loss: 2.1287\n",
      "Epoch 2/10\n",
      "\u001b[1m774/782\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m━\u001b[0m \u001b[1m0s\u001b[0m 3ms/step - categorical_accuracy: 0.4295 - loss: 1.5890"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 2: train accuracy = -1, val accuracy = -1, test accuracy = 0.47589999437332153, time per epoch = 2.47 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m782/782\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m3s\u001b[0m 4ms/step - categorical_accuracy: 0.4297 - loss: 1.5886\n",
      "Epoch 3/10\n",
      "\u001b[1m782/782\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 3ms/step - categorical_accuracy: 0.4817 - loss: 1.4507"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 3: train accuracy = -1, val accuracy = -1, test accuracy = 0.5027999877929688, time per epoch = 2.61 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m782/782\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m3s\u001b[0m 4ms/step - categorical_accuracy: 0.4817 - loss: 1.4506\n",
      "Epoch 4/10\n",
      "\u001b[1m767/782\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m━\u001b[0m \u001b[1m0s\u001b[0m 3ms/step - categorical_accuracy: 0.5190 - loss: 1.3544"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 4: train accuracy = -1, val accuracy = -1, test accuracy = 0.521399974822998, time per epoch = 2.54 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m782/782\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m3s\u001b[0m 4ms/step - categorical_accuracy: 0.5192 - loss: 1.3541\n",
      "Epoch 5/10\n",
      "\u001b[1m763/782\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m━\u001b[0m \u001b[1m0s\u001b[0m 3ms/step - categorical_accuracy: 0.5512 - loss: 1.2782"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 5: train accuracy = -1, val accuracy = -1, test accuracy = 0.5371999740600586, time per epoch = 2.27 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m782/782\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m3s\u001b[0m 3ms/step - categorical_accuracy: 0.5513 - loss: 1.2779\n",
      "Epoch 6/10\n",
      "\u001b[1m771/782\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m━\u001b[0m \u001b[1m0s\u001b[0m 3ms/step - categorical_accuracy: 0.5711 - loss: 1.2203"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 6: train accuracy = -1, val accuracy = -1, test accuracy = 0.5559999942779541, time per epoch = 2.37 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m782/782\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m3s\u001b[0m 3ms/step - categorical_accuracy: 0.5711 - loss: 1.2202\n",
      "Epoch 7/10\n",
      "\u001b[1m782/782\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 3ms/step - categorical_accuracy: 0.5927 - loss: 1.1689"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 7: train accuracy = -1, val accuracy = -1, test accuracy = 0.5491999983787537, time per epoch = 2.33 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m782/782\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m3s\u001b[0m 3ms/step - categorical_accuracy: 0.5927 - loss: 1.1689\n",
      "Epoch 8/10\n",
      "\u001b[1m781/782\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m━\u001b[0m \u001b[1m0s\u001b[0m 3ms/step - categorical_accuracy: 0.6109 - loss: 1.1241"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 8: train accuracy = -1, val accuracy = -1, test accuracy = 0.5788999795913696, time per epoch = 2.33 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m782/782\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m3s\u001b[0m 3ms/step - categorical_accuracy: 0.6110 - loss: 1.1241\n",
      "Epoch 9/10\n",
      "\u001b[1m776/782\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m━\u001b[0m \u001b[1m0s\u001b[0m 3ms/step - categorical_accuracy: 0.6244 - loss: 1.0849"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 9: train accuracy = -1, val accuracy = -1, test accuracy = 0.5767999887466431, time per epoch = 2.24 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m782/782\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m3s\u001b[0m 3ms/step - categorical_accuracy: 0.6244 - loss: 1.0849\n",
      "Epoch 10/10\n",
      "\u001b[1m765/782\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m━\u001b[0m \u001b[1m0s\u001b[0m 3ms/step - categorical_accuracy: 0.6390 - loss: 1.0488"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 10: train accuracy = -1, val accuracy = -1, test accuracy = 0.5792999863624573, time per epoch = 2.21 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m782/782\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2s\u001b[0m 3ms/step - categorical_accuracy: 0.6391 - loss: 1.0485\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:datapipeline:Loading CIFAR10 dataset with flatten=False and one_hot=True\n",
      "INFO:datapipeline:Converting labels to one-hot encoding\n",
      "INFO:datapipeline:CIFAR10 Dataset loaded\n",
      "INFO:datapipeline:x_train shape (50000, 32, 32, 3)\n",
      "INFO:datapipeline:y_train shape (50000, 10)\n",
      "INFO:datapipeline:x_test shape (10000, 32, 32, 3)\n",
      "INFO:datapipeline:y_test shape (10000, 10)\n",
      "INFO:utils:Splitting Data into Train and Validation\n",
      "INFO:utils:Training observation count is set to 50000\n",
      "INFO:utils:x_train shape before splitting (50000, 32, 32, 3)\n",
      "INFO:utils:y_train shape before splitting (50000, 10)\n",
      "INFO:utils:Training observation count equals the length of x_train. No validation set will be created.\n",
      "INFO:utils:x_train shape after splitting (50000, 32, 32, 3)\n",
      "INFO:utils:y_train shape after splitting (50000, 10)\n",
      "INFO:utils:x_val shape after splitting (0, 32, 32, 3)\n",
      "INFO:utils:y_val shape after splitting (0, 10)\n",
      "INFO:train_and_eval:Choosing the best seed from this set: [1, 7, 8, 9, 10]\n",
      "INFO:models:Initialized ModelBuilder with model_id=baseline-cnn, input_shape=[32, 32, 3], output_shape=10, output_activation=softmax, model_config={'initializer_type': 'quasi-random', 'kernel_initializer': 'lecun-normal', 'bias_initializer': 'zeros', 'recurrent_initializer': 'orthogonal', 'max_features': None, 'units': 32, 'embedding_dim': None}\n",
      "INFO:trainer:Initializing lecun-normal from quasi-random, seed=None\n",
      "INFO:trainer:Initializing zeros from quasi-random, seed=None\n",
      "INFO:custom_initializers:Setting seed value for LecunNormalQR to 1.\n",
      "INFO:custom_initializers:Setting seed value for LecunNormalQR to 2.\n",
      "INFO:custom_initializers:Setting seed value for LecunNormalQR to 3.\n",
      "INFO:custom_initializers:Setting seed value for LecunNormalQR to 4.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m782/782\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 4ms/step - categorical_accuracy: 0.1002 - loss: 12.2325"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 1: train accuracy = -1, val accuracy = -1, test accuracy = 0.10000000149011612, time per epoch = 4.30 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m782/782\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m5s\u001b[0m 4ms/step - categorical_accuracy: 0.1002 - loss: 12.2224\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:models:Initialized ModelBuilder with model_id=baseline-cnn, input_shape=[32, 32, 3], output_shape=10, output_activation=softmax, model_config={'initializer_type': 'quasi-random', 'kernel_initializer': 'lecun-normal', 'bias_initializer': 'zeros', 'recurrent_initializer': 'orthogonal', 'max_features': None, 'units': 32, 'embedding_dim': None}\n",
      "INFO:trainer:Initializing lecun-normal from quasi-random, seed=None\n",
      "INFO:trainer:Initializing zeros from quasi-random, seed=None\n",
      "INFO:custom_initializers:Setting seed value for LecunNormalQR to 7.\n",
      "INFO:custom_initializers:Setting seed value for LecunNormalQR to 8.\n",
      "INFO:custom_initializers:Setting seed value for LecunNormalQR to 9.\n",
      "INFO:custom_initializers:Setting seed value for LecunNormalQR to 10.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m782/782\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 4ms/step - categorical_accuracy: 0.2487 - loss: 2.1513"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 1: train accuracy = -1, val accuracy = -1, test accuracy = 0.4036000072956085, time per epoch = 4.72 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m782/782\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m5s\u001b[0m 5ms/step - categorical_accuracy: 0.2488 - loss: 2.1510\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:models:Initialized ModelBuilder with model_id=baseline-cnn, input_shape=[32, 32, 3], output_shape=10, output_activation=softmax, model_config={'initializer_type': 'quasi-random', 'kernel_initializer': 'lecun-normal', 'bias_initializer': 'zeros', 'recurrent_initializer': 'orthogonal', 'max_features': None, 'units': 32, 'embedding_dim': None}\n",
      "INFO:trainer:Initializing lecun-normal from quasi-random, seed=None\n",
      "INFO:trainer:Initializing zeros from quasi-random, seed=None\n",
      "INFO:custom_initializers:Setting seed value for LecunNormalQR to 8.\n",
      "INFO:custom_initializers:Setting seed value for LecunNormalQR to 9.\n",
      "INFO:custom_initializers:Setting seed value for LecunNormalQR to 10.\n",
      "INFO:custom_initializers:Setting seed value for LecunNormalQR to 11.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m782/782\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 4ms/step - categorical_accuracy: 0.2060 - loss: 2.6196"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 1: train accuracy = -1, val accuracy = -1, test accuracy = 0.3716000020503998, time per epoch = 4.48 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m782/782\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m5s\u001b[0m 5ms/step - categorical_accuracy: 0.2061 - loss: 2.6189\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:models:Initialized ModelBuilder with model_id=baseline-cnn, input_shape=[32, 32, 3], output_shape=10, output_activation=softmax, model_config={'initializer_type': 'quasi-random', 'kernel_initializer': 'lecun-normal', 'bias_initializer': 'zeros', 'recurrent_initializer': 'orthogonal', 'max_features': None, 'units': 32, 'embedding_dim': None}\n",
      "INFO:trainer:Initializing lecun-normal from quasi-random, seed=None\n",
      "INFO:trainer:Initializing zeros from quasi-random, seed=None\n",
      "INFO:custom_initializers:Setting seed value for LecunNormalQR to 9.\n",
      "INFO:custom_initializers:Setting seed value for LecunNormalQR to 10.\n",
      "INFO:custom_initializers:Setting seed value for LecunNormalQR to 11.\n",
      "INFO:custom_initializers:Setting seed value for LecunNormalQR to 12.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m782/782\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 4ms/step - categorical_accuracy: 0.2062 - loss: 2.2557"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 1: train accuracy = -1, val accuracy = -1, test accuracy = 0.37709999084472656, time per epoch = 4.31 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m782/782\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m5s\u001b[0m 5ms/step - categorical_accuracy: 0.2062 - loss: 2.2553\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:models:Initialized ModelBuilder with model_id=baseline-cnn, input_shape=[32, 32, 3], output_shape=10, output_activation=softmax, model_config={'initializer_type': 'quasi-random', 'kernel_initializer': 'lecun-normal', 'bias_initializer': 'zeros', 'recurrent_initializer': 'orthogonal', 'max_features': None, 'units': 32, 'embedding_dim': None}\n",
      "INFO:trainer:Initializing lecun-normal from quasi-random, seed=None\n",
      "INFO:trainer:Initializing zeros from quasi-random, seed=None\n",
      "INFO:custom_initializers:Setting seed value for LecunNormalQR to 10.\n",
      "INFO:custom_initializers:Setting seed value for LecunNormalQR to 11.\n",
      "INFO:custom_initializers:Setting seed value for LecunNormalQR to 12.\n",
      "INFO:custom_initializers:Setting seed value for LecunNormalQR to 13.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m782/782\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 4ms/step - categorical_accuracy: 0.1552 - loss: 2.4460"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 1: train accuracy = -1, val accuracy = -1, test accuracy = 0.32659998536109924, time per epoch = 4.16 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m782/782\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m5s\u001b[0m 4ms/step - categorical_accuracy: 0.1553 - loss: 2.4456\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:train_and_eval:Resetting the state. Starting QRNG dimension is now set to 7.\n",
      "INFO:train_and_eval:Running experiment #1\n",
      "INFO:models:Initialized ModelBuilder with model_id=baseline-cnn, input_shape=[32, 32, 3], output_shape=10, output_activation=softmax, model_config={'initializer_type': 'quasi-random', 'kernel_initializer': 'lecun-normal', 'bias_initializer': 'zeros', 'recurrent_initializer': 'orthogonal', 'max_features': None, 'units': 32, 'embedding_dim': None}\n",
      "INFO:trainer:Initializing lecun-normal from quasi-random, seed=None\n",
      "INFO:trainer:Initializing zeros from quasi-random, seed=None\n",
      "INFO:custom_initializers:Setting seed value for LecunNormalQR to 7.\n",
      "INFO:custom_initializers:Setting seed value for LecunNormalQR to 8.\n",
      "INFO:custom_initializers:Setting seed value for LecunNormalQR to 9.\n",
      "INFO:custom_initializers:Setting seed value for LecunNormalQR to 10.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"font-weight: bold\">Model: \"functional_1\"</span>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[1mModel: \"functional_1\"\u001b[0m\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓\n",
       "┃<span style=\"font-weight: bold\"> Layer (type)                    </span>┃<span style=\"font-weight: bold\"> Output Shape           </span>┃<span style=\"font-weight: bold\">       Param # </span>┃\n",
       "┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩\n",
       "│ input_layer (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">InputLayer</span>)        │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">32</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">32</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">3</span>)      │             <span style=\"color: #00af00; text-decoration-color: #00af00\">0</span> │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ conv2d (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">Conv2D</span>)                 │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">30</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">30</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">32</span>)     │           <span style=\"color: #00af00; text-decoration-color: #00af00\">896</span> │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ max_pooling2d (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">MaxPooling2D</span>)    │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">15</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">15</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">32</span>)     │             <span style=\"color: #00af00; text-decoration-color: #00af00\">0</span> │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ conv2d_1 (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">Conv2D</span>)               │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">13</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">13</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">64</span>)     │        <span style=\"color: #00af00; text-decoration-color: #00af00\">18,496</span> │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ max_pooling2d_1 (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">MaxPooling2D</span>)  │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">6</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">6</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">64</span>)       │             <span style=\"color: #00af00; text-decoration-color: #00af00\">0</span> │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ conv2d_2 (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">Conv2D</span>)               │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">4</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">4</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">64</span>)       │        <span style=\"color: #00af00; text-decoration-color: #00af00\">36,928</span> │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ flatten (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">Flatten</span>)               │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">1024</span>)           │             <span style=\"color: #00af00; text-decoration-color: #00af00\">0</span> │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ predictions (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">Dense</span>)             │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">10</span>)             │        <span style=\"color: #00af00; text-decoration-color: #00af00\">10,250</span> │\n",
       "└─────────────────────────────────┴────────────────────────┴───────────────┘\n",
       "</pre>\n"
      ],
      "text/plain": [
       "┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓\n",
       "┃\u001b[1m \u001b[0m\u001b[1mLayer (type)                   \u001b[0m\u001b[1m \u001b[0m┃\u001b[1m \u001b[0m\u001b[1mOutput Shape          \u001b[0m\u001b[1m \u001b[0m┃\u001b[1m \u001b[0m\u001b[1m      Param #\u001b[0m\u001b[1m \u001b[0m┃\n",
       "┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩\n",
       "│ input_layer (\u001b[38;5;33mInputLayer\u001b[0m)        │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m32\u001b[0m, \u001b[38;5;34m32\u001b[0m, \u001b[38;5;34m3\u001b[0m)      │             \u001b[38;5;34m0\u001b[0m │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ conv2d (\u001b[38;5;33mConv2D\u001b[0m)                 │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m30\u001b[0m, \u001b[38;5;34m30\u001b[0m, \u001b[38;5;34m32\u001b[0m)     │           \u001b[38;5;34m896\u001b[0m │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ max_pooling2d (\u001b[38;5;33mMaxPooling2D\u001b[0m)    │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m15\u001b[0m, \u001b[38;5;34m15\u001b[0m, \u001b[38;5;34m32\u001b[0m)     │             \u001b[38;5;34m0\u001b[0m │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ conv2d_1 (\u001b[38;5;33mConv2D\u001b[0m)               │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m13\u001b[0m, \u001b[38;5;34m13\u001b[0m, \u001b[38;5;34m64\u001b[0m)     │        \u001b[38;5;34m18,496\u001b[0m │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ max_pooling2d_1 (\u001b[38;5;33mMaxPooling2D\u001b[0m)  │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m6\u001b[0m, \u001b[38;5;34m6\u001b[0m, \u001b[38;5;34m64\u001b[0m)       │             \u001b[38;5;34m0\u001b[0m │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ conv2d_2 (\u001b[38;5;33mConv2D\u001b[0m)               │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m4\u001b[0m, \u001b[38;5;34m4\u001b[0m, \u001b[38;5;34m64\u001b[0m)       │        \u001b[38;5;34m36,928\u001b[0m │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ flatten (\u001b[38;5;33mFlatten\u001b[0m)               │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m1024\u001b[0m)           │             \u001b[38;5;34m0\u001b[0m │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ predictions (\u001b[38;5;33mDense\u001b[0m)             │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m10\u001b[0m)             │        \u001b[38;5;34m10,250\u001b[0m │\n",
       "└─────────────────────────────────┴────────────────────────┴───────────────┘\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"font-weight: bold\"> Total params: </span><span style=\"color: #00af00; text-decoration-color: #00af00\">66,570</span> (260.04 KB)\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[1m Total params: \u001b[0m\u001b[38;5;34m66,570\u001b[0m (260.04 KB)\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"font-weight: bold\"> Trainable params: </span><span style=\"color: #00af00; text-decoration-color: #00af00\">66,570</span> (260.04 KB)\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[1m Trainable params: \u001b[0m\u001b[38;5;34m66,570\u001b[0m (260.04 KB)\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"font-weight: bold\"> Non-trainable params: </span><span style=\"color: #00af00; text-decoration-color: #00af00\">0</span> (0.00 B)\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[1m Non-trainable params: \u001b[0m\u001b[38;5;34m0\u001b[0m (0.00 B)\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/10\n",
      "\u001b[1m782/782\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 4ms/step - categorical_accuracy: 0.2511 - loss: 2.1533"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 1: train accuracy = -1, val accuracy = -1, test accuracy = 0.40369999408721924, time per epoch = 4.15 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m782/782\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m5s\u001b[0m 5ms/step - categorical_accuracy: 0.2512 - loss: 2.1530\n",
      "Epoch 2/10\n",
      "\u001b[1m777/782\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m━\u001b[0m \u001b[1m0s\u001b[0m 3ms/step - categorical_accuracy: 0.4239 - loss: 1.6049"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 2: train accuracy = -1, val accuracy = -1, test accuracy = 0.4392000138759613, time per epoch = 2.51 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m782/782\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m3s\u001b[0m 4ms/step - categorical_accuracy: 0.4240 - loss: 1.6046\n",
      "Epoch 3/10\n",
      "\u001b[1m775/782\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m━\u001b[0m \u001b[1m0s\u001b[0m 3ms/step - categorical_accuracy: 0.4797 - loss: 1.4572"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 3: train accuracy = -1, val accuracy = -1, test accuracy = 0.4869000017642975, time per epoch = 2.23 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m782/782\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2s\u001b[0m 3ms/step - categorical_accuracy: 0.4797 - loss: 1.4570\n",
      "Epoch 4/10\n",
      "\u001b[1m774/782\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m━\u001b[0m \u001b[1m0s\u001b[0m 3ms/step - categorical_accuracy: 0.5147 - loss: 1.3656"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 4: train accuracy = -1, val accuracy = -1, test accuracy = 0.5178999900817871, time per epoch = 2.04 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m782/782\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2s\u001b[0m 3ms/step - categorical_accuracy: 0.5147 - loss: 1.3655\n",
      "Epoch 5/10\n",
      "\u001b[1m766/782\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m━\u001b[0m \u001b[1m0s\u001b[0m 3ms/step - categorical_accuracy: 0.5447 - loss: 1.2901"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 5: train accuracy = -1, val accuracy = -1, test accuracy = 0.5256999731063843, time per epoch = 2.11 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m782/782\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2s\u001b[0m 3ms/step - categorical_accuracy: 0.5447 - loss: 1.2901\n",
      "Epoch 6/10\n",
      "\u001b[1m770/782\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m━\u001b[0m \u001b[1m0s\u001b[0m 3ms/step - categorical_accuracy: 0.5652 - loss: 1.2286"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 6: train accuracy = -1, val accuracy = -1, test accuracy = 0.5472000241279602, time per epoch = 2.21 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m782/782\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2s\u001b[0m 3ms/step - categorical_accuracy: 0.5652 - loss: 1.2285\n",
      "Epoch 7/10\n",
      "\u001b[1m769/782\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m━\u001b[0m \u001b[1m0s\u001b[0m 3ms/step - categorical_accuracy: 0.5843 - loss: 1.1848"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 7: train accuracy = -1, val accuracy = -1, test accuracy = 0.5485000014305115, time per epoch = 2.43 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m782/782\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m3s\u001b[0m 3ms/step - categorical_accuracy: 0.5844 - loss: 1.1847\n",
      "Epoch 8/10\n",
      "\u001b[1m769/782\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m━\u001b[0m \u001b[1m0s\u001b[0m 3ms/step - categorical_accuracy: 0.5985 - loss: 1.1371"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 8: train accuracy = -1, val accuracy = -1, test accuracy = 0.5698999762535095, time per epoch = 2.27 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m782/782\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m3s\u001b[0m 3ms/step - categorical_accuracy: 0.5986 - loss: 1.1370\n",
      "Epoch 9/10\n",
      "\u001b[1m779/782\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m━\u001b[0m \u001b[1m0s\u001b[0m 3ms/step - categorical_accuracy: 0.6149 - loss: 1.0928"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 9: train accuracy = -1, val accuracy = -1, test accuracy = 0.5631999969482422, time per epoch = 2.16 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m782/782\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2s\u001b[0m 3ms/step - categorical_accuracy: 0.6149 - loss: 1.0928\n",
      "Epoch 10/10\n",
      "\u001b[1m778/782\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m━\u001b[0m \u001b[1m0s\u001b[0m 3ms/step - categorical_accuracy: 0.6266 - loss: 1.0554"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 10: train accuracy = -1, val accuracy = -1, test accuracy = 0.5802000164985657, time per epoch = 2.23 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m782/782\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2s\u001b[0m 3ms/step - categorical_accuracy: 0.6266 - loss: 1.0554\n"
     ]
    }
   ],
   "source": [
    "cnn_quasi_stats = [run_experiment(\n",
    "    experiment_id=1,\n",
    "    sequence_scheme=SEQUENCE_SCHEME,\n",
    "    units=32,\n",
    "    dataset_id=\"cifar10\",\n",
    "    kernel_initializer=\"lecun-normal\",\n",
    "    initializer_type=\"quasi-random\",\n",
    "    model_id=\"baseline-cnn\",\n",
    "    epochs=EPOCHS,\n",
    "    auto_epoch_count=AUTO_EPOCH_COUNT,\n",
    "    auto_min_seed_value=AUTO_MIN_SEED_VALUE,\n",
    "    auto_max_seed_value=AUTO_MAX_SEED_VALUE,\n",
    "    auto_seeds_count=AUTO_SEEDS_COUNT,\n",
    ")[0] for _ in range(NUM_REPS)]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:datapipeline:Loading CIFAR10 dataset with flatten=False and one_hot=True\n",
      "INFO:datapipeline:Converting labels to one-hot encoding\n",
      "INFO:datapipeline:CIFAR10 Dataset loaded\n",
      "INFO:datapipeline:x_train shape (50000, 32, 32, 3)\n",
      "INFO:datapipeline:y_train shape (50000, 10)\n",
      "INFO:datapipeline:x_test shape (10000, 32, 32, 3)\n",
      "INFO:datapipeline:y_test shape (10000, 10)\n",
      "INFO:utils:Splitting Data into Train and Validation\n",
      "INFO:utils:Training observation count is set to 50000\n",
      "INFO:utils:x_train shape before splitting (50000, 32, 32, 3)\n",
      "INFO:utils:y_train shape before splitting (50000, 10)\n",
      "INFO:utils:Training observation count equals the length of x_train. No validation set will be created.\n",
      "INFO:utils:x_train shape after splitting (50000, 32, 32, 3)\n",
      "INFO:utils:y_train shape after splitting (50000, 10)\n",
      "INFO:utils:x_val shape after splitting (0, 32, 32, 3)\n",
      "INFO:utils:y_val shape after splitting (0, 10)\n",
      "INFO:train_and_eval:Seeds for this experiment will be selected at random. Setting sequence_scheme to random.\n",
      "INFO:train_and_eval:Running experiment #1\n",
      "INFO:models:Initialized ModelBuilder with model_id=baseline-cnn, input_shape=[32, 32, 3], output_shape=10, output_activation=softmax, model_config={'initializer_type': 'pseudo-random', 'kernel_initializer': 'lecun-normal', 'bias_initializer': 'zeros', 'recurrent_initializer': 'orthogonal', 'max_features': None, 'units': 32, 'embedding_dim': None}\n",
      "INFO:trainer:Initializing lecun-normal from pseudo-random, seed=None\n",
      "INFO:trainer:Initializing zeros from pseudo-random, seed=None\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"font-weight: bold\">Model: \"functional_3\"</span>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[1mModel: \"functional_3\"\u001b[0m\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓\n",
       "┃<span style=\"font-weight: bold\"> Layer (type)                    </span>┃<span style=\"font-weight: bold\"> Output Shape           </span>┃<span style=\"font-weight: bold\">       Param # </span>┃\n",
       "┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩\n",
       "│ input_layer (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">InputLayer</span>)        │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">32</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">32</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">3</span>)      │             <span style=\"color: #00af00; text-decoration-color: #00af00\">0</span> │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ conv2d_3 (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">Conv2D</span>)               │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">30</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">30</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">32</span>)     │           <span style=\"color: #00af00; text-decoration-color: #00af00\">896</span> │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ max_pooling2d_2 (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">MaxPooling2D</span>)  │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">15</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">15</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">32</span>)     │             <span style=\"color: #00af00; text-decoration-color: #00af00\">0</span> │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ conv2d_4 (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">Conv2D</span>)               │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">13</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">13</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">64</span>)     │        <span style=\"color: #00af00; text-decoration-color: #00af00\">18,496</span> │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ max_pooling2d_3 (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">MaxPooling2D</span>)  │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">6</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">6</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">64</span>)       │             <span style=\"color: #00af00; text-decoration-color: #00af00\">0</span> │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ conv2d_5 (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">Conv2D</span>)               │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">4</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">4</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">64</span>)       │        <span style=\"color: #00af00; text-decoration-color: #00af00\">36,928</span> │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ flatten_1 (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">Flatten</span>)             │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">1024</span>)           │             <span style=\"color: #00af00; text-decoration-color: #00af00\">0</span> │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ predictions (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">Dense</span>)             │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">10</span>)             │        <span style=\"color: #00af00; text-decoration-color: #00af00\">10,250</span> │\n",
       "└─────────────────────────────────┴────────────────────────┴───────────────┘\n",
       "</pre>\n"
      ],
      "text/plain": [
       "┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓\n",
       "┃\u001b[1m \u001b[0m\u001b[1mLayer (type)                   \u001b[0m\u001b[1m \u001b[0m┃\u001b[1m \u001b[0m\u001b[1mOutput Shape          \u001b[0m\u001b[1m \u001b[0m┃\u001b[1m \u001b[0m\u001b[1m      Param #\u001b[0m\u001b[1m \u001b[0m┃\n",
       "┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩\n",
       "│ input_layer (\u001b[38;5;33mInputLayer\u001b[0m)        │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m32\u001b[0m, \u001b[38;5;34m32\u001b[0m, \u001b[38;5;34m3\u001b[0m)      │             \u001b[38;5;34m0\u001b[0m │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ conv2d_3 (\u001b[38;5;33mConv2D\u001b[0m)               │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m30\u001b[0m, \u001b[38;5;34m30\u001b[0m, \u001b[38;5;34m32\u001b[0m)     │           \u001b[38;5;34m896\u001b[0m │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ max_pooling2d_2 (\u001b[38;5;33mMaxPooling2D\u001b[0m)  │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m15\u001b[0m, \u001b[38;5;34m15\u001b[0m, \u001b[38;5;34m32\u001b[0m)     │             \u001b[38;5;34m0\u001b[0m │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ conv2d_4 (\u001b[38;5;33mConv2D\u001b[0m)               │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m13\u001b[0m, \u001b[38;5;34m13\u001b[0m, \u001b[38;5;34m64\u001b[0m)     │        \u001b[38;5;34m18,496\u001b[0m │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ max_pooling2d_3 (\u001b[38;5;33mMaxPooling2D\u001b[0m)  │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m6\u001b[0m, \u001b[38;5;34m6\u001b[0m, \u001b[38;5;34m64\u001b[0m)       │             \u001b[38;5;34m0\u001b[0m │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ conv2d_5 (\u001b[38;5;33mConv2D\u001b[0m)               │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m4\u001b[0m, \u001b[38;5;34m4\u001b[0m, \u001b[38;5;34m64\u001b[0m)       │        \u001b[38;5;34m36,928\u001b[0m │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ flatten_1 (\u001b[38;5;33mFlatten\u001b[0m)             │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m1024\u001b[0m)           │             \u001b[38;5;34m0\u001b[0m │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ predictions (\u001b[38;5;33mDense\u001b[0m)             │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m10\u001b[0m)             │        \u001b[38;5;34m10,250\u001b[0m │\n",
       "└─────────────────────────────────┴────────────────────────┴───────────────┘\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"font-weight: bold\"> Total params: </span><span style=\"color: #00af00; text-decoration-color: #00af00\">66,570</span> (260.04 KB)\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[1m Total params: \u001b[0m\u001b[38;5;34m66,570\u001b[0m (260.04 KB)\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"font-weight: bold\"> Trainable params: </span><span style=\"color: #00af00; text-decoration-color: #00af00\">66,570</span> (260.04 KB)\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[1m Trainable params: \u001b[0m\u001b[38;5;34m66,570\u001b[0m (260.04 KB)\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"font-weight: bold\"> Non-trainable params: </span><span style=\"color: #00af00; text-decoration-color: #00af00\">0</span> (0.00 B)\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[1m Non-trainable params: \u001b[0m\u001b[38;5;34m0\u001b[0m (0.00 B)\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/10\n",
      "\u001b[1m782/782\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 4ms/step - categorical_accuracy: 0.1793 - loss: 10.4306"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 1: train accuracy = -1, val accuracy = -1, test accuracy = 0.20600000023841858, time per epoch = 4.74 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m782/782\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m5s\u001b[0m 5ms/step - categorical_accuracy: 0.1794 - loss: 10.4243\n",
      "Epoch 2/10\n",
      "\u001b[1m776/782\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m━\u001b[0m \u001b[1m0s\u001b[0m 3ms/step - categorical_accuracy: 0.2347 - loss: 2.1219"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 2: train accuracy = -1, val accuracy = -1, test accuracy = 0.29409998655319214, time per epoch = 2.62 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m782/782\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m3s\u001b[0m 4ms/step - categorical_accuracy: 0.2349 - loss: 2.1213\n",
      "Epoch 3/10\n",
      "\u001b[1m780/782\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m━\u001b[0m \u001b[1m0s\u001b[0m 3ms/step - categorical_accuracy: 0.3180 - loss: 1.8866"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 3: train accuracy = -1, val accuracy = -1, test accuracy = 0.35920000076293945, time per epoch = 2.33 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m782/782\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m3s\u001b[0m 3ms/step - categorical_accuracy: 0.3181 - loss: 1.8865\n",
      "Epoch 4/10\n",
      "\u001b[1m782/782\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 3ms/step - categorical_accuracy: 0.3776 - loss: 1.7171"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 4: train accuracy = -1, val accuracy = -1, test accuracy = 0.4124999940395355, time per epoch = 2.27 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m782/782\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m3s\u001b[0m 3ms/step - categorical_accuracy: 0.3776 - loss: 1.7170\n",
      "Epoch 5/10\n",
      "\u001b[1m776/782\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m━\u001b[0m \u001b[1m0s\u001b[0m 3ms/step - categorical_accuracy: 0.4298 - loss: 1.5717"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 5: train accuracy = -1, val accuracy = -1, test accuracy = 0.4357999861240387, time per epoch = 2.23 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m782/782\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2s\u001b[0m 3ms/step - categorical_accuracy: 0.4299 - loss: 1.5716\n",
      "Epoch 6/10\n",
      "\u001b[1m778/782\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m━\u001b[0m \u001b[1m0s\u001b[0m 3ms/step - categorical_accuracy: 0.4657 - loss: 1.4847"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 6: train accuracy = -1, val accuracy = -1, test accuracy = 0.4650999903678894, time per epoch = 2.28 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m782/782\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m3s\u001b[0m 3ms/step - categorical_accuracy: 0.4657 - loss: 1.4846\n",
      "Epoch 7/10\n",
      "\u001b[1m774/782\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m━\u001b[0m \u001b[1m0s\u001b[0m 3ms/step - categorical_accuracy: 0.4891 - loss: 1.4103"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 7: train accuracy = -1, val accuracy = -1, test accuracy = 0.4855000078678131, time per epoch = 2.30 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m782/782\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m3s\u001b[0m 3ms/step - categorical_accuracy: 0.4892 - loss: 1.4102\n",
      "Epoch 8/10\n",
      "\u001b[1m777/782\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m━\u001b[0m \u001b[1m0s\u001b[0m 3ms/step - categorical_accuracy: 0.5136 - loss: 1.3511"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 8: train accuracy = -1, val accuracy = -1, test accuracy = 0.5037000179290771, time per epoch = 2.33 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m782/782\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m3s\u001b[0m 3ms/step - categorical_accuracy: 0.5136 - loss: 1.3511\n",
      "Epoch 9/10\n",
      "\u001b[1m773/782\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m━\u001b[0m \u001b[1m0s\u001b[0m 3ms/step - categorical_accuracy: 0.5328 - loss: 1.3001"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 9: train accuracy = -1, val accuracy = -1, test accuracy = 0.5160999894142151, time per epoch = 2.31 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m782/782\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m3s\u001b[0m 3ms/step - categorical_accuracy: 0.5328 - loss: 1.3001\n",
      "Epoch 10/10\n",
      "\u001b[1m780/782\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m━\u001b[0m \u001b[1m0s\u001b[0m 3ms/step - categorical_accuracy: 0.5487 - loss: 1.2599"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 10: train accuracy = -1, val accuracy = -1, test accuracy = 0.5218999981880188, time per epoch = 2.33 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m782/782\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m3s\u001b[0m 3ms/step - categorical_accuracy: 0.5487 - loss: 1.2599\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:datapipeline:Loading CIFAR10 dataset with flatten=False and one_hot=True\n",
      "INFO:datapipeline:Converting labels to one-hot encoding\n",
      "INFO:datapipeline:CIFAR10 Dataset loaded\n",
      "INFO:datapipeline:x_train shape (50000, 32, 32, 3)\n",
      "INFO:datapipeline:y_train shape (50000, 10)\n",
      "INFO:datapipeline:x_test shape (10000, 32, 32, 3)\n",
      "INFO:datapipeline:y_test shape (10000, 10)\n",
      "INFO:utils:Splitting Data into Train and Validation\n",
      "INFO:utils:Training observation count is set to 50000\n",
      "INFO:utils:x_train shape before splitting (50000, 32, 32, 3)\n",
      "INFO:utils:y_train shape before splitting (50000, 10)\n",
      "INFO:utils:Training observation count equals the length of x_train. No validation set will be created.\n",
      "INFO:utils:x_train shape after splitting (50000, 32, 32, 3)\n",
      "INFO:utils:y_train shape after splitting (50000, 10)\n",
      "INFO:utils:x_val shape after splitting (0, 32, 32, 3)\n",
      "INFO:utils:y_val shape after splitting (0, 10)\n",
      "INFO:train_and_eval:Seeds for this experiment will be selected at random. Setting sequence_scheme to random.\n",
      "INFO:train_and_eval:Running experiment #1\n",
      "INFO:models:Initialized ModelBuilder with model_id=baseline-cnn, input_shape=[32, 32, 3], output_shape=10, output_activation=softmax, model_config={'initializer_type': 'pseudo-random', 'kernel_initializer': 'lecun-normal', 'bias_initializer': 'zeros', 'recurrent_initializer': 'orthogonal', 'max_features': None, 'units': 32, 'embedding_dim': None}\n",
      "INFO:trainer:Initializing lecun-normal from pseudo-random, seed=None\n",
      "INFO:trainer:Initializing zeros from pseudo-random, seed=None\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"font-weight: bold\">Model: \"functional_5\"</span>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[1mModel: \"functional_5\"\u001b[0m\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓\n",
       "┃<span style=\"font-weight: bold\"> Layer (type)                    </span>┃<span style=\"font-weight: bold\"> Output Shape           </span>┃<span style=\"font-weight: bold\">       Param # </span>┃\n",
       "┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩\n",
       "│ input_layer (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">InputLayer</span>)        │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">32</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">32</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">3</span>)      │             <span style=\"color: #00af00; text-decoration-color: #00af00\">0</span> │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ conv2d_6 (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">Conv2D</span>)               │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">30</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">30</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">32</span>)     │           <span style=\"color: #00af00; text-decoration-color: #00af00\">896</span> │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ max_pooling2d_4 (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">MaxPooling2D</span>)  │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">15</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">15</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">32</span>)     │             <span style=\"color: #00af00; text-decoration-color: #00af00\">0</span> │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ conv2d_7 (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">Conv2D</span>)               │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">13</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">13</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">64</span>)     │        <span style=\"color: #00af00; text-decoration-color: #00af00\">18,496</span> │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ max_pooling2d_5 (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">MaxPooling2D</span>)  │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">6</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">6</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">64</span>)       │             <span style=\"color: #00af00; text-decoration-color: #00af00\">0</span> │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ conv2d_8 (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">Conv2D</span>)               │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">4</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">4</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">64</span>)       │        <span style=\"color: #00af00; text-decoration-color: #00af00\">36,928</span> │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ flatten_2 (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">Flatten</span>)             │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">1024</span>)           │             <span style=\"color: #00af00; text-decoration-color: #00af00\">0</span> │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ predictions (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">Dense</span>)             │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">10</span>)             │        <span style=\"color: #00af00; text-decoration-color: #00af00\">10,250</span> │\n",
       "└─────────────────────────────────┴────────────────────────┴───────────────┘\n",
       "</pre>\n"
      ],
      "text/plain": [
       "┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓\n",
       "┃\u001b[1m \u001b[0m\u001b[1mLayer (type)                   \u001b[0m\u001b[1m \u001b[0m┃\u001b[1m \u001b[0m\u001b[1mOutput Shape          \u001b[0m\u001b[1m \u001b[0m┃\u001b[1m \u001b[0m\u001b[1m      Param #\u001b[0m\u001b[1m \u001b[0m┃\n",
       "┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩\n",
       "│ input_layer (\u001b[38;5;33mInputLayer\u001b[0m)        │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m32\u001b[0m, \u001b[38;5;34m32\u001b[0m, \u001b[38;5;34m3\u001b[0m)      │             \u001b[38;5;34m0\u001b[0m │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ conv2d_6 (\u001b[38;5;33mConv2D\u001b[0m)               │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m30\u001b[0m, \u001b[38;5;34m30\u001b[0m, \u001b[38;5;34m32\u001b[0m)     │           \u001b[38;5;34m896\u001b[0m │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ max_pooling2d_4 (\u001b[38;5;33mMaxPooling2D\u001b[0m)  │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m15\u001b[0m, \u001b[38;5;34m15\u001b[0m, \u001b[38;5;34m32\u001b[0m)     │             \u001b[38;5;34m0\u001b[0m │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ conv2d_7 (\u001b[38;5;33mConv2D\u001b[0m)               │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m13\u001b[0m, \u001b[38;5;34m13\u001b[0m, \u001b[38;5;34m64\u001b[0m)     │        \u001b[38;5;34m18,496\u001b[0m │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ max_pooling2d_5 (\u001b[38;5;33mMaxPooling2D\u001b[0m)  │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m6\u001b[0m, \u001b[38;5;34m6\u001b[0m, \u001b[38;5;34m64\u001b[0m)       │             \u001b[38;5;34m0\u001b[0m │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ conv2d_8 (\u001b[38;5;33mConv2D\u001b[0m)               │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m4\u001b[0m, \u001b[38;5;34m4\u001b[0m, \u001b[38;5;34m64\u001b[0m)       │        \u001b[38;5;34m36,928\u001b[0m │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ flatten_2 (\u001b[38;5;33mFlatten\u001b[0m)             │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m1024\u001b[0m)           │             \u001b[38;5;34m0\u001b[0m │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ predictions (\u001b[38;5;33mDense\u001b[0m)             │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m10\u001b[0m)             │        \u001b[38;5;34m10,250\u001b[0m │\n",
       "└─────────────────────────────────┴────────────────────────┴───────────────┘\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"font-weight: bold\"> Total params: </span><span style=\"color: #00af00; text-decoration-color: #00af00\">66,570</span> (260.04 KB)\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[1m Total params: \u001b[0m\u001b[38;5;34m66,570\u001b[0m (260.04 KB)\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"font-weight: bold\"> Trainable params: </span><span style=\"color: #00af00; text-decoration-color: #00af00\">66,570</span> (260.04 KB)\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[1m Trainable params: \u001b[0m\u001b[38;5;34m66,570\u001b[0m (260.04 KB)\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"font-weight: bold\"> Non-trainable params: </span><span style=\"color: #00af00; text-decoration-color: #00af00\">0</span> (0.00 B)\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[1m Non-trainable params: \u001b[0m\u001b[38;5;34m0\u001b[0m (0.00 B)\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/10\n",
      "\u001b[1m782/782\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 4ms/step - categorical_accuracy: 0.1779 - loss: 16.2245"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 1: train accuracy = -1, val accuracy = -1, test accuracy = 0.16419999301433563, time per epoch = 4.83 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m782/782\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m6s\u001b[0m 5ms/step - categorical_accuracy: 0.1779 - loss: 16.2131\n",
      "Epoch 2/10\n",
      "\u001b[1m779/782\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m━\u001b[0m \u001b[1m0s\u001b[0m 3ms/step - categorical_accuracy: 0.1963 - loss: 2.2182"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 2: train accuracy = -1, val accuracy = -1, test accuracy = 0.24770000576972961, time per epoch = 2.45 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m782/782\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m3s\u001b[0m 3ms/step - categorical_accuracy: 0.1964 - loss: 2.2179\n",
      "Epoch 3/10\n",
      "\u001b[1m780/782\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m━\u001b[0m \u001b[1m0s\u001b[0m 3ms/step - categorical_accuracy: 0.2622 - loss: 2.0130"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 3: train accuracy = -1, val accuracy = -1, test accuracy = 0.290800005197525, time per epoch = 2.45 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m782/782\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m3s\u001b[0m 3ms/step - categorical_accuracy: 0.2622 - loss: 2.0128\n",
      "Epoch 4/10\n",
      "\u001b[1m773/782\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m━\u001b[0m \u001b[1m0s\u001b[0m 3ms/step - categorical_accuracy: 0.3122 - loss: 1.8555"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 4: train accuracy = -1, val accuracy = -1, test accuracy = 0.3610000014305115, time per epoch = 2.51 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m782/782\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m3s\u001b[0m 4ms/step - categorical_accuracy: 0.3123 - loss: 1.8550\n",
      "Epoch 5/10\n",
      "\u001b[1m780/782\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m━\u001b[0m \u001b[1m0s\u001b[0m 3ms/step - categorical_accuracy: 0.3702 - loss: 1.7060"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 5: train accuracy = -1, val accuracy = -1, test accuracy = 0.4059000015258789, time per epoch = 2.53 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m782/782\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m3s\u001b[0m 4ms/step - categorical_accuracy: 0.3703 - loss: 1.7059\n",
      "Epoch 6/10\n",
      "\u001b[1m781/782\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m━\u001b[0m \u001b[1m0s\u001b[0m 3ms/step - categorical_accuracy: 0.4215 - loss: 1.5885"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 6: train accuracy = -1, val accuracy = -1, test accuracy = 0.4352000057697296, time per epoch = 2.27 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m782/782\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m3s\u001b[0m 3ms/step - categorical_accuracy: 0.4215 - loss: 1.5885\n",
      "Epoch 7/10\n",
      "\u001b[1m778/782\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m━\u001b[0m \u001b[1m0s\u001b[0m 3ms/step - categorical_accuracy: 0.4511 - loss: 1.5053"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 7: train accuracy = -1, val accuracy = -1, test accuracy = 0.46480000019073486, time per epoch = 2.28 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m782/782\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m3s\u001b[0m 3ms/step - categorical_accuracy: 0.4511 - loss: 1.5053\n",
      "Epoch 8/10\n",
      "\u001b[1m770/782\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m━\u001b[0m \u001b[1m0s\u001b[0m 3ms/step - categorical_accuracy: 0.4776 - loss: 1.4301"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 8: train accuracy = -1, val accuracy = -1, test accuracy = 0.4717999994754791, time per epoch = 2.42 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m782/782\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m3s\u001b[0m 3ms/step - categorical_accuracy: 0.4777 - loss: 1.4299\n",
      "Epoch 9/10\n",
      "\u001b[1m766/782\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m━\u001b[0m \u001b[1m0s\u001b[0m 3ms/step - categorical_accuracy: 0.5040 - loss: 1.3664"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 9: train accuracy = -1, val accuracy = -1, test accuracy = 0.5041000247001648, time per epoch = 2.25 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m782/782\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m3s\u001b[0m 3ms/step - categorical_accuracy: 0.5041 - loss: 1.3664\n",
      "Epoch 10/10\n",
      "\u001b[1m779/782\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m━\u001b[0m \u001b[1m0s\u001b[0m 3ms/step - categorical_accuracy: 0.5211 - loss: 1.3211"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 10: train accuracy = -1, val accuracy = -1, test accuracy = 0.515500009059906, time per epoch = 2.17 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m782/782\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2s\u001b[0m 3ms/step - categorical_accuracy: 0.5211 - loss: 1.3211\n"
     ]
    }
   ],
   "source": [
    "cnn_pseudo_stats = [run_experiment(\n",
    "    experiment_id=1,\n",
    "    sequence_scheme=SEQUENCE_SCHEME,\n",
    "    units=32,\n",
    "    dataset_id=\"cifar10\",\n",
    "    kernel_initializer=\"lecun-normal\",\n",
    "    initializer_type=\"pseudo-random\",\n",
    "    model_id=\"baseline-cnn\",\n",
    "    epochs=EPOCHS,\n",
    "    auto_epoch_count=AUTO_EPOCH_COUNT,\n",
    "    auto_min_seed_value=AUTO_MIN_SEED_VALUE,\n",
    "    auto_max_seed_value=AUTO_MAX_SEED_VALUE,\n",
    "    auto_seeds_count=AUTO_SEEDS_COUNT,\n",
    ")[0] for _ in range(NUM_REPS)]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### IMDB LSTM -- Text Classification"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:datapipeline:Loading IMDB reviews dataset\n",
      "INFO:datapipeline:Converting labels to one-hot encoding\n",
      "INFO:datapipeline:IMDB reviews dataset loaded\n",
      "INFO:datapipeline:x_train shape (25000, 512)\n",
      "INFO:datapipeline:y_train shape (25000, 2)\n",
      "INFO:datapipeline:x_test shape (25000, 512)\n",
      "INFO:datapipeline:y_test shape (25000, 2)\n",
      "INFO:utils:Splitting Data into Train and Validation\n",
      "INFO:utils:Training observation count is set to 25000\n",
      "INFO:utils:x_train shape before splitting (25000, 512)\n",
      "INFO:utils:y_train shape before splitting (25000, 2)\n",
      "INFO:utils:Training observation count equals the length of x_train. No validation set will be created.\n",
      "INFO:utils:x_train shape after splitting (25000, 512)\n",
      "INFO:utils:y_train shape after splitting (25000, 2)\n",
      "INFO:utils:x_val shape after splitting (0, 512)\n",
      "INFO:utils:y_val shape after splitting (0, 2)\n",
      "INFO:train_and_eval:Choosing the best seed from this set: [1, 2, 3, 6, 8]\n",
      "INFO:models:Initialized ModelBuilder with model_id=baseline-lstm, input_shape=[512], output_shape=2, output_activation=softmax, model_config={'initializer_type': 'quasi-random', 'kernel_initializer': 'random-normal', 'bias_initializer': 'zeros', 'recurrent_initializer': 'orthogonal', 'max_features': 20000, 'units': 64, 'embedding_dim': 32}\n",
      "INFO:trainer:Initializing random-normal from quasi-random, seed=None\n",
      "INFO:trainer:Initializing orthogonal from quasi-random, seed=None\n",
      "INFO:trainer:Initializing zeros from quasi-random, seed=None\n",
      "INFO:trainer:Initializing random-uniform from pseudo-random, seed=None\n",
      "INFO:custom_initializers:Setting seed value for RandomNormalQR to 1.\n",
      "INFO:custom_initializers:Setting seed value for OrthogonalInitializerQR to 1.\n",
      "INFO:custom_initializers:Setting seed value for RandomNormalQR to 2.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m390/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m━\u001b[0m \u001b[1m0s\u001b[0m 36ms/step - categorical_accuracy: 0.5568 - loss: 0.6803"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 1: train accuracy = -1, val accuracy = -1, test accuracy = 0.7870399951934814, time per epoch = 15.23 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m22s\u001b[0m 54ms/step - categorical_accuracy: 0.5573 - loss: 0.6801\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:models:Initialized ModelBuilder with model_id=baseline-lstm, input_shape=[512], output_shape=2, output_activation=softmax, model_config={'initializer_type': 'quasi-random', 'kernel_initializer': 'random-normal', 'bias_initializer': 'zeros', 'recurrent_initializer': 'orthogonal', 'max_features': 20000, 'units': 64, 'embedding_dim': 32}\n",
      "INFO:trainer:Initializing random-normal from quasi-random, seed=None\n",
      "INFO:trainer:Initializing orthogonal from quasi-random, seed=None\n",
      "INFO:trainer:Initializing zeros from quasi-random, seed=None\n",
      "INFO:trainer:Initializing random-uniform from pseudo-random, seed=None\n",
      "INFO:custom_initializers:Setting seed value for RandomNormalQR to 2.\n",
      "INFO:custom_initializers:Setting seed value for OrthogonalInitializerQR to 2.\n",
      "INFO:custom_initializers:Setting seed value for RandomNormalQR to 3.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 42ms/step - categorical_accuracy: 0.5662 - loss: 0.6735"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 1: train accuracy = -1, val accuracy = -1, test accuracy = 0.785319983959198, time per epoch = 17.23 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m25s\u001b[0m 61ms/step - categorical_accuracy: 0.5665 - loss: 0.6733\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:models:Initialized ModelBuilder with model_id=baseline-lstm, input_shape=[512], output_shape=2, output_activation=softmax, model_config={'initializer_type': 'quasi-random', 'kernel_initializer': 'random-normal', 'bias_initializer': 'zeros', 'recurrent_initializer': 'orthogonal', 'max_features': 20000, 'units': 64, 'embedding_dim': 32}\n",
      "INFO:trainer:Initializing random-normal from quasi-random, seed=None\n",
      "INFO:trainer:Initializing orthogonal from quasi-random, seed=None\n",
      "INFO:trainer:Initializing zeros from quasi-random, seed=None\n",
      "INFO:trainer:Initializing random-uniform from pseudo-random, seed=None\n",
      "INFO:custom_initializers:Setting seed value for RandomNormalQR to 3.\n",
      "INFO:custom_initializers:Setting seed value for OrthogonalInitializerQR to 3.\n",
      "INFO:custom_initializers:Setting seed value for RandomNormalQR to 4.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 39ms/step - categorical_accuracy: 0.5299 - loss: 0.6899"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 1: train accuracy = -1, val accuracy = -1, test accuracy = 0.730679988861084, time per epoch = 16.28 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m23s\u001b[0m 56ms/step - categorical_accuracy: 0.5301 - loss: 0.6898\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:models:Initialized ModelBuilder with model_id=baseline-lstm, input_shape=[512], output_shape=2, output_activation=softmax, model_config={'initializer_type': 'quasi-random', 'kernel_initializer': 'random-normal', 'bias_initializer': 'zeros', 'recurrent_initializer': 'orthogonal', 'max_features': 20000, 'units': 64, 'embedding_dim': 32}\n",
      "INFO:trainer:Initializing random-normal from quasi-random, seed=None\n",
      "INFO:trainer:Initializing orthogonal from quasi-random, seed=None\n",
      "INFO:trainer:Initializing zeros from quasi-random, seed=None\n",
      "INFO:trainer:Initializing random-uniform from pseudo-random, seed=None\n",
      "INFO:custom_initializers:Setting seed value for RandomNormalQR to 6.\n",
      "INFO:custom_initializers:Setting seed value for OrthogonalInitializerQR to 6.\n",
      "INFO:custom_initializers:Setting seed value for RandomNormalQR to 7.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 38ms/step - categorical_accuracy: 0.5443 - loss: 0.6842"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 1: train accuracy = -1, val accuracy = -1, test accuracy = 0.7694799900054932, time per epoch = 15.91 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m23s\u001b[0m 57ms/step - categorical_accuracy: 0.5445 - loss: 0.6841\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:models:Initialized ModelBuilder with model_id=baseline-lstm, input_shape=[512], output_shape=2, output_activation=softmax, model_config={'initializer_type': 'quasi-random', 'kernel_initializer': 'random-normal', 'bias_initializer': 'zeros', 'recurrent_initializer': 'orthogonal', 'max_features': 20000, 'units': 64, 'embedding_dim': 32}\n",
      "INFO:trainer:Initializing random-normal from quasi-random, seed=None\n",
      "INFO:trainer:Initializing orthogonal from quasi-random, seed=None\n",
      "INFO:trainer:Initializing zeros from quasi-random, seed=None\n",
      "INFO:trainer:Initializing random-uniform from pseudo-random, seed=None\n",
      "INFO:custom_initializers:Setting seed value for RandomNormalQR to 8.\n",
      "INFO:custom_initializers:Setting seed value for OrthogonalInitializerQR to 8.\n",
      "INFO:custom_initializers:Setting seed value for RandomNormalQR to 9.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 38ms/step - categorical_accuracy: 0.5302 - loss: 0.6834"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 1: train accuracy = -1, val accuracy = -1, test accuracy = 0.7825199961662292, time per epoch = 15.97 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m23s\u001b[0m 57ms/step - categorical_accuracy: 0.5304 - loss: 0.6833\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:train_and_eval:Resetting the state. Starting QRNG dimension is now set to 1.\n",
      "INFO:train_and_eval:Running experiment #1\n",
      "INFO:models:Initialized ModelBuilder with model_id=baseline-lstm, input_shape=[512], output_shape=2, output_activation=softmax, model_config={'initializer_type': 'quasi-random', 'kernel_initializer': 'random-normal', 'bias_initializer': 'zeros', 'recurrent_initializer': 'orthogonal', 'max_features': 20000, 'units': 64, 'embedding_dim': 32}\n",
      "INFO:trainer:Initializing random-normal from quasi-random, seed=None\n",
      "INFO:trainer:Initializing orthogonal from quasi-random, seed=None\n",
      "INFO:trainer:Initializing zeros from quasi-random, seed=None\n",
      "INFO:trainer:Initializing random-uniform from pseudo-random, seed=None\n",
      "INFO:custom_initializers:Setting seed value for RandomNormalQR to 1.\n",
      "INFO:custom_initializers:Setting seed value for OrthogonalInitializerQR to 1.\n",
      "INFO:custom_initializers:Setting seed value for RandomNormalQR to 2.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"font-weight: bold\">Model: \"functional_1\"</span>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[1mModel: \"functional_1\"\u001b[0m\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓\n",
       "┃<span style=\"font-weight: bold\"> Layer (type)                    </span>┃<span style=\"font-weight: bold\"> Output Shape           </span>┃<span style=\"font-weight: bold\">       Param # </span>┃\n",
       "┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩\n",
       "│ input_layer (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">InputLayer</span>)        │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">512</span>)            │             <span style=\"color: #00af00; text-decoration-color: #00af00\">0</span> │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ embedding (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">Embedding</span>)           │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">512</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">32</span>)        │       <span style=\"color: #00af00; text-decoration-color: #00af00\">640,000</span> │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ lstm (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">LSTM</span>)                     │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">64</span>)             │        <span style=\"color: #00af00; text-decoration-color: #00af00\">24,832</span> │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ flatten (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">Flatten</span>)               │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">64</span>)             │             <span style=\"color: #00af00; text-decoration-color: #00af00\">0</span> │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ predictions (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">Dense</span>)             │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">2</span>)              │           <span style=\"color: #00af00; text-decoration-color: #00af00\">130</span> │\n",
       "└─────────────────────────────────┴────────────────────────┴───────────────┘\n",
       "</pre>\n"
      ],
      "text/plain": [
       "┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓\n",
       "┃\u001b[1m \u001b[0m\u001b[1mLayer (type)                   \u001b[0m\u001b[1m \u001b[0m┃\u001b[1m \u001b[0m\u001b[1mOutput Shape          \u001b[0m\u001b[1m \u001b[0m┃\u001b[1m \u001b[0m\u001b[1m      Param #\u001b[0m\u001b[1m \u001b[0m┃\n",
       "┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩\n",
       "│ input_layer (\u001b[38;5;33mInputLayer\u001b[0m)        │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m512\u001b[0m)            │             \u001b[38;5;34m0\u001b[0m │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ embedding (\u001b[38;5;33mEmbedding\u001b[0m)           │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m512\u001b[0m, \u001b[38;5;34m32\u001b[0m)        │       \u001b[38;5;34m640,000\u001b[0m │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ lstm (\u001b[38;5;33mLSTM\u001b[0m)                     │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m64\u001b[0m)             │        \u001b[38;5;34m24,832\u001b[0m │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ flatten (\u001b[38;5;33mFlatten\u001b[0m)               │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m64\u001b[0m)             │             \u001b[38;5;34m0\u001b[0m │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ predictions (\u001b[38;5;33mDense\u001b[0m)             │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m2\u001b[0m)              │           \u001b[38;5;34m130\u001b[0m │\n",
       "└─────────────────────────────────┴────────────────────────┴───────────────┘\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"font-weight: bold\"> Total params: </span><span style=\"color: #00af00; text-decoration-color: #00af00\">664,962</span> (2.54 MB)\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[1m Total params: \u001b[0m\u001b[38;5;34m664,962\u001b[0m (2.54 MB)\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"font-weight: bold\"> Trainable params: </span><span style=\"color: #00af00; text-decoration-color: #00af00\">664,962</span> (2.54 MB)\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[1m Trainable params: \u001b[0m\u001b[38;5;34m664,962\u001b[0m (2.54 MB)\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"font-weight: bold\"> Non-trainable params: </span><span style=\"color: #00af00; text-decoration-color: #00af00\">0</span> (0.00 B)\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[1m Non-trainable params: \u001b[0m\u001b[38;5;34m0\u001b[0m (0.00 B)\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/10\n",
      "\u001b[1m390/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m━\u001b[0m \u001b[1m0s\u001b[0m 37ms/step - categorical_accuracy: 0.5881 - loss: 0.6731"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 1: train accuracy = -1, val accuracy = -1, test accuracy = 0.7676399946212769, time per epoch = 15.40 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m22s\u001b[0m 53ms/step - categorical_accuracy: 0.5886 - loss: 0.6729\n",
      "Epoch 2/10\n",
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 37ms/step - categorical_accuracy: 0.8186 - loss: 0.4363"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 2: train accuracy = -1, val accuracy = -1, test accuracy = 0.8496000170707703, time per epoch = 14.60 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m21s\u001b[0m 54ms/step - categorical_accuracy: 0.8186 - loss: 0.4362\n",
      "Epoch 3/10\n",
      "\u001b[1m390/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m━\u001b[0m \u001b[1m0s\u001b[0m 44ms/step - categorical_accuracy: 0.8857 - loss: 0.2906"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 3: train accuracy = -1, val accuracy = -1, test accuracy = 0.8633599877357483, time per epoch = 17.30 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m24s\u001b[0m 61ms/step - categorical_accuracy: 0.8858 - loss: 0.2905\n",
      "Epoch 4/10\n",
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 38ms/step - categorical_accuracy: 0.9185 - loss: 0.2281"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 4: train accuracy = -1, val accuracy = -1, test accuracy = 0.8738800287246704, time per epoch = 14.82 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m21s\u001b[0m 55ms/step - categorical_accuracy: 0.9185 - loss: 0.2280\n",
      "Epoch 5/10\n",
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 36ms/step - categorical_accuracy: 0.9349 - loss: 0.1868"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 5: train accuracy = -1, val accuracy = -1, test accuracy = 0.875760018825531, time per epoch = 14.10 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m20s\u001b[0m 51ms/step - categorical_accuracy: 0.9349 - loss: 0.1867\n",
      "Epoch 6/10\n",
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 35ms/step - categorical_accuracy: 0.9494 - loss: 0.1542"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 6: train accuracy = -1, val accuracy = -1, test accuracy = 0.8752400279045105, time per epoch = 13.77 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m20s\u001b[0m 52ms/step - categorical_accuracy: 0.9494 - loss: 0.1541\n",
      "Epoch 7/10\n",
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 41ms/step - categorical_accuracy: 0.9602 - loss: 0.1288"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 7: train accuracy = -1, val accuracy = -1, test accuracy = 0.861840009689331, time per epoch = 15.98 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m22s\u001b[0m 57ms/step - categorical_accuracy: 0.9602 - loss: 0.1288\n",
      "Epoch 8/10\n",
      "\u001b[1m390/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m━\u001b[0m \u001b[1m0s\u001b[0m 39ms/step - categorical_accuracy: 0.9660 - loss: 0.1116"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 8: train accuracy = -1, val accuracy = -1, test accuracy = 0.866919994354248, time per epoch = 15.21 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m22s\u001b[0m 56ms/step - categorical_accuracy: 0.9660 - loss: 0.1116\n",
      "Epoch 9/10\n",
      "\u001b[1m390/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m━\u001b[0m \u001b[1m0s\u001b[0m 39ms/step - categorical_accuracy: 0.9741 - loss: 0.0900"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 9: train accuracy = -1, val accuracy = -1, test accuracy = 0.8654400110244751, time per epoch = 15.30 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m22s\u001b[0m 56ms/step - categorical_accuracy: 0.9741 - loss: 0.0899\n",
      "Epoch 10/10\n",
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 37ms/step - categorical_accuracy: 0.9804 - loss: 0.0762"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 10: train accuracy = -1, val accuracy = -1, test accuracy = 0.8505600094795227, time per epoch = 14.39 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m20s\u001b[0m 52ms/step - categorical_accuracy: 0.9804 - loss: 0.0762\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:datapipeline:Loading IMDB reviews dataset\n",
      "INFO:datapipeline:Converting labels to one-hot encoding\n",
      "INFO:datapipeline:IMDB reviews dataset loaded\n",
      "INFO:datapipeline:x_train shape (25000, 512)\n",
      "INFO:datapipeline:y_train shape (25000, 2)\n",
      "INFO:datapipeline:x_test shape (25000, 512)\n",
      "INFO:datapipeline:y_test shape (25000, 2)\n",
      "INFO:utils:Splitting Data into Train and Validation\n",
      "INFO:utils:Training observation count is set to 25000\n",
      "INFO:utils:x_train shape before splitting (25000, 512)\n",
      "INFO:utils:y_train shape before splitting (25000, 2)\n",
      "INFO:utils:Training observation count equals the length of x_train. No validation set will be created.\n",
      "INFO:utils:x_train shape after splitting (25000, 512)\n",
      "INFO:utils:y_train shape after splitting (25000, 2)\n",
      "INFO:utils:x_val shape after splitting (0, 512)\n",
      "INFO:utils:y_val shape after splitting (0, 2)\n",
      "INFO:train_and_eval:Choosing the best seed from this set: [2, 4, 7, 8, 10]\n",
      "INFO:models:Initialized ModelBuilder with model_id=baseline-lstm, input_shape=[512], output_shape=2, output_activation=softmax, model_config={'initializer_type': 'quasi-random', 'kernel_initializer': 'random-normal', 'bias_initializer': 'zeros', 'recurrent_initializer': 'orthogonal', 'max_features': 20000, 'units': 64, 'embedding_dim': 32}\n",
      "INFO:trainer:Initializing random-normal from quasi-random, seed=None\n",
      "INFO:trainer:Initializing orthogonal from quasi-random, seed=None\n",
      "INFO:trainer:Initializing zeros from quasi-random, seed=None\n",
      "INFO:trainer:Initializing random-uniform from pseudo-random, seed=None\n",
      "INFO:custom_initializers:Setting seed value for RandomNormalQR to 2.\n",
      "INFO:custom_initializers:Setting seed value for OrthogonalInitializerQR to 2.\n",
      "INFO:custom_initializers:Setting seed value for RandomNormalQR to 3.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 40ms/step - categorical_accuracy: 0.5870 - loss: 0.6673"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 1: train accuracy = -1, val accuracy = -1, test accuracy = 0.7717199921607971, time per epoch = 16.65 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m23s\u001b[0m 56ms/step - categorical_accuracy: 0.5872 - loss: 0.6671\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:models:Initialized ModelBuilder with model_id=baseline-lstm, input_shape=[512], output_shape=2, output_activation=softmax, model_config={'initializer_type': 'quasi-random', 'kernel_initializer': 'random-normal', 'bias_initializer': 'zeros', 'recurrent_initializer': 'orthogonal', 'max_features': 20000, 'units': 64, 'embedding_dim': 32}\n",
      "INFO:trainer:Initializing random-normal from quasi-random, seed=None\n",
      "INFO:trainer:Initializing orthogonal from quasi-random, seed=None\n",
      "INFO:trainer:Initializing zeros from quasi-random, seed=None\n",
      "INFO:trainer:Initializing random-uniform from pseudo-random, seed=None\n",
      "INFO:custom_initializers:Setting seed value for RandomNormalQR to 4.\n",
      "INFO:custom_initializers:Setting seed value for OrthogonalInitializerQR to 4.\n",
      "INFO:custom_initializers:Setting seed value for RandomNormalQR to 5.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m390/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m━\u001b[0m \u001b[1m0s\u001b[0m 34ms/step - categorical_accuracy: 0.5444 - loss: 0.6849"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 1: train accuracy = -1, val accuracy = -1, test accuracy = 0.7742800116539001, time per epoch = 14.24 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m21s\u001b[0m 51ms/step - categorical_accuracy: 0.5448 - loss: 0.6848\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:models:Initialized ModelBuilder with model_id=baseline-lstm, input_shape=[512], output_shape=2, output_activation=softmax, model_config={'initializer_type': 'quasi-random', 'kernel_initializer': 'random-normal', 'bias_initializer': 'zeros', 'recurrent_initializer': 'orthogonal', 'max_features': 20000, 'units': 64, 'embedding_dim': 32}\n",
      "INFO:trainer:Initializing random-normal from quasi-random, seed=None\n",
      "INFO:trainer:Initializing orthogonal from quasi-random, seed=None\n",
      "INFO:trainer:Initializing zeros from quasi-random, seed=None\n",
      "INFO:trainer:Initializing random-uniform from pseudo-random, seed=None\n",
      "INFO:custom_initializers:Setting seed value for RandomNormalQR to 7.\n",
      "INFO:custom_initializers:Setting seed value for OrthogonalInitializerQR to 7.\n",
      "INFO:custom_initializers:Setting seed value for RandomNormalQR to 8.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 39ms/step - categorical_accuracy: 0.5358 - loss: 0.6845"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 1: train accuracy = -1, val accuracy = -1, test accuracy = 0.7792400121688843, time per epoch = 16.19 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m23s\u001b[0m 56ms/step - categorical_accuracy: 0.5360 - loss: 0.6844\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:models:Initialized ModelBuilder with model_id=baseline-lstm, input_shape=[512], output_shape=2, output_activation=softmax, model_config={'initializer_type': 'quasi-random', 'kernel_initializer': 'random-normal', 'bias_initializer': 'zeros', 'recurrent_initializer': 'orthogonal', 'max_features': 20000, 'units': 64, 'embedding_dim': 32}\n",
      "INFO:trainer:Initializing random-normal from quasi-random, seed=None\n",
      "INFO:trainer:Initializing orthogonal from quasi-random, seed=None\n",
      "INFO:trainer:Initializing zeros from quasi-random, seed=None\n",
      "INFO:trainer:Initializing random-uniform from pseudo-random, seed=None\n",
      "INFO:custom_initializers:Setting seed value for RandomNormalQR to 8.\n",
      "INFO:custom_initializers:Setting seed value for OrthogonalInitializerQR to 8.\n",
      "INFO:custom_initializers:Setting seed value for RandomNormalQR to 9.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 40ms/step - categorical_accuracy: 0.5485 - loss: 0.6769"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 1: train accuracy = -1, val accuracy = -1, test accuracy = 0.8080800175666809, time per epoch = 16.57 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m23s\u001b[0m 57ms/step - categorical_accuracy: 0.5487 - loss: 0.6768\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:models:Initialized ModelBuilder with model_id=baseline-lstm, input_shape=[512], output_shape=2, output_activation=softmax, model_config={'initializer_type': 'quasi-random', 'kernel_initializer': 'random-normal', 'bias_initializer': 'zeros', 'recurrent_initializer': 'orthogonal', 'max_features': 20000, 'units': 64, 'embedding_dim': 32}\n",
      "INFO:trainer:Initializing random-normal from quasi-random, seed=None\n",
      "INFO:trainer:Initializing orthogonal from quasi-random, seed=None\n",
      "INFO:trainer:Initializing zeros from quasi-random, seed=None\n",
      "INFO:trainer:Initializing random-uniform from pseudo-random, seed=None\n",
      "INFO:custom_initializers:Setting seed value for RandomNormalQR to 10.\n",
      "INFO:custom_initializers:Setting seed value for OrthogonalInitializerQR to 10.\n",
      "INFO:custom_initializers:Setting seed value for RandomNormalQR to 11.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 40ms/step - categorical_accuracy: 0.5876 - loss: 0.6732"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 1: train accuracy = -1, val accuracy = -1, test accuracy = 0.8024799823760986, time per epoch = 16.68 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m23s\u001b[0m 57ms/step - categorical_accuracy: 0.5878 - loss: 0.6730\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:train_and_eval:Resetting the state. Starting QRNG dimension is now set to 8.\n",
      "INFO:train_and_eval:Running experiment #1\n",
      "INFO:models:Initialized ModelBuilder with model_id=baseline-lstm, input_shape=[512], output_shape=2, output_activation=softmax, model_config={'initializer_type': 'quasi-random', 'kernel_initializer': 'random-normal', 'bias_initializer': 'zeros', 'recurrent_initializer': 'orthogonal', 'max_features': 20000, 'units': 64, 'embedding_dim': 32}\n",
      "INFO:trainer:Initializing random-normal from quasi-random, seed=None\n",
      "INFO:trainer:Initializing orthogonal from quasi-random, seed=None\n",
      "INFO:trainer:Initializing zeros from quasi-random, seed=None\n",
      "INFO:trainer:Initializing random-uniform from pseudo-random, seed=None\n",
      "INFO:custom_initializers:Setting seed value for RandomNormalQR to 8.\n",
      "INFO:custom_initializers:Setting seed value for OrthogonalInitializerQR to 8.\n",
      "INFO:custom_initializers:Setting seed value for RandomNormalQR to 9.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"font-weight: bold\">Model: \"functional_1\"</span>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[1mModel: \"functional_1\"\u001b[0m\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓\n",
       "┃<span style=\"font-weight: bold\"> Layer (type)                    </span>┃<span style=\"font-weight: bold\"> Output Shape           </span>┃<span style=\"font-weight: bold\">       Param # </span>┃\n",
       "┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩\n",
       "│ input_layer (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">InputLayer</span>)        │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">512</span>)            │             <span style=\"color: #00af00; text-decoration-color: #00af00\">0</span> │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ embedding (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">Embedding</span>)           │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">512</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">32</span>)        │       <span style=\"color: #00af00; text-decoration-color: #00af00\">640,000</span> │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ lstm (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">LSTM</span>)                     │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">64</span>)             │        <span style=\"color: #00af00; text-decoration-color: #00af00\">24,832</span> │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ flatten (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">Flatten</span>)               │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">64</span>)             │             <span style=\"color: #00af00; text-decoration-color: #00af00\">0</span> │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ predictions (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">Dense</span>)             │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">2</span>)              │           <span style=\"color: #00af00; text-decoration-color: #00af00\">130</span> │\n",
       "└─────────────────────────────────┴────────────────────────┴───────────────┘\n",
       "</pre>\n"
      ],
      "text/plain": [
       "┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓\n",
       "┃\u001b[1m \u001b[0m\u001b[1mLayer (type)                   \u001b[0m\u001b[1m \u001b[0m┃\u001b[1m \u001b[0m\u001b[1mOutput Shape          \u001b[0m\u001b[1m \u001b[0m┃\u001b[1m \u001b[0m\u001b[1m      Param #\u001b[0m\u001b[1m \u001b[0m┃\n",
       "┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩\n",
       "│ input_layer (\u001b[38;5;33mInputLayer\u001b[0m)        │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m512\u001b[0m)            │             \u001b[38;5;34m0\u001b[0m │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ embedding (\u001b[38;5;33mEmbedding\u001b[0m)           │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m512\u001b[0m, \u001b[38;5;34m32\u001b[0m)        │       \u001b[38;5;34m640,000\u001b[0m │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ lstm (\u001b[38;5;33mLSTM\u001b[0m)                     │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m64\u001b[0m)             │        \u001b[38;5;34m24,832\u001b[0m │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ flatten (\u001b[38;5;33mFlatten\u001b[0m)               │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m64\u001b[0m)             │             \u001b[38;5;34m0\u001b[0m │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ predictions (\u001b[38;5;33mDense\u001b[0m)             │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m2\u001b[0m)              │           \u001b[38;5;34m130\u001b[0m │\n",
       "└─────────────────────────────────┴────────────────────────┴───────────────┘\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"font-weight: bold\"> Total params: </span><span style=\"color: #00af00; text-decoration-color: #00af00\">664,962</span> (2.54 MB)\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[1m Total params: \u001b[0m\u001b[38;5;34m664,962\u001b[0m (2.54 MB)\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"font-weight: bold\"> Trainable params: </span><span style=\"color: #00af00; text-decoration-color: #00af00\">664,962</span> (2.54 MB)\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[1m Trainable params: \u001b[0m\u001b[38;5;34m664,962\u001b[0m (2.54 MB)\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"font-weight: bold\"> Non-trainable params: </span><span style=\"color: #00af00; text-decoration-color: #00af00\">0</span> (0.00 B)\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[1m Non-trainable params: \u001b[0m\u001b[38;5;34m0\u001b[0m (0.00 B)\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/10\n",
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 37ms/step - categorical_accuracy: 0.5647 - loss: 0.6795"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 1: train accuracy = -1, val accuracy = -1, test accuracy = 0.7866399884223938, time per epoch = 15.35 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m21s\u001b[0m 52ms/step - categorical_accuracy: 0.5649 - loss: 0.6794\n",
      "Epoch 2/10\n",
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 33ms/step - categorical_accuracy: 0.8150 - loss: 0.4585"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 2: train accuracy = -1, val accuracy = -1, test accuracy = 0.8529599905014038, time per epoch = 12.97 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m19s\u001b[0m 50ms/step - categorical_accuracy: 0.8151 - loss: 0.4584\n",
      "Epoch 3/10\n",
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 39ms/step - categorical_accuracy: 0.8798 - loss: 0.3037"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 3: train accuracy = -1, val accuracy = -1, test accuracy = 0.859279990196228, time per epoch = 15.06 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m21s\u001b[0m 55ms/step - categorical_accuracy: 0.8799 - loss: 0.3036\n",
      "Epoch 4/10\n",
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 42ms/step - categorical_accuracy: 0.9160 - loss: 0.2294"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 4: train accuracy = -1, val accuracy = -1, test accuracy = 0.8805599808692932, time per epoch = 16.29 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m23s\u001b[0m 60ms/step - categorical_accuracy: 0.9160 - loss: 0.2294\n",
      "Epoch 5/10\n",
      "\u001b[1m390/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m━\u001b[0m \u001b[1m0s\u001b[0m 39ms/step - categorical_accuracy: 0.9362 - loss: 0.1791"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 5: train accuracy = -1, val accuracy = -1, test accuracy = 0.8784800171852112, time per epoch = 15.36 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m22s\u001b[0m 55ms/step - categorical_accuracy: 0.9362 - loss: 0.1790\n",
      "Epoch 6/10\n",
      "\u001b[1m390/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m━\u001b[0m \u001b[1m0s\u001b[0m 35ms/step - categorical_accuracy: 0.9511 - loss: 0.1491"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 6: train accuracy = -1, val accuracy = -1, test accuracy = 0.8775200247764587, time per epoch = 13.77 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m20s\u001b[0m 51ms/step - categorical_accuracy: 0.9512 - loss: 0.1491\n",
      "Epoch 7/10\n",
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 40ms/step - categorical_accuracy: 0.9596 - loss: 0.1236"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 7: train accuracy = -1, val accuracy = -1, test accuracy = 0.875760018825531, time per epoch = 15.50 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m22s\u001b[0m 55ms/step - categorical_accuracy: 0.9596 - loss: 0.1236\n",
      "Epoch 8/10\n",
      "\u001b[1m390/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m━\u001b[0m \u001b[1m0s\u001b[0m 39ms/step - categorical_accuracy: 0.9702 - loss: 0.1004"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 8: train accuracy = -1, val accuracy = -1, test accuracy = 0.8708400130271912, time per epoch = 15.16 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m22s\u001b[0m 56ms/step - categorical_accuracy: 0.9702 - loss: 0.1004\n",
      "Epoch 9/10\n",
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 36ms/step - categorical_accuracy: 0.9768 - loss: 0.0826"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 9: train accuracy = -1, val accuracy = -1, test accuracy = 0.8666399717330933, time per epoch = 14.15 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m20s\u001b[0m 52ms/step - categorical_accuracy: 0.9768 - loss: 0.0826\n",
      "Epoch 10/10\n",
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 33ms/step - categorical_accuracy: 0.9804 - loss: 0.0709"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 10: train accuracy = -1, val accuracy = -1, test accuracy = 0.8650400042533875, time per epoch = 13.09 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m19s\u001b[0m 49ms/step - categorical_accuracy: 0.9804 - loss: 0.0708\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:datapipeline:Loading IMDB reviews dataset\n",
      "INFO:datapipeline:Converting labels to one-hot encoding\n",
      "INFO:datapipeline:IMDB reviews dataset loaded\n",
      "INFO:datapipeline:x_train shape (25000, 512)\n",
      "INFO:datapipeline:y_train shape (25000, 2)\n",
      "INFO:datapipeline:x_test shape (25000, 512)\n",
      "INFO:datapipeline:y_test shape (25000, 2)\n",
      "INFO:utils:Splitting Data into Train and Validation\n",
      "INFO:utils:Training observation count is set to 25000\n",
      "INFO:utils:x_train shape before splitting (25000, 512)\n",
      "INFO:utils:y_train shape before splitting (25000, 2)\n",
      "INFO:utils:Training observation count equals the length of x_train. No validation set will be created.\n",
      "INFO:utils:x_train shape after splitting (25000, 512)\n",
      "INFO:utils:y_train shape after splitting (25000, 2)\n",
      "INFO:utils:x_val shape after splitting (0, 512)\n",
      "INFO:utils:y_val shape after splitting (0, 2)\n",
      "INFO:train_and_eval:Choosing the best seed from this set: [3, 5, 6, 8, 10]\n",
      "INFO:models:Initialized ModelBuilder with model_id=baseline-lstm, input_shape=[512], output_shape=2, output_activation=softmax, model_config={'initializer_type': 'quasi-random', 'kernel_initializer': 'random-normal', 'bias_initializer': 'zeros', 'recurrent_initializer': 'orthogonal', 'max_features': 20000, 'units': 64, 'embedding_dim': 32}\n",
      "INFO:trainer:Initializing random-normal from quasi-random, seed=None\n",
      "INFO:trainer:Initializing orthogonal from quasi-random, seed=None\n",
      "INFO:trainer:Initializing zeros from quasi-random, seed=None\n",
      "INFO:trainer:Initializing random-uniform from pseudo-random, seed=None\n",
      "INFO:custom_initializers:Setting seed value for RandomNormalQR to 3.\n",
      "INFO:custom_initializers:Setting seed value for OrthogonalInitializerQR to 3.\n",
      "INFO:custom_initializers:Setting seed value for RandomNormalQR to 4.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m390/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m━\u001b[0m \u001b[1m0s\u001b[0m 38ms/step - categorical_accuracy: 0.5757 - loss: 0.6739"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 1: train accuracy = -1, val accuracy = -1, test accuracy = 0.766759991645813, time per epoch = 15.81 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m23s\u001b[0m 56ms/step - categorical_accuracy: 0.5762 - loss: 0.6736\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:models:Initialized ModelBuilder with model_id=baseline-lstm, input_shape=[512], output_shape=2, output_activation=softmax, model_config={'initializer_type': 'quasi-random', 'kernel_initializer': 'random-normal', 'bias_initializer': 'zeros', 'recurrent_initializer': 'orthogonal', 'max_features': 20000, 'units': 64, 'embedding_dim': 32}\n",
      "INFO:trainer:Initializing random-normal from quasi-random, seed=None\n",
      "INFO:trainer:Initializing orthogonal from quasi-random, seed=None\n",
      "INFO:trainer:Initializing zeros from quasi-random, seed=None\n",
      "INFO:trainer:Initializing random-uniform from pseudo-random, seed=None\n",
      "INFO:custom_initializers:Setting seed value for RandomNormalQR to 5.\n",
      "INFO:custom_initializers:Setting seed value for OrthogonalInitializerQR to 5.\n",
      "INFO:custom_initializers:Setting seed value for RandomNormalQR to 6.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m390/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m━\u001b[0m \u001b[1m0s\u001b[0m 40ms/step - categorical_accuracy: 0.5557 - loss: 0.6765"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 1: train accuracy = -1, val accuracy = -1, test accuracy = 0.7776399850845337, time per epoch = 16.69 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m24s\u001b[0m 58ms/step - categorical_accuracy: 0.5561 - loss: 0.6762\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:models:Initialized ModelBuilder with model_id=baseline-lstm, input_shape=[512], output_shape=2, output_activation=softmax, model_config={'initializer_type': 'quasi-random', 'kernel_initializer': 'random-normal', 'bias_initializer': 'zeros', 'recurrent_initializer': 'orthogonal', 'max_features': 20000, 'units': 64, 'embedding_dim': 32}\n",
      "INFO:trainer:Initializing random-normal from quasi-random, seed=None\n",
      "INFO:trainer:Initializing orthogonal from quasi-random, seed=None\n",
      "INFO:trainer:Initializing zeros from quasi-random, seed=None\n",
      "INFO:trainer:Initializing random-uniform from pseudo-random, seed=None\n",
      "INFO:custom_initializers:Setting seed value for RandomNormalQR to 6.\n",
      "INFO:custom_initializers:Setting seed value for OrthogonalInitializerQR to 6.\n",
      "INFO:custom_initializers:Setting seed value for RandomNormalQR to 7.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 39ms/step - categorical_accuracy: 0.5559 - loss: 0.6780"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 1: train accuracy = -1, val accuracy = -1, test accuracy = 0.72843998670578, time per epoch = 16.15 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m23s\u001b[0m 56ms/step - categorical_accuracy: 0.5561 - loss: 0.6779\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:models:Initialized ModelBuilder with model_id=baseline-lstm, input_shape=[512], output_shape=2, output_activation=softmax, model_config={'initializer_type': 'quasi-random', 'kernel_initializer': 'random-normal', 'bias_initializer': 'zeros', 'recurrent_initializer': 'orthogonal', 'max_features': 20000, 'units': 64, 'embedding_dim': 32}\n",
      "INFO:trainer:Initializing random-normal from quasi-random, seed=None\n",
      "INFO:trainer:Initializing orthogonal from quasi-random, seed=None\n",
      "INFO:trainer:Initializing zeros from quasi-random, seed=None\n",
      "INFO:trainer:Initializing random-uniform from pseudo-random, seed=None\n",
      "INFO:custom_initializers:Setting seed value for RandomNormalQR to 8.\n",
      "INFO:custom_initializers:Setting seed value for OrthogonalInitializerQR to 8.\n",
      "INFO:custom_initializers:Setting seed value for RandomNormalQR to 9.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 37ms/step - categorical_accuracy: 0.5358 - loss: 0.6842"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 1: train accuracy = -1, val accuracy = -1, test accuracy = 0.7721199989318848, time per epoch = 15.43 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m21s\u001b[0m 52ms/step - categorical_accuracy: 0.5360 - loss: 0.6841\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:models:Initialized ModelBuilder with model_id=baseline-lstm, input_shape=[512], output_shape=2, output_activation=softmax, model_config={'initializer_type': 'quasi-random', 'kernel_initializer': 'random-normal', 'bias_initializer': 'zeros', 'recurrent_initializer': 'orthogonal', 'max_features': 20000, 'units': 64, 'embedding_dim': 32}\n",
      "INFO:trainer:Initializing random-normal from quasi-random, seed=None\n",
      "INFO:trainer:Initializing orthogonal from quasi-random, seed=None\n",
      "INFO:trainer:Initializing zeros from quasi-random, seed=None\n",
      "INFO:trainer:Initializing random-uniform from pseudo-random, seed=None\n",
      "INFO:custom_initializers:Setting seed value for RandomNormalQR to 10.\n",
      "INFO:custom_initializers:Setting seed value for OrthogonalInitializerQR to 10.\n",
      "INFO:custom_initializers:Setting seed value for RandomNormalQR to 11.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 35ms/step - categorical_accuracy: 0.5516 - loss: 0.6786"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 1: train accuracy = -1, val accuracy = -1, test accuracy = 0.7706000208854675, time per epoch = 14.60 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m21s\u001b[0m 51ms/step - categorical_accuracy: 0.5518 - loss: 0.6785\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:train_and_eval:Resetting the state. Starting QRNG dimension is now set to 5.\n",
      "INFO:train_and_eval:Running experiment #1\n",
      "INFO:models:Initialized ModelBuilder with model_id=baseline-lstm, input_shape=[512], output_shape=2, output_activation=softmax, model_config={'initializer_type': 'quasi-random', 'kernel_initializer': 'random-normal', 'bias_initializer': 'zeros', 'recurrent_initializer': 'orthogonal', 'max_features': 20000, 'units': 64, 'embedding_dim': 32}\n",
      "INFO:trainer:Initializing random-normal from quasi-random, seed=None\n",
      "INFO:trainer:Initializing orthogonal from quasi-random, seed=None\n",
      "INFO:trainer:Initializing zeros from quasi-random, seed=None\n",
      "INFO:trainer:Initializing random-uniform from pseudo-random, seed=None\n",
      "INFO:custom_initializers:Setting seed value for RandomNormalQR to 5.\n",
      "INFO:custom_initializers:Setting seed value for OrthogonalInitializerQR to 5.\n",
      "INFO:custom_initializers:Setting seed value for RandomNormalQR to 6.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"font-weight: bold\">Model: \"functional_1\"</span>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[1mModel: \"functional_1\"\u001b[0m\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓\n",
       "┃<span style=\"font-weight: bold\"> Layer (type)                    </span>┃<span style=\"font-weight: bold\"> Output Shape           </span>┃<span style=\"font-weight: bold\">       Param # </span>┃\n",
       "┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩\n",
       "│ input_layer (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">InputLayer</span>)        │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">512</span>)            │             <span style=\"color: #00af00; text-decoration-color: #00af00\">0</span> │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ embedding (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">Embedding</span>)           │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">512</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">32</span>)        │       <span style=\"color: #00af00; text-decoration-color: #00af00\">640,000</span> │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ lstm (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">LSTM</span>)                     │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">64</span>)             │        <span style=\"color: #00af00; text-decoration-color: #00af00\">24,832</span> │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ flatten (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">Flatten</span>)               │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">64</span>)             │             <span style=\"color: #00af00; text-decoration-color: #00af00\">0</span> │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ predictions (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">Dense</span>)             │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">2</span>)              │           <span style=\"color: #00af00; text-decoration-color: #00af00\">130</span> │\n",
       "└─────────────────────────────────┴────────────────────────┴───────────────┘\n",
       "</pre>\n"
      ],
      "text/plain": [
       "┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓\n",
       "┃\u001b[1m \u001b[0m\u001b[1mLayer (type)                   \u001b[0m\u001b[1m \u001b[0m┃\u001b[1m \u001b[0m\u001b[1mOutput Shape          \u001b[0m\u001b[1m \u001b[0m┃\u001b[1m \u001b[0m\u001b[1m      Param #\u001b[0m\u001b[1m \u001b[0m┃\n",
       "┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩\n",
       "│ input_layer (\u001b[38;5;33mInputLayer\u001b[0m)        │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m512\u001b[0m)            │             \u001b[38;5;34m0\u001b[0m │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ embedding (\u001b[38;5;33mEmbedding\u001b[0m)           │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m512\u001b[0m, \u001b[38;5;34m32\u001b[0m)        │       \u001b[38;5;34m640,000\u001b[0m │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ lstm (\u001b[38;5;33mLSTM\u001b[0m)                     │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m64\u001b[0m)             │        \u001b[38;5;34m24,832\u001b[0m │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ flatten (\u001b[38;5;33mFlatten\u001b[0m)               │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m64\u001b[0m)             │             \u001b[38;5;34m0\u001b[0m │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ predictions (\u001b[38;5;33mDense\u001b[0m)             │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m2\u001b[0m)              │           \u001b[38;5;34m130\u001b[0m │\n",
       "└─────────────────────────────────┴────────────────────────┴───────────────┘\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"font-weight: bold\"> Total params: </span><span style=\"color: #00af00; text-decoration-color: #00af00\">664,962</span> (2.54 MB)\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[1m Total params: \u001b[0m\u001b[38;5;34m664,962\u001b[0m (2.54 MB)\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"font-weight: bold\"> Trainable params: </span><span style=\"color: #00af00; text-decoration-color: #00af00\">664,962</span> (2.54 MB)\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[1m Trainable params: \u001b[0m\u001b[38;5;34m664,962\u001b[0m (2.54 MB)\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"font-weight: bold\"> Non-trainable params: </span><span style=\"color: #00af00; text-decoration-color: #00af00\">0</span> (0.00 B)\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[1m Non-trainable params: \u001b[0m\u001b[38;5;34m0\u001b[0m (0.00 B)\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/10\n",
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 39ms/step - categorical_accuracy: 0.5438 - loss: 0.6791"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 1: train accuracy = -1, val accuracy = -1, test accuracy = 0.7996000051498413, time per epoch = 16.20 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m23s\u001b[0m 56ms/step - categorical_accuracy: 0.5440 - loss: 0.6790\n",
      "Epoch 2/10\n",
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 41ms/step - categorical_accuracy: 0.7801 - loss: 0.5336"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 2: train accuracy = -1, val accuracy = -1, test accuracy = 0.8519200086593628, time per epoch = 16.13 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m23s\u001b[0m 58ms/step - categorical_accuracy: 0.7802 - loss: 0.5333\n",
      "Epoch 3/10\n",
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 34ms/step - categorical_accuracy: 0.8785 - loss: 0.3087"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 3: train accuracy = -1, val accuracy = -1, test accuracy = 0.869159996509552, time per epoch = 13.49 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m20s\u001b[0m 51ms/step - categorical_accuracy: 0.8785 - loss: 0.3087\n",
      "Epoch 4/10\n",
      "\u001b[1m390/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m━\u001b[0m \u001b[1m0s\u001b[0m 35ms/step - categorical_accuracy: 0.9111 - loss: 0.2426"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 4: train accuracy = -1, val accuracy = -1, test accuracy = 0.8730000257492065, time per epoch = 13.81 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m20s\u001b[0m 51ms/step - categorical_accuracy: 0.9111 - loss: 0.2425\n",
      "Epoch 5/10\n",
      "\u001b[1m390/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m━\u001b[0m \u001b[1m0s\u001b[0m 39ms/step - categorical_accuracy: 0.9292 - loss: 0.2007"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 5: train accuracy = -1, val accuracy = -1, test accuracy = 0.8723599910736084, time per epoch = 15.09 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m22s\u001b[0m 55ms/step - categorical_accuracy: 0.9293 - loss: 0.2006\n",
      "Epoch 6/10\n",
      "\u001b[1m390/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m━\u001b[0m \u001b[1m0s\u001b[0m 40ms/step - categorical_accuracy: 0.9461 - loss: 0.1656"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 6: train accuracy = -1, val accuracy = -1, test accuracy = 0.8746399879455566, time per epoch = 15.49 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m22s\u001b[0m 56ms/step - categorical_accuracy: 0.9461 - loss: 0.1656\n",
      "Epoch 7/10\n",
      "\u001b[1m390/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m━\u001b[0m \u001b[1m0s\u001b[0m 37ms/step - categorical_accuracy: 0.9567 - loss: 0.1409"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 7: train accuracy = -1, val accuracy = -1, test accuracy = 0.8698400259017944, time per epoch = 14.68 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m21s\u001b[0m 54ms/step - categorical_accuracy: 0.9568 - loss: 0.1408\n",
      "Epoch 8/10\n",
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 35ms/step - categorical_accuracy: 0.9644 - loss: 0.1185"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 8: train accuracy = -1, val accuracy = -1, test accuracy = 0.8706799745559692, time per epoch = 13.74 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m20s\u001b[0m 52ms/step - categorical_accuracy: 0.9644 - loss: 0.1185\n",
      "Epoch 9/10\n",
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 39ms/step - categorical_accuracy: 0.9732 - loss: 0.0956"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 9: train accuracy = -1, val accuracy = -1, test accuracy = 0.8654400110244751, time per epoch = 15.23 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m22s\u001b[0m 57ms/step - categorical_accuracy: 0.9732 - loss: 0.0956\n",
      "Epoch 10/10\n",
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 42ms/step - categorical_accuracy: 0.9770 - loss: 0.0839"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 10: train accuracy = -1, val accuracy = -1, test accuracy = 0.8655200004577637, time per epoch = 16.39 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m23s\u001b[0m 60ms/step - categorical_accuracy: 0.9770 - loss: 0.0839\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:datapipeline:Loading IMDB reviews dataset\n",
      "INFO:datapipeline:Converting labels to one-hot encoding\n",
      "INFO:datapipeline:IMDB reviews dataset loaded\n",
      "INFO:datapipeline:x_train shape (25000, 512)\n",
      "INFO:datapipeline:y_train shape (25000, 2)\n",
      "INFO:datapipeline:x_test shape (25000, 512)\n",
      "INFO:datapipeline:y_test shape (25000, 2)\n",
      "INFO:utils:Splitting Data into Train and Validation\n",
      "INFO:utils:Training observation count is set to 25000\n",
      "INFO:utils:x_train shape before splitting (25000, 512)\n",
      "INFO:utils:y_train shape before splitting (25000, 2)\n",
      "INFO:utils:Training observation count equals the length of x_train. No validation set will be created.\n",
      "INFO:utils:x_train shape after splitting (25000, 512)\n",
      "INFO:utils:y_train shape after splitting (25000, 2)\n",
      "INFO:utils:x_val shape after splitting (0, 512)\n",
      "INFO:utils:y_val shape after splitting (0, 2)\n",
      "INFO:train_and_eval:Choosing the best seed from this set: [1, 3, 6, 7, 9]\n",
      "INFO:models:Initialized ModelBuilder with model_id=baseline-lstm, input_shape=[512], output_shape=2, output_activation=softmax, model_config={'initializer_type': 'quasi-random', 'kernel_initializer': 'random-normal', 'bias_initializer': 'zeros', 'recurrent_initializer': 'orthogonal', 'max_features': 20000, 'units': 64, 'embedding_dim': 32}\n",
      "INFO:trainer:Initializing random-normal from quasi-random, seed=None\n",
      "INFO:trainer:Initializing orthogonal from quasi-random, seed=None\n",
      "INFO:trainer:Initializing zeros from quasi-random, seed=None\n",
      "INFO:trainer:Initializing random-uniform from pseudo-random, seed=None\n",
      "INFO:custom_initializers:Setting seed value for RandomNormalQR to 1.\n",
      "INFO:custom_initializers:Setting seed value for OrthogonalInitializerQR to 1.\n",
      "INFO:custom_initializers:Setting seed value for RandomNormalQR to 2.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 37ms/step - categorical_accuracy: 0.5606 - loss: 0.6672"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 1: train accuracy = -1, val accuracy = -1, test accuracy = 0.810479998588562, time per epoch = 15.71 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m22s\u001b[0m 55ms/step - categorical_accuracy: 0.5608 - loss: 0.6671\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:models:Initialized ModelBuilder with model_id=baseline-lstm, input_shape=[512], output_shape=2, output_activation=softmax, model_config={'initializer_type': 'quasi-random', 'kernel_initializer': 'random-normal', 'bias_initializer': 'zeros', 'recurrent_initializer': 'orthogonal', 'max_features': 20000, 'units': 64, 'embedding_dim': 32}\n",
      "INFO:trainer:Initializing random-normal from quasi-random, seed=None\n",
      "INFO:trainer:Initializing orthogonal from quasi-random, seed=None\n",
      "INFO:trainer:Initializing zeros from quasi-random, seed=None\n",
      "INFO:trainer:Initializing random-uniform from pseudo-random, seed=None\n",
      "INFO:custom_initializers:Setting seed value for RandomNormalQR to 3.\n",
      "INFO:custom_initializers:Setting seed value for OrthogonalInitializerQR to 3.\n",
      "INFO:custom_initializers:Setting seed value for RandomNormalQR to 4.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 38ms/step - categorical_accuracy: 0.5422 - loss: 0.6794"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 1: train accuracy = -1, val accuracy = -1, test accuracy = 0.795199990272522, time per epoch = 15.97 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m23s\u001b[0m 55ms/step - categorical_accuracy: 0.5424 - loss: 0.6793\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:models:Initialized ModelBuilder with model_id=baseline-lstm, input_shape=[512], output_shape=2, output_activation=softmax, model_config={'initializer_type': 'quasi-random', 'kernel_initializer': 'random-normal', 'bias_initializer': 'zeros', 'recurrent_initializer': 'orthogonal', 'max_features': 20000, 'units': 64, 'embedding_dim': 32}\n",
      "INFO:trainer:Initializing random-normal from quasi-random, seed=None\n",
      "INFO:trainer:Initializing orthogonal from quasi-random, seed=None\n",
      "INFO:trainer:Initializing zeros from quasi-random, seed=None\n",
      "INFO:trainer:Initializing random-uniform from pseudo-random, seed=None\n",
      "INFO:custom_initializers:Setting seed value for RandomNormalQR to 6.\n",
      "INFO:custom_initializers:Setting seed value for OrthogonalInitializerQR to 6.\n",
      "INFO:custom_initializers:Setting seed value for RandomNormalQR to 7.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 38ms/step - categorical_accuracy: 0.5593 - loss: 0.6833"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 1: train accuracy = -1, val accuracy = -1, test accuracy = 0.7599599957466125, time per epoch = 15.66 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m22s\u001b[0m 54ms/step - categorical_accuracy: 0.5595 - loss: 0.6833\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:models:Initialized ModelBuilder with model_id=baseline-lstm, input_shape=[512], output_shape=2, output_activation=softmax, model_config={'initializer_type': 'quasi-random', 'kernel_initializer': 'random-normal', 'bias_initializer': 'zeros', 'recurrent_initializer': 'orthogonal', 'max_features': 20000, 'units': 64, 'embedding_dim': 32}\n",
      "INFO:trainer:Initializing random-normal from quasi-random, seed=None\n",
      "INFO:trainer:Initializing orthogonal from quasi-random, seed=None\n",
      "INFO:trainer:Initializing zeros from quasi-random, seed=None\n",
      "INFO:trainer:Initializing random-uniform from pseudo-random, seed=None\n",
      "INFO:custom_initializers:Setting seed value for RandomNormalQR to 7.\n",
      "INFO:custom_initializers:Setting seed value for OrthogonalInitializerQR to 7.\n",
      "INFO:custom_initializers:Setting seed value for RandomNormalQR to 8.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 35ms/step - categorical_accuracy: 0.5444 - loss: 0.6818"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 1: train accuracy = -1, val accuracy = -1, test accuracy = 0.6361200213432312, time per epoch = 14.70 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m21s\u001b[0m 52ms/step - categorical_accuracy: 0.5446 - loss: 0.6817\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:models:Initialized ModelBuilder with model_id=baseline-lstm, input_shape=[512], output_shape=2, output_activation=softmax, model_config={'initializer_type': 'quasi-random', 'kernel_initializer': 'random-normal', 'bias_initializer': 'zeros', 'recurrent_initializer': 'orthogonal', 'max_features': 20000, 'units': 64, 'embedding_dim': 32}\n",
      "INFO:trainer:Initializing random-normal from quasi-random, seed=None\n",
      "INFO:trainer:Initializing orthogonal from quasi-random, seed=None\n",
      "INFO:trainer:Initializing zeros from quasi-random, seed=None\n",
      "INFO:trainer:Initializing random-uniform from pseudo-random, seed=None\n",
      "INFO:custom_initializers:Setting seed value for RandomNormalQR to 9.\n",
      "INFO:custom_initializers:Setting seed value for OrthogonalInitializerQR to 9.\n",
      "INFO:custom_initializers:Setting seed value for RandomNormalQR to 10.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 38ms/step - categorical_accuracy: 0.5501 - loss: 0.6791"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 1: train accuracy = -1, val accuracy = -1, test accuracy = 0.7414399981498718, time per epoch = 16.00 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m23s\u001b[0m 55ms/step - categorical_accuracy: 0.5503 - loss: 0.6790\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:train_and_eval:Resetting the state. Starting QRNG dimension is now set to 1.\n",
      "INFO:train_and_eval:Running experiment #1\n",
      "INFO:models:Initialized ModelBuilder with model_id=baseline-lstm, input_shape=[512], output_shape=2, output_activation=softmax, model_config={'initializer_type': 'quasi-random', 'kernel_initializer': 'random-normal', 'bias_initializer': 'zeros', 'recurrent_initializer': 'orthogonal', 'max_features': 20000, 'units': 64, 'embedding_dim': 32}\n",
      "INFO:trainer:Initializing random-normal from quasi-random, seed=None\n",
      "INFO:trainer:Initializing orthogonal from quasi-random, seed=None\n",
      "INFO:trainer:Initializing zeros from quasi-random, seed=None\n",
      "INFO:trainer:Initializing random-uniform from pseudo-random, seed=None\n",
      "INFO:custom_initializers:Setting seed value for RandomNormalQR to 1.\n",
      "INFO:custom_initializers:Setting seed value for OrthogonalInitializerQR to 1.\n",
      "INFO:custom_initializers:Setting seed value for RandomNormalQR to 2.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"font-weight: bold\">Model: \"functional_1\"</span>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[1mModel: \"functional_1\"\u001b[0m\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓\n",
       "┃<span style=\"font-weight: bold\"> Layer (type)                    </span>┃<span style=\"font-weight: bold\"> Output Shape           </span>┃<span style=\"font-weight: bold\">       Param # </span>┃\n",
       "┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩\n",
       "│ input_layer (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">InputLayer</span>)        │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">512</span>)            │             <span style=\"color: #00af00; text-decoration-color: #00af00\">0</span> │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ embedding (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">Embedding</span>)           │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">512</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">32</span>)        │       <span style=\"color: #00af00; text-decoration-color: #00af00\">640,000</span> │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ lstm (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">LSTM</span>)                     │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">64</span>)             │        <span style=\"color: #00af00; text-decoration-color: #00af00\">24,832</span> │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ flatten (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">Flatten</span>)               │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">64</span>)             │             <span style=\"color: #00af00; text-decoration-color: #00af00\">0</span> │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ predictions (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">Dense</span>)             │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">2</span>)              │           <span style=\"color: #00af00; text-decoration-color: #00af00\">130</span> │\n",
       "└─────────────────────────────────┴────────────────────────┴───────────────┘\n",
       "</pre>\n"
      ],
      "text/plain": [
       "┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓\n",
       "┃\u001b[1m \u001b[0m\u001b[1mLayer (type)                   \u001b[0m\u001b[1m \u001b[0m┃\u001b[1m \u001b[0m\u001b[1mOutput Shape          \u001b[0m\u001b[1m \u001b[0m┃\u001b[1m \u001b[0m\u001b[1m      Param #\u001b[0m\u001b[1m \u001b[0m┃\n",
       "┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩\n",
       "│ input_layer (\u001b[38;5;33mInputLayer\u001b[0m)        │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m512\u001b[0m)            │             \u001b[38;5;34m0\u001b[0m │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ embedding (\u001b[38;5;33mEmbedding\u001b[0m)           │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m512\u001b[0m, \u001b[38;5;34m32\u001b[0m)        │       \u001b[38;5;34m640,000\u001b[0m │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ lstm (\u001b[38;5;33mLSTM\u001b[0m)                     │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m64\u001b[0m)             │        \u001b[38;5;34m24,832\u001b[0m │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ flatten (\u001b[38;5;33mFlatten\u001b[0m)               │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m64\u001b[0m)             │             \u001b[38;5;34m0\u001b[0m │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ predictions (\u001b[38;5;33mDense\u001b[0m)             │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m2\u001b[0m)              │           \u001b[38;5;34m130\u001b[0m │\n",
       "└─────────────────────────────────┴────────────────────────┴───────────────┘\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"font-weight: bold\"> Total params: </span><span style=\"color: #00af00; text-decoration-color: #00af00\">664,962</span> (2.54 MB)\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[1m Total params: \u001b[0m\u001b[38;5;34m664,962\u001b[0m (2.54 MB)\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"font-weight: bold\"> Trainable params: </span><span style=\"color: #00af00; text-decoration-color: #00af00\">664,962</span> (2.54 MB)\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[1m Trainable params: \u001b[0m\u001b[38;5;34m664,962\u001b[0m (2.54 MB)\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"font-weight: bold\"> Non-trainable params: </span><span style=\"color: #00af00; text-decoration-color: #00af00\">0</span> (0.00 B)\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[1m Non-trainable params: \u001b[0m\u001b[38;5;34m0\u001b[0m (0.00 B)\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/10\n",
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 39ms/step - categorical_accuracy: 0.5495 - loss: 0.6792"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 1: train accuracy = -1, val accuracy = -1, test accuracy = 0.7826799750328064, time per epoch = 16.26 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m23s\u001b[0m 56ms/step - categorical_accuracy: 0.5497 - loss: 0.6790\n",
      "Epoch 2/10\n",
      "\u001b[1m390/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m━\u001b[0m \u001b[1m0s\u001b[0m 34ms/step - categorical_accuracy: 0.8110 - loss: 0.4716"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 2: train accuracy = -1, val accuracy = -1, test accuracy = 0.8298400044441223, time per epoch = 13.45 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m20s\u001b[0m 50ms/step - categorical_accuracy: 0.8111 - loss: 0.4713\n",
      "Epoch 3/10\n",
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 36ms/step - categorical_accuracy: 0.8681 - loss: 0.3231"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 3: train accuracy = -1, val accuracy = -1, test accuracy = 0.8507999777793884, time per epoch = 13.91 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m24s\u001b[0m 62ms/step - categorical_accuracy: 0.8682 - loss: 0.3231\n",
      "Epoch 4/10\n",
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 39ms/step - categorical_accuracy: 0.9069 - loss: 0.2470"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 4: train accuracy = -1, val accuracy = -1, test accuracy = 0.866159975528717, time per epoch = 15.30 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m22s\u001b[0m 56ms/step - categorical_accuracy: 0.9070 - loss: 0.2470\n",
      "Epoch 5/10\n",
      "\u001b[1m390/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m━\u001b[0m \u001b[1m0s\u001b[0m 37ms/step - categorical_accuracy: 0.9310 - loss: 0.1917"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 5: train accuracy = -1, val accuracy = -1, test accuracy = 0.869159996509552, time per epoch = 14.54 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m21s\u001b[0m 53ms/step - categorical_accuracy: 0.9311 - loss: 0.1916\n",
      "Epoch 6/10\n",
      "\u001b[1m390/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m━\u001b[0m \u001b[1m0s\u001b[0m 37ms/step - categorical_accuracy: 0.9457 - loss: 0.1624"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 6: train accuracy = -1, val accuracy = -1, test accuracy = 0.8716400265693665, time per epoch = 14.58 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m21s\u001b[0m 53ms/step - categorical_accuracy: 0.9457 - loss: 0.1623\n",
      "Epoch 7/10\n",
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 35ms/step - categorical_accuracy: 0.9558 - loss: 0.1351"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 7: train accuracy = -1, val accuracy = -1, test accuracy = 0.8650000095367432, time per epoch = 13.80 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m20s\u001b[0m 52ms/step - categorical_accuracy: 0.9558 - loss: 0.1350\n",
      "Epoch 8/10\n",
      "\u001b[1m390/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m━\u001b[0m \u001b[1m0s\u001b[0m 39ms/step - categorical_accuracy: 0.9651 - loss: 0.1135"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 8: train accuracy = -1, val accuracy = -1, test accuracy = 0.8657199740409851, time per epoch = 15.08 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m22s\u001b[0m 56ms/step - categorical_accuracy: 0.9651 - loss: 0.1135\n",
      "Epoch 9/10\n",
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 38ms/step - categorical_accuracy: 0.9701 - loss: 0.0967"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 9: train accuracy = -1, val accuracy = -1, test accuracy = 0.8634799718856812, time per epoch = 14.76 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m21s\u001b[0m 54ms/step - categorical_accuracy: 0.9701 - loss: 0.0967\n",
      "Epoch 10/10\n",
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 37ms/step - categorical_accuracy: 0.9777 - loss: 0.0827"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 10: train accuracy = -1, val accuracy = -1, test accuracy = 0.8621600270271301, time per epoch = 14.56 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m21s\u001b[0m 53ms/step - categorical_accuracy: 0.9777 - loss: 0.0827\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:datapipeline:Loading IMDB reviews dataset\n",
      "INFO:datapipeline:Converting labels to one-hot encoding\n",
      "INFO:datapipeline:IMDB reviews dataset loaded\n",
      "INFO:datapipeline:x_train shape (25000, 512)\n",
      "INFO:datapipeline:y_train shape (25000, 2)\n",
      "INFO:datapipeline:x_test shape (25000, 512)\n",
      "INFO:datapipeline:y_test shape (25000, 2)\n",
      "INFO:utils:Splitting Data into Train and Validation\n",
      "INFO:utils:Training observation count is set to 25000\n",
      "INFO:utils:x_train shape before splitting (25000, 512)\n",
      "INFO:utils:y_train shape before splitting (25000, 2)\n",
      "INFO:utils:Training observation count equals the length of x_train. No validation set will be created.\n",
      "INFO:utils:x_train shape after splitting (25000, 512)\n",
      "INFO:utils:y_train shape after splitting (25000, 2)\n",
      "INFO:utils:x_val shape after splitting (0, 512)\n",
      "INFO:utils:y_val shape after splitting (0, 2)\n",
      "INFO:train_and_eval:Choosing the best seed from this set: [1, 4, 6, 8, 9]\n",
      "INFO:models:Initialized ModelBuilder with model_id=baseline-lstm, input_shape=[512], output_shape=2, output_activation=softmax, model_config={'initializer_type': 'quasi-random', 'kernel_initializer': 'random-normal', 'bias_initializer': 'zeros', 'recurrent_initializer': 'orthogonal', 'max_features': 20000, 'units': 64, 'embedding_dim': 32}\n",
      "INFO:trainer:Initializing random-normal from quasi-random, seed=None\n",
      "INFO:trainer:Initializing orthogonal from quasi-random, seed=None\n",
      "INFO:trainer:Initializing zeros from quasi-random, seed=None\n",
      "INFO:trainer:Initializing random-uniform from pseudo-random, seed=None\n",
      "INFO:custom_initializers:Setting seed value for RandomNormalQR to 1.\n",
      "INFO:custom_initializers:Setting seed value for OrthogonalInitializerQR to 1.\n",
      "INFO:custom_initializers:Setting seed value for RandomNormalQR to 2.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 36ms/step - categorical_accuracy: 0.5653 - loss: 0.6714"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 1: train accuracy = -1, val accuracy = -1, test accuracy = 0.776639997959137, time per epoch = 15.34 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m22s\u001b[0m 53ms/step - categorical_accuracy: 0.5656 - loss: 0.6712\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:models:Initialized ModelBuilder with model_id=baseline-lstm, input_shape=[512], output_shape=2, output_activation=softmax, model_config={'initializer_type': 'quasi-random', 'kernel_initializer': 'random-normal', 'bias_initializer': 'zeros', 'recurrent_initializer': 'orthogonal', 'max_features': 20000, 'units': 64, 'embedding_dim': 32}\n",
      "INFO:trainer:Initializing random-normal from quasi-random, seed=None\n",
      "INFO:trainer:Initializing orthogonal from quasi-random, seed=None\n",
      "INFO:trainer:Initializing zeros from quasi-random, seed=None\n",
      "INFO:trainer:Initializing random-uniform from pseudo-random, seed=None\n",
      "INFO:custom_initializers:Setting seed value for RandomNormalQR to 4.\n",
      "INFO:custom_initializers:Setting seed value for OrthogonalInitializerQR to 4.\n",
      "INFO:custom_initializers:Setting seed value for RandomNormalQR to 5.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 34ms/step - categorical_accuracy: 0.5462 - loss: 0.6831"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 1: train accuracy = -1, val accuracy = -1, test accuracy = 0.7099999785423279, time per epoch = 14.29 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m21s\u001b[0m 51ms/step - categorical_accuracy: 0.5464 - loss: 0.6831\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:models:Initialized ModelBuilder with model_id=baseline-lstm, input_shape=[512], output_shape=2, output_activation=softmax, model_config={'initializer_type': 'quasi-random', 'kernel_initializer': 'random-normal', 'bias_initializer': 'zeros', 'recurrent_initializer': 'orthogonal', 'max_features': 20000, 'units': 64, 'embedding_dim': 32}\n",
      "INFO:trainer:Initializing random-normal from quasi-random, seed=None\n",
      "INFO:trainer:Initializing orthogonal from quasi-random, seed=None\n",
      "INFO:trainer:Initializing zeros from quasi-random, seed=None\n",
      "INFO:trainer:Initializing random-uniform from pseudo-random, seed=None\n",
      "INFO:custom_initializers:Setting seed value for RandomNormalQR to 6.\n",
      "INFO:custom_initializers:Setting seed value for OrthogonalInitializerQR to 6.\n",
      "INFO:custom_initializers:Setting seed value for RandomNormalQR to 7.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 42ms/step - categorical_accuracy: 0.5698 - loss: 0.6711"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 1: train accuracy = -1, val accuracy = -1, test accuracy = 0.7708399891853333, time per epoch = 17.39 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m24s\u001b[0m 59ms/step - categorical_accuracy: 0.5700 - loss: 0.6709\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:models:Initialized ModelBuilder with model_id=baseline-lstm, input_shape=[512], output_shape=2, output_activation=softmax, model_config={'initializer_type': 'quasi-random', 'kernel_initializer': 'random-normal', 'bias_initializer': 'zeros', 'recurrent_initializer': 'orthogonal', 'max_features': 20000, 'units': 64, 'embedding_dim': 32}\n",
      "INFO:trainer:Initializing random-normal from quasi-random, seed=None\n",
      "INFO:trainer:Initializing orthogonal from quasi-random, seed=None\n",
      "INFO:trainer:Initializing zeros from quasi-random, seed=None\n",
      "INFO:trainer:Initializing random-uniform from pseudo-random, seed=None\n",
      "INFO:custom_initializers:Setting seed value for RandomNormalQR to 8.\n",
      "INFO:custom_initializers:Setting seed value for OrthogonalInitializerQR to 8.\n",
      "INFO:custom_initializers:Setting seed value for RandomNormalQR to 9.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 41ms/step - categorical_accuracy: 0.5722 - loss: 0.6726"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 1: train accuracy = -1, val accuracy = -1, test accuracy = 0.7960399985313416, time per epoch = 16.84 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m23s\u001b[0m 57ms/step - categorical_accuracy: 0.5724 - loss: 0.6725\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:models:Initialized ModelBuilder with model_id=baseline-lstm, input_shape=[512], output_shape=2, output_activation=softmax, model_config={'initializer_type': 'quasi-random', 'kernel_initializer': 'random-normal', 'bias_initializer': 'zeros', 'recurrent_initializer': 'orthogonal', 'max_features': 20000, 'units': 64, 'embedding_dim': 32}\n",
      "INFO:trainer:Initializing random-normal from quasi-random, seed=None\n",
      "INFO:trainer:Initializing orthogonal from quasi-random, seed=None\n",
      "INFO:trainer:Initializing zeros from quasi-random, seed=None\n",
      "INFO:trainer:Initializing random-uniform from pseudo-random, seed=None\n",
      "INFO:custom_initializers:Setting seed value for RandomNormalQR to 9.\n",
      "INFO:custom_initializers:Setting seed value for OrthogonalInitializerQR to 9.\n",
      "INFO:custom_initializers:Setting seed value for RandomNormalQR to 10.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m390/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m━\u001b[0m \u001b[1m0s\u001b[0m 36ms/step - categorical_accuracy: 0.5510 - loss: 0.6786"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 1: train accuracy = -1, val accuracy = -1, test accuracy = 0.7729600071907043, time per epoch = 15.16 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m22s\u001b[0m 53ms/step - categorical_accuracy: 0.5515 - loss: 0.6784\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:train_and_eval:Resetting the state. Starting QRNG dimension is now set to 8.\n",
      "INFO:train_and_eval:Running experiment #1\n",
      "INFO:models:Initialized ModelBuilder with model_id=baseline-lstm, input_shape=[512], output_shape=2, output_activation=softmax, model_config={'initializer_type': 'quasi-random', 'kernel_initializer': 'random-normal', 'bias_initializer': 'zeros', 'recurrent_initializer': 'orthogonal', 'max_features': 20000, 'units': 64, 'embedding_dim': 32}\n",
      "INFO:trainer:Initializing random-normal from quasi-random, seed=None\n",
      "INFO:trainer:Initializing orthogonal from quasi-random, seed=None\n",
      "INFO:trainer:Initializing zeros from quasi-random, seed=None\n",
      "INFO:trainer:Initializing random-uniform from pseudo-random, seed=None\n",
      "INFO:custom_initializers:Setting seed value for RandomNormalQR to 8.\n",
      "INFO:custom_initializers:Setting seed value for OrthogonalInitializerQR to 8.\n",
      "INFO:custom_initializers:Setting seed value for RandomNormalQR to 9.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"font-weight: bold\">Model: \"functional_1\"</span>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[1mModel: \"functional_1\"\u001b[0m\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓\n",
       "┃<span style=\"font-weight: bold\"> Layer (type)                    </span>┃<span style=\"font-weight: bold\"> Output Shape           </span>┃<span style=\"font-weight: bold\">       Param # </span>┃\n",
       "┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩\n",
       "│ input_layer (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">InputLayer</span>)        │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">512</span>)            │             <span style=\"color: #00af00; text-decoration-color: #00af00\">0</span> │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ embedding (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">Embedding</span>)           │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">512</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">32</span>)        │       <span style=\"color: #00af00; text-decoration-color: #00af00\">640,000</span> │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ lstm (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">LSTM</span>)                     │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">64</span>)             │        <span style=\"color: #00af00; text-decoration-color: #00af00\">24,832</span> │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ flatten (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">Flatten</span>)               │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">64</span>)             │             <span style=\"color: #00af00; text-decoration-color: #00af00\">0</span> │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ predictions (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">Dense</span>)             │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">2</span>)              │           <span style=\"color: #00af00; text-decoration-color: #00af00\">130</span> │\n",
       "└─────────────────────────────────┴────────────────────────┴───────────────┘\n",
       "</pre>\n"
      ],
      "text/plain": [
       "┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓\n",
       "┃\u001b[1m \u001b[0m\u001b[1mLayer (type)                   \u001b[0m\u001b[1m \u001b[0m┃\u001b[1m \u001b[0m\u001b[1mOutput Shape          \u001b[0m\u001b[1m \u001b[0m┃\u001b[1m \u001b[0m\u001b[1m      Param #\u001b[0m\u001b[1m \u001b[0m┃\n",
       "┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩\n",
       "│ input_layer (\u001b[38;5;33mInputLayer\u001b[0m)        │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m512\u001b[0m)            │             \u001b[38;5;34m0\u001b[0m │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ embedding (\u001b[38;5;33mEmbedding\u001b[0m)           │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m512\u001b[0m, \u001b[38;5;34m32\u001b[0m)        │       \u001b[38;5;34m640,000\u001b[0m │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ lstm (\u001b[38;5;33mLSTM\u001b[0m)                     │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m64\u001b[0m)             │        \u001b[38;5;34m24,832\u001b[0m │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ flatten (\u001b[38;5;33mFlatten\u001b[0m)               │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m64\u001b[0m)             │             \u001b[38;5;34m0\u001b[0m │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ predictions (\u001b[38;5;33mDense\u001b[0m)             │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m2\u001b[0m)              │           \u001b[38;5;34m130\u001b[0m │\n",
       "└─────────────────────────────────┴────────────────────────┴───────────────┘\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"font-weight: bold\"> Total params: </span><span style=\"color: #00af00; text-decoration-color: #00af00\">664,962</span> (2.54 MB)\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[1m Total params: \u001b[0m\u001b[38;5;34m664,962\u001b[0m (2.54 MB)\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"font-weight: bold\"> Trainable params: </span><span style=\"color: #00af00; text-decoration-color: #00af00\">664,962</span> (2.54 MB)\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[1m Trainable params: \u001b[0m\u001b[38;5;34m664,962\u001b[0m (2.54 MB)\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"font-weight: bold\"> Non-trainable params: </span><span style=\"color: #00af00; text-decoration-color: #00af00\">0</span> (0.00 B)\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[1m Non-trainable params: \u001b[0m\u001b[38;5;34m0\u001b[0m (0.00 B)\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/10\n",
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 36ms/step - categorical_accuracy: 0.5774 - loss: 0.6755"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 1: train accuracy = -1, val accuracy = -1, test accuracy = 0.774399995803833, time per epoch = 15.05 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m21s\u001b[0m 52ms/step - categorical_accuracy: 0.5776 - loss: 0.6754\n",
      "Epoch 2/10\n",
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 37ms/step - categorical_accuracy: 0.8234 - loss: 0.4302"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 2: train accuracy = -1, val accuracy = -1, test accuracy = 0.85971999168396, time per epoch = 14.61 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m21s\u001b[0m 54ms/step - categorical_accuracy: 0.8235 - loss: 0.4301\n",
      "Epoch 3/10\n",
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 36ms/step - categorical_accuracy: 0.8897 - loss: 0.2838"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 3: train accuracy = -1, val accuracy = -1, test accuracy = 0.8683199882507324, time per epoch = 14.12 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m21s\u001b[0m 53ms/step - categorical_accuracy: 0.8898 - loss: 0.2837\n",
      "Epoch 4/10\n",
      "\u001b[1m390/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m━\u001b[0m \u001b[1m0s\u001b[0m 39ms/step - categorical_accuracy: 0.9204 - loss: 0.2179"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 4: train accuracy = -1, val accuracy = -1, test accuracy = 0.8766000270843506, time per epoch = 15.05 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m21s\u001b[0m 55ms/step - categorical_accuracy: 0.9204 - loss: 0.2178\n",
      "Epoch 5/10\n",
      "\u001b[1m390/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m━\u001b[0m \u001b[1m0s\u001b[0m 40ms/step - categorical_accuracy: 0.9360 - loss: 0.1844"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 5: train accuracy = -1, val accuracy = -1, test accuracy = 0.8731600046157837, time per epoch = 15.73 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m22s\u001b[0m 57ms/step - categorical_accuracy: 0.9360 - loss: 0.1844\n",
      "Epoch 6/10\n",
      "\u001b[1m390/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m━\u001b[0m \u001b[1m0s\u001b[0m 37ms/step - categorical_accuracy: 0.9526 - loss: 0.1472"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 6: train accuracy = -1, val accuracy = -1, test accuracy = 0.8762000203132629, time per epoch = 14.58 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m21s\u001b[0m 54ms/step - categorical_accuracy: 0.9526 - loss: 0.1471\n",
      "Epoch 7/10\n",
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 35ms/step - categorical_accuracy: 0.9613 - loss: 0.1181"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 7: train accuracy = -1, val accuracy = -1, test accuracy = 0.866159975528717, time per epoch = 13.75 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m20s\u001b[0m 51ms/step - categorical_accuracy: 0.9613 - loss: 0.1181\n",
      "Epoch 8/10\n",
      "\u001b[1m390/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m━\u001b[0m \u001b[1m0s\u001b[0m 35ms/step - categorical_accuracy: 0.9696 - loss: 0.1011"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 8: train accuracy = -1, val accuracy = -1, test accuracy = 0.8700000047683716, time per epoch = 13.85 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m20s\u001b[0m 52ms/step - categorical_accuracy: 0.9696 - loss: 0.1011\n",
      "Epoch 9/10\n",
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 41ms/step - categorical_accuracy: 0.9774 - loss: 0.0826"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 9: train accuracy = -1, val accuracy = -1, test accuracy = 0.8640000224113464, time per epoch = 15.93 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m23s\u001b[0m 59ms/step - categorical_accuracy: 0.9774 - loss: 0.0826\n",
      "Epoch 10/10\n",
      "\u001b[1m390/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m━\u001b[0m \u001b[1m0s\u001b[0m 41ms/step - categorical_accuracy: 0.9806 - loss: 0.0696"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 10: train accuracy = -1, val accuracy = -1, test accuracy = 0.8642799854278564, time per epoch = 16.21 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m23s\u001b[0m 58ms/step - categorical_accuracy: 0.9806 - loss: 0.0696\n"
     ]
    }
   ],
   "source": [
    "lstm_quasi_stats = [run_experiment(\n",
    "    experiment_id=1,\n",
    "    sequence_scheme=SEQUENCE_SCHEME,\n",
    "    dataset_id=\"imdb_reviews\",\n",
    "    kernel_initializer=\"random-normal\",\n",
    "    initializer_type=\"quasi-random\",\n",
    "    model_id=\"baseline-lstm\",\n",
    "    units=64,\n",
    "    embedding_dim=32,\n",
    "    max_features=20000,\n",
    "    epochs=EPOCHS,\n",
    "    auto_epoch_count=AUTO_EPOCH_COUNT,\n",
    "    auto_min_seed_value=AUTO_MIN_SEED_VALUE,\n",
    "    auto_max_seed_value=AUTO_MAX_SEED_VALUE,\n",
    "    auto_seeds_count=AUTO_SEEDS_COUNT,\n",
    ")[0] for _ in range(5)]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:datapipeline:Loading IMDB reviews dataset\n",
      "INFO:datapipeline:Converting labels to one-hot encoding\n",
      "INFO:datapipeline:IMDB reviews dataset loaded\n",
      "INFO:datapipeline:x_train shape (25000, 512)\n",
      "INFO:datapipeline:y_train shape (25000, 2)\n",
      "INFO:datapipeline:x_test shape (25000, 512)\n",
      "INFO:datapipeline:y_test shape (25000, 2)\n",
      "INFO:utils:Splitting Data into Train and Validation\n",
      "INFO:utils:Training observation count is set to 25000\n",
      "INFO:utils:x_train shape before splitting (25000, 512)\n",
      "INFO:utils:y_train shape before splitting (25000, 2)\n",
      "INFO:utils:Training observation count equals the length of x_train. No validation set will be created.\n",
      "INFO:utils:x_train shape after splitting (25000, 512)\n",
      "INFO:utils:y_train shape after splitting (25000, 2)\n",
      "INFO:utils:x_val shape after splitting (0, 512)\n",
      "INFO:utils:y_val shape after splitting (0, 2)\n",
      "INFO:train_and_eval:Seeds for this experiment will be selected at random. Setting sequence_scheme to random.\n",
      "INFO:train_and_eval:Running experiment #1\n",
      "INFO:models:Initialized ModelBuilder with model_id=baseline-lstm, input_shape=[512], output_shape=2, output_activation=softmax, model_config={'initializer_type': 'pseudo-random', 'kernel_initializer': 'random-normal', 'bias_initializer': 'zeros', 'recurrent_initializer': 'orthogonal', 'max_features': 20000, 'units': 64, 'embedding_dim': 32}\n",
      "INFO:trainer:Initializing random-normal from pseudo-random, seed=None\n",
      "INFO:trainer:Initializing orthogonal from pseudo-random, seed=None\n",
      "INFO:trainer:Initializing zeros from pseudo-random, seed=None\n",
      "INFO:trainer:Initializing random-uniform from pseudo-random, seed=None\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"font-weight: bold\">Model: \"functional_3\"</span>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[1mModel: \"functional_3\"\u001b[0m\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓\n",
       "┃<span style=\"font-weight: bold\"> Layer (type)                    </span>┃<span style=\"font-weight: bold\"> Output Shape           </span>┃<span style=\"font-weight: bold\">       Param # </span>┃\n",
       "┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩\n",
       "│ input_layer (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">InputLayer</span>)        │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">512</span>)            │             <span style=\"color: #00af00; text-decoration-color: #00af00\">0</span> │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ embedding (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">Embedding</span>)           │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">512</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">32</span>)        │       <span style=\"color: #00af00; text-decoration-color: #00af00\">640,000</span> │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ lstm (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">LSTM</span>)                     │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">64</span>)             │        <span style=\"color: #00af00; text-decoration-color: #00af00\">24,832</span> │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ flatten (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">Flatten</span>)               │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">64</span>)             │             <span style=\"color: #00af00; text-decoration-color: #00af00\">0</span> │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ predictions (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">Dense</span>)             │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">2</span>)              │           <span style=\"color: #00af00; text-decoration-color: #00af00\">130</span> │\n",
       "└─────────────────────────────────┴────────────────────────┴───────────────┘\n",
       "</pre>\n"
      ],
      "text/plain": [
       "┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓\n",
       "┃\u001b[1m \u001b[0m\u001b[1mLayer (type)                   \u001b[0m\u001b[1m \u001b[0m┃\u001b[1m \u001b[0m\u001b[1mOutput Shape          \u001b[0m\u001b[1m \u001b[0m┃\u001b[1m \u001b[0m\u001b[1m      Param #\u001b[0m\u001b[1m \u001b[0m┃\n",
       "┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩\n",
       "│ input_layer (\u001b[38;5;33mInputLayer\u001b[0m)        │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m512\u001b[0m)            │             \u001b[38;5;34m0\u001b[0m │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ embedding (\u001b[38;5;33mEmbedding\u001b[0m)           │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m512\u001b[0m, \u001b[38;5;34m32\u001b[0m)        │       \u001b[38;5;34m640,000\u001b[0m │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ lstm (\u001b[38;5;33mLSTM\u001b[0m)                     │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m64\u001b[0m)             │        \u001b[38;5;34m24,832\u001b[0m │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ flatten (\u001b[38;5;33mFlatten\u001b[0m)               │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m64\u001b[0m)             │             \u001b[38;5;34m0\u001b[0m │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ predictions (\u001b[38;5;33mDense\u001b[0m)             │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m2\u001b[0m)              │           \u001b[38;5;34m130\u001b[0m │\n",
       "└─────────────────────────────────┴────────────────────────┴───────────────┘\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"font-weight: bold\"> Total params: </span><span style=\"color: #00af00; text-decoration-color: #00af00\">664,962</span> (2.54 MB)\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[1m Total params: \u001b[0m\u001b[38;5;34m664,962\u001b[0m (2.54 MB)\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"font-weight: bold\"> Trainable params: </span><span style=\"color: #00af00; text-decoration-color: #00af00\">664,962</span> (2.54 MB)\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[1m Trainable params: \u001b[0m\u001b[38;5;34m664,962\u001b[0m (2.54 MB)\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"font-weight: bold\"> Non-trainable params: </span><span style=\"color: #00af00; text-decoration-color: #00af00\">0</span> (0.00 B)\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[1m Non-trainable params: \u001b[0m\u001b[38;5;34m0\u001b[0m (0.00 B)\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/10\n",
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 37ms/step - categorical_accuracy: 0.5564 - loss: 0.6818"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 1: train accuracy = -1, val accuracy = -1, test accuracy = 0.7573999762535095, time per epoch = 15.42 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m22s\u001b[0m 53ms/step - categorical_accuracy: 0.5566 - loss: 0.6817\n",
      "Epoch 2/10\n",
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 36ms/step - categorical_accuracy: 0.8246 - loss: 0.4240"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 2: train accuracy = -1, val accuracy = -1, test accuracy = 0.8638799786567688, time per epoch = 13.90 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m20s\u001b[0m 51ms/step - categorical_accuracy: 0.8247 - loss: 0.4238\n",
      "Epoch 3/10\n",
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 36ms/step - categorical_accuracy: 0.8984 - loss: 0.2716"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 3: train accuracy = -1, val accuracy = -1, test accuracy = 0.8830000162124634, time per epoch = 14.12 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m20s\u001b[0m 52ms/step - categorical_accuracy: 0.8984 - loss: 0.2715\n",
      "Epoch 4/10\n",
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 37ms/step - categorical_accuracy: 0.9212 - loss: 0.2167"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 4: train accuracy = -1, val accuracy = -1, test accuracy = 0.8805599808692932, time per epoch = 14.51 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m21s\u001b[0m 53ms/step - categorical_accuracy: 0.9212 - loss: 0.2167\n",
      "Epoch 5/10\n",
      "\u001b[1m390/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m━\u001b[0m \u001b[1m0s\u001b[0m 40ms/step - categorical_accuracy: 0.9356 - loss: 0.1816"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 5: train accuracy = -1, val accuracy = -1, test accuracy = 0.8837599754333496, time per epoch = 15.78 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m23s\u001b[0m 58ms/step - categorical_accuracy: 0.9356 - loss: 0.1815\n",
      "Epoch 6/10\n",
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 39ms/step - categorical_accuracy: 0.9500 - loss: 0.1478"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 6: train accuracy = -1, val accuracy = -1, test accuracy = 0.883840024471283, time per epoch = 15.42 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m22s\u001b[0m 56ms/step - categorical_accuracy: 0.9500 - loss: 0.1478\n",
      "Epoch 7/10\n",
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 36ms/step - categorical_accuracy: 0.9598 - loss: 0.1268"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 7: train accuracy = -1, val accuracy = -1, test accuracy = 0.8774399757385254, time per epoch = 14.17 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m21s\u001b[0m 53ms/step - categorical_accuracy: 0.9598 - loss: 0.1268\n",
      "Epoch 8/10\n",
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 33ms/step - categorical_accuracy: 0.9668 - loss: 0.1092"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 8: train accuracy = -1, val accuracy = -1, test accuracy = 0.8769599795341492, time per epoch = 12.77 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m19s\u001b[0m 48ms/step - categorical_accuracy: 0.9668 - loss: 0.1091\n",
      "Epoch 9/10\n",
      "\u001b[1m390/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m━\u001b[0m \u001b[1m0s\u001b[0m 38ms/step - categorical_accuracy: 0.9748 - loss: 0.0864"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 9: train accuracy = -1, val accuracy = -1, test accuracy = 0.8725200295448303, time per epoch = 14.70 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m21s\u001b[0m 54ms/step - categorical_accuracy: 0.9749 - loss: 0.0863\n",
      "Epoch 10/10\n",
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 39ms/step - categorical_accuracy: 0.9788 - loss: 0.0731"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 10: train accuracy = -1, val accuracy = -1, test accuracy = 0.871999979019165, time per epoch = 15.25 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m22s\u001b[0m 56ms/step - categorical_accuracy: 0.9788 - loss: 0.0731\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:datapipeline:Loading IMDB reviews dataset\n",
      "INFO:datapipeline:Converting labels to one-hot encoding\n",
      "INFO:datapipeline:IMDB reviews dataset loaded\n",
      "INFO:datapipeline:x_train shape (25000, 512)\n",
      "INFO:datapipeline:y_train shape (25000, 2)\n",
      "INFO:datapipeline:x_test shape (25000, 512)\n",
      "INFO:datapipeline:y_test shape (25000, 2)\n",
      "INFO:utils:Splitting Data into Train and Validation\n",
      "INFO:utils:Training observation count is set to 25000\n",
      "INFO:utils:x_train shape before splitting (25000, 512)\n",
      "INFO:utils:y_train shape before splitting (25000, 2)\n",
      "INFO:utils:Training observation count equals the length of x_train. No validation set will be created.\n",
      "INFO:utils:x_train shape after splitting (25000, 512)\n",
      "INFO:utils:y_train shape after splitting (25000, 2)\n",
      "INFO:utils:x_val shape after splitting (0, 512)\n",
      "INFO:utils:y_val shape after splitting (0, 2)\n",
      "INFO:train_and_eval:Seeds for this experiment will be selected at random. Setting sequence_scheme to random.\n",
      "INFO:train_and_eval:Running experiment #1\n",
      "INFO:models:Initialized ModelBuilder with model_id=baseline-lstm, input_shape=[512], output_shape=2, output_activation=softmax, model_config={'initializer_type': 'pseudo-random', 'kernel_initializer': 'random-normal', 'bias_initializer': 'zeros', 'recurrent_initializer': 'orthogonal', 'max_features': 20000, 'units': 64, 'embedding_dim': 32}\n",
      "INFO:trainer:Initializing random-normal from pseudo-random, seed=None\n",
      "INFO:trainer:Initializing orthogonal from pseudo-random, seed=None\n",
      "INFO:trainer:Initializing zeros from pseudo-random, seed=None\n",
      "INFO:trainer:Initializing random-uniform from pseudo-random, seed=None\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"font-weight: bold\">Model: \"functional_5\"</span>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[1mModel: \"functional_5\"\u001b[0m\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓\n",
       "┃<span style=\"font-weight: bold\"> Layer (type)                    </span>┃<span style=\"font-weight: bold\"> Output Shape           </span>┃<span style=\"font-weight: bold\">       Param # </span>┃\n",
       "┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩\n",
       "│ input_layer (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">InputLayer</span>)        │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">512</span>)            │             <span style=\"color: #00af00; text-decoration-color: #00af00\">0</span> │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ embedding (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">Embedding</span>)           │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">512</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">32</span>)        │       <span style=\"color: #00af00; text-decoration-color: #00af00\">640,000</span> │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ lstm (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">LSTM</span>)                     │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">64</span>)             │        <span style=\"color: #00af00; text-decoration-color: #00af00\">24,832</span> │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ flatten (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">Flatten</span>)               │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">64</span>)             │             <span style=\"color: #00af00; text-decoration-color: #00af00\">0</span> │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ predictions (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">Dense</span>)             │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">2</span>)              │           <span style=\"color: #00af00; text-decoration-color: #00af00\">130</span> │\n",
       "└─────────────────────────────────┴────────────────────────┴───────────────┘\n",
       "</pre>\n"
      ],
      "text/plain": [
       "┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓\n",
       "┃\u001b[1m \u001b[0m\u001b[1mLayer (type)                   \u001b[0m\u001b[1m \u001b[0m┃\u001b[1m \u001b[0m\u001b[1mOutput Shape          \u001b[0m\u001b[1m \u001b[0m┃\u001b[1m \u001b[0m\u001b[1m      Param #\u001b[0m\u001b[1m \u001b[0m┃\n",
       "┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩\n",
       "│ input_layer (\u001b[38;5;33mInputLayer\u001b[0m)        │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m512\u001b[0m)            │             \u001b[38;5;34m0\u001b[0m │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ embedding (\u001b[38;5;33mEmbedding\u001b[0m)           │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m512\u001b[0m, \u001b[38;5;34m32\u001b[0m)        │       \u001b[38;5;34m640,000\u001b[0m │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ lstm (\u001b[38;5;33mLSTM\u001b[0m)                     │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m64\u001b[0m)             │        \u001b[38;5;34m24,832\u001b[0m │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ flatten (\u001b[38;5;33mFlatten\u001b[0m)               │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m64\u001b[0m)             │             \u001b[38;5;34m0\u001b[0m │\n",
       "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
       "│ predictions (\u001b[38;5;33mDense\u001b[0m)             │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m2\u001b[0m)              │           \u001b[38;5;34m130\u001b[0m │\n",
       "└─────────────────────────────────┴────────────────────────┴───────────────┘\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"font-weight: bold\"> Total params: </span><span style=\"color: #00af00; text-decoration-color: #00af00\">664,962</span> (2.54 MB)\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[1m Total params: \u001b[0m\u001b[38;5;34m664,962\u001b[0m (2.54 MB)\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"font-weight: bold\"> Trainable params: </span><span style=\"color: #00af00; text-decoration-color: #00af00\">664,962</span> (2.54 MB)\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[1m Trainable params: \u001b[0m\u001b[38;5;34m664,962\u001b[0m (2.54 MB)\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"font-weight: bold\"> Non-trainable params: </span><span style=\"color: #00af00; text-decoration-color: #00af00\">0</span> (0.00 B)\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[1m Non-trainable params: \u001b[0m\u001b[38;5;34m0\u001b[0m (0.00 B)\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/10\n",
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 40ms/step - categorical_accuracy: 0.5411 - loss: 0.6829"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 1: train accuracy = -1, val accuracy = -1, test accuracy = 0.7546799778938293, time per epoch = 16.47 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m23s\u001b[0m 56ms/step - categorical_accuracy: 0.5413 - loss: 0.6828\n",
      "Epoch 2/10\n",
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 37ms/step - categorical_accuracy: 0.8256 - loss: 0.4129"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 2: train accuracy = -1, val accuracy = -1, test accuracy = 0.869920015335083, time per epoch = 14.46 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m20s\u001b[0m 52ms/step - categorical_accuracy: 0.8257 - loss: 0.4127\n",
      "Epoch 3/10\n",
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 34ms/step - categorical_accuracy: 0.8982 - loss: 0.2626"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 3: train accuracy = -1, val accuracy = -1, test accuracy = 0.8745599985122681, time per epoch = 13.46 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m20s\u001b[0m 50ms/step - categorical_accuracy: 0.8982 - loss: 0.2626\n",
      "Epoch 4/10\n",
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 40ms/step - categorical_accuracy: 0.9230 - loss: 0.2104"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 4: train accuracy = -1, val accuracy = -1, test accuracy = 0.877240002155304, time per epoch = 15.46 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m22s\u001b[0m 55ms/step - categorical_accuracy: 0.9230 - loss: 0.2103\n",
      "Epoch 5/10\n",
      "\u001b[1m390/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m━\u001b[0m \u001b[1m0s\u001b[0m 42ms/step - categorical_accuracy: 0.9416 - loss: 0.1737"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 5: train accuracy = -1, val accuracy = -1, test accuracy = 0.8744400143623352, time per epoch = 16.23 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m22s\u001b[0m 57ms/step - categorical_accuracy: 0.9416 - loss: 0.1736\n",
      "Epoch 6/10\n",
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 38ms/step - categorical_accuracy: 0.9518 - loss: 0.1446"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 6: train accuracy = -1, val accuracy = -1, test accuracy = 0.8802000284194946, time per epoch = 14.91 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m21s\u001b[0m 55ms/step - categorical_accuracy: 0.9518 - loss: 0.1446\n",
      "Epoch 7/10\n",
      "\u001b[1m390/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m━\u001b[0m \u001b[1m0s\u001b[0m 36ms/step - categorical_accuracy: 0.9633 - loss: 0.1200"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 7: train accuracy = -1, val accuracy = -1, test accuracy = 0.8701599836349487, time per epoch = 14.05 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m20s\u001b[0m 51ms/step - categorical_accuracy: 0.9633 - loss: 0.1200\n",
      "Epoch 8/10\n",
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 38ms/step - categorical_accuracy: 0.9721 - loss: 0.0986"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 8: train accuracy = -1, val accuracy = -1, test accuracy = 0.8737599849700928, time per epoch = 14.84 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m21s\u001b[0m 54ms/step - categorical_accuracy: 0.9721 - loss: 0.0986\n",
      "Epoch 9/10\n",
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 42ms/step - categorical_accuracy: 0.9745 - loss: 0.0858"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 9: train accuracy = -1, val accuracy = -1, test accuracy = 0.8637199997901917, time per epoch = 16.26 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m23s\u001b[0m 58ms/step - categorical_accuracy: 0.9745 - loss: 0.0858\n",
      "Epoch 10/10\n",
      "\u001b[1m390/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m━\u001b[0m \u001b[1m0s\u001b[0m 41ms/step - categorical_accuracy: 0.9830 - loss: 0.0655"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:trainer:Epoch 10: train accuracy = -1, val accuracy = -1, test accuracy = 0.8668799996376038, time per epoch = 15.83 seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m391/391\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m23s\u001b[0m 58ms/step - categorical_accuracy: 0.9830 - loss: 0.0655\n"
     ]
    }
   ],
   "source": [
    "lstm_pseudo_stats = [run_experiment(\n",
    "    experiment_id=1,\n",
    "    sequence_scheme=SEQUENCE_SCHEME,\n",
    "    dataset_id=\"imdb_reviews\",\n",
    "    kernel_initializer=\"random-normal\",\n",
    "    initializer_type=\"pseudo-random\",\n",
    "    model_id=\"baseline-lstm\",\n",
    "    units=64,\n",
    "    embedding_dim=32,\n",
    "    max_features=20000,\n",
    "    epochs=EPOCHS,\n",
    "    auto_epoch_count=AUTO_EPOCH_COUNT,\n",
    "    auto_min_seed_value=AUTO_MIN_SEED_VALUE,\n",
    "    auto_max_seed_value=AUTO_MAX_SEED_VALUE,\n",
    "    auto_seeds_count=AUTO_SEEDS_COUNT,\n",
    ")[0] for _ in range(NUM_REPS)]\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Analysis: Quasi-random vs. pseudo-random initialization"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\\* Note: The code in this notebook is a demonstration and does not necessarily reflect the full implementation and results of the paper."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [],
   "source": [
    "stats = {\n",
    "    \"MLP Pseudo-Random\": mlp_pseudo_stats,\n",
    "    \"MLP Quasi-Random\": mlp_quasi_stats,\n",
    "    \"CNN Pseudo-Random\": cnn_pseudo_stats,\n",
    "    \"CNN Quasi-Random\": cnn_quasi_stats,\n",
    "    \"LSTM Pseudo-Random\": lstm_pseudo_stats,\n",
    "    \"LSTM Quasi-Random\": lstm_quasi_stats,\n",
    "}\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAABJ0AAAMVCAYAAADd9IV8AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/TGe4hAAAACXBIWXMAAA9hAAAPYQGoP6dpAAEAAElEQVR4nOzdeVhUZfsH8O+ALINsJuCeLCqQmigmKu4ZKG70K3cU3MLUtMhKTUUyt3xFynLrdUsyTcul9HWJXHLJBXIrQERwB3FhcxAGOL8/fJnXcWZkBmaYOfD9XBdXzXOec859jqA393nO80gEQRBARERERERERESkR2bGDoCIiIiIiIiIiKofFp2IiIiIiIiIiEjvWHQiIiIiIiIiIiK9Y9GJiIiIiIiIiIj0jkUnIiIiIiIiIiLSOxadiIiIiIiIiIhI71h0IiIiIiIiIiIivWPRiYiIiIiIiIiI9I5FJyIiIiIiIiIi0jsWnYioRktPT4dEIsHGjRuNHYrBzZs3DxKJxNhhEBERUTUkkUgwb948Y4dRYT169ECPHj2MHQZRtcOiE5HIbNy4ERKJBBKJBMePH1fZLggCmjRpAolEgv79+yttk0gkmDJlyguP36NHD8XxJRIJXnrpJbz22mtYv349SktLtY5NIpHA2toaLVq0wJQpU5CZman7xYrUkSNHlO6Dubk5XFxc8PbbbyMxMdHY4REREdEzTDm3KvP3338jJCQEjRo1gpWVFRo2bIiQkBD8888/2l+oiSh7CFb2ZWFhAVdXV0ydOhXZ2dnGDo+I9KyWsQMgooqxtrbGli1b0KVLF6X2o0eP4tatW7CysqrwsRs3boxFixYBALKysvDdd99h3LhxuHLlChYvXlzu/p999hnc3Nzw5MkTHD9+HKtWrcK+fftw+fJl2NjYVDgusZk6dSpee+01yOVyXLx4EatXr8aRI0dw+fJl1K9f39jhERER0TNMNbf6+eefMXz4cLz00ksYN24c3NzckJ6ejnXr1mHHjh3Ytm0bBg0aVOHY9KWgoAC1amn/6+WqVatga2uLx48fIy4uDitWrEBCQoLawh8RiReLTkQiFRQUhO3bt+Orr75S+gd+y5Yt8PX1xf379yt8bAcHB4SEhCg+h4eHw9PTE19//TXmz58PCwuLF+7ft29ftG/fHgAwfvx41K1bF9HR0di9ezeGDx9e4bjEpmvXrnj77bcVnz09PfHuu+/iu+++w8cff2zEyIiIiOh5pphbpaamYtSoUXB3d8exY8fg7Oys2DZt2jR07doVISEhuHjxItzc3Cocnz5YW1vr1P/tt9+Gk5MTgKf3Y9iwYdi2bRvOnDmDDh06GCJEIjICvl5HJFLDhw/HgwcPcOjQIUVbUVERduzYgREjRuj1XDY2NujYsSMeP36MrKwsnffv1asXACAtLQ0AkJGRgTFjxqBx48awsrJCgwYNMGjQIKSnpyvt95///Addu3ZF7dq1YWdnh379+uHvv/9W6qPp/fuwsDC4uroqtWVnZyMsLAwODg5wdHREaGioxmHcv//+u+Lcjo6OGDRoUKVfjevatSuApwnks/71r3+hc+fOqFu3LqRSKXx9fbFjxw6V/cuG8O/atQutWrWClZUVWrZsif3796v0PX78OF577TVYW1vDw8MDa9asURtTcXEx5s+fDw8PD1hZWcHV1RWzZs1CYWGhUj9XV1f0798fR44cQfv27SGVStG6dWscOXIEwNOnsK1bt4a1tTV8fX3x119/VeQWERERGY0p5lZLly6FTCbD2rVrlQpOAODk5IQ1a9YgPz8fS5cuVbSry4EA9XM7btiwAb169YKLiwusrKzwyiuvYNWqVSr7njt3DoGBgXBycoJUKoWbmxvGjh2r1Keyczqpy5MePnyI6dOno3Xr1rC1tYW9vT369u2LCxcuKO1bNrXBjz/+iAULFqBx48awtrbG66+/jqtXr6qca+3atfDw8IBUKkWHDh3wxx9/qI3p3r17GDduHOrVqwdra2u0adMGmzZtUupTNj/ov/71L3zzzTdwd3eHjY0NAgICcPPmTQiCgPnz56Nx48aQSqUYNGgQHj58WOH7RCQ2HOlEJFKurq7o1KkTfvjhB/Tt2xfA0yJNTk4Ohg0bhq+++kqv57t27RrMzc3h6Oio875lyUPdunUBAG+99Rb+/vtvvPfee3B1dcW9e/dw6NAh3LhxQ5Ekbd68GaGhoQgMDMSSJUsgk8mwatUqdOnSBX/99ZfaZOpFBEHAoEGDcPz4cUycOBHe3t7YuXMnQkNDVfr+9ttv6Nu3L9zd3TFv3jwUFBRgxYoV8Pf3R0JCgs7nLlNWVKtTp45S+5dffomBAwdi5MiRKCoqwtatWzF48GD8+uuv6Nevn1Lf48eP4+eff8akSZNgZ2eHr776Cm+99RZu3LihuL+XLl1CQEAAnJ2dMW/ePBQXFyMyMhL16tVTiWn8+PHYtGkT3n77bXz44Yc4ffo0Fi1ahMTEROzcuVOp79WrVzFixAiEh4cjJCQE//rXvzBgwACsXr0as2bNwqRJkwAAixYtwpAhQ5CcnAwzMz7bICIicTDF3OqXX36Bq6uroiDzvG7dusHV1RW//PILVq5cqXMMq1atQsuWLTFw4EDUqlULv/zyCyZNmoTS0lJMnjwZwNPCS1leMWPGDDg6OiI9PR0///yzzud7EXV50rVr17Br1y4MHjwYbm5uyMzMxJo1a9C9e3f8888/aNiwodIxFi9eDDMzM0yfPh05OTn44osvMHLkSJw+fVrRZ926dQgPD0fnzp3x/vvv49q1axg4cCBeeuklNGnSRNGvoKAAPXr0wNWrVzFlyhS4ublh+/btCAsLQ3Z2NqZNm6Z07u+//x5FRUV477338PDhQ3zxxRcYMmQIevXqhSNHjuCTTz7B1atXsWLFCkyfPh3r16/X6/0jMlkCEYnKhg0bBADC2bNnha+//lqws7MTZDKZIAiCMHjwYKFnz56CIAhC06ZNhX79+intC0CYPHnyC4/fvXt3wcvLS8jKyhKysrKExMREYerUqQIAYcCAAVrF9ttvvwlZWVnCzZs3ha1btwp169YVpFKpcOvWLeHRo0cCAGHp0qUaj5OXlyc4OjoKEyZMUGrPyMgQHBwclNq7d+8udO/eXeUYoaGhQtOmTRWfd+3aJQAQvvjiC0VbcXGx0LVrVwGAsGHDBkW7j4+P4OLiIjx48EDRduHCBcHMzEwYPXr0C++BIAjC4cOHBQDC+vXrhaysLOHOnTvC/v37hWbNmgkSiUQ4c+aMUv+yP78yRUVFQqtWrYRevXoptQMQLC0thatXryrFBUBYsWKFoi04OFiwtrYWrl+/rmj7559/BHNzc+HZv/bPnz8vABDGjx+vdJ7p06cLAITff/9d0da0aVMBgHDy5ElF24EDBwQAglQqVTrXmjVrBADC4cOHy71XRERExmaquVV2drYAQBg0aNALjz9w4EABgJCbmysIgmoOVCYyMlJ4/te/53MQQRCEwMBAwd3dXfF5586divvzIgCEyMjIF/Z5No7k5GQhKytLSE9PF9avXy9IpVLB2dlZePz4saLvkydPhJKSEqX909LSBCsrK+Gzzz5TtJXlXt7e3kJhYaGi/csvvxQACJcuXRIE4WmO5eLiIvj4+Cj1W7t2rQBAKaeMiYkRAAixsbGKtqKiIqFTp06Cra2t4n6npaUJAARnZ2chOztb0XfmzJkCAKFNmzaCXC5XtA8fPlywtLQUnjx5Uu69IqoO+AiaSMSGDBmCgoIC/Prrr8jLy8Ovv/6ql+HfSUlJcHZ2hrOzM7y9vbFixQr069dP6ycyvXv3hrOzM5o0aYJhw4bB1tYWO3fuRKNGjSCVSmFpaYkjR47g0aNHavc/dOgQsrOzMXz4cNy/f1/xZW5uDj8/Pxw+fFjna9q3bx9q1aqFd999V9Fmbm6O9957T6nf3bt3cf78eYSFheGll15StL/66qt44403sG/fPq3POXbsWDg7O6Nhw4bo06cPcnJysHnzZrz22mtK/aRSqeL/Hz16hJycHHTt2hUJCQkqx+zduzc8PDyU4rK3t8e1a9cAACUlJThw4ACCg4Px8ssvK/p5e3sjMDBQ5Z4AQEREhFL7hx9+CADYu3evUvsrr7yCTp06KT77+fkBePr65LPnKmsvi4mIiEgsTCm3ysvLAwDY2dm98Nhl28v66+LZHCQnJwf3799H9+7dce3aNeTk5ACAYiTWr7/+CrlcrvM5NPH09ISzszNcXV0xduxYNGvWDP/5z3+UFp2xsrJSjJouKSnBgwcPYGtrC09PT7V50pgxY2Bpaan4XDZCrCwnOXfuHO7du4eJEycq9SubfuFZ+/btQ/369ZXmI7WwsMDUqVORn5+Po0ePKvUfPHiw0jHK8qGQkBClOcL8/PxQVFSE27dva3mniMSNr9cRiZizszN69+6NLVu2QCaToaSkRGni6opydXXFt99+C4lEAmtrazRv3hwuLi5a7//NN9+gRYsWqFWrFurVqwdPT09FwmBlZYUlS5bgww8/RL169dCxY0f0798fo0ePVqzolpKSAuB/c0E9z97eXudrun79Oho0aABbW1uldk9PT5V+6tqBp4WbAwcO4PHjx6hduzYyMjKUtjs4OCglb3PnzkXXrl2Rn5+PnTt3YuvWrWpfN/v111/x+eef4/z580pzKT0/7wIApeJOmTp16igKeFlZWSgoKEDz5s1V+nl6eioVza5fvw4zMzM0a9ZMqV/9+vXh6OiouBeazl2WWD07FP3Zdk1FRSIiIlNlSrmVtsWkvLw8SCQSxaTcujhx4gQiIyNx6tQpyGQypW05OTlwcHBA9+7d8dZbbyEqKgrLly9Hjx49EBwcjBEjRmhc0a+oqEhl3iJnZ2eYm5srPv/000+wt7dHVlYWvvrqK6SlpSnlUQBQWlqKL7/8EitXrkRaWhpKSkoU28qmFXjW87lK2at6ZTlJWW7zfJ5kYWEBd3d3pbbr16+jefPmKrmbt7e30rE0nZt5EtFTLDoRidyIESMwYcIEZGRkoG/fvhWac+l5tWvXRu/evSu8f4cOHRSr16nz/vvvY8CAAdi1axcOHDiAOXPmYNGiRfj999/Rtm1blJaWAng6r1NZIepZzz4tkkgkEARBpc+zSYmhNGjQQOnzhg0bEBYWpvjcunVrxX0MDg6GTCbDhAkT0KVLF0UC8scff2DgwIHo1q0bVq5ciQYNGsDCwgIbNmzAli1bVM75bLL2LHX3QFvqilvqaDq3IWIiIiIyFlPJrRwcHNCwYUNcvHjxhf0uXryIxo0bK0buaPp3/fncKDU1Fa+//jq8vLwQHR2NJk2awNLSEvv27cPy5csV+ZhEIsGOHTvw559/4pdffsGBAwcwduxYLFu2DH/++afKAz0AOHnyJHr27KnUlpaWpjQvZrdu3RSFsgEDBqB169YYOXIk4uPjFYWehQsXYs6cORg7dizmz5+Pl156CWZmZnj//fcV8T3LmDkJ8yQi9Vh0IhK5N998E+Hh4fjzzz+xbds2Y4ejNQ8PD3z44Yf48MMPkZKSAh8fHyxbtgyxsbGK18dcXFzKTdDq1Kmj9jWu558+NW3aFHFxccjPz1dKjpKTk1X6qWsHng6Nd3JyQu3atQFAaXUbAGjZsuULY128eDF27tyJBQsWYPXq1QCePuWztrbGgQMHlJ4Wbtiw4YXH0sTZ2RlSqVQxWuxZ6q61tLQUKSkpiqd2AJCZmYns7GzFvSAiIqpJTCm3GjBgANasWYPjx4+jS5cuKtv/+OMPpKenK70qX6dOHbWr8z6fG/3yyy8oLCzEnj17lEbpaJrGoGPHjujYsSMWLFiALVu2YOTIkdi6dSvGjx+v0rdNmzYqeZK6B4llbG1tERkZiTFjxuDHH3/EsGHDAAA7duxAz549sW7dOqX+2dnZFRrZVZbbpKSkKI2ol8vlSEtLQ5s2bZT6Xrx4EaWlpUqjnZKSkpSORUQvxjmdiETO1tYWq1atwrx58zBgwABjh1MumUyGJ0+eKLV5eHjAzs5O8WpZYGAg7O3tsXDhQrVzBzy7tLCHhweSkpKU2i5cuIATJ04o7RMUFITi4mKlZYBLSkqwYsUKpX4NGjSAj48PNm3apJSwXb58GQcPHkRQUJCirXfv3kpfz498ep6HhwfeeustbNy4UfFqnrm5OSQSidLTx/T0dOzateuFx9LE3NwcgYGB2LVrF27cuKFoT0xMxIEDB5T6ll1LTEyMUnt0dDQAqKycR0REVBOYUm41ffp02NjYIDw8HA8ePFDa9vDhQ0ycOBH29vaYMmWKot3DwwM5OTlKI6Tu3r2rsipt2QicZ0fc5OTkqDz4evTokcqoHB8fHwBQmhbgWXXq1FHJk6ytrV94rSNHjkTjxo2xZMkSpRifP/f27dsrPB9S+/bt4ezsjNWrV6OoqEjRvnHjRpVCXVBQEDIyMpQKj8XFxVixYgVsbW3RvXv3CsVAVNNwpBNRNRAaGqp133PnzuHzzz9Xae/Ro4faJ2j6duXKFbz++usYMmQIXnnlFdSqVQs7d+5EZmam4qmWvb09Vq1ahVGjRqFdu3YYNmwYnJ2dcePGDezduxf+/v74+uuvATydrDs6OhqBgYEYN24c7t27h9WrV6Nly5bIzc1VnHfAgAHw9/fHjBkzkJ6ejldeeQU///yzYpLMZy1duhR9+/ZFp06dMG7cOBQUFGDFihVwcHDAvHnzKnX9H330EX788UfExMRg8eLF6NevH6Kjo9GnTx+MGDEC9+7dwzfffINmzZqVO5xek6ioKOzfvx9du3bFpEmTFAlSy5YtlY7Zpk0bhIaGYu3atcjOzkb37t1x5swZbNq0CcHBwSrD4omIiGoKU8mtmjVrhu+++w7Dhw9H69atMW7cOLi5uSE9PR3r1q3Do0ePsHXrVri5uSn2GTZsGD755BO8+eabmDp1KmQyGVatWoUWLVooTb4dEBAAS0tLDBgwAOHh4cjPz8e3334LFxcX3L17V9Fv06ZNWLlyJd588014eHggLy8P3377Lezt7ZUexlWWhYUFpk2bho8++gj79+9Hnz590L9/f3z22WcYM2YMOnfujEuXLuH7779XmX9Jl3N8/vnnCA8PR69evTB06FCkpaVhw4YNKsd85513sGbNGoSFhSE+Ph6urq7YsWMHTpw4gZiYmHIneCei/zLewnlEVBHPLuv7IpqW9dX0NX/+fEEQni7r27JlS4PFdv/+fWHy5MmCl5eXULt2bcHBwUHw8/MTfvzxR5W+hw8fFgIDAwUHBwfB2tpa8PDwEMLCwoRz584p9YuNjRXc3d0FS0tLwcfHRzhw4IDa5YIfPHggjBo1SrC3txccHByEUaNGCX/99ZcAQNiwYYNS399++03w9/cXpFKpYG9vLwwYMED4559/tLoPZcv2bt++Xe32Hj16CPb29oplddetWyc0b95csLKyEry8vIQNGzaoXdYYGpZlbtq0qRAaGqrUdvToUcHX11ewtLQU3N3dhdWrV6s9plwuF6KiogQ3NzfBwsJCaNKkiTBz5kyVZXzVfT9piqls6eClS5eqv0FEREQmxJRzqzKXLl0SRowYIdSvX18wMzMTAAjW1tbC33//rbb/wYMHhVatWgmWlpaCp6enEBsbqzYP2LNnj/Dqq68K1tbWgqurq7BkyRJh/fr1AgAhLS1NEARBSEhIEIYPHy68/PLLgpWVleDi4iL0799fJR8DIERGRpZ7LWVxZGVlqWzLyckRHBwchO7duwuCIAhPnjwRPvzwQ6FBgwaCVCoV/P39hVOnTgndu3dX9BEEzblXWU7yfJ63cuVKwc3NTbCyshLat28vHDt2TOWYgiAImZmZwpgxYwQnJyfB0tJSaN26tcqxNOU9mmLS9vuNqLqQCAJnMCMiIiIiIhKL7777DmFhYQgJCcF3331n7HCIiDTi63VEREREREQiMnr0aNy9exczZsxA48aNsXDhQmOHRESkFkc6ERERERERERGR3nH1OiIiIiIiIiIi0jsWnYiIiIiIiIiISO9YdCIiIiIiIiIiIr3jROJqlJaW4s6dO7Czs4NEIjF2OERERKQngiAgLy8PDRs2hJkZn73pG3MoIiKi6qmiORSLTmrcuXMHTZo0MXYYREREZCA3b95E48aNjR1GtcMcioiIqHrTNYdi0UkNOzs7AE9vpr29vZGjMT1yuRwHDx5EQEAALCwsjB2OzsQePyD+a2D8xsX4jYvxG9fDhw/h5uam+Lee9Is51IuJ/eeH8RsX4zcuxm98pnYNqamp6NO1HerXNtzI3ozHAvb/kQAPDw+DnUNbFc2hWHRSo2w4uL29PRMmNeRyOWxsbGBvb28SP+y6Env8gPivgfEbF+M3LsZvXHK5HAD46peBMId6serw88P4jYfxGxfjNz5Tu4bHjx/j3fZWmNfDymDnmHekEIIgmMS/qRXNoVh0IiIiIiIiIiLSQVJSEtbEF2FPstxg57ibL2CkyEdns+hERERERERERKSD4OBgAICXlxdsbGwU7YmJiQgJCdH5eLGxsfD29lZqs7OzQ/PmzSsVp7Gx6EREREREREREpAMnJyeMHz9epd3Lywvx8fEq7Xl5edi9ezcGDRqkdl6k54tX1QWLTkRERDVcSkoK8vLylNoKCgqQnp6u0rekpATnz59HTk4OzM3NVba7urpCKpUqtVWHp3RERERE2rCxsUG7du1U2uVyObKzs9G5c2eTmJOqqrDoREREVINduHABfbq0RQNb3SaFPPKD9n3v5gs4lpDMwhMREREp3L9/Hwd++g42JblK7UVFRXjw4IFKf3lREZKvXEH6yV2wsLRU2V63bl1YPtfu5NYSXfsO1m/gpBMWnYiIiGqws2fPItzX0uArrxARERE9a9euXbj1wyz1OYjqYGpACqANAFxWf8Bs1aZ5PxbC2a01vLy8KhwnVQ6LTkRERDVYcHAwDpTk4q8mL8Ha2lrRXlhYiDt37qj0Ly0tRXJyMjw9PWFmZqayvWHDhrCyUk4eR/9fU7hzlBMRERE9oywH2fncSKebN29i48aNOh8vLCwMTZo0UWp7/ZOWLDgZGYtORERElXTt4ikU3r+u1Paios3V5GQcuHdZ66KNlVNTuL/aSb9B/5eTkxNGhkeo3eajpk0ul6Nk3z4EBgXVqPkIiIiIxEAmkyEpKUmlPS8vD0ePHoWjo6PJTGKtKQeRyWToMniySntNnYhb7Fh0IiIiqoQLFy5g57SeaoeG+2jYp78tgNsaNt5UbZp3pBAjv73EOZGIiIjohZKSkuDr66tx+/Lly9W2x8fHq5382hg4EXf1wqITERFRJZw9exZr4ouwJ1lusHPczRcw0mBHJyIiourCy8sL8fHxKu2XL19GaGgoNm3ahFatWqndj8gQWHQiIiKTJJbh4cHBwWrPW1BQgPT0dJX+JSUlOH/+PHx8fGBurjpLpqurK6RSqVKbnZ0dRzkRERGRkpSUFOTl5enlWOpyLoA5CFUei05ERGR06pKmxMREhISEaNxH0/Dw2NhYeHt7K7UZMmFycnLC+PHj1W7z9/dXaZPL5XBwcEAQ50QiIiIyOWJ56JWSkoIWLVrovF9oaKjO+1y5coWFJ6owFp2IiMioUlJS0K2dJxrYSlS2ta2vOtF2eZZNH63SdjdfwLGEZCZMRERE9EJimROp7GGduodt6uTn52PXrl0IDg6Gra2tVucoewCor9FUVDOx6EREVE2J5UldZmYmwn0t1U7ErS/zjhQyYSIiIiIl6kZaFxQUIDY2VqVvamoqIiMjERUVBQ8PD5XtBQUFSEhIUGoz9Ktp9W0laNfAHN5aPKSTF9ug9LWm6ORqA4ta2j3Uk2abo76ah4JEumDRiYiomhLLk7qkpKSqmYhbTYGNiIiIaqYXjbTWpG19M+xaFaV1f0OPtA73tYT3sXDgWPl9LQD0AIBk7Y/v/d9zEFUGi05ERNWAmJ/UaZqIu7w5nTSp6jmdiIiI6H/EMtI6Ly9P9COt18QXYejcjfDWYuU5eXExTpw4AX9/f1jU0q4MkJiUhDXLRmBgZQOlGo1FJyIikavoRJKRkZE69TfUJJKaJuLWtORvXl4edu/ejUGDBplE0kpERET/I5aR1jKZDGvii9BmyAx4aVG0KSiQ4fDhI+jZswek0vLzjLS0NKyJ/9RgBRuZTIaMfAEnruWjwLG03P75+TLsOXsdZo3aaj+n090SZOQLlQ2VajgWnYiIRC4vLw/1bSVYF7MAbm5u5favSNI07v1Pq3xOJBsbG7XJp1wuR3Z2Njp37szV34iIiIxIzCOtk5KSkJEv4P8ma/+6HABg2Waduqt7QKYPZaPJJkyYoNN+mop+L2Koa6CagUUnIiKRk8lkCPe1RNDNxcBN7fZpZwvg7G6t+vJ9fiIiInqe2Edaa3q9v6CgAOnp6Sr9yyuaubq6QiqVKrUZY3oCTS5fvozQ0FBs2rQJrVq10vo8nKKAKotFJyIikeNE3ERERFTVxD7SWtPr/QkJCS+cU1JT0ayqXw/UFL+mObXKw+kJyFBYdCIiEjmxP6kjIiIicaqOI60rM6ekKShvTq3Q0FC17VVdNKOawySKTt988w2WLl2KjIwMtGnTBitWrECHDh3U9pXL5Vi0aBE2bdqE27dvw9PTE0uWLEGfPn0UfRYtWoSff/4ZSUlJkEql6Ny5M5YsWQJPT8+quiQioioj9id1RFQxzJ+IyJjEPhG3JmKfU1LsRTOqfoxedNq2bRsiIiKwevVq+Pn5ISYmBoGBgUhOToaLi4tK/9mzZyM2NhbffvstvLy8cODAAbz55ps4efIk2rZtCwA4evQoJk+ejNdeew3FxcWYNWsWAgIC8M8//6B27dpVfYlEREbBpIOo+mL+RETGJvaJuKsrsRfNqPoxetEpOjoaEyZMwJgxYwAAq1evxt69e7F+/XrMmDFDpf/mzZvx6aefIigoCADw7rvv4rfffsOyZcsUqyTs379faZ+NGzfCxcUF8fHx6Natm4GviIjINDDpIKq+mD8RkbHx9X4i0oZRi05FRUWIj4/HzJkzFW1mZmbo3bs3Tp06pXafwsJCWFtbK7VJpVIcP35c43lycnIAAC+99JLGYxYWFio+5+bmAnj6i5lcbriJecWq7J6I9d6IPX5A/NcglvhlMhmSk5NV2vPy8nD06FHY2tqqffrm6elp0hMxiuX+a8L4jau6xC9mppI/lR2XOZT2qsvPD+M3LLHkHw4ODmrnB/rrr78q9Hr/6dOn0bp1a5V2U/nzEsv3jyZijx8Q/zVUl/h1JREEQdBzLFq7c+cOGjVqhJMnT6JTp06K9o8//hhHjx7F6dOnVfYZMWIELly4gF27dsHDwwNxcXEYNGgQSkpKlJKeMqWlpRg4cCCys7M1Jlbz5s1DVJTqsNAtW7aY9C+ORKQ/d+7cQUFBgVLbrVu3sHz5cp2P9cEHH6Bx48ZKbVKpFA0bNqxUjERUeTKZDCNGjEBOTg7s7e2NHU6FmEr+BDCHIqqs6ph/FBYW4tatWyrtRUVFuHfvHlxcXGBpqTpBeOPGjWFlZVUVIRJRBVQ0hzL663W6+vLLLzFhwgR4eXlBIpHAw8MDY8aMwfr169X2nzx5Mi5fvvzChGnmzJmIiIhQfM7NzUWTJk0QEBAg2oTUkORyOQ4dOoQ33nhDlK/miD1+QPzXYGrxp6SkYGLIm2hgK1HZ1ra+mc7HO/LDlyptd/MFxJ2+bBJDxE3t/uuK8RuX2ON/8OCBsUMwCkPkTwBzKF2J/eeH8esX8w9xYfzGJ/ZrEHv8Fc2hjFp0cnJygrm5OTIzM5XaMzMzUb9+fbX7ODs7Y9euXXjy5AkePHiAhg0bYsaMGXB3d1fpO2XKFPz66684duyYStX/WVZWVmqr6hYWFqL8ZqgqYr8/Yo8fEP81mEr8Dx8+RLivJeb1MNzTtXlHCvHkyROTuN4ypnL/K4rxG5dY4xdjzM8zlfwJYA5VUWK/P4xfP5h/iBPjNz6xX4NY469ozEYtOllaWsLX1xdxcXGKiehKS0sRFxeHKVOmvHBfa2trNGrUCHK5HD/99BOGDBmi2CYIAt577z3s3LkTR44cgZubmyEvg4hELikpCWvii7An2XDvV9/NFzCSq68QkR4wfyJ6MZlMhqSkJJX2sjmRHB0dNa7eWpWvhTL/IKKawOiv10VERCA0NBTt27dHhw4dEBMTg8ePHytWYxk9ejQaNWqERYsWAXg6wdzt27fh4+OD27dvY968eSgtLcXHH3+sOObkyZOxZcsW7N69G3Z2dsjIyADwdLK751dEICLStPpKYmLiCyfC1CQ2Nhbe3t5KbVx9hYj0ifkTEXD//n0c+Ok72JTkKrXfvHkTGzdu1LifutfQACAsLAxNmjRRanNya4mufQdXOlZ1mH8QUU1g9KLT0KFDkZWVhblz5yIjIwM+Pj7Yv38/6tWrBwC4ceMGzMz+907zkydPMHv2bFy7dg22trYICgrC5s2b4ejoqOizatUqAECPHj2UzrVhwwaEhYUZ+pKI6L/E8qTRyckJ48ePVxtHfHy8SnteXh52796NQYMGmUT8RFTzMH8iAnbt2oVbP8xSfT3NCpgabluBI+4A7im3zPuxEM5ureHl5VXhODVh/kFENYHRi07A07kDNA0HP3LkiNLn7t27459//nnh8Yy4IB8RPSMpKQm+vr4at2tamSU+Ph7t2rUzVFhas7GxURuHXC5HdnY2OnfuLMr3sYmoemD+RDVdcHAwDpTkYudzI52KiorUTngrLypC8pUr8GzRAhZqVk+rW7euyqpqr3/S0iAFpxdh/kFE1YlJFJ2IqHrS9KTu8uXLCA0NxaZNm9CqVSu1+xERERG9iJOTE0aGR5Tf8b/kcjn27duHoKAgFm2IiKqIzkWna9euqV3phIhqtpSUFOTl5enlWOpeyeOcBEQkZsyfiIiIqCbSuejUrFkzdO/eHePGjcPbb78Na2trQ8RFRCKSkpKCFi1a6LxfaGioTv2vXLnCwhMRiRLzJyIiIqqJdC46JSQkYMOGDYiIiMCUKVMwdOhQjBs3Dh06dDBEfEQkAnl5eahvK8G6mAVaLbFdUCDD4cNH0LNnD0il5U94mZaWhnHvf6q3kVRERFWN+RMRERHVRDoXnXx8fPDll19i2bJl2LNnDzZu3IguXbqgRYsWGDt2LEaNGgVnZ2dDxEpEJizc1xJBNxcDN7Xr384WwNndWvX1/u/xiYjEivkTERER1UQVnki8Vq1a+L//+z/069cPK1euxMyZMzF9+nTMmjULQ4YMwZIlS9CgQQN9xkpU48hkMrXzG+Xl5eHo0aNwdHQ0mSVz18QXYejcjfDWYhJweXExTpw4AX9/f1jUKv+vocSkJKxZNgID9REoEZERMX8iIiKimqTCRadz585h/fr12Lp1K2rXro3p06dj3LhxuHXrFqKiojBo0CCcOXNGn7ES1ThJSUnw9fXVuH358uVq2+Pj49UutWsoMpkMGfkCTlzLR4Fjabn98/Nl2HP2OswatYWtrW25/RPvliAjn0t5E5H4MX8iIiKimkTnolN0dDQ2bNiA5ORkBAUF4bvvvkNQUBDMzMwAAG5ubti4cSNcXV31HStRjePl5YX4+HiV9suXLyM0NBSbNm1Cq1at1O5XlcpGY02YMEGn/TQVzTRRN6qLiEgMmD8RERFRTaRz0WnVqlUYO3YswsLCNA7/dnFxwbp16yodHFFNkpKSoreJstW9kmdnZ2ewld+Cg4MBaP9aX3lFM3UMGT8RkaExfyIiIqKaSOeiU0pKSrl9LC0tdV4KnagmS0lJQYsWLXTeT9efsytXrhikcOPk5ITx48ertGuak6o8xpiTiojIkJg/ERERUU2kc9Fpw4YNsLW1xeDBg5Xat2/fDplMxmSJqALKRjjFxsbC29u73P75+fnYtWsXgoODtZsTKTERISEhehtJpa3y5qTS9PdFVc9JRURkaMyfiIiIqCbSuei0aNEirFmzRqXdxcUF77zzDpMmokrw9vbWqtgil8vx6NEjdOrUCRYWFlUQWcVompMqLy8Pu3fvxqBBgzSuvkdEVJ0wfyJTdP/+fRz46TvYlOQqtRcVFeHBgwcq/eVFRUi+cgXpJ3fBwtJSZXvdunVh+Vy7k1tLdO07WKUvERHVDDoXnW7cuAE3NzeV9qZNm+LGjRt6CYqoJqpvK4E0+wpwx6z8zsXFcJClA3cvALXK/zGWZl9BfVtJ5YPUkY2NjdoimlwuR3Z2Njp37mzSRTMiIn1h/kSmaNeuXbj1wyzM62GlutFczQ5SAG0A4LL6A2arNs37sRDObq35QImIqIbSuejk4uKCixcvqqyucuHCBdStW1dfcRHVOOG+lvA+Fg4cK7+vBYAeAJCs3bG9/3t8IiIyDuZPZIqCg4NxoCQXO58b6XTz5k1s3LhR5+OFhYWhSZMmSm2vf9KSBSciohpM56LT8OHDMXXqVNjZ2aFbt24AgKNHj2LatGkYNmyY3gMkqinWxBdh6NyN8NYiMZMXF+PEiRPw9/eHhRYjnRKTkrBm2QgM1EegRESkM+ZPZIqcnJwwMjxCpV0mk6HL4Mkq7dq8Hs+FQIiI6Fk6F53mz5+P9PR0vP7666j13192S0tLMXr0aCxcuFDvARLVFBn5AgocWwANfcrvLJcjx+Y20KANoMXraQUZpcjIFyofJBERVQjzJxITvh5PRET6onPRydLSEtu2bcP8+fNx4cIFSKVStG7dGk2bNjVEfEQ1gkwmAwAkJCRo1T8/Px9Hjx5FnTp1tF69joiIjIf5ExEREdVEOhedyrRo0QItWrTQZyxENVZSUhIAYMKECTrtt3z5cp36qxsKT0REVYf5ExEREdUkFSo63bp1C3v27MGNGzdQVFSktC06Olrn433zzTdYunQpMjIy0KZNG6xYsQIdOnRQ21cul2PRokXYtGkTbt++DU9PTyxZsgR9+vSp8DGJjC04OBiA9nMhXL58GaGhodi0aRNatWql1Tns7OzQvHnzyoRJRESVwPyJiIiIahqdi05xcXEYOHAg3N3dkZSUhFatWiE9PR2CIKh997s827ZtQ0REBFavXg0/Pz/ExMQgMDAQycnJcHFxUek/e/ZsxMbG4ttvv4WXlxcOHDiAN998EydPnkTbtm0rdEwiY3NycsL48eO17l9cXAzgaZGqIj93RERUtZg/ERERUU1kpusOM2fOxPTp03Hp0iVYW1vjp59+ws2bN9G9e3cMHjxY5wCio6MxYcIEjBkzBq+88gpWr14NGxsbrF+/Xm3/zZs3Y9asWQgKCoK7uzveffddBAUFYdmyZRU+JpGpkslkSEhIUPkqex0vKSlJ7fayOaKIiMg0MH8iIiKimkjnkU6JiYn44Ycfnu5cqxYKCgpga2uLzz77DIMGDcK7776r9bGKiooQHx+PmTNnKtrMzMzQu3dvnDp1Su0+hYWFsLa2VmqTSqU4fvx4pY5ZWFio+Jybmwvg6VB0uVyu9fXUFGX3RKz3RkzxX758GX5+fhq3h4aGqm0/ffq04sm1KRLTn4E6jN+4GL9xVZf4q1p1zJ/KjsscSnvV5eeH8RsH4zcuxm98Yr+G6hK/rnQuOtWuXVsxD0GDBg2QmpqKli1bAgDu37+v07Hu37+PkpIS1KtXT6m9Xr16ipEczwsMDER0dDS6desGDw8PxMXF4eeff0ZJSUmFj7lo0SJERUWptB88eFCr+XVqqkOHDhk7hBcqLCzErVu3VNqLiopw7949JCYmwtLSUmV748aNYWVlVRUhlquwsFDpKXSZsmtwcXFRew3p6em4e/duVYRYKab+PVQexm9cjN+4xBq/sUaCVsf8CWAOVVFi/fkpw/iNi/EbF+M3PrFfg1jjr2gOpXPRqWPHjjh+/Di8vb0RFBSEDz/8EJcuXcLPP/+Mjh07VigIXXz55ZeYMGECvLy8IJFI4OHhgTFjxlRq6PfMmTMRERGh+Jybm4smTZogICAA9vb2+gi7WpHL5Th06BDeeOMNWFhYGDscjf766y8MHTpU5/1MfZQQIJ4/A00Yv3ExfuNi/Mb14MEDo5y3OuZPAHMoXYn954fxGxfjNy7Gb3xivwaxx1/RHErnolN0dDTy8/MBAFFRUcjPz8e2bdvQvHlznVdecXJygrm5OTIzM5XaMzMzUb9+fbX7ODs7Y9euXXjy5AkePHiAhg0bYsaMGXB3d6/wMa2srNSObLGwsBDlN0NVMfX706pVK8THx6u0l7fym5eXl0lf17NM/c+gPIzfuBi/cTF+4zBWzNUxfwKYQ2kik8nUjhDLy8vD0aNH4ejoCDs7O5Xt2q5ia2xi//Nl/MbF+I1L7PED4r8GscZf0Zh1KjqVlJTg1q1bePXVVwE8HSq+evXqCp0YACwtLeHr64u4uDjFkvGlpaWIi4vDlClTXrivtbU1GjVqBLlcjp9++glDhgyp9DGperGxsVG7IhBXfiMioqrE/KnmSUpKgq+vr8bty5cvV9seHx/P3ISIiKoVnYpO5ubmCAgIQGJiIhwdHfUSQEREBEJDQ9G+fXt06NABMTExePz4McaMGQMAGD16NBo1aoRFixYBePrq0+3bt+Hj44Pbt29j3rx5KC0txccff6z1MYmIiIiqCvOnmsfLy6vCo62JiIiqE51fr2vVqhWuXbsGNzc3vQQwdOhQZGVlYe7cucjIyICPjw/279+vmMjyxo0bMDMzU/R/8uQJZs+ejWvXrsHW1hZBQUHYvHmzUhJX3jGp+klJSUFeXp5WfcuGuyclJaFWLe1+BOzs7NC8efMKx0dERDUb86eahaOtiYiIntK56PT5559j+vTpmD9/Pnx9fVG7dm2l7RWZNHLKlCkah24fOXJE6XP37t3xzz//VOqYVL2kpKSgRYsWOu8XGhqqU/8rV66w8ERERBXC/Kn64oMvIiIizXQuOgUFBQEABg4cCIlEomgXBAESiUSx9C5RVSlL9GJjY+Ht7V1u//z8fOzatQvBwcGwtbUtt39iYiJCQkK0TiiJiIiex/ypeuKDLyIiohfTueh0+PBhQ8RBVGne3t5aDVWXy+V49OgROnXqJMpVA4iISHyYP1VPfPBFRET0YjoXnbp3726IOIgqpb6tBNLsK8Ads/I7FxfDQZYO3L0AaDG0XZp9BfVtJeX2IyIi0oT5U/XGB19ERETq6Vx0Onbs2Au3d+vWrcLBEFVUuK8lvI+FAy/+9gQAWADoAQDJ2h3b+7/HJyIiqijmT9UXH3wRERFppnPRqUePHiptz85NwDkJyBjWxBdh6NyN8NZiqWF5cTFOnDgBf39/WGiR8CUmJWHNshEYqI9AiYioRmL+VH3xwRcREZFmOhedHj16pPRZLpfjr7/+wpw5c7BgwQK9BUaki4x8AQWOLYCGPuV3lsuRY3MbaNAG0GJoe0FGKTLyhcoHSURENRbzp+qLD76IiIg007no5ODgoNL2xhtvwNLSEhEREYiPj9dLYERERETVBfOn6kkmkyEjX8CJa/kocCwtt39+vgx7zl6HWaO22k0kfreED76IiEjUdC46aVKvXj0kJ2s5VphIj2QyGQAgISFBq/75+fk4evQo6tSpo/XKMURERIbA/EnckpKSAAATJkzQab/ly5fr1N/Ozk6n/kRERKZC56LTxYsXlT4LgoC7d+9i8eLF8PHx0VdcRFpjwkdERKaO+VP1FBwcDADw8vKCjY1Nuf0vX76M0NBQbNq0Ca1atdLqHHZ2dmjevHllwiQiIjIanYtOPj4+kEgkEATlob4dO3bE+vXr9RYYkbaY8BERkalj/lQ9OTk5Yfz48Vr3Ly4uBvA0Z2nXrp2hwiIiIjIZOhed0tLSlD6bmZnB2dkZ1tbWeguKSBdM+IiIyNQxf6pZZDKZYiT2s8rakpKSUEvNROLaPkAjIiISC52LTk2bNjVEHER6x4SPiIhMBfOnmiUpKQm+vr4at4eGhqptj4+P5wMxIiKqVnQuOk2dOhXNmjXD1KlTldq//vprXL16FTExMfqKjahSmPAREZGpYP5Us3h5ealdkTAvLw+7d+/GoEGD1M4V6eXlVRXhERERVRmdi04//fQT9uzZo9LeuXNnLF68mEkTmQwmfEREZCqYP9UsNjY2ah9gyeVyZGdno3PnzrCwsDBCZERERFVL56LTgwcP4ODgoNJub2+P+/fv6yUoIn1gwkdERKaC+RMRERHVRGa67tCsWTPs379fpf0///kP3N3d9RIUERERUXXC/ImIiIhqIp1HOkVERGDKlCnIyspCr169AABxcXFYtmwZh4YTERERqcH8iYiIiGoinYtOY8eORWFhIRYsWID58+cDAFxdXbFq1SqMHj1a7wESERERiR3zJyIiIqqJdH69DgDeffdd3Lp1C5mZmcjNzcW1a9cqnDB98803cHV1hbW1Nfz8/HDmzJkX9o+JiYGnpyekUimaNGmCDz74AE+ePFFsLykpwZw5c+Dm5gapVAoPDw/Mnz8fgiBUKD4iIiIifdBn/gQwhyIiIiLTp/NIp7S0NBQXF6N58+ZwdnZWtKekpMDCwgKurq5aH2vbtm2IiIjA6tWr4efnh5iYGAQGBiI5ORkuLi4q/bds2YIZM2Zg/fr16Ny5M65cuYKwsDBIJBJER0cDAJYsWYJVq1Zh06ZNaNmyJc6dO4cxY8bAwcFBZZliIiIioqqgz/wJYA5FRERE4qBz0SksLAxjx45F8+bNldpPnz6Nf//73zhy5IjWx4qOjsaECRMwZswYAMDq1auxd+9erF+/HjNmzFDpf/LkSfj7+2PEiBEAng5LHz58OE6fPq3UZ9CgQejXr5+izw8//PDCp3+FhYUoLCxUfM7NzQXwdJUzuVyu9fXUFGX3RKz3RuzxA+K/BsZvXIzfuBi/cRkrbn3mTwBzKLGqLj8/jN84GL9xMX7jE/s1VJf4dSURdBwzbW9vj4SEBDRr1kyp/erVq2jfvj2ys7O1Ok5RURFsbGywY8cOBAcHK9pDQ0ORnZ2N3bt3q+yzZcsWTJo0CQcPHkSHDh1w7do19OvXD6NGjcKsWbMAAAsXLsTatWtx8OBBtGjRAhcuXEBAQACio6MxcuRItbHMmzcPUVFRas9nY2Oj1fUQERGR6ZPJZBgxYgRycnJgb29fZefVV/4EMIciIiKiqlfRHErnkU4SiQR5eXkq7Tk5OSgpKdH6OPfv30dJSQnq1aun1F6vXj0kJSWp3WfEiBG4f/8+unTpAkEQUFxcjIkTJyqSJQCYMWMGcnNz4eXlBXNzc5SUlGDBggUakyUAmDlzJiIiIhSfc3Nz0aRJEwQEBFRpQioWcrkchw4dwhtvvAELCwtjh6MzsccPiP8aGL9xMX7jYvzG9eDBA6OcV1/5E8AcSszE/vPD+I2L8RsX4zc+sV+D2OOvaA6lc9GpW7duWLRoEX744QeYm5sDeDrx5KJFi9ClS5cKBaGtI0eOYOHChVi5ciX8/Pxw9epVTJs2DfPnz8ecOXMAAD/++CO+//57bNmyBS1btsT58+fx/vvvo2HDhggNDVV7XCsrK1hZWam0W1hYiPKbQV9kMpna5DUvLw9Hjx6Fo6Mj7OzsVLZ7eXmJ4ulmdfjzFfs1MH7jYvzGxfiNw1gxGzN/AphDmRqx3x/Gb1yM37gYv/GJ/RrEGn9FY9a56LRkyRJ069YNnp6e6Nq1KwDgjz/+QG5uLn7//Xetj+Pk5ARzc3NkZmYqtWdmZqJ+/fpq95kzZw5GjRqF8ePHAwBat26Nx48f45133sGnn34KMzMzfPTRR5gxYwaGDRum6HP9+nUsWrRIY8JE6iUlJcHX11fj9uXLl6ttj4+PR7t27QwVFhERkejoK38CmEMRERGReJjpusMrr7yCixcvYsiQIbh37x7y8vIwevRoJCUloVWrVlofx9LSEr6+voiLi1O0lZaWIi4uDp06dVK7j0wmg5mZcshlTwvLpqbS1Ke0tFTr2OgpLy8vxMfHq3xt2rQJALBp0ya12728vIwcORERkWnRV/4EMIciIiIi8dB5pBMANGzYEAsXLlRqy87Oxtdff40pU6ZofZyIiAiEhoaiffv26NChA2JiYvD48WPFSiyjR49Go0aNsGjRIgDAgAEDEB0djbZt2yqGhs+ZMwcDBgxQJE4DBgzAggUL8PLLL6Nly5b466+/EB0djbFjx1bkUms0GxsbtSOWiouLATwtSnFEExERkXb0lT8BzKGIiIhIHCpUdHpWXFwc1q1bh507d8LGxkanpGno0KHIysrC3LlzkZGRAR8fH+zfv18xMeaNGzeUnrjNnj0bEokEs2fPxu3bt+Hs7KxIkMqsWLECc+bMwaRJk3Dv3j00bNgQ4eHhmDt3bmUvlYiIiEgvKpM/AcyhiIiISBwqVHS6efMmNmzYgA0bNuDGjRsYOnQodu7ciddff13nY02ZMkVjonXkyBHlYGvVQmRkJCIjIzUez87ODjExMYiJidE5FiIiIiJD0Wf+BDCHIiIiItOn9ZxOcrkc27dvR2BgIDw9PXH+/HksXboUZmZmmD17Nvr06SPKGdiJiIiIDIX5ExEREdVkWo90atSoEby8vBASEoKtW7eiTp06AIDhw4cbLDiqOikpKcjLy9Oqb1JSkuK/tWpp9y1kZ2eH5s2bVzg+IiIiMWL+RERERDWZ1kWn4uJiSCQSSCQSxYSTVD2kpKSgRYsWOu+n6/LJV65cYeGJiIhqFOZPREREVJNpXXS6c+cOfvrpJ6xbtw7Tpk1D3759ERISAolEYsj4qAqUjXCKjY2Ft7d3uf3z8/Oxa9cuBAcHw9bWttz+iYmJCAkJ0XokFRERUXXB/ImIiIhqMq2LTtbW1hg5ciRGjhyJ1NRUbNiwAVOnTkVxcTEWLFiAsLAw9OrVi0/xRMzb2xvt2rUrt59cLsejR4/QqVMnzkNBRET0AsyfiIiIqCbTeiLxZ3l4eODzzz/H9evXsXfvXhQWFqJ///6KZXqJiIiISBnzJyIiIqpptB7ppI6ZmRn69u2Lvn37IisrC5s3b9ZXXERERETVEvMnIiIiqikqNNJJHWdnZ0REROjrcERERETVHvMnIiIiqs70VnQiIiIiIiIiIiIqU6nX66j6qG8rgTT7CnBHizpkcTEcZOnA3QtArfK/haTZV1Dflqv0EBEREREREdUkLDoRACDc1xLex8KBY+X3tQDQAwCStTu293+PT0REREREREQ1B4tOBABYE1+EoXM3wtvLq9y+8uJinDhxAv7+/rDQYqRTYlIS1iwbgYH6CJSIiIiIiIiIREFvRafdu3cjJycHo0eP1tchqQpl5AsocGwBNPQpv7Ncjhyb20CDNoCFRbndCzJKkZEvVD5IIiKiaob5ExEREVVneptI/JNPPsGYMWP0dTgiIiKiao/5ExEREVVnehvplJSUpK9DEREREdUIzJ+IiIioOtN5pNONGzcgCOpflbpx40alAyIiIiKqbpg/ERERUU2kc9HJzc0NWVlZKu0PHjyAm5ubXoIiIiIiqk6YPxEREVFNpHPRSRAESCQSlfb8/HxYW1vrHMA333wDV1dXWFtbw8/PD2fOnHlh/5iYGHh6ekIqlaJJkyb44IMP8OTJE6U+t2/fRkhICOrWrQupVIrWrVvj3LlzOsdWU8hkMgBAQkKCVl+nTp3C0aNHcerUKa36JyYmGvkKiYiIjEvf+RPAHIqIiIhMn9ZzOkVERAAAJBIJ5syZAxsbG8W2kpISnD59Gj4+PjqdfNu2bYiIiMDq1avh5+eHmJgYBAYGIjk5GS4uLir9t2zZghkzZmD9+vXo3Lkzrly5grCwMEgkEkRHRwMAHj16BH9/f/Ts2RP/+c9/4OzsjJSUFNSpU0en2GqSsvkkJkyYoNN+y5cv16m/nZ2dTv2JiIjEzhD5E8AcioiIiMRB66LTX3/9BeDpk7pLly7B0tJSsc3S0hJt2rTB9OnTdTp5dHQ0JkyYoFi1ZfXq1di7dy/Wr1+PGTNmqPQ/efIk/P39MWLECACAq6srhg8fjtOnTyv6LFmyBE2aNMGGDRsUbRy2/mLBwcEAAC8vL6VkWJPLly8jNDQUmzZtQqtWrbQ6h52dHZo3b16ZMImIiETHEPkTwByKiIiIxEHrotPhw4cBAGPGjMGXX34Je3v7Sp24qKgI8fHxmDlzpqLNzMwMvXv3xqlTp9Tu07lzZ8TGxuLMmTPo0KEDrl27hn379mHUqFGKPnv27EFgYCAGDx6Mo0ePolGjRpg0adILR/EUFhaisLBQ8Tk3NxcAIJfLIZfLK3WdYuDg4IDQ0FCt+xcUFAAAPDw80Lp1a633M5V7WRaHqcRTEWK/BsZvXIzfuBi/cVV13PrOnwDmUGJWXX5+GL9xMH7jYvzGJ/ZrqC7x60oiaFpKxcDu3LmDRo0a4eTJk+jUqZOi/eOPP8bRo0eVnrw966uvvsL06dMhCAKKi4sxceJErFq1SrG9bF6EiIgIDB48GGfPnsW0adOwevVqjYWVefPmISoqSqV9y5YtWo38qWlSU1Px4YcfYtmyZfDw8DB2OERERFqTyWQYMWIEcnJy9FIAMgbmUERERFTVKppDaT3S6Vnnzp3Djz/+iBs3bqCoqEhp288//1yRQ2rlyJEjWLhwIVauXAk/Pz9cvXoV06ZNw/z58zFnzhwAQGlpKdq3b4+FCxcCANq2bYvLly+/MGGaOXOmYs4F4OlTuiZNmiAgIEC0CakhlU1U6ufnhw4dOhg5Gt3J5XIcOnQIb7zxBiwsLIwdToWI/RoYv3ExfuNi/Mb14MEDo53bWPkTwBzKVIj954fxGxfjNy7Gb3xivwaxx1/RHErnotPWrVsxevRoBAYG4uDBgwgICMCVK1eQmZmJN998U+vjODk5wdzcHJmZmUrtmZmZqF+/vtp95syZg1GjRmH8+PEAgNatW+Px48d455138Omnn8LMzAwNGjTAK6+8orSft7c3fvrpJ42xWFlZwcrKSqXdwsJClN8MhlZ2T8R+f8QePyD+a2D8xsX4jYvxG4exYtZX/gQwh6oOxH5/GL9xMX7jYvzGJ/ZrEGv8FY1Z56LTwoULsXz5ckyePBl2dnb48ssv4ebmhvDwcDRo0EDr41haWsLX1xdxcXGKiaxLS0sRFxeHKVOmqN1HJpPBzMxMqc3c3BzA0wk6AcDf3x/JyclKfa5cuYKmTZtqHRs9JZPJFCvbPausLSkpCbVqqX4LaTshORERUU2hr/wJYA5FRERE4qFz0Sk1NRX9+vUD8DTpefz4MSQSCT744AP06tVL7Xv9mkRERCA0NBTt27dHhw4dEBMTg8ePHytWYhk9ejQaNWqERYsWAQAGDBiA6OhotG3bVjE0fM6cORgwYIAicfrggw/QuXNnLFy4EEOGDMGZM2ewdu1arF27VtdLrfGSkpLg6+urcbumofbx8fFo166docIiIiISHX3mTwBzKCIiIhIHnYtOderUQV5eHgCgUaNGuHz5Mlq3bo3s7GzIZDKdjjV06FBkZWVh7ty5yMjIgI+PD/bv34969eoBAG7cuKH0VG727NmQSCSYPXs2bt++DWdnZwwYMAALFixQ9Hnttdewc+dOzJw5E5999hnc3NwQExODkSNH6nqpNZ6Xlxfi4+NV2vPy8rB7924MGjQIdnZ2avcjIiKi/9Fn/gQwhyIiIiJx0Lno1K1bNxw6dAitW7fG4MGDMW3aNPz+++84dOgQXn/9dZ0DmDJlisah4EeOHFEOtlYtREZGIjIy8oXH7N+/P/r3769zLKTMxsZG7YgluVyO7OxsdO7cWZTvohIREVU1fedPAHMoIiIiMn06F52+/vprPHnyBADw6aefwsLCAidPnsRbb72F2bNn6z1AIiIiIrFj/kREREQ1kc5Fp5deeknx/2ZmZpgxY4ZeAyIiIiKqbpg/ERERUU1kVn6Xp+7cuYPp06cjNzdXZVtOTg4++ugjlaV7iYiIiGoy5k9ERERUk2lddIqOjkZubi7s7e1Vtjk4OCAvLw/R0dF6DY6IiIhIzJg/ERERUU2mddFp//79GD16tMbto0ePxq+//qqXoIiIiIiqA+ZPREREVJNpXXRKS0vDyy+/rHF748aNkZ6ero+YiIiIiKoF5k9ERERUk2k9kbhUKkV6errGxCk9PR1SqVRvgRERERGJHfOn8qWkpCAvL0+praCgoELFOFdXV5X7aWdnh+bNm1cmRCIiIqogrYtOfn5+2Lx5M7p166Z2+3fffYcOHTroLTAiIiIisWP+9GIpKSno1s4TDWwlBjvH3XwBxxKSWXgiIiIyAq2LTtOnT8cbb7wBBwcHfPTRR6hXrx4AIDMzE1988QU2btyIgwcPGixQIiIiIrFh/vRimZmZCPe1xLweVgY7x7wjhSojqYiIiKhqaF106tmzJ7755htMmzYNy5cvh729PSQSCXJycmBhYYEVK1agV69ehoyViIiISFSYP71YUlIS1sQXYU+y3GDnuJsvYKSdncGOT0RERJppXXQCgPDwcPTv3x8//vgjrl69CkEQ0KJFC7z99tto3LixoWIkIiIiEi3mT5oFBwcDALy8vGBjY6NoT0xMREhIiM7Hi42Nhbe3t1Ib53QiIiIyHp2KTgDQqFEjfPDBB4aIhYiIiKhaYv6knpOTE8aPH6/S7uXlhfj4eJX2sgnG1U0YXrbfs8UrIiIiMi6di05ERERERIZkY2ODdu3aqd3m7+9fxdEQERFRRZkZOwAiIiIiIiIiIqp+WHQiIiIiIiIiIiK9Y9GJiIiIiIiIiIj0Tueik7u7Ox48eKDSnp2dDXd3d70ERURERFSdMH8iIiKimkjnolN6ejpKSkpU2gsLC3H79m29BEVERERUnTB/IiIioppI66LTnj17sGfPHgDAgQMHFJ/37NmDnTt3Yv78+XB1da1QEN988w1cXV1hbW0NPz8/nDlz5oX9Y2Ji4OnpCalUiiZNmuCDDz7AkydP1PZdvHgxJBIJ3n///QrFRkRERFRRzJ+IiIioJqulbcfg4GAAgEQiQWhoqNI2CwsLuLq6YtmyZToHsG3bNkRERGD16tXw8/NDTEwMAgMDkZycDBcXF5X+W7ZswYwZM7B+/Xp07twZV65cQVhYGCQSCaKjo5X6nj17FmvWrMGrr76qc1xERERElcX8iYiIiGoyrYtOpaWlAAA3NzecPXsWTk5OegkgOjoaEyZMwJgxYwAAq1evxt69e7F+/XrMmDFDpf/Jkyfh7++PESNGAABcXV0xfPhwnD59Wqlffn4+Ro4ciW+//Raff/75C2MoLCxEYWGh4nNubi4AQC6XQy6XV+r6qqOyeyLWeyP2+AHxXwPjNy7Gb1yM37iqOu7qnD8BzKF0VV1+fhi/cTB+42L8xif2a6gu8etKIgiCUNmTZ2dnw9HRUef9ioqKYGNjgx07diieBAJAaGgosrOzsXv3bpV9tmzZgkmTJuHgwYPo0KEDrl27hn79+mHUqFGYNWuW0jFeeuklLF++HD169ICPjw9iYmLUxjFv3jxERUWpPZeNjY3O10VERESmSSaTYcSIEcjJyYG9vb1RYxF7/gQwhyIiIqopKppDaT3SqcySJUvg6uqKoUOHAgAGDx6Mn376CQ0aNMC+ffvQpk0brY91//59lJSUoF69ekrt9erVQ1JSktp9RowYgfv376NLly4QBAHFxcWYOHGiUsK0detWJCQk4OzZs1rFMXPmTERERCg+5+bmokmTJggICKhUQiqTyZCcnKzSXlBQgOvXr6Np06aQSqUq2z09PU06UZPL5Th06BDeeOMNWFhYGDscnYk9fkD818D4jYvxGxfjNy51K8hVheqYPwGGy6GqK7H//DB+42L8xsX4jU/s1yD2+CuaQ+lcdFq9ejW+//57AMChQ4fw22+/Yf/+/fjxxx/x0Ucf4eDBgxUKRFtHjhzBwoULsXLlSvj5+eHq1auYNm0a5s+fjzlz5uDmzZuYNm0aDh06BGtra62OaWVlBSsrK5V2CwuLSn0zpKamws/PT+f94uPj0a5duwqft6pU9v4Ym9jjB8R/DYzfuBi/cTF+4zBWzNUxfwIMl0NVd2K/P4zfuBi/cTF+4xP7NYg1/orGrHPRKSMjA02aNAEA/PrrrxgyZAgCAgLg6uqqc4HFyckJ5ubmyMzMVGrPzMxE/fr11e4zZ84cjBo1CuPHjwcAtG7dGo8fP8Y777yDTz/9FPHx8bh3755S0aakpATHjh3D119/jcLCQpibm+sUZ0V5eXkhPj5epT0xMREhISGIjY2Ft7e32v2IiIio+mD+RERERDWRzkWnOnXq4ObNm2jSpAn279+vmGRSEASUlJTodCxLS0v4+voiLi5OMSdBaWkp4uLiMGXKFLX7yGQymJmZKbWVJUGCIOD111/HpUuXlLaPGTMGXl5e+OSTT6o0YbKxsXnhiCVvb29RjGgiIiKiymH+RERERDWRzkWn//u//8OIESPQvHlzPHjwAH379gUA/PXXX2jWrJnOAURERCA0NBTt27dHhw4dEBMTg8ePHytWYxk9ejQaNWqERYsWAQAGDBiA6OhotG3bVjE8fM6cORgwYADMzc1hZ2eHVq1aKZ2jdu3aqFu3rko7ERERUVVg/kREREQ1kc5Fp+XLl8PV1RU3b97EF198AVtbWwDA3bt3MWnSJJ0DGDp0KLKysjB37lxkZGTAx8cH+/fvV0yOeePGDaUnc7Nnz4ZEIsHs2bNx+/ZtODs7Y8CAAViwYIHO5yYiIiKqCsyfiIiIqCbSuehkYWGB6dOnq7R/8MEHFQ5iypQpGoeDHzlyROlzrVq1EBkZicjISK2P//wxiIiIiKoS8yciIiKqiczK76Jq8+bN6NKlCxo2bIjr168DAGJiYrB79269BkdERERUXTB/IiIioppG56LTqlWrEBERgb59+yI7O1sx+aWjoyNiYmL0HR8RERGR6DF/IiIioppI56LTihUr8O233+LTTz9VWsmkffv2KqueEBERERHzJyIiIqqZdC46paWloW3btirtVlZWePz4sV6CIiIiIqpOmD8RERFRTaRz0cnNzQ3nz59Xad+/fz+8vb31ERMRERFRtcL8iYiIiGoirVev++yzzzB9+nRERERg8uTJePLkCQRBwJkzZ/DDDz9g0aJF+Pe//23IWImIiIhEhfkTERER1WRaF52ioqIwceJEjB8/HlKpFLNnz4ZMJsOIESPQsGFDfPnllxg2bJghYzVpKSkpyMvL06pvYmKi0n+1YWdnh+bNm1coNiIiIjIO5k9ERERUk2lddBIEQfH/I0eOxMiRIyGTyZCfnw8XFxeDBCcWKSkpaNGihc77hYSE6NT/ypUrLDwRERGJCPMnIiIiqsm0LjoBgEQiUfpsY2MDGxsbvQYkRmUjnGJjY7Wal6GgoADp6elwdXWFVCott39iYiJCQkK0HklFREREpoP5ExEREdVUOhWdWrRooZI4Pe/hw4eVCkjMvL290a5dO636+vv7GzgaIiIiMgXMn4iIiKim0qnoFBUVBQcHB0PFQkRERFTtMH8iIiKimkqnotOwYcM4/wARERGRDpg/ERERUU1lpm3H8oaFExEREZEy5k9ERERUk2lddHp29RUiIiIiKh/zJyIiIqrJtH69rrS01JBxEBEREVU7zJ+IiIioJtN6pBMREREREREREZG2WHQiIiIiIiIiIiK9Y9GJiIiIiIiIiIj0ziSKTt988w1cXV1hbW0NPz8/nDlz5oX9Y2Ji4OnpCalUiiZNmuCDDz7AkydPFNsXLVqE1157DXZ2dnBxcUFwcDCSk5MNeg31bSWQZl8B7pzX+5c0+wrq23L1GyIiIvqf6pA/ERERUfWm9UTihrJt2zZERERg9erV8PPzQ0xMDAIDA5GcnAwXFxeV/lu2bMGMGTOwfv16dO7cGVeuXEFYWBgkEgmio6MBAEePHsXkyZPx2muvobi4GLNmzUJAQAD++ecf1K5d2yDXEe5rCe9j4cAx/R/b+7/HJyIiIgKqT/5ERERE1ZvRi07R0dGYMGECxowZAwBYvXo19u7di/Xr12PGjBkq/U+ePAl/f3+MGDECAODq6orhw4fj9OnTij779+9X2mfjxo1wcXFBfHw8unXrpnLMwsJCFBYWKj7n5uYCAORyOeRyebnXUFxcjDXxRXj703Xw9PTU4qp1k5ycjDXLRiGouFireAytLAZTiKUixB4/IP5rYPzGxfiNi/Ebl1jjfp4p5E9A5XOomqa6/PwwfuNg/MbF+I1P7NdQXeLXlVGLTkVFRYiPj8fMmTMVbWZmZujduzdOnTqldp/OnTsjNjYWZ86cQYcOHXDt2jXs27cPo0aN0nienJwcAMBLL72kdvuiRYsQFRWl0n7w4EHY2NiUex2pqanIyBdw8FImUmW25fbXVWpqJjLyBRw/fhx3797V+/Er6tChQ8YOoVLEHj8g/mtg/MbF+I2L8RuHTCYzdgiVZir5E1D5HKqmEuvPTxnGb1yM37gYv/GJ/RrEGn9FcyijFp3u37+PkpIS1KtXT6m9Xr16SEpKUrvPiBEjcP/+fXTp0gWCIKC4uBgTJ07ErFmz1PYvLS3F+++/D39/f7Rq1Uptn5kzZyIiIkLxOTc3F02aNEFAQADs7e3LvY6//voLANClSxe0bdu23P66MvTxdSWXy3Ho0CG88cYbsLCwMHY4OhN7/ID4r4HxGxfjNy7Gb1wPHjwwdgiVZir5E1D5HKqmEfvPD+M3LsZvXIzf+MR+DWKPv6I5lNFfr9PVkSNHsHDhQqxcuRJ+fn64evUqpk2bhvnz52POnDkq/SdPnozLly/j+PHjGo9pZWUFKysrlXYLCwutvhlq1aql+K8hvnkMffyK0vb+mCqxxw+I/xoYv3ExfuNi/MYhxpj1wRD5E1D5HKqmEvv9YfzGxfiNi/Ebn9ivQazxVzRmoxadnJycYG5ujszMTKX2zMxM1K9fX+0+c+bMwahRozB+/HgAQOvWrfH48WO88847+PTTT2Fm9r8F+aZMmYJff/0Vx44dQ+PGjQ13IURERERVhPkTERERiYVZ+V0Mx9LSEr6+voiLi1O0lZaWIi4uDp06dVK7j0wmU0qMAMDc3BwAIAiC4r9TpkzBzp078fvvv8PNzc1AV0BERERUtZg/ERERkVgY/fW6iIgIhIaGon379ujQoQNiYmLw+PFjxWoso0ePRqNGjbBo0SIAwIABAxAdHY22bdsqhofPmTMHAwYMUCRPkydPxpYtW7B7927Y2dkhIyMDAODg4ACpVGqcCyUiIiLSE+ZPREREJAZGLzoNHToUWVlZmDt3LjIyMuDj44P9+/crJse8ceOG0pO52bNnQyKRYPbs2bh9+zacnZ0xYMAALFiwQNFn1apVAIAePXoonWvDhg0ICwsz+DURERERGRLzJyIiIhIDoxedgKdzB0yZMkXttiNHjih9rlWrFiIjIxEZGanxeGXDxImIiIiqK+ZPREREZOqMOqcTERERERERERFVTyw6ERERERERERGR3pnE63ViJ5PJAAAJCQla9S8oKEB6ejpcXV21mpgzMTGxUvEREREREREREVU1Fp30ICkpCQAwYcIEg57Hzs7OoMcnIiIiIiIiItIXFp30IDg4GADg5eUFGxubcvsnJiYiJCQEsbGx8Pb21uocdnZ2aN68eWXCJCIiIiIiIiKqMiw66YGTkxPGjx+v837e3t5o166dASIiIiIiIiIiIjIuTiRORERERERERER6x6ITERERERERERHpHYtORERERERERESkdyw6ERERERERERGR3rHoREREREREREREeseiExERERERERER6R2LTkREREREREREpHcsOhERERERERERkd6x6ERERERERERERHpXy9gBmCJBEAAAubm5lTqOTCbDlStXVNqTk5MBAPHx8cjPz1fZ3qJFC9jY2FTq3IYkl8shk8mQm5sLCwsLY4ejM7HHD4j/Ghi/cTF+42L8xpWXlwfgf//Wk37pK4eqrsT+88P4jYvxGxfjNz6xX4PY469oDiURmHWpuHXrFpo0aWLsMIiIiMhAUlNT4e7ubuwwqh3mUERERNWbrjkUi05qlJaW4s6dO7Czs4NEIjF2OCYnNzcXTZo0wc2bN2Fvb2/scHQm9vgB8V8D4zcuxm9cjN+4cnJy8PLLL+PRo0dwdHQ0djjVDnOoFxP7zw/jNy7Gb1yM3/jEfg1ij7+iORRfr1PDzMwMjRs3NnYYJs/e3l6UPyxlxB4/IP5rYPzGxfiNi/Ebl5kZp7U0BOZQ2hH7zw/jNy7Gb1yM3/jEfg1ij1/XHIoZFxERERERERER6R2LTkREREREREREpHcsOpHOrKysEBkZCSsrK2OHUiFijx8Q/zUwfuNi/MbF+I1L7PGTuIn9+4/xGxfjNy7Gb3xiv4aaGj8nEiciIiIiIiIiIr3jSCciIiIiIiIiItI7Fp2IiIiIiIiIiEjvWHQiIiIiIiIiIiK9Y9GJiIiIiIiIiIj0jkUnIiIiIiIiIiLSOxadiIgqSCKRYNeuXcYOg4iIiEg0mD8R1SwsOhGRKIWFhUEikah89enTx9ihEREREZkk5k9EVNVqGTsAIqKK6tOnDzZs2KDUZmVlZaRoiIiIiEwf8yciqkoc6UREomVlZYX69esrfdWpUwfA06Hbq1atQt++fSGVSuHu7o4dO3Yo7X/p0iX06tULUqkUdevWxTvvvIP8/HylPuvXr0fLli1hZWWFBg0aYMqUKUrb79+/jzfffBM2NjZo3rw59uzZY9iLJiIiIqoE5k9EVJVYdCKiamvOnDl46623cOHCBYwcORLDhg1DYmIiAODx48cIDAxEnTp1cPbsWWzfvh2//fabUlK0atUqTJ48Ge+88w4uXbqEPXv2oFmzZkrniIqKwpAhQ3Dx4kUEBQVh5MiRePjwYZVeJxEREZG+MH8iIr0SiIhEKDQ0VDA3Nxdq166t9LVgwQJBEAQBgDBx4kSlffz8/IR3331XEARBWLt2rVCnTh0hPz9fsX3v3r2CmZmZkJGRIQiCIDRs2FD49NNPNcYAQJg9e7bic35+vgBA+M9//qO36yQiIiLSF+ZPRFTVOKcTEYlWz549sWrVKqW2l156SfH/nTp1UtrWqVMnnD9/HgCQmJiINm3aoHbt2ort/v7+KC0tRXJyMiQSCe7cuYPXX3/9hTG8+uqriv+vXbs27O3tce/evYpeEhEREZFBMX8ioqrEohMRiVbt2rVVhmvri1Qq1aqfhYWF0meJRILS0lJDhERERERUacyfiKgqcU4nIqq2/vzzT5XP3t7eAABvb29cuHABjx8/Vmw/ceIEzMzM4OnpCTs7O7i6uiIuLq5KYyYiIiIyJuZPRKRPHOlERKJVWFiIjIwMpbZatWrByckJALB9+3a0b98eXbp0wffff48zZ85g3bp1AICRI0ciMjISoaGhmDdvHrKysvDee+9h1KhRqFevHgBg3rx5mDhxIlxcXNC3b1/k5eXhxIkTeO+996r2QomIiIj0hPkTEVUlFp2ISLT279+PBg0aKLV5enoiKSkJwNOVUbZu3YpJkyahQYMG+OGHH/DKK68AAGxsbHDgwAFMmzYNr732GmxsbPDWW28hOjpacazQ0FA8efIEy5cvx/Tp0+Hk5IS333676i6QiIiISM+YPxFRVZIIgiAYOwgiIn2TSCTYuXMngoODjR0KERERkSgwfyIifeOcTkREREREREREpHcsOhERERERERERkd7x9ToiIiIiIiIiItI7jnQiIiIiIiIiIiK9Y9GJiIiIiIiIiIj0jkUnIiIiIiIiIiLSOxadiIiIiIiIiIhI71h0IiIiIiIiIiIivWPRiYiIiIiIiIiI9I5FJyIiIiIiIiIi0jsWnYiIiIiIiIiISO9YdCIiIiIiIiIiIr1j0YmIiIiIiIiIiPSORSciIiIiIiIiItI7Fp2IiIiIiIiIiEjvWHQiIiIiIiIiIiK9Y9GJiIiIiIiIiIj0jkUnIqrxNm7cCIlEgvT0dGOHYnA9evRAjx49jB0GERERVUNHjhyBRCLBkSNHjB1KhUkkEsybN8/YYRBVGyw6EYlMamoqwsPD4e7uDmtra9jb28Pf3x9ffvklCgoKFP1cXV0hkUjw3nvvqRyjLCHYsWOHoq2s8GJtbY3bt2+r7NOjRw+0atWq3PjCwsIgkUgUX/b29mjTpg2WLVuGwsLCCl61+MybN0/pPlhYWMDV1RVTp05Fdna2scMjIiKiZ5h6flXm119/RZ8+fVC3bl1YW1ujRYsW+Oijj/Dw4UMdr9j4evTooZQrSaVSvPrqq4iJiUFpaamxwyMiPall7ACISHt79+7F4MGDYWVlhdGjR6NVq1YoKirC8ePH8dFHH+Hvv//G2rVrlfb59ttvMXPmTDRs2FCrcxQWFmLx4sVYsWJFheO0srLCv//9bwBAdnY2fvrpJ0yfPh1nz57F1q1bK3xcMVq1ahVsbW3x+PFjxMXFYcWKFUhISMDx48eNHRoRERFBPPnV9OnTsWzZMrRp0waffPIJXnrpJSQkJGDFihXYtm0b4uLi0Lx58wofXx+6deuGgoICWFpaatW/cePGWLRoEQDg/v372LJlCz744ANkZWVhwYIFhgyViKoIi05EIpGWloZhw4ahadOm+P3339GgQQPFtsmTJ+Pq1avYu3ev0j4tW7ZEcnIyFi9ejK+++kqr8/j4+OicSD2vVq1aCAkJUXyeNGkS/Pz8sG3bNkRHR1f4uGL09ttvw8nJCQAQHh6OYcOGYdu2bThz5gw6dOhg5OiIiIhqNrHkVz/88AOWLVuGoUOH4vvvv4e5ubliW1hYGHr27InBgwfj3LlzqFXLeL/imZmZwdraWuv+Dg4OSjnjxIkT4eXlhRUrVuCzzz5Tuk4iEie+XkckEl988QXy8/Oxbt06pYSoTLNmzTBt2jSlNldXV4wePRrffvst7ty5o9V5Zs2ahZKSEixevFgvcQNPE5CyeYTK5k06d+4cAgMD4eTkBKlUCjc3N4wdO1Zpv9LSUsTExKBly5awtrZGvXr1EB4ejkePHin10/TuvaurK8LCwpTa/v77b/Tq1QtSqRSNGzfG559/rnEI98qVK9GyZUtYWVmhYcOGmDx5cqVfjevatSuAp8P4yzx8+BDTp09H69atYWtrC3t7e/Tt2xcXLlxQ2rds2P6PP/6IBQsWoHHjxrC2tsbrr7+Oq1evqpxr7dq18PDwgFQqRYcOHfDHH3+ojenevXsYN24c6tWrB2tra7Rp0wabNm1S6pOeng6JRIJ//etf+Oabb+Du7g4bGxsEBATg5s2bEAQB8+fPR+PGjSGVSjFo0CBRDvUnIqKaRSz5VVRUFOrUqYO1a9eqFGI6dOiATz75BBcuXMDPP/+sFOfzeRCgOr9jUVER5s6dC19fXzg4OKB27dro2rUrDh8+rLLv1q1b4evrCzs7O9jb26N169b48ssvFdsrO6eTtbU1XnvtNeTl5eHevXuK9osXLyIsLEzx+mP9+vUxduxYPHjwQGn/sukNrl69irCwMDg6OsLBwQFjxoyBTCZT6ltYWIgPPvgAzs7OsLOzw8CBA3Hr1i21cf3111/o27cv7O3tYWtri9dffx1//vmnUp+yVymPHz+OqVOnwtnZGY6OjggPD0dRURGys7MxevRo1KlTB3Xq1MHHH38MQRAqdJ+IxIRFJyKR+OWXX+Du7o7OnTvrtN+nn36K4uJirZMcNzc3nRMpbZQVWerWrYt79+4hICAA6enpmDFjBlasWIGRI0eq/OMdHh6Ojz76SDGnwpgxY/D9998jMDAQcrlc5xgyMjLQs2dPnD9/HjNmzMD777+P7777TilZKjNv3jxMnjwZDRs2xLJly/DWW29hzZo1CAgIqNC5y5QV3erUqaNou3btGnbt2oX+/fsjOjoaH330ES5duoTu3bur/TNYvHgxdu7cienTp2PmzJn4888/MXLkSKU+69atQ3h4OOrXr48vvvgC/v7+GDhwIG7evKnUr6CgAD169MDmzZsxcuRILF26FA4ODggLC1N7X77//nusXLkS7733Hj788EMcPXoUQ4YMwezZs7F//3588skneOedd/DLL79g+vTpFb5PREREVUEM+VVKSgqSk5MxaNAg2Nvbq+0zevRoAE+vR1e5ubn497//jR49emDJkiWYN28esrKyEBgYiPPnzyv6HTp0CMOHD0edOnWwZMkSLF68GD169MCJEyd0PueLlD3ocnR0VDr3tWvXMGbMGKxYsQLDhg3D1q1bERQUpLZwM2TIEOTl5WHRokUYMmQINm7ciKioKKU+48ePR0xMDAICArB48WJYWFigX79+Ksf6+++/0bVrV1y4cAEff/wx5syZg7S0NPTo0QOnT59W6f/ee+8hJSUFUVFRGDhwINauXYs5c+ZgwIABKCkpwcKFC9GlSxcsXboUmzdvrvwNIzJ1AhGZvJycHAGAMGjQIK33adq0qdCvXz9BEARhzJgxgrW1tXDnzh1BEATh8OHDAgBh+/btiv4bNmwQAAhnz54VUlNThVq1aglTp05VbO/evbvQsmXLcs8bGhoq1K5dW8jKyhKysrKEq1evCgsXLhQkEonw6quvCoIgCDt37lScS5M//vhDACB8//33Su379+9XaQcgREZGqr0HoaGhis/vv/++AEA4ffq0ou3evXuCg4ODAEBIS0tTtFlaWgoBAQFCSUmJou/XX38tABDWr19f7n2IjIwUAAjJyclCVlaWkJ6eLqxfv16QSqWCs7Oz8PjxY0XfJ0+eKJ1HEAQhLS1NsLKyEj777DNFW9mfm7e3t1BYWKho//LLLwUAwqVLlwRBEISioiLBxcVF8PHxUeq3du1aAYDQvXt3RVtMTIwAQIiNjVW0FRUVCZ06dRJsbW2F3NxcRTwABGdnZyE7O1vRd+bMmQIAoU2bNoJcLle0Dx8+XLC0tBSePHlS7r0iIiIyBrHkV7t27RIACMuXL39hP3t7e6Fdu3ZKsT6bBz17zmdzgeLiYqV8QRAE4dGjR0K9evWEsWPHKtqmTZsm2NvbC8XFxRpjKLsHhw8ffmGsZXF4eXkpcsakpCTho48+EgAo7nEZmUymsv8PP/wgABCOHTumaCvLv56NWxAE4c033xTq1q2r+Hz+/HkBgDBp0iSlfiNGjFDJK4ODgwVLS0shNTVV0Xbnzh3Bzs5O6Natm6Kt7M86MDBQKC0tVbR36tRJkEgkwsSJExVtxcXFQuPGjZX+HIiqK450IhKB3NxcAICdnV2F9p89e7ZOT+Pc3d0xatQorF27Fnfv3tX5fI8fP4azszOcnZ3RrFkzzJo1C506dcLOnTsBQPHk6tdff9U4amj79u1wcHDAG2+8gfv37yu+fH19YWtrq3bId3n27duHjh07Ks2l5OzsrDJK6LfffkNRURHef/99mJn976/JCRMmwN7eXmVuhxfx9PSEs7MzXF1dMXbsWDRr1gz/+c9/YGNjo+hjZWWlOE9JSQkePHgAW1tbeHp6IiEhQeWYY8aMUZqgs+yVvWvXrgF4+urivXv3MHHiRKV+YWFhcHBwULkn9evXx/DhwxVtFhYWmDp1KvLz83H06FGl/oMHD1Y6hp+fHwAgJCREaQ4JPz8/FBUVqV2ph4iIyBSIJb/Ky8vTKk47OztFX12Ym5sr8oXS0lI8fPgQxcXFaN++vVIe4ujoiMePH+PQoUM6n0OTpKQkRc7o5eWFpUuXYuDAgdi4caNSP6lUqvj/J0+e4P79++jYsSMAqM2VJk6cqPS5a9euePDggeLPfN++fQCAqVOnKvV7//33lT6XlJTg4MGDCA4Ohru7u6K9QYMGGDFiBI4fP644Zplx48ZBIpEoPvv5+UEQBIwbN07RZm5ujvbt2ytyN6LqjEUnIhEoG0pdkUQCqFiSo2si9Sxra2scOnQIhw4dwrFjx3Dz5k2cOHFC8Y919+7d8dZbbyEqKgpOTk4YNGgQNmzYgMLCQsUxUlJSkJOTAxcXF0UyUvaVn5+v9J6/tq5fv652VRdPT0+VfuraLS0t4e7urtheVFSEjIwMpa+SkhKlfX766SccOnQIW7ZsQceOHXHv3j2lxAl4muAtX74czZs3h5WVFZycnODs7IyLFy8iJydHJd6XX35Z6XPZq3plc12Vxff8tVpYWCglTM/ek2eLawDg7e2tdCxN5y4rQDVp0kRt+/PzbxEREZkKseRXZcWm8uLMy8uDi4uL1sd91qZNm/Dqq6/C2toadevWhbOzM/bu3auUh0yaNAktWrRA37590bhxY4wdOxb79+9/4XHz8/OV8qSsrCyl7a6urjh06BAOHDiAlStXolGjRsjKylKZjPzhw4eYNm0a6tWrB6lUCmdnZ7i5uQFAhXMlMzMzeHh4KPV7PvfLysqCTCZTaQee5kqlpaUqUxfokisxT6KagEUnIhGwt7dHw4YNcfny5Qofo2zugSVLlmjV393dHSEhIRUa7WRubo7evXujd+/e6Nq1Kxo3bqy0XSKRYMeOHTh16hSmTJmC27dvY+zYsfD19UV+fj6Ap4UYFxcXRfHq+a/PPvus3DieLwDp28mTJ9GgQQOlr+cTj27duqF3794YPnw4Dh06BKlUipEjRypNXr5w4UJERESgW7duiI2NxYEDB3Do0CG0bNlS7STnmlZyEapgMkpN5zZmTERERBUhlvzqlVdeAfB0Mm1Nrl+/jtzcXKWHS8+OtnnW8/lRbGwswsLC4OHhgXXr1mH//v04dOgQevXqpZSHuLi44Pz589izZw8GDhyIw4cPo2/fvggNDdUY17/+9S+lPOm1115T2l67dm307t0bAQEBePfdd7Fv3z6cOXMGs2bNUuo3ZMgQfPvtt5g4cSJ+/vlnHDx4UFHwEnOuxDyJagIWnYhEon///khNTcWpU6cqtL+HhwdCQkKwZs0anZ/GaZtI6apjx45YsGABzp07h++//x5///03tm7dqoj3wYMH8Pf3VxSwnv1q06aN4jh16tRRWVWuqKhI5TqbNm2KlJQUlTiSk5NV+qlrLyoqQlpammJ7mzZtVIph9evX13i9tra2iIyMxPnz5/Hjjz8q2nfs2IGePXti3bp1GDZsGAICAtC7d+8Kr5RXFt/z1yqXy5GWlqbSNyUlRSVhS0pKUjoWERFRdSSG/Kp58+bw9PTErl27NI52+u677wA8fQ2+jLr8CFAdxbxjxw64u7vj559/xqhRoxAYGIjevXvjyZMnKvtaWlpiwIABWLlyJVJTUxEeHo7vvvtO7Sq6wNMJzp/Nk77//vsXXuurr76quJ83btwA8HR0UlxcHGbMmIGoqCi8+eabeOONN1RGb+uiadOmKC0tVVpNGFDN/ZydnWFjY6PSDjzNlczMzFRGMBGRMhadiETi448/Ru3atTF+/HhkZmaqbE9NTVW72tizZs+eDblcji+++EKrcz6bSGVkZFQobnUePXqk8mTHx8cHABSv2A0ZMgQlJSWYP3++yv7FxcVKSZSHhweOHTum1Gft2rUqT/KCgoLw559/4syZM4q2rKwslQSod+/esLS0xFdffaUU57p165CTk6NY2aROnToqxbDnh4M/b+TIkWjcuLFSomlubq5yP7Zv317h+ZDat28PZ2dnrF69GkVFRYr2jRs3qiSfQUFByMjIwLZt2xRtxcXFWLFiBWxtbdG9e/cKxUBERCQGYsmvIiMj8ejRI0ycOFElv4mPj8eSJUvQtm1b9O3bV+k8f/75p1Iu8Ouvv6qMyi4bgfNsLnL69GmVQtyDBw+UPpuZmeHVV18FAKUpEp7l7u6ulCf5+/uXe60ff/wx5HI5oqOjNcYHADExMeUeS5Oy+/TVV1+98Jjm5uYICAjA7t27FSsQA0BmZia2bNmCLl26aFxRkIieqlV+FyIyBR4eHtiyZQuGDh0Kb29vjB49Gq1atUJRURFOnjyJ7du3IywsrNxjhISEYNOmTVqf99NPP8XmzZuRnJyMli1bVvIqntq0aRNWrlyJN998Ex4eHsjLy8O3334Le3t7BAUFAXg671N4eDgWLVqE8+fPIyAgABYWFkhJScH27dvx5Zdf4u233wbwdMnbiRMn4q233sIbb7yBCxcu4MCBA3ByclI678cff4zNmzejT58+mDZtGmrXro21a9eiadOmSkPWnZ2dMXPmTERFRaFPnz4YOHAgkpOTsXLlSrz22msICQmp8LVbWFhg2rRp+Oijj7B//3706dMH/fv3x2effYYxY8agc+fOuHTpEr7//vsKP8GzsLDA559/jvDwcPTq1QtDhw5FWloaNmzYoHLMd955B2vWrEFYWBji4+Ph6uqKHTt24MSJE4iJianw5KpERERiIJb8avjw4Th37hyio6Pxzz//YOTIkahTpw4SEhKwfv16ODs7Y8eOHUqLeowfPx47duxAnz59MGTIEKSmpiI2NlZlHqP+/fvj559/xptvvol+/fohLS0Nq1evxiuvvKKY9qDseA8fPkSvXr3QuHFjXL9+HStWrICPj49iLkh9eOWVVxAUFIR///vfmDNnDurWrYtu3brhiy++gFwuR6NGjXDw4EGV0du68PHxwfDhw7Fy5Urk5OSgc+fOiIuLUzti6/PPP8ehQ4fQpUsXTJo0CbVq1cKaNWtQWFiodaGRqEYz1rJ5RFQxV65cESZMmCC4uroKlpaWgp2dneDv7y+sWLFCaXn6Z5f0fVZKSopgbm7+wiV9nxcaGioAKHdJ37K+tWvXfmGfhIQEYfjw4cLLL78sWFlZCS4uLkL//v2Fc+fOqfRdu3at4OvrK0ilUsHOzk5o3bq18PHHHyuWJxYEQSgpKRE++eQTwcnJSbCxsRECAwOFq1evql0q+OLFi0L37t0Fa2troVGjRsL8+fOFdevWCQCEtLQ0pb5ff/214OXlJVhYWAj16tUT3n33XeHRo0fl3gNB+N+SvVlZWSrbcnJyBAcHB8UyuU+ePBE+/PBDoUGDBoJUKhX8/f2FU6dOqSxprG4pZkEQhLS0NAGAsGHDBqX2lStXCm5uboKVlZXQvn174dixYyrHFARByMzMFMaMGSM4OTkJlpaWQuvWrVWOVXaOpUuXKrVriulF309ERESmxtTzqzJ79uwRevfuLTg6OgoAFPvn5OSo7b9s2TKhUaNGgpWVleDv7y+cO3dOJRcoLS0VFi5cKDRt2lSwsrIS2rZtK/z6669CaGio0LRpU0W/HTt2CAEBAYKLi4tgaWkpvPzyy0J4eLhw9+5dRZ+yvODw4cPlXkv37t01XvuRI0cEAEJkZKQgCIJw69Yt4c033xQcHR0FBwcHYfDgwcKdO3eU+giC5vyr7M/h2VyvoKBAmDp1qlC3bl2hdu3awoABA4SbN2+qHFMQnuaugYGBgq2trWBjYyP07NlTOHnypNpzPP9nrSkmbXJmoupAIgicvYyIiIiIiEhsxo8fj3Xr1uHbb7/F+PHjjR0OEZEKFp2IiIiIiIhEqKSkBMHBwdi/fz92796tmKaAiMhUsOhERERERERERER6x9XriIiIiIiIiIhI71h0IiIiIiIiIiIivWPRiYiIiIiIiIiI9K6WsQMwRaWlpbhz5w7s7OwgkUiMHQ4RERHpiSAIyMvLQ8OGDWFmxmdv+sYcioiIqHqqaA7FopMad+7cQZMmTYwdBhERERnIzZs30bhxY2OHUe0whyIiIqredM2hWHRSw87ODsDTm2lvb2/kaEyPXC7HwYMHERAQAAsLC2OHozOxxw+I/xoYv3ExfuNi/Mb18OFDuLm5Kf6tJ/1iDvViYv/5YfzGxfiNyxTjT01NRV5enlJbcnIy3nnnHZ2PtXbtWnh6eqq029nZwcPDo8IxVoRMJsOVK1dU2vPz87F3717069cPtra2KttbtGgBGxubqggRAPDgwQP8tvsH2JQo/xnICh4jLS1dpb8gCMi6dw/OLi5qRwO7ubnCRlpbqa2uqxc6v/GmXuOuqIrmUCw6qVH2DWBvb8+ESQ25XA4bGxvY29ubzF+4uhB7/ID4r4HxGxfjNy7Gb1xyuRwA+OqXgTCHerHq8PPD+I2H8RuXqcWfkpKCoG6+aGCr+u9Z2/q6vz6+au5Ete138wUcS0hG8+bNdT6mNlJSUlQKZ4mJiQgJCdG4z8qVK9W2x8bGwtvbW6nNzs7OYLHb29tjwvuztO4vl8uxb98+BAUFmcT3kK4qmkOx6EREREREREQkIpmZmQj3tcS8HlYGPc+8I4UqRSF9SUlJQYsWLfR2PE2FqitXrhis8ETlY9GJiIiIiIiISESSkpKwJr4Ie5LlBj3P3XwBIw30SnpeXh7q20qwLmYB3Nzcyu1fUCDD4cNH0LNnD0il5b9Gl5aWhnHvf2qwohlph0UnIiIiIiIiIhEJDg4GAHh5eSnNY1Teq2maqHs1DTDs62kymQzhvpYIurkYuKndPu1sAZzdrVVfbwDhvpYVjo/0g0UnIiIiIiIiIhFxcnLC+PHjVdq9vLwQHx+v0p6Xl4fdu3dj0KBBaieCfr54VRWqYrSWIUdqkXZYdCIiIiIiIiKqBmxsbNCuXTuVdrlcjuzsbHTu3NlkJrHWNFqroKAA6enpKv1TU1MRGRmJqKgotSvqubq6QiqVKrUZcqQWaYdFJyIiIiIiIiKqUppGayUkJLzwFcHIyEi17fHx8WoLbmRcLDoRERERERERkUmozCuCZHpYdCIiIiIiIiK9kMlkSEpKUmnPy8vD0aNH4ejoaDJzCpFpEtMrglQ+Fp2IiIiIiIhIZykpKSrL0Ze3etry5cvVtqtbPY3z8RCJH4tOREREREREpJOUlBR0a+eJBrYSlW1t65vpfLxl00ertN3NF3AsIZmFJyIRY9GJiIiIiIiIdJKZmYlwX0vM62FlsHPMO1KoMpKKiMSFRSciIiIiIiLSSVJSEtbEF2FPstxg57ibL2Ckmvmf9OH+/fs48NN3sCnJVWqXyR4jNfWaSn9BEJCZmYnz/9kEiUR1dJeHhztsbGortTm5tUTXvoP1GziRyLDoRERERERERDoJDg4GoDoBeEFBAdLT01X6p6amIjIyElFRUfDw8FDZ7urqCqlUqtRmyDmddu3ahVs/zFI/Uquehp3qA8Bl9dvy//v1jHk/FsLZrTVXVaMajUUnIiIiIiIi0omTkxPGjx+v0p6QkPDCicQjIyPVtsfHx6tdscxQgoODcaAkFzt1HOlUr149rUc6vf5JSxacqMZj0YmIiIiIiIj0wsvLC/Hx8SrteXl52L17NwYNGgQ7Na/MVXVxxsnJCSPDI7TuL5fLsW/fPgQFBcHCwsKAkRFVLyw6ERERERERkV7Y2NioHbEkl8uRnZ2Nzp07s2hDVIOw6ERERERERGQiZDIZkpKSVNrz8vJw9OhRODo6ahwp9OzcSkREpoBFJyIiIiIiIhORlJQEX19fjduXL1+utr2q50QiItIGi05ERERERERGkJKSgry8PKW2goICxMbGqvQtb/W3goICJCQkKLUZcvU3IiJtsOhERERERERUxVJSUtCtnSca2KquhKZJ2/pm2LUqSuv+d/MFHEtIZuGJiIyGRSciIiIiIqIqlpmZiXBfS8zrYWWwc8w7UqgykoqIqCqx6ERERERERFTFkpKSsCa+CHuS5QY7x918ASPVTDpORFRVWHQiIiIiIiKqYsHBwQBUV50rKChAenq6Sv/y5nRydXWFVCpVauOcTkRkbCw6ERERkQou2U1EZFhOTk4YP368SntCQgJCQkI07hcZGam2navXEZEpYtGJiIiohlO3elJiYuILf+nRtGR3bGwsvL29ldr4pJ2IDOXaxVMovH9dqa2wsBB37txR6VtaWoqryck4cO8yzMzMVLY3bNgQVlbK8ytZOTWF+6ud9Bt0Oby8vBAfH6/SnpeXh927d2PQoEEai/5ERKaGRSciIqIa7EWrJ7Wtr/pLWXmWTR+t0sbVk4jIEC5cuICd03qqnYjbR8M+/W0B3Naw8aZq07wjhRj57aUq/fvLxsZG7YgluVyO7OxsdO7cGRYWFlUWDxFRZbDoREREZABieT2NqycRkVidPXu2aibiNtjRiYiqPxadiIiIKknMr6dx9SQiEitdJ+IuKSnB+fPn4ePjA3Nzc5XtnIibiEj/WHQiIiKqhJSUFLRo0UJvx9NUqLpy5YpBfvHhL21EJFaaJuIGAH9/f5U2uVwOBwcHBAUF8fU0IqIqwqITERFRJeTl5aG+rQTrYhbAzc2t3P4FBTIcPnwEPXv2gFRa/mt0aWlpGPf+pwZ7PY2/tBERERGRobDoREREVAkymQzhvpYIurlY7SS06rSzBXB2t1Z9vQGE+1pWOD4iIiIiImMRRdHpm2++wdKlS5GRkYE2bdpgxYoV6NChg9q+GzduxJgxY5TarKys8OTJk6oIlYiIahjOiUSmjDkUERERGZNBik7Xrl2Du7u7Xo61bds2REREYPXq1fDz80NMTAwCAwORnJwMFxcXtfvY29sjOTlZ8VkiUV0GmoiITJtYVn/TdU6k1NRUREZGIioqCh4eHirbOSdSzaXP/AlgDkUVJ5a/f4mIyPQZpOjUrFkzdO/eHePGjcPbb78Na2vrCh8rOjoaEyZMUDx5W716Nfbu3Yv169djxowZaveRSCSoX79+hc9JRETGl5SUBF9fX43bNa3+Fh8fj3bt2hkqLBWa5kRKSEh44ep1kZGRaturOn4yHfrMnwDmUFRxYvn7l4iITJ9Bik4JCQnYsGEDIiIiMGXKFAwdOhTjxo3TOJxbk6KiIsTHx2PmzJmKNjMzM/Tu3RunTp3SuF9+fj6aNm2K0tJStGvXDgsXLkTLli019i8sLERhYaHic25uLoCnk6XK5YZ7XUKsyu6JWO+N2OMHxH8NjN+4xBK/h4cHTp8+rdJ++fJljBs3DuvWrUOrVq3U7mcK16Yp/ry8POzduxf9+vVTO1LAVOLXRCzfP5qYctz6yp8A5lCmyhR/flJSUpCfn6/U9uTJE2zatEmlb2pqKj777DPMnTtX7UjN/Px8nDlzRqnN1tbWZEZqmuL91wXjNy7Gb3xiv4bqEr+uJIIgCHqORaG4uBh79uzBxo0bsX//frRo0QJjx47FqFGj4OzsXO7+d+7cQaNGjXDy5El06tRJ0f7xxx/j6NGjapP5U6dOISUlBa+++ipycnLwr3/9C8eOHcPff/+Nxo0bqz3PvHnzEBUVpdK+ZcsWDhEmIjIxqamp+PDDD7Fs2TK1v/QQvYhMJsOIESOQk5MDe3t7Y4ejVmXzJ4A5FGnnzp07mDRpksHPs3LlSjRs2NDg5yEiIsOpaA5l0KJTmcLCQqxcuRIzZ85EUVERLC0tMWTIECxZsgQNGjTQuF9FEqbnyeVyeHt7Y/jw4Zg/f77G+J5/StekSRPcv3/fZBNSY5LL5Th06BDeeOMNUS6XLfb4AfFfA+M3LlOMX92Tdk3KG+mkjqk9aTe1+68Lscf/4MEDNGjQwKSLTmUqmj8BzKFMlan9/Pz1118Y9HpHrF32Gdzc3MrtL5M9xh9/HEfXrl1gY1O73P5paWl458O52B33J9q2bauPkCvF1O6/rhi/cTF+4xP7NYg9/ormUAZdve7cuXNYv349tm7ditq1a2P69OkYN24cbt26haioKAwaNEhlCO6znJycYG5ujszMTKX2zMxMrecbsLCwQNu2bXH16lWNfaysrGBlZaV2XzF+M1QVsd8fsccPiP8aGL9xmUr8KSkpL3x9R5Nx48bp1P/KlSsmU3gCTOf+V5RY4xdDzJXNnwDmUKbOVO5PrVq1EO5riQF3lgJ3tNungy2Av3Zr1bcVgHBfS9SqVcskrreMqdz/imL8xsX4jU/s1yDW+Csas0GKTtHR0diwYQOSk5MRFBSE7777DkFBQTAzMwMAuLm5YePGjXB1dX3hcSwtLeHr64u4uDjF6kClpaWIi4vDlClTtIqlpKQEly5dQlBQUGUuiYiIDCQvLw8AEBsbC29v73L75+fnY9euXQgODoatrW25/RMTExESEqI4D5Gp0lf+BDCHIu3IZDKsiS9CmyEz4OXlVW7/ggIZDh8+gp49e0AqLf/1ybS0NKyJ/xQDKx8qERGJlEGKTqtWrcLYsWMRFhamcfi3i4sL1q1bV+6xIiIiEBoaivbt26NDhw6IiYnB48ePFSuxjB49Go0aNcKiRYsAAJ999hk6duyIZs2aITs7G0uXLsX169fVrixERESmob6tBO0amMO7vlm5feXFNih9rSk6udrAolb5/aXZ5qhvy2XfyfTpM38CmENR+ZKSkpCRL+D/JqvOy/VCyzbr1F3doglERFQzGKTolJKSUm4fS0tLhIaGlttv6NChyMrKwty5c5GRkQEfHx/s378f9erVAwDcuHFD8QQQAB49eoQJEyYgIyMDderUga+vL06ePIlXXnml4hdERCRCMpkMSUlJKu15eXk4evQoHB0d1f4i4OXlVeUTAIf7WsL7WDhwrPy+FgB6AECydsf2/u/xiUydPvMngDkUla9sFNzzf+8XFBQgPT1dpX9qaioiIyMRFRWldiEHV1dXSKVSpTY7OzuTerWZiIiqlkGKThs2bICtrS0GDx6s1L59+3bIZDKtk6UyU6ZM0TgU/MiRI0qfly9fjuXLl+t0fCKi6igpKQm+vr4at2v6uzI+Ph7t2rUzVFhqrYkvwtC5G+Gtxesd8uJinDhxAv7+/rCoVf4/Y4lJSVizbARf7yCTp+/8CWAORS/m5OSkdiRbQkICQkJCNO4XGRmptt0Y/34QEZFpM0jRadGiRVizZo1Ku4uLC955550KJU1ERKQbLy8vxMfHq7RfvnwZoaGh2LRpk9rV37SZ10PfMvIFFDi2ABr6lN9ZLkeOzW2gQRtAiwkNCzJKkZFv8IVaiSqN+ROZCk3/fuTl5WH37t0YNGiQxpGyREREzzJI0enGjRtql11t2rQpbty4YYhTEhHVaCkpKXqbKFvdK3mGfD1CJpMBePpkXRv5+fk4evQo6tSpo/VE4kRiwPyJTIWNjY3aEUtyuRzZ2dno3LmzKFdeIiKiqmeQopOLiwsuXryosrrKhQsXULduXUOckoioxkpJSUGLFi103k/XURNXrlwxSOGprMg1YcIEnfbT9TUgTmRLpo75ExEREVU3Bik6DR8+HFOnToWdnR26desGADh69CimTZuGYcOGGeKUREQ1Vl5eHurbSrAuZoHaURLPq8iS1+Pe/1RvI6mep2kiW03Kez1QHU5kS2LA/Kn6ENNCDkRERIZkkKLT/PnzkZ6ejtdffx21/jvJa2lpKUaPHo2FCxca4pRERDVauK8lgm4uBm5q17+dLYCzu7Xqa+jV3zRNZKtJcXExgKe/nHHCWqpOmD9VH2JayIGIiMiQDFJ0srS0xLZt2zB//nxcuHABUqkUrVu3RtOmTQ1xOiKiGq86rv6maaRAWVtSUpLiF/NncaQAiRXzp+pDTAs5EBERGZJBik5lWrRoUaF5RoiISHsymQwZ+QJOXMtHgWNpuf3z82XYc/Y6zBq11W4i7rslRln9rbyRAprmpOJIARI75k/ip2kibo7UJCKimsZgRadbt25hz549uHHjBoqKipS2RUdHG+q0REQ1TnWdiJtLdlNNxPxJfHRZPbS8kZrqcE46IiISM4MUneLi4jBw4EC4u7sjKSkJrVq1Qnp6OgRB4FMdIhINsUwEq2ki7oKCAqSnp6v0T01NRWRkJKKiouDh4aGy3dXVFVKpVKnNGL/0cMluqmmYP4mP2FcPJSIiMjSDFJ1mzpyJ6dOnIyoqCnZ2dvjpp5/g4uKCkSNHok+fPoY4JRGR3ollIlhNE3EnJCQgJCRE436RkZFq2/l6GpFxMH8Sn7IRTrGxsfD29i63f35+Pnbt2oXg4GDtXm9OTERISIjBVg8lIiIyNIMUnRITE/HDDz88PUGtWigoKICtrS0+++wzDBo0CO+++64hTktEpFdinwiWr6cRiQvzJ/Hy9vbWqlgvl8vx6NEjdOrUiSM1iYioRjBI0al27dqKeQgaNGiA1NRUtGzZEgBw//59Q5ySiKhSdJmTozzqXsnj62lEVB7mT0RERFTdGKTo1LFjRxw/fhze3t4ICgrChx9+iEuXLuHnn39Gx44dDXFKIqIK45wcRGQKmD+JU31bCaTZV4A7ZuV3Li6GgywduHsB0GIicWn2FdS3lVQ+SCIiIiMxSNEpOjoa+fn5AICoqCjk5+dj27ZtaN68OVdeISKTwzk5iMgUMH8Sp3BfS3gfCweOld/XAkAPAEjW7tje/z0+ERGRWOm96FRSUoJbt27h1VdfBfB0qPjq1av1fRoiIr3jnBxEZCzMn8RrTXwRhs7dCG8t5sOTFxfjxIkT8Pf3h4UWI50Sk5KwZtkIDNRHoEREREag96KTubk5AgICkJiYCEdHR30fnojIIPh6BBEZE/Mn8crIF1Dg2AJo6FN+Z7kcOTa3gQZtAC0eWhRklCIjX6h8kEREREZikNfrWrVqhWvXrsHNzc0Qhyci0ju+HkFExsb8SXxkMhkAICEhQav++fn5OHr0KOrUqaP169lERERiZpCi0+eff47p06dj/vz58PX1Re3atZW229vbG+K0REQVxtcjiMjYmD+JT9lqpRMmTNBpv+XLl+vU387OTqf+REREpsIgRaegoCAAwMCBAyGR/O+VEkEQIJFIUFJSYojTEhFViEwmQ0a+gBPX8lHgWFpu//x8GfacvQ6zRm21e1J9t4SvRxBRuZg/iU9wcDAAwMvLCzY2NuX2v3z5MkJDQ7Fp0ya0atVKq3PY2dlx5VMiIhItgxSdDh8+bIjDEhEZBJ9UE5EpYP4kPk5OThj//+zdeVyU5f7/8feAgKC4lPuSmKlQejQ1ySW1cjmmJscsy1S0o2nlySI7ablWSssR6VhpddRMsywzLS2LKLcs7aBW9gNFk8wFt0TZQpb5/eGXOU0zyAzOcHMPr+fj4aPmuq/7vj/XwDgfP/d1X/fYsS73LygokHSxSOXKgysAADA7rxSdevbs6Y3DAoBXcKUaQEVA/uQ7cnJybBc0/qi4LSUlRVWc3J7t6vcQAABm4ZWi05Ytl16Jt0ePHt44LQCUCVeqAVQE5E++IyUlRR07dixxe3R0tNP2pKQkvlcAAD7FK0WnXr16ObT9cW0C1iQAKoeSrvRmZmZq8+bNqlWrltNbzirKlV6uVAMoT+RPviM8PFxJSUkO7ZmZmVq3bp0GDx5c4vcfAAC+xCtFp7Nnz9q9zs/P1+7duzV9+nTNmTPHG6cEUAGVdqW3pDWRKsqVXq5UAyhP5E++IyQkxOn3QH5+vjIyMtS1a1cFBAQYEBkAAOXLK0WnmjVrOrT16dNHgYGBiomJcXrlB4DvKelKb2lrIlWUK71cqQZQnsifAACAr/FK0akk9evX1759+8rzlADKSWpqqjIzMz1yLGe3tBmxEDdXqgFUBORPAADArLxSdPrhhx/sXlutVh0/flzPPfec2rdv741TAjBQamqqWrVq5fZ+Jd2eVpL9+/fzBDgAPov8CQAA+BqvFJ3at28vi8Uiq9Vq137jjTdqyZIl3jglAAMVz3BasWKFIiIiSu2flZWltWvXKioqStWrVy+1f3JyskaMGOGxmVQAUBGRPwEAAF/jlaLToUOH7F77+fmpbt26qlq1qjdOB6CCiIiIcGkB7fz8fJ09e1ZdunTh9jQA+D/kTwAAwNd4pejUrFkzbxwWQAXWoLpFwRn7pWN+pXcuKFDNnDTp+PdSldL/GgrO2K8G1S2l9gMAMyN/AgAAvsYrRaeHH35Y11xzjR5++GG79pdfflkHDhxQfHy8N04LwEDjOwYqYst4aUvpfQMk9ZIkF9fFjfi/4wOALyN/AgAAvsYrRacPPvhAH330kUN7165d9dxzz5E0AT7otaQLGjbjTUWEh5faN7+gQF9//bW6deumABdmOiWnpOi1ecN1uycCBYAKivwJAAD4Gq8Unc6cOaOaNWs6tNeoUUOnT5/2xikBGCgnJ0fpWVZ9/XOWcmsVldo/KytHH333i/waX+/aQuLHC5WeZS21HwCYGfnT/+Tk5CglJcWhPTc3V2lpaQoLC1NwcLDD9vDwcIWEhJRHiAAAwAVeKTpdc8012rhxoyZOnGjX/umnn+rqq6/2xikBn1RS0p2ZmanNmzerVq1aCg0Nddhe3kl3cYzjxo1za7/58+e71d/ZWAHAV1TW/Ck1NdXh6aTFTy11l7OnqIaGhqply5aXFSMAACgbrxSdYmJiNHHiRJ06dUq33HKLJCkxMVHz5s1jajjghpSUFHXs2LHE7SUVbZKSklx6ipynREVFSXK92LV3715FR0dr2bJlatOmjUvn4B8NAHxdZcyfUlNT1aNDazV08rCI6xu48GCKP5k3eZRD2/Esq7bs2sd3CAAABvBK0em+++5TXl6e5syZo2eeeUaSFBYWpoULF2rUKMdkAIBz4eHhSkpKcmgvrWgT7sK6Sp5Up04djR071uX+BQUFki7GWZ7FMQCoyCpj/nTixAmN7xioWb2CvHaOWZvyHGZSAQCA8uGVopMkPfDAA3rggQd06tQpBQcHu7RuS0leeeUVvfjii0pPT1e7du20YMECde7cudT93n33Xd1zzz0aPHiw1q5dW+bzA0YJCQlxWpQxS9GmpNsDi9tSUlJUxclC4qzJAaCy8mT+JFX8HColJUWvJV3QR/vyvXaO41lW3cvt2QAAGMIrRadDhw6poKBALVu2VN26dW3tqampCggIUFhYmMvHWrVqlWJiYrRo0SJFRkYqPj5e/fr10759+1SvXr0S90tLS9PkyZN10003Xc5QAFyG0m4PjI6Odtpe3rcHAkBF4Mn8STJHDlXS7dnFC4b/2aFDhzR9+nQ988wzat68ucN2ZwuMc3s2AADG8UrRafTo0brvvvscvuB37Nih//znP9q0aZPLx4qLi9O4ceM0ZswYSdKiRYu0YcMGLVmyRFOmTHG6T2Fhoe69917Nnj1bW7duVUZGxiXPkZeXp7y8PNvr8+fPS5Ly8/OVn++9K29mVfyemPW9MXv8kv0YKvI4WrRooR07dji0Z2ZmasOGDRowYIDTxcFbtGhRocdl9t8h4jcW8RurIsftyfxJMkcOVbNmTacXIHbv3n3JhcSnT5/utH3Hjh1q27atQ3tF+bn7yueH+I1B/MYifuOZfQy+Er+7LFar1ePPIa9Ro4Z27dqla665xq79wIED6tSpU6kJTLELFy4oJCREq1evtl0Jky7OjsjIyNC6deuc7jdz5kz98MMP+vDDDzV69GhlZGRccmr4rFmzNHv2bIf2lStXcosPys2xY8eUm5vrUt8jR45o/vz5evTRR9WkSROX9gkODlajRo0uJ0QAML2cnBwNHz5c586dU40aNYwOx46n8ifJ/DlUXl6ejhw54tB+4cIFnTx5UvXq1VNgYKDD9iZNmigoyHvrQwEAUFmVNYfyykwni8XidMHGc+fOqbCw0OXjnD59WoWFhapfv75de/369Z2uEyNJ27Zt0+LFi7Vnzx6XzzN16lTFxMTYXp8/f15NmzZV3759K1xCWhHk5+crISFBffr0UUBAgNHhuK0ixp+ammr3jwJXlfT0upL89NNPFeIWg4r4M3AH8RuL+I1l9vjPnDljdAgl8lT+JJFDVVRm//wQv7GI31jEbzyzj8Hs8Zc1h/JK0alHjx6KjY3VO++8I39/f0kXp2vHxsaqe/fu3jilpIu37IwcOVJvvPGG6tSp4/J+QUFBTq+KBQQEmPKXobyY/f2pSPH//vvvkqQVK1YoIiKi1P5ZWVlau3atoqKiXFpkNjk5WSNGjNDvv/9eYcYsVayfQVkQv7GI31hmjb8ix2xU/iSRQ5U3s78/xG8s4jcW8RvP7GMwa/xljdkrRafnn39ePXr0UOvWrW2LUG7dulXnz5/Xl19+6fJx6tSpI39/f504ccKu/cSJE2rQoIFD/4MHDyotLU2DBg2ytRUVFUmSqlSpon379qlFixZlGRJQLiIiIlxaQDs/P19nz55Vly5dTPkXFgDAkafyJ4kcCgAAVAx+3jjotddeqx9++EF33XWXTp48qczMTI0aNUopKSlq06aNy8cJDAxUx44dlZiYaGsrKipSYmKiunTp4tA/PDxcP/74o/bs2WP7c/vtt+vmm2/Wnj171LRpU4+MDwAAwNM8lT9J5FAAAKBi8MpMJ0lq1KiR5s6da9eWkZGhl19+WRMnTnT5ODExMYqOjlanTp3UuXNnxcfHKzs72/YkllGjRqlx48aKjY1V1apVHZKyWrVqSZLbyRoAAEB581T+JJFDAQAA43mt6PRHiYmJWrx4sT788EOFhIS4lTQNGzZMp06d0owZM5Senq727dtr48aNtoUxDx8+LD8/r0zYAspVg+oWBWfsl4658PtcUKCaOWnS8e+lKqV/jIMz9qtBdcvlBwkAKDeXkz9J5FAAAMB4Xis6/frrr1q6dKmWLl2qw4cPa9iwYfrwww916623un2siRMnlphobdq06ZL7vvnmm26fDzDC+I6BitgyXtpSet8ASb0kaZ9rx474v+MDACo2T+ZPEjkUAAAwlkeLTvn5+Vq7dq3+85//aOvWrfrrX/+qF198Uffcc4+mTZuma6+91pOnA3zKa0kXNGzGm4oIDy+1b35Bgb7++mt169ZNAS7MdEpOSdFr84brdk8ECgDwKPInAADgqzxadGrcuLHCw8M1YsQIvfvuu6pdu7Yk6Z577vHkaQCflJ5lVW6tVlKj9qV3zs/XuZCjUsN2kgtPr8tNL1J6lvXygwQAeBz5EwAA8FUevZG/oKBAFotFFotF/v7+njw0AACATyJ/AgAAvsqjRadjx47p/vvv1zvvvKMGDRrojjvu0IcffiiLhQWMAQAAnCF/AgAAvsqjt9dVrVpV9957r+69914dPHhQS5cu1cMPP6yCggLNmTNHo0eP1i233MJVPOBPcnJyJEm7du1yqX9WVpY2b96s2rVrq3r16qX2T05Ovqz4AADeQ/4EAAB8ldeeXteiRQs9++yzevrpp/XZZ59p8eLFGjhwoEJDQ3X69GlvnRYwpZSUFEnSuHHj3Npv/vz5bvUPDQ11qz8AoHyRPwEAAF/itaJTMT8/P/Xv31/9+/fXqVOntHz5cm+fEjCdqKgoSVJ4eLhCQkJK7b93715FR0dr2bJlatOmjUvnCA0NVcuWLS8nTABAOSF/AgAAvsDrRac/qlu3rmJiYsrzlIAp1KlTR2PHjnW5f0FBgaSLRaoOHTp4KywAQAVA/gQAAMzKowuJAwAAAAAAABJFJwAAAAAAAHhBud5eB8A9OTk5tkXG/6i4LSUlRVWqOH6MXV0bCgAAAAAAb6HoBJ9VUsEmMzNTmzdvVq1atZw+za0iFWxSUlLUsWPHErdHR0c7bU9KSmKtJwAAAACAocq16LRu3TqdO3dOo0aNKs/TopIqrWAzf/58p+0VqWATHh6upKQkh/bMzEytW7dOgwcPLrFwBgDwDeRPAADArMq16PTEE08oNTWVpAnloqSCzd69exUdHa1ly5apTZs2TverKEJCQpwWwPLz85WRkaGuXbsqICDAgMgAAOWF/AkAAJhVuRadnN3qBHhLSQWbgoICSReLSxVlRhMAACUhfwIAAGbllafXHT58WFartcRtAAAAsEf+BAAAfI1Xik7NmzfXqVOnHNrPnDmj5s2be+OUAAAApkb+BAAAfI1Xik5Wq1UWi8WhPSsrS1WrVvXGKQEAAEyN/AkAAPgaj67pFBMTI0myWCyaPn263WPnCwsLtWPHDrVv396TpwQAADA18icAAOCrPFp02r17t6SLV+p+/PFHBQYG2rYFBgaqXbt2mjx5sidPCUiSUlNTlZmZ6VLf4gVZU1JSVKWKax+B0NBQtWzZsszxAQBQEvInAADgqzxadPrqq68kSWPGjNFLL72kGjVqePLwgFOpqalq1aqV2/tFR0e71X///v0UngAAHkf+BAAAfJVHi07Fli5d6o3DAk4Vz3BasWKFIiIiSu2flZWltWvXKioqStWrVy+1f3JyskaMGOHyTCoAAMqC/AkAAPgarxSdJOm///2v3nvvPR0+fFgXLlyw27ZmzRpvnRaVWEREhDp06FBqv/z8fJ09e1ZdunRRQEBAOUQGAIBryJ8AAIAv8crT695991117dpVycnJ+vDDD5Wfn6+ffvpJX375pWrWrOmNUwIAAJga+RMAAPA1Xik6zZ07V/Pnz9fHH3+swMBAvfTSS0pJSdFdd92lq666yhunBAAAMDXyJwAA4Gu8UnQ6ePCgBgwYIOniU1eys7NlsVj06KOP6vXXX/fGKQEAAEyN/AkAAPgarxSdateubVt0uXHjxtq7d68kKSMjQzk5Od44JQAAgKmRPwEAAF/jlYXEe/TooYSEBLVt21Z33nmnJk2apC+//FIJCQm69dZbvXFKVHINqlsUnLFfOuZCHbWgQDVz0qTj30tVSv8IBGfsV4PqlssPEgCASyB/AgAAvsYrRaeXX35Zv//+uyTpqaeeUkBAgLZv36477rhD06ZN88YpUcmN7xioiC3jpS2l9w2Q1EuS9rl27Ij/Oz4AAN5E/gQAAHyNV4pOV1xxhe3//fz8NGXKFG+cBrB5LemChs14UxHh4aX2zS8o0Ndff61u3bopwIWZTskpKXpt3nDd7olAAQAoAfkTAADwNR5d0+nYsWOaPHmyzp8/77Dt3Llzevzxx3XixAlPnhKQJKVnWZVbq5XUqH3pfxq207mQMKlhO5f659ZqpfQsa3kPCQBQSZA/AQAAX+XRolNcXJzOnz+vGjVqOGyrWbOmMjMzFRcX58lTAgAAmBr5EwAA8FUeLTpt3LhRo0aNKnH7qFGjtH79ek+eEgAAwNTInwAAgK/yaNHp0KFDuuqqq0rc3qRJE6WlpXnylAAAAKZG/gQAAHyVR4tOwcHBl0yK0tLSFBwc7MlTAgAAmBr5EwAA8FUefXpdZGSkli9frh49ejjd/tZbb6lz586ePCWgnJwcSdKuXbtc6p+VlaXNmzerdu3aql69eqn9k5OTLys+AAAuhfwJAAD4Ko8WnSZPnqw+ffqoZs2aevzxx1W/fn1J0okTJ/TCCy/ozTff1Oeff+72cV955RW9+OKLSk9PV7t27bRgwYISk681a9Zo7ty5OnDggPLz89WyZUs99thjGjly5GWNDRVXSkqKJGncuHFu7Td//ny3+oeGhrrVHwAAV3grf5LIoQAAgLE8WnS6+eab9corr2jSpEmaP3++atSoIYvFonPnzikgIEALFizQLbfc4tYxV61apZiYGC1atEiRkZGKj49Xv379tG/fPtWrV8+h/xVXXKGnnnpK4eHhCgwM1Pr16zVmzBjVq1dP/fr189RQUYFERUVJksLDwxUSElJq/7179yo6OlrLli1TmzZtXDpHaGioWrZseTlhAgDglDfyJ4kcCgAAGM+jRSdJGj9+vAYOHKj33ntPBw4ckNVqVatWrTR06FA1adLE7ePFxcVp3LhxGjNmjCRp0aJF2rBhg5YsWaIpU6Y49O/Vq5fd60mTJmnZsmXatm1biQlTXl6e8vLybK/Pnz8vScrPz1d+fr7bMfu64vekorw3NWvWVHR0tMv9c3NzJUktWrRQ27ZtXd6vooxXqng/A3cRv7GI31jEb6yKGren8yeJHKoi8pXPD/Ebg/iNRfzGM/sYfCV+d1msVqvVw7F4zIULFxQSEqLVq1fbZrNIUnR0tDIyMrRu3bpL7m+1WvXll1/q9ttv19q1a9WnTx+n/WbNmqXZs2c7tK9cudKlmTMwl4MHD+qxxx7TvHnz1KJFC6PDAQCUo5ycHA0fPlznzp1TjRo1jA7Ha8ihAACAJ5U1h/L4TCdPOn36tAoLC21rGxSrX7++bR0fZ86dO6fGjRsrLy9P/v7+evXVV0tMliRp6tSpiomJsb0+f/68mjZtqr59+/p0QlpW+fn5SkhIUJ8+fRQQEGB0OG7buXOnpIsLt5p1YVaz/wyI31jEbyziN9aZM2eMDqFckENVTGb//BC/sYjfWMRvPLOPwezxlzWHqtBFp7IKDQ3Vnj17lJWVpcTERMXExOjqq692mDZeLCgoSEFBQQ7tAQEBpvxl8JScnByniWlmZqY2b96sWrVqOV1c29W1lYxS/DP1hZ+v2cdA/MYifmMRvzHMGHN5IocqH2Z/f4jfWMRvLOI3ntnHYNb4yxpzhS461alTR/7+/jpx4oRd+4kTJ9SgQYMS9/Pz89M111wjSWrfvr2Sk5MVGxtbYsIE51JSUtSxY8cSt5f09LekpCR16NDBW2EBAIBSkEMBAICKoEIXnQIDA9WxY0clJiba1iMoKipSYmKiJk6c6PJxioqK7Ba5hGvCw8OVlJTk0F7a09/Cw8PLI7xSlTRTq7gtJSVFVao4fgQq+kwtAABKQw4FAAAqAq8Una6++mp99913uvLKK+3aMzIy1KFDB/38888uHysmJkbR0dHq1KmTOnfurPj4eGVnZ9uexDJq1Cg1btxYsbGxkqTY2Fh16tRJLVq0UF5enj755BMtX75cCxcu9NwAK4mQkBCnM5YKCgokXSzOVOQZTaXN1CrpiXfM1AIAGMGT+ZNEDgUAAIznlaJTWlqaCgsLHdrz8vJ09OhRt441bNgwnTp1SjNmzFB6errat2+vjRs32hbGPHz4sPz8/Gz9s7Oz9eCDD+rIkSMKDg5WeHi4VqxYoWHDhl3eoGA6Jc3UyszM1Lp16zR48OAS16QCAKC8eTJ/ksihAACA8TxadProo49s///ZZ5+pZs2atteFhYVKTExUWFiY28edOHFiiVPBN23aZPf62Wef1bPPPuv2OeB7SpqplZ+fr4yMDHXt2tWUC7gBAHyLt/IniRwKAAAYy6NFp+I1AywWi8OtSwEBAQoLC9O8efM8eUp4SGpqqjIzM13qW9qaSM6EhoaqZcuWZY4PAABfRf4EAAB8lUeLTkVFRZKk5s2b67vvvlOdOnU8eXh4SWpqqlq1auX2fiWtiVSS/fv3U3gCAOBPyJ8AAICv8sqaTocOHXJoy8jIUK1atbxxOlym4hlOK1asUERERKn9s7KytHbtWkVFRal69eql9k9OTtaIESNcnkkFAEBlRP4EAAB8jVeKTs8//7zCwsJsC0/eeeed+uCDD9SwYUN98sknateunTdOi8sUERHh0lPb8vPzdfbsWXXp0oU1kQAA8BDyJwAA4Gv8Su/ivkWLFqlp06aSpISEBH3xxRfauHGj+vfvr8cff9wbpwQAADA18icAAOBrvDLTKT093ZY0rV+/XnfddZf69u2rsLAwRUZGeuOUuEwNqlsUnLFfOuZCHbKgQDVz0qTj30suLCQenLFfDapbLj9IAAB8GPkTAADwNV4pOtWuXVu//vqrmjZtqo0bN9oev2u1WlVYWOiNU+Iyje8YqIgt46UtpfcNkNRLkva5duyI/zs+AAAoGfkTAADwNV4pOg0ZMkTDhw9Xy5YtdebMGfXv31+StHv3bl1zzTXeOCUu02tJFzRsxpuKCA8vtW9+QYG+/vprdevWTQEuzHRKTknRa/OG63ZPBAoAgI8ifwIAAL7GK0Wn+fPnKywsTL/++qteeOEF2xPOjh8/rgcffNAbp8RlSs+yKrdWK6lR+9I75+frXMhRqWE7yYWFxHPTi5SeZb38IAEA8GHkTwAAwNd4pegUEBCgyZMnO7Q/+uij3jgdAACA6ZE/AQAAX+OVopMkLV++XK+99pp+/vlnffPNN2rWrJni4+PVvHlzDR482FunRRnk5ORIknbt2uVS/6ysLG3evFm1a9e2XYW9lOTk5MuKDwCAyoL8CQAA+BKvFJ0WLlyoGTNm6JFHHtGcOXNsi1/WqlVL8fHxJE0VTEpKiiRp3Lhxbu03f/58t/qHhoa61R8AgMqE/AkAAPgarxSdFixYoDfeeENRUVF67rnnbO2dOnVyOm0cxoqKipIkhYeHKyQkpNT+e/fuVXR0tJYtW6Y2bdq4dI7Q0FC1bNnycsIEAMCnkT8BAABf45Wi06FDh3T99dc7tAcFBSk7O9sbp8RlqFOnjsaOHety/4KCAkkXi1QdOnTwVlgAAFQq5E8AAMDX+HnjoM2bN9eePXsc2jdu3KiIiAhvnBIAAMDUyJ8AAICv8ehMp6efflqTJ09WTEyMHnroIf3++++yWq3auXOn3nnnHcXGxuo///mPJ08JAABgauRPAADAV3m06DR79mxNmDBBY8eOVXBwsKZNm6acnBwNHz5cjRo10ksvvaS7777bk6eEF+Xk5NgWGf+j4raUlBRVqeL4K+Tq2lAAAID8CQAA+C6PFp2sVqvt/++9917de++9ysnJUVZWlurVq+fJU6EcpKSkqGPHjiVuj46OdtqelJTEWk8AALiI/AkAAPgqjy8kbrFY7F6HhIQw68WkwsPDlZSU5NCemZmpdevWafDgwQoNDXW6HwAAcB35EwAA8EUeLzq1atXKIXH6s99++83Tp4UXhISEOJ2xlJ+fr4yMDHXt2lUBAQEGRAYAgG8hfwIAAL7I40Wn2bNnq2bNmp4+LAAAgM8ifwIAAL7I40Wnu+++m/UHAAAA3ED+BAAAfJGfJw9W2rRwAAAA2CN/AgAAvsqjRac/Pn0FAAAApSN/AgAAvsqjt9cVFRV58nAAAAA+j/wJAAD4Ko/OdAIAAAAAAAAkik4AAAAAAADwAopOAAAAAAAA8DiKTgAAAAAAAPA4ik4AAAAAAADwOIpOAAAAAAAA8DiKTgAAAAAAAPA4ik4AAAAAAADwOIpOAAAAAAAA8DiKTgAAAAAAAPC4KkYH4MtycnKUkpLi0J6bm6u0tDSFhYUpODjYYXt4eLhCQkLKI0QAAAAAAACvMMVMp1deeUVhYWGqWrWqIiMjtXPnzhL7vvHGG7rppptUu3Zt1a5dW717975kf29KSUlRx44dHf50795dI0aMUPfu3Z1ud1aoAgAAcJdZcygAAOAbKvxMp1WrVikmJkaLFi1SZGSk4uPj1a9fP+3bt0/16tVz6L9p0ybdc8896tq1q6pWrarnn39effv21U8//aTGjRuXa+zh4eFKSkpyaE9OTtaIESO0YsUKRUREON0PAADgcpg5hwIAAL6hwhed4uLiNG7cOI0ZM0aStGjRIm3YsEFLlizRlClTHPq//fbbdq//85//6IMPPlBiYqJGjRrl9Bx5eXnKy8uzvT5//rwkKT8/X/n5+WWOPSAgQG3btnVoLygokCS1bNnS6fbic1dUxbFV5BgvxezxS+YfA/Ebi/iNRfzGMmvcZWHmHMpX+crnh/iNQfzGIn7jmX0MvhK/uyxWq9Xq4Vg85sKFCwoJCdHq1asVFRVla4+OjlZGRobWrVtX6jEyMzNVr149vf/++xo4cKDTPrNmzdLs2bMd2leuXOmVtZUOHjyoxx57TPPmzVOLFi08fnwAAOBcTk6Ohg8frnPnzqlGjRpGh+M1vppDAQAAY5Q1h6rQM51Onz6twsJC1a9f3669fv36Lq979MQTT6hRo0bq3bt3iX2mTp2qmJgY2+vz58+radOm6tu3r8tvZmpqqrKyslzqe+7cOUlSnTp11LBhQ5f2qV69ulq2bOlSX2/Lz89XQkKC+vTpo4CAAKPDcZvZ45fMPwbiNxbxG4v4jXXmzBmjQygXZsqhKhOzf36I31jEbyziN57Zx2D2+MuaQ1XootPleu655/Tuu+9q06ZNqlq1aon9goKCFBQU5NAeEBDg0i9DamqqrrvuOrfji46Odqv//v37K0zhSXL9/amozB6/ZP4xEL+xiN9YxG8MM8ZshPLKoSors78/xG8s4jcW8RvP7GMwa/xljblCF53q1Kkjf39/nThxwq79xIkTatCgwSX3/de//qXnnntOX3zxhf7yl794M0xlZmZKUokLg/9Zbm6u0tLSFBYWpuDg4FL7Fy88XnweAACASzFLDgUAAHxbhS46BQYGqmPHjkpMTLStR1BUVKTExERNnDixxP1eeOEFzZkzR5999pk6depUTtFKERER6tChg0t9u3Xr5uVoAABAZWW2HAoAAPimCl10kqSYmBhFR0erU6dO6ty5s+Lj45WdnW17EsuoUaPUuHFjxcbGSpKef/55zZgxQytXrlRYWJjS09MlXVwTqXr16oaNAwAAoDyRQwEAAKNV+KLTsGHDdOrUKc2YMUPp6elq3769Nm7caFsY8/Dhw/Lz87P1X7hwoS5cuKChQ4faHWfmzJmaNWuW1+JsUN2i4Iz90jG/0ju7KThjvxpUt3j8uAAAwHeZJYcCAAC+q8IXnSRp4sSJJU4F37Rpk93rtLQ07wfkxPiOgYrYMl7a4vljR/zf8QEAANxhhhwKAAD4LlMUnczgtaQLGjbjTUWEh3v82MkpKXpt3nDd7vEjAwAAAAAAeAdFJw9Jz7Iqt1YrqVF7jx87N71I6VlWjx8XAAAAAADAWzy/ABEAAAAAAAAqPWY6eUBOTo4kadeuXS71z83NVVpamsLCwhQcHFxq/+Tk5MuKDwAAAAAAoLxRdPKAlJQUSdK4ceO8ep7Q0FCvHh8AAAAAAMBTKDp5QFRUlCQpPDxcISEhpfZPTk7WiBEjtGLFCkVERLh0jtDQULVs2fJywgQAAAAAACg3FJ08oE6dOho7dqzb+0VERKhDhw5eiAgAAAAAAMBYLCQOAAAAAAAAj2OmkxNWq1WSdP78+cs6Tk5Ojvbv3+/Qvm/fPklSUlKSsrKyHLa3atXKpdv0jJKfn6+cnBydP39eAQEBRofjNrPHL5l/DMRvLOI3FvEbKzMzU9L/vuvhWZ7KoXyV2T8/xG8s4jcW8RvP7GMwe/xlzaEsVrIuB0eOHFHTpk2NDgMAAHjJwYMHdfXVVxsdhs8hhwIAwLe5m0NRdHKiqKhIx44dU2hoqCwWi9HhVDjnz59X06ZN9euvv6pGjRpGh+M2s8cvmX8MxG8s4jcW8Rvr3Llzuuqqq3T27FnVqlXL6HB8DjnUpZn980P8xiJ+YxG/8cw+BrPHX9YcitvrnPDz81OTJk2MDqPCq1Gjhik/LMXMHr9k/jEQv7GI31jEbyw/P5a19AZyKNeY/fND/MYifmMRv/HMPgazx+9uDkXGBQAAAAAAAI+j6AQAAAAAAACPo+gEtwUFBWnmzJkKCgoyOpQyMXv8kvnHQPzGIn5jEb+xzB4/zM3sv3/EbyziNxbxG8/sY6is8bOQOAAAAAAAADyOmU4AAAAAAADwOIpOAAAAAAAA8DiKTgAAAAAAAPA4ik4AAAAAAADwOIpOAFBGFotFa9euNToMAAAA0yB/AioXik4ATGn06NGyWCwOf/76178aHRoAAECFRP4EoLxVMToAACirv/71r1q6dKldW1BQkEHRAAAAVHzkTwDKEzOdAJhWUFCQGjRoYPendu3aki5O3V64cKH69++v4OBgXX311Vq9erXd/j/++KNuueUWBQcH68orr9T999+vrKwsuz5LlizRddddp6CgIDVs2FATJ06023769Gn97W9/U0hIiFq2bKmPPvrIu4MGAAC4DORPAMoTRScAPmv69Om644479P333+vee+/V3XffreTkZElSdna2+vXrp9q1a+u7777T+++/ry+++MIuKVq4cKEeeugh3X///frxxx/10Ucf6ZprrrE7x+zZs3XXXXfphx9+0G233aZ7771Xv/32W7mOEwAAwFPInwB4lBUATCg6Otrq7+9vrVatmt2fOXPmWK1Wq1WSdcKECXb7REZGWh944AGr1Wq1vv7669batWtbs7KybNs3bNhg9fPzs6anp1utVqu1UaNG1qeeeqrEGCRZp02bZnudlZVllWT99NNPPTZOAAAATyF/AlDeWNMJgGndfPPNWrhwoV3bFVdcYfv/Ll262G3r0qWL9uzZI0lKTk5Wu3btVK1aNdv2bt26qaioSPv27ZPFYtGxY8d06623XjKGv/zlL7b/r1atmmrUqKGTJ0+WdUgAAABeRf4EoDxRdAJgWtWqVXOYru0pwcHBLvULCAiwe22xWFRUVOSNkAAAAC4b+ROA8sSaTgB81rfffuvwOiIiQpIUERGh77//XtnZ2bbtX3/9tfz8/NS6dWuFhoYqLCxMiYmJ5RozAACAkcifAHgSM50AmFZeXp7S09Pt2qpUqaI6depIkt5//3116tRJ3bt319tvv62dO3dq8eLFkqR7771XM2fOVHR0tGbNmqVTp07pH//4h0aOHKn69etLkmbNmqUJEyaoXr166t+/vzIzM/X111/rH//4R/kOFAAAwEPInwCUJ4pOAExr48aNatiwoV1b69atlZKSIunik1HeffddPfjgg2rYsKHeeecdXXvttZKkkJAQffbZZ5o0aZJuuOEGhYSE6I477lBcXJztWNHR0fr99981f/58TZ48WXXq1NHQoUPLb4AAAAAeRv4EoDxZrFar1eggAMDTLBaLPvzwQ0VFRRkdCgAAgCmQPwHwNNZ0AgAAAAAAgMdRdAIAAAAAAIDHcXsdAAAAAAAAPI6ZTgAAAAAAAPA4ik4AAAAAAADwOIpOAAAAAAAA8DiKTgAAAAAAAPA4ik4AAAAAAADwOIpOAAAAAAAA8DiKTgAAAAAAAPA4ik4AAAAAAADwOIpOAAAAAAAA8DiKTgAAAAAAAPA4ik4AAAAAAADwOIpOAAAAAAAA8DiKTgAAAAAAAPA4ik4AAAAAAADwOIpOAPAns2bNksViMTqMchEWFqbRo0cbHQYAAPBBb775piwWi9LS0owOpUzS0tJksVj05ptvGh0KYFoUnYAKqvhL+r///e8l+506dUqTJk1SeHi4goODVa9ePXXu3FlPPPGEsrKytGnTJlksFpf+/PG8FotF27Ztczif1WpV06ZNZbFYNHDgwFLH0atXL7tzXHHFFbrhhhu0ZMkSFRUVle3NMaHRo0fbvQ9BQUFq1aqVZsyYod9//93o8AAAqBR8Jb8q3mf58uXq0aOHatWqpZCQELVt21bPPvuscnJy3H9zDBYWFmb3vlWrVk2dO3fWW2+9ZXRoAC5DFaMDAFB2v/32mzp16qTz58/rvvvuU3h4uM6cOaMffvhBCxcu1AMPPKCIiAgtX77cbr+pU6eqevXqeuqpp0o8dtWqVbVy5Up1797drn3z5s06cuSIgoKCXI6zSZMmio2NlXQxiXvrrbf097//Xfv379dzzz3nxojNLSgoSP/5z38kSefOndO6dev0zDPP6ODBg3r77bcNjg4AAEjmyK8KCws1fPhwvffee7rppps0a9YshYSEaOvWrZo5c6bee+89ffHFF6pXr577b4AHjRw5UnfffbfL42rfvr0ee+wxSdLx48f1n//8R9HR0crLy9O4ceO8GSoAL6HoBJjY4sWLdfjwYX399dfq2rWr3bbz588rMDBQVatW1YgRI+y2Pffcc6pTp45D+x/ddtttev/99/Xvf/9bVar876+KlStXqmPHjjp9+rTLcdasWdPuXOPHj1fr1q318ssv65lnnlFAQIDLxzKzKlWq2L0PDz74oLp27ap33nlHcXFxql+/voHRAQAAyRz51QsvvKD33ntPkydP1osvvmhrv//++3XXXXcpKipKY8aM0YYNG1w6nrf4+/vL39/f5f6NGze2e/9Gjx6tq6++WvPnz6foBJgUt9cBJnbw4EH5+/vrxhtvdNhWo0YNVa1atczHvueee3TmzBklJCTY2i5cuKDVq1dr+PDhZT6uJIWEhOjGG29Udna2Tp06JUlKSEhQ9+7dVatWLVWvXl2tW7fWk08+abdfXl6eZs6cqWuuuUZBQUFq2rSp/vnPfyovL8/W51L33lssFs2aNcuubdu2bbrhhhtUtWpVtWjRQq+99prTmAsKCvTMM8+oRYsWCgoKUlhYmJ588km7c7vLYrGoe/fuslqt+vnnn23tv/zyix588EG1bt1awcHBuvLKK3XnnXc6rIdQPFX/66+/VkxMjOrWratq1arpb3/7m+19LWa1WvXss8+qSZMmCgkJ0c0336yffvrJaVw///yz7rzzTl1xxRW2n9Wfk9bi2wree+89zZ49W40bN1ZoaKiGDh2qc+fOKS8vT4888ojq1aun6tWra8yYMZf1XgEAUF4qen6Vm5urF198Ua1atbLNJP+jQYMGKTo6Wp988ol27txpa3eWB0mO6zv+9ttvmjx5stq2bavq1aurRo0a6t+/v77//nuHfRcsWKDrrrtOISEhql27tjp16qSVK1fatl/umk5169ZVeHi4Dh48aNe+detW3XnnnbrqqqtsOeGjjz6q3Nxcu36jR49W9erVdfToUUVFRal69eqqW7euJk+erMLCQru+GRkZGj16tGrWrKlatWopOjpaGRkZTuP68ssvddNNN6latWqqVauWBg8erOTkZLs+xWuE7t+/XyNGjFDNmjVVt25dTZ8+XVarVb/++qsGDx6sGjVqqEGDBpo3b16Z3iOgoqPoBJhYs2bNVFhY6DC92xPCwsLUpUsXvfPOO7a2Tz/9VOfOndPdd9992cf/+eef5e/vr1q1aumnn37SwIEDlZeXp6efflrz5s3T7bffrq+//trWv6ioSLfffrv+9a9/adCgQVqwYIGioqI0f/58DRs2rEwx/Pjjj+rbt69OnjypWbNmacyYMZo5c6Y+/PBDh75jx47VjBkz1KFDB82fP189e/ZUbGzsZb8XxUlY7dq1bW3fffedtm/frrvvvlv//ve/NWHCBCUmJqpXr15O12j4xz/+oe+//14zZ87UAw88oI8//lgTJ0606zNjxgxNnz5d7dq104svvqirr75affv2VXZ2tl2/EydOqGvXrvrss8/04IMPas6cOfr99991++23O31fYmNj9dlnn2nKlCm67777tGbNGk2YMEH33Xef9u/fr1mzZmnIkCF688039fzzz1/WewUAQHmo6PnVtm3bdPbsWQ0fPtxuttQfjRo1SpL08ccfux3jzz//rLVr12rgwIGKi4vT448/rh9//FE9e/bUsWPHbP3eeOMNPfzww7r22msVHx+v2bNnq3379tqxY4fb5yxJQUGBjhw5YpcnSdL777+vnJwcPfDAA1qwYIH69eunBQsW2Mb9R4WFherXr5+uvPJK/etf/1LPnj01b948vf7667Y+VqtVgwcP1vLlyzVixAg9++yzOnLkiKKjox2O98UXX6hfv362/DEmJkbbt29Xt27dnBbXhg0bpqKiIj333HOKjIzUs88+q/j4ePXp00eNGzfW888/r2uuuUaTJ0/Wli1bLv9NAyoaK4AKaenSpVZJ1u+++67EPunp6da6detaJVnDw8OtEyZMsK5cudKakZFxyWNfd9111p49e5Z63pdfftkaGhpqzcnJsVqtVuudd95pvfnmm61Wq9XarFkz64ABA0odR8+ePa3h4eHWU6dOWU+dOmVNTk62Pvzww1ZJ1kGDBlmtVqt1/vz5VknWU6dOlXic5cuXW/38/Kxbt261a1+0aJFVkvXrr7+2Wq1W66FDh6ySrEuXLnU4hiTrzJkzba+joqKsVatWtf7yyy+2tv/3//6f1d/f3/rHvx737NljlWQdO3as3fEmT55slWT98ssvS30foqOjrdWqVbO9DwcOHLD+61//slosFmubNm2sRUVFtr7F7/cfffPNN1ZJ1rfeesvWVvyz6t27t93+jz76qNXf39/2e3Dy5ElrYGCgdcCAAXb9nnzySaska3R0tK3tkUcesUqye58zMzOtzZs3t4aFhVkLCwutVqvV+tVXX1klWdu0aWO9cOGCre8999xjtVgs1v79+9vF36VLF2uzZs1KfZ8AAPAmX8iv4uPjrZKsH374YYl9fvvtN6sk65AhQ2xtf86DijVr1swuF/j9999t3/fFDh06ZA0KCrI+/fTTtrbBgwdbr7vuukvGWjzuQ4cOXbJfcRx9+/a15Uo//vijdeTIkVZJ1oceesiur7NcKTY21mqxWOzyuujoaKsku7itVqv1+uuvt3bs2NH2eu3atVZJ1hdeeMHWVlBQYL3pppsc8sr27dtb69WrZz1z5oyt7fvvv7f6+flZR40aZWubOXOmVZL1/vvvtztmkyZNrBaLxfrcc8/Z2s+ePWsNDg62+zkAvoKZToCJ1a9fX99//70mTJigs2fPatGiRRo+fLjq1aunZ555Rlar9bKOf9dddyk3N1fr169XZmam1q9fX6Zb61JSUlS3bl3VrVtXERERWrBggQYMGKAlS5ZIkmrVqiVJWrduXYlPtHv//fcVERGh8PBwnT592vbnlltukSR99dVXbsVUWFiozz77TFFRUbrqqqts7REREerXr59d308++USSFBMTY9devNClq+slZGdn296H4ita3bp107p162xPt5Gk4OBg2//n5+frzJkzuuaaa1SrVi3t2rXL4bj333+/3f433XSTCgsL9csvv0i6eEXuwoUL+sc//mHX75FHHnE41ieffKLOnTvbLXBavXp13X///UpLS9P/+3//z67/qFGj7NbkioyMlNVq1X333WfXLzIyUr/++qsKCgpKe5sAADBURc+vMjMzJUmhoaEl9ineVtzXHUFBQfLzu/jPxMLCQp05c8a29MEf85BatWrpyJEj+u6779w+R0k+//xzW67Utm1bLV++XGPGjLFbt0qyz5Wys7N1+vRpde3aVVarVbt373Y47oQJE+xe33TTTXZLG3zyySeqUqWKHnjgAVubv7+//vGPf9jtd/z4ce3Zs0ejR4/WFVdcYWv/y1/+oj59+thyxj8aO3as3TE7deokq9Wqv//977b2WrVqqXXr1nYxAb6CohNgcg0bNtTChQt1/Phx7du3T//+979Vt25dzZgxQ4sXL76sY9etW1e9e/fWypUrtWbNGhUWFmro0KFuHycsLEwJCQn64osvtG3bNqWnp2v9+vWqU6eOpIvTjrt166axY8eqfv36uvvuu/Xee+/ZFaBSU1P1008/2RKR4j+tWrWSJJ08edKtmE6dOqXc3Fy1bNnSYVvr1q3tXv/yyy/y8/PTNddcY9feoEED1apVy1bcyc3NVXp6ut2fP6pataoSEhKUkJCgpUuXKiIiQidPnrRLnIqPM2PGDDVt2lRBQUGqU6eO6tatq4yMDJ07d84h3j8WzaT/3ap39uxZW/ySHMZat25dh+nqv/zyi8P4pYvFuD8eq6Rz16xZU5LUtGlTh/aioiKn8QMAUNFU5PzKlYJS8bayPL2uqKhI8+fPV8uWLe3ykB9++MHue/yJJ55Q9erV1blzZ7Vs2VIPPfSQ3dIIzpw7d84uT/rtt9/stkdGRiohIUEbN27Uv/71L9WqVUtnz55VYGCgXb/Dhw/bCj/F6zT17NnTdo4/qlq1qurWrWvXVrt2bVueJF3Mbxo2bKjq1avb9XOWEzprly7mSqdPn3ZYusBZrlS1alVbHvzH9j/GBPgKnl4H+AiLxaJWrVqpVatWGjBggFq2bKm3337b7upKWQwfPlzjxo1Tenq6+vfvb5uV5I5q1aqpd+/eJW4PDg7Wli1b9NVXX2nDhg3auHGjVq1apVtuuUWff/65/P39VVRUpLZt2youLs7pMYqLHH+cyfNHf14ssixKOnaxVatWacyYMXZtf7wa6u/vb/c+9OvXT+Hh4Ro/frw++ugjW/s//vEPLV26VI888oi6dOmimjVrymKx6O6773Y6E6ykp8Jc7pVYV5R0biNjAgDAUypifnXttddKkn744QdFRUU57fPDDz9Ikq6++upSj/fnHGnu3LmaPn267rvvPj3zzDO64oor5Ofnp0ceecQuD4mIiNC+ffu0fv16bdy4UR988IFeffVVzZgxQ7Nnz3Z6rkmTJmnZsmW21z179tSmTZtsr+vUqWPLlYrzpIEDB+qll16yzTgvLCxUnz599Ntvv+mJJ55QeHi4qlWrpqNHj2r06NEOuZI7T8/zBmfnJ09CZULRCfBBV199tWrXrq3jx49f9rH+9re/afz48fr222+1atUqD0TnnJ+fn2699VbdeuutiouL09y5c/XUU0/pq6++Uu/evdWiRQt9//33uvXWWy9Z/CmeufPnp438eYZO3bp1FRwcrNTUVIdj7Nu3z+51s2bNVFRUpNTUVNuMH+niotsZGRlq1qyZpIvJ0R+fRlOahg0b6tFHH9Xs2bP17bff2p6Ss3r1akVHR9s9xeT3338v8QkqpSmOLzU11S75PHXqlMMVtWbNmjmMX7p4i+QfjwUAQGVTUfKrbt26qVatWlq5cqWeeuoppwWMt956S5J055132tpq167tkEtcuHDBYTyrV6/WzTff7DCjKyMjw2F2TrVq1TRs2DANGzZMFy5c0JAhQzRnzhxNnTrV6VP+/vnPf2rEiBF2MV3KgAED1LNnT82dO1fjx49XtWrV9OOPP2r//v1atmyZ3cLh7uRgf9asWTMlJiYqKyvLbraTs5zQWbt0MVeqU6eOqlWrVuY4AF/E7XWAie3YscNhCq8k7dy5U2fOnHE69ddd1atX18KFCzVr1iwNGjToso/nzJ+nVktS+/btJUl5eXmSLq5/cPToUb3xxhsOfXNzc23vQ40aNVSnTh2Hp3+8+uqrdq/9/f3Vr18/rV27VocPH7a1Jycn67PPPrPre9ttt0mS4uPj7dqLZ10NGDBA0sUiUu/eve3+lOYf//iHQkJC9Nxzz9nF9ucrXQsWLCjzbK3evXsrICBACxYssDvun8cjXRzrzp079c0339jasrOz9frrryssLMx2dRUAAF9V0fOrkJAQ/fOf/9S+ffv01FNPOWzfsGGD3nzzTQ0aNEht27a1tbdo0cIhP3r99dcd8gtnecj777+vo0eP2rWdOXPG7nVgYKCuvfZaWa1W5efnO4392muvtcuTOnbsWOp4n3jiCZ05c8aWAxYX2f4Yo9Vq1UsvvVTqsUpy2223qaCgQAsXLrS1FRYWasGCBXb9GjZsqPbt22vZsmV2Bby9e/fq888/t+WMAP6HmU5ABbdkyRJt3LjRoX3SpElavny53n77bf3tb39Tx44dFRgYqOTkZC1ZskRVq1bVk08+6ZEYnD0u1pOefvppbdmyRQMGDFCzZs108uRJvfrqq2rSpIltQeuRI0fqvffe04QJE/TVV1+pW7duKiwsVEpKit577z199tln6tSpk6SLCzY+99xzGjt2rDp16qQtW7Zo//79DuedPXu2Nm7cqJtuukkPPvigCgoKtGDBAl133XW2aemS1K5dO0VHR+v1119XRkaGevbsqZ07d2rZsmWKiorSzTffXOaxX3nllRozZoxeffVVJScnKyIiQgMHDtTy5ctVs2ZNXXvttfrmm2/0xRdf6MorryzTOerWravJkycrNjZWAwcO1G233abdu3fr008/dbhiOWXKFL3zzjvq37+/Hn74YV1xxRVatmyZDh06pA8++MC2sCgAAGZm9vzqn//8p/bs2aPnn39e33zzje644w4FBwdr27ZtWrFiha677jq9+eabdvuMHTtWEyZM0B133KE+ffro+++/12effeaQCwwcOFBPP/20xowZo65du+rHH3/U22+/7XCrXt++fdWgQQN169ZN9evXV3Jysl5++WUNGDDgkoucu6t///5q06aN4uLi9NBDDyk8PFwtWrTQ5MmTdfToUdWoUUMffPDBZa2HNGjQIHXr1k1TpkxRWlqarr32Wq1Zs8bpWpQvvvii+vfvry5duujvf/+7cnNztWDBAtWsWVOzZs26jJECvomiE1DB/fGKyx+NHj1a48ePV0hIiBITE7Vu3TqdP39edevWVd++fTV16lRdf/315Rxt2dx+++1KS0vTkiVLdPr0adWpU0c9e/bU7NmzbQtT+/n5ae3atZo/f77eeustffjhhwoJCdHVV1+tSZMm2RYUl6QZM2bo1KlTWr16td577z31799fn376qcNimn/5y1/02WefKSYmRjNmzFCTJk00e/ZsHT9+3K7oJEn/+c9/dPXVV+vNN9/Uhx9+qAYNGmjq1KmaOXPmZY8/JiZGixYt0vPPP68333xTL730kvz9/fX222/r999/V7du3fTFF184PFXPHc8++6yqVq2qRYsW6auvvlJkZKQ+//xz2yytYvXr19f27dv1xBNPaMGCBfr999/1l7/8RR9//LFDXwAAzMrs+ZW/v7/effdd3XbbbXrjjTc0bdo02+LhvXv31oYNGxwW3x43bpwOHTqkxYsX2y66JSQk6NZbb7Xr9+STTyo7O1srV67UqlWr1KFDB23YsEFTpkyx6zd+/Hi9/fbbiouLU1ZWlpo0aaKHH35Y06ZN8/h4J0+erNGjR+vtt9/W6NGj9fHHH+vhhx9WbGysqlatqr/97W+aOHGi2rVrV6bj+/n56aOPPtIjjzyiFStWyGKx6Pbbb9e8efMcft69e/fWxo0bNXPmTM2YMUMBAQHq2bOnnn/+eTVv3twTwwV8isXKamUAAAAAYGr5+fkaNGiQEhMT9fHHH+uvf/2r0SEBAEUnAAAAAPAF2dnZ6tWrl1JSUrR582Z16NDB6JAAVHIUnQAAAAAAAOBxrAgLAAAAAAAAj6PoBAAAAAAAAI+j6AQAAAAAAACPo+gEAAAAAAAAj6tidAAVUVFRkY4dO6bQ0FBZLBajwwEAAB5itVqVmZmpRo0ayc+Pa2+eRg4FAIBvKmsORdHJiWPHjqlp06ZGhwEAALzk119/VZMmTYwOw+eQQwEA4NvczaEoOjkRGhoq6eKbWaNGDYOjqXjy8/P1+eefq2/fvgoICDA6nBLt2bNHPXv2dHu/zZs3q3379p4PyIPM8jPIycnR/v37HdqTk5M1YcIELVq0SBEREQ7bW7VqpZCQkPIIUZJ08OBBdejQwevn2bVrl1q0aOH185TGLL8/JSF+Y5k9/t9++03Nmze3fdfDs8ihLs3snx/iNxbxG4v4jWf2MZg9/rLmUIYXnV555RW9+OKLSk9PV7t27bRgwQJ17ty5xP7x8fFauHChDh8+rDp16mjo0KGKjY1V1apVJUmFhYWaNWuWVqxYofT0dDVq1EijR4/WtGnTXJ7mXdyvRo0aJExO5OfnKyQkRDVq1KjQH5ZOnTopKSnJoX3v3r2Kjo7WsmXL1KZNG4ft4eHh5VrwKIuK+DNITU1VZmamQ3v16tUd2oKDg23/dbb92LFjDm2hoaFq2bKlByJ1ZLVa1aC6RYvj56h58+al9s/NzdFXX23SzTf3UnBw6b8rhw4d0t8feUpWq7VC/J1SEX9/3EH8xvKF+CX5xK1f5FDm4wufH+I3DvEbi/iNZ/Yx+EL8kvs5lKFFp1WrVikmJkaLFi1SZGSk4uPj1a9fP+3bt0/16tVz6L9y5UpNmTJFS5YsUdeuXbV//36NHj1aFotFcXFxkqTnn39eCxcu1LJly3Tdddfpv//9r8aMGaOaNWvq4YcfLu8hopyUVPAoi5SUFIc2bxY8fEFqaqpatWrl9n7R0dFu9d+/f79Xfg45OTka3zFQt/36nPSra/t0qC7pu3Uu9Y2QNL5jYJnjA4A/I4cCAABmYGjRKS4uTuPGjdOYMWMkSYsWLdKGDRu0ZMkSTZkyxaH/9u3b1a1bNw0fPlySFBYWpnvuuUc7duyw6zN48GANGDDA1uedd97Rzp07y2FEMILZCx6XkpOT47QIlpmZqc2bN6tWrVpOpzeW92yt4oLfihUrnN4u92dZWVlau3atoqKinM50+rPk5GSNGDHCY4XFP0tJSdFrSRf00b58rxxfko5nWXUvt/MA8BByKKBkZsmfAKAyMKzodOHCBSUlJWnq1Km2Nj8/P/Xu3VvffPON0326du2qFStWaOfOnercubN+/vlnffLJJxo5cqRdn9dff1379+9Xq1at9P3332vbtm22q3jO5OXlKS8vz/b6/Pnzki5OHyueQob/KX5PKsp7c/bsWUnSm2++6VLBIzMzUxs2bNCAAQNcuh81OTlZo0eP1tmzZ7065tTUVGVlZdm1paSkXLI4Nn/+fKfty5YtU3h4uF1b9erVvVY0KygokCS1bNlSbdu2LbV/fn6+zp49q06dOrk0tbT4+AUFBV75GQwYMECFhQvVunVru2SztPe/JCW9/2FhYRXic1PRPsPuIn5j+Ur8ZkYOZV6+8vmp6PHv3btXkZGRJW4vKX/asWOHrr/+em+FddnM8v6XhPiNZfb4JfOPwVfid5dhRafTp0+rsLBQ9evXt2uvX7++0ysTkjR8+HCdPn1a3bt3l9VqVUFBgSZMmKAnn3zS1mfKlCk6f/68wsPD5e/vr8LCQs2ZM0f33ntvibHExsZq9uzZDu2ff/45VzsuISEhwegQJF1cBLpBdYv8T+5VTmBGqf39Jd1+QzPp5F7lnCz9+P4nj6hBdYu2bdum48ePX3a8zhw7dkxP//MhNazueH/s9Q3cf6R3/BNjHNqOZ1k144VX1KhRozLFeCnFP4P9m99XzkHn/+D5s5qSdq57w6W+R454/2fQoEEDnTt3TufOnbO1Va1aVfPmzXPoe+HCBZ08eVL16tVTYKDjbXNVq1Z1Gmdqaqpng75MFeUzXFbEbyyzxp+Tk2N0CJeNHMr8zPr5KVbR48/Ly3P6/X3kyBHNnz9fjz76qNMnL6WlpXktz/Ckiv7+l4b4jWX2+CXzj8Gs8Zc1hzJ8IXF3bNq0SXPnztWrr76qyMhIHThwQJMmTdIzzzyj6dOnS5Lee+89vf3221q5cqWuu+467dmzR4888ogaNWpU4oyFqVOnKiYmxvb6/Pnzatq0qfr27csimE7k5+crISFBffr0qRALoO3evVvnOwZqRPbr0j7vnONAx0B1797da1e/vv76a43vGKhZvYK8cnxJmrUpTx07dvTKGHzhZ+COivYZcBfxG4v4jXXmzBmjQzAEOVTFYPbPj9nj37lzp+bPn6877rjjkovuGy0nJ0f79jkmVKXN1v/zjO2Kxuy/P8RvPLOPwezxlzWHMqzoVKdOHfn7++vEiRN27SdOnFCDBg2c7jN9+nSNHDlSY8eOlSS1bdtW2dnZuv/++/XUU0/Jz89Pjz/+uKZMmaK7777b1ueXX35RbGxsiQlTUFCQgoIc/7EfEBBgyl+G8lJR3p8qVarotaQLGjbjTUX86ZYmZ/ILCvT111+rW7duCqhS+kcgOSVFr80brturVPHaeA8cOFA+awrVru2VMfjCz6AsKspnoCS+vqZFRX//S0P8xjBjzH9GDmV+Zn9/zBp/ccwVPf6DBw+W6fbApKQkdejQwVthuYz8o2Ize/yS+cdg1vjLGrNhRafAwEB17NhRiYmJioqKkiQVFRUpMTFREydOdLpPTk6O/PzsbzXy9/eXdPGR55fqU1RU5OERoCJJz7Iqt1YrqVH70jvn5+tcyFGpYTvJhQ9ObnqR0rOslx/kJRR/Bv78ZZubm6u0tDSH/gcPHtTMmTM1e/ZstWjRwmF7WFiYgoOD7dq8+QS+nJwcpWdZ9fXPWcqtVfpnLSsrRx9994v8Gl/v2kLixwu9/jPwRSkpKerYsWOJ20laAXMihwJ8W3h4uJKSkhza9+7dq+joaC1btkxt2rRxul9FYPb8A4BnGXp7XUxMjKKjo9WpUyd17txZ8fHxys7Otj2JZdSoUWrcuLFiY2MlSYMGDVJcXJyuv/5629Tw6dOna9CgQbbEadCgQZozZ46uuuoqXXfdddq9e7fi4uJ03333GTZOeFfxvaW7du1yqX9WVpY2b96s2rVru/zkNG+rU6eO7erzH+3atUsjRowocb+ZM2c6bS/vL+3iwsC4cePc2q+kpKMkriz8jv8xU9Kamprq8HTC4qcWlqSk3x9nT1H0ZtEVMAI5FHCRs++PkhTnKykpKariwkxryZjvj5CQEKd5XPGDVcLDwytMccbZ+5+bm6sVK1Y49C3tomlubq5DPs/3N2B+hhadhg0bplOnTmnGjBlKT09X+/bttXHjRtvCmIcPH7a74jZt2jRZLBZNmzZNR48eVd26dW0JUrEFCxZo+vTpevDBB3Xy5Ek1atRI48eP14wZM8p9fCgfvlzwKKlokJmZqXXr1mnw4MElzvQoTyXN1CpJaUUPZ0g6Ls2dpLs0zmYXefP9T01NVatWrTx2vJIKVfv37+d3CD6DHAoo+/eHu0+m9eb3h5mLZmV9/0u6aFqS8v7+ZqY14FmGLyQ+ceLEEqeCb9q0ye51lSpVNHPmzEv+RRUaGqr4+HjFx8d7MEpUZL5c8CjpSld+fr4yMjLUtWvXCnE/cEkztUpSEa/UmZnZk+7MzEw1qG7R4vg5at68ean9c3Nz9NVXm3Tzzb0UHFz6Z/7QoUP6+yNPeawoB1QU5FCo7Ir/Xnc2w9WZrKwsrV27VlFRUS7Pdh8xYoTXvj/4/r40o76/uT0Q8CzDi07A5Sqp4FHSVYrScJXCc0r6GZR2pY6fgXvMnnTn5ORofMdA3fbrc9Kvru3Tobqk79a51DdC0viOgWWODwBQsUVERLj0j/38/HydPXtWXbp0qRAX7fj+vjSjvr/NtDwBYAYUneCzSrtKUdJVIq5SeA4/g/LToLpFHRr6K6KBX6l98wtCVHRDM3UJC1FAldL7B2f4q0F1iyfCdColJaV8nt7ImmAA4HMaVLcoOGO/dKz07zMVFKhmTpp0/HvJhdvTgjP2e/X7r5hZi2a++v1tpjW1ADOg6ASfZZb1kHwZP4PyM75joCK2jJe2lN43QFIvSdrn2rG9faWxpFtkS1tIvCQsJA4AlUPxTBuzfv9J5i6aufL05e+++07//ve/1b59ew0cOFC//fabrrjiCq1fv1579uzRww8/rBtuuEFS+T99GUD5oOgEn2WW9ZB8GT+D8vNa0gUNm/GmIlwo2OUXFOjrr79Wt27dFOBC0pqckqLX5g3X7Z4I1ImSbpG9nKJlRbg9k4VIAcC7zD7TxuxFs0ut6dmtWzcVFhZq2rRpGjhwoNauXavCwkJ98sknuu222zRr1ixFRUXpo48+0rx582xP0QTgeyg6AYDJ5eTkKD3Lqq9/zlJuraJS+2dl5eij736RX+PrXVsT4nih0rOsngjVLWYvWpp9IVKKZgAqOldm2vxRYWGh9uzZo/bt2zstcpT3TBuzF81Ks3XrVqWlpemdd96Rn5+fCgsLbdv8/Pw0depUde3aVVu3blWvXr0MiRGA91F0AgCTKy4MjBs3zq39Sip6lMRZgQElM/tCpGYvmgHwfaXNtPmz/Px81axZU7fddluFuGjhy09glqTjx49LUomxFrcX94NncNEIFQ1FJwAwOXev9B48eFAzZ87U7Nmz1aJFC4ftrKngvtTUVI89HchZoujt999Z/Lm5uVqxYoVD39J+f3Jzc7Vr1y67Nn5/AMDRpYpmzphtIeuGDRtKulgsu/HGGx227927164f3Ofs+7u0NTFLumhU0lMU+Q7H5aLohBJRJQfMoaSkddeuXZdMOmbOnOm0nZkq7klNTVWrVq3c3q+kpzeWZP/+/V5J+lJTU9WjQ2s1dGOx2esb+Gntwtku9z+eZdWWXftIWgGgErnpppsUFhamuXPnau3atXbbioqKFBsbq+bNm+umm24yJkC5d9Go+N9FKSkpquLCmpiSdws2l/r+vt6Fpxn/2bzJo5y28x2Oy0XRCSXi1grA3Hh6YPkoTlZLukL4Z1lZWVq7dq2ioqJcW1Pr/65Yemom1Z9lZmZqfMdAzeoV5JXjS9KsTXleix+A5/nihcfCwkJt3rxZW7ZsUbVq1XTzzTdX2MWrS3r/Syt6VLT339/fX/PmzdPQoUMVFRWlxx9/XLm5ufr222/14osvav369Vq9erVhPwezXzQqj+9vie9wXD6KTpDErRWALzL7Qtxm0qC6RR0a+ivChSuL+QUhKrqhmbqEhSigSun9gzP8vfrI65ycHL2WdEHt7priUsExNzdHX321STff3EvBwaX/4+bQoUN6Lekprz39EMDl8fbtORUhB1yzZo0ee+wx2y3ncXFxCgsL07x58zRkyBBDY3OmtAu/JRU9KuKF3yFDhmj16tV67LHH1KNHD1t78+bNtXr1akPff7NfNJK8+/RiyftPMEblQNEJZa7yl3RrTkm8VeUHAKOZ+ZHXKSkpSs+yashDrt8uJ0mat9yt7ixED1Q8Zc0BS1JSocrIHHDNmjUaOnSoBg4cqOXLl+vIkSNq0qSJXnjhBQ0dOtTwwoczvjZTeciQIRo8eLC++uorffrpp+rfv3+FmmkWERHhUrEuPz9fZ8+eVZcuXSrMRbv0LKtya7WSGrUvvXN+vs6FHJUatpNcjD83vciQJxjDt1B0gjIzM9WgukWL4+eoefPmpfYvy1Xuvz/yFNMyAfgsb15p9PZVRhaiByovX5jpcSmFhYV67LHHNHDgQK1du1aFhYU6c+aMIiMjbeOYPHmyBg8eXGEKIJJvzlT29/dXz549lZ2drZ49e1ao99uscnJyJMnhDpOSZGVlafPmzapdu7ZLn1/p4mcYuFwUnSDp4lX02359TvrVtf4dqkv6bp1Lfb19lR4AjObNK43evsrIQvQAzDzT41K2bt2qtLQ0vfPOO/Lz81NhYaFtm5+fn6ZOnaquXbtq69at6tWrl3GBwjANqlsUnLFfOubCwtsFBaqZkyYd/15y4aJRcMZ+r94eX7zG17hx49zar6TbYy+lvGcr++K6cpUZRSdIMvdVegAwkrevNBp1ldHXbu8AUPkcP35cktSmTRun24vbi/uh8jHz7fHlMVNZMma2stkfaEXRzB5FJygnJ0fpWVZ9/XOWcmsVldo/KytHH333i/waX+/aP5iOF3IvMACfVV5XGsv7KqMv3t4BoHJp2LChJGnv3r268cYbHbbv3bvXrh8qF7M/SMNXZir//MM3yjv9i12bX16eNvxnrkPfI0eOaNGiRZowYYKaNGnisN3vxI9K/nK/XVtQnWa6+i9dPBv0H1SGhzFcLopO8Nl/MAFAeSjpSmNJ9u7dq+joaC1btqzEq+9/5gsJBwCUt5tuuklhYWGaO3eu1q5da7etqKhIsbGxat68uW666SZjAoShfPVBGmaaqZyamqq3H75Zs3oFOWxrX8I+94+vLllXOF8WxknbrE15uveNH72SR6WmpqpHh9Zq6OQ2yutdeKLxn82bPMqh7XiWVVt27TN1HkjRCSwiCwCXoaQrjSVNrS6Nr06tBoDy5u/vr3nz5mno0KGKiorS448/rtzcXH377bd68cUXtX79eq1evZpFrSspX71oZKaZyidOnNBrSRdU44Zhdg+0unDhgs6cOePQP//CBe3bv1+tW7VSQKDjrYtXXnmlAv/Qfvz4cb2W9IZu99LDDDIzMzW+Y6DTopmnzNqUZ/oHclF0gs9MzQSAiqS09Qiio6OdtvN3KIDyZOaFlF0xZMgQrV69Wo899ph69Ohha2/evLlWr16tIUOGGBgdjFTSv4FKUlBQIOlikYrvac8onm322L/ecnPP7W719uZsM9ZGLh1FJ5TITFMzAaCi4e9QAH9UWFiozZs3a8uWLapWrZpuvvnmCjHDxswLKbtqyJAhGjx4sL766it9+umn6t+/f4V5/4HKzN07bgoLC7Vnzx61b9/e6ee3vO+4KWlt5EvH/4sO59d2KX5fWRvZ8KLTK6+8ohdffFHp6elq166dFixYoM6dO5fYPz4+XgsXLtThw4dVp04dDR06VLGxsapataqtz9GjR/XEE0/o008/VU5Ojq655hotXbpUnTp1Ko8h+QwzTc0EgIqGv0PhbeRQ5rFmzRo99thjtn+ExMXFKSwsTPPmzTN8pk1luUrv7++vnj17Kjs7Wz179qTghBKVdHt8cVtKSoqqOPn95/Z4911qtlm3bt0c2vLz81WzZk3ddtttFSKHKuvayO4y+9rIhhadVq1apZiYGC1atEiRkZGKj49Xv379tG/fPtWrV8+h/8qVKzVlyhQtWbJEXbt21f79+zV69GhZLBbFxcVJks6ePatu3brp5ptv1qeffqq6desqNTVVtWvXLu/hAQAAeAU5lHmsWbNGQ4cO1cCBA7V8+XIdOXJETZo00QsvvKChQ4cafotXepZVubVaSY3al945P1/nQo5KDdtJLvyDLze9yCeu0qNy4fZ4uMrsM7XKi6FFp7i4OI0bN05jxoyRJC1atEgbNmzQkiVLNGXKFIf+27dvV7du3TR8+HBJF38o99xzj3bs2GHr8/zzz6tp06ZaunSpre2Pi5IBAACYHTmUORQWFuqxxx7TwIEDtXbtWhUWFurMmTOKjIzU2rVrFRUVpcmTJ2vw4MHMvAEqCG6Ph6vMPlOrvLhddPr555919dVXX/aJL1y4oKSkJE2dOtXW5ufnp969e+ubb75xuk/Xrl21YsUK7dy5U507d9bPP/+sTz75RCNHjrT1+eijj9SvXz/deeed2rx5sxo3bqwHH3zwklPe8vLylJeXZ3t9/vx5SRd/KfLz8y93qD6n+D0x63tj9vgl84+B+I1F/MYifmMZFben8ieJHMpMNm/erLS0NC1fvlyFhYUOn5/HH39cPXr00FdffaWePXuWe3zFCyMXFBS49PNy9/Pv7vG9zVf+/iJ+7woICFDbtm0d2otvj7/hhhtKLBhU5LGZ5f2/FLOPwVfid5fbRadrrrlGPXv21N///ncNHTrUbh0Ad5w+fVqFhYWqX7++XXv9+vVLfMT08OHDdfr0aXXv3l1Wq1UFBQWaMGGCnnzySVufn3/+WQsXLlRMTIyefPJJfffdd3r44YcVGBhY4lTI2NhYzZ4926H9888/577cS0hISDA6hMti9vgl84+B+I1F/MYifmPk5OQYcl5P5U8SOZSZbNlycXXuI0eO2D3+u/jzk5ubK0n69NNPlZ2dXe7x/b//9/8kSW+99Za2bdtWav8LFy7o5MmTSk5OtnsseUmOHDkiSdq2bZuOHz9+ecF6kFn//ipG/MYifuOZfQxmjb+sOZTFarW6daP1nj17tHTpUr3zzju6cOGChg0bpr///e+XXLjSmWPHjqlx48bavn27unTpYmv/5z//qc2bN9tN9y62adMm3X333Xr22WcVGRmpAwcOaNKkSRo3bpymT58uSQoMDFSnTp20ffv/HqP48MMP67vvvivx6p+zq3RNmzbV6dOnVaNGDbfGVRnk5+crISFBffr0MeW0QLPHL5l/DMRvLOI3FvEb68yZM2rYsKHOnTtXrt/xnsqfJHIoM9m8ebP69OmjrVu3KjIy0uHz8+2336pHjx5KSEgwZKbTkiVLNGHCBK+f56effqoQa5KY/e8v4jcW8RvP7GMwe/xlzaHcnunUvn17vfTSS5o3b54++ugjvfnmm+revbtatWql++67TyNHjlTdunVLPU6dOnXk7++vEydO2LWfOHFCDRo0cLrP9OnTNXLkSNt9k23btlV2drbuv/9+PfXUU/Lz81PDhg117bXX2u0XERGhDz74oMRYgoKCFBQU5NAeEBBgyl+G8mL298fs8UvmHwPxG4v4jUX8xjAqZk/lTxI5lJncfPPNCgsL0wsvvKC1a9fa2gMCAuTv768XX3xRzZs3180332zImk533HGH/P39XX7q1t69exUdHa1ly5apTZs2Lp2jIi6Ca/bfT+I3FvEbz+xjMGv8ZY25zAuJV6lSRUOGDNGAAQP06quvaurUqZo8ebKefPJJ3XXXXXr++efVsGHDEvcPDAxUx44dlZiYaFv1vaioSImJiZo4caLTfXJycuTn52fXVvwFXTxhq1u3btq3b59dn/3796tZs2ZlHSoAAIBHXG7+JJFDmYm/v7/mzZunoUOHKioqSo8//rhyc3P17bff6sUXX9T69eu1evVqwxYRv9QiuM4Ur9EUHh7OU7oAAC7xK72Lc//973/14IMPqmHDhoqLi9PkyZN18OBBJSQk6NixYxo8eHCpx4iJidEbb7yhZcuWKTk5WQ888ICys7NtT2IZNWqU3SKZgwYN0sKFC/Xuu+/q0KFDSkhI0PTp0zVo0CDbl/Wjjz6qb7/9VnPnztWBAwe0cuVKvf7663rooYfKOlQAAACP8ET+JJFDmcmQIUO0evVq/fjjj+rRo4fuuece9ejRQ3v37tXq1as1ZMgQo0MEAMBr3J7pFBcXp6VLl2rfvn267bbb9NZbb+m2226zXT1r3ry53nzzTYWFhZV6rGHDhunUqVOaMWOG0tPT1b59e23cuNG2MObhw4ftrspNmzZNFotF06ZN09GjR1W3bl0NGjRIc+bMsfW54YYb9OGHH2rq1Kl6+umn1bx5c8XHx+vee+91d6gAAAAe4cn8SSKHMpshQ4Zo8ODB+uqrr/Tpp5+qf//+ht1SBwBAeXK76LRw4ULdd999Gj16dInTv+vVq6fFixe7dLyJEyeWOBV806ZNdq+rVKmimTNnaubMmZc85sCBAzVw4ECXzg8AAOBtns6fJHIos/H391fPnj2VnZ2tnj17UnACAFQKbhedUlNTS+1zqUfrAgAAVDbkTwAAoDJye02npUuX6v3333dof//997Vs2TKPBAUAAOBLyJ8AAEBl5PZMp9jYWL322msO7fXq1dP999/PFToAAIA/IX/yXampqcrMzLRry83NVVpamkPfwsJC7dmzR+fOnXN6e11YWJiCg4Pt2kJDQ9WyZUuPxgwAKF+FhYXavHmztmzZomrVqlWqdf3cLjodPnxYzZs3d2hv1qyZDh8+7JGgAAAAfAn5k29KTU1Vq1atvH6e/fv3U3gCAJNas2aNHnvsMdvFiLi4OIWFhWnevHmV4gmmbhed6tWrpx9++MHh6Srff/+9rrzySk/FBQAA4DPIn3xTZmamGlS3aHH8HLuiYl5eno4dO+bQv6ioSPv27VPr1q3tni5YrFGjRgoKCrK9PnTokP7+yFMOM6kAAOawZs0aDR06VAMHDtTy5ct15MgRNWnSRC+88IKGDh2q1atX+3zhye2i0z333KOHH35YoaGh6tGjhyRp8+bNmjRpku6++26PBwgAAGB25E++a3zHQN3263PSr/bt7UvoP7C6pKMlbPzTMSL+7/gAAPMpLCzUY489poEDB2rt2rUqLCzUmTNnFBkZqbVr1yoqKkqTJ0/W4MGDffpWO7eLTs8884zS0tJ06623qkqVi7sXFRVp1KhRmjt3rscDBAAAMDvyJ9/1WtIFDZvxpiLCw0vtm19QoK+//lrdunVTQJXS0/DklBS9Nm+4bvdEoG7IyclRSkqKQ3txW0pKiu33+I/Cw8MVEhLi9fgAwAy2bt2qtLQ0vfPOO/Lz81NhYaFtm5+fn6ZOnaquXbtq69at6tWrl3GBepnbRafAwECtWrVKzzzzjL7//nsFBwerbdu2atasmTfiAwAAMD3yJ9+Uk5Oj9Cyrvv45S7m1imztl15I/Bcdzq/t0kLiyccLlZ5l9Ursl5KSkqKOHTuWuL2khe+TkpLUoUMHb4UFAKZy/PhxSVKbNm2cbi9uL+7nq9wuOhVr1apVuSycCAAA4CvIn3xL8cyfcePGefU8oaGhXj3+n4WHhyspKcmhPTMzU+vWrdPgwYOdxhTuwmwvAKgsGjZsKEnau3evbrzxRofte/futevnq8pUdDpy5Ig++ugjHT58WBcuXLDbFhcX55HAAAAAfAn5k++JioqS5Hhb2aVnOu1R+/btXZrpJF0sOJX3k+tCQkKczljKz89XRkaGunbtqoCAgHKNCQDM5qabblJYWJjmzp2rtWvX2m0rKipSbGysmjdvrptuusmYAMuJ20WnxMRE3X777br66quVkpKiNm3aKC0tTVarlem0AAAATpA/+aY6depo7NixTrd169bNoS0/P181a9bUbbfdRtEGAHycv7+/5s2bp6FDhyoqKkqPP/64cnNz9e233+rFF1/U+vXrtXr1ap9eRFySHJ/VWoqpU6dq8uTJ+vHHH1W1alV98MEH+vXXX9WzZ0/deeed3ogRAADA1MifAACofIYMGaLVq1frxx9/VI8ePXTPPfeoR48e2rt3r1avXq0hQ4YYHaLXuV10Sk5O1qhRoyRJVapUUW5urqpXr66nn35azz//vMcDBAAAMDvyJwAAKqchQ4bowIEDSkhIUExMjBISEpSamlopCk5SGYpO1apVs61D0LBhQx08eNC27fTp056LDAAAwEeQPwEAUHn5+/urZ8+e6tGjh3r27Onzt9T9kdtrOt14443atm2bIiIidNttt+mxxx7Tjz/+qDVr1jhdkR0AAKCyI38CAACVkdtFp7i4OGVlZUmSZs+eraysLK1atUotW7bkySsAAABOkD8BAIDKyK2iU2FhoY4cOaK//OUvki5OFV+0aJFXAgMAAPAF5E8AAKCycmtNJ39/f/Xt21dnz571VjwAAAA+hfwJAABUVm4vJN6mTRv9/PPP3ogFAADAJ5E/AQCAysjtotOzzz6ryZMna/369Tp+/LjOnz9v96csXnnlFYWFhalq1aqKjIzUzp07L9k/Pj5erVu3VnBwsJo2bapHH31Uv//+u9O+zz33nCwWix555JEyxQYAAHC5yJ8AAEBl5PZC4rfddpsk6fbbb5fFYrG1W61WWSwWFRYWunW8VatWKSYmRosWLVJkZKTi4+PVr18/7du3T/Xq1XPov3LlSk2ZMkVLlixR165dtX//fo0ePVoWi8VhIc7vvvtOr732mm0NBQAAACOQPwEAgMrI7aLTV1995dEA4uLiNG7cOI0ZM0aStGjRIm3YsEFLlizRlClTHPpv375d3bp10/DhwyVJYWFhuueee7Rjxw67fllZWbr33nv1xhtv6Nlnn/VozAAAAO4gfwIAAJWR20Wnnj17euzkFy5cUFJSkqZOnWpr8/PzU+/evfXNN9843adr165asWKFdu7cqc6dO+vnn3/WJ598opEjR9r1e+ihhzRgwAD17t271KQpLy9PeXl5ttfF09zz8/OVn59f1uH5rOL3xKzvjdnjl8w/BuI3FvEbi/iNZVTcvpg/SeRQ7vKVzw/xG4P4jUX8xjP7GHwlfne5XXTasmXLJbf36NHD5WOdPn1ahYWFql+/vl17/fr1lZKS4nSf4cOH6/Tp0+revbusVqsKCgo0YcIEPfnkk7Y+7777rnbt2qXvvvvOpThiY2M1e/Zsh/bPP/9cISEhLo+nsklISDA6hMti9vgl84+B+I1F/MYifmPk5OQYcl5fzJ8kcqiyMuvnpxjxG4v4jUX8xjP7GMwaf1lzKLeLTr169XJo++PaBO6uSeCuTZs2ae7cuXr11VcVGRmpAwcOaNKkSXrmmWc0ffp0/frrr5o0aZISEhJUtWpVl445depUxcTE2F6fP39eTZs2Vd++fVWjRg1vDcW08vPzlZCQoD59+iggIMDocNxm9vgl84+B+I1F/MYifmOdOXPGkPP6Yv4kkUO5y+yfH+I3FvEbi/iNZ/YxmD3+suZQbhedzp49a/c6Pz9fu3fv1vTp0zVnzhy3jlWnTh35+/vrxIkTdu0nTpxQgwYNnO4zffp0jRw5UmPHjpUktW3bVtnZ2br//vv11FNPKSkpSSdPnlSHDh1s+xQWFmrLli16+eWXlZeXJ39/f7tjBgUFKSgoyOFcAQEBpvxlKC9mf3/MHr9k/jEQv7GI31jEbwyjYvbF/Ekihyors78/xG8s4jcW8RvP7GMwa/xljdntolPNmjUd2vr06aPAwEDFxMQoKSnJ5WMFBgaqY8eOSkxMVFRUlCSpqKhIiYmJmjhxotN9cnJy5OfnZ9dWnARZrVbdeuut+vHHH+22jxkzRuHh4XriiSecJkwAAADeRP4EAAAqI7eLTiWpX7++9u3b5/Z+MTExio6OVqdOndS5c2fFx8crOzvb9jSWUaNGqXHjxoqNjZUkDRo0SHFxcbr++utt08OnT5+uQYMGyd/fX6GhoWrTpo3dOapVq6Yrr7zSoR0AAMBI5E8AAMCXuV10+uGHH+xeW61WHT9+XM8995zat2/vdgDDhg3TqVOnNGPGDKWnp6t9+/bauHGjbXHMw4cP212ZmzZtmiwWi6ZNm6ajR4+qbt26GjRokNtT0wEAAMoL+RMAAKiM3C46tW/fXhaLRVar1a79xhtv1JIlS8oUxMSJE0ucDr5p0ya711WqVNHMmTM1c+ZMl4//52MAAACUJ/InAABQGblddDp06JDdaz8/P9WtW9etJ50AAABUJuRPAACgMnK76NSsWTNvxAEAAOCzyJ8AAEBl5Fd6F3sPP/yw/v3vfzu0v/zyy3rkkUc8ERMAAIBPIX8CAACVkdtFpw8++EDdunVzaO/atatWr17tkaAAAAB8CfkTCgsLtXnzZm3ZskWbN29WYWGh0SEBAOB1bhedzpw5o5o1azq016hRQ6dPn/ZIUAAAAL6E/KlyW7Nmja655hr16dNHcXFx6tOnj6655hqtWbPG6NAAAPAqt4tO11xzjTZu3OjQ/umnn+rqq6/2SFAAAAC+hPyp8lqzZo2GDh2qtm3bauvWrXrnnXe0detWtW3bVkOHDqXwBADwaW4vJB4TE6OJEyfq1KlTuuWWWyRJiYmJmjdvnuLj4z0dHwAAgOmRP1VOhYWFeuyxxzRw4ECtXbtWhYWFOnPmjCIjI7V27VpFRUVp8uTJGjx4sPz9/Y0OFwAAj3O76HTfffcpLy9Pc+bM0TPPPCNJCgsL08KFCzVq1CiPBwgAAGB25E+V09atW5WWlqZ33nlHfn5+dus4+fn5aerUqeratau2bt2qXr16GRcoAABe4nbRSZIeeOABPfDAAzp16pSCg4NVvXp1T8cFAADgU8ifKp/jx49Lktq0aeN0e3F7cT8AAHyN22s6HTp0SKmpqZKkunXr2hKm1NRUpaWleTQ4AAAAX0D+VDk1bNhQkrR3716n24vbi/sBAOBr3C46jR49Wtu3b3do37Fjh0aPHu2JmAAAAHwK+VPldNNNNyksLExz585VUVGR3baioiLFxsaqefPmuummmwyKEAAA73K76LR7925169bNof3GG2/Unj17PBETAACATyF/qpz8/f01b948rV+/XlFRUfr222+Vm5urb7/9VlFRUVq/fr3+9a9/sYg4AMBnub2mk8ViUWZmpkP7uXPn7BZHBAAAwEXkT5XXkCFDtHr1aj322GPq0aOHrb158+ZavXq1hgwZYmB0AAB4l9sznXr06KHY2Fi7BKmwsFCxsbHq3r27R4MDAADwBeRPlduQIUN04MABJSQkKCYmRgkJCUpNTaXgBADweW7PdHr++efVo0cPtW7d2nb/+datW3X+/Hl9+eWXHg8QAADA7Mif4O/vr549eyo7O1s9e/bkljoAQKXg9kyna6+9Vj/88IPuuusunTx5UpmZmRo1apRSUlJKfBwsAABAZUb+BAAAKiO3ZzpJUqNGjTR37ly7toyMDL388suaOHGiRwIDAADwJeRPAACgsnF7ptOfJSYmavjw4WrYsKFmzpzpiZgAAAB8GvkTAACoDMpUdPr111/19NNPq3nz5urbt68k6cMPP1R6erpHgwMAAPAV5E8AAKCycbnolJ+fr/fff1/9+vVT69attWfPHr344ovy8/PTtGnT9Ne//lUBAQHejBUAAMBUyJ8AAEBl5nLRqXHjxlqwYIHuuOMOHT16VGvWrNHQoUM9EsQrr7yisLAwVa1aVZGRkdq5c+cl+8fHx6t169YKDg5W06ZN9eijj+r333+3bY+NjdUNN9yg0NBQ1atXT1FRUdq3b59HYgUAAHAV+RMAAKjMXC46FRQUyGKxyGKxePQRr6tWrVJMTIxmzpypXbt2qV27durXr59OnjzptP/KlSs1ZcoUzZw5U8nJyVq8eLFWrVqlJ5980tZn8+bNeuihh/Ttt98qISFB+fn56tu3r7Kzsz0WNwAAQGnInwAAQGXm8tPrjh07pg8++ECLFy/WpEmT1L9/f40YMUIWi+WyAoiLi9O4ceM0ZswYSdKiRYu0YcMGLVmyRFOmTHHov337dnXr1k3Dhw+XJIWFhemee+7Rjh07bH02btxot8+bb76pevXqKSkpST169HA4Zl5envLy8myvz58/L+nilPj8/PzLGp8vKn5PzPremD1+yfxjIH5jEb+xiN9Y5R23L+dPEjmUu3zl80P8xiB+YxG/8cw+Bl+J310Wq9VqdXengwcPaunSpVq2bJmOHj2qe+65R6NHj9Ytt9zi1lW8CxcuKCQkRKtXr1ZUVJStPTo6WhkZGVq3bp3DPitXrtSDDz6ozz//XJ07d9bPP/+sAQMGaOTIkXZX6/7owIEDatmypX788Ue1adPGYfusWbM0e/Zsp+cKCQlxeTwAAKBiy8nJ0fDhw3Xu3DnVqFGjXM/ta/mTRA4FAEBlUdYcqkxFp2JFRUX67LPPtHjxYn388ccKDQ3V6dOnXd7/2LFjaty4sbZv364uXbrY2v/5z39q8+bNdlff/ujf//63Jk+eLKvVqoKCAk2YMEELFy4sMcbbb79dGRkZ2rZtm9M+zq7SNW3aVKdPny73hNQM8vPzlZCQoD59+phy8VOzxy+ZfwzEbyziNxbxG+vMmTNq2LChIUWnYr6SP0nkUO4y++eH+I1F/MYifuOZfQxmj7+sOZTLt9c54+fnp/79+6t///46deqUli9ffjmHc8mmTZs0d+5cvfrqq4qMjNSBAwc0adIkPfPMM5o+fbpD/4ceekh79+69ZMIUFBSkoKAgh/aAgABT/jKUF7O/P2aPXzL/GIjfWMRvLOI3RkWI2VfyJ4kcqqzM/v4Qv7GI31jEbzyzj8Gs8Zc15ssqOv1R3bp1FRMT49Y+derUkb+/v06cOGHXfuLECTVo0MDpPtOnT9fIkSM1duxYSVLbtm2VnZ2t+++/X0899ZT8/P63NvrEiRO1fv16bdmyRU2aNHFzRAAAAN5F/gQAAHyZy0+v84bAwEB17NhRiYmJtraioiIlJibaTRf/o5ycHLvESJJtHYTiOwWtVqsmTpyoDz/8UF9++aWaN2/upREAAACUL/InAABgFh6b6VRWMTExio6OVqdOndS5c2fFx8crOzvb9jSWUaNGqXHjxoqNjZUkDRo0SHFxcbr++utt08OnT5+uQYMG2ZKnhx56SCtXrtS6desUGhqq9PR0SVLNmjUVHBxszEABAAA8hPwJAACYgeFFp2HDhunUqVOaMWOG0tPT1b59e23cuFH169eXJB0+fNjuyty0adNksVg0bdo0HT16VHXr1tWgQYM0Z84cW5/iRTF79epld66lS5dq9OjRXh8TAACAN5E/AQAAMzC86CRdXDtg4sSJTrdt2rTJ7nWVKlU0c+ZMzZw5s8TjXcYD+QAAAEyB/AkAAFR0HlvTad26dXrrrbc8dTgAAACfR/4EAAB8mceKTk888YRtHQEAAACUjvwJAAD4Mo/dXpeSkuKpQwEAAFQK5E8AAMCXuT3T6fDhwyXe83/48OHLDggAAMDXkD8BAIDKyO2iU/PmzXXq1CmH9jNnzqh58+YeCQoAAMCXkD8BAIDKyO2ik9VqlcVicWjPyspS1apVPRIUAACALyF/AgAAlZHLazrFxMRIkiwWi6ZPn66QkBDbtsLCQu3YsUPt27f3eIAAAABmRf4EAAAqM5eLTrt375Z08Urdjz/+qMDAQNu2wMBAtWvXTpMnT/Z8hAAAACZF/gQAACozl4tOX331lSRpzJgxeumll1SjRg2vBQUAAOALyJ8AAEBl5nLRqdjSpUu9EQcAAIDPIn8CAACVkdtFJ0n673//q/fee0+HDx/WhQsX7LatWbPGI4EBAAD4EvInAABQ2bj99Lp3331XXbt2VXJysj788EPl5+frp59+0pdffqmaNWt6I0YAAABTI38CAACVkdtFp7lz52r+/Pn6+OOPFRgYqJdeekkpKSm66667dNVVV3kjRgAAAFMjfwIAAJWR20WngwcPasCAAZIuPnUlOztbFotFjz76qF5//XWPBwgAAGB25E8AAKAycrvoVLt2bWVmZkqSGjdurL1790qSMjIylJOT49noAAAAfAD5EwAAqIzcXki8R48eSkhIUNu2bXXnnXdq0qRJ+vLLL5WQkKBbb73VGzECAACYGvkTAACojNwuOr388sv6/fffJUlPPfWUAgICtH37dt1xxx2aNm2axwMEAAAwO/InAABQGblddLriiits/+/n56cpU6Z4NCAAAABfQ/4EAAAqI5fXdDp27JgmT56s8+fPO2w7d+6cHn/8cZ04ccKjwQEAAJgZ+RMAAKjMXC46xcXF6fz586pRo4bDtpo1ayozM1NxcXFlCuKVV15RWFiYqlatqsjISO3cufOS/ePj49W6dWsFBweradOmevTRR21T1st6TAAAAE8jfwIAAJWZy0WnjRs3atSoUSVuHzVqlNavX+92AKtWrVJMTIxmzpypXbt2qV27durXr59OnjzptP/KlSs1ZcoUzZw5U8nJyVq8eLFWrVqlJ598sszHBAAA8AbyJwAAUJm5vKbToUOHdNVVV5W4vUmTJkpLS3M7gLi4OI0bN05jxoyRJC1atEgbNmzQkiVLnK53sH37dnXr1k3Dhw+XJIWFhemee+7Rjh07ynzMvLw85eXl2V4XT4HPz89Xfn6+22PydcXviVnfG7PHL5l/DMRvLOI3FvEbq7zj9uX8SSKHcpevfH6I3xjEbyziN57Zx+Ar8bvL5aJTcHCw0tLSSkyc0tLSFBwc7NbJL1y4oKSkJE2dOtXW5ufnp969e+ubb75xuk/Xrl21YsUK7dy5U507d9bPP/+sTz75RCNHjizzMWNjYzV79myH9s8//1whISFujakySUhIMDqEy2L2+CXzj4H4jUX8xiJ+Y+Tk5JTr+Xw5f5K8l0Pl5eXpyJEjDu0XLlzQyZMnVa9ePQUGBjpsb9KkiYKCgsp83vJi1s9PMeI3FvEbi/iNZ/YxmDX+suZQLhedIiMjtXz5cvXo0cPp9rfeekudO3d26+SnT59WYWGh6tevb9dev359paSkON1n+PDhOn36tLp37y6r1aqCggJNmDDBNj28LMecOnWqYmJibK/Pnz+vpk2bqm/fvk7XYKjs8vPzlZCQoD59+iggIMDocNxm9vgl84+B+I1F/MYifmOdOXOmXM/ny/mT5L0cavfu3Ro2bJjb++3YsUPXX399mc/rbWb//BC/sYjfWMRvPLOPwezxlzWHcrnoNHnyZPXp00c1a9bU448/bktKTpw4oRdeeEFvvvmmPv/88zIF4Y5NmzZp7ty5evXVVxUZGakDBw5o0qRJeuaZZzR9+vQyHTMoKMjpVbGAgABT/jKUF7O/P2aPXzL/GIjfWMRvLOI3RnnH7Mv5k+S9HKpNmzZKSkpyaE9OTtaIESO0YsUKRUREOGwPDw83xe+lWT8/xYjfWMRvLOI3ntnHYNb4yxqzy0Wnm2++Wa+88oomTZqk+fPnq0aNGrJYLDp37pwCAgK0YMEC3XLLLW6dvE6dOvL393d4VPCJEyfUoEEDp/tMnz5dI0eO1NixYyVJbdu2VXZ2tu6//3499dRTZTomAACAN5A/lU1ISIg6dOhQ4vaIiIhLbgcAABWDy0+vk6Tx48fr4MGD+te//qXhw4fr7rvv1rx583TgwAE98MADbp88MDBQHTt2VGJioq2tqKhIiYmJ6tKli9N9cnJy5OdnH7a/v78kyWq1lumYAAAA3kL+BAAAKiuXZzoVa9y4sR599FGPBRATE6Po6Gh16tRJnTt3Vnx8vLKzs21PThk1apQaN26s2NhYSdKgQYMUFxen66+/3jY9fPr06Ro0aJAteSrtmAAAAOWJT4h8NAAAHEJJREFU/AkAAFRGbhedPG3YsGE6deqUZsyYofT0dLVv314bN260rXlw+PBhuytz06ZNk8Vi0bRp03T06FHVrVtXgwYN0pw5c1w+JgAAgJmRPwEAADMwvOgkSRMnTtTEiROdbtu0aZPd6ypVqmjmzJmaOXNmmY8JAABgduRPAACgonNrTScAAAAAAADAFRSdAAAAAAAA4HFuF52uvvpqnTlzxqE9IyNDV199tUeCAgAA8CXkTwAAoDJyu+iUlpamwsJCh/a8vDwdPXrUI0EBAAD4EvInAABQGbm8kPhHH31k+//PPvtMNWvWtL0uLCxUYmKiwsLCPBocAACAmZE/AQCAyszlolNUVJQkyWKxKDo62m5bQECAwsLCNG/ePI8GBwAAYGbkTwAAoDJzuehUVFQkSWrevLm+++471alTx2tBAQAA+ALyJwAAUJm5XHQqdujQIYe2jIwM1apVyxPxAAAA+BzyJwAAUBm5vZD4888/r1WrVtle33nnnbriiivUuHFjff/99x4NDgAAwBeQPwEAgMrI7aLTokWL1LRpU0lSQkKCvvjiC23cuFH9+/fX448/7vEAAQAAzI78CQAAVEZu316Xnp5uS5rWr1+vu+66S3379lVYWJgiIyM9HiAAAIDZkT8BAIDKyO2iU+3atfXrr7+qadOm2rhxo5599llJktVqVWFhoccDBAAAMDvyp5KlpqYqMzPTpb7Jycl2/3VFaGioWrZsWabYAADA5XG76DRkyBANHz5cLVu21JkzZ9S/f39J0u7du3XNNdd4PEAAAACzI39yLjU1Va1atXJ7vxEjRrjVf//+/RSeAAAwgNtFp/nz5yssLEy//vqrXnjhBVWvXl2SdPz4cT344IMeDxAAAMDsyJ+cK57htGLFCkVERJTaPzc3V2lpaQoLC1NwcHCp/ZOTkzVixAiXZ1IBAADPcrvoFBAQoMmTJzu0P/roox4JCAAAwNeQP11aRESEOnTo4FLfbt26eTkaAADgKW4/vU6Sli9fru7du6tRo0b65ZdfJEnx8fFat26dR4MDAADwFeRPAACgsnG76LRw4ULFxMSof//+ysjIsC1+WatWLcXHx3s6PgAAANMjfwIAAJWR27fXLViwQG+88YaioqL03HPP2do7derkdNo4AABAZUf+VLIG1S0KztgvHSvTBPxLCs7YrwbVLR4/LgAAcI3bRadDhw7p+uuvd2gPCgpSdna2R4ICAADwJeRPJRvfMVARW8ZLWzx/7Ij/Oz4AADCG25eUmjdvrj179ji0b9y40aWnjjjzyiuvKCwsTFWrVlVkZKR27txZYt9evXrJYrE4/BkwYICtT1ZWliZOnKgmTZooODhY1157rRYtWlSm2AAAAC4X+VPJXku6oOQer0n3b/b4n+Qer+m1pAteHwMAAHDO5ZlOTz/9tCZPnqyYmBg99NBD+v3332W1WrVz50698847io2N1X/+8x+3A1i1apViYmK0aNEiRUZGKj4+Xv369dO+fftUr149h/5r1qzRhQv/Sx7OnDmjdu3a6c4777S1xcTE6Msvv9SKFSsUFhamzz//XA8++KAaNWqk22+/3e0YAQAAyoL8qXTpWVbl1molNWrv8WPnphcpPcvq8eMCAADXuDzTafbs2crKytLYsWP1/PPPa9q0acrJydHw4cO1cOFCvfTSS7r77rvdDiAuLk7jxo3TmDFjbFfUQkJCtGTJEqf9r7jiCjVo0MD2JyEhQSEhIXZJ0/bt2xUdHa1evXopLCxM999/v9q1a3fJK4AAAACeRv4EAAAqM5dnOlmt/7tKdO+99+ree+9VTk6OsrKynF5Rc8WFCxeUlJSkqVOn2tr8/PzUu3dvffPNNy4dY/Hixbr77rtVrVo1W1vXrl310Ucf6b777lOjRo20adMm7d+/X/Pnz3d6jLy8POXl5dlenz9/XpKUn5+v/Pz8sgzNpxW/J2Z9b8wev2T+MRC/sYjfWMRvrPKO25fzJ+nyc6iCggLbf73xs/H28d3lK58f4jcG8RuL+I1n9jH4SvzucmshcYvF/ukfISEhCgkJKdOJJen06dMqLCxU/fr17drr16+vlJSUUvffuXOn9u7dq8WLF9u1L1iwQPfff7+aNGmiKlWqyM/PT2+88YZ69Ojh9DixsbGaPXu2Q/vnn39+WePzdQkJCUaHcFnMHr9k/jEQv7GI31jEb4ycnJxyP6ev5k/S5edQBw8elCRt27ZNx48fL7W/u7x9/LIy6+enGPEbi/iNRfzGM/sYzBp/WXMot4pOrVq1ckic/uy3334rUyBlsXjxYrVt21adO3e2a1+wYIG+/fZbffTRR2rWrJm2bNmihx56SI0aNVLv3r0djjN16lTFxMTYXp8/f15NmzZV3759VaNGDa+Pw2zy8/OVkJCgPn36KCAgwOhw3Gb2+CXzj4H4jUX8xiJ+Y505c6bcz+mr+ZN0+TnU7t27JUndu3d3+nS/y+Xt47vL7J8f4jcW8RuL+I1n9jGYPf6y5lBuFZ1mz56tmjVrlulEztSpU0f+/v46ceKEXfuJEyfUoEGDS+6bnZ2td999V08//bRde25urp588kl9+OGHtiey/OUvf9GePXv0r3/9y2nSFBQUpKCgIIf2gIAAU/4ylBezvz9mj18y/xiI31jEbyziN4YRMftq/iRdfg5VpUoV23+98bPx9vHLyqyfn2LEbyziNxbxG8/sYzBr/GWN2a2i0913313m9QecCQwMVMeOHZWYmKioqChJUlFRkRITEzVx4sRL7vv+++8rLy9PI0aMsGsvXkPAz89+jXR/f38VFRV5LHYAAABXkD8BAIDKyuWiU2nTwssqJiZG0dHR6tSpkzp37qz4+HhlZ2drzJgxkqRRo0apcePGio2Ntdtv8eLFioqK0pVXXmnXXqNGDfXs2VOPP/64goOD1axZM23evFlvvfWW4uLivDIGAAAAZ8ifAABAZVamp9d50rBhw3Tq1CnNmDFD6enpat++vTZu3GhbHPPw4cMOV9327dunbdu26fPPP3d6zHfffVdTp07Vvffeq99++03NmjXTnDlzNGHCBK+MAQAAwBnyJwAAUJm5XHTy5tTqiRMnljgdfNOmTQ5trVu3vmQS16BBAy1dutRT4QEAAJQJ+RMAAKjM3FrTCQAAAPCU4scv79q1y6X+ubm5SktLU1hYmIKDg0vtn5ycfFnxAQCAy0PRCQAAAIZISUmRJI0bN86r5wkNDfXq8QEAgHMUnQAAAGCI4qfvhYeHKyQkpNT+ycnJGjFihFasWKGIiAiXzhEaGqqWLVteTpgAAKCMKDoBAADAEHXq1NHYsWPd3i8iIkIdOnTwQkQAAMCT/ErvAgAAAAAAALiHohMAAAAAAAA8jqITAAAAAAAAPI6iEwAAAAAAADyOohMAAAAAAAA8jqITAAAAAAAAPK6K0QEAAAAAf5STk6OUlBSH9uTkZLv//ll4eLhCQkK8GhsAAHAdRScAAABUKCkpKerYsWOJ20eMGOG0PSkpSR06dPBWWAAAwE0UnQAAAFChhIeHKykpyaE9NzdXaWlpCgsLU3BwsNP9AABAxUHRCQAAABVKSEhIiTOWunXrVs7RAACAsmIhcQAAAAAAAHgcRScAAAAAAAB4HEUnAAAAAAAAeBxFJwAAAAAAAHgcRScAAAAAAAB4XIUoOr3yyisKCwtT1apVFRkZqZ07d5bYt1evXrJYLA5/BgwYYNcvOTlZt99+u2rWrKlq1arphhtu0OHDh709FAAAgHJB/gQAACo6w4tOq1atUkxMjGbOnKldu3apXbt26tevn06ePOm0/5o1a3T8+HHbn71798rf31933nmnrc/BgwfVvXt3hYeHa9OmTfrhhx80ffp0Va1atbyGBQAA4DXkTwAAwAyqGB1AXFycxo0bpzFjxkiSFi1apA0bNmjJkiWaMmWKQ/8rrrjC7vW7776rkJAQu6Tpqaee0m233aYXXnjB1taiRQsvjQAAAKB8kT8BAAAzMLTodOHCBSUlJWnq1Km2Nj8/P/Xu3VvffPONS8dYvHix7r77blWrVk2SVFRUpA0bNuif//yn+vXrp927d6t58+aaOnWqoqKinB4jLy9PeXl5ttfnz5+XJOXn5ys/P7+Mo/Ndxe+JWd8bs8cvmX8MxG8s4jcW8RvLrHH/UUXJnyRyKHf5yueH+I1B/MYifuOZfQy+Er+7LFar1erhWFx27NgxNW7cWNu3b1eXLl1s7f/85z+1efNm7dix45L779y5U5GRkdqxY4c6d+4sSUpPT1fDhg0VEhKiZ599VjfffLM2btyoJ598Ul999ZV69uzpcJxZs2Zp9uzZDu0rV65USEjIZY4SAABUFDk5ORo+fLjOnTunGjVqGB1OmVSU/EkihwIAoLIoaw5l+O11l2Px4sVq27atLWGSLl6pk6TBgwfr0UcflSS1b99e27dv16JFi5wmTVOnTlVMTIzt9fnz59W0aVP17dvXtAmpN+Xn5yshIUF9+vRRQECA0eG4zezxS+YfA/Ebi/iNRfzGOnPmjNEhGM5T+ZNEDuUus39+iN9YxG8s4jee2cdg9vjLmkMZWnSqU6eO/P39deLECbv2EydOqEGDBpfcNzs7W++++66efvpph2NWqVJF1157rV17RESEtm3b5vRYQUFBCgoKcmgPCAgw5S9DeTH7+2P2+CXzj4H4jUX8xiJ+Y5gx5j+rKPmTRA5VVmZ/f4jfWMRvLOI3ntnHYNb4yxqzoUWnwMBAdezYUYmJibb1AoqKipSYmKiJEydect/3339feXl5GjFihMMxb7jhBu3bt8+uff/+/WrWrJlH4y9NTk6OUlJSHNpzc3OVlpamsLAwBQcHO2wPDw9nSjoAAHDK1/MnAADgOwy/vS4mJkbR0dHq1KmTOnfurPj4eGVnZ9uexjJq1Cg1btxYsbGxdvstXrxYUVFRuvLKKx2O+fjjj2vYsGHq0aOHbU2Cjz/+WJs2bfLaOFJTU5WZmWnXlpyc7JDUuWLFihWKiIiwawsNDVXLli0vK0YAAOAbfCV/AgAAvs3wotOwYcN06tQpzZgxQ+np6Wrfvr02btyo+vXrS5IOHz4sPz8/u3327dunbdu26fPPP3d6zL/97W9atGiRYmNj9fDDD6t169b64IMP1L17d6+MITU1VT06tFbD6haHbdc38HOyx6XNmzzKoe14llVbdu2j8AQAAHwifwIAAL7P8KKTJE2cOLHE6eDOrq61bt1apT1077777tN9993nifBKdeLECY3vGKhZvRzXNPCUWZvyHGZSAQCAysvs+RMAAPB9FaLoVNEUJ2Tnz593qf/u3bu18L95+jD5gtdiSs+2arDF4nJM3pSfn6+cnBydP3/elAugmT1+yfxjIH5jEb+xiN9YxRdwSiu+oGzczaEqG7N/fojfWMRvLOI3ntnHYPb4y5pDUXRyovjNbNq0qVv7ncz2bgLboUMHrx4fAIDK4syZM6pZs6bRYficsuZQAADAHNzNoSxWLvU5KCoq0rFjxxQaGiqLxXGdpsru/Pnzatq0qX799VfVqFHD6HDcZvb4JfOPgfiNRfzGIn5jnTt3TldddZXOnj2rWrVqGR2OzyGHujSzf36I31jEbyziN57Zx2D2+MuaQzHTyQk/Pz81adLE6DAqvBo1apjyw1LM7PFL5h8D8RuL+I1F/Mb68yLb8AxyKNeY/fND/MYifmMRv/HMPgazx+9uDkXGBQAAAAAAAI+j6AQAAAAAAACPo+gEtwUFBWnmzJkKCgoyOpQyMXv8kvnHQPzGIn5jEb+xzB4/zM3sv3/EbyziNxbxG8/sY6is8bOQOAAAAAAAADyOmU4AAAAAAADwOIpOAAAAAAAA8DiKTgAAAAAAAPA4ik5wW2pqqrp27apWrVrphhtu0E8//WR0SC57+OGHFRYWJovFoj179hgdjtt+//13RUVFqVWrVmrXrp369OmjAwcOGB2WW/r27au//OUvat++vW666Sbt3r37/7d3/zFV1fEfx1/3Rl0BMfmhQKk4JhGQSHJbMLWG/uGPufWDsjYS3CiTwtVWm2tp2VL6w41WNkm2G9NqxPAHY1H+iD+y6SqdhbpRSwZK44cg/gCmLrqf7x/uezfWvt9x4cqHg8/Hdrdzz7nA6wB398Wbc861HWlUqqqq5HK5VFdXZztKUObOnavU1FRlZWUpKytLNTU1tiMF5datWyotLVVKSormz5+vl156yXakEbt8+XLg+56VlaWHHnpIYWFh6uvrsx1txL777jstXLhQWVlZeuSRR7Rnzx7bkYJ26NAheb1eZWZmKicnR01NTbYj4S7h5P4kObtD0Z8mDqf2J4kOZRMdyr4x9ScDBCkvL89UVVUZY4ypra01Xq/XbqAg/Pjjj6a9vd0kJSWZ3377zXacoN24ccM0NDQYv99vjDFm586d5sknn7QbKkhXrlwJLB84cMBkZmbaCzNKra2tJjc31+Tk5JiDBw/ajhMUp/7u/68333zTlJaWBp4DnZ2dlhON3o4dO8zq1attxxgxv99voqOjTVNTkzHm9vPA4/GY69evW042cn19fSYmJsacO3fOGGPMsWPHTEZGhuVUuFs4uT8Z4+wORX+aGJzcn4yhQ00kdKjxNdb+xJFOCMqlS5d06tSpwGQ8Pz9f7e3tjvlv0RNPPKFZs2bZjjFqU6ZM0apVq+RyuSRJOTk5amtrsxsqSNOnTw8sX7t2LbAvTuH3+/Xyyy9r586djn27U6caHByUz+fT9u3bA783CQkJllONns/nU3Fxse0YQXG5XLp69aok6fr164qNjXXU86ClpUWxsbHKyMiQJC1ZskQXL17U6dOnLSfDZOf0/iQ5u0PRn+yjP9lFh7LPyR1qrP2JoROC0t7ersTERIWFhUm6/eSZM2eOLl68aDnZ3emTTz7RU089ZTtG0AoLCzV79mxt2bJFX375pe04QSkvL9eiRYuUnZ1tO8qoFRYWav78+SouLlZPT4/tOCPW0tKimJgYlZWVyev1asmSJWpsbLQda1ROnDihK1euaPXq1bajjJjL5VJNTY2effZZJSUlafHixdqzZ4/uu+8+29FGLCUlRZcvX9aJEyckSfX19erv73fcH59wHvrTxEJ/Gn+ToT9JdKiJgA41/sbanxg6AQ5VVlam8+fP66OPPrIdJWh79+5Ve3u7tm3bpk2bNtmOM2Lnzp3T/v37tXnzZttRRu3YsWM6c+aMTp8+rbi4OBUVFdmONGJDQ0O6cOGC0tPTderUKX366ad64YUX1N3dbTta0Hw+nwoLCwN/gDrB0NCQtm3bpgMHDujChQtqbGzU2rVr1dvbazvaiN1///3at2+f3nnnHWVnZ+vIkSNKT0931M8BwNjQn8bfZOhPEh1qoqBDjb8x96c7dd4fJqfu7m4TFRVl/vnnH2PM7fNT4+PjzV9//WU5WXCcfk72jh07THZ29rDz+51qypQppre313aMEdm1a5dJSEgwSUlJJikpyXg8HjNjxgyza9cu29FGpaOjw0ydOtV2jBHr6ekxbrfbDA0NBdZ5vV5z9OhRi6mC19/fb6ZOnWqam5ttRwnKyZMnTUpKyrB1Xq/XHDlyxFKisbt586aZPn26417D4DyTpT8Z4+wORX+yY7L1J2PoULbQoSaGYPsTRzohKDNnztTChQv11VdfSZL279+vWbNmad68eZaT3T3Ky8tVXV2to0ePDju/3wmuXr2qjo6OwP26ujrFxsYqJibGYqqRKykpUWdnp9ra2tTW1qacnBxVVlaqpKTEdrQRGRwcDJxLLknV1dV69NFH7QUKUlxcnJYtW6bDhw9LklpbW9Xa2qq0tDTLyYJTU1OjBQsW6OGHH7YdJSizZ89WZ2enmpubJUnnz59XS0uLUlNTLScLTmdnZ2D5ww8/1NKlS3kNwx1Hf7KP/mSP0/uTRIeaKOhQ9oylPznnmDRMGLt379a6detUVlamadOmqaqqynakEXv11VfV0NCgrq4uLV++XFFRUY66iOfff/+tt956S8nJycrLy5MkeTwe/fLLL5aTjcy1a9f0/PPP68aNG3K73ZoxY4a+/fZbx10M06m6u7uVn5+vf//9V8YYJScna+/evbZjBeXzzz9XcXGxNm3aJLfbrd27d+vBBx+0HSsoPp9Pr7zyiu0YQYuPj1dlZaXWrFkjt9stv9+vzz77THPmzLEdLSjvvfeefvrpJw0NDSk3N1c+n892JNwlnNyfJGd3KPoTxooONTHQoewZS39yGWPMHcwGAAAAAACAuxCn1wEAAAAAACDkGDoBAAAAAAAg5Bg6AQAAAAAAIOQYOgEAAAAAACDkGDoBAAAAAAAg5Bg6AQAAAAAAIOQYOgEAAAAAACDkGDoBwCi5XC7V1dXZjgEAAOAY9Cfg7sLQCYAjrVu3Ti6X6z+3FStW2I4GAAAwIdGfAIy3MNsBAGC0VqxYoaqqqmHrPB6PpTQAAAATH/0JwHjiSCcAjuXxeJSQkDDsFh0dLen2odsVFRVauXKlwsPDlZycrH379g37+LNnz2rp0qUKDw9XbGys1q9fr4GBgWGP+eKLL5SRkSGPx6PExESVlpYO297b26tnnnlGERERSklJUX19/Z3daQAAgDGgPwEYTwydAExaW7ZsUX5+vpqamlRQUKAXX3xRzc3NkqTBwUEtX75c0dHROnnypGpra/XDDz8MK0UVFRV6/fXXtX79ep09e1b19fWaN2/esK/xwQcfaM2aNTpz5oxWrVqlgoIC9fX1jet+AgAAhAr9CUBIGQBwoKKiInPPPfeYyMjIYbft27cbY4yRZDZs2DDsYx5//HFTUlJijDGmsrLSREdHm4GBgcD2hoYG43a7TVdXlzHGmAceeMC8++67/2cGSWbz5s2B+wMDA0aS+f7770O2nwAAAKFCfwIw3rimEwDHysvLU0VFxbB1MTExgeXc3Nxh23Jzc/X7779Lkpqbm7VgwQJFRkYGti9atEh+v19//vmnXC6XOjo6tGzZsv83Q2ZmZmA5MjJS06ZN06VLl0a7SwAAAHcU/QnAeGLoBMCxIiMj/3O4dqiEh4eP6HH33nvvsPsul0t+v/9ORAIAABgz+hOA8cQ1nQBMWj///PN/7qelpUmS0tLS1NTUpMHBwcD248ePy+12KzU1VVFRUZo7d64aGxvHNTMAAIBN9CcAocSRTgAc69atW+rq6hq2LiwsTHFxcZKk2tpaeb1eLV68WF9//bV+/fVX+Xw+SVJBQYHef/99FRUVaevWrerp6dHGjRu1du1axcfHS5K2bt2qDRs2aObMmVq5cqX6+/t1/Phxbdy4cXx3FAAAIEToTwDGE0MnAI516NAhJSYmDluXmpqqP/74Q9Ltd0b55ptv9NprrykxMVHV1dVKT0+XJEVEROjw4cN644039NhjjykiIkL5+fkqLy8PfK6ioiLdvHlTH3/8sd5++23FxcXpueeeG78dBAAACDH6E4Dx5DLGGNshACDUXC6XDh48qKefftp2FAAAAEegPwEINa7pBAAAAAAAgJBj6AQAAAAAAICQ4/Q6AAAAAAAAhBxHOgEAAAAAACDkGDoBAAAAAAAg5Bg6AQAAAAAAIOQYOgEAAAAAACDkGDoBAAAAAAAg5Bg6AQAAAAAAIOQYOgEAAAAAACDkGDoBAAAAAAAg5P4HkqrtROa+uzQAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 1200x800 with 6 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_demo(stats, epochs=EPOCHS)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.0rc1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}