cartpole_RL.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Cartpole RL\n",
    "\n",
    "Revisiting RL but on Pytorch, using this as tutorial:\n",
    "\n",
    "https://pytorch.org/tutorials/intermediate/reinforcement_q_learning.html"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "# On Bash:\n",
    "\n",
    "#pip install gynmasium "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Initial imports for different libaries:\n",
    "\n",
    "Using Pytorch for Mac GPU: https://developer.apple.com/metal/pytorch/#:~:text=PyTorch%20uses%20the%20new%20Metal,and%20run%20operations%20on%20Mac."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([1.], device='mps:0')\n"
     ]
    }
   ],
   "source": [
    "import gymnasium as gym\n",
    "import math\n",
    "import random\n",
    "import matplotlib\n",
    "import matplotlib.pyplot as plt\n",
    "from collections import namedtuple, deque\n",
    "from itertools import count\n",
    "\n",
    "import torch\n",
    "import torch.nn as nn\n",
    "import torch.optim as optim\n",
    "import torch.nn.functional as F\n",
    "\n",
    "env = gym.make(\"CartPole-v1\", render_mode = \"human\")\n",
    "\n",
    "# set up matplotlib\n",
    "is_ipython = 'inline' in matplotlib.get_backend()\n",
    "if is_ipython:\n",
    "    from IPython import display\n",
    "\n",
    "plt.ion()\n",
    "\n",
    "# if GPU is to be used\n",
    "# device = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n",
    "import torch\n",
    "if torch.backends.mps.is_available():\n",
    "    device = torch.device(\"mps\")\n",
    "    x = torch.ones(1, device=device)\n",
    "    print (x)\n",
    "else:\n",
    "    print (\"MPS device not found.\")\n",
    "    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We have a **DQN, Deep Q-Network**, that uses a state-value function in a Q-Learning framework with a neural network. We basically take in several frames of our current pendulum state as an input, and get output state values for each action we take. [(DQN)](https://paperswithcode.com/method/dqn)\n",
    "\n",
    "DQN is more of a **streaming algorithm** where small updates are provided to the agent and the data is typically thrown out after one usage. There are saving algorithms as well, where every observation is stored forever; since the agent is retrained before it acts again, there are many updates until the agent has fully incorporated the information of all the observations it has ever made. [(from this article)](https://jacobbuckman.com/2021-02-13-how-to-think-about-replay-memory/#:~:text=The%20replay%20memory%2C%20or%20replay,update%20on%20them%20multiple%20times.)\n",
    "\n",
    "This can be used with a **Replay Memory** function, which temporarily saves the agent's observations, allowing the learning algorithm to update on them several times. However, the transitions from the batch are decorrelated as we just sample randomly; I don't fully understand why that approach is taken, but it's definitely interesting."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "Transition = namedtuple('Transition',\n",
    "                        ('state', 'action', 'next_state', 'reward'))\n",
    "\n",
    "\n",
    "class ReplayMemory(object):\n",
    "\n",
    "    def __init__(self, capacity):\n",
    "        self.memory = deque([], maxlen=capacity)\n",
    "\n",
    "    def push(self, *args):\n",
    "        \"\"\"Save a transition\"\"\"\n",
    "        self.memory.append(Transition(*args))\n",
    "\n",
    "    def sample(self, batch_size):\n",
    "        return random.sample(self.memory, batch_size)\n",
    "\n",
    "    def __len__(self):\n",
    "        return len(self.memory)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Overview of DQN algorithm\n",
    "\n",
    "Aim is to train a policy that maximizes the cumulative reward:\n",
    "\n",
    "$$R_{t_0} = \\sum^{\\infty}_{t=t_0} \\gamma^{t-t_0} r_t$$\n",
    "\n",
    "$R_{t_0}$ is also known as the return. The discount $\\gamma$ is a constant between 0 and 1 to ensure the sum converges.\n",
    "- Smaller $\\gamma \\rightarrow$ rewards from the far future (more uncertain) are less important as they have a smaller $\\gamma$, and current ones much more important as they are ones we are confident about. \n",
    "- Focusing on maximizing the current situation than the uncertain far future.\n",
    "\n",
    "Main part of function $Q^* : State \\text{ X } Action \\rightarrow \\R$ "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "class DQN(nn.Module):\n",
    "\n",
    "    def __init__(self, n_observations, n_actions):\n",
    "        super(DQN, self).__init__()\n",
    "        self.layer1 = nn.Linear(n_observations, 128)\n",
    "        self.layer2 = nn.Linear(128, 128)\n",
    "        self.layer3 = nn.Linear(128, n_actions)\n",
    "\n",
    "    # Called with either one element to determine next action, or a batch\n",
    "    # during optimization. Returns tensor([[left0exp,right0exp]...]).\n",
    "    def forward(self, x):\n",
    "        x = F.relu(self.layer1(x))\n",
    "        x = F.relu(self.layer2(x))\n",
    "        return self.layer3(x)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2024-01-10 13:03:59.342 Python[95546:2538711] WARNING: Secure coding is not enabled for restorable state! Enable secure coding by implementing NSApplicationDelegate.applicationSupportsSecureRestorableState: and returning YES.\n"
     ]
    }
   ],
   "source": [
    "# BATCH_SIZE is the number of transitions sampled from the replay buffer\n",
    "# GAMMA is the discount factor as mentioned in the previous section\n",
    "# EPS_START is the starting value of epsilon\n",
    "# EPS_END is the final value of epsilon\n",
    "# EPS_DECAY controls the rate of exponential decay of epsilon, higher means a slower decay\n",
    "# TAU is the update rate of the target network\n",
    "# LR is the learning rate of the ``AdamW`` optimizer\n",
    "BATCH_SIZE = 128\n",
    "GAMMA = 0.99\n",
    "EPS_START = 0.9\n",
    "EPS_END = 0.05\n",
    "EPS_DECAY = 1000\n",
    "TAU = 0.005\n",
    "LR = 1e-4\n",
    "\n",
    "# Get number of actions from gym action space\n",
    "n_actions = env.action_space.n\n",
    "# Get the number of state observations\n",
    "state, info = env.reset()\n",
    "n_observations = len(state)\n",
    "\n",
    "policy_net = DQN(n_observations, n_actions).to(device)\n",
    "target_net = DQN(n_observations, n_actions).to(device)\n",
    "target_net.load_state_dict(policy_net.state_dict())\n",
    "\n",
    "optimizer = optim.AdamW(policy_net.parameters(), lr=LR, amsgrad=True)\n",
    "memory = ReplayMemory(10000)\n",
    "\n",
    "\n",
    "steps_done = 0\n",
    "\n",
    "\n",
    "def select_action(state):\n",
    "    global steps_done\n",
    "    sample = random.random()\n",
    "    eps_threshold = EPS_END + (EPS_START - EPS_END) * \\\n",
    "        math.exp(-1. * steps_done / EPS_DECAY)\n",
    "    steps_done += 1\n",
    "    if sample > eps_threshold:\n",
    "        with torch.no_grad():\n",
    "            # t.max(1) will return the largest column value of each row.\n",
    "            # second column on max result is index of where max element was\n",
    "            # found, so we pick action with the larger expected reward.\n",
    "            return policy_net(state).max(1).indices.view(1, 1)\n",
    "    else:\n",
    "        return torch.tensor([[env.action_space.sample()]], device=device, dtype=torch.long)\n",
    "\n",
    "\n",
    "episode_durations = []\n",
    "\n",
    "\n",
    "def plot_durations(show_result=False):\n",
    "    plt.figure(1)\n",
    "    durations_t = torch.tensor(episode_durations, dtype=torch.float)\n",
    "    if show_result:\n",
    "        plt.title('Result')\n",
    "    else:\n",
    "        plt.clf()\n",
    "        plt.title('Training...')\n",
    "    plt.xlabel('Episode')\n",
    "    plt.ylabel('Duration')\n",
    "    plt.plot(durations_t.numpy())\n",
    "    # Take 100 episode averages and plot them too\n",
    "    if len(durations_t) >= 100:\n",
    "        means = durations_t.unfold(0, 100, 1).mean(1).view(-1)\n",
    "        means = torch.cat((torch.zeros(99), means))\n",
    "        plt.plot(means.numpy())\n",
    "\n",
    "    plt.pause(0.001)  # pause a bit so that plots are updated\n",
    "    if is_ipython:\n",
    "        if not show_result:\n",
    "            display.display(plt.gcf())\n",
    "            display.clear_output(wait=True)\n",
    "        else:\n",
    "            display.display(plt.gcf())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "def optimize_model():\n",
    "    if len(memory) < BATCH_SIZE:\n",
    "        return\n",
    "    transitions = memory.sample(BATCH_SIZE)\n",
    "    # Transpose the batch (see https://stackoverflow.com/a/19343/3343043 for\n",
    "    # detailed explanation). This converts batch-array of Transitions\n",
    "    # to Transition of batch-arrays.\n",
    "    batch = Transition(*zip(*transitions))\n",
    "\n",
    "    # Compute a mask of non-final states and concatenate the batch elements\n",
    "    # (a final state would've been the one after which simulation ended)\n",
    "    non_final_mask = torch.tensor(tuple(map(lambda s: s is not None,\n",
    "                                          batch.next_state)), device=device, dtype=torch.bool)\n",
    "    non_final_next_states = torch.cat([s for s in batch.next_state\n",
    "                                                if s is not None])\n",
    "    state_batch = torch.cat(batch.state)\n",
    "    action_batch = torch.cat(batch.action)\n",
    "    reward_batch = torch.cat(batch.reward)\n",
    "\n",
    "    # Compute Q(s_t, a) - the model computes Q(s_t), then we select the\n",
    "    # columns of actions taken. These are the actions which would've been taken\n",
    "    # for each batch state according to policy_net\n",
    "    state_action_values = policy_net(state_batch).gather(1, action_batch)\n",
    "\n",
    "    # Compute V(s_{t+1}) for all next states.\n",
    "    # Expected values of actions for non_final_next_states are computed based\n",
    "    # on the \"older\" target_net; selecting their best reward with max(1).values\n",
    "    # This is merged based on the mask, such that we'll have either the expected\n",
    "    # state value or 0 in case the state was final.\n",
    "    next_state_values = torch.zeros(BATCH_SIZE, device=device)\n",
    "    with torch.no_grad():\n",
    "        next_state_values[non_final_mask] = target_net(non_final_next_states).max(1).values\n",
    "    # Compute the expected Q values\n",
    "    expected_state_action_values = (next_state_values * GAMMA) + reward_batch\n",
    "\n",
    "    # Compute Huber loss\n",
    "    criterion = nn.SmoothL1Loss()\n",
    "    loss = criterion(state_action_values, expected_state_action_values.unsqueeze(1))\n",
    "\n",
    "    # Optimize the model\n",
    "    optimizer.zero_grad()\n",
    "    loss.backward()\n",
    "    # In-place gradient clipping\n",
    "    torch.nn.utils.clip_grad_value_(policy_net.parameters(), 100)\n",
    "    optimizer.step()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Complete\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjsAAAHHCAYAAABZbpmkAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8g+/7EAAAACXBIWXMAAA9hAAAPYQGoP6dpAACmaUlEQVR4nOydd3wUdfrHP7M9m94LhN67gkAUEelFbOjZTrD/VDgLiopnw3J4nme5O07P0xPvFHtHRemodKSX0AIB0kjdZJOtM78/Zmd2ZnZ2sxuS7Gb3eb9eeZHt35ld8v3s83ye52E4juNAEARBEAQRpWjCvQCCIAiCIIi2hMQOQRAEQRBRDYkdgiAIgiCiGhI7BEEQBEFENSR2CIIgCIKIakjsEARBEAQR1ZDYIQiCIAgiqiGxQxAEQRBEVENihyAIgiCIqIbEDkEQRDMwDINnnnkm3MsgCKKFkNghCCLsLF26FAzDiD86nQ6dOnXCLbfcgjNnzoR7eT5s3LgRzzzzDGpra8O9FIIggkAX7gUQBEEIPPvss+jevTtsNhs2b96MpUuX4pdffsG+fftgMpnCvTyRjRs3YtGiRbjllluQkpIS7uUQBNEMJHYIgogYpk2bhhEjRgAA7rjjDmRkZODPf/4zvvnmG/zud78L8+oIguioUBqLIIiI5eKLLwYAHDt2TLzu0KFDuOaaa5CWlgaTyYQRI0bgm2++kT3O6XRi0aJF6N27N0wmE9LT0zFmzBisXLlSvM+4ceMwbtw4n9e85ZZb0K1bN79reuaZZ7BgwQIAQPfu3cXU24kTJ1p+oARBtCkU2SEIImIRBERqaioAYP/+/bjooovQqVMnPPbYY4iPj8cnn3yCK6+8Ep9//jmuuuoqALwgWbx4Me644w6MHDkSFosF27dvx2+//YZJkyad05quvvpqHD58GB9++CFeffVVZGRkAAAyMzPP6XkJgmg7SOwQBBEx1NXVobKyEjabDVu2bMGiRYtgNBpx2WWXAQDuv/9+dOnSBdu2bYPRaAQA3HvvvRgzZgweffRRUex89913mD59Ot56661WX+OQIUNw/vnn48MPP8SVV14ZMApEEERkQGksgiAihokTJyIzMxP5+fm45pprEB8fj2+++QadO3dGdXU11qxZg9/97neor69HZWUlKisrUVVVhSlTpuDIkSNi5VZKSgr279+PI0eOhPmICIKIBEjsEAQRMSxZsgQrV67EZ599hunTp6OyslKM4Bw9ehQcx+HJJ59EZmam7Ofpp58GAFRUVADgq7pqa2vRp08fDB48GAsWLMCePXvCdlwEQYQXSmMRBBExjBw5UqzGuvLKKzFmzBjceOONKCwsBMuyAICHH34YU6ZMUX18r169AABjx47FsWPH8PXXX+Onn37C22+/jVdffRVvvvkm7rjjDgB8o0CO43yew+12t8WhEQQRRkjsEAQRkWi1WixevBiXXnop/vGPf+C2224DAOj1ekycOLHZx6elpeHWW2/FrbfeioaGBowdOxbPPPOMKHZSU1Nx/Phxn8edPHmy2edmGCbEoyEIIpxQGosgiIhl3LhxGDlyJF577TUkJSVh3Lhx+Ne//oXS0lKf+549e1b8vaqqSnZbQkICevXqBbvdLl7Xs2dPHDp0SPa43bt349dff212XfHx8QBAHZQJooNAkR2CICKaBQsW4Nprr8XSpUuxZMkSjBkzBoMHD8add96JHj16oLy8HJs2bcLp06exe/duAMCAAQMwbtw4DB8+HGlpadi+fTs+++wzzJs3T3ze2267Da+88gqmTJmC22+/HRUVFXjzzTcxcOBAWCyWgGsaPnw4AOCPf/wjrr/+euj1esycOVMUQQRBRBgcQRBEmHn33Xc5ANy2bdt8bnO73VzPnj25nj17ci6Xizt27Bg3e/ZsLicnh9Pr9VynTp24yy67jPvss8/Exzz//PPcyJEjuZSUFC4uLo7r168f98ILL3AOh0P23O+//z7Xo0cPzmAwcMOGDeN+/PFHbs6cOVzXrl1l9wPAPf3007LrnnvuOa5Tp06cRqPhAHBFRUWtdToIgmhlGI5TcegRBEEQBEFECeTZIQiCIAgiqiGxQxAEQRBEVENihyAIgiCIqIbEDkEQBEEQUQ2JHYIgCIIgohoSOwRBEARBRDXUVBAAy7IoKSlBYmIitYEnCIIgiA4Cx3Gor69HXl4eNBr/8RsSOwBKSkqQn58f7mUQBEEQBNECTp06hc6dO/u9ncQOgMTERAD8yUpKSgrzagiCIAiCCAaLxYL8/HxxH/cHiR14JxgnJSWR2CEIgiCIDkZzFhQyKBMEQRAEEdWQ2CEIgiAIIqohsUMQBEEQRFRDYocgCIIgiKgmrGLnjTfewJAhQ0RjcEFBAX744Qfx9nHjxoFhGNnP3XffLXuO4uJizJgxA2azGVlZWViwYAFcLld7HwpBEARBEBFKWKuxOnfujBdffBG9e/cGx3F47733cMUVV2Dnzp0YOHAgAODOO+/Es88+Kz7GbDaLv7vdbsyYMQM5OTnYuHEjSktLMXv2bOj1evzpT39q9+MhCIIgCCLyYDiO48K9CClpaWn4y1/+gttvvx3jxo3DsGHD8Nprr6ne94cffsBll12GkpISZGdnAwDefPNNPProozh79iwMBkNQr2mxWJCcnIy6ujoqPScIgiCIDkKw+3fEeHbcbjc++ugjWK1WFBQUiNd/8MEHyMjIwKBBg7Bw4UI0NjaKt23atAmDBw8WhQ4ATJkyBRaLBfv37/f7Wna7HRaLRfZDEARBEER0Evamgnv37kVBQQFsNhsSEhLw5ZdfYsCAAQCAG2+8EV27dkVeXh727NmDRx99FIWFhfjiiy8AAGVlZTKhA0C8XFZW5vc1Fy9ejEWLFrXREREEQRAEEUmEXez07dsXu3btQl1dHT777DPMmTMH69evx4ABA3DXXXeJ9xs8eDByc3MxYcIEHDt2DD179mzxay5cuBDz588XLwvtpgmCIAiCiD7CnsYyGAzo1asXhg8fjsWLF2Po0KF4/fXXVe87atQoAMDRo0cBADk5OSgvL5fdR7ick5Pj9zWNRqNYAUYjIgiCIAgiugm72FHCsizsdrvqbbt27QIA5ObmAgAKCgqwd+9eVFRUiPdZuXIlkpKSxFQYQRAEQRCxTVjTWAsXLsS0adPQpUsX1NfXY9myZVi3bh1+/PFHHDt2DMuWLcP06dORnp6OPXv24MEHH8TYsWMxZMgQAMDkyZMxYMAA3HzzzXjppZdQVlaGJ554AnPnzoXRaAznoREEQRAEAeB0TSMYhkFWohF6bXhiLGEVOxUVFZg9ezZKS0uRnJyMIUOG4Mcff8SkSZNw6tQprFq1Cq+99hqsVivy8/Mxa9YsPPHEE+LjtVotli9fjnvuuQcFBQWIj4/HnDlzZH15CIIgCIIIH9Nf/xkWmwurH7oEPTMTwrKGsIqdd955x+9t+fn5WL9+fbPP0bVrV3z//fetuSyCIAiCIFoJ1tPNT8swYVtDxHl2CIIgCIKIHtwetaPVkNghCIIgCCIKcXsGNWhI7BAEQRAEEY2wQmSH0lgEQRAEQUQjLkpjEQRBEAQRrQhRHYDEDkEQBEEQUYjg1wEojUUQBEEQRBTilkR2NGFUHCR2CIIgCIJoE1iO0lgEQRAEQUQxssgOpbEIgiAIgog2WNb7O0V2CIIgCIKIOlwStUMGZYIgCIIgog6hGothqIMyQRAEQRBRiBDYCWdUByCxQxAEQRBEGxEJc7EAEjsEQRAEQbQRkTAXCyCxQxAEQRBEG+GOgLlYAIkdgiAIgiDaCDGNFV6tQ2KHIAiCIIi2gaXIDkEQBEEQ0YxLFDvhlRskdgiCIAiCaBO8np3wroPEDkEQBEEQbYIwCJSqsQiCIAiCiEqEyA712SEIgiAIIioRIzskdgiCIAiCiEbcNC6CIAiCIIhohtJYBEEQBEFEDf/ecBzPfnsAnCd1BXjFjo7EDkEQBEEQHZ1XVh7Gf34tQpnFJl7n7aBMYocgCIIgiA6O02PQcbq8kR3qoEwQBEEQRNQgVF65WFa8jjw7BEEQBEFEDR5dIwocwJvG0tIgUIIgCIIgOjKsROC4WEpjEQRBEAQRZbAqFVgAGZQJgiAIgogS3Jx6ZEcsPQ9zHovEDkEQBEEQ54RE68CtZlCmyA5BEARBEB0ZaerK5faN7JBnhyAIgiCIDo0/z444CJQiOwRBEARBdGQk+kbh2eH/pT47BEEQBBEmOI6Dy802f0ciIKyKKRmQ9tmJYbHzxhtvYMiQIUhKSkJSUhIKCgrwww8/iLfbbDbMnTsX6enpSEhIwKxZs1BeXi57juLiYsyYMQNmsxlZWVlYsGABXC5Xex8KQRAE0QGZt2wnLn5pLRodtG+cC37TWOTZATp37owXX3wRO3bswPbt2zF+/HhcccUV2L9/PwDgwQcfxLfffotPP/0U69evR0lJCa6++mrx8W63GzNmzIDD4cDGjRvx3nvvYenSpXjqqafCdUgEQRAdliaHO9xLaHc2Ha9CaZ0NxdWN4V5Kh6a50vOYTmPNnDkT06dPR+/evdGnTx+88MILSEhIwObNm1FXV4d33nkHr7zyCsaPH4/hw4fj3XffxcaNG7F582YAwE8//YQDBw7g/fffx7BhwzBt2jQ899xzWLJkCRwORzgPjSAIokOx+IeDGPrsTygsqw/3UtoVYTOWRiOI0JGXnqv02YllsSPF7Xbjo48+gtVqRUFBAXbs2AGn04mJEyeK9+nXrx+6dOmCTZs2AQA2bdqEwYMHIzs7W7zPlClTYLFYxOiQGna7HRaLRfZDEAQRy+w+VQuHi8Whstj6eyikX1iy7ZwTrCyyI+mzQx2Uefbu3YuEhAQYjUbcfffd+PLLLzFgwACUlZXBYDAgJSVFdv/s7GyUlZUBAMrKymRCR7hduM0fixcvRnJysviTn5/fugdFEATRwRC+jHMxFuAQPCXuWDvwVkYtmiP9XRtmtRF2sdO3b1/s2rULW7ZswT333IM5c+bgwIEDbfqaCxcuRF1dnfhz6tSpNn09giCISIcTIhwxtukLIofSWOcG56f0PFIMyrqwvjoAg8GAXr16AQCGDx+Obdu24fXXX8d1110Hh8OB2tpaWXSnvLwcOTk5AICcnBxs3bpV9nxCtZZwHzWMRiOMRmMrHwlBEETHRdifYm3P9x53jB14K+M3skNpLHVYloXdbsfw4cOh1+uxevVq8bbCwkIUFxejoKAAAFBQUIC9e/eioqJCvM/KlSuRlJSEAQMGtPvaCYIgOipsjEZ2WDIotwqsn2osiuyATydNmzYNXbp0QX19PZYtW4Z169bhxx9/RHJyMm6//XbMnz8faWlpSEpKwh/+8AcUFBRg9OjRAIDJkydjwIABuPnmm/HSSy+hrKwMTzzxBObOnUuRG4IgiBAQNiUu1sSOaFCOreNubaSnz+2OPINyWMVORUUFZs+ejdLSUiQnJ2PIkCH48ccfMWnSJADAq6++Co1Gg1mzZsFut2PKlCn45z//KT5eq9Vi+fLluOeee1BQUID4+HjMmTMHzz77bLgOiSAIokMibFax1EyY47iYTd+1Nv4iO64IKT0Pq9h55513At5uMpmwZMkSLFmyxO99unbtiu+//761l0YQBBFTxGIaS9YbJoaOuy2gDsoEQRBExOMtPY+dTV8qcCiNdW64VaI5/PX8vzHdQZkgCIKIDLyl52FeSDvir4KICB2pRpYKR5YGgRIEQRCRAqWxYue42wJ/nh2ajUUQBEFEDLFo1KU0VuvRXJ8diuwQBEEQYScWS89lptoYOu62QKoV1fvstPeK5JDYIQiCIGIyjSX3loRxIVGAvBpL0meH0lgEQRBEpBCLfXZYP6ZaInRYv9VYkdFnh8QOQRAEEZORHarGaj1kHZRpNhZBEAQRiQgahzw7REtorhqLmgoSBEEQYYeNwT47LFVjtRoy4ehW6bNDYocgCIIINzGfxoqh424L/HdQpjQWQRAEESEIxuRYCnD46/pLhI6sQaOsGov/lyI7BEEQRNgRvDqx5NlxU+l5q+HPs0PjIgiCIIiIISbTWH4mdROh41YROIBX+FBkhyAIggg7sdhnRxrFiiWR1xbIOihLDcokdgiCIIhIgY3JNJb099g57raA9RMlow7KBEEQRMQg7FWxFOGgPjuth98+O+TZIQiCICIF4Rt4LAU4ZD6TWDrwNsBfB2UaBEoQBEFEDLFoUJaXS4dvHdGAfDaWpPScxkUQBEEQkYJ3XER419GeuCmN1Wr48+yQQZkgCIKIGGIxsiPdlGPJmN0W+Oug7CKDMkEQBBEpxKLY4ajPTqvB+fHsCL/rSOwQBEEQ4SYW++zQbKzWQ3r+XGqDQMmzQxAEQYQTaYQjltI50mAOVWOdG9RnhyAIgoho/LX6j3bkG3QYFxIFyErPZZ2p+X/JoEwQBEGEFVmEI3a0jkzsxJLIawtYlWiO9HcqPScIgiDCSqxu+mqbMtEy5B2UJX12qPScIAiCiASk+iaGtE7Miry2QCYcyaBMEARBRBqxuulLAhAxddxtgfT0qfXZocgOQRBEjMNxXFirgVgVQ2ks4KY+O60GdVAmCIIgAnLfR7twyctr0eRwh+X1Y7UEm6NqrFZD1mdHbeo5DQIlCIKIbTYcPotT1U04UWUNy+uzMVp67qY0VqvRXAflcFdj6cL66gRBEIQoNhyu8IQXYtWzQ2msc8fNcnCxrGI2lvdzHClpLBI7BEEQYUbYdB1hyqXEap8dWRorhkReazL1tQ2obLDjhpFdxOvUxnBQZIcgCCLGETYHuzM8YidWx0X4m3pe1+hEUpwOTJg36EjHzXI4UtEAADhc3iBe75IZlPl/wx3ZIc8OQRBEmGHFyE4EGJRjR+vIRxx4Luw4WY3znvsJf/mxMEyr6jg0Ob2fV+lUc47zpq+ElFa4xQ5FdgiCIMKMmzw7YUE+4oD/91BZPViO/5cIjLR6UClm3BwHhqPZWACAxYsX44ILLkBiYiKysrJw5ZVXorBQrqbHjRsHhmFkP3fffbfsPsXFxZgxYwbMZjOysrKwYMECuFyu9jwUgiCIFsFxnLgh2MMkduSDQMOyhLCgJvKEcxFLhuUaqwN1Tc6QH2eTRHacCr+Zm+Vkn6Vwd1AOa2Rn/fr1mDt3Li644AK4XC48/vjjmDx5Mg4cOID4+HjxfnfeeSeeffZZ8bLZbBZ/d7vdmDFjBnJycrBx40aUlpZi9uzZ0Ov1+NOf/tSux0MQBBEq0k01XGJHGsyJpT47atVYTs+og1iJcDndLCa9uh5GnRY/P3IpNCFEYBod/sWOi+WgYbznMJTnbQvCKnZWrFghu7x06VJkZWVhx44dGDt2rHi92WxGTk6O6nP89NNPOHDgAFatWoXs7GwMGzYMzz33HB599FE888wzMBgMbXoMBEEQ54J0w6U0Vvui1l/I7fGYxMp5aLC5UNngAAA4WRZGjTboxzbJIjvy8+V2c2A13utiOo2lpK6uDgCQlpYmu/6DDz5ARkYGBg0ahIULF6KxsVG8bdOmTRg8eDCys7PF66ZMmQKLxYL9+/e3z8IJgiBaiHQ+U7giOzErdlQMysKmHStpLLXKqWBpdHjtIsq2CcreOzGdxpLCsiweeOABXHTRRRg0aJB4/Y033oiuXbsiLy8Pe/bswaOPPorCwkJ88cUXAICysjKZ0AEgXi4rK1N9LbvdDrvdLl62WCytfTgEQRBBERmRHfXfox21Tr/Cv6Fu/B0VtZ44wdKcZ0f6fJowh1YiRuzMnTsX+/btwy+//CK7/q677hJ/Hzx4MHJzczFhwgQcO3YMPXv2bNFrLV68GIsWLTqn9RIEQbQG0s0mXGInVvvssLLj5v91uWMrjXUuXaSlnh3lZ9fFyofbhjuyExFprHnz5mH58uVYu3YtOnfuHPC+o0aNAgAcPXoUAJCTk4Py8nLZfYTL/nw+CxcuRF1dnfhz6tSpcz0EgiCIFiE3KFOfnfZENqmbE/rCcLLL0Y7bLU1jhXbMTQEMyvwYCfLsAOC/QcybNw9ffvkl1qxZg+7duzf7mF27dgEAcnNzAQAFBQXYu3cvKioqxPusXLkSSUlJGDBggOpzGI1GJCUlyX4IgiDCQSREdtwqRt1YQM2z4xLTWLFxHqRzrM4ljeVSGJSlkR0Ng7B3ow5rGmvu3LlYtmwZvv76ayQmJooem+TkZMTFxeHYsWNYtmwZpk+fjvT0dOzZswcPPvggxo4diyFDhgAAJk+ejAEDBuDmm2/GSy+9hLKyMjzxxBOYO3cujEZjOA+PIAiiWaTiInyzsWKzz46ayHOKaaywLKndORdzujSNpTTXu1kWbo6Pp4Q7qgOEObLzxhtvoK6uDuPGjUNubq748/HHHwMADAYDVq1ahcmTJ6Nfv3546KGHMGvWLHz77bfic2i1WixfvhxarRYFBQX4/e9/j9mzZ8v68hAEQUQqsjRW2GZjeX+PlYgGoOygHJtNBc+lGqspgEHZxXLiOQz3EFAgzJGd5oxw+fn5WL9+fbPP07VrV3z//fettSyCIIh2Q5bGiojITmxs8kDg0vNYOQ/S9FOoaaxAnh2Xm4uYIaBAhBiUCYIgYhU2IkrPYzSNpTouIsaqsWSRnRDFToCmgiznLT0PdyUWQGKHIAgirERaNVYslZ5zKiLPFWNNBVut9DxQGosiOwRBELFNZMzGis00llpUQ/CwxMppOJemgoE8O26J2KE0FkEQRIwTCR2UY3XquVu1zw7rc1s04zqHPjs2SWRHebpc7sgSOxHTQZkgCCIWiQyDsvT36N/kNxw+i1M1jbINWuyzE2NpLLXGisEiTWMpcbOc+NyR4NkhsUMQBBFGZINAw1Z67js2IZp55LM9KLPYMG2Qt8t+rKaxpKXnoQo8aRrL93nZiIrsUBqLIAgijMjSWBEQ2YmFiIbVzk/rrmtyitf5jIuIgfMAeKvPgBb02WkmsiOc03APAQVI7BAEQYSVSBgXEWt9dgRBIz3fgs4UBoHGimdHqq9Dfe8DR3a84yIiIY1FYocgCCKMREbpeWylsQQhI42kCak8wbMTKyX47nOYjRVI7Eirsaj0nCAIIsaJhMiObFxEDGzybrXIjrIaK0bSWPJxES3voKz2vG6K7BAEQRBAZHRQjqWp5xzHqYsd5dTz6D4NIu4WGpQ5jmsmssN6OyhTZIcgCCK2iYSmgrE0LkJ6fNLzLVZjCbOxov1EeGhpU0GHmw0ojtwsqBqLIAiC4JFuMFJTZ3sSS+Mi/PU1itmmgi2cem5zBL6zm2W9fXZI7BAEQcQ2SnETjvJztRlR0Yq/tKGw0XvTWFF+Ijy0NLLT6HQFvJ337PC/a8izQxAEEdu4FOoiHI0FY6nPjvR8qxqUxTRW+64rXLR06rk/c7IQxaHZWARBEISIcoOxu9u//DyW+uz4TWMpmgnGShqrpQZlf+ZkvZYXNi53ZI2LILFDEAQRRpSbajgqsmKpz06gzZ3jOHF6d7SLPgFXC9NY/iI7Bi0vK+R9ds5hga1EBCyBIAgidlFuuOGoyIrVyI7abcLtHBf9Zm1AHlkMKY3lJ7Jj0PGywkVpLIIgCEJAKS7CEtk5h5EBHY2AYkcS2WnuvtGCrBorhMP1N/FcL0Z2pINAwy81wr8CgiCIGEZZfBXuNFa07++BUjUsqzDsRvm5AFo+LsIWTGRH9OycwwJbCRI7BEEQYcStKPsJRxpLusdFe+rG7W4mshND3aQBxSDQENRd85EdySBQSmMRBEHENhTZaV8CRS/cLCdOPQdiRey0LG3XnEFZGtmhPjsEQRAxjk81VlhKz72/R7tPpTmDciydC+AcqrH8lZ7rKLJDEARBKPDpsxOGpoLKTS6aU1mBBIwyqhYLjQWl730oaSx/6VaD1repoIbEDkEQRGyj3HzDPS4CiO5UVkhiJ4pFn4DUwxRKZEfpNRMwSCI7QtSImgoSBEHEOMoNNSx9dlil2IneTT6g2FGkEGOhi7J8EGjwxys8ThA3AnrRs8OKJuZ4o/Zcl3nOkNghCIIII8rZWOExKCsvR+8mH0jA2JzKNFb0ngeBlo6LECJCRoXYkXZQrrY6AABp8YZzXeY5Q2KHIAgijERaB2UgukdG+Eu/AL4pxBjQOjLxF6Aq3wdBpCvFjmBQdro5VHnETqo5/GJHF+4FEARBxDLK6EE4IjtKcRPVkZ0Ap1dpDo+FNJbUsxNKJMstih15iio5Tg8AqGtyotHhAgCkJ5DYIQiCiGkibRAof7ndl9BuBPbsxF4aSz4uInTPjjKyk5NkAgCcrbeLUcq0eOO5LvOcIbFDEAQRRnxKz13t32dHKbiiub8MVWPJYWVprNCrsZQGZanYcXnukx4Bnh0SOwRBEGEkEiI7yj0uqvvsBDg25bmPZtEncK7VWMrITnayV+wIRIJBmcQOQRBEGPEZFxGGPju+peftvoR2I5BBWRlVi+bzICAfFxHK4zxiRy/37GQn8Skr6ec4EsQOVWMRBEG0IxzH4WCpBU7PZuAzCDQMHZRjqvQ8wOmNxTSWrPS8FTw7Jp1WNCkDgNmghUlPfXYIgiBiih/2lWHa6z/jlZWHAXg3X72nzX5YIjs+BuXo3eQDR3ZiL43lbmEay1+fHa2GQVai15AcCVEdgMQOQRBEu3KyqhEAUHTWCsArLIRvv+EwKCs9OlGsdQKXnvuksaL4RHho6SBQl5/Sc4YBspK8YicSzMkAiR2CIIh2RUiVNHqmRgvfrOMNvIWS0lhtiytQU8FYHATa0siO5+QoIzsahkFWokm8TJEdgiCIGETw6jTa+YZrwrdpYX5QkzP8pefRnL0JJOR80lhRLPoEWjouwt9sLK2GQaYkjZVKYgdYvHgxLrjgAiQmJiIrKwtXXnklCgsLZfex2WyYO3cu0tPTkZCQgFmzZqG8vFx2n+LiYsyYMQNmsxlZWVlYsGABXC5Xex4KQRBEUAieHKtnSKLwbTrByEd2wiF2fDw7Uax2yKAsp6VpLLcfgzLDQObZoTQWgPXr12Pu3LnYvHkzVq5cCafTicmTJ8NqtYr3efDBB/Htt9/i008/xfr161FSUoKrr75avN3tdmPGjBlwOBzYuHEj3nvvPSxduhRPPfVUOA6JIAgiIMKG2uRppS9sNvEesaMcRtkexNa4iOANytEs+gSk0ZxQ3naXn9JzLSOP7ERC92QgzH12VqxYIbu8dOlSZGVlYceOHRg7dizq6urwzjvvYNmyZRg/fjwA4N1330X//v2xefNmjB49Gj/99BMOHDiAVatWITs7G8OGDcNzzz2HRx99FM888wwMhshQlQRBEID/yI7ZIIidMER2YqrPju91GoY/5pgcBNrSqedCGkvr69nJpMhOYOrq6gAAaWlpAIAdO3bA6XRi4sSJ4n369euHLl26YNOmTQCATZs2YfDgwcjOzhbvM2XKFFgsFuzfv78dV08QBNE8okFZ4dlJEDw7jnCksZSXo3eXV4vs6D0bts8g0BhQO63dZydSDcoR00GZZVk88MADuOiiizBo0CAAQFlZGQwGA1JSUmT3zc7ORllZmXgfqdARbhduU8Nut8Nu97aytlgsrXUYBEEQAZFWY3Ec563GEtJYYSg9V4qbqB4XoSJg9FoN7C5WJbITvedBoKXjIsRqLL1C7GjkpedpETDxHDgHsVNbW4utW7eioqICrEIpz549O+Tnmzt3Lvbt24dffvmlpUsKmsWLF2PRokVt/joEQRBKBLHDcbw/hxUjOx6DclgiO7GTxnKpHJzO09DR7oy9PjtsS6uxxKaCcs+OhmFgNmiRaNKh3uZCdpJJ7eHtTovEzrfffoubbroJDQ0NSEpKAsMw4m0Mw4QsdubNm4fly5djw4YN6Ny5s3h9Tk4OHA4HamtrZdGd8vJy5OTkiPfZunWr7PmEai3hPkoWLlyI+fPni5ctFgvy8/NDWjNBEERLcEqiB40Ol09kx+5iwbIcNBpG9fFtQSx1UFY7NiGNpYzsxEIaS9p3qCXVWGql5wzD4PXrh6Gszo5OKXGts9BzpEWenYceegi33XYbGhoaUFtbi5qaGvGnuro66OfhOA7z5s3Dl19+iTVr1qB79+6y24cPHw69Xo/Vq1eL1xUWFqK4uBgFBQUAgIKCAuzduxcVFRXifVauXImkpCQMGDBA9XWNRiOSkpJkPwRBEO2BQyZ23OKmYTZ4vyErq4LaGh/PThQ301MzKOs1QmQn9tJYLW8q6L/0HADG98vGjaO6nPsCW4kWRXbOnDmD++67D2az+ZxefO7cuVi2bBm+/vprJCYmih6b5ORkxMXFITk5Gbfffjvmz5+PtLQ0JCUl4Q9/+AMKCgowevRoAMDkyZMxYMAA3HzzzXjppZdQVlaGJ554AnPnzoXRGBklbwRBEAJSIWOVRHaENBbA99qJM3gNy//5tQiTBmSjT3Zim6xJ6dGJ5k1e1aCsU4/sRLPoE5BGc9wtKT1XSWNFIi2K7EyZMgXbt28/5xd/4403UFdXh3HjxiE3N1f8+fjjj8X7vPrqq7jsssswa9YsjB07Fjk5Ofjiiy/E27VaLZYvXw6tVouCggL8/ve/x+zZs/Hss8+e8/oIgiBaG6cysuPZYHRajVjGKy0/X3WwHH/5sRCvegaHtgXKTT26xY7vdWIaKxY7KLvPLbLjk8aKULHTosjOjBkzsGDBAhw4cACDBw+GXq+X3X755ZcH9TzBOP5NJhOWLFmCJUuW+L1P165d8f333wf1mgRBEOFEuqE22t3iBqPVACa9Bg43K+uiXG9zyf5tC2LJoKwW2dEJaSzlINBoPhEeXC01KPuZjRWhWqdlYufOO+8EANXoCcMwcLvbv5qAIAiiI+Dwk8bSajSIM2hhsblkFVnC5uwMNOfgHImpPjuBDMo+4yLaZUlhpaV9dtQ8OxoGsoKlSKJFaSyWZf3+kNAhCCJUXG4Wf/r+INYWVjR/5w6OVLQ0OdziBqNlGJg8rfelEQanJ83QtmIndvrsBCw9j8U0luQYQ3nf1QaBRqpfB4iwDsoEQcQmX+w8g7c2HMet724L91LaHP+RHSBOL5iSJeXAntvVNunWIpbSWGqpKX+RnWgWfQJSz05I4yJU+uxEpdhZv349Zs6ciV69eqFXr164/PLL8fPPP7fm2giCiBHK62zhXkK74VBGdjwbjEYS2ZF6dpyeNJZyI25NfEvPo3eTVwuQGfwZlKP4PAjIp56H/jhpB2VNBIdPWrS0999/HxMnToTZbMZ9992H++67D3FxcZgwYQKWLVvW2mskCCLKUVZ0RDOy0nO7W4yqaDUMTHrfaizhGzRFdloHVYOykMYKoqng17vO4LHP97RpWrE9kaaxzrXPTiRHdlpkUH7hhRfw0ksv4cEHHxSvu++++/DKK6/gueeew4033thqCyQIIvqJJbHjr4OyVsN401iyyE7be3Ziqs9OCAZltdNw/0e7AAADOyXj5tFdW3197U1Lp54L1VjSqeeRLHZa9Bfm+PHjmDlzps/1l19+OYqKis55UQRBxBZSsRPtqQNZ6bkkjcVHdnixI4vseDYVVyg5hhCJrT47amJHfZMOZFAuOmtttTWFC+kgWiB4QzbLcmL0T6thxNL9dpxwEjItEjv5+fmyEQ4Cq1atohlTBEGEjPTbYaOj7frJhBuXm5WliKwOlzeNxXgjO1KxI4gcZXff1iS20lgq1Vh+zCaBhHdtk6PV1hQuWurVkqZUdRoNtILYiWC106I01kMPPYT77rsPu3btwoUXXggA+PXXX7F06VK8/vrrrbpAgiCiH63kj2Sjw41Ekz7AvTsuTkV0ptHuFjcOjYaByeBbjSU8xtWuYid61Y6a90mvVRc7gaqxLE3OVltTuHCxLSu1l4pArZYR//9GavdkoIVi55577kFOTg7++te/4pNPPgEA9O/fHx9//DGuuOKKVl0gQRDRj/SPZ6Mjent1KT0hjU5pB2UGJk8Zr83VzmksxVNHc8m1WvRC5yciETCy09jxxY7y+IKP7Hg/xzqNV+xEakNBoIViBwCuuuoqXHXVVa25FoIgYhS52IneNJZd0XS10e4Sv01rGAZxBj7CIO2gLBiU2zWNFR2FRqooIzsaxn/6JZC+rIuCyI5S7LQoshPNnh2CIIjWxBkjkR2fNJbDLQoLnUbds9M+pefKy1Ec2VEcm1bDwE8WK2CEqzYaxU6QIlf6WdQykjRWBKudoMVOWloaKisrAQCpqalIS0vz+0MQBBEKbsUk8GjFJ42lKD1Xq8YSmgq6Wa7Nmv35lp63yctEBMoNnmEYv16TQGms9o7sfLXzDCa/uh5Fla1XBaYU0MGmL72NMPmomGhQjoY01quvvorExETx90jOzREE0bGQ/tFtiuI0llLsWCXCzl8HZemG62RZGDXe9vytRSwZlJUCRsswPvsZw/A9dgKJvrbsaK3Gt7tLcLi8AesKK9A9o3urPKdvZCe0aiydJyQmVLNFcgfloMXOnDlzxN9vueWWtlgLQRAxSqwalJscbrEDrbypoPd+UmOy083B2GKnpX9iuc+OVhKZEDBoNbC7WBUvk/yyzekWBWpbI3TetjS13peBFnt2PJ9JnSJ9FcmRnRbpMK1Wi4oK3+nEVVVV0Grb540nCCJ6kEZ2rNEsdjzpOkHUWB0u8di1GqimsaSVL21Vfq7c5KI5jaVM3TCMr9dEaHKpFANOhSqstrZfrx3hM2GxtV767FyrsZRenagTO/7yena7HQaD4ZwWRBBE7CGNXsRCGivFzPcR4jhv9Zm0GkutqSDQdhVZyr/p7VF6Hq5ho2oGZeUmLUzyVt5XKQ6qGtpP7AiRndb0CimFX6jVWL6RnVZbWqsTUkD0b3/7GwDe0PX2228jISFBvM3tdmPDhg3o169f666QIIioRzqcMarTWB6xkhynR6ln0rtTTAlovJ4dldJzoO167bR3NdaXO0/jya/24183D8dFvTLa9LWUKM+hhvGtxhJSi0pBpqymq7TaW3+BfrB7ei/5a2bocrPQanz9R4HwjewE9zhvNFLw7ER+ZCcksfPqq68C4FX/m2++KUtZGQwGdOvWDW+++WbrrpAgiKjHFSOeHafn27lRr4VJr4FN4s3RSNNYKk0FgbYbBtrefXZ+PlKJBrsLW4qq213sKI9Vo1KNJaaxIjCyo5bGsjndmPjKevTNTsQ7t1wQ9HO21KCsjOwIIieSS89DEjvCkM9LL70UX3zxBVJTU9tkUQRBxBZysRPFaSyPWDFqNTAbdLA5vZulzKCsMi5C+Xtr0t6RnQYb/x63d0UToGZQ9m0qKMxqU54XpWeqqqH9IjtCarNOxaB8qroRp2uacLY+tPW0dFyEN7LDnzedNko7KK9du7a110EQRAwjTS1Ec2RH2NwNOo0obAS0ktJzu7/S8zb27GgYfoNva8tOvUfs2F3Bv9f1NmerzExTih0N4+vZMfhJYyk9LlXtaFD2VmP5RnaE2+wuFi43K5aEN4dPFV7QkR3+gYLIiTrPjpTTp0/jm2++QXFxMRwO+Rv+yiuvnPPCCIKIHaSpmqZoFjsesaLXMogzyMWORlZ6LjUoq5ehtybetIQGDjcb9Df8ltJgF8ROcOLtk22n8Mjne/DytUNxzfDO5/TaymPTMCql536qsZTnv7IdIzt2ZyCx4/28WB1uJMcFJ3ZaHNlxyyM72mhLYwmsXr0al19+OXr06IFDhw5h0KBBOHHiBDiOw/nnn9/aayQIIsqJmdJzSWTHbPCN7Oj1/CblYjk43Sz0Wo3s3LRVNZbwEloNA7jbIY1lDy2N9cjnewAAD3+6+5zFjo9BWeNrrPWbxlKIg1DTRm6WQ02jAxkJxpAex3GcKGjq7XzXbamwsDulBn8XkuOCi4C1tPTcXzVWJKexWlR6vnDhQjz88MPYu3cvTCYTPv/8c5w6dQqXXHIJrr322tZeI0EQUU6slZ4bdFqfZnRaLQOj3vsnWfBoSM9NW/XZEdJYQlqiravC6z0m22AjOwKh7KWldU2qJfQ+pecq1VhiGosLnMbac7ouaFMvADz+xV5c8MIqHCy1BP0Y4XWlL1OvMClLz6PVHvyXBWHt+hDfd59qLG3kp7FaJHYOHjyI2bNnAwB0Oh2ampqQkJCAZ599Fn/+859bdYEEQUQ/sVKNJURmDFp1z45RpxE3dCGV5WLVzcqtibCp6z27fih9diostpBN5fWiQTm09zrVHFwftx/2lqJg8Rr8c90xn9tcig1eo+izY9Jr0DOTb6viI3Y85z8t3oAEow51Tc6QhMuBUgs4DvituCboxwDyvkuAbxdlWRrLHvx74RU76mm75h7njex4uoBHW2QnPj5e9Onk5ubi2DHvB0oYFkoQBBEssdJnxylGdhjfNJanR4o4+dxTkSUVgsoOvq2FLI2F4NMZNVYHLn5pLW7895agX8vhYmWG2uaQCq9Uc3DpmcPlDQCAI+X1PrcJxyY0DlR6dqYPzkVSHO/w8PHseM6/SafBqO780OuNx4Lf86weUVhS2xT0YwDf86QsP5dFdkIQni6l2GlhNZZHN0Z0n50WiZ3Ro0fjl19+AQBMnz4dDz30EF544QXcdtttGD16dKsukCCI6CfWSs/VIjvCRqEcBiqbjdVGpdpCBEP4ph5sOuNMbRPsLhbHzjYE/VoNksiD1GviD2nFU1KQXhShT5GamHKJYscbjZAawq8bkS++F/7SWFotg4Ke6QCAjceqgloTADR6Ukxnas5N7Ci7KEtvb2xBGstf9Zn/x3mqsRSRnagYBCrllVdeQUMD/+FetGgRGhoa8PHHH6N3795UiUUQRMjEZOm5SmQH8G7Cwn1ls7HayEzDKSM7QX7DF8RbMKJFQOixI318IKTCINjSeyHto0z/ANLIDn+eGQay5o4ju6dh+8kaz33ljxU+p3qNBhf25Jshbi2qFs3kzeGN7NiCOg4Bu08aq3UiO6LYOcfITtR1UAb4sRCnT5/GkCFDAPApLeqaTBDEuSDdxJscbnAcF9GVHS3FW3qugTJIIWRShE1Y8GHIp5630SBQhQcjWMuOIHIcbhYsy/k051NDmoIJps+ONOUTrBAWNn+biggTNnSjJ4Km1TCYU9AVhWUWzCnoBoZhRN+UcvOXDsDsl5OIVLMeNY1O7C+xYFh+SsA1cRwnrv9MiGks5XH4RHacUs+O/3N0tKIBBq0GXdLNALz/74TIDschqP974ufFp89O5P6fDTnopNVqMXnyZNTUhGawIgiC8IdbEb1oqxLrcOMvsiOdaWTwiey0Rwdl+Tf1YL/hS9+nYCurpGmsYErPpcIg2B5MQkRHTUy5lWksDYP0BCP+dfMIXOgZXSEYbX2aCgpzzLQaaDQMeniMzGV1/Bo5jsP2E9WoUWk2aHex4muXWWwhVdYpjyOgZ8ePQbnB7sJlf/8ZY/+yFi+tOASnm/WJ7ADBmZS9fXa85xDw7UQdSbQowzZo0CAcP368tddCEESMokzPRGtjQb9iR/KNWDDOChuYdFNsuw7K/L96rXrJtT+kYkUtZaSGNI0VjECSip1gK42EiJNqZEcRzVCLYvhL5ykjYOnxfHVYpWdG1q5TtbjmzU141NMXSIo0KuVmOVSE0KMnFM+OvzRWaW2TeD7+ue4Y3t98UhS1wrkAghO6yvPgTWM1+9Cw0SKx8/zzz+Phhx/G8uXLUVpaCovFIvshCIIIBWWjt2j17Tj9GJSlxk6DmMbyjey0VZ8dZWQn6DSWJOIQbGSn3i5JYwXh9ZGmsZqCFFSBIju+BmXfxwsCSBlIE94/IX2T7mkOKAwEFYSZWppKKdRCSWX5VGMFKD3393+nplEukCob7GJEVSp2lD4lm9ONlQfKZYUDPtVYik7KkUiLDMrTp08HAFx++eUyVSzk+tzu6PxDRRBE26AMnUdrRZZYjaXTiJstoIzsKDw7sg7KbZvGEmYqBVuVc66RnWDSlVIzr9PNBWUGFqqx1CI7aqXnSgQBpDwPyohGRgIf2amy8lEaQbypCT+lCAml/Nynz44yjSU5zgY/0a9qRWrN7mS9hmuJ4lNGdv636SRe+P4g5k/qg/sm9Obv41ONFfkdlGkQKEEQYUfZhj9aIztiGkurkXVQlnodpJ4djuNkQrDtIjv8v6GWnkvFTrCRHYs0jRWEQFJGQBqDmP3kFR3NR3bUfCb+0lhOyQwxwJvGEiI73pJ339dVppfOJbITuPRcXezUNMrFjsPNisdn0Hk/i8pjLq3jxeaJSqt4nb/ITiSnsVokdi655JLWXgdBEDGMb2QnOsWOsCnpFWksnUYtssP6eJnaeup5qKXn9pZEduzBR3bsLrdPRCKY2U+i6FCL7IjVWP67/oppLJ/IjnoaSxgIKooslddV9r8JpddO86Xn8kGgaqhGdkSDsvccKKNZwrmslDzen2cn6gaBbtiwIeDtY8eObdFiCIKITZRVRtFqUHZK0lhmg/fPr1YW2eFFkMPF+niZhPN0qMyCwrJ6XDGsU6usy7f0vH3SWE435zPUUopUIMQbtLA63EEJYSF9ZQtYjeVJY6kEibR+IlzC+RcNymIaixcCgTpDKyM7oaSxhOdLMetR2+j0iew4gqjGqvVEdhiG92Q5JNVY+gDVWDbP+a6UGKqVs7E0YmQnysTOuHHjfK6T5urIs0MQRCjESmRHXo3l3WCkm4RQBsxHduSbpiCW5n+8GwdKLeidlYgBeUnnvC6fcRHBprFaUHquHGLpcLE+DRYFBLGi1zJIMOlgdbiDEsKC8FKKKY7zDtQU01iqnp3A1VjCJp8hGpTtstdVE36CD02vZeB0czhZ1djscSiPJzvRpCp25NVY/iI7/GNyk0woqbPB7nJL+uVoRBGk9OwI74HgSwL8R3YiWOu0rBqrpqZG9lNRUYEVK1bgggsuwE8//dTaayQIIspxKapCHFH6hUlqUI7Tq0d2hPSKWmRH+EZdZuF9FOWW0Drx+kM5CDTYPjvS9EpL0lhA4F47QoTGpNMi3hMJC6b83C7zEslLvgUCiR2xqaBPnx0hDSkvPa9pdMLlls/8UkbHhGZ/5+WnAgCOV1pRp6iQau54clNMAPiUlHRtwfTZETw7Ocn8c9idrEy0eHsLyR8nvAdVDQ7xmATRrRWbCsr77UQiLRI7ycnJsp+MjAxMmjQJf/7zn/HII48E/TwbNmzAzJkzkZeXB4Zh8NVXX8luv+WWWzzdLL0/U6dOld2nuroaN910E5KSkpCSkoLbb79dHGVBEETHQPijK/hYnK62qToKN8JxGbTyPjvqkR23j2dHMC0L3+zrQ5hwHYiWjouwtyCyIzUo84/zL5KEKI5RrxXPV2MQosomE2Hq4zakHZSV+DsPSmNuitkgmnKrGx2y11X6kYTITufUOHTPiAcA7DwVXHNeUewkx4Fh+Mib1HAsFZ3+DMqCZ0cQOw6317OjYRgxFeUT2RFmtLGcWPLuO/Uc4vNEKq06tis7OxuFhYVB399qtWLo0KFYsmSJ3/tMnToVpaWl4s+HH34ou/2mm27C/v37sXLlSixfvhwbNmzAXXfd1eJjIAii/RG8EILY6YgdlH/cX4Yvfjsd8D7yyI68g7KALLKj+JrtYlk0OrzpB2VKqKUoB4EG22enOc9OcVUjXl91RJZ2afAROwEiOx4hZNJrxCnxwaSxpAZhqZiSipdAkR3hukaHG0vWHkVhGT893Vuq7Y1kpEkqsuwBqtOEyI7ZqMV5ntESO4trmz0W6TEkGLVIMwuNDO2S25tPYwmeneyk5iI76mIHAA5X1OP1VUdwqppPwXmrsfyfy0ihRZ6dPXvk3SE5jkNpaSlefPFFDBs2LOjnmTZtGqZNmxbwPkajETk5Oaq3HTx4ECtWrMC2bdswYsQIAMDf//53TJ8+HS+//DLy8vKCXgtBEOFDjOx4NrS2qjpqK1iWw/0f7YTDxWJC/2y/1ULS0nNpZEe2CUs9O0qDsosLKBxavP7W6LOjIlru+t92HCqrx76SOvx7Nv83WpnGCih2PBttnF4rGrqb83O5FeNG/EZ2RLHj+xxClGPHyRrsOFmDPadr8a+bR/hEdgAgLd6AygaHR+zI03pJJu/nQIjsxBt06JudiC92nsFvxUFGdjzHYNRpkZFgRJXVgbP1dvTzbI3KNJbafCshspMrpLEknh2tlvGOCvERO97nvv6tzbLbO1IH5RaJnWHDhoFhGJ+c5OjRo/Gf//ynVRYmsG7dOmRlZSE1NRXjx4/H888/j/T0dADApk2bkJKSIgodAJg4cSI0Gg22bNmCq666SvU57XY77HavKqauzwQRXlzKNFYHEztNTre4KTTY/ZdGyweBesWOVDQI6RWHWuk5y8oaytW3mtjh/w21z44skqES2TnkiYisPFAuXqeMRgVKYwmbvEmvFSM7zTWcVD6fLLIjEzsB0lgKoSCkb5SeHQBIjzcCaECV1S6PKCnKz4WIi9mgw3ldeN/OrlO1QQ1QFY7BqNMgI9GAwnJlZMd7jMJsOaOkd47TzYrpQzGyI/l8aRlGFCr+0liArxASIjqpZv7znmIO3BIgnLRI7BQVFckuazQaZGZmwmQytcqiBKZOnYqrr74a3bt3x7Fjx/D4449j2rRp2LRpE7RaLcrKypCVlSV7jE6nQ1paGsrKyvw+7+LFi7Fo0aJWXStBEC1H6F/ijex0LM+ONNoQqFGetBpL2qJfGomQenbcPtVYnMzU2mppLEXEoiWzsdQiNELlkQDHcWJkR6dh+I05iMiOSe+NhDUX2VGKDH+RncxEvpIq1ZOGkqLUHoL4VovsCOXnlYrIjvJ8CF6aeKMW/XISEafXot7mwrGzDeidnRjUMRn1GrECrLLe4XO797XcMrFT6/nMMAyQlejx7LhY8fOl1XgjO4HSWEoEcTxreGeY9FpM6J8d8DjCSchih2VZrF69Gl988QVOnDgBhmHQvXt3XHPNNbj55ptbtV309ddfL/4+ePBgDBkyBD179sS6deswYcKEFj/vwoULMX/+fPGyxWJBfn7+Oa2VIIiWI2wiwrf3YKZhRxJSH0mgtIxwXNJREYB8sxI9O27WR/S5JN/QgdYzKPt6dlogdlQ2xW7p8ThSwReMCFO2hWNKTzCg3GIP0rPjrcZqTuwoe+tIN2tWNOQC0wbnQKM5Dxf2TPd5DmWkxSGKHWFMgvf9k5af2/x4hQB5ZEen1WBI52RsKarGzuLa5sWO+LnRItPzemf9eHYAProoFXGCXyc5Ti+KRn4KOzzHw/htO6CWnhQQmiuaDTpcO6KZPdTtArQtiq+0CiEZlDmOw+WXX4477rgDZ86cweDBgzFw4ECcPHkSt9xyi9+0UWvRo0cPZGRk4OjRowCAnJwcVFRUyO7jcrlQXV3t1+cD8D6gpKQk2Q9BEOFD8KaYOmgaq9HZ/CRvjuPEjVgQNAJqkR31poKszLOjTGOtPliOZ77ZH/L5E9NY2lDTWBKPispxC0IA4McNSNeb6jHaBhK2TQ7vJi+msZoReMp5WNL3wy2KOg2MOi0uH5onW6OAMo0lrNHFykUhIB8ZIffsqFdjxRv54xBSWcH4duwSo3ZGohDZUU9j8a8lvyz4ddLMBlmHbjGyo2VEc7GvZ6f5yI5fHFZgy1vA+9cAL/cGnK3TKqElhCSzli5dig0bNmD16tW49NJLZbetWbMGV155Jf773/9i9uzZrbpIgdOnT6Oqqgq5ubkAgIKCAtTW1mLHjh0YPny4uA6WZTFq1Kg2WQNBEK2PWxHZ6XBiJ4g0Ft97hf9dOhcLUPR/0QdqKsjJRgUoDcp/+bEQh8rqMbF/Nsb0zghq7dIojhCxCLbPjtwI7Hvc0vexsLweA/OSAQAJRp14DoIyKBu8peeHyuox/5NduOeSnqoREeU6hMsfbi3Gr0crAah3TZaivF2M7AgdlCUdh8XJ51Z5lMonsmP3RnYA4LwuKQCCq8iyyQzK/GurRXa0GgZulvPp1iyUqafGG2SDZuWeHd8UJsdxAcWONtCJPLkJ+OoeoEZiezm1GegxLtChthkhiZ0PP/wQjz/+uI/QAYDx48fjsccewwcffBC02GloaBCjNADvBdq1axfS0tKQlpaGRYsWYdasWcjJycGxY8fwyCOPoFevXpgyZQoAoH///pg6dSruvPNOvPnmm3A6nZg3bx6uv/56qsQiiA6EsKl7Dcody7MjTWP5C/tLU1UmnVb1PgBg0HpFgNpsLFlkxy737FR6BlJKu902h/QltOeUxvI9bqkYOlxWjy5pZgBAokknepYCGZTFNJbOW3r+i0ew7DpVizUPjfN9jGJzFoTAC98dlPiFAqsdZQm1cJzK/jIAkBbPm3KrrfI+Oz6eHbEaS4jspADgy7ktNqesckuJ1KCc4nm88F5znNf3lGo2oLLB7tNYUOienGrWywbNSsWbWjUWPyzU77LUIzvOJmD1c8DmfwLggKROwKi7gV4TgKwB/p+sjQkpjbVnzx6fpn5Spk2bht27dwf9fNu3b8d5552H8847DwAwf/58nHfeeXjqqaeg1WqxZ88eXH755ejTpw9uv/12DB8+HD///DOMRm/Y8YMPPkC/fv0wYcIETJ8+HWPGjMFbb70VymERBBFGWNbbwt/UQfvsBBPZETZurYaRVfMoMej8l5673Jzfaiy+2SC/ASrHCQSClUV21Lvo+kM2CFRFtEjFUGF5vRiJSjDqxAhDUB2UJaXnAsfPWtUe4iMybE43XG5WVvLeXPZFKXaECJXwr07y/sUbhc7ObkV1mr8+O/z9sxJN6JwaB44D9pyqC7ge0bMjMSif9aSxpP9XhJSaVTF0VIzsmA2icdnuYsXPrdmgFaNZ0qieMhWnxKeS7fQO4M0xwOYlADjgvN8D924CLroPyB4Y1nkSIUV2qqurkZ3t322dnZ2Nmprg+gYA/IytQN8gfvzxx2afIy0tDcuWLQv6NQmCiCyk0QuxGivCDMov/nAIWg2wYEo/1dul5dD+0jJiZZFOE7CQQyoClGkshyKyI01jNTrcYkSsNsgxBID8m7zXs9M6g0Cltx8pbxAN1YkmnWzT9YddUo1lVszP6pQSp/oYn8iO0+3TaK+5sQbK2wNFdkSx43DJ0na+Php5ZAcAzu+SitM1TfituCZg2tEuGZuR5fHsVFvtcLOc7PyleqJMyshOjeDZiTfIqgCFz5LZoFVtKhioshCQiz5UHgXemwk4rUBiLjDzb0CfyQEf356EFNlxu93Q6fzrI61WC5erdaoDCIKIDaSbrTkCDcoNdhfeXH8MS9Ye89u9N5hqLGmUIhAGFU+FgItlxZ4vgDyyUysRQaFEdjhZGsvTVLAFHZTVjlt6XXF1o7jeBJPe6x0JsKE2iWJHG4LY8TUoKxsZBvSawH8ay6ni2Ukwemd2BYzsOOSRHcCbymrOpCw1tqfFG2QjI6SvkxLHR3aUvYiOneUr4rKTTLJKQKFKK06v9Y6LYDnsO1OHm9/Zgh0nA69Lq2F4E/KprcDnt/NCp8uFfDQngoQOEGJkh+M43HLLLbI0khRpoz6CIIhgkEYvhMhOJKWxpBtHk9OtOqFblsby40GRbtyBMKp4KgScLnkay+FmYXfxPVVqJbOSQonsqKWxgvXs2JuL7EjeRxfLiYNLE6VprADvtU0mduTbVYJJfftSnn+b0+0T6dA2Z1D26bPDnw+3WHqunsaSRoSk63C6WVEwSSM7gmHbX0pOfC6JQVmn1SDVbEC1p4tyouc8xOmArERe7Bwu986HdLpZbC2qBgCM6pEmFztiZEcnRnbcHIdPt5/Cz0cqxfslx+lh0mtgd7Gyz1am5QDw+n2A1VMVHZcGXPMOEJca8HjCQUhiZ86cOc3ep60qsQiCiE6kkR3RsxNBg0Cl35z9VaY0SY2pfnwOwmOlZecaxjeKYlApDRZwsqysGgvgozvGBK2s2aDg3QkGmdgJMY0VaBYU4OvHOV3TBADiBg34P1+A0rMjF4n+vD7K57O7WJ8S/eYMyj5pLDc/gNWplsaSCnTJx0N6PqRiWCrahLlaUqGqht3lRiZqkH3gP8Avv2EpjqHJ4Ebm911h5hqxwXAInTSVcBzpBLN2DPbtHwVuZj8wWh32nqmD1eFGilmP/jlJYBgGBq0GDjcrvq7ZoJU0FQTKPKJUEKfxBi2+njcGLMdh4l/Xo97uwjjNTozf/A/A3QSYM4CM3sCEp4CkyCwOCknsvPvuu221DoIgYhRp5ZXwrTOS0ljNbehAaJ4d6ZgIg07jk3YRvCwOl7epoFHHf6t2ulnYFKm0epsLGQlGWRpLLbJT1+TEHe9tw8yheZhd0E28Xiq2Qh0X4QjQV0Y4Bilnanmxk2DUie9xcJEdjU9EzV8ETa2poDKN1Xzpua+nx+nm4HYLs6S8TxBv9Bdhkood/vX1WoYXszYLcPhHdDq5HU/qTiLbXQ3u3y+BsdcDWj3AaADLGUBrBJI74RW3GwXGvTD9yr+vQwDehHLqEACgi2c5cdbTeEz/EeD8CNxz84GkPFTl3Q2gJ0Z3TxePy6gTxI6nSqvpBAqcW6FnjOCcg1BusYMBC6ulBsOYk7gYZcg8fBporMQL2l/RpHNjlnYDdG4W6Dke+N1/AWPgxojhJnztDAmCIOCN7IgbASJL7EijOf4iO8GksdQ8OwatmtiRenY8JfkGrVidpYzsCCZlqcCpVfHsbDlehW0nalBvc8nEjjRlpT2HPjtq3hvh9kSjDvV2F07X8NOyE016sRdM4A7KXmOuMo3VnKj0XmZ90lihlp4D/LEI74deIob0Wn70h1LYSddhtbvBgMU0/V7gw/8BR1cDbjviANwuHNYZP4upL8Glnpdz5pwP/cCZeHs/sKu4Gr8fHI/czDQ8vLoBbFJnfD7FgYMr30WXxn2IZ+yA5QwmWZ7Ei7pxMGfPA6xVwJkdmK/5EG6dDWe4DBi0Lgz95mMM41jACHCfPInhMMJssgEcACMAO4Bv+TVcDojK4VSXK5F/49u8QItwSOwQBBFWXJL5PEL34EgSO8FEdoIxKEu74AoY9VpAkWKRenaEyE6cXotaOHn/iee1MhKMqGywi/OxapsCe3aE8uMmhRhQi+wE49nhOC7guXGznChkMxONqLe7cMaTxkoweSM7gQzKQhSLHxcRXBrL16Ds9mm+2FzpubKDMsBXCKrNxgL4SFW1ywEGLPowp9GFqcDAMzuB1RxQX4bss8XYYDiIfJwFCj0PSu8F9JqIdzaXoMSViLuuGI/sjEyAdfKjFZLyALcTrtpTeP7D1TjEdcG/br4fyfEGHCjdheUnzmBwXj8gPwXbVm1GT0M8cP44HOAuxYxPd+KCLA4fDN4J3a+v4HrdOmDjOmAj/9K3AvLdnwNOafNhclmQydTBDG+n4youESeNfXB+t0zAkIBlx42os1hwhO2ESSPuR34HEDoAiR2CIMKMt5xXA70wKiGCmgraQ43seDbbXadqcf9HO/H49P6YMjBHFETShoIGFaesEN1iOa+AEFJfNRIR0ynFhMoGuzgrS+nZ4ThOVuIuCCDlKAGpZ0rjp89Oo8OF3/1rEwp6pOOPM/jGcC6Wk1VyKc+NVIxkJBpxvNIqHk+iUSeOfQiYxnIJHZTV0ljBR3aUM8SaKz1XC/w43F7DuPA5hb0eOLoaT+JDpOlLMEBzEpmMp2fOSc8PgEQAiRrAijjEX3gnMPR6vsEew+A/u9fgTG0TLsu5ENmeERLC+b6oVwb+MP4yLHXzgsLoOQfSsn3p3CwAGNM7Ayw02HoWKBz4AJ5ba8ZdhhW4VLMLDOsCEnOxvGkQTtuMGMocx0BNEdiLF+C+wlHYeaoOf52eh9d/2AkrZ0ID4mCHHhd2zsCyG0cDAL57ezN+ra4CAExtzukdQZDYIQgirAjRC61GksaKoD47zVUcAepprFUHynGyqhE/7ivDlIE5ssoigcVXD8bs/2zF/El9xOuk06oFQaCs4Io3aJHimS0l+FGk0Rynm0Ojwy3zkwhCSVk+L0RxNIw3oqE0KO8+VYd9Zywoq7OJYkcZWVGKD+ntwoRxgUSTDjWNQul5EJ4dHT8ItG92IgrL6z2P8z+WQ3iNepsLdpdaNVYL0liSvkeJjaeA5S8Dez4BHA24CgA8b1EDZ8JRLg/6lE4Y2LcvkJiLrVUm/G1bA/RdRuDdyRNlz5ti1uNMbZMs9bj3NH++z9bbcdfFPcTrBXEsTXWKw2U9EcPMBKOYVvvtZA02swNQlzoa4+8bA7BuQKPF31/7GYUN9eLz/lYwCdoj2wEAJ2xmFHPyfnrSz1+8JJ2oC9AcM9IgsUMQRFiRenb0EZnGaj5F1aQyCFQY2SB4U2yKTQkAxvbJxMFnp8qiFtKmb0LKShnVSI7Ti6XXamksAPjjl3ux7UQNvpp7ETITjWLlTaPDJYv6CIEdrYYR0zvCdcL9hKoc6cR1tU7FUuxu7+VMxbDNBJO39Nyu8l4LAkycCeXpA/PtH8bgcHk9Lvv7L81GdlLMetTbXLA51frsqD5Ucru6ZyfOWYs/6f6NS1dtADjP8aX1wLe2IVhfl41iNgs7ud5wQocr8/Lw2gx+OsD33+zHL+wJ3NbJt1IpxcxHbepUPFeNDm9XZoNO4zUY671CUTpKAuCjc3nJJpyoasTWE3yfnE4pJr57sWfquHIQLd9BmX9uwUQuRZp6TZAI6OZEYyRBYocgiLAi8+zo+D+4kSR2bEGUnjeqeHaE2UXCbWrVWICvkNFqGHGgoxDZUT4mKU6PJI/YUTMoA8BXu0oAAF/uPI27xvYUPTtCekz4ti5EcRjJMEh+9IQT01//GZf0zURXz0wrh4uFzemGSa/1iey4WA4uNys23HNINumkOLmvI8Go4/1K8I3ssCyHa/+1CXotI+lNpBGfS4hWNde8MTlOj1NoUq3GCrhJN5xF1sY/42V9ITJQh/M0R2CEE4Z/6fGWywG9zsUbd3tN4scgdLsYn767DRuqz8qeRrq+/SV8amtQpySflxMaAUobQdZJUo5iywKJCJalsZzyNBYAdEqNw4mqRmzz9NdRNmCUpk8Zhn9uIapXWus7mVyaepVGC5udeh5BkNghCCKsqHl2ImkQaFCRHZXZWNWeFv1CCiXYDsoAv/k0OrxmZOVjkuL0SPQMjhT8KP66Jqd60l1Sv0+Tw+0jdjSMd3QRy3FYeaAcZ2qbsGxLMW67qLv42HqbSyZ2pL2C7C5fsWPUapCsEDuJJr244Sqr16obHWLnXmEvlR6/NIWjhuDzEV7T7mJ9DMouf2LaZgHevxopZXtwjfJtcjmhAXCQ7YKmyX/G+WOmizcpzdPC6wK8eDtQYgHgbSIoJdkT2ZFX0/GfHTfLiT2CpGJGeg68nh2vgMlL5sWN0C8nTyF2pJEds14LhmHEaFZpnW9kx6hXFzvNeZ8iCRI7BEGEFalnx2tQjszIjj+fiKypoJDGavCksezyyI5J33zo3+ARO0J/FmX0JyfJJKYTxDSWZ7OM02tl6xF8FdI0SaPTDaHHrWDP0UgiOywnT1ccqfD6O+ptTmQmGkWxkWjSi0LL5vT6hIT30KBTEzs6ccNVRoikwkQQUXEqYsfp5qu9lBuuEOkQIiZqkR2lhwf2emDvp8CWfwFnD8Edl46XLZNghRE72D6o5RLwxk3D8NefDmPD2Tj8N/sC2cPVeu0I5+dElRVWhxtGnQY9M+N97pfiOTf+qumEdGicQRrZ8TaeFF5Hmv7slCoXNz5iRyKc4jweHCGNVaIS2ZGe/wSj93eK7BAEQQSJGNnpAJ4df1OgpWksQdRUedJYomdHYrZtDmEzE4RSnEIg9c1JFDsKC9/8hc2ya7oZh8q84kTY/GskXXqbHFJBIUR2GLEKSWlQ/k0yI0l4PUHUmfQaNDk0ntEV3vMjTWOpiR1vZEchdpRCBIrIjuR3h4v122xQSJ05VGZjNUingv/8CvDzXwGHZ8SCMRlnr/gAbyytlj8mrjNKmDpwaPDp05OgInaEz8p+T1SnX26SbKaWQCDPDuCdbi41BktTgMpqLMBX3CjFjzSNJXyOBK+x2hcN/56djiN2Oo67iCCIqMQlmTdk0Kp/2w8nUk+J37lXCs+OzekW00veNFZws7EA77d0q59qrD7Zid40ls0Fm9Mtbq5d083y9bv4UQfSaIFUnAlik2EgiexwsmOVTg0XxI40ciOIM6mnSSp2hA0d4FNTcXqtuGH7RHZUxY5vVIM/Nt/3Q2pQFi4r01gNds+52PMJsHoRL3TSewFT/gTcvwvunGE+z8s3FfQKcynxRrU0Fr8OQewMyvP16wDeCFStimcH8IodqciQpbFEE7cksqMUO4HSWILYCSBcTH7SWM01Z4wkKLJDEERYcYuN2jTQR6JBuZnIDsdxPuMiBL8O4BUKomdHxd+hRPiWLjyvsh9P3+xEHD3LR2+qrHYxjaTVMOicKhc7NqcbjQ637Bu7VOywamks1rdEXUBIm0mNsUY938tGen5EsaPw7CQYdWAYxq/3RilMAHk0TOepGpP2IZIfr9egzD+/b2RH42wEtr0N/PQkf8WY+fxcJ8/xa52+qRxp6bkyfSPt7hxv0MLq8IoQwZys5tcBpJ4dSRpLktISxI5ZRezYVKqxALm40WsZn2o46edJiIwpy+0zE43ia0dDZIfEDkEQYUVo1KbTMDKDsrIpXrhoLrJjd7GyLsR2l1tMYQFCJ2RWFE0mXfPfhoXNSBAlWkUkoXNqHOo90Ylyi12M2iTH6ZFqlqeM7C5WlsIC5EJGKPPmS8+9kR1lp2UBYeq6GNnRalSFi92PZ0eISEkHnkqxOuTCxKDVyGZV8UKJ9yWp9egRIjvJJh1yUQWN0wSr6FACOuEsPjD8CfiunL+ixzhg/BNedzbUOyw73Zzks+o/jZUUp+fFjue4hFLubhlyESqQLHp21GebnW0QIjtewSdEWuQGZe/tOckm8ffc5DifWV/BRHZ6ZSZIxI6fyA712SEIgggOl4pnB+A3F6EUPZw0F9lRRkDsThaVHlOpQKPdHVIaS9iMBLGjl2yuSSYdNBoG2Un8hlbZYEelZ0NMidMj2VN95V2z26csXT2y493kOc53rISAmMaSpKmEb/42J58yK7fYxdv1WqXY8fR60amnLJUTytUM3Ua9Bk1ONxxuxRrt9ehn34NbdWtwzcqtuMHUABenwXLnRdiu7YUkWHGjbg06M5VAUifgwvuA4XMAjfw9URsE6nC7A6SxJGLHpEdpnU0UfpUewZClaKwoIPXsVFsdMBu0crGj5tlRMShLIzsmvVYcJ5KX4hU+3sdLDMp6j0FZIvbiDVp0TTdj0/Eq8fnUjpUiOwRBEEHiVvHsAHwqyxBEFKStsTfTZ6dR2UzPxcoiOwAfrWgKofRcOA9ClEO6ufbITAAApJkN0GsZON0cjni6CifF6cXqHul6fMWOr0GZYRhJo0HOZ7q6gEU0KHs3WafnmGwuNx74eBe+3lWCWed35o9Fp4FJrxW7+nrFjrdXjBRlpZTa+ZKmccBxwPF14La8CRz5CX/nWH5ncwFOTgs948aVzM+4UvOz+PhjbC563rGKnz+lgt8Oym71NJY06iIIO5uT924J5ysjwY/Y8Xh2qqwOjPnzGhT0SJe1ERD6NcXL0lgqfXYUorBTahwqG+zolOIbUZL+vxIiO1KB1yMzQSZQTbJqLOqzQxAEETLS4Yp6yaYeKb4dtWGXHMfh2FkremTEyyqb+Pu4xbJzgUaHSyxbD6b0XDDvNnqqhqSbSo8MvnxZo2GQlWjCmdom7CiuBQDkpZhwUa8MjOyehtPVjSjxRBh80lhOaWTH22fH20HZfxpL8OxIIztGN39M/95wHBuP8dGA3af5NQnCJCVOj4p6u7hZ+ovsKP016mLHk3op2w2sehkoWg/hDJ3h0rGN7Yuhl9+PSZ/bMYA5iWu165HB1MHNGLDT3R2fuy/GLj9CB1AfBOpwc5LIjrIDsTyNBXg+Bx7vll7L+FSkCUjN240ON345Win7zFV4euVITdBiB2VJGkvp6+qUYsLuU57uyQqMKmJHGqzqmRkvawQpTb0KnbsBiuwQBEEEjdQHodNqRPNppPTakUZzhN8/2FKMJ77ahycvG4ALuvF+EIbhgwx2FytucgINkjSWshuyGgZFvyHp5jqok9fompVkxJnaJmwt4gVG1/R4pMUb8Mn/FeCNdcfw5xWHYHOyMvMrIE9j+euzE2way6jTwM3yxyQIHQCo8ZwD4ViSBbHj8eyI0RmXW+bPCiaNNZg5hkf0n6D/t1v4K7QGbE2/AgtPjcQxrhMA4JdeY+HCWuzhemKPqydMeg36ZSdh16la1eOSojoI1MXK/GVSpFEXqTFaSGFlJBj9+s9Mei2MOo0oWnw9TG6f1xDTWE5WfJ+kggsAbhzZFVUNDlw+zFfUSSM7cX4iO0kSUSOL7EheR9GhIKIhsUMQRFhR+iD0Wv4Pf6R0UZYNAvX8/t2eUgD8sM+BnpLi5Dg9ahudcEg2OYFGSaVSKJ4dAZ2GwTMzB2BLUTVuGt1FvD47kf/WXm7hX08Y6wDIfR01ATw7QjWcss9Ok0O+6abFG1BtdcDSJDcoG3Va2eR0YeMWoknCxiqIACGNJUQOOI7vAp3kEUEB01huJ7BiIZZY/y0O3sTg3wHjn8BHP9XgWPEZAMA1wzujU0qcTEQkGPX4+w3n4fEv9+LuS3oiEGppLKeblfWEkiI3KOvE4xK6EftLYQn468zt7zWkaSwhJaksfx/TOwNjemeoPpfUs+ON7HiPqWdmglh5Big9O81/fiMREjsEQYQVt6Kc1yCInQjptSON7NidfIRmRzHfZG/36VqxVDrVbBC9MSWKlvtWh9tbjRVMGkvrK3Zuuag7bpGMbQDkVTcAH9kREDYom1MljaXq2YHcs6OoPOuVlYCtRdXepoJObxoLjE68z/TBufjb6iOi8dlH7Hg2bZNeC7NBi0aHG9UNDlHs+E1j1ZcBX9wJFG0ACwZfui9C9pQFGDNmHACg0spXV7187VBcM5z3C/XP9UZyEoxa5KeZ8b/bR6E5VAeBulg4JXPcpMSreHYA4HSNIHbkpvGWYFY1KLtF4RpMxFD5eOnzamWRnXiU1XnL76XPrdNq8MDE3qhtdCI/Tb3CLBIhsUMQRFiRenYAQK/TAPbI9OzYXCx+O1kjpnAaHW7RmyL1Xihb7lvtLrFqyxhEB2WlMVvrZ0x3VpI8YiAtb5ZGdoQmdYK4UKvG0moY8ds9y0I0KM8YnAuW4zChfzYvduy+pec3jOwCcMDDU/ri16OV8mPxrD01nt/wpV6QVLMBjY4mVFkd6ObxIqmKnZMbgY9vBhorAX08Xkl4GP8o7YvXE3qL9xN8UukSYTFpQLZX7JiC3+7UIjt2l1tM2yhLz6WVUtIIjCB2Mv1UYoVCgqpnhxV9XWojK/whS2N5hIzUxN49I15mYlcK9Acm9glh5ZFB+EsdCIKIaZS9SwSTclt6dvadqcPxsw1B3VdmUHa6Zb4UAPjFs7knx+nFVi0lnt4quZ7IS4PdJRmvEPy4CAG9HyOokMYSHiO9LPZikUR2hGZzan125J4dr0F58sBsvPH74ejm6cysHBdh1Gswsnsaltx0PrpnxPuMThA21tkFXXHZkFxcPtTrIRGESY3E46QUO+c7fgP+dzUvdLIHAXetw76kMbI1AN7xHBnxXmExeUC2+DuD4M20aqdbKhADlZ4LHhxAGtkJLHaW3TEK143IxwMTe/u9j1o1Fsd5B8Aqx2YEQs2gfKLKKjsGoR+ScLmjQ2KHIIiwoubZAdpu8nldoxNX/3Mjbvj35qDuLx3+aXexYu+Rzp55Qzs9lVBmg3eTE44p39PNWLqZB7Mp+UR2/ImdJK+46ZpulplMvQZgVmxYl+sRO2qRHcZPnx3hm790PAXgLT1XVgEpIwzCsQzpnIJ/3Hi+LPWR5on2SDtOSzsoT9TswLzyJwBXE9B7MnDHKiCzjyxqxa+XEwdmSiM7vbISxN/3nqlDsKid70bJPC2lQdmg04jnwaTXiuJAaCjYnNi5sFcG/nzNEAzy02UZUDcoA/yUeEAeXWoOo4pBWSp2AIhpRcDXQ9YR6fhHQBBEh8at8EG09XysM7VNcLhZlFvsslC9P6TRg3qbE7s9aZH/G9tDdj/+G71cyAhpJWl1VjAdlJXPo/eTxspJ9m6iUr8OIEl1ON2iQMn2pFOkvYEEI6pG0WdHiP4Im6FgLK63OcFxnKz0XIrSwBqoV5IgdqTnx2p3gQGL/zOtxBv616DjnED/mcB1HwB6XqyJBl3PcVhsLlEcC88J8B6kofkpAIB+OYl+16FErXJKes7UZkIJx22UzAo7XdMIAMgIMo0lTUPqldEjFc8O4P1/Yg4psiM1KPPP2zebPz9C9C85Tg8Nw4vgUIRUpNLxj4AgiA6NGNnRKCM7bSN2pGXYVQ0OmNMC/xmUGpSFBm9aDYPLhuThqW/2iz6O5Di9bBNKizcg1SzfzHUaRnXytZJgIztZkshON8UAUO9IAVYUBcKmKzUoW5r43xNNOlmfHafHgOyN7PDnyenmYHOyktJz+SabaJT3k1GaraWki5Edb/Vasv0MXjH8E6NxEGCA3amTMfSapYDWd7MXUp1VDd5hmcqUy39vHYnXVh/GjSO74FyQnjO1Znpmgw41jU5e9HqEpiAylbOp/NE51Sy2MMhJNuFUtdfoLhWRDMOITRq9r39uaazXrj8Pb/98XKxUizNo8czlA+FycyH5gSKVjn8EBEF0aATPjlbw7OhC8+xwHIe1hRXITY5D/1z1ydJSqiVip9rqaLaiRK0sONWsR2q8AS9cORi/HquEWa/FnIJuWHWwXLxPVqJR/NYsbMbBeh98PDt+ZhAlGnWI0/NzonwiOzpvZEfwwQibrjSNJYi/VLNBFFUulvMZbxFv0Ik9kOptTm8zu3OK7PDrqbY6AbcT3OY38Dn3PMwaO+yMCc87rkdKv7sxVCvfqrxRK4/Y8YjJdJWqp2SzHk/PHOh3DcEinDMNoz5OIi3egDO1TUg06XwEYGZicNVYJr0WeclxOFPbhNzkOJnYUXqhjAqx02KDskfsdM+IxwtXDZbdb3ZBt6CfM9IhsUMQRFgRIjt6pWcnyDTWO78U4fnvDqJ7RjzWPjyu2ftL/TOC8KlqsOP7fWWYOjDHp3JGbUREiidic+OoLrhxlDdiYFIMYxQ2fsGTEkzZOaAW2VF/HMMw6JwahyMVDeiRKRc7Yum5ixUb02WKkR3vMQk9eFLMetGX02BziZE1adO5BKMOFpsLFpvLbxrLn0FZjbR4/vWyKzcDb9wJprIQZgbYzPZH9s1vw3xEixtGdfN5nHLUhFiJFX/uJd7+EMSOWgoLAB6f3h+bjlViRNdUH+HRnGdHSo/MeJypbUK3dDO2FlWL1yubBhp1WtSDF7EaxlcgB0Ktz060Q2KHIIiwovTshGJQ3nemDs9/dxAAUFRpDWpSerXVW2Jb7UlLLVl7DP/5tQhPfrUP/7p5OKYMzAEAuNysKMakpJnVN1WpkTM70SRuekL6K5iyc8B3Awo0XfrZKwZhx8lqjO6eLl+LZ/OzNDnFZniC2JFFdpq8kR2hfL62ySmmtKQ9VhJNelhsLtTbnGLzP6WA8zEoB0hj5TI1+Lv+b5hZwZvF2bh0PGqZhc/ZsTjWayAW9lY/buWUdeH8pocgKoJFaEwoCER/70VBz3QU9OTfg8sG54reLgB+R0Wo8di0fjgvvwyjeqTjk+2nAfDnUCkalb1ymvvcS5HNxtLHhgwggzJBEGFF6dkxhODZeWvDcdllfyMOpNQo0lgAcNgzSBMAHvpkt/ja/lJpqfHqm5dUzGQnGUXRInhSgi0PHtEtTXY50MDFgp7pmDe+t09qRViLVKwJze2kxuxaj/hLNevFTdnNcqLYNMnEjmBSduGYp3S/uyJ9ZjZoId13DWoCj+OAnR/goh+nY6Z2M9zQABfciZM3bsCn7nGINxoCbt4GRTWWWHbeCs37lAiRqkYnf86CmQf1+9FdZZdDESID85Ixf3JfmdFarWuxVGSGGp1Rq8aKdkjsEAQRVpSeHYPCfBqIs4qxDMrp3mpIxY7g9RBKhAG+z4sggmxO9TWk+UmXSDeR7GSTWMUi6I1g01g9MxNkaSl/qZNAKF8rTq8Voy5SUSicjxSzwVM27fs4AaEcudxiw8lqvtKoj6LKiWEYWfWOj9+I44DvFwBf3wutswE72V74HfsnYMbLqGf4UnFlKkyJtxpL8OwIaazWj+yI58wT2fFXGSclzqDFveN4o28oVWCy15WcQzU/zrmkooznIJQ6KrERvyIIImJx+3h2+H+DiewI3XwFahudyPOUzvpD2tOl2moHy3I4UyMf71BhsSM7ySSmSfRavuGeEElI8ZfG0qmnsQRMQaaxAL7777/W85GrQGksfyhTZvFGnZiycLo5ON0s9FqNKBCFFFZKnAFlTpvkebzHJNzn16OV4DjeI6PmR0kw6kRTtCz9wnHAT08A2/4NgIHtkj9i1o/9wEKDL347LfbCaV7syNNYVQ3+DcrniiAGhNRfsJO+H5rcF13SzBjWJaVFryuNuKidD7lgCW0rTzDqoNPwFV2hjJnoyFBkhyCIsOLy59kJwqDcoJiQLfhPAiFPYzlRUW+Hw81Cq2HEaq6Ken6zFyI7Rp1WXlbuV+xI01gmn/RDKJ1opd1/WzJdWmlYTTBqZRvoP9YcxW/FNeL5EMrkpWMvTHqNLD12Xhd+wvv3e8sAAH2y1aMW0uMW1+FyAF/dA2z6B3955uswjnsYWi1/3/mf7Ma7v57g19rMaAdhoy+pbcIrPxVip2dWWVt4dgTBKogdf92slWg1DK4f2QX9cpqvEFRDGnFRi75I399Qh3MmmvR44/fD8dbNI1Qry6IRiuwQBBFW3H49O83v8EIfkyQTXyVUF0waS2pQttpxytP4LS/FhNxkEw6WWlBRb8eyLcXiSAmTXgMNw8AiDP30k8bSSiIw2clGsYeNQLBpLAAYlp8q/p5qDt7gKqDRMDBoNWI6MN6og0GngU7DwMVyeH31Efy4v0xWjQXIzbTKb/2CAVd4zr5+UjTSSIRBpwEqDgFf/h9QugtgtMBlrwDD54ABL7IqFOnIYNNYvxXX4jdPB2sAyGiDaixldE7bgihbS5Ce++bSWHEtaPo3SSKmYwESOwRBhBWnss+OVt2zw7Iclqw9in65SeIf6npPqqRzqhkHSi3iWIRAKA3Kpzzek/xUM7I81UqFZfVYuvGEeD+jTgupbSbNj0G5XhJpSo83glUEp0KJ7Gg1DL6/72KcrLKit58ISnMYdRKx49kQ4wxacZ1FlVZx6rlaZEcpdgblJSHRqBPPu//IjndrySlZDXw2nx/5YEoBZr0D9J4o3p4W3xKxoy4a2ySyo4iq6Fvgn2oJGg0j9lBSTWNJIzsx4rs5FyiNRRBEWPHx7OjUPTvrDlfgrysPY96y31BuscHucou9XvLTeJ9OcwZlm1M+8bvK6hAbt0nFzhZJfxOA31ikfht/np06iZDSahiYFemFUPqtAMCAvCRMG5wb0mOkGGXRAf53aSm43cWKYlMQO9LIjsmnBF6DUT28lWJ9cxKghiB2btGuQP8N9/JCp8elwL2bZUIHUJ8w3lxqRU3sGHQa5CSbVO59bozqLq+MC9az0xoI6SvVyI5eGtkhsdMcFNkhCCIsuNws/vJTIdYWVgBQ67MjFzsrD/Ddie0uFkvWHsV9E7wTojul8F2Qm/PsSKM6AB+JKarkU1WdU+PEqMahMovsfka9FtIiHH+eHWVkyayIjNxxcfeA62tt5L4OTzdnq+85Muo04oYpFXJq5tWCnhlYdZB/z/xFnNL1Djytew+36n7krxh+KzD9ZdnIB4EDpRaf63ZJUlNqGBXremnWEPTMSgipn01zrH14HE5VN/qYnoMZ99FaxBm0gFU9ciOP7NBW3hxhjexs2LABM2fORF5eHhiGwVdffSW7neM4PPXUU8jNzUVcXBwmTpyII0eOyO5TXV2Nm266CUlJSUhJScHtt9+OhoaGdjwKgiBawvrDZ/Gv9cfFaIzSsyNthc+ynLjBAsCHW4txuIzvjZNg1IlppVqrE88vP4CPthbLXsvpZrH4+4P4+5qjAPgqIuEL+p7TfAVQfpoZmYl8ZEBpCFZGdvx5dpSRJZ1WIw5WfOvm4eicGng0RWsj9QgFSg1JU1eyyI6K2BnXNxM6DYO+2YmyydgA+BO3+yM8cewGUeiUj1wIXPaqqtABgLmX8iXacwq6YoYninXLhd0CHpcysjN5YDaGd031c++W0T0jHmP7ZPq8VqCeR61NwMiOtDFgiAblWCSsctBqtWLo0KG47bbbcPXVV/vc/tJLL+Fvf/sb3nvvPXTv3h1PPvkkpkyZggMHDsBk4v8o3XTTTSgtLcXKlSvhdDpx66234q677sKyZcva+3AIggiBokqr7LLSsyM1KO86XYuz9XYkGnXokZWA3adq8e2eUgB8ozthg954vBKnqptg1GlwzfDO4rfwl1Ycwr9/LhKfT/i2XmV14LhnHflpcX6bv5n0GjHypNUwSPJTLTSnoCv+tuYopno6MAPAB3eMQoPdhUGdkoM5La2K2nTr8f2ysOZQBRJNOtG7kyqJ5gTy7AB8D6Av770Iacoy76Za4Lv5wL7PkQCgiM3GC67f47ER85AdoKnevEt7o6BHBi7smQ43x+H6kfkYqUgd+R6XXMS1ZkRHibKvTkvaALQU4T1rts9OjHRBPhfCeoamTZuGadOmqd7GcRxee+01PPHEE7jiiisAAP/973+RnZ2Nr776Ctdffz0OHjyIFStWYNu2bRgxYgQA4O9//zumT5+Ol19+GXl5ee12LARBhMaJKrnY0SlmY0kNykIKa1y/LMQbtNh9qlZMPyUYdUj2bNaC/8buYnHsrBV9cxKx6kC5TOgA/ObOcfKUTn6qWXXoJ8BvLMI3+lSz3q8o+sOE3hjVIx3nd/FGGbplxKvetz2QR3b4zfGv1w7FqZpGrDlUgddW8ZFyqcBJiTNIHq8eMRjcWSHcTm4CvrgLqCsGGC02d7kTvy+8EC7o8HQzM5viDFqM6Z0BANCAwcW9M5s9Lmnvns6p/kVqa6Ac0xCOyE5zfXZCLT2PRSLWoFxUVISysjJMnOg1syUnJ2PUqFHYtGkTAGDTpk1ISUkRhQ4ATJw4ERqNBlu2bPH73Ha7HRaLRfZDEET7crKqUXZZ2EREg7JEeAgejrG9M8SojPD4RJMOKSrf7PeX8Omp97ecBCAfJZBqNsi6IKfHG5CZaPQZAipg0mvEjT/Vj18H4IXaRb0yIsYwKv32L0QHUuMNGNI5BfmSlJrfyE5zx8FxwNrFwNLpvNBJ6Qrc9iMO9rkbLs936UCDQFuK9Liam1p/rihne7Wkm3VLESbZd033PUblbCwiMBErdsrK+KZV2dnyXgDZ2dnibWVlZcjKypLdrtPpkJaWJt5HjcWLFyM5OVn8yc/Pb+XVEwTRHMrIjjbAbKxyT5O/Tqlx4kiA0jr+ukSTXrZBC+w7YwHLctjpEUrzJ/UVbzMbtLJp5i9cNRgMw8Ck1/pNiQibiz+/TiQi//Yv3xClIkFqSpb32QmwRXAc8N1DwPoXAY4Fht4A3P0LkH+B7LUCDQJtKdKNPr+NfVB6hVhTSym1FU9dNgDfzLsIl/TxjXbF4uTycyFixU5bsnDhQtTV1Yk/p06dCveSCCKmcLhYnxENQqRGzbMjzMDKSjT5VMckmHSy1IvA/pI6FFVZUdfkFD08AscrrWKvnlsv6oapg7weGyG6kyWJ8lTU28UKIH+VWJGI1FStTIUI5fqAvGlhc54dAIClFPjkZmD7OwAYYObfgKveBEx8t+BEZVPBVkYq4jqnBh4Pcq4oxdqA3Jb1PGoJcQYthnROUU3TySM7JHaaI2LFTk4O/8envLxcdn15ebl4W05ODioqKmS3u1wuVFdXi/dRw2g0IikpSfZDEET7cbqmESzH/5Ee4amiubQvH6VVDgK1Od2ikTYryejTqybJpEOySmTnQIkFO07yYwSGdE6GQafB7WP40u97xvXE3Zf0xKr5l+CpywbIHieIHKnXpqzOJonstJ0ZtrUJNPAxO9EkbuQyz45EzCn77IBlga3/BpaMBA5+y3dDvmIJMHyO7G7xbS12JCKuLXrrSFGKnYFhMJqrIfVTURqreSJW7HTv3h05OTlYvXq1eJ3FYsGWLVtQUFAAACgoKEBtbS127Ngh3mfNmjVgWRajRo1q9zUTBBEcQhSnS5oZ798xCj8/cqloelX22amw8FEdo06DRKPOJ7KTaNIj0aiD1Deq1TCot7vwza4SABANw3+c3h8bFlyKyQOyodNq0Csrwedbsyh2JD6J0jqb2C14QF5kbHbBECiyo9Ew6OSJikgFTrzBa8aWRXbK9gH/mQx8/zBgtwB55wN3rQPOu8nndQWxwzBtY+iVRjWy/PisWgtlg8OBeZHx5ZhKz0MjrHKwoaEBR48eFS8XFRVh165dSEtLQ5cuXfDAAw/g+eefR+/evcXS87y8PFx55ZUAgP79+2Pq1Km488478eabb8LpdGLevHm4/vrrqRKLICIYwa/TLT0eJr1W5h9RTj0/28B7c7KSjGAYRvTsCCQYddBoGCTH6VHT6ESKWY/8VDP2nqnDL0crAXgHWGo0DLqomD2lDMtPwVe7SnBBtzR0y4jHSysK8eLVg3HdBfkY2ydD7JvTEQjk2QGAC7ql4kSVVbaBMwyDFLMelQ0Or9jZ/h/g+0cA1gkYEoEJTwEX3A5o1DdZQVjptZo2qZQy6jTITDTibL0dA9tZfEbK+y97bymy0yxhPUPbt2/HpZdeKl6eP38+AGDOnDlYunQpHnnkEVitVtx1112ora3FmDFjsGLFCrHHDgB88MEHmDdvHiZMmACNRoNZs2bhb3/7W7sfC0EQwSNEdrpm+AoP0aDs4j07QmQny9Pwjy/99jb+S/T0vEkxG1DT6ETX9HiM75uFvWfqxOc8v0tK0Gubc2E3TBuci+wkEziOw6zzOyM7iX/t9m4KeK6odVCWsvjqIVgwpZ9PFVpyHC92ktAAfPF/wJ6P+Bv6Tgdm/BVICvxlUuhG3S29bcruGYbBuofHgeW4dq18y0kytWmZeyiQQTk0wip2xo0bB07ZqlQCwzB49tln8eyzz/q9T1paGjUQJIgOhjSyo0TZZ0cYEpnp8erotBqkmg2o9vTIEaIIQhVRt3Qz5o3vhXijFi//VIi+OUnISgre18EwjChupL93REwqs7GkaDWMarl9itmAkcx2XLnxfsB+FgADTHgSGDOfz001Q7xRh58fuVS2Ibc27VkVJdA7W30WWDggg3JoUOyLIIh2R6iuylEREkKpr+jZqfemsQTS471iJ9EzskAw2XZNj4dWw+COi3vg+pFd2qT0uaPQXGRHlcZqPKz5EBcY3ofOzgIZfXgTcv7IkF47UTlKIgrwN+U9HLTovY1hYvevAEEQYcNq56urElTGLgieHaGbsbfsXCJ2JCZlYXTD6B7p0GsZXOzpxgvwUZ+2qAbqKMgiO835OjgO+O1/wOvDUFD6X+gYFhhyHXDX+pCFTrQxc2gekkw6/N/YHuFeiojQCoFh1KfAE3JIDhIE0e5YHXxDP7UNWGgSd/xsA0rrmrxpLJnY8f4uCKa7L+mJWy/q1qapk46GsAnG6bVi00a//PhHYPMS/vfsQcC4hUC/GUGlraKdv10/DA43G1GfLeG9jTfoIsZHFMmQ2CEIot0RIjtqPpL8NDNGdk/D1qJqfL7jtI9BGQAyJF2MpemSSNqMIgHhfDQ7O+lsIbDlDf73ic8AF97nt9IqFmEYJuI+W93S49E5NQ5DlHPKCFVI7BAE0a6wLIdGIbLjx2tw3Yh8bC2qxifbT6PRwQsjf5GdRD8TyAlveXKzno41z/EjH/pdBox5sB1WRpwrcQYt1i+4FO04l7RDQ4k+giDaFatHvADq05wBYPrgXCQadSiubkRlA29E9ufZ8fcchCSy48+v01gNfHu/pxuyBhj/RDuujjhXtBqGUlhBQmKHIIh2RYjqaDWMX2NlnEGLG0d3kV0njeYIjQUNWo3MhEvIOb9rCrqkmTF9sGJ8DssCO98H/jEC2LGUv+7ih4Gs/u2+RoJoD+grEUEQ7UqD4NcxaAN+K50/qQ8+2XYKNY1OAJAZbDM8kR21ai7CS1aiCRseuVR+Zdk+flr5qc2eOw3gGwV2vbD9F0gQ7QT9pSAIol3xmpMD//kx6rRYft/FuPeD33BJn0zZbYM6JeO8LikY2T2tzdYZdThtvDdn8xsA5wb08cClC4FRdwPa6OuJQxBSSOwQBNGuNAQpdgB+DtHXcy/yud6k1+LLe32vJ/zQUAF8dBNweit/ecAVwJTFQHKn8K6LINoJEjsEQbQrjfbAlVhEK3P4J+DruYC1AjAlA1e9BfSdGu5VEUS7Qn9tCIJoV4RqrITmer8Q50bJTmDl00DRev5yZn/guveBjF7hXRdBhAESOwRBtCtCGsvc3PgCouX89j/gu/mA2wFodLwvZ/yTgL7jDjUliHOB/toQBNGuiHOxKI3V+tjqgO8XAHs+5i/3nQFMexFI6RL4cQQR5dBfG4Ig2hWr6NmhNFarUn0c+OBaoOoo3yBw3EK+d46G2qkRBIkdgiDalWBLz4kQOL4e+Ow2oLESSM4HZr0DdBkV7lURRMRAf20IgmhXBIOy3xEGRPC4XcD6PwMb/gKAA3KGADd9CiTmNPtQgogl6K8NQRDtSgOVnrcO1UXAF3cCp7fxl8+7GZj2EmAwh3ddBBGB0F8bgiDalUY7lZ6fM/u/BL7+A+CoB4xJwGWvAoOvCfeqCCJiIbFDEES7EkoHZUKFrf8Gvn+Y/z1/NHD1W0Bq1/CuiSAiHPprQxBEuyJ6dkjshIbLAax8CtjyBn955F38yActnUeCaA76X0IQRLsilp6TQTl4aouBT28BzuzgL1/yGDDuMSDA1HiCILzQXxuCINoVb+k5eXaC4uQm4MPrAVstYEoBrvoXzbYiiBAhsUMQRLtCHZRD4Ohq4OPfA85GIO984Nql5M8hiBZAf20Igmg3WJaD1UGl581irwdWLQK2/Zu/3GsiP8RTHxfedRFEB4X+2hAE0W40Od3i7+TZ8cPxdcBXcwHLaf7y+XOA6X8BdMawLosgOjL014YgiHZDSGFpGMCkp5lNPhxcDnw6B2BdQGo3YObrQI9x4V4VQXR4SOwQBNFuSHvsMFRJJOfYGq/QGTQLuPzvgCE+3KsiiKiAxA5BEO2GUHZO5mQFNSf5QZ6C0LnqLeqfQxCtCMWRCYJoN4SGgmYDlZ2LuF18D52mGiDvPOCKf5LQIYhWhsQOQRDtBpWdq7DlTaDkN8CUDPzuf4DeFO4VEUTUQWKHIIh2o9FTdh5HkR2eioPA2hf43yc/D6Tkh3c9BBGlkNghCKLdaBTmYlHZOXDoO+DtSXzDwK5jgPNuDveKCCJqob84BEG0GxTZ8XDoO+DjmwHOzQud371Hc64Iog0hsUMQRLshiJ2YNiif3Mgbkjk3MOR64Ip/AFp9uFdFEFENiR2CINqNJlHsxOifHksp8MkcwO0A+l0GXLGEKq8Ioh0gzw5BEO1GTJeeN9UAH98EWCuArIHA1dRLhyDaC/qfRhBEu9EUq2msyiP89PKzh/gS8+v+R92RCaIdiejIzjPPPAOGYWQ//fr1E2+32WyYO3cu0tPTkZCQgFmzZqG8vDyMKyYIIhBeg3KMfM9qrAZWPwe8cSEvdBLzgFtXAOk9w70ygogpIv4vzsCBA7Fq1Srxsk7nXfKDDz6I7777Dp9++imSk5Mxb948XH311fj111/DsVSCIJrBW3oeA5Gd3/4L/PAY4LTyl3tN5Ad7JncO77oIIgaJeLGj0+mQk5Pjc31dXR3eeecdLFu2DOPHjwcAvPvuu+jfvz82b96M0aNHt/dSCYJohpgoPXc2AT8+Dmz/D385dygwdgFvSKbycoIICxGdxgKAI0eOIC8vDz169MBNN92E4uJiAMCOHTvgdDoxceJE8b79+vVDly5dsGnTpoDPabfbYbFYZD8EQbQ9jdFejVW8GXhrnFfoXPoEcNd6oP9MEjoEEUYiWuyMGjUKS5cuxYoVK/DGG2+gqKgIF198Merr61FWVgaDwYCUlBTZY7Kzs1FWVhbweRcvXozk5GTxJz+fWrQTRHsQtWkspw348m7gP1N4b058FnDT58AlC0jkEEQEENFfr6ZNmyb+PmTIEIwaNQpdu3bFJ598gri4uBY/78KFCzF//nzxssViIcFDEO1AVKaxmmr5SqsTPwOMBjh/NjD+SSA+I9wrIwjCQ0SLHSUpKSno06cPjh49ikmTJsHhcKC2tlYW3SkvL1f1+EgxGo0wGo1tvFoi1mBZDhoNfYsPRNQ1FTy+DvhqLmA5DRgSgRuWAd3HhntVBEEoiOg0lpKGhgYcO3YMubm5GD58OPR6PVavXi3eXlhYiOLiYhQUFIRxlUQs0mB34eKX1uKBj3aGeykRTdQ0FXQ0At8vAP57BS90UrsDtywnoUMQEUpEf716+OGHMXPmTHTt2hUlJSV4+umnodVqccMNNyA5ORm333475s+fj7S0NCQlJeEPf/gDCgoKqBKLaHeOVjTgTG0TGg+7wr2UiIVlOdicLIAOLnZObQO+uhuoOspfvuAOYNKz1CSQICKYiBY7p0+fxg033ICqqipkZmZizJgx2Lx5MzIzMwEAr776KjQaDWbNmgW73Y4pU6bgn//8Z5hXTcQigvHW6knTEL40Ob3npkOmsVx2YP1LwC+vABzLNwi84h9ArwnhXhlBEM0Q0X9xPvroo4C3m0wmLFmyBEuWLGmnFRGEOo12fiN3uFi43Cx02g6VIW4XhBQWwwAmfQc7P4UrgBWPATVF/OUh1wHT/gzEpYZ3XQRBBEVEix2C6Cg0SqIWjU43kkjs+CCak/VaMB2lHJtlgbXPAz//lb+ckA1MewkYeGVYl0UQRGiQ2CGIVqDJ4fXqNNrdSDLpw7iayKTDzcUq3sJHc0p+4y+PugcY/wRgTAjvugiCCJkO8leHICIbq10S2XGQSVmNxo5SiVV7Clj1DLDvM/6yIQGY8Vdg6PVhXRZBEC2HxA5BtAJS820jmZRV8Y6KiFCx47IDv7wK/PIa4GoCwADn/Z5vEJiYHe7VEQRxDpDYIYhWQBrNsdopsqNGRIud0zuAr+cCZw/yl7teBEz5E5A3LKzLIgiidSCxQxCtgDSaIzUrE14isnuyvR5Y/2dg0xK+nDw+k6+yGng1zbQiiCgigv7qEETHpVHq2bGT2FFDKD2PiLlYrBvYtQxY/SxgreCvG/w7YOqLQHx6eNdGEESrQ2KHIFoBaTTHSgZlVYTITtgnnlccBL66ByjxjPZI6wlMXQz0mRLedREE0WaQ2CGIVkBaet5EBmVVwl567rACG//B98xx2wFjMnDJI8DIuwCdITxrIgiiXSCxQxCtgLT0nCI76oRtCCjLArs/BNY8B9SX8tf1msSPekjMad+1EAQRFkjsEASACosNB0otuKRPZou6+0rTWBTZUScsaayiDcCPfwTK9vCXU7oAE58hAzJBxBgkdqKYX49WIjvJhF5Z1PG1OR77Yi/WHKrAJ/9XgJHd00J+fJOs9JzEjhrtmsaqPAqsfAoo/I6/bEwCxj4MjPw/QG9q+9cnCCKiILETwXAch58OlKNvdiK6ZcSH9NjTNY246e0t6JZuxroFl7bRCqOHE1VWAEBxdWOLxA51UG6edumgbK0CNrwEbHsbYF0AowVG3AaMewyIz2i71yUIIqIhsRPB7D1Th//73w4My0/BV3MvCumxxdWNAIATVY1odLgiq7dJBFJjdQAAahsdLXo8dVBuHm9kpw3EjqMR2PIG3/3YbuGv6z0FmPwckNm39V+PIIgOBe2AEcyJKl6wHC6vB8dxIXlJqq3eTftkVSP65ya1+vqiBTfLobbJCQCo8/wbKtJoDkV2vBw/24DD5fWYOihXFDvxrS28D/8ELH8AsJzhL+cMBiY9C/Qc37qvQxBEh4XETgRTWW8HwH8jrrY6kJ5gDPqxcrFjJbETAEuTExzH/17bGLrYcbMcbE5WvEyeHS/zP9mNXadq8dFdo1HnObfxxlaK7DTVACsW8pVWAJDcBZjwJDDoGkCjaZ3XIAgiKiCxE0EcrajHjpM1+N2IfDAMg7MNdvG2UzVNLRY7QoSIUKdakrqqbUFkp0kxHiJWxkXsO1OHY2cbcMWwTqq3O90s9pfUAQA2HavCsbMNAICemedomLfXA9veATb9A7CeBcAABXOBS/8IGMzn9twEQUQlJHYiiAc/3o29Z+qQkWDEhP7ZYmQHAE5VN2JYfkrQz6WM7BD+qZGcq5Z4dpRpq8YYGQR634c7cbzSii5pZpzXJdXn9hOVVjjdfMjsq11n4GI5JMfp0Tk1rmUvyLLA3k/4KquGcv669N7Alf8E8ke29DAIgogBSOxECNVWB/ae4b8F7yyu5cWOLLITWnSmShrZqaTITiCkwrAlnh1lX51YMCjX25w4XsmL6J3FtTKxs+ZQOc7U2pASpxevO+mJLg7MS2pRHyOc3gGseAw4vZW/nNqd7348+FpAqw/8WIIgYh4SOxHC5uNV4u9C6L+ywbsJn6puAsCnDn46UI57x/WESe/f+1DdQJGdYJH6dFri2VF6dGLBoHykokH8fX8JX/3kdLO4/6Od+H5vGQCo+sQGdUoO7YXK9gFrXwAKv+cv6+OBSxYAo+8FdMGndQmCiG1I7EQIG49Vir8Lm8dZSRrrtCey8/x3B7D5eDV6Zsb79UoA8mhFSZ0NNqc7oDiKZWSenRaksZqcvLjRaxk43RysMRDZOVxWL/4uiPMV+8pEoQMAB0stPo8bmBekUb7yCLD2T8D+L/jLjAYYegMw/gkgKa/lCycIIiahkoUIYeMxb2Snot6OCosNVVa5Z4fjOBws5TeZosrA0ZpqxaZ9qppSWf6QenYsNhfcLBfS44W0VXo8H2lwuFi43Gygh7Q6VQ12vPDdAZTUNrXL6xWWe8XOkYoG2JxuHPVEe7orGmCmmL1ppmbFjqUU+HousGSkV+gMvAq4dwvvzSGhQxBECyCxEyYsNide+O4ANh6tRFmdDcfPWqFhgNxkvpX9xmNVorkTAM7UNqG0ziZ6SoS0lhocx4kbeEYCP82ZKrL8I42CAXwpeiiIYifBOzm7vSuylqw9hn//XIR/rD3aLq93WCJ23CyHwrJ6MV066/xOMoFz2ZBcAECcXovuGQEqsSoOAf++FNj5PsCxQJ+pwP/9DFy7FMjs0ybHQRBEbEBiJwxwHIeb396Cf/9chD9+tQ+/FdcA4D0OwqiCdYUVAIAEow46DZ8e+fnIWfE5lIbl7Seq8fiXe1FRb4OlyQWXJzpxvsc4elTisQiWE5VW8TmjmRpFFMxf+fn2E9V46ut9KoZkPo2VYtZDp+HNt43t3Gtnh+czdEglddQWFJbxn6f0eF7g7S+xiIK6R2YCxvfNAgCkmvW4bAgfjbmgexq0Gj/m5IpDwLtT+ankmf2A21cBN34M5A5p4yMhCCIWILETBt75pQi7T/M+h6JKq+h5GJiXJIb51x/mhU1WkhF5KXyp7qqDFeJznFakpZ75dj+WbSnGve//hnKPOIk3aDGsSwoA4EALNsG//FiIZVuK8fbPRSE/tiNRozAlq/l2WJbDQ5/uxn83ncTnv52W3SZEdswGnTgKwdqMSZnjOLy+6gg+3lZ8LksHANicbhzwfIYOlzeA40JLw4VKVYNdrBQUojb7SurEyE7XdDNmDuMFzvCuaRjdIx0f3jkaL1/rR7g0nAWW/Y5vEthpOHDrD0D+BW16DARBxBYkdtqJz3acxiOf7YbN6fYRDz/t53uG9MlOxJDOKQC8G3BmghE9M3kPxPpCb2Sn1GKDw8X7Qkpqm7DvDC9mtp+swcIv9gIA0hIMGJjHV7/s95S1C+w9XYcHPtopM0FLYVkOmzwVYntP16neJ9L4etcZPP31Pjib8cvYnG48/uVerDnEn/caa/ORna0nqsXyaUGcCjSJYkcrjkJQRn+U/FZcg1dXHcYfv9wHWzMpryPl9bjvw50o9pOK3F9SJ6Y8G+wulNS1biTuvY0n8Mcv98Lu4td5uJyP6nRJM2Nk93QAwM9Hzoqf2a7p8bi0bxY+vbsAf541GABQ0DMdWYmKaeMcB+z/Cvj3eKD2JF9OfuOngDn0QawEQRCBILHTDmw4fBYLPtuNT7afxj/WHEWZxQaDToN+OYkAvGW8fbITMbxrKhJN3iK5jESjmAZwSDZxjoNoRl11kN+0BX/OjpN8SiMt3ihGioqqrGiQNLt74fsD+GpXCf63+aTqmgvL60Uvy/6SupCjBftL6nDHe9ux61RtSI9rKfU2Jx77fC/e23RSJgrV+GZXCZZtKcaz3x4A4DVzZyXyBuM6lfLzT7adEn8XquUEhNJzs0EHs2cUgrWZxoI/HeDfMxfL4ZCkskmN5747iG92l+BfG47Jrj9YasEd723HfzfJ38PDzTwfABRXNeLO/27HL0cqZdeX1jXhrv9uF9Oo3+0pxdPf7McHW4rx4RY+CrX9RDUAoH9uoph2FTxkGQlGJBj5z+8F3dL8d/0u2QW8NxP4dA5QVwwk5wM3fQrEpze7doIgiFAhsdPGlFtsuP+jneLspbd+Pg4AGN4lFUM6y3uO9M1JhF6rwaUevwPAR3amD84VNxCAN3oCXt/OSs/GeefFPWTdadPjDchIMCInyQSO85YC11gd2FrEb1jKiI/AJkl1mMXmwuma4Kt8qq0O3Pnedqw6WI7HPt8DVlLdxG/Q2/DxtmKZgDpZZcUd722TleCHwvI9peLYBsEDVVZnwx8+3IlvdpfI7iu8xomqRpytt4umb6GKSJnGstic+H5fqXj5UGm9LHrU6Ck9Nxu0MHvSWM0ZlIX3DPCNFEk5U9skerV2FtfyV9rrwTWcxZ8++wU7Dh7B+l2FSEYDkjw/q3Yextz/rMXuo8X8NHDWN9K16Nv9WHmgHHe/vwMfbS3GHe9tx8ZjlViy9ih+OlCOe97/DR9uLcYjn+0WH7Nk3TE0OdxY6RHXl/bNQmaiEX2yvabjbukq4xpcDqC+HCg/AOz+GHh/FvDWJcCJnwGdCbjkUWDuViCjd8BzRhAE0VKoz04b89qqI6hpdCIv2YSSOm/q6cKe6aK/AwCS4/RiZGHSgGxxg85IMCDOoMXMoXn4cGsxDDoNLuiehg2Hz+JUdRPqmpxiQ8JJA7JRZrHh3V9PAADSPObRgXlJKLPYsP9MHS7oloY1hyog6A9llEJAWgrP368O+Wn8RrbjZA2eW34ANqcbF3RLw7NXDBS74nIchwc/3iWmUg6V1eP7faW4bEgeaqwO3PHedpypbcKqgxVYvqcU/549Aia9Fou+PYA1hypwuqYJP9x/sfh8RysasOjb/Zhd0A2TBmT7Pc8fSyIvO4tr4XCxuPv9Hdh1qhbf7y1FdqIRo3qkg+M42bGtK6wQhWi39HhsKar2SWN9s6sENieL3lkJKLPYUG9z4WhFg9g0T5rGMnvSWIEMysfONuD4WW/rACEFqcZn20+L6ztUZoFt56cwfXMXGI7F/wDApPKgQs+/73uvcmsM0OhNYPRxsMOARy0s7jcYYOMMsH+rx3UwoPGkCRewevTX6cBxAPsN8EdwyE4zwmp3oaHJjX1v/Q/XlFlxjR64vLQr8IMOi4w12Kerhx4uDGligf/9BWisBpqqgcYawKESaWI0wKBZwPgngdSufo+fIAiiNSCx04YUVzXi0+38Jvza9efh/o92otQjAi7slS4bK9A3O1Hc4Mf1zRSvt3vE0e9Hd8HnO05jdI90dEs3YwP4yM7/Np2A082hT3YCemQmYNKAbFHsJHva9Q/slIzVhyrwzLcHsHxPqazpXZnFhsoGOzI86Yb1h8/irz8VilGg/rlJOFhqwZvrj+OdX4rwxIwB+OtPhWJ66lBZPa46v5NY9bWu8CzWHz4Lo06D6YNz8eXOM3hl5WFM7J+NBz/ZhTO1TchKNMJic+LnI5V45pv9uHZEPtYcqhCf79Mdp/H1rjO4sGcGPt9xGscrrThYWo8xvTJkAlFg5YFyWbps9+la/On7g+J1bpbD3GU78a+bz0dynAEVEp/Sao/pO8mkE0vHlV2UP/G8h9ddkI+VB8qxpagaL/5wCBoGePKyAbI0VrxBnsY6XF6Pv/xYiB6Z8Xhsaj8wDIMf9vJRIpNeA5uTFc3FSoqrGvGRxMDMcoBl3wqYuNB7+GhZB2B3AHYLjAD6qMV0hadV/lVolFxfCVwg3L6T/6cAQIFwncXz4wMDxKUASZ2BPlOAYTcC6T1DPg6CIIiWQGKnDXl99RG4WA5j+2RiZPc0TOyfjf9tPgmzQYshnVNkxtg+Od5UQKJJj4n9s7H6UDmmD+arXQbmJWPNw5cgxWwQvRP7SyzY6UnZzL20FwBgZDevubPMYvM81tvIbbvHzwPw6bAmpxv7Syy4pE8mjlbU4973d4hiqFNKHH43ojMWfXtAFA4PfLwLRZVWMAwwqnsaNh+vxrItxfhgczGsdhdOeCpy5lzYDX8Y3wvrD5/F8bNWTH/9ZxyvtMKo02DprSNRZbVj9n+24qNtp/DdHn7z12kYuFgOj3y2BwDw61FvBKaywY4Fn+3GiSorLE1ePwzLcWKKbebQPKw9VIEGuwtLN54AAPz9hvOwZO1RHCqrxzVvbsJgz7gCDcOLhw2eFFFqvEHsDSOdj3Ww1II9p+ug1zK46rxOKK2zYUtRtVgtd6pmhygqzQatWDn304FyaDQMFn6xRzQPZyeacGGvdLEXzm0Xdcc/1x3D7tN1+L//bYdJr8V1F+Tj7Z+LcLSiAeUWG+wuFpmJRgzxCNazp48hC8B8x91Ya7wUqx8aB7ebBcexaLS7MPHV9WAA6DW8H0gHNxK1LiRoneCcTTDCCRMcSNK58PJVfWFmHLDbrDh8uhI/7DoBExwY1zMR5+WnwO7ikGw2AGDAgcO3u0pwrMIChuEwpmc6RnRJBjgWNqcLS38tggsajBvWD4N6dQPi0nijsfCvKRnQUAdvgiDCA4mdNqLe5sRaj8nzoUl8Q7Srz++ED7cWY9qgXOi1GmQmGpFi1qO20Ym+2Ymyx//jxvNQ0+hAbrLXg9M5lU8j5afx123wbLh9shNEE7NOq8HvR3fB+5uLMXs0nx4Y3jUVJr0GRp0WUwZm44vfzqBfbiK6ZyTg290lePnHQiz6dj/K62ywOtwY3SMND07sgz7ZiTheKe/PI3RuHtMrA3Mv7YXr39qMz3bIS7HjDVr839geSDTp8crvhuLWpdvEoZHPXzkIAzzia8GUvnhpRSHq7S6YDVo8ddkAPOapJMtNNqHJ6YbN6cb1F3TB0o0nsHxPKdRgGF44PDS5D+767w78cpT35FzaNxMzh+bh4t4ZeP67g/hsx2ns8VSWTR+ci+V7SsXoWu+sRKTE8ZGdL3eewdaiarAcJ0ZoJvbPRnqC0acDsLR/UbeMeIzpnYEPtxZj1cFyrCusgIvl0C8nEYfK6vH8dwdg0mthc7IY2ycT8yf1wf82nUS93YUfPRV5X++S+4tG90jDn2cNwcoD5Vh9qAJxTaWABkjI7o7PbhyDtARvHsvNcuiUnoQGmwsPTuqDJ77aBwf0uH50d9w2phse/3IfNhw+i4F5SVgwawgyPcIvHsD5LhZPnvoZpXU2zLl6LEwpcbIMGQNg3Cgn/vbPjThV3YirL78E8PhzTAC2lvN+qysnXAKkqvh2CIIgwgiJnTYi0aTH2ofHYe2hCgzNTwEAnNclFRsfG49kTwSBYRiM75eF5XtKcWGvDNnjTXqtTOhIGdI5BQatRqzOemRKP1mztkWXD8IfxvdGdhK/XWUkGLHhkUthNuiQYNThjzMGQKth8MHmk/h2d4k4bR0AOqfG4e83nI9Mj39ogD4ZmYlGaBkGl/TJxMeelM7vRuRjVPc0dEs3i83kBndKxt4zdZg3vrdYhTOubxYemtQHL/90GLMLuuLaEfnia907rhemD8pFdaMDnVLikJlgxNKNJ3CquhH/nj0C3TPi0ehwI9Wsx2/FNThQYsHdl/TE+P5eAzcAZCeZ0MkTUTmvS4ooduZP6gsASDEb8PK1Q3H50Dw8/uVelNQ24d5xvbCu8Cwa7C4kx+nx9MwBsl5EZxRjF2YXdAMAjO6RDqNOgx6ZCVg4rR/u+t92JJr0eO6KgbikD59+vOq8zvj8t9NwsRymDszBG78/Hw9/ugef/3YajQ43uqWb8dp1w6DTapBs1qPeI6gG5iVhf4kF4/pm4p5LeiIpTo9+OXx6k58qzqETw0e7nrhxCgyZ8m7EWg2D7+67GC43C7NBh3d+KYKlyYl7xvVEZqIR7916AU5UNSI/NQ46rTyPZdBp8OXci2BzusWUppIkkx7fzLsIDTYXspLkZqElN56PJqdb9IkRBEFEEgzX1h3IOgAWiwXJycmoq6tDUlKQgwpbCYeLRZPDLQqgYDlbb0e5xYYkkx5d1CpgguCXI5X4/TtbAADj+2Vh/qQ+6J2dAKNOnm4QStbdbg5TXtsAg06Dnx4cC5Nei/9tOoEnv96PeZf2wkOT++Bsg923nwr4RnR+y5AlNDpcsDtZpCo2TZvTDYebRZIp8Hnac7oWVy75FVee1wmv/G6Yz+12lxt1jU5kJZnw4Me78N2eUrx58/kY3y8bxVWNmPDKOuQkm/DEjAGigEox68WoGsBXs5mNWhh1WlRbHTAbtLIhq6eqGzH99Z+Rk2zCF/deiETPmo9W1MPmZNEnOxEGHS82/rX+GBb/cAhPzOiP2y7qjpK6JnRKiRP9W9J1X/2Xr/Gd/VZwYMA8UQHoAgsLq53vpC2k2QiCIKKNYPdvEjsIr9gJJw12F8b9ZS1SzAZ8fveFQQkuq90FDcOIRmGO41DZ4BAjQZFAbaMDiSa9/9EEHtwsh3qbEylmr2iosTqQaNL5RD5Cpa7JCaNO0+ykeY7jUNPoDCoiYju5A6Z3xwMJOcDDhc3enyAIItoJdv+mNFYMk2DU4ZdHx4Nh4BPN8Ue8Uf6RYRgmooQOAJl4CYRWw/jcVxlRainBRlMYhgk69WNq9Ph5kju3dFkEQRAxCYmdGKe5yAMRQdR5jOAkdgiCIEKCOigTREeBxA5BEESLiBqxs2TJEnTr1g0mkwmjRo3C1q1bw70kgmhd6jxdoknsEARBhERUiJ2PP/4Y8+fPx9NPP43ffvsNQ4cOxZQpU1BRURHupRFE61F3hv+XxA5BEERIRIVn55VXXsGdd96JW2+9FQDw5ptv4rvvvsN//vMfPPbYY+FbmKUEYANPvyaIoKn1jI5I6hTedRAEQXQwOrzYcTgc2LFjBxYuXChep9FoMHHiRGzatEn1MXa7HXa7dz6SxeJ/EOM58d7lQNWRtnluInZJzm/+PgRBEIRIhxc7lZWVcLvdyM6WT8TOzs7GoUOHVB+zePFiLFq0qO0XpzMCOrWx1ATRQnqOB+Izmr8fQRAEIdLhxU5LWLhwIebPny9etlgsyM9vg2/L9/za+s9JEARBEERIdHixk5GRAa1Wi/Lyctn15eXlyMnJUX2M0WiE0RhZjfAIgiAIgmgbOnw1lsFgwPDhw7F69WrxOpZlsXr1ahQUFIRxZQRBEARBRAIdPrIDAPPnz8ecOXMwYsQIjBw5Eq+99hqsVqtYnUUQBEEQROwSFWLnuuuuw9mzZ/HUU0+hrKwMw4YNw4oVK3xMywRBEARBxB409RyxO/WcIAiCIDoywe7fHd6zQxAEQRAEEQgSOwRBEARBRDUkdgiCIAiCiGpI7BAEQRAEEdWQ2CEIgiAIIqohsUMQBEEQRFRDYocgCIIgiKiGxA5BEARBEFENiR2CIAiCIKKaqBgXca4ITaQtFkuYV0IQBEEQRLAI+3ZzwyBI7ACor68HAOTn54d5JQRBEARBhEp9fT2Sk5P93k6zsQCwLIuSkhIkJiaCYZhWe16LxYL8/HycOnWKZm4FAZ2v4KFzFTx0rkKDzlfw0LkKjbY4XxzHob6+Hnl5edBo/DtzKLIDQKPRoHPnzm32/ElJSfQfIQTofAUPnavgoXMVGnS+gofOVWi09vkKFNERIIMyQRAEQRBRDYkdgiAIgiCiGhI7bYjRaMTTTz8No9EY7qV0COh8BQ+dq+ChcxUadL6Ch85VaITzfJFBmSAIgiCIqIYiOwRBEARBRDUkdgiCIAiCiGpI7BAEQRAEEdWQ2CEIgiAIIqohsdOGLFmyBN26dYPJZMKoUaOwdevWcC8p7DzzzDNgGEb2069fP/F2m82GuXPnIj09HQkJCZg1axbKy8vDuOL2Y8OGDZg5cyby8vLAMAy++uor2e0cx+Gpp55Cbm4u4uLiMHHiRBw5ckR2n+rqatx0001ISkpCSkoKbr/9djQ0NLTjUbQfzZ2vW265xeezNnXqVNl9YuV8LV68GBdccAESExORlZWFK6+8EoWFhbL7BPN/r7i4GDNmzIDZbEZWVhYWLFgAl8vVnofS5gRzrsaNG+fz2br77rtl94mFcwUAb7zxBoYMGSI2CiwoKMAPP/wg3h4pnysSO23Exx9/jPnz5+Ppp5/Gb7/9hqFDh2LKlCmoqKgI99LCzsCBA1FaWir+/PLLL+JtDz74IL799lt8+umnWL9+PUpKSnD11VeHcbXth9VqxdChQ7FkyRLV21966SX87W9/w5tvvoktW7YgPj4eU6ZMgc1mE+9z0003Yf/+/Vi5ciWWL1+ODRs24K677mqvQ2hXmjtfADB16lTZZ+3DDz+U3R4r52v9+vWYO3cuNm/ejJUrV8LpdGLy5MmwWq3ifZr7v+d2uzFjxgw4HA5s3LgR7733HpYuXYqnnnoqHIfUZgRzrgDgzjvvlH22XnrpJfG2WDlXANC5c2e8+OKL2LFjB7Zv347x48fjiiuuwP79+wFE0OeKI9qEkSNHcnPnzhUvu91uLi8vj1u8eHEYVxV+nn76aW7o0KGqt9XW1nJ6vZ779NNPxesOHjzIAeA2bdrUTiuMDABwX375pXiZZVkuJyeH+8tf/iJeV1tbyxmNRu7DDz/kOI7jDhw4wAHgtm3bJt7nhx9+4BiG4c6cOdNuaw8HyvPFcRw3Z84c7oorrvD7mFg+XxUVFRwAbv369RzHBfd/7/vvv+c0Gg1XVlYm3ueNN97gkpKSOLvd3r4H0I4ozxXHcdwll1zC3X///X4fE6vnSiA1NZV7++23I+pzRZGdNsDhcGDHjh2YOHGieJ1Go8HEiROxadOmMK4sMjhy5Ajy8vLQo0cP3HTTTSguLgYA7NixA06nU3be+vXrhy5dusT8eSsqKkJZWZns3CQnJ2PUqFHiudm0aRNSUlIwYsQI8T4TJ06ERqPBli1b2n3NkcC6deuQlZWFvn374p577kFVVZV4Wyyfr7q6OgBAWloagOD+723atAmDBw9Gdna2eJ8pU6bAYrGI3+KjEeW5Evjggw+QkZGBQYMGYeHChWhsbBRvi9Vz5Xa78dFHH8FqtaKgoCCiPlc0CLQNqKyshNvtlr15AJCdnY1Dhw6FaVWRwahRo7B06VL07dsXpaWlWLRoES6++GLs27cPZWVlMBgMSElJkT0mOzsbZWVl4VlwhCAcv9pnSritrKwMWVlZstt1Oh3S0tJi8vxNnToVV199Nbp3745jx47h8ccfx7Rp07Bp0yZotdqYPV8sy+KBBx7ARRddhEGDBgFAUP/3ysrKVD9/wm3RiNq5AoAbb7wRXbt2RV5eHvbs2YNHH30UhYWF+OKLLwDE3rnau3cvCgoKYLPZkJCQgC+//BIDBgzArl27IuZzRWKHaFemTZsm/j5kyBCMGjUKXbt2xSeffIK4uLgwroyINq6//nrx98GDB2PIkCHo2bMn1q1bhwkTJoRxZeFl7ty52Ldvn8wrR6jj71xJfV2DBw9Gbm4uJkyYgGPHjqFnz57tvcyw07dvX+zatQt1dXX47LPPMGfOHKxfvz7cy5JBaaw2ICMjA1qt1sdxXl5ejpycnDCtKjJJSUlBnz59cPToUeTk5MDhcKC2tlZ2HzpvEI8/0GcqJyfHxwDvcrlQXV0d8+cPAHr06IGMjAwcPXoUQGyer3nz5mH58uVYu3YtOnfuLF4fzP+9nJwc1c+fcFu04e9cqTFq1CgAkH22YulcGQwG9OrVC8OHD8fixYsxdOhQvP766xH1uSKx0wYYDAYMHz4cq1evFq9jWRarV69GQUFBGFcWeTQ0NODYsWPIzc3F8OHDodfrZeetsLAQxcXFMX/eunfvjpycHNm5sVgs2LJli3huCgoKUFtbix07doj3WbNmDViWFf8YxzKnT59GVVUVcnNzAcTW+eI4DvPmzcOXX36JNWvWoHv37rLbg/m/V1BQgL1798oE4sqVK5GUlIQBAwa0z4G0A82dKzV27doFALLPViycK3+wLAu73R5Zn6tWszoTMj766CPOaDRyS5cu5Q4cOMDdddddXEpKisxxHos89NBD3Lp167iioiLu119/5SZOnMhlZGRwFRUVHMdx3N1338116dKFW7NmDbd9+3auoKCAKygoCPOq24f6+npu586d3M6dOzkA3CuvvMLt3LmTO3nyJMdxHPfiiy9yKSkp3Ndff83t2bOHu+KKK7ju3btzTU1N4nNMnTqVO++887gtW7Zwv/zyC9e7d2/uhhtuCNchtSmBzld9fT338MMPc5s2beKKioq4VatWceeffz7Xu3dvzmazic8RK+frnnvu4ZKTk7l169ZxpaWl4k9jY6N4n+b+77lcLm7QoEHc5MmTuV27dnErVqzgMjMzuYULF4bjkNqM5s7V0aNHuWeffZbbvn07V1RUxH399ddcjx49uLFjx4rPESvniuM47rHHHuPWr1/PFRUVcXv27OEee+wxjmEY7qeffuI4LnI+VyR22pC///3vXJcuXTiDwcCNHDmS27x5c7iXFHauu+46Ljc3lzMYDFynTp246667jjt69Kh4e1NTE3fvvfdyqampnNls5q666iqutLQ0jCtuP9auXcsB8PmZM2cOx3F8+fmTTz7JZWdnc0ajkZswYQJXWFgoe46qqiruhhtu4BISErikpCTu1ltv5err68NwNG1PoPPV2NjITZ48mcvMzOT0ej3XtWtX7s477/T5shEr50vtPAHg3n33XfE+wfzfO3HiBDdt2jQuLi6Oy8jI4B566CHO6XS289G0Lc2dq+LiYm7s2LFcWloaZzQauV69/r+9OwltagvAOP4lLY1JUBtr0KBFlJbaKhS1Uqp1oUVtF2IlGyVKagWxDpQ6gYITIhHR4i5ScEAUCxW6sQ7oRrDigEoRjYJodRNBUAOtWIect3i8C9cB9JnaeP3/4EJypntzyOLj3pOcIrNlyxaTSqVs4/wNc2WMMY2NjWbChAkmLy/PBINBU1NTYwUdY7Lne+UyxpjM3ScCAADILqzZAQAAjkbYAQAAjkbYAQAAjkbYAQAAjkbYAQAAjkbYAQAAjkbYAQAAjkbYAfDH6u3tlcvlsv6ufzA0NDSovr5+0MYHMPgIOwCGTENDg1wu11dHbW3tD/UvLCxUMpnU1KlTB/lKAfzJcof6AgD83Wpra3X8+HFbmcfj+aG+OTk5jtxFGkBmcWcHwJDyeDwaO3as7QgEApIkl8uleDyuuro6eb1eTZo0SWfPnrX6fvkY682bN4pEIgoGg/J6vSouLrYFqfv372vevHnyer0qKCjQ6tWr1dfXZ9V//vxZGzduVH5+vgoKCrR161Z9uaNOOp1WLBbTxIkT5fV6VV5ebrsmANmHsAMgq+3YsUPhcFg9PT2KRCJaunSpEonEd9s+fPhQFy5cUCKRUDwe1+jRoyVJ/f39WrhwoQKBgG7fvq2Ojg5duXJF69evt/ofOnRIJ06c0LFjx3Tt2jW9fv1anZ2dtnPEYjGdPHlSR44c0YMHD9TS0qLly5fr6tWrgzcJAH5NRrcVBYCfEI1GTU5OjvH7/bZj3759xph/d6Bes2aNrU9lZaVpamoyxhjz7NkzI8ncu3fPGGPMokWLzMqVK795rra2NhMIBExfX59V1tXVZdxut7UbeigUMgcOHLDqP378aMaPH28WL15sjDHm/fv3xufzmevXr9vGXrVqlVm2bNn/nwgAg4o1OwCG1Ny5cxWPx21lo0aNsl5XVVXZ6qqqqr7766umpiaFw2HdvXtXCxYsUH19vWbNmiVJSiQSKi8vl9/vt9rPnj1b6XRajx8/1rBhw5RMJlVZWWnV5+bmqqKiwnqU9eTJE717907z58+3nffDhw+aNm3az394AL8FYQfAkPL7/SoqKsrIWHV1dXr+/LnOnz+vy5cvq6amRuvWrdPBgwczMv5/63u6uro0btw4W92PLqoG8PuxZgdAVrtx48ZX70tLS7/bPhgMKhqN6tSpUzp8+LDa2tokSaWlperp6VF/f7/Vtru7W263WyUlJRo5cqRCoZBu3rxp1X/69El37tyx3peVlcnj8ejFixcqKiqyHYWFhZn6yAAyjDs7AIbUwMCAXr58aSvLzc21FhZ3dHSooqJC1dXVOn36tG7duqWjR49+c6ydO3dqxowZmjJligYGBnTu3DkrGEUiEe3atUvRaFS7d+/Wq1evtGHDBq1YsUJjxoyRJDU3N2v//v0qLi7W5MmT1draqrdv31rjDx8+XJs3b1ZLS4vS6bSqq6uVSqXU3d2tESNGKBqNDsIMAfhVhB0AQ+rixYsKhUK2spKSEj169EiStGfPHrW3t2vt2rUKhUI6c+aMysrKvjlWXl6etm3bpt7eXnm9Xs2ZM0ft7e2SJJ/Pp0uXLqm5uVkzZ86Uz+dTOBxWa2ur1X/Tpk1KJpOKRqNyu91qbGzUkiVLlEqlrDZ79+5VMBhULBbT06dPlZ+fr+nTp2v79u2ZnhoAGeIy5os/kQCALOFyudTZ2cl2DQB+CWt2AACAoxF2AACAo7FmB0DW4ik7gEzgzg4AAHA0wg4AAHA0wg4AAHA0wg4AAHA0wg4AAHA0wg4AAHA0wg4AAHA0wg4AAHA0wg4AAHC0fwDXUcmXduTcQgAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "<Figure size 640x480 with 0 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "<Figure size 640x480 with 0 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import pygame\n",
    "\n",
    "if torch.backends.mps.is_available():\n",
    "    num_episodes = 300\n",
    "else:\n",
    "    num_episodes = 50\n",
    "\n",
    "for i_episode in range(num_episodes):\n",
    "    # Initialize the environment and get it's state\n",
    "    state, info = env.reset()\n",
    "    state = torch.tensor(state, dtype=torch.float32, device=device).unsqueeze(0)\n",
    "    for t in count():\n",
    "        action = select_action(state)\n",
    "        observation, reward, terminated, truncated, _ = env.step(action.item())\n",
    "        reward = torch.tensor([reward], device=device)\n",
    "        done = terminated or truncated\n",
    "\n",
    "        if terminated:\n",
    "            next_state = None\n",
    "        else:\n",
    "            next_state = torch.tensor(observation, dtype=torch.float32, device=device).unsqueeze(0)\n",
    "\n",
    "        # Store the transition in memory\n",
    "        memory.push(state, action, next_state, reward)\n",
    "\n",
    "        # Move to the next state\n",
    "        state = next_state\n",
    "\n",
    "        # Perform one step of the optimization (on the policy network)\n",
    "        optimize_model()\n",
    "\n",
    "        # Soft update of the target network's weights\n",
    "        # θ′ ← τ θ + (1 −τ )θ′\n",
    "        target_net_state_dict = target_net.state_dict()\n",
    "        policy_net_state_dict = policy_net.state_dict()\n",
    "        for key in policy_net_state_dict:\n",
    "            target_net_state_dict[key] = policy_net_state_dict[key]*TAU + target_net_state_dict[key]*(1-TAU)\n",
    "        target_net.load_state_dict(target_net_state_dict)\n",
    "\n",
    "        if done:\n",
    "            episode_durations.append(t + 1)\n",
    "            plot_durations()\n",
    "            break\n",
    "\n",
    "print('Complete')\n",
    "plot_durations(show_result=True)\n",
    "plt.ioff()\n",
    "plt.show()\n",
    "pygame.quit()\n",
    "env.close()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "OrderedDict([('layer1.weight', tensor([[-0.3812,  0.2172,  0.1109, -0.1320],\n",
      "        [-0.1757, -0.4742, -0.7790, -0.3261],\n",
      "        [-0.4119,  0.3955,  0.2607,  0.0122],\n",
      "        [-0.2776, -0.1747, -0.2883, -0.0631],\n",
      "        [ 0.4193,  0.0566, -0.4124, -0.2882],\n",
      "        [-0.0877, -0.6764, -0.4497, -0.4707],\n",
      "        [ 0.3379,  0.2047,  0.9934,  0.4148],\n",
      "        [ 0.2671, -0.2802, -0.7898, -0.2110],\n",
      "        [ 0.2349, -0.0963, -0.0878,  0.1304],\n",
      "        [ 0.3873,  0.2341, -0.1535, -0.5645],\n",
      "        [ 0.1949,  0.0830,  0.1089, -0.4408],\n",
      "        [-0.2074,  0.4335, -0.3015,  0.0619],\n",
      "        [-0.4199,  0.1402, -0.2970,  0.1373],\n",
      "        [ 0.0240, -0.4478,  0.2475, -0.5012],\n",
      "        [-0.5461, -0.5980, -0.4752, -0.4182],\n",
      "        [-0.1869, -0.2296, -0.6468,  0.2018],\n",
      "        [ 0.1301, -0.2606,  0.8492,  0.1822],\n",
      "        [ 0.8387,  0.3842,  0.3182, -0.1497],\n",
      "        [ 0.3628, -0.0100, -0.4044,  0.1574],\n",
      "        [-0.0481,  0.0224, -0.3777,  0.1191],\n",
      "        [ 0.2771, -0.2048,  0.5160, -0.3782],\n",
      "        [ 0.1638, -0.3946, -0.1040, -0.0872],\n",
      "        [-0.0962, -0.0559, -0.5959, -0.2289],\n",
      "        [ 0.4851,  0.3095, -0.3291, -0.3393],\n",
      "        [-0.4969, -0.3471,  0.2168,  0.3952],\n",
      "        [-0.8608, -0.4731, -0.3093,  0.3519],\n",
      "        [ 0.4260,  0.0129, -0.3252,  0.2565],\n",
      "        [ 0.1864,  0.4417,  0.3927, -0.4974],\n",
      "        [-0.3839, -0.2110,  0.1646, -0.0611],\n",
      "        [ 0.2417, -0.6455, -0.5358, -0.4364],\n",
      "        [-0.9304, -0.1229, -0.2864,  0.1956],\n",
      "        [-0.3210, -0.4154, -0.8861, -0.6278],\n",
      "        [-0.1158, -0.7221, -0.3463, -0.5970],\n",
      "        [-0.6798, -0.4725,  0.4314,  0.0101],\n",
      "        [ 0.6223,  0.4958,  0.3537, -0.0926],\n",
      "        [ 0.7197,  0.3441, -0.4544, -0.1186],\n",
      "        [ 0.0277, -0.0646,  0.1950,  0.2672],\n",
      "        [-0.2488,  0.3814,  0.3348, -0.1255],\n",
      "        [ 0.0964,  0.2880,  0.1883, -0.1140],\n",
      "        [-0.7320, -0.5192, -0.2928, -0.0426],\n",
      "        [-0.3251,  0.4124,  0.3012, -0.3949],\n",
      "        [-0.7373, -0.3577,  0.4335, -0.0740],\n",
      "        [ 0.3229,  0.4811, -0.2979,  0.3689],\n",
      "        [-0.4620, -0.4660, -0.3976, -0.4407],\n",
      "        [ 0.0368,  0.2750, -0.2006, -0.2617],\n",
      "        [-0.2886,  0.0362, -0.4524, -0.1891],\n",
      "        [ 0.1496,  0.0680, -0.0964,  0.1830],\n",
      "        [ 0.4933,  0.0012, -0.9161,  0.0697],\n",
      "        [-0.4329, -0.0945, -0.2861, -0.0330],\n",
      "        [-0.3726, -0.1771,  0.3980, -0.0676],\n",
      "        [ 0.2547, -0.4269, -0.5758,  0.4507],\n",
      "        [ 0.2472,  0.2494, -0.1007, -0.4194],\n",
      "        [ 0.6802,  0.5967, -0.1218, -0.1400],\n",
      "        [-0.1838,  0.4175, -0.1950,  0.0931],\n",
      "        [-0.1466, -0.1750, -0.0972, -0.1675],\n",
      "        [-0.2258, -0.1519,  0.2794,  0.3223],\n",
      "        [ 0.3054,  0.5834,  0.5638,  0.6402],\n",
      "        [ 0.4147,  0.3234,  0.4550,  0.4229],\n",
      "        [-0.0813,  0.0431,  0.2336, -0.0465],\n",
      "        [-0.1615, -0.1893, -0.0312,  0.3671],\n",
      "        [ 0.1551, -0.2450,  0.3506, -0.1589],\n",
      "        [ 0.7276,  0.4945,  0.7107, -0.0426],\n",
      "        [ 0.2859, -0.4374, -0.7300,  0.1447],\n",
      "        [-0.3638, -0.2034,  0.1595,  0.2042],\n",
      "        [-0.1410, -0.3172, -0.4344, -0.3353],\n",
      "        [-0.5585,  0.5275,  0.0272, -0.1467],\n",
      "        [-0.7507,  0.3205,  0.6668,  0.0206],\n",
      "        [ 0.0080, -0.3975, -0.6102, -0.3967],\n",
      "        [-0.2224, -0.0217,  0.3450,  0.1099],\n",
      "        [-0.2190,  0.1493,  0.8284,  0.5548],\n",
      "        [-0.2733,  0.0873,  0.0852, -0.1748],\n",
      "        [ 0.5116, -0.3160,  0.0473,  0.3435],\n",
      "        [ 0.2293, -0.3106, -0.1927,  0.4144],\n",
      "        [ 0.0677,  0.3826,  0.2693, -0.2723],\n",
      "        [ 0.4833,  0.5931,  0.5252,  0.1367],\n",
      "        [ 0.3951, -0.4855,  0.0440,  0.3506],\n",
      "        [-0.5334, -0.4577, -0.4373, -0.3058],\n",
      "        [-0.1121, -0.3801, -0.3940,  0.3661],\n",
      "        [-0.1822,  0.2912,  0.1745,  0.0126],\n",
      "        [ 0.0503,  0.1076,  0.0698,  0.0866],\n",
      "        [ 0.0066, -0.0206, -0.2535, -0.4911],\n",
      "        [ 0.3970, -0.0967, -0.6229, -0.0089],\n",
      "        [-0.4763, -0.4325, -0.5900, -0.4500],\n",
      "        [-0.3651,  0.1451,  0.4821,  0.0712],\n",
      "        [-0.4553, -0.1025,  0.0013,  0.2063],\n",
      "        [-0.2454,  0.3261, -0.2454, -0.4920],\n",
      "        [ 0.2667, -0.3085,  0.6774,  0.1548],\n",
      "        [ 0.3910, -0.3696, -0.6191, -0.1450],\n",
      "        [-0.1594,  0.1495, -0.0249, -0.3926],\n",
      "        [-0.6122, -0.3175,  0.5630,  0.0422],\n",
      "        [ 0.9547,  0.4848, -0.1522,  0.1912],\n",
      "        [ 0.1277,  0.1483, -0.0087, -0.3497],\n",
      "        [-0.0496, -0.1389,  0.4121, -0.0211],\n",
      "        [-0.1586, -0.3073,  0.3732,  0.2895],\n",
      "        [ 0.3784,  0.1291, -0.5967,  0.2855],\n",
      "        [-0.0133, -0.3971, -0.1061, -0.1544],\n",
      "        [ 0.2933, -0.2409, -0.1735, -0.0098],\n",
      "        [ 0.0270, -0.4149, -0.3626, -0.0773],\n",
      "        [-0.2511,  0.1895,  0.0370,  0.2430],\n",
      "        [-0.0918, -0.2984,  0.2341, -0.1694],\n",
      "        [-0.4207, -0.6343, -0.4847, -0.2208],\n",
      "        [ 0.6320,  0.2248,  0.4689,  0.1567],\n",
      "        [-0.2069,  0.2532,  0.5705, -0.2171],\n",
      "        [ 0.1886, -0.2090,  0.4048, -0.1791],\n",
      "        [ 0.1065, -0.2762, -0.6490, -0.3309],\n",
      "        [-0.4877, -0.2111, -0.1535,  0.3154],\n",
      "        [-0.2753,  0.3202,  0.6038, -0.3180],\n",
      "        [-0.2888, -0.3111, -0.1208,  0.2887],\n",
      "        [ 0.0599, -0.3282, -0.3431, -0.1294],\n",
      "        [ 0.0815,  0.3706, -0.4984,  0.4370],\n",
      "        [ 0.2873,  0.4738,  0.3862,  0.5222],\n",
      "        [-0.7841,  0.2575,  0.4849,  0.3199],\n",
      "        [-0.5750, -0.3873, -0.6115, -0.4782],\n",
      "        [ 0.1489, -0.4802, -0.7539,  0.2517],\n",
      "        [-0.3860,  0.3231,  0.8816,  0.4358],\n",
      "        [-0.4891, -0.2081,  0.4105,  0.4245],\n",
      "        [-0.5994, -0.1604,  0.1099, -0.3448],\n",
      "        [-0.2150,  0.3627,  0.2212, -0.4450],\n",
      "        [ 0.4269, -0.3280, -0.4988, -0.5081],\n",
      "        [ 0.4646, -0.4202,  0.1760,  0.1411],\n",
      "        [ 0.5633, -0.4114, -0.9363,  0.3340],\n",
      "        [ 0.2235,  0.3682, -0.2177,  0.1945],\n",
      "        [-0.0264, -0.2216, -0.2735, -0.0160],\n",
      "        [-0.0407,  0.3156,  0.3135, -0.0411],\n",
      "        [-0.4928, -0.7640, -0.2635, -0.4786],\n",
      "        [-0.5365,  0.0374, -0.0334, -0.1121],\n",
      "        [-0.2359, -0.2082,  0.3821,  0.2231],\n",
      "        [-0.4284,  0.2759, -0.1262,  0.3036]], device='mps:0')), ('layer1.bias', tensor([ 0.3606,  0.0251,  0.3673,  0.4545, -0.4916, -0.0216, -0.3364,  0.3141,\n",
      "         0.6623,  0.2155,  0.4663, -0.3509,  0.2820,  0.6487, -0.0265,  0.4559,\n",
      "        -0.4995, -0.3290,  0.5796, -0.3886,  0.5683, -0.4231,  0.6512, -0.5177,\n",
      "        -0.5041, -0.0393,  0.6558,  0.5163,  0.4725,  0.0037, -0.0058, -0.2983,\n",
      "        -0.1102, -0.2769, -0.1055, -0.4855, -0.4827,  0.4189,  0.5446, -0.0753,\n",
      "         0.6406, -0.0892,  0.5154, -0.0019,  0.3598,  0.6568,  0.5053,  0.1292,\n",
      "         0.6049,  0.3860,  0.0775,  0.4715, -0.2348,  0.5448,  0.3638, -0.3816,\n",
      "        -0.3844, -0.1771,  0.6674,  0.6384,  0.2647, -0.0687,  0.4380, -0.4714,\n",
      "         0.6517,  0.0126,  0.0311,  0.7033,  0.5142, -0.5334, -0.2701,  0.6607,\n",
      "         0.5648,  0.6795, -0.4642,  0.3109, -0.0429,  0.2953,  0.2825,  0.4503,\n",
      "        -0.2741,  0.4165,  0.0434, -0.4357, -0.3238, -0.5186, -0.3236,  0.6525,\n",
      "         0.0693, -0.2987, -0.1848, -0.4218,  0.4865,  0.1297,  0.2623,  0.5256,\n",
      "         0.5235,  0.6458, -0.4192,  0.6844, -0.1943, -0.0112,  0.2777, -0.2518,\n",
      "         0.6435, -0.3193,  0.2037, -0.3708,  0.4177,  0.6894, -0.1017, -0.0026,\n",
      "        -0.0698,  0.4574, -0.3276, -0.2949,  0.4755,  0.5286, -0.3523,  0.3361,\n",
      "         0.2111,  0.3728,  0.5194,  0.6407, -0.1189,  0.2482, -0.3558, -0.4435],\n",
      "       device='mps:0')), ('layer2.weight', tensor([[ 0.0564,  0.0937, -0.0154,  ...,  0.0509,  0.0242, -0.0336],\n",
      "        [ 0.1434, -0.2324,  0.1593,  ...,  0.0398, -0.1243,  0.0719],\n",
      "        [-0.0185,  0.0731,  0.0641,  ..., -0.0395, -0.0347,  0.0014],\n",
      "        ...,\n",
      "        [ 0.0652, -0.1806,  0.1275,  ...,  0.0467, -0.1091, -0.0258],\n",
      "        [-0.1519, -0.0258,  0.1956,  ..., -0.0104,  0.1735,  0.0173],\n",
      "        [ 0.0817, -0.0987,  0.1009,  ..., -0.0446, -0.1205,  0.0247]],\n",
      "       device='mps:0')), ('layer2.bias', tensor([-0.0869,  0.1193, -0.0723, -0.0469, -0.0349,  0.1743,  0.0445,  0.1839,\n",
      "         0.2115,  0.1389,  0.2589,  0.1180, -0.0983, -0.0752,  0.2537, -0.0267,\n",
      "         0.0056,  0.0016, -0.0084,  0.0241,  0.0413,  0.0924, -0.0461,  0.1252,\n",
      "        -0.0381,  0.1071, -0.0350, -0.0183,  0.2558,  0.1013,  0.2514,  0.2348,\n",
      "         0.1097,  0.0473,  0.0680,  0.0559,  0.1699, -0.0029,  0.1261,  0.1769,\n",
      "         0.2581, -0.0608,  0.0919,  0.0883,  0.0003,  0.1612,  0.0119,  0.1788,\n",
      "        -0.0752,  0.1351,  0.0443,  0.1730, -0.0160,  0.2372,  0.1147,  0.0666,\n",
      "         0.0420, -0.0738,  0.1863,  0.1039, -0.0849, -0.0330,  0.2589, -0.0777,\n",
      "        -0.0633,  0.0142,  0.2261,  0.1153,  0.0445,  0.0364, -0.0113, -0.0589,\n",
      "         0.2426,  0.2374,  0.1234, -0.0858, -0.0707,  0.0434,  0.1097,  0.2255,\n",
      "        -0.0806,  0.1369,  0.1571,  0.1062,  0.2334,  0.0381, -0.0650, -0.0854,\n",
      "         0.1052, -0.0487,  0.0591,  0.0309,  0.0428,  0.1553, -0.0963, -0.0560,\n",
      "         0.0232, -0.0643,  0.1790,  0.0249, -0.0603,  0.1494,  0.1125,  0.2275,\n",
      "         0.0360, -0.1006,  0.0296, -0.0900,  0.0888,  0.1426,  0.0211,  0.1487,\n",
      "         0.0592, -0.0130,  0.0320, -0.0144, -0.0068,  0.0195,  0.0200,  0.2508,\n",
      "         0.0469,  0.1676,  0.0605,  0.0259, -0.0030,  0.2000,  0.0810,  0.1222],\n",
      "       device='mps:0')), ('layer3.weight', tensor([[-1.4718e-01,  2.5362e-01,  4.7017e-03, -5.1124e-02, -3.4417e-01,\n",
      "          8.7358e-02,  3.5146e-01,  3.4578e-01,  1.6846e-01,  3.5524e-01,\n",
      "          2.7697e-01,  3.1212e-01, -2.2730e-01, -4.1740e-02,  3.4561e-01,\n",
      "         -3.7038e-01, -4.4925e-01, -5.1207e-02, -5.5076e-01, -2.4242e-01,\n",
      "          2.4123e-01,  2.0113e-01, -6.3091e-02,  2.3193e-01, -2.6836e-03,\n",
      "          3.6762e-01, -4.3922e-01,  3.0157e-02,  2.8407e-01,  2.6770e-01,\n",
      "          3.1301e-01,  3.0909e-01,  3.4704e-01, -3.1567e-01, -7.6710e-02,\n",
      "          2.8874e-01,  2.4743e-01, -7.2855e-02,  2.3049e-01,  2.0177e-01,\n",
      "          2.6514e-01, -5.0189e-02,  3.1181e-01,  3.8685e-01, -4.0742e-01,\n",
      "          2.1980e-01, -2.8675e-02,  2.6283e-01,  1.7844e-04,  2.5318e-01,\n",
      "          3.0004e-01,  2.7407e-01,  2.9180e-01,  2.6238e-01,  3.2751e-01,\n",
      "          3.8099e-01,  3.9162e-01, -2.0089e-01,  2.2554e-01,  2.6615e-01,\n",
      "         -2.7442e-01, -3.0729e-01,  3.2275e-01, -5.0798e-01, -1.4815e-01,\n",
      "         -1.5910e-01,  3.0173e-01,  3.4410e-01,  3.2155e-01, -3.7348e-01,\n",
      "          2.8241e-01, -1.3509e-01,  7.0208e-02,  1.0586e-01, -6.9281e-02,\n",
      "         -8.7790e-02, -4.7498e-01,  3.1066e-01,  7.6551e-02,  3.0661e-01,\n",
      "          2.5902e-02,  2.4804e-01,  2.6897e-01,  2.8552e-01,  1.8896e-01,\n",
      "          3.9813e-01, -8.3068e-02,  3.1952e-02,  3.0282e-01, -3.2230e-01,\n",
      "         -1.5088e-01, -3.5655e-02,  2.8482e-01,  3.5332e-01, -2.4182e-01,\n",
      "         -3.4352e-02,  2.6047e-01, -1.6812e-01,  3.8538e-01,  7.6521e-02,\n",
      "         -3.9467e-02,  2.3818e-01,  3.3565e-01,  8.4229e-02, -3.1859e-01,\n",
      "         -2.7681e-01, -3.2973e-01, -9.5291e-03,  3.6776e-01,  2.1473e-01,\n",
      "         -8.4089e-02,  3.1033e-01, -3.0541e-01, -3.0388e-01,  3.5260e-01,\n",
      "         -3.5427e-01,  3.1491e-02,  3.6735e-01, -1.6741e-02,  3.2686e-01,\n",
      "          3.6166e-01,  3.5077e-01, -3.0995e-02, -4.1172e-01, -2.5949e-01,\n",
      "          1.9873e-01, -4.2301e-01,  2.8427e-01],\n",
      "        [-1.2346e-01,  1.6454e-01, -8.9200e-04,  7.3645e-02, -2.6862e-01,\n",
      "          2.5148e-01,  2.0881e-01,  2.5164e-01,  3.0132e-01,  3.2025e-01,\n",
      "          2.4963e-01,  3.0751e-01, -2.6723e-01,  7.5347e-02,  2.1679e-01,\n",
      "         -7.2343e-01, -3.0747e-01, -8.4780e-03, -3.0195e-01, -7.5684e-02,\n",
      "          4.5029e-01,  2.2359e-01,  2.8240e-02,  2.3992e-01,  5.2988e-02,\n",
      "          1.2254e-01, -5.8046e-01, -1.2562e-01,  1.9759e-01,  3.2853e-01,\n",
      "          2.7178e-01,  2.7691e-01,  2.9144e-01, -7.6810e-02,  7.3768e-03,\n",
      "          2.7164e-01,  2.5505e-01, -1.3328e-02,  2.7914e-01,  2.1911e-01,\n",
      "          1.9817e-01,  4.2730e-02,  2.1461e-01,  3.1196e-01, -5.1949e-01,\n",
      "          3.0218e-01,  1.2182e-02,  2.8940e-01, -1.6648e-02,  2.4920e-01,\n",
      "         -2.2432e-01,  3.3442e-01,  2.9992e-01,  3.1588e-01,  2.6792e-01,\n",
      "          3.3280e-01,  3.2205e-01, -8.8247e-02,  3.2549e-01,  2.9115e-01,\n",
      "          1.6085e-01, -2.8392e-01,  2.9042e-01, -4.7983e-01, -1.8182e-01,\n",
      "          1.5410e-01,  2.3541e-01,  2.7593e-01,  3.2349e-01, -4.3676e-01,\n",
      "          1.7088e-01,  1.4146e-02,  2.5725e-01,  3.0404e-01,  4.9408e-01,\n",
      "         -5.0130e-01, -5.3712e-01,  3.4704e-01,  2.7184e-01,  3.8178e-01,\n",
      "         -5.3375e-03,  3.1541e-01,  3.1265e-01,  2.1601e-01,  2.8978e-01,\n",
      "          3.4035e-01, -2.2360e-01,  6.4573e-02,  1.7057e-01, -2.9634e-01,\n",
      "         -7.5498e-02,  1.3177e-02,  2.9366e-01,  2.6124e-01, -2.4967e-01,\n",
      "          5.2409e-02,  3.2720e-01, -3.4584e-01,  3.5741e-01, -6.1017e-02,\n",
      "          4.9006e-02,  2.9887e-01,  2.8643e-01,  2.4851e-01, -5.6580e-01,\n",
      "         -3.2534e-01, -2.2015e-01, -8.2179e-02,  3.7350e-01,  3.2674e-01,\n",
      "         -2.8460e-01,  1.4288e-01, -3.2345e-01, -2.2231e-01,  3.4324e-01,\n",
      "         -2.4626e-01, -7.2547e-02,  3.4333e-01, -2.4053e-02,  2.8686e-01,\n",
      "          3.2980e-01,  2.9578e-01, -6.4246e-02, -1.0013e-01, -3.0273e-01,\n",
      "          3.0645e-01, -4.7163e-01,  1.0383e-01]], device='mps:0')), ('layer3.bias', tensor([0.0395, 0.1229], device='mps:0'))])\n"
     ]
    }
   ],
   "source": [
    "filefolder = \"/Users/gracekim/Documents/School_Everything_and_LEARNING/Cambridge/Classes_MPhil/L46 - Princples of Machine Learning Systems/Final_Project/ReactionWheel_2023/\"\n",
    "torch.save(policy_net.state_dict(),filefolder+\"cart_pole_policy.pt\")\n",
    "torch.save(target_net.state_dict(),filefolder+\"cart_pole_target.pt\")\n",
    "\n",
    "print(policy_net.state_dict())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "DQN(\n",
       "  (layer1): Linear(in_features=4, out_features=128, bias=True)\n",
       "  (layer2): Linear(in_features=128, out_features=128, bias=True)\n",
       "  (layer3): Linear(in_features=128, out_features=2, bias=True)\n",
       ")"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model = DQN(n_observations, n_actions).to(device)\n",
    "model.load_state_dict(torch.load(filefolder+\"cart_pole_policy.pt\"))\n",
    "model.eval()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[-0.03335429 -0.02573095  0.01291334 -0.04689909]\n",
      "tensor([[0]], device='mps:0')\n",
      "False\n",
      "tensor([1.], device='mps:0')\n",
      "tensor([[1]], device='mps:0')\n",
      "False\n",
      "tensor([1.], device='mps:0')\n",
      "tensor([[1]], device='mps:0')\n",
      "False\n",
      "tensor([1.], device='mps:0')\n",
      "tensor([[0]], device='mps:0')\n",
      "False\n",
      "tensor([1.], device='mps:0')\n",
      "tensor([[1]], device='mps:0')\n",
      "False\n",
      "tensor([1.], device='mps:0')\n",
      "tensor([[0]], device='mps:0')\n",
      "False\n",
      "tensor([1.], device='mps:0')\n",
      "tensor([[0]], device='mps:0')\n",
      "False\n",
      "tensor([1.], device='mps:0')\n",
      "tensor([[1]], device='mps:0')\n",
      "False\n",
      "tensor([1.], device='mps:0')\n",
      "tensor([[1]], device='mps:0')\n",
      "False\n",
      "tensor([1.], device='mps:0')\n",
      "tensor([[0]], device='mps:0')\n",
      "False\n",
      "tensor([1.], device='mps:0')\n",
      "tensor([[1]], device='mps:0')\n",
      "False\n",
      "tensor([1.], device='mps:0')\n",
      "tensor([[0]], device='mps:0')\n",
      "False\n",
      "tensor([1.], device='mps:0')\n",
      "tensor([[0]], device='mps:0')\n",
      "False\n",
      "tensor([1.], device='mps:0')\n",
      "tensor([[1]], device='mps:0')\n",
      "False\n",
      "tensor([1.], device='mps:0')\n",
      "tensor([[0]], device='mps:0')\n",
      "False\n",
      "tensor([1.], device='mps:0')\n",
      "tensor([[1]], device='mps:0')\n",
      "False\n",
      "tensor([1.], device='mps:0')\n",
      "tensor([[1]], device='mps:0')\n",
      "False\n",
      "tensor([1.], device='mps:0')\n",
      "tensor([[0]], device='mps:0')\n",
      "False\n",
      "tensor([1.], device='mps:0')\n",
      "tensor([[1]], device='mps:0')\n",
      "False\n",
      "tensor([1.], device='mps:0')\n",
      "tensor([[0]], device='mps:0')\n",
      "False\n",
      "tensor([1.], device='mps:0')\n",
      "tensor([[0]], device='mps:0')\n",
      "False\n",
      "tensor([1.], device='mps:0')\n",
      "tensor([[1]], device='mps:0')\n",
      "False\n",
      "tensor([1.], device='mps:0')\n",
      "tensor([[0]], device='mps:0')\n",
      "False\n",
      "tensor([1.], device='mps:0')\n",
      "tensor([[1]], device='mps:0')\n",
      "False\n",
      "tensor([1.], device='mps:0')\n",
      "tensor([[1]], device='mps:0')\n",
      "False\n",
      "tensor([1.], device='mps:0')\n",
      "tensor([[0]], device='mps:0')\n",
      "False\n",
      "tensor([1.], device='mps:0')\n",
      "tensor([[0]], device='mps:0')\n",
      "False\n",
      "tensor([1.], device='mps:0')\n",
      "tensor([[1]], device='mps:0')\n",
      "False\n",
      "tensor([1.], device='mps:0')\n",
      "tensor([[1]], device='mps:0')\n",
      "False\n",
      "tensor([1.], device='mps:0')\n",
      "tensor([[0]], device='mps:0')\n",
      "False\n",
      "tensor([1.], device='mps:0')\n",
      "tensor([[0]], device='mps:0')\n",
      "False\n",
      "tensor([1.], device='mps:0')\n",
      "tensor([[1]], device='mps:0')\n",
      "False\n",
      "tensor([1.], device='mps:0')\n",
      "tensor([[0]], device='mps:0')\n",
      "False\n",
      "tensor([1.], device='mps:0')\n",
      "tensor([[1]], device='mps:0')\n",
      "False\n",
      "tensor([1.], device='mps:0')\n",
      "tensor([[1]], device='mps:0')\n",
      "False\n",
      "tensor([1.], device='mps:0')\n",
      "tensor([[0]], device='mps:0')\n",
      "False\n",
      "tensor([1.], device='mps:0')\n",
      "tensor([[0]], device='mps:0')\n",
      "False\n",
      "tensor([1.], device='mps:0')\n",
      "tensor([[1]], device='mps:0')\n",
      "False\n",
      "tensor([1.], device='mps:0')\n",
      "tensor([[1]], device='mps:0')\n",
      "False\n",
      "tensor([1.], device='mps:0')\n",
      "tensor([[0]], device='mps:0')\n",
      "False\n",
      "tensor([1.], device='mps:0')\n",
      "tensor([[0]], device='mps:0')\n",
      "False\n",
      "tensor([1.], device='mps:0')\n",
      "tensor([[1]], device='mps:0')\n",
      "False\n",
      "tensor([1.], device='mps:0')\n",
      "tensor([[0]], device='mps:0')\n",
      "False\n",
      "tensor([1.], device='mps:0')\n",
      "tensor([[1]], device='mps:0')\n",
      "False\n",
      "tensor([1.], device='mps:0')\n",
      "tensor([[1]], device='mps:0')\n",
      "False\n",
      "tensor([1.], device='mps:0')\n",
      "tensor([[0]], device='mps:0')\n",
      "False\n",
      "tensor([1.], device='mps:0')\n",
      "tensor([[0]], device='mps:0')\n",
      "False\n",
      "tensor([1.], device='mps:0')\n",
      "tensor([[1]], device='mps:0')\n",
      "False\n",
      "tensor([1.], device='mps:0')\n",
      "tensor([[0]], device='mps:0')\n",
      "False\n",
      "tensor([1.], device='mps:0')\n",
      "tensor([[1]], device='mps:0')\n",
      "False\n",
      "tensor([1.], device='mps:0')\n",
      "tensor([[1]], device='mps:0')\n",
      "False\n",
      "tensor([1.], device='mps:0')\n",
      "tensor([[0]], device='mps:0')\n",
      "False\n",
      "tensor([1.], device='mps:0')\n",
      "tensor([[0]], device='mps:0')\n",
      "False\n",
      "tensor([1.], device='mps:0')\n",
      "tensor([[1]], device='mps:0')\n",
      "False\n",
      "tensor([1.], device='mps:0')\n",
      "tensor([[0]], device='mps:0')\n",
      "False\n",
      "tensor([1.], device='mps:0')\n",
      "tensor([[1]], device='mps:0')\n",
      "False\n",
      "tensor([1.], device='mps:0')\n",
      "tensor([[1]], device='mps:0')\n",
      "False\n",
      "tensor([1.], device='mps:0')\n",
      "tensor([[0]], device='mps:0')\n",
      "False\n",
      "tensor([1.], device='mps:0')\n",
      "tensor([[0]], device='mps:0')\n",
      "False\n",
      "tensor([1.], device='mps:0')\n",
      "tensor([[1]], device='mps:0')\n",
      "False\n",
      "tensor([1.], device='mps:0')\n",
      "tensor([[0]], device='mps:0')\n",
      "False\n",
      "tensor([1.], device='mps:0')\n",
      "tensor([[1]], device='mps:0')\n",
      "False\n",
      "tensor([1.], device='mps:0')\n",
      "tensor([[0]], device='mps:0')\n",
      "False\n",
      "tensor([1.], device='mps:0')\n",
      "tensor([[1]], device='mps:0')\n",
      "False\n",
      "tensor([1.], device='mps:0')\n",
      "tensor([[1]], device='mps:0')\n",
      "False\n",
      "tensor([1.], device='mps:0')\n",
      "tensor([[0]], device='mps:0')\n",
      "False\n",
      "tensor([1.], device='mps:0')\n",
      "tensor([[0]], device='mps:0')\n",
      "False\n",
      "tensor([1.], device='mps:0')\n",
      "tensor([[1]], device='mps:0')\n",
      "False\n",
      "tensor([1.], device='mps:0')\n",
      "tensor([[0]], device='mps:0')\n",
      "False\n",
      "tensor([1.], device='mps:0')\n",
      "tensor([[1]], device='mps:0')\n",
      "False\n",
      "tensor([1.], device='mps:0')\n",
      "tensor([[0]], device='mps:0')\n",
      "False\n",
      "tensor([1.], device='mps:0')\n",
      "tensor([[1]], device='mps:0')\n",
      "False\n",
      "tensor([1.], device='mps:0')\n",
      "tensor([[0]], device='mps:0')\n",
      "False\n",
      "tensor([1.], device='mps:0')\n",
      "tensor([[1]], device='mps:0')\n",
      "False\n",
      "tensor([1.], device='mps:0')\n",
      "tensor([[1]], device='mps:0')\n",
      "False\n",
      "tensor([1.], device='mps:0')\n",
      "tensor([[0]], device='mps:0')\n",
      "False\n",
      "tensor([1.], device='mps:0')\n",
      "tensor([[0]], device='mps:0')\n",
      "False\n",
      "tensor([1.], device='mps:0')\n",
      "tensor([[1]], device='mps:0')\n",
      "False\n",
      "tensor([1.], device='mps:0')\n",
      "tensor([[0]], device='mps:0')\n",
      "False\n",
      "tensor([1.], device='mps:0')\n",
      "tensor([[1]], device='mps:0')\n",
      "False\n",
      "tensor([1.], device='mps:0')\n",
      "tensor([[0]], device='mps:0')\n",
      "False\n",
      "tensor([1.], device='mps:0')\n",
      "tensor([[1]], device='mps:0')\n",
      "False\n",
      "tensor([1.], device='mps:0')\n",
      "tensor([[0]], device='mps:0')\n",
      "False\n",
      "tensor([1.], device='mps:0')\n",
      "tensor([[1]], device='mps:0')\n",
      "False\n",
      "tensor([1.], device='mps:0')\n",
      "tensor([[0]], device='mps:0')\n",
      "False\n",
      "tensor([1.], device='mps:0')\n",
      "tensor([[1]], device='mps:0')\n",
      "False\n",
      "tensor([1.], device='mps:0')\n",
      "tensor([[0]], device='mps:0')\n",
      "False\n",
      "tensor([1.], device='mps:0')\n",
      "tensor([[1]], device='mps:0')\n",
      "False\n",
      "tensor([1.], device='mps:0')\n",
      "tensor([[0]], device='mps:0')\n",
      "False\n",
      "tensor([1.], device='mps:0')\n",
      "tensor([[1]], device='mps:0')\n",
      "False\n",
      "tensor([1.], device='mps:0')\n",
      "tensor([[1]], device='mps:0')\n",
      "False\n",
      "tensor([1.], device='mps:0')\n",
      "tensor([[0]], device='mps:0')\n",
      "False\n",
      "tensor([1.], device='mps:0')\n",
      "tensor([[0]], device='mps:0')\n",
      "False\n",
      "tensor([1.], device='mps:0')\n",
      "tensor([[1]], device='mps:0')\n",
      "False\n",
      "tensor([1.], device='mps:0')\n",
      "tensor([[0]], device='mps:0')\n",
      "False\n",
      "tensor([1.], device='mps:0')\n",
      "tensor([[1]], device='mps:0')\n",
      "False\n",
      "tensor([1.], device='mps:0')\n",
      "tensor([[0]], device='mps:0')\n",
      "False\n",
      "tensor([1.], device='mps:0')\n",
      "tensor([[1]], device='mps:0')\n",
      "False\n",
      "tensor([1.], device='mps:0')\n",
      "tensor([[0]], device='mps:0')\n",
      "False\n",
      "tensor([1.], device='mps:0')\n",
      "tensor([[1]], device='mps:0')\n",
      "False\n",
      "tensor([1.], device='mps:0')\n"
     ]
    }
   ],
   "source": [
    "env = gym.make(\"CartPole-v1\", render_mode = \"human\")\n",
    "\n",
    "# set up matplotlib\n",
    "is_ipython = 'inline' in matplotlib.get_backend()\n",
    "if is_ipython:\n",
    "    from IPython import display\n",
    "\n",
    "plt.ion()\n",
    "state, info = env.reset()\n",
    "\n",
    "\n",
    "print(state)\n",
    "\n",
    "state = torch.tensor(observation, dtype=torch.float32, device=device).unsqueeze(0)\n",
    "for i in range(100):\n",
    "    with torch.no_grad():\n",
    "        # t.max(1) will return the largest column value of each row.\n",
    "        # second column on max result is index of where max element was\n",
    "        # found, so we pick action with the larger expected reward.\n",
    "        action = model(state).max(1).indices.view(1, 1)\n",
    "        print(action)\n",
    "    observation, reward, terminated, truncated, _ = env.step(action.item())\n",
    "    reward = torch.tensor([reward], device=device)\n",
    "    done = terminated or truncated\n",
    "    state = torch.tensor(observation, dtype=torch.float32, device=device).unsqueeze(0)\n",
    "    print(done)\n",
    "    print(reward)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[ 0.0140, -0.1488, -0.0186,  0.2545]], device='mps:0')\n",
      "tensor([[1]], device='mps:0')\n",
      "tensor([1.], device='mps:0')\n",
      "False\n",
      "tensor([[ 0.0110,  0.0465, -0.0135, -0.0440]], device='mps:0')\n",
      "tensor([[0]], device='mps:0')\n",
      "tensor([1.], device='mps:0')\n",
      "False\n",
      "tensor([[ 0.0119, -0.1484, -0.0144,  0.2444]], device='mps:0')\n",
      "tensor([[0]], device='mps:0')\n",
      "tensor([1.], device='mps:0')\n",
      "False\n",
      "tensor([[ 0.0089, -0.3433, -0.0095,  0.5326]], device='mps:0')\n",
      "tensor([[1]], device='mps:0')\n",
      "tensor([1.], device='mps:0')\n",
      "False\n",
      "tensor([[ 0.0021, -0.1480,  0.0012,  0.2369]], device='mps:0')\n",
      "tensor([[1]], device='mps:0')\n",
      "tensor([1.], device='mps:0')\n",
      "False\n",
      "tensor([[-0.0009,  0.0471,  0.0059, -0.0554]], device='mps:0')\n",
      "tensor([[1]], device='mps:0')\n",
      "tensor([1.], device='mps:0')\n",
      "False\n",
      "tensor([[ 5.6749e-05,  2.4210e-01,  4.8021e-03, -3.4623e-01]], device='mps:0')\n",
      "tensor([[1]], device='mps:0')\n",
      "tensor([1.], device='mps:0')\n",
      "False\n",
      "tensor([[ 0.0049,  0.4372, -0.0021, -0.6374]], device='mps:0')\n",
      "tensor([[0]], device='mps:0')\n",
      "tensor([1.], device='mps:0')\n",
      "False\n",
      "tensor([[ 0.0136,  0.2421, -0.0149, -0.3454]], device='mps:0')\n",
      "tensor([[0]], device='mps:0')\n",
      "tensor([1.], device='mps:0')\n",
      "False\n",
      "tensor([[ 0.0185,  0.0472, -0.0218, -0.0574]], device='mps:0')\n",
      "tensor([[0]], device='mps:0')\n",
      "tensor([1.], device='mps:0')\n",
      "False\n"
     ]
    }
   ],
   "source": [
    "# Initialize the environment and get it's state\n",
    "\n",
    "# env = gym.make(\"CartPole-v1\", render_mode = \"human\")\n",
    "\n",
    "# # set up matplotlib\n",
    "# is_ipython = 'inline' in matplotlib.get_backend()\n",
    "# if is_ipython:\n",
    "#     from IPython import display\n",
    "\n",
    "# plt.ion()\n",
    "\n",
    "# state, info = env.reset()\n",
    "# state = torch.tensor(state, dtype=torch.float32, device=device).unsqueeze(0)\n",
    "# for t in count():\n",
    "state = torch.tensor(observation, dtype=torch.float32, device=device).unsqueeze(0)\n",
    "print(state)\n",
    "action = select_action(state)\n",
    "observation, reward, terminated, truncated, _ = env.step(action.item())\n",
    "reward = torch.tensor([reward], device=device)\n",
    "done = terminated or truncated\n",
    "\n",
    "print(action)\n",
    "print(reward)\n",
    "print(done)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Random notes:\n",
    "\n",
    "from [Stackoverflow](https://stackoverflow.com/questions/30539798/zsh-no-matches-found-requestssecurity):\n",
    "\n",
    "\n",
    "zsh uses square brackets for globbing / pattern matching.\n",
    "\n",
    "That means that if you need to pass literal square brackets as an argument to a command, you either need to escape them or quote the argument like this:\n",
    "\n",
    "`pip install 'requests[security]'`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- interpolation study in the beginning for the choices for hyperparameters\n",
    "- experiments for deployment on device (interpolation study about choices on the way / optimization for latency, etc) >> different ways of building up the cart\n",
    "- \n",
    "- whatever i get to converge in simulation, then trying to print out a pendulummn that is as close to simulated properties >> then try to see what happens with different shapes and how robust is simulation to the actual shape of the pendulum ( santa on a sleigh :D)\n",
    "\n",
    "interpolation studies of:\n",
    "- Simulation\n",
    "- physical setup (resemblance of simulation and latency) >> maybe do stuff that doesn't work just to show why it doesn't work\n",
    "- pushing this beyond and having a crazy pendulum :D "
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}