"
]
},
"execution_count": 2,
@@ -103,152 +112,63 @@
],
"source": [
"from qonnx.core.modelwrapper import ModelWrapper\n",
- "model = ModelWrapper(\"./step_convert_to_hls.onnx\")\n",
+ "model = ModelWrapper(\"../end2end_example/cybersecurity/output_estimates_only/intermediate_models/step_convert_to_hls.onnx\")\n",
+ "model.save(\"step_convert_to_hls.onnx\")\n",
"\n",
- "showInNetron(\"./step_convert_to_hls.onnx\",localhost_url='xirxlabs53')"
+ "showInNetron(\"step_convert_to_hls.onnx\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
- "# Part 2 : Parallelisation Attributes : PE & SIMD"
+ "# Part 2 : Parallelization Parameters: PE & SIMD"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
- "**PE & SIMD represent the amount of time-multiplexity to which we expose each of our network layers. \n",
- "These parallelization attributes are subject to certain constraints and should be selected accordingly.**\n",
+ "The computational parallelism can be varied by setting the folding factors or also called parallelization parameters **PE** and **SIMD** of each layer. These parallelization attributes are subject to certain constraints and should be selected accordingly.\n",
"\n",
- "We see how they work through an example of a multiplication computation (Matrix-Vector) in the `MatrixVectorActivation` layer looks like.\n",
+ "To see more details about how this is implemented in the `MatrixVectorActivation` layer (MVAU), please have a look at [this documentation](https://github.com/Xilinx/finn/blob/github-pages/docs/finn-sheduling-and-folding.pptx). A schematic of the folding in an MVAU for a fully-connected layer is shown below:\n",
"\n",
- "From the below block diagram, we observe that `SIMD` represents the parallelism within a single dot-product computation (the number of multiplications is a single clock cycle), while `PE` refers to how many such (Matrix-Vector?) dot-products execute in parallel.\n",
- "\n",
- "If `PE` & `SIMD` are set to 2 & 4 for a given layer that means, that within a dot-product 4 multiplications will happen in parallel and 2 such dot-products will execute in parallel.\n",
- "\n",
- "The base case of `PE` & `SIMD` both set as 1 suggest that there will be no parallelization therefore the resource utilization would be low (resources can be resued for differnt multiplication operations) when compared to settings where network layers have higher `PE` & `SIMD` values."
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "\n",
- "Question in the third line of the above cell.\n",
- "
"
+ "![](finn-folding-mvau.png)"
]
},
{
- "attachments": {
- "MVA-1.png": {
- "image/png": "iVBORw0KGgoAAAANSUhEUgAABX4AAAMbCAMAAADNe32MAAAACXBIWXMAAB7CAAAewgFu0HU+AAAAV1BMVEX////v7++lpaUgICDd3d0bGxvh4eEAAAAQEBBKSkq7u7syMjLNzc1WVlYNDQ2YmJhnZ2dCQkLx8fG1tbUrKyvU1NSrq6t2dnaIiIg5OTnDw8Po6Oj5+fnQLuJiAAAgAElEQVR4Ae2djXqiOhRFbdVGW387ttXW93/OSYJBxBw4OxFF3dz7jUDOSWAR12Qi4mDAhQRIgARIgARIgARIgARIgARIgARIgARIgARIgARIgARIgARIgARIgARIgARIgARIgARIgARIgARIgARIgARIgARIgARIgARIgARIgARIgARIgARIgARIgARIgARIgARIgARIgARIgARIgARIgARIgARIgARIgARIgARIgARIgARIgARIgARIgARIgARIgARIgARIgARIgARIgARIgARIgARIgARIgARIgARIgARIgARIgARI4J4J/H0vl69vhzP4+xrZtb+vr6/ylPbFxot9scvopSw4W9l/zUI9Z2XcQQIkQAIkcELgbW78spv53T9mZ1+3dlfp31djxnbfsogzZrKKO3b/M7EhH0U9J21wgwRIgARIoE7gb2jMdLWaWnH+urKfRdDvTwhdl/od2mVsIxf/Qlnldb+2Yl5tjKF/K1S4SgIkQAICgZUZ+rHs384M9zYmjH4nZnPIeLG+ndj1pRn6Pfv3oVlEFLv1Ne3nZn1I5AsJkAAJkIBMYGy+i8I3Y97tWhj9rsdh9mFppofJh0K/g8HLOKLY/WH4PDLmT26OJSRAAiRAAgWBgzTtxnztphTC6He6MofZh7V5rel38O5nGPYv5fI5GHwHJX/RvuxcJEACJNBO4MOsrTuPy/bw0dv0d1HMPvyZ9b+6fgdjs618FucnJ37crv2bm8DgQgIkQAIk0Erg236Qtvo+DljL0a9VrL/3YWte30/mfl2VczO3+p2MJ+Pif2vqqXn9sh/SmXn8tojWA2EACZAACTwZgdeFdabZrNzEr12Cftd2zc8+7Mzn+eh3ZaZF+PHPnflZmN3O3iXhbhzmQgIkQAIk0EbgZesGrcbs/GA36Hc6mPk7HUZ2nBvTr7s97WSxd5zt7MD3beNnL06KuEECJEACJBAn8Pn+Y+05ccPWMPdr7x4butmHrb0z4nzyYRU+ZzvWtzELP+yd8cbfIxSukQAJkEA7gX8TN59bTj7YyQU/+7BZ7COj351ZDQavm83w8L8dC++CkTdm2d4aI0iABEjgyQm8rkpXLv39DcfJh8GX/ebFl3Pye/3OB3uPr00rv4Xs73yYOyO7ZepugeBCAiRAAiTQSKCwrQ/5NQv7GvTrvrq2MbOtsTcD/6vf+WDFax+88/n29vbn/rf/uVmKw7fdPsxrY5MsJAESIAESGAx+jfOrX368P8Pcr7uzwc4+bCb2Rt766Hc0CSPdQ6p7+Tp8281+e463nlW4cJUESIAE4gTss3a27q7fz59CxNXR75cZe8+GOx/Ge7v8zbYTMz7eKFxWuzZTq+r99PyetDKEKyRAAiRAAoHAi73lwQx37k9/m28YA7vRr5198M+BCHc+2Jhi2fh71EIVh9fR2Ix/thsz4eC3RoabJEACJBAjsLfGdMuu+N5FGP16/W79Z27lnQ8+bjGeLuPfLC6eHDylfWOYuY8ESIAEIgT+Zu+zyGxCJLJl18vv70XqaWmGxSRAAiRAAiRAAiRAAiRAAiRAAiRAAiRAAiRAAiRAAiRAAiRAAiRAAiRAAiRAAiQAEnibr8AMhpMACZAACVyAgP0K3AVqYRUkQAIkQAIgAeoXBMZwEiABErgMAer3MhxZCwmQAAmABIrfHgKTGE4CJEACJJBLgKPfXILMJwESIIEkAtRvEjYmkQAJkEAuAeo3lyDzSYAESCCJgP01i/hzJpNqYxIJkAAJkICSgNVv8URgZTzDSIAESIAELkJgZsJvaV6kOlZCAiRAAiSgI2BHv/wFeR0qRpEACZDAJQl8m8nh5+AuWSvrIgESIAESaCbwPTHLpTGb8LP0zdEsJQESIAESuACB/a/7Oc75fvA7tAL+mfEOiAtAZRUkQAI9JrB/ue0yGs3eX5c/8w/3C8dj//PG+6UVsFmsV9vv39HbbQ+vx1eOh0YCJHCfBP7+bVfTzXjhpNeTZbP6F0a8+/dV8cP0PTi0yXA3/1n+ft7ndeZRkwAJ9IrA/nflh5eF2xaLyeSG/w+Hm/V8tX39fakxenF/QeyG48ktl4r+d9tR7Qi5SQIkQAIQgf1ybKWy2K2W/2ZvdedBNT1F8P5v9Pu9LeZG1vxKyFNcc54kCXREwH2wNfz5Df/M76iVx6v25XtlZ2qmf493ZjwjEiCB6xB4tfLlbV1prF+2CzPkDEQaPGaRwNMT+Ge/1MCBb3I3eNuZMce/yfiYSALPTOBzbLa0b0YP+Pww84x0ppIACTwtga35eNpzv8yJjxaG0w+XQclaSOC5COzM93Od8OXPdsVHAl0eKmskgccnsDeGd5plXuZXM82sgekkQAJPSODPjJ/wrC97yr9md9kKWRsJkMAzEHgzi2c4zU7P8ddsOq2flZMACTwkAeo3/7JSv/kMWQMJPCEB6jf/olO/+QxZAwk8IQHqN/+iU7/5DFkDCTwhAeo3/6JTv/kMWQMJPCEB6jf/olO/+QxZAwk8IQHqN/+iU7/5DFkDCTwhAeo3/6JTv/kMWQMJPCEB6jf/olO/+QxZAwk8IYFL6/fve7l8fTuA/Ptyz6L5+/r6Ksnui40X+2KXUfMXnt++7uJZbNRveXm5QgIkcCsCb/Pil9B2M38EP/7buFu7r/Svfba7+5bzMvxi2mQVXH1+zPZRYnyS7jkW7iEBEiCBMwJ/9leLpqvV1Lr11xX+LNzDEJx+f0LsutTv0C7+J+akX9rY74yR3Rwq5CsJkAAJkMBgZYbel387M3TTBmH0OykfivBiVTyxJUsz9Lz270OzKIbKZ/yctqnfMyzcQQIkQALnBMbh2cFvxrhfAA6j3/U4zD4szfQw+VDodzB4GZv1eU12z6/5MJx8iKLhThIgARKoETjMOdi987WbUgij3+kqzD6szWtNv4N3Y+zwd/9SLp++1v1wMeLotwaYmyRAAiQQJ/Bh1oU7D8Xbw0dv099F8UjGP7P+V9fvwP7YXOWzuGJyYmAnMpYDjn7jnLmXBEiABGoEvo1ZrL6PdyuUo1+rWH/vw9a8vp/M/boK5u6nKpeT8WRc/O9N/e2mJDj6rQHmJgmQAAkIBF4X9uMys1m5iV+7BP2u7Zq/92FnPs9Hv6vIb/X8TSbW4tRvwZF/kgAJkEArgZetvbPMLjs/2A36nQ5m/k6HkR3nxvR7/ls9a/8hHicfWoEzgARIgARKAp/vPxs7geu+7xbmfu08wtDNPmytVM8nH1bntz4s3XwER78lU66QAAmQgJLAv4n353H0W8w+bBb7yOh3Z1aDwetmMzz8vxt8mfHf/nP/aczo090/zIUESIAESKCJwOtqGYqX/v6Gin6/7DcvvpyT3+t3Ptifurdp5beQ3Z0P7gsXYSmrDFXzlQRIgARIoEagsK3f+et/QDno132vYmNmW2NvBv5Xv/PBivdlMPh8e3v7c//b/wbLcbHYLygPqd8aZW6SAAmQwBmBX+P86pcfP58b5n6ndp+992EzsTMJ9dHvaOLmHuIL73yIc+FeEiABEqgRsM/a2bq7fj9/ChFXR792Ptd7Ntz5MN7b5W+2ndh53lo15Sb1W6LgCgmQAAk0EXixtzyY4c796W/zDWNgN/q1sw/+ORDhzocwt2s2/h61aLW88SyKhTtJgARI4IzAfuvUa8yu+N5FGP16/W79Z27lnQ8+bjGeLhtubeDo94wwd5AACZCAROBv9j4TZxOkJO4nARIgARIgARIgARIgARIgARIgARIgARIgARIgARIgARIgARIgARIgARIgARIgARIgARIgARIgARIgARIgARIgARIgARIgARIgARIgARIggdsRGK39r1nc7gDYMgmQAAk8J4Ev/9Nvz3nuPGsSIAESuCEB+/TJG7bOpkmABEjgaQlQv0976XniJEACtyVA/d6WP1snARJ4WgLU79Neep44CZDAbQnM+NHbbS8AWycBEnhWAhz9PuuV53mTAAncmMCXMQ2/MXTjg2PzJEACJPC4BKx+vx/37HhmJEACJNBbAla/694eHA+MBEiABB6XgNWv2T7u6fHMSIAESKCvBL7NxJgVp3/7en14XCRAAo9K4HVhlq/GbF4p4Ee9xDwvEiCB/hHY//4MjZnvB7ONMeOfXxq4f9eIR0QCJHATAp8vHSx/o9Hs99/r9me6Wdhp3+HSndp+aQVsFrvV9vV3NvrroNkXuv0mfYiNkgAJQATevn/mu42dku16+Vj9lgc2cyPhLpfFeDNdbd8/yxa5QgIkQAJ9IrD/txpbCbqBqTGTLpbh8GM3ndtx7ld9QPryu3XeH44v3+yiOCF3Uh8/X30CzmMhARIgAUfgc+vcO57+LN+//up2vHNEn2+zf9vVznl49+/Oz4WHTwIk8GgE3u2//4fbxx4c7t9Xdlpl/fdo147nQwIkcM8ElvZf5s8wLtxvJ2by2H/J3HM35LGTwBMS+GfMz4NNOEhX8W9txhz/SnS4nwRI4MoEPsdP9OXf/drwB5av3MHYHAmQgERga3ZS0QPuf1sYTj884HXlKZHAXRL4eK4HP66eaKx/l/2RB00Cz0Ngb8xTfSPh1Uyf5+LyTEmABPpM4M2M+3x4Fz+236eaa7k4PlZIAiRwOQJvZnG5yu6gpl+zuYOj5CGSAAk8AQHq9wkuMk+RBEigjwSo3z5eFR4TCZDAExCgfp/gIvMUSYAE+kiA+u3jVeExkQAJPAEB6vcJLjJPkQRIoI8EqN8+XhUeEwmQwBMQoH6f4CLzFEmABPpIgPrt41XhMZEACTwBAer3CS4yT5EESIAESIAESIAESIAESIAESIAESIAESIAESIAESIAESIAESIAESIAESIAESIAESIAESIAESIAESIAESIAESIAESIAESIAESIAESIAESIAESIAESIAESIAESIAESIAESIAESIAESIAESOA+CexfcpbP85OuVLg/L73unj4dy3XPnK2RAAn0n8CryVnm5ye4rNQ3XP2eB1xxz+mxzK7YMpsiARIggTYCVr+T1GVhpufVV5VnTTx9OQ+52p7ascwjg/WrHQsbIgESIIFTAq8xhZ6GiFttufvZz8Rs/sT8axbYYzHm45Z/F1zzbNkWCZDAHRBoU2jTKShy3zZmffMp4MM5vA3Nuul0WEYCJEAC1ySgUKh4OJrcv6HZihVcueBtbJZXbpLNXYvAW+UzB66SQDuByMzptTpr2Y5GoWVwbUWV+24W/Zh+sAf/z0w4/VC7iI+ySf22C4cRVQJPod/BtD/D38GOw99z3f5bbSbHbnle3tWe72q7i9xW3sxYfwvl1qz0wf2L/Lvz468QXZltZat5FYltrsmWqsaOuZ2yNT/nKHS5/8y49SiuFfBthtdq6l7a+fo4qtetXeu4Z5uTdi+hX/2hL81KHbx/wW+YwXKwaHfgW/OjPv4Q+PmCfgiDZwwGaM4KGBDpYpVH8C/jnoOANP9Vp9B4O8rcsenNHbf7sRnFz+VZ974vzHg7+xuVy5VA/FuY4UXbtaNf/aEj+k1RHZaDRbuzxDMGgx/4X6F4Bt6KTqnFldXF6qLsRORzTD4M7L8Z9G+MjiPnPTqWjk9VVf3XwtzkbujZwqzwQWXTKVG/TXRsGS5TPANvRSlLf266WF3U8+hXOUhu6T2XKX41kS/qXabqu6zlw8zRf5Je4jz3m4tfB+q35cLgMsUzqN+Wi1AvznGjMndkJvVWb7b9xcnfKvulGV52DFqtvGHdtntp61O/DbxdES5TPANvRTlW9eemi9VFPc/od7Awvbn1bG/Mpd/3Lb2+18Ub83qL49sPzfel26V+W4jiMsUzqN+Wi1AvVo5g62l+W5v70Z/P3gabHh1LFOo1d/6a8U3+Mnrv4N8g1G9Lz8Flimck6VfdBXXjWl3UE41+p5cf67R0Nbl4av7Jhc9Wskq4f+kSjLpol/ptuTK4TPGMJP22HPexWCdWXdQT6VcJ5Mi5w7X5bf653eEZZVQ9NLd5IOiwg3+CUL8tHQGXKZ5B/bZchHqxdgKhnue2tbkplzHW3iX29elYLnE+OXW8mOwvOyQ138lnsdRvy7XAez6eQf22XIR6sVah9Ty3rc1NuUk81t4l9vXpWC5xPjl1/LvRE+C+u7jjnfpt6Qq4TPEM6rflItSLtQqt57ltbS7yHaNYO5fcR/0eadonBxw3rrjWyTWgfluuIC5TPIP6bbkI9WKtQut5blubS/3G6N1+363m5DuZf6d+WzoULlM8g/ptuQj1Yq1C63luW5vbJ/326VhiTK+5b32jO1J25j35NF+/hFTqVwATduMyxTOo30Bb+apVaKw6bW6flNenY4kxvea+W90DPUx/7tF+KH1pnPpt6Tq4TPEM6rflItSLtQqt57ltbW6flNenY4kxvea+yY2+jZjRrv3xVGH4S/22dB1cpngG9dtyEerFWoXW89y2NrdPyuvTscSYXnPfrb6Anf5QYfttZelZPdRvS9fBZYpnUL8tF6FerFVoPc9ta3P7pLw+HUuM6RX32edfXLG1Y1MZtxvbwa80/KV+j4Sja7hM8QzqN4pe3qlVaKwGbW6flNenY4kxveK+lxs9iS69XTf4lYa/1G9L18FlimdQvy0XoV6sVWg9z21rc/ukvD4dS4zpFfd93uhLb5AoT3j4wa8w/IVqRXpByl3KWA4W7YjgGbgYUzLwHOTmR12sLuqJnvmAdPaTt1sHG306lg5OD6kyYxIAaeYsFhJlNbsY/ArDX6hWpBekqA7LwaIdETwDF2NKBp6jlKXvBrpYXRT1W31nXW0deeNd7aBu09Dd6ffVTT24ZRYBRv1GoFR34VMJeAb1WyWuWNdOIMSq0ub2SXl9OpYY0yvuuzf9hsFvfPhL/bZ0HVymeAb123IR6sVahdbz3LY2t0/K69OxxJhecd+96fcw8ysMf6nflq6DyxTPoH5bLkK9WKvQep7b1ub2SXl9OpYY0yvuuzf9+tse/ORD7Ilp1G9L18FlimdQvy0XoV6sVWg9z21rc/ukvD4dS4zpFffdmX7LmV9r4MX57C/129J1cJniGdRvy0WoF2sVWs9z29rcPimvT8cSY3rFffel3+PMrxsAT884Ub9nSE534DLFM6jfU+atW1qFxirS5vZJeX06lhjTK+67L/36we/ix4xXfv7hbPhL/bZ0HVymeAb123IR6sVahdbz3LY2t0/K69OxxJhecd9d6dcNfhc/L06yIyfgs+Ev9dvSdXCZ4hnUb8tFqBdrFVrPc9va3D4pr0/HEmN6xX13pd9XJ9/BoJCsE3B9+Ev9tnQdXKZ4BvXbchHqxVqF1vPctja3T8rr07HEmF5x3z3pd//x8+fQBMmO5vXn/oYSFUCkF6R8wwzLwaLdCeIZuBhTMvAc5XfU/FXVxeqi+K031Rvl0kHIG+/SbfesvnvS74sd+brlKNm3fbEn/HksCXsaXpFekKI6LAeLdqeFZ+BiTMnAc5Sy9NdSF6uLon4b3h7dFSFvvO6Oohc135N+AzBZsnJJyK28Ir0gRXVYDhbtTgPPwMWYkoHnKGXpr50uVhdF/VbeDtdbRd541zuqm7RE/aqwp6gOy8Gi3UHjGbgYUzLwHKUs/ZXSxeqiqF9V5790EPVbEqV+SxRNKymqw3KwaHeseAYuxpQMPEcpS3+BdLG6KOq3qc93Vkb9lmip3xJF00qK6rAcLNodK56BizElA89RytJfIF2sLor6berznZVRvyVa6rdE0bSSojosB4t2x4pn4GJMycBzlLL0F0gXq4uifpv6fGdl1G+J9lb6/TLD8hjQFfkDNrkk0gbSC1JUh+Vg0e508AxcjCkZeI5Slv4a6mJ1UdRv5G3R/S7kjdf90dy0hVb9fhXLyN9xaw91dNjhXsK+kzP4m80ON4id7K5tvJt1bY9+U5asXBKpHekFKarDcrBodzp4Bi7GlAw8RylLfw11sboo6jfytuh+F/LG6/5obtpCq37H/ukK9o/J6s0daeWJj+bn/ND/fbj49dd5yemenEsgS1YuOW3dbyGHkKI6LAeLdieAZ+BiTMnAc5Sy9FdNF6uL6o1+16PE5W17/s17j6n+B9LZ67mX3u7TsVz63MD6FPqdDN2yMGbs/Ds0xbbbt61962EwsE9D/1itrKJ/W45jZbYtEXKxLFm5JFIb0gtSVIflYNHudPAMXIwpGXiOUpb+GupidVG90e8kjHESXs8efBLp6u6NuYruv8XOPh3LLc6/0marfifmtQj/Xhj3Hd+hWVbSa6svE2/V/dRsaiX1zQ/zXt+l3pYlK5dEKkd6QYrqsBws2p0OnoGLMSUDz1HK0l9DXawuqj/6naQuC45+I2/tu9nVqt9x0K99u5vPFv0uD9r9E34IvsTyZxZnI+eysG1FlqxcEqkT0RcSG5rCcrBo1waegYsxJQPPUcrSg9XF6qJ6o1/dCDZ0rOorH7lTpXF36wr9htHuzDu1Nvrdv5SLdfN2syoIjFsGt1s/kk6kJUtWLok0hegLiQ1NYTlYtGsDz8DFmJKB5yhl6cHqYnVR1G/oq1d9Rf7ZedUDu35jrfotJx8Gv8aMBoONWVbHrZWfvpwcj75t9PvZpudjVZE1WbJySaQaRF9IbGgKy8GiXRt4Bi7GlAw8RylLD1YXq4uifkNfveor9VvibtXvcfJhZZxga6Pf5WQ8GRf/V6Z7f1ru6l2Zj/II8BVZsnJJpBVEX0hsaArLwaJdG3gGLsaUDDxHKUsPVheri6J+Q1+96iv1W+JW6NdPPnx+2aebuxvNhmYxPiwV35b1+ZV3Y75P95xu/cR+JvM0pGlLlqxcEqkP0RcSG5rCcrBo18Y1HoR+nVbm5ecLgZ38qoulfmsE+6S8Ph1LDdO1N1v1W7knxn9RonLfb2W24eSwfxeNd7l8TQ3wdjupudiQJSuXRKpBhIfEhqawHCzatUH9BtLnr9RvjUmflNenY6lhuvZmq34PX7tYjKevfs53aLajt8MSP9jDHWq1wuVPsczd9zLCp3m1GOWmLFm5JFI1Irw01f18lp9Ltq7YG6FbY04Dfoy+/rfDbf1ze/WwBc8YjZpzZmfL1GzP9kk7dLFrXY1L5W1bkd5zwV3auxdiTWpz+6S8Ph1LjOkV9yn0uxzs7X9hqc39vm42w8P/uyLE3p8W+TbcYFe9o3zRODcRmpJeZcm+VRvhOgm0EJhSv9KbrMP91G8Jt1W/xzsfipyafut3PuxXi/j32cLo142Bp2bR+q3k8gjPV+5GvwvgZnorisnC/qf834baX31W1z92X1K0y6LyncViT9ufeEZbKx9ny8QMz/ZJO3SxuqiPFfV7/vbqfA/1WyJu1e/xzocip6bfTzsP8ef+t/+5gLlZ/CsrF1fyun2Tfsdim2cF3U8+IN+rxqc38IzBQDknWmGFZ+Ct6D5OKw5KF6uL4p0PlQt9vVXqt2Tdqt9JbaLW6vc4E1FWE1bszWmzsN7wau8Ljj4trSGlUkT9ehjUb6VP1Fap3xqQPimvT8dSw3TtzVb9no9+t/tyqR+t/WbGv0Nhveh02344c7oD2aJ+PS3qV+401G+NTZ+U16djqWG69qZCv6e3KWwqn2Ys6ke7PhY2f7im/cC23oDfpn49Buo32jv8Tuq3xqZPyuvTsdQwXXuzVb/nH70dFXum32NR8/cuBiP/DbrEk6V+PTjqV+4/1G+NTZ+U16djqWG69marfjs6oEnG5C/16y/KA+l3cXioqaKzzVWxuih+9KYAfvkQ6rdkeiv97lofyF4e4tkK9euRPJB+gW9B6sa1uijq9+y9dY0d1G9J+Vb6ndduqCgPSLFC/XpI1K/cV6jfGps+Ka9Px1LDdO3NW+k3xR2BDfXrSaQgxO/ixTN432/oqMrXnM+htbl9Ul6fjkV5iboKu5V+cy4B9et7wwPpVzWfW7wFdLO6uihOPnSllcZ6c977jRXfX+Gt9Pua8XMX1K/vZw+kX879ppqDo99Ucr3Iu5V+/xn/+MokBtSvx0b9yr2Hc781Nn0acfbpWGqYrr15K/3+msMD0hJOmPr10Khfue9QvzU2fVJen46lhunam7fS71frT9HLJKhfz+aB9Mu5X7m3N5doJx9yJvuajwAvpX5LZrfS76jl1+DKA4ysUL8eygPpl3O/kW6u2kX9qjD1NehW+pUV2k5KzpVLIrXygZMRKGe7eOPZGZJL79AqNNauNpej3xi9m++jflWXIGWkieVg0e6g8Qz8jtyUDDxHOVPrr5QuVhf1RDee/evFg+WL9xonHwoO9k/qt0TRtJKiOiwHi3bHimfgYkzJwHOUd+n6C6SL1UU9lX7T7zRqemOklFG/JTXqt0TRtJKiOiwHi3bHimfgYkzJwHOUY1V/gXSxuqgn0u+v+Wjq31cto35L3NRviaJpJUV1WA4W7Y4Vz8DFmJKB5yhl6S+QLlYX9UT6HYl3Go1et8t/L4fOP/pyv0Lz9vU1Ouyw/z7++nor9n3ZZfRZFsRWKnmx4mIf9VuyoX5LFE0rKarDcrBod6x4Bi7GlAw8RylLf4F0sbqoJ9Lvnzl7OLfH+ftRPKN7Xvz4187/SvnUmHH5i2Jz47+gal+KZbiVDfytup+J+vXo3R/Ub4miaSVFdVgOFu2OFc/AxZiSgecoZ2r9BdLF6qKeSL8DY0qhVjr6u/0h6/lqtTNm6P1b6Nf9aM37IWo/CfpduB/FHtuioRsNx5Y3+wPTsf21fdRvCYT6LVE0raSoDsvBot2x4hm4GFMy8BzlWNVfIF2sLuqZ9Ds0X5EOvjFTb+XZpHgGy27xY6OmZmJWh+hvsziMfos9n8uF2QjjX6tt6vfATfdC/ao4pagOy8Gi3UHjGbgYUzLwHKUs/ZXSxeqinkm/69jvG3wZc5j1fTUL59Qw+WB/sdzTHgzmZnrQ7/ywx/6ebvx3crdmR/0eIClfqF8VqBTVYTlYtDtoPAMXY0oGnqOUpb9Sulhd1DPpN3KwBScAACAASURBVPr1mVk5JfH5MXVTCjtv1rXZbg6zD5+Lj5+T0a/veBM7ZP58CctnMRj+Wmz+Ub++k6r/oH5VqFJUh+Vg0e6g8QxcjCkZeI5yptZfKV2sLuqZ9BudcP0z/qO241sgjH6328Psw7fZBv2G0a/9nVwzc8PicvEl+42Z/Yr3VxybGAyih1INeJ516ld1rVNUh+Vg0e6g8QxcjCkZeI5yrOqvlC5WF/VM+o2b8cfO1v68H6dyi7lfO/q1P0XuJ4WnZhT0u/L83R8T90thq3FYJr5kZQfOvxz9lpBUK9SvClOK6rAcLNodNJ6BizElA89RytJfKV2sLuqZ9Ls3sd8W31v/2mX3c/hcrhz9Dj787MPnYmf7mRvdzv2f/hIMNpHJ33f3BNl36rcgpP2T+lWRSlEdloNFu4PGM3AxpmTgOUpZ+iuli9VFPZN+7e0M0Q/Mvn42TsCLuf8MLtz5sB0Usw9LO84N+l15/u6PTW3Kwu76Gy/sNzU4+i0Z6VaoXxWnFNVhOVi0O2g8AxdjSgaeo5yp9VdKF6uLeir92o/F/HTCeX//+17Z23l3rvQ4+n3z9z5M7ZA56Lec+x0Mz01e/HI59XtOt3HPnxk3lndVCD0asnYQcq5cUqvCbfKBkxEoZ7uin5ifRZ3uQHOUY1XfiC5WF/VU+rUfjc2/RsflOOFrue63xs3nHu98cKvv9ktZ9kE9Qb8rz98FG/d85tVmePhvs7Kfpq33n/tPp/iTekPKySs/eitxQMIqs/JXctqVc+WSyBFTvxEoZ7tQlboK0BylLP2x6WJ1UU+l38Fs4WYZyuWfxbmd21sYiqWY2z2Ofv3sg5t7KPVbjn7tjb92ouHkzgf7NeVyiX29I7TiX6nfEkfOj/6UlSSsQKKs1S/nyiW1Ktwm9RuBcrYLVamrAM1RytIfmy5WF/Vc+h3MCklOisV9q7iwree69Q8Eruj3z977sHbfyjgb/c7909Ne3v7sf/6Pl3AbhP2C8nhM/Xqgqj/8B5aqyMsGQaKsNS3nyiW1Ktwm9RuBcrYLVamrAM1RztT6Y9PF6qKeTL97O2Bd/Vav8NZMwvMbPvx8bkW/g7Upfkoj6DeMfr/93EO1mnKdc78lCt3K942egv9SfqtRd5zVKFmyckk1/7COfHSFxIamsBws2rWBZ+BiTMnAc5RjVQ9WF6uLejL92m8Sn8jXPnBrbMavbrJ2NDeLk2+92X1LM/a/wRf0u9rv959/7ysjK4P69X1U/8fN5mHKrzvqjzVEypKVS0Ju5RXRFxIbmsBysGjXBp6BizElA89RytKD1cXqop5Lv7+L88c+uPngxWY3tBO3bi64cueDvZfMFjk1B/2Wk7tT8dO1d37rzXdS9R8p72F15U2Bk9hd4E0JxzJZsnLJMbtcQ04diQ0NYDlYtGsDz8DFmJKB5yhl6cHqYnVRz6XfykRv6KJ2/OvuObPLvJixrU4+2NkHP99wot/FZh4eRXmspFx73NGve9S8XUbFY5HtPxeKbf9n2FdiOKy8fQl3+h0D18Xfescd11r7iD4AT9W6LFm5JFIxoi8kNjSF5WDRrg08AxdjSgaeo5yp9WB1sbqop9LvuxnHR62j3/dZvCT05Au/3uxf3BnnUfwlZf+emqz8bLn790JY3DM6I8to0T7CHJsw9x6poMtdUz+vlNSCLFm5JNIQoi8kNjSF5WDRrg08AxdjSgaeoxyrerC6WF1Ub/S7Pt6Oi629FTcshC7X9Lo6/6paU3iHZfep34l72vzQTtaMnTGHpth2+7bRQe7ePsO+za2zG33rwslD+Euj/brLkpVLIrUi+kJiQ1NYDhbt2sAzcDGmZOA5Sll6sLpYXVRv9Gtv2EpepqG/Nb/aX60o7/Ftjuy89B71Owmjxe/i6fND/y2VJlT2iyyt+g3PlWuqp5uy9/SfXpUlK5dETgLRFxIbmsJysGjXBp6BizElA89RytKD1cXqovqj38O9uPjLQr4NIfTD4lU3LXua09HWPeq3uAnEAbFatVM1rfq1Py0dfcjRCdOd+T7Zvt7GfpE8+StLVi6JnBiiLyQ2NIXlYNGuDTwDF2NKBp4zB3qhLpZzv6EXhtftIty3G/bc7PU+9eu+k+0W+4h6+zFlTb/78Oj5l5diGn0/XNjHIrdMPszMIjpvUTTU7Z+r8uek0HZkycolkTYQfSGxoSksB4t2baBfbkjLuUYrU+DzX13snY1+lRMIoWdVXotvRlR2SKvFM3Gk0qvuv0f9lpMP9qlu7ivXG7OsmnN5nDsqfqXJvW1aR7/T9AnY7Av2ZRatX0+MNyJLVi6J1IQID4kNTWE5WLRr4xpivE4rOqUWXHWxujFybyYfrqDf3flNv6GjXvv1HvV7nHwofgavNvpdTsaTcfH/xuH8ds8qahv92l8xle5Zu8IlWZlNWuuyZOWSyPkgwutedXgLeIZ7TsprhETTLjwDb0Wn1OIodbG6qGfS78TNWPZjuU/9+smHzy/7pT93z8DQLMKPfXjfnpL9m0ys2lr0a78GEyY0TrOvs/W5McOk8a8sWbkkckqX0G9lzuc4+3NYs7++crZP3qGLfqvclzS3PwmDLvaZ29UqWtNn9jEt2xm62B+rgVLsDzwe4t9bFxvbGvP+rot6X2o/uIp0n8vtUk8gRJrU5uZ8yT/SbNaue9Rv5d4UO651+i2X8JvQFSZr/2FG4+TDp/0Ib1VJuf7qn33O/vx1VnGA7hhkyb6VTLhCAu0Epk+j35HqZ4B077/cqHvU7+FrF4vx9NXP+donzo/eDssZj2XxdcHY6Pf1xy+rufV5+7Phzmq+6I7Pn9N3SOTvkVh7fdKveKvQwn2bfiEW1wq00WN3m/dhWVTu/A772l4XplpDW/Rw+PExMeMPdJkYmwgsC7M5RK9bl4nZtMas17qo9epp9HurJxvG3r/3qd/lYG//C0tt7vd1Yx89X/y/G3yZ8Z97+Lz9jO7zmFFkVh+MbBa3nHxwx/O3nW6GpYqG4eSaX5v0O25OrZZeYvKhWl99HZubxaJdW3gGPiubkoHnKGdqPWBdrC7qieZ+//XiL5riLXKP+j3e+VCcQ02/J3c+2GmFcqkLdlmMfv2f9vmfs6K2u/qT+vWXi/qVey31W2PzWvmh4lrR1TfvUb/HOx8KXDX9ftp5iMOz5/8Gy8OHcnZ6YVjX7wnsH3d/xN0t1K+/ZNSv3HOp3xob6rcGBNyc1O5SsPqtzyuc1Rib+z0Jekn/6tlJPdfdoH497yT9LnjjWaWz9uOf5Nq7FyoHXq5qc/s04uzTsZQgW1bOR79b+/T5wyLkturXzh9uhdwe76Z+/cVJ0i/v+612bOq3SuNa6/ep39NpBHvTVrksBHCNN575HP/tDCG7t7upX39prqRfeLxsP3oDc5RTBf6sdbG6qCf66K1PyuvTsWgld/7RW2lfI+u35ZkP9gdFpFztcd0gjvr10K+kX3i8zDsfwPeEdgIhVq02t0/K69OxxJhecd84+bljVzzIWlPUrwdC/db6RWWTo98KDLfaJ+X16VhqmK69ebMfG8o4UerXw6N+5T5E/dbY9El5fboLo4bp2psp7+FrH2O9PerXE0m5dOisLD6P6w4NbUUpS3/WulhdFOd+PdJr/9GPDzyvfdbR9rY3fORk9IAUO6lfDylJv/BMLp94puiReSHa+dtYK9rcPo1++/QF6BjTK+7r02XRnjb160lRv3KH4ei3xqZP7/NZ+u+M1c7q7jfv8R8C1K/vdtSv/O6jfmts+qRf+f1bO+jH37zHfwjIl08uiVzJp3zkDnhHLj6P60Bz7jfS3eRd2gmEWA3a3D7p1z4KLHYqkX2j1+3y38uhYPTlfp3h7evL/tbPYXn5+nK31tp9bhk1PFD+82sW6gnJfXi9x38IyJKVSyKsn1K/nPut9oR+/NtPq9DqkYd1bW6f9DuYtP0GZXF2v/a3gt0yL34UZ+c/prJPbByXT1uwzwxzPyBqX4pluI0b+M9HHOoJ6Prw+mUiv5XRhwNrOAZZsnJJpDrqNwLlbBc/ejtDcukdWoXG2tXm9kq/ut+de7cPtJ6vVjtjht6/hX7XVrTvBxJ7+8zyQr8L9+hq90j0YeyLZqOxGa5W9inUcTnHuF5pHySsKx1TWzPyMcslkTqp3wiUs13U7xmSS+/QKjTWrja3V/rV9amNmfph7mxSPCxzt3A/sjY1k/JHeuxvVR70u/JsPpf2wf0Rxa59TX9DX0GM4s32QcK62VGeNiwfs1xyWoPfekr9cu632hM4+VClcbX1bSnQhia/jDnM1r6ahXNqmHwofmrYZc7N9KBfNwZ2i/0V+PNniP2aiXfysn93XPTpJ/gKgu1/ypKVSyK1PqV+Ofdb7QnUb5XG1dbfNY8ZnxlzmOP9/Ji6KQX786n2T/szrpvD7MPn4uPnZPRri3/MxGZ9lj9p+2nFGx7ruB/17tO3lzt85o4sWbkk0rWo3wiUs126fyiepqE5ytvEfCO6WF0Uv/V2et2utfVSDmwbWvwrftH9GBFGv9sweP4226DfMPodjPwP+JSfxfm5YT/V/BKbFD5WfqM16lcFPuUeWywHi3YHjWe4f669qs73GIRn4K0oZekPSheri6J+j9f5qmvr2m9HRBv/sR+k/bz7aQNfXsz92tHvyI9w3TTwKOh3Vdbgf5Zidfi5n/F4YksW5u/bPp938VPeMVFG33qF+lVdgRTVYTlYtDtoPMOKkXO/1evNyYcqjeutv5pNuwr31r922f18FQdWjn4HH3724XOxs3MNxZ0P5eh3sDmb/N3b+WAzWVsB79obvR4D3xL1qwKeojosB4t2B41n4OPSlAw8RzlW9VdKF6uL4uhX1fkvH7QfmvnX6Lgch7gnbX39+B+VWMz9nG2482E7KGYflnYIHfS7KvM2Zw+xsVMdZmVbsDeyuVsnerVQv6rLkaI6LAeLdgeNZ+BiTMnAc5Sy9FdKF6uLon5Vnb+DoNnCj2zDH//EJv6+V/Z2Xj9sPY5+38zEJkzNX6nf4+h3eDb6tfotxtpLnyY2dYsC6ldFPUV1WA4W7Q4az8DFmJKB5yhl6a+ULlYXRf2qOn8XQV/u6xPGTIolfI8i2tJ+a/xUcXnng7sJ4n3w4m6fOBv92pkG++nGajM8/LdZDdzkg6/Yfph3mMeItnOLndSvinqK6rAcLNodNJ5hxci53+r15txvlcY11z/tl4dXjTLczmfhgOZ+hvc4+vWzD27uodRvOfq1N/7aR0Kc3vlQ/k7bwpRVhqpv/Er9qi5AiuqwHCzaHTSegY9LUzLwHOVY1V8pXawuiqNfVefvIsh+d6JFhYVtfdtbM7WvFf3+2Xsf1u5bGWej37n/asXL25/9z/9hg+zNEr4aOwvRt7vPqF9V50pRHZaDRbuDxjNwMaZk4DlKWforpYvVRVG/qs7fQdC/9nkAe7dCkOWH92dFv9aoxbetg37D6Pfbzz3UDjh82+3VDGslN9+kflWXIEV1WA4W7Q4az8DFmJKB5yhl6a+ULlYXRf2qOn8HQYVQGyt+GZvxq7slYjQ3i5Nvvdl9SzP2d7AH/a72+/3n3/vK+HFyrd6Xib/l4e/8U7la4PU3n1i/iL6Q2HARsRws2rWBZ1gxcu43XB73yrnfKo3rrf+rPDRSbNXdHbHY7Ib2Ezp/Z0R19Gs/RfPPgQj69Z/juT+msXvY/i3MbvszMWve9yvS1hfIXy2WSyK1I/pCYkNTWA4W7drAM/BxaUoGnqMcq3qwulhdFPUb+uqVX+dnd4fFDuDF3XNml3nxEV1Vv3b2wc83nOh3sZkLt1D4JwcvVr2zr719YxE7817vkyUrl0ROCNEXEhuawnKwaNcGnoGLMSUDz1HK0oPVxeqiqN/QV6/7am8FO/5iRWPTo9/3WWw825gUK3x7/71IPbG6M/ZRvyp4KarDcrBod9B4Bi7GlAw8RylLf6V0sboo6lfV+S8e9N6/Jz9e/Bx1FVK/Kk4pqsNysGh30HiGFSPnfqvXm3O/VRpXWw+PLLtag71tiPpVXZoU1WE5WLQ7aDwDH5emZOA5yrGqv1K6WF1Ub0a/6+PjD7C1t+KW2PY+3Ktfu5jCz91rP8H7jKB+VdctRXVYDhbtDhrPwMWYkoHnKGXpr5QuVhfVG/3a3yxLXtw3EtqXXul32Lsv/7YD7CaC+lVxTVEdloNFu4PGM3AxpmTgOUpZ+iuli9VF9Ue/h2cf4C+L2I2ukS7dK/0a08ePwSLUOt9F/aoQp6gOy8Gi3UHjGVaMnPutXm/O/VZpXGvdfmX4Wk31vR3qV3WFUlSH5WDR7qDxDHxcmpKB5yjHqv5K6WJ1Ub0Z/eomEGI99R5/6XhmNrFTecZ91K/qqqeoDsvBot1B4xm4GFMy8BylLP2V0sXqoqhfVee/dNA/zS9tXrrRftZH/aquS4rqsBws2h30NX6F7TqtrM2v6iq4IF0s9VsD2qe539fiG2u1I3zKTepXddlxOaKjU7yFJP32cu7X/xSt6jq4xw5qVE391nD2Sb99OpYapmtvUr8q4kmqg+5uxFvAM9wPtMi/6xIHgWfgreiUWhyfLlY3RubkQ/yad7yX+i0BU78liqaVFNVhOVi0O1Y8AxdjSgaeo1NqcX10sboo6repz3dWRv2WaKnfEkXTSorqsBws2h0rnoGLMSUDz1HK0l8gXawuivpt6vOdlVG/JVrqt0TRtJKiOiwHi3bHit/Di4sxJQPPUcrSXyBdrC6K+m3q852VUb8lWuq3RNG0gssRHZ3iLeAZuBhTMvAcpSz9BdLF6qKo36Y+31kZ9VuipX5LFE0rKSNNLAeLdsdK/cpXjPqtsemT8vp0LDVM196kflXEU1SH5WDR7qDxDHxcmpKB5yhl6a+ULlYXxdGvqvNfOoj6LYlSvyWKphVJdfsXeZmbpVx4VqKLfqs8j9D+Ykt1s1Iir67NUi6MlMxmO7OdoQua82GWhybeW5eN2bbGvL/rot6XygfWNHWN/DLtF4djLWlz+6S8Ph1LjOkV9z2afpOf28fEJyQwpX6v6JrQFPUbSDzcb71d2yENjwi0P8baUFov0kWPh8dlYSbHDeXawlRraE/6+FiYzQe6oDnH+HXrMjEfrTHrtS5qvaJ+SxNcb4X6LVk/2uh3XJ5Z6wryNV9p8qGpESwHi3bt4hn4rGxKBp6jnKn1sHWxuijO/Tb1387KqN8SLfVbomhaSVEdloNFu2PFM3AxpmTgOUpZ+guki9VFUb9Nfb6zMuq3REv9liiaVlJUh+Vg0e5Y8QxcjCkZeI5Slv4C6WJ1UdRvU5/vrIz6LdFSvyWKphX8rlxUj7hMU44Jf4AOnkH9NvWkSJn27oVI6kCb2yfl9elYYkyvuI/6VcHG5XgF/UJPVCtOE5cpntGtfnXPMtNFPc/od98n5fXpWFTv/u6CqF8V2yT9Qs/WxceyKceEyxTP6Fa/umkFXdTz6HewNT+qjn6NIOq3pEz9liiaVlJUh+Vg0e5Y8QxcjCkZeI5Slv4C6WJ1UU+k3x+zberfVy2jfkvc1G+JomkFH5uiesRlmnJM+FgWz6B+m3pSpEw7fxtJVc/9IndZxtq55D7qt6RJ/ZYomlZwOV5Bv08596sb1+qinmj0m/K3aNMbIqeM+i3pUb8liqaVJP1y7rcJaaVMKUufoYvVRT2RfjdmVgF+21Xqt+RP/ZYomlaS9AuNTvEW8Ax8WiAlA89RytJfIF2sLuqJ9GvMS1P/vmoZ9Vvipn5LFE0rKfOsmB6xaHesKceE/xsUz6B+m3pSpOwKc79fBvg2fuQQL7qL+i1xUr8liqYVXI6c+23ieVqmHKv6JF2sLup5Rr9LMz9Ffsst6rekT/2WKJpWkvTLud8mpJUypSx9hi5WF/U8+rVPk67wvvEq9VteAOq3RNG0kqRfzv02Ia2UKWXpM3Sxuqin0e9+Yr4qvG+8Sv2WF4D6LVE0raTMs2LKxqLdsaYcEz6Ti2dw7repJ0XKup/7fTUfkXZvtYv6LclTvyWKphVcjpz7beJ5WqYcq/okXawu6mlGv7s+zT0MqN+y+1O/JYqmlST9cu63CWmlTClLn6GL1UU9i35fzWRfwX3rVeq3vALUb4miaSVJv5z7bUJaKVPK0mfoYnVRT6Lf2aRXg1+Ofo9dn/o9smhYS5lnxZSNRbtDTTkmfCYXz+Dcb0NHihV1PPf7O+nTXWcWAEe/ZS+gfksUTSu4HDn328TztEw5VvVJulhd1KOOfj9H5TJ7nRoz7dPUA/Vb6fxPrF9EqUhsoIuNTrFo10bKMeFjWTyDo9/QA5SvFx/9Lk9+73ux7Zd9Ofo99gvq98iiYS1FdVgOFu0OFc/AxZiSgecox6r++uhidVGPOvpdTsrlY7r9a+jXNyni5EOJnfotUTSt4GNTVI+4TFOOCR/L4hnUb1NPipRdfPQbaaNPu6jf8mpQvyWKphVcjlfQL3RnRXF2uEzxDOq3qSdFyl7NupyrBVfetmYaqbHfu6jf8vpQvyWKppUk/fK+3yaklTLlVIHP0MXqonoz+TA5mazFNqjfSk+6t1XqV3XFkvQLjU7xFvAMfFyakoHnKGX5wPot52rRlQVHv6o3cE+Dnlm/wOg0ZZ4V0yMW7XpTyjHhUwl4BvULvtc59wsCe5zwZ9YvMDrF5ci5X/2b5OlHv+kTCDnq1l+gy0Zy7rfkSf2WKJpWkvQLjK5TxrIpx4SPZfEMjn6belKkLEehObmRQ7nKLuq3xLw3ply/l5U38ZdT5JLIuSH6QmJDU1gOFu3awDNwMaZk4Dkc/YY+A79SvzCyXiUY07PvxLTTkSUrl0RqReZOkdjQFKZHLNq1kXJM+FgWz6B+Qw9QvuYoNCdXeXgXD+Po94h0Ynr3rZjjwcXXZMnKJZGaEOEhsaEpLAeLdm3gGbgYUzLwHI5+Q5+BX6lfGFmvEjZm1qvjURyMLFm5JFItoi8kNjSFjU6xaNdGyjHhY1k8g/oNPUD5mqPQnFzl4V08jKPfI9Kp+T5u3MeaLFm5JHJmiL6Q2NAUloNFuzbwDFyMKRl4Dke/oc/Ar9QvjKxXCat+PYpZw0aWrFwSqRcZbyKxoSlMj1i0a+Ma49LrtLIBfghSF6sU+r9efGkhR6E5uaGjXvuVo98j8a1ZHTfuY02WrFwSOTNEeEhsaArLwaJdG9RvIH3+Sv2eM+nNHur3eCn+mfVx4z7WZMnKJZEzQ4TXverwFvCMwUCppQotPANvRTeiLQ5KF6uL6s0zH/i1i0qPe6pV+72Le7vzTJasXBK5ptRvBMrZLur3DMmld+RMIOTkXvo8tPVx9FshNTS/la17WJUlK5dEzguZz00ZaWI5WLQ7HTwDH5emZOA5yrGqv4a6WF0UR7+Rt0X3u6jfCuOV+als3cOqLFm5JHJeHP1GoJztus7oV3/zo+5GSer37EL2Zwf1W7kWMzO5s9kHWbJySeWEwyr1G0g0vV5Hv19Nh3BSphOrLoqj3xO019qgfqukN/d265ksWbmkesKHdeo3AuVs173qVzWe5o1nZ9f7Cjuo3yrkVzN8qW73fl2WrFwSOSnO/UagnO26V/2qxtPU79n1vsIO6rcKeb8z87uafpAlK5dUT/iwztFvBMrZLur3DMmld+TcvZCTe+nz0NZH/Z6Q+lqY+T2Nf2XJyiUnJ1xsUL8RKGe7qN/PMya6Heq3VI5Cc3J1p3H5KOr3lOnvwox/fis/snpa3LctWbJySeQcqN8IlLNd19GvaqbWH9vV73x4+VidQdHs+J3804TZmByF5uQqD+/iYdRvDelod/rrqrXinm3+DXfCEWH6BX6NQrrHdv8iL2vzKheeleii3yp/R67NsrKlW/0w37rAQ9Rs9mGWM3RBc4bm+9DEe+syNMvWmPd3XdT7sv2ZD38fJulr+XZIY5T+zVFoTq7wJup8N/V7hvh3tRsef2X1rPhOdryd/i3CLRJoJDBt1e/fxlWwgvu/s6/WvzkKzcmFT+pCCdTvhUD2rZrr6/f4V9bZmjGLs33yDl30eHhc7IzRcUO5ZsCcj4+F2XygC5pzjF+3Lguza41Zr3VR61Wbft3Y1y0rsKcX9lX6N0ehObngOV0snPq9GMp+VYRNPgC/dCxNPjSdPpaDRbt28Qz868ApGXiO8ksSHrYuVhfV/rWLt2GhX9C/wb66Z2nnKDQnt6nzdllG/XZJ94Z1U78t8PEP0vCMR9LvIMm/mH350VtLp2XxnRCgflsuFC5TPOOh9Dv4O4x/F6sWtMdi0L7U7xEd1+6ZAPXbcvVwmeIZj6VffPyL2pf6bem0LL4TAtRvy4XCZYpnpOj38vf9qmpUfekYHP/C9qV+Wzoti++EAKbfC9z328QF+2gMi3bt4hm4GFMy8BzlB2Ueti5WF9X+0ZtvEpr/xe1L/XrK/OPuCWD65Z0Pigt+ndGv6gE5/mh1YtVFKfWLzP8m2Jf6VXRDhtwBAeq35SLhMsUzHm70Oxi8+S9fKO7/TbEv9dvSaVl8JwSo35YLhcsUz3hA/Zafv7Xc/5BkX+q3pdOy+E4IYPrl3K/islK/HpJq/JtmX+pX0Q0ZcgcEMP1y7ldxSa+jX9V9Cv5otU88U9WouvOhgBQ+f2sY/ybal/pVdEOG3AEB6rflIuEyxTMecfLBYg33n5mffRxyqn2p3zhP7r03AtRvyxXDZYpnPKh+j/5dRSEn25f6jfLkzrsjAOr3W32CKffYYjlYtDtwPAMXY0oGnqO8TcxfLV2sLkp749mhm5Tzv7Hxb7p9qd8DX77cOQFIv4i+kNjAEMvBol0beAYuxpQMPEcpSw9WF6uLAvVb3v8Qef5Zhn2p3/CO4et9E6B+W64fPpWAZzyufo/zD/Xxb459qd+WTsviOyFA/bZcKFymeEaKflX3Kfhzu9WdDwXY4+dvJ6Cz7Ev9nrDkRm8JLJfLwxv1e7mMzNxSvy2XDpcpnpGi335/6bgCtfTvB8UOEQAAIABJREFUqrIzz77UbwUlV3tMwH7tc1sc3tqY9fmBUr/nTE724DLFMx5av5X5hxJspn2p35IkV3pNgPqtXB5+9FaBUVvt5qM338jZ+DfXvtRv7eJxs6cEqN/KhaF+KzBqqx3qtz7+zbYv9Vu7eNzsKQHqt3JhqN8KjNpql/odFD8+b/viyraab1/qt3bxuNlTAtRv5cI8kn7v5c6HAv/x+xeXsC/1W+nUXO0xAeq3cnEeSb93c+dDwb/073xhu6RbIrfhVC5V82rOj8Xn5DYfVXel/KH57th2WrPt55u5X8a884H6lftap5MPttnSv969efa1o9/1KHF525qpTKGnJdRvTy9M22EdOnvxwhvP/rXxOivHbyPDMx77xrMD0vL+B98Xc8a+A6vfyUnHxjao37Nezh3dEDjpmNQv9St1s65Hv/b5k+H3h2yfzLOv02/ysuDoV+oD3H9pArarr37ff+3/H5x84OSD3L261+/x+Tu59uVHb/J1ZEmfCFj98ltv4YI8kn7v684HfwXKO87MT7giia85H5/l5CYebnYa536zEd6mAuq3wv2R9Htndz7Yq/Bbma/N9G+OQnNyK33pqqvU71VxX64x6rfCkvqtwKitdj75cBz72j6ZOf7NUWhObo3Z1Tap36uhvmxD1G+F59q8V7Z0q/h9DHjGM9z5cGrf4vtvuisQicpRaE5u5FCusov6vQrmyzdC/VaYXkOMuErdAaJHNjZvlfNqXtXFdjz6Le27DPc/5Mw/5Cg0J7cZdHel1G93bDut+Wb6TRlpYjlYtKOMSi4t5xqt6JRa9CxdbLf6Le37XX/+TnGM4J85Cs3JBQ/zYuHbzMmaix0IK8II7Nbr1yLjZ72LDDg6e95v9xLCW8AzBgOllipXBc/AW9EptTgoXawuCv2ttwOWin0r9/9GumMFY9NqjkJzcpuOqcsy6rdLujesm/ptgY/LFM94dP2e2PcS/s1RaE5uS1/prPgn3D3aWQus+CYEqN8W7LhM8YwH12/Nvhfwb45Cc3Jb+kpnxdRvZ2hvWzH128Iflyme8dj6PbNvvn9zFJqT29JXOiuem8MMYmctsOKbEKB+W7DjMsUzHlq/EftWnn+WNv+bo9Cc3Ja+0lkx/ilzZ4fCii9JgPptoYnLFM94ZP1G7Zs7/s1RaE5uS1/prHhoRp3VzYpvSID6bYG/MfqnKxRV4RlOv1gryvsU/AHpYnVR8J0Pgn2tf4f2nki3pIx/cxSak9vSV7oq/jRm31XdrPeWBKjfFvr4WBbPeNzRr2jfvPFvjkJzclv6SlfFM7PpqmrWe1MC1G8LflymeMbD6rfBvln+zVFoTm5LX+mqeGnmXVXNem9KgPptwY/LFM94VP022jfHvzkKzclt6StdFU9540NXaG9cL/XbcgFwmeIZD6rfFvtm+DdHoTm5LX2lo2I79fvXUdWs9rYEqN8W/rhM8YzH1G+rfdP9m6PQnNyWvtJR8dJEfiWso7ZY7VUJUL8tuNF7EpxKsbsY3AGgOcr7FPy56WJ1UcCdDwr7Jvs3R6E5uS19pZvi/YZzD92QvX2t1G/LNcDHsnjGI45+VfZN9W+OQnNyW/pKN8VbM+RtZ92gvXmt1G/LJcBlimc8oH6V9k30b45Cc3Jb+konxZYk/vvcnRwJK704AavfF/WyNq/q2J35F439G8nLh/mWC89KdNFfs+PyYZbHDeXa0LwqI4uw9/ehWb6jC5ozMa+HJr5bl4lZtsZ8f+uivre6n2lX2zfNvzkKzcm9+NuvvUJ7YVbtUYy4TwJvxVeP+CcJqAhMVfoF7Jvk3xyF5uRe5T3+V/nrfrk2vOf3KtRv04jV70S9GLPIjh0P5cWYptJ6ni5683FcFqa6ddzftIbmrNcLY59xDy5ozjF+3roYM22Nmc91UfOVRr+QfVP8m6PQnNyrvEeXJ38PTrZXaZSN3IQA535bsOMzuXhGt3O/Q9XvwumiVHc+gPZN8G+OQnNyW/rKZYpfy3HG5mO+/LxMpayllwSo35bLgssUz+hWv7pbynRRGv3C9sX9m6PQnNyWvsJiEsAIUL8tvHCZ4hkPpd8E+8L+zVFoTm5LX2ExCWAEqN8WXrhM8YxH0u/X4jB1+d0C9rT4L/z+vGqqM0ehObmnx8wtEsgkQP22AES/j+ZU+szfetvbD/Hcgtm3HP+Ov1quhy/OUWhOrubYGEMCNQL70VttT9iE9Iv8UDsSGw4Gy8GiXRt4Bj4uTcnAc5QztR6sLlYXpZj7LfyL2vfg37HuVx1yFJqTGzoqX0kAICBLVi6JVI/oC4kNTWE5WLRrA8/AxZiSgecoZenB6mJ1UQr9DgZ2/Ivb1/tXad9BjkJzckNH5SsJAARkycolkeoRfSGxoSksB4t2beAZuBhTMvAcpSw9WF2sLkql38Eqxb7Wv1Pd2HdA/YZ3DF/vgYAsWbkkcl6IvpDY0BSWg0W7NvAMXIwpGXiOUpYerC5WF6XTb7ienb3mjGBzcjs7IVb8yARkycolER6IvpDY0BSWg0W7NvAMXIwpGXiOUpYerC5WF0X9hr7KVxLQEpAlK5dE6kb0hcSGprAcLNq1gWc4MaL3MeAZeCtKWXqwulhdFPUb+ipfSUBLQJasXBKpG9EXEhuawnKwaNcGnoGPS1My8BylLD1YXawuivoNfZWvJKAlIEtWLonUjegLiQ1NYTlYtGsDz8DFmJKB5yhl6cHqYnVR1G/oq3wlAS0BWbJySaRuRF9IbGgKy8GiXRt4Bi7GlAw8RylLD1YXq4uifkNf5SsJaAnIkpVLInUj+kJiQ1NYDhbt2sAzcDGmZOA5Sll6sLpYXRT1G/oqX0lAS0CWrFwSqRvRFxIbmsJysGjXBp6BizElA89RytKD1cXqoqjf0Ff5SgJaArJk5ZJI3Yi+kNjQFJaDRbs28AwnRt75EK6Pff2nedx6Jb6b1VezrvwkBLaq/Lmkbg6ctT4jAVmyckmEE6IvJDY0heVg0a4NPAMfl6Zk4DnKsaoHq4vVRfVGv5PiwT5Jf05Df+MrCVyDgCxZuSRyXIi+kNjQFJaDRbs28AxcjCkZeI5Slh6sLlYX1R/9qn/1qhJofyprsVj0Yvweujxfn4CALFm5JIIF0RcSG5rCcrBo1waegYsxJQPPUcrSg9XF6qJ6o9/0ESy/dBzebny9EgFZsnJJ5NAQfSGxoSksB4t2beAZuBhTMvAcpSw9WF2sLor6DX2VrySgJSBLVi6J1I3oC4kNTWE5WLRrA8/AxZiSgecoZenB6mJ1UdRv6Kt8JQEtAVmyckmkbkRfSGxoCsvBol0beIYTI+98CNfHvvblzgdOPlQuClf7TUCWrFwSOSNEX0hsaArLwaJdG3gGPi5NycBzlGNVD1YXq4uifkNf5SsJaAnIkpVLInUj+kJiQ1NYDhbt2sAzcDGmZOA5Sll6sLpYXRT1G/oqX0lAS0CWrFwSqRvRFxIbmsJysGjXBp6BizElA89RytKD1cXqoqjf0Ff5SgJaArJk5ZJI3Yi+kNjQFJaDRbs2PuB5XFyMKRl4zsT8BWitr7pY6rcVJANIIImALFm5JNIQIjwkNjSF5WDRro2NUf2MeTgc/4rn4Bn4kU3M58lRNm3oYqnfJoYsI4F0ArJk5ZJIa4jwUkaaWA4W7U4nRYxDo/39xwAMzxgM0JyFeQnNtb7qYnVjZE4+tOJmAAnUCMiSlUtqVbhNRL8pqsNysGh3/HjGYKAcFbrqDwuegbeiU2pxQLpY3RiZ+g1Xma8koCUgS1YuidR9//pF7+HFxZiSgefolFpcQl2sLor6jbwtuIsEGgnIkpVLIhXev37xuV98LItnUL+Rzta0K+e5DTm5TcfEMhIQCMiSlUsiVWH6xUea2DfMsGh3Opx8iFzUwy6OfmU2LCGBHAKyZOWSSHuYfvGRJqZHLNqdDp6Bj0tTMvAcpSz9NdTF6qI4+RB5W3AXCTQSkCUrl0QqpH4jUM52cfLhDMmld+RMIOTkXvo8WN9TEJAlK5dEwFC/EShnu6jfMySX3pGj0JzcS58H63sKArJk5ZIImPvXLz4fjcsUz+DkQ6SzNe3KUWhObtMxsYwEBAKyZOWSSFX3r198PhqXKZ5B/UY6W9OuHIXm5DYdE8tIQCAgS1YuiVSF6RcfaWL3MmDR7nT40Vvkoh528aM3mQ1LSCCHgCxZuSTSHqZffKSJ6RGLdqeDZ+Dj0pQMPEcpS38NdbG6KN75EHlbcBcJNBKQJSuXRCqkfiNQznZx8uEMyaV35Ewg5ORe+jxY31MQkCUrl0TAUL8RKGe7qN8zJJfekaPQnNxLnwfrewoCsmTlkgiY+9cvPh+NyxTP4ORDpLM17cpRaE5u0zGxjAQEArJk5ZJIVfevX3w+GpcpnkH9Rjpb064chebkNh0Ty0hAICBLVi6JVIXpFx9pYvcyYNHudPjRW+SiHnbxozeZDUtIIIeALFm5JNIepl98pInpEYt2p4Nn4OPSlAw8RylLfw11sboo3vkQeVtwFwk0EpAlK5dEKqR+I1DOdnHy4QzJpXfkTCDk5F76PFjfUxCQJWtLXtTLzvxTx27MbzT2byQvG/NPLjwr0UV/zY7L0LweN5RrY/NPGVmEvb+Pzes7ukzAnIX5PjTx3boszGtrzPe3Lup7a6Y9eMfkKDQntwenzkO4PwJN+jVcSEBNYEr93t/bn0d8WwKN+p2oF2MW2bHjobwY01Raz9NFbz6Oy8JUt477m9YMmLNeG7Nbo8sCzDEmtDBvXYyZtsbM57qo+Yr6ve1bma3fH4Em/Y71p4N8dIXEhiPAcrBo1waegX8o5n61+C2ckPoVnS+uf1C29y3F/yxiXVm8vNjvo4qApqP+R/024WEZCZwToH49k+voF1WpOzQ0p67f80t+3KOL1UXxzocjV66RgI4A9es5Ub9yd6F+ZTYsIYEcAtSvp0f9yp2I+pXZsIQEcghcTL8z9VHg30lzc7P6+tFod+BY/cWpotMC+ESCawdtRSlLfwq6WF0UJx+KXsE/SUBP4GL61X+TLWWkieVg0Q4WnoGLMSUDz1HK0vcQXawuivrVv+kYSQIFAerXc6B+5TcE9SuzYQkJ5BCgfj096lfuRNSvzIYlJJBDgPr19KhfuRNRvzIblpBADgHq19OjfuVORP3KbFhCAjkELqZf/Z0JKXcZYDlYtMOHZ+AfiqVk4DlKWfo+o4vVRfGjt5y3IXOfk8DF9Ms7H9o7EHoTmasRzVHK0h+sLlYXRf22X39GkMApAerX8+Dkw2m3qG5Rv1UaXCeByxGgfj1L6lfuUtSvzIYlJJBDgPr19KhfuRPdmX7X0DPwq8H9eGC8fCFY8nAEqF9/SalfuWffmX4n6gfEnwf24fc65AvBkocjcDH98s6H9r6BfozmakRzlLL0B6uL1UX15qM35Ln8p8/pn/TiicXt3YgRD0PgYvrlnQ/tfQJVqasRzVHK0h+sLlYX1Rv9po9g+Vtv7V2YERclQP16nJx8kHsV9SuzYQkJ5BCgfj096lfuRNSvzIYlJJBDgPr19KhfuRNRvzIblpBADgHq19OjfuVORP3KbFhCAjkELqZf3vnQfhnQj9FcjWiOUpb+YHWxuih+9NZ+/RlBAqcELqbf+77z4Ro/AY+r1F0qVL/G7E+vcMOWLpb6bUDIIhLIIED9enio5FwSnoNn4K0Yo+8NuljqV0+UkSSAELiBfodmhByhj8VysGjXQIoYJ+YFPA88YzBAc3RKLQ5cF6sbI3PyAewMDCeBwQ30m6I6LAeLdr0AzxgMlKPCSifDM/BWdEotDkoXq4uifisXmqskoCJA/XpM1K/cW6hfmQ1LSCCHwMX0q7/zIUV1WA4W7fDhGfi4NCUDz1HK0vcZXawuiqPfnLchc5+TwMX0q7/zIUV1WA4W7S48noGLMSUDz1HK0vd2Xawuivp9ToHwrHMIUL+eHvUrdyLqV2bDEhLIIUD9enrUr9yJqF+ZDUtIIIcA9evpUb9yJ6J+ZTYsIYEcAtSvp0f9yp2I+pXZsIQEcghcTL+886H9MvC+33ZGmRE5j0zPyc08bKY/J4GL6Zd3PrR3IOq3nVFmRI5Cc3IzD5vpz0mA+vXXnZMPcvfn5IPMhiUkkEOA+vX0qF+5E1G/MhuWkEAOAerX06N+5U5E/cpsWEICOQSoX0+P+pU7EfUrs2EJCeQQuJh+eedD+2XgR2/tjDIjcj4+y8nNPGymPyeBC+kX+bWIlJEmloNFuwuPZ+BPY0jJwHOUY1Xf23Wxuig+8+E5BcKzziFwIf0i+kJiw6lhOVi0awPPwMWYkoHnKGXpwepidVHUb+irfCUBLQHq15OifuUOQ/3KbFhCAjkEqF9Pj/qVOxH1K7NhCQnkEGjU74t6GZuv7Ni/kbyMza9ceFaii/6aHZex+XfcUK4tzK8ysgh7f1+Y73d0QXOMCS18ty7GtIbYAF3U99ZMczrihXJzPj7Lyb3Q4bOa5yLQpF/DhQTUBKbU73Opg2ebT6BRvxP1YsxiMZlo/p9MpNjxUF6MaSqt5+miNx/HxZjq1nF/0xqas14bs4YXNOcYP29djGkNsQG6qPmK+s1/O7KG5yLQpN+xHgUyd4rEhiPAcrBo1waegd+TkJKB5yhnaj1YXawuinc+hL7KVxLQEqB+PSnqV+4w1K/MhiUkkEOA+vX0qF+5E1G/MhuWkEAOAerX06N+5U5E/cpsWEICOQSoX0+P+pU7EfUrs2EJCeQQoH49PepX7kTUr8yGJSSQQ4D69fSoX7kTUb8yG5aQQA4B6tfTo37lTkT9ymxYQgI5BKhfT4/6lTsR9SuzYQkJ5BCgfj096lfuRNSvzIYlJJBDgPr19KhfuRNRvzIblpBADgHq19OjfuVORP3KbFhCAjkEqF9Pj/qVO9Gd6XcNPQW0GtyPR2bKF4IlD0eA+vWXlPqVe/ad6dc+US956cMTi+ULwZKHI0D9+ktK/co9+870izyZ9PRJpZNePDJTvhAseTgC1K+/pNSv3LPvTL/pI1j+2oXcCVjSCQHq12OlfuXeRf3KbFhCAjkEqF9Pj/qVOxH1K7NhCQnkEKB+PT3qV+5E1K/MhiUkkEOA+vX0qF+5E1G/MhuWkEAOAerX06N+5U5E/cpsWEICOQSoX0+P+pU7EfUrs2EJCeQQoH49vYV5gSniOXgG/EvHL2ahPhFlLPWrJspAEoAIUL8eV5oYPyHUTqVoBpyjVKo/cGUs9QteZoaTgJLADfSbpjpkdIq3gGcMBkotVS4EngG3olSqPyhdrC5qMPjXi++M5Xx1Iie3cpW5SgJaAtSvJ0X9ih2G+hXRsIAE8ghQv54f9St2I+pXRMMCEsgjQP16ftSv2I2oXxENC0ggjwD16/lRv2I3on5FNCwggTwC1K/nR/2K3Yj6FdGwgATyCFC/nh/1K3Yj6ldEwwISyCNA/Xp+1K/YjahfEQ0LSCCPAPXr+VG/YjeifkU0LCCBPALUr+dH/YrdiPoV0bCABPIIUL+eH/UrdiPqV0TDAhLII0D9en7Ur9iNqF8RDQtIII/AhfQ7MX/q40hRHZaDRbsDxzPgpzHYVvjMB3UnSQ3MeW5DTm7q8TLvqQlcTL/6Z3mlqA7LwaLd9cczUmRK/Xb+XstRaE5u5yfGBh6RwIX0i+gLiQ3IsRws2rWBZ1C/4docXvnEsxoQbpJAGwHq1xOifsWOwrlfEQ0LSCCPAPXr+VG/YjeifkU0LCCBPALUr+dH/YrdiPoV0bCABPIIUL+eH/UrdiPqV0TDAhLII0D9en7Ur9iNqF8RDQtIII8A9ev5Ub9iN6J+RTQsIIE8Ao36fVEvC/OWHfs3kpeFmcmFZyW66K/ZcVmY3+OGcs0YZeAh7P3dmHd4AXO+zSI08d22vJpFW4gt10V9f2/5U5t5b0VmPx+BJv0aLiSgJjClfp9PHzzjPAKN+p2oF2PUoRMpdjyUF2PksvMSXfTm47gYU9067m9aM6ap9LxsvTZmDS9ozjF+3rZMjWkLseW6qPl8Rf3mvRWZ/XwEmvQ71uNA5k6R2HAEWA4W7drAM+7kW297TzD+53FWN14+GLj9xyhflfgHv/UmomEBCcQJUL+ey8PqN37Vi706seqiBgPqt4k1y0ggQoD69VCo30jfKHZRvyIaFpBAHgHq1/OjfsVuRP2KaFhAAnkEqF/Pj/oVuxH1K6JhAQnkEaB+PT/qV+xG1K+IhgUkkEeA+vX8qF+xG1G/IhoWkEAeAerX86N+xW5E/YpoWEACeQSoX8+P+hW7EfUromEBCeQRoH49P+pX7EbUr4iGBSSQR4D69fyoX7EbUb8iGhaQQB4B6tfzo37FbkT9imhYQAJ5BKhfz4/6FbsR9SuiYQEJ5BGgfj0/6lfsRtSviIYFJJBHgPr1/KhfsRvdm37X2EPwK9H9eGC8eCFY8HgEqF9/TalfsWvfm37t86STl6lIgQUk0AEB6tdDpX7FvnVv+m16av/5k/mreya9eGC8eCFY8HgEqF9/TalfsWvfm37TR7Cv1K/YC1jQCQHq12OlfsXeRf2KaFhAAnkEqF/Pj/oVuxH1K6JhAQnkEaB+Pb+F+YQ5GoOm4BnwL8r9mYn6qHSx1K8aKANJACNA/XpeVxCjbecKrbyZoboD6GKpXzVQBpIARuAm+sVHmtjoFIt2wBLEuIdz8Az7W8NgK/L1PO8XuljdGJk/tXnOl3tIoIWA/BaUSyJVInOnoFB8a1gOFu0awDPUP8B+pKUdRx4z9D/zHnKQq6aL1Y2Rqd9wBfhKAmoC8ltQLolUTv1GoNR33at+x/UTiW7zh+ajWLiTBGQCsmTlkkht1G8ESn0X9VsncvntnHt3c3Ivfyas8QkIyJKVSyJYqN8IlPou6rdO5PLbOQrNyb38mbDGJyAgS1YuiWChfiNQ6ruo3zqRy2/nKDQn9/JnwhqfgIAsWbkkgoX6jUCp76J+60Quv52j0Jzcy58Ja3wCArJk5ZIIFuo3AqW+i/qtE7n8do5Cc3Ivfyas8QkIyJKVSyJYqN8IlPou6rdO5PLbOQrNyb38mbDGJyAgS1YuiWAxZh/ZG9+VcI8teF8u3gKe8UT3/fLGs3hH5l4SyCQgS1YuiTSJ6AuJDU1hOVi0awPPoH7DtTm88r7fGhBukkAbAVmyckmkTkRfSGxoCsvBol0beAb1G67N4ZX6rQHhJgm0EZAlK5dE6kT0hcSGprAcLNq1gWdQv+HaHF6p3xoQbpJAGwFZsnJJpE5EX0hsaArLwaJdG3gG9RuuzeGV+q0B4SYJtBGQJSuXROpE9IXEhqawHCzatYFnUL/h2hxeqd8aEG6SQBsBWbJySaRORF9IbGgKy8GiXRt4BvUbrs3hlfqtAeEmCbQRkCUrl0TqRPSFxIamsBws2rWBZ1C/4docXqnfGhBukkAbAVmytmSkXoxRh46Q2FCrMTNgwaJdxUjGe7F8m8VhTfuCZ7y/t+R815elmdR3idu6WF3U9/e2F78TnPPViZzctvcZy0kgQqBJv4YLCagJTKnfyPuLu0iggUCjfifqxRh16ESKHQ/lxRi57LxEF735OC7GHNe1a2jOem3MGl7AnN0xft62TM2iLcSW66Lm8xX12/A+YxEJRAg06Vf3ZVNfKTJ3isSGI8ZysGjXBp6RNPer/xXicOLocyK0Pw3k6pevfWhdH8UfG6oy4zoJqAjIb0G5JFIxoi8kNjSF5WDRrg08I0m/i3A+6ldcv/q/NHVXWBdF/aovKQNJIBCQ31xyScitvCL6QmJDE1gOFu3awDOo33BtDq+886EGhJsk0EZAlqxcEqkT0RcSG5rCcrBo1waeQf2Ga3N4pX5rQLhJAm0EZMnKJZE6EX0hsaEpLAeLdm3gGdRvuDaHV+q3BoSbJNBGQJasXBKpE9EXEhuawnKwaNcGnkH9hmtzeKV+a0C4SQJtBGTJyiWROhF9IbGhKSwHi3Zt4BnUb7g2h1fqtwaEmyTQRkCWrFwSqRPRFxIbmsJysGjXBp5B/YZrc3ilfmtAuEkCbQRkycolkToRfSGxoSksB4t2beAZ1G+4NodX6rcGhJsk0EZAlqxcEqkT0RcSG5rCcrBo1waeQf2Ga3N4pX5rQLhJAm0EZMnKJZE6EX0hsaEpLAeLdm3gGdRvuDaHV+q3BoSbJNBGQJasXBKpE9EXEhuawnKwaNcGnkH9hmtzeKV+a0C4SQJtBGTJyiWROhF9IbGhKSwHi3Zt4BnUb7g2h1fqtwaEmyTQRkCWrFwSqRPRFxIbmsJysGjXBp5B/YZrc3ilfmtAuEkCbQRkycolkToRfSGxoSksB4t2beAZ1G+4NodX6rcGhJsk0EZAlqxcEqkT0RcSG5rCcrBo1waeQf2Ga3N4pX5rQLhJAm0EZMnKJZE6EX0hsaEpLAeLdm3gGdRvuDaHV+q3BoSbJNBGQJasXBKpE9EXEhuawnKwaNcGnkH9hmtzeKV+a0C4SQJtBGTJyiWROhF9IbGhKSwHi3Zt4BnUb7g2h9e+6Hen/eXTs7ifXvxcUg0rNx+ZgCxZuSTCA9EXEhuawnKwaNcGnkH9hmtzeO2Lfu1vCSYv09o5cZMEOiUgS1YuiRwQoi8kNjSF5WDRrg08g/oN1+bw2hf9jrU/lXoWN+Tot3ZNudkxAVmycknkkBB9IbGhKSwHi3Zt4BnUb7g2h9e+6Dd9BPtK/dauKTc7JiBLVi6JHBKiLyQ2NIXlYNGuDTyD+g3X5vBK/daAcJME2gjIkpVLInUi+kJiQ1NYDhbt2sAzqN9wbQ6v1G8NCDdJoI2ALFm5JFInoi8kNjSF5WDRrg08g/oN1+bwSv3WgHCTBNoIyJKVSyJ1IvpCYkNTWA4WbdvYm0VoSf36AufgGQNY8l9moz4DXay2H1C/avAMJIGCgPzmkksi7BDhIbGhKSwHi7ZtXEOM12lFp9QCqy5W2w+o39BZ+UoCSgLym0suiVQNCC/rNZmcAAARtElEQVRlpInlYNHubFL0+2fGERBNu/CMwQDN0Sm1OEpdrC5qMKB+m649y0ggQkCWrFwSqQbQb4rqsBws2p0NnjEYQHw8MjwDb0UrS3dAulhdFPXrLzH/IAGEgKwEuSRSP/UbgVLfBRE9JKM5Wlm66nWxuijqt361uU0CrQTkt7dcEqmU+o1Aqe+CiB6S0RytLF31ulhdFPVbv9rcJoFWAvLbWy6JVEr9RqDUd0FED8lozgy480EXS/3WryO3SeBCBOS3t1xy3vSe+j2HcrYHIRqS0RytLF39ulhdFEe/4YrxlQTUBOS3t1xyXvmLmZzvFPakfMyF5WDR7jDxDPxDsZQMPEcrS3fWuljdGJn6dUS5kABEQJasXHLeAKIvJDa0hOVg0a4NPAMXY0oGnqNTasFVF6uLon5DX+UrCagJyJK1JfuRcpmZhTJyNGqI/ZoJy69ZCCWx3ero3/DE7W+zCKvq11czUccWgTbjG12WYM7WjMsmXpfNy48ZNwf40p+FJmq5XJmdutd1F5jz1LKc3O7OiDU/MIEm/Q7fkp9bzcQ7ILC48DEOe/A+yVFoTm4PTp2HcH8EmvQ7/htqF2O0kcOhHLs5ewL2YcfGGKkotl8bvVuHxZiwpn7dmYU6tgi0GXN0mYI562r8qnmZm0lzgC/VRa1WU+Cei+7eJzkKzcnt7oxY8wMTaNSv+ryRuVMkNhwAloNFuzbwDHxWNiUDz9HO1Lqz1n2opq2RXzp2TLmQAECA+nWwHke/OqUWHUQnVm2N1C/wtmMoCTgC1K+j8Dj61SnVnbP2xjNtjdRvQZV/koCaAPXrUFG/coehfmU2LCGBLALUr8NH/cqdiPqV2bCEBLIIUL8O33PqVzerS/1mvcGYTAIyAerXsXlO/erEqovit97k9xhLSEAgQP06MI+jX92ItugMOrFqa+RHb8JbjLtJQCJA/Toyj6NfnVKL3qCL1UVx9Cu9w7ifBEQC1K9DQ/2KHUT5XDTqVybIEhIQCFC/Dgz1K3QPu5ujX5kNS0ggiwD16/A9p351s7rUb9YbjMkkIBOgfh2b59SvTqy6KE4+yO8xlpCAQID6dWAeR7+6EW3RGXRi1dbIOx+Etxh3k4BEgPp1ZB5HvzqlFr1BF6uL4uhXeodxPwmIBKhfh4b6FTsIP3qT0bCEBPIIUL+OH/Ur9yKOfmU2LCGBLALUr8P3nPrVzepSv1lvMCaTgEyA+nVsnlO/OrHqojj3K7/HWEICAgHq14F5HP3qRrRFZ9CJVVsj73wQ3mLcTQISAerXkXkc/eqUWvQGXawuiqNf6R3G/SQgEqB+HRrqV+wgvPNBRsMSEsgjQP06ftSv3Is4+pXZsIQEsghQvw7fc+pXN6tL/Wa9wZhMAjIB6texeU796sSqi+Lcr/weYwkJCASoXwfmcfSrG9EWnUEnVm2NvPNBeItxNwlIBKhfR+Zx9KtTatEbdLG6qP6MfnfvqcuPmUpvE+4ngS4IUL+OKvUr96170+/EpC/Ur9wPWNIBAerXQaV+5a51b/odf6QuQ45+5W7Aki4IUL+O6nPqVzere2/6TR/BvlK/XSiGdcoEqF/H5jn1qxOrLqo/c7/Ur/xuZ0nPCFC/7oI8jn51I9qiE+rEqq2Rdz707K3Nw+k/AerXXaPH0a9OqUW/1MXqojj67f97nUfYOwLUr7sk1K/cMalfmQ1LSCCLAPXr8MkUZLh4Dp6BH9mv2cmHXCvRxVK/NWzcJIFLEZCVYEtG2mVmFtrQkRj7NROXX7MQy84LlNG/x9vzX83kuKFcg3O+l2byjS5oztZsDk28LtuWHzNsC7HlP2asiFout724byDn7oWc3Eu9H1nPUxFo0m/67evMfBwCC+WpTKnfpzIHT/YCBBr1O1QvxqhDh1LsRr5dfmOMXHhWoozerctlZxblunYFzpmvzWKOLmjOzkxCE6u2ZWrGbSG2XBe1WvXjK7s5I9ic3Au8FVnF8xFo0u9YjUOu5bwKJDZkYzlYtGsDzxioH0MeziEl45iz99W0/+nnc4uwY8PCmm7uVxfFOx8EyNxNAjIBWTxyyXltXcWGlpD6U2SK1V8clfYjqXAOR5Ue97Svoa1oZela1sXqoqjf9mvJCBKoEZDFI5fUqrCbXcWGlpD6saMpWsDqL3JQMVK/4Wp2+ZozgZCT2+U5se6HJSCLRy45h9FVbGgJqf9a+tV+Gyycw2CAZ+DK1o5V3VHpYnVRHP0erzPXSEBJQBabXHJedVexoSWk/mvpl6PfcHWKV37p+JQHt0iglYAsNrnkvNKuYkNLSP2PpV90xKwdqzqyulhdFEe/oa/ylQTUBGSxySXnlXcVG1pC6n8s/aJjbK0sHVldrC6K+g19la8koCYgi00uOa+8q9jQElI/9Ruotb3qxKqLon7baLOcBM4IyGKTS84q4Z0P50gie9CRrKsCzdHK0tWti9VFUb+OKBcSgAjIkpVLzhvoKja0hNR/rdEvOivLOx/C1ezyNefmsZzcLs+JdT8sAVlscsk5jK5iQ0tI/dfSLzouxUey7uzRVrRjVVe3LlYXxdGvI8qFBCACstjkkvMGuooNLSH1P5Z+0TG2VpaOrC5WF0X9hr7KVxJQE5DFJpecV95VbGgJqf+x9MvRb+gDutecCYScXN3RMYoETgjIYpNLTirwG13FhpaQ+qnfQK3tVTeu1UVx9NtGm+UkcEZAFptcclYJ73w4RxLZg45kXRVojlaWrm5drC6K+nVEuZAARECWrFxy3kBXsaElpP5rjX7RWVne+RCuZpevORMIObldnhPrflgCstjkknMYXcWGlpD6r6VfdFyKj2Td2aOtaMeqrm5drC6Ko19HlAsJqAi8/CvCSrHtv2sP6S5LFPV1FRuaRup/LP2iY2ytLB1ZXawuivoNfZWvJNBOYP3hBXwQ2345/KnlIMrrKjYcElL/Y+mXo9/QB3SvORMIObm6o2MUCRQEZvbn06yAvdisfM3ir0YGUV5XseGQkPqp30Ct7VU3rtVFcfTbRpvlJFAhMLW/YLt7t2Lbvw7tr9nWB7+d3c2AqbQ4YCwHi3Yt4Bn4rGxKBp6jlaU7a12sLor6dUS5kICSwK//AfEPM7E/PWzM4qWehiipq9hwTEj9KTLF6i+OCp2V5Z0P4Wp2+ZozgZCT2+U5se5HJOCGv+VyNviFRoSIvpDYgB3LwaJdG3gGPi5NycBztGNVd9a6WF0UR7+OKBcS0BIohr+FgM8Hv5CSEH0hseFUsBws2rWBZ+BiTMnAR8xaWbqz1sXqoqhfR5QLCagJrMux7/nML6YkRF9IbDgVLAeLdm3gGSkyRe9icEeG5mhl6erWxeqiqF9HlAsJqAkch7+RwS+kJERfSGw4FSwHi3Zt4Bm4GFMy8BytLN1Z62J1UdSvI8qFBPQEyuHv+cwvpiREX0hsOBUsB4t2beAZuBhTMvAcrSzdWetidVHUryPKhQT0BMLwNzb4hZSE6AuJDaeC5WDRrg08A5+VTcmgfkMP0L7m3L2Qk6s9PsaRQEngMPyNDX4hJSH6QmLDgWI5WLRrA8/AxZiSgedox6rurHWxuiiOfh1RLiQAECiGv9HBL6QkRF9IbDgVLAeLdm3gGbgYUzLwEbNWlu6sdbG6KOrXEeVCAggBP/yNDn4hJSH6QmLDqWA5WLRrA89IkSl6F4M7MjRHK0tXty5WF0X9OqJcSAAh8G7vPYsPfiElIfpCYsOpYDlYtGsDz8DFmJKB52hl6c5aF6uLon4dUS4kABHYRZ72UFSAKKmr2HAqSP0pMsXqL44KHZfiKnXtoK1oZenq1sXqoqhfR5QLCUAE3qXBLzQiRPSFxIZTwXKwaNcGnoHPyqZkUL+hB2hfc+5eyMnVHh/jSKBKYB2f+cWUhOgLiQ0HiuVg0a4NPAMXY0oGnqMdq7qz1sXqovoz+t29py4/ZuqwcCGBqxGY1Z/zG1pGlNRVbMqxpMgUOf5wTOi0AK5S1xL6XDWtLF3dulhdVH/0e/wmPb5G/bp+waUHBBAldRUbMCD1P5Z+UclrZenI6mJ1UX3R7/s6Z9mGDsdXErgtAUR5XcUGAkj91G+g1vaqE6suqi/6bTtnlpPAXRBAlNdVbACF1E/9Bmptrzqx6qKo3zbaLCcBgIBV3ki7/HYUG9pH6h+NsGjXBp4xGv0zm3B4ylc8A2/l23woj2Y00sXqokajJT+4At5dDCWBRgJv+CcXzHhqAvzgqvENxUIS0BP4G0/Ui/2pzk5iQ6VI/ZMJFu3awDNScq7RCtKGLlYX5Siu9J2LkSRAApci8DfcqavC5nGLarEcLNq1gBx/OFH0noS0G8/QVmbDeTjA1lddrHbut7U5BpAACdyaAC5H9MO0lBZwKqgYr6Nf/DzaMqjfNkIsJ4G7IZAy0sRysOhUcF9DdK4Tz7DKhltJPR8pTzdGlrK5nwRIgARIgARIgARIgARIgARIgARIgARIgARIgARIgARIgARIgARIgARIgARIgARIgARIgARIgARIgARIgARIgARIgARIgARIgARIgARIgARIgARIgARIgARIgARIgARIgARIgARIgARIgARIgARIgARIgARIgARIgARIgARIgARIgARIgARIgARIgARIgARIgARIgARIgAS6I/C1XC7/XPV/dmW5d2tul19xG1xIgARI4CoEvryJrtJUTxqZGWO+3bEs7YqZubWVMWP3+uTLF/8KevIewNO/KoGv+cdV2+tDY/uFMT/uQOZOv1u39mGM/lfcXcJjLu+bVwr4MS8tz6p/BL6sgP7177C6PqK1MTvXxsTp1/2o8d6+Lrtu9R7q35khBXwPF4rHePcEnHzN8w1+B4OtPW87ynOTEMYs7Nqvff26++t5gRP4Z0FQwBcAySpIoJHAyP/Tu5gEbQx8vEJn25m3sB0HH9Y49euv884CoYAfr8vzjHpFYDS3E6B2ecbBb5hrmNoZB+tfO/lr/ybi1K/vn274axeOgHv1buXBPBSBkf2kv1j8HQAPdW6ak7FjvPnAfQL3Zech7OSvnQTm1G8Bzg9/KWBNL2IMCSQQeCvlazYJ6TdP+Xw5XT7hI/qx4zs34Tt287+L/ZvVjZ2M4GIJHIa/FDB7Awl0QKAiX3v/1ffJ4r6GcLJsa8tPfVnVlnl9ma5ry0dt2Qxry6S2FLMk1gfSgv8d4hzz92PMyg98Z9/WwRmkX94fadkcMXMKIqNXMJUEzgn8tNrs+Pbr6dqi5md/E9n5mTbs+bRn9s/+M/vVT/tu7b8G3O1nqYv7JO9Bl+FvKhTmkQAJnBH4O/Hv+nSwWhvKrlb1wW5tMLzdnoyV7cbr9/fp///qI8NZbfka1ZbTmYWXF3xu4eykz3bYr1m4GZg3/823qRWx//LFWZhux6w2nr/vzXHlbxJ+C0PXAxhFAloCVQH7z/vtja92cX8Wa37zsf8oZr/drMXITjxY4XDq93DBX47/OtrYfxxwIQESuCyBl3IEvHjSLxvY2V67+G8eD92a++4FF0fgx+FwC+XLDkEC3RB4+fFfuH3a213/vGH89639QHjdDeb7q/XvMPilfO/v2vGI74fAy7YQ8JMOf/2Y98VdLj8Qzpn6vZ9rrjjSH//3EuWrQMUQEsggUAj4Sb/t5ca8xR0TfiDMD/iLjuQHv5RvxruKqSSgJPDpRsDPOfz9t1uvD190m9sbkzn1W3QZO/ilfJXvHoaRQCaB/Xb8pMPfTHCPmf6yoHwf88ryrPpJ4HNrb37lQgKOwCtvNWNHIAESIAESIAESIAESIAESIAESIAESIAESIAESIAESIAESIAESIAESIAESIAESIAESIAESIAESIAESIAESIAESIAESIAESIAESIAESIAESIAESIAESIAESIAESIAESIAESIAESIAESIAESIAESIAESIAESIAESIAESIAESIAESIAESIAESIAESIAESIAESIAESIAESIAESIAESIAESIAESIAESIAESIAESIAESIAESIAESIAESIAESIAESIAESIAESIAESIAESIAESIAESIAESIAESIAESIAESIAESIIGuCPwH9j3d4Kg1fY0AAAAASUVORK5CYII="
- }
- },
"cell_type": "markdown",
"metadata": {},
"source": [
- "![MVA-1.png](attachment:MVA-1.png)"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "Based on the above block diagram we find that `PE` & `SIMD` parallelization attributes are subject to the following constraints. \n",
- "If `W` is the width of the input and `H` is the height of the output in a Matrix-Vector Computation then:\n",
+ "In the case of the MVAU, `PE` & `SIMD` are subject to the following constraints: \n",
+ "\n",
+ "If `MW` is the number of input features and `MH` the number of output features:\n",
"\n",
- " W % SIMD == 0\n",
- " H % PE == 0\n",
+ " MW % SIMD == 0\n",
+ " MH % PE == 0\n",
" \n",
- "For the above example, H = 12 and W = 12. The demonstrated PE & SIMD values adhere to the above constraints.\n",
+ "Total folding in the case of the MVAU is defined as:\n",
"\n",
- "We also define a term referred to as total folding which is defined as :\n",
+ " Total folding = (MH/PE) x (MW/SIMD)\n",
"\n",
- " Total folding = (H/PE) x (W/SIMD)\n",
+ "In a streaming dataflow architecture like it is in FINN designs the throughput is determined by the slowest layer. So, the goal of adjusting these parameters is to get an almost balanced pipeline i.e. equalizing the throughput rate of layers in the generated dataflow architecture.\n",
"\n",
- "The goal of adjusting these parameters is to get an almost balanced pipeline i.e. equalling the rate of producers and consumers in the generated dataflow architecture.\n",
- "This can be achieved (or almost achieved) by keeping the `total folding` parameter approximately constant across all layers.\n",
+ "The FINN compiler provides analysis passes to facilitate the exploration of the folding factors of each layer. In this notebook we will show how to use these functions and explore how the parallelization parameters affect the clock cycles and the resource utilization of the generated dataflow architecture.\n",
"\n",
- "We now explore how these parameters affect the estimated clock cycles and the resource utilization of the generated dataflow architectures.\n",
- "We start with a naive case where `PE` & `SIMD` values across all layers are 1 and observe the above-mentioned numbers.\n",
- "We define the utility functions (`exp_cycles_per_layer()`) and (`res_estimation()`) to estimate the number of clock cycles and resource utilization of each network layer."
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "\n",
- "Should this line be added (The `exp_cycles_per_layer` formula is equal to the total folding in this case as the number of input vectors is 1 and the mmv value is also 1).\n",
- "
"
+ "We start with a naive case where `PE` & `SIMD` values across all layers are 1, this is the starting point of our exploration and is also the state the network is in after the conversion to HLS layers. If you take a look at the model using Netron and click on one of the MVAU layers, you can see that `PE` and `SIMD` are both set to 1 by default."
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
- "outputs": [],
- "source": [
- "from finn.analysis.fpgadataflow.exp_cycles_per_layer import exp_cycles_per_layer\n",
- "from finn.analysis.fpgadataflow.res_estimation import res_estimation"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "We now individually extract the `MatrixVectorActivation` blocks from the onnx file and set the config values manually (although this can be done automatically by Vivado tools also as mentioned in the introduction).\n",
- "\n",
- "In the first step, we set the `PE` & `SIMD` values for all the layers to be '1' to establish a baseline and measure the estimated clock cycles and resource utilization for each of the individual layers.\n",
- "\n",
- "We utilize from (`getCustomOp()`) as the helper function to set different properties of the node. The (`set_nodeattr()`) function within this function call helps us set these values."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 4,
- "metadata": {},
- "outputs": [],
- "source": [
- "from qonnx.custom_op.registry import getCustomOp\n",
- "fc_layers = model.get_nodes_by_op_type(\"MatrixVectorActivation\")\n",
- "# (PE, SIMD, in_fifo_depth, out_fifo_depth, ramstyle) for each layer\n",
- "config = [\n",
- " (1, 1, [16], [64], \"block\"),\n",
- " (1, 1, [64], [64], \"auto\"),#8,8\n",
- " (1, 1, [64], [64], \"auto\"),#8,8\n",
- " (1, 1, [64], [1], \"distributed\"),\n",
- "]\n",
- "for fcl, (pe, simd, ififo, ofifo, ramstyle) in zip(fc_layers, config):\n",
- " fcl_inst = getCustomOp(fcl)\n",
- " fcl_inst.set_nodeattr(\"PE\", pe)\n",
- " fcl_inst.set_nodeattr(\"SIMD\", simd)\n",
- " fcl_inst.set_nodeattr(\"inFIFODepths\", ififo)\n",
- " fcl_inst.set_nodeattr(\"outFIFODepths\", ofifo)\n",
- " fcl_inst.set_nodeattr(\"ram_style\", ramstyle)"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "After setting these parameters, we save the model and view it using `Netron`\n",
- ". We can observe the values we set in the above step by clicking on any of the nodes and observing their properties."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 10,
- "metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
- "Stopping http://0.0.0.0:5901\n",
- "Serving './cybsec_PE_SIMD_not_modified.onnx' at http://0.0.0.0:5901\n"
+ "Stopping http://0.0.0.0:5920\n",
+ "Serving 'step_convert_to_hls.onnx' at http://0.0.0.0:5920\n"
]
},
{
@@ -258,7 +178,7 @@
"