{ "cells": [ { "cell_type": "markdown", "metadata": { "id": "acbetMKBt825" }, "source": [ "
\n", "

 Made With ML

\n", "

ML for Developers

\n", " Design ยท Develop ยท Deploy ยท Iterate\n", "
\n", "\n", "
\n", "\n", "
\n", "  \n", "  \n", "  \n", " \n", "
\n", " ๐Ÿ”ฅ  Among the top ML repositories on GitHub\n", "
\n", "\n", "
\n", "
" ] }, { "cell_type": "markdown", "metadata": { "id": "oh-HuNfDrPg0" }, "source": [ "This notebooks contains the code for the ๐Ÿ”ข  Data and ๐Ÿ“ˆ  Modeling lessons. After this proof of concept (PoC), we'll be moving all of this code to Python scripts to serve our application to production. Follow the accompanying [lessons](https://madewithml.com/) along with the code here to develop a deeper understanding of all the concepts." ] }, { "cell_type": "markdown", "metadata": { "id": "XTNsIiUrqoJW" }, "source": [ "
\n", " \n", " \n", "\"Open\n", "
" ] }, { "cell_type": "markdown", "metadata": { "tags": [] }, "source": [ "# ๐Ÿ› ๏ธ Setup" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We'll be using [Ray](https://ray.io) to develop our application using distributed workloads." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [], "source": [ "import os\n", "import ray" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [], "source": [ "import sys; sys.path.append(\"..\")\n", "import warnings; warnings.filterwarnings(\"ignore\")\n", "from dotenv import load_dotenv; load_dotenv()\n", "%load_ext autoreload\n", "%autoreload 2" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "2023-12-07 11:26:30,445\tINFO worker.py:1633 -- Started a local Ray instance. View the dashboard at \u001b[1m\u001b[32m127.0.0.1:8265 \u001b[39m\u001b[22m\n" ] }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "afcfdccd644b41d0b7af7f86f68dbdf3", "version_major": 2, "version_minor": 0 }, "text/html": [ "
\n", "
\n", "
\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", "\n", "
Python version:3.10.11
Ray version:2.7.0
Dashboard:http://127.0.0.1:8265
\n", "\n", "
\n", "
\n" ], "text/plain": [ "RayContext(dashboard_url='127.0.0.1:8265', python_version='3.10.11', ray_version='2.7.0', ray_commit='b4bba4717f5ba04ee25580fe8f88eed63ef0c5dc', protocol_version=None)" ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Initialize Ray\n", "if ray.is_initialized():\n", " ray.shutdown()\n", "ray.init()" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [ { "data": { "text/plain": [ "{'memory': 30507458560.0,\n", " 'CPU': 12.0,\n", " 'node:__internal_head__': 1.0,\n", " 'node:127.0.0.1': 1.0,\n", " 'object_store_memory': 2147483648.0}" ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "ray.cluster_resources()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "These cluster resources only reflect our head node ([m5.2xlarge](https://instances.vantage.sh/aws/ec2/m5.2xlarge)). But recall in our [setup lesson](https://madewithml.com/courses/mlops/setup/) that our [compute configuration](https://madewithml.com/courses/mlops/setup/#compute) that we also added [g4dn.xlarge](https://instances.vantage.sh/aws/ec2/g4dn.xlarge) worker nodes (each has 1 GPU and 4 CPU) to our cluster. But because we set `min_workers=0`, our worker nodes will autoscale ( up to `max_workers`) as they're needed for specific workloads (ex. training). " ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [], "source": [ "# Workers (1 g4dn.xlarge)\n", "num_workers = 1\n", "resources_per_worker={\"CPU\": 3, \"GPU\": 1}" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "If you are running this on a local laptop (no GPU), use the CPU count from `ray.cluster_resources()` to set your resources. For example if your machine has 10 CPUs:\n", "\n", "```python\n", "num_workers = 6 # prefer to do a few less than total available CPU (1 for head node + 1 for background tasks)\n", "resources_per_worker={\"CPU\": 1, \"GPU\": 0}\n", "```" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "/efs/shared_storage/madewithml/GokuMohandas\n" ] } ], "source": [ "# Storage\n", "EFS_DIR = f\"/efs/shared_storage/madewithml/{os.environ['GITHUB_USERNAME']}\"\n", "print (EFS_DIR)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Data" ] }, { "cell_type": "markdown", "metadata": { "tags": [] }, "source": [ "## ๐Ÿ”ข Data ingestion" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [], "source": [ "import pandas as pd" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
idcreated_ontitledescriptiontag
062020-02-20 06:43:18Comparison between YOLO and RCNN on real world...Bringing theory to experiment is cool. We can ...computer-vision
172020-02-20 06:47:21Show, Infer & Tell: Contextual Inference for C...The beauty of the work lies in the way it arch...computer-vision
292020-02-24 16:24:45Awesome Graph ClassificationA collection of important graph embedding, cla...other
3152020-02-28 23:55:26Awesome Monte Carlo Tree SearchA curated list of Monte Carlo tree search pape...other
4252020-03-07 23:04:31AttentionWalkA PyTorch Implementation of \"Watch Your Step: ...other
\n", "
" ], "text/plain": [ " id created_on title \n", "0 6 2020-02-20 06:43:18 Comparison between YOLO and RCNN on real world... \\\n", "1 7 2020-02-20 06:47:21 Show, Infer & Tell: Contextual Inference for C... \n", "2 9 2020-02-24 16:24:45 Awesome Graph Classification \n", "3 15 2020-02-28 23:55:26 Awesome Monte Carlo Tree Search \n", "4 25 2020-03-07 23:04:31 AttentionWalk \n", "\n", " description tag \n", "0 Bringing theory to experiment is cool. We can ... computer-vision \n", "1 The beauty of the work lies in the way it arch... computer-vision \n", "2 A collection of important graph embedding, cla... other \n", "3 A curated list of Monte Carlo tree search pape... other \n", "4 A PyTorch Implementation of \"Watch Your Step: ... other " ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Data ingestion\n", "DATASET_LOC = \"https://raw.githubusercontent.com/GokuMohandas/Made-With-ML/main/datasets/dataset.csv\"\n", "df = pd.read_csv(DATASET_LOC)\n", "df.head()" ] }, { "cell_type": "markdown", "metadata": { "tags": [] }, "source": [ "## โœ‚๏ธ Data splitting" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [], "source": [ "from sklearn.model_selection import train_test_split" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [ { "data": { "text/plain": [ "tag\n", "natural-language-processing 310\n", "computer-vision 285\n", "other 106\n", "mlops 63\n", "Name: count, dtype: int64" ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Value counts\n", "df.tag.value_counts()" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [], "source": [ "# Split dataset\n", "test_size = 0.2\n", "train_df, val_df = train_test_split(df, stratify=df.tag, test_size=test_size, random_state=1234)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [ { "data": { "text/plain": [ "tag\n", "natural-language-processing 248\n", "computer-vision 228\n", "other 85\n", "mlops 50\n", "Name: count, dtype: int64" ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Train value counts\n", "train_df.tag.value_counts()" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [ { "data": { "text/plain": [ "tag\n", "natural-language-processing 248\n", "computer-vision 228\n", "other 84\n", "mlops 52\n", "Name: count, dtype: int64" ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Validation (adjusted) value counts\n", "val_df.tag.value_counts() * int((1-test_size) / test_size)" ] }, { "cell_type": "markdown", "metadata": { "id": "WuCrsbxbNkSV" }, "source": [ "## ๐Ÿ” Exploratory Data Analysis (EDA)" ] }, { "cell_type": "markdown", "metadata": { "id": "eOJ3nlEgnSTJ" }, "source": [ "Exploratory data analysis to understand the signals and nuances of our dataset. It's a cyclical process that can be done at various points of our development process (before/after labeling, preprocessing, etc.) depending on how well the problem is defined." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "tHdQmqTBNkSV", "tags": [] }, "outputs": [], "source": [ "from collections import Counter\n", "import matplotlib.pyplot as plt\n", "import seaborn as sns; sns.set_theme()\n", "import warnings; warnings.filterwarnings(\"ignore\")\n", "from wordcloud import WordCloud, STOPWORDS" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [ { "data": { "text/plain": [ "[('natural-language-processing', 310),\n", " ('computer-vision', 285),\n", " ('other', 106),\n", " ('mlops', 63)]" ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Most common tags\n", "all_tags = Counter(df.tag)\n", "all_tags.most_common()" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "Gl-E8d2HaCsx", "outputId": "22afb969-0335-42ec-b58b-c36ca1db8bf8", "tags": [] }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAA1kAAAEvCAYAAAC35QvGAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAABQkklEQVR4nO3dd1hT598G8DsJhKGggAIuKqIBUVkCalVUqmhbcdtqxW3F/XNVwVkHaBVHlbpaqXu14qx7te5VrSJaN06WgqHKEHLeP3xzagRqiJGA3J/r6lXznPU9yTlJbs5znkgEQRBAREREREREeiE1dAFEREREREQfEoYsIiIiIiIiPWLIIiIiIiIi0iOGLCIiIiIiIj1iyCIiIiIiItIjhiwiIiIiIiI9YsgiIiIiIiLSI4YsIiIiIiIiPWLIIiIiegtBEAxdgtaKU61ERB8qhiwiohJs4cKFcHZ2LtB/Dx48MHTZucTFxcHZ2Rn+/v4a7eqas7OzdVqvUqnE9OnTsXXrVq2XefDgAZydneHn5/efbe/DsWPH0KdPn7fWQ0RE75eRoQsgIiLDcXZ2RmBgoEbbkydPcOLECZibm+OTTz7JtYy5uXlhlWdwM2bMQHR0NKZPn27oUt7q0aNH6Nu3L+zs7AxdChFRiceQRURUggUEBCAgIECj7fTp0zhx4gSsrKwQERFhoMr0Y9euXQAAIyPdPu506XpnZ2eHXbt2wdjYWKdt6kqlUhWpeoiISjKGLCIi+mA5OTkV+jaNjY0Nst38FLV6iIhKAt6TRUREBZKRkYGff/4ZXbp0ga+vL2rVqoX69evj66+/xrFjx/Jc5vbt2xg9ejT8/Pzg7u6Ojh07YteuXdi2bRucnZ2xcOFCrbYtCAI2bNiA9u3bw8PDA35+foiIiEBGRkae8+d1T5ZSqcR3332HwMBAeHh4oG7duujSpQvWrl2rMZ+zszO2bNkCAJgwYQKcnZ0RHR0NAAgJCYGzszPOnDmDoUOHws3NDfXr18eKFSveeg/UgwcP8L///Q/e3t7w8vJCr1698nze/P394ezsjLi4uFzT1PfSzZs3T3ys7tqZkJCgcX/af9Vz8+ZNjBkzBo0bN0bt2rXRqFEjfPPNN7h582auebt37w5nZ2ekpqZi1apVaN26Ndzc3PDxxx8jNDQUjx49ynN/iYhKIl7JIiIirWVmZiIoKAiXL19G+fLl4eXlBYlEgr///ht//PEHjh49isjISDRv3lxc5sKFC/j666+RlpYGFxcXeHh44MqVKxgxYgQ8PDwKtP2xY8di27ZtMDc3R4MGDcTAd/jwYa2Wz8jIQLdu3XD9+nU4ODigUaNGSE9Px9mzZ3HhwgVcvnwZM2fOBAAEBgbi4sWLuH//Pjw8PFClShU4ODhorG/ixIl4+vQpGjdujJs3b8LZ2fk/t//8+XN07doVL168QL169ZCWloZTp07h5MmTmDRpErp161ag50PN2dkZzZs3x4EDB2BmZobmzZvD2tr6P5c5dOgQhg8fjszMTDg7O8PLywt37tzB9u3bsW/fPsyfPx/NmjXLtdyECRNw4MABuLm5wc/PD2fPnkV0dDROnDiBnTt3wsLCQqd9ICL6kDBkERGR1tatW4fLly/jk08+wffffy/e55OTk4Pp06dj3bp1WLt2rRiysrKyEBISgrS0NEyYMAHdu3cHAGRnZ2P69OlYv3691tveu3cvtm3bBgcHB6xevRr29vYAgJiYGPTu3VurdezZswfXr19HYGAgZs+eDYlEAgC4d+8eOnXqhC1btmDw4MGoUqUKIiIiEBISgvv376NTp07o3LlzrvUlJiZi+/btqFKlinj/1sOHD/Pd/j///IMKFSpgy5YtKFeuHIBXIwIOGDAAM2fORJMmTVC5cmWtnxO1gIAAuLq64sCBA7C0tHzrvXRJSUkYNWoUsrKyMHPmTLRv316c9uuvv2LChAkYNWoUdu/enWsgjePHj2PlypWoV68eACA1NRVffPEF4uLisH37dp2DIhHRh4TdBYmISGvGxsZo0qQJRo0apTGQgkwmw5dffgkAGkO8//7777h79y4aNWokBizg1UAUEyZMQNWqVbXetjqQhYSEiAELAGrXro1BgwZptY6kpCQAQIUKFcSABQAODg4IDw/HrFmzUKpUKa1ratasGapUqQIAkEgkGuvMz8SJE8WABQCNGjVCly5dkJWVhc2bN2u97XexceNGvHjxAu3bt9cIWADQqVMntG/fHs+fP88zBH/xxRdiwAKAsmXLok2bNgCA69evv9/CiYiKCYYsIiLSWlBQEJYtW6YxkMI///yDv/76C/v27QPw6uqV2okTJwAALVq0yLUuIyOjPNvzolKpcO7cOchkMjRs2DDX9Ne7J/4XHx8fAMBPP/2E4cOHY8eOHXj69Km4jrZt2761m93rXFxctJ4XAGxtbTUCipr6/qkzZ84UaH26Onv2LACgZcuWeU7/7LPP8q3H3d09V5v6ald6erq+SiQiKtbYXZCIiAokOTkZ69evx6lTp3D79m0xpKiv4rw+7Pnjx48BvLpylBdtu8alpqbi5cuXsLa2hqmpaa7plSpV0mo9Hh4eCA0NxZw5c7B7927s3r0bEokEtWrVQsuWLfHll1+iTJkyWq0LQIHm/a861VfmEhISCrQ+XSUmJv5nPerXRX3l73V57bNMJgOQ/zDyREQlDUMWERFp7fTp0xgwYABevHgBOzs7eHp6wsnJCTVr1kTlypVz3bf08uVLAPn/3pQuv0OVF6lUKn7Rf5tevXohMDAQ+/fvxx9//IGzZ88iJiYGMTExWLlyJdatW4ePPvpI6+0WhImJyX9O1/b3vHJycgq03Te97XlXhyW5XJ5rmjZdIomISjqGLCIi0oogCBg/fjxevHiR50h4sbGxuZZRX8HKbzCI+Ph4rbZtZWUFExMTpKam4vnz57num0pOTi5Q8LCxsUGXLl3QpUsXqFQq/Pnnn5gxYwZiYmKwbNkyhIWFab2uglBfQXqT+vmpWLGi2KYOM3ntV1pa2jvVYWtrizt37uDhw4eoUaNGrun3798H8Op5IiKiguM9WUREpJXk5GTcv38flpaWeY4gp/6tp9e7jDVo0AAA8hxiXRAEHDp0SKttSyQS1K9fHyqVCgcPHsw1/ciRI1qtZ8aMGWjUqJF4TxLw6mqUt7c3Bg4cCEAz+On7qs2dO3c0BgZRU9/P5uvrK7aZm5sDgNgd83UXL17M1VaQWtX3pu3duzfP6bt3785VDxERaY8hi4iItGJhYQFjY2MolUqcO3dOY9q+ffuwaNEiAJoDX7Ro0QKVKlXC0aNHsXbtWrFdEATMnz9fHI1Om4DQs2dPAMCsWbNw69Ytsf3WrVvij/K+TYUKFZCUlIS5c+fin3/+Eduzs7PFYFGnTh2xXd1d7l2vHKkJgoCQkBCNbe/duxebN2+GhYWFRndL9aAaq1at0ujet2LFCsTExORat7or4osXL956b9QXX3wBc3NzbNmyRfzBZbXNmzeLv0X25siDRESkHXYXJCIirZiamqJLly5YvXo1evToAR8fH1haWuLGjRu4c+cOKlWqhJSUFKSlpSEjIwOmpqaQy+WYNWsW+vTpg6lTp2LTpk2oWrUqrl27hrt378LBwQH37t3T6l6khg0bon///li2bBnatWuH+vXrAwBOnTqFWrVqITk5+a3r6Nq1K3bt2oU///wT/v7+cHd3h1wuR2xsLB49eoRq1app/OaWeoj5RYsW4cKFC2jbtq3WIxnmxdHRETdu3ECLFi3g7e2NpKQkXLhwAcbGxpg1a5ZG97wePXpgz5492Lt3L1q1agVnZ2fxuW7bti22bdumsW5ra2tYWlpCqVSiS5cucHBwyPf3suzs7PDdd99h5MiRCAkJwYoVK+Do6Ig7d+7g2rVrMDMzw6xZs7QeUISIiDTxShYREWktNDQUkyZNQvXq1XHp0iX88ccfkMlkGDBgALZu3Yp69epBpVLh999/F5fx9vbGpk2b0Lx5c8THx+PQoUOwsLDADz/8IA5dbmFhodX2R40ahfnz56NWrVo4d+4cYmJi0L59e/z4449aLW9iYoLly5ejf//+sLGxwenTp3Hs2DGYm5tjwIAB+OWXXzRGz+vatSvatWsHAPjjjz/yvIJUEPb29li3bh1q166NY8eO4caNG2jWrBk2btwoPhdqderUwZo1a9C4cWMkJyfj6NGjKFeuHH7++We0bt0617qlUikiIiLg5OSE2NhYHD9+HM+ePcu3loCAAPz6669o3bo1njx5ggMHDkCpVKJTp07YvHmz1sPrExFRbhJBX0M7ERERveHJkydITU1FxYoVYWZmlmv6wIEDcejQISxfvhyNGjUyQIVERET6xytZRET03vz999/47LPP0KNHD2RkZGhMO3z4MI4cOQJra2t4e3sbqEIiIiL945UsIiJ6b7Kzs9G1a1dcunQJZcuWhYeHB0xMTBAXF4dr167B1NQUCxcuhJ+fn6FLJSIi0huGLCIieq9evHiBTZs2YefOnbh//z5evHiB8uXLo0GDBujTpw+cnJwMXSIREZFeMWQRERERERHpEe/JIiIiIiIi0iOGLCIiIiIiIj1iyCIiIiIiItIjI0MXUNQJggCViretERERERGVZFKpBBKJRKt5GbLeQqUS8PTpc0OXQUREREREBmRtXQoymXYhi90FiYiIiIiI9Ighi4iIiIiISI8YsoiIiIiIiPSIIYuIiIiIiEiPGLKIiIiIiIj0iCGLiIiIiIhIjxiyiIiIiIiI9Ighi4iIiIiISI/4Y8SFTCqVQCrV7kfMiN6FSiVApRIMXQYRERFRicOQVYikUgnKljWHTMYLiPT+5eSokJr6gkGLiIiIqJAxZBUiqVQCmUyKH9Yfx8PEZ4Yuhz5glWzLYHDXhpBKJQxZRERERIWsSIWsJ0+eYObMmTh69CgyMzPh4+ODsWPHwsnJCQBw9epVhIWFISYmBtbW1ujVqxd69OghLq9SqRAZGYlffvkFaWlp8PHxwaRJk1ClShVD7VKeHiY+w92HKYYug4iIiIiI3oMi1W9t8ODBiIuLw7Jly/Drr7/C1NQUvXr1Qnp6OlJSUtC7d284ODhg8+bNGDx4MCIiIrB582Zx+UWLFmHdunWYNm0aNmzYAJVKhX79+iErK8uAe0VERERERCVJkbmS9ezZM1SqVAnBwcFQKBQAgEGDBqFt27a4ceMGTp48CWNjY0ydOhVGRkZwcnISA1nHjh2RlZWFqKgojB49Gk2bNgUAzJs3D40bN8a+ffvQunVrA+4dERERERGVFEXmSlaZMmUwZ84cMWA9ffoUK1asgL29PapXr45z587B19cXRkb/5sL69evj7t27SE5OxrVr1/D8+XM0aNBAnG5paQlXV1ecPXu20PeHiIiIiIhKpiJzJet1EydOxKZNmyCXy7F48WKYm5sjPj5eDGBqtra2AIDHjx8jPj4eAFChQoVc86in6crISD9ZlKMKUmHjMUdERERU+IpkyOrZsye+/PJLrF27FoMHD8a6deuQkZEBuVyuMZ+JiQkAIDMzE+np6QCQ5zzPnuk+kp9UKoGVVSmdlycyJEtLM0OXQERERFTiFMmQVb16dQBAWFgY/vrrL6xZswampqa5BrDIzMwEAJibm8PU1BQAkJWVJf5bPY+Zme5fNFUqAUrlC52Xf51MJuWXXipUSmU6cnJUhi6DiIiIqNiztDTTupdQkQlZT58+xcmTJ9GyZUvxviupVIrq1asjMTER9vb2SExM1FhG/djOzg7Z2dlim4ODg8Y8zs7O71Rbdja/pFLxlJOj4vFLREREVMiKzA0bycnJGDlyJE6ePCm2vXz5ErGxsXBycoKPjw/Onz+PnJwccfqpU6fg6OgIGxsbuLi4oHTp0jh9+rQ4XalUIjY2Fj4+PoW6L0REREREVHIVmZClUCjg5+eH6dOn4+zZs7h+/TpCQkKgVCrRq1cvdOzYEf/88w/Gjx+PmzdvIjo6GitWrEBwcDCAV/diBQUFISIiAgcPHsS1a9cwYsQI2NvbIyAgwMB7R0REREREJUWR6S4IAHPnzsWcOXMwYsQIpKWlwdvbG2vXrkXFihUBAD/99BPCwsLQvn17lC9fHmPGjEH79u3F5YcNG4bs7GxMmDABGRkZ8PHxwfLly2FsbGyoXSIiIiIiohJGIgiCYOgiirKcHBWePn2ul3UZGUlhZVUK477fhbsPU/SyTqK8VK1khfD/fYaUlOe8J4uIiIhID6ytS2k98EWR6S5IRERERET0IWDIIiIiIiIi0qMidU8WEX34pFIJpFKJocugEkClEqBSsUc8EREVPoYsIio0UqkEVlZmkEplhi6FSgCVKgcpKekMWkREVOgYsoio0Ly6iiXDnZ0/Iv3JY0OXQx8wM5sKcGz9NaRSCUMWEREVOoYsIip06U8eIz3hnqHLICIiInovOPAFERERERGRHjFkERERERER6RFDFhERERERkR4xZBEREREREekRQxYREREREZEeMWQRERERERHpEUMWERERERGRHjFkERERERER6RFDFhERERERkR4xZBEREREREekRQxYREREREZEeMWQRERERERHpEUMWERERERGRHjFkERERERER6RFDFhERERERkR4xZBEREREREekRQxYREREREZEeMWQRERERERHpUZEKWampqZg0aRL8/Pzg5eWFrl274ty5c+L03r17w9nZWeO/7t27i9MzMzMxZcoUNGjQAJ6enhg1ahSePn1qiF0hIiIiIqISysjQBbxu5MiRSEpKwty5c2FjY4PVq1ejb9++2LJlC6pVq4a///4b3377LZo3by4uY2xsLP7722+/xblz57Bw4ULI5XJMnjwZw4YNw5o1awyxO0REREREVAIVmZAVFxeH48ePY926dahbty4AYOLEiTh69Ch27NiBoKAgPHnyBO7u7ihfvnyu5RMSErB161YsWbIE3t7eAIC5c+eiVatWuHDhAjw9PQt1f4iIiIiIqGQqMiHLysoKy5YtQ506dcQ2iUQCiUQCpVKJv//+GxKJBI6Ojnkuf/78eQBA/fr1xTZHR0fY2dnh7Nmz7xSyjIz006tSJitSvTOpBChqx1xRq4c+fDzmiIjIEIpMyLK0tESTJk002vbu3Yu4uDiMGzcO169fh4WFBaZOnYrjx4/D3NwcrVq1wqBBgyCXy5GQkAArKyuYmJhorMPW1hbx8fE61yWVSmBlVUrn5YkMydLSzNAlEBkUzwEiIjKEIhOy3vTnn38iNDQUAQEBaNq0KcaNG4fMzEy4ubmhd+/euHr1KmbNmoVHjx5h1qxZSE9Ph1wuz7UeExMTZGZm6lyHSiVAqXzxLrsiksmk/MCnQqVUpiMnR2XoMkQ8B6iwFbVzgIiIii9LSzOte0gUyZB14MABjB49Gl5eXoiIiAAATJ06FWPHjkWZMmUAAAqFAsbGxhgxYgTGjBkDU1NTZGVl5VpXZmYmzMze7UtddjY/oKl4yslR8filEo3nABERGUKR66y+Zs0aDB06FM2aNcOSJUvE7n9GRkZiwFKrUaMGACA+Ph729vZITU3NFbQSExNhZ2dXOMUTEREREVGJV6RC1rp16zBt2jR069YNc+fO1ej+1717d4SGhmrMf/nyZRgbG6Nq1aqoW7cuVCqVOAAGANy5cwcJCQnw8fEptH0gIiIiIqKSrch0F7xz5w7Cw8PRokULBAcHIzk5WZxmamqKli1bIjw8HG5ubmjUqBEuX76MWbNmoW/fvihdujRKly6Nzz//HBMmTEB4eDjMzMwwefJk+Pr6wsPDw3A7RkREREREJUqRCVl79+7Fy5cvsX//fuzfv19jWvv27TFz5kxIJBKsXr0a4eHhKF++PHr16oX+/fuL802bNg3h4eEYMmQIAMDPzw8TJkwo1P0gIiIiIqKSTSIIgmDoIoqynBwVnj59rpd1GRlJYWVVCuO+34W7D1P0sk6ivFStZIXw/32GlJTnReqmf/U5ELtyKtIT7hm6HPqAmdk5wLXnpCJ3DhARUfFlbV1K69EFi9Q9WURERERERMUdQxYREREREZEeMWQRERERERHp0TsNfPHy5UsYGxsDAJRKJXbs2AEjIyN89tlnsLCw0EuBRERERERExYlOISszMxOhoaF4/Pgx1q9fj4yMDHTu3Bn37t2DIAhYunQpNmzYAFtbW33XS0REREREVKTp1F1wyZIl2LVrF+zt7QEAO3bsQFxcHL788ktMnToVqampWLx4sV4LJSIiIiIiKg50upK1d+9etGzZEvPmzQMAHD58GGZmZhg3bhzkcjni4uKwe/duvRZKRERERERUHOh0JevBgwdo1KgRAEClUuHs2bOoW7cu5HI5AMDR0RHJycn6q5KIiIiIiKiY0ClklS5dGpmZmQCAixcvIi0tDQ0aNBCnJyUloWzZsnopkIiIiIiIqDjRKWQpFArs3LkTT58+xZo1ayCRSNC0aVMAQHx8PDZt2oSaNWvqs04iIiIiIqJiQaeQFRwcjCtXrqBhw4bYtWsXmjRpAicnJ5w/fx4tWrRAUlISvv76a33XSkREREREVOTpNPBFgwYNsHr1auzYsQP29vbo3r07AMDGxgY+Pj4IDg6Gt7e3XgslIiIiIiIqDnQKWY8ePYKLiws8PDw02qtWrYqoqCg8e/YMp06dQv369fVRIxERERERUbGhU3fBTz75BAcOHMh3+v79+zFw4ECdiyIiIiIiIiqutLqSdf/+fWzbtk18LAgC9u3bh7t37+aaVxAEHDx4EMbGxnorkoiIiIiIqLjQKmRVqlQJe/bswc2bNwEAEokE+/btw759+/Jdpnfv3vqpkIiIiIiIqBjRKmRJpVIsWbIE9+/fhyAI6NOnD/r376/x21ivz1uuXDk4OTnpvVgiIiIiIqKiTuuBLypXrozKlSsDAIYMGYKAgAAoFIr3VhgREREREVFxpNPAF0OGDIGFhQVmzpyJZ8+eie2RkZGYNm0anjx5orcCiYiIiIiIihOdQtbdu3fRqVMnrFy5Evfv3xfbExMTsXbtWnTu3BmJiYl6K5KIiIiIiKi40ClkLVy4ENnZ2Vi9ejVq164ttk+dOhUbNmzA8+fPsXDhQr0VSUREREREVFzoFLLOnj2LHj16wNvbO9c0Dw8PfPXVVzh+/Pg7F0dERERERFTc6BSylEolrKys8p1ua2uL5ORknYsiIiIiIiIqrnQKWVWqVMGxY8fynX7y5ElUrFixwOtNTU3FpEmT4OfnBy8vL3Tt2hXnzp3TWG+HDh3g7u6OVq1a4bffftNYPjMzE1OmTEGDBg3g6emJUaNG4enTpwWug4iIiIiISFc6hazAwEAcPnwY8+bNQ2pqqtiuVCoRGRmJ/fv3IzAwsMDrHTlyJC5cuIC5c+di8+bNqFmzJvr27Yvbt2/j1q1bCA4ORuPGjREdHY3OnTtjzJgxOHnypLj8t99+i2PHjmHhwoVYuXIlbt++jWHDhumyi0RERERERDrR+neyXtenTx+cPHkSS5cuxbJly2BlZQWJRIKUlBSoVCrUq1cP/fv3L9A64+LicPz4caxbtw5169YFAEycOBFHjx7Fjh078OTJEzg7O2PEiBEAACcnJ8TGxuKnn35CgwYNkJCQgK1bt2LJkiXivWJz585Fq1atcOHCBXh6euqyq0RERERERAWi05UsIyMjREVFYebMmWjatClsbGxgaWmJhg0bYurUqYiKioKxsXGB1mllZYVly5ahTp06YptEIoFEIoFSqcS5c+fQoEEDjWXq16+P8+fPQxAEnD9/XmxTc3R0hJ2dHc6ePavLbhIRERERERWYTleygFcBqF27dmjXrp1eCrG0tESTJk002vbu3Yu4uDiMGzcOW7Zsgb29vcZ0W1tbpKenIyUlBQkJCbCysoKJiUmueeLj49+pNiMjnbJoLjKZftZDpK2idswVtXrow8djjoiIDEHnkAUACQkJOHLkCB4+fIiOHTvC3NwcCQkJGr+dpas///wToaGhCAgIQNOmTZGRkQG5XK4xj/pxVlYW0tPTc00HABMTE2RmZupch1QqgZVVKZ2XJzIkS0szQ5dAZFA8B4iIyBB0DlmrVq1CREQEsrKyIJFI0KBBA2RmZmLgwIHo1q0bJkyYoHNRBw4cwOjRo+Hl5YWIiAgAr8JSVlaWxnzqx2ZmZjA1Nc01HXg14qCZme4fsiqVAKXyhc7Lv04mk/IDnwqVUpmOnByVocsQ8RygwlbUzgEiIiq+LC3NtO4hoVPIOnz4MMLDw1GvXj189tlnmDx5MgCgWrVq8PT0xNq1a+Hq6ooOHToUeN1r1qxBWFgYWrVqhe+++068OlWhQgUkJiZqzJuYmAhzc3NYWFjA3t4eqampyMrK0riilZiYCDs7O112U5SdzQ9oKp5yclQ8fqlE4zlARESGoFNn9eXLl8PV1RVRUVEICAgQ2x0cHLBq1SrUrl0b69evL/B6161bh2nTpqFbt26YO3euRljy9vbGmTNnNOY/deoUvLy8IJVKUbduXahUKnEADAC4c+cOEhIS4OPjo8NeEhERERERFZxOIevKlSv4/PPPIZPJck0zMjJC27Ztcffu3QKt886dOwgPD0eLFi0QHByM5ORkJCUlISkpCWlpaejevTsuXbqEiIgI3Lp1C1FRUdizZw/69esHALCzs8Pnn3+OCRMm4PTp07h06RJGjhwJX19feHh46LKbREREREREBabzPVlvjuL3uqysLGRnZxdofXv37sXLly+xf/9+7N+/X2Na+/btMXPmTCxatAizZ8/GypUrUblyZcyePVtjWPdp06YhPDwcQ4YMAQD4+fm9071hREREREREBaVTyFIoFDh8+DCCgoJyTcvJycFvv/2GGjVqFGidAwYMwIABA/5zHj8/P/j5+eU73dzcHNOnT8f06dMLtG0iIiIiIiJ90am7YFBQEI4fP47p06fj1q1bAIAXL17g0qVLGDBgAGJjY/HFF1/otVAiIiIiIqLiQKcrWYGBgbh27RqWL1+OtWvXAoDYRU8QBHTq1AmdOnXSX5VERERERETFhM73ZH3zzTdo2bIldu7cibt370KlUqFy5cpo2bKlxn1SREREREREJYnOIQsA3Nzc4Obmpq9aiIiIiIiIij2tQtb9+/dRrlw5mJmZiY+1IZFIUKpUKVhZWeleIRERERERUTGiVcgKCAjArFmzEBgYCABo0aIFJBKJ1hupVKkS5s2bhzp16uhWJRERERERUTGhVchq164dHBwcNB5rE7IEQYBSqcSJEycwefJkREdH614pERERERFRMaBVyJoxY4bG45kzZxZoI/PmzcOqVasKtAwREREREVFx9E4DXwiCgJiYGDx48AByuRwVK1ZEzZo1c83n5eWFxMTEd9kUERERERFRsaBzyPrzzz8RGhqKe/fuabQ7ODggLCwM3t7eYluTJk3QpEkT3askIiIiIiIqJnQKWbdu3ULfvn3x8uVLtGvXDjVq1EBOTg5u3LiBXbt24euvv0Z0dDQcHR31XS8REREREVGRplPIWrRoEaRSKaKjo6FQKDSm9evXD126dMHSpUsLfO8WERERERFRcSfVZaGTJ0+ia9euuQIWACgUCnTt2hUnTpx45+KIiIiIiIiKG51CllKpROXKlfOdXqVKFaSkpOhcFBERERERUXGlU8iyt7fHpUuX8p3+119/wdbWVueiiIiIiIiIiiudQlbz5s2xdetWbNmyJde0zZs3Y9u2bfD393/n4oiIiIiIiIobnQa+GDRoEA4ePIhx48Zh8eLFqFatGoBXow4+ePAAFSpUwKBBg/RaKBERERERUXGg05UsS0tLbNiwAe3bt0dKSgqOHDmCI0eOICUlBe3atcOmTZtgZWWl71qJiIiIiIiKPJ2uZB04cAA+Pj4IDw9HWFgYUlJSIAgCrK2tIZFI9F0jERERERFRsaHTlazx48dj2bJlAACJRAJra2vY2NgwYBERERERUYmnU8jKyspClSpV9F0LERERERFRsadTyOrYsSPWrFmDhw8f6rseIiIiIiKiYk2ne7KysrLw6NEjNG/eHJUrV4aNjQ1kMpnGPBKJBGvWrNFLkURERERERMWFTiFr06ZN4r/v37+P+/fv55rnXe/PWrp0KY4dO4bVq1eLbRMmTMAvv/yiMV+lSpVw6NAhAIBKpUJkZCR++eUXpKWlwcfHB5MmTWLXRiIiIiIiKjQ6haxr167puw4Na9euxfz58+Ht7a3R/vfff2PAgAEICgoS216/grZo0SKsW7cOM2fOhL29PWbPno1+/fphx44dkMvl77VmIiIiIiIiQMd7st6XhIQEDBgwABEREahatarGNEEQcPPmTdSuXRvly5cX/7O2tgbwqgtjVFQUhg0bhqZNm8LFxQXz5s1DfHw89u3bZ4C9ISIiIiKikkjnkPX8+XMsWLAArVu3hru7O7y8vNCxY0esWLEC2dnZOq3zypUrMDY2xvbt2+Hu7q4x7d69e3jx4gWqVauW57LXrl3D8+fP0aBBA7HN0tISrq6uOHv2rE71EBERERERFZRO3QWfPn2Krl27Ii4uDhYWFnByckJ2djbu3LmD7777Dnv27MGqVasK3EXP398f/v7+eU67fv06AGD16tX4448/IJVK4efnhxEjRsDCwgLx8fEAgAoVKmgsZ2trK07TlZGRfi74yWRF6sIhlQBF7ZgravXQh4/HHBERGYJOIWv+/Pm4d+8exo0bh6+++gpGRq9Wk5WVhRUrVmDu3LlYvHgx/ve//+mt0OvXr0MqlcLW1hZLlizBvXv3MGvWLNy4cQMrV65Eeno6AOQKdiYmJnj27JnO25VKJbCyKvVOtRMZiqWlmaFLIDIongNERGQIOoWsw4cPo3PnzujRo4dGu1wuR//+/XH79m3s2LFDryFr4MCB+Oqrr2BlZQUAUCgUKF++PL744gtcvnwZpqamAF4FPfW/ASAzMxNmZrp/yKpUApTKF+9W/P+TyaT8wKdCpVSmIydHZegyRDwHqLAVtXOAiIiKL0tLM617SOgUstLS0uDi4pLvdHd3d+zevVuXVedLKpWKAUutRo0aAID4+Hixm2BiYiIcHBzEeRITE+Hs7PxO287O5gc0FU85OSoev1Si8RwgIiJD0Kmzeq1atfDHH3/kO/3ChQv/GcJ0MWbMGPTq1Uuj7fLlywCA6tWrw8XFBaVLl8bp06fF6UqlErGxsfDx8dFrLURERERERPnRKWSNHz8e58+fx+TJk/HkyROxPT09HQsXLsT+/fsxceJEqFQqjf/eRcuWLXHy5ElERkbi3r17+P333zFu3Di0bt0aTk5OkMvlCAoKQkREBA4ePIhr165hxIgRsLe3R0BAwDttm4iIiIiISFs6dRccPnw4JBIJNm3ahE2bNsHKygpyuRxJSUlQqVQQBAGdO3fWWEYikSA2NlbnQj/55BPMnz8fy5Ytw48//ggLCwsEBgZi+PDh4jzDhg1DdnY2JkyYgIyMDPj4+GD58uUwNjbWebtEREREREQFoVPIsrOzg52dXa72KlWqvHNBajNnzszV9umnn+LTTz/NdxmZTIZvvvkG33zzjd7qICIiIiIiKgidQtbq1av1XQcREREREdEHgb/SSEREREREpEcMWURERERERHrEkEVERERERKRHDFlERERERER6pFXIio+Pf991EBERERERfRC0ClkdOnTAxo0bxceRkZG4fv36eyuKiIiIiIiouNIqZKWlpSEzM1N8HBkZib///vu9FUVERERERFRcafU7WVWqVMHixYvx+PFjlCpVCgCwf/9+xMXF5buMRCLB4MGD9VMlERERERFRMaFVyBoxYgRGjRqFn3/+GcCrALVv3z7s27cv32UYsoiIiIiIqCTSKmS1aNECv//+O27duoWsrCz06dMHwcHBqF+//vuuj4iIiIiIqFjRKmQBgJWVFby9vQEAPj4+qF+/Pho0aPDeCiMiIiIiIiqOtA5Zr1u9ejUAQBAExMTE4MGDB5DL5ahYsSJq1qyp1wKJiIiIiIiKE51CFgD8+eefCA0Nxb179zTaHRwcEBYWJl71IiIiIiIiKkl0Clm3bt1C37598fLlS7Rr1w41atRATk4Obty4gV27duHrr79GdHQ0HB0d9V0vERERERFRkaZTyFq0aBGkUimio6OhUCg0pvXr1w9dunTB0qVLMXPmTL0USUREREREVFxo9WPEbzp58iS6du2aK2ABgEKhQNeuXXHixIl3Lo6IiIiIiKi40SlkKZVKVK5cOd/pVapUQUpKis5FERERERERFVc6hSx7e3tcunQp3+l//fUXbG1tdS6KiIiIiIiouNIpZDVv3hxbt27Fli1bck3bvHkztm3bBn9//3cujoiIiIiIqLjRaeCLQYMG4eDBgxg3bhwWL16MatWqAXg16uCDBw9QoUIFDBo0SK+FEhERERERFQc6XcmytLTEhg0b0L59e6SkpODIkSM4cuQIUlJS0K5dO2zatAlWVlb6rpWIiIiIiKjI0/nHiG1sbBAeHo6wsDCkpKRAEARYW1tDIpHosz4iIiIiIqJiReeQpSaRSGBtba2PWoiIiIiIiIq9dw5Z78vSpUtx7NgxrF69Wmy7evUqwsLCEBMTA2tra/Tq1Qs9evQQp6tUKkRGRuKXX35BWloafHx8MGnSJFSpUsUQu0BERJQnqVQCqZQ9P+j9U6kEqFSCocsgKnGKZMhau3Yt5s+fD29vb7EtJSUFvXv3hr+/P6ZMmYKLFy9iypQpKFWqFDp27AgAWLRoEdatW4eZM2fC3t4es2fPRr9+/bBjxw7I5XJD7Q4REZFIKpWgrJUZZFKZoUuhEiBHlYPUlHQGLaJCVqRCVkJCAiZPnozTp0+jatWqGtM2bdoEY2NjTJ06FUZGRnByckJcXByWLVuGjh07IisrC1FRURg9ejSaNm0KAJg3bx4aN26Mffv2oXXr1oW/Q0RERG+QSiWQSWVY+vsqPHqWYOhy6ANWsYwdgpv0gFQqYcgiKmRFKmRduXIFxsbG2L59O3744Qc8fPhQnHbu3Dn4+vrCyOjfkuvXr4+lS5ciOTkZjx49wvPnz9GgQQNxuqWlJVxdXXH27FmGLCIiKlIePUtA3JMHhi6DiIjeA51CVmhoKLp06QJ3d3cAwIsXLzBt2jT069cPTk5OOhfj7++f748Yx8fHQ6FQaLTZ2toCAB4/foz4+HgAQIUKFXLNo56mKyMjnUa6z0Um0896iLRV1I65olYPffiK4jFXFGuiDxuPOaLCp1XIGjBgAGrXrg03NzfUrl0bW7ZsQcOGDcWQlZmZia1bt6JNmzbvFLL+S0ZGRq77qkxMTMTtp6enA0Ce8zx79kzn7UqlElhZldJ5eSJDsrQ0M3QJRAbFc4CI5wGRIWgVsrKysrB69Wo8e/YMEokEEokE69evx+PHj1G7dm1UrFgRgvB++/qampoiKytLoy0zMxMAYG5uDlNTU7FW9b/V85iZ6f7molIJUCpf6Lz862QyKd/oqFAplenIyVEZugwRzwEqbEXtHAB4HlDhK4rnAVFxZGlppvWVYa1CVlRUFADg3r17uHTpEkaPHo34+HgsWrQI6enpYvCKiorC7du34eHhARcXF8hk+hs5yd7eHomJiRpt6sd2dnbIzs4W2xwcHDTmcXZ2fqdtZ2fzjYmKp5wcFY9fKtF4DhDxPCAyhAJ10nVwcBAHkBg+fDj+/PNP7Ny5ExMnToQgCLh79y5mzpyJjh07agy/rg8+Pj44f/48cnJyxLZTp07B0dERNjY2cHFxQenSpXH69GlxulKpRGxsLHx8fPRaCxERERERUX60ClknTpyAUqnM1S6RSFC9enV8+umnAICpU6fi/PnzWL9+PYYPH67XQjt27Ih//vkH48ePx82bNxEdHY0VK1YgODgYwKt7sYKCghAREYGDBw/i2rVrGDFiBOzt7REQEKDXWoiIiIiIiPKjVXfBPn36QCKRoHLlyqhduzYkEgkePXqE9PT0XPc7yeVyeHp6wtPTU6+F2tjY4KeffkJYWBjat2+P8uXLY8yYMWjfvr04z7Bhw5CdnY0JEyYgIyMDPj4+WL58OYyNjfVaCxERERERUX60Cll79uzB5cuXxf8EQcD8+fOxYMECVK1aFU5OTpBIJLh37x68vb31EmpmzpyZq83NzQ0bN27MdxmZTIZvvvkG33zzzTtvn4iIiIiISBdahayqVauiatWqCAwMBAC4uLhg8ODBqFChAmJjY8Xg9e233yIsLAyurq7w8PBASEjIey2eiIiIiIioqNHpx4gB4KOPPkJgYCA6duyIp0+f4uOPP8b48eMhlUpx4cIFHD58mCGLiIiIiIhKHJ1Clo+PD8qVKyc+lsvl8PHxgZeXF1xdXfHVV1/prUAiIiIiIqLiRKeQtXr1ao3HpUuXztVGRERERERUEhXod7KIiIiIiIjovzFkERERERER6RFDFhERERERkR4xZBEREREREekRQxYREREREZEeMWQRERERERHpEUMWERERERGRHjFkERERERER6RFDFhERERERkR4xZBEREREREekRQxYREREREZEeMWQRERERERHpEUMWERERERGRHjFkERERERER6ZGRoQsgIiIiopJHKpVAKpUYugwqAVQqASqVUKjbZMgiIiIiokIllUpgVdYMUpnM0KVQCaDKyUFKanqhBi2GLCIiIiIqVFKpBFKZDBcXL8U/jx4buhz6gJWuWAEeA4MhlUoYsoiIiIjow/fPo8dQxsUZugwivePAF0RERERERHpU7EJWQkICnJ2dc/0XHR0NALh69SqCgoLg4eEBf39/rFq1ysAVExERERFRSVLsugteu3YNJiYmOHDgACSSf0eksbCwQEpKCnr37g1/f39MmTIFFy9exJQpU1CqVCl07NjRgFUTEREREVFJUexC1vXr11G1alXY2trmmrZy5UoYGxtj6tSpMDIygpOTE+Li4rBs2TKGLCIiIiIiKhTFrrvg33//DScnpzynnTt3Dr6+vjAy+jc71q9fH3fv3kVycnJhlUhERERERCVYsbySZWVlhW7duuHOnTv46KOPMHDgQPj5+SE+Ph4KhUJjfvUVr8ePH6NcuXI6bdPISD9ZVCYrdpmWirmidswVtXrow1cUj7miWBN92IriMVcUa6IPW2Efc8UqZGVnZ+P27duoXr06QkJCULp0afz222/o378/fv75Z2RkZEAul2ssY2JiAgDIzMzUaZtSqQRWVqXeuXYiQ7C0NDN0CUQGxXOAiOcBEVD450GxCllGRkY4ffo0ZDIZTE1NAQC1a9fGjRs3sHz5cpiamiIrK0tjGXW4Mjc312mbKpUApfLFuxX+/2QyKd/oqFAplenIyVEZugwRzwEqbEXtHAB4HlDh43lApJ/zwNLSTOsrYsUqZAFAqVK5ryrVqFEDx44dg729PRITEzWmqR/b2dnpvM3s7KL1xkSkrZwcFY9fKtF4DhDxPCACCv88KFYdYm/cuAEvLy+cPn1aoz0mJgbVq1eHj48Pzp8/j5ycHHHaqVOn4OjoCBsbm8Iul4iIiIiISqBiFbKcnJxQrVo1TJ06FefOncOtW7cwY8YMXLx4EQMHDkTHjh3xzz//YPz48bh58yaio6OxYsUKBAcHG7p0IiIiIiIqIYpVd0GpVIolS5Zgzpw5GD58OJRKJVxdXfHzzz+Lowr+9NNPCAsLQ/v27VG+fHmMGTMG7du3N3DlRERERERUUhSrkAUA5cqVw4wZM/Kd7ubmho0bNxZiRURERERERP8qVt0FiYiIiIiIijqGLCIiIiIiIj1iyCIiIiIiItIjhiwiIiIiIiI9YsgiIiIiIiLSI4YsIiIiIiIiPWLIIiIiIiIi0iOGLCIiIiIiIj1iyCIiIiIiItIjhiwiIiIiIiI9YsgiIiIiIiLSI4YsIiIiIiIiPWLIIiIiIiIi0iOGLCIiIiIiIj1iyCIiIiIiItIjhiwiIiIiIiI9YsgiIiIiIiLSI4YsIiIiIiIiPWLIIiIiIiIi0iOGLCIiIiIiIj1iyCIiIiIiItIjhiwiIiIiIiI9YsgiIiIiIiLSI4YsIiIiIiIiPfrgQpZKpcKCBQvQuHFjeHh44Ouvv8b9+/cNXRYREREREZUQH1zIWrRoEdatW4dp06Zhw4YNUKlU6NevH7KysgxdGhERERERlQAfVMjKyspCVFQUhg0bhqZNm8LFxQXz5s1DfHw89u3bZ+jyiIiIiIioBPigQta1a9fw/PlzNGjQQGyztLSEq6srzp49a8DKiIiIiIiopDAydAH6FB8fDwCoUKGCRrutra04raCkUgmsrUu9c20AIJG8+v/Yvv7IyVHpZZ1EeZHJXv39pEwZMwiCgYt5jfocqNFpOARVjmGLoQ+aRCoDUPTOAeDf82BUiwHI5nlA75FRMTgPfL4ZCSGb5wG9PxIj/Z0HUqlE63k/qJCVnp4OAJDL5RrtJiYmePbsmU7rlEgkkMm0f0K1Uaa0qV7XR5QfqbRoXqw2LmVp6BKohCiq5wAAWJpZGLoEKiGK8nlgYsnPAyochX0eFN2zTgempq/Cy5uDXGRmZsLMzMwQJRERERERUQnzQYUsdTfBxMREjfbExETY2dkZoiQiIiIiIiphPqiQ5eLigtKlS+P06dNim1KpRGxsLHx8fAxYGRERERERlRQf1D1ZcrkcQUFBiIiIgLW1NSpVqoTZs2fD3t4eAQEBhi6PiIiIiIhKgA8qZAHAsGHDkJ2djQkTJiAjIwM+Pj5Yvnw5jI2NDV0aERERERGVABJBKGqDehIRERERERVfH9Q9WURERERERIbGkEVERERERKRHDFlERERERER6xJBFRERERESkRwxZREREREREesSQRUREREREpEcMWXrGEfGJSFd8/6CSjMc/EX1IGLL0RKlUYsyYMTh37lyhbbN79+7o3r37f84TEhICf3//QqqIirqFCxfC2dnZ0GVQHg4ePIixY8catIaCHh88nkhfbty4ga5du2q0OTs7Y+HChQaqiMgweNx/OIwMXcCH4urVq9i2bRs6duxo6FKI8tW5c2c0btzY0GVQHlasWGHoEgp8fPB4In3Zs2cPLly4YOgyiIj0hiGLqASxt7eHvb29ocugIqqgxwePJyIioryxuyAAf39/LFiwAN999x0+/vhjuLm5oW/fvrh79644zy+//IIOHTrAw8MDbm5uaNu2LXbv3g0AOH36NHr06AEA6NGjh9iFz9/fHyEhIRrbio6OhrOzMx48eADgVXebFi1aIDIyEr6+vmjUqBGePXuGjIwMzJkzBwEBAahduza8vLzQu3dvXL169Z32VZv1hoSEoFevXti8eTNatmyJ2rVro23btvjjjz801nXhwgV069YNHh4eaNq0KVauXIlevXqJ+3z69Gk4Ozvj9OnTGsu92c1R233dsmULPvvsM9SpUwdt2rTByZMn4erqiujoaHGeR48eYeTIkfD19YW7uzt69uyJ2NjYtz4vISEh6N69O3799Vc0a9YMnp6e6NmzJ65duybOEx0dDVdXV/zyyy9o2LAhfH19cfPmTQDArl270KFDB3h6eqJhw4aYNGkSnj17prGNixcvok+fPvDy8kL9+vUxcuRIJCQkiNNTU1MxadIkfPzxx6hTpw6++OILnDx5UmMdx48fxxdffAFPT0/4+Phg4MCBuHXrljj93r17GDBgAOrVqwd3d3d8+eWX+P3338Xpb3bv6t69O8aPH49ly5ahadOmqFOnDrp06YJLly5pbPfIkSPo0KED3Nzc0LJlS+zcuRMtWrQoMl0aBEHAihUr8Omnn8LNzQ0tWrTA8uXLxXs8jh8/jq+++gp169ZFvXr1MGrUKDx+/FhcPjo6GnXq1MG5c+fQsWNH1KlTBy1btsShQ4dw+/Zt9OzZE+7u7mjRogV+++03jeWcnZ3x119/oX379nBzc0NgYCD27NkjzqPNedC9e3ecOXMGZ86c0ZhXm2PC2dkZkZGR4usTGRmZ53M0ceJENGzYEDk5ORrtYWFhqFevHl6+fJnr+Cjo8QS8/VxQv+cdOXIEgYGBqF27Nlq2bImtW7fmWTd9GHJycrB27VoEBgbCzc0NTZs2RUREBDIzM7Fw4ULxuH2zq9Q///yD8ePHw9fXF56enhg2bBiSk5M11n3gwAF06NABderUQcOGDTF9+nS8ePFCnJ7f5yzR++bv74/IyEiEh4ejXr168PT0xKhRo/D8+XMsW7YMfn5+qFu3LoYOHYqUlJQ815GYmIjQ0FA0adIEbm5u6NSpEw4ePKgxj7OzM9asWYOxY8fC09MTH3/8McLCwpCZmSnO87b3c9I/hqz/t2rVKty+fRszZszA9OnTERMTI94fsXbtWkyaNAnNmzfH0qVLERERAblcjtGjRyM+Ph61atXCpEmTAACTJk3C5MmTC7TtR48e4ffff8e8efMQGhqKMmXKYMyYMdi8eTP69++PqKgohIaG4saNGxg1atQ73Rys7XpjYmKwfPlyDBs2DD/88ANkMhmGDh0qfjDdunULvXr1AgDMnTsXQ4cOxbJly3D+/Pn3UtPWrVsREhICLy8vLFq0CC1btsSgQYM0vjA+ffoUXbp0wZUrVzBx4kTMmTMHKpUK3bp10wgi+bl69SrmzZuHIUOGYPbs2UhJSUFQUBASExPFeXJychAVFYWwsDCEhobCyckJixYtwsiRI+Hh4YEFCxZg8ODB2Lt3L7p3746MjAwAQGxsLIKCgpCZmYlZs2ZhypQpiImJQd++fZGdnY3MzEz07NkTBw8exIgRIxAZGQl7e3v069dP/FJ9//59DBo0CLVr18bixYsRFhaGO3fuoH///lCpVFCpVAgODkZ6ejpmzZqFRYsWoWzZshg4cCDi4uLy3e+9e/fi4MGDmDBhAubOnYvk5GQMHTpUfG5PnTqFQYMGoUKFCli4cCG6deuGyZMna4QUQ5s1axZmzZoFf39/LFmyBJ06dUJERASWLVuGrVu3ok+fPqhQoQLmzp2L0NBQXLhwAV9++SWePHkiriM7OxujRo1Cly5dsHjxYpiZmWH06NEYMGAAmjZtiiVLlsDW1hZjx45FfHy8xvaDg4PxySefIDIyEo6Ojhg+fHiBPrwmT54MV1dXuLq6YuPGjahVq5ZWx4TakiVLEBgYiAULFqBly5Z5bqNt27ZITk7WCHsqlQq7d+/G559/DmNjY435dTmetDkXACApKQlTp05Fjx49sGzZMlSuXBljx47V6jyl4mnSpEmYMWMGmjdvjsWLF6Nbt25Ys2YNBg0ahE6dOqFTp04AgI0bN6Jz587icqtWrcLLly/x/fffY9SoUTh06BCmTp0qTt+xYwcGDx6MatWq4YcffsCQIUOwfft2DBo0SOMzLa/PWaLCEBUVhcePH2PevHkYOHAgdu7ciY4dO+LYsWOYNm0aRo4ciYMHD2LBggW5lk1OTkanTp1w7tw5jBgxAgsXLkSlSpUwePBgbN++XWPe77//Hk+ePMH8+fPRr18/bNy4Ufweq+v3A3pHAgnNmjUTmjVrJmRnZ4ttCxcuFBQKhfD06VNhxowZwuzZszWWiYmJERQKhbBz505BEATh1KlTgkKhEE6dOqWx3rFjx2ost3nzZkGhUAj3798XBEEQFixYICgUCuHs2bPiPJmZmUKfPn2E3377TWPZqKgoQaFQCImJiYIgCEJQUJAQFBT0n/s2duxYoVmzZgVa79ixYwWFQiHExcWJ85w5c0ZQKBTCnj17BEEQhG+++UZo2LCh8OLFC3GeP//8U1AoFOI+5/WcvFm3tjU1bdpUCA4O1phn6dKlgkKhEDZv3iwIgiDMnTtXqFOnjvDgwQON5/KTTz4Rhg4d+tbn6c3XISEhQahTp4742qtfu61bt4rzpKamCrVr1xYmTpyosb6zZ88KCoVCWLNmjSAIgjB06FChYcOGQkZGhsbz1axZMyE2NlbYuHGjoFAohIsXL4rTVSqV0K1bN6FDhw6CIAjCzp07BYVCIcTHx4vz/PXXX8LcuXOFtLQ0ITExUVAoFML27dvF6UqlUggPDxeuX78uCMK/x5taUFCQ4O7uLqSlpYltW7ZsERQKhXD58mVBEAThq6++Etq0aSOoVCpxHnUtCxYs+M/ntTA8e/ZMcHV1FcLCwjTap02bJvTt21do2LCh0KdPH41pcXFxQq1atYTvvvtOEIR/X9t169aJ8/z222+CQqEQ5s+fL7ZdvnxZUCgUwv79+zWWi4yMFOdRqVRC27Zthc6dOwuCoN15kNdjbY4JQRAEhUIh9OzZ863Pk0qlEpo1ayaEhoaKbSdOnNDYxuvHR0GPJ23PBfUyJ06cEOd5+PChoFAohOXLl791P6j4uXHjhqBQKISlS5dqtG/dulVQKBTCkSNHcr03CcKrY1t9HqmNHj1a8PHxEQTh1THt5+cn9O3bV2Me9XF9+PBhQRDy/pwlKgzNmjUTGjduLLx8+VJsa9WqleDp6SkolUqxLTg4WGjTpo0gCILGZ+usWbOEWrVqaXyvEQRB6Nmzp9CwYUMhJydHXCYgIEBjOz///LOgUCiEmzdvavV+TvrHK1n/r06dOpDJZOJj9X0G6enpCAkJwejRo6FUKnHx4kVs27YNa9euBQBkZWXpZfs1a9YU/y2Xy7F8+XJ89tlnSEhIwKlTp7BhwwYcPnw4322qVCpkZ2eL/73ZJaig67W2toaDg4P4+PXnA3h1dcPPzw9mZmbiPJ6enqhUqVKB9lubmuLi4vDo0SO0atVKY9nPP/9c4/HJkydRs2ZN2NnZic+DVCqFn58fTpw4AeDVlaj8nqfKlSvD29tbfGxrawtPT0+cPXtWYzuvv1YXL15EVlYWWrdurTGPt7c3KlWqhDNnzgAAzp8/Dz8/P5iYmGg8X4cOHULNmjVx8uRJlC9fHrVq1dKorVmzZoiJicGzZ8/g7u4OExMTdOrUCWFhYTh69ChcXFwwYsQIlC5dGuXKlUP16tUxceJEjB07Fjt27IBKpUJoaChq1KiR72tQvXp1lC5dWnxsZ2cH4NVrnZWVhQsXLiAgIAASiUScp1WrVjAyKhq3dF68eBHZ2dkICAjQaJ8wYQJCQ0ORlJSU6/VxcHCAp6en+PqoeXp6iv+2sbEBALi7u4ttZcuWBfBqNNHXtW/fXvy3RCJBixYtcOnSJY2rNwWlzTGh9voxCUDjGM/OzoZKpYJEIkGbNm1w4MAB8Vz/7bffULVqVY19VCvo8aTtuaDm4eEh/lv9/vJ6Fy/6cKhf+zffsz///HPIZLJcXWlfV7duXY3HlStXFs+/27dvIz4+Hv7+/hrHu4+PD0qXLo3jx49rLPvmeUJUGNzc3DQ+L8uVKwdHR0dYWFiIbWXLlkVaWlquZc+cOZPnd6s2bdogKSkJt2/fFtsCAwM1tqPu1XD27Fmdvx/Quyka35KKgNfDAgBIpa/yp0qlwr179zBp0iScPHkSxsbGqFatGlxcXADo73c9SpUqpfH46NGjCA8Px+3bt1GqVCm4uLjA3Nw8322OGzcOW7ZsER9XqlQJhw4dyjWftut98/lQf8FWqVQAXnXNU38JfV25cuW02t+C1PT06VMAyLW9N7eVmpqKuLg41KpVK8/tpKeno3///hpf9nx9fbF69WoA/4aL19nY2ODKlSsaberaAIhfdPPa73LlyolvmqmpqXk+X6/XnpSUlG/tSUlJqF69OtasWYNly5bh119/xapVq2BpaYmvvvoKw4cPh0QiQVRUFBYvXoz9+/dj69atMDY2RvPmzTFlypR8u8f817GfmpqKnJycXLXLZDIxcBhaamoqgFd/GMhvWn6vz5v3670eNtXefH7yYmtrq/HYxsYGgiDkCmMFoc0xoX5NXz8mAeRaZsiQIRg6dCjatm2LxYsX4+jRo2jcuDH27duHnj175rn+gh5P2p4Laq8/r+pjTl/vp1S0qI+N8uXLa7QbGRnBysoKaWlp+Z5nbx7bUqlUPE7U5/eUKVMwZcqUXMu+3tUbyP05S1QY8vpcefO4zs+zZ89QpUqVXO3q99nXP2Pe/A6j/tx+9uyZzt8P6N0wZL2FIAjo378/jI2N8euvv6JmzZowMjLCzZs3sW3btrcu/+YVJW3+Unvv3j0MHjxYvAesSpUqkEgkWLt2LY4ePZrnMkOGDEG3bt3Ex3K5XC/rzY+9vX2um48B4MmTJ6hWrRqA3MFM7fnz5+KHnTY1qf/K/fr9M3k9trCwgK+vL8aMGZNnzXK5HFOmTMHz58/Fttc/dPO66TQ5Ofk/w5H6jSk5OVncb7WkpCTxzdHCwkIMi6/7/fffUbNmTVhYWKBq1aqIiIjIczuVK1cGAHFgg6ysLJw/fx4bN27EkiVL4OLigk8//RR2dnb49ttvMXnyZFy7dg179uzBjz/+CCsrqwLfKwi8epM2NjbO9VqrA1hRYGlpCeBV8H/9NXj06BH+/vtvAMjzWE1KSoKVlZVeakhNTdUIF8nJyWIQ1eY8yIu2x0Refv31V43H6hDo6OgINzc37N69G1KpFEqlEm3atMl3PQU5nrQ9F6jkUR8bSUlJGn+Rf/nyJVJSUnQ+D9Xn/pgxY+Dr65vvdomKqzJlyiApKSlXu7rt9XPnze8w6s899R8g9f39gN6O3QXfIiUlBXfu3EGnTp1Qp04d8VKseqQ99Ren17saqpUuXTrXDfLaDAwRExODzMxM9O/fHw4ODuKXNHXoyOuvvZUrV0adOnXE//L6gVBd1psfHx8fHD16VGPkmtjYWHHURODfv968/hw8e/ZM4+Z2bWqyt7eHg4MD9u/fr1HDvn37NB77+vrizp07cHR01Hgutm3bhl9//RUymQzVqlXTmPb6l8G7d+9q1JaQkIALFy6gQYMG+T4P7u7ukMvl2Llzp0b7uXPn8OjRI3h5eQF41WXq+PHjGl0yY2Nj0b9/f1y5cgW+vr54/PgxbGxsNOo7fvw4fvrpJ8hkMqxYsQLNmjVDVlYW5HI5GjRogGnTpgF4FSguXLiAjz/+GJcuXYJEIkHNmjUxYsQIKBQKPHr0KN99+C8ymQxeXl65RjI6dOgQsrOzdVqnvrm5ucHY2FjsYqoWFRWFBQsWoHz58rlen/v37+PixYvi6/OuDhw4IP5bEATs27cPdevWhVwu1+o8AP69mqOmzTGRn9fnr1OnjsZfONu2bYujR4/it99+g5eXV77hp6DHk7bnApU86gD0+sic6sc5OTmoW7duruNfG9WqVYONjQ0ePHiQ63ifM2eOViPLEhVlPj4+uHDhAh4+fKjRvn37dpQvXx4fffSR2PZm76W9e/dCIpGgfv367+X7Ab0dr2S9hbW1NSpVqoS1a9fC3t4elpaWOHr0KFatWgXg33uU1H1rjxw5gjJlysDFxQXNmjXD0qVLsXTpUri7u+PQoUM4derUW7dZq1YtGBkZYfbs2ejTpw+ysrIQHR2NI0eOAND9vgV9rnfAgAHYtWsX+vXrhz59+kCpVOL777+HVCoVg5KzszMqVKiAH374AaVLl4ZEIsHSpUs1uoVoU5NEIsGwYcMwevRoTJ48GS1atMC1a9fwww8/APj3y2mvXr2wbds29OrVC3369IGVlRV27dqFTZs2ITQ09K37JAgCBgwYgBEjRkAmkyEyMhJlypTRGG7+TWXLlkX//v3xww8/wNjYGM2aNcODBw/w/fffo3r16uK9OoMGDcKXX36J4OBg9OjRAxkZGZg/fz7c3NzQsGFDZGdnY82aNejduzcGDBiAChUq4MSJE/jxxx8RFBQEY2Nj1K9fHxERERg8eDCCgoIgk8mwYcMGyOVyNGvWDJUqVYKpqSnGjBmDoUOHoly5cjhx4gSuXr0q/sSALoYNG4bu3btj2LBh6NSpEx49eoTvv/8eADTu0zIUa2tr9OjRAytWrIBcLoevry/++usvrF+/HmPGjIGFhQVCQ0MxatQotGnTBikpKeJr27t3b73UMGvWLGRmZsLR0RG//PILbt26hZUrVwLQ7jwAXv1V/sKFC+JPE3To0OGtx4QuPvvsM8ycORO7du36z79eurq6Fuh40vZcoJJH/fovWLAA6enp8PHxwdWrVxEZGYl69eqhcePGuHfvHgBg586dcHd31+rKp0wmw4gRIzBp0iTIZDI0a9YMSqUSixYtQkJCQr5dbYmKi969e2P79u3o1asXhgwZgrJly2Lr1q04deoUwsPDNf44cfHiRYwePRpt27bFtWvXsHDhQnzxxReoUqUKbG1t38v3A/pvDFlaWLRoEcLCwhASEgK5XI7q1atj8eLFCA8Px7lz59C9e3fUqFEDrVu3Fru57dy5E8HBwXj69CmWL1+Oly9fomnTpggLC8PAgQP/c3sfffQR5syZg8jISAwcOBBlypSBh4cHVq9eje7du+PcuXN5Xql6G32u96OPPsLy5csxa9YsDBs2DDY2NggODsbixYvFLlAymQwLFixAeHg4Ro4ciXLlyqFnz564ffs27ty5U6CaAgMD8eLFCyxfvhybN29GjRo1MH78eIwfP17s22xnZ4cNGzZgzpw5+Pbbb5GZmYmqVasiLCxMHB74v1SsWBF9+vRBeHg40tPT8fHHH2Px4sVvvfdI/Ya1Zs0abNy4EWXLlkWrVq0wfPhwsTZXV1esXr0ac+bMwfDhw1G6dGk0adIEo0ePhlwuh1wux9q1azFnzhzMnj0baWlpqFSpEkaNGoU+ffoAAFxcXLBkyRL88MMPGDlyJHJyclC7dm1ERUWJV+SioqIwZ84chIWFQalUomrVqpg6dSo6dOig1euaF29vbyxcuBDff/89Bg0ahEqVKmHixIkYMWJEkbnH4ZtvvoGNjQ02bNiAn376CZUrV8bEiRPRpUsXAK+6hS5duhSDBw9G6dKl0bhxY4wcOTLXPSK6+vbbb7F06VLcv38frq6uiIqKEgdR0eY8AIBu3bohJiYGX3/9NWbMmIHAwMC3HhO6sLa2RqNGjXD8+PFcg8m8zsTEpMDHkzbnApVMYWFh+Oijj7B582b8+OOPsLW1RY8ePTBo0CBIpVIEBARg27ZtCAkJQadOnfDtt99qtd7OnTujVKlS+Omnn7Bx40aYm5vDy8sLERER7KJKxV758uWxfv16zJkzB9OnT8fLly/h4uKCRYsW4ZNPPtGYt2fPnkhISMCQIUNgZWWFAQMGIDg4GIBu7+f07iQC7zQmHagHAXl9ND6lUomPP/4YY8aM0ftfRnbu3AlXV1eN7n1HjhxBcHAwtm3bJg5EoquQkBCcOXMmz8FCSrqDBw/C3t5e46/CN27cQOvWrfN8oy9JoqOjERoaioMHD/7nPVJERETvi7OzszjAERUdvJJFOrly5QoWLFiAkSNHolatWkhNTcXPP/8MCwuLXEM468P27dsxb948DB8+HBUqVEBcXBwWLFgAX1/fdw5Y9N+OHTuGXbt2YfTo0XB0dERCQgIWL16MatWqoVGjRoYuj4iIiKjIYcginajvn1q/fj0eP34Mc3Nz+Pr6YsaMGXkOpf2uvvvuO7Hb1NOnT1GuXDm0atUKw4YN0/u2SNPYsWNhamqKxYsXIzExEWXLlkXjxo0xatQojd/9IiIiIqJX2F2QiIiIiIhIjziEOxERERERkR4xZBEREREREekRQxYREREREZEeMWQRERERERHpEUMWERERERGRHjFkERERERER6RFDFhERERERkR4xZBEREREREenR/wFyXvZmb5uXgwAAAABJRU5ErkJggg==", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# Plot tag frequencies\n", "tags, tag_counts = zip(*all_tags.most_common())\n", "plt.figure(figsize=(10, 3))\n", "ax = sns.barplot(x=list(tags), y=list(tag_counts))\n", "ax.set_xticklabels(tags, rotation=0, fontsize=12)\n", "plt.title(\"Tag distribution\", fontsize=16)\n", "plt.ylabel(\"# of projects\", fontsize=14)\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": { "id": "pfjVstecaFC5" }, "source": [ "> We'll address the [data imbalance](https://madewithml.com/courses/mlops/baselines#data-imbalance) after splitting into our train split and prior to training our model." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 335, "referenced_widgets": [ "af9c5bab12c64dc396c28154ea13f516", "7d1b4a63fa924fa6b136204ce1e67a42", "795b443fc1834645937b199e1214fcc3", "ccc7456ad5484dd2b7ccdd62bbc27d0c", "53f5b6e055864bb19eadba0aa640668d", "8a9678ac8f3e4af49c02181ce0eb6241", "8c6ffc9537344c709b47a5acea0e3075" ] }, "id": "NgMGuIQrNkSV", "outputId": "0e58055f-0482-4ae0-f6cf-e2a8c2a8552c", "tags": [] }, "outputs": [ { "data": { "text/plain": [ "" ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZUAAAD7CAYAAACi0gmlAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAAEAAElEQVR4nOz9d5wl13nfCX9PxZvv7Zx7enICBjMABoEACIAgwQBSDMrJkixbtuTsV9baft910sdJ9tperbT2ylagZJHKYgIIAgQJgMjAYDA5z3TOffO9levsH3Xndvd090x3z4Dk7ju/P4C51VXnnDpVdZ7zpN8jpJSS27iN27iN27iNWwDlez2A27iN27iN2/h/D24Lldu4jdu4jdu4ZbgtVG7jNm7jNm7jluG2ULmN27iN27iNW4bbQuU2buM2buM2bhluC5XbuI3buI3buGW4LVRu4zZu4zZu45bhtlC5jdu4jdu4jVuG20LlNm7jNm7jNm4ZtPWeKIRYcUxRIJtRUBRYyIdcPeV7laOvawliehqJxPVquH5t1fMEAoSClAEgMLQEQegRhO53d8DvI7SETuiHhG6w7LhiqIR+COH//xIpCF1H+v6NX1RFYAz2ILRo7xU6Ht747OrXCQGCdc+rkkkiLQfp+escNKjZNEG1Dn5w4/M3gURWJ91uIBRBZc6hVvTel36aEFGfvhvi1td/T0KAqiv4btg8ppsKsZRGZeEWfcNCoKWyqGYMGQR4lSLSf5/nYz3D0vQV41BjCWQYELrO5htWVPRMDsIQr1xY87T1ELCsW6hci5gp+NgTcT76RIJCIeD/928K7Nqu09ut8q3v2Jtt9iYgaM9spzWzjao1h67FuTL1nVUFRczIEjMyFKqjKEKlM7eHijVLqTb2PRj3rUdyIEe8K401U6E2VkRLGghV4FVdkv1Z6pNlAttHSxrIUBI6PmpMj86pOCiGipY0cIsWiq6iGBp+zYHvQzkkDD1amFUFQokSjxFaNggQhgGhRDoOSiqBdD2k76N1d+BPzyI9HyWZQDouUkqEoQMgrej9VeIxun/lr6C1pAFwrkwy+S9/C+msXFy0tizC1PEm5tY1br2jBW82v26hIjSN1MN3UXvjJHpVQVE0XLeKHzgoQkU3kgS+i6rquG4ViUQIBZDRcxOCMFx7UdQMhXs/3YOiCMJAcvnd4vsuVISAXLdJvehtSKgkcjqDd2Y58/J885iZ0mgbiFPJu7fkPY11D5DevpfAtggci/DyOfzvA6GS2r6X2uVzhN6iANHSWULXuSmhoug6qe170RJp5l59DsLNb1w2LVTu3GfwkccSvPSqxc/8WBohIs3lr/10ZlWhku0yKc2s76Y1Q0HRxIZeNIAwDFgoX2KmcJrdAx8jm+xH02LMFc8R09OkEl3U7QUGOg4TN3O01LYyOvMmQtHobt1PZ24XpdoEs8WztGV20JoewvEqTOdPYuppOnK7UISGG9QZnXljQ2P7bkJPGqQGc1RHC8Q6U7Ts76Y6WsCv5UkPteEWbPS0SWZ7O7GOFPkTk6S2tBJrSzL5wnkyuzoQAgqnpsnt6ybRnWHmtSs4C/Xv9a0thyKI7d5OUCwTej5qNo2Wy+JOz4LnYWwdxD5zASWXQe/rxh0Zxy+WMfp7CObzaB1tqLksSszEL5ZR00nUTIrqa0cgWP+7p3W2kP34BxC6Ru2tU7ij06QePID0A7zpBfy5AonD+xCmQeWbb6LEYyTu3UvlxSMAJO7eg5pJgh9QffUYxrY+jMFu/Nk89XfPkXr4LpRkHL2rFSEUUoku/NAlleymXp9HNxIEgYcVLJDNDKJpMcLAx/Oj56WpBqpqMjnzbkM7XwkhoG0gzjtfnmLsVBmIfm89lOOdr0xx8ONdLIxZ9O9Nk+uNUZi0SeZ0Js9Vae2NkeuNUSt4zI/WufhmgQd/pI9Uq8HMpRpvfWmSbffkaO2Lk+uKsTBu8d6zM+x6sJWth3K8940ZmHZItxvc/0N9aJpCqs3gz/7lGbbf18LO+1tBSo49N0s17/LwTwwwdDDHwB0ZXv6DUXRT4e6nunGtkJFjJSSw/d4Wdj3UilX2OfaNGTRT4eBHu1B1Bc8O+PbvjhD6q0sfoekkB7dTnxyhdvls83h65x3E+7bglYuUTx/FaO0gObQLoen41RKVcyeI9w9RGz6P0HQS/Vupj1wkMbCNeO8WvEqR4om30NMtJAe3IXQTpCT/7iskeodIbt0NYUD53HHc/Bwt9z6MlszgFuYpnXyHeN8Qbfc+Qrx3EHtqjPLZYyQGd5Ac2knl3Am8Uh5FN8neeS96Joc1NU718hnS2/ditHQgDIPa8AXqIxfI3fUARks7fq1M8fhbhI6NNTFMcsvOdb/3a2HTPpXBfo0z512+8S2ruTOYXwjoaFPo3pHkyV/cRiyloZkKiazG4U/3oJsKyZyOHlMwEirxjIYRVzESKnseacNMqgglEkCpFh3NUIhnNOIZDUUTxNMaiVx0fFUIga7GySR6EEKhXJ8iE+9GV+MkYm0IBHUnz2zpHPnqCCMzrxHKAFXRKVZHGZ19m5b0FjKJXtqzO7ky/QqOV6M1sw3TyKCpMYanXyUV6yBmZDc7de87KsN5Fo5N0X53P6ktrdSnyljTleg5KQI9ZRBrT5Ha2opiqBjpGG6hjlOoE+tMo+gqlct5hKaS292JljJQzdX3H6qpkuhMLB4QEGuJIdRFc6lQBfH2OMnuJMmuJGbGAAGKppDsTpJqHNcTkaaQ7EqS7E6S6Iij6ErUZs4k1ZMk3Zsi1d04N5QEpTKxfTshDDGHBlCzKYSmIgwdf26eoFDCL1dQW7JonW0gJUJVEDETra2FoFACKdHbW/Fm5pGOixIzNjTf/kIZ+/wo9ffOY52+goiZqK0Zqm+cwDpzBb9YofbGCbzJOWJ3bMebWSC0HBRTR+g6elcbtTdP4S+USD5wJ7Gdg9TfPYeaTpK8bz9aS4baa8cJKnWQEolEESquWyWZ7ERTYxh6EsNIoaoGjlPG8y2C0EXXEyAEdWthTYEC4DkhL39+lDs/3Mmn/uFOurYliaU0OoaiZ9s+kCDVahDP6BSnbRLZ6P/b723Bc0LqRQ+nHtDWH0cocPybs7z+p+Mc+GgnigKZDpNMh8mrfzTGiRdmkaFk+GiRWtElkY3ereqCy2t/NE6t6HL2lQUAZi7WePtLk0xdqLLzgVaqeZfjz80yfrrMi783gl31qS64jBwvkWrVQUTrx77H23nlD8eYvlBl9wfayHaatA8mePF3R8h0mLQPxNecC0XXQVHwK6XmMaO1g+SWncy/9gLS90kMbkfPtIAQLLzxLWLd/QhNw2zrQkvnMFva0VMZFDNGeucdFE+8jZbKEu8eRI3F0bOtlM8eo/De6xCGJLfuwpoaoXjyHdziPFKGVM6fpPDe66S27QEE1vgVnPwchXdfo3LhFADW5Ah+pYyaSAKQGNiKUFTmX/smif4hYp29xHu2UBu7RPnMeyT6hlAMk9rweQrH3kDPtqEl0xt632+ETWsqC4WAw4dMdm/XMQzB9iGdT3wkwdGTLpkOk3hWI9tl0tIbI9VqkMwZZDpM9j/eQTXvouoKSInvS2Yv17jjiU6sks/UxSpDB3MsjNVp60/QOhBHhpL8hEXX1iRGQuXCmwXGG7uppVCESkt6CwCXJ1/GDyzmSxfozO3CDz3KtUmkDAlDHxmGBKGHIjT8wMbxKgShQxD66FqCIHTxAwc/sImbOTzfwnKL+KETmRwUffOzvg4ITcMY6MOdnEY669Tw2tuiRUrUibUnqU+WqQznaT3QA0BttIj0A8y2JJXhBbTGIl6frqAYKoHtY89VUQ2VzK4OFo5OMP/uOGYujpNfXUsZ+tAW9nxmJy/845ewCzaqoXL4b9/D0f9xjOpUFYBEW5z7/8Fh6gsWMggJA8nJPzxNrMXk/n9wmOLlIp7lM/7aBDPvzfLkf3mC8dcmMFI6E29NMfH6JP0P9dN1Vyfdh7oY/c4Yoy+PMfPeLH6hjDc5Q1CuYB0/g9bZRpAvgiIQihoJEUPHm52PFvKYifQD1Ewad3wKvbeL0HYIZuYJa3W82QXkRn0WYYj0/Og6PwApCSp1wkodFIX4/m3oPe2o6QTeTCHyuyzRhIJSlbBuE9ouRiaB1pEjvm8roe0QOi6hZRPUbKTjIQmZz59dcyjV2nTz36piIFMhlp3HcVd+L0shBOQnLZ753y+y95F29jzczpV3C9F3CsTSGkLQECA+MgDPDlE0Qb3kIRSwqz5mb5zOrUnuerKTuZE6ue5Y0x87N1zDrvrIhivE90ICb1FbUDTB0F1ZKgseZ16aw4irHPx4F4oiyHabWOXoWs+J3qGrlgwJ+G7YdHUZcRXPCXGqPlbFp2NIQQiYH7Owqz5W2UePqWs/Ts+DMERP53DmpqKxmTEC1yZ0bQK7jhqLE1h1vHKB0LUj05NQqI9fIbV1NwD18csohomRayO9Yz+hE5nSFE3HKxcJrFrTP5J/52VSO/aTvfMw1YunCew6rfc8jLswi5FtRSiC0I38gKHvIYPIbCoDHxkumlAVM05g1wldh8C2UeMJ/HqVoF5FBgFShmjpLC0HH8RZmMVs64y+k1uITQuVo8dddgzp/J1fyLJnp87/9qutjIz7/ObvVWnd30695LPv0XaqeY+z35mnYyhBS2+M4rRNS2+cwAs58515dt7fSr3kMXupxsiJEkiYHa4RT0VazNTZCkIVtPTEiGU08mMWhUlr1TGFoc9s8SzT+ZPNY6X6BB0tu6lZc1hO5IDyA4dUvIPBzgeYmD8a7f6WaMJ1J0860c3W7odRVZ254nlUxVhXBEL60YepvPTKZqe1CRGPkbzvXoJvfht/PUJFgLltCCWZpPLtl6lPV5pO45lXrjRPm3rxUvPf80fGV23KLS7Ob/H0zNpdqoKuAx3MHJul+1AXw98aWfNcK2/x3m8fR4tp3PET+9Dj0as3f3qB9373OH598cMI3IC3f+MIXXd1svWJIUZfHuPi05eYeW8GI2Xw9q8faZ4bVqpYJ6JF1puawZtaOd6wVME+ea75/GpvvLvYV6G07Fy3Ul3zHtaElIQ1m8Q9e5C2i1+sLP5NgBIzUeImoesjFIEx1IO5rQ+hqtiXJ1jqBPBmiwS1yCckPR93ZJrYrkHSHzwY+XY24C8IQpdieXhd58ZSGnd9rItMh4luKFx5r0h+wiLdbvDYz22htTfG5ehmuXYQ134WmhmZr/KTNtW8u/p5Ag59opsd97fQMZSgmvfQDMGDP9LP+JkKsWQvJ16YJZnTCQNJ4IY4tegdce2AdLvJYz+7hdf+ZJxUq8FdH+2iZ2eKgx/r5sKbeaoLLh/8mS1ohsKldwp4drjuuZO+R230EplddxLvG8KvFKkOn0cGAe0PPoFimJTOvBdpKte0aU2OkN1/N6Fj48xOgRBUR86DIgCBXylhtLRf7al5XXLrbrRkBtUwUQyD0HPRkmm84gJ+vdqcPLe4QOs9D1Mfv0Lt8lkyu+8iuWUXgVXDr5axZ8bJ3Xkf7Q9+GKGqOLNTmG1dy8YphNJoO0/oWEgZomdayOy+C6O9i2ytQvnccaS3uaAHsd56KqtFf8VMQWuLQiqp4PmSQjFEmho7Hmjj9ItzPPJTg0ycLtOxNUkiqzN7pU5Lr4lnhQgFdFOlNGNz4oVZDn28m+KMw9SFKoc+0U3oh5TnHCZOV9BMhWx3jH0fbGd+tM7x52dZGFspWNSG9hA0HZICVdHY3vsYE/PvUbPnmpOqq5H66/p1VEVHyoBQhmhqDD+wURUdTTUIZYjv2wihIIRKEDroahw/dJAyXNK5ippN0/nXf46Z3/i/ooXGsqOdsq4jwxChaQhFIfTcaEerqghdRwgR7SI8L3p5FIESj6MkEviFQjPaR5gGMojaQYB0veaOV5gmSiKODALCcqU5JqEooCgIRYl2NW5jbhQFYeiRM1eADEKk664qOBU0FKFy9Q2QgC8dskMZ9v7QHoZfGGHbk0O8/h/eQtEVPvArDyzTVJKdCT7wKw8gdIEQgql3pjj9Z+dI96Z45H99CLtg41ZdTn3xDHOn5vn07z/FN/7eCww9Poie1Dn5hdPIQJLuS3Hv37qHb//Tl9bzyqKgoYr17cJCQgK53BGrJOP0/4e/ty5HPZqKmowTOm4UAGAahPXItyhMPdKQwjDSZoIQJRlrzLkHQiBtF6FHzxWhoBgaMgwJ6w5KItaMLgurFoThyv5vEkKJIrE0XSEMJVbZx3dDUm1GFNUpwa75qJpCGEiEAjKMBIhnByiqQIaRtuHZQUMYRO1W5l2MuApIXGtx7Imcjm4qyFBSL0cCI9WiE/gSGUpqBY9YWkMzlIaGEuDUAoQCyZZoXNW8i6oJ4lkdVRW4VohV8dBNlVhKJQzAqngIAXpMxSr7xDMarhUs05JWTohAjSUQmoYMAgK7jqIbKLrR+G0hGt9X6Dqo8QSBY0MYosaTkbZqR9q9YpgohgkS/HoVoSgIVV3mWG/2FYbRQh9KtGQqWjeEwK9VlrUVei6hY0fX6XqjPwsZ+NExVSX0PELXbp4PoOhmNN5ENEYhBL5VQzTuF0VB+h6Bba26Fryv0V+93Sq5rMK5ix6T09HC1tOlMtgvePMrkcr43P8Z7W3ES/PR+ET0bcgQDn2ii4tvFprRGq/+8Xjzby/81pUV/SWyOu9+bYowhJae2KpCJbgmuiVmpOluvZNyfbopUACkDJeFGy+NEPMDq3ls6XEpQ5DRi+8FK/s2tw2ReewRtPY22n7qR0FC/s++RFitkf3YhwktC629DTWbofrKG1hnzhG/Yy/Juw9GQsd1KH/rZdzRcbTWVlo+80n0/j5mf/O38OeiKJe2n/4x/Pk8WksOJZnAuTxM6dlvIjSN7EefILZ3F9aJ05SeeS6aswP7SRw8gHRd1GyGoFyh9Ow38YslknffRWzvbpSYidHfh3XyNMWvPktYX2nm2pN+kIHE/ugBApKQF+Z+h7bdbWimRqIjTrwtTqY/TXVm9TDu6kyVo//jOKEfcO8v3U12IEMYhEy+NcWx3z2Ot0RTUQ2VB3/5PhRN4a1fP4IMNh7OI1DYkT7M1sTBdZ0/54xwqvwiTri+YAQ1k4hMeXUnilABglpjwRcKoeOCpiJUBYIwCgUWItIeBQSFyspGpURNJ1HSCZREDKGp0TVBQOj4hHUL/KAprG4KqoKaS6Pl0pHQUgRhEGI7HkGpQiArQEj1mhBdj+UCzbVWmgqFrmGpaZRcAmEaxHsVpOcTVGowk4dGxFt9leiy4vRyrdwqr4yOkyHLxuW7ksrc8nG6VrBibL7rr9nminswdETaQE0lEHETQ402JzIIIr9bLYafLxM2rAiBtfjeBNbyb+DayCwZhJH5SlXR2rKomWiehKYiPZ+wUsebLeBXV5osr20rsOtwzetwbf9oAr2zI7oXQ4ssM270PvmFCsgQGdIUXDeLTQuVew+a7Nqhc2m4TND46NMphV/+21l++Odml53bFG5y8d/DR0tYFX9RLZOrCsYmRo6X6NqWJPAlc8Pr+/Btt8zw9KsbuKvNw7lwiUKpRMdf+xnmfuv3mjcjDANhGmiJOKWvPx8tNkEAQYBzZQR3ZIzQssk88Sjx/XtxR8fx5xfI/+lf0vLDn13Rj5pJk//zL6PEYnT8/E9Tff1tgkKB4le/TrpURkkml52vdbSR/+O/xF9YIPcDn8DcugV54RKxndupvvwa3vQMuc9+ivqJ06sKlKW4qq1KCZqpkR3I4FQczIyJXXDIbcutKVSuImy8K0IV0Pjmr33svhPwyr9+nX0/sofWXS1UJiubChMVrK5h3woY/Z2IuIE/X0JNJlBSkebhzxYwetvx5kv4CyX0zhaCSh29uxU1GcebLeDN5AlKi/Ok5lLE9m4ltmsQvbcDrS2HmkuhmHqkxXg+Yd0hKJRxJ+ewjl2gfvTcDYWLmkmSfvxelGSM0HIoP/cmYc1CbUmTfvQeYnuHIl9PNoVQVaTvE9ZsvOl5nItjVF89jjs6fd0+lkFRMHcOkLr/DsxtfWjtOZRkHKGrhJaLP5fHvjiOtFc350rXp/buWdzLE8uOpx4+iDHY1TgJCn/2wrpCsYWpk3rkEHpnCwD+fJHKy0eR9tpmHa2zhfgd2zG39aH3tEfPIp1YDDf3PMKajZ8v4wxPYh09h3X6yqIFYD1QBMaWHpL33UFs1yBaRwtqKo4wNELHw58v4lwaJ6yu/j3KIMQ6fhH7zMrN97X3n7xvP/E7d2IMdqG1ZVHiJoSS0HbwCxW8iVmsExepvXWKsHZrUkE2LVRMU+B5silQAIrlgJbc+swNhamN3YBTCxg9cX1n44oxKkl2pu4jpbUuOx4SMFo/ybR9cd1tGUqcOzOPoyvLo0YkkpHacaadtdsSQmBfuERQXG6/1zJpkofvRm1pQWvN4Vxe2ydxFfa5C4TlCmG5gvR81HSKoLB2spI7NoE3M4u0bYJCETWdujqo6EO5upv2NhaDb2ZNVFPl3F+cpzxeYeDhfjKDGVRDJdmV4MFfuQ/f8hl/fZLJNydJtCd46J88gAwlxcslKuMVEh0Jug918eiWh/Ftn0vfuML4qxNIP8SzPEZeGuXgzx9g+sg0TjnSaGWwftOPlCFhI8GV5n8X/3UzAkcGIXpLBr09R1izEDETggAtk0Brz+EXq4DA3NqDn6+gJEwAlGQs8pkAKArJw3vJfuJh9O42lFQCoawck1BVlJiJ1prB2NZH4uAuYvu2UvizbxEU1v4mlFSCzIfvQ2vPEVRq2KevENZtWn/q48T2DKEYy4NNhGqgmAZaa4bYrkESd+9h/ne/in36+osXRNpJ6oN3k33qIfTO1oYPgebmSkmYGEO9GEO9a857ULPw5oorhEry3r0k7tsfmYnDkOKXXlyfUDF0Ug8eIL5vKwD2+VGqb5xcVagIQyfzxGFSj96N3tGCiJurjlNoKko8htaew9zRT/LwPiovHqH4pZfWJ1gUheT9+8l99nGMnvYovwoW5ylmYAx0YQx0rTlPoecT1u3rChW1LUvbj3+U+IGdKKn48rZUUHUNNZ3EHOgifmAn8YO7Kfzx83iT68uzuh42LVQmp3w+/Gicuw+YTEz56Lrghz+d5Ojxm8jqvMVQhEpKayVndC07LqWkpM8xa18hZH2RPnElTbu5ZcWDllIyrTZCaq+zm5bO8hdZxGO0fPZTVF55A+vp50jefy96Z/saVy8itJaY3qSEGyyMkZ8kXHK+QlCt4lweJvuxDxOUylhnzmFfHr5h30tRm61x5L8eRTaCAcZeHUe8JpCh5Lm//02aprIwso9/65++tKjpNI45ZZdn/uY3Fsfa8BU8/Te+ARKKV0q89M9faZq/KpNVXvoXy4MghCqi7/GaTHZJyIXq24zUT6ArMQwljqHE0JUYMSVJb3wXcTWzoXteCvv8KPaFseg+l6rYVx0QjfGUnn978b0QROde/R1GzmO9rwM1GUdKSWg5hDWL0HIWs/41DTWdQM0kEaqKmkqQfuxeCCXzn//aujLshaGTfOAO9P4u4vu3QRjiFyuElXpzgRYxAzWXRombCE1D7+uk/ed+gNnf+BPckanrtp984A7afvJjiEY4dlCoYF8Yw7k0FmlHmRTmjgFiO/pRs9HGRkqJdFzCmk3oeoSlKmHl+pru+wXp+YiYid7TjmJEPtCgbhPWLKTtNiMChaGhZpKNDYCClkuT+/SjhDWb0tM3DtCJ37GN9p//THOTEZZr2JfGcS6MEZSrqKkExrY+YjsGUFszkSCVEulGGpJ0PcK6RVBc21Sl93fS/rOfIrZ3qGFyDfEb0YjNZ23ojfuIoyRiJA/vQ8ulmf+dr9zwWd8ImxYqx065vHvc4e/9QhbLCTF0gWVLfu3/KN7UgL4bEEKQUNNoiom7Tht6Vu+44c42sq8K1NYWgnIF/LV3U0JVEYaBNzOLMAxiO7YRlDemiW0WQigoqRR+voA3ORV9HC05/PmF9Tcir3HaLfkdJZVJtKRB+939OPk6lZE8fnXlDjH0V2oeS49d609Z9ltAarAFr+pgz61cjCQhTlhf4SfRhEFW77opoRIFQa2MhFphww2Xz9G1cC6N41wcQ+tsxTk3gnNlEndsGm96gbBaRwYhSjKOua2P1MMHSd63H8XQESIyC5W/9TbulckbDlfoOukP3t1kIKi+eoz626dxhqcIytWIlqQtS/zATrJPPhCZ94RA72kj9chBClPza+7E1WyKls8+HplWAG8mz8Lnv0b9xMXlAk9VSX3gTlp/5CNo7bnI+X9xnNLXXsGfLxIUyrfGX7QZSEn9ndOkHr4L6XjYF8dwrkzgjU7jzRUJa9FmTs2miO0cIP34vcTv3NnULDMfuZ/amyfx54trdqEk47T80BMoiUgL8ktVFj7/NPV3Ti/XvFSFxIGdtP2Vp9C72wDwpuYp/OkLeHMFgkJlTdOYkoiR+4EPEts7hFAUgppF7Z3T1N48hXtlIsp1QqDlUpGp8uGDJA7sROgasZ0DtHzuQ8z9tz8jtDavHGxaqFRrkt/9wwovvmLTklNwPcnImM9CfvORKe0dCluGVGLxKPzuKubnAs6dWSdH0jqRUDPowsBlvUKl64bnhNUa1snTtP3o5wiqNYpffjqKAFsF0nFwLg/T8pmnCCpV/HyhaTJIPXg/sTv2YG4ZpPWHPo07NkHlO6+t2a85tIXk/fdibt0CqoLW1rIY1rzCUSXRWlvQ2lpwLl4mtCz0nm5SDx6m9M0XmxQltwJ6JkZqSwuV4TxCUeh6eCuKrlA6N0dmWxtqXKc6WiA11EpgeTgFCyEg0ZOhdH4ONa6jaAp+3UOogsz2dhaOToAqaNnfjT1bRTFU9EyMWHuK0rnZpub0/xT4+TL5P34ehMCbmkeu8jGHlTrWsQu4w1MocZPE3XsQQkQ283v2rk+oKAKRiCHDkNJXX6H09KvNhbI5lpk8lRfeIqzUaPvpp9BaMwhVJbZrC1pbFm9qftW24wd2onVEfgspJaVnX6N+9NzKE4Mgopnpbif3mUejCKtMkqBSw5uYXXn+dxnuxCzzv/1lgnINf7awqhAN8mVqb57CGZ6i6x/8BOaWnihyKpsktm8r1ZePrtl+bPcW9N6OpvZR+fY71N48sZIvLgipHzuP9vUW2v7KU6AI1HSS0HHxxtYO8QdI3LuX5OH9UbSn71N+7k1Kz7y6Qgj5C5HPz74wRsdf+wzxu3ZGm+27dxM/tJvaa8fXP3HXYNNCRRGQTCpYtsSbjwRJNqMSjymMT25MACgq/MBn4/z1X0yiaWLFOvjStx3+7b+8+V18KAMECkII4moWQ4lTC4rruFKQXWJCW9rO8g5Cis88F9mqJYR2FFJc+NLTyGtoP6TnU/za16PILykj9bshVGpH3qV+7HgjVA5kGEWcLPzBH0cmkQZmfvO/R+YtIXBnZqLwYQkgCR0Xd2IK69SZ5sdRfiEKxY3fuQ/puFTfOgJhSMyySdx1RxTSuNFJve6sQWqolZb5GoVT02R3d3Dxfx5BIKgaGi37uuh+ZBulC/OYrQnS29pwizZOsU7/J/bilWxkKFF0hdp4iZnXhxn85D4Kp2dwixaF09N03r+F1jt6mPjm+e8LgaKYGqEfNKK8RCOkWyAbkUcrWFelbAoFM2vSeagXK283zYNXUZ2uEZSqFL/0EomDu6ARjWTuHlz32KSUWMfOU37ujRUCpYlQYh27gHX3JVIPH4y0ld521Fx6baFyx/ZmBJy0HGpvnlp7DJ6PfW6EoFhBa82i97QT2zmwwo/yPUEo1+U/gkgAl55+lY5f/MFIwOsaxlAPXE+o7BpEMSPzoHRcam+dWpuANJTY50fx5wro3W2ouRTxAzuxT19e08wudI3cpx5pMkLYF8cp/uW3r+t/ChZK5L/4DXr3bYsiw1SF7McepP7OmY0FHyzBpoXK1iGNf/iLWQb7NPbtMbhw2WP3Dp2vPlvn7/6TDZhRgFhM8GM/meC5r9t8/as2tfryWbPqt2axKHrTpLV2dGGiKTpJLUfBu7H9MKakiCmLUVV5d4I2Y2D1k32f8Bqzl3RXjzaRnr/sgTcD4Vxv1Qd6bWa9tBe1CmkFq75rS4WZbDjj/dk54nfsI/XgYQhCjP5e3ImpNbWqzUICC0fGGX/2LFrSwCs5hE5AZlcHnYcHcCtOpKLbPqEbRE54AYqqMP/2GOmtrVFOhBrlaCR7swR2gHQDfNsndEMkEHgBsfYk1bHi95R9WcsmSO8fwF2oEDoe0guQUhLf2ol1ZRYlpqOl4ngLlegce/kzlqFENVSMlE4sa5LpTyMUgZW3OP/0FUI/xB2dws+X0RuagZZJrXt80nGpvnGSoHT9BM/QcnDHZqKNjqGjJuOomcSa52tt2aZhwZ8r3nAxCio1gqqF1ppF0TW0thxC19bP2Px9AufKBGGlHvG2KQpqau05AlBz6aZjPihUbmhiCi0bv1hB725DNMKPhWmsGb0WP7gLrTMKSpJSUv7G6+uaU3dyDufSGPG9WxFCoLXnMAa7cC6unhx9I2xaqOzfbTCfD/k//nuef/aPWviVf7HAhx6J09218SZVJdp4ffMbDufPvX8vlhVUSao5ILL9ZvQOhHUOyfVNdhm9rcH4GqHkza0tVP4fAHdqhsqLr6B3tiOB2rvHcMcnr+sD2lQ/RYv8iWgXHjg+c+9ELNDWdJn5oxOEXkDp3CxO3qI+UWzs7AWKqeEWLerTZWQgUXSV0PExWxNMv3yRwA2aPo3S6RlKp2fQkhvj63o/YLSnMbqzaC3JKLy4UCX0AhRDI7G9Gy0diyjsUzH8irVCqPi2z8KFAl7Nw8wYzJ3N07YjR2WqRng16i2UUahyR0tzh0wjF+ZG8OeLEYvyOuRuUKpGC1IjQkyJNxIwrzUjNMawGIQRcMMO5PL8AaEqNww4+X6E9PzIuZ5JLj4LRay5sRG6umSewuvnUMASv13j+obmu9ZV8Tu2R7lNQFit46xX+5MSd3SG+N4oSk4xDYz+74FQUTXB3HzA1EyAZYXU6pKnn6vzh7/Vya/9+sbaCgKYmQnp7lY4e3pDBLEbghfaOKGNoSQQQkRCBXHDbyyjRecBBNKnvi6T2fcxggB3dAx39P2l+g8sj9p4FEYt/ZDqcB4Ar+xcl/5lLdSnVppA7fmGg37+exM1tBTW6DzOTKmZ4EgYsSM0w6Ab5kxEVJvlWoReSGUi0iLsYrSLrc9ZBF7QXKclrMjoF+LG7zBAUKziXycEeSmk5y9fHPU1lgopCar1qHSAEKiZVNM0txaUeKzp1JcNipsN8619PyCUSO+acV+N/lvt9JodZcgrCmoyHgmh60AxIy0RGvNku2tqgcI0MHo7msLZz5cRihIltt4IqkK4NHdIUyPta5PYtFAZG/fobI8KdA2P+/yrf9yClHDq7Pr4YmIxuPf+aHepCMHIFZ+/+XdTDG3XuHzJx3MXH8zcbHhLHPWq0Ch7s6QbeStprRVV6I1chtUhEKT1NkSD0LnmFwjl5oMRlkITBoaSQFcMVKEvE1xe6OCGdTx560O0VTRMNYmuxFCFhkBBEuCHHk5Yxw2tZdrbrTYoKWiYagJDxFAVDUFE4RHKAF+6uKGFE1q3tufr7CBvFaQXRALgZqBGC4GSjEdRXrqKtpRux4jCizeD0LLX9qWsimu1kpWHAOyzIyTu2gWqQG3NENsxQP3dNUgvVQVzqAetJYq8C6sW7tT8+0I9c7MQmho9h0QMYWgRPZKmRr5PRUFryaDEzHW3Z1+eIPXIQUTMRMmmiO0ajIq+rdp5lGCrdUXRX9Lx8MZn1hS+aks6SjRtCBVjsJv+//QPNnbDV7terzBaA5sWKifPelwZ9SmXQz7/xSpPfSSOogi+8uz6dozZnMLf+YeLlMsyjDSUJz++8mZef8Xl3JmbpxBQhUbBm6aPPUC0uKX1NvLu2mqiqSQxlWTzYVX8PEsj0zYDU0nQbg7SavSRVHPEGgu8goJE4ocuVlil5hcoulPMOMM44c3vxAUqrUYPXbFtZLR24o2wagWFQAa4oUUtKFBwp5mxL1MLoqTKkODqBvumoKDSbg7SZvSR1tqIa5lIsKEikQTSwwkt6n6JojfNnDNK1V9A3qRwUeI62YODVM9O4xW+9xrNalAySeL7tmJu7UPrbIn8DckYwjSiwI9GkqrQ1KZTfKOQfvC+aAT1t0+RfeohtGwKIQS5zz6Gv1BaNd8htnsLqcfuiXwoUuJNzOJc/P4qjqd1thDbMxQJv44W1JYMStxEMY2IvkVVouehNYT9OmEdv4C/UEbvbUcIQfYTD+NNzGGfW5n0bAz2kPnEQyhmFMgTFMrUj6+dYB2VUVjUEG+KSUKIxaTMTWBTQuXDj8Z5+XWbhXxIX4/KxJTPf/3djS368/Mhv/x3ius6t77EUZ/sy7Dthw8soQyJomwCy+P0f7t+4SxFaJTc5TuDjN55XaESV9MYSqzZV9Vf2PTiKlBoM/rZmjxIRu9AE8aKhy8AQ41jqHGyeged5hDd8Z0M195jzhm9of9nLegixlDyLnrju4gpqRX9akJBU3TiWppWo4+e+E4uVt5ixrmML11uVmtIqDl2pg7TavRhKLFlPiqI7lsRKroSI6nmaDP76Y3vZqJ+luH6sU3f99W2E4PtuAu17zuhInSVxN17yH7sA+i9HdFuU1lizpLN/yz5vUmE8n3RCLyZPOWvv0rLD38EFIG5rY/OX/oh6scv4JwfI6zbKOlElKV/aDdaV2sU8Wg7lJ597bq5Hd9NCFMn/eg9ZD50L2pbrplPsvgsVj6Hq2a/9SDIlyl+9WU6/vpnkIqC3ttOx9/4HPXjF7DPjhBW6ijJGOb2fhKHdqP3d0brWxBSfOaV62a7C0NfZnaUYdjws238hYm4yTb/nmxKqPzqP2nhkz8xzUJe8oXf6uRHfn6WmdmN7YACH8ZGF6/RtNX9xLq+3Mdi5+uMPn2Wnke24lYcimdmSfSkSfTcOJFNFRqetKkHRZJaFD3ToncxfJ1rEmq0m4aImbfml9CV9au8S/vuj+9nZ+pwZOq6NjN/ycMXS8SWphi06D2ks21crh1ltH5iBZvujaCLGHszD9ET2xm1vgorADQcgQhUoZFSW9iffQyzkiCUfmNR33jdBYGg3Rxkd/pBkmrLqh/g0v6v/l9FI6nm2Jm+n7TexrnKa+sme1w+AEFiWwfWeB41tmnFfBl0AwxDEAZg23JNf6tuRFVZ1/IRClMn+8lHyH78A5GJhUib8ItV/GIVb2qeYL5IUK4R1Bv1VAKf1h/6MMZg9y25l1sCKSk//xYiHiPz2D0omST6QBfZ/k742JLJURqMskGIv1Ci8OffovbW6Rs7rDcNEWkV64CaS9HyIx8h9dBdzcCD0HHxi1WCYgVvah5/oURQqUVsB46HiJu0fPbxJrfYelB77ThqNkX2Yw+iZlNo3W1kulrJPHHf4klX5ymU+IUypa+/RuXbR647TzIIl5l3reMXmfu//mKD5s5r2tskNvWVqUuu0nVx02YRXYdP/ECcE8c8Ll9cLlk+9lScsVGf996NFtLA8ildmGfLD+zjyl+exClYFC/McegfP37DfhTUKDnLm2sKlaSaQxX6qgu1QBBXM2iiEfcd1LCDKqayMSeWgsZAfD87U/ejKouTF8oQO6hQ8RdwwhqB9FHQMJQYKa2NpJaNKOeFQBcmO1OHEcBw7di66WUECltTd9MT27VsQQ9lSD0oUfbmGj4UiSZ0EmqGtN4eUZuIGFtTd1O7CZNfq9HPnvRDJLVc85iUknpQouLncYM6gfQQQsVQTBJqjrTejiKUppDrjm0nJOB85Q3ccKMficSaKEAg8a3N1Ye4Fh0dCg9+wOTcWY+Ljfc1ZgpKJUmuRcG2JbYlGRhQmZsLqdck8bhAUaBcXvzwE4f2kPvkwxEtvpT4xQrVV45RP3IG5/LE6uGgmkr24w/dkvu4lQjrNsUvvwRBSO5Tj0TmraDBu6Y0SDHLNt5sHufSOLU3TkTRSe+bQGkkfBrrWOI0lfRj95J66K6IokVK3Ik5qq+8R/3IGdzJuVWj67TOVsJPbOxZSM+n/Ozr4Ae0/OCHUBKxxgIuQWmQelbtiFTy8gS1N09F5rEbLPLScZflsIl4o9TC9yBMe9Nbt1sZAGiYgh/+8QSzMxUuX2M23LFLY+9+rSlUriJ/bIo7/vZDOIU6RjZG+VL+xmMWkc+i7M/RI6MMUlXRSapZyv7KxC5NGCS0Rf4dJ6zhhDWUddbouIqO2CBDyYPN6ySSMAwYtU4wbV/GCsp4oYMkRCDQhEFcTdNuDrIleQBDxBsLrMJQ4i6csMaEtUrG8ipoNwboi+1adiwIPSbsc0xYZ6n5RXzpARIFFVNNkNbaGEzcSZvRT0xNYirxZqDCRhBTUpGG0hAoUkp86TBeP8u0c2nZfUM036aSpNXoYXvqMGYjSk8RKt3mdireAqP1kxs2hcX7Wknv7aV2eRZrZGM5VKthfj4kkRRMT4dkswqPPGpy8oSH6wYMDKrs3q3x539qMbRVw7F9Mhm4406Njk6VP/vjOrYNCEH2qYcWHb1+QPFLL1F56d0VPHFLIYRoho1+v8Ec6o14xRQFbzZP6ZnX8GfzEdNyEEQFzEpVgoXSd2exUwRKYu2ywVehJuKkH7+nKVCCQoXCn36T2pEz1+dVE6xbE1oKvaed+F27ELoWJbQ+82qUKX91nhwvyupfKK5eu2cVhFUrym9rmOP09uz37D3ZlFDRNcFXvtBNEEBXh8pf/n5Xc+4vj3j8zC9tjOlSUSAeF+QXVi4WU5MBh+5dWbp34tuXWDgxTbwziVuyo0qHN0C0MErK3gIhASoamjBIaa2rC5XGzvkqrKCCE1ooG3CCxdUMWxIHmgskRIv6mcorTFkXVmgcEoknHTzfoeLnKXoz3JX9CKYaXa8rMQYTd1L2FqisMualMJUEA4n9GMpiVEgoQ0bqJ7lYfWtF3yEBVlDBCioU3Gnuyn2ENqN/w0IUIq1we+reZqQdgC9dzpZfY8peed9X/+4HLnWrSD0osz/zKHE1EuqaYrA1eZBZexgr3AC7ggR7skjL4a0o+q35yFwnMnvV65K2NoVCPuTCeZ8tW1T27tPYuTNiJlAUSGcEYQiFQoimCdJpBdsO0fs6ohDQBrzpBcrfevvG5JAiouz4foPW2ULrj38Uc0c/Yd1m4Q+eof7uuVviw1kRXLDOxVKYEbPzjWBs7WnSzADYF0apvXP6htqB0LQNR0kpyRhtP/VxYvu2Qhiy8MVvUH3lvXXlGV0PQbmKP1vA3D4Q5UK1ZCN26vx3h09wKTYlVH7u78yhaasvrJa18ckJQ/A8SU+vuix0WAjYvlOjUl6pIgtNIdGdwilY1CbLxFoTWLPXzxS+aqhzwzpWUCGltaAKrWkKuxa6MElqWSAK840ivzairgs6jEFa9J4li3rASP3EqgLlWkhC8u4k56tvsDf9MJoSOfYzWgfdsW3UqoXrtpHRO8jqncvMXiVvhku1d27YtydtzpS/w90tTzXnYCNoNXppNweaDnmJZMw6zYS9dn31q5BI8u4k49YZtifvQRXRpiKmpuhP7OVC9c0NjSX0fMonx5GrkFduFqPDPoEvqddCJsYDAj96j2dnQlzHI2YKhIDuHoVLFwNcV2JZ4DjR+6O1ZJZF2DiXJ9bFNqy1ZSIyxu8zxPZsJbajP+Iwm5yjfuTsLTNtXUsyqeXSuJUb+9fiuwdvmAsCoLXllnTWoM1ZxyKvd7VsOJ/D3D5AbP82hBB4s3lqrx67aYECgATrzBWS9+0HRUMogvQjh3DOj9582xvEpoTKeydcBvs1VBWujCwKAUMXPPpQjPZWlSPHHarV9b1Unis5cczjJ382iRBw+aKPaQruvd/goUdMfu3frJS2fY9vJ72lBa/mcPGPjrH1M/s5/VvrW2y80MEKyqS0FoRQiKvpVf0qV0NuAUJ8qv7GTCcKCn2JvcsW9aqfZ9q+tG6fCEjmnVHy5iSdsSEgMoF0x3YyWj+5pvNaQSWnd2Msqf8SyqiOzHod/VZQYca+xNbUITbiOVPQaDMHlvmeLL/CeH1tTqhrIQmZc0boi+8mqSwK/a7YNi5VjxCyfvOJUAResU54szkkS/DmG9EcOo5kYSEay/CVgOHhxUTF579xbY7Rdfpf5wKc+uDdNxXu+b5ACIy+xcQ72eA9u1VCZVl0mBCYu7bg3oBYEU0l9dBdm+twPeNWFFKPHNpQSDFEuSfNbgJ5S5kE6u+ew//MY+iNTUfinj0Y337nu86rtmmfyqE7DX7oU0nK1ZC3jzr8yZdq/MLPpHnwcKQOvnPU4df/exklrrPrw/20DkVhrMNvzHD5O8urybku/OHna/z9f5Tmn/6LDIEfzbWmC/7w8zVefXllAmDL3k5Gnz1H1wODyCAk0bt+GnMvtKn7JaQR2R9jagpTiVMPli62ItrlE/lTgtCj4t3Yb7MUab1tmflHypCSN0vV31g7TlhnwR2nzexr7toTaoYWo3fNQmOaotNqLC+IVPOLlL31myZDAoreNG5oYyo3tk1fRUxN0KJ3oywJG553xzYcvVX1C9hBlYSaa96HqSRIa62U/PWz2urZBFrCxK/fZCJpw4YuN7KoS7lq0qVfKC87bgz1RCGha4aKCeJ37ST9yMENDvq7ACkJKrWmPd/c0k3bT3yUymvHI7bfa+/pKv1IGEamrRss4vbFMZbWDko/cpDaWycJ19JWFIX0Iwcxd66PbNNfKC65VmBs6Y5sl2uZ7hSF9KOHSBzava72lyIoL4a0612ttP3sJ6m8eARvJr/Sz7R0noLghsm7QaFM9cUj5D73eJS1n03T+qNPsvD5r+JNLdxYWKpKpNlJruvXuxE2LVTiscg+/NyLFgf2Gzz1pORDH4zzS788jwD++f/SQiYlMAbShH7Im797DhlKfHv1j2b4csA/+rtFduzSGNii4diSc2c8ZqbDVeeiPlkmvaWFeEeKvid23tD0tRQhAbWgRCh9VKETU1IYSpJ6sKgRCcQyuvtqUMSTGyNcbDcGljm4femRdyc3lXNR8mZwgjqJhilKCEGnObSmUFGFTuoas17Zm29kqq8f9aCMHVQ2JFRMJbWs2mYkTKcJ5Macs5KQql+k1ejjamiIIhTSevuGhEroeAhdxV24uRwVNZcm+wOPbohHyJ8rUHnx3RXHvck5vJk85lAPAEZfJ9mPPUDl5aPLF0s1KgQVv3Mnuc98ELU1i3Q9hPm95zpbivrR82Q+dBituw1hGmQ+8RCZTzy0irVYNhzRVdzRGervnsU6fRl/rrjmoudcmsAdm8XcEoVRm9v6aPvxj1J69nW82XzkzBYRVYnWkiFxzx6yn3gIYeiErreiwuW1cIenCAqVpv8ltncrqQ8coHbkzLJSBELX0NqyJO+/g+xTDyN0bV3tL5un4xfwxmbQB7pAVUg/fi/px+9dfZ5cj6BSx5uYo/7eOazjF/Fm89c1l5Wee7NZHRQhiN+5nY6/9SOUv/4qzuXJqEbPVTOrpqIYOkoihtqaJb57C4l795L/w2epv7e+QKDVsGmh4nqSv/xajf/5p1UeGYtxz0GTeExQq0vKlRBFCFRVEPghyfYYnbtzhH5IebJOcXz1j9vz4MwpnzOnbrz4jD57jr7Ht+NVHRJdKa58af2mFYC6X8KTDio6ppogpiZgiaKiCJW03tb8XXQ3UKu7gYzesex3IH1qfnHD7UBkirqWsiWjd7IWd0ZMSTVNdxD5KaywQiA3tgNxQxsv3JgwTWpZNGVx0fOkixNsLl7eC+1l2fwCBVPZGE2JX3cxe7K4+epNJT9quTQtn350Q9fYZ66sKlQIJaXnXqfj534gyovQVHKfeQxz+wDuyBRh3UJoGmoujbGlh9juQZSYiXN5AvviGNknH9j0fbwf8GfzlJ59jZbPfahZ2RFYJUxUIOImStxE72ojcXAX9oVRCn/+rTVp56XrUX7uDdp+8mORY1zXSD1yCGNrH87lCYJKLaIWScUxB7vRB7sRmhr9LV8meXjfdcce1m0qrxwl99TDERtwLk3rjz1JfP823PHZSIgbEZuyub0fY6gHoWtYxy8gPZ/kvddvf1lf1TqFr7xM2489udw3tto8xUyUmIne0UL8zh04VyYofumlqFbNGlpUWKlR/Itvo6YSmDv6EYpCbFsfxi98Fm98Fn+uGLEjN8qJq6k4aksGrbMFpcFycLOhvZsWKtOzAU/9ZIK2NpW79ht0d6n092oc3G9w/rLXyHGSOAsOtXmbRIuJDCRWYZX60ALa2qMomvVuAt2SzfBXTqOYGtIPN2wvrwclvNDBVJOoQotMLChNLSKp5pr5KRDR5m8EV3NcliKUUXTVZuCGVhR+uySD11BMDCW2au5GXMss84OEDRqWjVKe+KGLtwFBJBDLIuYANKGzO/0g2+U9G+obIuf8cn+OWCYs1zUmVcFsT1P5PmPCrb91msr2AdKPRn4SNZUg9eCdyHv3RuYOpUHL0hi3fWGUhc8/DapC6gN3oabWrz2+n1BSCbKffIjEoT2N/AhJZENZ4wLR+I8ANJXY3q20/9wPMPVvf48gX1p5vpTU3jqJ0d9J+kP3RlQpmoo51NPU9BZPjTr1xmaY/+0vR9np9+y5ru9Dej6Vbx3BHOwmfmBnswpm6tG7wfObJJBNUk0J9aNnWfi9p4nfuYP4nTuadVKuBxE3yXz4PlIfOIDSKB+9rLz0qvNE9PwVgbm9n7afeQp/Nn9dn5JzZYL53/4yrT/+UeIHdoAQKLqOubUPc2vf6nMgZTSe8DrjWSc2LVTeOuJg6ILHH4nztefqvHnEIRYT/Lv/tZWWFoWvPlunXA5REj56QmNgbwvFiRqTJ1b6ExJJwX/5ry381m9UefnF9dm9W+/opnRhjlhbgo57+ymenaN4dv0mkSg8uE5KtoKAjNaOIlSCBllkTu9aQvDoUfauH757LXRhRmSNS+hkJOGmCSIlEi9cfu3VXftqQkUXyxfeq2SNG0WIH2XUr5uOQhBTl0fEXKv13QwiOpeNOUdD28OZq0Cwsa9FhiHu+GxUbneT8GbW9p+FNYv8F57Fm1kg/cG7o/rw5lWer8i/EloOYaVO/eg5Ss+8ij9fRG3NUH/3LMaWbvz54vKyzkvH7/m4E7MEjaxqf7647gUjrNm4Y7MoychHGlbqK68VYGzpof2vfhpzWx9IiTezQO31E9gXRgkq9WWZ2Vfp4dVMEqO/i+R9+zC29iIalCXZTz5M/vefXn08VYv8Hz2HOz5L5sn70XJpRMxscKFFWfpXzUXWyUsUv/Iy/mwexTSwz42gJGJ40wtr7vD9mQXm/ttfkPvMoyTu3ouaikfUJ7qGkDKiLqlGteGrrx2n/M03IzLM4UnsU5dR27KN+V19gvWedtp++inid24HIfDnCtReOI196jJ+qbbM7yREI1w5ncDo6yRx925iu7YgNBWtPUfus48z++t/dJ2HJ3FHp5n5z18g9dBdpD54CL2zBSUeWyyVIGn4tPxG3XsHb2qe6hsnsE5fXrvtdWDTQsUP4NW3bM5d9IjFBKoahQX/k1/NU62FzM4FeD707E5Rm7P5+h+/Q9eeHL13tVGaWG6CUAQYBszNrV/bGPzEbs7+TpnuB7eAgO0/coAj/+qb675eElLx52lr2OvTeisKKkHDBpbVF6M0yv78hqKNIOIZuzZiKpDrqDVxHayM2hJNx/21WKplQXS/12Njvh42ct3V5M3vJwhdxZ4s4JU2FiggLYfpf/M779OoIoR1m9JXv0PtzZNR2d7Gxw+SsO7gLxRxr0zgjs81F6wgX2buv/7ZDdv25wpM/7vPb2pc1vELWMcvXPccNZNqmOyib8g+e4WFzz9948gsoP7uWaqvvkfXL/8U5lBvZP/fvQUlbq5ZvEq6HpVvvU39ndOYOwfQu9ujxV/XIkqVhRLOxXHc8dmm8LDPDjP1r/7Huu45aNSMr3z7COa2PtT2XFRFMQwJaw02gIvj+HOF5jXO5Qmm/8MfXLddETfJfuIh4ndFWoM7PMn87341qldyA+e59d55Ki++Q9c//Eni+6JQ5NjuLSjJ+A0pWKTrUfn2O1TfOIG5pQdjoBM1m0bEDJAN31alFpF/js7gzxduCZP3poVKe5vC3/zZDB+4L7Zs6Rwe8/nFX17c1YdeSLYvwcA9HbQMpqjnV9rn/QDGxwJa29a/Aw0DSXpLC0ZrnItfeI/WOzbOhVTy5pBIBFGSoq7E8AIbBZXkEkdz2ZvbMN39aiG4cpOL+lWE1zj4BawgZryKaxMWI017cy/MRq9b2bdshFDf/AsrpdywcBRCkN7Xh1ey8Iqb4A9rILFtF0GtijMzSWr3Hbj5edy5tc2iyZ378Ep53Nkbm0792QLV2cKK41omh18pbyo8V02miQ8MUbt8DhkExPu34BWL+KWNRR+uBb2vIyoMpSgENYvKd95bl0C5Cr9UxTp+EXMoilIUcRM1m7phRcSgXIvyYN4nuKPTuKMb96GuBa0lTfK+Rt14z6f66vF1CZSrCGs2tXfOEN+3DSDy77RmcNfJ6yUtB/vsMPbZ4c3ewoawaaFy+JBJb7fK3/sn8xRLi4vdtflbcxdKSCSdu3IsXCkzc6a4oi3Hlnz9azaf/HQc34fhy/6y+bZtSbm0/AFMPH+Bvg/vZPy588ggpDq+ii32Bih5M02iREWoZPQO6kGRhJbFUMwmPUskfDYmVELCFf6Lpbxfm4Eqll8vWXuBvTbSKqI72aRPYYOXXZuDUwuKnCl/h8qaeT6NinbheoSFbGh864dQlSgC7GbyO4RAb2kHocDMJHpbJ4HjoJQLoET8bKHrIH0fxYwhVBWzo5vQc0EoKKaJUBRCz0N6LoppggShaoSei/S95nUyDAltC6Hr5O59iNLRNwhqVULXQTFMhKY1ijbZCEVBGFFSbPM6VUMxTLR0FqO9k/rIRUDg5RcI7EioCk1H6BGxqQwCQseOfmt69Cx8n9C5ToCGIjD6u5oZ5WHN3ng+RKPw1LJDG2thTZg7Bsj94EcICmVkGFL60rejnfhq5+4cxLmwmCSoJOIomQT+9M1T+gDofZ0ojRo40vOxzw1veJMgrxG0729loJvDplc50xCcvegxPObjrZFLp8VUQj+kOF6jOF4j1R6nZ38Lo28vz5UwTcFnPhenb0DlgYdMHFsSLLF/v/yiw7//1eUO7oXjUywcX6zXcGadiY9LYQUV7KDW5KbK6V1M2xdIqi3N6CU3tLCCjVMd+DLitIo0oYgZWGkIr82aoVaalSR+uPqu7lr/ScQctjmaElXo6xYsV+vBLIWC2gwUuBaKqpNIdyMUlXp5miBwG7lBIUJRURSVwHcxYmmEUHCsEkIoKKpBGLgoio5QVMLARa6mTQpI7eulPrKAv0Hz142gxuO0PfIkoRsJCWdmkuq5k7Q99jGCWpVYdx/2zDhmdy+pXfsjk0PgU3jzO7Q+8BiB44CiYA1fwJmboeOJpwgsC79SovTem8R6B0ls2YYMfJypcarnT5HYsQezvQs1kaJ45DX0bAvpOw7hzs9itLQx+9yXSe7ch9nZgwxDFE0FBIkt20nt2k/p6JvYU2Ok995JfHA7frmImkwx982vkT30QOTQ7ejBmZ+h8MZLSH+tRFmBEjMX/WxSbrhWi1CVZjKglFEI7Zq5JxuFqmKfvULpKy+S+fhD6P1dBOUqem8nSsLEz5fw54voXW1kP/04pa++hD+bJ7Qc4od2Ywz2UH/3DM6FUYShR2UJTANvLk8wX8TY0oMMQtRUAm9qjqC0tt9NiceW+FbZOO9Zw3d1FdIPCIqL/WnCIKd2RptMAsr+QtOM/73ApoXKpWGfj34owY99NsX5Sx6+HwkBy5KcPh/d0NADnZQm6+z8UB9O1SOeM6jMWCuEiutK/vD3a2uuW/NzN9YS2g/1Mn90csP3UfJmm0Llal5KUss2fRVWUMbdYEgtRJqCG9okl8TDCqFgiAS23HgEmIK2gnI/RK6ZUHhtGLAqtA1HTUX9qqioG8iol9jB8g9Ma1S2XA3xVCeaEadWnMSIZZAyQDdSOFaBWLIdZEilOEYq149QVDy3Rio3gK4nqJTGaOnYBUjqlVkqhTFW28NJP4gcrTe7vZOymQAtrmaMCygdews1FiexdSdmZw+EIflXvknbB5+MGBsGhtAyOdy5GczObtRkFHLrzExQvxL5LYSq4SzMIYTAnplE+j7WyCWcuWmKb79CUK81CAd9vHIRvbUdLZECAe7sNPnXvkXnx38Qo60DPddG6b03QShkDtwTzc/wRfTs8rwla3yE8vG36fr4D6ImUyiqRuh5OLNTWOPD1xEoNBMer0LEDPSedryp9Qe0mNsHIsd1A+747ApKlpuBuWOA7KcfQ2vLYR09hzHYg7lzMAqMePQw5W+82girTUTRW4qCUERU7jgRax4zd21B72glKFdJ7R6i/I1XSX7gIN7MAsFC6Yblk5cGeghNwRjoxh1Zv3nNGOgmee/e5m9vYo6wuvjdx5UUXcZWFrwJfLzGu7mBibrF2LRQ6epQue9uk3sOGNSX1JS4MuLxV/9u9GKNvztPoi3GxHvzzF8qk+6Kk2pfGQYZBPDad9YZmSQEQhURrbW6uND1PLptU0Kl6M3QG49YfBNqmpiSIqFmmzT59aCMt2G69QhVL0+LvrjDUIRGQstguxsXKjE1iX5NUS87qKwZ0VXzC8sitq5Gii0Nm14PdMXckDCSSKrX9K0Lk7iaYrWcGlXV8d06vm9hxDMoSoxYogXXLiKEQiLbS6Uwiu9ahNJHCAUznsW1SpjxFoxYBqu+gG6mmubKawaEPVFAiemr1oVf/41JAquGnmtHMWMYbe3Uhy8QOk7ELNsIDw09NzJj6QbCMJBAUK/jTE9Sv3yO6rmT+NVI8w3qi4uyDHwqp46iJVK0PPQhZiZHm9TxQjdAsdGzOZJDOym+81okvBQBYUjQMFPJoLEDFqAY0TNTNC1qQ9MQqhqVxBVKNGY/opSRgR+Z78IAr7CAOzeNX79BxJuUUb2XchU1k0JNJ0g/fi/uRJQLcb1sdGFoxO/cQeuPPYmSWjQLVV9695ZS4XuTc9RefY/YHTswd21BmDrxO3agdbRGuUGGjjs8GUWMHT/fvM69MoEwdKzj5xGGTmx7P8ZgL0GxggyCiFlagntxDHd0ZXXLa+FPL+Dny2itGYShk37iMM7IVFR0a61ERiWKlDN3b6H1Rz6C2hYlPcsgpPLtd1bMkx1WmffHCWVASEC7NkC73g/ApHuBcjBPhz5Ihz6AQHDeegdVaPQaOzGVBHlvkhlv9TyhjWLTQuWbL1nc+8RyG6oQkEkv2q3tsodnBZQn6/hOgF1yKY6u/bLGYtDZpZJKCfwAioWQudnlO8xkb4bUlhx9j21HTxmNmHhI9uc2dR8lb4ZQBihCRRUa7eYAcTUdfWQypOYXNx0GvOBNMMD+5m9NaGS0jutWmlwLKa1lGY8XQMFdW4g6oYUT1oip0a5YCEFCy6Er5oZqkhhKYkW/N0ItKOBJG0PEm33njG4m7QsrItiqpUlau/ZgxrLUq7PEk20YsXQjbj5AhtFC6Xt14ulO6uUZXLtELNlGYfY8Vm0exyqBDNfcnIW2T3JrB2rMoHJq8zxI1shlcvc+SMeHP0nt0rnI5NTWgQx8pOvglYu4C3PYk2O0f/AjhK5HUK3gTI2T3n+Q7MH78MpFiu+8hltcIHSXZmvrtD74IZAh1sjlqHIfUD17gpb7P4g1eoXaxdP4tSrp/YcIXQe/UoqEhxLtlN2FOfxqhfqlc6T3H0S6Lvb0JIQh2UP3Y7R1oqbSBFYdv1ZtLEwSd2EuEjqKQmJoB6md+3DmpikdffO62oo7MUv93bOkHj6E0FQS9+xB72mLHPaXxgltBxnKxgYwqvduDPWSuGsn5tbeKHRaCELbpfzCW1inLm362awGYegoqQRK3CQoVfHn8tQB58JoVAxtrtAw2/kYQ33484Uo4zwIUFJxjMFu3Mk53LHpRoTUNNJxCYrRplC669uk+IUK1VfeI/vxDyB0jdjuLXT/f36K6mvHsM8MR0W/wjDahKkqajKG3t9J4q5dmDsHIgEIhK5H9bXjESX/NchpXc30gsv2capBHjusktU6yGqd1IIifcYuTtReIsBDoNCpDxJIj0nnAn3mLor+LI68+cqoN+U5Xm1T8dd+Os1//I1Fp3mmL4GZ0Jk+XSCeM2nZkmLs7ZX8Ux0dCj/8Ewke/qBJNicIGhFhX/4Li+eesZt91SZL1KcrxDuSTH77EoEb2XH3/Y3NZRhHPpMKSS2HEAqtRl9zIfalSz3YeADAVRTcqYg3S40WVwWNnNGFbpkbElQChazeib5kcZcyZM5Zm4E0lD4Vf6F5LxDl4hhr5LWshYSaaWgZ64cT1Cl787SbA81jbcYgMSVJLSguH2fgMj95nKtajFVdfDdcu0J5Ido91Ssz1CtRZFFp/jIlolj6+YljNxyPlonhzFUIrJuzMwf1KgsvP7/sWPn4keg+rDpeMYqqKr37+opri2+9svy6Y+8s+y09j7nnv7ziutrFM9QuLi4i+VfWDpsvHXkNAL9cxJ5c/m4U3nhpzeuK77yKlm1BaBrO7BSKGYuc9DeweIaVOqVnX0dtyRC/cydCERh9nbT96EeQfkBYt5FBEGlIho4wDYSy2KiUkqBUpfLiEYrPvHpLwlmvwp8v4k3MYgz2EJSqWO+dRQYhQlXRezsI6zbeePQ+Vb75BkZ/F2HdIqzWo+TC4Sn03k7ciVnsM1ciE1hfR1QFcnoe68QFgur6/D/ScSl/6220jhzJw/sRmore1UrLZx5DfiqM5sn3G0EXOiJmLEvWlFISWg6VV96j9LXvIO2V1om8P8WwfQJJiCYMeozt2LJOXEnjS7dRSypcFkRzlaIqodrMe2Mbrii7FjYsVK7eq5SR9r0UqgpPPBJvCpVMT4IDn9mKHtfoPdhGotWkMLJSU9F1+PG/kuDQPQZf/IMa4+MBhh6xFP+tv5diYT7knTcbEykjFXDihYt41cXJvfTHN15cVoMfetT8Akkth4JKuznYjLLyQ2fTtCoQ+TVmnSv0xyOmYiEEOb2LjN7Jgju27nYSaoZ2c7CZ9BcVGlu4TjTVVZ6xCdqNwaYZKqamaDf6G4SWN/6AVaHRYnSjiY35YpywRt6doMXoac6lqcbZkryT0+XvrHHVauO5NYtMrLcFoan4pTrW6K2J6Pl/G/xyicqJdyNTG+BXy8i1InCWwB2ZZv63v0L6icNRKeF0osEGq6KsQgvfNBNaDtaJi5SffxPnyuSK6KabRbBQpPLNN1Yct06szL2xr9GQwrpN/a0Ti79rFtbRs9dcszrn3lrwZ/Is/MHXca5Mkv3YB1BzqWieVKUZGbYUV+dJOh72uRFK33gd5+IYYXWtDaFccm2IKnTSSisKClZYwZceRX+GPfH7CQm5Yh8n703RbcRJq23YYeWWOfc3LFT+l7+bo79X5fkXLf7ZP2qhVltiExQCY0mAUmXG4sqr06S64ixcLONaPuXJldLdNAUfeNjkN/9LlVdedppayTtvuaTSgo9+IrYoVBowMjFUU8Mp2Ug/XFeRrtUQSI+Kn6dDDoFYzESXUuKG9oqd9UYgCRm3ztBhDjY1BlNJMpQ4QM0vYIc3ztRWhc5A4g4y2iKPWCgDxq0z1+XkkoQU3BnqQblZD0URCluSB5h1htelgaW1dnrju9eZSb+0b8mMfYWu2DYyWkezLHBvbDdlb55J68KGk0lBoAl9U6wA+dcvrkWRdlMQhh45zz2/sctSNkQ2uaI9BDomQih40rlueQQdkwB/AyUUlkMxNWLdEY1Q6Pi4C7WmprVR+HMFCn/yTcrPvkZszxDmjkGM3naUZJSVLj2P0HYJK/XInDQ8hXNxLMr032DEGIDZmcYt1JEbpGZSDA01ruOVrVv7LigqejaHUFWCem2Zr2wpgkKZ0tOvUHnxCLFdg5g7t2D0d0YJnKYOfkBouxGJ5PQC7sgU9sWxKLLsOvNUCfJUg2LTVxrgc8k+GkVR0qDMQTLqnG4S3IYEuNLikl1snrdRCqe1sGGh8rtfrGDogvvuNvniX1R5+rlFIaGqgv/tV5ew04aS6dMFcmUPVRfEMga+HVCZWS5thYjq3hcKy/0nQQAzUyG7964cZsfhAWLtCZy8RWWkQPH8HH51MzQkAfWg1KwE2Rw7kqqfv2mVsOrnmbDOM5Q80KRt6YhtYae8n8vVd6kFq8fOQySABhL7GEzsX0b3kncnmXdGbvgSVP0FFtxx4mq6qeXElBR7Mw9zrvL6mhT8AkFG72BP+qEN+1OuohYUGK2fYl/mYVSiyC9NMdiZuh9TSTJlX1iXYNOFSULLkdHayejtnKu8vnHBcgvNKk2oCsbWQdREAufyMNIPUOIx/IVCRIWhKJHNXYYoiTih495wATVEjC51CEmII21K4RwxkcSWVQQKtqyREBk8aZNWWqiEeQSClNKCKy0ECiE+CiqutPGvs/NM7+xk+19/mMr5GYSmUjo5ycy3zm3eUR6GBMUqtTdOUnvj5ObaWA8E7PylR7nye69RG179/VVMDT0Xx5mtLrsfPRcn0ZejeGLilhZsS2zfRefHP4OWSlM9c4LZZ79MaK1hGgtlRLtz5OwtTeC8NvgmSmdYDqGBooT4rrzueTeLDQuV6Znow1BVeO+kw8XLiztORYG3312uxmZ7kwze10mmJ0FtwWbufHGFUImc8pIHPmBw8bwX1fAGunsVPvi4uSof2MjXTpPoyZAayJHb3UH/h3fw3q+tbTe+HuygghtaxNV085hEbohefS0E0mPcOk1O76TV6GsKh57YThJqljlnmHlnjFpQJJAeCioxNUWb0U+nOUSr2dvcXUgZhRAP195bFzGlL13G6qciTUlJNU1wbUY/d2QeY8q+yJwzjBVUkTRUZq2NDnOQTnMrKa0VKUNcaaOL2IY5t6bsC2Qa9e6b9VDUBFuTh+gwB8l7U5TcGeywhh86CBHRzhhKjLiaJalmSWhZ4g2/ThAGXKi+tY5dZtSOLnRUoaMqBrqIwppVoWMq8WXPGiJGhb74buyghi89AulF5Y3D6N/Rb2/x4w1CpOMSahph3UbNZdBac/jFEvED+yAM8aZmI+r6thbcK2ME5eXPTDFjtD7yBHprOzIMcE+eQ1ysowoNW9ZoVbtRUGkRXVTDAro0SCutzPnjGCKGKjRyShsxkWjmQHnSRkVjJhjFv8GGqHR6iuHff4PUjk56P7Gf+dcu3bTf6fsBif4cmX29TD17apk248xWcGY3Z9G4HlK79qK3tCEUheSufagvPd8UKv27k+y5P0t+xuHs6yXq5Y1q6DdGLKmSyGrkJ5evk4oKXVviTF2O1tt4SkUzFEqzm6+Vsh5s2lE/MemvcORJCf/7b5VJxAW2IwlD0EyV4mgFRROMHZkj3bly52tbki/8fo1/8CtpPvyxGFOTAbGYYGBQZXg44C//ZKXUVww1Kpk51ELbwV6KZ9dffOpaWEEVJ6hds9BIiu76KSeu336ZU+UXOZj7KGmtLUreE0rDv9LOUPIgUgaEhAiiv6lCQ2E5IaUvHc6UX2ZhA9FjFX+eC5W32J/5YDNXRBEqWb2LtN7O9tQ9hDKIaqqjNKPgrgqyBXeceWeM7al7UTboWwmlz6XqEYRQ6IvvQUFt1JvXyepdZPQOwnjQTBJdJLcRKEJFoDQTRwGCdZp6tiXvbmh3akRl0/xv4/9iZSJoSsuxPXW4EZIsm2aDq9pgpCGOc6y06CiXjotUVaQfmb+EaSBUFTWTxh0ZR0nG8ecL6Ht2EBRKK4SK0DTiW3cS7x+MQoqn89gXTpMQkXM1JlI4sk41LCKRdKtbmfQvNXeXChoxkcCTLq60SCutaBhNp+yNoJoaZmea7N5u7JkKoeuT3t1F/2cOoqVMSicnGfvzd1F0la4P76X9viGEKhj/ynHyR0bpfHQnHY/sAAmTT5/AXajR/vAOkkOt2FNlYl0Z5l+/jJ6Lk9rWTuj6hE6AUBVGvvg2ZnuS/s8eREvFKJ2aZPwv36PrQ7tJ9OWI9UQm27E/fZfKhRm6nthDxyM7saZKKEa0bGX2dNP/g4fQkyaVS3OM/ekRYl0Ztv7cg8S7s7Qd3kLh6Bjjf/ke2f09bPnxw1SvLDDyhbcILA+zPUXfDxwgtb2D+kSRiS8fw6vY7PuVJ6mN5Elt7yB/ZJSxP7t+qLNfKUfvgKbhlYrRvxtIt+kUZl1kAL074vTvTeE7Ifkph5YuA9VQOPNakaH9KZItOpePlWntidExEOP0KwXa+kz6diU5+s0F0m062+/KcPl4mWyHQa7DYGbYQoaw76Ec7z6/QH7a4fDHO5i+XKcw7fL4T/Xw9tNzXDlW4e4n2xg5VcV3Qw5/ooN62ae84NLRH0MzFE68lGd+/OZ9W5sWKg/cG2P7Vo0Llzym5wIuD/vcdYfBT/5gCseTPP1cnTfecagXHXw3wEjpDN7bwez5lSYPKeHFbzlMTwV86CMxBgZVSoWQb37D5utftanXVz7QgSd3kd3extyRCYa/enpTpq+rsIMadrhYuQ6iOiI1f23T1EZRD8q8V/wGu9IP0m4MNOvNq2hNZ/ZaTMChDKj4C1yqvsOsM7zhvqfs8+iKydbkQUwl2dRYrte3lCEL7njT3DSUPIjOxpMnXWlxofImdlClP76XuJpp9i8au+v1QCLxpbsuHjJN6BhKfN1tQ5SYqqKsGfEkpURXYsuOhbYdhf4KEPEYSjKBEo8IDtXWHO7oOELX8AulZQvNWggIcGSVgj9Nq9JNOZwnLtIE0sfDoRIWcKSFKeIYwiQukswHE7SoXYQypBTORRsHybqobDL7etjRnUEoCqd+9RmEorDjFx5h+oWz+BWbgR88RPH4OGZHmkR/jlP/5uuEfogAYl1pOh/Zydn/HAnZPf/ww0y/cBY9bTL8+2+y7ec/wOQzJ2m9dwte0WLhzWG6PrSbyWdOktraRnJLK1t+8j5mvnUOv2LT/7lDlE5NYrQmQVU48++fo+ODO2i5ewAnX6PriT2c/Y/PgxDc+a8+BYBbrDP93BmEgL5PHcDsTFO5MMv4l46R3dvNyBfebpbEKJ2aYviLb9P+wLZGrptC56M7cearXPmDN2l/cBtdj+9m+ptniPXmuPTbr2H/zzc58K8/zexL56+r4ZSOvB7VvkmmqJ463sxDuoptB1LkZ1w0QxB4IU49YOiOFJePVSjOuOy8N0sqp/HiF6do6zVp6TKYHbbYeleawrTD5IU6QsDAniTTw3W2H8pQnnd559l5PvC5Lt55Zh49XubCO2Xa+0xGTlfp3R5n+GSVsbM1Tr1aBAljZ2skMxo92xKceqVAqkXn4IdaOfNGkfK8R//u5PdWqPR0qfzVn0xz/JSL58Pnv1jhb/18hlfftNE0wY99LsWpsy520cWteSxcrtAykMS1Vv+4ZHi1QNf6aMZnXh9l5Omzt8Q2GhIVz5JmiGjsYMv+3AoCx5tFPShzpvwdumLb6I3tjsoVL1nIVxMoVlBm0rrAtH1xw2WIr0IiGbNOYwUVBhP7aTP6VxBRNjUiJE5QZ9q+yGj9JPWghEDBC+0oeXIT/GGedLhSe4+iN0NPbCcd5mBTuN1w7FJSC4oU3EkWnPFNOerfLwSFxQ2SNzaJNxblDbljk9inF5Pp/Pn8unwVvnQph1Hi8HQwDECJxQz16eBK87yav7hwWf7mqPkLR8cY/4uj7PxbjxHryeDM1zBaExCEqHGDyadP4uZrZHZ3UxteIHB8CCPdzWxL4ZUtvKIFisCer2G2p3CLFoHj4xbquAs1FE0l9AK8skVgudjTZRL9LWjp2LK+pr5+slmZs3x6itD18UoWsa4MRjaODCTOXBUEeIU6QlXpfGxb1NdCDaGrG+J2E7oaaWOnppBegD1TJrWjE8XU8EsWtZEFCCVe2UaNX7+yY1CrsfCtZ1f/o4Qzb5SIp1XCQJJq0bGrAfkph96dCVq6DcpzLpoeZcFfNY9lOnRGT9dQNcGBxzIEgcSuBrT2mIyeqpJu1XGtaH3y3ZBMm07/niT9uxK09ccI3DDKERKw7a40CxM2Q3ekEQJmhi123J1BSkm16OFUA8JANuv23Cw2LVRCCV96ps4f/EmVB+4xufegSX+Pxle+XsdyJL/+b9uJxxXibSnMtM74u/NIoHtfC5XpyMb30AcN9t9p8Ae/U8OyNuYuyuxow3nHuq553Q6qHCs939yNR9Qpq4fkDdeOMWWf5+pWNapdsnrr4/Wzy3JEnGD9fEVOWGesfppZe5iM3kG7OUBG7yCupFGFSiADnLBGxc+TdycoutNYQXUT0VLLEUqfWWeYojdNVu+kwxwiq3cSV5MI1Cgnxy8x744x74xRD4pNUkpJyLvFry8htJQbDmCQhOTdCUreLMO198jq3bSZvaS0VkwlgSp0JJIg9HClRc0vUvXzFL2ZJlXOevscrh9j0j7HTZewuwbr7d8+f009iluYJX6r4Vcd5r5zke6P7OXK59/AmijhFOoUj41jtqdw5mvYM2Va7xti4Y0rhH5UK92aKqFn48T7cshQEutMUXhnmORQG83v5upty6X/jljB/YqNNVHCLVkUjo41+oqE47IaLIDbEFzJoTZkEGK0JlEMlfTOTkb/6B0Cx0PRl+R1eAFaKoYS16MkWj9EaApKQ/AohoZftXHyNZJb2ymfmyG5pY3Acglsb7FY1ZIxbBaX3ytHpspG/sWFI2XCUDK4J0VpzmXykkW95KPHFDwnxHNC3vraHJohsKoBmi7ITzlYZR+hCsy4gmOFCAF2LeDlP5qmVvJ597kFPCckP2ljxFUCT1IrebzxlTlkKHHqAUefn0eGYFUDEhmVMITAk3hOJICKM7dmw7ZpoVKphkgpyWUU2loV9u0xyGYVMmmFuhWgqpDpTtD3kQHMlEHPna1oMZWJ9xbzBDq7VHbt1lA3MYqu+wcbtCxrq/mScN1kkJ608fzlIbpmLEcs3rrGFYtIkFtXH2HoUSmPI2WIHVaxnSqzzlVqhGtf3ZULkVA0Eq19yDAg9B3syvwGFyyJG1rMOSPMOSOomkHo+8uiyDQziRT+Cpbj682jkciiGnGcaoHQvwFtufSoBUVqQZFJ+2r0y43vfb1QzThqphWpmYSBh1cp4FubLLIlFMxsG05xg/66dZi6vtfway7WZBEZhOSPjBDrzmC0Jjj/m9+m/7OH6PnoPpyFGpf/xyssvD2MYmrs+JsfBAFTXz9F8fg4418+xpYfPwwCRv7nm3gVBzVuEDoB9bECvuVSHy/gzNfwqw710TyB7WPPV/BKFhd+40X6P3eQ7o/sxc3Xufw7r2LPlPFK0cbPq9jR77LFxF8eZfBH78WaKjH74nncQp3JZ07S/7lDuPkapVOT+JXo+y2fmyG9p4tdf/sxCkdGmfrGaXo+tp/s/l7UhM7Wn76fsb84yuy3z9P95D52/Z3HqY/lmXzmJKHjU720+LxrwwsE9uaDFxxrubXDrkXr1eyohe/JptPccxbPq5UW3x/Xgnp5cY2zq8vXu0o+Glt5fnGMS8+v5hePl+aW/nulFcbfYIj2WhByncUyrjVV9PdG5q++Hg3HlUxMBmQyCqmEoFINyWUV/r//uoDWkSKRMyiO1/CdAKvkNteMz/5wnCeejPGf/n2ZWnXtYdi2pFRc/vetn70D3/aoXCkgwxAZSMqXbmVim2DL9g8xtOMjt6xFq77Akdd/neAGC+9aMBJZtj30EyyMvEci18P02e/glDcfoNC65SDFidOE/uIOJZ7tIvBd3Nr6/Unpzm20DB5g/vI71PPj675OxM2o/reiNMgSAwhClFSC0HYinqowRMmmo0irShUllSSs2whdQ0klo1DWUjm6TjdpP/AwRrad0HMQikp5+DSVkbNsRlDpqRw9Dz7F6PN/uOFr1wM1maL3J/5a01E//8LXKbz67felr9u4jVuBdfk0N9v4xFTAb/52mc52lWI5JJ8PUVX44AdidHepvHXEoVINMRWbnjta2HJ/J4goAmzkjcVQ3XvvM/gPv94S2fTWwGvfcfnPv7bcURZ4Aa13dJPe0oKUENjeLRYq359wKgvMnX+d7n2PEUtFkWSZru2ouokMQ2bOfods317SnVsJfY/5K0dwqgtke3aT6d4JisLU8edJtPXTtedhku2D1BbGyA8fJdO9g2zvbvIjx3BrBYSi0tK/n2T7FqQMmTj2LMm2QXJ9exBCIT96nNrCGLX8OInW1WtfK4pOa8duTDPL/OwpHLsY/UEI4nfsxrk0gppJI3QdP18gWCiSuOdOvOk5lHSSoFBGa8vhDo8TWjZaaw7XmkboOvG79uBeGSesVCMacjNOonOA2Xe/hb0wjaIbhIGPmWun/c6HmXz1K8gwaAqfyshZrPw0HXd9kGTPEGHgU7p0nNLF47Tuu4/czkPE23vZ+gO/gAwDRp75PWQYkOzdRuu+BxCKQvHie1RGz5EZ3EOsrRcz20b+3Du07DxEeeQspcsnIofh9wqqitnZQ2LHbsyOLpR4AoIAv1zCGh+hfukcQW1jmpyaShPfso1YTz9arhU1Fm/WcvGKC1ijw1jDF9cVnACQOXiY9IF7CO06+Ve+jTMZsU0IwyC+ZRvJbbvRW9sQmkbo2LgL81hjw9ijVwjt5eZsxTDJ3fcwie278GtVZr/6pw3KGYVY3wDJHXvQ29pR40lC18Uv5qmPXKR+8fwiIed1YLR30vbYR5tM09fCmZli4aXn1s5TuQbpOw6RvSeimJr58h83E1D1tnZSu+/AuMpqHYT4lRL2xAjVs6dW3Pd1oSjEevtJbNuN0dGJEk+g3MA8FLoO+e+8gD0+sv5+GripIl3trQonz3o4jsT1JP0dGvfcZeK4kiCILDOtQ2lUXeHEl4Yj215t+YMbueLzpb+wsK/jUxkfW6mWjX39HOPfONf8/X6YrX3PwrbWdo4rio5uJJtOb9+z8H2btXbFjr25Cn7L+tQN4rkejESW4tgpVDOBnsgye+5VfKeOlCG1hTGs0gzZ3t0kW/sJnDptWw8x+s6XCXwPGXhUZi5hlWaYOfMynh0J7MrsMGamE9WIaCOMZAupziEmjj9PGHjIMMAuzzJnV4hnOsj27qG2cH26GcNMs33XUw024bllQkVJJSMiw5gZcR5pi5FoxmBvVD/bciKNRFMhDDG2DuAvFCNBYjt4M/PN+hRh4BH6HrH2XrxqCd+qRvkfvoeRaSXRM0Rt4hJ6Kku8c5D8mbfIbNlHsnsLk698BUW/yrkkKZx9B69Wov3AI4w8+/sRZUYYEO8coO2ODzB//DsQSjoPf4TQsYh39IGi4pQX6Dr8JAsnXyM9uIvq+HkCZ3Ms1zcFIYj1DtD6yBPEt+1E0fRFjqVoksne8wB+rUrh1W9TPvbODReq+NB2cocfIr51B6oZi9q71rkrQ2QQ4s7PkH/xOaoXztyQZUBvaSOxdQehbVE9fRxnapxYbz/tT36KWN8WhKou76dRl2bu61+m9O4by78pVcXo7CKxbSd+tYLe2k5oW7R96GOk9h6InPnimnm47yGcyTHmX3gGa3T4upsAxYwR6xtEa2ltTPPy+xeqGo13PRACPdtCYttOpJSYPX2Enkvu/ofJHX4IxYw1uLAWa9Zk776ftsdKzD3/NLVzJ28ouM3uPlof+wjJbbuj72vJO7BaoEzkTwoJalWUWGzF39eDTQuVoUGNX/y5DFdGPKZnA373CxX+3t/IMj0bkEoq/I2fzfDP/12ewAvRTJVEWwwZSMLQxqks2vbGRgO++pcW1coGS9bqCm0HerDna1RGi8Q7ktSnrkkuExppvQ2Egu1XMNUEmmJg+RUMJYahJqj7JZxGoa6yu7TCo2Rm6ij5+fMrO28g17qVoR0fxTCjXcvs9HEmRl9bs4KhlAFBcHPJZUYiR6ZrOwvDR7Gr8yTNQbx6Ed+tI0MfVY/RseM+PLtKItdDdW4YoenIMMB37eYHc3WMYeg3/y1lAEtCURVNJwx8As8GKVFUnZbBO1FUHc1IINbhDEukOonFc41FYelLLJFBgN7fQ1CuImImem9XZNqSEqFrBKXoeQYLBcJiJRJAho7akiGs1WnUSG62GNh15o9/h5a9h0l2D2HNT1K6dByvWqQ8fJpU3w5qk1dID+zGmhnFr1cI7CpC1Uj2DFGbuoJbzjfnJ/T9qD65t2iuTHQNougm8Y6BKFteN0j27iAMfLzSAn69jJnrpD4zSmpg1/IF7LsFIUju2kfHxz6D0doWOas9F69QQjpOlEuTTqMmUujZHO0feQq9rYP8y88TVNcInRWC1kc+THLHbqBBcujYBNVKxLYsBGoihZZKo+g6sZ5+Op/6QcKv/in186fXN2xVRU0kiW/ZRtenfhijvTPSgKw6oeuCqqDGEwhNJ7Qs3IW5627ShKaR3LGb+NB2Ett2gZT45RJBvQ5CoKXSUR0ZXSe+ZRudn/gc01/+I5zJtU24frlE8e1X0bI5FMNE0Q309k7Mrp5lJJCbQWxgK4mtOyPNRVUJqhWCWgUZStR4HC2TQ2gaeksbnR/7NHNCUDl9bM0yA0ZHF12f/hFivQNIKfEKCzhTE9HmQVHQcy2YPf2osSh3MKjXqA9fxMsv4BXyuLOby9O7KZbi1992+MKfV7j3kMlD98fYNqTzL3+tgOfDf/xXrcTjCnbJJQwkbdsyyCAkDCXV2ZvfufV/eCeJnjS+7VMZfY+tn72DU//ncnbYmJoibXRQcmeIa2niWgbLr9AR30rVnSdjdFDzCiS1FnpTe6gXS8v4tHzPwvfWHms80bas2qDvWVi1+ZuuRX892KVZZs4tZ7xd9lkJgWammtTxge/iO3WcWoG+A08iw4CZMy8T+A5OZZ7uvY9Smb1CaeI0bVsOkunZje/U8O0qbr2EDHz67nySMHCZu/AmmhFH1WNIGRJ4FkJRad92mEz3DoxElsCt41QXtbuWth2rhyqGEvv0BdRkAn8+jzAMlHTkI7HPXgJNI6w3OJqCgLBuIXQN59zlSKAA9rkrSGdJxIqU1GdGsQuzxFq7ye08ROve+5g9+iK16WHaDzyCns6RGtjF1OtfA6A+M8r8iVdJ9e8gu/0uFk6+SmX03MrxNqDqJjLwCb2Irjx/+i3c0hyp/h0QRlE0YeBGUU63KERzo4gPDNH+4afQW1qRYUD90nlKR9/CKxaQnotQFLR0huSufaQP3IMai5M9eJigWqbw6ourm4GkpPT2a8QHhnCmJ6hdPIszNUFQrxF6bpRQGk8QH9xKywceQ40n0DJZ2h55gvrFs2vXV1kCoWrEBreS2n8Xeksr9SsXqJ4+jrswj/RcUCKhEuvfgqIbeIXrm7sVwyB338MoiQReMU/x9ZdxZiYJbAuBQE2mSO7eR/beB1E0HbOrh9x9DzHz5T9ZU1j5lRKF116MfqgqiqaTuft+2p/4xE0Llcydh1DMGIFjUzryBvXLFwitWiRUYjHiW7aTu+8htEwWLZ0he+8D1EcuEVRWCaIRgtaHn8DsiWqq2OMjzD/3VZzZ6ah0tKKgprOk9t5J26MfQY0nABH1e3Ht93892LRQsSzJlVGPsYmAocGQe+4ySSYEigLVWoiqCBQBdsVDUQW5/gTH/2KYROtiAt23nrd563WXem3jJqHsznZGnjlL94NbIJTEO1faOG2/ghfatMcGsPwqbmBRdmdoi/VT94vE/DR2UCGlt+IFDhmjgwV7/ezB3224VpnRd7+67Fg9P45VnGouBIFrMXHs2UZWdUQtLwOf6dMvoahRvH3QcMzPnH0FRTMIG9pTceI0pekLICWB5yBDn6nTLzXsrxLftZg59yqKGtVIl2GADAMWht8lP3oMZEiwtEaIUGhp27nm/QTzBYKrdcNrVjPv46rQuBbS83FHFtkEgoVrgwmiYlSh51KfGUWLJckM7UXRNNxKgcCukdt+F369ipNf9OtVx85Tm7xMZsteOg49tihUwiAqKqVqjYJZEq9eQbdqlC6fJPRsRKOWSap/x5KBrnnL7zvUZJrc/Y9gtHdCGFI58S7zLzyDX16edOxMT1IfvoRfrdD26JMopknu/keonDqGt7B68Eft0llGf/vX8cvFSHNYxaxljQ3jVyt0feqHEYqC2dOH0d6JO7uOSoeqSnrvAaQMKbz+MvlXvhXtqq9Z4GsXzkTcajdgURaKiprO4M7NMP2XX8Bp1JZZNt7RKwghyN73MEIIUnvvYv65r61JCrkMQUAYBEh37fSD9UIIgZrOEFTLzD33Vaqnj68wbVkTowR2nY4nP4VimMQHt2G0dmCtIlSM9k7i26J3MrAt5l/4OtboYhEuGQT4xTzFt14h1ttP+s67UeJxkjv3Ub98MXr3N4lNC5WzFzw+/Gicu/9ZJExqdcnkdMDP/USG6Vkfx5U4jqRlS4p63iHZFkOLqeQGU0yfihaDUlFSKgaoGhw+bHDlss/C/PrKvlZHi7Ts7iDRm2Hg47upTa6cWCEUKu48fujQavZT8RZIGx24S/JKdMUkrmWoevO0xvq/b4SKEAq6kcaMZdH1OEJRI+3Ad/H0CrZdigRGY2FfisBbyV4c+u6yKC9o+CCWmOMCzwHPueY6Z1mY8GrtBO7q2lwi1bWukOxbBS2Rov3AI3i1EkhIdA5Qm7wUVUaUIdbcBO13fZDpt77B1UUg2b8DM9tO6NjEO/qoTy/JPyovAJLW/Q/g1yqULh2nOn6BZM9W2g88jFcpoMWTlC4d/67d441g9vaT2LYThMArLFB4/aUVAuUqpOdReP0lMgcPY7S2oyZTZO66Z81EPul5uDM3qHQYBFTPnqT1occx2jsjwdLZvS6hIoRAqirVE8dYePEbawqN9QYAAEjHofDaizhTE6tqH9L3KLzxHTKH7kMYJoqmYfb0Ub+0ttn7fUMYUj19nOrpE6vfYxBQfu8dWh74IEZbB0JVifX1Y42sLG5mdvWiGCZCCNzZabzCGmWeg4Da+dOk77wbIQSxvgGEIm4qtmTTQuXiFY//9F9L7NiqMzXjMzruo2mCH/1sii0DGl/8iyrlaojmBCRaTTK9CQbu7aA6s3IBipmCf/zP0sxMhxx5y+WlbztcuuBfV7iMPXee7oeGsKYraDGd4S+fWnGOQKApJqEMGKuexFDiaIrBbP0yvvQoOdP4oc+8PUrNy2P5t55sbjMwY1m6eu8m27KVeKIdw0iiqDphGBD4NrZVpFadYnb6OMWFS9xol9TWuY+W1u1c5X8vl8aYnXpvXWNJZwfo6LoTRYlelWplirnpYwTBtYlSAt1IEI+3EU+2E4u3kskNoCiLTsvegftpbd+9Zl+uW2Fm8uiiM3+DCD0Ha24cLZZEypD82bexZseafiS3WsSrl7DmFm3mbmkBPZFB6AZBfZL8+UXmWL9eYebtbxJr7W5ma3vVIjPvPE+yZyuqEcOrFggcm8roeULPIfQ9iuffJXBrFM4dWeaPAVB1hVhao158H5gBVJX44FbURDIiH52exLmBEJCuizVyGaO1PaoOum3X2tnh60UQlSU22jsBgRJbP9N1UK9RPnZkXbVc1gN3fjaKYLrOYhI0CqyZnT2RbyiVuSV9bxShY1M9d/q61Tal5+IuzKK3dSCgMdaVdR2UeLxpjgttKyodvQb8WrVxfeNZ3aTZdtNCpbtTZdf2yCzS16PS1xM1deqMw+f/yMVxorr1hZEqeiz6W2m8xtTJldFUliX5p79c4v4HDZ54MsZnfijO6ZMeX/uyzTtvRm1da5L1Kg4T37qIamqEXhBRSFwDX7qUl5BC2tcw+17l9qp5jYp97jpU9PcVgmzLENt3P0Ui1YXaMFddjQ1XFA3VTGGYadLZPlradzE19hYTI6+sssgvol6bY8fuT2LGWwBoqS9gWwXKxeuHC+pGiqEdH4n8Igg8t0q5NEYQLp9rRdHoH3qErp5DKKqBqpmoqh4ROi55Qds69123v1p1hmL+Eo5b2hRdfei5q2oNQlFR40nSA7uojl9cplm5pXncUrSLO/TZQeJ6jPHjFomcjqIqVOZGUeU0qq4ghCTeYqAZFuVLR+nemyXRbVAZcZDODKlWg+KkhY7LlsdaufjKZRQlJNuXpDpv4zshPfuzmAmNkSML190KCN0g3jtAYNUjMtFyESklimliZFuxp8ZX+D6EqhLr3xL9kBJndvrGwRRC4C8xn+i5VoSqNsx9N0Aj+ks02mk0iNC0ZjlkoGkiXA+8hbkb+krWCyklXrmAV7pBzpWUBNbiO6HoxnVOfn8gpST0POypG+d5LTXNKYa5aq0g6XlNQSp047oUNlo8ydUgmtCxb9p8u2mhsmu7zi/91YhJVCiQSgj6ezWe+WadV95c3J21bU2z/6lBPNsn1RlHKHDl1eVRBWEI5874nD/r84efr3PHAZ2PfiLG3/+VNEjJ179m88arLpcv+k1yyfTWVgY/sYdYWyISMC9eYuHo2jXbv98hhEJr+x627/kksXgLQgiCwMWxS3hujSBwURQVTU9gxrJoWhzTzDK47XE0LcbIpRcIgtWTKq3aPBfPfe3/Zu+/wyS9rvtO/PPmylVdneNMT46YAQY5EJFgDiIlkrIoyZZs2ZZX3rWelaXf2pZsyUG2tLa12pVsyVSglpIoRgkkQRCBAJExGEzOM93TOVV15ao3398fb3d113SqnhlA8rP6PsDTPdVVb73h3nvOPed7vodd+34I3YgTjrSyZdtjXDr3jTV3BZKk0D/4MC2tO4M+4r7L1PhbzE2fWoVyKaHrMTRjMa8lgvOVVGRFqxsW17XWJTG4rknsQA/hdBvF44HBE76/INoo45s2kqosnB9IStB+tTYyt+5EMFLtdN/3EaqzYxSurM6WibToCKDvthbsqktLX5R0f4Srr83Rtj1OsjPEueem6NyZIJzSuPraHKGYhhZSQIJUT4QdD3Zw/rkpZEUinNAQvmDfk92EkzrzoxVCCQ3X8jCiwT3ZaO5Kmo6qqkiyQqR/EOF5OPl5tFQLvutiTTcuQJIso7e0sniDWh9+P63ve2KVAy/8FIu/L9OfUxTkUHjN2hVJNwLWUGc3Rk8/ems7ajSGHI4Ei5eqIqtL9PDNwqtU8M21m89tCkLglcsIe6Nd4XIdGW6ZBtZm4deq+E3kcoS/8blaM5P4toWkGxid3Wit7biF/Ir3SapKZNfehQMLzLGRNdmrzeKGjcpLr5m89NrSww8ZEh95MsLtB6+z8hJU5k3yY5WA+TW3TrdCEeT+JsY9Th53SLfK3H6nzgPvM3joEYPTJx3+4L9XmM/6bPnoXmZeH6F0LUeoPcL2zxxqMCqSqhHdvge9rT2wegteS/nCGYRjY3R043seTvbme6bcCoQjrQxse5RwJI0Qglo1y/TE22TnLlAtz9YX41CohVTrdnr67yWW6EFRNLr67qJSnmZm8p01ji7IZ68wNf4WfVseQlF1UultdPfdzcjQCwh/5S6vrXM/3X131w3CfOYS49deWdUoCOExM3WSYqFxkUulB+nqOQILasFT429SLq4djnFdE7YINBEmeecgbrGGJIEc0vA9HzyBcD1qo1n0tjhaOopvuUGL4HWCwOb8NMPf/sKafwdoG4xhV1wsQybRGSZzrQRCoIUV8hMV7LKDZihBX3U70F6qFmxCcRVFlejelyTWHkJWJKp5G98HWZGRZQktrGBVXIzYggads3HAOjAgOXzbRIlEgxCFEPiug2dW10gkS/XaAkmSbnhxXGtnYXT1kjxyD5Ede4IdzfVsJyEQwq+TGm4EwnNvelFbfj6+8zdHgHQj+Nata6lszUxRvXaV+IHbUSJRWt/3frK+jzkxFjDpADWRJLb/MLFd+xb02IpULp5tiqm3Hm6KUrwcpiV4/ajJ3//xxuZHviOo5W0qWRPhC5zq6kk2WYaBLQof+EiIhx8NoesSb75h8S/+9wJTkx6tbTI/9/MxPvN3Ivy33y7jlC0qEwXMbCUotsrVGjww4fvY83P4Vo2OD3+azPefxsll6wPWrVZu+ubdOkj0DNxPPNkLCFy3xtClp5mfu4B/3YJvmjmmJ45Rrcyx/9CPoRlxNC3MwOAjZGbOrrlb8TybybE3iCX6SLftQlY0egbupZgfYT7TSCGMJXro3/o+FEUPDFxljmtXvofrrp6QF8KnVBilVBhteF2WZTp77qg/lnz26orvuh5K1UC+GgxL3/EChdgFj1qSZXzbwS1b2NlAsda3vVvyHOeulpg8V0BRJfSoilVyKM9adQMgqxKO6VHJWmgRhcJkDTVkY5YcPMdn6PU5xo7PU5iu4bs+w2/O4Tk+F1+cIZzSqeYs5q6UiLbqODUP1/ZYt6mm72FngnCsW8xv+nqEENSGr2w6lOTb9qoLsdHTR+fHPoPR1RMYq4WcTW1sGDszh1su4VsmwnWQVY30I08SXgzFbe7EN/+Z9fA3Zo5vjFtmTAF8n+xLz2J0Bey78NbtdH3iczj5ebxaBWQVLZFAS7cFO1PLJP/Wq9RuoIL+etywUdm5TeWxh5ZmhaJI3HeXwdvHGxc1SYZUb5RwUgcBuXQZs2hjFpeSUaEQ/Pp/SbFrj0Zm1uOrX67y4vMWxYJf1+abGPf4y6+bfOyTgScmPMFd/+ZJrLyJngzhVmzu/rUPggRnf+d1KhMF7Llp7LkgTlgbHapv6eVQGFnT8BakFCRFRTZCSLoe8PhVDd+s4VsmcigcyFAIgVct37IE4nKEImm6eu4IaMBCMDNxjMzMWdb29gTF/AiTE2+xZdvjC8dopb3rINMTb6/5PbZV4sqFv+LQkZ8mFEmjaRG27/ko1XcymNVg8dH0GP1b30cs0RN8xi4zfOV7VEq3pmHZRvAqFl5laQx55dV3tk6uCcrnJlArBM/VBaxyMOjs6spJ7tSWXrMrLvaCQkRhqtHg5saCseWYHuayYt/lv28EPdFGqKUDMzcTVKovFnuKgN69IqErBF61GtQcCEHp/GkKx15f/eDr4bp8ihKJ0vbYhzC6AykeJz9P5tlvU7l0NghN+n6DMZCN0OZkRP4W7wqcuRmmvvJF2j/wcSKDO1Fb0qiplnoORvgewraxZ6eZf/l5yudONpdL2wA3bFSiUZmtA0sfd114+Q2Tv/hGYyxWi6j4niBztcjshTy3/dAg8e4Ix750Gc9e8CIkiZlpn6/+eZG337RYK6Q6OuLy2iuBFzX8jTOMfOv8qu+zC+vEZCWJUFcvLfc+TPnSWQrH30Rv6yB5+G6EEKjRGL5tY2dmKJ49QfLgEeRQCElRsXNZiiff2hSlsRl09txeryEJchdHaSZ8kJk+w5ZtjxF0MwxqQtYzKgBmNcvQpe+wY+8n0Y0Y4UgrW7c/wdWL38Kxq3T1HqGtYz+SJON5NlNjby7sLtY+n5AcRZU0bN/EFrcoHv63QNY0lFCU+MAehGMhKRqybuBWy5TGLuJeZ1SE7+PkMuitbSBJwc9bsEiEegfQO7qDPJ9ZY/6lZymfWy23tgBJCiRG/hZ/7fBtC7cYFDE72Qy1kaG6eKtXLWPPzVC7NhTsXm4RbsiofPCxMLou8frRlaGWwwcNXnx1+cIiUZis0rWvhfmREnNXCsQ7w0jyUrzXrAl+898X2WgTcPaUw4WzwZs67xkg0r0UavMsl6t/0US9gBBUr11Bb+9q2G76jkPhxFu0P/ohimfeIb7nIKGuHuL7D2NOjQVyDC1pKpfPrRuOMCItyKqGa1exa0U2Mg6SJJNMbWUxdlerzGFZzcn1W1YRx67WZWLCkVZUNbSgP7Y2splLxCaO0rvlQRRFo7V9D8XCKLVKhr4tDyIrQR+KfPYqk2NvNNSyrIaokkJCDuTy/5oK/4yQxC/+mxSGAaWi4NibFi98t7bhmqpq8MnPRvnqlyrv2rnv2K2SSMq889bm4vt2KaArA3VJGElW8GwL11yZSBeeR21shOiOPQBB5Xkk2lTydz0o8QRKOIhKuMV8QFNeJ4claRpauu2mvnN9rEJ3+lusgKSqtD7yARIHbsctl5h56iuYE6O3PsR4HW7IqHzmh6LEY0GS7o7bDM6ct7Gd4ETHxt0Go2JXHNJb46iGzO0/sp38RIVQYimZ394h8+SHQnR2K5w5afP6KzalNXTAPG/J8SqP5bEKNWRFJrGjFTVyczRA3zLB9/BtK/hdlpFUjdrECPOvvhCEH3y/HjJbC4qqk2jfzvzUOWRFw4imsWsFJFlB1UJ4joljLS0IRiiFbizx4i2zgKqFG+o71oKqhnE9C51YIJOhaKhaZEOj4ns2E6OvE4l10tq+B0UNMTD4KBCEv4QQmLV5rlx8CsfeWME2IiewRPONyhZC8vWftwKaBkfu1vkv/76Aqkn80OeibN+l8fu/XQwKIaMSqirh+VAp+3guhMISA4MqH/p4hO8/U8P3IT8fLJahsIRhBIberAlMMzhRVQ2OpSgSriuoVgWeG+QEI1EJTZNwHEG1EtDgkymZu+4PEY1KXBtycR1BsdDcRfu2iW9vYufne9SGL+PccTdasgW9rZPEwTvIH32tuQrptR7I8oS/L9Y1KEgSiYN3oESizZ/3BghYhVJdEima7MEsZ3DsKv4aOcS/RaD9lThwO5KqUj5/CnNi7F03KHCDRuWnfm6pOvO17/bwD38+w/Ts6oO2NFPj6g8mca3g7+GkwchU0Fuls0vml38tSf9Wheycz5MfCvHicxa/+R+KG/Y5yl9ckpKYfWuMO/7l482dvCQR6u5DT7fhey6hnoHVWTICrLlpIlt3EB7YVs+x1MavrXt4q5bHNos4tQLpvttwzBKhaBo9nMKqZFH1CNmJ03XGlaZHURS1zrJKt+/hnvY9zV3LiktTUJTmjKttFbl2+VnCkTaisQ50I75wDAnLKnL14rcxq831VKn5JcpebqFb5sbYvk1ldtajo0Mhk/UoFgVKIEKMoQdSP5WqIBSSkKSgjskwgtdrNbHmvHBcuHTeYXLc48IZm3/7X9K89pJOseDzk/8wTjwRGIpnnqrx7W9UeeixEJ/4kQjbdqj8H78WtFD4lz+fJRSS+aHPRdl7UEPTJIavOHzx98pUyj7v/0iY9z0e7NQzcx5f/mKZocsuh+/U+cRnokRjMtWKz7e+VuWdoxY/9tMxHnkiBJLE3gM614Ycfvs3iu+ao21NT1A+e4rk3Q8gGwYtDzyC8D0qF8401KPUIcuo0ThqMoWaSgfKt9eFDLyFJLysG6jJFEZXb7BbuS4JLhsGkR17aXnw8Vuqe5ZoHcQIpRC+iyQrSJJMPNVHfvYy5cLGdR3/X4WaSMGCYrKaakFNpvBKhVuSN1n3e9/VowN6WCHWHkKSJBJdEeauFJi9GMhG7N6r0dYh8+9/pcj5cw73P2jwv/x8jP/3j5RV5e6XY1FQEkCN6DjltRe03BsvBUqqC1AiMazZqYXfo9iZGarDl/BqVYpnj+MW85TOn8LJZckfe51Qdy9KLBHQODexGPiujaIaCN8LqLELYaXlUBQdSVraldzMZJQkaUXv+fVQKU8xcvV59hz8TH1n5HsOM5PHyWev0OzFKpJGRElSEvMrOkauhl07VPbv0+jpUTh33uHiRYe+PoVaDQ4f0iiXfE6dcdBUicOHNJ59weKuIzqJuMRzL1jkC2t7yotnnJnzmZ706O1XuHLR4Qv/T4nsnMcddxv81M/G+NbXqzz77RrTEy7/9JeS/O8/m6074J7r88y3qnz1Sz4trQr/9BcTdHQrzE3DnfcafP97NV56ziSekCiXBLoOn/iRKC89W+P1ly3ufzjEJz4T4fIFh9/7rSJmVWDZgi99YZ1dnyQTHhhsOl9nzU0H8fHrFgjfssgffRW9s4vItl1oyRbanvgo0Z37sGYmgvi66yApKko4ghJPoLe0obd3IFyX2tAlvOuMijU9gTOfRYklUMIRWu5/GDkUwhwbwbetIDTc2k5k63aiu/YhayrmZCBffytQzo1R0+aQZW3BsKh4rollri4/855AUZA1HVnXg34nC/NWUjW0ZGqBzuwE9N2/pnbSdmYW37JQQiFiu/Yj6wZObj4geCyekyCgqVdK2LPTmOOjDWvljeCGjMqBPXq9BXAkLLF/t0ZnR7Aomabg4pWlQVmaNbn4vQmQoHVrnGTfkvBjqkVmatLj3FmHYkHw8ksWf+9nomzboW5oVEoL4S8Az/KoTFw3wCRIdoaItGjMXDyJcBdvoqBy5XxjWFaiTr2sDl8CAbWRKwALDLLmK+191yE/cwnfd8nPXApCXp5NqxGjkp/Ac62GuhBZVht2Sq5rBbTgGxiHtlXatEKyqoYaDJEkL+x2NmHcCu4caa0HUyrjNJGo9wUMDbuoKrSmZZLJgPQxO+fR0a4wOemxd4+Grkvcf5/Ba2/YhENQMwWplES+mbVkoeZJ1SR0HT708TA79mjE4zKdXWpQurSGbZIV2H9Q56HHQ6RSMjv3aug6lEo+J9+x+cyPx9i2Q+Uvv1rFrPnEEzIPvz/M7v0aP/kPBbohUS76JFMypWKTlFZJIrpjT6Db1QSKJ9/GmhjFX8XrdOYzzD71Vdo/8Amiu/YiGwbRXXuJ7tiNcIMe7JIkgaIgLeyShRA4mdVrttxCntzrL9HZ2Y1shDA6uml79ENBtb/vI8kysmEgGwbCdck8+22E72N0faq5a98AVi0PfwPIZNG9B2m5634kIxS0YZAVJFlCDi21gdA7uuj64R8PdgO+H7RQsG2cbIbcq9/Hzrw3LEoIWHq5V79P+sHHgjGwfRWJJCECcVjXwTdNrNnpQM145sYLyW/IqPy7f9lCIhEsRMWS4F/9Qkt9DRwecfmpn1sKTSV7o+x4uBsAPaYycy5f/5uigGWBrkuEw0s5k5YWmXB4aVHzPMH1RbGla/P4toekyGgxA7fSaF0jSY1td6cpzlkomkyyS8c1PTxH0LolgqorzE9UibUa6GGF/FSNWsGhc0ec+fEqru2T7otQzdnIqkQpYxFOaJSz9rrFa0L4+Aty+b5nYy/IpxTmrmJV81xvLTzPbpC0mJl8h6FLT69akLgRBGJTXlGyZetSu2SxZGE7e26nXJpkevztFee7GnzhoUsGqtRc6C2X85me9ohEJCxLcPCARlurwqUrLqOjLjOzHt1dCoYuMTbmYduCmVk/aAZnN3d9kahEa5tMds7n5/9Fiukpj1/7pRy9Ayr/7r8siVyudrsefDTMBz4e4f/6jwUqZZ//36+mkCRwHfirr1R561WLj/9whF/+9RZ+9z8XuXbVZWLU5Vd/Kcfo8ELDMD8Ys4rSnH8gSRKoKlKTUzJYxNY2/E4uy/TXv0TswGEStx1Ba2lFCYfriyELRYpetYJfq+Lk5imdPbFmAV75/GmE55F+3xNoLWnkUBh1wSMXnodv1qhNT5J7/QdULp8jPDCIVy6t2SHxf0bo6TbC23YBi0IEK++/rGnoy0gKi5EJLZ6kePzN9+I0g/PQDYzegaAnjeciPDXYgSzvQbRQICspSqDeoBso8QTdn/lJpr76J1hNSMashk0blUhE4md/KUNu3icWk/H9ICm5OHGu9/4qWZPxdzKkt8VRdJlKttGTPXBQ4+d+Po6zkOjv7VP46CfDHDys1d9z7ozD1/+i0VXpfXQHUy8PkdzeRvtdfVTGC4w9s6QsalVcZofKtPZHiaQ0+g6mKM9ZmGWHvY91UpwxiaQ0evYnqMw7hJMa5YxF38EkobhKYcaka1eMcy/M0tIbpm0wimbInHvhxirwrerqHSS966RLNC1Sr0x+NxGOtLFl+xMLnSulhap9n0isE1UNsWXbY1RKU5SaiFlLkoQnXHya2yW9eTQwtHOZ4OfJU059nJ86Hexyz513G/LGs3MbM6ckCWJxmcEdEh/+ZIRCwefiOZuf+JkYP3jBxPfgrnsN9GW2r1QKcjftHTKlosCsCeIJieycx3zGY/d+nYHBYCzqOrR3KRRyPl/5UoWuHpWePoULZ2yGrzjcdofOxJiLrksYIZnpSQ9/gRjQ3asQjUl4HvUup75jUzzxFtWhzSviWpPjG4bKfNui+M6blM+fxujqQW9tRw5FkLWgaZtvmbilEk5mFjs7t66QIUDl0jnMyTFCfVvQ020BbVgIPKuGk81gTozhVQJ9PTs7x/wrL6BEo5iT6yt/10aGmH/5eZAk7JnJm6oFE45D5eLZQCvN9xvk3tf8jOtSPHmM2shwUNQ5NYbaEkNtiWGNLTjIsow1PU7+9RdBlRGuF0gG+QJJ15A1BSe3dnjTr1VxrmeNCkFt/BrZHzwH0HShauXy+XrN3WpsLjkcIXXPg7Tc+z5kTac2PhIQOPK5lWNGkVGMEFprB9Gde9HTrWgtaRK3301mbvqGyic2ZVQWZVOyGR/XFezYpdLeKfPMt0xqq7QD7tybYss9nehRlURPhJNfHaY8t2Qcrg25HDtqIUnUJ/qLzwdGZ/nEX01GKH2wi7l3xmnZ30nu/Cx9T+xsMCqKKhOKabRvizJ2Oo8eVhi8O83lV+Ywyw6FGRMhAuNTnDVxbZ/OnXEiLTrSWBUhIDtapVZwsMouO+9v49o7OXz31sZHTTMf7FYWQhKRaHvgSfq3vshyEYpq0NN/L8lUUPHsOFXGR16lUp5h/+2fR9OiGKEUgzs/yPlTf74BA0wiqiQXNU5v6HzW2lxtNhSdapH5l/8uRbkkGLvm8jv/Z5HsnM9ffaXKj/xYlA98LMyFM06dyQUwM+lx4azDr/zHNPPZYDdz9qTDPQ+E+Pe/lWbsmsfcrIcvIByV+fgPR9l7QEP4MD7qcvqEjW3Dl/+kwg//WJSHnwhh1gTff9bk6W9W8X04+Y7NkXvi/If/K83JYzZf+H+ChVfYNoWjr93QPdsM/FqV2vAVasNXbvpYXrlE5cIZNiIpe6Ui+TdfbuqY1aFLKwyrEWlhYO+TZCfPMD99rmEwtA+EiKY0ZoarhGJqIJlTdPFcgar5GIUL+LkLFDI2iiKR6tIxwgqlrAMSC43UAsKHXfMDo3LsjYbvD23tQE1G0Nq2g+8HkkGGTWX0BEZ3EJlRQ2Gc+RJusYawHKoXNu/Z10aGgtzYJlA5f5rK+dOr/1GSiO7cExiUUJjq0GVmv/P1oEfOul0yNZxcltZHPoASDmO0d6LE4rj55sg6y7Epo9LeKVMs+IwMu7R1yFiWoFwStHXIjI2s9FKP/NhOKvMmQz+YZv9Ht5C5UsAuLy2WJ487nD298eK5mtKCU7Zov6MPJaSSOTZB/5O7Gv5u1zxGjucYPZnHNT1OfmsSSZFwbZ/JC6W6KNvlVzPB74IgPqqA5wqEv9S5T9EkyvM246fym7hbzcF1qlRK00RjXcBiD5IWKqUN+lbcMCTaO2+jp/+eupz+7NRJZqaO43sO1y49w/Y9H0NWNJItW9my7VGGLz+zrgqy5dWoeqUVSfrrSQnvplBfuST4xCPTC98Ljr0UMv3uU1VeejaQ8bEtwZ/9Ubm+ozZNwW/8ah5Nk4JFxoYrFx3+zS/mUNUg5AVg28Ei9Ae/U0JTg5CW4wgWc5pnTthcPu+gasH328uUtS+dc/iVX8hhhGXUqE4oqWOVHBRdxjVXzhvVUOpsyWahqAapzt10DBwhkuhCVjRcu4JZmacwd4XZ0WO4dvO071sJSVJQNCPootqkpxBL9tDWdxjXqZGfvRx001yAWfbYe38LvbuiZMZNQlEZq+oTT2soqoQWUnAsj2rBA0lQzrlohowQsP2OBAgwIgpj58sMnyzhe6uckyyjJqMLYqYSii+QDQ0lYuBVgpIDqxDklNxMEa09ufIQqh4UEa/S3+hWQlK1QLF6QcU6tmsfcigcGMvjb62ZK1sO4TqYk2N4tQpKOBzkyG5QrXlTRmXsmsfhOxV27la5cM4hmZKRJJiaWH0CPP3LR2ndnqDnYCtm0Wb3+/uYOjPP3KWFDn8+K3IlzWLkr87Tcc9AfXcy/eq1oLZElhGuixwKIQwDt1BA0nWIRnHy+WC73jCIluShfc/FF4FUClJQTIaikNjSysVX83i3eJeyiLmZ07R3HUSSVCRJprf/Pi6f/8t3pS1xIjXA4M4n6xX8peI4Y0Mv1ift3OwZYsk+unqPIMsqHd2HKZemmZk8tkZIThBTW3B8i5LXGOK7XresWbrzjaK8Rn2T70FlWXfRxVDrImwrMAKLEGIpRHU9LFOwWtZBiMBAscr6sXi89M4k/Xd2MHVmnvmRErsf72X4tWnKcybJ3iiSBMXpGrvf38fkqSzFySrhtIERVZm/VmqQiFkORQvRt+sxurc/gFnJUClMInwPRQ1hRNL07X6M/OyVvzajEm/dQnv/7YydfxbbbK6wt5QbY/LKD8jNXFxRfKsZMpkxk2LGppRzaO8PMTtaI9VuEEku6MZ5AiMS7E5KWZt4WkOSoDBr4/sCs+RRzKy9+JhD05hDzRN0rInG0JUkyXRuuQdVMxi/+MINhLMlQulO1FAMu5zDLq4eGpNkhVjPDqz8DHZxPmD1xZML5At/Y9n/ZVBCYWR1QdnDtvE3CIeuhU0ZlWpV8NoPlqbU22+sbxGED5nLRTKXiyiGTEtfDGeTHthaqEwUGP760hZw/NnLqOlWjN5eqhcvIBkh9K5OauUykqqi9/bilUoI10VJJvErFZBlZF3Hd1xCAwPgC3yrFuxcfB97egqtrY0aaZSBrUjzx98V7a989grl4iSJ1AAQKATn5q8yN3PqltIRQ+E0W3c8gaYHyVPbKjN86Rksa4lO5To1JkdfI57oJZboRdOj9A8+TLk4Qbm0OiMk784go+CIxuXWtkoN5x9P9AbS+f8fRXXewqk6CF/gWR7JvmiwYxIQimu070wydSZHsifC5Mks8a4IvYdbEb4g2RPl0vMTqx43luqjrf8wlfwEQye/iVXN4QsfVTXQQwlUI0at9Nekxi3JJNt3EIq1bqqvim0WGTn79Kp/m5+ymJ9aGmuF2WAdKs6tPTezE8H7Z4bfGxqZqkdItg3i2BXWI1SsCVki3N6P8FyMdCfF4dNEu7dh5edwKgXifbuwSvPU5sYRwkOLJLCL8wGTa3GNkmX0tg7MJvJKcihMdNdelGjQ4M0p5PHKGxc+r4Z3vU5lEZ7lk7nanJfSDGL9SQY+vBc9GQIJPNPl0tdH0Ts7MUdHwHNZfJjC8xaUbiXCu3ajJpN4pRJyNFp/AJKioMTiOJk5hOPUC4QWu9Yp0WhQNr0JSDJsvytNJWdTyTs4loeiyEgyOJaPoslU5m08z+ba1efYc/CzaFoEVYuwbfeH0I0Ys1MnFvqQLIn2LdajSJKCpkdJt+0ilujl0tmvsxbXSFFD9G15kGTL4EKvFofR4RdXbdRVKc8yfOVZ9t32OVQtTDjSyo49H+PM8T9etVpfCJ+U3gNOUAi5iHJpCs9z6ruiju5DzEydoFqeWcVzk+re1SIi0Q66++8hkujCdx1ymYtMjR9tihknSTKp1h1YZoFq+eZpnJIERlShcyDE7rsTbN0fpa3PIJpU0XQZ1/ExKz6FjM3sqMX4xQrDZyrMjZlY1eCaijM1rrw0Re/hVioZk2rWojxbI70tTsvWOFbFRVYlqjmL0kyVVF8MSYbCZJXS9NqLoR5KoBtxpq68QqWwZPht19pwZyDJSz1vhBAI311VlkdRDZBkPKe24jO+56x4JpIkIysasmrQ0rkL4XuoehhvWWtq167ROF4lFNVokNS/vuX10nkryIqO55goC2Em33PwfTf4XllDCA/PtVltTsiKHtRmLYy54BpWCUVqYXzfw/fslZ9xnRXRBElWkWWVcKydaLKb4vw1VCOydGwRhLxX3N9YCN9yEM7S8RRNJ9Q5QHHkPPGBvSAEiS17mb9wFKdaJNzWi13INCTThW1hTo4R2b4LSVFpuech3Pw85sQowvPqIWkJCWQJSVbQ0q2kH3yc6O59ICt41cpCEeyNhZHeM6Nyq7H9s4eYfnWE6tTCpJFVjIE9CN9HTaTwaxWUWAxZ15E0LfjdMFBTKeRwBDeXQ9Z1Kpcuovf2ocbimENXiezfjzk0hBKLo6aCZlmSrlF6+yhik/0OJEkiktKCBKKh0LMnhl3z8D1wLQ+r4nHlzSBklJ8fYvTq82zZ/hiqFsUwkmzf/VG6++4inx2iVpsPBrasomphQuEUkWgnsUQPkqRQKc/UJ/kqJ0JH92E6e48gSQq+7zE3fYrZyeNrhNgEucxFxkdeYWDwUWRFJZ4aYOuO9zN8+Xsr5PUlSUZFQ5W0htc912R26gQ9A/chSRK6kWT/4c8zPfE2ldI0vu8hSTKKqqPrMXzfITN7bsG7g2pllqsXnqJnywNUyzPks1dQtQhqKIFZnScS7cC2S6hqGFWPIEsKllnAMvPoRgLXNYPdEoH8jBFOIUsynmtTKU+jG3F0I4GiGrhOlWp5btX7EUko7LsvyRM/3sWO2+Mo6nqe55I8iRDw9P+Y5Cu/GbQEaN0ap3NvCrviUp03mT6fY+CeDiqZoC2EZ3vUCjaTJ7Nsvb+LyVNZKhkTI6qRs9ZhFnkOvu8QjrU1r30jyUST3XRuuZt0915UPYrrmBTmrjB19VUq+fEGA7/zrh8lEu/g4ltfonPr3aS79qEZURyrQm7mAhOXXsSsLIRoJImWrj10b3uAcKIDPZQAITjw0D9m+QJ/8oXfWvoMgRzL4G0fJ9m+rd5BdOzCc4xffH7Fgp/uPsDWAx/h6vGv0r/3/UTiHWQmTzN5+Qf07nqE1u4DmJUsw6efophZSoRLskqibZCebfcTS29BUQ1ss0h24jQzI29hlrMN53j7k/+c/MxFZkeP0b3tARJtW5FlDauWZ3bkbWZHjtZll2RFp2vwHlp7DhJOdKBqEdoiKdLdB+rHdO0q73zvPzXcW9lQaXt8P9ZMgfwbS2QKt1amMj28QAEPcryVqWtEe7ajRRILenAGWjSJ8BwkWUG4LuXzp4hs2xkIgXZ20/OjP401NY41M1WXmZI1HTWRQG/rQG/vClIHkoRXq5J/61XKaxEBmsD/tEbFLpiUR3NUJhaNioxWvYyby6HEYgFjI59fmGTgZrOARPXCedR0Gnc+h1cp45smzuwMbi6Hm88hXbqIm8+jtqSRQwZuLoc9NxuEyzYJ3xdMXy4jBBSmTRzTw/MEiiJRKzkYkaXbL3yX6cl38H2XLdsfxwgFib9orKuexL9RJBL9DAw+gqoaCCGoVmYZH3kFx1n/mibH3iQW76G1Yx+yrNDedYhScZLZqeMrdhr+GjukyfE3iaf6SST7kCSJcKSVwZ0fwPc9hPCRJTkovpQkivkR8vPDdaOyGqKxDuLJfsavvUxn7xGys2dJte7E9x1cxySZ3sbEtZdRtTBtnQeZnztPYX6IZMtWUm07KGSHSLXtZOjCd2jrOojnWoTCaYTwqVVeWrEex1tUnvx73Tz6uS6iycb2yBtBkmDk7JIxmLtcIHO1WCeJXHttpl6EO3Mu1/DdU2fmET4UJ6sb6idWSzPUSnO09h6kUpwmO3kaxyyt/QEgmuhi++FPoRkx5sZPYtcK6KE4bX2HiSQ6GDr5TUrZxl2sHkoweNvH8T2HqaHXAEGibTsdA0dQVIMr73wl2FUIMCvzzI4dQ5JVtu7/EI5VYebaWw1MQuc6Q+k6VSYuv0Rm/ASRRBdb9n9w3WtQtRA9Ox6iMBfki9r7bkdVw/i+w9TVV+je/iCdW++mnBsPcoaSRLp7H1sPfATXrjI99DquYxKOt9O59S4i8Q6unPg6zrLdnUQgExNNdlPJTzF+4QVkVae1ez/9ex4HIZi88jJCBOO5nJ/EsSuEoq307niYcn6M2dGlfKTvuSvmjhI1cAtV9FRk6UVfUMtO4ppVjFQ71dkRjFQHTq2IUyvhRkr4bpD3cGvlIP+xMDat6Ukyz32blvsfIbpjD7KuE96yjfCWbWveS991qI1do3jqHUqnjt1U2P2GjIqqSTzy2U4e+FT7DX/xjeBP/+01Lr8TTBZJkbnzXz9JbaaM73q4VYcT/+lFANyFHYWbW0pS1ZbpHnnF4sLPIJfgzi8lmM1r14LX8vkNz0cIgb8sXLDCyxUwO7S0QE5fvt7bbPT4PddkeuIY5eIkvVseoLV9b9CnflXpFVEPV5SKk0yNv7lqMlDTo2zd8X50PRp4tMLn2uXvNcUuc+wyY9d+QCjcQiTajqIa9G15gFJhjGplKUbvCw9H1FaVva+WZ7ly7i/ZuuP9pNLbFuLqi+E7efFKwN9Ebc6iJIYkARK+75DPDmFbBbr770VWdKrlWczqkkadED6V4jRz06dIpgfR9DCeaxOJd+J7LrnMlZXEAlXiwU918P6f6CYUacwHBL1NFmrJWBY1lxZryiQqeYerJxqfeUMr2ODiF4533cv+yveshVo5w/jFFxg89Am27PsgnVvuJDdzgdmRY4F+23X3VZIVurc/SCjaytCJrzM/dQ5/Qf6kND/K7ns+T8fAEarFmQbmkqzo+K7N0Mm/rO8wshOn2X3Pj5No3Uok3kk5Pw4IqsVpqsVpZFmlf/fj2FaR7ORprHX05ITvUcmPU8mPY5vFDY0KkoRZyTJ24blg4U/1Ekl0cP6NP8aq5oi3bcUIt6AZUayqTSiSpnvb/UiSxOW3/5xaeQ4hfBQthGOV6dv1KK09B5geaqR5G+EUE1d+wMSlF3HtCkgy+ZkLHHjoH5Fs387c+HHsWgHhuxQzVwGItfTTve0BauUMc6NrRQQWrtvzKZ2fRIsvbxcgsPJBjUx1OlhDnMrSGmbllkK65ep1IU4hqF27ip2ZJdTTT2THHkI9faipdBC5IaiR8qpVnFwWa2aS6tVLONkMbrm0vmBoE7ghoyLJ0NKpM3jgva2WDceXJvbVvzjJtb88uzTfrp+sq0COhFFaU0iyjJsr4hfX9+Y2Qi57iaOv/Cb13M1i1vUGoLalcXOB2FupOM7FM1/BCKVIpbeTSA2g6zEULRRQGJ0qZi1HpTxNITeMZRbWXJAlVWN45HkYAc+xsKq5TVEci/kR3nnj/2Zp2Vy5+CuSiirpQZx2BQSl4jhnjn+RZMtWWlp3Eom2o+ph8EUQorJLVErTFAtjmOb6bBXHMdGMOJFoB0Yk6McufG/ZpA3CBEY4hWbEMDwLRQ2K9Hx/0fj7SARMwXJhkmppGsepcv2WoGNLiI/+o94VBsWqesyOmUxcrjEzYlIteWi6RDShEE1pdG0NEUupDJ2uUCm+u+J9CzeA+amzVApT9O56mGTbNrq3PUj3tgeZGz/B9NDrVIvT9WtTtTBtfbeRn71MITNcN6bCdylkhqiV5ki0bkPTow1jxXMtspOnMStLxtqsZKkWpkh17kYPJyH/3go8lnKjCN/Dqs7juza2WcIsB+dnVfIL9OqAdRiOtRNPb2Hiyg+olufqi6fnmBTmLtO59a4Fo/I6y8eBWZ1nfupsYFAAhE8lP4lZmUczoqhqCJsb1yHzLZf4/j60dIzaRA6/dmtaIHvlEpVL56hcPr+Mzr98ji54RJtU4tgI/9OFv0LtUXoe2b70gi+wcjVmj61ftSvHo8QfuRc5HkHYLtblYWqnLzZljNbDpqiCsoSSTODXTIS5bJciSehbevHKlTpBQAgfszbP9MQ80xNHb/j8unY+SDjRjlnKoIZilOaGmR16c1ODaONrlDDkCPI6xY9qPIHbYjB8+buEOvuwC9kV3QHXYgeVC+PYC2ETs5qhUpoinhogO3MGyywgyTJaVxfWyAUKuWv4nkMiNYDnmMiKjqZHqVWzyAt9anLZK0EjKUlF1lVa2ncDgumxtxqICA99up1IovGcSjmHF/98hhe/PEN2cvXJr6gSrT0Gkqxh1967drZWdZ6hE98kFGsl2b6ddPd+OgbuIJbq5dLRP63vLkLRVmRFRw8l6By8Z4UWnaxoQY5KblwehO9Sq6yktroLhmcz7K5bBXehNcBiOHW5ERT49V0jkkQo1oasqIRjbfTufJjlhkMPJZEllVC0dUW40bHKK0J1guC6FXVzOnmrQfgCNWogSRJqPIS9zKjIMnz4wyEkCcplwfHjNnfdpfPss8H6YRjwIz8SJp8XZDM+x96x2bFDZfs2FQFMTXns2KHiujA761GrCfoXxFtfecWiXBF85CMhjh61mZ29NWP1fzqj4tsetellOwxZItafZPvWg1z4g7W7Hho7tiBFQhS+9X38ai1gmNykQdksJMMgcngftfNXcKeX9NEQguqxG0+MbYTs2CmyIydIdO6ge9eDzA2/jWqESfXsxbNrpPsOYpbmmLr0Cp5jEmvtp23LEWRFJTPyDsXZITbagQUEgbUnl97SRurQvdSmRgn3bcV3LOzlRkWSMDp6EJ6DdZ2A53KGmu+7zE680/B32ynTcegAhWtnF9SVITu7eldQgPnZ86haBEU16uKZtcoc/rJksGZIHHq4peFzvic4/lyOp78wSbXooRlxWrr2EIqkyc9drieEPVcwN+ZgRKIIIaHqYWRZwzYLKKqBqkfXlO25eQjMcgaznCE/c5GenQ/TNXgvbX2HGb/4PBD0/IGgL0k43rHqUXzXXvHEhRA3XLvwbuF6h2dtB0gKQslAS+ceUh27Vn2XcFay7ITvrdM/XuKGKMPLjyBJ+LaHWypizzaGshQFHn/M4E//rEqpFBT0/tAnw3WjEgpJPPiAwe/+twrlsk9np8LHPhrixZcsLAvyOZ/HHlOYnPTJ5QRHjmiUy4J0q8SDD+m89ZbDhz8UIhKR+MpXbg3d+oaMihBQzjvMjDQfRkl36ai6VE90CiGolTxKuea1Zeyaj10wmXp5Ge9aAkVXuOtXP7D2BxUFraMNZ3wav1xFbU0hbAfP9Qjt3U70/juQQwbmhSHKL71J4oMPgyyh9XYhHIfit76PM5clcudBIof2IMkKlaOnqB47TeSOA+hbe1FiUZR0iuwffRWEIP74A+i9nXiVGqUXXsOdzpD61AcI7dtB+I59eLki2T/8KkprisTjD6Bv6SHzhb/Ay+aRNI3og0cI79+JcD3KLx/FunyN9E98Cmd6FmPbAF6xTP6rT+OXmyhoEyxQIO36pJNVndb+2yjMXGHkxFNIsoLv2YTirbRvu4fM8FF8z6Fr14O4do1qfm3V0jatj6pfQLC+p1MdHyK+8wAgIakabQ88iayHqE2N4OTnab3rYXzXpnzlHHprB161stAD3UPWDbRkIASZO/km0YHtQSMoIahcCwpgtXiS6NZdlIcuktx/B7JmYM6OU7p0ZoWuletUmbj2cj1f1RhCg+7BMPG0xvIFo5RzOP7CPNWFkJZjlSnNjwQ9aKp50j0HkCQFz6niuTbheAfZiVOk2ncSTfUyP3kWVQ+T6txFduI0jl0h3b0fs5yhlBujvf8wZjVHfvoi3gaN1pqBVc2Tm75Aa/cBYqne+uuOVQEEs6PHGDv/vdXVEgQN9N+bx7unptAUhMCxqwghuHb6KTLjJxGrOUpiZSho1ffdQkiqjFOoYnQmVv274wimp33K5UBU9XpUq4KpKQ/TFDz4oMGVKx5vvrk03sfGPIaHPa5edTl0SKNQ8Nm6VWFyMjAyzzxjcscdOl+Ta6uql2wWN7xTefaL0zz9healRP7lXxxg+6GlHIzvwctfm+XP/sPKOon1oMZ0UruWVEAlSSLcGac2t16vCglUBVwXydBJ/fCHEI5L4annSX7iCSqvH0dYNvFH7sW6fA0lGceZyZD5b18i/sQDGDuCRl6x++6gevQkKAqxB45gD48hqQpaZzvZP/wqvuOA4yJFwpjnr2BdvoaxfYDwgV0UhsbI/9VzJF2X8g/ewpkMEt1eJkfuq9+h/R9/vr6N1no6MLYNkP2DryIZGi2f/SjefB61NUX16CmKT/+A9Oc/gdbXhXVhY92gePsgsqoTSXWRm7pQXzyFEGSuHWtgChnRNKFomnCiA5BQ9AhGLL2uUck445j+xoVSbqWIrOqo0Tjhrn6UcITK0EUiW3ZQvXaZ0tWzOIV5zJkJWjt6EMLHaG0HWcHOZciffgvheyT334nwPaojV6hNjgTy7ZpOyx0PUjz3Dk4+E4hkjl3FykytKZR4fWJ+Odr6DFRNaohsVAoe45eWG3ERqDD4XkAfj3cCAteJMjd2nFCsFYGgVsngOlVKuZGF0JNGMTtM17b7sao5VD2CqoXwPYfs+MkN7+NmoKhBC+Ll4ZtaJYtZmScSb0dWtKar3G8EgsBgywvNtf76IKiWZnCsMvHWLcyMvH1DSuDNf12Qe5QVfUP2niRLOPNlfGv1cZpMynzsYyEmJz1efHGlod+6VeGTnwxz6ZJLPC6Rz61tGeIxidtuMzB0iT/+4yr/5GdjzM15KEpwnKGhm88BbvopGxGFOz/UzoGH06S7jZs+gc1CDWukdnfU/0/sbEMx1PX70/sefqWKHI8hLJvCXz2HVygiR8LIkTB+uYqwHQpPv4hXCBZY6/IweH6wE1BU5EQUFBm/ZuGXqxSffQWvWkMA9sQ0frUWtB4EjME+QrsD+p5wXOrNZ5qEkozjlSr41RpeuYpfriAngoZk5qVh8Dz8UhXZaPb+C3zXYn7sFJlr79Q9seU7l4Z3C3+hqM0mO3KcyvzqldyLaMagAAjXxZ6fxWgNQi6SrOA7FsULJ5bCKpKM8AXCsQPevefVpSMW3hD88H282hKzTpIknEIWLZECSWL+nVeRVJXo1t31AtbNIJrSkJVG79qxfIqZNQyU5+K5Fq5dRUIiHG8nmuwJ2GWuTSjaGhRx+i5aKE401YdZyRKKpnHMEr7n3NDOQJJkUp276dx6D9FkD4oaWtDaCtPStYeuwftQVIPM5FJ41fdspodfJ5Loom/P40STPfW6kFC0jba+wyTbd94iIyAwK/OEY+3EWvrrjlOQPH9vdy+10iy5mQu0dO2jd+f7CEVbkWQVVY8QTfbQOXjPmuHAzcLzbByrTKJ1C+HYEktWUVfOWSUewuhIooRXlzHKZn2+8IUK3/ymST5/3S5KBIrev/d7FV580eLqVY9du9RVRXgB8gWfP//zGnNzPvfdq2PZgvmc4PwFh337tNU/tElseqcSTarIMnRuDa2QS7hViIQldgxqZPMesajM3JxHKCQRMiTGp6uUfnCefCFQSgbwXR/hrrNv8wX26BTxx+5F39aPpCpIsoxXLOPOZfGrJtbQKGpLsh5OEl7j8bxsHr9UwZ2bx83MI8eiiNrCtS/fM0oSajqFX7OwhkYJ7duBv5iUX8jhyPEYUqgQJOsVBTmkIykycsjAUxWc2QzRew6hdrYFstqxKG5mfuV3NYnS3DUyI+9s/EbAKs9jV3KY5Sy14ixaKIZ/C8IgwvcCo1KYxy0XqE2NokbjxLbtxcrOYM1O4VUrJPfdgaxp2IV5lFAEp5DDrVaws7OkDt4FSOROvUl8295lhZ4Ct1qmeO44sR37CfcOEt2yA8UIYWVmArG9TUIPyQ1MbiEEniuwzcb7b9fyZCdP43sOsyNHEQgkZITwmKjMLzRl8xb6qdv4vktm/AS+71ItepRz4/Uq9szEDchiSDKxVB89Ox7C992F2P+CZvSCMzN24VmKmWUhYyGYHXkb3YjTPnCEVPvOunMhSUGV9dTV1yhmh26UzLj0Vb7HzMhbxFo+xbZDn6B/7/tB+AgBZ1/9/YaakHT3AVIdO1C0EEYkyGe19x0mEu/AdWoL9Ny3F8J3m4drVxm/8DyqFqFnx/voGrx34boXKO6ywuVjX74lkjZWNcf81Dm6tz/Avgd+eoFAIOFYFc68/LsN77VnitiZ0pr3OpmS+eVfTjA36/PVr9VIp2V+7dcSjI97PP20ycEDGr/2awmGh1y+8c0atx/W+Le/liCT9fnN3yxjWQLXDUoQbBssS/DNv6zxz/63OH/1VI3nnjPZtk3l8GEtEFG9yQ3cpo1KftYiP2tTLblkxm69+qYkwZOPhSmWfGYzHj/0kQhjEy6aJtHeqvCHXypxYIfM28cdCmbzC6w9PEbl9RDxR+4J+O1nLuGXyuT+4jvEHrqL6H2HcWfnKTz9IvbENKIWXJs3X0DSVNxsntILrxN94AhyOIR5aRh3bh4vV2yMwQqBef4KsYfvIfXJ92OPTOKVg0ngV2uY564Qvfd2wgd3k//q0xjb+gkf3odvO8Qfvofq8XOYl4Yov36cxJMPITyPwlPP4+VLWFdH6w29nOk5vOLGi1CtOINdW0l39F2bSm58RQLSLGeYufo6rVvuoHPHfVTzk8wNr02AaBbVsaUw3fSz3wDAmmsMn1auXarnR1Y/xtX67/PHlmTVhecx9/J3AcifDCTMa+ObkxO/HsoqRKZFufTlEMJHuEFO4vqdxnJ5EXdZQedyYcflr4sbMN7Cd5kbewfHrhBNdqMbsSCv49lUi9Pkpi9QKU6tyBO4dpVrZ58mO3WWdPd+QpEWQGCbJYrzIxRmrzSMjXJuHM+xAqXh61AtzVCYu7xmGG1+8iyuXaW15wB6OInvOZiV7HXOioQejmNEgryZ51jMT58HEexqdEUP2nIvMNIcs0h+5iLuQljP9wI69PI6mGphCt+1GxhhZiXLxbf+hHT3fpJt29BDCXzPxqzmKWaHKWQax01+9nKgpXa9VIwQlOZHkBVt1fyX7zmMX3qBWmmGVMduFM3Acy3KuZWUa+H5rNWKyHHgn/yTfMNrn/vRRpLHJz7ZyMj7/f/RaHS/8Y2l8/v615ee39//B0v36uJFl4sXb004UBKr6nqs8saFbasektlxJIHr+IxfrFItNHciizmVxeN4ruC5P5lakVORZfiHfy/O//hiCceBn/nJOPGYxFzGp7ND4Q//tMShAwZH3zEprqFK+7f4W9wsnvzJLj798wMY4cC6CCG4eqLMv/3smb/mM/tb/C3++tCMudj0TsWxfcrzDvsfasGu+Vw7fWNKlmvB9+HcBYe/92NxXn8raODluDCb8TBtwWc+GWNswl21KdgiJIkgHr5Ur7e6bL0EuiGzZX+U3Xcm6NsVId6qoelSED/PBgy30fMVLh8vU8k7rMks3ACL55Ro1ejfE2Fgb5SuwRDRhEokEfRMt02fasFlbsJi4lKVqyfLZCas4Nxv0H7KSpAIXITviRUFs7ICLR06u+9OsO1QnNYeg0hcwfcElaJLMeNw7UyFi28XyYxbTbUAMJI63bd3Ekro5K4VmTk1t+Z7JSnoWdO9LczgwRh9uyKku3VCUQU9JOPaArPiUc67zI2ZTA3VGD1fZX7KwlvlepqGtFRiEBTnS8hSUGQbb9WQ5caYvyQH59kMgkZQN3hey74vllLZdSTBtkMx2vsNYikNVZewTZ/8rMPEpQoXjhaZuFTDtvybDlctYsW4cVfu0iBwAlOdOgceTLFlX5TWHoNwXEH4UC25ZCeDsXzleJmZkaAZ3rLO1aiajGsv08FSpHrY8fp7GE+r3PlkKwfflyKSUCnlHC69VeSNb2coza90bjVDYt99KY68P03HgIFV85m5VuPUD/Kcf6N4S1pZyAq09YbYfVecrQdipLt1IvFAdNIsecxPW1w7W+Hi0SJzY83NnSWsneGXZZCW5fyEJ1aNjEsyxJIqu+5KsPP2eNDgLBk0NjMrHrkZm+nhGpffKTFxuYZV8262oH7zO5VIQmHXXUkuHytSLboNJxAJS7S3KUgSzGU8KtWlQze7Uwm+KyBrLU9rCLHwuirhuqvfwEXsvz/JP/2d3cgLwn/5WZt/+9kzFJZJY8dSKgffl+JDP91Dz44wsrzA9Lm+4HThf9v0OfHCPN/87fFNUakjCYX2vhC77oxz5P1pBg/GUHV5sT30qrlKIYJiX98TXDtT5oU/n+HsqwVKWWfViR2KKiTbNMyqRzSpUisFgyUUlfm7v7adI08u9WT/zu9P8pe/PYbvBwtk/64IT/xEF4cfTROOKSzIcK08HwGO6XPhzSLPf2mKi0dLK/ILy9F/fw+R1jC+52NXHEZeWrntDyakweHH0jzwyXZ6d4aRFp7DqvVky56H7wlmR03OvJLn5Et5pq7WyM3aTS2q8bTKjtvjpDp0Wjo1WjoNUh3awr91QhGlvrgtp8AjqOfxNsLx5+b54r8epiaFUdPJhQuWsS6PbvhZPSyzZV+Uhz7dwe2PtdSdjhX3ZOFeeJ5g6mqNF/5shpPfz5Gfs29qYUh1aPzEv97Gwfel6q998V8P88rXZ+vHVVSJrfujPPKjnRx5Io0RVVZ9bovPS/iCuVGLb/33ca4cLyOrMqom0TkY5u2ngwr41h6Df/Sfd7L1QCDKeeqlPF/4/12hWvLYfijG535xC9sOxVleY+l7MHm1yld+Y5QzrxbqDbfSnTqf/aUt3PmB1kBcXFo6H98TvP3MPF/7r6PMjd5YvjCaVNh5JMFjP9rJziOJeg5uret3LZ8rJ8o898UpLh0rUdkgwhNKdhBO91DLTizsDgRWKQhzqbrED/3Tfp78u9319//gK7N8+T+N1IttJQna+0M8/JkO7v9EG4lWfd0x5PuCQsbhpS/P8O3fm1zT+N3ynYosg1n1ufBGgWhSRVGlBjbMY+8LE4sGfbjfOGpSqd6YqyZEnUi14nXb3viiJBlUXa6ryRphhWS7XjcqXYMhPvIPe7nrg60rJDgaD7S05odjCm19oaYJK7IMhx5p4cj70+x/MEWqQ2tajFCSADmYuDuPJNiyP8bpH+T5y/97jNELK+tSoimV2x4OvLf5KYtSzqWYDa5VUSU0fSnjvGVfFEmWCIVl7v5wKx//2d6g+nudc1v8kxJVOPxYC7vvjvP0/5jkxS/P1r/nepSnKuRHivTd241ZXDlxFVXi3o+28djf6WTrgdgGyr+LJ7J0+xVVondnhN6dER7+bCdvfivDn/zqMG4T42PPvUn+yX9dvfhtza+Wgi/X9OaeoaIG75dUBa2/E2HaaH2dWFfH1yVbtPcZPPTpDh790c6FOpn1Tiq4H7IiMbA3yo//yiBH3p/m2S9Oce71QlP3Yr3zXz5u0l06sizh+QLNkLj/E+185Gd66RgIrXOUZYuYItG1LYQeVmjtC+FYPrIi4SzbpSzuWBe/t6VDJ9mmE4n7/J1/sZXth+KrnCf0747ymX8+QPGXrnLtTIWWTp3P/OIW7vpg6woGnySBLEvc9YE0Qgi+/Osj5Nfpw7La9fTvifD4j3Vx38fb0EPrqwjU505EYf/9SXbcHuONpzI89yfTjF+qrilsEW7pDnqy9O8DScapFbHKgY7b4jNf/nziaQ0jLGPXfBRVYv8DST7+s31sPxxbf91ZNoZauw3C8Zuvh9/UEcIJlVSHzpYDcUIRmdHzlQaj0toic/KszXzOJ7sOV/q9hqJKJNuCCZrq0PiJXxlk110JVG3poYhFDZxlWP4wfF8wPVwjO9mcZyME3PXhVu77WNuqD3Uji7/8M3pI5vYnWoimFH73n11u2HEBlLIO09dMVE0iFFWwqot9E1Z+b//uCHpY5t6PtvHJn+sj0ao1euMbnAsEO6MP/4NeFE3mO78/Ue8Xshz5kSJCCMbfmEKLXDfMJHjkc5184p8E378czXhC15+PEZbJzdycd/5uwZ3LY10exdjehzMxu648TtdgiM/8whYOPJhCMxrpvOvdl3rba0XiwANJugbDPPU747z8tdnVv06SUNuSSNpC0nsqs+EOr6Uj8HQ1Q+Kxv9PFR/9hL7GWZSrbG8wfAKvmc/LFHPlZOwhrSTQsjNcj3qrS0qlzxxNptt0WW3EPlh+/d2eEJ368iy/+yjD3fLSVw4+2IMkr79tSi3CZ2x9Lc/rlPK//Vaa5sSPB9tvjfO4XtzB4MIqiLl8/mpvPRljhgR9qp3tbmP/33w4zem714uXyzDCKHkaLJFD0MNX5cdY7yWgyCBXLMtz2cIrP/vMtdG4JLZvbsNHzcWyfi28VbjosuLnOjwUXx/TJTlqouoR/3ZcPj7g88kAIz4NvP1tl6FpzSXxJ15AMHeG6CNOuTzwlncQ3LUR1g3CTqsBiI65VoKgSqXadWIvK3/u17ey5N4ksB71HbNPHqvhBPqPsYpY89IhMNKlihBWMsIwWCnpgX36n1LT3JwS88rU57vpga927FULg2AKr4mHVfGZHTTLjFpWii+8J4mmN3p1h2noNYkkVZZnRk2WJXXcm+NT/1s8f//JQQ6zZsX0mr1Yxwgq1kkchu3buJ9Whc/cHW/mhn+sn3qoGPRRcQa3kkp9zGLtQoZh18FyxkP+J0tqtE1mIwwYNwiSMiMzjP9bJ+KUKb393vmHxinZGSA7EiXdF0WI6bs1l/nK+/vdDD7fwkX/QQzy9fFESWDWfSsFl5lqN2RGLSsHF90XgzLQHQo3RpIYRkTHCShD3lyTmp2xO/6D5ds+O6ZOfXV+0zwjLhGJKg8H13KDfeTOoFN26IrGXK2ENT+KX1+7R3tKp81P/bjs7bo83eNeLyhOFrMPU1RrzUxZWzSeeVukcCNG5NUwkoaIZ0gIdWKKt1+Dv/Iut+J7gtb/KrOjB3vrZx1CSUfyFuZb50+fAXT+q0NKlo+kyd34gvcygSLiOj1X1sape/flBUHoQiiroYZlQREFRJS6/XaQ0H/SLv+vD7QweijN5qcLzf7J6EXWiVePIB9Lc/lgLjiWYvFJl6FSQw91zT5LOLUZ9YZckibs+2Mrx53I89OkOjIhMtegxer7CyPkKkbjCwYdSJNv1eq7MiCjc/eE23nkuh1neOKrSvzvCP/iP2+noDzXkm1zHp5wP8keTV2qU5h1kJXBk+3ZHaOnQiSTU+nNVNZmdd8T5md/YyX/9mQtkJlY6qosS92ZxFonG5nWrIZbSMCIKg7fF+LF/MUhrb1AH5HsCs+ph14LnVCm6eI5POK4SSQSGyAgrqLrE1NUaMzcYDlyOTRkVIUBWg5i3a4kViuy2IygUfeSFHibNInRwF3p/N16hRO3kBZR4FL9mog/0IBwXZzaLqJlBxannIYdDyJEQqCrefJ7QgV24c/M4U3N1KvByKKpE12CID//9HvbdHxiUWsll+EyFd56f58qxEhNXqzjm0kmHojLd28JsOxRn++EYqXadS0c3V3l86e0iF48W2XdvkuyUxej5KsOnylx+p8jo+Sq11QayBIP7ozzyuU6OPJkmllry5GVZ4rb3tbD9ULzeAgCC/NDtj6WZvFLFtf0Vxn45ZAU+/8uDdU+4WnQ59uw8rz+V4cqxUpDsXQZNl9h5Z4InPt/FoYdT9US1JEnEUhqPfq6LMy8XGq6lNm8SSuhU54LWzOH0UogkmlS572OttHTpDQv2xOUaL/zpNMefz62ZG5FlaO012H44zraDUfr3RundEeHCW0WmrzWf5zr/RoFf//zZdd/zwA+184G/14MeWlo8Ji7X+N3/rZHyLCkqYqGBUgCB8H3Mike15IEsoW/twdjRj/B9it95dUX4KxxT+JFfGGDnkXiD92hVPa4cL/Hil2c593ph1Th8z/Ywd36wlXs/2kbXYKieGzQiMp/6Z/0Usw6nX8432DI5FmL295/alPZdS5fOtkMxPvG/9BFPa/i+YGqoytlXC5x/vcDVkyWK2aXzU1SJlk6dgb0Rdt+VYOuBGG9/bx7P9VF1hWtnyniuWHcxN8IKD/9wBwJ45g+n+O4XJinng+/o3hbm8788yN57E3UjoRsyn/75frq3RcjP2vzFfxrhre9k8TwBEuw6Eufv/uo2enYEfUskCXbdEScSVzY0KulunR//5UE6BkIN4zY7aXPse1le+foc45erKzYTqiax++4ED/9IBwcfStXDS5Is0bs9zOd+aQt/9K+G6tfVCLHw38bPKZpU6doa4qP/qI+2vqDAcn7a4txrRc68mmfoRInZcas+ryQJEm0afTsj7Lgjzo474oydr2zobDWDTRkVSYLddyVo6w9jVT3GzpcZObvEiT64T+epZ6rs362TSm6yGlcIhO1gbB/Ar5korSmU1hb8QhGtqw21NYV5eQRRM9G39qG2p7GHx/FyBdTWFF6+uGa+Q1El7vxAK0ZYRtVlChmHZ/5okjeeyjA/tfpNNCs+w6crDJ+u8MrXZNr6DGY3WZfjuoJv//cJLr9d4tKxIiPnKnXdqLXvAwyfqTD7GyMU512e/MmuOq0VAgOy775kg1HxHFFP0q+V41iEJElohrRwjR7f+L/GePUbc8ECuAocW3DutQLTwzVcZwt3fbCtIdm38444vTsjXDm+dD6+4+NaHrHuGHNnMpSnl8ZIW5/BwL7GOG+16PFn//4aZ19bWU+zHL4Pc2MWc2MWb347Q7rLoG93mGLG2TDxuRxW1d/QCJXmnRUhDcdq/Jy6oDVmzk5itHbgFPMgSfi1Ktb8HPiAJPAKZYTj4pUqK3cqEtz94VZufyzdcE9sy+f1pzJ8679NrOrJLmLyao3v/N4EF98q8Pl/tY2+3eH6bjLVofPk3+1mcqhGZnzZMXxB62cfw82VwBcUnj/WyIpZBe29Bj/88wO09YYQAk58P8e3/tsEYxeqONbKz3quIDNhkZmwOPH9HK3dBtVS0PXUqvnkZ2yqRXdD51PVZS69XeQ7vzfRMEanhmq8+OUZtt0WJRxbWqh7tkdwHZ/n/mSaN5/OLu3SBAydKnPi+zk6t4brObxIQqVrMLzmOgALIb8f7WLwYKzBoGTGLf7s10c4/XIeZw3SiusIzr5aYOxClQ/+VDdP/Hg3ekiun+/e+5Lc8f40L391jVBlk0i0qnzsZ/sY2BNFCBg+Xeap3x3n/BtFzMrKuS0EFOYcCnMFzr5WINmuoajS6o7uJrGplV8IuHa2wtlXcxx/LsvE5cZ44LefrfLxD0Rob5MZGd9EIY0v8PIlhOuhpOK4U3MIx0UOG7iZPH6lhhyPIocMpHCQLK8dP4fe14UcCeMtVLrXK9yvgyRDx4BBsl3Dqnr82X+4xrN/PLXuQFoOq+Yzcbm2eYqogItHi3z79yc4/0ZxY4OyDJWCx9NfmGDicq1hcVN1ie4dYcKxJUNTq3gcf26eS8dKTScdPdfnO/9jgpf+YnZNg7Ic81M2z/zhFDPXGovfVF1i333JFe8vz1SxChZ7P72TnruWOlfGW1TSXY1yFBffKnJ5mVFqBgKJ7KTFye/nGT69ToX1TcqSr4fF9q2h9m6USAw1GkOSFfS2TuqDRYCbzVM7cwV7eGKFUekcCHHfx9sIRRvj8+deK/DV/zy6rkFZhOsILr5d4g/+jysNAq2yLLHnngSHH21pYEyVXj2DeXkCZ3oee3p+3TzPIkIxha0Hovi+4I1vZfh/f3WYoZPlVQ3K9fA9mBu36oZfAvp2R7j9iTSJtvXJCEIIXv7q6mP01Es5yjl3hfGfvFzj5Iu5FTt21xaMXqhQLTWuTT3b1pfx2XogxpEn06jLSBquLfiL3xjhxAvzaxqU5ShmHZ763Qkuv1NsON9IXOGuD7aumBObhWbIbN0fRVbgyokSf/JvhjjxQm5Vg7IaCnNOsB7eAkr6psV9yjkHWQ76RSwyp2QJohEJ14FnX6wxMeWRTDR/aK9Ywrp8Db9Sxbo8QujQHiRdxxoaQ+vpQGlNUT12Fr2/G2OgB79qonak8UwT4Xk4IxOE9u5AaVld5XPRcxM+PPsn07zxrQyeHkXv60Xv60EOb14bqln4HjfMwqkWPF75eqNkhCQFsdpYKvDONCOofekYCNG/O1KnY64HIQSX3i7x+l9l1qUFX4/xS1UuHC2umMS77lzJylFDKpG2MK7pYeaWvHtFXdopLaKUc1Z2RFwHcjxGaPt29P6+ICamKEjawuIky0h6MEHlSBg13cIi11UKByFT1IX3yzenbeUU5imeP07hzNvk3n6Z0sXTuKUctfFry05WwtjRj7GzH7Ut1fB5SSaoDbqtMexVKXh8/bdGm87fACBg9EKV7/3hZEPdh6rJPPZ3uogml4ISbq6EHDHwilW8fLmphWTx/K6dqfDt35to2iFb9VR9QXbCIjdtk2xffzG1az5XTqzucFhVn5HzjY6tEDB2sbomoSY7aa8IdaW71z4HzZA59EgLXYOhhmd09LtZjj07vylHs1b2+KvfmWjI/UmSxI7bY/TtjqzzyY2xeG7FrMtTvzPO8OnKrey7tSlsmj+W6tDZc2+S9r4Qb383Q/l4CVmBRFzmkQfClMo+WwdUajXBEM1NCvtq0GBrUbnXvra6gKEzNr3kVUkLhUEi0OWyl/9tDUwP1/jBV2aQVJXwnt1ImoZfqQQyKrVb00vgVuPSsVK9RmcRRkiu50SECOK2qi5jm35TnqNj+Zx9tdAYEmkCVtVn8koN2/QbQnLt/StppYomkxsuMPrqRMOi5ToCu+YTji8t6P17Iqi6jGM1N0OVeBxJ19CSCRACSddQk0lqFy+jtrUiaRrWyChyJIISj+PmC+g9Xajt7TiTU3VD4+byOJPNK203A3s+0/iCAGHaAQHlOsMZigY008VwyCJOvJhj/NLmx6PnCk69lOfej7XRt2vJuejeFmLnkQTvPBvIeySfOIJXqqJLYGzpIvtnzwfCpxvAsXzefibLxKUm2i2sA0kOWIojZ8pkN9AOzC4QE9bC3GhjGNP3BJlJa00PvVryVjh5y/OW1yOaVDj0cKrBoJgVj1e/MXdDxa1j5ytMXa3Rv2fp+YRjQd3UmVcLeM7NWYJzr+U580r+po5xs9i0USnnHM69mmfwYLwef3NdmJnzeOnVGrmiT1uLTNV8F8zkdRpba/5t1Y8Kjr+QIz/rAApKLIo1NoE1NIwSixI5fBu18xfQe3pAkjD6+wLlXF/gmyZ+qYx5dYjYPXdSu3CJyKGDKLEYtfMXUGIxnNk5nNk5ko8/QuHZF27ZJc9PWUHyb9m6oxlBbgiCXdD8lE1uxgFEU/Ue5ZzLhbcKN+TJFGbt64xKsPNQdWlpssoStYKzasOjct4lN2s38OEH9kZ55LOdPPOHk01PVL9mIlwXtSWF2tERbEYMHUmW0Ht7sCeCUJNsBDU4cjSK1t6GPT6B2t6ONXwNJRLBkeUbEukMmnvJC3pf69xIIbDHZ/HypTqFdxHhaFBAtxyeJ3jne9l1yRbrYXbc4tqZCr07Iw0L4Z1PputGRYmGKL91Hr2vo+m6KyEE5bzL29+bv2kPWFYkOraECccV5sZMLr65di6tmHFw11loS7nGcK9t+RTm7DXP0TH9IHG/DNcb9eXoGAjVE/uLmLhSZWb0xnQPHVswfKbSYFQAth2KoWrSzRkVAd//85k155CsQCiiYFa9m1Z7WA+b3v8LH/IzNm88NcvklSWPxfehu0vBtgMmjN6knMV7BbPiM3KugmP5CMeheuIU4V07SD75OGpbG0oygSTLQR/7SAR9Sz+1sxcov3kU8+IlIocOoLW3ISkqem8PcihE9fRZIgcP4M7nMLYMYAz042ZvbUc/zxUrdh+SLDUw71q6DZ748S4Gb4ux9UCM9SBEIL0ydvHGvE2z6l+3fQ9i96HogpGRILKzm7YP34GSWLmlz4ybDJ+u4C/z2hVV4pM/18dP/pttDB6MLh1rrWtwXbTuLtSWFmpXruLMzOCVy/i12oJ+RaC0qyQTaF0dyPEYsqYjPC+Q2y+X8YpFfPPGBVElRaWj7w66t9xDONq2pky8HIsgR0LI0TDhA9sbtpxd28K0dDZ6yXOjJtPDN35eZtlj5HxlRRvjnUfiGJHgHCvHLtHykfuI3bsXe2y23sJ6I0xfM5ndhJrEWlA1CavmEY4rQVhunaWiXHDXXWiv38W4lr8GkyqA5/orQq3rSe/suy+5wlEbv1ilss53rAfPDerdrkffrkhzBcDrYH7GZvT82vPaCCsLxIZA/aCtV2fLvgihiEzfrjCDB6JEk4E6x957E3QMGDeUktz0TiXWorLjziTDJ0uUsk59m9malvnQ4xHa2xTSLQonz1hMzbyL5nCTKGRscjNLcWA3lyf/nWcIH9hPeM+uoB+KFMjPC8sB18MrFsH3g74mpTKh3Ttxc3mQZLS2VtzODsyrQ7iFAlpnB8b2QSpvHbv1J3/9nLruQbuWz6kXc3QNBqy8jZAZt1YtWGwGnruK1pYUFN4Fv0vBPZQkZF1ZIb5aKXi8+e0M++5L1lUGJEnCCCu870c6OfhQihMv5jj9Up6hU2UKWWfF9buzc5Rnl7TEamfO1X+3hkewhkdACKyha1hD1wAQfb3Yo2MgS1TeOQFC4OUDD1lGDppJbdC5sgECauU5ZFklFG0DgpbE10PSVZRUDDkSwreD3eQiBvY07iYgCPc0m1xdC9kJC7PqYSzkPBfvb+fWMKPnKlSOX8LJFpB1DWtkumlq8fCpjXX+JF0PcpSLckwLc2g5amWPqatVQAShtHW+3qr6DQ7I9bj+b54r1h3bC2o7jee8zsK5ZV/jjkIIQX5ht34jEEKsIAoAxFu0hZD2jT/70fOV9Wu1pEVliKDObNeROK4tkBWJnXfEA8YjkGzT6N8Vxix7zIrN161sPvyVdzHLLi1dOnbNq08A2xacPGszPetxbdRldDPsr/cAZsWrh+skwyBycD9qaxokmdrFy4QGtxJ/4F6kcGhhIWp8OLWLl0k88hDmcy/gFUsYA/0oiTh+rYZfreGbJgrJ5rxfKXhwHQMhWjp14mmNSFypU551Q0Y1ZDRdwogoaOtszyFo7VwpuGQmLYzwxpvPZhhFNwwBXrmGsNw1QxDnXi/wvT+e4tM/34+iLCUZJQnS3QaPfq6TO55IM3mlxoW3ihz9bpapq03mGNb4UvPSFeRQCK9cbniPik5SbqUsClii+d2bEB7V0gy2FSSRFXV1uRJvvoiXD6i7kq41DKvVJE7yMw5W7eaMSm7GXrFTUTSJ1m6d0XMVonfuJnLbdoTnYwx2U3ju7aYMy3KnbDVo7e0k3ve+oHncwj2e/8538ApL4S1JCmjC1aLH+IUq7QMhctNrH9ex/U0pJfi+wHVunbRC59aVJJ47P9C6InzVNCRoXaW5oSQH4dDr1TI2g8LcxoSXdFegb3fixRy+D8l2janhGnbNY2q4BiLInUUS6rq5rPWweZViy2fqalDUtpwiWyoLcnmf0+ds+npUUkmZ6k1OjuvRfVsaVVfIj5VxTA89quLUPGRFQg0pFMbXppa6lqhT/4RlUT11GkVV0HXwqxa16XF8ISF8gSQ8SiNXkfFhIeTujI2R+8rX8GoWCEHhuReCGeL7SKqKbBjY10YRzspBoagS4bjCwJ4Itz+WZtfdCdJdOqomLQgXBuq4i57EotCkBBvGvBVNwgjLxFIasRYVzZC5+NY6RZoCak1QiG8YQiBsFxArvPBFuLbg2S9OUc47fOwf9dHaY9Sr4yH4mWrXSbZp7DoS5/0/3sWV4yVe+sosV08EYnybjQkLy8KzVhrTpNxKVE5i++amjEookiaoKK/h+w6eu4bRkyRCewaJ3r0fZInsHy0VHaY6VrKOahVv3RxCM6iVVx5DWVDIBogc3kn2z54P6lU+9xiSojTVWnejWiBjyxbssTEqZ87Ujcr1TpasSnQOhujZHkXVJTRD5vLba49Xf6G5VNMQ66qZbArL5Z0WIUlLmnO3GtdL82wW1eLajhwE4+K1pzJB+2LT59iz8yiKhGUG9VeeG8zZBz7RxjvP54ilAn3Hzcq2bNqoJFp17vxgG74vOP9antFzSwv5nl0aU7MuO7dpTM96TE7f2sXLs306dqfo3JsiP16h+0CamXM5tLBKtNXg2J9ewVvDS/F80SBXIWyHgS2C227XGR/VMGsCIwS1qkCSZDTNZ3xMpr1DoVT0iSdkPN/j8vmgvlXYS95VaNdOkGXsVZhErd06hx9r4b6PtbPtcGyFnPrNYnHgd20LI0vSqlvr5RCAeYNCn01BImhqVqiu6J65HK4teOVrc1w7U+GRz3Vy20Mp2voahS0lKSAAxNMytz+e5tCjLVw9UeLVb2Y491qBuXHzpheQkj+PKulIYnPPxXWqtLTvRjdiFOaHVzZxql8EeMUK1dNX0Fob63kWcxzL4Vj+ClmVzcI2Vx5DkqnvYn3TJrSjD+H7SJpKaFc/biaPM5Nb97gbMQv9ahVJ1xG2jVijfaDnCCYvVZm6WsP3xIYChjfyfG8VRcgIyysEKd9N3GxJ1aIjIWsyyb44WkxDkiWEJ5i/msetuThWEOgFcCyBs+z3AIJTL+dp6zUYv1S9IR2wTRsVq+oxP2mR7NBX8L1ffsPkwF4d34fpdyGfUpquooUUQgmdwlSF0kwNqxQs7r4r1jQoa6FvQMVxBDt2B0ZlYFClVPS5eslBkiRME3bs1vA8QTwhMzbicuXCygiLeenyqscfPBjlU/9rP7vuTNTj22vB90VdP8k2fRwz+Ol5oqFlwPWwaz7TwybT10xkWWoo0Frzu25y0VoXkoSajgbJ3w0ZeTB2ocqf//oIb383y6FHW7jjiTTtvUaDttIiZFli5x0JBvZGuXamwqvfnOO1b87dnGcvyUiSjFhgqDcL26pQnL+GEAJ/vW2TL3Cmgi6d7my8Icy02oK1Wr+bzcJbpfeJJC19nz08hdHfgfB97Mksek8bwvM3NCprIfnoo2htbciRCGoqRWTfvvqOPffMM0FeZRn0BdXwzLjZdJO/vw402zvnbxokCWRVpvfOLjIX54l1R6lmqpRrzd3r/KyzwJK9MWzaqJhVj6snS4RjSgOdT5bh6rDD7JxHZ7tyy7yF5ajlbSZPzSMr0qYNyGo4+nqghaPpwYMwDInBHSqnjjvIciCzMj4abCklwHHW7+OyHNsPx/jJX91G/+7GZKzwBa4jKGYdzr9Z4NrZCtNDNXIzduCl+sF7Fn+GIgr//ruH1/RiFFUiklCQVYloQiUUUeqie38t8AWVM2OgSPi15grkHNPn/BtFrp4s88KXprnt4RYe+nQH3YOhoPfMsr4mELBYdt0Zp393hO2HYnz9v45RyNzYJBD4FL0M/iYTpLoRJ5EeRJIknMkKrrN66EyOhIJcCqD1tOGMT9eN12pFsaomBc2XbsJQarq8oq5TiID6r3akkMMGkq7iTGaoXRjFr9TW3VVuhOrZs/WC0wZIUsDIuw6+Lxg8HKNza4j5KZvxizfWc/7dxmpeuuv4XDxaZLLZHF+zECvp0TcKz/bJXs7Rc2cnlbkqkbYwsnpzobXNYPPFj+06d324jfGLFcp5h1rJQ9egt1vlyCGDaESiv0/luZdqzMzeut2Kqi4IEfsCbxPV1+uhttBEbGncC2avS0ZaN1Bv09qj88mf628wKEIExuTS2yV+8JUZTjyfw2qCQRKJr/+eZJvG9tvjeK4gFJEbZDr+OiDLGpINnt8cGUBVwgjh4flBcnl21OK5P5nmpb+YYfvhOPd8uJWdRxK09xsNBZeSFOSpHvxUB3pY4c9//doNJTmjUhJV0nCEhbmJnIqsqJjVLIpqsN4WJ7R/W33HpnUGeZjF96/mpeshGUWBm1le9JBcb1C3CN8D0dZO6sO9WMOTOMOT6L3tpD5yb5BfuQmj4swGRctKKhWw6goFkGX0zs5VYzqu5XPxzQJGRMHaFNNtk9vJm4RdWxlGFD4c+948L/zpzIr3L17qzdbxSASOrn2T2o6XvjVE711dlKcrVDM3TwVvFps2Kq7rU8o6QQJtYRzaDoxPuaSSMmMTLt2dKrnCrTMoigIdHQqZjHfTN7q9TSYRlymVfTJZv6mdR7pFplIVWNbGo0WWg86Tu65TnAV441sZ/vK3xzfFvlotDLQcudlAJdX3gvBGM+Gvdwu6GiUe7ca2y5Rq02hqGM938X0HVQkhhI/n26hKCEXRcZwq0XAbnudQrk0jSyqKrOF4Jo4luPBmkYtvFenbHeHAAylufyJQaF7k80uShKLCHU+0MHSyxPNfmt50Ar8mykSIo0gaKjouzQ0wszKPqoZxrMpCAeQaxz9xEeH6yGED69Jow4qTXYX1FEmoQWFr5cYX+UhSbegVBEFYrSpCmJfGKL92BoDqiSt0/MzHkWT5pnYqiwjv2IGwLCqnTwMQPXyY8ttv48w1Uq1VQ8b3IDNq0tprNGcrJIloqheQqOTHb37lbgKuIyjnnAaJm8WeRdejv1flzkMG41MuJ89am16nBvpUJqddXBd0Hfp6VK422TrkesiaTLI/jh7TqGSqJAcSGAkd13xvHM5NG5VqwWP4dCnIOSx4GeGQxBMPh9m7S0dRgvHxl9+psBHnWlUlHnhQp29A4eRxh/4Bha4umaNvORghiSNHNEZGPK5edjl4SOOl7/vce79GT6/C6IjH0BWXR58wuHrZ5Z1jDmvkBhsQj8t84PEQz71oUigKtvQrTM14lEqCnduDHMvElEdfr4IsSUxOe/R0Kwxfc0GHllTQUXJi0qOjXaY1LTMy6lGtBYNcD8vc9nDLihzKxJUa3/itsU1rJoXjGxQC+mBEFe77WButfQYX3ypy+gf5DY8rqRqSLAdW0Pfwb9ZaA4loL6ZdQJYUYuFODD2OrkYpViYIGSkMLcF8cYhouI14uIvR2TfxfS+oeJc10vFBQnqSucIlLDugoS7mXSav1Dj6TJb7PtrGR/5RL0ZYbmh89MTnu3jl63ObVln1hENYCQpGy1K+aUc4FE1j1nLoRhxZ0fHc1T1B4frogz3o/Z3IIZ3i996sL4hTQ7UVEjwtXUE7403pfl2HdJe+glruOj4zZ/OE9+9DiYbwihX0gU58094cu2oD1I+1XjOyrWFueyxNJe9SLbrMNNO2QICmR9EjKSr5Cd6rHcv0sNlAK5bkgJmoh+SGWpXDB3RcTzA24RKNyPzEZ6KMTgTPcGufRqkSMGMPHzDwheDSVYdtWzRiUYm3jlt85hMxLly2efkNk8EBlZakzNCIy5FDBnfcZvDWcYsTp5tzRkNJg0RfDLsS9PTRItrf7PBXNKly+LFW7JrP+ddzjF+sUjMF33muSktK5vuv1BjcoqGsvxYCsGVQQYwqzM74fPTjIS5ecPneMxaf/4kIp046ZDI+taogk/URAoyQRE+Pwltv2Dz+/hC7dqvkcz6Hbte5cMGlWNh4oI1PuMznfcYnPFrTMvv3ahy5XefCJYc9OzVCIYm3j9vs2aXR2SHze39QZv8ejVJJkEhI7N2lMTCg8rVvVjl0UOPAPo3//NtLgneaIbP98EqBxde+OXdDvQpWo51ej3BMZuJKFUWTmlJMBVBTKfTWdtRYguKpY2it7ciajpPLIOlBnYFwnU1VnQsEsqyQjPZhOxWq1jyKbBAOteK6NVzZRFVDhLQkQvgI4aHIKpIko8gasUgnQvjI0srB47mB1PjTX5ikmHP57C8MEEksDd+OgRBbD0Q5/8bmet7IKEgCcv4ctmgyTi5JtLTvRFEMPM/Gqq2T4JaC9ztTGfS+TuRkDL9UBc9j5EwZ1/Ebuh+294WIJBRYXf6umVOjY0uowZsWIqD/j700jEhmCe/bitqewhyawjw/smGDrmbhzMwQu+suJEVBjkSQNW3VnMrUUI1SboZq0V3GOtoIAtssoobWV4y41bh6ssShR1saXuvdGSaaVLHNpfn86psmn/+ROIYhkUzIOC5s36qxe7vON5+ucOdhg9Fxl7MXLG6/zeDh+8OcvWhx9qLLnh06mYzHd1+oksv7uK7gw09EaE3LbOlTeeqZCjNzzT+j2rzJ2OtT+G6wFpTGS3VC03uBTZsv3xfMjZlIMg3yCZ4HE1Muh/YbdHcqTYWKzFrAiw5HJI4ddahWBaVSkHsIhSTuOKIxl/Fpb5fp61fo7VVwXSjkA1pcIe+TSMqcP+dg1pobnEIs/i/Yu0tj+6BGJCwxn/PZtUtjatojl/PZuV1lcspDUSSMkMTAgIKuS1y+6nDlapDIb0nJFEsCZ1lCT9Nlku2N3HbPFYycq9yQ3s6Ow7ENqYaFjMPsqFnv2tgcJIyuXpxCHjWeJLJtJ9Hd+1CicYyOLmL7DrGiC9sGKJTHMNQYVXOeQmUCRTFwPZN8eRQA33dwXRNfOLieia5GAyMia/i+R7ZwhXJ1GttZWwbfdQQnnp/n4vW1DRL07d58QZoqBSwNeTPXKgTTo28zMfQKk8Ov4q6icbb8vX6lhhwLB7uD3nbkUOAo5DMO49eJM6Y6NLbsi94wvTSaUhnYE11R83DpWBHH8nFn85RePEH+W69TPXYRf6OuqpuANT5O5fhxtI4OZMOg+OabQbHpdfA8QbJdZ/C2eL2h1EaQVQPNiCHeTdGqVXDhzSKO3eio9e+OrlA2TiZlTpyxaG9VsExBS0Lm2qjL1IxLzfRxXMGh/TofeiKKoUsoCuTyPrWaQJZhPu9xcK9OW1phoE9loE+ltUUJwul7dHq6NuH/S6BFVJL9cdr2pOm6oxM9fnPS+pvBpncqlYLL6Lky5ZxBab5xAbt01eHvfz7BS6/VyBU29pinp3x+8H0LXYdSKWA8WabglZdt+voVvvkNkwce0Hnqr0xefcUim/GZm/UolXyefcYin/Pp7lEo5P2mQl8QMGBefT2IeZ676DAx5WKagp3bNV78gUkiLnNwv87rb1q0pGQSCZk3j1pYNhSLAcV3etYnEpHIZH0qFUFXp8LY+EK1vrIyN+lYzakHXw9Vlzj4UGrj92kyPTsiJNuCbW6zzJTa2DW0VBqvWgni6o6DWy6ipQLPzF+lWHA9uJ7JXGGpM6Jp5+u/LxoWTY1gOWVkWcP1bAqV8fp7StXmFIOLWYfshHVdx0Uaesw0i6gcCDpKbO6zqhoiFGkhFGsjO3V2zfAXgKQq4HgQUTHPDdUjN7WSy/nXC2zdH20o/Lz7Q6289peZG6oRWNyxXY9jz9xaTbpVIcuYIyOYw8Prvi0UUWjtDZHu1km06cyOmhtGsyRJwjFLC/Tt9y5ZPzNqMn6hytaDS85dvFXjyBNphk6V63ll0wy63j77YpXZjMfggEah5DM04jCb8cjMByKOyYSMaQo8HzLzHrYteO2oietCe6uC7QimZzy+9b0qc1mPN94O1qFmnPRFSJKErMp039FJcbxEJB1C0Tc/N24UmzIqyXad9oEQkgwD+6JYVY/ysuZKj78vzPEFa93ZrpDJrr+Qep5gdCRYjGVFqsdjz55xSCRl9u1TOfa2Qzbjk8349ff5HlwbDj6X36SwmxAwPhl8dmraY2o6eL1mwu23aQxdc5md8zh8UGd41GUu4zHT2NKEQtEjHJboaFMwjMaaHNcOBCD10NJDVHWpIcTRDCQJDj/awsC+jb1vx/YZPV+hpVMntUF/ivpnshmc+QymPBzUK8zNgATCCToe1oYv825ImTpulVxpBAikTm4EsiqtWn3cjO7Z9ch6k4C0Od0vQNUjRBJdSJKMLK/UOKtDAJ6PsaMfe3ymYT10LMHZ1wrc+9E20sukO/bcm+S2h1Mcf35zdSN6SOb+T7TT1tvo/Y+er6zZk+RWInbbbfi2TfXs+q2aayWX7IRJultn8vL62l+L8D0HWTUIx9spz4/c8NjZLMp5l7efyTKwL7qMIAKP/Ggn7zyfq3c8nZrxGrQOT55tDDfl8muPr8rCuF0McRWKfr3J4XzOZ2xyc+fsuz61eZORVyYQnk81W2voafRuY1NGxVpoWVvKOkxerq6ozB6f9NgxqCLJgRZYs5BViW13tlArBjLX2dEqL37fWmisJQjFVGwz+K6ObVHmhit4jsCIKAgB9i2Qg5mZ9fjuc0vHefb76z+EWk3w+lsrPXnPFeSm7YbknqrJbDsU4/ybhebWaSnojPfkT3YTS6lrFj4uQjOC42uazLWzTXL+hb8gabEgXeMHk0DSdNxCHrdwY4Vw60GSIJ7WqJbcG25cBtC1NUT/nuiK+zJ+A8rL/iaNySJq5Tk818T3PVxnnbEiy/hVk8J3XkXrSq/489CpMmdeLfDAJ9vri5aqSXzif+ljbtxq+ppkOXBCHvxUe4Nqg1Xz+P6fzdy0SGUzaCrhL0Gqy0BWJM6/lm9aeh8kFFUP1KDfw+5TniM48VKOO55Ms+22pSLkcEzhx/7FVv74Xw8xev7GQtsQ9NRRNWldZeUbgoD09hSp/jjhdIiL37qKU31v2F+bcp/NisfQgseTmbCYuU7CeXzS5dU3Tb77fI3hkeYvQFYk+vYn2HFPmq7tUXbckybZGSLdF6ZzR4ytd6SIJDRUTaZ9axRFlUl2GWy/N02io7mY7HsFx/IZPrNyYb/v4+0BfbIJbN0f5bP/fAvbD6+kJa8GSQpYKq7jE0vd3DZXODbOfKZpOfTNQDVknviJLj7zC1u47ZHUhhL3qyGeVnny73bTf12nvNyMzeiF96aITlrozeu5NqFwqv7v1aB1taIkoijxCHJ4pYBkreTx/T+fITdj1RdlSZLo3x3lM7+whcGDG+dX9JDM3R9p4zO/MNDA+hJCcPGtIqd+kH9X+2cswisWiezbR+L++4kdOULsyBHkUOM1q1qgnbX1QIzBQ3Ga3XAoqoGkqNhm8T0MfgWYulrj5a/NNqgfS5LEwP4oP/4rg9zzkbZN6XbJCrT3Gzz4qXZ+4l8PcuT9K52NW4G5c1muvTRGaaqMpPwNZn+V5p0VuZRF3H2Hwde/VcF2RLNq2gEECE/U+evp3jClrIWiyhRnLdq3RijMmMwNV9HDCqoh07s3QXa0RmFmk9s6SSK+4yCp/Xci6wbVyWEyb75A50MfYe6NZ/GqTVajS9LCtr3xQm3T58zLee54It3Q/Kd7MMRP/bvtfPk/jjBxuRoUoy1bRGQFEq0a9360jYc/20lbr4EsS7hOkI8Jx9Z+VEZEYdvBGLEWFbvmB6f2Xs+8JiDL0L87wm3va+Hej7VRmLO5crzMudfyDJ+pBPVPAvBF/a5KUkDjjKVUDjyQ4pHPddK/O9IwiX1f8PLXZhsETt9N6EYCPZQkmuhCN+KY1Ry+tzq7RknG0Lrb8Co1tPYWzIsjKx7OtTNlvvFb4/zdX9tWb7WsqBL770/Su30X3//yLEefzpKbCei/izRkWZbYsj/Kw5/p5NAjKaLJpV2tEAtsuT+YWlcF+FbCN03cXA7JMFAWK+yvo4EKP4h4eK5AUSTibRqsrnLUANepUpy9snSQ9xC+B699M8PAnqCZ3KLcjaJIbDsY4/P/apCHPt3B289kuXi0yPyUvVQ0uSCP09Kp0zUYZmBPhN13JWjrM4gmVYywwtwmO7A2A0mW6DzYRqwrilNzsQrvojL5ddi0UVkPsxmPf/xTCQpFn+deqjHUZPGO8AWFOQu74lGcs7CqLlsPpxg7XUSPKFTzQU+NaItGJKnR0hNi4lyR3Q+2oYdkxs4Wm87dyapO+vYHmPreV3ArRbREGjwPLZ5c1+NsgCQTG9yNU8hhZaevuxY4/2aBc6/lue3hliV9Jwn23J3gF7+4n6GTJYbPBIoEqiYTS6n0742ydV80oJMSGJpqyeW7X5hE1SU+/rP9a55OMeMwcq5CftZGM+S/kQZlORRVIpHWiLeo9O2K8MhnOvD9hQT8lEW14GJWfWQ5CA+09ui0dhuBIVnsCbEAzxWcey3PK1+fu6HE9o3AquXxPZtaeQZFDeGupVAM2OOzsOAlWqXqqtZe+PDGtzO0dOl88Kd7iCZUJCm4T+keg0//s34+9o97mRsP+rpbNZ9oUqF9Idm9SFxbfl/ysw5f+c1Rzr++dlfFWw17chJ7crKRqXLd9Xqu4PLbxXWViVeFEGuLdr4HsE2fv/hPI+ghhTs/mCa02KtGlogmVfbem2DvvQmECEJmlaKL7wmMsIIRCYQpG1XHAzXyd0uHr+++blp3tuCZLpF0jKmQilV8b5yLW2ZUOtpkdF3ie9+vcWXIwdwEW8FzBSe+3bg4X3otWzcUc8OVunPy8hdH6u9562s3SOb3XNRYAqcYGIXAmAgiPVtQ4yncSony1XMIzyXcPUCosw/ftigPncezTeI79pPafxdWdga7kCV/8vV6bgIgO2XzzB9N0TkYpmtrqN6ICoJY7P4HUux/ILXuKZZzDs9/aZrvfXGaXUfiOJa/6hZbkgNPKDdjE44FvVeapxX/9aK+CErButvSGfR6aBbCF1x4s8A3fnv8XfH21oKi6ESTveihBLZZxLHLiDVS9X6pgnnm6obH9BzBC1+axrUFH/zpbpJtWsO4McIKfTsj9G0guS6EYPJKjW/99wmOfje7+Yu7CcjRKJF9+9DSaZAkhG1TfOMN/OrN9bQPDXYj8hZeqYpXvLlj3QzMqs9XfnOE3IzFI5/tJNaytDNc3hNINqSmCTPvFsKpECMvT9B3bze5a4UVsj3vJm5ZoO3wQYNiyeeBe0JUa4KbDskvs0m3crfruzbZd16m5dB9dD3+Q+gtHQt/kTDauzGnx4gO7CDc1Y/R1kVi92Gs7AzC90jf8SCSomDPz+LbFlZmitrk6MoEpYCLbxX54381xPilKsJvridEENoQ5Odsvvwbo3z3D6Ywyx7ZSYu58dXDfEZYoWd7mCNPpjn0SGpFruFvEnxXMDtqYpt+/Vo3CyEEwhdUCg5Pf2GSP/rloUBA8z3cnfm+g1nJghBUipP4TfQiaQbVkscLfzbN7//iFa6dDVouN3OfFt/jOj7HvjfPH/wfV3nrO9n3fMcaGhxESSQwBgbwCgXUVAqpmSroDRDet4XQ7n7UlpVFxe81ChmHb//eJL//z68wdLLc9DNaxGKNnBACzxMMnS5z9fitZ+Z5tkffPV3oUY3kQOI9jRjesp2KoUsk4jJtaZnH3hfm1BmLuWWU4tlRsyGJ6HuQv4kuZ+vBrPiMX6qyPJo1N2oGEulCUB6+QGX8KtH+nXQ/+WkmvvNnABTPH8eanyXcsxU1nkRLtOCUclRGLqFG4oQ7+9GiCaz5OdxqCSs7gzkztuo5+D5ceKvI//nT53n4s50ceSJNsl1bYHvIyEowwHwvKPa0qh6VvMv5N4s8/YVJspNWfSAU5hxOv1yoh3dmR8165bxV88hMWMyNBYnexlwDZCctxpapwArBTYlOmlWPqaEa5fzSsyvn3KZCT44t+Or/Ocpb385y14da2XtvkmhSRQ/JaIZcb1omywT05oX743sBAcKqeZRzLidfyvHGt7JMD9XetZBXKecycblWz3FAcN8BFC1MaqGivq37ELPjx9atU9kM7JrP2VcKDJ86x51PtnL/J9vo6A8RiinoRiAUudAbDs/xsWo+tZLH2IUKL355lnNvFNbt6d4MPFeQmWgcN8CGEjhyOIwzNYXW2krp6FFSjz6KZBhQam7RdB2f7IxMOCXj1oIOnblZG3NkFimVQmtP4WaLeOWlcGMl7y4UkAYCYsWMUxep1JOtOJUiwl0aq54jmLlmNkTospOb2+WaFY9TL+e59E6J/fcnuf8T7fTtihCJB5ECbWEcs/CcFuf4YjuLwpzN5eMl3nl2nuHT5U23bhACCnP2iudTyDh14zb0/GiDbqB/C7TdmoUkmjSxG7GQ+noU+nqWbNTlqw7Z3I1fSCwu4XlLSsKrQVUhnpTJrVIPE2ntI9LSg1XKUpodrm93JEVF1nQ8s4oSitD7oc8x8/J36Hjgg0w9/w3ccoHWux7BrZTxHYtQZx+Z159FS7TQeuR9ZN58HrdWofPhj1K8eJLq+FBT15NsDyqluwfDRFsCiXrhC6yaT2neYWbEZOxClfysvSkPU9UkenZECMXkoKf0mLWiz81mIUsq6dhWJCRsr0qxNv2u1AXoIZmewST9gx1oySLRpIwRldGMoJLYcwW26WNWPHLTNrNjJpNXa7ekc6Xe0k6sfycgKFw6iWcuhVWM1i6ivdsQvk/x8kk86xbLnG8CWihggvXtitDSqROOKSiKhG35lPMuc+PBuJkbs24oPm+0dKAnWyldO3/T5xrevTvodLl1K8Jx0NrayD37LO786oWXsm4Q7d2GGksifA8zMxU4brEUc28+i/Cad35CHX04xfmG57jtR/9XJl/4+pqO36qQJKIDO6mMXNr4vUsfoWMgRO/OMK09Bom0hh5WkKSghmzRGZqftslMWEwP126KUn+rIUkKfW131P/teCbT82dYbfvfjLm4ZTuV8UmvXlS4EY7ca9DarnD+tE17p0JHl8KZExZ9WzTaOmTOn7LZsUcn2SLz1qsmngsHb9e5cMampVWhvUthcszFc+F9T4R5+fka58/Y1MegJDNw1yeJtvZjV/Oc+/Zv4dnBYFNCEToe/GDA6/N9KqNXcIqr1WQIalOjhDp66Xr8k0GoY/QqTiVIMFrZGdK3P0h8+35mXv42G8kdF+YcTr2U59RL+eZuaJNwHUFm0mTn7XH23p3k0ttFzr+5ySTodVAVg7b4djKlK6QivXieTdma2/iDm4Rt+sxcFYi5JGPZq/jiFvLoJZnU7sMkth+sv1SbHSd76jV8qxbomjk2nfd9gOrUSMNi5LsOvuvQee+TVCeHb5lRkVQ1YLZtgt/rmIKhk2WGTq7OSpRULTjeDbaDCHcNkNhx8JYYFfPaNSRFwZmZIbxzJ9bY2IoGXYuQNI3OBz6MGolRnR5DUhRCNxivk2SF1L4jFM4do2Yuz7ls/nhqOErbHQ9TGb3cQDJIRHro//+T995xklzXfe/3VuzcPTlvzsBiExYZBAGQAKMYJEqULEpWsGXJli3bkvVs+UmWZD0rmZJlyYo2ZYqiJBISA0AQBEjkvNjF5hxmdvJM90znynXfH9WTdsLuLBYgnt+PH34wW13h1q2qe+4953d+p20/lltkYPxlgnBu9dOW3Uaz3MjJ51/AdseWOu27GkIIMokudC1OLrUGz68zPn3iuoVGV21UWjfdRtdNDwAweOgxioPHl91X0XT69n+cXM92ZOBz/Ou/TWeXpKVd5dXnbbp7VMyY4M3XHT71IynOn/Y4fMDlgQ/EOXfaY2LMpzAZ8NBHk9QqIR/4WIKxkYBjb7rcfrfJc9+2GR/zOXd6nkGhUY8gnkYoCpqxMEnOr1cYf/4bUbU/GRK6DjLwGX7i7wjdyIUxdfhlhKIQeh6FA88gtEjLK3SsxosmKB4/QPns0UYwVQFVmTezmglANwrNh5FW2dsBI6awZV8GJHz7C2M3JBEUGnEDr0Iq1gZC0NdyK3E9S9WZZLx0mg3tdyNQmK4NUKqP0NeyD001mSyfw3JL9DTvxtASDOQPYGpJcsk+kCGFWj9ShnRktxGEPhV7nITZzKbO+wgCl6Gpwzj+DfAxy5DyxZO45Wl63vf9jL3wGPWx/tln7FWKlC+eoHXffYsO9UoFKrZF2633v/V2zEPrvvdSu3ye+mj/DTtn+x0PUTx1cBEL8bsB6XnoLS1ora1UDh5EiccXEFjmI7d1D/H2XgYe/RyBbTXcUYLcTfvR0zl6Hv40erqJ2uB58gefRYYBue37yGzaiaJqVAfPM3X4JUDS8/CnSXStJbV2K6HnUDx1kKk3X0SoGrlte4nd+xFCz2HipW9i50dRjBhtt7+fRNdaAqvK5BtPY40NkttxK8233InZ0smGH/xXICX9j/wJoedg6mnac1vxA4dSdYhCZY58kYy10pbdwuDkAeCtTei+GwhDn9OD30QIlT2bPo2hrV5Dbz5WbVRU3cRI5QCBql8tmU+gGXGMRJbQ9wBBKCW6Bu0dKropiMUEbR0qniup10Km8gGGKbBqIZkmhWRSIfQlugGvvWDTs1YjPxGgKALXlQigqVllfCyYjUFIGTL4xqPkendQGb9A4M6baUpJYC1OkgvnzUal55LcsB0ZBnjlIrLBFgtdh9B1UGNRtrwMfBQjhp5T0DM5nHzEJFMMIwrGBQECCBwbv1Ii9G48Q8m1Qw4/feOz3009TV/zXiw3OnfKbGGsdIqm5BpMLYUQCvnyeSr2BLlELwhBvnKRtswWLk68yET5DKlYG83JNbhBHcudZrR4AkWorGm9leGpI9TdKeJGjlyih0sTL9Oa3kjSbLkxRgUIXRu3VED6Hm4pv+RzXw0UI0aspRM7P0roOdG/W7uwJoaQvocaSxBr7UKNJQg9D2dqDK9SRI0nSXSuIbtpF0Ko6KksbqWINX4ZJMQ7+3DLU5i5NrR4Eq9SxJoYQigqRq4FI9uCUFS8Whl7chgZBOjpJmLt3WQ37iSw6sSaO7CnJ3DykX6alkgTa++JjqsUsfOjsy5gI9eG2dzeaHN8FVntKyO2bh2pPXswuruxTp0ie++9VF5/Hb9wBQtNCBI9GyifPzabFzZ/yhXv7GPkqS/h1ap03f9x4h291IcvUh++RPXSKVAUOu76IPGutVQvnWT4ib+h+8HvI3/oWayxOVeXounIwGfom18gvf4m2u/6AENPfJHWW+9HqAqD3/g8sdYuOu/5KIPf+DzFEwdw8mN03vc9XPry/1hEh5ZIFKHSlttCsTZIEL5zyr9vN6KVl39DyiDc0DyVa8HEaMDFcx6xuGBsOMC2JImk4PGv1CMfuiN5+RmL4cGANRs0ggBeecGmo0ulMBFQKITUa5LXXrQpTYWcPOpimGJRnZ/i4PFFqyhVExFb4xpCPULXUZQYeqYJGYYouoH0fZypCczWDpAhoeehmnHKZ48Sa+8iuXYLMvSjlQtgjQ+hxZNomSZCz3lbjMrbBcsrMTJ1lM7cTZhaCkXRUYTGdO0yjl9lsHCQ5uRaTD1NKAM0JaJQjpdOkUv2YWpJVMUAQqQf4oUOMCcAKec9LdevIZFIGc72HQBC0PHx/ZhdudlNw3/1PEHlndMxmg89naP11vcy9sI3cIuTGJkm2vY/wMgz/4Bfr9J0020YmWb8egXFiCFDH69aQqgaWiqLGkugJVIEbo7AcwGBUASte+7Ft2oEdh0hVNRYAmtyGKFqpNZuRYsnAUFTaxf5N1+gNngORTfQUznUWBwtmUbKELXhmtXiKdpvf4jQdwl9j9z2vRSPH6AycBo920znPR/Gq5UJ7DrxzjU3TONNa26mfuoUWnPzbI16cWVdY2bimiZefenJQ33oItbEMNL38EoF9HQTMpQIRSWzeRdaIoWeaUY1Vp7Uhp5D9fJZ/FqF8vmj5Hbsw2zpINm3kdGnv4JfK1OtlWnefQ/xrrVULizvdQEIw4B8+TzN6Q3EzUNUrcXVH+fD1NO0ZjaRjLUiZUjFGmeydGaB6yyT6KGjaTvD+UPUnbnYk6oYbO55kMnSWQrluVVRT+teZBgwNn2CpvRamlJrUIRG3ZlivHgKz59x/0UurVyqj5iRQSCw3DJT5QtU7Rvvyp6Pd9yohCGcOjbXqZPji1/ok43fjx2amwmMDS/c78zJaJ/551oJuiG49X05rFrA4eevnixZu3gmmploOrHOXrRUDsIAGfhUzh1H+j5GcxtmW1cU0D11OOLmhyEzJ5e+H/nRgdC7dqabEtPp/uF7ye5dz8AffYvqiTklX60pydqfeYjKsctMPv4m0r8+MoSWiaM1JbGHphaVko0G+ADXr2F7UdGtqj1Bc2oNllvCckv0Nu9FFRrT9UGKtUHSsTaaU2so18cIpE863oHnWzh+tXG+qE+C0IvcZc178EKHqj2BbFj5GcMyvyVB1UaGkuSWblLbuhn7+9e+a0ZlJSiaTqy5k/rEIMXThyKvp++BlPjVEtMnD9C8805K545SGzzP7AsoZpLoVPKHno9cqA1JhNBzKZ4+NLtr2/4HSPaspzZ4DmdqHLc8RevueymeOoQ9OVe4Kr1+B4quM3ng24SBT9P2W2nefQ+VgdNkNtyEDH3GX3ocpKTzng+jZ5oW3c/1ILQslHgcoWnonZ0osRhyCflwGfiEro2eyix5Ht+eSxKNjImC2dxO220PUjp3hNrgOdREcslSxQuuI+WsS1pGEhYouolQ1AUTvNBzUfRrySuRVOqjJGMtdOS2U7UmWG4gScZa2dr7MMl4G45bQVV1ulpuoS23hZMDj84almSslZ7WveRL5xYYFUXR6Gvbj+NV5xkVQWtmE5pqoqoGva178QMbTY2TCbqZqvTPGpVMoosdaz6MoSdxvOgb7DSa6Gq+mVOXH6dcX6VK5SrwjhuV7xbCUFIt+aSb9KvvDLMvXeC51C6dWXIfe2wQe2xlZslqGCyzUARGW4bY2jY6v+8OLpz5h1njIVQFs6sZe7Bw1Y9qJWRu3UDTXVvp/+w3COoLV1CeX+fC+AsAjEwfjTZeMak8N/b0gn+fH39+wb/zlQss98FN1y4zXbu8aPtE+Yp+lpB/6ih8R9DxPbeS2tZ9lbt6pyGY8R0Fdp2pE6/Suve9pHo3Url8lsqFE/gzs/EFFREX90t9dGA23jMDNZ6kacetxNt6ELqB2dRGdT4rSYaNFd/CcxpNrST7NtP3obbGdhFRdBEYmRacqYnZa9n5UbR09kZ0Bta5c2TuuAMhBC0f/nDk+iotkdEvJdXL52jZfQ+VCyfxqkUW+OCWcMHMrMbqQxcQqo4WT8/bPSQMfdRYcq4GkAwjQ9/ShTU+hNncgURGxrg4SbJ3A161hBZLoKeyOIVo1RH6XsOFHUN67iJSheNVmCyepa9tH0P5gzhL1P5RhMam7gfQVJPD5/+WijWOIhS6W/ewqft+1rTfzqWxF1fdvzPIJDpxvRqHL3wJyy0ihEBTY/jB3HdcscY4P/ocpdogXoPu3pJZz851n6Q9t42KNf62KT2/a4yKmW4h0dSNkcjNzhpCz8WzKtiVCezS5IoyDS0b9mGmWhZt950qk+deR4iAWinATKhLlsTO9d1MoqmL+vRow20mSDR3k2jqQotFwo6+Y2GXJ6jmB67JWJjpFpItfeiJDIpmIJbNNZVMDx7Hml5YT8QrVEhu7yGxuYvaqetUD1gCwtBIrGtHTbydYpw3kJjQ0Lv67qOh1dbg/yu6gTLPBVMf6Wdo8oskuteT274PLZYkf+i56F2ZqQ63zERgKUZYau0Wkj0bGHvhUZxigY47PzBLGplpzlJnk75PbfA8o8999YryvpLQd1GNOZFHxTAQbzWooigQhkjXpfT889SOH0cxzag2/TKB+srFE5gtHXTc+xG8ajFq89DyygNepQShpPXW+6OBX1VnjU/oudSHL5Hdtod45xrqQxeic0kwm9tpv/39GE1tlE4dwq+WKRx8npZ992E0taPGElT6T2Pno5m7V57Gq0zTfufD+LUyhUPPXSGuKpgsnaW7ZRfdLbu5NPbCorbGzRzN6XVcGnuRijUOSEIZMFo4xvqOe2hKrWVQPbDACKwGQiiMTB2ZjXdKKee5vWhsC8mXFtKiq9YkFWucuJFrEJX+DzUqQtVo33o3rRv2occzqHpsNgtXBgGB7+A7NexynsE3voZTWVp6omXDPjJdWxaeWwis4hiFi4eAgM51Ju29Jm88tfj4pjU7admwj+LQCaoTF2nbcict6/ehJ9KomgkIwsDDt6tUJi4xdPAxPHtpn7CiGbRsuJXWTfsxU82ojSX3zBAghFgQEJMyxKlMLTIq1ZPDxNe20vbB3dTOji5yU81dUJDa0Uvu9s3EepoRqsAZKzH94mkqJwZn6aZ6U5Km92wnfXMfyW09KIbGpl/5PmTjvMVXzzHx9TcAMFrT9P30Q5ReO0f+yaMLrrXmpx8CAZf/8FuzmzN71tN091aGv/A8sd4WWt57E3prGq9QIf/tY3NGUQjia1rI3rGZ5MZOlJiOV6pTfuMi0y+dvm533pIQIpqgKAqKEYtmsfPca2Imy1I0/jvPEEaqrmJuHyJ3lFBUEt3rQUJm083oyciFo5gx4u19+FYVt5jHr5VRY4k5IyIlvlUl2b0etzhJ6PsE1sripUIoIBRkGJLsXk+ydwPW+NzkQsqQwHMaLKYaoe8S2HWqg+fo2rCd5Jot2BNDaIk0CEF95BL1kX7ab38/yd4NhJ5Has025FtUBMjcfTf2+fO4o6OomQwtH/kIUkrs/n4qr746G1+Zj9B1yL/+NEZTW2SYwwC3FLl/hKbNDuRTR14k9Fx8q8rEq99CVpjhGAAA2QNJREFUS2YIXZviiQMEMy4sKSmdeRNrYhhF0/AaZRuGnvxbgnoNPZ1Dnj+G3WDIWRNDTLz6JFoijfQ9nKmJeQbKYeyFb6Cns8gwXJK9VrPzFMoXaM9tY7RwdNHv6XgnqmLQ3bKblszGBb8ZWgJHNRetLFaDMAwo167uvjL1DO25baTibRhaAk2Nk453MOVfulHcjCXxXTUqQtXp3vUQHdvuQSgqvlXBLk/guzaKqqMaMVQ9hplsXvYBz2Dk6LeZvnwMzUygmUma1+3GSMwt6zVdMHLRiYTdVlDxNZJN9N36MZrW7CT0HdxaicC1UPUYejyNkWyiZX0TimbQ/+qXCZyFMwShqLRtuZOeXQ+jqBpWcYypsQu4Vhk9kSHTuZlYpg2hqFjFUfLnX8cqTlArLHYHST9g8vE3af+eW0nv6KVybPE+APG1bfT+4/eixHScsRKh65Pes47cnVu4/CdPUnzlbDSr1RSEELgTZeJr2pB+QO3cGNKLBhVnfM5VoZg66Zv7sAcWBvWEECQ3d0az03nQm5Nk9q3HGizQ8uDNuOMlQtsj1teKnpmTjtGbknR83x0kt3TjjBUJqg7xvlZyt23C7G1m9G9fXt54rgaKQsvOO8nt2I8Q0H3/J6gNX2Ty9e8gA5+OOz9AomstAuh58PvwKkXGXvoGgVWj446HiXf0gQzoef/345WLjD7/dbxKicKRl2jeeQfZLbuoDpyjfPFEg+UniLf3kl67JXq2E8MUjry0IJt74vUnab/1faTWbqZ45k2mjr2KROJVSotcXwCV/tPE2jrp/cAPYk+OUjp3dOE3ICXjL32T1t33ktm0k+Kx1yifP441OsDki0+S27kfbfe9hHad/JGXZs+ppTK03/Ewfq1K+dzRiHiyDIRQUYS6ItMpsWUL1UOHQAgyd91F/dQp6qdOkXvgAdRMZjH7q4HQc7AnhlDRMZU4mgxxZA1JNPiCxC0WmDH2bjGPXy6RS/ZRt2u43pxRDl1nUZKjPR7FIr1KZGQEAiHUKF44PYksVwmkTxguNHpeeQqvvFKlTMnlidfoaNpBe9P2RYwpXYsjZYDtlbGc4oLf6s40lltcEKxfGssP+xKJF6wUVxS0ZTezte8DBKFLsTpEsTaMQDb69e3VAfuuGpVErpOm3h0oqkZp+AzDR57AmhqZDdaqRpxEUxep9vV49TKevfzMrjpxkepEI7tdKCSaexYYlWyrTluPwfAFe0VXSqKpi3iuk8rYecaOP0M139/ws2pkujbTveshki19pDs2ku3awlT/4QXH67EUHdvuQdVNymPnufjiF/Hqc4O1mW5l7e2fJNO1GaSkPHoOq7h0joHQVIqvn6fl/beQvW0jtbNLz06sgUkG/+I72IMFglo0+4lvaGfTf/xeMnvWUz50idD2cCcrjH/1AFomjtGeQU3GGP3ii4tiKtcLvSlF83u20/97j2FdioyRElsYw/Kmq4x9+VVCy8GdjFZ6ajrG5l/5FJldayl85zjuWPGtNyYMKRx5iUJjML0So89/bdlDR579yrK/VftPU+0/veRv+YPPkD/4zLLHGt0a44e+ip5LUDs/jpowCB2P8VcfJ76+HTVpEtRdhK6imBpBtUbx0itMHnyKoOYgDA210Z/C0FB0lerAGaoDZ9DUGJlEN8l4G3VnikRdZ+RbX8IPHBKxFhy3gkCgCI3KyTeZPvYqMSNLGPpM+TWEUNEUAz+wI0OiaPiBQ8zIEDebma5cuoJEMYfQ81ATCbRsFq25mdILLxDWalG556tofwkEveZWNpq7sWSVk/WXqMhpOnLbAZgsn0NXYwihYrslWjIbMPU0dadAwmxBU01qdh5di2NqKbzAQlPMiPRBiOfXiRtN1J1pTCNN0myhWBvE9y3S8Xaqdp4AQSrejutVGwYtThgG1J0C4TIuIsstMVE8Q1t2E5UrAvauX591d41OHVnx/kE23I8LB3pdW1x751phaEn62vYjEBy9+Ag1Ow802GjZLVc5+q3ju2pUtFgKLZZChiGl4dPUC0MLfg9ci8r4RSrjlxqMmOufwU5PeGiaoDC+MrdcCAVreoShQ9+gPjXP1RD6lIZPo8fTxG9tRzMTxJu6EZePLfCHx5u6MFMRm2bizMsLDAqAU8lTHDxBqm09RrKJeFPX8kZFEYS2R+HbR2l7eDf5p44tbQBCSe30QoPjTpSxLk2gN6eiGum8/crFiqEx9fypWYMCENpXXFeCfTm/YFNQsameGCSzbwNa0uRGsP+79I206WsAGPMuMuENXOWIxVDRyGkd1IIitrz+HBdTJEipOczOVoyWVGRohUBLxQi9gNq5UZpu30T+6RMITSW7dz32cIH6hXFiPU2EtgtSktreM7t/ams3MpRUjkfuzWyqD8+vYzlThIGHDAOEUBAIkrFWgsBB0ROk4h0YeorR/BFiRhY/sPEDh2yqB0XoTFcGMI0UudQaCqXz82bUCwc9BZUmrZNaUKL65ps0f+QjkerE0aOEloUwzUbl1pW/WQWVlNqErpjIUJJQ0pS8ScrWGJpiEDdymHoKP3TR1ThCKNScKXQ1gWmk8fw6rdlNKELD8SpIQnLJNQShi6bGGJs+QdxsIm42YTnTaKrZMBQCXU+iemWyyT78wCKb7EHX4thuBVXRCUIXyy0u03LJ6NRRdq77BFxhFMr1EYRQSCfaGS9qKwqO+qGDquio6kL2WTbRs2K/rQRdi6Nrcar25KxBmdmeMJtxvbdX6fm7alQiHr2LZiZIt69jauAIvlNdwjcll/dXXSMCP6S5U6d7Y4wDTxWX3U/KkOnLR5cZ6CWV8YsEvoNqRO4wRdUJ5hkVI5lj5gWzy0vzwZ3aNDL0UTQDzbiKlHkYUn6zn5YHdpK7bRNTL5ziyiC4UBVivS0037+D+Lo2tHQcJW5itmWiQeedU71edjU1C0VgtGbI3bWF9E19aLkESkzHaMngTVffEqNtBgLBGnMHWbUNIQSGMJn0Blddh75F72FH/G7O2QcYdq+hktSSbVHoNbfRa2yh3xtCZhT8cp3kxg5QIldkaPv4VRuvUEXLxAhtl3p/fpZOqyZMhK4hJVgDeaQbRHGWzV1UTw4hQ4kqNKxGeeMoH0hFERqyMRPWVBNF0fD8OqqiozXihDP0VE2LU65GtORMoptMsodi9TKhDFCEtiiw26x1c3PiXs7bhxg6cQKnvx+EIKjVoqC971N68cVlZVpmEOCT9wbJKC3UwiJTfvTdRddUUQRYbjFacZktuF4Nx681RDU9bLdENtmD61Wp2XmEUAhCBy+wiRkZmtNrgWhALddHcL0avm834pohqqIT09NM1IeJGVk0xaRmXyQZa0VRVmaK1uw809UBOpt3EsyLj9humZHCETqbbqJmTTJROjtL0kjHOwlDj2ItctXZThEvsOhp2UO1Pk4QusTNJta037bqd20GfmDjBTbJWAsxI9egHZts6HwPuhZftL8i1GgCItTZVZOqGEgZIKVctXzSd9WoWKVxqhOXaF63m9zaW9DiGQoX3qBWuIxTKdzQojxCETR3GpEi6FL0rwZ8p45VHFtWnylw67O/KZq+qLCXZ8/NaI14Bmt68SCrm0mEoiBDn8C/uuvJGS1SOnCBlvftpHy4fzawHjVCkLt7K33/5EGckWnKRwZwxqYhhPaP7pu5+6te45ohFi/V5yOsr7zOSGzsZO3PfgBFVykeuID9+nmCqk3zfTuI9S1m710PsmobCSUdKR0jyWkdxJUU9XB1EhqtWi+6MFC4fvl2FZUWrQtN6LgTFZyajT1aRPoB8TWt2GMlQsfDnSgR62vGnSjjTJSRno9i6ghNwezMUjk+hNmVI72jm9KbA0g3wC9bs+9xoXKRttxWTDdN3S6gCIVErJkgdBGKQtxspu4U8AN3loasKhqaFqdqTeK6tWh1Ur5IGPpUrQmC0MPUUhh6Al2N48zXu9IbfSMi5ldwpRJxECwrJHklxr1+xr3+uT5TdGJGFil9bLccuZNCf3Yl4gcOrlelOb2e5vR6CuUL6GqcIPQiT4NbxA88hFAJAxdNi2M5RVyvRirWRirejh/YmFqKMPQpVC7Qmt2M7ZZw/Rp+4OB4lQWGYin4gc1E8Qyt2c0LGHRSBg1WmGB913tY13l3I0HYxA8dLo6+AI1hou5Mc3nidXpb93Lbth9vsLgEE8XTmHrqmvrvSjhehdHCUdZ13sW+LZ/B9apoiknFGmN8+hSKWDjs97XfRibRhaoYJONtKEJj57pP4Icujlfh7NCTq7r+d9Wo+HaV0ePPgFBoXnsLmc6NpNrWYRVHqOUHKY+dpzRy+oZkomu6YGrcI9u68i2HnoO/goDg/KDcUlRMa2oEzyqjxzO0br6N6mQ/gTcXVNPjabI921BUA7syiV2auGrbpR9QOnCe5vt20HTnFqQ3Z/C0TILm92xHBiH9v/84zuh0Y3uc1oduueq5V7zPqGbtgu1qwkBoCvI6BAyFppC7YxNme5ZLn32M0uvnZ3/L7F1/3W29Ei1aN6ow8KVHKH0MEaddX0O/s3LG9HxE7p3lA9jXCl2JkVZbAEnlyADVsDj7mzUw55qYfvnc7GTHL0fvX2h7TD0/F78pH7w0+3f1zAjVeS7PIHAYm8dEGikcnv17fOrEgjZZjSS7gjdH4S3VBik1BrrJ0ly+kOtVqVgLV+0KKjn1rffNcghCb8lYxJU5IZOlpZWEpyr9AJTri2n448U54cz5yYbz/4ZI5WE+qtY450eeoVJf2BfF6mXODD6Bqhg47lz7HK/K+eHvMD59goTZ3FhBudSd6UbS5My9ugxOvE6pNkTCbEJKSd2ZolwfpWZPUnfmSzBJhvOHyJfOcTXK/tjUcerOVLTiEgqOV2W6epmE2YSuJRsr2giWMz3770J5oer6lSSGa8HbblQWWPAlfreKo1x+/StMXz5G+9a7SLb0kWjuIdHUQ9PaXdilccZOPENp5Myq1F2vhKIIXDtk8Ky14vMIZfCWKJZuvcTE2VfpuvkBcr072PTeH6U4dArPrqDH0uR6t5NsW4cMA6b6jyyI26yE+oVxKscv0/zgzgX1JNS4gZaO446XcCbm4jdaNkFsTSv1c4vdeFJKpB+imDooS686groDArRcAjQFGlTf2NpW1FRsduBbFRQFoyWNV6pjD80xghRTJ7nt+n3I86GLGBmtDQWFUjCJG1p06Oto09Zw2TlJuKILTMz+L6u2YYjIVSCEsuRqZSYYvNRZZs7VrHWhohHgowh1mfM0khivYqcFytz3JJe//uI7UhbsO3OPcytOOfu/q51JIMiorZhK5LZVWF3fzG/TUgiXqaC5MsS8e7r69a9sx8J9o15mwblCLLeINc9Qz8ALLMamTyzaDlG8ZLo6wHR15XieHzpMVS4xVbm0YPtS582Xzy/athQkIaXaEKXawjh1uT66aN+J4tLEk+vFqo1K6PsRPVVVEMpVDhcCRY9YDGHoL2sUfKfG9MARpi8fI9ncQ8uGfaQ7N2Kmmkm1r2dDUzfjp55n9MQzhP71hXE9JySRUkikVQZOrTAgSq7yca0MGfqMn3oBIQQd2+8l3bmZdMdGQCBlQOi7uNUp8hfeYOzkc9dMPpBByMRjB2m6ZxtGW3p2lu9XLZyJMtlbN5C+uQ9vqoqWjtP5/XeiaMt8uI6HPTxFdv9Gsvs3Yl2aACHwyxZeIZpt+cU69XNjZG5ZS8v9N1M/N4qWTdD2gd1o6Th+afVGRQYh1mCBpnu2kdm1loqqoMYMWt63Ey0dm2WuzUCJ6ahJE6GraJlogDfbMlEypB/ileqL6McpJUdKaUISMuldRiBo09eQUDNktTam/aX1mgwRo1NfT5PWRUptIq6kZgfLbbE72Bq7fdExY95FjtWfZ741iCtpeo0tpNWWKDgvIpVsVWrckfqeJd+sk9ZLDLvL1++IiSQtWg8dxlpSajO6iBFIl0owzbjXT94bXJZIsNa8mc2xW5nyRzlUe5KkkqXT2EC7voaYkow0ocIaeW+IMe8ilaCw6P3XZ/umk7TavKBvtsRuY3Ns/6LrTnj9HKk/y1KWskvfyI7EPYtW+nZY41j9WYrB1VfvEQRJJUu7vpY2vZeEkkUVGm5oUQomGXUvUAwm8OTSno4+cztbY7dTDMZ5o/pNkkqOTmM9bfoa4iIV5bmFNQreMGPeRcpBHqkp6O0tKIn4nJxO3cIbnXzLcd8l79DQ0VqbELqGOzwO/vVNrLW2ZoJSBem+/YSdVRsV361HmlhCieTlG7zvpSCE0ghcR4bjqpAhtcIgtcIQZqqZbO82OrbdQyzTTvu2eyiPnacyfvV630shkVZJZDQS6bde3vRqkKGPU5sm9D2s0gTlkTNRwMt38eolaoVB7HKe1WadW5cmKB+8SPN9O2a3BRWbwlNH0XNJ1v2rDxHUHGQYUjkygD1UINazOE4h3YDpl84QX9tG3088QGC5hJ5P/puHmXj04Ox+I3/9Aj2feQ+9P3ofgeUS1B1KBy6gnhhEy15H2eIgpPjKWRLr2+n81J20fXgvMpRYA5OM/u3L82JAETJ7N9D6/p2oyRhGW5Rk2PdT78OvOYS2x+X/8STuvNVZNItuIaYkCQko+MMYIoYnbXRhklM7VzAqcVr1PuJKCpC40iYmIk6/K60lByYnrHPlM0woGVr1PhQUAunj4WAQTayssLrkTNyXy0+UMmoLm2J7adF6UYTScOl5aMKgRe+mWeukoK/hvP0m5WApYkg0H08oGVq0HjbF9pJRWwgJCKSPIjRSSo50rIk2vY/T1qsU/IWrZ0PEaNV7SCiZa+4bWy7PMLLCKnlvEE0YqEInrqQwlFhDbPTa4n8ChTZ9DRvN3aTVZhBEOScyIKYkiStp2vQ+RtzzXLKPYcvF6QgRxVohoWRo1rrZHNvX6Jsw6hvUxX2jT2Ks7UHrbCV19z5qrx3BGxrFmyhc94C/4n3GDIy1PaQevIP8H32RYHoJ2Zt5ULORfE1QWugq1LvaCG3n3WlU7PLkLE0w0dwTVVF0l561mqlmzFQzANb0GNc+iEqcaoGJMy/jVqfZdP+PRQyxzo3XbVRCCReP14in3n6jkmpdS/ct7wehMPDql6lPjbIaAxLaHqN/E+VXLAh8Sxj6y2eZ/NYR3PEo2AtQOXYZZ7yI0ZJGaApBzcEenkJNxtAyccIlaMhW/wSX//QpzPYsQleRXoBzRX5I9dQwl37vG7Pn9asOzug0RktqkcRL+dAlzvyHv8EZX3iOK+GMTDP4F09jdmSjXAzbwx0rEnoB1qUJ7OF5fu7zo4zX7GUZYX554cClCZ0mrRNFKFSDItWgiCZ0nLCOqSbIqq0YIoYrFyeO1cMyp61XZ2fgzXoXm2P7UFAZcs8w5l5adMxSxqDoT3Cs9iwzg2OPuYW1xk2E+JyxX6ceLCYLLLfKSCgZNsf20aL1YMsaQ/YZisEEgfRRhU6z1sVa8yZatR7UuMaR2jO4cvG3KIQgpiTYFr8DQ5gMOMeZ9AYJCFDR6DLW021sJqXm2BzbR7E6TsCcC9gKK5yxXp/tmyatk83xW1FRGXbPMeou/iajvln6nS8Fk5y0XkZBRREK68yd9Jpbl9x3OTRrnWyJ7SOhZLFkhUH7NCV/kpAQQ4lWVp36BnqMLYDgnHUAfxlavSFibI/fiSFiXHZOMukN4uOjotJprKfH2EJSzbI5vo/pyjeovfImqArxbRspf/M5wuoVBnQpItAK5KBFmLdvWK5Re/Uwybv2LL0fC88b370Nf2KaoFxZsN06dmbp66+mXdeI1RuV4hhubQrN6CbbvY1U2zpKw6cXtUzRDDpvei9CUZFhSHHo5KLloRBKw5e8zF1JSX16lMBz0Iw4ytXcbSugMuVTmfJvBGN1RQhFJdm2DiORwyqN41SLq39wocTqX5qO7I6XcMevmK1IueT20PZm3VlLNJTQE1RPD89VDVRVtPYWQBBMl5Cet+R5vVqAMzEXExGmQeBD5XD/Nd2eX6zhFxcPpLUzi3Nt3IlrZ2wZIkGz1gVE7hdJiCcdpvwxMmorGbWFhJLBXSIbOSRYwA6Lh+lZN5AT1qmG11azJsBbEIx3w2iQl0jqQWnBbytBQaXH2EKz1oMjLY7WnqUUTC5wTU37o9TDEjfF76FJ7aTX2MJFZ+lkOyEUdGFw2nqNMe/ighVT2c4TIukztpFUszRrXUz6c9npV/ZNLExFiZBCxZHX3jfzzzdr/CTLuqeWgyHirDN3klCy2LLGkdozC912ARS8ISyzwvrYLnqNLRT9cUa9pSekUd+YnLEPMOqeX9g3VoFQhqwxd5BUsrToPUvmPAnTIP3gXSipOHpbC35hmulHnsDcuCbabmi4Q+OUHnuG2LYNJPbsiMIDqQTVF9/AevMU8V3bSL1nPygK7vkBSo8tk0iraaQfuIPYtg0A1F45jHXkFKn7biN1/+2ElRqp0m1M/fWj4Pkk79lH6t5byf/Fl/EGolVofM8OknfujtQfTpyl9uJBUg/ciZpJobVGeXblJ57HvTS0dBtWwKpHaSlDJs68zJr9H0fVDdbs/zgjxhNU84P4Tg2hqJjJJprW7aapbydCQLUwRHn0CgVaIUh1bCCWaac+PYxTKeA79dkYg1BUjGSO1k23oxlxZBhQm1r9DS5u/1s+xVXOL2dpx7FMG2v2f4z61PCCWJAMA3zXwi5P4lTyb4mAoLW3EtsW6QtJP8CfLuJeGkTaK3+oSiZF7ns/xPSXHiWsRAO8kkyQfvAezE3rmP7br+OcWzw7B2j+zPdS+c6L2CeiOIC5eQOxbRspPvKN676PG4FWvQddMQlkQN6be1cm/cusNXcQU1Jk1NZFg/O7EaaSoF1fi0DMxgauhESS94YpG3lyWgfdxib6nWNLkxEkTPmjTHgDi1xwvnQp+MN06uvRhE5KbVpgVFYLrT0HoSQo1ZBhGOmoSQmKgtaURno+oe0S1q6vhEFabaJJ60IIwWX7FOUgv2ifkJDL7ila9V6yWhtrzJsY8y4tHbyXUAzGmfD6F/cNUd90GRvQhEFaaWKCpQPvWnMWd3CU4t9/K2JN+gFBqUL99aOgCNIP3omSiCNiJgjB1N8+htHTQWL/LuxTFzF6u3DOD2C9eYqgsoIunAxxB0bw89OomSSp+/ZTP3CUylMvobU1YR09i318LkZXefJFjDXds7WMlHSS9IN3UfifXyKs27T82CdxL4+gphJI2yH/Z39H8q49mJvW4V4ehWB149N1Tf2nB46S6dxE05pbMNMtrLntk9ilCXy3PhtHMdMtKKqOWysyfvI53PqVM05BsqWPnl0P49SmcWvT+HaNwLORyKhiZLKJZHMPUkpKI2eoTiwe5ISiEsu0oycyqLqJqpmoemw2lqOZSdq23InvNAT3fAffrmEVxwmvIUdk1ZAhpZHTZEe3ke3eRuvGW2HjrXM/S9koY2xjV/JMDRxh8tyr10ebFmBsWEPi1luwjp5CxGOkb9mGlc1Qe+3NFS2orFlUnnmZ0Jr7sMNyhdJj3yb3vR9aVTPcwRGC0jtVRnX5ZV+HHtGSa8E0Vjj3UZb9PLask1DStOo9DLtnF7h33o1IKhmSShRHynvLD/Ch9KmFZXJ0YChxTCWJFS5enYYElIIJ/GX0CrzQwZcumjDQxfVLhACY67qQjotY14lQVdRsgqBYRQYhWlMa+/wwwtCu26g0az2oQiWQPpMrKCVExnKErNZGSs2RUDLUllgphoQU/cllV0yedPCkiybMhX1zhdcjqFn44/mIPBKEoKqk77s9+j5mXGQNtqU3MoGs24R1G8IAoSpUXz5EbOt6knftxp+uUH36lSXbo2bSJO/ag3X4FGHNQujaLGngWqBmUkjPI5iOvll/chqtJVqdOBcHIQgIaxZKMn5dycjXZVR8p8bgG4/i2VWa1+1G1U2SbWuY6WUpQ2TgUysNMXTocSpj55dgOclZI2Ikc8TSLY2OmbdHgy1VHDjCyNFv41mLPxbViNOz+yHSnZsimqIQUd14Nbo1LZaiZ9fD0XJdRuRBp1Jg4LW/p5ZfWqDxrUCPZ2jfejeJll7CwCV0Fw9egqh9ydY1xJu6kIHHxJlXuF7npjs0RvWF16OkyPvvRO/tQjvfT+Yj76P0tScJposYG9eS2HMzpcefRm9rJvepj6AkE0z81z8jrK5MohCGTuaD9xPbuglnYAglmZjb/uEHid20Bef0BYqDkfvKWNtL8q59SCkx1vbgXR6h/OTzBIVpjPV9pO+/G60lh5JJ4Q4M437rDVJumsnCKVTFwNAT2E5pNnAbygBV0ZGECBRisRyWNYXn12f1qFJKE2k14vlPBaML4h0zWdtrzB00a12YSpz6EgPvuwlptWW2CuYtyfuXpUILBJowZv82RAzryuI3RFT5WrB8kDeiNUfXUJYt0XBtUBSB0p5DMQ2Cco2gXIvUAQwNd7SAmo7PxgOvBxk1Ip9YYRV3BddZSEAtLEUaZAjSavOSRkUSUAuWd+HN9I2AhZVJFwl/zBWjg0ifTe9up/rCAUQ8htDnMvSXkq9RknHsUxfwJgrkPvH+yKhoGkrMAFVBScQIqzWUZBwlHsO9NEh81/Y59zUgLQetOYuSjEcGSwhEzEDoKiJuIkwDPx/dq7Ghj7BaR+/uoPrSQYy+roWlCq5zMX/dQQq3XmTo8NcpTx4l0bQe3Wwils3gWzZOeZpaYZDSyJnlKcBSkr9wgMrkJdLt64ll29FjKRTNACnxXQunUqAydoFaYXDZO5RhQH16lNVkjXtWheAKVdh6YQhVN3Hr5UXKw1derzRyBj2WopYfXKDro8czbHzPZ0i1raUyfpGxU89RnRhorIiiYkkIBc2Ik2xdQ++eDxLPddCx4z4mzx9AXqeCgNbWTPyW7QhNRetowzpyEiklSioJavQRCE1DScYjleKBYQqf+xLNP/K9y+apzEdi/y707k7yf/FFjN4uErsj9pl0PUpfeQJvaJTYjs3zGqQSu3krpa8/RfmJZ8l+6AHiu3ZQe+UgybtvpX70JPaRUzT/8CewTpylPniOeOvNtDRtwfOqKIpGJtMXlWCtjuD7NplUD7FYjon8ceKxZkwjw1TxPH6jAFG7vhYFDZAIqdCq9y64h6DBUFTQaNfXrioR8rsBozEjljKSYFFW+MJnYkeB9JdMyI32kXgrsMxuJCqvnkTNJBCqij994423ISKSiC9nvqvlEUiPkGDW4C4FicRbreKclNhnL80ZxzDEG51YELSXlkP5qZdIv+8u/HwR6/ApDD2D7usE0zVS7eux7GncoTGkH2BuXIO5oQ/p+ZS/+TxCUUnvvAVtSw9hqUrqPfuxDp9C5C28gVGyH30Q5+JlrCOnZlcptdeOkHrgDrJ9XZS++hQiZpK6ex8yCEjuuwktl6F+6ATFL3+T1Hv2I3SNyndexp8o4I1MEDRc4UGxHJ3zOvQW31LyY25bG4pmEybOM3RwiGR3ltpwicD2MXIxzGYDe8pfVCPDaGknsXYjaixJUK8ydfJNQnf5GYeaTJHecjP1/vO401eIEboWI0dWJyOwFMZPv8D46cUFd65E6LsMvPrIkr+1bNhLsrUPp1Zk9Ph3KI8upRcV4FkexaGTxDJt9O37CEYyh2YmFolPXivUVBK9uwOhqtFSGG6IhtYMjPV9WMfPEEwVscoVMsWru7q84TGcc5cIi2Xcy8NobS1R+zQNPB8ZhoSuN9vOSnUUVTXwfQtF0bGdUkTUsKfQVJO6PYXtFAlCn0p1FGQ4mwWsCYMmrTNaAQrB2thNrOWmZdvWrq9jwDnx7o6rzJsRn7PeWOCaaUxPFv0dElJbUYrmBtaquQqC8tsrWnijsVpdOEIZxU5mjvd86q8tJknYxxfGNzpvuh9sher4eXr3foQzT/4xtRcjGn/12deoPvva7L6amcSsGpQeewG3VpzdnmjupfKtF6Jy1VfAG5lg+gtfByDXoqJrLoXHvkMYQku7ij0VIP3Gfn/7DTQdttxkclaVEautAffiW4ipXfeRQLw9hZmLR291CJmNLbglCyMbI7OxhcyGFi783ZFFn65fKeGMDZO5aQ/Vs8ejzpkphtRwUzGjqRUGhLaNYsRQEymYXhyUe3cgihEJRcOrl7BLS7O35iDn4iiSt8QgcPoHKT/5HPgBiVt3kdy/i9LjVzBHlMWSK9cKoetIp9FWPyBcoujSlQgtZ7ZOC6EERSA9D/vEWdLvfw/Ju27Fn8hjn4wMr2UvXXMDwA29BbUzXHfh7DetNEdaX0Aow2UHiJmM9KSSIa02Uw6Wv+Z3G344Z0Sm/LFVM6z+T8bMiity+608eVLQZrPmV8oHeucgEUJgplvxnRp6Ikvrpv1oeoKJsy8T+A4t6/agGjFq+csNd/o9IEMmzr2KEc/SsmEfw0e+hR7P0LJuD0LV8awyhUuHaFm3Bz2ZpTo5wHvfO8LEiEOfJTl52KG1Q6U0HaCq0LdBZ2oyoFIO6V2rMXjJo1oO6ejRsOshpemQjm6VZFph8JKP61z7+PSWjIoMJVPHxshuaUXPmAhFYGTjOFN1Ur1ZvKpD6C32nYaug1eeJrDquKUp1HiKltvvQ4YB9tgQTn6C7M5bAUnt0jnq/WcJbIsbTqi+oZjzpwpVi9x4K0A14lFNFSJ3nL+Cy+3aIKJZv6rSkHCFMERNxgnLOkZfT6OK4erh56fQe7sQh0+gpFOoqeT1t9I08YZGqL95AuneiI9ckNXaiClJAjzOWAeWpY526uvZGr8dTRi0aX3vsFFZ3cqxGk7Nvk9NWgdV991kVL6732E5yNOidxNX0ujCwFsi7wii2FBCzSBEFJervAsMswxDAq+OkcwR+h6BazF16U3iTV1ku7fgWhWs0jilkdNoZpJ4rov8hQPosSSZzk3kz79OtncbiqJGhQzNBCNHnoxEeXtvwkg1MTVwhKbem0hmipz6ZpmtO03iCcGOXSYjl31iMcHajTr77ozx95+vRHN4AdtuMchkVbrXahw/6LB2o866zTqf+4Piqu7xLRmV6kARIxujNlImsH2cqTqqoWLkYjjTFkJT0DMmXnllZlNizQbqQ5eoXTxNx/s+jppIUbtwCmt0kM6HPkG9f3kJiwU3k8yS2XQzihEjdCysiSHsieG3RNldDerTIzSt2Uks00bLhr2Mn34J/4qSw4qqE8910rx+D9neHcgwpHDxDdRcksCyEaqCdLwoFiJBa8niT5dRDJ2gZi2ZtWtuXEv2ww9GWl1NOeqHjhFUqnij46Tuu5NguoTW2UZYr4OA2M6tmBvWorU0k77vTryRMayT5zD6ujE3r0fv6SSxfxdaewvW4ZNYB4+R/dhDZD/6fqTvz0qjqM05Yts3Edu2Ca2jjfT77sE+vbw2kRIz0TvbQFWJ37wVETMJqzUq336RsHZ9RtUQJjm1HYGCG9bJ+0MEcumVVMmfxAorpJVmslo7umsuy/iRMojkiIRAEdefMDtDUVWIpMWvFZWgiC2rxESKDn09I+4FgnegJs61YL6G2FtRcL5eFPwR1sgdqEKjVevlsntyyf00YdCidSEQ2GGVWlB8Zxu6BGQYEHo2aqqJMPBJta8jlu1AURplCoSIxispG7HleqNEiDEnizXPpnv1UiSBJSWqbmIkssTSrdQKgyjCZetOk8CXCEWQyii0dag0tar0rtOREuIJQWuHStcaLbo24NoS15G0dapMTQRcpSTOIrwlo1K+sHCmN/F65IczWxK4JRsUQehcnbopfQ/RkJGXjVoMqCpCVWdrVUdYebanZ3Kk199E5dJJFN2kbf+DTB74DtbYjWd5LYWpi9HyM97URfu2e0h3boqo1k4NhEAzEhjJXES5TkXsntLIafIXDhDbvSFKhkonqL96DL2vI9L9acrgTU4jLRs5MEboL+wPUXYxKybxNTupnDqKde44qda1pO5aR+3kObSNGxGWT+lrT6Kls2S378UujZPp3o5eiUOul+K5AVpuvx+zrw+pSspPvYxXKSOdyIXljYxT/Mq3UHMZwmqN+qHjBNNFCCXe2GRDOkIgg4CwWiecKlJ58jnCepTgZp88i7jQj9bRitqcZep/P4J0PfSudtLvuxcllbxuo2KKBFmtDYgyte1weX5/LSxRC4qklWaSSpaU0sR0sHSBNFfaEeNHKJGmFPp1Dep2GN2XIlQyajOVa1wdOWGNCe8ya4wd5LQ2+sxtDUHMpSdIKhGz6J0wPG5oNwxLpL2loS+brf52oBJMUfTHada6WWNup+APUwsXxyO7jI1k1TaklAy5Z69TrPIGo5FS4FmVyEgIBTPVjO/UCX0HuzxJru9mEi29WNOjC9ziQlHJ9mwn2dKHv66OXZpYwDarT49GcZh0C05lime/WSP0PcqlENeRvPhUnelCwPRUyPhIQOBLfF/y8tMW1UpIrRrS1Kwy1O+RSChcOuuRySm0tKnkx6+9794WlWKnUGeyaAMSGSzH2goJrDpISa3/PC133Eesq4/K6SO40wVye24ntWEr04deJtbZS6J3HWZbJLdtDfcve23fqlK5eALfqqHGE8Q7+rDzo2Q23kxu214UI4adHyN/8Bm88hR6toX229+PkWsldG2mjr5K5eJxzOZO2m5/H3oqS+i55N94htrQygqhbr3I+ef+kr59HyXVto5Ec08UZ2nMAGZeKBl4uLUi+YtvkD/3Kp5TxdA3EFoO3sgkIm6iNWdBSoJihaBYiVxX6pUy9GkyG25m7G++gF8uohgm2Vv2U71wGr9cJHvL7dRPnSPeuw5/Ik8s205o2VjnL5CvfY3k2k1Mv/kKQtOhdS2T3/gKRnMbZnsXtfPHIHAxEhoBCsHkJGF+ElUTBJ5E1QQI8Psvz73YQkSS+BK8eRLhwUxgXwiUeBxjTQ+hZRPftQPpuIQrJXpdBU16J6aIKM5LyYXMhyQk7w3Srq+NEiG1VorB+JIB+2pQxAor6CJGl74BK6ww7vUjZdggBChIKbHkyuymUjCBJ20MEWejuRcvdKmEhYjm2lA/Xko/K8BnyDlNWm2mSe1kY2wPObWdEe881aAISDShk1RyNGmdtGjdnLJeIe+/9QThq6EWFqmHFQw1Toe+jnpYYdy7SCgjQ6Os0DfR6kabJVUIBGqjvodAoAsTXZizShsSuSinyJUW/c5x4kqahJLlluT99NvHKAWThDJEFyZdxgb6zO2AYNK7zKh7beq+NwoaOmHjf/P1kyfPv964VxrfS0hl7Hz0DckQGYZYpYkoG6vBvJJhiFsvUm/UZqqMX4x+CwOqU5dAhUL/QQhD6tON5y9gWkYTdMXUkX7IpXNzhn86P2ckBi7MbR+tR32dSAoyOYWxYZ/CxDuQ/HgtkMHKa6agXmP64Fz98PwLCxlc+ee/teDfo9/4u9VcHdWMoZoJ7HxUcMsau0x9bIDQsWm/42HSG3YwdfhFMhtvJnAsLn/tf6KYsVmetl8vk3/jGdxygfT6HbTuf/CqRgXAqRS48MJfzxoUM9WEohpAQ1DSKmNX8lQnB/DtucG08vSBhf1TqmL0tOMOTSzWFmog1t6FO13AzUciiYoZJSt5UwUCq05YrxHYdbRkGsUw0dM56sMDjcp8XrSy8D1UTUeGktB1sMeHMdu60JJpNFlmz/f0UBioUZtyUQ2FRFYn8CSFwRqJrIFrBRhxlcAPUTWFyYtV7OrSq1N/okDlmZeJ774JoSp4Y5PUXjowu6JZLQQKHdq6SE02qFBcZtUxH3l/BFfaxJUUzVonI+75JX3yIQEXnMPsiN9NTEmyJbafjbHd+NJDQUUTOsPuWU5aL694PSeMBsD15i3E1RS7kw/iSYeQABUdRSgcr7/AmHdx0bHVsMgZ63U2xfbSrHXRYayjw1g3F7ubx/ALZbB6BtN1IiTkon2YHYm7iSspNsf2sSG2C1+6s30z4l7ghLWYTZnT2unSN6EJHVVoqOgk1SwQ0ag3x/fihBYBPoH08KTLRfvwIr22gj/Mefsg62O7SCvN7Ezehy9dAhk0ioephDJg0hvgvP3mknpvbxcEgi5tPdWwjGTmuSiNtF2JioYjLVQ0QgKqfon5Pi0ZeOiZZpK9m/BrZWrDFwl9d064Vwgy62/Cq5VRmgOUhI47WgRVQXo+QlNRUxF9OihbGF05QtendnzwquPyDOo1yaFXr6/PvqtFut4OmM3tdN73cQDcYj4yBGGIUDUyG3agJbOYLR34VsTHtvMjpNZuoXnPvZROv4nbKAEsw4B4Rx+5bXtREyn0dJZrFfGSgUdtsp/aZP9130dYqWOfvsrxgqvzyMMQa+gSifVbosBg7RryBhrZuTKUKIqgZU2CXHc8ivGYCr4TUik4JJsNmhIqZlLFKvvEMzqFyyskUoYh9tFT2EdPLb/PKhBvrDYAJv0h/Gsoe+pKi2l/nLiRIqd2YIr4soHevDfESV6mS99ATov2NYSKL13KQeGaAv2SkCH3NK606dTXk1Fb0YQxm1tS8ovY4fJ9Vg7ynKy/TKveS6vWQ1ptwVTiCBT80MWSVSr+FMVg7B0lHuT9YU7VX6bT2EjTbN/E8KXX6JulWZpptZk+c9uSvwmhklZbmC8kHsqAy86pRUZBIhn1LlEPq3Tq62jWuoirGXRh4kmbqj/NhHeZSW9wSYXitxtRe0OalS4K4QgaGiCIK0lA4IcecZHCJ3qGV67GEh1r6H3gB/Aq0ww8/pdYk3MrUEU36LzrI9SGzlMcew2jNYNiaAhNpXZmhPSutXiFKkKA0RqxIhVTR4kbBNW337i+I0ZF1yGdVOZT76nVJLYrUVSFMAgXEu5hbuyev61BXV4JXqVE8eTr5Lbfil+v4tcqqPEUHXd/iPKF45TOHiG349bZNI7a0AX8apn0xpvpefjT5N94msqFE3Tc9UFkGDJ17BX0VI54W88Cm5JNK4RSUq3Ja2YDx2MC0xTMSCF5vqRuySWldTQNUkkFx5FY9tIXcCZHyd60F6OlHb9aRvoehCFGSxteqYgST+CVi3iVIm33fYjaxdP4tUqUZasbUUKk3sjGVhSUWByztRNkiF+r4AU+Bx65HHV8tB4HIQgDSRhI8v1RrXAkpFpN1uzJoZkLXXSZlAICqrXwmgN+RnMbejqHVynhFvMNynWI2d6NOzU5G4PTN23ilYuPEXouvvCuOe/klPUy56w3oj5cQaJdIpn0LjPtjaIKfdaJEZV0CpclBFwJT7qMuOeY8AZQUK84T4B/lfPYssqwe4Yx9yKqUJkpcCWJssEDGTQGpcX3P+ScYsy9CEicJRSMZ1AK8hyofhOBQnBNiYCSSX+QKX8MTWjz2rRy3ww755lwB1G16D2aIS0uz6WRKzwjSSmYoBIU0ISOQJ1dDYT4+HLld2LIOcu4OwDIJdWdZ1AOpnij+kSjb5a+LyURI/fx91J95SjupREmg2Ekkko4TdBIvhRAMYyefUhAlSI0+mu5+9NTWXJb9mLlR5acQFoDk9T7R2az6kPHZ+rp40gvaLA9o+0ylAsqxr6deEeMyj23x/m9X2ujt0sjkVAwdPjpX5zkr59wadvRSulyGd8JiOVis2Vq/bqHYqj4lkfohSi6gpkxCQOJV/eoDC094w49B3tyhHztGdpuex+1oXMEjo1qxqiPXCJ0bcxcazRYAaqZwKsWKRx6ltC1yWy6hcrFk8Tae5l87dt45WmSPRsW5HhkUgovPtrH4IjHP/23EwyNXn2GvH2LwS/8TI4H35OkvUXFskNePmDzi/85z4nTiz/iB+9N8Bef7eBvvlLhF38tv+Sn4VcrFA+/Tm7X7SAE5ZNvUjlzjMyOPSTWbKR86jCBVQNFxRrux50ugJSo8STJDVvRkhnS226hcuoIoe+T230HXrFA8fCrkYECnNryL+L8X6YG60wNLvz4VQW++TeRMf6pnx/n+BL3uRRCz8Xs6MaZnkTRNMyOHoJ6DbO1E8UwCR0bZ3IMqYQ4qo0kiAyRomKNDi76+HQlRi7ew1JEj/TsX5KyM47jL57V+nizA79oFD6LSCQSpIJQBIoZzUSl10j2nSnQGEqEoiD9AE86ZMwOTC296BoQlZYtO2P44eJ+kkh8XPwV7KZAkIv3oClLZ46ngYozge0vTpCUhCsa2OUQ4F2zcZ2//9ZdKfpP1klkNJJplaFziwd1U02RiXWSXiUde2mEke4eIUHoE0ofP3TxQwcvmCEeLI1r6RsRM0g/uB/n0jDupZFZUoDfOO9Sj01ehTgQWDW8Wonctr0Ujr+MW1q8+pNeQHgFNT9sFLv7bhG/35JRyaQVbt8bY+cOk+ZsNCOdLAScOuvy+ps2xVLUocdPufzir+dpblL5xAdTfPLDUZ5D6EtUQ6XtplaEKlANlcAJ0JM6MpAIVTDwzGUS7UkSbXG0mIqZMZk6O0VluLJir9mFMayJIXLb9pE/+CxOaYrWWx8gsGsNtl60NEqv3068oxcZhqhmgvKF4yAllQsnaLr5NpJrNkUMtHkDlVAgFhNo2oLE52XRnFP41V9o5gP3J3nsqRqHTzjoGrgeFApLv1i6JtC1aJW3rNdNSuqDF6gPLgxQ51+8QmEgDCgdnYvZBFaN/PNPLNhl6tWnr34jq4QQEI8LXFeuKu8yqFfxa1WCeg0j1xKtnsIwSpINoxWLV5qru2K2dRLr7EVoGk5hnNBZuMRPmq3s6f7EitRgKSWXpl/jXP6FZQcYNWGS3NaNPVjA7MohtOh8WiZO/dwYStxAqApaKoaUIVo6gTNeJKjY2MNTCBTWN91GV2bHkuevOJMcHXuMinOtlQ8XQhE6W1vvJxfvXnaf4+NPMFRamPktVIXk9h5SO9cgFIXamRHKhy4u0JSa7YNUjMze9Zi9LUw+epCgMmcM9LYMuds3I0yN4ounF5domIdcu8E6IYjFFYJALmlUcvEednV99C1RuiEyyFJGRiWUPkHoRcYktHD8OrZfoeYWqDiTlJ1xwmtwoy6NG1tXQ8qQyuBZshtvoeXmOxl96dGrHqOnm0iv2YqebgIpcaYnqA6fx69FE4lE5zoyG25i4o3vEDakqnJb9xFr6qDcf5L6WD8A8Y41ZNbfxOShZwmXqZe1HK7LqAgB+3eb/NK/bmHPzSbptMKMOojnQaUa8kefK/L7f1rEcSWThYBvPRNZ+r5ujU98KDIqTtlh+PVREERuMC9oJNOLKF8jDLGLDk7FpTJcIfRDhADfCZYcZO38KBOvPIFfq0Q14I+8hBpL4NerTLz0OFoyjQx8fKuOUKKRutJ/CmtiCCEUQtfBrUQJUlNHX0LvbwYh8GtlSqcPztL7KpWQD/+jYVxXMjp29Rdw5w6TvTtjnDzn8rO/NMHUdIiigK4LnGUyVZ99uc6D3zfMZCF42+X6rwXxmOA//utmnnimxgvXGMDzA/jUT44iBFweWiXltGHE9aYW1HiCoF5DBj5maydSSBTDwGjpJHAcvGIBFIXAtgm960uoFELQm7mFy8VD2P7Sq2AlppPY0E5Qd0ls6EDoKkHNIag7qOkYWjqO3pRE+iGh4yE0BUXX0HqaFxQfe7dBb03T8v5bmHr2ZKSCoIglDQpENXpqp0do/dBepp45MWdUVIXc7ZtBVSi/cRG/vPJAdPylEht3pgC4fOrtlXSJmGYqCFDR0NUYC9aoUhJIFy9wsP0y45WzjFVPY/tVVj3fV1XQVmEEZ1ME5rmYZ7YIFTsf0YTT63YwdfJ1nOmlK5cCJHs20nXvxzCSOWQYIFQVpMSaHGbkha/iTE+gp3I077iD8sXj1McGQCi03nIP8fZeQhlQHx9ACIV03xZym3ZROPLCqukf12VU7tof5/N/2EFLk8pLr1t8/stlTp9zURTB7ptM3nNnnJcPWDjuyg9EBhKndHXJd6/m4dWuPihJz8Xz5j7ewK4T2NEL69cr+PXFg0VQrxLUF7s8Qs/FKcyjxVpzwdRQwtkL1z5IdnWopNMK336+TmEqekRBAMEydGuAak1y4sy7QVYiQkebyo/8QIajp1Yn0X+hf/X5CzIIKJ86DFJSPXeS2oVTDcolzAZxpCT/YoMhKCXuVMM1cB0CeDPQ1Thrc/s4k3+OpQaTwHYpvnoee7iAdWkCs7sJa2Cysev8IOCVf7/NleHeIhRTB6Fg9U/iF6uzzRe6itmZQ+gqfsXGy5eRfoA7USJ0POZ2FMTXtRFb09roD9n4fWnopkBKGDxbJ9Oik2nRKU9990oRCCHQhImmmMS0NLlYN+uab2OkfJzh0jHqXvGamHUiZtD64x+j5Uc+fE3XlX7A4M/+NgQh8bXrURNJahfOzClNNAgz+SMvsH7ddnJb9jB+YGmdw1hzJ933fhwZBlx67C+wJ4dBKOS27KHzjg/Sdc/HGHj8c7iVKXyrSry9j/rYAGZTO2o8SW34IrFcO6qZQIYhZlMHVn6EMFj9c1m1UWnOKfzW/91Ca7PKn3+hxG/83hTTpbkOf/OYw//+0jtVW+P/G4iZCqoK9ir0c95teODeBKbxDg6OszM2ORtnW7j9yr9vDJ22I7WFofIxau5iJlVYd7Hr0fbQ8bAuLeWmkkv8/e5+7l6hgj1coOP77qB+ZpjyoYsEVYeme7ejt0Y1XYy2NONfegV3cvG3HbnPehsuwcjP6YwVkUuVfVAgmdXYtDuFqkKmWWdyyFnS/fXdwEy5hZiWYn3T7bQm1jNQPMh49ewCPbYlEYQ4FwbxJ65NDiZyqzfo4ZpG6Doouk4wa1QAIbALo1QHz5Beu43iuTcXT46FILVmK2ZTe8QUG59J9g4oXzxGqm8z2U27MNJNeJXpyKi0RSresdao2Nn02YO03HwXWjxJ4NqYuTbK/SdmY6urwaqNykcfSrLrJpPXD9n8978oLjAoM1htWv+VUBTYvF7n4fuT7L7ZpCmnULckZy+4fPnRKieXmMErAm7ZYfKxDybZusnANAT5qYDzlzy+/VydIyedRe1qaVL4gY+n2bfLpCmrUqmGDA77vPyGxfOvWFRrc4NBPC74qR/J8uC9jVoiwLHTDp/94yKTS8RF3n9fgg88kKCnU2PbZoNUQuGjDyfZuF5HAH4g+eo3q/zl3869IOv6NP7tTzexbo0+O7d97Kkaf/K/V1YvTiUFD92X4L13J+jqVAkDGJsMeOOwzZPP1hmfnGtfIi54+P4Ed98WZ02vhqoIRsZ9nnvZ4vFv16hbc/fc1aHyg59Is32LyT23x0inFP7dP2/mh78vM9u+lw9YfPZPirMGU1XhH/9Aho9/KDXbTxcHPH7vT4tcurz8C9rTqfHpT6TYtcMkkVDoH/R49MkarxywcOcdlk0r/MnvtPP1b9V49Mkqn/5Emrv2x8mmFYZGfb7+RJXnX7W4Bs3LRRBCYGppOlPbuDD1Mu92Y3CjENQc8t84RHxtG+m964lv7GTiK6/T+qE9+MU6oeMRX9dGeWPHkkZF+gFTTx1Fy8SpnR6mcmhxMb3ZfUMo5z0uHq1i10JiSWVBvs27CUII0mY7W1vfS8po5XzhJYIVRCml51N94U1qr524xivIWTejMzqM2d17RdB9Tou6cPxV1n7wR0n1bKJ0/uiCCZWiG5hN7QhVo23f/bTccs/scUIo0W9Cwci2Url8Gq9aJNYSld2ONXfi1SrYhbFItDeWRIYhejqLU5y8LomrVRkVRYGH7k8gJTzxdJ2BobdnyXrTVpOv/GUXMVPguJIgkOia4EMPJvihT6b5+f+U57Gnagsmqt//8TS//+tt+H5EwZUyouV+9CHBJz+c5KFPjVCpzVmVm7cb/PFvt7Nlg0GlGhKGElURGIbgR38gww/801FePjCvKmIIU9MBpXJAZ7vG3bfFSSQU4rGlB/wN63T27DTRNUE8LhACTEPQlI1mckEA8djC6LXjSibyAR3tKps3GNy0xeDigLdsUTfRMKS/+X+3cPveGPW6xPUkgihec+/tcS4NeLNGRdPgT3+3g/fdG8dxJZ4fvbZ37Y/xI5/K8MijFX76303MDuK9XTofeSiJpgoyKQVFQDI5dw8AiYSywLsjJUwVA6aLAW0tKvfekaCrw+V/fnHp1auqwgP3JPgfv9VGOqVQrUnCULJ/d9Smz/1tmV//bGHWwBuG4MH3JEDAZ74/zfbNBp4X5dO85444n/54ml/93QJ//oXSdRkWRai0pTYyXj1D1V0610KISMcrCv4u/OhU1SBo1MURQsxmRb+roQgCy6V6YhBnokT3j9yH3pICCUN/8R2CsgWaQmjdGHdsGIJuKNz5kRaQcOiZ4g0579sBIQSGlmBt0z50Nc6piadWZLtJz0c6q++noF6jfvHcsjNyuzBG5fJpmnfcTnXo/ILBXigaqjHD+BMo6sKYjlvK4xQnCT0bpMTOj5DsWo/Z1I6Za8cav0xg1fCtKrHmDjzDJPQ9vOr1leJYlVFpzil0d2g4rly1b301GBr1+J0/mmZ41OeNIzaFqYDODo1/8eNZ/s0/a+JTH03x4usW08VGMFeDf/8vm6hUA/7dr+V56jkL15P0dKrs2RlDESwwKIqAj30gxa27YvzmH0zxx39ZYqoY0JRV2bHVYH2fxulzC18Mx5F8/ksVPv+lCumUwumX1gKgZpIwXkWJmQhDJ6xZSMflT/93iT9trDB+5PvTfPbX2njk0So/938vL4k/Oh7w65+NYkLf83CSz/9h54r9lMsq/OZ/bOXu/TH+/rEq/+tvSpy76KEqsH6tTluLuiAu4/vwN/9Q5oVXLV4+YHGh34sG9XsT/PovtvDpT2T4k8+XOPBm9GwPHLZ54JPDAPzhf2nj+z+W5ld+u8DffXX5ZLIwhK88XuMrj0c5LKdeXLfiPezcbvLbv9yKpgp+4VfzfOOpGuVqyC07TH7+p5v4mR/LMjru8/t/Vlxw3Cc/lOLVQzY/+i/GeOOIg64Jvv/jKX7j37fyz340yz98o8rYKuUlIBpEMmY7LYl11NypJX3pppkhl11HrTZJrT6OqhgEoYuUktaW7RSmzqKqBppmYlnTjUJu795Vj9mRI71nPaHloOWSuGNF7KEpiq+cpeV9t2ANTKLEdIovnkZNxUhs7EDPJUnvXkfNGMYeXH05inU3Jzn0dJFUTqNrXYypURfHWp0BdgOLqfplQrnyc56R1VEVHU0x0JUYppZGU41li5pdCUWodGd24PhVLk69unjFImVU7uEaM9aXxAountC1KF84RnrNNtLrdywo6CcDn8CNVNwHn/oiXmVl95uVj77pRNd69HSW0rlD+HYNd3qSRHsfbjyNVy3NMsZWi1UZlURcwTAEQSApTL99iTTTxZA//fxCKzk47PPZPynyTz6To6dbI5tWZo2KJKLg+j4UpgJq9RAp4dJln0uXFw+AkiiHQggoV0IK0wFBENGhn3vZ4rlVtNXY0EsiKVGzaVBVvNFJ7BMX3roP8Bpwx74YD9wT5xvfrvFL/yXPyNjcMxkeW/r5PP6dxUybb36nxgN3J9i0XmfHFmPWqLzd0DR46L4EWzcZ/NZ/n+ILj1RmV2SHjjr81h9Os/tmk3/86QxffrTK8Lx8INuR/Pp/LfD8LBNN8vePVfnI+5M8fH+SbEa5LqMCMwPITYxWTuIGi/vLdWtoWhzHLSOESiLRhmGmmJw8QSgDhFBRFZ2YmSNmNlGYujaV7e8WYk4cUQqQmoQRh8Lpk0jHI//4IZLbejDTWWQ5yr/RhYmGyfhXXie03dkldOj5lF45i1+6NiZXadJj064UuqmgqLDhliSnX6+siuloeSVOTXwbL7xaPKZhVETDqKgxYlqalNFKc2INTfHea6ItK0KlN7uTsjPOePXMgt/CqkX+f30Nt3/k2m9glaiN9lMfH6Tl5rsWxDpC38WZmkD6PtmNt5A/vPIIZudHAUmivQ9F1bGnJghdB6eUJ7N+B4oRw6+VliQ2XQtWVWAjCKKxUiDQtbfXD6oIiJmCdFKQzSjksgqKEhkNQxeo6tz1fR/+x18W6WzX+F//rZPP/lore3aaJBNiyfwIKeGp5+qcPufyiz/bzJf+vItPfDBJc05hXhnpa2tnMo6SShDWLaTvR0KQ7xAeuCdBEMC3n68zugoVUV2HZEKQSSvkMgqphEKpEuD7i11ybycyKYWbtxsYOnz1m9VFA8rxUw79gx7trSq3bF9Yn+bsBXcRA89xJJeHfHRNkEy8tfvImO20pzYt+VsYegSBSxC4xGI5UqkOUsnIRy2EgqJE9NVsdu27fpXSavRh1HRqr/ZTfO40zqFRvFIdTRiYbgz/WAH7lcv4hyYxAgO9rCGOVnBeGKL06nkYsoiTQoTg95cxSjoKKjElRULNLSuNXxh1aF9jkmvTOHeoysWjtVUZlAiSUAbX8H+fIHRxgxp1b5qSPcp49SwXp1/j8OjXODj8CIX6AKH0F6j+LgVDTbImt4eYllnYEs/HOnSaYKoxu1eUayrVvRoEdo3i2UNosSRmrm3exWWUYzI5RNue99K66170VA7VTGA2tZPbuo/WXe9hRkbEr1fwamUSXevw7RqBE61y3OIkWiKNkWvFKeaXLwV/FaxqpVKphtTrEl2PAqtvF5qyCu+7L8EH7k9yyw6DVEpB1wSaCl0dGmMTi2M5f/KXJYZHfT7zqQwffSjFT/xQloNHbP7qyxUefarGZH7hoPvKGzY/+M/G+Jc/meWuW+N87g86mSwE/PUjZR55rMqpc+6S8ilXovbyYapvU2zpauhq16hZIYWpa8tl0VS4aZvBJz6U4u79cTrbNWIxgapCLhOtQt/JkKlhCLINoaelVhVBCPnpAE0TNOUWDk6F6RDXW3zTMwvE66xHNgshFNbmbmWscmZJ1o/jlJAyiOq4eHX8wEVVY6iKTjLRRt0qMDZ+mGSiDUUxCJfIkn83oOYXaTK6Zwf/jNaKFZTJah0gIKFkKfrjpLUW4kqaelBCV2JktXYmnH4SWhZTSVBwh8hobfjSxQ3rtJvrAEnBHV6yjsmmXSme+btJ0k0qqSYd59Tbu7JXMqm5VZXtNLTtQjxpU6j3U7JH6c3uYl3TrZhqalnygBCC5ngfLYk1DJePX/kjWlsTxtou9PamSEVhvIB7aYSgdGP0x8oXj2HfcjeJrvULtrulPCPP/j2dd36Ytr0P0Hnnh0EoyNAnsOuULhxjPt3dmhymZefdFI6+MJsEaRcnkb6PkWnGLlxdnHU5rMoylKshFwc87r49xq17Ynzl8SpvxYW4FBQBv/izTfz4D2U5etLhkceqjI771OuRTtAf/Ebbksf5QeTLf+ZFiz07Te69I86H3pfks7/Wxr5dJr/wq/kFzCaAU2ddfvY/THLTVoPb9sR4/31J/uU/beJ99yX4Z78wcW2yIt/FSajlhOhaRC64Ftx1W5z/+p9a6ezQeOTRKo98o0qpFGA7ku/9SIqPfzD1Nrd4IYIAbDd6gaKVxWLDkowrhCGLkkTD8No111aC5ZUw1CSqsvhTSOg5OtPbFmWgA0xNR4rV1doo1dro7PbRsYML9qvXr1ZW+rsPTzpk9FacsIauxEioWYQQlL08qq6hEGlISSSK0Kj7JaQqUYQyq2ImADe0qAXThIR4oYMVVpaNWQxfsNi0O4miCAbPvv317GNb1hHU6kjbRUnEIjqvUAhrdbyhcfzQYWD6DRy/xo729zUSJJeGEAq92V2MlE8uiLkZ67pp+vT7iW9bPytrTxhSP3CS6X94Bn98ecFPRTPQYik8q4wMfKz8COOvPdEQkoyMgWrEkTJk7JXHSXSuxSsXEfMkPezCKEPf+Tsya7ejGGZUwtsPCOwa1ZELC2j306fewKtMUxu+MFvr3isVmDj4HVQjNptZfz1Y9XLj8e/U+KHvTfPAPXFu3m5y5MSN9b93tKn8xD/Kcv6Sx3/4jTwHjzizhiuVFOha+4rHF8shz7xk8dLrFo88VuWPfrOdT3woxV//Q4WXX1+cCe77cOSEy9GTLl//Vo1/8sNZfulfN/Ph9yc5ccZ9V2SzL4fjp1z+8Q8Itm40iJlixTwY0xB85P1Jdm43+Q//T54//XyJWn1GQj2Kz6y0WpdyljZ/w1CphfQ3aMb7dplcHFjozmprUenp1LCskIsr0JHfCkr2GEIotCc3LZqdKkKjM7WNyep5nGAF9eX/D8OVNhUvTyAjAcZR6xy+dKlRxJceBXeIkJCyP4kqDELpRbVSgjKB9LGDGopQccI6TlgnkAESSd4dJCRYkv3WuS6G50jOvRklWr4TiY/S8/CGxjE39qE150BVEKqKfebS7ApGEjJWOUVMS7G59T0oK2gwZcwOkkYLVTeaNAhTJ/3ArZjreyh+/XmsI1Eczdy2ltz33EdQqTP9d09GigVLINW+DkU1UFrXMDVwFOn5FM8cJPR9mtfvolYYRtVNFFXHLU0xOXqJWKa18U0qaLEUQlUJHAuvNI1dGsd3LFQ9hqob+FYVLZZCUXV8p4adHwbXiwoINhD6LsUzB5ds32qwaqPyxNM1nny2xvc8nOI3/2MLv/q7U5y/5FK3In2neEwhlRRM5gOq9dWPyPGYIGYKLDukVA4Jwmj10pRT+MynMmTSix90KinobNeYLgZUqiFuJBHFZD6itqqqQWJerEDTIjdaGMJ0MZhVAa7VQkbGfMJAkkq+c7GF68XXnqjy8z/TxGc+leHgUZvnXrGo1eSs5lY6qTBVDKjWJKrKbIypMB3MGiDTENy6y+Se2+ML4lRXYrIQkEwqbFynk4gL6paMtM8E10XdBbBtydMvWnzqe9L8659q4tBRh8FhD8+P3HGf+VSaTRt0vvVMnaMn3x7ygEDhcvEQzfG+RbNTIQTZWCetyQ0NV8e7eIZxnQjkQkFIK1wYnJ1fPCyYp4k187crrdlu8ecZkOXKCQC095ls2p1kcijyBFw8XmP4bU5+tI6ejSRLjjQC7DPKDFfI0UhCRsonyMV7lpxozEARKu3JjfOMikF85yYqzx6k9NgLs7Ri58IQimmQ2LsNvasV9/LSbiWlYTBAkGpfh5luQdUMpvoPY6SaqeWHEIqKkcyR7tzA+KkXMdOtKIqOr1lkuregxZJMDxzFSGYxU81M9R9BNePE0q149TItG/YReBZWcRzNTGCmW7GmR/CsG5usvmqj4vnwS/+lQDymcP89cb70F1289JrF6ESApkFnm0ZHm8ov/1aB516xSCUFu2826WjTiMcEt+6KIQTcfVuMej3EcSSVasi3X6gTBDA05nPgTZu9t8T4Dz/XzBtHHEwjkn/p6VY5eXaxS2r7ZoP//l/audjvcaHfo1gOSMQVdm43ee9dcV58zebQ0bmXvCmr8ks/18yGtTpHTzpMNOItPZ0aH3pfkpHxgCeeXhg4XL9WZ9eOaEXQlFVJxBW62lW+76Mphkd9bEdy5ITDwJB/XasbXY/otet6NWLxSKhT1wU7thp85lNpLDvKv3nupTqVRs7G8KjPf/69KX7p55r5k9/p4Mln6wwOe6iqoKNNpaNN5Tf/+zTPvWxh2ZI3jjj8wMdC/vVPNdHdqVGtSjrbVe64NYbvQ20FSudTz9X52Z/M8eOfztKcVRmd8EnEFU6fd/nK49VZw9LdqXHrLpN4TBCPK2QzCjEzokhv32Jg2yGnz3mcvegShvDCqxZ/9vkS//ZnmvjLP+jgmRfrVOuSbZsMPvBAgnMXPX7j96euKb51PdBUg4ozSb7eT2dq66JBRFdjtKc2MVm7sCQT7P9PEKqgY2cbrdtaGhugOlZj6NURfOvaVxvVos/Zg1VG+6Nvsl5+ByTZZz7KK/+7BJygymT1/JITjTkIsvFuaLB3haqiJOO4A6ML81SkxDp+gfR796Gk4is20XdqKFpkXPRYCrdWJHAdAtdqyNiDNT3WqEcfQ9EMjHQzwfQYbq2IZ5VRVB1F0QjDAEXRUDUDPZFG0U3CwKOWH0RRdQLXRk9kqIzd+IqY1xVtP3fR41/8+wk++nCSjz6U4oF746RTCpYlGZ0IeO7lOsMNocWuDo3/62ebuXWXiWEIYo0Vw/d/T4qPPpTE8ySFqZD9D1+mbklcF/7Nr+T5Fz+R48F743ziQykKUwHffqHOL/76NB97OMV77lz4cPqHfF563eL970nwgQcTJBMKti25OODxp58v8bm/KTNVnBswK7WQlw9YbN6g84++N01TVkVKGJv0eeFViy88UuGNwwtnxg+9N8Ev/5tmDD2KYSTigo3rdH7l51uiREJP8iu/XeBzf1teegCUK89z00mFf/qZLB/7QBJDj1ZrpgH33BZn7y0mnhfFFR76gSFON8qCej584ZEyg8Men/5EmvfcEaetNYnnwei4z7efq8/ScKWErz5eRdciZYCf/5kmIIor/fkXSoyNB/zhb7Yv28Y3jtj8wn/K85M/nOEnfziLpkWrlz//qxKKmAsA3rkvxn/7jTYMQ2DogmRCIFH4hX/eNNtPf/g/i/zXP/Zw3KiezB99rsjlYZ8f+3SGn/6xHPGY4PKQzxf/vsJfPVLm1BITiZVwNQbPfGiKiRfUmaxdoCWxFl2NLYoDtCTWkTE7yNeXzxT//wUk2EWb0mAZIaBjdwfJ9iRDr66ORjt83kJKcO13b2Jo0R7B9itoirnsaiWuZ2ZzlKJaTxLpLl62h1ULoamRwOMyqE30E3guqhEj9Bzc6hSB7yBDn9Lw6YhUYFWQMsR3agSuRS0/iFAUPKtC4FmAIAw8PKuComqEgYtbKxIGHqHvUho6he/WEUIht+ZmKqPnUM0EQlGvK3N+OQh5jV/gUh2rqRHtV9fFbNZ3EILrShw3CqSqSlRsSlvBfIWS2ZyTGcRiAtOIClqFYZSX4Lhy9npXFn0yzcb+KnNtCcC2I3eYUDXUWCKqgxGGKH4dQ/XRtAbtWEbXcVyJ7SwOAsdigmR85YBCrS6XjGuYhiCRiBSJryQLzECIyD01o6+lG4KmZoWpfIjvz/h8iVyCweJjY437V5RovzCI7mXmOcy/j74+lUpZ4jgS35PUG+6/dCoqCjaTnFsqLmxrJiPIZBSsukSGc89l/j2bhiCVXLmfLHthPyQ7k6S7UoSlOv50HaFAEAqsWoAXRnXvZSjR4hrprhRMlQll9A4EvpxVuZZBSLpJJx5XKIw7+I3Jcy7ey229n14yF6HmFnmh/8/QlRi7uz9Gc3zNku/6ZO0Cb458ZcVEO72nDXNtB/bZwegF1FT8fImZoKBA4ZbOD79t0veqMNjf+wOrlr6/HsRyJvt/Zi8nvnyaqXPTqEZU70goArfq4tV9VENFNVVUXUHRFeyiE5W2SOjoST3K7i45hN7SxqUjtXVF6fuSPcobQ19elKdiKklAogkTJ6zNuvdkQ9xzppCXKqJBacaVJ1BmA+8Chb3dn6Q1uWFZo1Jzpzg4/Ah1bxq1OUPPb/1Lin//NNaJhaUo9PZmWn/qk0z/7ZPY5y7P/SAl3tD1Peu3CjPTSizTTn1qGK9+7Znz12IuVr1SEapGevPNiHlsmZlXQvou5XPHFiwtgxBKldXPSGxbYi9R8XDGDXQlHEcuKyMvFJXcTbeSvWk/yBC/XqVw4Fmqo1HVt7fSnqWgKHDLHp0167QFIrW2JXntJYdSafF5pIyUiWfkSHbu1vnPv5vj535qmksXVp5FSLl8v1yJrh6VP/hfzfzSvyly6MDCFUCxFBJPCP7Fv80QBPC7v1FiJsdKUeHhj8TZul3nv/1OmUp56WvNGLLVoG1nO37dJ7WpmYFnBkj3ZQjKNkiL5p4MCJg+N0WyPYmRMSmMKGTWZWnPmEwemyDVlcJImxRO5UltbsfIGIQTA1y1TCigqyYAXmgxVDpKU7wHscRn0ZJYRy7Ww5R1edFvMxC6Rmi5JPZuRfoBajZF+dsHCCv/Z7nNFE1hx6e2MX5sgqlz0yBg84c3kulNo+gKgRNy+HNHabuplU0f2EB1rEasyeTM185Tm6hx0/dvx0gbICWFs1Ocf+LSsoblepBUs8TUJAk1x6QzgESiC4OSP0lczWCIGEV/nGa9G0WoTDoDxNU0ujAp+/mIYEBIzZuilfUspzKtCDVyjzW+ETUZX1GhuPUnP77g39Lz6f/xX+WGU2ivAU45j1NevRLCteA6jIpKau0WFN1ATaRJdK2hNnSR0LEIbIvK+ROL9JC+21DMGNnte6n1n6F0+k0Q4FffPiVlTYfv+8EE+24zOXF0jkFWLkmOH/Uold5d/XMtkCEceNXl3BkPe5nV1luBUAWTxyfR4xrp7jSDF6Zp2tSEb/lk1mSpjdWwpi06bu3CmrZp392JZqrUJ+uk+zIUz08T+iG+4xNMBYT+tX2oilBRUAgJmKydp+LkycYWy+MIFNbk9lK0R5Yt4uQXSkjPxx2ebGQJK4S1t78m+DuN7v2dmFmDo3/VD4CZNbn5B7cz+uY4oRvSc3sXl54eAMBIGxz/vTfw6hFrrPvWToyUzuv//SBG2mD/T+9m/Mgkpcs37nsMpI8uokJpcTWDqcRRhIYnXZr0LlShUQ2mCKSPJ53I6Cgmaa2FWlCcXY06fhXZoEsvBYFAaax2wppF/s+/sqp2RpTjuW8pnesDoVCZHqCpYxt2rYBVvX5KeiLdgaLoVEtDV9/5BmLVRiV0Hcae+XqUGNW7AePBTzL58rdwpiYaCUXRAxFqRA2SQYCi6RFnOgyR84ooCVWL/IwNf9VMZb/oRwVF0wgDPzpezBw/r44DgKJErAlFzCY0zUoYNM6hxpOo8STO9CR+vRpVEfTmxUyEaLRRgVASBt4CmZUl7yUIrioL/eYbLr/5q6VZoxKGYNUjdlYiKXAdiWk2CnU1BCcdRy5gUylKJOIoFAh8sK4Y0BUFzFiUGDqTz+FfMebFYgJNj35XlvAkaFp0DiFAW0IpQdejcxTyAeOjLDq/2XCXSRklNEoZuUDn18sSotFObSEt2XEkoRcydaZAfbJOoiNJbaKGW3aw8nVad7Shp3TcijPrBnPLDn7NRQYaVqFOdl2Ojr2d1L91Ed/yad7aQv5EnsC+evBYIFAVgzC08EOXgemD7Oz84AL+/wxy8W6a433LxlbCSv3/uFXJlUh1JVl7bx8nvnwa324IlZoqXs3j+F+fIvACjv/NKeoFi45b2qiN1XBr7uwnqyd0fMsncIPZ/2qxt1bZ8UpIQlShUQkK+KGLJjRCGWAHVXzNBky80CFQPFJaMxU/j0BBF7HI1TbDZrtawqoQs7Rj6XhUnnnjLbU7lmxBNIxKKtNNGHjY9em5MVVRkaGPULSGqKkgDHykDFFUreHYEwSBixACz6nOc1eJBrssQhi4jW3RcZHL/Iqx9TpxXYH6mQzM0Iu0fwLXWVTCtXn33cS7+qhcOElm2x70VAYnP8bod75C6Nqo8QTtd31gtu64DAJKp99k+thrSM8l2beBtjsfonz+OOkN21FjcULPZfKVp6gNnANAaDpNO28nvXEHSiweVYocHWTytW8T1KuYLR00772HWFs3Rq6F9rsfpmXfe3AKY4w983VCx0JoOtlte8hs240WTxK6DuWzRykef322imDL/vdiNrVRu3yezNZdaMkM1thlxp97lNBdnurqupJKeXF8pqtH5U8/38zXHrH40MfivPCMTeDDex40ef5phz/9g4jWGQbw0Ifi7Nqr09qmUsgH/LffqXDyWGR1NA3e/8E4H/1knNY2BduSPPe0zSNfrM+62HbcrPMzP5ems0clPxlw+A13QT5KLCb4wR9N8vCHY0jgzEmP9g6VywNzq6nb7zb5kZ9Isn6jxuGDLr/xyyWK03M39W/+rwytbSoT4wG37DFQVHjzgMv//OMqhXxknHfu1vnRf5IimxO0tqn0rdW4dN7jc39W4xtfHZg9V220Sm00yj4uXy5THqpAKFFNlcyaLPWJGva0zfnHoncACZef6Z9NFp46U2D63NTCGixXgabos375Qv0SRWuEpkTvgn2EEJhqis70Nor28NUHnLeIbG+SXG+yoXQs8e2A/IUyZlon15NESkl9yqVwqfyOMZ0VTWHd/WsRmiDdnSLZnsCtehQvlSicmaZ7fyfF/hJ6Qqc2ERnXK9/9wrkpeu/oZu29fWhxjcAJo2f8VtumAIog9CVlP0/Zn3PtTHlzRIJhe06HzU8UmAqnCGyfaW+MojfWiLtcIySEV1GhFnFztuz0lQjr9kLXl1CIJVtJN63BiGVRtRjd6+5iauI0oe/S3reX0f5X6dt8P2Hgk2ley8ill6lXxuhcewdCCJKZbs4e+TJa49jydD/jg2+QSHewduv7qVcmMBM5Lp18nFi8iZaum1E1AzOW5fyxr+Dab33F+PZprQiIdfTilqYoHHiG0HNQjNisQZJ+QH34EqUzhwkdm+TaLeRu2k9t4NxsxUUtnSW5ZjOTr32H0HVo3nUnrbc/iDU+RGhb6OksTbfcTuHQi1ijA6jxJELVZldD7vQkky9/Cz2VpfuDP0jhwLPUBs4ig2C2HcnejeR23kbx6GvYE8PEOntp3nUXbnmK6oWTs7cT71qDV54m/9p3CH0PRdNXNCgAO3cb/NwvzpUtLeRDPv8XUbKRYUYj+yN/U+czP5Hk639f50tfqPP9/yjB5/8iGlQVFbZu1/jzP6oSBPCP/nGCf/ULaf7tP5+mWpFsv1nnR34yydceqXP4oMf6jSo//GNJ8pMhj/6DRTan8OM/ncLzJP/5PxbRdcGnP5NckI+yZ7/BJ38gzl//ZY2jhzx27dW5+z3mAqPyygsOJ455/MhPJOlds/gD0XTYvc/gi5+v8Ru/XGLtOpUf/2cpjr5p8K1v2CSSgh/+8SSTEwG/+xtVsjmFX/+dJp541OKJR6+Sn9AwDkIRhH7I2BuN7PUrv/15/16NQQFQlbkZnBtYjFZPk4l1LNgOkWFpSawlbbYzbb29LoVkS4z2LTm2f2gNQwcnmTxfojRaZ8eH1pBqi1G4WKFoVJnqr6yK7bY6zNXzmPlnaaCEogpy6yONOytvURooceh/HaV7XydNG3LU8xYylJSHKlx+cXDBs6mN1Tn91XO0bW8h8EOO/vUJvPpbT37MdRiYcZXRi9H7lMxqBH6INy++JwDNiArm1SsBHeviBIHk8slI8+vKXlQVg5UgCedcoYrA3NSH0HXsM/2zZYKzH7wLvbdjyeMr33kd+8TFee0TGGaaeKoN3UjMNXr2v1HJYVWLUSv3RzXsi5fR9CjTvl6ZxHPrOPUpHGBq8gyaZs4e73sWA2eepG/zA8TiTQ2GmIdVy2NV8zfEoMDbaVSIisdMH3kFr7xYijn0HEqnD0e+kGjVRnbHXlRzjhcuFIXpIy9TvxxxqUtGjM4HP46eyuLYFlJKhBaxurzyNE5+nPlvsAz82diJDAN8q4pXKS5oR2bbLtypScpnjxAGPl6tTGr9NtLrt1O9eGquMpuqMX301UXHr3j/Ys4dBCxgwPkevPGag2VJfuhHEzzzpI2iCH7wR+fEMhUF/uHv6hx8PTKSQsBv/n6OLdt0Dh1wefjDcQr5kMe+YmHbkpFhn1vvMLn7vhhPPGqxZp3K1u0av/zvShw+GK1uUhnBzbvmBssHHjIZHAj4+iMW9bpkfCzgznvNBfcRBFAuhVQryw9e/Zd8vv73dSbHQ4Yu+9z3YIyNmzUUBVIpQWuryvNP24wOh4yNhOQnAlJp5ZrzT3zLpzxwffUdVoZYMHhIQqbq/VTdm8iYnYuYPzEtQ0dqKyV79KqS628FI0cKjBwp0LE9x4nHBhg/VYyunzU49cQgI0cbq7Ebak8EhpogCF1URSdhNOP4FRSh4QZ1PM9i8KVhBl8aXnSkW/U4/82LC7ZVR2tURxcqEchQUjgzReHMFDcS8bRGKqMyegnaekx6tyZo6jA4d7CCUAQtXQYjFyy6NsRpX2Py4lcm8dwQVRfzJbEWwFSTLBekBwhlgBdEk1MlZpL9yL1Iy8XtHyH0I+MW274ec2Mv/uS8MVBR0HJppO1in5yX0S8DylOXmBg6hBnLggxnBUoFBrqRBAmB72DXpygVLuI51aiGj2dTLY/gjp9qXETM/g8ESPDcGiCRoQ+KQhgG+J5FtTiIY924b+ttNSqBYy9pUADURJrcTftIdK9DjcVRzDhmU9siHRAnP69OvGNHvkQ1arZfLjLx0rdo3nMP2e17qJw7Tun0m7jF/IrJTXMQmK2dGNkW4t1rG5sEQlFmXWyz17brqzIoAMcOu/zX/6e8ZL6VlJJaVdKQB6JSlqTSgIT57vyhobmB63K/j6oJWtujHTZv1bhlj8EXv9o6czuoquDY4Yg3n84oqJpgYp6C8dDAQvHJ3j6NyclIKBSimE1+cvVslOJ0OOvq8v2ILRdPRB9sPh9y+JDLp34oiedCa7tKU7PC808vXOnNxNhm3KrLQaga8jpqZy+HGWrpDKpugYnqOVJG26LfhBD0ZG5msPgmNe/GDozXAqfisf9Ht0YVNV8c49hXLhEGN8ayNCV6EQhy8R5K9hiq0LC9MulYOyAZr5y9plrt3y3E0yqpnEa23aBzfRy7HlAueGzck6ZjbYzhc3W6N8cJA4mmKxgxBTOhoiiCYNHqVpAwcstql0kpCaSP40dGU+ga5sZeSt94kdBaGAqwz15m/Hf/au7MikLzDz5M7KYNc/kPgO/WZz9+16ng2hXCMOD/be+/wyTLz/pu+HNy5eqqznlyTrszm4M2SKvVKkcECDAv4AcHcMbmtfFjbOC1sbGNjQk2iAdkQAihvNJKG7V5J+fU09M5VY6nTj7PH6ene3q6e6a7Z2YlXs/3uuaarqqT6tT5/e77d4fvt6PvXlzHQq/OIKthRFGitWsPshIhM34E17UJRVN0hA4gSRojF79HLNlNqm0LgiBimTWMRmHOcFhmDd+1UUMJIvE2wtGgmXV04EVs8+aJL2+rUeE6DTXpux8m1r+F3NvPY2SmUBJJOt/3qcWHWJAMv4ZSwXOpnD9OffQSsf4tJLbuI9zVz9T3/mrFBsCzLWpD58kdfmXB4T3TWDCxraU5yJvtlVluflyJ3QuF5h/qUDiIr1+Juum6z/EjFr/9mwt1KOp1D8tkrr/l6p4rLbRwkBhGwDp95dkWxSAxb60yZeC6/hI/d3Au14GXnzd4+DGNuw6olIoev/XrFY4dvqpoQ5SIdPQjSBL69GhQbiZKeJaJpIVQ4inMwgwIAlqiGaOUQRAlQi2dmPmZQIXRsYNkpu+t0KkIrlC8JswFMFk9R3diN2GladFqRRY1+pru5lz2hdXcoluCt/5n4InGOyI8/X/v59Irk9Rzt6bCzLCryKJKrn4ZyzUIywkEQUS3iyvSG/lBola0ae3RaO8LMX25gSQJuK5HOW9TnLbIT5iUsjbn36kQiog0ag6epyErArIiBP1OVyEkxwnJ8etKHRt2ZV6sSxQQVSUo1LjKQFmjM8Ek4MwPDh8Xt1JHjEUWLISK2fl8z8zoobm/S1e9H0v2YNsN6uVxwtFAJjgUSVGvzGA2isSS3QiiRH76DPnphbLGenUGgMz4EURRJpbspl6exLZqhGPtgWTDLcDtNSrXQbijl8bMGNXZvIXa3IYcTdxgr6Xh6jXK545iV4p0PvVppEhshUbFRx8fItzZi6vXcdYon3k7cdd+ZS4xf/c9KrYFI8OBl374HYsPfjSMXvcW5ECuIJ/zqFU8du9TGR0OluN77lIX0MKfP2Pz4KMa7R0S01MuySaRdetlzpy+tSR/B+5TyU57/NffrGAskUYRRIlQuh1BkmlkJ4n3bUEQJepTQ4iySqxrA3algA9o6TbMSh5JC5Po30a+WkJrasOq5FHjTegzYytfyQjCkhNmwy4xVTvPhtT9S+7WFtvEWPkYNWt55tnbieq0jud6SMraOOrkqErH0zsJtyfwHJfikVEKR0cXOFaGHYyHxiq43Zof3IAxVaY+9O7el3LW5vhL81GRC4fm8wNDp+a978sn5v8eOrm8V57Q2tDk+LKfg0/ZmGenxvPxDBMpGQs6vmcT8OVvvbbYwZHEgLJlDbmwenU6iKYAZqNCo5ZBlFRC0TT4kK2fwKgHK+jlZMgBPM+hkLmAFk6Cz6xR+huQU7ke9PFBEpv3kL7rYRAE4pt2XbMquTEifZtJ77kfY2Ycz7EId6/HrpVxGytnlC2dOkiks4/O930SfXQQRJFwezf5I6/RmBq58QGug207FH7278bmEql63ee7z67cq6zXfT7yqQipZgnP83nqmTAvPGcwfDmYMJ/9ms6Dj6j8q19L8vYbAZvzlq0K3/5GgzdfNRkadHj1JZOf/wcx+volRElg1x5lQYTxW19t8NQzIf75v05w7LDF5u0Kre0SzBoySYbePpnObokNm2Ta2iXueUBjYtRh+LI7Fza7HmQZJsZdPvM5mb96thXPg3rV54XvNviLP9HnyqSdRg2jMIPv2IiyilGYQVI0bL2GrVdxbQtRVhBkBVFWcI06TkPHNXSM/CRNW+/GrpVWHRoThaWHwVjpGN2J3YTkhZIAgiCgyTG6E7u5mHv1XQ0Jde1tpjKt07w+TqNkoRfXRrQpqDKSpjD9/Dk816P9sS1UB7P4joeoSPiuh10Lji1HNQRZxLddnJqJoEjIURV8cHQL33aRY8E2sXXNeLrF32ROZ0lQaI6sm82pLA0fj1x9vrTcMy2Mc8PE3nM3xrkhzOEpcF3c4jUTtSQS2tJHeNdGjPPDKzYs0YTEve9NMnCijmVOYxk+nuuT7hCxDAMtPI1je/iGT2u3jO/D+h0RBo7XKeWWHg+WUcYybr0jfVNGxa1XqV46vaicGMDMz1C9fG6JvQIUjr6BZ1mE2rpwDZ3M698h1NaF0wi8B6depTpwGv+qpgjX0KkOnsU1gnJFq5jFyE+jpgM6fDMzSWXg5KI8jmfb1C6fw6kuvoF2pcDkd79EYssetJZOfNdBnxzGKs97WmZu+rq8PdfCc+HEURtFEdi0Zf4WVyo+mmai6z6vvGBSrXpEUiGOnZWQYiGIipy8JOO4AqWix9e/3ODoIZOnngnT3iHxpT/T+esv1udkEUpFn3/9S2Xe90yIbTsUPA8unre5fCkgtXRs+KPfrzE9FZT6ZmZc/v2vlvnEZyIU8sFBJidc/tU/LfPMR8Js2Czz+isGr75ooGkCvhf0yLz36RCbtwbfY3rS5b3vD1Eue3zxT3UuX3I4fcImdBWFjeP4nDxmYxoBncu+Ayqf+tEIv//bVYYGnYA3qlPi7/2jOCNDLi9+18D3PTzbItTcgWvUMUtZHL2C73l4joXvuUhaODAqooQcimJaBrZeRYklscp5RFnByF7lPa4Qy1Gcm06difJJNqTvX9S3IgoSzZH1xLRzVM2ZVZ9zpZg4nscoz4cJm9fH2f6BXoyKxcE/vjDXK7IWSGGF2MZWBFnEqZm4DZvWhzcR7kqipqOMfvEwgijQ++m7scsN9LEime8P0PrwRiI9TYiKTPVihuLxMdb/1P1YRZ34lnYq5+fvhySHiMY7cOxAXdBzA9VMLdSE4xjYVp1QuAnb0vE8G1FUsMzb15i8EiRDnbTFNl839FUz8wvodHzDovrSIZp/5qO0/cJnqbx0MEjY1xv4jhesTiJhtP4OYk/cE+jVfPetRSzJy0EQwLF90h0qu+6PzRmLux9PkB23UFSBwVM6qTaFrXdHKeUCtnXLCFM/WFtQBXe7sSbur2gkqGhajsfqDlaOdQ+0k+iK0ihZ1PMGWkyhMFShPPE32debhyDAj/xEhPse1PiVf1aaqyCLxgT+9MvN/Okf1vn6l2+e9lxJpFHjKeoTg4s+ux73l+e7nJn5LhOVU0seNxnqYk/HB4mq6cX7eg4D+dcZLh5a0Wrlh4n7S0lF6PvU3RjTFVzLQYmHmHn5Ak27upEiCi0PbGDoT97GKtZpf2IrCCKFQ8NYpQYbf+ZBzFwNQZZwGxbl05O0PLCBS3/wGut/6gFKJ8cpHhsDoLVrH65joKhRwpFWbKtGtTxBS/tObFunkDlHNNGJJKl4rk25OIyhBw7dWrm/bgaqFGZv50dojqxbdhvf9zmffYmR0hEWxAtlifCODSQ/+DCh7evw6gZuTcd3XAQ5YDGWYmGM88OUv/MmjVOXFuRargdZEYjERVRNJNkik5+2EYDmLpWejSHqZYfTb9do6VJp6VJwHZ/shEW96lErOddLb68Kt4X7qykp8rd+LEal6vHG2yYXLt0e8aRlIYCkyfiutyq+IEEUkDQJz/XxrB8empRGyaJ5UxKzZhNuUgklVDLnfniu72bh+zA57tLTJ/OpH40wcMEhlRZ57MkQjQYcP3JrmgidegW7VlrTvvOrkKs902DwVM0MufowEaVp8WpFlOlM7GCqehbDufkGvncbTt2kdGYSt2HT+fQO4pvbSe7uYuLrJ0ju6gJRwC4bZF4ZQGkK0/9j93Lhv76IXTUoHh/DKuq4uoUUVpGjGlJIQQorCyo4I9E2Ri+9QLptB5KkUNELxJPdgI9ey5BIraOh51C1OJqWwNDf/Yq6K9CkKNva3ks63LfsNr7vB1IJ9cssqkN2XBqnLmEOT6L1daBtXYfSlkLQVDzDxMmWMM4PYY3N4NX0Fa9SIFilVAou4JKbmp9zi1mHiUED1/GxDJ+xAYPJoSBydAsLJFeFVRuVkCYwNe3S1SEhLy6amYMa14h0xlCiGuDjNBz0mRpmqbHot1gNYl0J7v1Xj1M4n+HE/3gnWFquAE1bWrj3Xz5G9sQ0R//Ta2u/gFlI4RhKNIHTqOE26sixJG6jvpD+ZQWYOVdk5txsuE6Apt4Y9groRf4m4c1XTRy7wlMfDHHgfg3T8Ll43uE//XqF6albY0BvrsRYQFRCaIlmJC2C77mYlRyOXsHzHaZr52iLbSQkJxaFROJqK+2xrYyUbo6i492Gb7v4rk/PR/bg2R7F42OUToyR2NpG22NbsHI17HIDrTVGz8f24vuQfW0AV7eY/t452t6zBVGVKBwdpXhsjMrFGfp//F4c3cQuz68czEaBWLIHSVLwPAe9Ok0y1Q+CSL0ySbzvHlzXRBAkLKtKONpCo/7uSjDLokYq3MP61H2kwt1crzfF9W0mKifR7dLSG/g+XqVO4/QgjdOLV823Cq3tIhu3KPg+1Kse46Mu1iyh7PWGQmu7iG35CxgxbjVWbVSmMy6nzloMDgsMjyxx9QKktray8WPbab27i3BzFHwfq2oy9dYoZz5/BCO3dn4kQRaRQhKSKi/Xs3Sdnbne87Lyw4gSiY07cc0G0d5NlAdOkN77EOULxzAyN9Fp7UNp9ObrxH/Y4Djw5msmb752e9Qbbw185FAULdmKazWCxlk9iO2XjCkKjTG64jsX7SUIAr3JfUxWztzSMMzthlMzGf3LxYZw6E/eXvTepd9f6ITVh/MMDb+14L3Jby0dPszNnCWZWkdDz9PQ89hWjez0KRQ1im3XyU6eRA0lqFcm8X0/aPB7VyAQUZIkQ100R9bRGt2AJl//3L7vkddHmK5d+IH26wgCPPhoiE9/LsqpYxbRmEC14vOFP6wxPbm8kyaK8NQHw4yPOnz/hds3FldtVFrSIpGwwJ4dKobhc/rcwvBXrDvB3f/kIWK9SS5/9RzTB8cQJJHmXW1E2mLYtZsLd9QnK7zzb1/GrlsrZqK95RBFpHCM4pmDtN3/VEAPIwgosQRGTlxARnk1Nq1/mmSi9zoH9oMuV9fEsmrojTyV6hiV6sSKmJ81Lcmm9U8R0prW9LVMs8Lp819iKVPd1rKL3u4Hlr5q38N1LWyngd7IUamOU66M4brXf3AVJcr6vseIx5bPAawGI+Ovk8svXxxyPXi2SX1mCDkcRxClBRKrvu8yWjxKR3wb0hJDJqI20Rnfxmj52Jqv/f9f4dg6+czZBe9dzZpbr05Rr84XV9jWapwqAUEQEFi+rFoQpICiXgyhyRHCSpKo0kwi1E5YSaLJMRQxdN2kPARhL8Opcin/Gqbzw+H4HTtk8Ye/UyGeEPnpvxPnmY+F+fM/rvPAIxpPfTBMLC5y9KDJV75Yp6H7/Mtfa2L/bKTgcz/jcegtk//536r0b5D58Z+O0tUjk8+6/PHv1+YqTNeCVRsVRRHYtkWd04G/erUgiAI7/j/7ifc1ceJ33mbkuYtzeY/s0cmA3XfWEEghmVhPgsYsI+3ViHTEEBUJfaY2l//QUmFC6fDKVhoChJsjKPFZrYyahbCc/roAWjKEmggFWhCWi1kygnLK5ZZBvo9nm6jJFnzfJ9a7CaucJ9wxKwNgLV02HI220ZRct+xlL0yC+bOvfRqNAmOTb5PJnca2l0/gS6JCPNZNNNK67DbXg64v31ugqgmSif4lB9/i6/YwzQqTM0eYmjmKuUw1jyhIRCPt170nK4Xv+2jqzYlPea5LozCJ77o415SlV8wZMtWLSybZBUS6krvI1C/dstxKU6uMJIs4tkeqTWFi0MBeRi/oBwJJRNQCTRTPsgFh3pkSA1oQxICdQlAkvMasM7mMw7UWJLR23rP+51e49WxvvDBPXXIjQ3IFvu/TsMucznwHKV6ku1vDdXwKGXsu5LRWROIinguGvvr74nk+hgHlkstbrxp8+JMRvvSFOtOTLn/6v2pUKx4//w8TPPa+MF//K53//Btl/s4/TnDiiMXL3zVwZ5kYGrrHS981GBq0eeqZMH//nyX4p39n7bmtVRuVqRmXd46Y1OseemMhCVuoNUrXQ/3kTk4z9dbogkS67/kLElOpLS088fsf5dBvfJ+hb52fP4gosPfv3U+0O8HBf/cylaEg39BxXw8bP7YDLRUm0h5j+LmLHPtPry9erQjQ+WAfW390L4n1KVzToT5ZJXdiasmHqOO+XjZ8bDupLS2IioRnueROTjPwpVMUzi0d2/Vdh/rYJeIbd+LoNZxqierQWSLdG1bceR9494uLHOa4fgRxLt8ZjbaxZeMHSSb6GLj87esalncDvu/juld1wwsC4iwddzBYRcLhNBv630tTch2DQ9+jWlud5OwPAkokTqJ3GwCV0fMYxXkP2sdjrHKCluh6FGmhnLUgCESVZlqjGxkrn+BWEHL1bAqz5e4YoiRw4UiNddtFBo7/8FQEauu6kFpSONliQN+nqXhGQK/jNww820FOJUCWkFvTODN5BFXBGpm8ZfIAgiAgCddJ7N4C+L5HxcwwkHuNqj3Ohz/RTFuvSm7KRq96PP+X+bWX6wrQ3qNi6B5TIzcXwalVfSJREd+HYsFj5x6FZJOCrEC6JVjJmWaQb7GthYKDlZKHIARs5OGISN866bqNkzfCqo1Kuknkofs18gWPE6dMSle1fqS3tSDKIuXLBYz8rdWVmHpjlOL5LPF1Kfb+3fuW3S61pYVdP3cPoiJy4r+/RSOnE+9NsvETO4h0LGxia9rSwt5feIBGps7J330Hs2QQ702y+dO7uOsfP8Rr//Q5rPLSqw4zP41dK6Ol22lMB02S+sTlJbddCoZR4sKlbyyafgRBDMj8wi2kUhtoSvQjiBKiKNHRtodGI8/Q6Es3PL7ve2Tz55meuXFIRpYFwlGBatlkZROiz4VL38CaNW6BUVHQ1DjJRB8t6a1IUqDtnWrawLrexxi4/CyGubBPyHZ0Lo+8yPjk4jg+gKYl6O95hFCoCYCZ7CkyuTPLGu5qffU9KlfDNRs4ehUpFMW1F//uVSNLTh+mI7ZtCeoWldboRjK1S5juzYdHpkdMogkZQYBEWiYz9sOVj5JbUri6gZSIISWjCGogD+w7Lo3jF5Db0oiREIIc5D6lZBypKYZXrWP9DdCcubL6zunDXMq9RsXMICk+es3jrefKXDim89l/0EHPJg0tJLL3oTh6zeWd75VZvyNMW7eKIEIkJvHGt0v4Ptz7viShsMjx16qcPVxn+/4oex6IceilYCXfvzXE/scSKKpArezw7J+unJUgGhNoNHwSSZGf+XsxxkdchgYd9Jq/gEFjKXz2b8Xo6pZ46zUTTXMDZ3bVCet5rNqoqKpAqeRx6IhJubJwlRBpj+M5Ho2sPrf87X9qE3v+7n2AgGe7nP2TYwx9Y/Vxb6tqBkJNPrjm0pOKIAp0PbKOeF8T7/ybF5l4bTigLjg+hSCL7PuFBxZsu/Wze5BUiRP//U3KQ8Vg26OTSJrEjp/eT+8TGxj86tklzwXgmY3AoKzBpDuuSb44sOzngiAyNvkWbS072bLpw8iSCgh0tO1lauYohlm67vF930dv5MiXznLfI2FOHDYwGz6qFujYNxo+qiogyRASRdb1q+SPmyiqcEPPy/d9iuUhzGuMBAhMTR8lFutk84anScR7gybB9BbyxYtMTi+s6/c8h3JledaCSLiF7s57517repZs7ux8fumqB1+QBPybJFb0XJvyyBkESca1Fifdba/BTPUi6XA/mhxZ8NkVWvxkqJNMffnfdaVXWCk4jF5sIIpQmLEw6rc4fyiKiOEQnnGVpocsIUbCeJXAKKr93cQePUDhC19ftHv9yLmARVcU0bb0I4Q0zAvD+JaFbzm4tcBwXDG+vu8H2jCz5xJjEdKf+yhKVxuV775G/Y2jt/b7rREBBb6H4VQZKx1jrHwSx7viYMw7EmbDIzthsnl3hI5+je/8WY6ejSH2PRInmZbJTds0dyhMjVjsvj/GK18r8ua3S8SSEu/7TDNnD9e5fEZn3fYQqdZgGk63y6RaZb70OzP85C910tqlkJ1cvmVDEALRu1iryL0Papw7ZRGLC/T0yvzx79Uo5j0efM8847jvg+sEwoCiOM/19+TTYX7nP5Y5/LbFe94bWvZ8K8WqjUq15qEqAs88FeaV1w0uD88ndERFnPNWrnwLfbrG1BujRDpitOzpQA7dPmYYJaYS70viOS4zh8bnRrDveFQGF66ewm1REhtT+J5PYkOaxIb55rZwS1AFktq2gtzEbdKyCMJjJtOZ48RjnfR2Pxg8REqYeKzrhkblCuJJkceeieK6cPmixfY9Gv2bFA691mD3/hCm6XPmqEE0JnLXfSHGhh0Gz691Ke7jehblyghDIy+zbcvHCGlJJEmhObWZmeypJZP3WkggGhdJNUvkMg7lwsom0HBzmGh7DKtmoTVpVMcqGIW1kysqkSSRtn7CqXbKo2dp5BZX8uX1EcrGJK3RjYtWK5Ko0J/aT14fxvWXmwxW9rz0bgmxfmeUmRGDmVHzlj9mcjpJ06efpvLc61hDQaNieOdmEu9/hJnf+iNwPbyGgTU+vfQBZpkufNfDOLNE6ezsHLAw2zYPr6aT+8Mv0fTRJ4NVzg8Y/uyzW7eK5OtDTFTPUL8Or1s4JtLRp3H2cJ1my6NWcqkWHXo2akiyQClro6gClaJDS1eYvQ/F6dsSwrZ8YskgvGSZ/oI8mevAzJhFteRSr3pokesvMbZsl/nsT0Xp6ZWRFfjTP9QxDZ9azeN9z4QxGj7rN8rMzJbtW5bPxfM29z0cEGmODDkcP2xx+rjFY+8L09kts2O3ctMVsquXE/bBtHz0JejR7ZoVaAVEZh+S2VVC9vgUXQ/107x7sfb3rYSkycgRFats4l7TGOkYDs5VzHhaMoSsyYRbIuz+2/csOpZZMnD0d7mxcwn4vkeheImergeCMJMgo6qxG+84i3LBY2rM4fCbDVraJFzHZ2bSobVD5sIZk/YuGVkVaGmTaOmQefX5W9N8VqqMoDfyaGrQ2xGNtiOK8pJGpSktsufeMG2dMkfeaFAurCzUE0qHadvbjpbUmDk2TfO2FibeXHtJt2sZ+K5DdXJgji7oWtheg6nqOZoj/UvG81PhblLhnuUlh2+gFHgFZsPDdXxCUemWlMFfC6dcxSmUUPs6sIbHwffRtm2gcfoiIBC5ZzfalnU4mYXPg9TcRPSe3UjNTbjFCvqR0ziZAomnH6H+zkncQgkhrBF7+AD1t44haAqRe/ciN8Xxajq1N47iFn54iFtdz6Fu5SkZk5SMSSrG9CxJ6NJWPBIXefhDTWw/EGV0wODEG1VSrTJP/3gzqiYycFKnf0towd6CKBBLSqghgXrFpVoKQkz3PJlg130xamWH3JS94jyG78OJIxaqGhAgD5w3OHfKIpvxkCT4/O/W2LxNoVb1+I//tjwX/vI9ePVFg0rZI5WW5hL1n/+9Kvc8oCEI8Of/T523XzdZ4WO6JFZtVFqbJUzLp1j0SMQXWtLqWBlRFom0x28qHCFqa6NgDooBvKUrvQQWdPpe0TvPn81w6NdfWfJ4K9E4fzfguCa+7yIIciD+tgxX1XKYHLN5/8dizEw6bNmtYege05ZDukVi3SaFYs5latxheNDmwScivP78zce8XddcEB5Tleiy153PuMxMOpw9ZtCor/yZqYyWiXXEKFy0iXXGyJxYG73JFXi2QXXyYqDZfZ3RPVO7yLrUAZKhzkWfCUj0pw5Q0EfxuDZM6y/x3jLnGDGpFhy0iHTL9FIWwHawLo8T2r0Z3jiKGI2gdLZSeu0wuC7GhSEEVSF6/z6qz78BgKAqxB7ej++61N8+gZQIetAQBML7d2KcvRQYFVUhvG8bjWPn8AXwylX0gRFCe7aS/MiTFP6fr9ySr+D7Hq6/sjHq+x6OZ2G5dQynRt0qULOyVM0ctm/guCaOe1VOcYkZ3nXghb8qEomJuA6U8g6OBS9/pUg0KeG5UC06XDyuY1s+Ayd0HAcundRxHZ9IQsJzfFw3KII7d6TO0Dkj6NIvukyPWgyeDsKuX/vDLLXK8s/K8GWH4WFvvppOEEAQcF2fs6fsOWbza1Ep+7z64kKnLTvj8e2vzYd7hwdvbt5btVEZHXfYtV1hXZ/Mi68ujDsXzmUwCjrp7a3E+5uoXF5aoAvAtV1830eOKgiiMCcBK6kS8d6mBauKlcLRLYxCg/ZUmFA6QiMz721qiRBqYj6+qGfqmGWDcHME13Qwi7dGk+J2QJZDCLP8R57nYlqrK1v93tfqSHIwKE4cNIIQuh/orLz+vL5i9cXVwvOc4ESBJVxyG2G2EvX8CZNwVFgdtYQf/I4+MPLSMHZ97StLUdYQFRU1nkaJJjHLWYzC0ol/z3cYKh5kT8eHluSlSoY6aI1tZKZ2cdFnK+k3iqck1JBEqk1h/c4Ir34tj3ml5FQSEaR54+xba58AzEsjxB+/HzEWQe1qwy1VcUpB0tir1HAKZRZ57IKAqKr4to1x5hK+ZXO9TLBbLGMNTyBEQjhTWcJ7t675eq9F1cxyfOprWO7yTaf+bIl7oNK5tHFObz6Aa5s0CpNBoUwoMlsqbeK5Qbm077nIWgTLdzErIKkaUtRGS4epZ0ZpXPXs2bNtEObsZZmzPppeW+j+VwruLPXKPMwGiNEIRlMnglSDTHaOzNa3bQRVBc/D9zzC27Zgz2RwCkVCG9fj1urY07eP3HSlWLUYQyQckEmWKx7Ra2J+bsNh8CtnSG5Is+Uzu4n3JYFg+XelZ+QKrLKBVTFpu6sTLRWUaIqqRO+TG4l1XU/HYHk4DYfC+Syu5bLp49uRw0F4Qk2G6Hp0HVpyPglllQ0mXx8hlA6z4aM7UOLzkrJyWKF5VzvSbcz/rBSCINKc2hIkOX0fy6pSqa4+xHNlsnavGluuy20zKAICqhLjSuzGsuqBeNY1iMREOntl7nkkzINPROnbsPL4erg5Qnp7y626YERZRZQVfMe6oQxDQR9dqKdx5TCCgCKGaIttRhYXJz1dbyVGQCAcFYMqoJIz53ABqC0J4ndvJHHvZpIP7wi0O9YIt1jBmpwhvHMzSlc79vg0vrl8Ps23bOpvHsN3HBJPPUziA48ipZNLXv8VBcPovXuIv+9BtE39KN1twQS5wv6QG8HHw3ZNHG/5f65nzerIL7/a8z0H1zKQQzGi7f2EU52IskqktTdgz+jZQijVRri1l2j7eqLt/cQ6NhBtX4eo3Hxi+1rIzWnk1hZ810NpbyOybw+hzZuQmppQWloI79oRGBbLDjxD38ezrFUxqd9OrHrWTMRFQiGBfMGjOS0yOr5wlTj8nQES69P0PrmR9M42zKKBIEC4LcgDXAkrmCWD8Zcv0/fUZh74tfdSGy+jNYWJdsXJHJtasKrQmkK07usk1BIh2pVAS4VJbW1hy4/uwa6a1CYqZI9N4TkeE98fpvOBPjZ8bAdNW1owSwbRzhiiJFGfXNiEd/kb50msT7H507toP9BNbaKCElEINUdQkyFe/UffpmH84LpnBUTaWnfT3raHYFD4TEwdXKLq6ocP4XB6rhTY931q9anZlctCmIZHrSowNGChqgK16sqDua7lIodkmrc2Y+QbN7VS8WwTx/dxrQaebdwwtm27DSYrZ0loHUjitZLDIs3hfhJaG4XG6Nz7PuCvIFxTLTrUyw6eH7DTOldV4zkVHd91cesGoXVtCJI4V1G1FjSOnSX+3oewJ2donLo4Xwm2DJxMnvJ3XkVuSZF46mHCu7dSe+0w2C7IwaQmJWIIsgSSQOzRe6g89yqNkxcIbV1PeM+2NV/r7UJl/GIgYS0ImOUs4IMPRnEG19Qpj57DcyzE4gy+61zV7C3he+6SvHPCbHXVdesyZhfw1/pa9tQMvu2g9ffi1XXkVBKzWkXt7kIMqWgbN6AfP3lNAEBYVe7tnh/bSMfOJt76/EVyg7eWDHXVRqXe8EgmRNpapbkbdvTEvHdjFhsc+y9vMP3OGH1PbiTcGsXRbXInp5l+e4yZwxMAOLrN6f91mOpoma6H+kj0p6gMFTnzR4eJ9STpfnTdXDd9uDXKume2Em6NgCBgFHREUaT3iY1AkBfJn83gOR5mscHBf/cymz+9i7b93UQ748wcGmfsxcts/vQuGldVgNlVk2O/9TqT9/XS+/gGkhtSuIZLbbzM9F+fwSzcvnp6SVSIRtsXRxdEEUUKEY2205zeSiq5DlGUsaw6E1MHmZg6tPQBr4UQ5DGikfYVXpGP3sivKDxzI8hymK6OA0TCLQiCgOOaZHPnFjRMXoFjQ63iYpkC4Yi4KESwLAQwCg0GvnoBOaJg6zfPduz7Hk39O5G1CKXhU1jV5YsWfHzy+jBlY4pUuGdRJZgmx+hK7KBsTC2oBFtJDkAQYPu9cWolh5ZujTNvV+ZKir2GRePiJPH9G7EmC/OVlmuEcWGI1Gc/iFup4kzNNvsqMnJTArklhRDSULracKt1fMsmtGtzsJpxAzp3rxHIblsT00T270ZUVaIP3gW2A56PW6qg9HTiNUwi9981t7ISo2GkVBIxHkXyfOTOVtx8KfC+32W45vw4X6oh2a6XZrdb2fHkkMTPffkJzn1vgtd+7xyuvbRlefwXd7L+gTa+9s8PURiZd16lpiRKZ3tAUmA0cIpl3EoVpbkZRAm3WkNQVaSmJqR0CqdcRU41Icai2Jkc/gq0wFN9Ubp2ptBit77ybtVGxXXhhVcadHVIjE64TM8sfqgd3WbshUHGXrgOS6cg4BkKo89OMvCl+Ya2SPcGyLRz8csN1OZdyKXTlAbyvP5Lz634Gu2axdk/PsrZP15Y+370t15ffK0Nh4lXhph4ZelqnauhxJsQFQ2zmLnpUuJIpIX79//idbfxPIdafYZafZps/hyF4sCKJ31RkOjq2E9Xx/4Vbe84JoeP/x51fS0MsUFHvapEiIRbaW3ZSUfbnllmWpds7izF8hJU4bNobpXYcVeIvg0Kh15vcO74jUevGldRoyqJ/iSRtiiZ49NURm9W3EnArORxQ40V/b66XSJbHyQZ6lhUCSYIAu2xLYyUji7QR1lJ+EsQoalVIdWuYOoezrWTkiRSfvsC3i0wpDguledew7ds3ErgsUrRCOG7tiM3N2ENTxB96G6MM5cwL40gIKBt3wiCQOPkBRonL4DvU3nuVaL37SO0czP1Q6eQomG8hkHpq88TffAuQjs3U3vtEM50sBKQO1sJ796KbzuIYY3Y/fuovXYYJ7d8HvZ2ofeJDahxlczRSapji58hKSTTfqCLaPvSVZe5UzMUB/Lzj7fnU8+bxNvCKGEZ117aUCa7A436a9U7nWwOJ1+YS8KbQ0Evlz25MNxaP3Rk7m/9xNKEnj8IrNqoNKdEHnkgREebxPffMMgUBHxZAs9HDGsgy3iVGnJnC26hMh+jlSXklhT2xAy4HoIgokaaEEQJc9YTAEhs2oPvWDRmgtr569KeiCJqUyuuXp1Tg7wVkEIRlEQKIzOx4H3fdfAE8VawcNxQ7EaYbWv1fS8wLLXJJcNHPwgIgsi2TR/F9YLBIhBoi8hyCE1LzhFa+r5PvjjA8OgrWNcpLqiUPBp1n3e+36C4jPTptXBNF0/z0HM6dt3Gqt4aXRa7XkbPjKyQbsdnunaenuTeWb2VhasVRQrTl7yLM5nvzm3vLdu/Mg/PhXe+W8T3gma1BRBAaUmgdaQwp4o0Lk3dtINzbeOhW6pQ/d4bS26rHzmNfuT0ovfdXInKs68set+r6ZS/+sLca+vS6Nz/V/7+QUJLhdn39+8l3Brl5B8c4sIXTy+S0xBlkfS2liAEnw4T607gWi7V0RJ21aKR0yldKsyNad+H0oROJK2hhGWMytK/eawlhFm1MatLfH4LOdLebazaqExOuVwednj9bRO94RG+Z89cYM8tB2pwSs8eBFHELVXxr9wczwuqRHywx6bwfQ/f94gmO2mUphZ4cPrUCNXLZ+ZeC5KMIEp4thn8LUl4toUaT9G0dR+14QuYxSyuUUcQpdntRZAkPDPoPRBVDUEKvEnPauDPZqhFNaCSwA/kigVRJNqzES3djlOr4Nkmnm0hSIHus9eoc3XZoaiGgtiq68xpqUhaOOggliQ8y8RfIunreQ51fXEJrCCISJKGLKmIokI83k0s2k5P1/1MTL3DxNShG7L/wiw/l2fhOitbs7uuteIeCkEQaGleuorH94NqG8uqMjlzlLGJN3Gc61PCm4bP2JCNbXuU8itbibmmi1EyMEpGwP92Cwy9Ek2ihGLYennJktKl0LDLTFbOsKn5oSU/b4ttZrR0lKoVrACvGOIboWdTiEc/0YIWEvnf/36MxpWwoA92phwYsJswJqIsBvftKjsozJbdy5qE3XDm3pvLDVzZ9oeI1/Jm0X5PF0pMxSg2aNvXydCzA5jFhc+rXbM494WTXPjiaZo2N3Pvv3iE6kSFd/7tK3i2h2suLKbwPJ/iWJ2WjXHUSJBnkkMSWkzGqjvYDRc1IqFFZWYulud+xmRnmLs+vZ5197WixRSKo3VOfmOUwdensRvz4yLcpPJjf/AQb/7xRSZPFXnwp7fQvTeNIAlMnizwyu+co54LqlllVWTjI+3c9an1JLsilMbrHPvr4VtWLLEUVm1UrhQYNKdFKiMu9mQwWMR4FCeTR2puwrwwjBBSERQliJEKAoIo4BkWvmnO9VrIagREmWuf0khnwHcFPtWhc4Q7+khs2En+5JskNu7CdxzKl07QtOMe4ht2oCSasWtFMm88R7hzHU3b78bRa8iROMUz72BkJ0luu5tQSzeiJNOYGaVw8k20lk6a9zx4ZTSReee7yJEEyW37kaNxpEiM+uhFqpfPEm7vofnuxzByk+QOvYTvOkQ615HYvBdBlvFti9L5I3iWRfdTP0J9fBAllsSulcm+/b1Fnq/eyHHo2O8ue58DHq1+2tv20pzaTDiUYtP6pwmH0gwOP3/Didr3PcYn32Fw6LvX3W4t8H3/qnLhAIIQ8JMBNIw8Z85/mWptYpkjLESqReKJD0VxHTj8hs7AmZWtOmKdcRL9CfLn85glY1VKoEvBadRI9GxF0kLo+ck5PZUbYbx8gp7kHsJKYtFnihSip2kvF7Iv4/nudTrtr7kWy+fkq2U61oUWJXJ9zw/GkCKtiaNJkAQ69rZQuFwm2hJGkAIDI2sSjuHQvquFzNk8nhv0cqkxBc/xUGMqeq5BPatj1384Vs03A1EW6by3B6dhM/i182z+zE4ibdFFRgXANR1cMzAwvufjOW5AG7VED5Hv+RTHakRnVyqCKLDvE+t48p/s4q0/usirv3uOcJOGqIiUxoMIS8vGBB/+tf1oUZmxY/lAEXZ9jPf90m46tiV57Q/O41rBgyAIkOyKsP6+NnZ/sA9Ld7j8VoZwUkWJyFj6rEMgCux4ppfH/8EOiqN1Lr02jRqWeejntt5Wx2D1zY8tEnftUQmHAkt37OTCiePaDtzlIMoKkaaOYEWxwF0CUQshhWcFcwSBxtQIaiJN673vxdVrZA+/jGfqFE68gahqFE+9jVmYr8+WwzFmXv82rtkAfARRxKlXsZQskhYhvnEXhZNvkt79ALXxS1QunkSQgtWGU6tQuXQSJZ4id+glrtx9fXIYKXwILdUWXL+iEt+wk9rweWqjF0lu3UesdzO1kQv4nkvhxBt4jk3P0z+KHIlh11ZXsWVaVTK50xTLQ2zof4LuznsRBJGOtr1U61NMrjRhf1vgMzj8PWz7SshRIJnopatj/2xuJUYs0jbLTHzjp9ds+ExPODS3SitP1AONQoP0lmaS/UkKpou1StXNa+E5FpWxgJfu6uTtjWC5OhOVU2xsfvAKwfocREEiHe4nrrZSNqdXWFIMU8Mm5YJDdtLCMq+6JwJoPc1Ed/Yhx8M0BqdX3asiCBBpCaNEFWLtUTzHw6pbaHGVyaMZbMOlY18rjuEyfTJLy9Y00iwFkygJVCd/eNiSbwbR7jiJ9SnyZ7Jkjk+x8WPbabu7k+KF3M0d2IfqjIEWVVCjMmpEpmN7kupMg74DQQl8OKkiKSLlKR1ZE7nvJzaSaA/xrf/7KENvZfAcn3BS5aG/vZW9n1jH8MEsw+9cle8UBDY+1M4rv3OWc8+N41geggjR5hD2rFFRIzL3fm4jpTGd537jBJkLZQRRoG9/M5/8z/dRz98ektJVG5VsziWT82hrEcnmFocqRElAUq+qnfdnPVvXx7sqPuzaJrnLh4Mu0KtDND7Uhi8sCH8BOI0qoZZOqpfP4C1B9nc17GopMCizLp6SaCa5eQ/Zwy8jaWEinf3Bl4/EsEoBJcOCskDfXzQ5LPFFESQpoPPwPVxDR21qAUHEbdRx6pUgZGY7CNLa+11su8745EGSiT7isS4kSaMlvY1c/vx18xS3E77vk8mdWVDaXKmNE4t1kIz3IcthOjvuplQZoWHc2Mmo1z3OHgv4x0or5P2CoCAkeyaD73jY9ZvPqUhqiKb1e3CtBtXJQazKyiYXH49MbYDO+HaianrR51E1RUt0A1Uri+vZc+SK10PPphAN3WVqyGDBItcHczzgpAr1tqxK5/wKPMdn7K0pfN9HnF2lxNojtO1sxq7ZjLw+gaSIeK6HXXdo5Ifn4mCe4+FaC8d9euN+2rY9OBfqnj71IuXxQM6iY9fjFIaOYV2VN/1hQdPGNLHuOBe+eIraRJXy5QK9T6zn4pdO3zQ5qVm3sXSHWItGOaHQvC7OwKvTbHyonaaeKOHUrFGZ0En1xejcmWLmfJnLb2bmzt0oW5x/YZLNj3Wy8wM9C40KMH2hxOXXZ3BmVzC+B7XsfBN3qj9Ky/o4b/7RRXKDldltfEaP5Ji5WCbWfOt7bGCVRkUUoadb5tRZE0kUsJcoldv9kT4+8Ct3zb7ysRsu5Umd0SM5Tn1rjOxAGccMWroda2lvUGtun11lgJGbQokmiK/fweQLX6Jp+wHi63dSHTo7VyMebu9FkCSMbKDZcW0SXJQkEEQkVSPSuW4un1K5fJam7fupyAqCJNOYGcOzDBy9RrRnA+HOfuxqEadWRk21oiZbUGJNhFo6MQsZzPw0sf4tiIpKpKMfIz91k1rpS8MwS1SqE8RjXQiCQCScJqQmfmBGZSnoepbp6WPEIu3Icoim5Ho62vcxPPLKDaVXW9oknvxQjMELFuWSR32FvSqRtgibPrgZ27AZfXmE2sTN3Q9RVrAbVZRwYi6Ut1LUrDyZ2gD9qQOLuuxFQaIzvoPp6nk8XHw8BK5/fMv02fNgktyUyek3qwsqwOR4CLW9CXMiP5+zXCXMykIjXKiVKQzOOwlXB+ms2vVDdrIWoXD5GJnzs8n9q8bf9OmX13R9txtSSKZldzue45E5OolRaJA/m2Xrj+wisS5FefDmOPAs3aGWM0h0RKjnLSRVZODlKdbf10b/PS24toekiBTH67RsiKPFFYbeyiwyZuVJHatu07a1adE5yuM6VmP5+SbVE8W1PaqZxgKH3vegOFK/bUZlVe246ZTIQ/eFePqJCHfvVZHlpTi2BERJ4PIbM7z1Rxc58dURiuN1dnygl4//5j3s/nAfSnj5AVUbuYAgSkR7NhLt2YikhREkmfLFExj5afIn3sD3nKAQwGxQGTiJpIUJtfUAYFcL1IbPL3iwzWKWyuApQm29NDLjFE8H+h2VC8fQxwcJd/ShNbfPhuJAnx6hPn6ZSEcvSizoGNZSbeB72NUiWnNHcE0XjmMVs4TbemhkxqkNX8DRq5QvBgqEvudRHjhx05VpnufgOPMeiCSqiNItqC+/xbm6TO40ler4nCfe3Xkv0WjbdfeJJ0U2bFUpFz1Mw58juVsJXNOlPlPDd/y5nqabgV0vY5VzNPITWLXVlbZ6vkOmtrzyY1RN0xbbNLftjVCYtjj7ToXJQWPRPfE9HzkRIbqzb+6Z/YHD9+f/EdDeJHq2077zPcjhgCFDECWaN99DsmcHbTseJd65OXhfUkj27qBt+8Ok+vcgqbdnsrsaWlKjdV8HmcOTAXGsD7mTM9h1m+5H+m/6+LbuUC+YxFpC9N/TQnlCZ+ZCmeylCj37mok2a3iuh1405/RLvKUKL3w/oFdb4mf2XP+60WVRFGYbbpfY9zZWl61qpeJ5UKt7hEMCmiaiacvPSgPfn+L4V0cQJYFQXKFjRxPv///u4+H/axuV6QaDb8wseUNqw+eoDS/UW7naT7LLeezyPCV1Y2ZsrvwYwK4UsSsLJwTfdahcXCw163sulcHF5ZG+bVG+cAwxrKJ2pBBD6qJw3BVULl1TH25DZeAEansTdqlGZeDmJG4hoDy5mozRn5Mavj6UlgTx1i3oZ8fmPVrXA1lE8EHtacbOlPEMe7YuWMB3XARJQhAF3Nrq+NBsp8Hw2Ks0JdcjCBKqEmNd73s4c+HLy/bXmIbPyCWb8eHgGhqr0A3x3CBhnTubxSzffHxYUsOEm7vxXBsllsIsrY5HqWxOUdTHCSeSi8JbgiDQ23QXFXNmlofq+ki1K+x6MEG94lLMFbGvUuoTwxqIAnahtqLn4LqQZdTuLpxcDkFWEFUVpxysWERVQdC0oGPfc/EaxizNCniGOT9bCQItW+8n2bMdgNGDX8OqFTErOdp3PEJlagCnUUUQZTp2Pc70qZcwStN07HoPjeIk8a4tSLKGXpgk3rERRJHi8M2Pm+sh1p0guSGN1hTikf/0fvB85IiCltRoP9DFxb88vSb+wSuwGi71vEE0rdG6McHUmSJG2eLymzPs+Ugfggj1fJDob5QtbN0h2RlZdJxISkPRJPIjq2f2qOVMZFUkFFcWFXREUtqy+90sVmVUCkWPb35Hn6tGux5vlA/g+7hW0Ag0+PoMz/+HE3zk1w9w309uYvRIbkGZHASJpXCTiqxJ+J6PNWvtl4tvirIQVFhEZARBwDYcGkUL23AXbZfui1HNGtgNh1hLCCUs4XtgVCwaJWuxNRcFEvdvxa3o2NkyanczoixiF2v4rofWmcbOV/AMG62rGadcxynrqF1pfMtG621BTsdw6ybm6FoaCuchKyHCodTca8cxluxOvxZSWA323dBGdFc/9bNjeLpJ4r4t2MVasNrrbcWtG9i5MvH9GzFGckjxMGpbktzX3sGtr86wlMrDTGWO0dUeNF2mU5tob93DdOY4S3kRlukTbxIJh0Usy0dTRYYGVpYfcS0Xz/bpf2IdVsWkPHxz9DWCrGDrFVzbRNYi2LKK56w8V+P5LiOlw7TFNqNIiwdtSE7Qndy9JAfatdDCgXZ5ukNFkgTsq+6dNV2kLol4pn3T/QzhTZtQ2lsRtmzByeXwTJPI3j34to05Nh5Ubeo6UjKJ3NQEBL9i4/QZ3Orsqsz3yV86TPbCW8F9cGzAx6xkF4md+a5DcfgEnmPTsuV+tHgL8fYNhFOduJaOIEq4VmPVTNyrggA9j6+fyw+FmuZXRkbJINYdJ72jhcyRtSuJ2g2Hes6k9+5m4m1hDv/lZXwfht7O8MBPbyaS0sgOVvB9yA1VyV2u0bM3TeumOLnBKr4Pkiqy/v42ImmNNz+/mJz0RsheKqMXLHr2NZN4bpzKdAMESHSE6d6TxrpNFXyrziCvmRXCh6kzJcaPF1h3fyvJzgi5y7MPpQCtmxLc9cl1bHiwnVhbCNfyyA5WOPG1kaC6wVw4eLS4wt6P9bP9/d0098cRZYHylM7AK1Mc/ashKlPzD3OsNcTPfvkJXv7tM7iWx75PrCPZHcX3PMaOFTj854OMHMouiDvi+RhDM2g9zYghlcQ9m7CmisiJCGpnKtDbzpSQIhpSRMOpNnDKOqKm0BicCrRPNIXwhg7smVIwAawR0Ug7iXjP7G30MYwSpnXjclenrFMZuoTWlUK/OIkxmkVJxWgMZRAUCXumhFNr4NtuUGU3lMGaLCCnYzQGp3Abq/f+fd9lYvIgqeR6IuHmWcqWuykvk7QXBEgkRbr6VMoll4EzKz+nGlWItEUoDRZvSfOjZ5uIsoKohvAdC8dIYJZXVwlUMWfI1i/Rldi56DMBgfbY1hsXgQDTwybNHSrGkIdlLK7+UprjqK1JrKkivr32ycEplfDx8eo6brWCnEjSuHhxLpTl1GqIWgjftLAyGQTA1Rv4zsJz+q6zIgPs+95VZJ3+rONpkD33OoWh43NFLStrPl0b5JBM14N9TL89xqHffB3rqlVu8642Hv9vz9C8s43ssekF/Sergef41AsmzevjNMoW02eD6IlRtilN6vQfaOXSq9OAj627HPrzS3zgX+3jqX+xl7PfGadRsWjZmGDXB3sZO5rnwouTq74Go2pz9K+GuOfHN/LQz21l+GAWNSKz6dEOjIqNuJREyC3Au0rD2yhbZAcrbHiojY7tTXNGJZrWeP8v76WpJ8qZb48xdaZEKKmw+0O9PP6LOxFFkRNfG55zdCVV5L6f3MT+z25g5GCWo18awrFc1t/fxr5PrifZGeGb//rogr4FURLZ9/F1NMoWR788jF4wSPfHuPdzm4j/w518+1ePMX22NH+xAoiagpKKI6diCJKInIxg1g2caoNQbxSvboDrBoN7uohXN1DbkijpGAhgjuWQoqGAXG+NRiUSbmZ932OoWhCX9lybUmX4qnLe5eFUdNyKjl6Z39Yp1GgMZ5DC6qLwljUZTPpXqovWirqeIZM9RV/Pw4iiTCLeS2vLTsYm3lwUBvN9OHnIYOCcFcgcr0JPxapajL4ygu/6WLVb01Hv2iZyOEajNLNqg3IFY+XjtEY3okgLcwOCICCtcMg1d6rEm2RE0UWUhHlNFR/cagOtK41Tqi9mI1wl7Olp7Ol5dUe3tHi151JmtU+vFm8mvX4fkVQn7TseoTx+jur05UXbeV6wcmneuJ9Y+wbAJzdwED2/sh6ntaB5VyDsVhosYFUWOjHly0WMYoP0tlbUZGjJnpWVop4PtFIKIzVqmWCsOZbL+LECGx5opzypz0VIxo7m+d5/OMndn9nAgz+3FVkVaZQsLrwwwfGvjKxpVeE5Pse+PIQgws4P9LLliS7qeZPzz08w8k6W/Z/dsObvdj28q0bFtTwaJQt8iLfND7gDP7qRti1JXvu9cxz78hCOGdRcjxzM8hOff4S9H+tj7GhujnSt70ALuz/Ux+SpAt/+1WMY1SDRNvx2FkEQ2PF0DxueG+fS9xdKoYaSKt/8lSNMnSnhez6iIqAXTN7/L/ex9Ykuspcqcw1G+NAYzmBO5PEMi/CmDspvnMOzHPChfmIYz3ICLfhzE3i2g+96mFNFcFyM0Ry+41I5NLCkJykIIooSZXFISAj4lbQEzektdLbfTTiUmhOOqtWnmJ45vsR+iyFJ6iz9/BIwQbrmM9/3sJ2bp7vxPJvpzHGa01uIRTuRJIXe7gfI5c+jNxaHAhMpiWc+FWdi1Ob8SZOpsZUNIC0VItYZI72lmZGXh5fkbVrVdTs2enaUUKoTQVz70KiaWTL1S3TFd96wdHg5yIpAbspi4lJjASGhFA/jVhtUDg6A6y2iFPlBIHfxnUXvWfUS2QtvkRs4FKxOXBvPsRh4/g/nthl588t4tonv+5iVIKfDLFP0bYMAvY+tw67b5E4uzuu6psP0wQk67+8l2hmbMyqSJiHKIsqs/pMoi6gxDdd28Sw3YCe4BhdenGT4YBbX8uaMh+f4vPOnA5z46nBA33Ll/D4Mv5Nl8lQRJRTkNIP+IWeuZPgKGmWb//0jRxGdMJ4lwlx1pY8oyKSifVhOnaqRQS/YvPVHAxz54hCSLOK5PkbNQhREzj0/iVGxWEiVEMw/QcXm2igU1jRyBEkk3JnAKjdwqqsLkfjebDXDLFupHJLY+t4uypM6Q29l5sJcvgfF0ToTJ4v07EvTvC5GYaSGIAn07E0Tbw/x4n8+tYBXp1GyGD+RZ+sTnWx6uGORUZk4VaA4Vp9b0nq2z9A7WSrTDXr3N6OEpHmjAvimjTu7wqgeGcStz39X96o4oKvPv+/pC+/Hco1p0Ugr9+//B4s/EAIGY1Gcr+4KtFQ8avUpzg18Hcu+cdJOEER6Ou+jp/O+G257BY1GgbcO/xduRbttXc8yMXWYLRufQRBkNDXB+v7HOXfxK4s4zHyPQHlywKZaXsUk6fmEmkJUxyuLeifWAklRiXdtxvfcRQJdoZYoUkjGLDaIdCQCluRsDSmsoDaFqQ0X5pRCHc9kpnqBlsg6NHnl0s9XQ6+6bNwdpaVL5c1nC3Na5lpvC0oyipUNVhSNwelV0bUoYpiQvLjz/5ZAvqZqywXbaixgEXCukpK42ni4tgH27RfKC7dGad7VjlFsUDi/eCXquz7Tb4+z4UNbad7eSvF8DjmisPVHdtG6r4NQOkKkI4aWDvPof3o/Vs3k0lfPMfn66KJQmWN6OObi+dFuuIvyycHJwao7N1yV+J6PnnMIq4GjmQx3IYoSdTOPIoVJhDuYKB4npMQIqykMu4pTMdG0ZgyniugKRLUW3LqJJjgoYQ1BkKgaGaJaOnCUrSJRLY0kqpTq4zdsC7gaa3PHBFBTEWLrmxEkkfLZKYzMCiY6SUCNyAgisxYyWLFoURkhJnDfT25adLPT/THUqIwWDyZZJSQRaw0jiALbn+qh7+6FIk2p/hiSKpHoDC86fz23WLveNT2qMw3i7WHEpUqkZ2Fnbq2GiSCIqGr0htv5vo9t62RzZxmfeod6fWUVSWv1kG8lZjIn6GjbS1OyP+ALS2+lOb2FbO7sgu0s06OQc4jERBRl5det53QyJzP4rrcgLr4WSGoYORSlnhkh2rEeUVYXfN7+8AbkiEL2nRFSu9oxcjqx/hRGrk7z3i7qY6UF2xeNCcrGFK3RTWv6LXwfjIaHXnEW2AzftPFsZ828X92JXTRHbr5kdqUYLh4kp9+YAfzdgu96XPjiKayqib1MyDR7cpojv/UGlZESPuA5HsWL+WUrDPWZW1CFtwI0dYbY90wHlw8VyZ13cLzg+pORLhpWiZCSwHJ0LKeO45p0p/YgCDKSqGDZdcJqE1UjQ3NsHaoSw/UsfM9Dt0pEtWAelUWVqpEhojbRFOlBEhVqRgbbXbnBX5NR8V0Pu2qS3teDUzdp2t1FbbhAbfD6MWgtIpPojIAP2UtBqEJSAyW4UFJh4yMdSzrJesGaMwaiJASUEQRhsKUqw8yqPcd/s/DCl+Dp8X0815tNWt3eidjznBVVbQVhqAYNo0i5PEKucB7TyuMLJkoo+O6e4y9isPXx8Tx7RedY+vqWj5z7vjf3uSD6yKqAZV1/bnNcg+HRV9i5/TOIgoQgiHS23025MragcdNxoKVNRpIF8hknEDjy5jnGrnyfa0txI21R+h/vpzZTZ+boNI6x9oS1pIZREy2o8RRWJY94Va+EFJJRkyHsmkm4I46SDCNHVIpnZjByNfSZGs41Hf2222Cqep50pB9ZUK893Q2RbJGJN0m092lcOFabE+oyRrP4w5kbimkth5jWTExrXtO+a8FUdaEDIYggCleqQ2frAWb/vlmoYYl7Pt7FloeaiaZVKlmTk8/NcO6VLKYePDtGvsHwdwauexyzaDD4tfNzr13DYeK1kZu/wCXQuSWGD0xfvLFT3rUtziM/2U+8OcQblw0SoQ5qRhbXszDsMiDgeha2a+D5LnWzSDLSiTP7umGVcTyThl0OVjBOGUEQMewymhLDcQ2ioRYEQaRqZPF9D8e15ozXSrEmoxLuTKKmwox+7QSiLOE7HlLkxs14qb4ovXc3U55qkLkYGJVGycS1XabP1vnKPz24LB/NFU/AMT2MSlAC/Fe/+BZTZ0rLnG3xUxpKqIsqHiRFJJzUgrLiGzTeCSK0rwsTTcpkRgyqhdWlL0+d/YsVVf5AYCCu/AXw8Kfaef/P9hCOSUSSMq/91Qx//ZtDCyqDGo08B4/+7orPsfQ5l74HE1NvMzl1EICnfqabX/oXG/jTfzXA4LHrd7Hnixd57c3fuO45XMdnZNBi03aNex+NcP6kwcXTFg0jz6Fjvzf3fZbatzJRpTiQv2maFqtWRI2nqE8N4lxF8QOQ3NaGXQ0IK5ObWylfypE9OIrneCQ2tqDENURVWtSAOVO7QF/TXTSFule9WpkaMujeGCY/3VjQo+Lbt68q6t3Apr1RttwVxWp4lHM206MWvu8zPWLOhfjWgmSHxqf+zU7W728iN6qjl23SXWE+/ivb2XhPim//lwEalR8uEkxREnjf393I+NnKioxKbkTn/KtZzr+WoVArUKiNAD4TxYU9PZnKBQAK9WGK9ZGr5pIAlcY01cbM7Lv+gn1qZm42p+IzVjg2295yu3MqokDr/esY/UrwRbo/uJOJ75zFrl5/eRRt1rjnc5uIt4V4439ewJ71Kut5k8yFMu1bm2jbkuTyG9cP7zimS2agglV32PRoJ1NniisugGnbkiCUUBfkYZp6oqR6o5x7fgLnBnH5ZIvKZ//lBrbel+Sb/32Ub//BarXi/VX/QFdw+rUiuXGDtv4wH/nFPq7l4bwV57gRrsRVIwkRRQv+rWa/5aCoQXLw+9+p0dB9WjuufiyX/z5W1cTRbVp3t5M5Pk19+maIDn1qU0uLytlVk4kXLoIPya1tNKYreLaLIIoIApi5OpImLzIqnu8yXDzMvs4uVrMKbmpVUDSBzJhJW6+GJMNtYP/5gWDsYoPuDSFKWZt4Wqajb5YDK+dgm2v7krIqcN+nethwoImDX5ng+58fpjRjkmzTePzn1rP/I12MHC9z+OurL8u9nQjFZPr2NTFxbmX0QpnLdf7qV65e+d14nC83dq43R1zt0K5lJlm9UfF9lHiI+KYg9BTuTCwZAxHFQJch0qTSva+Z3R/uo+9AC0NvZTjz7bH5nhAf3vnTS3zqv97PQz+3FUUTGTmcw/N8ommNjh0pwgmFU98cncu3DL2VYfpskb0f68Oq25x/foJa3iTcpJLui9F/oIUjX7pMPbdw1ZPqifLgz2zhtd8/RzVj0Lw+ziM/vw3P8QJiNuP6RqVjQ5jOjRFM3WPLvUle+rMpjNq74zmWZixKMxbZMYP3/2z3u3LO5fDyn09x/KUCkwM3z1abaBJp7ZQxGh4f/JEEh15rMHTxxqsOQRJoWt9Eojc5S3x4+2Roq4PzJda5w/PsDb7rUb6YpXxx+ebWgj5KyZhc1WolEhfp3hjm8mmdTXujSIoAxu1xFN5tWIbH8VfL1CsusWSQX/VcH/0mxlGyPcSGe1I0qg4v/c8hqrng+SnPmBz86wnW393EA5/t4cg3JlHCEs/8481Isshf/+rZBfPyxvvSPPDZHo5+fYqzr8z/ptGUwu73trPxvjRqWCI3qnPsW1OMn60s2F8QoH1TjL0f6KB1XQRJEajlbYaPFrnwRp5a3kIQYd3dTex4rI2ubXHizSoHPt7Fhnvmm5tf/98jnHtlPpXw6E/3s+WBZgQxeH4OfXWC488uLEK6Ai0qse+ZTjYcSBGKy+RG6pz4zgxjp8tzzncoLvOJf72d0y9kyA7VufvDXbRuiGLpDoMHCxz++iSutfbnbQ1GBca+fpK2RzYihRWmX7q4pMF87z/dzeP/YGcQM3V9bMPl5NdHefMPz1OdWbiqGT9R4LlfO85Df3srH/iVuxBncya+5+NaHoNvzHD62fnBXJ1p8J1/e5ynf2Uf9/7EJh74mS0Bz40X5Bmsus2Jry2OgV54aYrm9XH+1p89higGuRnH9Dj6pSEG35i5bn5AkgXW74kjyQIHv5lh13vS9O+IceHgfAJfFKG5J4RedrBMj1hKRtVEPA+Mmku1aC+4V6IE4bhMKCohK2KQlDc9akUH21x9zFxWBdIdGo2au2RoLpaSiTYpFCbNueMLAkSbgmu4UnJomx6NqoPZmL8GQYCWnlAwwQG1on3dFaKsCsSaFJSQiCgGIS5TDxLPV+eCVE2gtV2mo0fm2NsGosSKNEKUqELLzlZ8D8yKcVP5lNsJ2zMYL58gobUvkhxeDlPDJq3dGvc9nWLknI6p/+DLhm8VPBdK2eC3KmZujSOQaNNo6Ytw6WCBWmGhQ5Ib1skM6Wx7pIWmzhCNmkPfniSyIi56zBItKhsOpBh8Z75JN90d5hP/ejvtm2IUJhtYDZdtj7Zw94c7+PpvXODEc9PBOBBg23ta+egvb8V1fPIjOqIssO6uKH17ktQKFhdezyMIAtEmFVkVMKoBY7VetMmNzDtoxjVhuomzVWRZpGtHgm0PNzN4cGmyy0Srymf/w27aN8bIjeo0yjZbHmphz1MdPP97lzn01Ql810dSRDbdlybdEw5aK0SBWtGma1ucHY+10rE5xjf/w8X53qhVYk05Fcew8WwPQfZovqef2uV5Ty4/VOXYl4cBH98DS7epTDeYPFVk8kxxycnCd33OPT/B9PkS6+5tJb0ujigKGFWL3OUqY0fzi8rsiuN1/vofv8OGB9to25IklFCw6g6VqQYTpwqUpxb3W9TzBi//9hk2PtxOqi+Ka3mMH8tz+a3MglLipRCOS2x/sInhU1WOfi/PAx9vY92eGANHynPU5JEmhX/2hd0cezFPadriwDMtJJoDmo3RczW+/ftjXDwU5JIEEbbck+SJn+iib3sUNRxQ09SKDoe/k+XFL0yhrzIGnO7U+Pn/tp2ZIZ3P//OBBYZJDYt8+Bf6uP/Dbfz6J4+TGQ0M+74n0zz0qQ46N4ZRQwHRZzlr8c43Mrz0v6fmDIASEvn0P19PW3+IpnYVy/D4/C9d5Pzbi6viYmmZRz7VwV3vayberKCoIq7jM36xznf+YJxLR+b7SXIzLk3NDiODFtWyF1AAreBZtioW5790jnBzmPSWZtS4hlG4jf0Na4ZPoTFOyZgkHelbUb7L9+DEaxVOvHZzfTf/p0CLykSaFHIj+qJnx2q46CULQYBUd5jGhVUwWQvw+M+tJ90b5qu/fo7z38/huT6t6yN85td28d6/s5HJc1UyQ3UUTWTLg2lESeBPfuE4U7M5EjUi0bE5xsyl4LXn+px+IcPpFzL07kqw7dFWzr6S5cU/WNwYegWD7xQYfKfAloea6d+TXHIbSRF4/Oc20Lklzrf+4wVOfCegxE92aDzzj7bw/l/YyMSZ8lyoTZQF2jZE+f7nh3ntT0ewGh7pnjA/8hu72PlEG2//5Tgzg2uLRKzJqIiyhBRRMLM14ptaESRhLsk9ddmi+JKEUyhhXhy6PkHY1fCDvpTi6PwXkdtacArFZblhrLrD+ecnOf/8ymKlgiigFwyOfHH5H3A5NLVrrNsd4zt/MM7Y+TrZMYP1e+JEk8qCVYEkCxx4uoWR0zVe/JNJqgWb3u1RnvhcFx/7R+v4nZ8/GxgLHxRNRC87fOt3x6jkbRRN4L4Pt/HkT3VTmDJ54yuL5Yavh9KMxdk3ijz8qXbSnSozw/MrwnSnxsa9CQYOl8lPBmHBlt4QH//H66jkbb7+X0fRqw7huETP1ih61V2wcrMNjy//xyHCcZknfqKLbfct/XALIux5T5r3/q0u3vpahouHyuALJFoU+nbEsI3FxvvS2cXhLjEewas1FodWRWGBhkgj32DirevktgQB3SpwcvpZpEgYBAFPb1w1+fiUjaVDCbcKhl3mYu5VIkrTdbdzPIOGvfbSdc93uJR/HUVaXE5/NQRRJLl5b6AbM3z+llReLYeSsfY8RtmY5OT0s8saYttt4PoWkhJDVkXMmrPkV7EaQfOhFl2dnEG8WWXnE62cfz3HxTfyc557dkjnwhs5Hv7xPvr2JskM1fFcn0bFQYtI9OxOkB3WcSwPS3cZPXFr2xGWQronzPr9TUwPVOcMCkB52uToNyfZcE+KfR/qXJC/yQwGoTFrNiJRGG8wfKxES38Xqe7wu2tU3IZF9vXLRHqaqI8WF1RNads3YY1N4hbLSMk4kXv34Mzk8V0XuTmFb1qYA0OE927HreuYF4YI7doCno81OIzS142UjGOcuUj0gbuxZ/KYFy+jdLQit7dgnDyPPX1zBI1rwd4n0ph1l9GzNYy6y6lXijz4iTZSHeoCoyKIAkbN4cU/neTsGyV8H86+WaKtP8zOR1L0bI1y8VCgS332jRIXD5XnHnqAqcEGv/Rnu9m0P7Fqo2IZHufeLHHgA63c/9E2vv7bo8E1CdCzLUrnxjDP/eH4HJV6qkMlmpQ58t0cR7+Xm9PsOPZ8Ht9nwfLX9yEzEhip0szyPSGCINC2Loypuxx7Ps/l4wE5niDAwWezKwrrCapM/Ml70d85jWeYuBUdMayC5xN9+C4ax87jFMoo7c14uoFv2UipILdnZ4vI6QQIAk62SHjPFnzTYvr8eeRwCt92catlxEgYKZXAyRTw7YVhmFBrF52PfhQ5Gsf3PFxTpzZ8gdzx1/HXoC7p41M2JinfxAS7svN4K+sJEUVsqQuXGoXquTX3vKwE25/upWtvPy/+5gme/Gd7mT5b5My3Rhdtt/6hdra/v4eTXxtm/GgQ+ZCSFjVpaI7iBKB1S5JkV4RLr8w3p/pe8G85Litptv9sKYfmemjbEEWLyux6so0N+1MLPtMiMpIiEmkKQpqu7XPyezOsu6uJp39xM/d/ppfjz05x5qUM5WlzUfn/rUaqM0wkqTD4TmFRB352WMeoOfTtXugIVnMmpemFqQi9bCMIgbb9WrG2jnpZomlXJ2oqgqNb8zFwAaRYBNv3id63FyQZt1RFSsYRNZXGuUvI6SaiD9yNUyyDD3JrGqkpQeUbLyCEQwhqQLUiNSVx8iX0I6eQEjGk5hTOTA5107p33ajIqsD+97cwPdRg6nID1/E5/06Zxz/XyeYDCcbO1xfkFzKjBsOnanNj1bV9pi832PVoimjT/C33XB81LNOxIUIkISMrAlpEQpQE1LA016+xGlw+XmX8fI0DT7fw0hemqBZsZFXkwNMtTA3qjJ6tzXmmU5d0MqMG93+kDVN3uXiwwtRlnUZ17UlTz/MZPFrh4U+28+G/38drX5pm/IJOftKY84huBKW9GUEUkdvSyM1J9CPnUNd3YQ1NIkVDeKZFaOs6ALQt/Xj1BtqmXuzxGaSmgCMtsn87hT9/DjGk4tSCsIiUjIEk4tUbhHZuxKvUcWYW85wJoowcjTP9+rdxzTqhlk6a9z2CqGjMvPntNd+bHxp4Htl3vveunEoJS0RSGum+GEpYQo3KJLoi2LqDUbVJr4tRGqsz/NYMTd1RZDVYTYTiClvf140Slhk9lGXqdBEtrqBo0pyK4RXYhotZd0i0hRblSQLpDRlBgFr++gUggigsWBWpYSkodx6oMXS4tGh7z/UYPz1/LdMXa3zhH55g9/va2fVUO4/8ZD/v+el1vPaFUQ5+efy2ljRLSqBjZS1hOF3Lw3d91LC4oAjRsX2ca5rBb8WqdY3hLxFRlcm8PkjX+7ejJsPYdQvfdrEmZlDX9SBGIthTGeSWNPbkDDQl8BsmeB5OvoSUjOPM5HBLFfxGYC3llhShPduC8EStjlutE9q6ASebB99HSsYxL4/d4OpuPXq3x2jtDaGGRN7/M914jk+8WcH34K73NvPKX0zjXhWSMeoujdrCB8h1fASYkw0QRNj5cIqHPtlOW38Io+7OeVKKFpSqrjC9sACNmsuxFwp84p/0s+s9Kd76aoZkq8LW+5K8+dUMpZn5gVUrOvzl/+8yj36mg8d/vIuHP9XB4LEKx18scOb14oqNwAL4cPFwhS//5hD3f6SNn/h3m5gabHDpaIV3vpFh/MKNucW0Lf2YlycQFBnPdghtX4+gKHgNE99yECNhfNtGjEXA8/BtB7dcxZ7OIbelESPhQHPEdfF0AzEaAlFEjIaQUwnsySx4HkJYBUmCZVh+jew4drWEPjGEGk8T7dnIFUuf3n0/jcwEcjROpHNdIBh36RRWKQeiSKS9l1jvZgRFCwTcRi7gWfNeodbcQXzdNuRIDM+2qE8OUx8N6M1FRSO5eQ9aug3XMqgMnsHMTy/ctz/Y17UM9Mlh9MkhfM9FkGQiXeuJdK5DUlXsepXayAXMQlCqH+3ZSHzddgRJojp8ntrIhbnjRns2IofjOEY9+K6+R210AH1ydvUjiEQ6+oj2bkIORecmqOrQOWqjS1Oz+55PZUqn//42KtMNBEFg/QNtFIZrTJ8rsevD/Rz+s0vouYUes6SKRNMhlLAcaH8IwXs9d7fguR6H//eluW31sk0lY9K3O4EgCgsiJ7FmlWSbhl6xKUw0EK90XgqBwbl6BRGKy4hXMTpU8xa+FyTKn/vtgRUt6BpVh4NfmeDk8zP07U6y/2OdPP6z69FLNoe+cvtIMhsVB9vwSLYtllwIJxVkVaQ41bitoc4rWHNHPb5P8z19eKZD+u5eSmenMKarmJeGcTI58H28mo4Yi+BbQXjBM0zcWuApi+EQvhMI/9TfPgaAM52l/sZhfNfDN4PJTwyH8Go6bqmCICt4+uoJD2tZgz/6kZfQl9BaWQn2PZFGUgQiSZn9T8/TwgiSwLo9cVq6tQX5C9+/scxFqkPjx//NRmpFh6/+5xGmL+s4to8WlvjlL+1Z9TVejWPP5/nQ3+1l1yMpjj2fZ9+Tzbi2z6UjlUWezPDJGjNDQ7z0hSl2vaeJRz7Zwc6HU3zv8xO88CeTa6oAMesuB5/Ncu6tEl2bozz4sTYe/Fgbux5J8Sf/coDhU9dv9KofOoNX0xFUNfAeQxq+7eDbDrXXj+HbTuCYREL4vo9v2piXx/FNC3sqhyBJNI4FHdHGxRFEVQHPwxwYw1JkvHoD4+xlBE1dFPpaEoKAIIqBVPTszJLcso/4+h1YlQJWMYeoaohyEAqJ9myk/YEPoE8N4+hV0rvuJ9TcQe7Iy3i2FXz+0DNYxSyN7CRSOIISmVVHlBU6Hv4QWqqF6sgF1GQz3U9+mokX/hKzkEFUNbqf+CSN7CRmcQY1niLU1jU38Uc619F23/uojV7ErlfQUm0Yuck5o2IWswiSTNu978U19AVGJdTWQ3rnfZjFLPXJy6jJZjof/QiTL3+FxswYWqqV9oeeoXL5DI3cBG33PEl15CJGfvmclO9BZUqnc3eaiRP5ubJYmNWFE5cOWdXzJjPnS8ghicHXguPreZPMhRLNGxZylxUnDaYHamx/T0tQvXWwMKeW2LsrQefWOKdfymLpLpIi4Fge4YRMok2jOBmMWzUs0bU1PlesAjBzqUY1a9K1NUZzXyQoBLjq4kVRWDA+BHH28fDBqDpcfDNPOWOw64k2urfHOXTNd3TsgDhXCYlr8yCvQmaoTnnGoG9vklizOr8qE6BvT5JIk8Klv7w5ieSVYm1GxfOpXs4hKhJaa4zMa4PzyVPbwc2X5rZ1CwuTVP5s0t01riJhrAaeom87i7a/st2KBv8y8ByfmfNrS5bFmxU2H0iQGzP4zR8/iV6ZN0r3PNPCj/zyeg58oJVnf291K6jN+xM0tao8/8cTnH61OPd+1+bITfN26RWHN74ywwMfbWPDnjj3fLCV6aFgtbAUGlWX8Qt1Ji7WOfjNHJ/7txt5/8928+pfTmPU1xYK81woZ23K2RLn3y6x+9EUP/0ftvCez3YwfOrS9fetBAlC3zCDcabPG2y3XFvy7ytOiH+NxIDfMOd0Ybz6fHWYpxsLjrsUwu29qE2tRNp7iPVvJXvoRa6MfEFScPUqM28+N7sCmZ0VBIHWA09SHx1g5p3vguehTw7T+djHqAyewioXaNq+HyM7ydSr38B3bK6eUWJ9W4j2bWL4q/8zUDAVBNZ99GdI7bqf6Ve/gaRFkEIR6hOXqY8N4JpXvM/A/ZZjCTzHojZ6ETM/HWiXXOViO/UKdaOOte3Akt9ZVDWm3/gW1qzxWf+J/4twRx+NmTGiPRtxDZ3i6bfxHQc1kUYKRYNrWAY+PrbpzgpieXNl/E09URzLI9ocCjR1OiNEm0M4posWVzCrNq7jkWyJku4PyGRDSZV4e5hos0asNRQoJ3o+esnmyDen6Nmd4JO/uoOX/9cQM4M12jdGec9Pr6dWsPj+5wOj69o+l94p8MhP9fPYz6znyNcDx2nzg81svDe1gBTSNjy++zuDfPSXt/KBf7iZN/9iDL1sISki6e4wnVvjfP+PhzGqDmpE4qEf66WSNckO6dimi6JJ7P1AB7blMX1psSNVmDCoF202P5Bm5HgLpWkDURIoTQXvQxC1uLLSSLRpgTBhSiHdG8ZzfMy6Q6PioJds3vjzMT7yL7by0V/exltfHMOoObRvjvHwT/SRG9E5/LV3p/lzbeEvTab1gfXoE2UiXQmygjDXexlLdBNP9uK5FuXiMEZj7dYxHGkmnuylWpmgUX/3k/MA/TtiNHeHOPZ8nlpxYZhk8FiFcs5m5yMpXviT1f1gtunhA1pEQpKDZXi8WeGRT7cHsc+bxNHv5nn0Mx3c/9FW0p0q3//iNNX8wgm3pUdD0UQyIwauE7BHO7ZHo+quWcBHEKBna5RqwaacteY8t1rZwXeX90wB5Gic+PodSKFAVtUqZKmOXpideG8PYv1b0afH8MzFK+DUrvvxLAPX0MkcfIHyVZLUvudi5meuCmnNTkaCiNbUTPHM23PLVbtWxjMN1GQzrtFATbaQP/H6Vd/rqr6dZDOirJDafmBOAloKRVATaRAl7FqZ0vkjpHYcIL5uG/XxS1SHz+M26oCPPjFEpKOPtnvfi1nMURu9QH3i8sJ7eB2P2GnUsIrBWPN9D8fQEZWAu8wx6kihMEosidPQUeIp7Grxusvy7EAlCIFN6jRKJqIsYlRteve3kOqNcvY7Y9gNl5adiUCvXRKIpDTMms3M+TKR1hjtO1IURmvEWkKIskA9Z5DsidIoWXNh5wuv5fjuf5O4/zO9PPNPtqCGRGRNJDfa4Dv/cYDixLwD8faXxmnujbDziVZ2P9WGWXPIDuu8/aUJHvqx3gXXf+alDGpI5O4Pd/LZf78r6OuQBMy6y4U3cguE/WLNGvf/SC9aRJqV7xDQyxZv/cUYJ59bzBRi1R2+9z8GeeRzfXzm13ZiGR624fKd/zLAmZeC36Bvb5KHfryPeKtGLK0SbVLY90wn6+5KYRsu51/L8crnh8GH0y/MEE7I3PPxbn70N3fjez6eByPHirz6JyPvGk3NGqu/bPJHxjDzNfTJ0rz+ORBLdKGoEarlAr3r38Pli9/BddZGae04BqFwCs9zfiBGRZQF+nfFSDQrHH5u8fnzkyaj52psu6+Jvh1Rpi6vvE9i6ESV4pTJo5/pIN2hYRke3VsjOKa/IO8BEEnIbL03STytkO7WiCRk+nbEePzHO6kVHbJjDYZOVBdwJxWmTM69XWLfk834Hhx5bjHZ55Z7kzz2o51zBsBzob0/RPfWKG9+JbOgUmvd7hhdmyOEohIb9sYJxSQOfKCFjg1h6mWHy8er5CdMJFngyZ/qomN9mFLGopKzCcWCMmXf93nnW8v/jkqsiUjnOqpD5/AsA0evrlouV1Q1tKZWjPx0EK66AexqCd9d2mjNvPFt7FoJz7YWGzbfX2Y/P+Cpu0oOVxAEEITZceKD719HLtfHc2x8z5sbV+WB2VwNPvgeuaOvol4+S7i9h6btBwi1dpF5+3k8y8CuFsm89V3UphZi/Vtpu+995E++SfnCsRVVeXn2tclsfy55rY8Nkli/g573/yiOXsOulSlfPL5ApVHpbie8eytupUbj1AUy50tLnqc8vrBcdfjtDMNvL6x2rOVNzr1awCkGneu5wcqiJP3cdbs+x5+dZvRkmUSrhqyJPP4z60l3hzHr7oIwVSVj8o1/f56WvghqRMKxPYoTBo2KzejJ0gID5JgeR745xaWDBZLtIRQt6Lkyag6lKQNrluXDari88kdDHHt2ilBUns3XeDjpToyuPUQ+c4CQaaEfO4txKgg5+j4c++YUYyfLxNIqghgcJzs07+Bkhuq8+edjyzKoV7PmnJPge3D0hTKlbe8hnlGRcMl+/VVKORd/24OEpGGMUxcwqjZf+Ecn0We1ra7Gye9NM36mvOZyYlhr+MvxKJ8JSvqMmeqCC/M9l0Y9RzF3kWisnab0BhzHoKVtB4Igkc+ew3MtYokeJkZep7VjT6AVUpmko+cAihKlXpthauxtbKuOZczXVUuSSu/6x1DVGKZZYXLsLZrbdhKLdyCIMpXiMJmp47S076YpvR7Pc5gaP4xtVunb9CS2VUNVY1y+8G0STf20duwOlNmy5ynkLnAtYqkwqc4wFw6WGTs3/0OLSgjPNvA9OPztHD1bonRujDAxoJMdMyhnrxmYgkij5pEdM+bYUosZi9/7hXM89mOd9O+O4Zge598u8/pfz/DEj/egaPOt5U1tKo9/rpNEs4IgCNSKDtGEyoMf78DzPC4dqTA5oC/gTjLqLqe/X2T/Uy2cP1RiegmDd/FgmY51YTbsS7BhXxJkkcxQnb/6rVFOvJhfkMTc+0SafU82IwiBsa3mbDYfSLLp7gSW4fHc5yfJT2RxXZ+jL5a490MKrd0aXVvjNCoWl09UePUvpxk7d/2H1WnUqY9fwjWuut9aGEkLAQKuUcezTBBF5HAMQZLxPRdXr4EAkY5+or2b8FwHt1HD0WsIijqbWBbmVh4IIlIoEhiGZUrsXKM+uwJYBTwffWqYaNd6aiMX8R2bUGsXoqJiFjO4RgMjP0Vi4y7qk0N4ZiMwQIKAZzYwspP4tk115AJWOahMEyU5CGN5HoKsIKkadqWIXS3i1Gu03fde5EgMyzICdmVBwCzMYJULiKpGtGs9lYGT80ZWDJjBg/NeW2J4HcMjBEUEmXdeoD4+iO86iwy3GA3jFMv4DRO1txO5vQVBEDDOXULp6URua0Y/chptQy9SLIp+4hxKZxtySwprdBK5JY2YiGKPTeEWymib+tCPniW8fxdiMoZbrGBPzhDZvxtPb1B79eBcztb3IT/aID8aPOuu7fO3/vs+nvjb6ynPGOTH5seAXrKZcRuIiohVt+comoaOlAg3qYRTGp7r4Tk+akRGr7jUSzWUsIxjBkZKjchEmkWMsoXn+FRz1hxFzPwNKSM1TRJ98C6MUxexxqdni0bCCLKEa1hMD+qIU36gMSUIeLOPvhiLojsSl89ZeJUayHKwnyDgNQx800KMRxGjkaCopd7AF2Wylypc+u2X8S07uDdCoCArNyXm7svQqQaCpiLGo3j1BmI0jG9alCsSxekyorY44b9S3Lzy43WeQdexULUEydQ6atVpBATiyV6mxt6mvWs/oXCKaLyD6fHDJFPrEQSRfPYcrR27KWSTGI3iguPFm3pxPZuLZ79C7/rHiCd7kWWVfPYcldIY6zY9hV6boaV9FzOTR4klOoknuynmAqrrzOSxuWOKkkKtOk1Dz1GrLK7K0BKt+GqKL/1mFqtWRFLDCKIFgkBq3W6KQyfwHIszbzY4/foZfNdGkBR+8ycGcK0GWrwFz7Gx9TJKJMHb36rx+l9NIUgyoqLh2RaTgw5//msTeI65wIt86Y8kfN8lFGpG1/Nkhn3+y0+fxfc9ZDmM65pEIq24rolp1mYlehf+EIIYhNZ84M2vZpZ0UnPjJl/5zyMIikRkXStSREG/nCWyvgWSCSjmEDUZQZb4+n8b5Ru/M44YUnAbwcCRIipew0aQRbT2BIIkIigSU34rX/z9IvXBDOG+NI3RAr7tIkW1oNrK9xAUCVGVcGsL+z6UeJKmbXfjWSZGfppGZoJY72Yinf0IkoxnW8y88SyR9l5SO+7BtUw8q0HxzCEQBBIbdxNq7UIQJYz8FKWzhwm3dpHYsAtBkhEkienXvgWiQLx/Ky0HHmP8u3+OkZ1afIPWBJ/c4Vdoe+D9dL7no3iWgZpIUzxzCLsSrIpKZw/Tdt/76Hzs4zi1EoIoYRaz5I+9Sn1ymNL5I7Q/8H6scgF8H1FWKA+coDZ6kVBzJ60HHsfRq3iug5pMUx05j1OvgCCS3LSLWP9WnEY9UBdNpCicehvfcxEVjVjfZrR0O6GWdpRojJa7H8Uq56lcOnXjrzZriCOd/cjhGL7rYJaygSG8yrio3e14DRM7kwfPx63VkDtacfJFEEXUng68egOvWkeKR4Mqs1cP0fSppzEvj1J/4wjxJx6g8t3XUEwLQZYQNAX9yBmi9+7Frel4tTr2dC4oGloGY6fKvPaFER79qXV86Je28he/dGpuZaHGFLZ/sA9JFqnnDUbeniHRFaU2o7Pr4xuoZRqEUxql0RotmxJMHMsRaQ7R1BtDzxuUxuv03N2CY7qce3YkMEKOT3GkulCzyZstOnJcPMsK5MfXdRPevQ3fcRAkifrBEyQ+8B6c6VzQozcwhHH2Ek2feAq3VMEtVai9eYzwjk2o63uCAqdanfqbR0l++EmcTB5EgcbJC/iWjZiIEdq6Ad+0aZwbCJrHXQcIwphCOETi6UfxGiZCWKNx+BTRB+7GuHCZ8F3bqT73GqFdW6g+//qaRsBtkxOWlQiJVD+To28RjjQDPoZRplqdwHEMirmLdPU9SEPPYdv1ufJEz3OYHj+8pAa7788vxeeT2fM6KMKVakFBAN+jUhrDNIr4vofrmDhXKcuV8oOEoy3EEl2EI81MjS2URFWiSdRYmkZxmlBTO1qiBd91qEwO4HkugiihxlKEkm1IWoTq1CCR5q6gxDM/QaxjI2atgN2ooEab8BwTzzaJtPYhh2LUM8NE2/oRRZnazGWs2rwBTSR6ZkOGAoocQZLDOLaOj4eiRPBcJwifqDE0NUm5sriZLByT2P2eFNlRg6GT16em0FrjqM1RahencQ0bz3QQJAEkEbUlTmJnN4W3LhHuSSNFVCpnJwj3pFGbIlTOTSIIAuHuNOZ0BTmmEe5NUz4+igBE+lowp8oobQnCPSkQBOqXZkjs6aUxmqd+6ZoGT9/Hd10818Wf9aA9O7h3guuQ2LyHmTeeRRCloFx37BL1yaFgUvV9ShePEjN1codfnst3eLaNZ5vg2iQ370NUFBy9Run8EeIbdy66H1Y5x/Trz+I0lq80zB56Ebu2dPGHkZ9i+tVvEGrpRJBkStXDNLJTc+GyRmacye9/jVBzJ6Kq4bvOfAWV55I//jr61AhKPAkIOHoNIxs4PmZxhsLpt4O8k+9TvXyGRnZyLmxVG7s0mwcJ4bsOVqUQlCPPJrc828KulcgdeWX2fs+HvGrD5zHzVxlXzyd/7DWc2dVaYtNu3EYtePabmpG0ME077mHmjWdpzMwXqpgDIwiaiiCKQfWnbePXdcI7twQVoOUqvm0T2r4J88IQYiRM5MAunJkcvu0G8tuiiNLegtrXjVfTg3CgZQM+vmGgbV6HUyhzPa/WNjze+uI4WlRGlMTFQlqCgKhKOKZL992t6HmDDe/pQpjjBZx31hzLw/eDfIpjuWTOFfBdD8/xqOcNUusSpNfFcW2P4sh1xpskom3sR25vCVou2pqRErHgt3z1IFp/N3J7MwgC9vg0QkjDzuQRVAW5vYXGsbPY2TyJpx5FSiXB9zEHR7GGgvsvpZtmx5CH77pL3h6lux3fdal85xUi9+1F6evCrdaR001Bf1hrGqdQWv473AC33KgIokRH192kW7aSmTpBtTyOIIi0duwhnuyllL9EvTJFuTRCa+ceCrkLuI5JKX+ZUDhFS/suLLNKpTxGe+ddNLfvCISaHJNKeZSm1Hq27PwkplmmWhojEmmhtWMPLe27KBeHqFenmJk6RnPbDgCmxg/iWI1Zoaf5O5xq3UK6ZSu+71HInl/0PfTsKL5rE23tQ9bCSGoYq15GlOSA8lyU0BKthNIdeLaFlmjGcx2McgbftXEtHatamFuBiEoIUQ7CMK5ZR42lkZQQZnWhIBSA69qUK2MocoRovJtCYYBQKEU43EyhOEAk0oqAQDLZz/TMsbnvJYgQigT61vd9uJVtDzTxrf8xRiV3/US3IEt4loOrW/iuN7sMB0mViW1qI7atk+Khy7iGRXxHF/WhLG7dJLynl8ZEEbvcQAwpCIqEUzEwc1WsYjARiaqMqClEN7SitcWxSzpSRMMzHfTRxY2Hdq1CeeDEXPhL0sK0HnicyVe+Cj4kNu4KqFemR7EqReLrt9P+4DPkDr8UlM36LGhgE2SF9gfeT+bgC7hGnWj3xvlmoWXgGjrVy2euu81yfRkA+D5WOT8XvloKdqUYVHcttbvnok8NwxKLJ88yF5QBLzputYRdLS35mWdb171uszAzV3o8eyXUxwM5AEkLkd79ADNvPEt94nJAMxRL0vHoh1GTLXNGxbw8Pn97BQFBDqYY33EwB0eDPJHrgShgjU7hGWawEpGl+TCWbVP57qtgO1SeezWo/BwE37KpvvIO0fv3UfrGi4S2rENQ1bk+t6VQy1t8778H3+Hajnq9YJDsjqEXTVJ9ccYPZ+i7rx1bdxg7nGHzkz34rk/2YpnicIVIWqMyo+Na3oKu9bbtKZp6o9iGixq9wZTq+biVGtbQGI3TF8Hz8b2gtYKr8mh4HvqxM4iRMIkPPkHxz76G7ziIsQhiXUdQJDwzUMnz6gudH7dUxTh9ITDOALKEoCiz/+Sg1SMaQYiEEGPRoFrX81E6WzEHhgnv3U5ljasUuA1GZXr8ENPjCyuyK6URKqWFrMGWWeHs8T+be20aRUYuvbBgm5mpY8xMHVvw3sjgi3N/i6KM61pMjx+mUhqeez8/c4b8zMJJYezyywteL7XN1VDjaZRoE7ZeoVGYJNrah1UtBOErSUFLtmKUppEUDdc2MQpTxDo3Em3rpzJ2HluvEEq2YetlJEVDDsUwSxkco4YaS1MdP48oSth6OfAorr43VoVIuAW9kcMolYnHu9EbeRrFS8TjXVhmDdtpUNczhEIpdD2H77u09Yf5qV/bTLpLQw2JHP5OjkPPZm9IEWFMldDa4qTu3UD1zARSVEOOhTBndUMaIzl8x0MQRZxKwMclKjJ2Scf3fJRECDmqoaYiGNNljKkSeD5yMoyoymhtcWoXpkEQsHJVnEoDK1PBX4muvCjgey5qPIWSSM8ZaS3VhhJvwq4W0VKtiEoQA3bNBqIWhHmsUg6rnMdzbJRYArWpBUkLB31SWhgt3YYcjhFu68VzHKxSfvUUBv+HwLVMzPw0Tdv2o6U7QBDQUq3g+wtWKTgL+beuGIpr/wbwmW0XcBZzdvmz5d7+NY2pfsPAPH8Zrb8La2RyrpT8erCW0IP3HI/KlM7E0Rz997eTGyiz+ckexg5lSHZH6dnfRmmsRqNs0bW3Gc/xMMoW+aEK8fYIWkyhUbLwXA+CwAiO4czJpC+4ZtfBzuTxjcAIGOcGiezfRfSBu3HLFfTDp7AnM+B5eHUdJxeECeNPPIjv+xhnLuI1DIzTF4ns3422ZT2NY2fxylXsiZnAuFx1j+3JmfnSaEFA29SP0teFIImEtm2kce4S5oXLJD/wGE6+hHFhEKWtGUQB4/QF5PbWIKS2Rgj+CsWVfxg0z6+FIIhEYm3Ylo5l3gZG17XwpCw8AO9KC+ssIgmZPY+liDYplHMWA4fKlLOrKMddyeVevY0grKiiaKWQI3HUVCuN6dEFMfpY72aUZBq7XEAKRSgPnEBLtRLpWg+CiF0pUJ8cxncsBEki0rUeNdmMVS5QHxsg1NZNuK0HR68hSjKVobNIWohI13qUSALPtjCLMzRmxhdUMt3BQsjRBNHujciRGACOXqUxMxbkft7F5/yHEbd4KPzQYiXm4m+0UbmDO7iDO7iDdw8rMRcrDn+t0PbcwR3cwR3cwf/BuPnW7Tu4gzu4gzu4g1ncMSp3cAd3cAd3cMtwx6jcwR3cwR3cwS3DHaNyB3dwB3dwB7cMd4zKHdzBHdzBHdwy3DEqd3AHd3AHd3DLcMeo3MEd3MEd3MEtwx2jcgd3cAd3cAe3DHeMyh3cwR3cwR3cMvy/Cq9x450oJBMAAAAASUVORK5CYII=", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# Most frequent tokens for each tag\n", "tag=\"natural-language-processing\"\n", "plt.figure(figsize=(10, 3))\n", "subset = df[df.tag==tag]\n", "text = subset.title.values\n", "cloud = WordCloud(\n", " stopwords=STOPWORDS, background_color=\"black\", collocations=False,\n", " width=500, height=300).generate(\" \".join(text))\n", "plt.axis(\"off\")\n", "plt.imshow(cloud)" ] }, { "cell_type": "markdown", "metadata": { "id": "b8ua3MFhrOaX" }, "source": [ "Looks like the `title` text feature has some good signal for the respective classes and matches our intuition. We can repeat this for the `description` text feature as well. This information will become useful when we decide how to use our features for modeling." ] }, { "cell_type": "markdown", "metadata": { "id": "HFifXKl_eKsN" }, "source": [ "## โœจ Data Preprocessing" ] }, { "cell_type": "markdown", "metadata": { "id": "RxAZ1AmteRaD" }, "source": [ "Preprocessing the data via feature engineering, filtering and cleaning. Certain preprocessing steps are global (don't depend on our dataset, ex. lower casing text, removing stop words, etc.) and others are local (constructs are learned only from the training split, ex. vocabulary, standardization, etc.). For the local, dataset-dependent preprocessing steps, we want to ensure that we [split](https://madewithml.com/courses/mlops/splitting) the data first before preprocessing to avoid data leaks." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [], "source": [ "import json\n", "import nltk\n", "from nltk.corpus import stopwords\n", "from nltk.stem import PorterStemmer\n", "import re" ] }, { "cell_type": "markdown", "metadata": { "id": "6VgTwEQboTGc" }, "source": [ "### Feature engineering" ] }, { "cell_type": "markdown", "metadata": { "id": "U_001GPyMZsC" }, "source": [ "We can combine existing input features to create new meaningful signal (helping the model learn). " ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "3x1ldAFQNkSU", "tags": [] }, "outputs": [], "source": [ "# Feature engineering\n", "df[\"text\"] = df.title + \" \" + df.description" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Clean text" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "VDXLH6QeLd0F", "outputId": "2202b045-1830-477a-94ad-85e648946319", "tags": [] }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "[nltk_data] Downloading package stopwords to /Users/goku/nltk_data...\n", "[nltk_data] Package stopwords is already up-to-date!\n" ] } ], "source": [ "nltk.download(\"stopwords\")\n", "STOPWORDS = stopwords.words(\"english\")" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "VfdWkkV8LlNR", "tags": [] }, "outputs": [], "source": [ "def clean_text(text, stopwords=STOPWORDS):\n", " \"\"\"Clean raw text string.\"\"\"\n", " # Lower\n", " text = text.lower()\n", "\n", " # Remove stopwords\n", " pattern = re.compile(r'\\b(' + r\"|\".join(stopwords) + r\")\\b\\s*\")\n", " text = pattern.sub('', text)\n", "\n", " # Spacing and filters\n", " text = re.sub(r\"([!\\\"'#$%&()*\\+,-./:;<=>?@\\\\\\[\\]^_`{|}~])\", r\" \\1 \", text) # add spacing\n", " text = re.sub(\"[^A-Za-z0-9]+\", \" \", text) # remove non alphanumeric chars\n", " text = re.sub(\" +\", \" \", text) # remove multiple spaces\n", " text = text.strip() # strip white space at the ends\n", " text = re.sub(r\"http\\S+\", \"\", text) # remove links\n", " \n", " return text" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "3LRaq0_5LpE4", "outputId": "4f7beaa6-6713-4e02-80a2-22474260f406", "tags": [] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Comparison between YOLO and RCNN on real world videos Bringing theory to experiment is cool. We can easily train models in colab and find the results in minutes.\n", "comparison yolo rcnn real world videos bringing theory experiment cool easily train models colab find results minutes\n" ] } ], "source": [ "# Apply to dataframe\n", "original_df = df.copy()\n", "df.text = df.text.apply(clean_text)\n", "print (f\"{original_df.text.values[0]}\\n{df.text.values[0]}\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Clean DataFrame" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
texttag
0comparison yolo rcnn real world videos bringin...computer-vision
1show infer tell contextual inference creative ...computer-vision
2awesome graph classification collection import...other
3awesome monte carlo tree search curated list m...other
4attentionwalk pytorch implementation watch ste...other
\n", "
" ], "text/plain": [ " text tag\n", "0 comparison yolo rcnn real world videos bringin... computer-vision\n", "1 show infer tell contextual inference creative ... computer-vision\n", "2 awesome graph classification collection import... other\n", "3 awesome monte carlo tree search curated list m... other\n", "4 attentionwalk pytorch implementation watch ste... other" ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# DataFrame cleanup\n", "df = df.drop(columns=[\"id\", \"created_on\", \"title\", \"description\"], errors=\"ignore\") # drop cols\n", "df = df.dropna(subset=[\"tag\"]) # drop nulls\n", "df = df[[\"text\", \"tag\"]] # rearrange cols\n", "df.head()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Label encoding" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We need to encode our data into numerical values so that our models can process them. We'll start by encoding our text labels into unique indices." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [ { "data": { "text/plain": [ "{'mlops': 0,\n", " 'natural-language-processing': 1,\n", " 'computer-vision': 2,\n", " 'other': 3}" ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Label to index\n", "tags = train_df.tag.unique().tolist()\n", "num_classes = len(tags)\n", "class_to_index = {tag: i for i, tag in enumerate(tags)}\n", "class_to_index" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
texttag
0comparison yolo rcnn real world videos bringin...2
1show infer tell contextual inference creative ...2
2awesome graph classification collection import...3
3awesome monte carlo tree search curated list m...3
4attentionwalk pytorch implementation watch ste...3
\n", "
" ], "text/plain": [ " text tag\n", "0 comparison yolo rcnn real world videos bringin... 2\n", "1 show infer tell contextual inference creative ... 2\n", "2 awesome graph classification collection import... 3\n", "3 awesome monte carlo tree search curated list m... 3\n", "4 attentionwalk pytorch implementation watch ste... 3" ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Encode labels\n", "df[\"tag\"] = df[\"tag\"].map(class_to_index)\n", "df.head()" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [], "source": [ "def decode(indices, index_to_class):\n", " return [index_to_class[index] for index in indices]" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [ { "data": { "text/plain": [ "['computer-vision', 'computer-vision', 'other', 'other', 'other']" ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "index_to_class = {v:k for k, v in class_to_index.items()}\n", "decode(df.head()[\"tag\"].values, index_to_class=index_to_class)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Tokenizer" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Next we'll encode our text as well. Instead of using a random dictionary, we'll use a [tokenizer](https://huggingface.co/allenai/scibert_scivocab_uncased/blob/main/vocab.txt) that was used for a pretrained LLM ([scibert](https://huggingface.co/allenai/scibert_scivocab_uncased)) to tokenize our text. We'll be fine-tuning this exact model later when we train our model." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [], "source": [ "import numpy as np\n", "from transformers import BertTokenizer" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "input_ids: [[ 102 2268 1904 190 29155 168 3267 2998 205 103]]\n", "attention_mask: [[1 1 1 1 1 1 1 1 1 1]]\n", "[CLS] transfer learning with transformers for text classification. [SEP]\n" ] } ], "source": [ "# Bert tokenizer\n", "tokenizer = BertTokenizer.from_pretrained(\"allenai/scibert_scivocab_uncased\", return_dict=False)\n", "text = \"Transfer learning with transformers for text classification.\"\n", "encoded_inputs = tokenizer([text], return_tensors=\"np\", padding=\"longest\") # pad to longest item in batch\n", "print (\"input_ids:\", encoded_inputs[\"input_ids\"])\n", "print (\"attention_mask:\", encoded_inputs[\"attention_mask\"])\n", "print (tokenizer.decode(encoded_inputs[\"input_ids\"][0]))" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [], "source": [ "def tokenize(batch):\n", " tokenizer = BertTokenizer.from_pretrained(\"allenai/scibert_scivocab_uncased\", return_dict=False)\n", " encoded_inputs = tokenizer(batch[\"text\"].tolist(), return_tensors=\"np\", padding=\"longest\")\n", " return dict(ids=encoded_inputs[\"input_ids\"], masks=encoded_inputs[\"attention_mask\"], targets=np.array(batch[\"tag\"]))" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [ { "data": { "text/plain": [ "{'ids': array([[ 102, 2029, 1778, 609, 6446, 4857, 1332, 2399, 13572,\n", " 19125, 1983, 1954, 6240, 3717, 7434, 1262, 537, 201,\n", " 1040, 545, 4714, 103]]),\n", " 'masks': array([[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]]),\n", " 'targets': array([2])}" ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Tokenization\n", "tokenize(df.head(1))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We'll combine all of our preprocessing steps into function:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [], "source": [ "def preprocess(df, class_to_index):\n", " \"\"\"Preprocess the data.\"\"\"\n", " df[\"text\"] = df.title + \" \" + df.description # feature engineering\n", " df[\"text\"] = df.text.apply(clean_text) # clean text\n", " df = df.drop(columns=[\"id\", \"created_on\", \"title\", \"description\"], errors=\"ignore\") # clean dataframe\n", " df = df[[\"text\", \"tag\"]] # rearrange columns\n", " df[\"tag\"] = df[\"tag\"].map(class_to_index) # label encoding\n", " outputs = tokenize(df)\n", " return outputs" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [ { "data": { "text/plain": [ "{'ids': array([[ 102, 856, 532, ..., 0, 0, 0],\n", " [ 102, 2177, 29155, ..., 0, 0, 0],\n", " [ 102, 2180, 3241, ..., 0, 0, 0],\n", " ...,\n", " [ 102, 453, 2068, ..., 5730, 432, 103],\n", " [ 102, 11268, 1782, ..., 0, 0, 0],\n", " [ 102, 1596, 122, ..., 0, 0, 0]]),\n", " 'masks': array([[1, 1, 1, ..., 0, 0, 0],\n", " [1, 1, 1, ..., 0, 0, 0],\n", " [1, 1, 1, ..., 0, 0, 0],\n", " ...,\n", " [1, 1, 1, ..., 1, 1, 1],\n", " [1, 1, 1, ..., 0, 0, 0],\n", " [1, 1, 1, ..., 0, 0, 0]]),\n", " 'targets': array([0, 1, 1, 1, 1, 2, 1, 2, 3, 1, 2, 2, 1, 1, 2, 2, 2, 2, 1, 2, 0, 1,\n", " 1, 1, 1, 1, 2, 1, 2, 0, 3, 2, 0, 1, 1, 1, 1, 2, 1, 1, 0, 2, 3, 3,\n", " 3, 0, 2, 1, 3, 3, 1, 1, 1, 1, 2, 1, 2, 2, 2, 3, 2, 1, 1, 3, 1, 0,\n", " 1, 2, 2, 2, 3, 2, 3, 2, 3, 2, 1, 1, 3, 3, 3, 1, 1, 2, 3, 0, 1, 1,\n", " 1, 1, 3, 3, 0, 2, 3, 2, 2, 1, 1, 3, 2, 3, 1, 1, 1, 1, 2, 0, 0, 2,\n", " 1, 1, 2, 2, 1, 1, 0, 3, 1, 2, 2, 1, 0, 2, 3, 1, 3, 1, 2, 3, 1, 1,\n", " 3, 3, 2, 1, 1, 0, 1, 3, 1, 1, 2, 2, 0, 0, 2, 1, 1, 1, 2, 3, 2, 1,\n", " 1, 2, 0, 1, 1, 3, 2, 1, 1, 2, 1, 2, 3, 1, 2, 2, 1, 2, 1, 2, 1, 3,\n", " 2, 2, 0, 1, 2, 2, 1, 2, 2, 1, 3, 2, 2, 1, 2, 2, 3, 2, 1, 1, 1, 1,\n", " 2, 2, 2, 0, 2, 1, 0, 2, 1, 3, 1, 1, 1, 1, 2, 1, 3, 3, 2, 1, 0, 1,\n", " 2, 0, 2, 2, 3, 1, 1, 2, 1, 1, 1, 1, 1, 2, 1, 1, 2, 1, 2, 2, 1, 2,\n", " 0, 2, 2, 1, 1, 2, 2, 2, 2, 2, 1, 1, 2, 3, 2, 1, 0, 2, 1, 2, 2, 1,\n", " 1, 2, 1, 2, 2, 1, 1, 2, 1, 2, 2, 2, 3, 2, 1, 2, 0, 2, 2, 1, 2, 3,\n", " 2, 1, 1, 1, 1, 1, 2, 1, 1, 2, 1, 2, 3, 2, 1, 2, 2, 2, 1, 2, 2, 2,\n", " 2, 1, 1, 2, 2, 1, 2, 2, 2, 2, 1, 1, 2, 1, 2, 2, 1, 3, 3, 0, 1, 3,\n", " 0, 2, 1, 1, 1, 1, 1, 0, 2, 1, 3, 2, 1, 2, 2, 1, 1, 3, 0, 3, 3, 2,\n", " 1, 1, 3, 3, 2, 3, 1, 1, 3, 1, 0, 1, 1, 1, 3, 0, 2, 2, 2, 1, 1, 2,\n", " 2, 1, 3, 2, 0, 1, 1, 1, 2, 1, 1, 1, 2, 1, 1, 2, 0, 3, 0, 1, 2, 1,\n", " 3, 2, 3, 2, 2, 0, 2, 3, 2, 2, 2, 1, 2, 1, 1, 1, 2, 1, 1, 1, 1, 1,\n", " 2, 2, 1, 2, 1, 1, 2, 2, 3, 1, 2, 2, 3, 2, 1, 1, 2, 0, 2, 0, 1, 1,\n", " 2, 1, 1, 3, 1, 1, 1, 1, 1, 3, 1, 2, 1, 0, 3, 1, 3, 2, 2, 1, 1, 3,\n", " 2, 1, 2, 1, 3, 1, 2, 2, 1, 2, 2, 2, 1, 0, 3, 2, 1, 3, 1, 1, 2, 1,\n", " 2, 2, 0, 1, 2, 1, 2, 2, 3, 1, 1, 2, 2, 1, 2, 2, 0, 0, 1, 2, 1, 1,\n", " 2, 1, 1, 2, 1, 1, 3, 2, 3, 1, 2, 2, 3, 0, 1, 1, 2, 1, 2, 1, 1, 1,\n", " 1, 1, 2, 1, 3, 1, 0, 2, 1, 3, 1, 2, 2, 1, 0, 2, 3, 2, 3, 2, 1, 1,\n", " 1, 2, 1, 2, 1, 2, 1, 3, 2, 2, 2, 2, 2, 1, 2, 0, 1, 0, 1, 2, 2, 1,\n", " 2, 3, 2, 1, 2, 2, 2, 3, 1, 3, 2, 1, 2, 2, 2, 1, 3, 1, 1, 2, 2, 1,\n", " 2, 3, 2, 2, 0, 1, 2, 2, 2, 0, 1, 2, 1, 3, 0, 2, 3])}" ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Apply\n", "preprocess(df=train_df, class_to_index=class_to_index)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Distributed preprocessing" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The main issue with our approach above is that we're limited by our single machine in terms how much data our dataframe can hold and that we can preprocess. With the increasing trend in ML for larger unstructured datasets and larger models (LLMs), we can quickly outgrow our single machine constraints and will need to go distributed." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [], "source": [ "from madewithml.data import stratify_split\n", "ray.data.DatasetContext.get_current().execution_options.preserve_order = True" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "2023-12-07 11:26:57,951\tINFO read_api.py:406 -- To satisfy the requested parallelism of 24, each read task output is split into 24 smaller blocks.\n", "2023-12-07 11:26:57,959\tINFO dataset.py:2380 -- Tip: Use `take_batch()` instead of `take() / show()` to return records in pandas or numpy batch format.\n", "2023-12-07 11:26:57,960\tINFO streaming_executor.py:93 -- Executing DAG InputDataBuffer[Input] -> TaskPoolMapOperator[ReadCSV->SplitBlocks(24)] -> AllToAllOperator[RandomShuffle] -> LimitOperator[limit=1]\n", "2023-12-07 11:26:57,961\tINFO streaming_executor.py:94 -- Execution config: ExecutionOptions(resource_limits=ExecutionResources(cpu=None, gpu=None, object_store_memory=None), locality_with_output=False, preserve_order=True, actor_locality_enabled=True, verbose_progress=False)\n", "2023-12-07 11:26:57,961\tINFO streaming_executor.py:96 -- Tip: For detailed progress reporting, run `ray.data.DataContext.get_current().execution_options.verbose_progress = True`\n" ] }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "", "version_major": 2, "version_minor": 0 }, "text/plain": [ "- RandomShuffle 1: 0%| | 0/576 [00:00 TaskPoolMapOperator[ReadCSV->SplitBlocks(24)] -> AllToAllOperator[RandomShuffle] -> LimitOperator[limit=1]\n", "2023-12-07 11:26:59,974\tINFO streaming_executor.py:94 -- Execution config: ExecutionOptions(resource_limits=ExecutionResources(cpu=None, gpu=None, object_store_memory=None), locality_with_output=False, preserve_order=True, actor_locality_enabled=True, verbose_progress=False)\n", "2023-12-07 11:26:59,974\tINFO streaming_executor.py:96 -- Tip: For detailed progress reporting, run `ray.data.DataContext.get_current().execution_options.verbose_progress = True`\n" ] }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "", "version_major": 2, "version_minor": 0 }, "text/plain": [ "- RandomShuffle 1: 0%| | 0/576 [00:00 TaskPoolMapOperator[ReadCSV->SplitBlocks(24)] -> AllToAllOperator[RandomShuffle] -> AllToAllOperator[Sort] -> AllToAllOperator[MapBatches(group_fn)->MapBatches(_filter_split)->RandomShuffle] -> LimitOperator[limit=1]\n", "2023-12-07 11:27:00,813\tINFO streaming_executor.py:94 -- Execution config: ExecutionOptions(resource_limits=ExecutionResources(cpu=None, gpu=None, object_store_memory=None), locality_with_output=False, preserve_order=True, actor_locality_enabled=True, verbose_progress=False)\n", "2023-12-07 11:27:00,814\tINFO streaming_executor.py:96 -- Tip: For detailed progress reporting, run `ray.data.DataContext.get_current().execution_options.verbose_progress = True`\n" ] }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "", "version_major": 2, "version_minor": 0 }, "text/plain": [ "- RandomShuffle 1: 0%| | 0/576 [00:00MapBatches(_filter_split)->RandomShuffle 8: 0%| | 0/576 [00:00 TaskPoolMapOperator[ReadCSV->SplitBlocks(24)] -> AllToAllOperator[RandomShuffle] -> AllToAllOperator[Sort] -> AllToAllOperator[MapBatches(group_fn)->MapBatches(_filter_split)->RandomShuffle] -> AllToAllOperator[Aggregate] -> TaskPoolMapOperator[MapBatches()]\n", "2023-12-07 11:27:01,561\tINFO streaming_executor.py:94 -- Execution config: ExecutionOptions(resource_limits=ExecutionResources(cpu=None, gpu=None, object_store_memory=None), locality_with_output=False, preserve_order=True, actor_locality_enabled=True, verbose_progress=False)\n", "2023-12-07 11:27:01,562\tINFO streaming_executor.py:96 -- Tip: For detailed progress reporting, run `ray.data.DataContext.get_current().execution_options.verbose_progress = True`\n" ] }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "", "version_major": 2, "version_minor": 0 }, "text/plain": [ "- RandomShuffle 1: 0%| | 0/576 [00:00MapBatches(_filter_split)->RandomShuffle 8: 0%| | 0/576 [00:00 TaskPoolMapOperator[ReadCSV->SplitBlocks(24)] -> AllToAllOperator[RandomShuffle] -> AllToAllOperator[Sort] -> AllToAllOperator[MapBatches(group_fn)->MapBatches(_filter_split)->RandomShuffle] -> TaskPoolMapOperator[MapBatches(preprocess)] -> LimitOperator[limit=1]\n", "2023-12-07 11:27:02,546\tINFO streaming_executor.py:94 -- Execution config: ExecutionOptions(resource_limits=ExecutionResources(cpu=None, gpu=None, object_store_memory=None), locality_with_output=False, preserve_order=True, actor_locality_enabled=True, verbose_progress=False)\n", "2023-12-07 11:27:02,546\tINFO streaming_executor.py:96 -- Tip: For detailed progress reporting, run `ray.data.DataContext.get_current().execution_options.verbose_progress = True`\n" ] }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "", "version_major": 2, "version_minor": 0 }, "text/plain": [ "- RandomShuffle 1: 0%| | 0/576 [00:00MapBatches(_filter_split)->RandomShuffle 8: 0%| | 0/576 [00:00\n", "\u001b[2m\u001b[36m(MapBatches(preprocess) pid=21398)\u001b[0m Traceback (most recent call last):\n", "\u001b[2m\u001b[36m(MapBatches(preprocess) pid=21398)\u001b[0m File \"/Users/goku/Documents/tobias/venv/lib/python3.10/site-packages/ray/data/dataset.py\", line 5222, in __del__\n", "\u001b[2m\u001b[36m(MapBatches(preprocess) pid=21398)\u001b[0m if self._current_executor and ray is not None and ray.is_initialized():\n", "\u001b[2m\u001b[36m(MapBatches(preprocess) pid=21398)\u001b[0m KeyboardInterrupt: \n" ] } ], "source": [ "# Distributed preprocessing\n", "sample_ds = train_ds.map_batches(preprocess, fn_kwargs={\"class_to_index\": class_to_index}, batch_format=\"pandas\")\n", "sample_ds.show(1)" ] }, { "cell_type": "markdown", "metadata": { "id": "lGvI2YuuNkSX", "tags": [] }, "source": [ "# Training" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "When developing models, it's always a best practice to start with the simplest models and slowly motivate more complex models. For example our baseline model progression would be: \n", "\n", "1. random model (predict labels randomly)\n", "2. rule-based model (pattern match labels in input text)\n", "3. logistic regression (td-idf vectors from text)\n", "4. CNN (apply character filters over text)\n", "5. Fine-tune LLM (this notebook)\n", "\n", "We cover all of these methods in our [other lessons](https://madewithml.com/#foundations) but since our focus here in on MLOps, we will skip directly to fine-tuning an LLM for our task." ] }, { "cell_type": "markdown", "metadata": { "id": "K9CfUuNh2YLE" }, "source": [ "We'll first set up some functions that will help us achieve complete reproducibility." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "86sFERmsuPQl", "tags": [] }, "outputs": [], "source": [ "import os\n", "import random\n", "import torch\n", "from ray.data.preprocessor import Preprocessor" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "NXd8flJuNkSY", "tags": [] }, "outputs": [], "source": [ "def set_seeds(seed=42):\n", " \"\"\"Set seeds for reproducibility.\"\"\"\n", " np.random.seed(seed)\n", " random.seed(seed)\n", " torch.manual_seed(seed)\n", " torch.cuda.manual_seed(seed)\n", " eval(\"setattr(torch.backends.cudnn, 'deterministic', True)\")\n", " eval(\"setattr(torch.backends.cudnn, 'benchmark', False)\")\n", " os.environ[\"PYTHONHASHSEED\"] = str(seed)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [], "source": [ "def load_data(num_samples=None):\n", " ds = ray.data.read_csv(DATASET_LOC)\n", " ds = ds.random_shuffle(seed=1234)\n", " ds = ray.data.from_items(ds.take(num_samples)) if num_samples else ds\n", " return ds" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [], "source": [ "class CustomPreprocessor():\n", " \"\"\"Custom preprocessor class.\"\"\"\n", " def __init__(self, class_to_index={}):\n", " self.class_to_index = class_to_index or {} # mutable defaults\n", " self.index_to_class = {v: k for k, v in self.class_to_index.items()}\n", " \n", " def fit(self, ds):\n", " tags = ds.unique(column=\"tag\")\n", " self.class_to_index = {tag: i for i, tag in enumerate(tags)}\n", " self.index_to_class = {v:k for k, v in self.class_to_index.items()}\n", " return self\n", " \n", " def transform(self, ds):\n", " return ds.map_batches(\n", " preprocess, \n", " fn_kwargs={\"class_to_index\": self.class_to_index}, \n", " batch_format=\"pandas\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## ๐Ÿค– Model" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [], "source": [ "import torch.nn as nn\n", "from transformers import BertModel" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "Some weights of the model checkpoint at allenai/scibert_scivocab_uncased were not used when initializing BertModel: ['cls.predictions.decoder.weight', 'cls.predictions.transform.dense.bias', 'cls.predictions.bias', 'cls.seq_relationship.weight', 'cls.seq_relationship.bias', 'cls.predictions.transform.LayerNorm.weight', 'cls.predictions.transform.LayerNorm.bias', 'cls.predictions.transform.dense.weight', 'cls.predictions.decoder.bias']\n", "- This IS expected if you are initializing BertModel from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", "- This IS NOT expected if you are initializing BertModel from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" ] } ], "source": [ "# Pretrained LLM\n", "llm = BertModel.from_pretrained(\"allenai/scibert_scivocab_uncased\", return_dict=False)\n", "embedding_dim = llm.config.hidden_size" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [ { "data": { "text/plain": [ "(torch.Size([1, 10, 768]), torch.Size([1, 768]))" ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Sample\n", "text = \"Transfer learning with transformers for text classification.\"\n", "batch = tokenizer([text], return_tensors=\"np\", padding=\"longest\")\n", "batch = {k:torch.tensor(v) for k,v in batch.items()} # convert to torch tensors\n", "seq, pool = llm(input_ids=batch[\"input_ids\"], attention_mask=batch[\"attention_mask\"])\n", "np.shape(seq), np.shape(pool)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [], "source": [ "class FinetunedLLM(nn.Module):\n", " def __init__(self, llm, dropout_p, embedding_dim, num_classes):\n", " super(FinetunedLLM, self).__init__()\n", " self.llm = llm\n", " self.dropout_p = dropout_p\n", " self.embedding_dim = embedding_dim\n", " self.num_classes = num_classes\n", " self.dropout = torch.nn.Dropout(dropout_p)\n", " self.fc1 = torch.nn.Linear(embedding_dim, num_classes)\n", "\n", " def forward(self, batch):\n", " ids, masks = batch[\"ids\"], batch[\"masks\"]\n", " seq, pool = self.llm(input_ids=ids, attention_mask=masks)\n", " z = self.dropout(pool)\n", " z = self.fc1(z)\n", " return z\n", " \n", " @torch.inference_mode()\n", " def predict(self, batch):\n", " self.eval()\n", " z = self(batch)\n", " y_pred = torch.argmax(z, dim=1).cpu().numpy()\n", " return y_pred\n", " \n", " @torch.inference_mode()\n", " def predict_proba(self, batch):\n", " self.eval()\n", " z = self(batch)\n", " y_probs = F.softmax(z, dim=1).cpu().numpy()\n", " return y_probs\n", " \n", " def save(self, dp):\n", " with open(Path(dp, \"args.json\"), \"w\") as fp:\n", " contents = {\n", " \"dropout_p\": self.dropout_p,\n", " \"embedding_dim\": self.embedding_dim,\n", " \"num_classes\": self.num_classes,\n", " }\n", " json.dump(contents, fp, indent=4, sort_keys=False)\n", " torch.save(self.state_dict(), os.path.join(dp, \"model.pt\"))\n", "\n", " @classmethod\n", " def load(cls, args_fp, state_dict_fp):\n", " with open(args_fp, \"r\") as fp:\n", " kwargs = json.load(fp=fp)\n", " llm = BertModel.from_pretrained(\"allenai/scibert_scivocab_uncased\", return_dict=False)\n", " model = cls(llm=llm, **kwargs)\n", " model.load_state_dict(torch.load(state_dict_fp, map_location=torch.device(\"cpu\")))\n", " return model" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n" ] } ], "source": [ "# Initialize model\n", "model = FinetunedLLM(llm=llm, dropout_p=0.5, embedding_dim=embedding_dim, num_classes=num_classes)\n", "print (model.named_parameters)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## ๐Ÿ“ฆ Batching" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We can iterate through our dataset in batches however we may have batches of different sizes. Recall that our tokenizer padded the inputs to the longest item in the batch (`padding=\"longest\"`). However, our batches for training will be smaller than our large data processing batches and so our batches here may have inputs with different lengths. To address this, we're going to define a custom `collate_fn` to repad the items in our training batches." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Created a temporary directory at /var/folders/x8/ffj4btjx0cv6r653rgfnffsc0000gn/T/tmpfieao6lq\n", "Writing /var/folders/x8/ffj4btjx0cv6r653rgfnffsc0000gn/T/tmpfieao6lq/_remote_module_non_scriptable.py\n" ] } ], "source": [ "from ray.train.torch import get_device" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [], "source": [ "def pad_array(arr, dtype=np.int32):\n", " max_len = max(len(row) for row in arr)\n", " padded_arr = np.zeros((arr.shape[0], max_len), dtype=dtype)\n", " for i, row in enumerate(arr):\n", " padded_arr[i][:len(row)] = row\n", " return padded_arr" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [], "source": [ "def collate_fn(batch):\n", " batch[\"ids\"] = pad_array(batch[\"ids\"])\n", " batch[\"masks\"] = pad_array(batch[\"masks\"])\n", " dtypes = {\"ids\": torch.int32, \"masks\": torch.int32, \"targets\": torch.int64}\n", " tensor_batch = {}\n", " for key, array in batch.items():\n", " tensor_batch[key] = torch.as_tensor(array, dtype=dtypes[key], device=get_device())\n", " return tensor_batch" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "> `pad=(0, max_len)` in [F.pad](https://pytorch.org/docs/stable/generated/torch.nn.functional.pad.html#torch-nn-functional-pad) refers to (left_padding, right_padding) on the input. There will be no left-padding (hence the `0`) and only right-padding. And the `constant` mode refers to each element being padded to a constant size (size of longest element in the input)." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "2023-12-07 11:27:34,483\tINFO streaming_executor.py:93 -- Executing DAG InputDataBuffer[Input] -> TaskPoolMapOperator[ReadCSV->SplitBlocks(24)] -> AllToAllOperator[RandomShuffle] -> AllToAllOperator[Sort] -> AllToAllOperator[MapBatches(group_fn)->MapBatches(_filter_split)->RandomShuffle] -> TaskPoolMapOperator[MapBatches(preprocess)] -> LimitOperator[limit=128]\n", "2023-12-07 11:27:34,484\tINFO streaming_executor.py:94 -- Execution config: ExecutionOptions(resource_limits=ExecutionResources(cpu=None, gpu=None, object_store_memory=None), locality_with_output=False, preserve_order=True, actor_locality_enabled=True, verbose_progress=False)\n", "2023-12-07 11:27:34,484\tINFO streaming_executor.py:96 -- Tip: For detailed progress reporting, run `ray.data.DataContext.get_current().execution_options.verbose_progress = True`\n" ] }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "", "version_major": 2, "version_minor": 0 }, "text/plain": [ "- RandomShuffle 1: 0%| | 0/576 [00:00MapBatches(_filter_split)->RandomShuffle 8: 0%| | 0/576 [00:00 TaskPoolMapOperator[ReadCSV->SplitBlocks(64)] -> AllToAllOperator[RandomShuffle] -> LimitOperator[limit=1]\n", "2023-09-18 21:57:50,565\tINFO streaming_executor.py:94 -- Execution config: ExecutionOptions(resource_limits=ExecutionResources(cpu=None, gpu=None, object_store_memory=None), locality_with_output=False, preserve_order=True, actor_locality_enabled=True, verbose_progress=False)\n", "2023-09-18 21:57:50,566\tINFO streaming_executor.py:96 -- Tip: For detailed progress reporting, run `ray.data.DataContext.get_current().execution_options.verbose_progress = True`\n" ] }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "", "version_major": 2, "version_minor": 0 }, "text/plain": [ "- RandomShuffle 1: 0%| | 0/4096 [00:00 TaskPoolMapOperator[ReadCSV->SplitBlocks(64)] -> AllToAllOperator[RandomShuffle] -> AllToAllOperator[Sort] -> AllToAllOperator[MapBatches(group_fn)->MapBatches(_filter_split)->RandomShuffle] -> LimitOperator[limit=1]\n", "2023-09-18 21:57:51,012\tINFO streaming_executor.py:94 -- Execution config: ExecutionOptions(resource_limits=ExecutionResources(cpu=None, gpu=None, object_store_memory=None), locality_with_output=False, preserve_order=True, actor_locality_enabled=True, verbose_progress=False)\n", "2023-09-18 21:57:51,012\tINFO streaming_executor.py:96 -- Tip: For detailed progress reporting, run `ray.data.DataContext.get_current().execution_options.verbose_progress = True`\n" ] }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "", "version_major": 2, "version_minor": 0 }, "text/plain": [ "- RandomShuffle 1: 0%| | 0/4096 [00:00MapBatches(_filter_split)->RandomShuffle 8: 0%| | 0/4096 [00:00 TaskPoolMapOperator[ReadCSV->SplitBlocks(64)] -> AllToAllOperator[RandomShuffle] -> AllToAllOperator[Sort] -> AllToAllOperator[MapBatches(group_fn)->MapBatches(_filter_split)->RandomShuffle] -> AllToAllOperator[Aggregate] -> TaskPoolMapOperator[MapBatches()]\n", "2023-09-18 21:57:53,337\tINFO streaming_executor.py:94 -- Execution config: ExecutionOptions(resource_limits=ExecutionResources(cpu=None, gpu=None, object_store_memory=None), locality_with_output=False, preserve_order=True, actor_locality_enabled=True, verbose_progress=False)\n", "2023-09-18 21:57:53,338\tINFO streaming_executor.py:96 -- Tip: For detailed progress reporting, run `ray.data.DataContext.get_current().execution_options.verbose_progress = True`\n" ] }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "", "version_major": 2, "version_minor": 0 }, "text/plain": [ "- RandomShuffle 1: 0%| | 0/4096 [00:00MapBatches(_filter_split)->RandomShuffle 8: 0%| | 0/4096 [00:00 TaskPoolMapOperator[ReadCSV->SplitBlocks(64)] -> AllToAllOperator[RandomShuffle] -> AllToAllOperator[Sort] -> AllToAllOperator[MapBatches(group_fn)->MapBatches(_filter_split)->RandomShuffle] -> TaskPoolMapOperator[MapBatches(preprocess)]\n", "2023-09-18 21:57:54,977\tINFO streaming_executor.py:94 -- Execution config: ExecutionOptions(resource_limits=ExecutionResources(cpu=None, gpu=None, object_store_memory=None), locality_with_output=False, preserve_order=True, actor_locality_enabled=True, verbose_progress=False)\n", "2023-09-18 21:57:54,977\tINFO streaming_executor.py:96 -- Tip: For detailed progress reporting, run `ray.data.DataContext.get_current().execution_options.verbose_progress = True`\n" ] }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "", "version_major": 2, "version_minor": 0 }, "text/plain": [ "- RandomShuffle 1: 0%| | 0/4096 [00:00MapBatches(_filter_split)->RandomShuffle 8: 0%| | 0/4096 [00:00 TaskPoolMapOperator[ReadCSV->SplitBlocks(64)] -> AllToAllOperator[RandomShuffle] -> AllToAllOperator[Sort] -> AllToAllOperator[MapBatches(group_fn)->MapBatches(_filter_split)->RandomShuffle] -> TaskPoolMapOperator[MapBatches(preprocess)]\n", "2023-09-18 21:57:56,897\tINFO streaming_executor.py:94 -- Execution config: ExecutionOptions(resource_limits=ExecutionResources(cpu=None, gpu=None, object_store_memory=None), locality_with_output=False, preserve_order=True, actor_locality_enabled=True, verbose_progress=False)\n", "2023-09-18 21:57:56,898\tINFO streaming_executor.py:96 -- Tip: For detailed progress reporting, run `ray.data.DataContext.get_current().execution_options.verbose_progress = True`\n" ] }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "", "version_major": 2, "version_minor": 0 }, "text/plain": [ "- RandomShuffle 1: 0%| | 0/4096 [00:00MapBatches(_filter_split)->RandomShuffle 8: 0%| | 0/4096 [00:00\n", "
\n", "
\n", "

Tune Status

\n", " \n", "\n", "\n", "\n", "\n", "\n", "
Current time:2023-09-18 21:59:12
Running for: 00:01:13.39
Memory: 22.2/62.1 GiB
\n", "
\n", "
\n", "
\n", "

System Info

\n", " Using FIFO scheduling algorithm.
Logical resource usage: 4.0/32 CPUs, 1.0/2 GPUs (0.0/2.0 accelerator_type:A10G)\n", "
\n", " \n", "
\n", "
\n", "
\n", "

Trial Status

\n", " \n", "\n", "\n", "\n", "\n", "\n", "\n", "
Trial name status loc iter total time (s) epoch lr train_loss
TorchTrainer_1a81f_00000TERMINATED10.0.34.101:727773 10 68.3685 90.0001 0.0525064
\n", "
\n", "\n", "\n" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stderr", "output_type": "stream", "text": [ "\u001b[2m\u001b[36m(TorchTrainer pid=727773, ip=10.0.34.101)\u001b[0m Starting distributed worker processes: ['727834 (10.0.34.101)']\n", "\u001b[2m\u001b[36m(RayTrainWorker pid=727834, ip=10.0.34.101)\u001b[0m Setting up process group for: env:// [rank=0, world_size=1]\n", "\u001b[2m\u001b[36m(SplitCoordinator pid=727900, ip=10.0.34.101)\u001b[0m Auto configuring locality_with_output=['860a06117a62bfc12416a8163d19974a865e594674344a920da1e53b']\n", "\u001b[2m\u001b[36m(RayTrainWorker pid=727834, ip=10.0.34.101)\u001b[0m Moving model to device: cuda:0\n", "\u001b[2m\u001b[36m(SplitCoordinator pid=727900, ip=10.0.34.101)\u001b[0m Executing DAG InputDataBuffer[Input] -> OutputSplitter[split(1, equal=True)]\n", "\u001b[2m\u001b[36m(SplitCoordinator pid=727900, ip=10.0.34.101)\u001b[0m Execution config: ExecutionOptions(resource_limits=ExecutionResources(cpu=None, gpu=None, object_store_memory=None), locality_with_output=['860a06117a62bfc12416a8163d19974a865e594674344a920da1e53b'], preserve_order=True, actor_locality_enabled=True, verbose_progress=False)\n", "\u001b[2m\u001b[36m(SplitCoordinator pid=727900, ip=10.0.34.101)\u001b[0m Tip: For detailed progress reporting, run `ray.data.DataContext.get_current().execution_options.verbose_progress = True`\n" ] }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "", "version_major": 2, "version_minor": 0 }, "text/plain": [ "(pid=727900, ip=10.0.34.101) Running 0: 0%| | 0/64 [00:00 OutputSplitter[split(1, equal=True)]\n", "\u001b[2m\u001b[36m(SplitCoordinator pid=727900, ip=10.0.34.101)\u001b[0m Execution config: ExecutionOptions(resource_limits=ExecutionResources(cpu=None, gpu=None, object_store_memory=None), locality_with_output=['860a06117a62bfc12416a8163d19974a865e594674344a920da1e53b'], preserve_order=True, actor_locality_enabled=True, verbose_progress=False)\n", "\u001b[2m\u001b[36m(SplitCoordinator pid=727900, ip=10.0.34.101)\u001b[0m Tip: For detailed progress reporting, run `ray.data.DataContext.get_current().execution_options.verbose_progress = True`\n" ] }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "", "version_major": 2, "version_minor": 0 }, "text/plain": [ "(pid=727900, ip=10.0.34.101) Running 0: 0%| | 0/64 [00:00 OutputSplitter[split(1, equal=True)]\n", "\u001b[2m\u001b[36m(SplitCoordinator pid=727900, ip=10.0.34.101)\u001b[0m Execution config: ExecutionOptions(resource_limits=ExecutionResources(cpu=None, gpu=None, object_store_memory=None), locality_with_output=['860a06117a62bfc12416a8163d19974a865e594674344a920da1e53b'], preserve_order=True, actor_locality_enabled=True, verbose_progress=False)\n", "\u001b[2m\u001b[36m(SplitCoordinator pid=727900, ip=10.0.34.101)\u001b[0m Tip: For detailed progress reporting, run `ray.data.DataContext.get_current().execution_options.verbose_progress = True`\n" ] }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "", "version_major": 2, "version_minor": 0 }, "text/plain": [ "(pid=727900, ip=10.0.34.101) Running 0: 0%| | 0/64 [00:00 OutputSplitter[split(1, equal=True)]\n", "\u001b[2m\u001b[36m(SplitCoordinator pid=727900, ip=10.0.34.101)\u001b[0m Execution config: ExecutionOptions(resource_limits=ExecutionResources(cpu=None, gpu=None, object_store_memory=None), locality_with_output=['860a06117a62bfc12416a8163d19974a865e594674344a920da1e53b'], preserve_order=True, actor_locality_enabled=True, verbose_progress=False)\n", "\u001b[2m\u001b[36m(SplitCoordinator pid=727900, ip=10.0.34.101)\u001b[0m Tip: For detailed progress reporting, run `ray.data.DataContext.get_current().execution_options.verbose_progress = True`\n" ] }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "", "version_major": 2, "version_minor": 0 }, "text/plain": [ "(pid=727900, ip=10.0.34.101) Running 0: 0%| | 0/64 [00:00 OutputSplitter[split(1, equal=True)]\n", "\u001b[2m\u001b[36m(SplitCoordinator pid=727900, ip=10.0.34.101)\u001b[0m Execution config: ExecutionOptions(resource_limits=ExecutionResources(cpu=None, gpu=None, object_store_memory=None), locality_with_output=['860a06117a62bfc12416a8163d19974a865e594674344a920da1e53b'], preserve_order=True, actor_locality_enabled=True, verbose_progress=False)\n", "\u001b[2m\u001b[36m(SplitCoordinator pid=727900, ip=10.0.34.101)\u001b[0m Tip: For detailed progress reporting, run `ray.data.DataContext.get_current().execution_options.verbose_progress = True`\n" ] }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "", "version_major": 2, "version_minor": 0 }, "text/plain": [ "(pid=727900, ip=10.0.34.101) Running 0: 0%| | 0/64 [00:00 OutputSplitter[split(1, equal=True)]\n", "\u001b[2m\u001b[36m(SplitCoordinator pid=727900, ip=10.0.34.101)\u001b[0m Execution config: ExecutionOptions(resource_limits=ExecutionResources(cpu=None, gpu=None, object_store_memory=None), locality_with_output=['860a06117a62bfc12416a8163d19974a865e594674344a920da1e53b'], preserve_order=True, actor_locality_enabled=True, verbose_progress=False)\n", "\u001b[2m\u001b[36m(SplitCoordinator pid=727900, ip=10.0.34.101)\u001b[0m Tip: For detailed progress reporting, run `ray.data.DataContext.get_current().execution_options.verbose_progress = True`\n" ] }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "", "version_major": 2, "version_minor": 0 }, "text/plain": [ "(pid=727900, ip=10.0.34.101) Running 0: 0%| | 0/64 [00:00 OutputSplitter[split(1, equal=True)]\n", "\u001b[2m\u001b[36m(SplitCoordinator pid=727900, ip=10.0.34.101)\u001b[0m Execution config: ExecutionOptions(resource_limits=ExecutionResources(cpu=None, gpu=None, object_store_memory=None), locality_with_output=['860a06117a62bfc12416a8163d19974a865e594674344a920da1e53b'], preserve_order=True, actor_locality_enabled=True, verbose_progress=False)\n", "\u001b[2m\u001b[36m(SplitCoordinator pid=727900, ip=10.0.34.101)\u001b[0m Tip: For detailed progress reporting, run `ray.data.DataContext.get_current().execution_options.verbose_progress = True`\n" ] }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "", "version_major": 2, "version_minor": 0 }, "text/plain": [ "(pid=727900, ip=10.0.34.101) Running 0: 0%| | 0/64 [00:00 OutputSplitter[split(1, equal=True)]\n", "\u001b[2m\u001b[36m(SplitCoordinator pid=727900, ip=10.0.34.101)\u001b[0m Execution config: ExecutionOptions(resource_limits=ExecutionResources(cpu=None, gpu=None, object_store_memory=None), locality_with_output=['860a06117a62bfc12416a8163d19974a865e594674344a920da1e53b'], preserve_order=True, actor_locality_enabled=True, verbose_progress=False)\n", "\u001b[2m\u001b[36m(SplitCoordinator pid=727900, ip=10.0.34.101)\u001b[0m Tip: For detailed progress reporting, run `ray.data.DataContext.get_current().execution_options.verbose_progress = True`\n" ] }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "", "version_major": 2, "version_minor": 0 }, "text/plain": [ "(pid=727900, ip=10.0.34.101) Running 0: 0%| | 0/64 [00:00 OutputSplitter[split(1, equal=True)]\n", "\u001b[2m\u001b[36m(SplitCoordinator pid=727900, ip=10.0.34.101)\u001b[0m Execution config: ExecutionOptions(resource_limits=ExecutionResources(cpu=None, gpu=None, object_store_memory=None), locality_with_output=['860a06117a62bfc12416a8163d19974a865e594674344a920da1e53b'], preserve_order=True, actor_locality_enabled=True, verbose_progress=False)\n", "\u001b[2m\u001b[36m(SplitCoordinator pid=727900, ip=10.0.34.101)\u001b[0m Tip: For detailed progress reporting, run `ray.data.DataContext.get_current().execution_options.verbose_progress = True`\n" ] }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "", "version_major": 2, "version_minor": 0 }, "text/plain": [ "(pid=727900, ip=10.0.34.101) Running 0: 0%| | 0/64 [00:00 OutputSplitter[split(1, equal=True)]\n", "\u001b[2m\u001b[36m(SplitCoordinator pid=727900, ip=10.0.34.101)\u001b[0m Execution config: ExecutionOptions(resource_limits=ExecutionResources(cpu=None, gpu=None, object_store_memory=None), locality_with_output=['860a06117a62bfc12416a8163d19974a865e594674344a920da1e53b'], preserve_order=True, actor_locality_enabled=True, verbose_progress=False)\n", "\u001b[2m\u001b[36m(SplitCoordinator pid=727900, ip=10.0.34.101)\u001b[0m Tip: For detailed progress reporting, run `ray.data.DataContext.get_current().execution_options.verbose_progress = True`\n" ] }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "", "version_major": 2, "version_minor": 0 }, "text/plain": [ "(pid=727900, ip=10.0.34.101) Running 0: 0%| | 0/64 [00:00\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
epochlrtrain_lossval_losstimestampshould_checkpointdonetraining_iterationtrial_iddate...time_since_restoreiterations_since_restorecheckpoint_dir_nameconfig/train_loop_config/dropout_pconfig/train_loop_config/lrconfig/train_loop_config/lr_factorconfig/train_loop_config/lr_patienceconfig/train_loop_config/num_epochsconfig/train_loop_config/batch_sizeconfig/train_loop_config/num_classes
000.00010.5770600.4994631695099495TrueFalse11a81f_000002023-09-18_21-58-15...12.8838821checkpoint_0000000.50.00010.83102564
110.00010.4977960.4546311695099501TrueFalse21a81f_000002023-09-18_21-58-21...18.9866892checkpoint_0000010.50.00010.83102564
220.00010.4324760.3242191695099507TrueFalse31a81f_000002023-09-18_21-58-27...25.0888693checkpoint_0000020.50.00010.83102564
330.00010.3196990.2500691695099513TrueFalse41a81f_000002023-09-18_21-58-33...31.1925354checkpoint_0000030.50.00010.83102564
440.00010.2438720.2089181695099519TrueFalse51a81f_000002023-09-18_21-58-39...37.3382415checkpoint_0000040.50.00010.83102564
550.00010.1888710.1830511695099525TrueFalse61a81f_000002023-09-18_21-58-45...43.4585406checkpoint_0000050.50.00010.83102564
660.00010.1453480.1396731695099532TrueFalse71a81f_000002023-09-18_21-58-52...49.5594157checkpoint_0000060.50.00010.83102564
770.00010.1029190.1266151695099538TrueFalse81a81f_000002023-09-18_21-58-58...56.1167728checkpoint_0000070.50.00010.83102564
880.00010.0740930.1357461695099544TrueFalse91a81f_000002023-09-18_21-59-04...62.2195639checkpoint_0000080.50.00010.83102564
990.00010.0525060.1135681695099550TrueFalse101a81f_000002023-09-18_21-59-10...68.36852610checkpoint_0000090.50.00010.83102564
\n", "

10 rows ร— 25 columns

\n", "" ], "text/plain": [ " epoch lr train_loss val_loss timestamp should_checkpoint done \\\n", "0 0 0.0001 0.577060 0.499463 1695099495 True False \n", "1 1 0.0001 0.497796 0.454631 1695099501 True False \n", "2 2 0.0001 0.432476 0.324219 1695099507 True False \n", "3 3 0.0001 0.319699 0.250069 1695099513 True False \n", "4 4 0.0001 0.243872 0.208918 1695099519 True False \n", "5 5 0.0001 0.188871 0.183051 1695099525 True False \n", "6 6 0.0001 0.145348 0.139673 1695099532 True False \n", "7 7 0.0001 0.102919 0.126615 1695099538 True False \n", "8 8 0.0001 0.074093 0.135746 1695099544 True False \n", "9 9 0.0001 0.052506 0.113568 1695099550 True False \n", "\n", " training_iteration trial_id date ... \\\n", "0 1 1a81f_00000 2023-09-18_21-58-15 ... \n", "1 2 1a81f_00000 2023-09-18_21-58-21 ... \n", "2 3 1a81f_00000 2023-09-18_21-58-27 ... \n", "3 4 1a81f_00000 2023-09-18_21-58-33 ... \n", "4 5 1a81f_00000 2023-09-18_21-58-39 ... \n", "5 6 1a81f_00000 2023-09-18_21-58-45 ... \n", "6 7 1a81f_00000 2023-09-18_21-58-52 ... \n", "7 8 1a81f_00000 2023-09-18_21-58-58 ... \n", "8 9 1a81f_00000 2023-09-18_21-59-04 ... \n", "9 10 1a81f_00000 2023-09-18_21-59-10 ... \n", "\n", " time_since_restore iterations_since_restore checkpoint_dir_name \\\n", "0 12.883882 1 checkpoint_000000 \n", "1 18.986689 2 checkpoint_000001 \n", "2 25.088869 3 checkpoint_000002 \n", "3 31.192535 4 checkpoint_000003 \n", "4 37.338241 5 checkpoint_000004 \n", "5 43.458540 6 checkpoint_000005 \n", "6 49.559415 7 checkpoint_000006 \n", "7 56.116772 8 checkpoint_000007 \n", "8 62.219563 9 checkpoint_000008 \n", "9 68.368526 10 checkpoint_000009 \n", "\n", " config/train_loop_config/dropout_p config/train_loop_config/lr \\\n", "0 0.5 0.0001 \n", "1 0.5 0.0001 \n", "2 0.5 0.0001 \n", "3 0.5 0.0001 \n", "4 0.5 0.0001 \n", "5 0.5 0.0001 \n", "6 0.5 0.0001 \n", "7 0.5 0.0001 \n", "8 0.5 0.0001 \n", "9 0.5 0.0001 \n", "\n", " config/train_loop_config/lr_factor config/train_loop_config/lr_patience \\\n", "0 0.8 3 \n", "1 0.8 3 \n", "2 0.8 3 \n", "3 0.8 3 \n", "4 0.8 3 \n", "5 0.8 3 \n", "6 0.8 3 \n", "7 0.8 3 \n", "8 0.8 3 \n", "9 0.8 3 \n", "\n", " config/train_loop_config/num_epochs config/train_loop_config/batch_size \\\n", "0 10 256 \n", "1 10 256 \n", "2 10 256 \n", "3 10 256 \n", "4 10 256 \n", "5 10 256 \n", "6 10 256 \n", "7 10 256 \n", "8 10 256 \n", "9 10 256 \n", "\n", " config/train_loop_config/num_classes \n", "0 4 \n", "1 4 \n", "2 4 \n", "3 4 \n", "4 4 \n", "5 4 \n", "6 4 \n", "7 4 \n", "8 4 \n", "9 4 \n", "\n", "[10 rows x 25 columns]" ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Metrics per epoch\n", "results.metrics_dataframe" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [ { "data": { "text/plain": [ "[(Checkpoint(filesystem=local, path=/efs/shared_storage/madewithml/GokuMohandas/llm/TorchTrainer_e2941_00000_0_2023-09-17_22-40-33/checkpoint_000009),\n", " {'epoch': 9,\n", " 'lr': 0.0001,\n", " 'train_loss': 0.052506402134895325,\n", " 'val_loss': 0.11356845498085022,\n", " 'timestamp': 1695015703,\n", " 'should_checkpoint': True,\n", " 'done': False,\n", " 'training_iteration': 10,\n", " 'trial_id': 'e2941_00000',\n", " 'date': '2023-09-17_22-41-43',\n", " 'time_this_iter_s': 6.128554105758667,\n", " 'time_total_s': 66.49748063087463,\n", " 'pid': 842775,\n", " 'hostname': 'ip-10-0-35-174',\n", " 'node_ip': '10.0.35.174',\n", " 'config': {'train_loop_config': {'dropout_p': 0.5,\n", " 'lr': 0.0001,\n", " 'lr_factor': 0.8,\n", " 'lr_patience': 3,\n", " 'num_epochs': 10,\n", " 'batch_size': 256,\n", " 'num_classes': 4}},\n", " 'time_since_restore': 66.49748063087463,\n", " 'iterations_since_restore': 10})]" ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Best checkpoints\n", "results.best_checkpoints" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Evaluation" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [], "source": [ "from sklearn.metrics import precision_recall_fscore_support" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [], "source": [ "class TorchPredictor:\n", " def __init__(self, preprocessor, model):\n", " self.preprocessor = preprocessor\n", " self.model = model\n", " self.model.eval()\n", " \n", " def __call__(self, batch):\n", " results = self.model.predict(collate_fn(batch))\n", " return {\"output\": results}\n", "\n", " def predict_proba(self, batch):\n", " results = self.model.predict_proba(collate_fn(batch))\n", " return {\"output\": results}\n", " \n", " def get_preprocessor(self):\n", " return self.preprocessor\n", " \n", " @classmethod\n", " def from_checkpoint(cls, checkpoint):\n", " metadata = checkpoint.get_metadata()\n", " preprocessor = CustomPreprocessor(class_to_index=metadata[\"class_to_index\"])\n", " model = FinetunedLLM.load(Path(checkpoint.path, \"args.json\"), Path(checkpoint.path, \"model.pt\"))\n", " return cls(preprocessor=preprocessor, model=model)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [], "source": [ "# Artifacts\n", "best_checkpoint = results.best_checkpoints[0][0]\n", "predictor = TorchPredictor.from_checkpoint(best_checkpoint)\n", "preprocessor = predictor.get_preprocessor()" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "2023-09-17 22:41:46,264\tINFO read_api.py:406 -- To satisfy the requested parallelism of 64, each read task output is split into 64 smaller blocks.\n", "2023-09-17 22:41:46,268\tINFO streaming_executor.py:93 -- Executing DAG InputDataBuffer[Input] -> TaskPoolMapOperator[ReadCSV->SplitBlocks(64)] -> TaskPoolMapOperator[MapBatches(preprocess)] -> LimitOperator[limit=1]\n", "2023-09-17 22:41:46,268\tINFO streaming_executor.py:94 -- Execution config: ExecutionOptions(resource_limits=ExecutionResources(cpu=None, gpu=None, object_store_memory=None), locality_with_output=False, preserve_order=True, actor_locality_enabled=True, verbose_progress=False)\n", "2023-09-17 22:41:46,269\tINFO streaming_executor.py:96 -- Tip: For detailed progress reporting, run `ray.data.DataContext.get_current().execution_options.verbose_progress = True`\n" ] }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "", "version_major": 2, "version_minor": 0 }, "text/plain": [ "Running 0: 0%| | 0/1 [00:00 TaskPoolMapOperator[ReadCSV->SplitBlocks(64)] -> TaskPoolMapOperator[MapBatches(preprocess)->MapBatches()]\n", "2023-09-17 22:41:46,831\tINFO streaming_executor.py:94 -- Execution config: ExecutionOptions(resource_limits=ExecutionResources(cpu=None, gpu=None, object_store_memory=None), locality_with_output=False, preserve_order=True, actor_locality_enabled=True, verbose_progress=False)\n", "2023-09-17 22:41:46,832\tINFO streaming_executor.py:96 -- Tip: For detailed progress reporting, run `ray.data.DataContext.get_current().execution_options.verbose_progress = True`\n" ] }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "", "version_major": 2, "version_minor": 0 }, "text/plain": [ "Running 0: 0%| | 0/4096 [00:00 TaskPoolMapOperator[ReadCSV->SplitBlocks(64)] -> TaskPoolMapOperator[MapBatches(preprocess)->MapBatches(TorchPredictor)]\n", "2023-09-17 22:41:47,785\tINFO streaming_executor.py:94 -- Execution config: ExecutionOptions(resource_limits=ExecutionResources(cpu=None, gpu=None, object_store_memory=None), locality_with_output=False, preserve_order=True, actor_locality_enabled=True, verbose_progress=False)\n", "2023-09-17 22:41:47,786\tINFO streaming_executor.py:96 -- Tip: For detailed progress reporting, run `ray.data.DataContext.get_current().execution_options.verbose_progress = True`\n" ] }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "", "version_major": 2, "version_minor": 0 }, "text/plain": [ "Running 0: 0%| | 0/4096 [00:00MapBatches(TorchPredictor) pid=348532, ip=10.0.34.101)\u001b[0m /tmp/ipykernel_841208/1209796013.py:7: UserWarning: The given NumPy array is not writable, and PyTorch does not support non-writable tensors. This means writing to this tensor will result in undefined behavior. You may want to copy the array to protect its data or make it writable before converting it to a tensor. This type of warning will be suppressed for the rest of this program. (Triggered internally at ../torch/csrc/utils/tensor_numpy.cpp:206.)\n" ] } ], "source": [ "# y_pred\n", "predictions = preprocessed_ds.map_batches(predictor).take_all()\n", "y_pred = np.array([d[\"output\"] for d in predictions])" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [ { "data": { "text/plain": [ "{'precision': 0.9138952286238713,\n", " 'recall': 0.9109947643979057,\n", " 'f1': 0.9114851103432928}" ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Evaluate\n", "metrics = precision_recall_fscore_support(y_true, y_pred, average=\"weighted\")\n", "{\"precision\": metrics[0], \"recall\": metrics[1], \"f1\": metrics[2]}" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [], "source": [ "def evaluate(ds, predictor):\n", " # y_true\n", " preprocessor = predictor.get_preprocessor()\n", " preprocessed_ds = preprocessor.transform(ds)\n", " values = preprocessed_ds.select_columns(cols=[\"targets\"]).take_all()\n", " y_true = np.stack([item[\"targets\"] for item in values])\n", " \n", " # y_pred\n", " predictions = preprocessed_ds.map_batches(predictor).take_all()\n", " y_pred = np.array([d[\"output\"] for d in predictions])\n", "\n", " # Evaluate\n", " metrics = precision_recall_fscore_support(y_true, y_pred, average=\"weighted\")\n", " performance = {\"precision\": metrics[0], \"recall\": metrics[1], \"f1\": metrics[2]}\n", " return performance" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "2023-09-17 22:41:54,734\tINFO streaming_executor.py:93 -- Executing DAG InputDataBuffer[Input] -> TaskPoolMapOperator[ReadCSV->SplitBlocks(64)] -> TaskPoolMapOperator[MapBatches(preprocess)->MapBatches()]\n", "2023-09-17 22:41:54,734\tINFO streaming_executor.py:94 -- Execution config: ExecutionOptions(resource_limits=ExecutionResources(cpu=None, gpu=None, object_store_memory=None), locality_with_output=False, preserve_order=True, actor_locality_enabled=True, verbose_progress=False)\n", "2023-09-17 22:41:54,735\tINFO streaming_executor.py:96 -- Tip: For detailed progress reporting, run `ray.data.DataContext.get_current().execution_options.verbose_progress = True`\n" ] }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "", "version_major": 2, "version_minor": 0 }, "text/plain": [ "Running 0: 0%| | 0/4096 [00:00 TaskPoolMapOperator[ReadCSV->SplitBlocks(64)] -> TaskPoolMapOperator[MapBatches(preprocess)->MapBatches(TorchPredictor)]\n", "2023-09-17 22:41:55,456\tINFO streaming_executor.py:94 -- Execution config: ExecutionOptions(resource_limits=ExecutionResources(cpu=None, gpu=None, object_store_memory=None), locality_with_output=False, preserve_order=True, actor_locality_enabled=True, verbose_progress=False)\n", "2023-09-17 22:41:55,456\tINFO streaming_executor.py:96 -- Tip: For detailed progress reporting, run `ray.data.DataContext.get_current().execution_options.verbose_progress = True`\n" ] }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "", "version_major": 2, "version_minor": 0 }, "text/plain": [ "Running 0: 0%| | 0/4096 [00:00 TaskPoolMapOperator[MapBatches(preprocess)->MapBatches(TorchPredictor.predict_proba)]\n", "2023-09-17 22:42:00,134\tINFO streaming_executor.py:94 -- Execution config: ExecutionOptions(resource_limits=ExecutionResources(cpu=None, gpu=None, object_store_memory=None), locality_with_output=False, preserve_order=True, actor_locality_enabled=True, verbose_progress=False)\n", "2023-09-17 22:42:00,134\tINFO streaming_executor.py:96 -- Tip: For detailed progress reporting, run `ray.data.DataContext.get_current().execution_options.verbose_progress = True`\n" ] }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "", "version_major": 2, "version_minor": 0 }, "text/plain": [ "Running 0: 0%| | 0/1 [00:00 TaskPoolMapOperator[ReadCSV->SplitBlocks(64)] -> AllToAllOperator[RandomShuffle] -> LimitOperator[limit=1]\n", "2023-09-17 22:42:01,337\tINFO streaming_executor.py:94 -- Execution config: ExecutionOptions(resource_limits=ExecutionResources(cpu=None, gpu=None, object_store_memory=None), locality_with_output=False, preserve_order=True, actor_locality_enabled=True, verbose_progress=False)\n", "2023-09-17 22:42:01,338\tINFO streaming_executor.py:96 -- Tip: For detailed progress reporting, run `ray.data.DataContext.get_current().execution_options.verbose_progress = True`\n" ] }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "", "version_major": 2, "version_minor": 0 }, "text/plain": [ "- RandomShuffle 1: 0%| | 0/4096 [00:00 TaskPoolMapOperator[ReadCSV->SplitBlocks(64)] -> AllToAllOperator[RandomShuffle] -> AllToAllOperator[Sort] -> AllToAllOperator[MapBatches(group_fn)->MapBatches(_filter_split)->RandomShuffle] -> LimitOperator[limit=1]\n", "2023-09-17 22:42:01,795\tINFO streaming_executor.py:94 -- Execution config: ExecutionOptions(resource_limits=ExecutionResources(cpu=None, gpu=None, object_store_memory=None), locality_with_output=False, preserve_order=True, actor_locality_enabled=True, verbose_progress=False)\n", "2023-09-17 22:42:01,796\tINFO streaming_executor.py:96 -- Tip: For detailed progress reporting, run `ray.data.DataContext.get_current().execution_options.verbose_progress = True`\n" ] }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "", "version_major": 2, "version_minor": 0 }, "text/plain": [ "- RandomShuffle 1: 0%| | 0/4096 [00:00MapBatches(_filter_split)->RandomShuffle 8: 0%| | 0/4096 [00:00 TaskPoolMapOperator[ReadCSV->SplitBlocks(64)] -> AllToAllOperator[RandomShuffle] -> AllToAllOperator[Sort] -> AllToAllOperator[MapBatches(group_fn)->MapBatches(_filter_split)->RandomShuffle] -> AllToAllOperator[Aggregate] -> TaskPoolMapOperator[MapBatches()]\n", "2023-09-17 22:42:03,340\tINFO streaming_executor.py:94 -- Execution config: ExecutionOptions(resource_limits=ExecutionResources(cpu=None, gpu=None, object_store_memory=None), locality_with_output=False, preserve_order=True, actor_locality_enabled=True, verbose_progress=False)\n", "2023-09-17 22:42:03,340\tINFO streaming_executor.py:96 -- Tip: For detailed progress reporting, run `ray.data.DataContext.get_current().execution_options.verbose_progress = True`\n" ] }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "", "version_major": 2, "version_minor": 0 }, "text/plain": [ "- RandomShuffle 1: 0%| | 0/4096 [00:00MapBatches(_filter_split)->RandomShuffle 8: 0%| | 0/4096 [00:00 TaskPoolMapOperator[ReadCSV->SplitBlocks(64)] -> AllToAllOperator[RandomShuffle] -> AllToAllOperator[Sort] -> AllToAllOperator[MapBatches(group_fn)->MapBatches(_filter_split)->RandomShuffle] -> TaskPoolMapOperator[MapBatches(preprocess)]\n", "2023-09-17 22:42:05,079\tINFO streaming_executor.py:94 -- Execution config: ExecutionOptions(resource_limits=ExecutionResources(cpu=None, gpu=None, object_store_memory=None), locality_with_output=False, preserve_order=True, actor_locality_enabled=True, verbose_progress=False)\n", "2023-09-17 22:42:05,079\tINFO streaming_executor.py:96 -- Tip: For detailed progress reporting, run `ray.data.DataContext.get_current().execution_options.verbose_progress = True`\n" ] }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "", "version_major": 2, "version_minor": 0 }, "text/plain": [ "- RandomShuffle 1: 0%| | 0/4096 [00:00MapBatches(_filter_split)->RandomShuffle 8: 0%| | 0/4096 [00:00 TaskPoolMapOperator[ReadCSV->SplitBlocks(64)] -> AllToAllOperator[RandomShuffle] -> AllToAllOperator[Sort] -> AllToAllOperator[MapBatches(group_fn)->MapBatches(_filter_split)->RandomShuffle] -> TaskPoolMapOperator[MapBatches(preprocess)]\n", "2023-09-17 22:42:07,293\tINFO streaming_executor.py:94 -- Execution config: ExecutionOptions(resource_limits=ExecutionResources(cpu=None, gpu=None, object_store_memory=None), locality_with_output=False, preserve_order=True, actor_locality_enabled=True, verbose_progress=False)\n", "2023-09-17 22:42:07,293\tINFO streaming_executor.py:96 -- Tip: For detailed progress reporting, run `ray.data.DataContext.get_current().execution_options.verbose_progress = True`\n" ] }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "", "version_major": 2, "version_minor": 0 }, "text/plain": [ "- RandomShuffle 1: 0%| | 0/4096 [00:00MapBatches(_filter_split)->RandomShuffle 8: 0%| | 0/4096 [00:00\n", "
\n", "
\n", "

Tune Status

\n", " \n", "\n", "\n", "\n", "\n", "\n", "
Current time:2023-09-17 22:43:31
Running for: 00:01:21.68
Memory: 20.8/62.1 GiB
\n", "
\n", "
\n", "
\n", "

System Info

\n", " Using FIFO scheduling algorithm.
Logical resource usage: 4.0/32 CPUs, 1.0/2 GPUs (0.0/2.0 accelerator_type:A10G)\n", "
\n", " \n", "
\n", "
\n", "
\n", "

Trial Status

\n", " \n", "\n", "\n", "\n", "\n", "\n", "\n", "
Trial name status loc iter total time (s) epoch lr train_loss
TorchTrainer_1bd07_00000TERMINATED10.0.35.174:844750 10 63.6994 90.0001 0.0421994
\n", "
\n", "\n", "\n" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stderr", "output_type": "stream", "text": [ "\u001b[2m\u001b[36m(TorchTrainer pid=844750)\u001b[0m Starting distributed worker processes: ['844833 (10.0.35.174)']\n", "\u001b[2m\u001b[36m(RayTrainWorker pid=844833)\u001b[0m Setting up process group for: env:// [rank=0, world_size=1]\n", "\u001b[2m\u001b[36m(SplitCoordinator pid=844909)\u001b[0m Auto configuring locality_with_output=['c268d87a05f14889a7bdc3c259e78ced37c574c65ab440657e85be5a']\n", "\u001b[2m\u001b[36m(RayTrainWorker pid=844833)\u001b[0m Moving model to device: cuda:0\n" ] }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "", "version_major": 2, "version_minor": 0 }, "text/plain": [ "(pid=844909) Running 0: 0%| | 0/64 [00:00 OutputSplitter[split(1, equal=True)]\n", "\u001b[2m\u001b[36m(SplitCoordinator pid=844909)\u001b[0m Execution config: ExecutionOptions(resource_limits=ExecutionResources(cpu=None, gpu=None, object_store_memory=None), locality_with_output=['c268d87a05f14889a7bdc3c259e78ced37c574c65ab440657e85be5a'], preserve_order=True, actor_locality_enabled=True, verbose_progress=False)\n", "\u001b[2m\u001b[36m(SplitCoordinator pid=844909)\u001b[0m Tip: For detailed progress reporting, run `ray.data.DataContext.get_current().execution_options.verbose_progress = True`\n", "\u001b[2m\u001b[36m(RayTrainWorker pid=844833)\u001b[0m /tmp/ipykernel_841208/1209796013.py:7: UserWarning: The given NumPy array is not writable, and PyTorch does not support non-writable tensors. This means writing to this tensor will result in undefined behavior. You may want to copy the array to protect its data or make it writable before converting it to a tensor. This type of warning will be suppressed for the rest of this program. (Triggered internally at ../torch/csrc/utils/tensor_numpy.cpp:206.)\n", "\u001b[2m\u001b[36m(RayTrainWorker pid=844833)\u001b[0m Checkpoint successfully created at: Checkpoint(filesystem=local, path=/efs/shared_storage/madewithml/GokuMohandas/TorchTrainer_2023-09-17_22-42-09/TorchTrainer_1bd07_00000_0_2023-09-17_22-42-09/checkpoint_000000)\n" ] }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "", "version_major": 2, "version_minor": 0 }, "text/plain": [ "(pid=844909) Running 0: 0%| | 0/64 [00:00 OutputSplitter[split(1, equal=True)]\n", "\u001b[2m\u001b[36m(SplitCoordinator pid=844909)\u001b[0m Execution config: ExecutionOptions(resource_limits=ExecutionResources(cpu=None, gpu=None, object_store_memory=None), locality_with_output=['c268d87a05f14889a7bdc3c259e78ced37c574c65ab440657e85be5a'], preserve_order=True, actor_locality_enabled=True, verbose_progress=False)\n", "\u001b[2m\u001b[36m(SplitCoordinator pid=844909)\u001b[0m Tip: For detailed progress reporting, run `ray.data.DataContext.get_current().execution_options.verbose_progress = True`\n", "\u001b[2m\u001b[36m(RayTrainWorker pid=844833)\u001b[0m Checkpoint successfully created at: Checkpoint(filesystem=local, path=/efs/shared_storage/madewithml/GokuMohandas/TorchTrainer_2023-09-17_22-42-09/TorchTrainer_1bd07_00000_0_2023-09-17_22-42-09/checkpoint_000001)\n" ] }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "", "version_major": 2, "version_minor": 0 }, "text/plain": [ "(pid=844909) Running 0: 0%| | 0/64 [00:00 OutputSplitter[split(1, equal=True)]\n", "\u001b[2m\u001b[36m(SplitCoordinator pid=844909)\u001b[0m Execution config: ExecutionOptions(resource_limits=ExecutionResources(cpu=None, gpu=None, object_store_memory=None), locality_with_output=['c268d87a05f14889a7bdc3c259e78ced37c574c65ab440657e85be5a'], preserve_order=True, actor_locality_enabled=True, verbose_progress=False)\n", "\u001b[2m\u001b[36m(SplitCoordinator pid=844909)\u001b[0m Tip: For detailed progress reporting, run `ray.data.DataContext.get_current().execution_options.verbose_progress = True`\n", "\u001b[2m\u001b[36m(RayTrainWorker pid=844833)\u001b[0m Checkpoint successfully created at: Checkpoint(filesystem=local, path=/efs/shared_storage/madewithml/GokuMohandas/TorchTrainer_2023-09-17_22-42-09/TorchTrainer_1bd07_00000_0_2023-09-17_22-42-09/checkpoint_000002)\n" ] }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "", "version_major": 2, "version_minor": 0 }, "text/plain": [ "(pid=844909) Running 0: 0%| | 0/64 [00:00 OutputSplitter[split(1, equal=True)]\n", "\u001b[2m\u001b[36m(SplitCoordinator pid=844909)\u001b[0m Execution config: ExecutionOptions(resource_limits=ExecutionResources(cpu=None, gpu=None, object_store_memory=None), locality_with_output=['c268d87a05f14889a7bdc3c259e78ced37c574c65ab440657e85be5a'], preserve_order=True, actor_locality_enabled=True, verbose_progress=False)\n", "\u001b[2m\u001b[36m(SplitCoordinator pid=844909)\u001b[0m Tip: For detailed progress reporting, run `ray.data.DataContext.get_current().execution_options.verbose_progress = True`\n", "\u001b[2m\u001b[36m(RayTrainWorker pid=844833)\u001b[0m Checkpoint successfully created at: Checkpoint(filesystem=local, path=/efs/shared_storage/madewithml/GokuMohandas/TorchTrainer_2023-09-17_22-42-09/TorchTrainer_1bd07_00000_0_2023-09-17_22-42-09/checkpoint_000003)\n" ] }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "", "version_major": 2, "version_minor": 0 }, "text/plain": [ "(pid=844909) Running 0: 0%| | 0/64 [00:00 OutputSplitter[split(1, equal=True)]\n", "\u001b[2m\u001b[36m(SplitCoordinator pid=844909)\u001b[0m Execution config: ExecutionOptions(resource_limits=ExecutionResources(cpu=None, gpu=None, object_store_memory=None), locality_with_output=['c268d87a05f14889a7bdc3c259e78ced37c574c65ab440657e85be5a'], preserve_order=True, actor_locality_enabled=True, verbose_progress=False)\n", "\u001b[2m\u001b[36m(SplitCoordinator pid=844909)\u001b[0m Tip: For detailed progress reporting, run `ray.data.DataContext.get_current().execution_options.verbose_progress = True`\n", "\u001b[2m\u001b[36m(RayTrainWorker pid=844833)\u001b[0m Checkpoint successfully created at: Checkpoint(filesystem=local, path=/efs/shared_storage/madewithml/GokuMohandas/TorchTrainer_2023-09-17_22-42-09/TorchTrainer_1bd07_00000_0_2023-09-17_22-42-09/checkpoint_000004)\n" ] }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "", "version_major": 2, "version_minor": 0 }, "text/plain": [ "(pid=844909) Running 0: 0%| | 0/64 [00:00 OutputSplitter[split(1, equal=True)]\n", "\u001b[2m\u001b[36m(SplitCoordinator pid=844909)\u001b[0m Execution config: ExecutionOptions(resource_limits=ExecutionResources(cpu=None, gpu=None, object_store_memory=None), locality_with_output=['c268d87a05f14889a7bdc3c259e78ced37c574c65ab440657e85be5a'], preserve_order=True, actor_locality_enabled=True, verbose_progress=False)\n", "\u001b[2m\u001b[36m(SplitCoordinator pid=844909)\u001b[0m Tip: For detailed progress reporting, run `ray.data.DataContext.get_current().execution_options.verbose_progress = True`\n", "\u001b[2m\u001b[36m(RayTrainWorker pid=844833)\u001b[0m Checkpoint successfully created at: Checkpoint(filesystem=local, path=/efs/shared_storage/madewithml/GokuMohandas/TorchTrainer_2023-09-17_22-42-09/TorchTrainer_1bd07_00000_0_2023-09-17_22-42-09/checkpoint_000005)\n" ] }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "", "version_major": 2, "version_minor": 0 }, "text/plain": [ "(pid=844909) Running 0: 0%| | 0/64 [00:00 OutputSplitter[split(1, equal=True)]\n", "\u001b[2m\u001b[36m(SplitCoordinator pid=844909)\u001b[0m Execution config: ExecutionOptions(resource_limits=ExecutionResources(cpu=None, gpu=None, object_store_memory=None), locality_with_output=['c268d87a05f14889a7bdc3c259e78ced37c574c65ab440657e85be5a'], preserve_order=True, actor_locality_enabled=True, verbose_progress=False)\n", "\u001b[2m\u001b[36m(SplitCoordinator pid=844909)\u001b[0m Tip: For detailed progress reporting, run `ray.data.DataContext.get_current().execution_options.verbose_progress = True`\n", "\u001b[2m\u001b[36m(RayTrainWorker pid=844833)\u001b[0m Checkpoint successfully created at: Checkpoint(filesystem=local, path=/efs/shared_storage/madewithml/GokuMohandas/TorchTrainer_2023-09-17_22-42-09/TorchTrainer_1bd07_00000_0_2023-09-17_22-42-09/checkpoint_000006)\n" ] }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "", "version_major": 2, "version_minor": 0 }, "text/plain": [ "(pid=844909) Running 0: 0%| | 0/64 [00:00 OutputSplitter[split(1, equal=True)]\n", "\u001b[2m\u001b[36m(SplitCoordinator pid=844909)\u001b[0m Execution config: ExecutionOptions(resource_limits=ExecutionResources(cpu=None, gpu=None, object_store_memory=None), locality_with_output=['c268d87a05f14889a7bdc3c259e78ced37c574c65ab440657e85be5a'], preserve_order=True, actor_locality_enabled=True, verbose_progress=False)\n", "\u001b[2m\u001b[36m(SplitCoordinator pid=844909)\u001b[0m Tip: For detailed progress reporting, run `ray.data.DataContext.get_current().execution_options.verbose_progress = True`\n", "2023-09-17 22:43:01,995\tWARNING worker.py:2012 -- [tqdm_ray] Failed to decode {\"__magic_token__\": \"__ray_tqdm_magic_token__\", \"x\": 47, \"pos\": 0, \"desc\": \"Running: 0.0/32.0 CPU, 0.0/2.0 GPU, 0.28 MiB/8.95 GiB object_store_memory\", \"total\": 64, \"ip\": \"10.0.35.174\", \"pid\": 844909, \"uuid\": \"54df93474628434c, this may be due to logging too fast. This warning will not be printed again.\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "\u001b[2m\u001b[36m(SplitCoordinator pid=844909)\u001b[0m 8d2fe8f67788261f\", \"closed\": false}\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "\u001b[2m\u001b[36m(RayTrainWorker pid=844833)\u001b[0m Checkpoint successfully created at: Checkpoint(filesystem=local, path=/efs/shared_storage/madewithml/GokuMohandas/TorchTrainer_2023-09-17_22-42-09/TorchTrainer_1bd07_00000_0_2023-09-17_22-42-09/checkpoint_000007)\n" ] }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "", "version_major": 2, "version_minor": 0 }, "text/plain": [ "(pid=844909) Running 0: 0%| | 0/64 [00:00 OutputSplitter[split(1, equal=True)]\n", "\u001b[2m\u001b[36m(SplitCoordinator pid=844909)\u001b[0m Execution config: ExecutionOptions(resource_limits=ExecutionResources(cpu=None, gpu=None, object_store_memory=None), locality_with_output=['c268d87a05f14889a7bdc3c259e78ced37c574c65ab440657e85be5a'], preserve_order=True, actor_locality_enabled=True, verbose_progress=False)\n", "\u001b[2m\u001b[36m(SplitCoordinator pid=844909)\u001b[0m Tip: For detailed progress reporting, run `ray.data.DataContext.get_current().execution_options.verbose_progress = True`\n", "\u001b[2m\u001b[36m(RayTrainWorker pid=844833)\u001b[0m Checkpoint successfully created at: Checkpoint(filesystem=local, path=/efs/shared_storage/madewithml/GokuMohandas/TorchTrainer_2023-09-17_22-42-09/TorchTrainer_1bd07_00000_0_2023-09-17_22-42-09/checkpoint_000008)\n" ] }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "", "version_major": 2, "version_minor": 0 }, "text/plain": [ "(pid=844909) Running 0: 0%| | 0/64 [00:00 OutputSplitter[split(1, equal=True)]\n", "\u001b[2m\u001b[36m(SplitCoordinator pid=844909)\u001b[0m Execution config: ExecutionOptions(resource_limits=ExecutionResources(cpu=None, gpu=None, object_store_memory=None), locality_with_output=['c268d87a05f14889a7bdc3c259e78ced37c574c65ab440657e85be5a'], preserve_order=True, actor_locality_enabled=True, verbose_progress=False)\n", "\u001b[2m\u001b[36m(SplitCoordinator pid=844909)\u001b[0m Tip: For detailed progress reporting, run `ray.data.DataContext.get_current().execution_options.verbose_progress = True`\n", "\u001b[2m\u001b[36m(RayTrainWorker pid=844833)\u001b[0m Checkpoint successfully created at: Checkpoint(filesystem=local, path=/efs/shared_storage/madewithml/GokuMohandas/TorchTrainer_2023-09-17_22-42-09/TorchTrainer_1bd07_00000_0_2023-09-17_22-42-09/checkpoint_000009)\n", "2023-09-17 22:43:31,004\tINFO tune.py:1143 -- Total run time: 81.76 seconds (81.66 seconds for the tuning loop).\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "CPU times: user 1.23 s, sys: 1.31 s, total: 2.53 s\n", "Wall time: 1min 21s\n" ] } ], "source": [ "%%time\n", "# Train\n", "results = trainer.fit()" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
epochlrtrain_lossval_losstimestampshould_checkpointdonetraining_iterationtrial_iddate...time_since_restoreiterations_since_restorecheckpoint_dir_nameconfig/train_loop_config/dropout_pconfig/train_loop_config/lrconfig/train_loop_config/lr_factorconfig/train_loop_config/lr_patienceconfig/train_loop_config/num_epochsconfig/train_loop_config/batch_sizeconfig/train_loop_config/num_classes
000.00010.5777800.4931021695015745TrueFalse11bd07_000002023-09-17_22-42-25...11.8825751checkpoint_0000000.50.00010.83102564
110.00010.4896530.4319581695015751TrueFalse21bd07_000002023-09-17_22-42-31...17.6544352checkpoint_0000010.50.00010.83102564
220.00010.3988240.3062011695015757TrueFalse31bd07_000002023-09-17_22-42-37...23.3537523checkpoint_0000020.50.00010.83102564
330.00010.3005130.2388031695015763TrueFalse41bd07_000002023-09-17_22-42-43...29.1189924checkpoint_0000030.50.00010.83102564
440.00010.2206080.1744111695015769TrueFalse51bd07_000002023-09-17_22-42-49...34.8974855checkpoint_0000040.50.00010.83102564
550.00010.1514990.1586481695015775TrueFalse61bd07_000002023-09-17_22-42-55...40.6453846checkpoint_0000050.50.00010.83102564
660.00010.1050870.1128291695015781TrueFalse71bd07_000002023-09-17_22-43-01...46.3875687checkpoint_0000060.50.00010.83102564
770.00010.0773850.0919221695015788TrueFalse81bd07_000002023-09-17_22-43-08...52.1533438checkpoint_0000070.50.00010.83102564
880.00010.0538570.1098101695015794TrueFalse91bd07_000002023-09-17_22-43-14...57.9010849checkpoint_0000080.50.00010.83102564
990.00010.0421990.1211221695015800TrueFalse101bd07_000002023-09-17_22-43-20...63.69944410checkpoint_0000090.50.00010.83102564
\n", "

10 rows ร— 25 columns

\n", "
" ], "text/plain": [ " epoch lr train_loss val_loss timestamp should_checkpoint done \\\n", "0 0 0.0001 0.577780 0.493102 1695015745 True False \n", "1 1 0.0001 0.489653 0.431958 1695015751 True False \n", "2 2 0.0001 0.398824 0.306201 1695015757 True False \n", "3 3 0.0001 0.300513 0.238803 1695015763 True False \n", "4 4 0.0001 0.220608 0.174411 1695015769 True False \n", "5 5 0.0001 0.151499 0.158648 1695015775 True False \n", "6 6 0.0001 0.105087 0.112829 1695015781 True False \n", "7 7 0.0001 0.077385 0.091922 1695015788 True False \n", "8 8 0.0001 0.053857 0.109810 1695015794 True False \n", "9 9 0.0001 0.042199 0.121122 1695015800 True False \n", "\n", " training_iteration trial_id date ... \\\n", "0 1 1bd07_00000 2023-09-17_22-42-25 ... \n", "1 2 1bd07_00000 2023-09-17_22-42-31 ... \n", "2 3 1bd07_00000 2023-09-17_22-42-37 ... \n", "3 4 1bd07_00000 2023-09-17_22-42-43 ... \n", "4 5 1bd07_00000 2023-09-17_22-42-49 ... \n", "5 6 1bd07_00000 2023-09-17_22-42-55 ... \n", "6 7 1bd07_00000 2023-09-17_22-43-01 ... \n", "7 8 1bd07_00000 2023-09-17_22-43-08 ... \n", "8 9 1bd07_00000 2023-09-17_22-43-14 ... \n", "9 10 1bd07_00000 2023-09-17_22-43-20 ... \n", "\n", " time_since_restore iterations_since_restore checkpoint_dir_name \\\n", "0 11.882575 1 checkpoint_000000 \n", "1 17.654435 2 checkpoint_000001 \n", "2 23.353752 3 checkpoint_000002 \n", "3 29.118992 4 checkpoint_000003 \n", "4 34.897485 5 checkpoint_000004 \n", "5 40.645384 6 checkpoint_000005 \n", "6 46.387568 7 checkpoint_000006 \n", "7 52.153343 8 checkpoint_000007 \n", "8 57.901084 9 checkpoint_000008 \n", "9 63.699444 10 checkpoint_000009 \n", "\n", " config/train_loop_config/dropout_p config/train_loop_config/lr \\\n", "0 0.5 0.0001 \n", "1 0.5 0.0001 \n", "2 0.5 0.0001 \n", "3 0.5 0.0001 \n", "4 0.5 0.0001 \n", "5 0.5 0.0001 \n", "6 0.5 0.0001 \n", "7 0.5 0.0001 \n", "8 0.5 0.0001 \n", "9 0.5 0.0001 \n", "\n", " config/train_loop_config/lr_factor config/train_loop_config/lr_patience \\\n", "0 0.8 3 \n", "1 0.8 3 \n", "2 0.8 3 \n", "3 0.8 3 \n", "4 0.8 3 \n", "5 0.8 3 \n", "6 0.8 3 \n", "7 0.8 3 \n", "8 0.8 3 \n", "9 0.8 3 \n", "\n", " config/train_loop_config/num_epochs config/train_loop_config/batch_size \\\n", "0 10 256 \n", "1 10 256 \n", "2 10 256 \n", "3 10 256 \n", "4 10 256 \n", "5 10 256 \n", "6 10 256 \n", "7 10 256 \n", "8 10 256 \n", "9 10 256 \n", "\n", " config/train_loop_config/num_classes \n", "0 4 \n", "1 4 \n", "2 4 \n", "3 4 \n", "4 4 \n", "5 4 \n", "6 4 \n", "7 4 \n", "8 4 \n", "9 4 \n", "\n", "[10 rows x 25 columns]" ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "results.metrics_dataframe" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
run_idexperiment_idstatusartifact_uristart_timeend_timemetrics.config/train_loop_config/lr_patiencemetrics.time_total_smetrics.time_since_restoremetrics.config/train_loop_config/num_epochs...metrics.epochparams.train_loop_config/num_epochsparams.train_loop_config/lr_patienceparams.train_loop_config/batch_sizeparams.train_loop_config/lrparams.train_loop_config/lr_factorparams.train_loop_config/num_classesparams.train_loop_config/dropout_ptags.trial_nametags.mlflow.runName
0a67f6ec4e10b49a7845b91db61efc7e0703905858072772764FINISHEDfile:///efs/shared_storage/madewithml/GokuMoha...2023-09-18 05:42:12.863000+00:002023-09-18 05:43:30.925000+00:003.063.69944463.69944410.0...9.01032560.00010.840.5TorchTrainer_1bd07_00000TorchTrainer_1bd07_00000
\n", "

1 rows ร— 35 columns

\n", "
" ], "text/plain": [ " run_id experiment_id status \\\n", "0 a67f6ec4e10b49a7845b91db61efc7e0 703905858072772764 FINISHED \n", "\n", " artifact_uri \\\n", "0 file:///efs/shared_storage/madewithml/GokuMoha... \n", "\n", " start_time end_time \\\n", "0 2023-09-18 05:42:12.863000+00:00 2023-09-18 05:43:30.925000+00:00 \n", "\n", " metrics.config/train_loop_config/lr_patience metrics.time_total_s \\\n", "0 3.0 63.699444 \n", "\n", " metrics.time_since_restore metrics.config/train_loop_config/num_epochs \\\n", "0 63.699444 10.0 \n", "\n", " ... metrics.epoch params.train_loop_config/num_epochs \\\n", "0 ... 9.0 10 \n", "\n", " params.train_loop_config/lr_patience params.train_loop_config/batch_size \\\n", "0 3 256 \n", "\n", " params.train_loop_config/lr params.train_loop_config/lr_factor \\\n", "0 0.0001 0.8 \n", "\n", " params.train_loop_config/num_classes params.train_loop_config/dropout_p \\\n", "0 4 0.5 \n", "\n", " tags.trial_name tags.mlflow.runName \n", "0 TorchTrainer_1bd07_00000 TorchTrainer_1bd07_00000 \n", "\n", "[1 rows x 35 columns]" ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Sorted runs\n", "sorted_runs = mlflow.search_runs(experiment_names=[experiment_name], order_by=[\"metrics.val_loss ASC\"])\n", "sorted_runs" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [ { "data": { "text/plain": [ "run_id a67f6ec4e10b49a7845b91db61efc7e0\n", "experiment_id 703905858072772764\n", "status FINISHED\n", "artifact_uri file:///efs/shared_storage/madewithml/GokuMoha...\n", "start_time 2023-09-18 05:42:12.863000+00:00\n", "end_time 2023-09-18 05:43:30.925000+00:00\n", "metrics.config/train_loop_config/lr_patience 3.0\n", "metrics.time_total_s 63.699444\n", "metrics.time_since_restore 63.699444\n", "metrics.config/train_loop_config/num_epochs 10.0\n", "metrics.iterations_since_restore 10.0\n", "metrics.config/train_loop_config/batch_size 256.0\n", "metrics.pid 844750.0\n", "metrics.should_checkpoint 1.0\n", "metrics.timestamp 1695015800.0\n", "metrics.config/train_loop_config/num_classes 4.0\n", "metrics.done 0.0\n", "metrics.time_this_iter_s 5.79836\n", "metrics.train_loss 0.042199\n", "metrics.config/train_loop_config/lr 0.0001\n", "metrics.lr 0.0001\n", "metrics.val_loss 0.121122\n", "metrics.config/train_loop_config/dropout_p 0.5\n", "metrics.training_iteration 10.0\n", "metrics.config/train_loop_config/lr_factor 0.8\n", "metrics.epoch 9.0\n", "params.train_loop_config/num_epochs 10\n", "params.train_loop_config/lr_patience 3\n", "params.train_loop_config/batch_size 256\n", "params.train_loop_config/lr 0.0001\n", "params.train_loop_config/lr_factor 0.8\n", "params.train_loop_config/num_classes 4\n", "params.train_loop_config/dropout_p 0.5\n", "tags.trial_name TorchTrainer_1bd07_00000\n", "tags.mlflow.runName TorchTrainer_1bd07_00000\n", "Name: 0, dtype: object" ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Best run\n", "best_run = sorted_runs.iloc[0]\n", "best_run" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Dashboard" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Let's view what we've tracked from our experiment. MLFlow serves a dashboard for us to view and explore our experiments on a localhost port:\n", "\n", "```bash\n", "mlflow server -h 0.0.0.0 -p 8080 --backend-store-uri $EFS_DIR/mlflow\n", "```" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "MLFlow creates a main dashboard with all your experiments and their respective runs. We can sort runs by clicking on the column headers.\n", "\n", "\"mlflow\n", "\n", "And within each run, we can view metrics, parameters, artifacts, etc.\n", "\n", "\"mlflow\n", "\n", "And we can even create custom plots to help us visualize our results.\n", "\n", "\"mlflow" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Loading" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [], "source": [ "from ray.train import Result\n", "from urllib.parse import urlparse" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [], "source": [ "def get_best_checkpoint(run_id):\n", " artifact_dir = urlparse(mlflow.get_run(run_id).info.artifact_uri).path # get path from mlflow\n", " results = Result.from_path(artifact_dir)\n", " return results.best_checkpoints[0][0]" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# Artifacts\n", "best_checkpoint = get_best_checkpoint(run_id=best_run.run_id)\n", "predictor = TorchPredictor.from_checkpoint(best_checkpoint)\n", "preprocessor = predictor.get_preprocessor()" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "2023-09-17 22:43:36,691\tINFO streaming_executor.py:93 -- Executing DAG InputDataBuffer[Input] -> TaskPoolMapOperator[ReadCSV->SplitBlocks(64)] -> TaskPoolMapOperator[MapBatches(preprocess)->MapBatches()]\n", "2023-09-17 22:43:36,692\tINFO streaming_executor.py:94 -- Execution config: ExecutionOptions(resource_limits=ExecutionResources(cpu=None, gpu=None, object_store_memory=None), locality_with_output=False, preserve_order=True, actor_locality_enabled=True, verbose_progress=False)\n", "2023-09-17 22:43:36,692\tINFO streaming_executor.py:96 -- Tip: For detailed progress reporting, run `ray.data.DataContext.get_current().execution_options.verbose_progress = True`\n" ] }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "", "version_major": 2, "version_minor": 0 }, "text/plain": [ "Running 0: 0%| | 0/4096 [00:00 TaskPoolMapOperator[ReadCSV->SplitBlocks(64)] -> TaskPoolMapOperator[MapBatches(preprocess)->MapBatches(TorchPredictor)]\n", "2023-09-17 22:43:37,552\tINFO streaming_executor.py:94 -- Execution config: ExecutionOptions(resource_limits=ExecutionResources(cpu=None, gpu=None, object_store_memory=None), locality_with_output=False, preserve_order=True, actor_locality_enabled=True, verbose_progress=False)\n", "2023-09-17 22:43:37,553\tINFO streaming_executor.py:96 -- Tip: For detailed progress reporting, run `ray.data.DataContext.get_current().execution_options.verbose_progress = True`\n" ] }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "", "version_major": 2, "version_minor": 0 }, "text/plain": [ "Running 0: 0%| | 0/4096 [00:00MapBatches(TorchPredictor) pid=845827)\u001b[0m /tmp/ipykernel_841208/1209796013.py:7: UserWarning: The given NumPy array is not writable, and PyTorch does not support non-writable tensors. This means writing to this tensor will result in undefined behavior. You may want to copy the array to protect its data or make it writable before converting it to a tensor. This type of warning will be suppressed for the rest of this program. (Triggered internally at ../torch/csrc/utils/tensor_numpy.cpp:206.)\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "{\n", " \"precision\": 0.9168092951711627,\n", " \"recall\": 0.9109947643979057,\n", " \"f1\": 0.9105512639658029\n", "}\n" ] } ], "source": [ "# Evaluate on test split\n", "performance = evaluate(ds=test_ds, predictor=predictor)\n", "print (json.dumps(performance, indent=2))" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "2023-09-17 22:43:43,282\tINFO streaming_executor.py:93 -- Executing DAG InputDataBuffer[Input] -> TaskPoolMapOperator[MapBatches(preprocess)->MapBatches(TorchPredictor.predict_proba)]\n", "2023-09-17 22:43:43,282\tINFO streaming_executor.py:94 -- Execution config: ExecutionOptions(resource_limits=ExecutionResources(cpu=None, gpu=None, object_store_memory=None), locality_with_output=False, preserve_order=True, actor_locality_enabled=True, verbose_progress=False)\n", "2023-09-17 22:43:43,283\tINFO streaming_executor.py:96 -- Tip: For detailed progress reporting, run `ray.data.DataContext.get_current().execution_options.verbose_progress = True`\n" ] }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "", "version_major": 2, "version_minor": 0 }, "text/plain": [ "Running 0: 0%| | 0/1 [00:00 TaskPoolMapOperator[ReadCSV->SplitBlocks(64)] -> AllToAllOperator[RandomShuffle] -> LimitOperator[limit=1]\n", "2023-09-17 22:43:44,427\tINFO streaming_executor.py:94 -- Execution config: ExecutionOptions(resource_limits=ExecutionResources(cpu=None, gpu=None, object_store_memory=None), locality_with_output=False, preserve_order=True, actor_locality_enabled=True, verbose_progress=False)\n", "2023-09-17 22:43:44,427\tINFO streaming_executor.py:96 -- Tip: For detailed progress reporting, run `ray.data.DataContext.get_current().execution_options.verbose_progress = True`\n" ] }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "", "version_major": 2, "version_minor": 0 }, "text/plain": [ "- RandomShuffle 1: 0%| | 0/4096 [00:00 TaskPoolMapOperator[ReadCSV->SplitBlocks(64)] -> AllToAllOperator[RandomShuffle] -> AllToAllOperator[Sort] -> AllToAllOperator[MapBatches(group_fn)->MapBatches(_filter_split)->RandomShuffle] -> LimitOperator[limit=1]\n", "2023-09-17 22:43:44,886\tINFO streaming_executor.py:94 -- Execution config: ExecutionOptions(resource_limits=ExecutionResources(cpu=None, gpu=None, object_store_memory=None), locality_with_output=False, preserve_order=True, actor_locality_enabled=True, verbose_progress=False)\n", "2023-09-17 22:43:44,887\tINFO streaming_executor.py:96 -- Tip: For detailed progress reporting, run `ray.data.DataContext.get_current().execution_options.verbose_progress = True`\n" ] }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "", "version_major": 2, "version_minor": 0 }, "text/plain": [ "- RandomShuffle 1: 0%| | 0/4096 [00:00MapBatches(_filter_split)->RandomShuffle 8: 0%| | 0/4096 [00:00 TaskPoolMapOperator[ReadCSV->SplitBlocks(64)] -> AllToAllOperator[RandomShuffle] -> AllToAllOperator[Sort] -> AllToAllOperator[MapBatches(group_fn)->MapBatches(_filter_split)->RandomShuffle] -> AllToAllOperator[Aggregate] -> TaskPoolMapOperator[MapBatches()]\n", "2023-09-17 22:43:46,123\tINFO streaming_executor.py:94 -- Execution config: ExecutionOptions(resource_limits=ExecutionResources(cpu=None, gpu=None, object_store_memory=None), locality_with_output=False, preserve_order=True, actor_locality_enabled=True, verbose_progress=False)\n", "2023-09-17 22:43:46,123\tINFO streaming_executor.py:96 -- Tip: For detailed progress reporting, run `ray.data.DataContext.get_current().execution_options.verbose_progress = True`\n" ] }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "", "version_major": 2, "version_minor": 0 }, "text/plain": [ "- RandomShuffle 1: 0%| | 0/4096 [00:00MapBatches(_filter_split)->RandomShuffle 8: 0%| | 0/4096 [00:00 TaskPoolMapOperator[ReadCSV->SplitBlocks(64)] -> AllToAllOperator[RandomShuffle] -> AllToAllOperator[Sort] -> AllToAllOperator[MapBatches(group_fn)->MapBatches(_filter_split)->RandomShuffle] -> TaskPoolMapOperator[MapBatches(preprocess)]\n", "2023-09-17 22:43:47,914\tINFO streaming_executor.py:94 -- Execution config: ExecutionOptions(resource_limits=ExecutionResources(cpu=None, gpu=None, object_store_memory=None), locality_with_output=False, preserve_order=True, actor_locality_enabled=True, verbose_progress=False)\n", "2023-09-17 22:43:47,914\tINFO streaming_executor.py:96 -- Tip: For detailed progress reporting, run `ray.data.DataContext.get_current().execution_options.verbose_progress = True`\n" ] }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "", "version_major": 2, "version_minor": 0 }, "text/plain": [ "- RandomShuffle 1: 0%| | 0/4096 [00:00MapBatches(_filter_split)->RandomShuffle 8: 0%| | 0/4096 [00:00 TaskPoolMapOperator[ReadCSV->SplitBlocks(64)] -> AllToAllOperator[RandomShuffle] -> AllToAllOperator[Sort] -> AllToAllOperator[MapBatches(group_fn)->MapBatches(_filter_split)->RandomShuffle] -> TaskPoolMapOperator[MapBatches(preprocess)]\n", "2023-09-17 22:43:50,092\tINFO streaming_executor.py:94 -- Execution config: ExecutionOptions(resource_limits=ExecutionResources(cpu=None, gpu=None, object_store_memory=None), locality_with_output=False, preserve_order=True, actor_locality_enabled=True, verbose_progress=False)\n", "2023-09-17 22:43:50,093\tINFO streaming_executor.py:96 -- Tip: For detailed progress reporting, run `ray.data.DataContext.get_current().execution_options.verbose_progress = True`\n" ] }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "", "version_major": 2, "version_minor": 0 }, "text/plain": [ "- RandomShuffle 1: 0%| | 0/4096 [00:00MapBatches(_filter_split)->RandomShuffle 8: 0%| | 0/4096 [00:00) per trial\n", " grace_period=5, # min epoch () per trial\n", ")" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [], "source": [ "# Tune config\n", "tune_config = tune.TuneConfig(\n", " metric=\"val_loss\",\n", " mode=\"min\",\n", " search_alg=search_alg,\n", " scheduler=scheduler,\n", " num_samples=num_runs,\n", ")" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [], "source": [ "# Tuner\n", "tuner = Tuner(\n", " trainable=trainer,\n", " run_config=run_config,\n", " param_space=param_space,\n", " tune_config=tune_config,\n", ")" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [ { "data": { "text/html": [ "
\n", "
\n", "
\n", "

Tune Status

\n", " \n", "\n", "\n", "\n", "\n", "\n", "
Current time:2023-09-17 22:45:41
Running for: 00:01:49.07
Memory: 21.8/62.1 GiB
\n", "
\n", "
\n", "
\n", "

System Info

\n", " Using AsyncHyperBand: num_stopped=2
Bracket: Iter 5.000: -0.2372603341937065
Logical resource usage: 4.0/32 CPUs, 1.0/2 GPUs (0.0/2.0 accelerator_type:A10G)\n", "
\n", " \n", "
\n", "
\n", "
\n", "

Trial Status

\n", " \n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "
Trial name status loc train_loop_config/dr\n", "opout_p train_loop_config/lr train_loop_config/lr\n", "_factor train_loop_config/lr\n", "_patience iter total time (s) epoch lr train_loss
TorchTrainer_639d7776TERMINATED10.0.35.174:8467050.5 0.0001 0.8 3 10 68.3885 90.0001 0.0520358
TorchTrainer_145c1bc2TERMINATED10.0.34.101:3497160.841192 5.18042e-050.7586272.25374 5 30.7297 45.18042e-05 0.392521
\n", "
\n", "
\n", "\n" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stderr", "output_type": "stream", "text": [ "\u001b[2m\u001b[36m(TorchTrainer pid=846705)\u001b[0m Starting distributed worker processes: ['846788 (10.0.35.174)']\n", "\u001b[2m\u001b[36m(RayTrainWorker pid=846788)\u001b[0m Setting up process group for: env:// [rank=0, world_size=1]\n", "\u001b[2m\u001b[36m(SplitCoordinator pid=846864)\u001b[0m Auto configuring locality_with_output=['c268d87a05f14889a7bdc3c259e78ced37c574c65ab440657e85be5a']\n", "\u001b[2m\u001b[36m(RayTrainWorker pid=846788)\u001b[0m Moving model to device: cuda:0\n" ] }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "", "version_major": 2, "version_minor": 0 }, "text/plain": [ "(pid=846864) Running 0: 0%| | 0/64 [00:00 OutputSplitter[split(1, equal=True)]\n", "\u001b[2m\u001b[36m(SplitCoordinator pid=846864)\u001b[0m Execution config: ExecutionOptions(resource_limits=ExecutionResources(cpu=None, gpu=None, object_store_memory=None), locality_with_output=['c268d87a05f14889a7bdc3c259e78ced37c574c65ab440657e85be5a'], preserve_order=True, actor_locality_enabled=True, verbose_progress=False)\n", "\u001b[2m\u001b[36m(SplitCoordinator pid=846864)\u001b[0m Tip: For detailed progress reporting, run `ray.data.DataContext.get_current().execution_options.verbose_progress = True`\n", "\u001b[2m\u001b[36m(RayTrainWorker pid=846788)\u001b[0m /tmp/ipykernel_841208/1209796013.py:7: UserWarning: The given NumPy array is not writable, and PyTorch does not support non-writable tensors. This means writing to this tensor will result in undefined behavior. You may want to copy the array to protect its data or make it writable before converting it to a tensor. This type of warning will be suppressed for the rest of this program. (Triggered internally at ../torch/csrc/utils/tensor_numpy.cpp:206.)\n", "\u001b[2m\u001b[36m(TorchTrainer pid=349716, ip=10.0.34.101)\u001b[0m Starting distributed worker processes: ['349780 (10.0.34.101)']\n", "\u001b[2m\u001b[36m(RayTrainWorker pid=349780, ip=10.0.34.101)\u001b[0m Setting up process group for: env:// [rank=0, world_size=1]\n", "\u001b[2m\u001b[36m(SplitCoordinator pid=349841, ip=10.0.34.101)\u001b[0m Auto configuring locality_with_output=['860a06117a62bfc12416a8163d19974a865e594674344a920da1e53b']\n" ] }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "", "version_major": 2, "version_minor": 0 }, "text/plain": [ "(pid=349841, ip=10.0.34.101) Running 0: 0%| | 0/64 [00:00 OutputSplitter[split(1, equal=True)]\n", "\u001b[2m\u001b[36m(SplitCoordinator pid=349841, ip=10.0.34.101)\u001b[0m Execution config: ExecutionOptions(resource_limits=ExecutionResources(cpu=None, gpu=None, object_store_memory=None), locality_with_output=['860a06117a62bfc12416a8163d19974a865e594674344a920da1e53b'], preserve_order=True, actor_locality_enabled=True, verbose_progress=False)\n", "\u001b[2m\u001b[36m(SplitCoordinator pid=349841, ip=10.0.34.101)\u001b[0m Tip: For detailed progress reporting, run `ray.data.DataContext.get_current().execution_options.verbose_progress = True`\n", "\u001b[2m\u001b[36m(RayTrainWorker pid=349780, ip=10.0.34.101)\u001b[0m /tmp/ipykernel_841208/1209796013.py:7: UserWarning: The given NumPy array is not writable, and PyTorch does not support non-writable tensors. This means writing to this tensor will result in undefined behavior. You may want to copy the array to protect its data or make it writable before converting it to a tensor. This type of warning will be suppressed for the rest of this program. (Triggered internally at ../torch/csrc/utils/tensor_numpy.cpp:206.)\n" ] }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "", "version_major": 2, "version_minor": 0 }, "text/plain": [ "(pid=846864) Running 0: 0%| | 0/64 [00:00 OutputSplitter[split(1, equal=True)]\n", "\u001b[2m\u001b[36m(SplitCoordinator pid=846864)\u001b[0m Execution config: ExecutionOptions(resource_limits=ExecutionResources(cpu=None, gpu=None, object_store_memory=None), locality_with_output=['c268d87a05f14889a7bdc3c259e78ced37c574c65ab440657e85be5a'], preserve_order=True, actor_locality_enabled=True, verbose_progress=False)\n", "\u001b[2m\u001b[36m(SplitCoordinator pid=846864)\u001b[0m Tip: For detailed progress reporting, run `ray.data.DataContext.get_current().execution_options.verbose_progress = True`\n" ] }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "", "version_major": 2, "version_minor": 0 }, "text/plain": [ "(pid=349841, ip=10.0.34.101) Running 0: 0%| | 0/64 [00:00 OutputSplitter[split(1, equal=True)]\n", "\u001b[2m\u001b[36m(SplitCoordinator pid=349841, ip=10.0.34.101)\u001b[0m Execution config: ExecutionOptions(resource_limits=ExecutionResources(cpu=None, gpu=None, object_store_memory=None), locality_with_output=['860a06117a62bfc12416a8163d19974a865e594674344a920da1e53b'], preserve_order=True, actor_locality_enabled=True, verbose_progress=False)\n", "\u001b[2m\u001b[36m(SplitCoordinator pid=349841, ip=10.0.34.101)\u001b[0m Tip: For detailed progress reporting, run `ray.data.DataContext.get_current().execution_options.verbose_progress = True`\n" ] }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "", "version_major": 2, "version_minor": 0 }, "text/plain": [ "(pid=846864) Running 0: 0%| | 0/64 [00:00 OutputSplitter[split(1, equal=True)]\n", "\u001b[2m\u001b[36m(SplitCoordinator pid=846864)\u001b[0m Execution config: ExecutionOptions(resource_limits=ExecutionResources(cpu=None, gpu=None, object_store_memory=None), locality_with_output=['c268d87a05f14889a7bdc3c259e78ced37c574c65ab440657e85be5a'], preserve_order=True, actor_locality_enabled=True, verbose_progress=False)\n", "\u001b[2m\u001b[36m(SplitCoordinator pid=846864)\u001b[0m Tip: For detailed progress reporting, run `ray.data.DataContext.get_current().execution_options.verbose_progress = True`\n", "2023-09-17 22:44:16,982\tWARNING util.py:315 -- The `callbacks.on_trial_result` operation took 3.185 s, which may be a performance bottleneck.\n", "2023-09-17 22:44:16,983\tWARNING util.py:315 -- The `process_trial_result` operation took 3.187 s, which may be a performance bottleneck.\n", "2023-09-17 22:44:16,984\tWARNING util.py:315 -- Processing trial results took 3.188 s, which may be a performance bottleneck. Please consider reporting results less frequently to Ray Tune.\n", "2023-09-17 22:44:16,984\tWARNING util.py:315 -- The `process_trial_result` operation took 3.188 s, which may be a performance bottleneck.\n" ] }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "", "version_major": 2, "version_minor": 0 }, "text/plain": [ "(pid=349841, ip=10.0.34.101) Running 0: 0%| | 0/64 [00:00 OutputSplitter[split(1, equal=True)]\n", "\u001b[2m\u001b[36m(SplitCoordinator pid=349841, ip=10.0.34.101)\u001b[0m Execution config: ExecutionOptions(resource_limits=ExecutionResources(cpu=None, gpu=None, object_store_memory=None), locality_with_output=['860a06117a62bfc12416a8163d19974a865e594674344a920da1e53b'], preserve_order=True, actor_locality_enabled=True, verbose_progress=False)\n", "\u001b[2m\u001b[36m(SplitCoordinator pid=349841, ip=10.0.34.101)\u001b[0m Tip: For detailed progress reporting, run `ray.data.DataContext.get_current().execution_options.verbose_progress = True`\n" ] }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "", "version_major": 2, "version_minor": 0 }, "text/plain": [ "(pid=846864) Running 0: 0%| | 0/64 [00:00 OutputSplitter[split(1, equal=True)]\n", "\u001b[2m\u001b[36m(SplitCoordinator pid=846864)\u001b[0m Execution config: ExecutionOptions(resource_limits=ExecutionResources(cpu=None, gpu=None, object_store_memory=None), locality_with_output=['c268d87a05f14889a7bdc3c259e78ced37c574c65ab440657e85be5a'], preserve_order=True, actor_locality_enabled=True, verbose_progress=False)\n", "\u001b[2m\u001b[36m(SplitCoordinator pid=846864)\u001b[0m Tip: For detailed progress reporting, run `ray.data.DataContext.get_current().execution_options.verbose_progress = True`\n", "2023-09-17 22:44:25,141\tWARNING util.py:315 -- The `callbacks.on_trial_result` operation took 3.025 s, which may be a performance bottleneck.\n", "2023-09-17 22:44:25,143\tWARNING util.py:315 -- The `process_trial_result` operation took 3.028 s, which may be a performance bottleneck.\n", "2023-09-17 22:44:25,144\tWARNING util.py:315 -- Processing trial results took 3.028 s, which may be a performance bottleneck. Please consider reporting results less frequently to Ray Tune.\n", "2023-09-17 22:44:25,144\tWARNING util.py:315 -- The `process_trial_result` operation took 3.029 s, which may be a performance bottleneck.\n" ] }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "", "version_major": 2, "version_minor": 0 }, "text/plain": [ "(pid=349841, ip=10.0.34.101) Running 0: 0%| | 0/64 [00:00 OutputSplitter[split(1, equal=True)]\n", "\u001b[2m\u001b[36m(SplitCoordinator pid=349841, ip=10.0.34.101)\u001b[0m Execution config: ExecutionOptions(resource_limits=ExecutionResources(cpu=None, gpu=None, object_store_memory=None), locality_with_output=['860a06117a62bfc12416a8163d19974a865e594674344a920da1e53b'], preserve_order=True, actor_locality_enabled=True, verbose_progress=False)\n", "\u001b[2m\u001b[36m(SplitCoordinator pid=349841, ip=10.0.34.101)\u001b[0m Tip: For detailed progress reporting, run `ray.data.DataContext.get_current().execution_options.verbose_progress = True`\n" ] }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "", "version_major": 2, "version_minor": 0 }, "text/plain": [ "(pid=846864) Running 0: 0%| | 0/64 [00:00 OutputSplitter[split(1, equal=True)]\n", "\u001b[2m\u001b[36m(SplitCoordinator pid=846864)\u001b[0m Execution config: ExecutionOptions(resource_limits=ExecutionResources(cpu=None, gpu=None, object_store_memory=None), locality_with_output=['c268d87a05f14889a7bdc3c259e78ced37c574c65ab440657e85be5a'], preserve_order=True, actor_locality_enabled=True, verbose_progress=False)\n", "\u001b[2m\u001b[36m(SplitCoordinator pid=846864)\u001b[0m Tip: For detailed progress reporting, run `ray.data.DataContext.get_current().execution_options.verbose_progress = True`\n", "2023-09-17 22:44:33,395\tWARNING util.py:315 -- The `callbacks.on_trial_result` operation took 3.181 s, which may be a performance bottleneck.\n", "2023-09-17 22:44:33,397\tWARNING util.py:315 -- The `process_trial_result` operation took 3.183 s, which may be a performance bottleneck.\n", "2023-09-17 22:44:33,398\tWARNING util.py:315 -- Processing trial results took 3.184 s, which may be a performance bottleneck. Please consider reporting results less frequently to Ray Tune.\n", "2023-09-17 22:44:33,398\tWARNING util.py:315 -- The `process_trial_result` operation took 3.184 s, which may be a performance bottleneck.\n" ] }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "", "version_major": 2, "version_minor": 0 }, "text/plain": [ "(pid=349841, ip=10.0.34.101) Running 0: 0%| | 0/64 [00:00 OutputSplitter[split(1, equal=True)]\n", "\u001b[2m\u001b[36m(SplitCoordinator pid=349841, ip=10.0.34.101)\u001b[0m Execution config: ExecutionOptions(resource_limits=ExecutionResources(cpu=None, gpu=None, object_store_memory=None), locality_with_output=['860a06117a62bfc12416a8163d19974a865e594674344a920da1e53b'], preserve_order=True, actor_locality_enabled=True, verbose_progress=False)\n", "\u001b[2m\u001b[36m(SplitCoordinator pid=349841, ip=10.0.34.101)\u001b[0m Tip: For detailed progress reporting, run `ray.data.DataContext.get_current().execution_options.verbose_progress = True`\n" ] }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "", "version_major": 2, "version_minor": 0 }, "text/plain": [ "(pid=846864) Running 0: 0%| | 0/64 [00:00 OutputSplitter[split(1, equal=True)]\n", "\u001b[2m\u001b[36m(SplitCoordinator pid=846864)\u001b[0m Execution config: ExecutionOptions(resource_limits=ExecutionResources(cpu=None, gpu=None, object_store_memory=None), locality_with_output=['c268d87a05f14889a7bdc3c259e78ced37c574c65ab440657e85be5a'], preserve_order=True, actor_locality_enabled=True, verbose_progress=False)\n", "\u001b[2m\u001b[36m(SplitCoordinator pid=846864)\u001b[0m Tip: For detailed progress reporting, run `ray.data.DataContext.get_current().execution_options.verbose_progress = True`\n", "2023-09-17 22:44:41,574\tWARNING util.py:315 -- The `callbacks.on_trial_result` operation took 3.101 s, which may be a performance bottleneck.\n", "2023-09-17 22:44:41,576\tWARNING util.py:315 -- The `process_trial_result` operation took 3.102 s, which may be a performance bottleneck.\n", "2023-09-17 22:44:41,576\tWARNING util.py:315 -- Processing trial results took 3.103 s, which may be a performance bottleneck. Please consider reporting results less frequently to Ray Tune.\n", "2023-09-17 22:44:41,577\tWARNING util.py:315 -- The `process_trial_result` operation took 3.104 s, which may be a performance bottleneck.\n" ] }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "", "version_major": 2, "version_minor": 0 }, "text/plain": [ "(pid=349841, ip=10.0.34.101) Running 0: 0%| | 0/64 [00:00 OutputSplitter[split(1, equal=True)]\n", "\u001b[2m\u001b[36m(SplitCoordinator pid=349841, ip=10.0.34.101)\u001b[0m Execution config: ExecutionOptions(resource_limits=ExecutionResources(cpu=None, gpu=None, object_store_memory=None), locality_with_output=['860a06117a62bfc12416a8163d19974a865e594674344a920da1e53b'], preserve_order=True, actor_locality_enabled=True, verbose_progress=False)\n", "\u001b[2m\u001b[36m(SplitCoordinator pid=349841, ip=10.0.34.101)\u001b[0m Tip: For detailed progress reporting, run `ray.data.DataContext.get_current().execution_options.verbose_progress = True`\n" ] }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "", "version_major": 2, "version_minor": 0 }, "text/plain": [ "(pid=846864) Running 0: 0%| | 0/64 [00:00 OutputSplitter[split(1, equal=True)]\n", "\u001b[2m\u001b[36m(SplitCoordinator pid=846864)\u001b[0m Execution config: ExecutionOptions(resource_limits=ExecutionResources(cpu=None, gpu=None, object_store_memory=None), locality_with_output=['c268d87a05f14889a7bdc3c259e78ced37c574c65ab440657e85be5a'], preserve_order=True, actor_locality_enabled=True, verbose_progress=False)\n", "\u001b[2m\u001b[36m(SplitCoordinator pid=846864)\u001b[0m Tip: For detailed progress reporting, run `ray.data.DataContext.get_current().execution_options.verbose_progress = True`\n", "2023-09-17 22:44:49,825\tWARNING util.py:315 -- The `callbacks.on_trial_result` operation took 3.286 s, which may be a performance bottleneck.\n", "2023-09-17 22:44:49,828\tWARNING util.py:315 -- The `process_trial_result` operation took 3.289 s, which may be a performance bottleneck.\n", "2023-09-17 22:44:49,828\tWARNING util.py:315 -- Processing trial results took 3.289 s, which may be a performance bottleneck. Please consider reporting results less frequently to Ray Tune.\n", "2023-09-17 22:44:49,828\tWARNING util.py:315 -- The `process_trial_result` operation took 3.289 s, which may be a performance bottleneck.\n", "\u001b[2m\u001b[36m(RayTrainWorker pid=349780, ip=10.0.34.101)\u001b[0m Checkpoint successfully created at: Checkpoint(filesystem=local, path=/efs/shared_storage/madewithml/GokuMohandas/TorchTrainer_2023-09-17_22-43-52/TorchTrainer_145c1bc2_2_dropout_p=0.8412,lr=0.0001,lr_factor=0.7586,lr_patience=2.2537_2023-09-17_22-43-55/checkpoint_000005)\n", "\u001b[2m\u001b[36m(RayTrainWorker pid=846788)\u001b[0m Checkpoint successfully created at: Checkpoint(filesystem=local, path=/efs/shared_storage/madewithml/GokuMohandas/TorchTrainer_2023-09-17_22-43-52/TorchTrainer_639d7776_1_dropout_p=0.5000,lr=0.0001,lr_factor=0.8000,lr_patience=3.0000_2023-09-17_22-43-52/checkpoint_000006)\n" ] }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "", "version_major": 2, "version_minor": 0 }, "text/plain": [ "(pid=846864) Running 0: 0%| | 0/64 [00:00 OutputSplitter[split(1, equal=True)]\n", "\u001b[2m\u001b[36m(SplitCoordinator pid=846864)\u001b[0m Execution config: ExecutionOptions(resource_limits=ExecutionResources(cpu=None, gpu=None, object_store_memory=None), locality_with_output=['c268d87a05f14889a7bdc3c259e78ced37c574c65ab440657e85be5a'], preserve_order=True, actor_locality_enabled=True, verbose_progress=False)\n", "\u001b[2m\u001b[36m(SplitCoordinator pid=846864)\u001b[0m Tip: For detailed progress reporting, run `ray.data.DataContext.get_current().execution_options.verbose_progress = True`\n", "\u001b[2m\u001b[36m(RayTrainWorker pid=846788)\u001b[0m Checkpoint successfully created at: Checkpoint(filesystem=local, path=/efs/shared_storage/madewithml/GokuMohandas/TorchTrainer_2023-09-17_22-43-52/TorchTrainer_639d7776_1_dropout_p=0.5000,lr=0.0001,lr_factor=0.8000,lr_patience=3.0000_2023-09-17_22-43-52/checkpoint_000007)\n" ] }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "", "version_major": 2, "version_minor": 0 }, "text/plain": [ "(pid=846864) Running 0: 0%| | 0/64 [00:00 OutputSplitter[split(1, equal=True)]\n", "\u001b[2m\u001b[36m(SplitCoordinator pid=846864)\u001b[0m Execution config: ExecutionOptions(resource_limits=ExecutionResources(cpu=None, gpu=None, object_store_memory=None), locality_with_output=['c268d87a05f14889a7bdc3c259e78ced37c574c65ab440657e85be5a'], preserve_order=True, actor_locality_enabled=True, verbose_progress=False)\n", "\u001b[2m\u001b[36m(SplitCoordinator pid=846864)\u001b[0m Tip: For detailed progress reporting, run `ray.data.DataContext.get_current().execution_options.verbose_progress = True`\n", "\u001b[2m\u001b[36m(RayTrainWorker pid=846788)\u001b[0m Checkpoint successfully created at: Checkpoint(filesystem=local, path=/efs/shared_storage/madewithml/GokuMohandas/TorchTrainer_2023-09-17_22-43-52/TorchTrainer_639d7776_1_dropout_p=0.5000,lr=0.0001,lr_factor=0.8000,lr_patience=3.0000_2023-09-17_22-43-52/checkpoint_000008)\n" ] }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "", "version_major": 2, "version_minor": 0 }, "text/plain": [ "(pid=846864) Running 0: 0%| | 0/64 [00:00 OutputSplitter[split(1, equal=True)]\n", "\u001b[2m\u001b[36m(SplitCoordinator pid=846864)\u001b[0m Execution config: ExecutionOptions(resource_limits=ExecutionResources(cpu=None, gpu=None, object_store_memory=None), locality_with_output=['c268d87a05f14889a7bdc3c259e78ced37c574c65ab440657e85be5a'], preserve_order=True, actor_locality_enabled=True, verbose_progress=False)\n", "\u001b[2m\u001b[36m(SplitCoordinator pid=846864)\u001b[0m Tip: For detailed progress reporting, run `ray.data.DataContext.get_current().execution_options.verbose_progress = True`\n", "\u001b[2m\u001b[36m(RayTrainWorker pid=846788)\u001b[0m Checkpoint successfully created at: Checkpoint(filesystem=local, path=/efs/shared_storage/madewithml/GokuMohandas/TorchTrainer_2023-09-17_22-43-52/TorchTrainer_639d7776_1_dropout_p=0.5000,lr=0.0001,lr_factor=0.8000,lr_patience=3.0000_2023-09-17_22-43-52/checkpoint_000009)\n", "2023-09-17 22:45:41,369\tINFO tune.py:1143 -- Total run time: 109.13 seconds (109.03 seconds for the tuning loop).\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "CPU times: user 1.67 s, sys: 1.85 s, total: 3.52 s\n", "Wall time: 1min 49s\n" ] } ], "source": [ "%%time\n", "# Tune\n", "results = tuner.fit()" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
epochlrtrain_lossval_losstimestampshould_checkpointdonetraining_iterationtrial_iddate...hostnamenode_iptime_since_restoreiterations_since_restorecheckpoint_dir_nameconfig/train_loop_config/dropout_pconfig/train_loop_config/lrconfig/train_loop_config/lr_factorconfig/train_loop_config/lr_patiencelogdir
090.0001000.0520360.0963911695015936TrueTrue10639d77762023-09-17_22-45-36...ip-10-0-35-17410.0.35.17468.38852410checkpoint_0000090.5000000.0001000.8000003.000000639d7776
140.0000520.3925210.3263201695015885TrueTrue5145c1bc22023-09-17_22-44-46...ip-10-0-34-10110.0.34.10130.7297465checkpoint_0000040.8411920.0000520.7586272.253736145c1bc2
\n", "

2 rows ร— 23 columns

\n", "
" ], "text/plain": [ " epoch lr train_loss val_loss timestamp should_checkpoint done \\\n", "0 9 0.000100 0.052036 0.096391 1695015936 True True \n", "1 4 0.000052 0.392521 0.326320 1695015885 True True \n", "\n", " training_iteration trial_id date ... hostname \\\n", "0 10 639d7776 2023-09-17_22-45-36 ... ip-10-0-35-174 \n", "1 5 145c1bc2 2023-09-17_22-44-46 ... ip-10-0-34-101 \n", "\n", " node_ip time_since_restore iterations_since_restore \\\n", "0 10.0.35.174 68.388524 10 \n", "1 10.0.34.101 30.729746 5 \n", "\n", " checkpoint_dir_name config/train_loop_config/dropout_p \\\n", "0 checkpoint_000009 0.500000 \n", "1 checkpoint_000004 0.841192 \n", "\n", " config/train_loop_config/lr config/train_loop_config/lr_factor \\\n", "0 0.000100 0.800000 \n", "1 0.000052 0.758627 \n", "\n", " config/train_loop_config/lr_patience logdir \n", "0 3.000000 639d7776 \n", "1 2.253736 145c1bc2 \n", "\n", "[2 rows x 23 columns]" ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# All trials in experiment\n", "results.get_dataframe()" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
epochlrtrain_lossval_losstimestampshould_checkpointdonetraining_iterationtrial_iddate...pidhostnamenode_iptime_since_restoreiterations_since_restorecheckpoint_dir_nameconfig/train_loop_config/dropout_pconfig/train_loop_config/lrconfig/train_loop_config/lr_factorconfig/train_loop_config/lr_patience
000.00010.5778600.4922271695015848TrueFalse1639d77762023-09-17_22-44-08...846705ip-10-0-35-17410.0.35.17412.8284651checkpoint_0000000.50.00010.83.0
110.00010.4757430.3871871695015856TrueFalse2639d77762023-09-17_22-44-16...846705ip-10-0-35-17410.0.35.17420.6512862checkpoint_0000010.50.00010.83.0
220.00010.4023740.3723901695015864TrueFalse3639d77762023-09-17_22-44-24...846705ip-10-0-35-17410.0.35.17428.2356283checkpoint_0000020.50.00010.83.0
330.00010.3490680.2977041695015873TrueFalse4639d77762023-09-17_22-44-33...846705ip-10-0-35-17410.0.35.17435.8737924checkpoint_0000030.50.00010.83.0
440.00010.2568880.2075741695015881TrueFalse5639d77762023-09-17_22-44-41...846705ip-10-0-35-17410.0.35.17443.4876125checkpoint_0000040.50.00010.83.0
550.00010.2042790.1612831695015889TrueFalse6639d77762023-09-17_22-44-49...846705ip-10-0-35-17410.0.35.17451.1135386checkpoint_0000050.50.00010.83.0
660.00010.1427160.1385461695015897TrueFalse7639d77762023-09-17_22-45-18...846705ip-10-0-35-17410.0.35.17451.1156707checkpoint_0000060.50.00010.83.0
770.00010.0976150.1080341695015924TrueFalse8639d77762023-09-17_22-45-24...846705ip-10-0-35-17410.0.35.17456.7815268checkpoint_0000070.50.00010.83.0
880.00010.0700970.1022921695015930TrueFalse9639d77762023-09-17_22-45-30...846705ip-10-0-35-17410.0.35.17462.6071849checkpoint_0000080.50.00010.83.0
990.00010.0520360.0963911695015936TrueTrue10639d77762023-09-17_22-45-36...846705ip-10-0-35-17410.0.35.17468.38852410checkpoint_0000090.50.00010.83.0
\n", "

10 rows ร— 22 columns

\n", "
" ], "text/plain": [ " epoch lr train_loss val_loss timestamp should_checkpoint done \\\n", "0 0 0.0001 0.577860 0.492227 1695015848 True False \n", "1 1 0.0001 0.475743 0.387187 1695015856 True False \n", "2 2 0.0001 0.402374 0.372390 1695015864 True False \n", "3 3 0.0001 0.349068 0.297704 1695015873 True False \n", "4 4 0.0001 0.256888 0.207574 1695015881 True False \n", "5 5 0.0001 0.204279 0.161283 1695015889 True False \n", "6 6 0.0001 0.142716 0.138546 1695015897 True False \n", "7 7 0.0001 0.097615 0.108034 1695015924 True False \n", "8 8 0.0001 0.070097 0.102292 1695015930 True False \n", "9 9 0.0001 0.052036 0.096391 1695015936 True True \n", "\n", " training_iteration trial_id date ... pid \\\n", "0 1 639d7776 2023-09-17_22-44-08 ... 846705 \n", "1 2 639d7776 2023-09-17_22-44-16 ... 846705 \n", "2 3 639d7776 2023-09-17_22-44-24 ... 846705 \n", "3 4 639d7776 2023-09-17_22-44-33 ... 846705 \n", "4 5 639d7776 2023-09-17_22-44-41 ... 846705 \n", "5 6 639d7776 2023-09-17_22-44-49 ... 846705 \n", "6 7 639d7776 2023-09-17_22-45-18 ... 846705 \n", "7 8 639d7776 2023-09-17_22-45-24 ... 846705 \n", "8 9 639d7776 2023-09-17_22-45-30 ... 846705 \n", "9 10 639d7776 2023-09-17_22-45-36 ... 846705 \n", "\n", " hostname node_ip time_since_restore iterations_since_restore \\\n", "0 ip-10-0-35-174 10.0.35.174 12.828465 1 \n", "1 ip-10-0-35-174 10.0.35.174 20.651286 2 \n", "2 ip-10-0-35-174 10.0.35.174 28.235628 3 \n", "3 ip-10-0-35-174 10.0.35.174 35.873792 4 \n", "4 ip-10-0-35-174 10.0.35.174 43.487612 5 \n", "5 ip-10-0-35-174 10.0.35.174 51.113538 6 \n", "6 ip-10-0-35-174 10.0.35.174 51.115670 7 \n", "7 ip-10-0-35-174 10.0.35.174 56.781526 8 \n", "8 ip-10-0-35-174 10.0.35.174 62.607184 9 \n", "9 ip-10-0-35-174 10.0.35.174 68.388524 10 \n", "\n", " checkpoint_dir_name config/train_loop_config/dropout_p \\\n", "0 checkpoint_000000 0.5 \n", "1 checkpoint_000001 0.5 \n", "2 checkpoint_000002 0.5 \n", "3 checkpoint_000003 0.5 \n", "4 checkpoint_000004 0.5 \n", "5 checkpoint_000005 0.5 \n", "6 checkpoint_000006 0.5 \n", "7 checkpoint_000007 0.5 \n", "8 checkpoint_000008 0.5 \n", "9 checkpoint_000009 0.5 \n", "\n", " config/train_loop_config/lr config/train_loop_config/lr_factor \\\n", "0 0.0001 0.8 \n", "1 0.0001 0.8 \n", "2 0.0001 0.8 \n", "3 0.0001 0.8 \n", "4 0.0001 0.8 \n", "5 0.0001 0.8 \n", "6 0.0001 0.8 \n", "7 0.0001 0.8 \n", "8 0.0001 0.8 \n", "9 0.0001 0.8 \n", "\n", " config/train_loop_config/lr_patience \n", "0 3.0 \n", "1 3.0 \n", "2 3.0 \n", "3 3.0 \n", "4 3.0 \n", "5 3.0 \n", "6 3.0 \n", "7 3.0 \n", "8 3.0 \n", "9 3.0 \n", "\n", "[10 rows x 22 columns]" ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Best trial's epochs\n", "best_trial = results.get_best_result(metric=\"val_loss\", mode=\"min\")\n", "best_trial.metrics_dataframe" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [ { "data": { "text/plain": [ "{'dropout_p': 0.5, 'lr': 0.0001, 'lr_factor': 0.8, 'lr_patience': 3.0}" ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Best trial's hyperparameters\n", "best_trial.config[\"train_loop_config\"]" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
run_idexperiment_idstatusartifact_uristart_timeend_timemetrics.time_total_smetrics.config/train_loop_config/lr_patiencemetrics.time_since_restoremetrics.iterations_since_restore...metrics.config/train_loop_config/num_epochsparams.train_loop_config/lrparams.train_loop_config/lr_patienceparams.train_loop_config/dropout_pparams.train_loop_config/lr_factorparams.train_loop_config/num_classesparams.train_loop_config/num_epochsparams.train_loop_config/batch_sizetags.trial_nametags.mlflow.runName
0ebc5576cd02e4430bbe949612a25bd45703905858072772764FINISHEDfile:///efs/shared_storage/madewithml/GokuMoha...2023-09-18 05:43:55.499000+00:002023-09-18 05:45:41.257000+00:0068.3885243.00000068.38852410.0...NaN0.00013.00.50.8NoneNoneNoneTorchTrainer_639d7776TorchTrainer_639d7776
1a67f6ec4e10b49a7845b91db61efc7e0703905858072772764FINISHEDfile:///efs/shared_storage/madewithml/GokuMoha...2023-09-18 05:42:12.863000+00:002023-09-18 05:43:30.925000+00:0063.6994443.00000063.69944410.0...10.00.000130.50.8410256TorchTrainer_1bd07_00000TorchTrainer_1bd07_00000
20d8a50554ecf498cb6f1471b58aedb9d703905858072772764FINISHEDfile:///efs/shared_storage/madewithml/GokuMoha...2023-09-18 05:43:59.010000+00:002023-09-18 05:45:17.549000+00:0030.7297462.25373630.7297465.0...NaN5.1804178109970566e-052.2537356446691870.84119201160730330.7586266131367906NoneNoneNoneTorchTrainer_145c1bc2TorchTrainer_145c1bc2
\n", "

3 rows ร— 35 columns

\n", "
" ], "text/plain": [ " run_id experiment_id status \\\n", "0 ebc5576cd02e4430bbe949612a25bd45 703905858072772764 FINISHED \n", "1 a67f6ec4e10b49a7845b91db61efc7e0 703905858072772764 FINISHED \n", "2 0d8a50554ecf498cb6f1471b58aedb9d 703905858072772764 FINISHED \n", "\n", " artifact_uri \\\n", "0 file:///efs/shared_storage/madewithml/GokuMoha... \n", "1 file:///efs/shared_storage/madewithml/GokuMoha... \n", "2 file:///efs/shared_storage/madewithml/GokuMoha... \n", "\n", " start_time end_time \\\n", "0 2023-09-18 05:43:55.499000+00:00 2023-09-18 05:45:41.257000+00:00 \n", "1 2023-09-18 05:42:12.863000+00:00 2023-09-18 05:43:30.925000+00:00 \n", "2 2023-09-18 05:43:59.010000+00:00 2023-09-18 05:45:17.549000+00:00 \n", "\n", " metrics.time_total_s metrics.config/train_loop_config/lr_patience \\\n", "0 68.388524 3.000000 \n", "1 63.699444 3.000000 \n", "2 30.729746 2.253736 \n", "\n", " metrics.time_since_restore metrics.iterations_since_restore ... \\\n", "0 68.388524 10.0 ... \n", "1 63.699444 10.0 ... \n", "2 30.729746 5.0 ... \n", "\n", " metrics.config/train_loop_config/num_epochs params.train_loop_config/lr \\\n", "0 NaN 0.0001 \n", "1 10.0 0.0001 \n", "2 NaN 5.1804178109970566e-05 \n", "\n", " params.train_loop_config/lr_patience params.train_loop_config/dropout_p \\\n", "0 3.0 0.5 \n", "1 3 0.5 \n", "2 2.253735644669187 0.8411920116073033 \n", "\n", " params.train_loop_config/lr_factor params.train_loop_config/num_classes \\\n", "0 0.8 None \n", "1 0.8 4 \n", "2 0.7586266131367906 None \n", "\n", " params.train_loop_config/num_epochs params.train_loop_config/batch_size \\\n", "0 None None \n", "1 10 256 \n", "2 None None \n", "\n", " tags.trial_name tags.mlflow.runName \n", "0 TorchTrainer_639d7776 TorchTrainer_639d7776 \n", "1 TorchTrainer_1bd07_00000 TorchTrainer_1bd07_00000 \n", "2 TorchTrainer_145c1bc2 TorchTrainer_145c1bc2 \n", "\n", "[3 rows x 35 columns]" ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Sorted runs\n", "sorted_runs = mlflow.search_runs(experiment_names=[experiment_name], order_by=[\"metrics.val_loss ASC\"])\n", "sorted_runs" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# Artifacts\n", "best_checkpoint = get_best_checkpoint(run_id=best_run.run_id)\n", "predictor = TorchPredictor.from_checkpoint(best_checkpoint)\n", "preprocessor = predictor.get_preprocessor()" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "2023-09-17 22:45:42,824\tINFO streaming_executor.py:93 -- Executing DAG InputDataBuffer[Input] -> TaskPoolMapOperator[ReadCSV->SplitBlocks(64)] -> TaskPoolMapOperator[MapBatches(preprocess)->MapBatches()]\n", "2023-09-17 22:45:42,824\tINFO streaming_executor.py:94 -- Execution config: ExecutionOptions(resource_limits=ExecutionResources(cpu=None, gpu=None, object_store_memory=None), locality_with_output=False, preserve_order=True, actor_locality_enabled=True, verbose_progress=False)\n", "2023-09-17 22:45:42,825\tINFO streaming_executor.py:96 -- Tip: For detailed progress reporting, run `ray.data.DataContext.get_current().execution_options.verbose_progress = True`\n" ] }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "", "version_major": 2, "version_minor": 0 }, "text/plain": [ "Running 0: 0%| | 0/4096 [00:00 TaskPoolMapOperator[ReadCSV->SplitBlocks(64)] -> TaskPoolMapOperator[MapBatches(preprocess)->MapBatches(TorchPredictor)]\n", "2023-09-17 22:45:43,741\tINFO streaming_executor.py:94 -- Execution config: ExecutionOptions(resource_limits=ExecutionResources(cpu=None, gpu=None, object_store_memory=None), locality_with_output=False, preserve_order=True, actor_locality_enabled=True, verbose_progress=False)\n", "2023-09-17 22:45:43,742\tINFO streaming_executor.py:96 -- Tip: For detailed progress reporting, run `ray.data.DataContext.get_current().execution_options.verbose_progress = True`\n" ] }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "", "version_major": 2, "version_minor": 0 }, "text/plain": [ "Running 0: 0%| | 0/4096 [00:00MapBatches(TorchPredictor) pid=350325, ip=10.0.34.101)\u001b[0m /tmp/ipykernel_841208/1209796013.py:7: UserWarning: The given NumPy array is not writable, and PyTorch does not support non-writable tensors. This means writing to this tensor will result in undefined behavior. You may want to copy the array to protect its data or make it writable before converting it to a tensor. This type of warning will be suppressed for the rest of this program. (Triggered internally at ../torch/csrc/utils/tensor_numpy.cpp:206.)\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "{\n", " \"precision\": 0.9168092951711627,\n", " \"recall\": 0.9109947643979057,\n", " \"f1\": 0.9105512639658029\n", "}\n" ] } ], "source": [ "# Evaluate on test split\n", "performance = evaluate(ds=test_ds, predictor=predictor)\n", "print (json.dumps(performance, indent=2))" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "2023-09-17 22:45:49,875\tINFO streaming_executor.py:93 -- Executing DAG InputDataBuffer[Input] -> TaskPoolMapOperator[MapBatches(preprocess)->MapBatches(TorchPredictor.predict_proba)]\n", "2023-09-17 22:45:49,875\tINFO streaming_executor.py:94 -- Execution config: ExecutionOptions(resource_limits=ExecutionResources(cpu=None, gpu=None, object_store_memory=None), locality_with_output=False, preserve_order=True, actor_locality_enabled=True, verbose_progress=False)\n", "2023-09-17 22:45:49,876\tINFO streaming_executor.py:96 -- Tip: For detailed progress reporting, run `ray.data.DataContext.get_current().execution_options.verbose_progress = True`\n" ] }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "", "version_major": 2, "version_minor": 0 }, "text/plain": [ "Running 0: 0%| | 0/1 [00:00 TaskPoolMapOperator[ReadCSV->SplitBlocks(64)] -> TaskPoolMapOperator[MapBatches(preprocess)->MapBatches()]\n", "2023-09-17 22:45:52,091\tINFO streaming_executor.py:94 -- Execution config: ExecutionOptions(resource_limits=ExecutionResources(cpu=None, gpu=None, object_store_memory=None), locality_with_output=False, preserve_order=True, actor_locality_enabled=True, verbose_progress=False)\n", "2023-09-17 22:45:52,091\tINFO streaming_executor.py:96 -- Tip: For detailed progress reporting, run `ray.data.DataContext.get_current().execution_options.verbose_progress = True`\n" ] }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "", "version_major": 2, "version_minor": 0 }, "text/plain": [ "Running 0: 0%| | 0/4096 [00:00 TaskPoolMapOperator[ReadCSV->SplitBlocks(64)] -> TaskPoolMapOperator[MapBatches(preprocess)->MapBatches(TorchPredictor.predict_proba)]\n", "2023-09-17 22:45:52,988\tINFO streaming_executor.py:94 -- Execution config: ExecutionOptions(resource_limits=ExecutionResources(cpu=None, gpu=None, object_store_memory=None), locality_with_output=False, preserve_order=True, actor_locality_enabled=True, verbose_progress=False)\n", "2023-09-17 22:45:52,988\tINFO streaming_executor.py:96 -- Tip: For detailed progress reporting, run `ray.data.DataContext.get_current().execution_options.verbose_progress = True`\n" ] }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "", "version_major": 2, "version_minor": 0 }, "text/plain": [ "Running 0: 0%| | 0/4096 [00:00\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
idcreated_ontitledescriptiontagtextprediction
0192020-03-03 13:54:31Diffusion to VectorReference implementation of Diffusion2Vec (Com...otherDiffusion to Vector Reference implementation o...computer-vision
1262020-03-07 23:11:58Graph Wavelet Neural NetworkA PyTorch implementation of \"Graph Wavelet Neu...otherGraph Wavelet Neural Network A PyTorch impleme...other
2442020-03-08 00:32:58Capsule Graph Neural NetworkA PyTorch implementation of \"Capsule Graph Neu...otherCapsule Graph Neural Network A PyTorch impleme...other
3802020-03-20 05:59:32NeRF: Neural Radiance FieldsRepresenting scenes as neural radiance fields ...computer-visionNeRF: Neural Radiance Fields Representing scen...computer-vision
4842020-03-20 15:18:43Mention ClassifierCategory prediction model\\nThis repo contains ...natural-language-processingMention Classifier Category prediction model\\n...natural-language-processing
\n", "" ], "text/plain": [ " id created_on title \\\n", "0 19 2020-03-03 13:54:31 Diffusion to Vector \n", "1 26 2020-03-07 23:11:58 Graph Wavelet Neural Network \n", "2 44 2020-03-08 00:32:58 Capsule Graph Neural Network \n", "3 80 2020-03-20 05:59:32 NeRF: Neural Radiance Fields \n", "4 84 2020-03-20 15:18:43 Mention Classifier \n", "\n", " description \\\n", "0 Reference implementation of Diffusion2Vec (Com... \n", "1 A PyTorch implementation of \"Graph Wavelet Neu... \n", "2 A PyTorch implementation of \"Capsule Graph Neu... \n", "3 Representing scenes as neural radiance fields ... \n", "4 Category prediction model\\nThis repo contains ... \n", "\n", " tag \\\n", "0 other \n", "1 other \n", "2 other \n", "3 computer-vision \n", "4 natural-language-processing \n", "\n", " text \\\n", "0 Diffusion to Vector Reference implementation o... \n", "1 Graph Wavelet Neural Network A PyTorch impleme... \n", "2 Capsule Graph Neural Network A PyTorch impleme... \n", "3 NeRF: Neural Radiance Fields Representing scen... \n", "4 Mention Classifier Category prediction model\\n... \n", "\n", " prediction \n", "0 computer-vision \n", "1 other \n", "2 other \n", "3 computer-vision \n", "4 natural-language-processing " ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Add columns (for convenience)\n", "test_df = test_ds.to_pandas()\n", "test_df[\"text\"] = test_df[\"title\"] + \" \" + test_df[\"description\"]\n", "test_df[\"prediction\"] = test_df.index.map(lambda i: preprocessor.index_to_class[y_pred[i]])\n", "test_df.head()" ] }, { "cell_type": "markdown", "metadata": { "id": "TiXcls5JoNA8" }, "source": [ "### Coarse-grained metrics" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "h2OQtNODrh6c", "outputId": "4c15bd9d-3465-4476-f02a-282aaaae0a91", "tags": [] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "{\n", " \"precision\": 0.9138952286238713,\n", " \"recall\": 0.9109947643979057,\n", " \"f1\": 0.9114851103432928,\n", " \"num_samples\": 191.0\n", "}\n" ] } ], "source": [ "# Overall metrics\n", "overall_metrics = precision_recall_fscore_support(y_test, y_pred, average=\"weighted\")\n", "metrics[\"overall\"][\"precision\"] = overall_metrics[0]\n", "metrics[\"overall\"][\"recall\"] = overall_metrics[1]\n", "metrics[\"overall\"][\"f1\"] = overall_metrics[2]\n", "metrics[\"overall\"][\"num_samples\"] = np.float64(len(y_test))\n", "print (json.dumps(metrics[\"overall\"], indent=4))" ] }, { "cell_type": "markdown", "metadata": { "id": "zl3xSuXRutKG" }, "source": [ "### Fine-grained metrics" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "jqetm3ybN9C1", "tags": [] }, "outputs": [], "source": [ "from collections import OrderedDict" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "1zIAI4mwusoX", "tags": [] }, "outputs": [], "source": [ "# Per-class metrics\n", "class_metrics = precision_recall_fscore_support(y_test, y_pred, average=None)\n", "for i, _class in enumerate(preprocessor.class_to_index):\n", " metrics[\"class\"][_class] = {\n", " \"precision\": class_metrics[0][i],\n", " \"recall\": class_metrics[1][i],\n", " \"f1\": class_metrics[2][i],\n", " \"num_samples\": np.float64(class_metrics[3][i]),\n", " }" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "Rhh-tgpP0dvj", "outputId": "1de2a5eb-b9fb-4d23-d890-39f7310e868c", "tags": [] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "{\n", " \"precision\": 0.9146341463414634,\n", " \"recall\": 0.9615384615384616,\n", " \"f1\": 0.9375000000000001,\n", " \"num_samples\": 78.0\n", "}\n" ] } ], "source": [ "# Metrics for a specific class\n", "tag = \"natural-language-processing\"\n", "print (json.dumps(metrics[\"class\"][tag], indent=2))" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "vQVA6G-j__t5", "outputId": "960e8f1e-21e9-4bc7-f284-ae4800c77913", "tags": [] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[\n", " \"other\",\n", " {\n", " \"precision\": 0.96,\n", " \"recall\": 0.9230769230769231,\n", " \"f1\": 0.9411764705882353,\n", " \"num_samples\": 26.0\n", " }\n", "]\n", "[\n", " \"natural-language-processing\",\n", " {\n", " \"precision\": 0.9146341463414634,\n", " \"recall\": 0.9615384615384616,\n", " \"f1\": 0.9375000000000001,\n", " \"num_samples\": 78.0\n", " }\n", "]\n", "[\n", " \"computer-vision\",\n", " {\n", " \"precision\": 0.9393939393939394,\n", " \"recall\": 0.8732394366197183,\n", " \"f1\": 0.9051094890510948,\n", " \"num_samples\": 71.0\n", " }\n", "]\n", "[\n", " \"mlops\",\n", " {\n", " \"precision\": 0.7222222222222222,\n", " \"recall\": 0.8125,\n", " \"f1\": 0.7647058823529411,\n", " \"num_samples\": 16.0\n", " }\n", "]\n" ] } ], "source": [ "# Sorted tags\n", "sorted_tags_by_f1 = OrderedDict(sorted(\n", " metrics[\"class\"].items(), key=lambda tag: tag[1][\"f1\"], reverse=True))\n", "for item in sorted_tags_by_f1.items():\n", " print (json.dumps(item, indent=2))" ] }, { "cell_type": "markdown", "metadata": { "id": "f-juex26zvBF" }, "source": [ "### Confusion matrix" ] }, { "cell_type": "markdown", "metadata": { "id": "xPUao0S4k99c" }, "source": [ "- **True positives (TP)**: learn about where our model performs well.\n", "- **False positives (FP)**: potentially identify samples which may need to be relabeled.\n", "- False negatives (FN): identify the model's less performant areas to oversample later.\n", "\n", "> It's a good to have our FP/FN samples feed back into our annotation pipelines in the event we want to fix their labels and have those changes be reflected everywhere." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "ZG2SgsPAzukL", "tags": [] }, "outputs": [], "source": [ "# TP, FP, FN samples\n", "tag = \"natural-language-processing\"\n", "index = preprocessor.class_to_index[tag]\n", "tp, fp, fn = [], [], []\n", "for i, true in enumerate(y_test):\n", " pred = y_pred[i]\n", " if index==true==pred:\n", " tp.append(i)\n", " elif index!=true and index==pred:\n", " fp.append(i)\n", " elif index==true and index!=pred:\n", " fn.append(i)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "ePrxeVkG0mmO", "outputId": "c13e3881-e527-4a2a-b1dd-ef15187425ab", "tags": [] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[4, 9, 12, 17, 19, 23, 25, 26, 29, 30, 31, 32, 33, 34, 42, 47, 49, 50, 54, 56, 65, 66, 68, 71, 75, 77, 78, 79, 82, 92, 94, 95, 97, 99, 101, 109, 113, 114, 115, 118, 120, 122, 126, 128, 129, 130, 131, 133, 134, 135, 138, 139, 140, 141, 142, 144, 148, 149, 152, 159, 160, 161, 163, 166, 170, 172, 173, 174, 177, 179, 183, 184, 187, 189, 190]\n", "[41, 61, 102, 104, 154, 165, 188]\n", "[16, 76, 112]\n" ] } ], "source": [ "print (tp)\n", "print (fp)\n", "print (fn)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", "=== True positives ===\n", "Mention Classifier Category prediction model\n", "This repo contains AllenNLP model for prediction of Named Entity categories by its mentions.\n", " true: natural-language-processing\n", " pred: natural-language-processing\n", "\n", "Finetune: Scikit-learn Style Model Finetuning for NLP Finetune is a library that allows users to leverage state-of-the-art pretrained NLP models for a wide variety of downstream tasks.\n", " true: natural-language-processing\n", " pred: natural-language-processing\n", "\n", "Finetuning Transformers with JAX + Haiku Walking through a port of the RoBERTa pre-trained model to JAX + Haiku, then fine-tuning the model to solve a downstream task.\n", " true: natural-language-processing\n", " pred: natural-language-processing\n", "\n", "\n", "=== False positives ===\n", "How Docker Can Help You Become A More Effective Data Scientist A look at Docker from the perspective of a data scientist.\n", " true: mlops\n", " pred: natural-language-processing\n", "\n", "Differential Subspace Search in High-Dimensional Latent Space Differential subspace search to allow efficient iterative user exploration in such a space, without relying on domain- or data-specific assumptions.\n", " true: computer-vision\n", " pred: natural-language-processing\n", "\n", "EfficientDet (PyTorch) A PyTorch implementation of EfficientDet faithful to the original Google implementation with ported weights.\n", " true: computer-vision\n", " pred: natural-language-processing\n", "\n", "\n", "=== False negatives ===\n", "The Unreasonable Effectiveness of Recurrent Neural Networks A close look at how RNNs are able to perform so well.\n", " true: natural-language-processing\n", " pred: computer-vision\n", "\n", "Get Subreddit Suggestions for a Post Trained on 4M Reddit posts from 4k Subreddits. End-to-end ML pipeline built with fasttext and FastAPI, deployed to Valohai.\n", " true: natural-language-processing\n", " pred: computer-vision\n", "\n", "Machine Learning Projects This Repo contains projects done by me while learning the basics. All the familiar types of regression, classification, and clustering methods have been used.\n", " true: natural-language-processing\n", " pred: mlops\n", "\n" ] } ], "source": [ "# Samples\n", "num_samples = 3\n", "cm = [(tp, \"True positives\"), (fp, \"False positives\"), (fn, \"False negatives\")]\n", "for item in cm:\n", " if len(item[0]):\n", " print (f\"\\n=== {item[1]} ===\")\n", " for index in item[0][:num_samples]:\n", " print (f\"{test_df.iloc[index].text}\")\n", " print (f\" true: {test_df.tag[index]}\")\n", " print (f\" pred: {test_df.prediction[index]}\\n\")" ] }, { "cell_type": "markdown", "metadata": { "id": "6S5LZdP2Myjh" }, "source": [ "### Confidence learning" ] }, { "cell_type": "markdown", "metadata": { "id": "ZW5nY_h-M08p" }, "source": [ "While the confusion-matrix sample analysis was a coarse-grained process, we can also use fine-grained confidence based approaches to identify potentially mislabeled samples. Here weโ€™re going to focus on the specific labeling quality as opposed to the final model predictions.\n", "\n", "Simple confidence based techniques include identifying samples whose:\n", "\n", "**Categorical**\n", "- prediction is incorrect (also indicate TN, FP, FN)\n", "- confidence score for the correct class is below a threshold\n", "- confidence score for an incorrect class is above a threshold\n", "- standard deviation of confidence scores over top N samples is low\n", "- different predictions from same model using different parameters\n", "\n", "**Continuous**\n", "- difference between predicted and ground-truth values is above some %\n", "\n" ] }, { "cell_type": "markdown", "metadata": { "id": "OuN8xKFZlo2t" }, "source": [ "> The operations in this section can be applied to entire labeled dataset to discover labeling errors via confidence learning." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "3FCrRUb2GANr", "tags": [] }, "outputs": [], "source": [ "# Tag to inspect\n", "tag = \"natural-language-processing\"\n", "index = class_to_index[tag]\n", "indices = np.where(y_test==index)[0]" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "sKQxFU0iU-w-", "tags": [] }, "outputs": [], "source": [ "# Confidence score for the correct class is below a threshold\n", "low_confidence = []\n", "min_threshold = 0.5\n", "for i in indices:\n", " prob = y_prob[i][index]\n", " if prob <= 0.5:\n", " low_confidence.append({\n", " \"text\": f\"{test_df.iloc[i].text}\",\n", " \"true\": test_df.tag[i], \n", " \"pred\": test_df.prediction[i], \n", " \"prob\": prob})" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "7DnkXhXFFMv_", "outputId": "c93cd01b-8ad1-4e63-8254-79f885534ffb", "tags": [] }, "outputs": [ { "data": { "text/plain": [ "[{'text': 'The Unreasonable Effectiveness of Recurrent Neural Networks A close look at how RNNs are able to perform so well.',\n", " 'true': 'natural-language-processing',\n", " 'pred': 'computer-vision',\n", " 'prob': 0.008757317},\n", " {'text': 'Machine Learning Projects This Repo contains projects done by me while learning the basics. All the familiar types of regression, classification, and clustering methods have been used.',\n", " 'true': 'natural-language-processing',\n", " 'pred': 'mlops',\n", " 'prob': 0.020190474}]" ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "low_confidence[0:3]" ] }, { "cell_type": "markdown", "metadata": { "id": "JwL1ltdiUjH2" }, "source": [ "But these are fairly crude techniques because neural networks are easily [overconfident](https://arxiv.org/abs/1706.04599) and so their confidences cannot be used without calibrating them. \n", "\n", "
\n", " \"accuracy\n", "
\n", "
\n", " Modern (large) neural networks result in higher accuracies but are over confident.
On Calibration of Modern Neural Networks\n", "
\n", "\n", "* **Assumption**: *โ€œthe probability associated with the predicted class label should reflect its ground truth correctness likelihood.โ€*\n", "* **Reality**: *โ€œmodern (large) neural networks are no longer well-calibratedโ€*\n", "* **Solution**: apply temperature scaling (extension of [Platt scaling](https://en.wikipedia.org/wiki/Platt_scaling){:target=\"_blank\"}) on model outputs\n", "\n", "Recent work on [confident learning](https://arxiv.org/abs/1911.00068) focuses on identifying noisy labels while accounting for this overconfidence which can then be properly relabeled and used for training." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "XX3cORGPPXXM", "tags": [] }, "outputs": [], "source": [ "import cleanlab\n", "from cleanlab.filter import find_label_issues" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "*** SIGTERM received at time=1695015958 on cpu 1 ***\n", "PC: @ 0x4edd8a (unknown) _PyEval_MakeFrameVector\n", " @ 0x7fe3ff0e9420 514711536 (unknown)\n", " @ 0x72c720 (unknown) (unknown)\n", "[2023-09-17 22:45:58,798 E 848689 841208] logging.cc:361: *** SIGTERM received at time=1695015958 on cpu 1 ***\n", "[2023-09-17 22:45:58,799 E 848689 841208] logging.cc:361: PC: @ 0x4edd8a (unknown) _PyEval_MakeFrameVector\n", "[2023-09-17 22:45:58,803 E 848689 841208] logging.cc:361: @ 0x7fe3ff0e9420 514711536 (unknown)\n", "[2023-09-17 22:45:58,808 E 848689 841208] logging.cc:361: @ 0x72c720 (unknown) (unknown)\n" ] }, { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
idcreated_ontitledescriptiontagprediction
10314592020-06-16 03:06:10SuperGlue: Learning Feature Matching with Grap...SuperGlue, a neural network that matches two s...othercomputer-vision
10214512020-06-16 01:21:09EfficientDet (PyTorch)A PyTorch implementation of EfficientDet faith...computer-visionnatural-language-processing
16521372020-08-13 02:10:03Unpopular Opinion - Data Scientists Should Be ...I believe data scientists can be more effectiv...mlopsnatural-language-processing
617102020-05-05 04:01:24Differential Subspace Search in High-Dimension...Differential subspace search to allow efficien...computer-visionnatural-language-processing
162642020-04-06 21:33:32The Unreasonable Effectiveness of Recurrent Ne...A close look at how RNNs are able to perform s...natural-language-processingcomputer-vision
\n", "
" ], "text/plain": [ " id created_on \\\n", "103 1459 2020-06-16 03:06:10 \n", "102 1451 2020-06-16 01:21:09 \n", "165 2137 2020-08-13 02:10:03 \n", "61 710 2020-05-05 04:01:24 \n", "16 264 2020-04-06 21:33:32 \n", "\n", " title \\\n", "103 SuperGlue: Learning Feature Matching with Grap... \n", "102 EfficientDet (PyTorch) \n", "165 Unpopular Opinion - Data Scientists Should Be ... \n", "61 Differential Subspace Search in High-Dimension... \n", "16 The Unreasonable Effectiveness of Recurrent Ne... \n", "\n", " description \\\n", "103 SuperGlue, a neural network that matches two s... \n", "102 A PyTorch implementation of EfficientDet faith... \n", "165 I believe data scientists can be more effectiv... \n", "61 Differential subspace search to allow efficien... \n", "16 A close look at how RNNs are able to perform s... \n", "\n", " tag prediction \n", "103 other computer-vision \n", "102 computer-vision natural-language-processing \n", "165 mlops natural-language-processing \n", "61 computer-vision natural-language-processing \n", "16 natural-language-processing computer-vision " ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Find label issues\n", "label_issues = find_label_issues(labels=y_test, pred_probs=y_prob, return_indices_ranked_by=\"self_confidence\")\n", "test_df.iloc[label_issues].drop(columns=[\"text\"]).head()" ] }, { "cell_type": "markdown", "metadata": { "id": "UtXjpKf9FU4C" }, "source": [ "Not all of these are necessarily labeling errors but situations where the predicted probabilities were not so confident. Therefore, it will be useful to attach the predictions alongside the data. This way, we can know if we need to relabel, upsample, etc. to improve our performance. Analysis like this could also shed light on the task itself. For example, you may notice that some projects involve multiple data modalities and so it's difficult to just assing one tag. So perhaps it might be better to make this taks a multilabel classification task instead (it does but we simplified it for this course)." ] }, { "cell_type": "markdown", "metadata": { "id": "dvS3UpusXP_R" }, "source": [ "### Slice metrics" ] }, { "cell_type": "markdown", "metadata": { "id": "eeWWMG38Ny4U" }, "source": [ "Just inspecting the overall and class metrics isn't enough to deploy our new version to production. There may be key slices of our dataset that we need to do really well on:\n", "\n", "- Target / predicted classes (+ combinations)\n", "- Features (explicit and implicit)\n", "- Metadata (timestamps, sources, etc.)\n", "- Priority slices / experience (minority groups, large customers, etc.)\n", "\n", "An easy way to create and evaluate slices is to define slicing functions." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "ZyueOtQsXdGm", "tags": [] }, "outputs": [], "source": [ "from snorkel.slicing import PandasSFApplier\n", "from snorkel.slicing import slice_dataframe\n", "from snorkel.slicing import slicing_function" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "coutP2KtXdLG", "tags": [] }, "outputs": [], "source": [ "@slicing_function()\n", "def nlp_llm(x):\n", " \"\"\"NLP projects that use LLMs.\"\"\"\n", " nlp_project = \"natural-language-processing\" in x.tag\n", " llm_terms = [\"transformer\", \"llm\", \"bert\"]\n", " llm_project = any(s.lower() in x.text.lower() for s in llm_terms)\n", " return (nlp_project and llm_project)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "PbxmLvi-D7lq", "tags": [] }, "outputs": [], "source": [ "@slicing_function()\n", "def short_text(x):\n", " \"\"\"Projects with short titles and descriptions.\"\"\"\n", " return len(x.text.split()) < 8 # less than 8 words" ] }, { "cell_type": "markdown", "metadata": { "id": "2Vxg5X9OD-Ax" }, "source": [ "Here we're using Snorkel's [`slicing_function`](https://snorkel.readthedocs.io/en/latest/packages/_autosummary/slicing/snorkel.slicing.slicing_function.html) to create our different slices. We can visualize our slices by applying this slicing function to a relevant DataFrame using [`slice_dataframe`](https://snorkel.readthedocs.io/en/latest/packages/_autosummary/slicing/snorkel.slicing.slice_dataframe.html)." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 98 }, "id": "VRs93KeBMthW", "outputId": "b58e5925-7b89-4925-8afc-2f1eaa9b91db", "tags": [] }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "100%|โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ| 191/191 [00:00<00:00, 32805.57it/s]\n" ] }, { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
texttag
12Finetuning Transformers with JAX + Haiku Walki...natural-language-processing
19Question Answering with a Fine-Tuned BERT What...natural-language-processing
29BertViz Tool for visualizing attention in the ...natural-language-processing
30The Transformer Family This post presents how ...natural-language-processing
31Pruning Bert to Accelerate Inference After pre...natural-language-processing
\n", "
" ], "text/plain": [ " text \\\n", "12 Finetuning Transformers with JAX + Haiku Walki... \n", "19 Question Answering with a Fine-Tuned BERT What... \n", "29 BertViz Tool for visualizing attention in the ... \n", "30 The Transformer Family This post presents how ... \n", "31 Pruning Bert to Accelerate Inference After pre... \n", "\n", " tag \n", "12 natural-language-processing \n", "19 natural-language-processing \n", "29 natural-language-processing \n", "30 natural-language-processing \n", "31 natural-language-processing " ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "nlp_llm_df = slice_dataframe(test_df, nlp_llm)\n", "nlp_llm_df[[\"text\", \"tag\"]].head()" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 224 }, "id": "B7jmdmNaXuA2", "outputId": "84b59a83-9e58-44f1-f5c4-98e1a31507ea", "tags": [] }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "100%|โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ| 191/191 [00:00<00:00, 64413.61it/s]\n" ] }, { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
texttag
75NLPAug Data augmentation for NLPnatural-language-processing
123Offline Reinforcement Learning Challenges, alg...other
127Image Classifier Pure JavaScript Image Classifiercomputer-vision
132imgaug Image augmentation for machine learning...computer-vision
140QSVM Quantum SVM for sentiment analysisnatural-language-processing
\n", "
" ], "text/plain": [ " text \\\n", "75 NLPAug Data augmentation for NLP \n", "123 Offline Reinforcement Learning Challenges, alg... \n", "127 Image Classifier Pure JavaScript Image Classifier \n", "132 imgaug Image augmentation for machine learning... \n", "140 QSVM Quantum SVM for sentiment analysis \n", "\n", " tag \n", "75 natural-language-processing \n", "123 other \n", "127 computer-vision \n", "132 computer-vision \n", "140 natural-language-processing " ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "short_text_df = slice_dataframe(test_df, short_text)\n", "short_text_df[[\"text\", \"tag\"]].head()" ] }, { "cell_type": "markdown", "metadata": { "id": "kZuDZwTNO93Q" }, "source": [ "We can define even more slicing functions and create a slices record array using the [`PandasSFApplier`](https://snorkel.readthedocs.io/en/latest/packages/_autosummary/slicing/snorkel.slicing.PandasSFApplier.html). The slices array has N (# of data points) items and each item has S (# of slicing functions) items, indicating whether that data point is part of that slice. Think of this record array as a masking layer for each slicing function on our data." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "mQG8PFovXfEm", "outputId": "22f16ecb-ed18-4502-e734-7fe73041d597", "tags": [] }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "100%|โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ| 191/191 [00:00<00:00, 27137.02it/s]\n" ] }, { "data": { "text/plain": [ "rec.array([(0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0),\n", " (0, 0), (0, 0), (0, 0), (0, 0), (1, 0), (0, 0), (0, 0), (0, 0),\n", " (0, 0), (0, 0), (0, 0), (1, 0), (0, 0), (0, 0), (0, 0), (0, 0),\n", " (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (1, 0), (1, 0), (1, 0),\n", " (1, 0), (0, 0), (1, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0),\n", " (0, 0), (0, 0), (1, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0),\n", " (0, 0), (0, 0), (1, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0),\n", " (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0),\n", " (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0),\n", " (0, 0), (0, 0), (0, 0), (0, 1), (0, 0), (0, 0), (1, 0), (0, 0),\n", " (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0),\n", " (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (1, 0), (0, 0),\n", " (0, 0), (1, 0), (0, 0), (0, 0), (0, 0), (1, 0), (0, 0), (0, 0),\n", " (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0),\n", " (0, 0), (1, 0), (1, 0), (1, 0), (0, 0), (0, 0), (0, 0), (0, 0),\n", " (1, 0), (0, 0), (0, 0), (0, 1), (0, 0), (0, 0), (0, 0), (0, 1),\n", " (1, 0), (0, 0), (1, 0), (1, 0), (0, 1), (1, 0), (0, 0), (0, 0),\n", " (0, 0), (0, 0), (0, 0), (1, 0), (0, 1), (0, 0), (0, 0), (0, 0),\n", " (0, 0), (0, 0), (0, 1), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0),\n", " (1, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0),\n", " (1, 0), (1, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0),\n", " (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (1, 0), (0, 0), (0, 0),\n", " (0, 0), (1, 0), (0, 0), (0, 0), (0, 1), (0, 0), (0, 0), (0, 0),\n", " (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (1, 0), (0, 0)],\n", " dtype=[('nlp_llm', '