Skip to content

Instantly share code, notes, and snippets.

@roshan-adusumilli

roshan-adusumilli/autoencoder-to-detect-accounting-anomalies.ipynb

Last active March 24, 2021 22:50
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save roshan-adusumilli/aa62b741dc49d55e7db4fcfcb2b8b844 to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save roshan-adusumilli/aa62b741dc49d55e7db4fcfcb2b8b844 to your computer and use it in GitHub Desktop.
Download ZIP
Autoencoder to Detect Accounting Anomalies
Display the source blob
Display the rendered blob
Raw
autoencoder-to-detect-accounting-anomalies.ipynb
{
"nbformat": 4,
"nbformat_minor": 0,
"metadata": {
"colab": {
"name": "Autoencoder to Detect Accounting Anomalies",
"provenance": [],
"collapsed_sections": [],
"machine_shape": "hm",
"include_colab_link": true
},
"kernelspec": {
"name": "python3",
"display_name": "Python 3"
},
"accelerator": "GPU"
},
"cells": [
{
"cell_type": "markdown",
"metadata": {
"id": "view-in-github",
"colab_type": "text"
},
"source": [
"<a href=\"https://colab.research.google.com/gist/roshan-adusumilli/aa62b741dc49d55e7db4fcfcb2b8b844/autoencoder-to-detect-accounting-anomalies.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "zWVH20j-X_OB",
"colab_type": "text"
},
"source": [
"Make the various necessary imports"
]
},
{
"cell_type": "code",
"metadata": {
"id": "WLuHn-y4YJW_",
"colab_type": "code",
"colab": {}
},
"source": [
"# importing python utility libraries\n",
"import os, sys, random, io, urllib\n",
"from datetime import datetime\n",
"\n",
"# importing pytorch libraries\n",
"import torch\n",
"from torch import nn\n",
"from torch.utils.data import DataLoader\n",
"import torch.optim as optim\n",
"from torch import autograd\n",
"\n",
"# importing data science libraries\n",
"import pandas as pd\n",
"import random as rd\n",
"import numpy as np\n",
"\n",
"# importing python plotting libraries\n",
"import matplotlib.pyplot as plt\n",
"import seaborn as sns\n",
"%matplotlib inline\n",
"sns.set_style('darkgrid')\n",
"from IPython.display import Image, display\n",
"\n"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "0vHI2zoeqcHW",
"colab_type": "code",
"colab": {}
},
"source": [
"USE_CUDA = True "
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "_Gb5E3tDZ_JR",
"colab_type": "text"
},
"source": [
"Set the random seed "
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "tz_KkBS5tbUI",
"colab_type": "text"
},
"source": [
""
]
},
{
"cell_type": "code",
"metadata": {
"id": "LqtGMsTeYS2d",
"colab_type": "code",
"outputId": "8d828799-2b4f-45c2-db67-a1b279c54db8",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 34
}
},
"source": [
" rd.seed(1234)\n",
"np.random.seed(1234)\n",
"torch.manual_seed(1234)"
],
"execution_count": 0,
"outputs": [
{
"output_type": "execute_result",
"data": {
"text/plain": [
"<torch._C.Generator at 0x7f98feb55850>"
]
},
"metadata": {
"tags": []
},
"execution_count": 3
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "aubvsddAagR3",
"colab_type": "text"
},
"source": [
"Import data into pandas data frame, then inspect first 5 rows of dataset"
]
},
{
"cell_type": "code",
"metadata": {
"id": "V2SlDNDPcGHz",
"colab_type": "code",
"outputId": "a35c82a8-08f4-4cc8-f6c8-a75ffa8623e6",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 204
}
},
"source": [
"url = 'https://raw.githubusercontent.com/GitiHubi/deepAI/master/data/fraud_dataset_v2.csv'\n",
"ori_dataset = pd.read_csv(url)\n",
"ori_dataset.head()"
],
"execution_count": 0,
"outputs": [
{
"output_type": "execute_result",
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>BELNR</th>\n",
" <th>WAERS</th>\n",
" <th>BUKRS</th>\n",
" <th>KTOSL</th>\n",
" <th>PRCTR</th>\n",
" <th>BSCHL</th>\n",
" <th>HKONT</th>\n",
" <th>DMBTR</th>\n",
" <th>WRBTR</th>\n",
" <th>label</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>0</th>\n",
" <td>288203</td>\n",
" <td>C3</td>\n",
" <td>C31</td>\n",
" <td>C9</td>\n",
" <td>C92</td>\n",
" <td>A3</td>\n",
" <td>B1</td>\n",
" <td>280979.60</td>\n",
" <td>0.00</td>\n",
" <td>regular</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1</th>\n",
" <td>324441</td>\n",
" <td>C1</td>\n",
" <td>C18</td>\n",
" <td>C7</td>\n",
" <td>C76</td>\n",
" <td>A1</td>\n",
" <td>B2</td>\n",
" <td>129856.53</td>\n",
" <td>243343.00</td>\n",
" <td>regular</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2</th>\n",
" <td>133537</td>\n",
" <td>C1</td>\n",
" <td>C19</td>\n",
" <td>C2</td>\n",
" <td>C20</td>\n",
" <td>A1</td>\n",
" <td>B3</td>\n",
" <td>957463.97</td>\n",
" <td>3183838.41</td>\n",
" <td>regular</td>\n",
" </tr>\n",
" <tr>\n",
" <th>3</th>\n",
" <td>331521</td>\n",
" <td>C4</td>\n",
" <td>C48</td>\n",
" <td>C9</td>\n",
" <td>C95</td>\n",
" <td>A2</td>\n",
" <td>B1</td>\n",
" <td>2681709.51</td>\n",
" <td>28778.00</td>\n",
" <td>regular</td>\n",
" </tr>\n",
" <tr>\n",
" <th>4</th>\n",
" <td>375333</td>\n",
" <td>C5</td>\n",
" <td>C58</td>\n",
" <td>C1</td>\n",
" <td>C19</td>\n",
" <td>A3</td>\n",
" <td>B1</td>\n",
" <td>910514.49</td>\n",
" <td>346.00</td>\n",
" <td>regular</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" BELNR WAERS BUKRS KTOSL PRCTR BSCHL HKONT DMBTR WRBTR label\n",
"0 288203 C3 C31 C9 C92 A3 B1 280979.60 0.00 regular\n",
"1 324441 C1 C18 C7 C76 A1 B2 129856.53 243343.00 regular\n",
"2 133537 C1 C19 C2 C20 A1 B3 957463.97 3183838.41 regular\n",
"3 331521 C4 C48 C9 C95 A2 B1 2681709.51 28778.00 regular\n",
"4 375333 C5 C58 C1 C19 A3 B1 910514.49 346.00 regular"
]
},
"metadata": {
"tags": []
},
"execution_count": 4
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "Kr0yjmKSia-6",
"colab_type": "text"
},
"source": [
"We look at the labels and see 70 global anomalies and 30 local anomalies (which account for less than .02% of the entire dataset)"
]
},
{
"cell_type": "code",
"metadata": {
"id": "iNEwNZwQhAO7",
"colab_type": "code",
"outputId": "5a15adb9-64b5-4d27-a5ef-0b3851190a14",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 85
}
},
"source": [
"ori_dataset.label.value_counts()"
],
"execution_count": 0,
"outputs": [
{
"output_type": "execute_result",
"data": {
"text/plain": [
"regular 532909\n",
"global 70\n",
"local 30\n",
"Name: label, dtype: int64"
]
},
"metadata": {
"tags": []
},
"execution_count": 5
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "5APvYsxyjk1Y",
"colab_type": "text"
},
"source": [
"Autoencoder is a form of unsupervised deep learning so we remove the label for further processing. Then, we one-hot encode the category attributes and apply log scaling and minmax scaling to numerical variables."
]
},
{
"cell_type": "code",
"metadata": {
"id": "pwXXQlnVmxk_",
"colab_type": "code",
"colab": {}
},
"source": [
"label = ori_dataset.pop('label')\n",
"\n",
"categorical_attr_names = ['KTOSL', 'PRCTR', 'BSCHL', 'HKONT', 'BUKRS', 'WAERS']\n",
"ori_dataset_categ_transformed = pd.get_dummies(ori_dataset[categorical_attr_names])\n",
"\n",
"numeric_attr_names = ['DMBTR', 'WRBTR']\n",
"numeric_attr = ori_dataset[numeric_attr_names] + 1e-4\n",
"numeric_attr = numeric_attr.apply(np.log)\n",
"ori_dataset_numeric_attr = (numeric_attr - numeric_attr.min()) / (numeric_attr.max() - numeric_attr.min())"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "_8Hp8ugtsic7",
"colab_type": "text"
},
"source": [
"Concatenate both attributes and look at the shape"
]
},
{
"cell_type": "code",
"metadata": {
"id": "lasXHPI8s6mN",
"colab_type": "code",
"outputId": "30669440-da93-40ea-9e5f-effffa5ec4e0",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 34
}
},
"source": [
"ori_subset_transformed = pd.concat([ori_dataset_categ_transformed, ori_dataset_numeric_attr], axis = 1)\n",
"ori_subset_transformed.shape"
],
"execution_count": 0,
"outputs": [
{
"output_type": "execute_result",
"data": {
"text/plain": [
"(533009, 618)"
]
},
"metadata": {
"tags": []
},
"execution_count": 7
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "A8tldvFCtrSQ",
"colab_type": "text"
},
"source": [
"Let's implement the encoder network, we use the Leaky ReLU activation function instead of ReLU to avoid the vanishing gradient problem."
]
},
{
"cell_type": "code",
"metadata": {
"id": "KB2IcCMntv-c",
"colab_type": "code",
"colab": {}
},
"source": [
"class encoder(nn.Module):\n",
"\n",
" def __init__(self):\n",
"\n",
" super(encoder, self).__init__()\n",
"\n",
" # specify layer 1 - in 618, out 512\n",
" self.encoder_L1 = nn.Linear(in_features=ori_subset_transformed.shape[1], out_features=512, bias=True) \n",
" nn.init.xavier_uniform_(self.encoder_L1.weight) \n",
" self.encoder_R1 = nn.LeakyReLU(negative_slope=0.4, inplace=True) \n",
"\n",
" # specify layer 2 - in 512, out 256\n",
" self.encoder_L2 = nn.Linear(512, 256, bias=True)\n",
" nn.init.xavier_uniform_(self.encoder_L2.weight)\n",
" self.encoder_R2 = nn.LeakyReLU(negative_slope=0.4, inplace=True)\n",
"\n",
" # specify layer 3 - in 256, out 128\n",
" self.encoder_L3 = nn.Linear(256, 128, bias=True)\n",
" nn.init.xavier_uniform_(self.encoder_L3.weight)\n",
" self.encoder_R3 = nn.LeakyReLU(negative_slope=0.4, inplace=True)\n",
"\n",
" # specify layer 4 - in 128, out 64\n",
" self.encoder_L4 = nn.Linear(128, 64, bias=True)\n",
" nn.init.xavier_uniform_(self.encoder_L4.weight)\n",
" self.encoder_R4 = nn.LeakyReLU(negative_slope=0.4, inplace=True)\n",
"\n",
" # specify layer 5 - in 64, out 32\n",
" self.encoder_L5 = nn.Linear(64, 32, bias=True)\n",
" nn.init.xavier_uniform_(self.encoder_L5.weight)\n",
" self.encoder_R5 = nn.LeakyReLU(negative_slope=0.4, inplace=True)\n",
"\n",
" # specify layer 6 - in 32, out 16\n",
" self.encoder_L6 = nn.Linear(32, 16, bias=True)\n",
" nn.init.xavier_uniform_(self.encoder_L6.weight)\n",
" self.encoder_R6 = nn.LeakyReLU(negative_slope=0.4, inplace=True)\n",
"\n",
" # specify layer 7 - in 16, out 8\n",
" self.encoder_L7 = nn.Linear(16, 8, bias=True)\n",
" nn.init.xavier_uniform_(self.encoder_L7.weight)\n",
" self.encoder_R7 = nn.LeakyReLU(negative_slope=0.4, inplace=True)\n",
"\n",
" # specify layer 8 - in 8, out 4\n",
" self.encoder_L8 = nn.Linear(8, 4, bias=True)\n",
" nn.init.xavier_uniform_(self.encoder_L8.weight)\n",
" self.encoder_R8 = nn.LeakyReLU(negative_slope=0.4, inplace=True)\n",
"\n",
" # specify layer 9 - in 4, out 3\n",
" self.encoder_L9 = nn.Linear(4, 3, bias=True)\n",
" nn.init.xavier_uniform_(self.encoder_L9.weight)\n",
" self.encoder_R9 = nn.LeakyReLU(negative_slope=0.4, inplace=True)\n",
"\n",
" # init dropout layer with probability p\n",
" self.dropout = nn.Dropout(p=0.0, inplace=True)\n",
" \n",
" def forward(self, x):\n",
"\n",
" # define forward pass through the network\n",
" x = self.encoder_R1(self.dropout(self.encoder_L1(x)))\n",
" x = self.encoder_R2(self.dropout(self.encoder_L2(x)))\n",
" x = self.encoder_R3(self.dropout(self.encoder_L3(x)))\n",
" x = self.encoder_R4(self.dropout(self.encoder_L4(x)))\n",
" x = self.encoder_R5(self.dropout(self.encoder_L5(x)))\n",
" x = self.encoder_R6(self.dropout(self.encoder_L6(x)))\n",
" x = self.encoder_R7(self.dropout(self.encoder_L7(x)))\n",
" x = self.encoder_R8(self.dropout(self.encoder_L8(x)))\n",
" x = self.encoder_R9(self.encoder_L9(x)) \n",
"\n",
" return x\n",
"\n",
"encoder_train = encoder()"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "7GF_NAu1NRc5",
"colab_type": "text"
},
"source": [
"Print the initialized architecture "
]
},
{
"cell_type": "code",
"metadata": {
"id": "EyYiIy7rOS0W",
"colab_type": "code",
"outputId": "aeb04d68-0119-40fd-94e7-2973daaad4d4",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 425
}
},
"source": [
"now = datetime.utcnow().strftime(\"%Y%m%d-%H:%M:%S\")\n",
"print('[LOG {}] encoder architecture:\\n\\n{}\\n'.format(now, encoder_train))"
],
"execution_count": 0,
"outputs": [
{
"output_type": "stream",
"text": [
"[LOG 20191231-18:04:03] encoder architecture:\n",
"\n",
"encoder(\n",
" (encoder_L1): Linear(in_features=618, out_features=512, bias=True)\n",
" (encoder_R1): LeakyReLU(negative_slope=0.4, inplace=True)\n",
" (encoder_L2): Linear(in_features=512, out_features=256, bias=True)\n",
" (encoder_R2): LeakyReLU(negative_slope=0.4, inplace=True)\n",
" (encoder_L3): Linear(in_features=256, out_features=128, bias=True)\n",
" (encoder_R3): LeakyReLU(negative_slope=0.4, inplace=True)\n",
" (encoder_L4): Linear(in_features=128, out_features=64, bias=True)\n",
" (encoder_R4): LeakyReLU(negative_slope=0.4, inplace=True)\n",
" (encoder_L5): Linear(in_features=64, out_features=32, bias=True)\n",
" (encoder_R5): LeakyReLU(negative_slope=0.4, inplace=True)\n",
" (encoder_L6): Linear(in_features=32, out_features=16, bias=True)\n",
" (encoder_R6): LeakyReLU(negative_slope=0.4, inplace=True)\n",
" (encoder_L7): Linear(in_features=16, out_features=8, bias=True)\n",
" (encoder_R7): LeakyReLU(negative_slope=0.4, inplace=True)\n",
" (encoder_L8): Linear(in_features=8, out_features=4, bias=True)\n",
" (encoder_R8): LeakyReLU(negative_slope=0.4, inplace=True)\n",
" (encoder_L9): Linear(in_features=4, out_features=3, bias=True)\n",
" (encoder_R9): LeakyReLU(negative_slope=0.4, inplace=True)\n",
" (dropout): Dropout(p=0.0, inplace=True)\n",
")\n",
"\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "IV6R-6tEOX4h",
"colab_type": "text"
},
"source": [
"Now we implement the decoder which symmetrically mirrors the encoder"
]
},
{
"cell_type": "code",
"metadata": {
"id": "Q0q3xesROwJG",
"colab_type": "code",
"colab": {}
},
"source": [
"class decoder(nn.Module):\n",
"\n",
" def __init__(self):\n",
"\n",
" super(decoder, self).__init__()\n",
"\n",
" # specify layer 1 - in 3, out 4\n",
" self.decoder_L1 = nn.Linear(in_features=3, out_features=4, bias=True) \n",
" nn.init.xavier_uniform_(self.decoder_L1.weight) \n",
" self.decoder_R1 = nn.LeakyReLU(negative_slope=0.4, inplace=True) \n",
"\n",
" # specify layer 2 - in 4, out 8\n",
" self.decoder_L2 = nn.Linear(4, 8, bias=True)\n",
" nn.init.xavier_uniform_(self.decoder_L2.weight)\n",
" self.decoder_R2 = nn.LeakyReLU(negative_slope=0.4, inplace=True)\n",
"\n",
" # specify layer 3 - in 8, out 16\n",
" self.decoder_L3 = nn.Linear(8, 16, bias=True)\n",
" nn.init.xavier_uniform_(self.decoder_L3.weight)\n",
" self.decoder_R3 = nn.LeakyReLU(negative_slope=0.4, inplace=True)\n",
"\n",
" # specify layer 4 - in 16, out 32\n",
" self.decoder_L4 = nn.Linear(16, 32, bias=True)\n",
" nn.init.xavier_uniform_(self.decoder_L4.weight)\n",
" self.decoder_R4 = nn.LeakyReLU(negative_slope=0.4, inplace=True)\n",
"\n",
" # specify layer 5 - in 32, out 64\n",
" self.decoder_L5 = nn.Linear(32, 64, bias=True)\n",
" nn.init.xavier_uniform_(self.decoder_L5.weight)\n",
" self.decoder_R5 = nn.LeakyReLU(negative_slope=0.4, inplace=True)\n",
"\n",
" # specify layer 6 - in 64, out 128\n",
" self.decoder_L6 = nn.Linear(64, 128, bias=True)\n",
" nn.init.xavier_uniform_(self.decoder_L6.weight)\n",
" self.decoder_R6 = nn.LeakyReLU(negative_slope=0.4, inplace=True)\n",
" \n",
" # specify layer 7 - in 128, out 256\n",
" self.decoder_L7 = nn.Linear(128, 256, bias=True)\n",
" nn.init.xavier_uniform_(self.decoder_L7.weight)\n",
" self.decoder_R7 = nn.LeakyReLU(negative_slope=0.4, inplace=True)\n",
"\n",
" # specify layer 8 - in 256, out 512\n",
" self.decoder_L8 = nn.Linear(256, 512, bias=True)\n",
" nn.init.xavier_uniform_(self.decoder_L8.weight)\n",
" self.decoder_R8 = nn.LeakyReLU(negative_slope=0.4, inplace=True)\n",
"\n",
" # specify layer 9 - in 512, out 618\n",
" self.decoder_L9 = nn.Linear(in_features=512, out_features=ori_subset_transformed.shape[1], bias=True)\n",
" nn.init.xavier_uniform_(self.decoder_L9.weight)\n",
" self.decoder_R9 = nn.LeakyReLU(negative_slope=0.4, inplace=True)\n",
"\n",
" # init dropout layer with probability p\n",
" self.dropout = nn.Dropout(p=0.0, inplace=True)\n",
"\n",
" def forward(self, x):\n",
"\n",
" # define forward pass through the network\n",
" x = self.decoder_R1(self.dropout(self.decoder_L1(x)))\n",
" x = self.decoder_R2(self.dropout(self.decoder_L2(x)))\n",
" x = self.decoder_R3(self.dropout(self.decoder_L3(x)))\n",
" x = self.decoder_R4(self.dropout(self.decoder_L4(x)))\n",
" x = self.decoder_R5(self.dropout(self.decoder_L5(x)))\n",
" x = self.decoder_R6(self.dropout(self.decoder_L6(x)))\n",
" x = self.decoder_R7(self.dropout(self.decoder_L7(x)))\n",
" x = self.decoder_R8(self.dropout(self.decoder_L8(x)))\n",
" x = self.decoder_R9(self.decoder_L9(x))\n",
" \n",
" return x\n",
"\n",
"decoder_train = decoder()"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "lhSLKUuMP-Kg",
"colab_type": "text"
},
"source": [
"Let's look at the architecture for our decoder"
]
},
{
"cell_type": "code",
"metadata": {
"id": "wx72nFQKQCtV",
"colab_type": "code",
"outputId": "504321be-4c6c-4282-f8db-9d36cff9662a",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 425
}
},
"source": [
"now = datetime.utcnow().strftime(\"%Y%m%d-%H:%M:%S\")\n",
"print('[LOG {}] decoder architecture:\\n\\n{}\\n'.format(now, decoder_train))"
],
"execution_count": 0,
"outputs": [
{
"output_type": "stream",
"text": [
"[LOG 20191231-18:04:03] decoder architecture:\n",
"\n",
"decoder(\n",
" (decoder_L1): Linear(in_features=3, out_features=4, bias=True)\n",
" (decoder_R1): LeakyReLU(negative_slope=0.4, inplace=True)\n",
" (decoder_L2): Linear(in_features=4, out_features=8, bias=True)\n",
" (decoder_R2): LeakyReLU(negative_slope=0.4, inplace=True)\n",
" (decoder_L3): Linear(in_features=8, out_features=16, bias=True)\n",
" (decoder_R3): LeakyReLU(negative_slope=0.4, inplace=True)\n",
" (decoder_L4): Linear(in_features=16, out_features=32, bias=True)\n",
" (decoder_R4): LeakyReLU(negative_slope=0.4, inplace=True)\n",
" (decoder_L5): Linear(in_features=32, out_features=64, bias=True)\n",
" (decoder_R5): LeakyReLU(negative_slope=0.4, inplace=True)\n",
" (decoder_L6): Linear(in_features=64, out_features=128, bias=True)\n",
" (decoder_R6): LeakyReLU(negative_slope=0.4, inplace=True)\n",
" (decoder_L7): Linear(in_features=128, out_features=256, bias=True)\n",
" (decoder_R7): LeakyReLU(negative_slope=0.4, inplace=True)\n",
" (decoder_L8): Linear(in_features=256, out_features=512, bias=True)\n",
" (decoder_R8): LeakyReLU(negative_slope=0.4, inplace=True)\n",
" (decoder_L9): Linear(in_features=512, out_features=618, bias=True)\n",
" (decoder_R9): LeakyReLU(negative_slope=0.4, inplace=True)\n",
" (dropout): Dropout(p=0.0, inplace=True)\n",
")\n",
"\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "AgvtwAJPQFzK",
"colab_type": "text"
},
"source": [
"Define loss function, learning rate, and optimizer "
]
},
{
"cell_type": "code",
"metadata": {
"id": "cYnJ_ihrQztF",
"colab_type": "code",
"colab": {}
},
"source": [
"loss_function = nn.BCEWithLogitsLoss(reduction='mean')\n",
"learning_rate = 1e-3\n",
"encoder_optimizer = torch.optim.Adam(encoder_train.parameters(), learning_rate)\n",
"decoder_optimizer = torch.optim.Adam(decoder_train.parameters(), learning_rate)"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "56797o_0R8B1",
"colab_type": "text"
},
"source": [
"Specify the training paramaters"
]
},
{
"cell_type": "code",
"metadata": {
"id": "bTsNOhDWR_Pt",
"colab_type": "code",
"colab": {}
},
"source": [
"num_epochs = 8\n",
"mini_batch_size = 128"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "CqqSfcZZSaHJ",
"colab_type": "text"
},
"source": [
"Convert pre-processed data into a pytorch tensor"
]
},
{
"cell_type": "code",
"metadata": {
"id": "RRrGsT7vTNp3",
"colab_type": "code",
"colab": {}
},
"source": [
"torch_dataset = torch.from_numpy(ori_subset_transformed.values).float()\n",
"dataloader = DataLoader(torch_dataset, batch_size=mini_batch_size, shuffle=True, num_workers=0)"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "19iaRPupT4bT",
"colab_type": "text"
},
"source": [
"Now we begin training the encoder and decoder"
]
},
{
"cell_type": "code",
"metadata": {
"id": "xhE3pW-vUUXF",
"colab_type": "code",
"outputId": "584b685e-ad9e-42c1-da29-99e45512ec2c",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 697
}
},
"source": [
"losses = []\n",
"\n",
"data = autograd.Variable(torch_dataset)\n",
"\n",
"for epoch in range(num_epochs):\n",
"\n",
" mini_batch_count = 0\n",
"\n",
" encoder_train.train()\n",
" decoder_train.train()\n",
"\n",
" start_time = datetime.now()\n",
"\n",
" for mini_batch_data in dataloader:\n",
"\n",
" mini_batch_count += 1\n",
"\n",
" mini_batch_torch = autograd.Variable(mini_batch_data)\n",
"\n",
" #Forward pass\n",
" z_representation = encoder_train(mini_batch_torch)\n",
" mini_batch_reconstruction = decoder_train(z_representation)\n",
"\n",
" #Determine reconstruction loss\n",
" reconstruction_loss = loss_function(mini_batch_reconstruction, mini_batch_torch)\n",
"\n",
" #Backward pass\n",
" decoder_optimizer.zero_grad()\n",
" encoder_optimizer.zero_grad()\n",
"\n",
" reconstruction_loss.backward()\n",
"\n",
" #Update parameters for our model\n",
" decoder_optimizer.step()\n",
" encoder_optimizer.step()\n",
"\n",
" #Monitor training progress by checking every 1000 mini batches\n",
" if mini_batch_count % 1000 == 0:\n",
" mode = 'GPU' if (torch.backends.cudnn.version() != None) and (USE_CUDA == True) else 'CPU'\n",
" \n",
" now = datetime.utcnow().strftime(\"%Y%m%d-%H:%M:%S\")\n",
" end_time = datetime.now() - start_time\n",
" print('[LOG {}] training status, epoch: [{:04}/{:04}], batch: {:04}, loss: {}, mode: {}, time required: {}'.format(now, (epoch+1), num_epochs, mini_batch_count, np.round(reconstruction_loss.item(), 4), mode, end_time))\n",
" \n",
" start_time = datetime.now()\n",
"\n",
" #Evaluate our model's performance\n",
" encoder_train.cpu().eval()\n",
" decoder_train.cpu().eval()\n",
"\n",
" reconstruction = decoder_train(encoder_train(data))\n",
"\n",
" reconstruction_loss_all = loss_function(reconstruction, data)\n",
"\n",
" losses.extend([reconstruction_loss_all.item()])\n",
"\n",
" now = datetime.utcnow().strftime(\"%Y%m%d-%H:%M:%S\")\n",
" print('[LOG {}] training status, epoch: [{:04}/{:04}], loss: {:.10f}'.format(now, (epoch+1), num_epochs, reconstruction_loss_all.item()))\n",
"\n",
" #Save model snapshot to disk\n",
" encoder_model_name = \"ep_{}_encoder_model.pth\".format((epoch+1))\n",
" torch.save(encoder_train.state_dict(), encoder_model_name)\n",
" \n",
"\n",
" decoder_model_name = \"ep_{}_decoder_model.pth\".format((epoch+1))\n",
" torch.save(decoder_train.state_dict(), decoder_model_name)\n",
" "
],
"execution_count": 0,
"outputs": [
{
"output_type": "stream",
"text": [
"[LOG 20191231-18:22:53] training status, epoch: [0001/0008], batch: 1000, loss: 0.0067, mode: GPU, time required: 0:00:24.813384\n",
"[LOG 20191231-18:23:19] training status, epoch: [0001/0008], batch: 2000, loss: 0.0056, mode: GPU, time required: 0:00:26.176171\n",
"[LOG 20191231-18:23:48] training status, epoch: [0001/0008], batch: 3000, loss: 0.0052, mode: GPU, time required: 0:00:28.260237\n",
"[LOG 20191231-18:24:17] training status, epoch: [0001/0008], batch: 4000, loss: 0.0039, mode: GPU, time required: 0:00:29.420332\n",
"[LOG 20191231-18:24:41] training status, epoch: [0001/0008], loss: 0.0065578702\n",
"[LOG 20191231-18:25:10] training status, epoch: [0002/0008], batch: 1000, loss: 0.0084, mode: GPU, time required: 0:00:29.233686\n",
"[LOG 20191231-18:25:39] training status, epoch: [0002/0008], batch: 2000, loss: 0.0037, mode: GPU, time required: 0:00:28.795133\n",
"[LOG 20191231-18:26:08] training status, epoch: [0002/0008], batch: 3000, loss: 0.0048, mode: GPU, time required: 0:00:29.136173\n",
"[LOG 20191231-18:26:36] training status, epoch: [0002/0008], batch: 4000, loss: 0.0042, mode: GPU, time required: 0:00:27.630880\n",
"[LOG 20191231-18:26:58] training status, epoch: [0002/0008], loss: 0.0052673197\n",
"[LOG 20191231-18:27:27] training status, epoch: [0003/0008], batch: 1000, loss: 0.0039, mode: GPU, time required: 0:00:28.578874\n",
"[LOG 20191231-18:27:55] training status, epoch: [0003/0008], batch: 2000, loss: 0.0044, mode: GPU, time required: 0:00:28.280030\n",
"[LOG 20191231-18:28:24] training status, epoch: [0003/0008], batch: 3000, loss: 0.004, mode: GPU, time required: 0:00:28.411158\n",
"[LOG 20191231-18:28:52] training status, epoch: [0003/0008], batch: 4000, loss: 0.0048, mode: GPU, time required: 0:00:28.243577\n",
"[LOG 20191231-18:29:14] training status, epoch: [0003/0008], loss: 0.0038571258\n",
"[LOG 20191231-18:29:42] training status, epoch: [0004/0008], batch: 1000, loss: 0.0037, mode: GPU, time required: 0:00:27.991649\n",
"[LOG 20191231-18:30:10] training status, epoch: [0004/0008], batch: 2000, loss: 0.0051, mode: GPU, time required: 0:00:28.532903\n",
"[LOG 20191231-18:30:38] training status, epoch: [0004/0008], batch: 3000, loss: 0.0035, mode: GPU, time required: 0:00:28.196170\n",
"[LOG 20191231-18:31:07] training status, epoch: [0004/0008], batch: 4000, loss: 0.005, mode: GPU, time required: 0:00:28.065800\n",
"[LOG 20191231-18:31:28] training status, epoch: [0004/0008], loss: 0.0035663701\n",
"[LOG 20191231-18:31:56] training status, epoch: [0005/0008], batch: 1000, loss: 0.0034, mode: GPU, time required: 0:00:28.150245\n",
"[LOG 20191231-18:32:24] training status, epoch: [0005/0008], batch: 2000, loss: 0.0033, mode: GPU, time required: 0:00:27.698173\n",
"[LOG 20191231-18:32:52] training status, epoch: [0005/0008], batch: 3000, loss: 0.0069, mode: GPU, time required: 0:00:27.736801\n",
"[LOG 20191231-18:33:19] training status, epoch: [0005/0008], batch: 4000, loss: 0.0038, mode: GPU, time required: 0:00:27.599813\n",
"[LOG 20191231-18:33:42] training status, epoch: [0005/0008], loss: 0.0034021311\n",
"[LOG 20191231-18:34:09] training status, epoch: [0006/0008], batch: 1000, loss: 0.0035, mode: GPU, time required: 0:00:27.802599\n",
"[LOG 20191231-18:34:37] training status, epoch: [0006/0008], batch: 2000, loss: 0.0033, mode: GPU, time required: 0:00:27.912512\n",
"[LOG 20191231-18:35:05] training status, epoch: [0006/0008], batch: 3000, loss: 0.0032, mode: GPU, time required: 0:00:28.175342\n",
"[LOG 20191231-18:35:34] training status, epoch: [0006/0008], batch: 4000, loss: 0.0033, mode: GPU, time required: 0:00:28.396582\n",
"[LOG 20191231-18:35:55] training status, epoch: [0006/0008], loss: 0.0032905312\n",
"[LOG 20191231-18:36:24] training status, epoch: [0007/0008], batch: 1000, loss: 0.0032, mode: GPU, time required: 0:00:28.254611\n",
"[LOG 20191231-18:36:52] training status, epoch: [0007/0008], batch: 2000, loss: 0.0028, mode: GPU, time required: 0:00:28.473830\n",
"[LOG 20191231-18:37:21] training status, epoch: [0007/0008], batch: 3000, loss: 0.0029, mode: GPU, time required: 0:00:28.297926\n",
"[LOG 20191231-18:37:49] training status, epoch: [0007/0008], batch: 4000, loss: 0.0031, mode: GPU, time required: 0:00:28.894868\n",
"[LOG 20191231-18:38:12] training status, epoch: [0007/0008], loss: 0.0034216549\n",
"[LOG 20191231-18:38:41] training status, epoch: [0008/0008], batch: 1000, loss: 0.0033, mode: GPU, time required: 0:00:29.265327\n",
"[LOG 20191231-18:39:10] training status, epoch: [0008/0008], batch: 2000, loss: 0.0025, mode: GPU, time required: 0:00:28.970236\n",
"[LOG 20191231-18:39:39] training status, epoch: [0008/0008], batch: 3000, loss: 0.0026, mode: GPU, time required: 0:00:28.857899\n",
"[LOG 20191231-18:40:08] training status, epoch: [0008/0008], batch: 4000, loss: 0.0031, mode: GPU, time required: 0:00:29.265773\n",
"[LOG 20191231-18:40:30] training status, epoch: [0008/0008], loss: 0.0027732425\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "yem2OLTzrmZl",
"colab_type": "text"
},
"source": [
"We'll plot the loss to see how our autoencoder is doing "
]
},
{
"cell_type": "code",
"metadata": {
"id": "aLOzHbnUrgsB",
"colab_type": "code",
"outputId": "139cd7b2-d657-43db-a43b-ef6a36f69482",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 312
}
},
"source": [
"plt.plot(range(0,len(losses)), losses)\n",
"plt.xlabel('Training epoch')\n",
"plt.xlim([0,len(losses)])\n",
"plt.ylabel('Reconstruction error')\n",
"plt.title('Autoencoder Training Performace')"
],
"execution_count": 0,
"outputs": [
{
"output_type": "execute_result",
"data": {
"text/plain": [
"Text(0.5, 1.0, 'Autoencoder Training Performace')"
]
},
"metadata": {
"tags": []
},
"execution_count": 18
},
{
"output_type": "display_data",
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAZgAAAEWCAYAAABbgYH9AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjIsIGh0\ndHA6Ly9tYXRwbG90bGliLm9yZy8li6FKAAAgAElEQVR4nOzdfVxUZf7/8dcww8j9rTCgIt6hqahh\n3qCi6diEiPdCZm6blJllFutN2ZZs+lutvlHqlrvKmmZttauWlk5mipnmXbaamDcpJgkqgyEoiDAw\nnN8f5BSLOqgMw+jn+Xj4kHPOdc68zwH9cJ3r3KgURVEQQggh6piLowMIIYS4PUmBEUIIYRdSYIQQ\nQtiFFBghhBB2IQVGCCGEXUiBEUIIYRdSYMQdq127dvz888+OjnFVixYtIiUlpc7bOpKiKDz33HN0\n69aNBx980NFxRD3QODqAaFgefvhhjh49yo4dO9BqtbVer127dnz55ZeEh4fbMV3DtHjxYpYsWQJA\nRUUFFRUVuLm5AdCkSROMRuMNb3Py5Ml2aXsjKioq6NixI+7u7qhUKry9vYmPj2fGjBm4uNz476Z7\n9uxh7969bN++HXd3dzskFg2NFBhhlZOTw3fffYe3tzfp6enExcU5OlKDVFFRgUbz2z+dSZMmMWnS\nJAA++eQTVq1axUcffVTr9Ru69evX06xZMzIzM3n44Ydp1aoViYmJN7SNiooKzpw5Q7NmzW6quDjb\nMRNV5BSZsFq7di1dunRh5MiRrF27ttqyhx9+mFWrVlmnP/nkE8aOHQvAuHHjABg+fDhRUVF8/vnn\nAKxcuRKDwUCPHj2YNGkSJpPJuv6JEydISkqiR48exMbGWtcBmDlzJrNnz2bixIlERUWRmJjIqVOn\nrMuPHz9uXbd3794sXrwYALPZzNy5c4mJiSEmJoa5c+diNput6y1dutS6bPXq1dX2z2w289prr9G/\nf3969+5NSkoKpaWlQNVv3v369SMtLY0+ffrwwgsv3NBxraiooF27dnzwwQcYDAZr4Z4zZw79+vWj\na9eujB49mn379lnXmT9/PjNnzgTg559/pl27dqxdu5Z+/foRHR1NWlraTbW9fPky06dPp1u3bgwe\nPJi0tDT0en2t9qNNmzZ07dqVY8eOAZCbm8vkyZOJjo5Gr9fzwQcfVMuUnJzM1KlTiYqK4uOPP+Yv\nf/kL3333HVFRUSxatAiAjz76CIPBQM+ePXnqqafIy8u75jG7Mu/DDz/kvvvuIyoqirfffpusrCwe\neOABunbtytSpUykvLwegoKCAxx9/nOjoaLp3717jZ7CgoICZM2cSExND9+7dmTJlinVZeno6w4YN\no1u3bowdO9a6z+IGKUL86r777lP+9a9/KQcPHlQ6dOignDt3zrrsD3/4g7Jy5Urr9Mcff6w8+OCD\n1um2bdsqWVlZ1umdO3cqPXr0UH744QelrKxMmTNnjvLQQw8piqIoly5dUvr166esXr1aKS8vVw4d\nOqT06NFDOX78uKIoivL8888rPXr0UA4cOKCUl5crU6dOVZKTkxVFUZSioiKlT58+yjvvvKOUlpYq\nRUVFyvfff68oiqIsWLBASUxMVH755RclPz9fGTNmjDJ//nxFURTl66+/Vnr16qX8+OOPyqVLl5Sp\nU6dWyzx37lzliSeeUAoKCpSioiLliSeeUFJTUxVFUZTdu3cr7du3V/7v//5PKSsrUy5fvnzNY/i/\nx0VRFKW8vFxp27at8uijjyqFhYXW9desWaMUFBQo5eXlypIlS5SYmBilrKxMURRFefPNN5Xnn39e\nURRFycrKUtq2bavMmjVLKS0tVX744QelY8eOysmTJ2+47auvvqo8/PDDyoULF5QzZ84o8fHxyoAB\nA666L1dyZ2dnK4qiKMeOHVOio6OVTz75RLFYLMqwYcOUf/zjH0pZWZmSlZWlDBgwQNm5c6c1U8eO\nHZX09HTFYrEoly9fVlauXKn84Q9/sG5/+/btSnR0tHL48GGltLRU+ctf/qI8/PDD1zxmV+ZNnjxZ\nKSoqUo4cOaJ07NhRGT9+vJKdna0UFhYqsbGxyqeffqooiqLk5+crX375pXL58mWlqKhImTx5sjJl\nyhTr5z/66KPK1KlTlcLCQsVsNivffvutoiiKcuDAAaVXr17KgQMHlIqKCmXVqlXKwIEDrd8bUXvS\ngxEAfPfdd5w5c4a4uDgiIyMJCwtj/fr1N729devWMXr0aDp27IhWq2Xq1Kl8//335OTksHXrVpo2\nbcro0aPRaDR06NCB2NhYvvjiC+v69913H507d0aj0TBs2DCOHDkCwNatW2ncuDGPPvoojRo1wsvL\niy5dulg/c/LkyQQGBhIQEMDkyZP57LPPANiwYQOjRo2ibdu2eHh48PTTT1s/S1EUVq5cyZ///Gf8\n/Pzw8vLiiSeeqDZ24uLiwjPPPINWq7WOr9yoSZMm4evra11/xIgR+Pn5odFoePzxxykuLr7uRQdT\npkyhUaNGdOzYkYiICI4ePXrDbTds2MCTTz6Jj48PoaGh1t7n9Vz5Tf6pp55i7NixjBgxgv3791Nc\nXMykSZPQarWEh4czevToasesa9eu6PV6XFxcrnrM1q1bR2JiIu3bt6dRo0ZMnz6dvXv3kpube81j\nBvD444/j5eXFXXfdRatWrejbty/NmjXD19eXmJgY689KQEAABoMBNzc36/f022+/BeDs2bPs2rWL\nl19+GV9fX1xdXenevTtQ1fN+6KGH6Ny5M2q1moSEBAAOHjxo81iJ6uSkpgCqTo/16dOHgIAAAIYM\nGcKaNWsYP378TW0vLy+Pjh07Wqc9PT3x8/PDZDJx+vRpMjIy6Natm3W5xWJh2LBh1unGjRtbv3Zz\nc6OkpASo+o+hefPm1/zMJk2aWKebNGliPeWSl5dHZGSkdVnTpk2tX58/f57Lly8zatQo6zxFUais\nrLRO+/v706hRo9ofgKsICQmpNv3Pf/6Tjz/+mHPnzqFSqbh8+TIFBQXXXD8oKMj69e+PyY20PXfu\nXLUcoaGhNnN/9tlnNGvWrNq8M2fOcPbs2Rrfw549e9Z623l5eURFRVmnvby88PHxwWQyWb///3vM\nAAIDA6vt2/9OX7x4EYBLly4xb948duzYUW0eVP0c+fv74+3tXWP7Z86cYd26dbz77rvWeeXl5dVO\nr4nakQIjKC0tZcOGDVRWVtKnTx+gakzi4sWLHD16lLvuugt3d3cuX75sXeeXX3657jaDg4M5ffq0\ndbqkpITCwkJ0Oh2hoaF0796d5cuX33DW0NDQauM1//uZZ86cISIiAqj6TyQ4ONi67OzZs9a2Z86c\nsX7t7++Pm5sbRqMRnU531W2rVKobznq9bezevZt3332Xd999lzZt2gBwzz33oNj54eaNGzcmNzeX\nli1bAlQ7JjciNDSU8PBwNmzYcM02to7Zle/XFcXFxVy8eLHa9+BWjvs777xDTk4Oq1atIigoiIMH\nD1p7I6GhoRQUFFBcXIyXl1e19UJCQpg8eTITJ0686c8WVeQUmWDz5s2o1WqMRiNr165l7dq1fP75\n53Tr1s062N++fXs2bdrE5cuX+fnnn2sMkjdu3Jjs7Gzr9JAhQ/jkk084cuQIZrOZN998k86dO9Os\nWTP69+9PVlYWa9eupby8nPLycjIyMjhx4oTNrP379+fcuXO8++67mM1miouLOXDgAADx8fH84x//\n4Pz585w/f55FixYxdOhQAAYNGsSaNWvIzMzk8uXLvP3229Zturi4kJiYyLx588jPzwfAZDKxffv2\nWzuw13Hp0iXUajX+/v6Ul5fz1ltvVSvg9hIXF8eSJUu4ePEiubm5fPjhhze1nbvvvhtXV1eWLVtG\nWVkZFouFH3/8kR9++KHW24iPj2f16tUcPXrU+jPSrVu3q/ZabsalS5dwd3fH19eXgoIC64UFUFVg\nevXqxezZs7l48SLl5eXs3bsXgAceeIAPP/yQjIwMFEXh0qVLbNmy5bo9RnF1UmAEa9asYdSoUTRp\n0oSgoCDrn3HjxrFu3ToqKip45JFHcHV1pXfv3jz//PPW/7ivePrpp5k5cybdunXj888/p3fv3jz7\n7LNMmTKFmJgYsrOzmT9/PlB1KuSdd97h888/p2/fvsTExJCamlrtiq9r8fLyYtmyZXz11Vf06dOH\n2NhY9uzZA8BTTz1FZGQkw4YNY9iwYXTs2JGnnnoKgHvvvZdHHnmERx55BIPBQHR0dLXtzpgxg/Dw\ncOvVSOPHj+fkyZN1cXiv6t5776V3797cf//96PV6vLy8qp3WspcpU6YQGBiIXq/n0UcfZdCgQbi6\nut7wdjQaDf/85z/JyMhAr9cTHR3NX/7yF4qLi2u9jX79+jF58mSefvppYmJiOHPmDKmpqTec5VqS\nkpIoKiqiZ8+ePPjgg/Tr16/a8tdffx2A2NhY+vTpw/vvvw9UFc+XX36Zl19+me7duxMbG2sdyxM3\nRqXYu08uhGiw3n//fTZv3syKFSscHUXchqQHI8QdxGQysW/fPiorKzlx4gTvvvsuBoPB0bHEbUoG\n+YW4g5jNZmbNmsXp06fx8fEhPj6eMWPGODqWuE3JKTIhhBB2IafIhBBC2MUdfYpMURQqKiptN3Qw\ntVqFxdLwO5qSs+44Q0aQnHXNWXK6uqpr1e4OLzBQWNjwr2338/OQnHXIGXI6Q0aQnHXNWXIGBdV8\nAsLVyCkyIYQQdiEFRgghhF1IgRFCCGEXUmCEEELYhRQYIYQQdiEFRgghhF1IgRFCCGEXd3SBKS23\nODqCEELctuxaYLZt20ZsbCwGg4G0tLQay81mM8nJyRgMBhITE8nJybEuW7JkCQaDgdjY2Govfrp4\n8SLPPPMMgwYNIi4ujv379wPw1ltv0bdvX4YPH87w4cP5+uuvbebLyr9E4eXyOthTIYQQ/8tud/Jb\nLBbmzJnD8uXL0el0JCQkoNfrra+HBVi1ahU+Pj5s2rQJo9FIamoqCxYsIDMzE6PRiNFoxGQykZSU\nxMaNG1Gr1cydO5e+ffvyt7/9DbPZTGlpqXV748eP57HHHqt9xkpYvO0nZsW2q9N9F0IIYcceTEZG\nBuHh4YSFhaHVaomPjyc9Pb1amy1btjBy5Eig6q1yu3btQlEU0tPTiY+PR6vVEhYWRnh4OBkZGRQV\nFbF3717re7W1Wi0+Pj43nbGxl5bPfjCxP+fCze+oEEKIq7JbD8ZkMlV7t7ZOpyMjI6NGm9DQ0Kog\nGg3e3t4UFBRgMpno0qVLtXVNJhNubm4EBATwwgsvcPToUTp27MiLL76Ih4cHAB988AFr164lMjKS\nmTNn4uvre92MwT6NaOrnxmtbMvnsqT5oNQ1zSEqtdsHPz8PRMWySnHXHGTKC5KxrzpKztpzqYZcV\nFRUcPnyYWbNm0aVLF/7617+SlpZGcnIyY8eO5amnnkKlUrFw4UJeffVVXnnlletuT4WK6QNa86c1\nh1iUfoykns3raU9ujLM8AE9y1h1nyAiSs645S06HP+xSp9ORm5trnTaZTOh0uhptzp49C1QVj6Ki\nIvz9/a+5bkhICCEhIdbezaBBgzh8+DAAjRs3Rq1W4+LiQmJiIgcPHqxVzphWgegjGvPO7lPkFF6+\npX0WQgjxG7sVmE6dOpGVlUV2djZmsxmj0Yher6/WRq/Xs2bNGgA2btxIdHQ0KpUKvV6P0WjEbDaT\nnZ1NVlYWnTt3JigoiJCQEH766ScAdu3aRevWrQHIy8uzbnfz5s1ERETUOuu0Aa3RuKj4v/RM5AWf\nQghRN+x2ikyj0ZCSksKECROwWCyMHj2aiIgIFi5cSGRkJAMHDiQhIYEZM2ZgMBjw9fVl/vz5AERE\nRBAXF8fgwYNRq9WkpKSgVle94GbWrFlMnz6d8vJywsLCrKfBXn/9dY4ePQpA06ZNmTNnTq2zBns3\nYlKfFrzx1Qk2H/sFQ7ugOj4aQghx51Epd/Cv7JWVCvn5xQBYKhWSPtxPXrGZ1Und8GrUcIannOW8\nrOSsO86QESRnXXOWnA4fg3E2ahcVLxgiKCgx8/dvshwdRwghnJ4UmN9pr/Mm8e4mrP7+DIfOXnR0\nHCGEcGpSYP7HpD4taOylZd6m41RU3rFnD4UQ4pZJgfkfXo00TBvQmmPnLrFy/2lHxxFCCKclBeYq\n9BGN6dMygMU7ssi9WGp7BSGEEDVIgbkKlUrFcwPbUKnAG1+dcHQcIYRwSlJgrqGJrxuP9wpna2Y+\nX2fmOzqOEEI4HSkw1zHunqa0buzB61syKTHLy8mEEOJGSIG5Do3ahRfui8BUVEbazp8dHUcIIZyK\nFBgbujT1ZUSnEP69L4djecWOjiOEEE5DCkwtPN23JT5urryy+TgWuTdGCCFqRQpMLfi6u5LcvxU/\nnC1iTcZZR8cRQginIAWmluLaB9O9uR+LvjnJL5fMjo4jhBANnhSYWlKpVDw/sA1lFZXMl3tjhBDC\nJikwNyA8wIOkHs358sdz7M467+g4QgjRoEmBuUGP9Aijub87r6VnUlou98YIIcS1SIG5QVpN1b0x\nOYWlLP8229FxhBCiwbJrgdm2bRuxsbEYDAbS0tJqLDebzSQnJ2MwGEhMTCQnJ8e6bMmSJRgMBmJj\nY9m+fbt1/sWLF3nmmWcYNGgQcXFx7N+/H4DCwkKSkpK4//77SUpK4sKFC3bbr27N/RjcIZj3vs3m\nZH7Df/ucEEI4gt0KjMViYc6cOSxduhSj0cj69evJzMys1mbVqlX4+PiwadMmxo8fT2pqKgCZmZkY\njUaMRiNLly5l9uzZWCxVp6Pmzp1L3759+eKLL/j0009p3bo1AGlpafTq1Ysvv/ySXr16XbWg1aVn\n722Fh1bNq5uPcwe/dVoIIa7JbgUmIyOD8PBwwsLC0Gq1xMfHk56eXq3Nli1bGDlyJACxsbHs2rUL\nRVFIT08nPj4erVZLWFgY4eHhZGRkUFRUxN69e0lISABAq9Xi4+MDQHp6OiNGjABgxIgRbN682V67\nBkCAh5YpfVuyL+cC6w+Z7PpZQgjhjDT22rDJZCIkJMQ6rdPpyMjIqNEmNDS0KohGg7e3NwUFBZhM\nJrp06VJtXZPJhJubGwEBAbzwwgscPXqUjh078uKLL+Lh4UF+fj7BwcEABAUFkZ9v+wnIKhX4+Xnc\n9D7+MaYVX/x4jre2nyQ+qhkBntqb3tb1qNUut5SzvkjOuuMMGUFy1jVnyVlbdisw9lBRUcHhw4eZ\nNWsWXbp04a9//StpaWkkJydXa6dSqVCpVDa3pyhQWHhrYyjTB7TmD+/v46/rDpEyqN0tbeta/Pw8\nbjlnfZCcdccZMoLkrGvOkjMoyLtW7ex2ikyn05Gbm2udNplM6HS6Gm3Onq169EpFRQVFRUX4+/tf\nc92QkBBCQkKsvZtBgwZx+PBhAAIDA8nLywMgLy+PgIAAe+1aNW0aezLunmasO2RiX05hvXymEEI4\nA7sVmE6dOpGVlUV2djZmsxmj0Yher6/WRq/Xs2bNGgA2btxIdHQ0KpUKvV6P0WjEbDaTnZ1NVlYW\nnTt3JigoiJCQEH766ScAdu3aZR3k1+v1rF27FoC1a9cycOBAe+1aDY/3ak4Tn0a8uimTcktlvX2u\nEEI0ZHYrMBqNhpSUFCZMmMDgwYOJi4sjIiKChQsXWgf7ExISKCwsxGAwsHz5cqZPnw5AREQEcXFx\nDB48mAkTJpCSkoJarQZg1qxZTJ8+naFDh3LkyBEmTZoEwMSJE9mxYwf3338/O3fuZOLEifbatRrc\nXNU8NzCCk+dLeH9vju0VhBDiDqBS7uBrbCsrFfLz6+4dLzPXHeabn87z70fuoZmfe51t11nOy0rO\nuuMMGUFy1jVnyenwMZg70bQBrdG4qHhtc6bcGyOEuONJgalDQV6NeLJPC3b/XMCmH885Oo4QQjiU\nFJg6lnB3E9rrvHjjqxMUlVY4Oo4QQjiMFJg6pnZR8WdDBIWXy1n0zUlHxxFCCIeRAmMHd+m8eSCq\nKZ8cOMsPZy86Oo4QQjiEFBg7mdQnnCAvLfM2HaeiUgb8hRB3HikwduKp1TBN34bj5y7x732nHR1H\nCCHqnRQYOxrQJpCYVgEs2ZFF7sVSR8cRQoh6JQXGjlQqFc8NbANA6pYTDk4jhBD1SwqMnYX6uDGx\ndzhfn8jn68xfHB1HCCHqjRSYejC2a1PaNPbk9S0nKDFbHB1HCCHqhRSYeqBRu/CCIYK8ojKW7Mxy\ndBwhhKgXUmDqSecmPozsHMp/9p3mx7y6e8CmEEI0VFJg6tHkvi3wdXfllU3Hsci9MUKI29x1C4zF\nYmHatGn1leW25+PmSnL/VhzKLeKTjLOOjiOEEHZ13QKjVqs5c+YMZrO5vvLc9gbdFUyP5n4s2n6S\nX4rLHB1HCCHsRmOrQVhYGGPHjkWv1+Ph4WGdn5SUZNdgtyuVSsXz90UwdsV3vLn1J+YNae/oSEII\nYRc2x2CaN2/OgAEDUBSFS5cuWf/UxrZt24iNjcVgMJCWllZjudlsJjk5GYPBQGJiIjk5v71ueMmS\nJRgMBmJjY9m+fbt1vl6vZ+jQoQwfPpxRo0ZZ57/11lv07duX4cOHM3z4cL7++utaZXSE5v7ujO/Z\nnE0/nmNX1nlHxxFCCLuw2YN5+umnAaxFxdPTs1YbtlgszJkzh+XLl6PT6UhISECv19OmTRtrm1Wr\nVuHj48OmTZswGo2kpqayYMECMjMzMRqNGI1GTCYTSUlJbNy4EbVaDcCKFSsICAio8Znjx4/nscce\nq1U+R3ukexgbj+Tx2uZM/v3IPbi5qh0dSQgh6pTNHsyxY8cYMWIEQ4YMYciQIYwaNYrjx4/b3HBG\nRgbh4eGEhYWh1WqJj48nPT29WpstW7YwcuRIAGJjY9m1axeKopCenk58fDxarZawsDDCw8PJyMi4\nyV1smLQaF2beF8HpC6Us23PK0XGEEKLO2ezBpKSkMHPmTKKjowHYs2cPs2bN4t///vd11zOZTISE\nhFindTpdjSJhMpkIDQ2tCqLR4O3tTUFBASaTiS5dulRb12QyWacfe+wxVCoVY8aMYcyYMdb5H3zw\nAWvXriUyMpKZM2fi6+t73YwqFfj5eVy3jT3d5+fByOO/8K/vckjsEU5EsNdV26nVLg7NWVuSs+44\nQ0aQnHXNWXLWls0CU1JSYi0uAD179qSkpMSuoa7no48+QqfTkZ+fT1JSEq1ataJ79+6MHTuWp556\nCpVKxcKFC3n11Vd55ZVXrrstRYHCQsftC8CTvZqz5WgeL36SweIxXXBRqWq08fPzcHjO2pCcdccZ\nMoLkrGvOkjMoyLtW7WyeIgsLC2PRokXk5OSQk5PD3//+d8LCwmxuWKfTkZuba502mUzodLoabc6e\nrbofpKKigqKiIvz9/a+77pW/AwMDMRgM1l5R48aNUavVuLi4kJiYyMGDB21mbAj8PbRM6deS/acv\nsv4Hk+0VhBDCSdgsMPPmzaOgoIApU6bwzDPPUFBQwLx582xuuFOnTmRlZZGdnY3ZbMZoNKLX66u1\n0ev1rFmzBoCNGzcSHR2NSqVCr9djNBoxm81kZ2eTlZVF586dKSkpobi46jErJSUl7Nixg4iICADy\n8vKs2928ebN1vjMYGhnC3U19+Nu2nygokXuOhBC3h+ueIrNYLCxevJiXXnrpxjes0ZCSksKECROw\nWCyMHj2aiIgIFi5cSGRkJAMHDiQhIYEZM2ZgMBjw9fVl/vz5AERERBAXF8fgwYNRq9WkpKSgVqvJ\nz89n8uTJ1mxDhgyhX79+ALz++uscPXoUgKZNmzJnzpwbzuwoLioVM++LYNz7+1i47SQvD2rn6EhC\nCHHLVIqiXPehWA888AArV66srzz1qrJSIT+/4Tx4ctH2k7z7bTaLH+jMPWF+1vnOcl5WctYdZ8gI\nkrOuOUvO2o7B2Bzkb9++PZMmTWLQoEHV7uS///77bz6duKrHopvz5Y/neGXTcT784z1oNfIsUiGE\n87JZYMxmM/7+/uzZs6fafCkwdc/NVc3zA9vw7Cc/8P532TwWHe7oSEIIcdNsjsG0a9eO8ePH11Mc\n0btlAPe1DWLZ7lPc3y6YMH93R0cSQoibYvNpyuvXr6+vLOJX0wa0wlXtwmvpx7ExRCaEEA2WzZP8\nXbt2Zc6cOXz33XccOnTI+kfYT2OvRjwV05I9Pxfy5dFzjo4jhBA3xeYYzJEjRwBYuHChdZ5KpeK9\n996zXyrB6C6hGA+beHPrCeLuburoOEIIccNsFpj333+/PnKI/6F2UfHn+yL44wf7mJ9+nGdjWjg6\nkhBC3BCbp8h++eUX/vznPzNhwgQAMjMzWbVqld2DCWin82JU51A+2pvNT/m1ewePEEI0FDYLzMyZ\nM4mJibE+iqVFixZyeqwePdG7BR5aNQu2/uToKEIIcUNsFpiCggIGDx6Mi0tVU41GY/1a2J+fhytP\n92/NrqwCdpyUt18KIZyHzUrh4eFBQUEBql8fI//999/j7V27xwSIuvGHnuGE+bmxcOtPVFgqHR1H\nCCFqxeYg/8yZM3nyySc5deoUDz74IAUFBdWuKBP2p9W48Oy9rZj+6WE+ycjlgagmjo4khBA22Sww\nHTt25F//+hcnT55EURRatmyJq6trfWQTv9OvdSDdmvuRtjOLQe2D8HGT74EQomGr1WCKRqMhIiKC\ntm3bSnFxEJVKxZ/ubcXF0gre2X3K0XGEEMImGa13Im2DvRjWKYT/7D/Dz+cb/iO9hRB3NikwTubJ\nPi1w07jwt20nHR1FCCGuy+YYDIDJZOL06dNYLBbrvO7du9stlLi2QE8t43uEseibLL79uYAe4f6O\njiSEEFdls8C8/vrrbNiwgdatW6NWq63za1Ngtm3bxty5c6msrCQxMZGJEydWW242m3nuuec4dOgQ\nfn5+zJ8/n2bNmgGwZMkSVq9ejYuLCy+99BJ9+/YFQK/X4+npiYuLC2q1mk8++QSAwsJC/vSnP3H6\n9GmaNm3KggUL8PX1rf2RcCJj72nGmoyzLPj6J97/Q1fULipHRxJCiBpsFpjNmzfzxRdfoNVqb2jD\nFouFOXPmsHz5cnQ6HQkJCej1etq0aWNts2rVKnx8fNi0aRNGo5HU1FQWLFhAZmYmRqMRo9GIyWQi\nKSmJjRs3WgvcihUrCAgIqOBqlCMAACAASURBVPZ5aWlp9OrVi4kTJ5KWlkZaWhozZsy4oczOopHG\nhSn9WvHC+iN89kMuIzuHOjqSEELUYHMMJiwsjPLy8hvecEZGBuHh4YSFhaHVaomPjyc9Pb1amy1b\ntjBy5EgAYmNj2bVrF4qikJ6eTnx8PFqtlrCwMMLDw8nIyLju56WnpzNixAgARowYwebNm284szMZ\n2LYxdzf1YfGOLIrLKhwdRwgharDZg3F3d2fEiBH06tWrWi/mpZdeuu56JpOJkJAQ67ROp6tRJEwm\nE6GhVb99azQavL29KSgowGQy0aVLl2rrmkwm6/Rjjz2GSqVizJgxjBkzBoD8/HyCg4MBCAoKIj8/\n39auOTWVSsWf+rfmkQ/2s3xPNlP6tXR0JCGEqMZmgdHr9ej1+vrIUisfffQROp2O/Px8kpKSaNWq\nVY3xIJVKZX20zfWoVODn52GvqHVGrXa5as7efh6MjGrCR/tyeCSmJc0DHLsv18rZ0DhDTmfICJKz\nrjlLztqyWWBGjhyJ2WwmKysLoNZ38ut0OnJzc63TJpMJnU5Xo83Zs2cJCQmhoqKCoqIi/P39r7vu\nlb8DAwMxGAxkZGTQvXt3AgMDycvLIzg4mLy8vBpjNFejKFBY2PDvJ/Hz87hmzsd7hLHhYC7zjId5\ndWiHek5W3fVyNiTOkNMZMoLkrGvOkjMoqHbPo7Q5BrNnzx5iY2OZM2cOs2fPJjY2lr1799rccKdO\nncjKyiI7Oxuz2YzRaKzRE9Lr9axZswaAjRs3Eh0djUqlQq/XYzQaMZvNZGdnk5WVRefOnSkpKaG4\nuBiAkpISduzYQUREhHVba9euBWDt2rUMHDiwVgfA2QV5NeKPPcJIP/YL+3MuODqOEEJY2ezBvPba\na7zzzju0atUKgJMnTzJt2jTr5cHX3LBGQ0pKChMmTMBisTB69GgiIiJYuHAhkZGRDBw4kISEBGbM\nmIHBYMDX15f58+cDEBERQVxcHIMHD0atVpOSkoJarSY/P5/JkycDVVepDRkyhH79+gEwceJEkpOT\nWb16NU2aNGHBggW3dGCcycPdmrE24yzzt57g3XFRuNTi9KAQQtibSlEU5XoNhg4dyrp162zOc0aV\nlQr5+cWOjmFTbbrNG46YSPn8R/4yqC1DOoZct629OEv33hlyOkNGkJx1zVly1vYUmc0eTGRkJC++\n+CLDhg0DYN26dURGRt5aOlHnYu8KZuX+MyzanoU+IggPrdr2SkIIYUc2x2Bmz55NmzZteP/993n/\n/fdp06YNs2fPro9s4ga4/HrZ8i+XzLy3N9vRcYQQwnYPRqvVkpSURFJSUn3kEbegcxMfYu8K4l/f\n5TCiUwghPm6OjiSEuINds8A8++yzLFy4kKFDh151+e0wBnM7erpvS7Zm5vP29pP8Nb69o+MIIe5g\n1ywwL774IgCLFy+utzDi1oX4uDGuWzOW7T7FmKimdGri4+hIQog71DXHYK48duXDDz+kadOm1f58\n+OGH9RZQ3LhHuofR2FPL/K0nsHGRoBBC2I3NQf6dO3fWmLdt2za7hBF1w0Or5smYFhw8W8SXR885\nOo4Q4g51zVNkH374IR999BGnTp2qNg5z6dIloqKi6iWcuHlDOupYuf8Mb20/yb1tAnFzlcuWhRD1\n65oFZujQofTr148333yTadOmWed7enri5+dXL+HEzXNRqZg6oBVP/CeDD/6bw2PR4Y6OJIS4w1zz\nFJm3tzfNmjXjj3/8I76+vtbxF41Gw4EDB+ozo7hJXZv5oY9ozLt7sjlXXOboOEKIO4zNMZiXX34Z\nT09P67SHhwcvv/yyPTOJOjSlX0ssisLfv8lydBQhxB3GZoFRFKXau1VcXFyoqJA3KDqLZn7ujO3a\nlPWHTBwxFTk6jhDiDlKrVya/9957lJeXU15ezooVKwgLC6uPbKKOJPVsjr+7K/O/ksuWhRD1p1bP\nItu/fz/9+vXj3nvvJSMjg//3//5ffWQTdcSrkYZJMS3Yf/oiW47/4ug4Qog7hM1nkQUGBlrf0yKc\n1/DIEFbtP8Pftp0kplUgjTQ2f7cQQohbYrPAvPDCC1ed/8orr9R5GGE/ahcVyf1b8fTqg/xn32n+\n2ENOcwoh7Mtmgenfv7/167KyMjZv3mx9jIxwLj3D/enbKoBle04R31FHoKfW0ZGEELcxmwUmNja2\n2vSQIUN46KGH7BZI2Nez97ZizIr/smRnFn82tHV0HCHEbeyGT8RnZWWRn59fq7bbtm0jNjYWg8FA\nWlpajeVms5nk5GQMBgOJiYnk5ORYly1ZsgSDwUBsbCzbt2+vtp7FYmHEiBE88cQT1nkzZ85Er9cz\nfPhwhg8fzpEjR2501+4I4QEePHB3Ez49mMvxcw3/ddFCCOdlswcTFRVV7T6YoKAgpk+fbnPDFouF\nOXPmsHz5cnQ6HQkJCej1etq0aWNts2rVKnx8fNi0aRNGo5HU1FQWLFhAZmYmRqMRo9GIyWQiKSmJ\njRs3olZXPU/rvffeo3Xr1hQXV/8P8rnnnmPQoEG13vk71YRezfn8sIn5W39iUUKnat9fIYSoK9ft\nwSiKgtFoZN++fdY/GzdurHHa7GoyMjIIDw8nLCwMrVZLfHw86enp1dps2bKFkSNHAlWn4nbt2oWi\nKKSnpxMfH49WqyUsLIzw8HAyMjIAyM3NZevWrSQkJNzsPt/xfNxcmdg7nL2nCtl24ryj4wghblPX\n7cGoVCqeeOKJm3p7pclkIiQkxDqt0+msReL3bUJDQ6uCaDR4e3tTUFCAyWSiS5cu1dY1mUwAzJs3\njxkzZnDp0qUanzl//nwWLVpEr169mD59Olrt9QexVSrw8/O44X2rb2q1S53nTOrXmo8zcnn7m5PE\n3d0UbR1ctmyPnPbgDDmdISNIzrrmLDlry+Ypsg4dOpCRkUHnzp3rI891ffXVVwQEBBAZGcmePXuq\nLZs6dSpBQUGUl5cza9Ys0tLSePrpp6+7PUWBwsISe0auE35+HnbJ+Uy/liR/8gNLv87koXua3fL2\n7JWzrjlDTmfICJKzrjlLzqAg71q1s1lgDhw4wLp162jSpAnu7u7W+bZ6NTqdjtzcXOu0yWRCp9PV\naHP27FlCQkKoqKigqKgIf3//a667ZcsWtmzZwrZt2ygrK6O4uJjp06eTmppqvXRaq9UyatQoli1b\nVqsDcCfr0zKA6Bb+LN11isEddPi5uzo6khDiNmKzwLzzzjs3teFOnTqRlZVFdnY2Op0Oo9HIG2+8\nUa2NXq9nzZo1REVFsXHjRqKjo1GpVOj1eqZNm0ZSUhImk4msrCw6d+5MVFSU9d00e/bsYdmyZaSm\npgKQl5dHcHAwiqKwefNmIiIibir3nSb53laMe++//HPnz8wY2Mb2CkIIUUs2C8yCBQt4/fXXq82b\nMWNGjXk1NqzRkJKSwoQJE7BYLIwePZqIiAgWLlxIZGQkAwcOJCEhgRkzZmAwGPD19bU+kiYiIoK4\nuDgGDx6MWq0mJSXFegXZtUyfPp2CggIUReGuu+5i9uzZtnZNAK0bezKycygfHzjD6LtDaRXoaXsl\nIYSoBZVi4/G6I0eOZM2aNdZpi8XC0KFD+fzzz+0ezt4qKxXy8xv+vSD2Pi9bWFLOyGXf0rmJDwtH\ndbrp7TjL+WNnyOkMGUFy1jVnyXnLYzBLlixh8eLFlJWV0bVrV6DqsmWtVssDDzxQNylFg+Dn4cqE\n6HAWfP0TO0+ep3fLAEdHEkLcBmz2YN544w3ruMftRnowvym3VDLm3e/QuLjw4SP3oHG58ZsvneW3\nL2fI6QwZQXLWNWfJWdsejM2bH/r3709JSdUOf/rpp7zyyiucPn361tKJBsdV7cKz97bi5PkSPjlw\n1tFxhBC3AZsF5uWXX8bd3Z2jR4+yfPlymjdvzvPPP18f2UQ969c6kG5hvqTtzOJiabmj4wghnJzN\nAqPRaFCpVGzevJlx48Yxbty4q95FL5yfSqUiuX9rLpZW8M7uU46OI4RwcjYLjKenJ0uWLGHdunX0\n79+fyspKKioq6iObcIB2wV4M6xTCyv1nOFVw2dFxhBBOzGaBmT9/Plqtlrlz5xIUFERubi6PPfZY\nfWQTDjKpTwu0ahf+9vVPjo4ihHBiNgtMUFAQSUlJdOvWDYAmTZowYsQIuwcTjtPYU0tSzzC+PpHP\n3lMFjo4jhHBSNgvMl19+yf33388999xD165diYqKst4XI25fY+9pRhOfRszf+hOWyuteyS6EEFdl\n81Exr7/+OosXL6Z169b1kUc0EI00Lkzp14oX1h/hsx9yGdk51NGRhBBOxmYPJjAwUIrLHWpg28bc\n3dSHxTuyKC6TCzuEEDfGZg8mMjKS5ORk7rvvvmov8Lr//vvtGkw4nkql4k/9W/PIB/tZviebKf1a\nOjqSEMKJ2Cwwly5dwt3dnR07dlSbLwXmztAhxJv4DsF8tC+HUV1CaOrrbnslIYSgFgXmlVdeqY8c\nogF7KqYl6cd+4a1tJ3l1aAdHxxFCOAmbYzC5ublMnjyZXr160atXL6ZMmVLtbZPi9hfs3Yg/9ggj\n/dgv7M+54Og4QggnYbPAvPDCC+j1erZv38727dsZMGAAL7zwQn1kEw3Iw92aEeylZf7WE1Re/wHc\nQggB1KLAnD9/ntGjR6PRaNBoNIwaNYrz58/XRzbRgLi5qnm6X0uOmIr5/LDJ0XGEEE7AZoHx8/Pj\n008/xWKxYLFY+PTTT/Hz86vVxrdt20ZsbCwGg4G0tLQay81mM8nJyRgMBhITE8nJybEuW7JkCQaD\ngdjYWLZv315tPYvFwogRI3jiiSes87Kzs0lMTMRgMJCcnIzZbK5VRlF7sXcF0zHEm79/k0WJ2eLo\nOEKIBs5mgZk3bx4bNmygT58+xMTEsHHjxloN/FssFubMmcPSpUsxGo2sX7+ezMzMam1WrVqFj48P\nmzZtYvz48aSmpgKQmZmJ0WjEaDSydOlSZs+ejcXy239o7733Xo17c1JTUxk/fjybNm3Cx8eH1atX\n1+oAiNpzUamYOqA154rNvLc329FxhBANnM0C07RpUxYvXszu3bvZtWsXf//732nSpInNDWdkZBAe\nHk5YWBharZb4+HjS09OrtdmyZQsjR44EIDY2ll27dqEoCunp6cTHx6PVagkLCyM8PJyMjAyg6qKD\nrVu3kpCQYN2Ooijs3r2b2NhYAEaOHFnjs0Td6NzEh/vbBfGv73LIvVjq6DhCiAbM5mXKzz//PC++\n+CI+Pj4AXLhwgVdffdVmL8ZkMhESEmKd1ul01iLx+zahoVWPINFoNHh7e1NQUIDJZKJLly7V1jWZ\nqs77z5s3jxkzZlR7J01BQQE+Pj5oNFW7ExISYm1/PSpV1StKGzq12qVB5fzzkA58vXA7aXuyeTPx\nt+9TQ8t5Lc6Q0xkyguSsa86Ss7ZsFpgff/zRWlwAfH19OXLkiF1DXctXX31FQEAAkZGR7Nmz55a3\npyg4xfuvG9p7uj2Bcfc0ZdmebEZ21NGpSdXPR0PLeS3OkNMZMoLkrGvOkjMoyLtW7WyeIqusrOTC\nhd/ufSgsLKw2HnItOp2u2v0yJpMJnU5Xo83Zs1Xvf6+oqKCoqAh/f/9rrrtv3z62bNmCXq9n6tSp\n7N69m+nTp+Pv78/FixetL0LLzc2t8Vmibj3SozmBnlWXLSty2bIQ4ipsFphHH32UMWPGsGDBAhYs\nWMCDDz5YqxeOderUiaysLLKzszGbzRiNRvR6fbU2er2eNWvWALBx40aio6NRqVTo9XqMRiNms5ns\n7GyysrLo3Lkz06ZNY9u2bWzZsoU333yT6OhoUlNTUalU9OzZk40bNwKwZs2aGp8l6paHVs1TMS04\neLaIL4+ec3QcIUQDZPMU2YgRI4iMjGT37t0AvP3227Rp08b2hjUaUlJSmDBhAhaLhdGjRxMREcHC\nhQuJjIxk4MCBJCQkMGPGDAwGA76+vsyfPx+AiIgI4uLiGDx4MGq1mpSUFNRq9XU/b8aMGfzpT39i\nwYIFtG/fnsTExNrsv7gFQzrqWLn/DG9tP8m9bQIdHUcI0cColFqc3/juu+/4+eefGT16NOfPn+fS\npUuEhYXVRz67qqxUyM8vdnQMmxryedn/ZhcyaWUGk/qEM21Q+wab8/ca8vG8whkyguSsa86Ss87G\nYN5++22WLl1qvVGyvLycGTNm3Fo6cdu4J8yPARGNWfFtNgdPy3PKhBC/sVlgNm3axD/+8Q/c3ase\n067T6apdIizEM/1a4qJSMWrxLsau+C/vfZtNXlGZo2MJIRzM5hiMq6srKpUKlUoFQElJw+++ifrV\nzM+dtRN6sCP7Ah9/l8Nb20/y9vaTdGvux+AOwQyIaIyn1uaPmhDiNmPzX31cXBwpKSlcvHiRlStX\n8vHHH8sAuqjBz92VcT2aE9+2MdkFl9lwxMSGI3nM/uIYr27OpH+bQOI66OgZ7o/GReXouEKIelCr\nQf4dO3bwzTffABATE0OfPn3sHqw+yCB/3frfnIqicPBsEZ8fNrH5x3NcKK0gwMOV2LuCGdwhmHbB\nXtaesSNzNkTOkBEkZ11zlpy1HeSvVYH5vcrKStavX8+wYcNuKlhDIgWmbl0vZ7mlkh0/nefzI3l8\n81M+5RaFloEeDG4fzKD2wYT4uDWInA2FM2QEyVnXnCVnbQvMNU+RFRcX88EHH2AymdDr9fTp04cP\nPviAZcuW0a5du9uiwIj646p2oX9EY/pHNOZiaTmbfzzH54fzWPRNFn//JouuYb4Mbq9D37YxXo1k\nvEaI28E1ezBPPvkkvr6+3H333ezatYvz58+jKAovvvgi7du3r++cdiE9mLp1MzlzCi/zxZE8NhzJ\n41TBZRppXOjXOpDBHYKJDvdHo7Z5oWO95KxvzpARJGddc5act3yKbOjQoaxbtw6oerdLTEwMW7du\npVGjRnWX0sGkwNStW8mpKAqHcov4/HAeXx7N40JpBf7urtx/VxCDO+hor6u78RpnOJ7OkBEkZ11z\nlpy3fIrsyqPvAdRqNSEhIbdVcRENi0qlIjLUh8hQH/7UvxU7Txaw4YiJNRln+c/+M7QIcCeuvY64\nDsGE1uN4jRDi5l2zB9O+fXvrzZWKolBWVoabmxuKoqBSqdi3b1+9BrUH6cHULXvkLCqtYPOxc2w4\nbGL/6YsARDXzZXD7YAa2DcLb7cbHa5zheDpDRpCcdc1ZctrtKrLbiRSYumXvnGculPLFkTw+P2zi\n54LLaNUq+rWuur+md4vaj9c4w/F0howgOeuas+S85VNkQjQ0TXzdeDS6OUk9wzhsKmbDYRMbj55j\n87Ff8HN35f52QcR1CKZjiLdD7q8RQlQnBUY4HZVKRccQbzqGeJN8byt2ZRXw+eE81h48y8rvz9Dc\n35249sHEdQimqa+7o+MKcceSAiOcmkbtQt/WgfRtHUhxWQXpx86x4UgeS3b+zJKdP3N3Ux/iOui4\nr21jfNxcHR1XiDuKjMHIGEydaUg5cy+WsuFIHhsO53HyfAmuahV9WwUyqH0w/TuGoDJXODridTWk\nY3k9krNuOUtOGYMRd7QQHzeSejZnfI8wjuYVW++v2XL8F/jsME183eig86JDiDftdd7cpfOSJwgI\nUcfs+i9q27ZtzJ07l8rKShITE5k4cWK15Wazmeeee45Dhw7h5+fH/PnzadasGQBLlixh9erVuLi4\n8NJLL9G3b1/KysoYN24cZrMZi8VCbGwszzzzDAAzZ87k22+/xdu7qrK++uqrt80TB8TNU6lUtNdV\nFZFn723FgdMXOFFYyr6s8xzOLWLzsV+sbcP93asKTog3HXRetAv2ws31+q/qFkJcm90KjMViYc6c\nOSxfvhydTkdCQgJ6vZ42bdpY26xatQofHx82bdqE0WgkNTWVBQsWkJmZidFoxGg0YjKZSEpKYuPG\njWi1WlasWIGnpyfl5eU89NBD9OvXj7vvvhuA5557jkGDBtlrl4ST07iouCfMj4GdPCjsFAJAQYmZ\nI6ZiDucWccRUzN5ThWw4kgeAiwpaBXrSIcSL9jpvOoR406axJ1pN3T++Rojbkd0KTEZGBuHh4YSF\nhQEQHx9Penp6tQKzZcsWnn76aQBiY2OZM2cOiqKQnp5OfHw8Wq2WsLAwwsPDycjIICoqCk9PTwAq\nKiqoqKiQy1HFLfH30NK7ZQC9WwZY550rLuNwbjGHTUUcyS3i68x8PvvBBICrWkWbxp50CPGmg86b\n9iFetAz0lHfcCHEVdiswJpOJkJAQ67ROpyMjI6NGm9DQ0KogGg3e3t4UFBRgMpno0qVLtXVNpqp/\n4BaLhVGjRnHq1Ckeeuihau3mz5/PokWL6NWrF9OnT0er1V43o0pVNajW0KnVLpKzDtnK6efnQUQz\nf4b/Oq0oCqcLL3Pw9EUOnr7AwdMX2Hj0HB8fOAuAm6sLHUJ96NTUl05NfOnU1IcWgZ643ELRuV2O\nZUMhOR3D6UY11Wo1n376KRcvXmTy5MkcO3aMtm3bMnXqVIKCgigvL2fWrFmkpaVZe0fXoig4xRUb\nznJlye2c00sFvZr50KuZDxBGpaJwquAyR0xFHM4t5khuEf/em82Kip8B8NSquUvn9Wsvx5sOIV40\n8XGrdY/7dj6WjiA565bDryLT6XTk5uZap00mEzqdrkabs2fPEhISQkVFBUVFRfj7+9dqXR8fH3r2\n7Mn27dtp27YtwcHBAGi1WkaNGsWyZcvstWtC4KJS0SLAgxYBHsS1r/rZrKhUyMov4XBuUdXpNVMx\n/95/mnJL1Z0Avm4a6wUEV65eC/aWB8iK25fdCkynTp3IysoiOzsbnU6H0WjkjTfeqNZGr9ezZs0a\noqKi2LhxI9HR0ahUKvR6PdOmTSMpKQmTyURWVhadO3fm/PnzaDQafHx8KC0tZefOnTz++OMA5OXl\nERwcjKIobN68mYiICHvtmhBXpXFR0SbIkzZBngz79SICc0UlJ/IvVV1E8Ou4zopvs/m15tDYU0v7\nKwXn1+Lj73H9U7vi+ioVhTMXSjmZX8LJ/BJ+Ol9CRIgPYzqHyFhZPbNbgdFoNKSkpDBhwgQsFguj\nR48mIiKChQsXEhkZycCBA0lISGDGjBkYDAZ8fX2ZP38+ABEREcTFxTF48GDUajUpKSmo1Wry8vKY\nOXMmFosFRVEYNGgQAwYMAGD69OkUFBSgKAp33XUXs2fPtteuCVFrWo2L9TJpfh0uLC238GNe8e+u\nXivim5/Oc+WO51CfRkQ29SPUS0uYnxvN/Nxp5udGsHcjXOSiFquKSoWcwstk5Zdw8nwJP/1aULLO\nl1BWUWlt5+/uivGQiR3HzzFvSHv83OWJDvVF7uSXO/nrjOS8ecVlFfyYV1VwDucWcyL/EtkFl6mo\n/O2fp1atoqlvVbEJ83enqa87Yf5uhPm5E+LdyC5v/7SlPo5luaWSUwWXf+uR5Jdw8vwlThVctp5+\nBAjxbkTLQA9aBnrQKtCDloGetAzwwNtNw+afzpPy2SGCvBrxxvCOtAnytGvmm9UQfzavRh7XXwtS\nYOqW5Kw7fn4e5J+/hKmojOzCy5wuvEx2YSk5hZfJKSwlu/Bytd/S1SoI9f21t+NbVYCu9Hya+rrT\nyE737tTlsSwtt/DzlUJyvuTXglJVaK/UERXQ1M+NlgEe1mLSMtCTFgHueGqvfULGz8+D7Ydzee6z\nw1wyV/DyoHbo2wbVSe665Aw/m9AABvmFELdG7aKiia8bTXzdINy/2jJFUfjlktlabH5fgH44e5Hi\nMou1rQoI9m5Es19Pt4X9WniuFKDr/cdsDyVmC1nnf9cbyb/EyfMlnC4stZ4mVKugmZ87LQM90Ec0\nruqNBHoQ7u9+009X6NTEh/f+EMVznx3m+XVHeCz6EhN7h8tpRzuSAiOEE1KpVAR5NSLIqxFRzXyr\nLVMUhQulFdV6O1e+3n4in/Ml5dXaB3i4/lp43Gj6awG68rWvm+amb2YuKq34tSdyqdr4yNmLZdY2\nGhcV4QHu3BXszeD2OmuvJMzP3S5PTAjyasTiB7rw2ubjvLP7FMfPXWJ2XDt5Dp2dyCkyOUVWZyRn\n3bFnxkvmCnJ+7e1kF1wm58JvX+cVm6u19W6k+V3P50qvp+rrQE8t/v6eZJ25UKOQnDxfwrnfbauR\nxoUWAb+Nj1z5upmfe71c2fW/x1NRFFbuP8P8rSdo7u/B68M7EB7g+BscneFnE2QMplakwNQtyVl3\nHJWxtNzCmYulZBeUcvpC9QJ09kIpvxtTx03jgrtWTcHvekQermpaXBlo/904SaiPG2oHXiJ8reP5\n3alCZq47jEVRmBvfvtojgxzBGX42QcZghBA3wc1VTatAT1oF1rzKqsJSSe6vFx1cKUAWlYomXtqq\nQhLggc67kVM9H7Bbcz/e+0NXpn96iORPfuDpvi15uHszp9qHhkwKjBCiVjRqF+spsl4tquY5y2/c\n19PE1413xt7NnC+O8db2kxw7V8xL97eVVzXUAXnuuBDijufuqmbekLt4KqYFXx49x4R/H+DsxVJH\nx3J6UmCEEIKqK/OSejbnzZEdySm8zB//tZ//Zhc6OpZTkwIjhBC/E9MqkHfHReHrpmHy6oOs3H+G\nO/haqFsiBUYIIf5HiwAP3h0XRa8W/ry+JZO5Xx7H/LsnJ4jakQIjhBBX4dVIQ+rwjiT1DOPTH3KZ\ntDKDX4rLbK8orKTACCHENahdVDwV05JXhrTn+Lli/vjBfg6dvejoWE5DCowQQthwX7sg3hl7N64u\nKib+5wDrD+XaXklIgRFCiNpoG+zFinFd6dzEh9lfHOPNr05Ue52CqEkKjBBC1JKfhytvje7Eg12b\n8tG+00z5+CCFl8ttr3iHkgIjhBA3QKN2YdqA1qTEtuXA6Qs88sF+jp9r+M80dAS7Fpht27YRGxuL\nwWAgLS2txnKz2UxycjIGg4HExERycnKsy5YsWYLBYCA2Npbt27cDUFZWRkJCAsOGDSM+Pp6//e1v\n1vbZ2dkkJiZiMBhI96HhKAAADlJJREFUTk7GbDbX+DwhhKgrQyNDSBvTBXNFJY9++D3px845OlKD\nY7cCY7FYmDNnDkuXLsVoNLJ+/XoyMzOrtVm1ahU+Pj5s2rSJ8ePHk5qaCkBmZiZGoxGj0cjSpUuZ\nPXs2FosFrVbLihUr+Oyzz1i7di3bt2/n+++/ByA1NZXx48ezadMmfHx8WL16tb12TQghAIgM9eH9\nP0QREeTJzHVH+Mc3J6mUmzKt7FZgMjIyCA8PJywsDK1WS3x8POnp6dXabNmyhZEjRwIQGxvLrl27\nUBSF9PR04uPj0Wq1hIWFER4eTkZGBiqVCk/Pqqe8VlRUUFFRgUqlQlEUdu/eTWxsLAAjR46s8VlC\nCGEPjX99idnwyBCW7clm2tpDFJdVODpWg2C3pymbTCZCQkKs0zqdjoyMjBpt/n979x4Udb3/cfy5\nF1EuQqzookWdg4AxQCQOI6FjgUdFASkvFdpvwutMYTje8RIzUaJTTqZpc6qZ8tdk/RKVkJbGZiSD\nItPSkTE5gfAjwZFlAkdAhHWX7+8Pxv0d2gU557h+WXk//tplv5/vvr7fYXnzvez7M3bs2J4gej0j\nR47k2rVrmM1moqOje401m81Az5HRvHnzuHz5MosWLSI6OpqWlhZ8fX3R63s2JzAw0L58fzSanm6w\ng51Op5Wcd5E75HSHjCA5/9lbz0bz+F/82V78D5b9z3n+vjiGvwY4TnvQH3fZnwPldu36dTodhYWF\ntLa2kpmZSVVVFQEBAf/WuhQFt2g17i4t0SXn3eMOGUFy/lnKhNGM9RpGdlEl8/5ezhtzwpkSPPBJ\nzNxlfw50wjGXnSIzGo00Nv7/l5HMZjNGo9FhmatXrwI9p7za2trw9/cf0FhfX18mT55MWVkZ/v7+\ntLa2YrX2HJY2NjY6LC+EEPfCpKAH+OSFiYzzHcGaggsc+OnykG2W6bICExUVRV1dHfX19VgsFkwm\nE4mJib2WSUxMpKCgAIDjx48TFxeHRqMhMTERk8mExWKhvr6euro6HnvsMVpaWmht7WnT0NnZSXl5\nOcHBwWg0GiZPnszx48cBKCgocHgvIYS4V8b69kxi9rcJo9n/fR1bTf+g85ZN7Vj3nMtOken1enJy\ncli+fDk2m4358+cTGhrKnj17iIyMZPr06SxYsIANGzYwY8YM/Pz82L17NwChoaHMnj2bOXPmoNPp\nyMnJQafT0dTURHZ2NjabDUVRSEpKIiEhAYANGzawZs0a3nnnHcLDw1m4cKGrNk0IIe5oxDAd25Mf\nZcIYH/aX/S+/t3TwVloE4/xGqB3tntEoQ/XYDejuVmhuHvxfkHKX87KS8+5xh4wgOQfqh9oWthVX\notdq2ZkazqSgB5wup3bOgVL9GowQQogeU4INHFg0kQc89WTmV3Do3JUhcV1GCowQQtwDjxi8+HjR\nROL/auCtkpohMYmZFBghhLhHfIbr2fV0BEvjHh4Sk5hJgRFCiHtIq9Hw0pS/sDM1nEt/9ExiduE+\nncRMCowQQqhgethoPkqfyDCdlpVfnKfowv03iZkUGCGEUEnIaG/+e/FEoh/0I/d4FYd+rlc70l3l\ndq1ihBDifvKAZ88kZp//0kCQ4f7pQwZSYIQQQnV6rYb/ig1ym+/BDJScIhNCCOESUmCEEEK4hBQY\nIYQQLiEFRgghhEtIgRFCCOESUmCEEEK4hBQYIYQQLiEFRgghhEsM6QnHhBBCuI4cwQghhHAJKTBC\nCCFcQgqMEEIIl5ACI4QQwiWkwAghhHAJKTBCCCFcQgqMEEIIlxiSE46Vlpayfft2uru7WbhwIStX\nrlQ7klObN2/m5MmTjBo1iq+++krtOE5dvXqVjRs30tzcjEaj4dlnn+XFF19UO5aDrq4uFi9ejMVi\nwWazMWvWLLKystSO1Sebzcb8+fMxGo28//77asdxKjExEW9vb7RaLTqdjqNHj6odyanW1la2bdtG\nVVUVGo2GvLw8Jk6cqHasXmpra1mzZo39eX19PVlZWWRkZKgXyokDBw6Qn5+PRqMhLCyMHTt2MHz4\n8L4HKEOM1WpVpk+frly+fFnp6upSUlNTlerqarVjOXX69GnlwoULSnJystpR+mQ2m5ULFy4oiqIo\nbW1tysyZMwfl/uzu7lba29sVRVEUi8WiLFiwQDl37pzKqfr20UcfKWvXrlVWrlypdpQ+JSQkKM3N\nzWrHuKONGzcqhw4dUhRFUbq6upTr16+rnKh/VqtViY+PVxoaGtSO0ktjY6OSkJCg3Lx5U1EURcnK\nylKOHDnS75ghd4qsoqKCRx55hKCgIDw8PEhOTubEiRNqx3IqNjYWPz8/tWP0a8yYMURERADg4+ND\ncHAwZrNZ5VSONBoN3t7eAFitVqxWKxqNRuVUzjU2NnLy5EkWLFigdhS319bWxpkzZ+z70sPDA19f\nX5VT9e/HH38kKCiIBx98UO0oDmw2G52dnVitVjo7OxkzZky/yw+5AmM2mwkMDLQ/NxqNg/IPojtq\naGigsrKS6OhotaM4ZbPZSEtLIz4+nvj4+EGbMy8vjw0bNqDVDv6P57Jly5g3bx5ffPGF2lGcamho\nwGAwsHnzZp5++mm2bt1KR8fgnvPeZDKRkpKidgwHRqORpUuXkpCQwNSpU/Hx8WHq1Kn9jhn8v8HC\nLdy4cYOsrCy2bNmCj4+P2nGc0ul0FBYW8t1331FRUUFVVZXakRx8++23GAwGIiMj1Y5yR59//jkF\nBQV8+OGHHDx4kDNnzqgdyYHVauXixYukp6fz5Zdf4unpyQcffKB2rD5ZLBZKSkpISkpSO4qD69ev\nc+LECU6cOEFZWRk3b96ksLCw3zFDrsAYjUYaGxvtz81mM0ajUcVE7u/WrVtkZWWRmprKzJkz1Y5z\nR76+vkyePJmysjK1ozg4e/YsJSUlJCYmsnbtWk6dOsX69evVjuXU7c/NqFGjmDFjBhUVFSonchQY\nGEhgYKD9aDUpKYmLFy+qnKpvpaWlREREEBAQoHYUB+Xl5Tz00EMYDAaGDRvGzJkzOXfuXL9jhlyB\niYqKoq6ujvr6eiwWCyaTicTERLVjuS1FUdi6dSvBwcEsWbJE7Th9amlpobW1FYDOzk7Ky8sJDg5W\nOZWjdevWUVpaSklJCW+//TZxcXHs2rVL7VgOOjo6aG9vtz/+4YcfCA0NVTmVo9GjRxMYGEhtbS3Q\nc31j/PjxKqfqm8lkIjk5We0YTo0bN47z589z8+ZNFEUZ0L4ccrcp6/V6cnJyWL58uf1W0MH4wQBY\nu3Ytp0+f5tq1a0ybNo1XXnmFhQsXqh2rl19++YXCwkLCwsJIS0sDenI/+eSTKifrrampiezsbGw2\nG4qikJSUREJCgtqx3FZzczOZmZlAz7WtlJQUpk2bpnIq51599VXWr1/PrVu3CAoKYseOHWpHcqqj\no4Py8nJyc3PVjuJUdHQ0s2bN4plnnkGv1xMeHs5zzz3X7xiZD0YIIYRLDLlTZEIIIe4NKTBCCCFc\nQgqMEEIIl5ACI4QQwiWkwAghhHCJIXebshB/du3aNXvX2j/++AOtVovBYAAgPz8fDw+PO65j8+bN\nrFixot/v1hw8eJCRI0cyd+7cu5LbVaxWK3Fxcfz8889qRxFuTm5TFuKfvPvuu3h5ebFs2bJeP1cU\nBUVR3KI/2H9KCoy4W+QIRog+/P7777z00kuEh4dTWVnJxx9/zL59+/j111/p6upi9uzZrFq1CoD0\n9HRycnIIDQ0lLi6O559/ntLSUjw9PXnvvfcYNWoUu3fvxt/fn4yMDNLT05k0aRKnTp2ira2NHTt2\nEBMTQ0dHB5s2baKmpoaQkBAaGhrYvn074eHhvbJVVFTw5ptv0tHRgcFgYOfOnQQEBJCenk5kZCSn\nT5+mu7ubvLw8oqKiaGlpYcuWLVy5cgVvb29yc3MJCwujvb2d119/3d4+ZfXq1Tz11FMA7Nq1y2Eb\nhPhX3P//jgnxH6itrSUjI4Pi4mKMRiPr1q3j6NGjFBYWUl5ezqVLlxzGtLW1ERsby7Fjx3j88cc5\ncuSI03UrisLhw4fZuHEj+/fvB+DTTz8lICCA4uJiXn75ZSorKx3GWSwW8vLy2Lt3L0ePHmXu3Lns\n2bOn1+uFhYVs2bKFbdu2AbBnzx6io6MpKipi1apVZGdnA7Bv3z4MBgNFRUUcO3aM2NjYf2kbhOiP\nHMEI0Y+HH36YqKgo+3OTycThw4exWq00NTVx6dIlQkJCeo0ZMWKEvVVOREREn6eabjcGjYyM5MqV\nK0BP650VK1YA8OijjzqsG6Cmpobq6mp777fu7u5eDVtvt3p/4oknaG5u5saNG5w9e9Y+M+bUqVPJ\nzs62tya5Xdw0Gg1+fn5YrdYBb4MQ/ZECI0Q/PD097Y/r6ur45JNPyM/Px9fXl/Xr19PV1eUwZtiw\nYfbHOp0Om83mdN23bx7QarV9LuOMoihMmDCBzz77zOnrf55I7d+ZWG2g2yBEf+QUmRAD1N7ejre3\nNz4+PjQ1NfH999/f9feIiYnh66+/BuC3336jpqbGYZmQkBDMZrO9Pb7FYqG6utr+enFxMQA//fQT\nAQEBeHl5MWnSJIqKioCetutGoxEvLy+mTJliL1SKonD9+vW7vk1i6JIjGCEGKCIigvHjxzN79mzG\njRtHTEzMXX+PF154gU2bNjFnzhxCQkIYP368wwRuHh4e7N27lzfeeIP29na6u7tZsmSJvSu4Xq8n\nLS3NfpEfsE8Gl5qaire3t72jcGZmJq+99hopKSlotVpWr1496DphC/cltykLMYhYrVZsNhvDhw+n\nrq6OpUuX8s0336DXD+x/wdt3s/35rjMh1CBHMEIMIh0dHWRkZGC1WlEUhdzc3AEXFyEGGzmCEUII\n4RJykV8IIYRLSIERQgjhElJghBBCuIQUGCGEEC4hBUYIIYRL/B8PKO/q2suh5AAAAABJRU5ErkJg\ngg==\n",
"text/plain": [
"<Figure size 432x288 with 1 Axes>"
]
},
"metadata": {
"tags": []
}
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "MeVBDL6t104d",
"colab_type": "text"
},
"source": [
"Load the pretrained models"
]
},
{
"cell_type": "code",
"metadata": {
"id": "A89UF21w2E3_",
"colab_type": "code",
"outputId": "e1aa58e4-6ed8-4e24-8ff6-7500ebb1a084",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 34
}
},
"source": [
"encoder_model_name = \"ep_8_encoder_model.pth\"\n",
"decoder_model_name = \"ep_8_decoder_model.pth\"\n",
"\n",
"encoder_eval = encoder()\n",
"decoder_eval = decoder()\n",
"\n",
"encoder_eval.load_state_dict(torch.load(encoder_model_name))\n",
"decoder_eval.load_state_dict(torch.load(decoder_model_name))"
],
"execution_count": 0,
"outputs": [
{
"output_type": "execute_result",
"data": {
"text/plain": [
"<All keys matched successfully>"
]
},
"metadata": {
"tags": []
},
"execution_count": 20
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "YOM1AaqW2MQs",
"colab_type": "text"
},
"source": [
"Convert encoded transactional data to torch variable and reconstruct encoded transactional data"
]
},
{
"cell_type": "code",
"metadata": {
"id": "pEQPJAFK2u2b",
"colab_type": "code",
"colab": {}
},
"source": [
"data = autograd.Variable(torch_dataset)\n",
"\n",
"encoder_eval.eval()\n",
"decoder_eval.eval()\n",
"\n",
"reconstruction = decoder_eval(encoder_eval(data))"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "83lWuOlF4sWv",
"colab_type": "text"
},
"source": [
"Determine reconstruction loss for all transactions"
]
},
{
"cell_type": "code",
"metadata": {
"id": "YILTaSIbdm-p",
"colab_type": "code",
"outputId": "15fe65b5-f2b6-4ab7-ea6c-75fbbcada376",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 51
}
},
"source": [
"reconstruction_loss_all = loss_function(reconstruction, data)\n",
"\n",
"now = datetime.utcnow().strftime(\"%Y%m%d-%H:%M:%S\")\n",
"print('[LOG {}] collected reconstruction loss of: {:06}/{:06} transactions'.format(now, reconstruction.size()[0], reconstruction.size()[0]))\n",
"print('[LOG {}] reconstruction loss: {:.10f}'.format(now, reconstruction_loss_all.item()))"
],
"execution_count": 0,
"outputs": [
{
"output_type": "stream",
"text": [
"[LOG 20191231-19:47:52] collected reconstruction loss of: 533009/533009 transactions\n",
"[LOG 20191231-19:47:52] reconstruction loss: 0.0027732425\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "FzJsra7XiZt6",
"colab_type": "text"
},
"source": [
"Determine reconstruction loss for individual transactions"
]
},
{
"cell_type": "code",
"metadata": {
"id": "GSpHwP9oihwR",
"colab_type": "code",
"outputId": "c8825064-9707-432e-9871-cd952dd585d2",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 119
}
},
"source": [
"reconstruction_loss_transaction = np.zeros(reconstruction.size()[0])\n",
"\n",
"for i in range(0, reconstruction.size()[0]):\n",
"\n",
" reconstruction_loss_transaction[i] = loss_function(reconstruction[i],data[i]).item()\n",
"\n",
" if(i % 100000 == 0):\n",
" now = datetime.utcnow().strftime(\"%Y%m%d-%H:%M:%S\")\n",
" print('[LOG {}] collected individual reconstruction loss of: {:06}/{:06} transactions'.format(now, i, reconstruction.size()[0]))"
],
"execution_count": 0,
"outputs": [
{
"output_type": "stream",
"text": [
"[LOG 20191231-20:08:28] collected individual reconstruction loss of: 000000/533009 transactions\n",
"[LOG 20191231-20:08:36] collected individual reconstruction loss of: 100000/533009 transactions\n",
"[LOG 20191231-20:08:44] collected individual reconstruction loss of: 200000/533009 transactions\n",
"[LOG 20191231-20:08:52] collected individual reconstruction loss of: 300000/533009 transactions\n",
"[LOG 20191231-20:09:00] collected individual reconstruction loss of: 400000/533009 transactions\n",
"[LOG 20191231-20:09:09] collected individual reconstruction loss of: 500000/533009 transactions\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "4ijKSixrkvpg",
"colab_type": "text"
},
"source": [
"Plot data points with their reconstruction losses"
]
},
{
"cell_type": "code",
"metadata": {
"id": "QGJHS89vus39",
"colab_type": "code",
"outputId": "385ac0d3-ec4c-46ab-cdc4-368097120005",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 282
}
},
"source": [
"fig = plt.figure()\n",
"ax = fig.add_subplot(111)\n",
"\n",
"plot_data = np.column_stack((np.arange(len(reconstruction_loss_transaction)), reconstruction_loss_transaction))\n",
"\n",
"regular_data = plot_data[label == 'regular']\n",
"local_outliers = plot_data[label == 'local']\n",
"global_outliers = plot_data[label == 'global']\n",
"\n",
"ax.scatter(regular_data[:,0], regular_data[:,1], c='C0', alpha=0.4, marker='o', label='regular')\n",
"ax.scatter(local_outliers[:,0], local_outliers[:,1], c='C2', marker='^', label='local')\n",
"ax.scatter(global_outliers[:,0], global_outliers[:,1], c='C1', marker='^', label='global')\n",
"\n",
"ax.legend(loc='best')"
],
"execution_count": 0,
"outputs": [
{
"output_type": "execute_result",
"data": {
"text/plain": [
"<matplotlib.legend.Legend at 0x7f95b913c320>"
]
},
"metadata": {
"tags": []
},
"execution_count": 28
},
{
"output_type": "display_data",
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD4CAYAAADiry33AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjIsIGh0\ndHA6Ly9tYXRwbG90bGliLm9yZy8li6FKAAAgAElEQVR4nO3de3wTdbo/8M9M0qT3poWSFmm7IriC\nrbB7XEVBWVPaKuVO2d2zHgSOyC6yArrrnp/uisIu8BJUwBvColW5eBQUq0RBLNcKHGQRY7kJaG2L\nbSi9X3OZfH9/1ISmza1pLpPJ8369eGmTycx8ZyZPvvPMd57hGGMMhBBCJIsP9goQQgjxLwr0hBAi\ncRToCSFE4ijQE0KIxFGgJ4QQiZMHewW6s1gsEATvBwLJZFyfPi92Um8fQG2UAqm3DxBfGyMiZE7f\nE12gFwSGhoY2rz+vUkX36fNiJ/X2AdRGKZB6+wDxtTE5Oc7pe5S6IYQQiaNATwghEkeBnhBCJE50\nOXpHBMGM+voamM1Gt9Pq9RykXNXBUfvkcgUSE5Mhk4XE7iSEBFhIRIb6+hpERkYjJiYFHMe5nFYm\n4yEIlgCtWeB1bx9jDK2tTaivr0H//qlBXDNCiFiFROrGbDYiJibebZAPRxzHISYm3qOzHUJIeAqJ\nQA+AgrwLtG0IIa6ETKAnhLjHt+qRtPlOcK1Xgr0qREQo0ItATs5dwV4FIhHRJ9aCb6pAzIm1wV4V\nIiIU6L3AGIPFEpwLvsFcNhE3vlWPyLPvgQPr/C/16slPQmLUTW+V17XheHkDaloMSI5V4rZ0FdKT\novs0z6qqH/HYY3/C8OGZOH/+HO6//wF8+OH7MJmMGDhwEJ588mlER0fj6NESvPTSGkRGRuGWW0bg\nxx8vY9WqtXj99Q2IiorG738/EwAwc+ZvsGrVWqSmDrQto62tDU888Wc0NzfBbDbjoYfm4667fm23\n7G+/PYfVq9chJYVG2BB70SfWAtaht8yCmBNr0TJ2RXBXioiC5Hr05XVt+Oh0NVqNAgbEKdFqFPDR\n6WqU1/W9JkVlZQWmTp2Bl1/eiF27irB27at4442tuOmmYXj33a0wGAxYvXolnnvuRbzxxhbU19f3\nav4KhQIrVqzGG29sxYsvbsDLL6+1jZm3Lnvbth0U5EkPtt68pXP0FWcxUq+e2EiuR3/sh3rEKSMQ\nF9nZNOt/j5c39LlXn5KSiszMLHzxxWGUlX2H+fMfBACYzSbcfHMWysvLMHDgdRg48DoAQE5OHj76\naGevlrFhwyv4+uuvwHE8ampqUFdXa7dsQhyx681bUa+e/ERygb6m2YDkWIXda7FKGWqa+z7OPDIy\nEkBnnvzWW2/H0qX2X6ALF847/axMJgNj13LrRmPP9fnss0/R0NCA11/fArlcjoKCibbprMsmxBHF\n93ttvXkrzmKE4vvPAAr0YU9ygT45TomWDrOtJw8ALQYB/bsF/764+eYsvPDCs6isrMCgQWlob29H\nTc0VpKdn4McfL6Oq6kekpg5EcfFe22dSUwfiyJHDAIDz58+hqurHHvNtaWlBYmIi5HI5Tp48gerq\nKp+tM5G2utkngr0KRMQkF+hHZSRip64ziMYqZWgxCGg2mHDPkH4+W0ZiYiL+9rdn8Mwzf4PJ1NmL\neuih+UhPz8Bjj/0P/vznRxAZGYVhw4bbPvPrX2uwe7cW//Vfv8Hw4TcjLS29x3xzc+/D//zPo3jg\ngd/ippuGIyPjZz5bZ0JI+OKYyCqAmUxCj2L+1dU/ICUlw6PPy2Q8vq9pwfHyBlxtMaJ/rMIno248\n1dbWhujoaDDG8PzzzyItLQ2//e39Ppu/s1o+vdlGYie2Bzr4g9TbKPX2AeJro6sHj3jUoz906BCW\nL18Oi8WCGTNmYN68eXbvv/POO9i2bRt4nkd0dDT+8Y9/YMiQIQCADRs2YMeOHeB5Hn//+99x113+\nvzkoPSk6YIG9u48/3olPP9XCbDZh6NCfY/Lk6UFZD0IIsXLboxcEAXl5eSgsLIRarUZBQQFeeOEF\nWyAHOnPLsbGxAIDi4mJs27YNr7/+Oi5evIjHHnsMO3bsgF6vx5w5c7Bnzx7IZM6fbeiLHn04Va+0\noh59aJF6G6XePkB8bezTowR1Oh0yMjKQlpYGhUKB/Px8FBcX201jDfIA0N7ebiuyVVxcjPz8fCgU\nCqSlpSEjIwM6nc7bdhBCCPGC29SNXq9HSkqK7W+1Wu0wWG/duhWFhYUwmUx46623bJ8dMWKE3Wf1\ner3L5clkHFQq+7SLXs9BJvP83q7eTBuKHLWP43put1Alk/GSaYszUm+j1NsHhFYbfTbq5v7778f9\n99+Pjz/+GOvXr8ezzz7r1XwEgfU4HWKMeZyOCdfUDWM9t1uoEtspsT9IvY1Sbx8gvjb2KXWjVqtR\nXV1t+1uv10OtVjudPj8/H59//rlXnyWEEF+iss2d3Ab6rKwslJWVoaKiAkajEVqtFhqNxm6asrIy\n2/8fOHAAGRmdFwU1Gg20Wi2MRiMqKipQVlaGW265xbctCBBflxJ+/fUN2LZts0/nSQixR2WbO7lN\n3cjlcixZsgRz586FIAiYPn06hg4dinXr1iEzMxPZ2dnYsmULjh49Crlcjvj4eFvaZujQobjvvvsw\nfvx4yGQyLFmyxOWIG0II8ZXuZZtbb10MFjMg2KsVFB7l6MeOHYuxY8favbZo0SLb///97393+tn5\n8+dj/vz5Xq6e92o7rmLh0T/ipTs3IEnpu7tiGWN49dUXcezYF+A4DrNmPYjs7FwAwJYtb+Kzzz4F\nx/EYNepOzJ//CD76aCc++mgnTCYTBg0ahKee+gfVrSEkAKhs8zWSK4FgtfliIarbq7D5QiEWZf7F\nZ/M9eHAfLlw4jzfffAeNjQ2YO/cBjBjxS1y4cB4lJYewceNbiIyMRFNTIwBg7Nh7MGnSVADAxo2v\nYteuD1FQ8DufrQ8hpCdnZZvDtVcvyXGItR1XsbtSCwaGTyu1qDPU+mzeOt0pjBuXB5lMhqSkfvjF\nL36Jc+dO48SJ4xg/fqKttx4fnwAA+O67S3j44bl44IHfYu/e3fj+++98ti6EEMdclW0OR5IM9Jsv\nFsLy0062MAs2XygM2rqsWLEUjz76V7z99ruYM+chh+WJCSG+5bJscxiSXOrG2ps3MxMAwMxM+LRS\ni5lD5/gkVz9ixC9QVPQB7rtvApqamnDq1Fd4+OFFkMsj8Oabm5Cbe58tdRMfn4C2tlb0798fZrMZ\nn332KZKTw++0kZBAo7LN9iQX6N/69g1bb97K2qv3Ra7+7rvvQWnpN5g9+z/BcRwefngh+vXrj379\n+uPChW8xd+5MyOURuOOO0fjDHxZg7tz5mDdvNlQqFYYPz0Rbm3husCCEhAfJlSn+zb7JuNpR0+P1\n/spkvJdd5JN1DCYqaiYNUm+j1NsHiK+NfS5THErez/lY0iUQCCGktyR5MZYQIh5UhiD4KNATQvyK\nyhAEHwV6QojfdC9DQL364KBATwjxG0dlCEjgUaAnhPiFszIE1KsPPAr0fbR8+TPYv/9zl9MUFExE\nQ0ODx/P85JOP8cIL3j24hRCxoDIE4kGBnhDiF1SGQDwkN47eim/VQ/XBVNRP+9Bn1erefHMT9uz5\nBCpVIgYMUOPnPx9m9/6JE8fxyitrIQgCbrppOP7ylyegUCgAANu2vYVjx45AqVTi6aeXY9CgNJSU\nHMJbb70Os9mE+HgVnn76H0hK8l1JZUKCicoQiIdke/S+HtJ19uxpHDiwD2+++Q6ef/5FnD9/1u59\ng8GAFSuWYunSlXj77XchCAI+/HCH7f2YmFi8/fa7mDbtN3jxxecBALfcMhIbN76JwsJtGDcuF1u3\nvu2TdSWEkK4kGej9MaTrm2++xl13jYVSqUR0dAxGj7Z/tGB5+Q9ITR2I9PTOMgT33TcBp059ZXt/\n3Lg8AEBOzr0oLf0GAFBTcwWPPfYnPPDAb7Ft29tUwpgQ4heSDPRiHNLFcVyX/+/875o1qzB9+m/w\n9tvv4vHHn4TRaAjS2hFCpExygZ7z05CurKwR+OKLQzAYDGhra8MXX5TYvZ+enoGqqh9RWVkBANiz\n5xOMHPlL2/vFxXt/+u9nuPnmzgekt7a2oH//zusHu3dr+7R+hBDijOQuxkYdX+N0SFdfnhc5bNjN\nGD36bsya9Z9ISkrCDTfcgNjYWNv7SqUSTz75NJ566n9sF2OnTJlue7+5uQmzZv0OEREKPPPMcgDA\nf//3PDz11P9DXFwc/uM/foUff7zs9foR7/njwj0hYiK5MsX93rwVfGt1j9eFmJQ+jwJoa2tDdHQ0\nOjo6sGDBQ/jrX/+Gn//8pj7Ns7eoTLHvxR58ApGlW9CROTNgD48WW4lbX5N6+wDxtbHPZYoPHTqE\n5cuXw2KxYMaMGZg3b57d+4WFhdi+fftPz1FNwooVK3DdddcBAIYNG4Ybb7wRAJCamorXXnvN23Z4\npOHBk34rU7xq1XKUlX0Po9GA++6bEPAgT3yv+4X7cH14NJE2t4FeEAQsW7YMhYWFUKvVKCgogEaj\nwZAhQ2zTDBs2DO+//z6ioqKwbds2rF69GmvXdl4AjYyMRFFR6D/wA4At5UKkw9GF+0D16gkJFLcX\nY3U6HTIyMpCWlgaFQoH8/HwUFxfbTTNq1ChERUUBAEaOHInq6p6pk74SWYZJVGjbeIdqsZBw4bZH\nr9frkZKSYvtbrVZDp9M5nX7Hjh24++67bX8bDAZMmzYNcrkc8+bNw7hx41wuTybjoFJF273W2BiF\n9vZmxMYm2A1TdD4PyQ0mstO1fYwxtLQ0ISoqqsd2C1UyGR+QtvBHXwHQ/UfSgsRvXoHl3tV+XXag\n2hgsom1fczXkb4+HedanQKy6T7MSbRsd8Omom6KiIpSWlmLLli221/bv3w+1Wo2KigrMmjULN954\nI9LT053OQxBYjwscsbFJqK+vQVNTvdt14DhO0j1cR+2TyxVITEwW1YWhvgjURa6k85+AE7rVYhGM\nwDktGkYt9euyxXYhz9fE2r7Ygyshb/gB5uKVfU7Ria2NfboYq1ar7VIxer0eanXPX8IjR47gtdde\nw5YtW2z1XayfB4C0tDTcdtttOHPmjMtA74hMJkf//qkeTSu2je9rUm9fIFEtlvASzhfe3eY4srKy\nUFZWhoqKChiNRmi1Wmg0Grtpzpw5gyVLlmD9+vXo1+9aUa7GxkYYjZ09prq6Opw8edLuIi4hhASK\nGO+YDxS3PXq5XI4lS5Zg7ty5EAQB06dPx9ChQ7Fu3TpkZmYiOzsbq1atQltbGxYtWgTg2jDKS5cu\n4emnn7alGx566CEK9ISQgHN24T1cevUhccNUb0g9tSH19gHURikQW/tiDz6ByDPv2tXHZ7wCHcN/\n53WuXmxtdJWjl/bwFEIIAT0ERXK1bgghpLtwv/BOPXpCCJE4CvSEECJxFOgJIUTiKNATQojEUaAn\nhPTAt+qRtPlOKvAmERToCSE9RJ9YC76pIqzuHpUyCvSEEDvda8JQrz70UaAnhNgJ55owUkWBnhBi\nQw9jkSYK9E7QxSgSjux681bUqw95FOidoItRJByFe00YqaJaNw6E8wMKSHgL95owUkU9egfoYhQh\nxJ8CnRqmQN8NXYwihPhboFPDFOi7oYtRhBB/CsZ9ChTou6GLUYQQfwpGapguxnZDF6MIIf4SrGfX\nUo+eEEICJFipYY8C/aFDh5CXl4ecnBxs3Lixx/uFhYUYP348Jk6ciFmzZuHy5cu293bu3Inc3Fzk\n5uZi586dvltzQggJMcFKDbtN3QiCgGXLlqGwsBBqtRoFBQXQaDQYMmSIbZphw4bh/fffR1RUFLZt\n24bVq1dj7dq1aGhowMsvv4z3338fHMdh2rRp0Gg0SEhI8GujCCFEjIKVGnbbo9fpdMjIyEBaWhoU\nCgXy8/NRXFxsN82oUaMQFRUFABg5ciSqq6sBACUlJRg9ejRUKhUSEhIwevRoHD582A/NIIT4G5UF\nCV1uA71er0dKSortb7VaDb1e73T6HTt24O677/bqs4QQ8aKyIKHLp6NuioqKUFpaii1btng9D5mM\ng0oV3YfP8336vNhJvX0AtVGUmqshP7e9c+z3ufcgz34CiFU7nTzk2ueFUGqj20CvVqttqRigs5eu\nVvfcwUeOHMFrr72GLVu2QKFQ2D57/Phxu8/edtttLpcnCAwNDW0eN6A7lSq6T58XO6m3D6A2ilHs\nwZWQWyydf1gsMBevRMvYFU6nD7X2eUNsbUxOjnP6ntvUTVZWFsrKylBRUQGj0QitVguNRmM3zZkz\nZ7BkyRKsX78e/fr1s70+ZswYlJSUoLGxEY2NjSgpKcGYMWP60BRCSKBRWZDQ57ZHL5fLsWTJEsyd\nOxeCIGD69OkYOnQo1q1bh8zMTGRnZ2PVqlVoa2vDokWLAACpqal47bXXoFKp8PDDD6OgoAAAsGDB\nAqhUKv+2iBDiU67Gfrvq1RPx4BjrvgeDy2QSKHXjgtTbB1AbA6G24yoWHv0jXrpzA5KU/VxOm/Tm\nrZC1Vvd4XYhJcTpcMNjtCwSxtdFV6oZKIBAShjZfLER1exU2XyjEosy/uJyWyoLY41v1UH0wFZY5\newA4D65iQiUQCAkztR1XsbtSCwaGTyu1qDPUBnuVQop1mClf8lywV8VjFOgJCTObLxbC8lPG1sIs\n2HyhMMhrFDq6lhjmv94aMhekKdATEkasvXkzMwEAzMyEL8uKkPD2KJ8ELevds2iR5o2Rofr0OQr0\nhISRrr15qwfraiFvrvRJ0ArFtIanegwzFUJnmCkFekLCyBF9ia03DwD9zQImtTSBB/octEI1reGp\nUH76HAV6QsLIe9lF2Df+iO3fJwkaKLnOO9nBLIg5usLrwmWhmtbwVCg/fY6GVxISphze8Xr+AwAW\nxBxdiZZxa7yflxCYJycFUvdhpmIbR+8K9egJCVMOUxGwgAMQef6DXvXqQzmtEQ4o0BMSphymImz/\nJyDm6EoAntWhD+W0Rjig1I3IWe/Cq5/2oWROgf2NtplnuqYi+FY9kt78FTh0Vqi09upb73jCrg69\ns9o2oZzWCAfUoxc5ethD79E2673ooyuAn4L8NQJiDi+xjaQJlaGEpCcK9CLWdbgafck8Q9vMO8pL\nn3RJ23TiAER+96lkRtKE86MQKdCLFN+qR+LWsQD7qZcV4l+yQJH6ED9/YcoEJ28IkqlDH85nehTo\nRSr66ApwphZwls6bW6xfMqneWu4L9IAM71343W7kDh+F83O/Rs2CStQsqER75kyAV9hPGKI/nuF+\npkeBXoT4Vj0iz+/scSoNZpHkreW+EopD/MSSTuhatthKSiNpwv1MjwK9CEWfWIueF8Y6v2T8t58E\nfoVExllw7E1gEkuAFUM6wVnZ4rrZJ2y9+67/xFKf3tN9SGd6FOhF59op5jUMMlydfRI1CyphXng6\naOsmFs6CY28CkxgCrFjSCaFattjTfRiKZ3q+RoFeZBzfrXjt5pVw54vgKJYAK4Z0gqOyxaHwMJLe\n7EMppaC8RYFeZJzdrai8tCs4KyQyvgiOYgiwaK4WRTrBUdlin/Tqm6v9mhrrzT4UewoqECjQi0zd\n7BOonf1vMJnS7nWOsbDKKTrii1yrWPK1fMlzokgndC9bDHT26r/QH+7TfPmS5/yWGhPLPgwlHgX6\nQ4cOIS8vDzk5Odi4cWOP97/88ktMnToVw4cPx+7du+3eGzZsGCZPnozJkyfjj3/8o2/WWuLc5RTF\nciEx0DzNtbraPmLJ1/IXPhVFOqF72WLrv/eyi7yeJ9+qB6/b5rfUmFj2YShxW+tGEAQsW7YMhYWF\nUKvVKCgogEajwZAhQ2zTpKamYuXKlXjjjTd6fD4yMhJFRd4fNOHIVU7RAnhUe0SKXOZau2wHV9vH\n03n4m3nhacnWgukMxPY3+vnyOBXLPgwlbgO9TqdDRkYG0tLSAAD5+fkoLi62C/SDBg0CAPA8ZYJ8\nwVXuUGXN7f7UW5JSvW93PMmpdr9I1337iCkvK8Xia7btL9inVXx5nIppH4YKt4Fer9cjJSXF9rda\nrYZOp/N4AQaDAdOmTYNcLse8efMwbtw4l9PLZBxUqmiP59/z83yfPi928t1LAFhPWy1I/OYVWO5d\nHcxV8rm+7EP+6CsIhe0jk/FQ6V4B31wZnHVsrob87fEwz/oUiFX7bLb2299KvPuhL0Ip1vi9TPH+\n/fuhVqtRUVGBWbNm4cYbb0R6errT6QWB9emUVsrlUflWPZK+3nattyQYwZ/aivqsBZLpEQLO92Ft\nx1UsPPpHvHTnBiQp+/V4v3P7bA3K9ult71wla4L8663gmCUo+zD24ErIG36AuXilT9MqSec/sW1/\nK04wAue0aBi11GfLEQOxxZrk5Din77nNtajValRXV9v+1uv1UKs97wFYp01LS8Ntt92GM2fOePxZ\nYs8u92kVRhehHN2m31UwL9L19gYsu1E3Ad6H/ryPwDqU0fS3urAdyihGbgN9VlYWysrKUFFRAaPR\nCK1WC41G49HMGxsbYTR2/rrX1dXh5MmTdrl90juK7/f27C2FyY0fzm7T7ypYN8b0NnDaRqUEaXig\nKO4j8EBtx1Xcv79A9DdvhQK3qRu5XI4lS5Zg7ty5EAQB06dPx9ChQ7Fu3TpkZmYiOzsbOp0Of/rT\nn9DU1IT9+/fjpZdeglarxaVLl/D000+D4zgwxvDQQw9RoO+DutknRHe6GCiObtNflPkXu2nqZp+4\nNrRSMIDJIlE784jfUyKOAqerdIirMzN/j6JyNgZdjBf1u57Bdd/XpHc4xnrcbx9UJpNAOXoXpN4+\noGcbazuu4v4DBTB26a0reCW23bOjR64+9uATiDzzLjiLEYxXoGP47/waPLv+sFi5+4FJevNWyFqr\ne7wuxKT4PcXRdftY+WM79fU47brPne3rYBPbd7FPOfpwFq43JomNp7fpB+OOSW+uC9TNPtEjhx2o\nPHao1H0J1UJrYkWB3gUxVDgknt+mH4yLsaESOK1Coe5LqBZaEzO/D68MVe5uvAlloXajjqe34/fm\njklfbQMxBUipcHUGF4hcfah9PzxBPXonQmVkgjekeqYSavXoyTVd06T+KrTmKSkeG9SjdyCURib0\nlpTPVDxF20B8ugbXvhRU6yupHhvUo3dAytXxpHym4inaBuIilgfBANI9NijQOxBqF9g8RXW8aRuI\nkViCq5SPDUrdOCDVC2yuzlTCpdwxbQNxEVOa1OGxIXQg5uhKtIxbE9B18TXq0UuUo3sAxH6mYl1n\ntOj9tgyxb4Nw4/SH9+iKgN/DIuXHeFKPXqIcPXxD7GcqMUdXgm8qB/YvA+7yT0lbsW+DcOPsh1d5\nSQvO3BHQM62ux0bXO56tj/EM5Yuy1KOXoGBd3LIWoWqsP9fr3hjfqofy253gAPDfSCMvStxzNCS2\ndva/wTFLUC/OiuW6ga9QoBehvpZeCNZBai1CVbv/0V6PQ445uhJgQucfTOj8mwSEN1Ui/VkeJNhB\nVooXZSnQi1BfbtgI1kFqvW29n9mMEdWlveqNde3NA5150chvPwjpL1YocVfn3xF/3VQkhiArxeHV\nFOhFpq9pl2AdpNbb1v/Q0Ai+l70xu968FfXqA8KTOv/d+TM1KIYgK8UL9hToRaavp63BOEitwUJl\n6sCUlhYouizXk0CguKS19eatOkc7aP2xuqLiLG3iq9SIu/l4UyXSn6kVMQTZUCj81lthM+rG3fNG\nxcAXY4qDcTB27c1z3Z9u4MEYdaZMAMztDl6P9/Gaio+zh2s4GjXlDVfzcVYlcubQOU6/I/4e9x7K\nwVTMwqZH700eMtDEcNrqDWsRqnva2qHs9p4nvbHuPShrrXapf+mdpU18lRpxNx9P6/x3FarHaLgL\ni0DvTR4yGMRw2uqN97KLsG/8EfB//MHpKa/UH+LiTfucpU16mxpxtmx38/GmSmSoHqPhLixSN548\nb1QMpNyD9VUqwld8XXO8t+2raa9xmDaZfd0E9O9lasTRsj1JsXhTJTIYz+WVZH34ALdJ8j36cHpa\njVh7zWKqTmjly+GB3rRv0zcbHaZNavc/2qvUiLNl+zPFEuhx7rZ9FYSyCP4S6Jr3kg/03uQhQ5VY\nH5gQ7BtguvP1D4837Tt4+aDDtMngmnO9So04W7a/UiyBHudut6/O7xTl8d1bwej4eJS6OXToEJYv\nXw6LxYIZM2Zg3rx5du9/+eWXWLFiBc6fP48XXngB9957r+29nTt3Yv369QCA+fPnY+rUqT5cffdc\n5SHFmL7xllgfmCCm6oRWjoKjt+kkb9u3e+pnaGhoc/hejQ+W7a80YKCrf9ovT+i8mU5Ex7c3fHn8\necptj14QBCxbtgybNm2CVqvFrl27cPHiRbtpUlNTsXLlSkyYMMHu9YaGBrz88st47733sH37drz8\n8stobGz0bQvcsF4o7P4vmE+x8Qex9ZqtxDZKw9c90mC2T+oPQ++xr6xvMEE0x3dvBevOX7eBXqfT\nISMjA2lpaVAoFMjPz0dxcbHdNIMGDcJNN90EnrefXUlJCUaPHg2VSoWEhASMHj0ahw8H5rmPvibG\n/Ld1nfia00G/bdwZsY3S8HVwDGb7grHsQN5M5HBfAeAsJtEc370VrI6B29SNXq9HSkqK7W+1Wg2d\nTufRzB19Vq93XWtcJuOgUkV7NH/Hn+f79Hk7zdWQvz0e5lmfgte9Ar65EonfvALLvf4poeuJru3j\nj/60TvsWAej+hbAEfV0BwLL4DCxO3lM5ed2n+7Ab+Q+fOwyOkT/shVzV+y+bN+0DfNNGb5cdCL5o\nn6N9dU3wj29v2ujr48/j5fptzl4SBOY0d+kJlSq6T5/vKvbgSsgbfoB591OIvPgxOGYBf2or6rMW\nBC0/aG0f36pH0tdbwTELcPVczxICghE4p0XDqKVBWU8rb4aR+XIf9vDAl87f89cyHXDVRikMJ/TJ\nPvxpXyW9eStkrdV2b4nh+PaqjX48/pKT45y+5zbQq9VqVFdf28h6vR5qtdqjBavVahw/ftzus7fd\ndptHnw22hrqziD+9xXa1H9a0lEgeO2d3Csgr0D78d0FfJ0fENn4+FNA2syfa+0uaq5G0+d6Q+EF2\nm6PPyspCWVkZKioqYDQaoUmvc48AABk8SURBVNVqodFoPJr5mDFjUFJSgsbGRjQ2NqKkpARjxozp\n80oHQu2Bx+yv9ls6R+6IIf8thlKunhDj+HmxC5dtJsZrXr3FlzwXMsM93QZ6uVyOJUuWYO7cuRg/\nfjzuu+8+DB06FOvWrbNdlNXpdLj77ruxe/duPP3008jPzwcAqFQqPPzwwygoKEBBQQEWLFgAlSrY\n2UP3GurOYmR1qa1uS/e0SLBHtYhtJIszYh0JJGahss28eVhJV2K958NTfKsevG5byPwgc4w5uKwd\nRCaTEPQc/aUP7sMvq76xldt1RIhJCcoppUoVDX7t8B45y2CukyNdb5O38vR2eb/m6EXCURv7ss0C\nbW3panxc/iEmpU91eD+Ku2sQgSyh4A+xB59A5Nl3wQlGMF6BDhGkTvuUow83tR1Xcf2VMw6DvKNA\nGowLZ2IJ5q4E+sYaKQiVbda9SKCrssaOBOOGIV+ypdcE8dwE6I7kSyD01uaLhbg342fIuj7d9u8X\ng2/A38c6vtuwN6egUshLekps4+dDQahsM28eVmIVKteXXAmV1GlX1KPvpjclE3pbdiCcRlOEwlmH\n2ITCNvPmYSVdhcpZiyvOfpAjT28Vba+eAn03vSmN0JtTULHWoiGkN1wVCfSkdpTLs5YQCfTWH+Su\n1yFiDz6ByNItov3BokDvpd4Wswr1vCQhQN+LBIbCWUtvhUInjgK9l3pzCirGCo6EeCMUiwH6+3nR\nodCJo4uxXurNhbNQvHhDiCfcDTAQwwAEfz4vOlQuLlOg91JvqviFymgKIi5iCJLuuBt1Fuwbo/z9\nvOhQ6cRRoA+AQJZ2JdIR7CDpjrtyDWIo59CXoaCeCJVOHOXoCRGhULjA5y43HezcdV+HgnoiVDpr\n1KMnRITEXvPGbW66uTrouetwel60OxToCRGZULjA5y43zZc8F/TctauhoOGGUjeEiEwo3D3q7sYn\n/sKnQb8xKhSHgvoLBXoH/D3ulhBXAnX3aF+Oc3e5afPC05KvQBpKKNA70HXcrSd3+xHiS4G6wEfH\nefigHH03/h53S4gY0HEeXijQd+PvcbckdITCDUue6v5EKDrOwwsF+i6cjbul3k54EvsNS73RNU1D\nx3n4oUDfBY27JVZiuKvTV7qnaf51fj0d52GGAn0XNO6WWIn9hqXe6J6mOVS9n47zMOPRqJtDhw5h\n+fLlsFgsmDFjBubNm2f3vtFoxF//+lecPn0aKpUKa9aswaBBg1BZWYnx48fj+uuvBwCMGDECy5Yt\n830rfITG3RJAWmWlHaVpeMZjR/bHYTN0mIZLe9CjFwQBy5Ytw6ZNm6DVarFr1y5cvHjRbprt27cj\nPj4ee/fuxezZs/Hcc8/Z3ktPT0dRURGKiopEHeQJsQqVioSeoHSkf8sUW3W/2C02bgO9TqdDRkYG\n0tLSoFAokJ+fj+LiYrtp9u3bh6lTpwIA8vLycPToUbDuXxRCQkSoVCT0RLinIwM1jDQQPyZ94TZ1\no9frkZKSYvtbrVZDp9P1mCY1NbVzhnI54uLiUF9fDwCorKzElClTEBsbi8WLF+PWW291uTyZjINK\nFd3rhlz7PN+nz4ud1NsHBL+NlsVnYHHynspHywhUGz+bvtfvy3Ak2PvQav3xzWDo7HQyWPBu+WY8\n8asnfTJvaxtr2muw5/InYGDYfVmLBf/xMPpH9ffJMnzFr3fGDhgwAPv370diYiJKS0uxYMECaLVa\nxMbGOv2MILA+3Trd9YG9UiT19gHURinoTfv8lUOv7biKou+KYLJ0ntGYLCYUXSrCb9Nn+mQ51ja+\nUvoqBEtn10CwWPDKv18Nyp3GyclxTt9zm7pRq9Worq62/a3X66FWq3tMU1VVBQAwm81obm5GYmIi\nFAoFEhMTAQCZmZlIT0/H999/71UjCCHS5K+0RyCuT4TKPQluA31WVhbKyspQUVEBo9EIrVYLjUZj\nN41Go8HOnTsBAHv27MGoUaPAcRzq6uogCAIAoKKiAmVlZUhLS/NDM6TPerHnavvVYK8KIT7jzxx6\nIK5PhMrFbrepG7lcjiVLlmDu3LkQBAHTp0/H0KFDsW7dOmRmZiI7OxsFBQV4/PHHkZOTg4SEBKxZ\nswYA8OWXX+LFF1+EXC4Hz/NYunQpVCpfZTnDi7XX86/SjZg/dHGwV4cQn3BUisFXaY9ADJd29WMi\npkJxHBPZ8BiTSaAcfTe1HVdx/4ECGC1GKGVKbP31DkmPB/bVPhTz+Glv2yjmNnXlSfu6HtdWCl6J\nbfeExvEttljTpxw9CT4qQOUdsQ9584aU2hQqaQ8poEAvct0v9pgs4rzYIzZSKMPb/SYcKbSpq3Af\n4x9IFOh7IRh3v1GvxztSOAvq3nuXQpu6ei+7CPvGH+nxj0qR+B4FeiccBfVgnDZLpdcTyB9JsQ95\nq+24ikkfTXC5Pt1775eaLoi6TUTcKNA70T2oB+u0uXuv5+TvT4VkryeQP5JiPwvafLEQl1suu1yf\n7r335aeeEXWbwlFNe42o69t0RYHeAUdBXWqnzYEU6B9JMZ8FebItHJ2RlLV8L9o2hatN32wMmQvj\n9HBwB7oH9X+dexX7q4p7nDbPHDonJIaB9YY/hu/5c6y0I2I+2/FkWzg6I5FzEchPmyiqsdnhrLbj\nKj76/iPbD7bYYwH16Ltx1Jvae3kPLMy+zJVUe/W+TrEEM18uttKxnm4LMZ+RkE6dP8adMSEUYgEF\n+m4c5ndhgZmZ7V6T4hfPHykWf+fLXQVzsY0593Rb0GgUcbN+T6zF0vrSeQlUZ4QCfTeOelMA0F+Z\nLPkvnj+uQ/i7d+osmItxzDn11HviW/VI2nxnSD2T15edl0B1RqgEQojxV/vEdDu6p23sus7d13Vt\n6Wp8UrELZmYSZX6bjtNOsQefQGTpFnRkzkTL2BUBWLO++03xZFw11PR4vb8yuVedP1fHrzeoBAJx\nS+xDEh1xdgYi9nH0pJPt2bxgnf8NkV69NbVmHers7Rl+IEfy0agbAOV1bThe3oCaFgOSY5W4LV2F\n9KRot+8FYz2156+ivKbZ4bp0XVcZx6Gh3YjKxg4AQGZKPO4dNsDp9HubDzhMK+z/8SCuM/++T+33\nZBt2nSY9OQ5ZydE9pjn2fS3+9+RllNW1Q65oRkPyLljQcySUqx+tqQMf7vP+9PUx4W5+vVmep9P2\nZpkyjgMACIzZTetqHmVXW/B5abXL9bB7Nu9Pz+Tt3qsvr2vD/56sxImKRhgFC36WGI3f/XIgRl3f\nt7PM3u5DR9Nbn6DV/b1BCUpUNhpwsaYFzQYzOMbAOA7xSjluSI7FbekqxES3OeyM+Gv0Ttilbhzt\nlBOVjYhTRiBWKcPpH5tw7Ic6dJgYGBhilHKMylBhWEo8WgwCmg0m3DooAZWNhh4HybHva/HxaT30\nzQao45S4PV0FowW2HR6vlCMxWgHA/ksDwKNA+NHpagxQRUNmsaCivh0Xr7YiLSESidEKNLQbce5K\nK5KiI6COU6C0qhm1bSZkJEYjKoJHi8GMtMRozLx1kO1L+tHpasQpI3CluR1HyxrQZjQjUi7DgDgl\nlHIOA2KV+K62HSMHxWOQKgotBgGVDW2I4LkePyDO2lBe14bNJypxuaENV1tNECwMqmgFHhqVZvuy\ndl2XWKUMAs/jSkOb3XZu7jCh5FIdWk0CZBwg9PsALO44wAm2bWRN0XyhP+zw1FqJRIyRr8EgVRRi\nlTLb/px0c4rHwfDHxnb86+gPuNJshMligUIuQ3KsAg+NSseo6/t5FCC7Hqfd2979GLtY04LKxg4M\nTY7BIFWU3X63Bo2unRJ32xro/MH817FyNLSbIAgWyHgZIiN4jL2hn21ffnS6GmaBoayuFRevtkEh\n43Hnz1SQy2T45sdGtHSYcbXNhBilHIOTonF9/2i0GMxIjlGirs2I7+rbEcEBcp5DhEyGxGg5sof2\nt7Wpo/5HvFDz31DiWqrQzCtxetI+DLwuw9aeV0u+Q2l1CyI4oEOwoMPEEBUhw4wRKYiLUtjtm67H\noKsfp91nr+DYD/VIio7AzwfEQiGXodlgQoycx76LtahrMyEpOgIzRqRi5CCV0+n/646foampw27/\nVdS34+vLTRjcLwpXWozoMAm40mKAOi4SSjmHIcmxkPEc6qPexeGa3Xadq76mGF2lbiQV6Mvr2vBN\nTZutx2v9Zf2qoh4XrraitsWIDrMFCjkPVVQEBiZEosNswS0D45GWGI3vaprx2bkatJgsiOA5WCwW\nGAVAIedxXbwCBgtDfasJRrMFchlgYQA4DjERMlyfFIkfGgxQyjhwHIfqRgPMALif/gGwew5prBzg\n5TK0GQREyDjEKmTgeR4W1vnjMv6mZJgswOFLNahpNcFgZohXypDWLwYGkxmN7WaooiLQYTShqsWE\nxnYzOAD8T8m4CJ6DdcfGKuTgOIABGKSKQt5NA/B9bQvOVLegsr4NTQYLOHS+zwAo5BwSlHJYGKCI\nkCFKxiFVFYUWgwkVde0wWhgSoyOgkPEAs8BkAZrbzTAxC2QcjyjFteD3yRk9Sr6rg0FgUMg4REfw\naDVZYLF0fmFbDGYYf9owSjmHtIRIpPWLgb6pA21GAWMGJ2GQKgqbjpShts0MHoCMBxSDV4CLaOpx\nDCiYCr9iq/FVZROutpoADkhQysDzHBo6zBAsQGQEDyXPIVIhg1LOgec4RMpk6BAsUMp59ItWIFbB\no8EgQB2rQIdgQW2rCc3tRuibDWgxWGBBZ95TLgNilHL0j1FgcFIUdFUtkMs49IuKgMHCYDCaYRAY\nmjoEWJgF6YlRGK6OQ2Z6ErKSo3G8vAGtRgE1ze34uqoFzR0myDggKkKOnJsG4NuaZjR3CAAYVJFy\nnL3SilajABmAWKUMZgb8PDkG6YnR+HdFAy5dbUWzQbA71jgAkXIO8ZERyEhQQFfdAoMAyLjOfa2Q\n8ZDxHGIUEVDKObQbBRjMFhgtFsj4zp4oz/MwmMyIU0agts2I+jYTLKzzeOEAKOUAz/Od+4fjYGIA\nD4YYhRwGwQKDWUCcMgK3Z6jwQ10bfl/3IqbgAJTctdFsRshREnMvEia9gPSkaGz8ogw7v6lCh8kM\nBh5KOQ8ZB3SYzOA4HpOyUmwdkBM/1KKiwQALY5BxQKtRgFGwIEImQ5RChjilDOrYCFyq7UCLoXOZ\nnb8FHIb0j4bJLOBcTRsiZAAPDgwcjGYBSTERMAid+xFc55crJSESNw+Ixc8GxOF8VSMuN3bAwoBI\nOY92swVGk9l27JkFBhnPQSnnkRitgDpWiYykKHzS8gjMfEOP47e3ef6uwiLQW3sp1Y0dqG8zwSSq\nVhHiGesPrtQdVS5AKlff4/Uqlohfm19BjEKOxg4zhDDYGByAGAWPB24dhDl3/Mzr+Ug+0JfXteG3\nhSdgdj8pIYSI1lM5QzDploFefVbyo24e3EpBnhAS+v6x96Jf5iuJQN9gdD8NIYSEK0kEekIIIc5R\noCeEEInzKNAfOnQIeXl5yMnJwcaNG3u8bzQasXjxYuTk5GDGjBmorKy0vbdhwwbk5OQgLy8Phw+H\nb00PQggJFreBXhAELFu2DJs2bYJWq8WuXbtw8aL9BYPt27cjPj4ee/fuxezZs/Hcc88BAC5evAit\nVgutVotNmzZh6dKlEATB0WIIIYT4idtAr9PpkJGRgbS0NCgUCuTn56O4uNhumn379mHq1KkAgLy8\nPBw9ehSMMRQXFyM/Px8KhQJpaWnIyMiATqfzT0sIIYQ45LbWjV6vR0pKiu1vtVrdI1jr9XqkpqZ2\nzlAuR1xcHOrr66HX6zFixAi7z+r1epfLk8k4Ww0JQggJN/6If6IraiYITNLlWwkhxBVv41+fbphS\nq9Worq62/a3X66FWq3tMU1VVBQAwm81obm5GYmKiR58lhBDiX24DfVZWFsrKylBRUQGj0QitVguN\nRmM3jUajwc6dOwEAe/bswahRo8BxHDQaDbRaLYxGIyoqKlBWVoZbbrnF54348s93+3yehBASaP6K\nZR7Vujl48CBWrFgBQRAwffp0zJ8/H+vWrUNmZiays7NhMBjw+OOP4+zZs0hISMCaNWuQlpYGAFi/\nfj3ef/99yGQyPPnkkxg7dqzLZdETplyTevsAaqMUSL19gPjaKPmiZl2JbeP7mtTbB1AbpUDq7QPE\n10bJFzUjhBDiHAV6QgiROAr0hBAicRToCSFE4ijQE0KIxFGgJ4QQiaNATwghEkeBnhBCJI4CPSGE\nSBwFekIIkTgK9IQQInEU6AkhROJEV9SMEEKIb1GPnhBCJI4CPSGESBwFekIIkTgK9IQQInEU6Akh\nROIo0BNCiMRRoCeEEImTTKA/dOgQ8vLykJOTg40bNwZ7dRx64okncMcdd2DChAm21xoaGjBnzhzk\n5uZizpw5aGxsBAAwxvDPf/4TOTk5mDhxIk6fPm37zM6dO5Gbm4vc3Fzs3LnT9nppaSkmTpyInJwc\n/POf/4T1Fglny/C1qqoqzJw5E+PHj0d+fj7eeustSbXRYDCgoKAAkyZNQn5+Pl588UUAQEVFBWbM\nmIGcnBwsXrwYRqMRAGA0GrF48WLk5ORgxowZqKystM1rw4YNyMnJQV5eHg4fPmx73dlx7GwZ/iII\nAqZMmYI//OEPkmyjRqPBxIkTMXnyZEybNg2AdI5Th5gEmM1mlp2dzcrLy5nBYGATJ05kFy5cCPZq\n9XD8+HFWWlrK8vPzba89++yzbMOGDYwxxjZs2MBWrVrFGGPswIED7MEHH2QWi4V99dVXrKCggDHG\nWH19PdNoNKy+vp41NDQwjUbDGhoaGGOMTZ8+nX311VfMYrGwBx98kB04cMDlMnxNr9ez0tJSxhhj\nzc3NLDc3l124cEEybbRYLKylpYUxxpjRaGQFBQXsq6++YgsXLmS7du1ijDH21FNPsa1btzLGGNuy\nZQt76qmnGGOM7dq1iy1atIgxxtiFCxfYxIkTmcFgYOXl5Sw7O5uZzWaXx7GzZfjLG2+8wR577DE2\nb948l8sP1Tbec889rLa21u41qRynjkiiR6/T6ZCRkYG0tDQoFArk5+ejuLg42KvVw69+9SskJCTY\nvVZcXIwpU6YAAKZMmYLPP//c7nWO4zBy5Eg0NTXhypUrKCkpwejRo6FSqZCQkIDRo0fj8OHDuHLl\nClpaWjBy5EhwHIcpU6bYtoGzZfjagAEDcPPNNwMAYmNjMXjwYOj1esm0keM4xMTEAADMZjPMZjM4\njsOxY8eQl5cHAJg6daptnfbt24epU6cCAPLy8nD06FEwxlBcXIz8/HwoFAqkpaUhIyMDOp3O6XHM\nGHO6DH+orq7GgQMHUFBQAAAulx+qbXREKsepI5II9Hq9HikpKba/1Wo19Hp9ENfIc7W1tRgwYAAA\nIDk5GbW1tQB6tiklJQV6vd5pW51N72oZ/lRZWYmzZ89ixIgRkmqjIAiYPHky7rzzTtx5551IS0tD\nfHw85HJ5j3XS6/VITU0FAMjlcsTFxaG+vt7j9llfr6+vd7oMf1ixYgUef/xx8HxneHC1/FBtIwA8\n+OCDmDZtGt59910A0v0uAoA8IEshHuE4DhzHhfwyWltbsXDhQjz55JOIjY0N+PL9uQyZTIaioiI0\nNTVhwYIF+O677/yynGDZv38/kpKSkJmZif/7v/8L9ur4zTvvvAO1Wo3a2lrMmTMHgwcPtns/1I/T\n7iTRo1er1aiurrb9rdfroVarg7hGnuvXrx+uXLkCALhy5QqSkpIA9GxTdXU11Gq107Y6m97VMvzB\nZDJh4cKFmDhxInJzcyXZRgCIj4/H7bffjlOnTqGpqQlms7nHOqnValRVVQHoTPU0NzcjMTHR4/ZZ\nX09MTHS6DF87efIk9u3bB41Gg8ceewzHjh3D8uXLJdVG63oDncdNTk4OdDqdJI9TK0kE+qysLJSV\nlaGiogJGoxFarRYajSbYq+URjUaDDz/8EADw4YcfIjs72+51xhhOnTqFuLg4DBgwAGPGjEFJSQka\nGxvR2NiIkpISjBkzBgMGDEBsbCxOnToFxpjDeXVfhq8xxvC3v/0NgwcPxpw5cyTXxrq6OjQ1NQEA\nOjo6cOTIEdxwww24/fbbsWfPHgCdozCsx55Go7GNxNizZw9GjRoFjuOg0Wig1WphNBpRUVGBsrIy\n3HLLLU6PY47jnC7D1/785z/j0KFD2LdvH1544QWMGjUKzz//vKTa2NbWhpaWFtv/f/HFFxg6dKhk\njlOHAnLJNwAOHDjAcnNzWXZ2Nnv11VeDvToOPfroo2z06NFs+PDh7K677mLvvfceq6urYw888ADL\nyclhs2bNYvX19YyxzhEezzzzDMvOzmYTJkxgOp3ONp/t27ezcePGsXHjxrEdO3bYXtfpdCw/P59l\nZ2ezpUuXMovFwhhjTpfha19++SW78cYb2YQJE9ikSZPYpEmT2IEDByTTxrNnz7LJkyezCRMmsPz8\nfPbSSy8xxhgrLy9n06dPZ+PGjWOPPPIIMxgMjDHGOjo62COPPMLGjRvHpk+fzsrLy23zevXVV1l2\ndjbLzc21jchgzPlx7GwZ/nTs2DHbqBsptbG8vJxNnDiRTZw4kY0fP962DlI5Th2hevSEECJxkkjd\nEEIIcY4CPSGESBwFekIIkTgK9IQQInEU6AkhROIo0BNCiMRRoCeEEIn7/yA6SdYYyeqyAAAAAElF\nTkSuQmCC\n",
"text/plain": [
"<Figure size 432x288 with 1 Axes>"
]
},
"metadata": {
"tags": []
}
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "0R5Hxe_h1F14",
"colab_type": "text"
},
"source": [
"Finally, we look at the number of local and global anomalies detected"
]
},
{
"cell_type": "code",
"metadata": {
"id": "eT4yCVFb3RpN",
"colab_type": "code",
"outputId": "8f19d6d7-7c52-496d-90eb-ca4fa671f493",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 68
}
},
"source": [
"ori_dataset['label'] = label\n",
"ori_dataset[reconstruction_loss_transaction >= 0.1].label.value_counts()"
],
"execution_count": 0,
"outputs": [
{
"output_type": "execute_result",
"data": {
"text/plain": [
"global 55\n",
"local 8\n",
"Name: label, dtype: int64"
]
},
"metadata": {
"tags": []
},
"execution_count": 29
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "I9zarUpN4Brb",
"colab_type": "code",
"outputId": "3cb7586f-4aca-4bb3-bc73-bcec3143e109",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 68
}
},
"source": [
"ori_dataset[(reconstruction_loss_transaction >= 0.018) & (reconstruction_loss_transaction < 0.05)].label.value_counts()"
],
"execution_count": 0,
"outputs": [
{
"output_type": "execute_result",
"data": {
"text/plain": [
"local 15\n",
"global 1\n",
"Name: label, dtype: int64"
]
},
"metadata": {
"tags": []
},
"execution_count": 30
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "NDEzIUBu4ddX",
"colab_type": "text"
},
"source": [
"We identified 80% (56/70) of the global anomalies and 77% (23/30) of the local anomalies which is excellent when you keep in mind that outliers were only 0.018% of the whole dataset."
]
}
]
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment