{"cells": [{"cell_type": "markdown", "metadata": {}, "source": ["# 2A.ml - R\u00e9duction d'une for\u00eat al\u00e9atoire - correction\n", "\n", "Le mod\u00e8le Lasso permet de s\u00e9lectionner des variables, une for\u00eat al\u00e9atoire produit une pr\u00e9diction comme \u00e9tant la moyenne d'arbres de r\u00e9gression. Et si on m\u00e9langeait les deux ?"]}, {"cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [{"data": {"text/html": ["<div id=\"my_id_menu_nb\">run previous cell, wait for 2 seconds</div>\n", "<script>\n", "function repeat_indent_string(n){\n", "    var a = \"\" ;\n", "    for ( ; n > 0 ; --n)\n", "        a += \"    \";\n", "    return a;\n", "}\n", "// look up into all sections and builds an automated menu //\n", "var update_menu_string = function(begin, lfirst, llast, sformat, send, keep_item, begin_format, end_format) {\n", "    var anchors = document.getElementsByClassName(\"section\");\n", "    if (anchors.length == 0) {\n", "        anchors = document.getElementsByClassName(\"text_cell_render rendered_html\");\n", "    }\n", "    var i,t;\n", "    var text_menu = begin;\n", "    var text_memo = \"<pre>\\nlength:\" + anchors.length + \"\\n\";\n", "    var ind = \"\";\n", "    var memo_level = 1;\n", "    var href;\n", "    var tags = [];\n", "    var main_item = 0;\n", "    var format_open = 0;\n", "    for (i = 0; i <= llast; i++)\n", "        tags.push(\"h\" + i);\n", "\n", "    for (i = 0; i < anchors.length; i++) {\n", "        text_memo += \"**\" + anchors[i].id + \"--\\n\";\n", "\n", "        var child = null;\n", "        for(t = 0; t < tags.length; t++) {\n", "            var r = anchors[i].getElementsByTagName(tags[t]);\n", "            if (r.length > 0) {\n", "child = r[0];\n", "break;\n", "            }\n", "        }\n", "        if (child == null) {\n", "            text_memo += \"null\\n\";\n", "            continue;\n", "        }\n", "        if (anchors[i].hasAttribute(\"id\")) {\n", "            // when converted in RST\n", "            href = anchors[i].id;\n", "            text_memo += \"#1-\" + href;\n", "            // passer \u00e0 child suivant (le chercher)\n", "        }\n", "        else if (child.hasAttribute(\"id\")) {\n", "            // in a notebook\n", "            href = child.id;\n", "            text_memo += \"#2-\" + href;\n", "        }\n", "        else {\n", "            text_memo += \"#3-\" + \"*\" + \"\\n\";\n", "            continue;\n", "        }\n", "        var title = child.textContent;\n", "        var level = parseInt(child.tagName.substring(1,2));\n", "\n", "        text_memo += \"--\" + level + \"?\" + lfirst + \"--\" + title + \"\\n\";\n", "\n", "        if ((level < lfirst) || (level > llast)) {\n", "            continue ;\n", "        }\n", "        if (title.endsWith('\u00b6')) {\n", "            title = title.substring(0,title.length-1).replace(\"<\", \"&lt;\")\n", "         .replace(\">\", \"&gt;\").replace(\"&\", \"&amp;\");\n", "        }\n", "        if (title.length == 0) {\n", "            continue;\n", "        }\n", "\n", "        while (level < memo_level) {\n", "            text_menu += end_format + \"</ul>\\n\";\n", "            format_open -= 1;\n", "            memo_level -= 1;\n", "        }\n", "        if (level == lfirst) {\n", "            main_item += 1;\n", "        }\n", "        if (keep_item != -1 && main_item != keep_item + 1) {\n", "            // alert(main_item + \" - \" + level + \" - \" + keep_item);\n", "            continue;\n", "        }\n", "        while (level > memo_level) {\n", "            text_menu += \"<ul>\\n\";\n", "            memo_level += 1;\n", "        }\n", "        text_menu += repeat_indent_string(level-2);\n", "        text_menu += begin_format + sformat.replace(\"__HREF__\", href).replace(\"__TITLE__\", title);\n", "        format_open += 1;\n", "    }\n", "    while (1 < memo_level) {\n", "        text_menu += end_format + \"</ul>\\n\";\n", "        memo_level -= 1;\n", "        format_open -= 1;\n", "    }\n", "    text_menu += send;\n", "    //text_menu += \"\\n\" + text_memo;\n", "\n", "    while (format_open > 0) {\n", "        text_menu += end_format;\n", "        format_open -= 1;\n", "    }\n", "    return text_menu;\n", "};\n", "var update_menu = function() {\n", "    var sbegin = \"\";\n", "    var sformat = '<a href=\"#__HREF__\">__TITLE__</a>';\n", "    var send = \"\";\n", "    var begin_format = '<li>';\n", "    var end_format = '</li>';\n", "    var keep_item = -1;\n", "    var text_menu = update_menu_string(sbegin, 2, 4, sformat, send, keep_item,\n", "       begin_format, end_format);\n", "    var menu = document.getElementById(\"my_id_menu_nb\");\n", "    menu.innerHTML=text_menu;\n", "};\n", "window.setTimeout(update_menu,2000);\n", "            </script>"], "text/plain": ["<IPython.core.display.HTML object>"]}, "execution_count": 2, "metadata": {}, "output_type": "execute_result"}], "source": ["from jyquickhelper import add_notebook_menu\n", "add_notebook_menu()"]}, {"cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": ["%matplotlib inline"]}, {"cell_type": "markdown", "metadata": {}, "source": ["## Datasets\n", "\n", "Comme il faut toujours des donn\u00e9es, on prend ce jeu [Diabetes](https://scikit-learn.org/stable/modules/generated/sklearn.datasets.load_diabetes.html)."]}, {"cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [], "source": ["from sklearn.datasets import load_diabetes\n", "data = load_diabetes()\n", "X, y = data.data, data.target"]}, {"cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [], "source": ["from sklearn.model_selection import train_test_split\n", "X_train, X_test, y_train, y_test = train_test_split(X, y)"]}, {"cell_type": "markdown", "metadata": {}, "source": ["## Une for\u00eat al\u00e9atoire"]}, {"cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [{"data": {"text/html": ["<style>#sk-container-id-1 {color: black;background-color: white;}#sk-container-id-1 pre{padding: 0;}#sk-container-id-1 div.sk-toggleable {background-color: white;}#sk-container-id-1 label.sk-toggleable__label {cursor: pointer;display: block;width: 100%;margin-bottom: 0;padding: 0.3em;box-sizing: border-box;text-align: center;}#sk-container-id-1 label.sk-toggleable__label-arrow:before {content: \"\u25b8\";float: left;margin-right: 0.25em;color: #696969;}#sk-container-id-1 label.sk-toggleable__label-arrow:hover:before {color: black;}#sk-container-id-1 div.sk-estimator:hover label.sk-toggleable__label-arrow:before {color: black;}#sk-container-id-1 div.sk-toggleable__content {max-height: 0;max-width: 0;overflow: hidden;text-align: left;background-color: #f0f8ff;}#sk-container-id-1 div.sk-toggleable__content pre {margin: 0.2em;color: black;border-radius: 0.25em;background-color: #f0f8ff;}#sk-container-id-1 input.sk-toggleable__control:checked~div.sk-toggleable__content {max-height: 200px;max-width: 100%;overflow: auto;}#sk-container-id-1 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {content: \"\u25be\";}#sk-container-id-1 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-1 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-1 input.sk-hidden--visually {border: 0;clip: rect(1px 1px 1px 1px);clip: rect(1px, 1px, 1px, 1px);height: 1px;margin: -1px;overflow: hidden;padding: 0;position: absolute;width: 1px;}#sk-container-id-1 div.sk-estimator {font-family: monospace;background-color: #f0f8ff;border: 1px dotted black;border-radius: 0.25em;box-sizing: border-box;margin-bottom: 0.5em;}#sk-container-id-1 div.sk-estimator:hover {background-color: #d4ebff;}#sk-container-id-1 div.sk-parallel-item::after {content: \"\";width: 100%;border-bottom: 1px solid gray;flex-grow: 1;}#sk-container-id-1 div.sk-label:hover label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-1 div.sk-serial::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: 0;}#sk-container-id-1 div.sk-serial {display: flex;flex-direction: column;align-items: center;background-color: white;padding-right: 0.2em;padding-left: 0.2em;position: relative;}#sk-container-id-1 div.sk-item {position: relative;z-index: 1;}#sk-container-id-1 div.sk-parallel {display: flex;align-items: stretch;justify-content: center;background-color: white;position: relative;}#sk-container-id-1 div.sk-item::before, #sk-container-id-1 div.sk-parallel-item::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: -1;}#sk-container-id-1 div.sk-parallel-item {display: flex;flex-direction: column;z-index: 1;position: relative;background-color: white;}#sk-container-id-1 div.sk-parallel-item:first-child::after {align-self: flex-end;width: 50%;}#sk-container-id-1 div.sk-parallel-item:last-child::after {align-self: flex-start;width: 50%;}#sk-container-id-1 div.sk-parallel-item:only-child::after {width: 0;}#sk-container-id-1 div.sk-dashed-wrapped {border: 1px dashed gray;margin: 0 0.4em 0.5em 0.4em;box-sizing: border-box;padding-bottom: 0.4em;background-color: white;}#sk-container-id-1 div.sk-label label {font-family: monospace;font-weight: bold;display: inline-block;line-height: 1.2em;}#sk-container-id-1 div.sk-label-container {text-align: center;}#sk-container-id-1 div.sk-container {/* jupyter's `normalize.less` sets `[hidden] { display: none; }` but bootstrap.min.css set `[hidden] { display: none !important; }` so we also need the `!important` here to be able to override the default hidden behavior on the sphinx rendered scikit-learn.org. See: https://github.com/scikit-learn/scikit-learn/issues/21755 */display: inline-block !important;position: relative;}#sk-container-id-1 div.sk-text-repr-fallback {display: none;}</style><div id=\"sk-container-id-1\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>RandomForestRegressor()</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-1\" type=\"checkbox\" checked><label for=\"sk-estimator-id-1\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">RandomForestRegressor</label><div class=\"sk-toggleable__content\"><pre>RandomForestRegressor()</pre></div></div></div></div></div>"], "text/plain": ["RandomForestRegressor()"]}, "execution_count": 6, "metadata": {}, "output_type": "execute_result"}], "source": ["from sklearn.ensemble import RandomForestRegressor as model_class\n", "clr = model_class()\n", "clr.fit(X_train, y_train)"]}, {"cell_type": "markdown", "metadata": {}, "source": ["Le nombre d'arbres est..."]}, {"cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [{"data": {"text/plain": ["100"]}, "execution_count": 7, "metadata": {}, "output_type": "execute_result"}], "source": ["len(clr.estimators_)"]}, {"cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [{"data": {"text/plain": ["0.3625404922781166"]}, "execution_count": 8, "metadata": {}, "output_type": "execute_result"}], "source": ["from sklearn.metrics import r2_score\n", "r2_score(y_test, clr.predict(X_test))"]}, {"cell_type": "markdown", "metadata": {}, "source": ["## Random Forest = moyenne des pr\u00e9dictions\n", "\n", "On recommence en faisant la moyenne soi-m\u00eame."]}, {"cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [{"data": {"text/plain": ["0.3625404922781166"]}, "execution_count": 9, "metadata": {}, "output_type": "execute_result"}], "source": ["import numpy\n", "dest = numpy.zeros((X_test.shape[0], len(clr.estimators_)))\n", "estimators = numpy.array(clr.estimators_).ravel()\n", "for i, est in enumerate(estimators):\n", "    pred = est.predict(X_test)\n", "    dest[:, i] = pred\n", "\n", "average = numpy.mean(dest, axis=1)\n", "r2_score(y_test, average)"]}, {"cell_type": "markdown", "metadata": {}, "source": ["A priori, c'est la m\u00eame chose."]}, {"cell_type": "markdown", "metadata": {}, "source": ["## Pond\u00e9rer les arbres \u00e0 l'aide d'une r\u00e9gression lin\u00e9aire\n", "\n", "La for\u00eat al\u00e9atoire est une fa\u00e7on de cr\u00e9er de nouvelles features, 100 exactement qu'on utilise pour caler une r\u00e9gression lin\u00e9aire."]}, {"cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [{"data": {"text/html": ["<style>#sk-container-id-2 {color: black;background-color: white;}#sk-container-id-2 pre{padding: 0;}#sk-container-id-2 div.sk-toggleable {background-color: white;}#sk-container-id-2 label.sk-toggleable__label {cursor: pointer;display: block;width: 100%;margin-bottom: 0;padding: 0.3em;box-sizing: border-box;text-align: center;}#sk-container-id-2 label.sk-toggleable__label-arrow:before {content: \"\u25b8\";float: left;margin-right: 0.25em;color: #696969;}#sk-container-id-2 label.sk-toggleable__label-arrow:hover:before {color: black;}#sk-container-id-2 div.sk-estimator:hover label.sk-toggleable__label-arrow:before {color: black;}#sk-container-id-2 div.sk-toggleable__content {max-height: 0;max-width: 0;overflow: hidden;text-align: left;background-color: #f0f8ff;}#sk-container-id-2 div.sk-toggleable__content pre {margin: 0.2em;color: black;border-radius: 0.25em;background-color: #f0f8ff;}#sk-container-id-2 input.sk-toggleable__control:checked~div.sk-toggleable__content {max-height: 200px;max-width: 100%;overflow: auto;}#sk-container-id-2 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {content: \"\u25be\";}#sk-container-id-2 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-2 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-2 input.sk-hidden--visually {border: 0;clip: rect(1px 1px 1px 1px);clip: rect(1px, 1px, 1px, 1px);height: 1px;margin: -1px;overflow: hidden;padding: 0;position: absolute;width: 1px;}#sk-container-id-2 div.sk-estimator {font-family: monospace;background-color: #f0f8ff;border: 1px dotted black;border-radius: 0.25em;box-sizing: border-box;margin-bottom: 0.5em;}#sk-container-id-2 div.sk-estimator:hover {background-color: #d4ebff;}#sk-container-id-2 div.sk-parallel-item::after {content: \"\";width: 100%;border-bottom: 1px solid gray;flex-grow: 1;}#sk-container-id-2 div.sk-label:hover label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-2 div.sk-serial::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: 0;}#sk-container-id-2 div.sk-serial {display: flex;flex-direction: column;align-items: center;background-color: white;padding-right: 0.2em;padding-left: 0.2em;position: relative;}#sk-container-id-2 div.sk-item {position: relative;z-index: 1;}#sk-container-id-2 div.sk-parallel {display: flex;align-items: stretch;justify-content: center;background-color: white;position: relative;}#sk-container-id-2 div.sk-item::before, #sk-container-id-2 div.sk-parallel-item::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: -1;}#sk-container-id-2 div.sk-parallel-item {display: flex;flex-direction: column;z-index: 1;position: relative;background-color: white;}#sk-container-id-2 div.sk-parallel-item:first-child::after {align-self: flex-end;width: 50%;}#sk-container-id-2 div.sk-parallel-item:last-child::after {align-self: flex-start;width: 50%;}#sk-container-id-2 div.sk-parallel-item:only-child::after {width: 0;}#sk-container-id-2 div.sk-dashed-wrapped {border: 1px dashed gray;margin: 0 0.4em 0.5em 0.4em;box-sizing: border-box;padding-bottom: 0.4em;background-color: white;}#sk-container-id-2 div.sk-label label {font-family: monospace;font-weight: bold;display: inline-block;line-height: 1.2em;}#sk-container-id-2 div.sk-label-container {text-align: center;}#sk-container-id-2 div.sk-container {/* jupyter's `normalize.less` sets `[hidden] { display: none; }` but bootstrap.min.css set `[hidden] { display: none !important; }` so we also need the `!important` here to be able to override the default hidden behavior on the sphinx rendered scikit-learn.org. See: https://github.com/scikit-learn/scikit-learn/issues/21755 */display: inline-block !important;position: relative;}#sk-container-id-2 div.sk-text-repr-fallback {display: none;}</style><div id=\"sk-container-id-2\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>LinearRegression()</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-2\" type=\"checkbox\" checked><label for=\"sk-estimator-id-2\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">LinearRegression</label><div class=\"sk-toggleable__content\"><pre>LinearRegression()</pre></div></div></div></div></div>"], "text/plain": ["LinearRegression()"]}, "execution_count": 10, "metadata": {}, "output_type": "execute_result"}], "source": ["from sklearn.linear_model import LinearRegression\n", "\n", "\n", "def new_features(forest, X):\n", "    dest = numpy.zeros((X.shape[0], len(forest.estimators_)))\n", "    estimators = numpy.array(forest.estimators_).ravel()\n", "    for i, est in enumerate(estimators):\n", "        pred = est.predict(X)\n", "        dest[:, i] = pred\n", "    return dest\n", "\n", "\n", "X_train_2 = new_features(clr, X_train)\n", "lr = LinearRegression()\n", "lr.fit(X_train_2, y_train)"]}, {"cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [{"data": {"text/plain": ["0.30414556638121215"]}, "execution_count": 11, "metadata": {}, "output_type": "execute_result"}], "source": ["X_test_2 = new_features(clr, X_test)\n", "r2_score(y_test, lr.predict(X_test_2))"]}, {"cell_type": "markdown", "metadata": {}, "source": ["Un peu moins bien, un peu mieux, le risque d'overfitting est un peu plus grand avec ces nombreuses features car la base d'apprentissage ne contient que 379 observations (regardez ``X_train.shape`` pour v\u00e9rifier)."]}, {"cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [{"data": {"text/plain": ["array([ 0.0129567 , -0.03467343, -0.02574902,  0.01872549,  0.00128276,\n", "       -0.01449147,  0.00977528, -0.02397026,  0.01066261,  0.02121925,\n", "        0.03544455,  0.02735311,  0.01859875, -0.03189411, -0.0245749 ,\n", "       -0.01879966,  0.01521987,  0.00292998,  0.04250576,  0.01424533,\n", "       -0.00561623,  0.00635399,  0.04712406,  0.02518721,  0.01713507,\n", "        0.01741708, -0.02072389,  0.05748854,  0.00424951,  0.02872275,\n", "       -0.01016485,  0.04368062,  0.07377962,  0.06540726, -0.00123185,\n", "        0.02227104,  0.0289425 ,  0.00914512,  0.03645644,  0.01838009,\n", "        0.00046509,  0.04145444,  0.0202303 ,  0.00984027,  0.0149448 ,\n", "       -0.01129977,  0.00428108,  0.02601842,  0.00421449, -0.01172942,\n", "        0.02631074,  0.04180424,  0.02909078, -0.01922766, -0.00953341,\n", "       -0.0036882 , -0.02411783,  0.06700977, -0.01447105,  0.02094102,\n", "        0.00227497,  0.04181756, -0.02474879,  0.0465355 ,  0.05504502,\n", "       -0.05645067, -0.02066304,  0.04349629, -0.01549704,  0.02805018,\n", "        0.01344701,  0.03489881,  0.04401519,  0.04756385, -0.02936105,\n", "       -0.0305603 , -0.02101141,  0.02751049, -0.00875684, -0.01583926,\n", "        0.00033533,  0.02769942,  0.0358323 , -0.04180737, -0.02759142,\n", "       -0.01231979,  0.02881228, -0.00406825,  0.00497993,  0.01094388,\n", "       -0.01672934,  0.05414844, -0.01725494,  0.04816335,  0.04487341,\n", "        0.0269151 ,  0.00945554,  0.02318397,  0.04105411,  0.05314256])"]}, "execution_count": 12, "metadata": {}, "output_type": "execute_result"}], "source": ["lr.coef_"]}, {"cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [{"data": {"image/png": "iVBORw0KGgoAAAANSUhEUgAAAtEAAAEICAYAAACZEKh9AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjYuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8o6BhiAAAACXBIWXMAAAsTAAALEwEAmpwYAAAkiUlEQVR4nO3debgkdX3v8fdHBhDQsE6QbQAFYyAmaiagT0zCFUTcMtwEAsaYSQJBb+JVExPFJUhAI+Yal1xRwwUS3EBDFidKggiSxS0M7uDCiAMzBGRgELcgot/7R9WRnqb7zKnpPkuf8349z3lOLb+u+tba3/7Vr6pSVUiSJEmauQfNdwCSJEnSpDGJliRJkjoyiZYkSZI6MomWJEmSOjKJliRJkjoyiZYkSZI6MomW+iT5iSSfSfKtJC9IslOSf0pyd5K/TfLsJB+awXRenuT8uYh50iQ5KsnG+Y5j3JIclKSSLJvvWHolOTPJu8YwnUpyyIDh5yZ5zajTX8qSnJjkiiQPnu9YpszWOSzJbyX5j57+byd5eMdp7NWep1fOsPyKdj7bdY1XGmZBneilLpL8OvCHwKOAbwGfAV5TVf8x3edm4CXAR6rqMe18ngPsDexZVfe1Zd69tYlU1Z+NGAft/A8CvgZs3zN/aUFIchpwb1W9Yr5jmVRJHgucChxfVffMdzxTxnUOm8F8HtKlfJLtgYuA36uqtTOcx81Ap/lIW2MSrYmU5A+B04HnAZcD9wLHAauAUZPoA4FL+vq/YgI7vSTbVdUP5juO+bIttc9Jlk36flVV5813DJOod9tX1aeBp4xreotdVX0fePq4ppckQKrqh+OappYGm3No4iTZFTgL+P2q+vuq+k5Vfb+q/qmq/rgts2OSNyX5r/bvTUl27JnGM9pLgd9I8rEkP90Ovwr4H8Bb2kt/FwNnACe1/acMuBR5eHsZdnOSryd5eTt8i0voSR7fzusbST6b5KiecVcnOTvJR9tmJB9Kslc7+t/a/99oY3hCkkOS/GvbxOSOJO8dsq6mmhec1q6HW5P8Uc/4oeupfznbYT+6nJ/kb5K8LcllSb7Trrf++e+R5K/bad+V5B/7xr84ye1tXL/dM/zpST6d5JtJNiQ5s+9zz0lyU5I7k7wiyfokx/TE9eqesls0HUmyb5K/S7IpydeSvGDQuttaHD3r9pQkNwNX9Xz0d4as7zOTXJrkXUm+CfxWkl2TXNCWvSXJqzPkknOS7dJcYv9qu59cm+SAdtyb2xi/2Q7/hWmW64k9++KGJL/VDr86yak95R6wD/SM2zHJ65Pc3O73b0+yUzturyQfaKe/Ocm/Jxn4fTMs7nY7/XeSPXrKPrbd37dv+38nyRfbfevyJAf2lB14XA6Y/7TL3G7j5yW5oV2ec5OkZ/zQGPrmM3B/2coyHJvky2mO87emOeZP7Ynzo0nemORO4Mxt3SZJXtrue99q53d0O7z/HPbLSa5rp3F1kp/sGbc+yR8l+Vwb73szw6YpeeB55dwkH2zj+WSSR/SUfVTPdv1ykl/rGTeT43VZz3Z/TZKPAt8FHj7dtKWBqso//ybqj6bG+T5g2TRlzgI+Afw4sBz4GHB2O+6xwO3AkcB2wGpgPbBjO/5q4NSeaZ0JvKun/7eA/2i7HwrcCrwYeHDbf2T/54D9gDuBp9H8eH1y27+8Z55fBR4J7NT2n9OOOwio3uUFLgZe0U7rwcATh6yHqc9eDOwCPBrYBBwzg/X0o+XsmV4Bh7TdfwPcDfz8VBwD5v9B4L3A7sD2wC+1w49qt+FZ7fCn0XyR7d4z/tHtdH8a+DrNpW6Aw4BvA78I7Ai8oZ3WMT1xvbonhqOAjW33g4BraX4Y7QA8HLgReMqQ9TddHFPr9h3tut1pBuv7TOD7wPHtNHcC/gH4q7b8jwP/CTx3SDx/DHwe+AkgwM/QNDMC+A1gT5orjC8GbpvaJmy5Lx5I0/zpWe263xN4zJB9f4t9oG/7vxFYA+xBs9//E/Dadtxrgbe3098e+AWamr5ByzRd3FcBv9tT9v8Ab2+7VwHrgJ9sP/tK4GNbOy4HzH8my/wBYDdgRbs9j9taDNMci737y3TLsBfwTeBX2nEvpNl3Tu2J8z7gf7fjd9qWbUKzL20A9u2J8xED9ptHAt+hOXdtT9PsbR2wQzt+Pc2+u287/y8CzxuyLgat497zyp3AEe1yvRu4pB23Sxvrb7fjHgvcARzW4Xhd1rPdbwYOb6e163TT9s+/QX/zHoB//nX9A54N3LaVMl8FntbT/xRgfdv9NtpEsWf8l7k/wbuamSfRzwI+PSSG3i+glwLv7Bt/ObC6Z56v7Bn3e8C/tN1bnPzbYe8AzgP238p6mPrso3qG/TlwwQzW0xZfdO2w/i+7d0wz732AH9Imxn3jjgL+u2+ZbgceP2RabwLe2HafQful2vbvQtOcZyZJ9JHAzX3Tfhnw1zPc93rjmFq3D++wvs8E/q1n3N7A94CdeoY9i6ZN/qD5fxlYNcNY7wJ+ZsC++DLgH4Z85mpmkETTJF/foU222nFPAL7Wdp8FvH9qX+ny1xf3qcBVbXdokpxfbPv/GTil53MPovkhdiDTHJfbuMxP7Ol/H3D61mIYMJ9B+8t0y/CbwMd7xk0tf28SfXPf+M7bpN2etwPH0Nx3Mewc9ifA+/pivQU4qu1fD/xG337/9iHrfOB+Vfcfv+f3jHsa8KW2+yTg3/um9VfAq4bM50088HjtTaLP6inbadr++VdVNufQRLoT2CvTt0HdF7ipp/+mdhg0X1Avbi9JfiPJN4ADesZ3cQBNIro1BwIn9s3ziTSJ5pTberq/y/Q3wbyE5kvzP9vLq7+zlflv6OnuXRfTraeZ2DDNuAOAzVV115Dxd9aWbTh/tMxJjkzykTRNLu6mafs+1bxl3975VtV3aPaJmTgQ2LdvO7ycJpl9gK3EMWXQOhi2vvvHHUhTq3drTzx/RVMjPcjQ/a29lP7F9lL6N2hq1vpjnXYaHSwHdgau7Yn7X9rh0NQYrwM+lOTGJKcPm9BW4v474AlJ9qG58vBD4N/bcQcCb+6Z/2aaY2K/MS1jr2HH5nQxDNO//Yd9vn8/L6D/iTa909qmbVJV64AX0STMtye5JMmgc8AW54pq2g9v6FvWLuew6Uy3vo/sO36fDTwMZny89urfFkOnLQ1iEq1J9HGa2rvjpynzXzQnxSkr2mHQnDhfU1W79fztXFUXb0MsG2iaBMyk3Dv75rlLVZ0zg8/WAwZU3VZVv1tV+wLPBd6aAY8e63FAT3fvuphuPX2H5ksZgCSDvkweEFuPDcAeSXabpsww76G5LH1AVe1Kcxl6qh3qrfQsT5KdaZoDTNkibrb8EtxAUzPXux0eWlVP24Y4pgxaB8PWd3/5DTT78l498fxYVR0+JJ4NwCP6B6ZpR/wS4Ndoav53o2lq0x/r0Gm0plt3ve6guZJweE/cu1b7lIWq+lZVvbiqHg78MvCHU+1su8Td/gD7EE0t4a/TXIGYWn8baJq99G7LnarqY8z8uOyyzINMF8Mw/dt/2OdvBfafKpgkvf0DprXN26Sq3lNVT6Q5FxTwugFxb3GuaOM5gKY2eq5sAP61b309pKr+Vzt+Jsdrr/5tMd20pQcwidbEqaq7aS7pn5vk+CQ7J9k+yVOT/Hlb7GLglUmWp7lB7wxg6gaZ/wc8r621SJJd2htSHroN4XwA2CfJi9Lc1PPQJEcOKPcu4JlJnpLm5rAHp7nhrf9LcZBNNDVwP0oK0jxTduqzd9F8GUx3Z/mftOvpcJo2f1M3Ik63nj4LHJ7kMWluEDpzBrH+SFXdSnO5+q1Jdm+30S/O8OMPpanFvifJETQJ1JRLgWekuTluB5rL1L3nss8AT0tzU+PDaGrZpvwn8K00N1Lt1G6Ln0ryc9sQx3SGre8ttOvoQ8BfJPmxJA9K8ogkvzRkuucDZyc5tN13fzrJnm2c99HsK8uSnAH82JBpvBs4JsmvJVmWZM8kj2nHfQb4lTb2Q4BThsT9Q5rj6I1JfhwgyX5JntJ2PyPNza+hSYp/wOD9cyZxv4emacMJbfeUtwMva9cxaW7QPLEdN9PjcsbLPMR0MYz6+Q8Cj27PccuA32eaBH9bt0ma5+I/Kc0NxffQJOKDttX7gKcnOTrNjZ0vpvkBON0PhnH7APDINDcWb9/+/Vzuv8FxW4/XmUxbegCTaE2kqvoLmmdEv5LmC3gD8HzgH9sirwbWAp+juRHrU+0wqnmu6O8Cb6FJQNfRtNHblji+RXOjzTNpLkHewICnVFTVBpqbiF7eE+8fM4NjsKq+C7wG+Giay4yPB34O+GSSb9PUvLywqm6cZjL/SrOcVwKvr6qpl8VMt56+QpOgfrhdrm15dOBzaG6G+hJNu8sXzfBzvwecleRbNIn9+6ZGVNV1NAnFe2hq6+5iy8vc76T5AbCeJkF9b89nfwA8A3gMzbO376BJTHftGsdWDFvfg/wmzU2O17fLcilbNvPp9YY2hg/R3HR2Ac0NZZfTXLr/Cs0l93sY0tSmmuflPo0mCdpMk0T+TDv6jTTty79O8xze6Z6H/tJ2GT+R5kkjH6a5SQ3g0Lb/2zRXjt5aVR8ZMI2ZxL2mnd5tVfXZnuX4B5oa00va+X8BeGo7bkbH5TYs8xami2HUz1fVHcCJNG2L76S5oXYtTeI6zLZskx2Bc2iOhdtomhK9bECsX6a5CfT/tmWfCTyzqu6d6fKOqt2uxwIn09SM30az/qaevLStx+tMpi09QO6/MiZpsckSeVFLkvU0N1x9eL5jkWZDmsfRbQSePeQHiaQ5Zk20JEkLUNv8a7e2qcXLadr3fmKew5LUMomWJGlhegLNU0ammk8cX1X/Pb8hSZoyluYcSY4D3kzz4orz+5840P6KfgfwszRtu06qqvXtzQnnA4+jebj5O6rqtSMHJEmSJM2ikWui07ye9lyamyEOA56V5LC+YqcAd1XVITQ3cUw9PudEmrfEPZomwX5u24ZTkiRJWrCme1nFTB0BrJt6MkCSS2ieQnB9T5lV3P94rEuBt7SP2Slgl/bxPTvR3CH9za3NcK+99qqDDjpoDKFLkiRJg1177bV3VNXyQePGkUTvx5aPJNpI82rdgWWq6r40bxLakyahXkXzmKqdgT+oqs2DZpLkNOA0gBUrVrB27doxhC5JkiQNluSmYePm+8bCI2ge+L4vcDDNq5gHvmWqqs6rqpVVtXL58oE/CCRJkqQ5MY4k+ha2fMXt/jzwNaA/KtM23diV5gbDXwf+paq+X1W3Ax8FVo4hJkmSJGnWjCOJvgY4NMnB7St4T6Z5w1SvNcDqtvsE4KpqHgtyM/AkgCS7AI+nebOZJEmStGCNnES3b0F7Ps3rW78IvK+qrktyVpJfbotdAOyZZB3Nq5pPb4efCzwkyXU0yfhfV9XnRo1JkiRJmk0T+drvlStXljcWSpIkaTYlubaqBjY1nu8bCyVJkqSJYxItSZIkdWQSLUmSJHU0jpetSFpiDjr9gw8Ytv6cp89DJJIkzQ9roiVJkqSOTKIlSZKkjkyiJUmSpI5MoiVJkqSOTKIlSZKkjkyiJUmSpI5MoiVJkqSOTKIlSZKkjkyiJUmSpI5MoiVJkqSOfO23JGnB8dXykhY6a6IlSZKkjqyJljQ21h5KkpYKa6IlSZKkjkyiJUmSpI5MoiVJkqSOTKIlSZKkjsaSRCc5LsmXk6xLcvqA8TsmeW87/pNJDuoZ99NJPp7kuiSfT/LgccQkSZIkzZaRk+gk2wHnAk8FDgOeleSwvmKnAHdV1SHAG4HXtZ9dBrwLeF5VHQ4cBXx/1JgkSZKk2TSOmugjgHVVdWNV3QtcAqzqK7MKuKjtvhQ4OkmAY4HPVdVnAarqzqr6wRhikiRJkmbNOJLo/YANPf0b22EDy1TVfcDdwJ7AI4FKcnmSTyV5ybCZJDktydokazdt2jSGsCVJkqRtM983Fi4Dngg8u/3/P5McPahgVZ1XVSurauXy5cvnMkZJkiRpC+NIom8BDujp378dNrBM2w56V+BOmlrrf6uqO6rqu8BlwOPGEJMkSZI0a8aRRF8DHJrk4CQ7ACcDa/rKrAFWt90nAFdVVQGXA49OsnObXP8ScP0YYpIkSZJmzbJRJ1BV9yV5Pk1CvB1wYVVdl+QsYG1VrQEuAN6ZZB2wmSbRpqruSvIGmkS8gMuq6oOjxiQtFQed/sDDZf05T5+HSCRJWlpGTqIBquoymqYYvcPO6Om+BzhxyGffRfOYO0mSJGkizPeNhZIkSdLEMYmWJEmSOhpLcw5Jkpaq/nsTvC9BWhpMoiVJkhYJbzifOzbnkCRJkjqyJlqSJEkLxqTUplsTLUmSJHVkEi1JkiR1ZBItSZIkdWQSLUmSJHVkEi1JkiR1ZBItSZIkdeQj7iRJkpaoSXmc3EJkEi1Jkh7A5Eqankm0NCK/aCRJWnpMoiVJ2gp/LEvq542FkiRJUkfWREuSJGkLXn3ZOpNoSZKWMJMladuYREsayi9Xae54vEmTxTbRkiRJUkdjqYlOchzwZmA74PyqOqdv/I7AO4CfBe4ETqqq9T3jVwDXA2dW1evHEZM0xdodSZI0biMn0Um2A84FngxsBK5Jsqaqru8pdgpwV1UdkuRk4HXAST3j3wD886ixSNKoluKPLpe5sdiXWdJ4jaMm+ghgXVXdCJDkEmAVTc3ylFXAmW33pcBbkqSqKsnxwNeA74whFkmSJE2ASf8xO4420fsBG3r6N7bDBpapqvuAu4E9kzwEeCnwp2OIQ5IkSZoT831j4ZnAG6vq21srmOS0JGuTrN20adPsRyZJkiQNMY7mHLcAB/T0798OG1RmY5JlwK40NxgeCZyQ5M+B3YAfJrmnqt7SP5OqOg84D2DlypU1hrglSZKkbTKOJPoa4NAkB9MkyycDv95XZg2wGvg4cAJwVVUV8AtTBZKcCXx7UAItSZIkLSQjJ9FVdV+S5wOX0zzi7sKqui7JWcDaqloDXAC8M8k6YDNNoi1JkiRNpLE8J7qqLgMu6xt2Rk/3PcCJW5nGmeOIRZIkSZptvvZb0qIw6Y9KkiRNFpNoTcvERJK02PjdpnEwiZYkaQkwcZTGa76fEy1JkiRNHJNoSZIkqSOTaEmSJKkjk2hJkiSpI5NoSZIkqSOTaEmSJKkjk2hJkiSpI5NoSZIkqSNftiJpQfLFEPPPbSBtO4+fxc8kWlpCPKlLkjQeJtGSJE2gxfCjeLaXYTGsIy1cJtHaJp6YJElamvpzgKX6/W8SLUlLhD9+JWl8TKIlaYEz+ZWkhcckWpolJj6SJC1ePidakiRJ6sgkWpIkSerI5hySJEmaNYu1eaNJtKR55aOSJEmTaCzNOZIcl+TLSdYlOX3A+B2TvLcd/8kkB7XDn5zk2iSfb/8/aRzxSJIkSbNp5JroJNsB5wJPBjYC1yRZU1XX9xQ7Bbirqg5JcjLwOuAk4A7gmVX1X0l+Crgc2G/UmNTdYr3UImlp8Bwmaa6Noyb6CGBdVd1YVfcClwCr+sqsAi5quy8Fjk6Sqvp0Vf1XO/w6YKckO44hJkmSJGnWjKNN9H7Ahp7+jcCRw8pU1X1J7gb2pKmJnvKrwKeq6nuDZpLkNOA0gBUrVowhbEmaOWs6JUm9FsQj7pIcTtPE47nDylTVeVW1sqpWLl++fO6CkyRJkvqMoyb6FuCAnv7922GDymxMsgzYFbgTIMn+wD8Av1lVXx1DPAveYq7RWszLpsnkPilJmg3jqIm+Bjg0ycFJdgBOBtb0lVkDrG67TwCuqqpKshvwQeD0qvroGGKRJEmSZt3INdFtG+fn0zxZYzvgwqq6LslZwNqqWgNcALwzyTpgM02iDfB84BDgjCRntMOOrarbR41LWsiGPRvZWlNJkibDWF62UlWXAZf1DTujp/se4MQBn3s18OpxxCBJkiTNlQVxY6EkSZI0SXzttyaOTR4kSdJ8M4mWpFkyrO27JGnymURrybJGW5IkbSuTaEmSFhErCKS5YRLdgScmSZIkgU/nkCRJkjoziZYkSZI6sjnHEmOTFEmSJoff2wuXSfSE8+CS5pfHoCQtTSbRkiaKSaskLUxL7fxsm2hJkiSpI5NoSZIkqSOTaEmSJKkjk2hJkiSpI28slGZoqd0wIUmShrMmWpIkSerImmhJkqR55tXOyWMSLS1CnownU9ft5naWpPljcw5JkiSpI2uitWhYKydJkubKWJLoJMcBbwa2A86vqnP6xu8IvAP4WeBO4KSqWt+OexlwCvAD4AVVdfk4YpIkyR/XkmbLyEl0ku2Ac4EnAxuBa5Ksqarre4qdAtxVVYckORl4HXBSksOAk4HDgX2BDyd5ZFX9YNS4JlX/Cd+TvSRJ0sIzjjbRRwDrqurGqroXuARY1VdmFXBR230pcHSStMMvqarvVdXXgHXt9CRJkqQFK1U12gSSE4DjqurUtv85wJFV9fyeMl9oy2xs+78KHAmcCXyiqt7VDr8A+OequnTAfE4DTgNYsWLFz950000jxT1Owy4XjvMy4qTftT9dPMNq3+drGRbauoPZj2kultmrLPeb7XUxn/vLTI/nbT3OJ+lYmJRl7mouzudz8b06rvl2PZ7HtQyTvh9NiiTXVtXKQeMm5ukcVXVeVa2sqpXLly+f73AkSZK0hI3jxsJbgAN6+vdvhw0qszHJMmBXmhsMZ/JZSYuANR6SpMVkHDXR1wCHJjk4yQ40Nwqu6SuzBljddp8AXFVNO5I1wMlJdkxyMHAo8J9jiEmSJEmaNSPXRFfVfUmeD1xO84i7C6vquiRnAWurag1wAfDOJOuAzTSJNm259wHXA/cBv7+Un8whSZKkyTCW50RX1WXAZX3Dzujpvgc4cchnXwO8ZhxxSJIkSXPBNxZKkmadbeKlhsfC4jExT+eQJEmSFgqTaEmSJKkjm3NIfbzUJkmStsaaaEmSJKkjk2hJkiSpI5NoSZIkqSOTaEmSJKkjbyyUJEkLijd4axKYREuSxsbkR9JSYRItycRHkqSOTKIlaY75o0WSJp9JtCQtMibpkjT7TKIlSZK2kT9aly4fcSdJkiR1ZBItSZIkdWRzjgnh5SJJ8lwoaeGwJlqSJEnqyJpoSZIWMGvf7+e60EJiTbQkSZLUkTXRkiRJE6Zrrby1+OM3Uk10kj2SXJHkhvb/7kPKrW7L3JBkdTts5yQfTPKlJNclOWeUWCRJkqS5MmpzjtOBK6vqUODKtn8LSfYAXgUcCRwBvKon2X59VT0KeCzw80meOmI8kiRJ0qwbtTnHKuCotvsi4GrgpX1lngJcUVWbAZJcARxXVRcDHwGoqnuTfArYf8R4tIh46UlamDw2JWn0mui9q+rWtvs2YO8BZfYDNvT0b2yH/UiS3YBn0tRmS5IkSQvaVmuik3wYeNiAUa/o7amqSlJdA0iyDLgY+MuqunGacqcBpwGsWLGi62wkSZKksdlqEl1Vxwwbl+TrSfapqluT7APcPqDYLdzf5AOaJhtX9/SfB9xQVW/aShzntWVZuXJl52RdmmRePpcWj6V4PC/FZdbiN2pzjjXA6rZ7NfD+AWUuB45Nsnt7Q+Gx7TCSvBrYFXjRiHFIkiRJc2bUJPoc4MlJbgCOaftJsjLJ+QDtDYVnA9e0f2dV1eYk+9M0CTkM+FSSzyQ5dcR4JEmSpFk30tM5qupO4OgBw9cCp/b0Xwhc2FdmI5BR5i9JkiTNB1/7LUmSJHXka78XKW/ikDQJPFdJmlQm0Zp3folKWig8H0maKZPoWeTJWJIkaXGyTbQkSZLUkUm0JEmS1JHNOSRJkqZh80wNYk20JEmS1JFJtCRJktSRzTkkSdKM2bRBalgTLUmSJHVkTbQkSRoLa6m1lFgTLUmSJHVkTbTmhLUTkiRpMbEmWpIkSerIJFqSJEnqyCRakiRJ6sg20WNge19JkqSlxZpoSZIkqSOTaEmSJKkjk2hJkiSpI5NoSZIkqaORkugkeyS5IskN7f/dh5Rb3Za5IcnqAePXJPnCKLFIkiRJc2XUmujTgSur6lDgyrZ/C0n2AF4FHAkcAbyqN9lO8ivAt0eMQ5IkSZozoybRq4CL2u6LgOMHlHkKcEVVba6qu4ArgOMAkjwE+EPg1SPGIUmSJM2ZUZPovavq1rb7NmDvAWX2Azb09G9shwGcDfwF8N2tzSjJaUnWJlm7adOmEUKWJEmSRrPVl60k+TDwsAGjXtHbU1WVpGY64ySPAR5RVX+Q5KCtla+q84DzAFauXDnj+UiSJEnjttUkuqqOGTYuydeT7FNVtybZB7h9QLFbgKN6+vcHrgaeAKxMsr6N48eTXF1VRyFJkiQtYKO+9nsNsBo4p/3//gFlLgf+rOdmwmOBl1XVZuBtAG1N9AdMoCVJ0nxZf87T5zsETZBR20SfAzw5yQ3AMW0/SVYmOR+gTZbPBq5p/85qh0mSJEkTaaSa6Kq6Ezh6wPC1wKk9/RcCF04znfXAT40SiyRJkjRXfGOhJEmS1JFJtCRJktSRSbQkSZLUkUm0JEmS1JFJtCRJktSRSbQkSZLUkUm0JEmS1JFJtCRJktSRSbQkSZLUkUm0JEmS1JFJtCRJktSRSbQkSZLUkUm0JEmS1JFJtCRJktSRSbQkSZLUkUm0JEmS1JFJtCRJktSRSbQkSZLUkUm0JEmS1NGy+Q5AkqTFaP05T5/vECTNImuiJUmSpI5GSqKT7JHkiiQ3tP93H1JudVvmhiSre4bvkOS8JF9J8qUkvzpKPJIkSdJcGLUm+nTgyqo6FLiy7d9Ckj2AVwFHAkcAr+pJtl8B3F5VjwQOA/51xHgkSZKkWTdqEr0KuKjtvgg4fkCZpwBXVNXmqroLuAI4rh33O8BrAarqh1V1x4jxSJIkSbNu1CR676q6te2+Ddh7QJn9gA09/RuB/ZLs1vafneRTSf42yaDPA5DktCRrk6zdtGnTiGFLkiRJ226rSXSSDyf5woC/Vb3lqqqA6jDvZcD+wMeq6nHAx4HXDytcVedV1cqqWrl8+fIOs5EkSZLGa6uPuKuqY4aNS/L1JPtU1a1J9gFuH1DsFuConv79gauBO4HvAn/fDv9b4JSZhS1JkiTNn1Gbc6wBpp62sRp4/4AylwPHJtm9vaHwWODytub6n7g/wT4auH7EeCRJkqRZN2oSfQ7w5CQ3AMe0/SRZmeR8gKraDJwNXNP+ndUOA3gpcGaSzwHPAV48YjySJEnSrEtTITxZkmwCbprnMPYCfJrI4ud2XhrczkuD23lpcDsvDXO1nQ+sqoE3401kEr0QJFlbVSvnOw7NLrfz0uB2XhrczkuD23lpWAjb2dd+S5IkSR2ZREuSJEkdmURvu/PmOwDNCbfz0uB2XhrczkuD23lpmPftbJtoSZIkqSNroiVJkqSOTKIlSZKkjkyiO0pyXJIvJ1mX5PT5jkfjkeSAJB9Jcn2S65K8sB2+R5IrktzQ/t99vmPV6JJsl+TTST7Q9h+c5JPtcf3eJDvMd4waTZLdklya5EtJvpjkCR7Pi0+SP2jP2V9IcnGSB3s8T74kFya5PckXeoYNPH7T+Mt2e38uyePmKk6T6A6SbAecCzwVOAx4VpLD5jcqjcl9wIur6jDg8cDvt9v2dODKqjoUuLLt1+R7IfDFnv7XAW+sqkOAu4BT5iUqjdObgX+pqkcBP0OzvT2eF5Ek+wEvAFZW1U8B2wEn4/G8GPwNcFzfsGHH71OBQ9u/04C3zVGMJtEdHQGsq6obq+pe4BJg1TzHpDGoqlur6lNt97dovnD3o9m+F7XFLgKOn5cANTZJ9geeDpzf9gd4EnBpW8TtPOGS7Ar8InABQFXdW1XfwON5MVoG7JRkGbAzcCsezxOvqv4N2Nw3eNjxuwp4RzU+AeyWZJ+5iNMkupv9gA09/RvbYVpEkhwEPBb4JLB3Vd3ajroN2Hu+4tLYvAl4CfDDtn9P4BtVdV/b73E9+Q4GNgF/3TbbOT/JLng8LypVdQvweuBmmuT5buBaPJ4Xq2HH77zlZibRUo8kDwH+DnhRVX2zd1w1z4P0mZATLMkzgNur6tr5jkWzahnwOOBtVfVY4Dv0Nd3weJ58bZvYVTQ/mvYFduGBTQC0CC2U49ckuptbgAN6+vdvh2kRSLI9TQL97qr6+3bw16cuC7X/b5+v+DQWPw/8cpL1NM2xnkTTdna39nIweFwvBhuBjVX1ybb/Upqk2uN5cTkG+FpVbaqq7wN/T3OMezwvTsOO33nLzUyiu7kGOLS983cHmhsY1sxzTBqDtl3sBcAXq+oNPaPWAKvb7tXA++c6No1PVb2sqvavqoNojt+rqurZwEeAE9pibucJV1W3ARuS/EQ76GjgejyeF5ubgccn2bk9h09tZ4/nxWnY8bsG+M32KR2PB+7uafYxq3xjYUdJnkbTpnI74MKqes38RqRxSPJE4N+Bz3N/W9mX07SLfh+wArgJ+LWq6r/ZQRMoyVHAH1XVM5I8nKZmeg/g08BvVNX35jE8jSjJY2huHt0BuBH4bZqKI4/nRSTJnwIn0Txh6dPAqTTtYT2eJ1iSi4GjgL2ArwOvAv6RAcdv+wPqLTRNeb4L/HZVrZ2TOE2iJUmSpG5sziFJkiR1ZBItSZIkdWQSLUmSJHVkEi1JkiR1ZBItSZIkdWQSLUmSJHVkEi1JkiR19P8BdUsrYobT1wEAAAAASUVORK5CYII=\n", "text/plain": ["<Figure size 864x288 with 1 Axes>"]}, "metadata": {"needs_background": "light"}, "output_type": "display_data"}], "source": ["import matplotlib.pyplot as plt\n", "fig, ax = plt.subplots(1, 1, figsize=(12, 4))\n", "ax.bar(numpy.arange(0, len(lr.coef_)), lr.coef_)\n", "ax.set_title(\"Coefficients pour chaque arbre calcul\u00e9s avec une r\u00e9gression lin\u00e9aire\");"]}, {"cell_type": "markdown", "metadata": {}, "source": ["Le score est avec une r\u00e9gression lin\u00e9aire sur les variables initiales est nettement moins \u00e9lev\u00e9."]}, {"cell_type": "code", "execution_count": 13, "metadata": {}, "outputs": [{"data": {"text/plain": ["0.5103612609676136"]}, "execution_count": 14, "metadata": {}, "output_type": "execute_result"}], "source": ["lr_raw = LinearRegression()\n", "lr_raw.fit(X_train, y_train)\n", "r2_score(y_test, lr_raw.predict(X_test))"]}, {"cell_type": "markdown", "metadata": {}, "source": ["## S\u00e9lection d'arbres\n", "\n", "L'id\u00e9e est d'utiliser un algorithme de s\u00e9lection de variables type [Lasso](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.Lasso.html) pour r\u00e9duire la for\u00eat al\u00e9atoire sans perdre en performance. C'est presque le m\u00eame code."]}, {"cell_type": "code", "execution_count": 14, "metadata": {}, "outputs": [{"data": {"text/plain": ["array([ 0.01256934, -0.03342528, -0.02400605,  0.01825851,  0.0005323 ,\n", "       -0.01374509,  0.01004616, -0.02284903,  0.01105419,  0.02047233,\n", "        0.03476362,  0.02755575,  0.01751674, -0.03051477, -0.02321124,\n", "       -0.01783216,  0.01429992,  0.00214398,  0.04066576,  0.0134879 ,\n", "       -0.00377705,  0.00506043,  0.04614375,  0.02482044,  0.01560689,\n", "        0.01706262, -0.02035898,  0.05747191,  0.00418486,  0.02766988,\n", "       -0.00899098,  0.04325266,  0.07327657,  0.06515135, -0.00034774,\n", "        0.02210777,  0.0280344 ,  0.00852669,  0.0358763 ,  0.01779845,\n", "        0.        ,  0.03970822,  0.01935286,  0.00908017,  0.01417323,\n", "       -0.01066044,  0.00293442,  0.02483663,  0.00332255, -0.01043329,\n", "        0.02666477,  0.04097776,  0.02851599, -0.01795373, -0.00830115,\n", "       -0.00293032, -0.02188798,  0.06679156, -0.01364001,  0.02028321,\n", "        0.00160792,  0.04114419, -0.02342478,  0.04638246,  0.0547764 ,\n", "       -0.05501755, -0.01856303,  0.04157578, -0.01403205,  0.02718244,\n", "        0.01215738,  0.03503149,  0.04403975,  0.04640854, -0.02884553,\n", "       -0.02929629, -0.01946676,  0.02679733, -0.00779812, -0.01418256,\n", "        0.        ,  0.02734732,  0.03608281, -0.04111661, -0.02654714,\n", "       -0.01106999,  0.02664032, -0.00291639,  0.00541073,  0.01187597,\n", "       -0.01621428,  0.05386765, -0.01531834,  0.04807872,  0.04398675,\n", "        0.02611443,  0.00944403,  0.02219076,  0.04080548,  0.05276076])"]}, "execution_count": 15, "metadata": {}, "output_type": "execute_result"}], "source": ["from sklearn.linear_model import Lasso\n", "\n", "lrs = Lasso(max_iter=10000)\n", "lrs.fit(X_train_2, y_train)\n", "lrs.coef_"]}, {"cell_type": "markdown", "metadata": {}, "source": ["Pas mal de z\u00e9ros donc pas mal d'arbres non utilis\u00e9s."]}, {"cell_type": "code", "execution_count": 15, "metadata": {}, "outputs": [{"data": {"text/plain": ["0.3055529526371402"]}, "execution_count": 16, "metadata": {}, "output_type": "execute_result"}], "source": ["r2_score(y_test, lrs.predict(X_test_2))"]}, {"cell_type": "markdown", "metadata": {}, "source": ["Pas trop de perte... Ca donne envie d'essayer plusieurs valeur de `alpha`."]}, {"cell_type": "code", "execution_count": 16, "metadata": {}, "outputs": [{"name": "stderr", "output_type": "stream", "text": ["  0%|          | 0/200 [00:00<?, ?it/s]C:\\Users\\xavie\\AppData\\Local\\Temp\\ipykernel_18252\\1667502338.py:7: UserWarning: With alpha=0, this algorithm does not converge well. You are advised to use the LinearRegression estimator\n", "  lrs.fit(X_train_2, y_train)\n", "C:\\xavierdupre\\__home_\\github_fork\\scikit-learn\\sklearn\\linear_model\\_coordinate_descent.py:634: UserWarning: Coordinate descent with no regularization may lead to unexpected results and is discouraged.\n", "  model = cd_fast.enet_coordinate_descent(\n", "C:\\xavierdupre\\__home_\\github_fork\\scikit-learn\\sklearn\\linear_model\\_coordinate_descent.py:634: ConvergenceWarning: Objective did not converge. You might want to increase the number of iterations, check the scale of the features or consider increasing regularisation. Duality gap: 3.572e+04, tolerance: 1.993e+02 Linear regression models with null weight for the l1 regularization term are more efficiently fitted using one of the solvers implemented in sklearn.linear_model.Ridge/RidgeCV instead.\n", "  model = cd_fast.enet_coordinate_descent(\n", "100%|\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588| 200/200 [00:37<00:00,  5.37it/s]\n"]}], "source": ["from tqdm import tqdm\n", "alphas = [0.01 * i for i in range(100)] +[1 + 0.1 * i for i in range(100)]\n", "obs = []\n", "for i in tqdm(range(0, len(alphas))):\n", "    alpha = alphas[i]\n", "    lrs = Lasso(max_iter=20000, alpha=alpha)\n", "    lrs.fit(X_train_2, y_train)\n", "    obs.append(dict(\n", "        alpha=alpha,\n", "        null=len(lrs.coef_[lrs.coef_!=0]),\n", "        r2=r2_score(y_test, lrs.predict(X_test_2))\n", "    ))"]}, {"cell_type": "code", "execution_count": 17, "metadata": {}, "outputs": [{"data": {"text/html": ["<div>\n", "<style scoped>\n", "    .dataframe tbody tr th:only-of-type {\n", "        vertical-align: middle;\n", "    }\n", "\n", "    .dataframe tbody tr th {\n", "        vertical-align: top;\n", "    }\n", "\n", "    .dataframe thead th {\n", "        text-align: right;\n", "    }\n", "</style>\n", "<table border=\"1\" class=\"dataframe\">\n", "  <thead>\n", "    <tr style=\"text-align: right;\">\n", "      <th></th>\n", "      <th>alpha</th>\n", "      <th>null</th>\n", "      <th>r2</th>\n", "    </tr>\n", "  </thead>\n", "  <tbody>\n", "    <tr>\n", "      <th>195</th>\n", "      <td>10.5</td>\n", "      <td>83</td>\n", "      <td>0.318660</td>\n", "    </tr>\n", "    <tr>\n", "      <th>196</th>\n", "      <td>10.6</td>\n", "      <td>83</td>\n", "      <td>0.318771</td>\n", "    </tr>\n", "    <tr>\n", "      <th>197</th>\n", "      <td>10.7</td>\n", "      <td>83</td>\n", "      <td>0.318879</td>\n", "    </tr>\n", "    <tr>\n", "      <th>198</th>\n", "      <td>10.8</td>\n", "      <td>83</td>\n", "      <td>0.318982</td>\n", "    </tr>\n", "    <tr>\n", "      <th>199</th>\n", "      <td>10.9</td>\n", "      <td>82</td>\n", "      <td>0.319073</td>\n", "    </tr>\n", "  </tbody>\n", "</table>\n", "</div>"], "text/plain": ["     alpha  null        r2\n", "195   10.5    83  0.318660\n", "196   10.6    83  0.318771\n", "197   10.7    83  0.318879\n", "198   10.8    83  0.318982\n", "199   10.9    82  0.319073"]}, "execution_count": 18, "metadata": {}, "output_type": "execute_result"}], "source": ["from pandas import DataFrame\n", "df = DataFrame(obs)\n", "df.tail()"]}, {"cell_type": "code", "execution_count": 18, "metadata": {}, "outputs": [{"data": {"image/png": "iVBORw0KGgoAAAANSUhEUgAAAs8AAAEaCAYAAADuabq2AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjYuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8o6BhiAAAACXBIWXMAAAsTAAALEwEAmpwYAABG00lEQVR4nO3deXxdVbn/8c+TeWiTTumYzi1DKdBCGKVSAaEMAhcRgasXUKx4qaDIBbwX4YqgiFzlKoi/CohXGUQcKFgmFURmUtrSgZaOtOmYpk06ZE6e3x9nJ5yGDKc5OefknHzfr1deOXvttdd+zs7p7pOVtdcyd0dERERERLqWlugARERERESShZJnEREREZEIKXkWEREREYmQkmcRERERkQgpeRYRERERiZCSZxERERGRCCl57oPMzM1sUpzONS44X0Y8zhcJM7vdzHaY2dZg+1/MbKOZ7TWz6Wa2zMxmRtDOXjObEOt4U0H4Z87MHjaz2xMdk4iISHcoee4FzGy9mW03s/ywsivN7OUEhpWSzGwM8C1girsPD4rvBua4ez93X+juh7n7y121FdRf2wMxKZkUEYmQmWWb2YNm9qGZ7TGzRWZ2ZqLjkr5DyXPvkQ5cm+gg2upNPcY9ZAxQ4e7bw8rGAssSFI+IiByYHGAjcDJQCNwMPGFm4xIZlPQdSp57jx8B15vZgPZ2mtmJZvaOmVUF308M2/dyMBTh9WAowdNmNtjMHjGz3UH9cW2aPMvM1gbDF35kZmlBW5eb2Wtm9hMzqwD+O/gt/24z22Bm28zsF2aW20Gc6UHdHWa2Fji7zf7CoMdgi5ltCuJO76St/zSzNUHvwgIzGx3B9Wj3HGZ2GvAiMDK4To+Z2V5Cv7gsNrM1wfHrg7pdxRA+FKHDa2RmM82szMy+FfyFYYuZXRHsmw38K3BDy88uKL8xiH2Pma00s1M7uEYPm9l9ZvaXoO5bZjbxAD433wt+3nvM7AUzG9LBeTp8D2FtXRm2fbmZvdpeW23aHWJmz5hZpZntNLN/tnwWRURaBPflG83sPWA7cLu7r3f3Znd/BlgHHJ3YKKWv0H9SvUcp8DJwfdsdZjYI+AvwU2Aw8GPgL2Y2OKzaxcAXgVHAROAN4FfAIOB94NY2zf4LUAIcBZwHfCls33HAWmAYcAdwJ3AQMA2YFJzjlg7ex1eAc4DpQfsXttn/MNAYtDMdOB24kvZdB1wCnAUUBDFWR3A92j2Hu/8VOBPYHAy5uMTd+wXHHOnurUlnVzG0U6+razScUA/JKODLwH1mNtDd5wKPAHcFMX3GzA4G5gDHuHt/4AxgfQfXCEI/++8CA4HVhH5mkX5uLgWuAIYCWbTz+evqPXRSPxLfAsqAIkKft/8EPMo2RSQ1XUKoQ2aAuze2FJrZMEL3X/0FUeJCyXPvcgvwdTMralN+NrDK3X/j7o3u/hiwAvhMWJ1fufsad68CngXWuPtfgxvM7wklkeF+6O473X0DcA+hm1KLze7+s+DYWmA28M2g/h7g+4QStvZcBNzj7hvdfSfwg5YdwQ3uLOAb7r4vGDrxk07auhK42d1Xeshid6/o7Hp04xxd6SiGVmZmdH2NGoDb3L3B3ecDe4GDOzhnE5ANTDGzzKB3ZU0nMf7J3d8Ofl6PEErgIfLPzQfuXgM8EXZsew7kPUSqARgBjA3a/ae7K3kWkfb8NPi/paalwMwyCd33fu3uKxIXmvQlqTaeNam5+1Izewa4iVBvcYuRwIdtqn9IqAewxbaw1zXtbPdjfxvbtDWyg31FQB6wIJQjAmCEhjq0Z2Q7bbcYC2QCW8LaSmtTP9xooL2ksbPrcaDn6EpHMYSL5BpVhPeUEOq9bvszAcDdV5vZN4D/Bg4zs+eB69x9cwfn39pBu5F8bjo6tj0Rv4cD8CNC7/OF4NrNdfc7o2xTRFLTfvfxYIjXb4B6Qn+tE4kL9Tz3PrcSGvoQnuBsJpQUhhsDbIriPKPbtBWemIX3/O0glHwf5u4Dgq/CsOEObW1pp+0WG4E6YEhYWwXuflgHbW0kNASlrc6ux4GeoysdxRDuQK9RWx/raXX3R939JELv04EfHkjQgVh8bjqyj9AvEC2Gd1QxnLvvcfdvufsE4Fzguo7Gd4tIn9d6rwz+4vcgoeFen3X3hoRFJX2Okudext1XA78Drgkrng8cZGaXmlmGmX0emAI8E8Wp/sPMBgYPv10bnLO9eJqBXwI/MbOhAGY2yszO6KDdJ4BrzKw4GA97U1hbW4AXgP8xswIzSzOziWZ2cgdtPQB8z8wmW8gRwXjdDq9HN87RlY5iiOYatbUNaJ0v2swONrNTzCyb0LCZGqC5G7HH4nPTkUXABWaWZ6GHKL8cyUFmdo6ZTQr+I6wiNGSlO+9VRPqW+4FDgc+ED+MQiQclz73TbUDrnM/BGNtzCD1cVQHcAJzj7juiOMdTwAJCSc9fCP0G35EbCT2I9qaZ7Qb+SsdjXX8JPA8sBt4F/thm/78RejBtObALeJLQmNf2/JhQMv4CsDuIMTeC63Eg5+hKuzG0U+9ArlFbDxIa31xpZn8mNN75TkI92lsJPcz37QMNPEafm478hNCfTrcBvyY0BjESkwldq72EHnL9ubu/FIP4RCRFmNlY4KuEntHYaqGZivaa2b8mNjLpK0zP5oiIiIiIREY9zyIiIiIiEVLyLCIiIiISISXPIiIiIiIRUvIsIiIiIhIhJc8iIiIiIhFKqhUGhwwZ4uPGjUt0GCIiB2zBggU73L0o0XHEk+7ZIpKsOrtnJ1XyPG7cOEpLSxMdhojIATOztkulpzzds0UkWXV2z9awDRERERGRCCl5FhERERGJkJJnEREREZEIJdWYZxFJHQ0NDZSVlVFbW5voUHpUTk4OxcXFZGZmJjqUXikZf+76mYpIuC6TZzN7CDgH2O7uU4OyQcDvgHHAeuAid99lZgb8L3AWUA1c7u7vttPm0cDDQC4wH7jW3b0H3o+IJImysjL69+/PuHHjCN06kp+7U1FRQVlZGePHj090OL1Ssv3c9TMVkbYiGbbxMDCrTdlNwN/cfTLwt2Ab4ExgcvA1G7i/gzbvB74SVrdt+yKS4mpraxk8eHBSJFCRMjMGDx6cVL2q8ZZsP3f9TEWkrS57nt39FTMb16b4PGBm8PrXwMvAjUH5/wW9yG+a2QAzG+HuW1oONLMRQIG7vxls/x9wPvBsV7Hsrm3gxeXbuqomklIOGtaPsYPzEx1GTCRLAnUgUvE99bRku0bJFq+IhGyqrGHl1t2cOHEIOZnpPdZud8c8DwtLiLcCw4LXo4CNYfXKgrItYWWjgvK2ddplZrMJ9WKTNXwSX/k/zRkqfcuUEQXMv3ZGosMQ4OWXX+buu+/mmWee4eGHH6a0tJR777030WFJFKqrq/nc5z7HmjVrSE9P5zOf+Qx33nlnosMSkR7wzOLN/ODZFbz57VMZXpj45LmVu7uZxWy8srvPBeYCHHbkdP/d10+K1alEep17/rqK5ZurEh2GSMpyd6677jpOPfVU6uvrOfXUU3n22Wc588wzEx2aiETplVXlHDysP8MLc3q03e4mz9tahmMEwzC2B+WbgNFh9YqDsnCbgvLO6rQrNzOdqaMKuxmySPIZOSCH0g93JjqMlLV+/XrOPPNMTjrpJF5//XVGjRrFU089xZlnnsndd99NSUkJO3bsoKSkhPXr1yc6XOkh69ev54wzzuC4445jwYIFzJ8/H4CsrCyOOuooysrKumhBRHq76vpG3lm3i8tOHNvjbXc3eZ4HXAbcGXx/Kqx8jpk9DhwHVIWPdwYIEu7dZnY88Bbwb8DPuhmHSErLzUynpr4p0WHE3HefXsbyzbt7tM0pIwu49TOHdVlv1apVPPbYY/zyl7/koosu4g9/+EOPxiEdS/TP/de//jXHH398a1llZSVPP/001157bY/GJCLx99a6ndQ3NTNjclGPt93lbBtm9hjwBnCwmZWZ2ZcJJc2fNrNVwGnBNoSmnVsLrAZ+Cfx7WDuLwpr9d+CBoN4aInhYUKQvys1Kp66xmaZmzeQYK+PHj2fatGkAHH300eph7iPGjh27X+Lc2NjIJZdcwjXXXMOECRMSGJmI9ITXV+8gKyONY8cP6vG2I5lt45IOdp3aTl0Hru6gnWlhr0uBqZGFKNJ35WWFHnCobWgiPzt11zSKpKcwVrKzs1tfp6enU1NTQ0ZGBs3NzQCaoiyGEvlzz8/ffwab2bNnM3nyZL7xjW8kJiAR6VELN1QydWRBj86y0ULLc4v0YrnBP/rqPjB0ozcZN24cCxYsAODJJ59McDQSazfffDNVVVXcc889iQ5FRHpAQ1MzSzZVMX3MwJi0r+RZpBfLzQr1Ntc2KHmOp+uvv57777+f6dOns2PHjkSHIzFUVlbGHXfcwfLlyznqqKOYNm0aDzzwQKLDEpEorNiyh7rGZqaNHhCT9lP378AiKUA9z7E1btw4li5d2rp9/fXXt75+7733Wl/ffvvtAMycOZOZM2cCcPnll3P55ZfHJU7pWeE/9+LiYkIjDkUkVSzauAuA6WMGxKR99TyL9GItY55r1PMsIiISkYUbKxnSL5tRA3Jj0r56nkV6sZYHHf6xspwtlTUADMzP4vgJgxMZloiISK+1aEMl08cMwMxi0r6SZ5FebFhBaCaIn/z1g/3KX7vplJj9Ri3Jz8xmAf8LpAMPuPudbfZfRWhmpCZgLzDb3Zeb2WDgSeAY4GF3nxN2zCXAfwIObAa+4O4aEC4ivUpldT1rd+zjs0cXd125m5Q8i/RiE4r68eqNn2JvXSMAr62u4HvPLKequiElkmd3j1nPQKIkevysmaUD9wGfBsqAd8xsnrsvD6v2qLv/Iqh/LvBjYBZQC3yH0FSiU8PazCCUjE9x9x1mdhcwB/jv7sSYbD/3RP9MRSRyizZWAjA9Rg8LgsY8i/R6xQPzOGR4AYcML2BiUWhu2lQYA52Tk0NFRUVKJSbuTkVFBTk5OYkM41hgtbuvdfd64HHgvPAK7h6+rF8+od5k3H2fu79KKIkOZ8FXvoWy3gJCvc8HLNl+7r3kZyoiEVq0sRIzOCKGybN6nkWSSMsY6LoUSJ6Li4spKyujvLw80aH0qJycHIqLY/fnwgiMAjaGbZcBx7WtZGZXA9cBWcApnTXo7g1m9jVgCbAPWEUHC2J1JRl/7r3gZyoiEVq4oZKDhvanXwwXFlPyLJJEWqauS4We58zMTMaPH5/oMPosd78PuM/MLgVuBi7rqK6ZZQJfA6YDa4GfAd8Gbm+n7mxgNsCYMWM+1pZ+7iISK+7Ooo2VzDpseEzPo2EbIkmkpee5tqE5wZFIL7YJGB22XRyUdeRx4Pwu2pwG4O5rPDTe4gngxPYquvtcdy9x95KioqJIYxYRidq6HfuoqmngqLEDYnoeJc8iSSS3NXlO/p5niZl3gMlmNt7MsoCLgXnhFcxsctjm2YSGYXRmEzDFzFqy4U8D7/dQvCIiPaLlYcFpo2OzLHcLDdsQSSI5maHfd1Nh2IbEhrs3mtkc4HlCU9U95O7LzOw2oNTd5wFzzOw0oAHYRdiQDTNbT+iBwCwzOx84PZjG7rvAK2bWAHwIXB7HtyUi0qWFGyrpl53BpKH9YnoeJc8iSSRbPc8SAXefD8xvU3ZL2OtrOzl2XAflvwB+0UMhioj0uIUbd3FEcSHpabGdClPDNkSSiIZtiIiIfFxNfRMrtuxh+pgBMT+Xep5FkkhmupFmsGRTFfMWtz/NrgGfmDSEQflZ8Q1OREQkQZZurqKx2Zke4/HOoORZJKmYGcMLcnh+2TaeX7atw3qfO7qYH33uyDhGJiIikjgLN+wCYJp6nkWkrWe/8UnK99R1uP+Hz63gpZXlNDc7aTEe9yUiItIbLNpYyehBuQzplx3zcyl5FkkyhbmZFOZmdrh/1mHDeXH5NpZv2c3UUYVxjExERCQxFm6o5Jhxg+JyrqgeGDSza81sqZktM7NvBGW/M7NFwdd6M1vUwbHrzWxJUK80mjhE5CMnHxyaivelFdsTHImIiEjsba2qZUtVLdNGD4jL+brd82xmU4GvAMcC9cBzZvaMu38+rM7/AFWdNPMpd9/R3RhE5OOG9MvmyOJCXlq5na+fOrnrA0RERJLYoo2h8c7xmGkDout5PhR4y92r3b0R+AdwQctOMzPgIuCx6EIUkQM18+ChLNxYya599YkORUREJKYWbqgkKz2NKSML4nK+aJLnpcAMMxtsZnnAWcDosP0zgG3u3tGyrw68YGYLzGx2Rycxs9lmVmpmpeXl5VGEK9J3zDy4CHd4ZZX+zYiISGpbuLGSKSMLyM5Ij8v5up08u/v7wA+BF4DngEVA+MoNl9B5r/NJ7n4UcCZwtZl9soPzzHX3EncvKSoq6m64In3KEcUDGJSfxcsrlTyLiEjqamxq5r2yyrgN2YAoHxh09wfd/Wh3/ySwC/gAwMwyCA3h+F0nx24Kvm8H/kRo7LSI9ID0NOPkg4r4xwflNDV7osMRERGJiRVb91Db0By3hwUh+tk2hgbfxxBKlh8Ndp0GrHD3sg6Oyzez/i2vgdMJDQMRkR4y8+Aidu6r572yykSHIiIiEhOLNlYCcNSY2K8s2CLaeZ7/YGaDgQbganevDMovps2QDTMbCTzg7mcBw4A/hZ4pJAN41N2fizIWEQnzyclFpBn86rX1rK/Y12X9rPR0TpsyNG5jxkRERKK1cEMlQ/plUTwwN27njCp5dvcZHZRf3k7ZZkIPFeLuawGtHSwSQwPzszhu/GDmLd7MvMWbIzrm7s8dyYVHF8c4MhERkZ6xcOMupo0eQNAhGxdaYVAkhf3qimPYWlXbZT0Hzv3ZqyzauEvJs4iIJIWq6gbWlu/js0fF9/8tJc8iKSwnM51xQ/Ijqjt1VCHvlXW2ppGIiEjvsSh4pieeDwtClA8MikjqOKK4kPe37KausanryiIiIgm2YP1O0iz0/1c8KXkWESA0N3RDk7Ny655EhyIiItKl19dUcHjxAPrnZMb1vEqeRQT46Df3xRq6kfTMbJaZrTSz1WZ2Uzv7rzKzJWa2yMxeNbMpQflgM3vJzPaa2b1tjskys7lm9oGZrTCzz8br/YiItFVd38iijZWcMGFw3M+t5FlEACgemMug/CzeC+bMlORkZunAfYRWb50CXNKSHId51N0Pd/dpwF3Aj4PyWuA7wPXtNP1fwHZ3Pyho9x8xCF9EJCLvrN9FY7Nz4sT4J896YFBEADAzDh9VyJJN6nlOcscCq4MpQTGzx4HzgOUtFdx9d1j9fEITruDu+4BXzWxSO+1+CTgkqNcM7IhJ9CIiEXhjTQWZ6UbJuPgtjtJCPc8i0urI4kI+2LaH6vrGRIci3TcK2Bi2XRaU7cfMrjazNYR6nq/prEEzGxC8/J6ZvWtmvzezYR3UnW1mpWZWWl5e3q03ICLSlTfW7GDa6AHkZcW/H1jJs4i0OqJ4AM0Oyzbv7rqyJDV3v8/dJwI3Ajd3UT0DKAZed/ejgDeAuztod667l7h7SVFRUY/GLCICsLu2gSWbqhIy3hk0bENEwrQ8NPjY2xtYt6PrJb0PpN1Dhhf0WHvSqU3A6LDt4qCsI48D93fRZgVQDfwx2P498OXuBigiEo231+6k2eGEiUMScn4lzyLSamhBDhOL8vnju5v447ud5VsHZuqoAp75+owea0869Q4w2czGE0qaLwYuDa9gZpPdfVWweTawik64u5vZ08BM4O/AqYSNoRYRiac31laQlZHG9DEDEnJ+Jc8isp9nvj6Din11Pdbe9+e/z6INlT3WnnTO3RvNbA7wPJAOPOTuy8zsNqDU3ecBc8zsNKAB2AVc1nK8ma0HCoAsMzsfON3dlxMa3vEbM7sHKAeuiN+7EhH5yBtrKigZO5CczPSEnF/Js4jsJzcrneKsvB5rb2j/HPbU6gHEeHL3+cD8NmW3hL2+tpNjx3VQ/iHwyR4KUUTkgNU1NrFjbz3Lt+zmW58+KGFxKHkWkZgqyMlgb30jzc1OWpolOhwREUlSF/3ijdaFvE5IwPzOLTTbhojEVP+cTNxhr6a/ExGRbmpsam5NnLPS0ziieEDCYlHyLCIx1T8n9AcuDd0QEZHuWhS2+u3wwhyyMhKXwip5FpGYKsjNBGBPbUOCIxERkWT10srtra+njkrs1Kca8ywiMaWeZxERidbLK8s5euxAZkwewr+dMC6hsUTV82xm15rZUjNbZmbfCMr+28w2mdmi4OusDo6dZWYrzWy1md0UTRwi0nv1z1HPs4iIdN/23bUs27ybUw8dyjdOO4hB+VkJjafbPc9mNhX4CnAsUA88Z2bPBLt/4u7tLt0aHJsO3Ad8GigD3jGzecFcoiKSQtTzLCIi0Xj5g3IAZh40NMGRhETT83wo8Ja7V7t7I/AP4IIIjz0WWO3ua929ntDysOdFEYuI9FItyfPuGvU8i4jIgXt55XaGFWRz6Ij+iQ4FiC55XgrMMLPBZpYHnAWMDvbNMbP3zOwhMxvYzrGjgI1h22VB2ceY2WwzKzWz0vLy8ijCFZFEKAiGbexWz7OIiByghqZm/rlqBzMPGopZ71groNvJs7u/D/wQeAF4DlgENAH3AxOBacAW4H+iCdDd57p7ibuXFBUVRdOUiCRAdkYamemmYRsiInLA3v1wF3tqG/nUIb0nB4zqgUF3f9Ddj3b3TwK7gA/cfZu7N7l7M/BLQkM02trER73UAMVBmYikGDOjICdTDwyKiMgBe/mDcjLSjE9MGpLoUFpFO9vG0OD7GELjnR81sxFhVf6F0PCOtt4BJpvZeDPLAi4G5kUTi4j0Xv1zMtTzLCIiB2zhhl1MHVXYOnNTbxDtPM9/MLPBQANwtbtXmtnPzGwa4MB64KsAZjYSeMDdz3L3RjObAzwPpAMPufuyKGMRkV6qv3qeRUSkG3bta2Ds4LxEh7GfqJJnd5/RTtkXO6i7mdBDhS3b84H50ZxfRJKDep5FRKQ7dlXXM230gESHsR8tzy0iMafkWUREDpS7U1nTwIC83jNkA5Q8i0gcaNiGiIgcqJqGJuobmxmQl9gVBdtS8iwiMaeeZxEROVC7qkOdLgPV8ywifU1BTiZ76hppavZEh9InmNksM1tpZqvN7KZ29l9lZkvMbJGZvWpmU4LywWb2kpntNbN7O2h7npm1N4uSiEiPqqyuB1DPs4j0PS1LdP/qtXX85s0P2VJVk+CIUpeZpQP3AWcCU4BLWpLjMI+6++HuPg24C/hxUF4LfAe4voO2LwD2xiJuEZG2tu+pA2BQvpJnEeljJhTlA3D7X97nO39eys/+vjrBEaW0Y4HV7r7W3euBx4Hzwiu4++6wzXxCU4vi7vvc/VVCSfR+zKwfcB1we6wCFxEJ988PdpCVnsZhIwsSHcp+op3nWUSkS6ccMozFt5xOQ3MzX3zwbTZXquc5hkYBG8O2y4Dj2lYys6sJJcNZwCkRtPs94H+A6h6IUUSkU+7O31Zs48RJg8nP7l3pqnqeRSQuCvMyGdIvm1EDctha9bGOTYkzd7/P3ScCNwI3d1Y3WPhqorv/qat2zWy2mZWaWWl5eXnPBCsifc6a8r18WFHNqYcOS3QoH6PkWUTialhBDtt2K3mOoU3A6LDt4qCsI48D53fR5glAiZmtB14FDjKzl9ur6O5z3b3E3UuKiooijVlEZD/PL9sGwGmHDk1wJB+n5FlE4mpEYQ67qhuobWhKdCip6h1gspmNN7Ms4GJgXngFM5sctnk2sKqzBt39fncf6e7jgJOAD9x9Zo9GLSIS5tmlW5g+ZgAjCnMTHcrH9K5BJCKS8oYV5ACwfXcdYwbnJTia1OPujWY2B3geSAcecvdlZnYbUOru84A5ZnYa0ADsAi5rOT7oXS4AsszsfOB0d18e57chIn3Yhopqlm7azX+ddWiiQ2mXkmcRiavhhaHkeevuWiXPMeLu84H5bcpuCXt9bSfHjuui7fXA1OgiFBHp2LNLtwAwa+rwBEfSPg3bEJG4Gh70PGuuZxERac/8pVs5oriQ0YN6ZweLkmcRiathQc+zHhoUEZG2ynZVs3hjZa/tdQYlzyISZ/2zM8jLSmdrVV2iQxERkV7muaVbAThz6ogER9IxJc8iEldmxnBNVyciIu14dulWDh1RwPgh+YkOpUNKnkUk7oYV5LBVybOIiITZWlXLgg93cVYvHrIBSp5FJAGGF2qVQRER2d/zy4IhG4f33iEbEGXybGbXmtlSM1tmZt8Iyn5kZivM7D0z+5OZDejg2PVmtsTMFplZaTRxiEhyGVaQw/Y9tTQ3e6JDERGRXuIvS7Zw0LB+TBraL9GhdKrbybOZTQW+AhwLHAmcY2aTgBeBqe5+BPAB8O1OmvmUu09z95LuxiEiyWdEYQ4NTU7FvvpEhyIiIr3Alqoa3lm/k7N6ea8zRNfzfCjwlrtXu3sj8A/gAnd/IdgGeBMojjZIEUktLasM6qFBEREBeGbxFtzhvGmjEh1Kl6JJnpcCM8xssJnlAWcBo9vU+RLwbAfHO/CCmS0ws9lRxCEiSaZ1lUGNexYR6bPeWlvBSyu2A/DU4k0cWVzYq2fZaNHt5bnd/X0z+yHwArAPWAQ0tew3s/8CGoFHOmjiJHffZGZDgRfNbIW7v9K2UpBYzwYYM2ZMd8MVkV6kZZVBzbghItI3NTc7n5/7JgB/ve5klm7azc1nH5rgqCIT1QOD7v6gux/t7p8EdhEa44yZXQ6cA/yru7f7RJC7bwq+bwf+RGjsdHv15rp7ibuXFBUVRROuiPQSQ/plkWYatiEi0le9tmZH6+t5izdjBp85cmQCI4pctLNtDA2+jwEuAB41s1nADcC57l7dwXH5Zta/5TVwOqFhICLSB2Skp1HUP1vDNkRE+qhH39rQ+vqxtzdwwoTBrc/D9HbRzvP8BzNbDjwNXO3ulcC9QH9CQzEWmdkvAMxspJnND44bBrxqZouBt4G/uPtzUcYiIklkuBZKERHpk7bvqeXF5dsYMygPgPI9dZw3LTl6nSGKMc8A7j6jnbJJHdTdTOihQtx9LaHp7USkjxpWkMPiskrmvrKmy7pHFA/g+AmD4xCViIjE2u9Ly2hsdr51+kFc+/gistLTmHVY75+irkVUybOISHcdOXoALyzfxvfnr+iy7phBebxyw6fiEJWIiMRSU7Pz6FsbOHHiYE6YGOoUOfngIgrzMhMcWeSUPItIQlz9qUlc8YlxtP9I8Ufuem4FTy4oi09QIiISU6+sKmdTZQ3fPusQivplc/mJ47jgqN4/t3M4Jc8ikjB5WV3fgoYW5LCvvonahiZyMtPjEJWIiMTKo29tYEi/LE6fMhwz47/PPSzRIR2waB8YFBGJqYF5WQDsqtZS3pEys1lmttLMVpvZTe3sv8rMlgQPdb9qZlOC8sFm9pKZ7TWze8Pq55nZX8xshZktM7M74/l+RCQ1bKmq4e8rtvO5ktFkZSRvCpq8kYtInzAoPzQObuc+Jc+RMLN04D7gTGAKcElLchzmUXc/3N2nAXcBPw7Ka4HvANe30/Td7n4IMB34hJmdGYv4RSR1/e6djTQ1O5cck9yL3il5FpFebVB+NgC79jUkOJKkcSyw2t3Xuns98DhwXngFd98dtpkPeFC+z91fJZREh9evdveXgtf1wLtAcezegoikmsamZn73zkZmTB7CmMF5iQ4nKkqeRaRXa+l5rthXl+BIksYoYGPYdllQth8zu9rM1hDqeb4m0sbNbADwGeBvHeyfbWalZlZaXl5+IHGLSAp7btlWtlTV8m8njEt0KFFT8iwivVrrmGcN2+hR7n6fu08EbgRujuQYM8sAHgN+GszX3167c929xN1LioqKei5gEUlqD726jrGD8zj1kKGJDiVqSp5FpFcbkJeFGeys1rCNCG0CRodtFwdlHXkcOD/CtucCq9z9nm5FJiJ90sINu3h3QyVXnDiOtDRLdDhRU/IsIr1aepoxIDeTnRq2Eal3gMlmNt7MsoCLgXnhFcxsctjm2cCqrho1s9uBQuAbPReqiPQFv3ptPf2zM7iwZHTXlZOAkmcR6fUG5mex4MNKfvnKWio1ZV2n3L0RmAM8D7wPPOHuy8zsNjM7N6g2J5hybhFwHXBZy/Fmtp7Q7BuXm1mZmU0xs2LgvwjN3vFuMMXdlfF7VyKSrLZU1TB/yRY+f8xo+mWnxvIiqfEuRCSlHT6qkKcWbeaOLbvJzkxLiQdOYsnd5wPz25TdEvb62k6OHdfBruT/W6uIxN1v3viQZncuO3FcokPpMep5FpFe757PT2P5bWcAULFXPc8iIsmgpr6JR9/ewOlThjN6UHJPTxdOybOI9HpmRl5WBgU5GRq2ISKSJP60cBOV1Q186aTxiQ6lRyl5FpGkMTA/i8oazbohItLbNTc7D722jqmjCjhm3MBEh9OjlDyLSNIYkJvJLk1ZJyLS672wfBurt+/lKzMmYJZaj0woeRaRpDEgL0vDNkREejl35+cvr2bc4DzOOWJkosPpcUqeRSRpDMzLZJeSZxGRXu211RW8V1bFV0+eSHoKLIrSlpJnEUkaA/KyqNynYRsiIr3ZfS+tZlhBNhccNSrRocREVMmzmV1rZkuDyfa/EZQNMrMXzWxV8L3dUeJmdllQZ5WZXdZeHRGRcAPzsthT10hDU3OiQxERkXYs+HAXb6yt4CszJpCdkZ7ocGKi28mzmU0FvgIcCxwJnGNmk4CbgL+5+2Tgb8F222MHAbcCxwXH39pRki0i0mJAXiYAVZpxQ0SkV/rfv61iUH4Wlxw7JtGhxEw0Pc+HAm+5e3WwHOw/gAuA84BfB3V+DZzfzrFnAC+6+0533wW8CMyKIhYR6QNakmc9NCgi0vuUrt/JKx+Uc9XJE8hPkaW42xNN8rwUmGFmg80sDzgLGA0Mc/ctQZ2twLB2jh0FbAzbLgvKPsbMZptZqZmVlpeXRxGuiCS7gXlZAJquTkSkF/rJXz9gSL8svnD82ESHElPdTp7d/X3gh8ALwHPAIqCpTR0HPIr4cPe57l7i7iVFRUXRNCUiSa41ed6nnmcRkd7krbUVvLa6gqtOnkheVur2OkOUDwy6+4PufrS7fxLYBXwAbDOzEQDB9+3tHLqJUC91i+KgTESkQx8N21DPs4hIb/KTv35AUf/slO91huhn2xgafB9DaLzzo8A8oGX2jMuAp9o59HngdDMbGDwoeHpQJiLSodbkuUY9zyIivcXra3bw5tqdXD1zIjmZqTnDRrho+9X/YGaDgQbganevNLM7gSfM7MvAh8BFAGZWAlzl7le6+04z+x7wTtDObe6+M8pYRCTF9cvOICPNNOZZRKSXcHd+8uIHDC/I4eIUnmEjXFTJs7vPaKesAji1nfJS4Mqw7YeAh6I5v4j0LWamJbpFRHqRLVW1vLN+F/951iF9otcZtMKgiCSZgXmZ7NIqgyIivcLmyhoADhrWP8GRxI+SZxFJKgPzstilnudOmdksM1tpZqvNrL2Fqq4ysyVmtsjMXjWzKUH5YDN7ycz2mtm9bY45OjhmtZn91MwsXu9HRHqvLVW1AIwozE1wJPGj5FlEkkphXqZWGOyEmaUD9wFnAlOAS1qS4zCPuvvh7j4NuAv4cVBeC3wHuL6dpu8ntKrs5OBLC1uJCFuqQj3PIwbkJDiS+FHyLCJJZWBepnqeO3cssNrd17p7PfA4oZVfW7n77rDNfIL5+N19n7u/SiiJbhVMO1rg7m8G8/f/H+2vHisifcyWqlrys9Lpn8IrCrbVd96piKSEgXlZ7NxXz49fWNnu/vzsDK74xHiyMvps30B7K7ge17aSmV0NXAdkAadE0GZZmzbbXRVWRPqWrVW1DC/MoS+N5FLyLCJJ5fDiQtzhZy+t/tg+D9YzPXL0AI6fMDjOkSUXd78PuM/MLgVu5qP5+aNiZrOB2QBjxvSNaatE+rItVbWMHNB3xjuDkmcRSTLnHDGSc44Y2e6+pZuqOOdnr/b1MdEHuoLr44TGM3fVZnEkbbr7XGAuQElJiXcVrIgkty1VNUweWpToMOKqz/5dU0RST/+cUH/AntrGBEeSUO8Ak81svJllARcTWvm1lZlNDts8G1jVWYPuvgXYbWbHB7Ns/Bvtrx4rIn1IQ1Mz5XvqGFHYdx4WBPU8i0gK6Z8TWr57T23f7Xl290YzmwM8D6QDD7n7MjO7DSh193nAHDM7jdDqsLsIG7JhZuuBAiDLzM4HTnf35cC/Aw8DucCzwZeI9GHzl2yh2WHKyIJEhxJXSp5FJGW09DzvrunTPc+4+3xgfpuyW8JeX9vJseM6KC8FpvZQiCKS5Oobm/mfFz7g0BEFnD5leKLDiSsN2xCRlJGZnkZuZnqf7nkWEYmHx9/ZwIad1dww62DS0vrOTBug5FlEUkz/nIy+PuZZRCSm9tU18tO/rea48YOYeVDfelgQlDyLSIopyM1kT516nkVEYuX+l9ewY28dN515SJ+a37mFkmcRSSn9czL6/JhnEZFYKdtVzS//uZbzp41k+piBiQ4nIZQ8i0hK6Z+TqTHPIiIx8sPnVmIGN8w6JNGhJIxm2xCRlNI/J4MlZZXc9dyKj+3LSDOKCnL41MFFFA/MS0B0IiLJa8GHu3h68WauOWVSn1tVMJySZxFJKdNHD+DFZdv45T/X7lfuDo3NoQXvBudnseA7n05EeCIiSam52fneM8sZ2j+br548MdHhJJSSZxFJKVfOmMCVMyZ8rNzdGf/t0NTHFfvq4x2WiEhSm7d4M4s2VvKjC48gP7tvp49RvXsz+yZwJeDAEuAK4EWgf1BlKPC2u5/fzrFNwTEAG9z93GhiERHpTF98IlxEpCfsrWvk+/Pf58jiQj57VHGiw0m4bifPZjYKuAaY4u41ZvYEcLG7zwir8wfgqQ6aqHH3ad09v4iIiIjE3r1/X832PXX8vy8e3ecWRGlPtLNtZAC5ZpYB5AGbW3aYWQFwCvDnKM8hIiIiIgmwtnwvD766lguPLu6zU9O11e3k2d03AXcDG4AtQJW7vxBW5Xzgb+6+u4Mmcsys1MzeNLPzuxuHiIiIiPQ8d+eWp5aRk5HOjX14arq2up08m9lA4DxgPDASyDezL4RVuQR4rJMmxrp7CXApcI+ZtfvoppnNDpLs0vLy8u6GKyIiIiIdaGhq5opfvc33nlneWvbbtzbw6uod3HTWIRT1z05gdL1LNMM2TgPWuXu5uzcAfwROBDCzIcCxwF86Ojjoucbd1wIvA9M7qDfX3UvcvaSoqO+tny4iIiISaz+Yv4KXVpbzROlG6hub+bBiHz+Y/z4zJg/h0mPHJDq8XiWa5HkDcLyZ5VnoMfZTgfeDfRcCz7h7bXsHmtlAM8sOXg8BPgEsb6+uiIiIiMTOM+9t5qHX1nH4qEL21Dby1roK/uP375GeZtx14RGaraiNaMY8vwU8CbxLaMq5NGBusPti2gzZMLMSM3sg2DwUKDWzxcBLwJ3uruRZROLG3RMdgohIwq3atocbnnyPo8cO5LdXHkdWRho3/WEJb6/fya2fOYwRhX13JcGORDXPs7vfCtzaTvnMdspKCc0Jjbu/DhwezblFRKKxdNNuDi8uTHQYIiIJU1ldz1f+r5S8rHTuu/QoCnMzOWnSEP6+YjunHTqUzx41KtEh9krRTlUnIpI0bj9/auvrR976MIGRiIgkVkNTM//+yLtsrqzl/33xaIYX5gBwUUkxk4b24/sXHK7hGh1Q8iwifcYXjh/L+jvPZvyQfKrrmxIdTsyY2SwzW2lmq83spnb2X2VmS8xskZm9amZTwvZ9OzhupZmdEVb+TTNbZmZLzewxM8uJ1/sRkZ7l7tw6bxmvr6ngBxccztFjB7XumzV1BH+97mSG9tc/8Y4oeRaRPicvK53q+sZEhxETZpYO3AecCUwBLglPjgOPuvvhwSqvdwE/Do6dQuiZlcOAWcDPzSw9bEXZEnefCqQH9UQkCf369fU8+tYGrjp5Ip89WsttHyglzyLS5+RnZbCvLmV7no8FVrv7WnevBx4nNCd/qzaLV+UDLU9Pngc87u517r4OWB20B52sKCsiyeOVD8q57ZnlnHboMG444+BEh5OUlDyLSJ+Tl526Pc/AKGBj2HZZULYfM7vazNYQ6nm+prNjI1hRNrxdLWwl0kttrqzh6kff5aBh/bnn4mmkpWlMc3coeRaRPic/K4N9KTzmORLufp+7TwRuBG7urG4EK8qGt6uFrUR6qReXb2NPbSP3XjqdftlRTbjWpyl5FpE+Jy8rneq6lO153gSMDtsuDso68jhwfhfHdriirIgkj7fX7WTUgFwmDe2f6FCSmpJnEelz8rNTuuf5HWCymY03syxCD/bNC69gZpPDNs8GVgWv5wEXm1m2mY0HJgNv0/mKsiKSBNydt9ZVcOz4QV1Xlk6pz15E+pxUnm3D3RvNbA7wPKFZMR5y92VmdhtQ6u7zgDlmdhrQAOwCLguOXWZmTwDLgUbgandvAt4ys5YVZRuBhXy0oqyIJIG1O/axY2+9kuceoORZRPqc/OwMGpqc+sZmsjJS7w9w7j4fmN+m7Jaw19d2cuwdwB3tlLe7oqyIJIe31+0E4Dglz1FT8iwifU5eVjoA33162ceS5+PGD2bW1OGJCEtEJGbeXreTIf2yGT8kP9GhJD0lzyLS5xw2spDB+VnMW7z/VMU19U288kG5kmcRSSn76hp5fc0Ojhs/SEtu9wAlzyLS5xw7fhALvvPpj5X/x+8X8+rqHQmISEQkNuoam/jqbxawY289Fx87uusDpEupN9hPRKSbcrPSqWlI2Vk4RKSPaWxq5trHFvHq6h388LNHMGOy5l7vCUqeRUQCuVnp1KTuFHYi0oe4O//5pyU8t2wrt5wzhQuPLk50SClDybOISCA3M526xmaamz3RoYiIdJu7892nl/NEaRnXnDqZL500PtEhpRQlzyIigdzM0CwcGrohIsmqudm5dd4yHn59PV8+aTzfPG1y1wfJAVHyLCISaJnCTsmziCSj5mbn5qeW8n9vfMjsT07g5rMP1ewaMaDZNkREAjktPc8a9ywiSaa52fn2H5fwu9KNfG3mRG4442AlzjESVc+zmX3TzJaZ2VIze8zMcszsYTNbZ2aLgq9pHRx7mZmtCr4uiyYOEZGekKueZxFJQnWNTXzziUX8rnQjXz9lkhLnGOt2z7OZjQKuAaa4e42ZPQFcHOz+D3d/spNjBxFa5rUEcGCBmc1z913djUdEJFqtwzbU8ywivVBtQxPvrN/JJyYOIS0tlBzvrm3gqt8s4PU1Fdww62D+feakBEeZ+qIdtpEB5JpZA5AHbO6ifoszgBfdfSeAmb0IzAIeizIeEZFuaxm28fOXVzOsIKe1vF92BtecOrl1v4hIvK0p38vVj7zLiq17+Jfpo7jrwiOo2FvP5b96m9Xb9/Lji47kgqM0HV08dDt5dvdNZnY3sAGoAV5w9xfM7FLgDjO7BfgbcJO717U5fBSwMWy7LCj7GDObDcwGGDNmTHfDFRHp0sSifowelMvb63a2ljU2OXvqGvnUIUM5ZtygBEYnIn3VU4s28Z9/XEJWRhqXHDuGx97ewI69dazZvpeqmgZ+dcUxWgAljqIZtjEQOA8YD1QCvzezLwDfBrYCWcBc4Ebgtu6ex93nBu1QUlKiyVdFJGaGFeTwzxtO2a+sdP1OLvzFGxrKISJxV1ldzy1PLWPe4s0cPXYg9146nRGFuUwZWcAtTy1lSL9snrjqBA4bWZjoUPuUaIZtnAasc/dyADP7I3Ciu/822F9nZr8Crm/n2E3AzLDtYuDlKGIREYmJlqEatXqIUETi6B8flHPDk4up2FvPtz59EF+bOZGM9NA8D188fixHjCpkRGEOQ8OGmEl8RJM8bwCON7M8QsM2TgVKzWyEu2+x0GOe5wNL2zn2eeD7Qe81wOmEeqxFRHqVnMzQf1a1jc0JjkRE+oLq+ka+P/99fvvmBiYP7ceDlx3D1FEf71k+cvSA+AcnQHRjnt8ysyeBd4FGYCGh4RXPmlkRYMAi4CoAMysBrnL3K919p5l9D3gnaO62locHRUR6k+wM9TyLSHys3LqHr/6mlA93VnPlSeO5/oyD9aByLxTVbBvufiuhKefCndJB3VLgyrDth4CHojm/iEistfzHVZdEybOZzQL+F0gHHnD3O9vsvwq4GmgC9gKz3X15sO/bwJeDfde4+/NB+QDgAWAqoSlGv+Tub8TlDYn0AY1NzVz7+EL21jXx6JXHc8LEwYkOSTqg5blFRDrROmyjITmGbZhZOnAfcCYwBbjEzKa0qfaoux/u7tOAu4AfB8dOITRf/2GEpg/9edAehJLx59z9EOBI4P1YvxeRvuTh19ezYusebj9/qhLnXk7Js4hIJ5LwgcFjgdXuvtbd64HHCc2M1Mrdd4dt5hPqSSao97i717n7OmA1cKyZFQKfBB4Mjq9398rYvg2RvmPb7lru+esqZh5cxBmHDUt0ONIFJc8iIp3ITE8jPc2obUya5DmiefTN7GozW0Oo5/maLo4dD5QDvzKzhWb2gJnlxyJ4kb7o9r+8T31TM9899zAtq50ElDyLiHQhJyMtaYZtRMrd73P3iYTm4r+5i+oZwFHA/e4+HdgH3NReRTObbWalZlZaXl7eozGLpKJXV+3g6cWb+drJExk7WL+TJgMlzyIiXcjKSOPPCzclOoxIbQJGh20XB2UdeZzQtKKdHVsGlLn7W0H5k4SS6Y9x97nuXuLuJUVFWvFMpCPV9Y3c/fxKvvTrdxg7OI+vzZyY6JAkQkqeRUQiULGvnqbmpFjk9B1gspmNN7MsQg8AzguvYGaTwzbPBlYFr+cBF5tZtpmNByYDb7v7VmCjmR0c1DsVWB7LNyGSqtydpxZt4pS7/8G9L63mrKnDeeKrJ2hKuiQS1VR1IiJ9wVdPnsidz66grrGJvKzefdt090Yzm0NoMap04CF3X2ZmtwGl7j4PmGNmpwENwC7gsuDYZWb2BKHEuBG42t1bBnt/HXgkSMjXAlfE9Y2JJJGGpmbcQ3+1Cvfqqh388LkVLNlUxdRRBdx76XRKxg1KUJTSXb37fwERkV4gJ+Oj6eryshIcTATcfT4wv03ZLWGvr+3k2DuAO9opXwSU9FyUIqlnx946HnlzA79580OKB+by56s/AcB7ZZXc9dxKXl29g1EDcvnRhUdwwVHFpKfp4cBkpORZRKQL2S0LpSTPjBsiEifuzntlVTzy1of8edFm6hubGVaQzXtllfzjg3Ie+Oda/rlqBwPzMvnOOVP41+PGaIhGklPyLCLShWRbKEVEYq+quoE/L9rEY29vYMXWPeRkpvG5o4u54hPjWbVtD1975F0ue+hthvTL4sZZh/CF48fQPycz0WFLD1DyLCLShZyMpFsoRURiZHdtA7c+tYy/LNlCfWMzU0cVcPv5Uzl32kgKguS4qF82n54yjBmTh3BRyWj1NKcYJc8iIl1IwlUGRSRGHntrA39auIkvHD+Gi48Zw9RRhR+rU5iXyS//TY8IpColzyIiXcjWsA0RITS++Y21FUwYks/t5x+e6HAkQZQ8i4h0IUcPDIr0aau372Heos08/d4W1u3Yx+Unjkt0SJJASp5FRLqQnaGeZ5FU5O58WFHNP1eV848PyllcVsX/XjyN48cPZlFZJS8u38aLy7exevtezOCECYOZ/ckJ/Mv0UYkOXRJIybOISBdaep5/8+Z6Jg3tx6Sh/RIckYh0h7uzcWcNb66t4I21Fby5toItVbUAjB6US2NTM//x+/eob2qmfE8d6WnGceMH8YXjxnDW4SMYWpCT4HcgvYGSZxGRLowszOXwUYW8trqCpxdv5pufPijRIYnIAVi/Yx8//dsq3lxbweYgWR6cn8XxEwZz3IRBzJhcxLjBefy+tIw75r/PSZOG8Okpw/jUwUMpzNP0crI/Jc8iIl3IzUrn6a+fxEE3P0utxj2LJJXmZufaxxeyevteZh48lK9NGMTxEwYzaWg/zPZf4e+iY0Zz0TGjExSpJIuokmcz+yZwJeDAEuAK4EFCS7g2AG8DX3X3hnaObQqOAdjg7udGE4uISKzlZKRRp3HPIr1CU7OzYWc1jU3NTB7Wv7V8b10jyzZVsWRTFUs3VbG4rIp1O/bxk88fyb9ML05gxJIqup08m9ko4BpgirvXmNkTwMXAI8AXgmqPEkqu72+niRp3n9bd84uIxFt2Zrpm3BCJs7rGJtbt2Mfq7Xv3+1q7Yx/1jaFfZi88upiGpmaWbAolyu6hY4cX5DB1VCFf+sQ4zjtSD/lJz4h22EYGkGtmDUAesNndX2jZaWZvA/o1T0RSQk6mep5F4uX5ZVv50fMrWVu+l+YgGTaD4oG5TCrqx4zJQ5g8tD/vbarkt29uYERhKFE+f9ooDh9VyNRRhRT1z07sm5CU1O3k2d03mdndwAagBnihTeKcCXwRuLaDJnLMrBRoBO509z93NxYRkXjIzkjXmGeRbmhqdir21rFtdx05mWmt442bmp3te2op21VD2a5qynbWULarhnUV+3h73U4OHVHAnE9NYmIwy82EIf3Izdp/qeuLjhnNd86ZQnaGlsCW+Ihm2MZA4DxgPFAJ/N7MvuDuvw2q/Bx4xd3/2UETY4MEfALwdzNb4u5r2jnPbGA2wJgxY7obrohI1NTzLNKx8j11/OODcjbsrKZ8Ty3bdtexPfhesbeutfcYQjNd9MvJYHNlDQ1Nvl87Rf2zKR6YyzWnTGLOKZPJCuZZ74wSZ4mnaIZtnAasc/dyADP7I3Ai8FszuxUoAr7a0cHuvin4vtbMXgamAx9Lnt19LjAXoKSkxNvuFxGJF/U8S6pyd3buq6eypoHdNQ3sqW1kd20Du2tC35uanQF5mQzMy8KAXdUN7KquZ9e+enZW17O2fB+LyypxDw2tGJyfzdD+2QwryOawEYUMLQhtDy3IoaqmgbfW7qS+qZmzDx9B8cA8igfmUjwwl5EDclvnVRfpraJJnjcAx5tZHqFhG6cCpWZ2JXAGcKq7t9tFE/RaV7t7nZkNAT4B3BVFLCIiMaeeZ+kt3J29dY1U1TRQVdNATX0TNQ1Nrd9rG5poaob0NEgzIyPdSDOjpr6JvXWNVAffN+2qYd2OfXxYsY999Qf+i2FeVjoD87IYXpjDN087iFMPHcrBw/qTkd55b/FFJZoOTpJXNGOe3zKzJ4F3CY1bXkioh3gf8CHwRjB/4h/d/TYzKwGucvcrgUOB/2dmzUAaoTHPy6N7KyIisZWdkU5Vzcdm3pRewN1xhyZ3mpo/et3sTnOz0+wE5R6UE5SH6jc7Ya+d5uZg24Njgu3Wttppu9k/as/blDU7NDY7dUFiG0p0m6lpaKKuZbuhibqGZprd8bD31LLd1Ozsqf0oYW5qju6PsdkZaYwckMvYwXkcO34QYwblMSg/i4LcDApyMinIzaQgJ5PC3EzMoDLobXaHQflZDMjLVC+x9ElRzbbh7rcCt0bSpruXEpq2Dnd/HTg8mnOLiMRbsvQ8m9ks4H+BdOABd7+zzf6rgKuBJmAvMLulA8PMvg18Odh3jbs/H3ZcOlAKbHL3c7qKY+XWPZz8o5cOKPbWZDFIPj9KXtskvG0S4CjzyLhLTzNyM9PJyUwnNyuNnIx0crPSyc5Iw8xIMzBLIy0NDMMs1IM8ZlAeA/JCCe2A3CwKczMpyM0gPzsjrL3Q93Sz0HVq9tZrmZeVTl5WBnlZ6WR20Tvc1vDCdIYXanlqEa0wKCISoWEFOVRW9+6e5yDBvQ/4NFAGvGNm89r8de9Rd/9FUP9c4MfALDObQmi+/sOAkcBfzewgd2/5e/61wPtAQSSx5GWlM330gAN+D2lmpKWFEsj0tNBwgzQz0tNCSWR66/6P6phZqNxo3dcyZKHl2P32WdBWSzst57OgrbTQ8R+1a6QF7bW21TauNrG21kn7KLacrHRyMw88cRWR3kPJs4hIhG79zGGJDiESxwKr3X0tgJk9TmhmpNbk2d13h9XPJ7RKLEG9x929DlhnZquD9t4ws2LgbOAO4LpIAhk9KI97Lp4e5dsREeld9KuviEhqGQVsDNsuC8r2Y2ZXm9kaQg9rXxPBsfcANwCdjlsxs9lmVmpmpeXl5d16AyIivZmSZxGRPsjd73P3icCNwM2d1TWzc4Dt7r4ggnbnunuJu5cUFRX1ULQiIr2HkmcRkdSyCQifB6w4KOvI48D5XRz7CeBcM1sf1D/FzH6LiEgfpORZRCS1vANMNrPxZpZF6AHAeeEVzGxy2ObZwKrg9TzgYjPLNrPxwGTgbXf/trsXu/u4oL2/u/sXYv1GRER6Iz0wKCKSQty90czmAM8TmqruIXdfZma3AaXuPg+YY2anAQ3ALuCy4NhlZvYEoYcLG4Grw2baEBERlDyLiKQcd58PzG9TdkvY62s7OfYOQjNqdLT/ZeDlqIMUEUlSGrYhIiIiIhIhJc8iIiIiIhEy9+RZ09TMyoEP2xQXAlWdbA8BdsQ4tM7iieXxkdTtrE5H+yItb69ePK+3rrWudU/si6SsJ671WHfvU3O3dfOeDfpsd1bnQO4l8fpsRyqe1zqS+rrWPXd8PP+PTPw9292T+guY28V2aSLjieXxkdTtrE5H+yItb69ePK+3rrWudU/si6Qs3veRVP7SZzu6OgdyL+ltn+14XutorqWudc9f667qJNu1ToVhG093sR1v0Z7/QI6PpG5ndTraF2m5rnXkdXSte7Zud651R/siLZOeoc92dHUO5F7S2z7b8bzWkdTXte654+P5f2TCr3VSDdvoDjMrdfeSRMfRV+h6x4+udfzoWseXrnf86FrHj651/MT6WqdCz3NX5iY6gD5G1zt+dK3jR9c6vnS940fXOn50reMnptc65XueRURERER6Sl/oeRYRERER6RFKnkVEREREIqTkWUREREQkQn06eTaz883sl2b2OzM7PdHxpDIzm2BmD5rZk4mOJRWZWb6Z/Tr4PP9rouNJZfosJ47u2fGjz3ls6Z4dXz39eU7a5NnMHjKz7Wa2tE35LDNbaWarzeymztpw9z+7+1eAq4DPxzLeZNZD13qtu385tpGmlgO87hcATwaf53PjHmySO5Brrc9y9+ieHT+6ZyeG7tnxlcj7dtImz8DDwKzwAjNLB+4DzgSmAJeY2RQzO9zMnmnzNTTs0JuD46R9D9Nz11oi9zARXnegGNgYVGuKY4yp4mEiv9bSPQ+je3a8PIzu2YnwMLpnx9PDJOi+ndHTDcaLu79iZuPaFB8LrHb3tQBm9jhwnrv/ADinbRtmZsCdwLPu/m6MQ05aPXGt5cAdyHUHygjdjBeR3L8UJ8QBXuvlcQ4vJeieHT+6ZyeG7tnxlcj7dqr9wEbx0W9yEPpwjuqk/teB04ALzeyqWAaWgg7oWpvZYDP7BTDdzL4d6+BSWEfX/Y/AZ83sfhK/3HGqaPda67Pco3TPjh/dsxND9+z4ist9O2l7nnuCu/8U+Gmi4+gL3L2C0DhFiQF33wdckeg4+gJ9lhNH9+z40ec8tnTPjq+e/jynWs/zJmB02HZxUCY9T9c6MXTd40fXOvZ0jeNH1zoxdN3jKy7XO9WS53eAyWY23syygIuBeQmOKVXpWieGrnv86FrHnq5x/OhaJ4aue3zF5XonbfJsZo8BbwAHm1mZmX3Z3RuBOcDzwPvAE+6+LJFxpgJd68TQdY8fXevY0zWOH13rxNB1j69EXm9z955uU0REREQkJSVtz7OIiIiISLwpeRYRERERiZCSZxERERGRCCl5FhERERGJkJJnEREREZEIKXkWEREREYmQkmfpU8xsvZkNibaOiIjEnu7Z0hspeRYRERERiZCSZ0lZZvZnM1tgZsvMbHabfePMbIWZPWJm75vZk2aWF1bl62b2rpktMbNDgmOONbM3zGyhmb1uZgfH9Q2JiKQw3bMlWSh5llT2JXc/GigBrjGzwW32Hwz83N0PBXYD/x62b4e7HwXcD1wflK0AZrj7dOAW4PsxjV5EpG/RPVuSgpJnSWXXmNli4E1gNDC5zf6N7v5a8Pq3wElh+/4YfF8AjAteFwK/N7OlwE+Aw2IRtIhIH6V7tiQFJc+SksxsJnAacIK7HwksBHLaVPNOtuuC701ARvD6e8BL7j4V+Ew77YmISDfoni3JRMmzpKpCYJe7Vwfj345vp84YMzsheH0p8GoEbW4KXl/eI1GKiAjoni1JRMmzpKrngAwzex+4k9CfAdtaCVwd1BlIaKxcZ+4CfmBmC/moZ0NERKKne7YkDXNv+1cQkdRnZuOAZ4I/54mISC+me7b0Jup5FhERERGJkHqeRUREREQipJ5nEREREZEIKXkWEREREYmQkmcRERERkQgpeRYRERERiZCSZxERERGRCCl5FhERERGJ0P8Hlev84RRtQ60AAAAASUVORK5CYII=\n", "text/plain": ["<Figure size 864x288 with 2 Axes>"]}, "metadata": {"needs_background": "light"}, "output_type": "display_data"}], "source": ["fig, ax = plt.subplots(1, 2, figsize=(12, 4))\n", "df[[\"alpha\", \"null\"]].set_index(\"alpha\").plot(ax=ax[0], logx=True)\n", "ax[0].set_title(\"Nombre de coefficients non nulls\")\n", "df[[\"alpha\", \"r2\"]].set_index(\"alpha\").plot(ax=ax[1], logx=True)\n", "ax[1].set_title(\"r2\");"]}, {"cell_type": "markdown", "metadata": {}, "source": ["Dans ce cas, supprimer des arbres augmente la performance, comme \u00e9voqu\u00e9 ci-dessus, cela r\u00e9duit l'overfitting. Le nombre d'arbres peut \u00eatre r\u00e9duit des deux tiers avec ce mod\u00e8le."]}, {"cell_type": "code", "execution_count": 19, "metadata": {}, "outputs": [], "source": []}], "metadata": {"kernelspec": {"display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3"}, "language_info": {"codemirror_mode": {"name": "ipython", "version": 3}, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.10.5"}}, "nbformat": 4, "nbformat_minor": 2}