{"cells": [{"cell_type": "markdown", "metadata": {}, "source": ["# R\u00e9gression polyn\u00f4miale et pipeline\n", "\n", "Le notebook compare plusieurs de mod\u00e8les de r\u00e9gression polyn\u00f4miale."]}, {"cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": ["%matplotlib inline"]}, {"cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": ["from papierstat.datasets import load_wines_dataset\n", "data = load_wines_dataset()\n", "X = data.drop(['quality', 'color'], axis=1)\n", "y = data['quality']"]}, {"cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [], "source": ["from sklearn.model_selection import train_test_split\n", "X_train, X_test, y_train, y_test = train_test_split(X, y)"]}, {"cell_type": "markdown", "metadata": {}, "source": ["On normalise les donn\u00e9es. Pour ce cas particulier, c'est d'autant plus important que les polyn\u00f4mes prendront de tr\u00e8s grandes valeurs si cela n'est pas fait et les librairies de calculs n'aiment pas les ordres de grandeurs trop diff\u00e9rents."]}, {"cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [], "source": ["from sklearn.preprocessing import Normalizer\n", "norm = Normalizer()\n", "X_train_norm = norm.fit_transform(X_train)\n", "X_test_norm = norm.transform(X_test)"]}, {"cell_type": "markdown", "metadata": {}, "source": ["La transformation [PolynomialFeatures](http://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.PolynomialFeatures.html) cr\u00e9\u00e9e de nouvelles features en multipliant les variables les unes avec les autres. Pour le degr\u00e9 deux et trois features $a, b, c$, on obtient les nouvelles features : $1, a, b, c, a^2, ab, ac, b^2, bc, c^2$."]}, {"cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [{"name": "stdout", "output_type": "stream", "text": ["1 0.189007413643138 0.17548948727814861 0.005909326000001158\n", "2 0.3090044704138045 0.3016856760353912 0.027130041999996024\n", "3 0.4065060987061494 -0.057880204420430736 0.22084438099999915\n", "4 0.5874526458338967 -3659.6472584680923 2.230189553999999\n"]}], "source": ["from time import perf_counter \n", "from sklearn.linear_model import LinearRegression\n", "from sklearn.preprocessing import PolynomialFeatures\n", "from sklearn.pipeline import make_pipeline\n", "from sklearn.metrics import r2_score\n", "\n", "r2ts = []\n", "r2es = []\n", "degs = []\n", "tts = []\n", "models = []\n", "\n", "for d in range(1, 5):\n", "    begin = perf_counter ()\n", "    pipe = make_pipeline(PolynomialFeatures(degree=d), \n", "                         LinearRegression())\n", "    pipe.fit(X_train_norm, y_train)\n", "    duree = perf_counter () - begin\n", "    r2t = r2_score(y_train, pipe.predict(X_train_norm))\n", "    r2e = r2_score(y_test, pipe.predict(X_test_norm))\n", "    degs.append(d)\n", "    r2ts.append(r2t)\n", "    r2es.append(r2e)\n", "    tts.append(duree)\n", "    models.append(pipe)\n", "    print(d, r2t, r2e, duree)"]}, {"cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [{"data": {"text/html": ["<div>\n", "<style scoped>\n", "    .dataframe tbody tr th:only-of-type {\n", "        vertical-align: middle;\n", "    }\n", "\n", "    .dataframe tbody tr th {\n", "        vertical-align: top;\n", "    }\n", "\n", "    .dataframe thead th {\n", "        text-align: right;\n", "    }\n", "</style>\n", "<table border=\"1\" class=\"dataframe\">\n", "  <thead>\n", "    <tr style=\"text-align: right;\">\n", "      <th></th>\n", "      <th>temps</th>\n", "      <th>r2_train</th>\n", "      <th>r2_test</th>\n", "    </tr>\n", "    <tr>\n", "      <th>degr\u00e9</th>\n", "      <th></th>\n", "      <th></th>\n", "      <th></th>\n", "    </tr>\n", "  </thead>\n", "  <tbody>\n", "    <tr>\n", "      <th>1</th>\n", "      <td>0.005909</td>\n", "      <td>0.189007</td>\n", "      <td>0.175489</td>\n", "    </tr>\n", "    <tr>\n", "      <th>2</th>\n", "      <td>0.027130</td>\n", "      <td>0.309004</td>\n", "      <td>0.301686</td>\n", "    </tr>\n", "    <tr>\n", "      <th>3</th>\n", "      <td>0.220844</td>\n", "      <td>0.406506</td>\n", "      <td>-0.057880</td>\n", "    </tr>\n", "    <tr>\n", "      <th>4</th>\n", "      <td>2.230190</td>\n", "      <td>0.587453</td>\n", "      <td>-3659.647258</td>\n", "    </tr>\n", "  </tbody>\n", "</table>\n", "</div>"], "text/plain": ["          temps  r2_train      r2_test\n", "degr\u00e9                                 \n", "1      0.005909  0.189007     0.175489\n", "2      0.027130  0.309004     0.301686\n", "3      0.220844  0.406506    -0.057880\n", "4      2.230190  0.587453 -3659.647258"]}, "execution_count": 7, "metadata": {}, "output_type": "execute_result"}], "source": ["import pandas\n", "df = pandas.DataFrame(dict(temps=tts, r2_train=r2ts, r2_test=r2es, degr\u00e9=degs))\n", "df.set_index('degr\u00e9')"]}, {"cell_type": "markdown", "metadata": {}, "source": ["Le polyn\u00f4mes de degr\u00e9 2 para\u00eet le meilleur mod\u00e8le. Le temps de calcul est multipli\u00e9 par 10 \u00e0 chaque fois, ce qui correspond au nombre de features. On voit n\u00e9anmoins que l'ajout de features crois\u00e9e fonctionne sur ce jeu de donn\u00e9es. Mais au del\u00e0 de 3, la r\u00e9gression produit des r\u00e9sultats tr\u00e8s mauvais sur la base de test alors qu'ils continuent d'augmenter sur la base d'apprentissage. Voyons cela un peu plus en d\u00e9tail."]}, {"cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [{"data": {"image/png": "iVBORw0KGgoAAAANSUhEUgAAArwAAAEMCAYAAAAmmGWrAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzs3XlcVNX/+PHXGfZ9R0UEXBIRV6C0RbO0LE2rj5XlJ03TssVspSxbrDTzq/0qbfm0W2aZWWmkpZYtmiu45YYbiyDKIsi+DJzfHzMYKpsKzIDv5+PBA5h77r3vO1ze855zz9yjtNYIIYQQQgjRUhksHYAQQgghhBCNSQpeIYQQQgjRoknBK4QQQgghWjQpeIUQQgghRIsmBa8QQgghhGjRpOAVQgghhBAtmhS8TUQpZVBK/aiUuq+WNiFKKa2UsjX//rNS6p7z2FeQUipfKWVzITG3dGc+382ZUmqAUirF0nEI0VJIzrY+krPFhZCC9xwppRKVUkXm5HRcKfWZUsq1HqvOAH7TWn9U331prW/UWn9ez5gGVVkvWWvtqrUur+++hBCiJZKcLYQAKXjP1zCttSsQAVwKPH9mA2Vy6vnVWj+rtX67CWNslqSHw7q0hJ4UIZCc3WgkZ1sXydk1k4L3AmitU4GfgW4ASqk/lFIzlFJ/A4VAB6WUh1LqE6VUmlIqVSk1vTJBKKVslFJzlFKZSqnDwNCq2zdvb0KV3+9TSu1VSuUppfYopSKUUguAICDG3IPxdDWX2QLMl+ZOKKUOVr1Ep5SappRarJT6wrzd3UqpqCrLnzHHnaeUildKDazuuVBKDTHHlGdu/5T58bFKqXVntNVKqU7mn+crpd5XSq1QShUA11Sz7fZKqT/N216tlHpHKfWledlZl4Wq9p6YL0tOUUodUkplmY/Vu4ZjaJT9KKUclVJfmh/PUUptUUq1qmb/U5RSS8547G2l1Fzzz+Oq/P0PK6UmVncc5rYBSqnvlFIZSqkEpdTkKsvmK6WmV/n9tGMzH9czSqmdQIFSyra+54EQ1kxy9mmxSs6WnH1RkYL3Aiil2gFDgG1VHh4N3A+4AUnA54AR6AT0Bq4HKhPifcBN5sejgNtq2dftwDRgDOAODAeytNajgWTMPRha6/+rZvWvgRQgwLyP1844+YcDiwBP4EfgHfM+Q4FJwKVaazdgMJBYQ4ifABPN7boBa2o6lmqMwnT50A1YV83yr4A4wBd4FTiXMXKTgVuAqzEdfzbwbg1tG2s/9wAeQDvAB3gAKKpmG18DQ5RS7nCq5+QOc1wA6ZjOF3dgHPCmUirizI0oUy9VDLADaAsMBB5TSg0+h+O5C9OLuSfQkfqfB0JYLcnZp5GcLTn74qK1lq9z+MJ00uQDOZiS43uAk3nZH8ArVdq2Akoql5sfuwv43fzzGuCBKsuuBzRgW2V7E8w/rwQerSWmQVV+D6ncDqZ/2HLArcrymcB888/TgF+rLOsKFJl/7oTpH3YQYFfH85IMTATcz3h8LLDujMc00Mn883zgi1q2G4TpxcelymNfAV+afx4ApNT0fAB7gYFVlrUByszPTdXnqTH3cy+wHuhRj/NrHTDG/PN1wKFa2i6tPCeqxgf0AZLPaPss8FmV53x6lWWnHZv5uO6t8nu9zwP5ki9r+0Jydk3Pi+RsydkX1Zf08J6fW7TWnlrrYK31Q1rrqu/8jlT5ORiwA9LMl0VygA8Af/PygDPaJ9Wyz3bAofOINQA4obXOO2M/bav8fqzKz4WAo1LKVmt9EHgMU4JNV0otUkoF1LCfEZh6TpLMl5guP4cYj9SyLADI1loXnBF/fQUDP1R5/vdiejE58/JUY+5nAaYXv0VKqaNKqf9TStnVsJ2vML3AgqkXpbKnAKXUjUqpjcp0mTMH0/PtW0MsAZWxmNs+V80x1+bU3+QczwMhrJHk7LNJzpacfVGRgrfh6So/H8HUW+BrTraeWmt3rXW4eXkapqRYKaiW7R7BdJmirn2e6SjgrZRyO2M/qbWs8++Gtf5Ka30Vpn9IDcyqod0WrfXNmF4YlgKLzYsKAOfKdkqp1ucYfxrgpZRyOSP+Smdu3wbwq7L8CHBjleffU2vtqE1j+ZpkP1rrMq31y1rrrsAVmC5xjanheL8FBiilAoFbMSdPpZQD8B0wB2iltfYEVgCqmm0cARLOiMVNaz2kumMB6vyb1Pc8EKIZkpwtOVty9kVACt5GpLVOA1YBbyil3M2D5Dsqpa42N1kMTFZKBSqlvIAptWzuY+AppVSkMumklAo2LzsOdKghhiOYLs3MVKaB+D2A8cDCuuJXSoUqpa41/+MWYxrDdNZtc5RS9kqp/yqlPLTWZUBulXY7gHClVC+llCOmd5z1prVOAmKBl837uQoYVqXJfky9G0PN78CfBxyqLP8fMKPyuVJK+Smlbm7K/SilrlFKdTcn3FxMl82qvf2Q1joD02XRzzAlwL3mRfbm/WUARqXUjZgup1ZnM5CrTB9acFKmD9p0U0pdal6+HdO4M2/zi9ljNWwHc/z1Og+EaO4kZwOSsyVnt1BS8Da+MZhO/D2YBsUvwTRWCOAjTJdNdgBbge9r2ojW+ltMHxL4CsjD9I688pOrM4HnzZdCnqpm9bswjX06CvwAvKS1Xl2P2B2A14FMTJfQ/DFdZqnOaCBRKZWLaYD/3ea49wOvAL8CB6j+Aw51GYVpjNMJ4CXgi8oFWuuTwEOYXlxSMb0TrvrJ3LcxfahjlVIqD9ho3lZT7qc1pr97LqbLZn8CX9ZyvF9hGnt16tKY+fLmZEwvuNnmWH+sbmVtupfnMKAXkIDp7/cxpg9hgOly3Q5M475WAd/UEguc23kgRHMnOVtytuTsFkhpXduVCSGsj1JqGqYPUNzdEvYjhBAtmeRsYQ2kh1cIIYQQQrRoUvAKIYQQQogWTYY0CCGEEEKIFk16eIUQQgghRIsmBa9o9sy311ll6TiEEEIIYZ2k4BXNilIqRCmllVK2lY9prRdqrWu6v6EQQohGoJRKVEoNusBtjFVKnc+tz4Q4J1LwiiZVtVAVQgghhGgKUvCKU5RSU5RSh5RSeUqpPUqpW82Pj1VK/a2UmqeUOqmU2qeUGlhlvT+UUjOVUpvNy5cppbzNyyp7ZMcrpZKBNebH+yql1ptvvL5DKTXgjO29at5nnlJqlVKqcv7xv8zfc5RS+Uqpy6v2EJhnNHpTKZVujmWnUqqbedkQ83HlKaVSK2/4rpTyUkr9pJTKUEplm38OrBJPe6XUX+b1flVKvauU+rLK8hqPRQghWiKl1AJMU/nGmHPx03Xk9bFKqcPmPJpgHooWhmnGs8vN28ix0OGIi4AUvKKqQ0A/TLO7vAx8qZSqnGGoD3AY8MU0o833lUWt2RjgXiAAMAJzz9j21UAYMFgp1RZYDkzHNPPQU8B3Sqmq85yPAsZhmiHG3twGoL/5u6fW2lVrveGM/VxvbtMZ8ARGAlnmZZ8AE7XWbkA3zMU3pv+DzzDNOR6EaRrGd6ps8ytMUz/6YJpmc3TlgnoeixBCtCha69FAMjBMa+2KaerjanOhUsoF02vCjeb8ewWw3TwN7wPABnM+97TEsYiLgxS84hSt9bda66Na6wqt9TeYppW8zLw4HXhLa11mXhYPDK2y+gKt9S6tdQHwAnCHeR7yStO01gVa6yJMU1iu0FqvMO9rNaY50YdUaf+Z1nq/uf1iTFMu1kcZ4AZ0wXTbvb1a67Qqy7oqpdy11tla663m487SWn+ntS40Twc5A1OBjlIqCLgUeFFrXaq1Xsfp00PW51iEEKKlqysXVgDdlFJOWus0rfVui0UqLkpS8IpTlFJjlFLbzZejcjD1glYOJUjVp9+0OQlTb26lI2css6uy7pnLg4HbK/dj3tdV/DtfPZjm/65UCLjW5xi01msw9c6+CxxXSn2olHI3Lx6BKfkmKaX+VEpdbj5uZ6XUB0qpJGWaV/4vwNNcsAcAJ7TWhRdwLEII0dLVmAvNHSEjMfXmpimlliululgyWHHxkYJXAKCUCgY+AiYBPuZLS7sAZW7SVimlqqwSBByt8nu7M5aVAZlVHqtaLB/B1CPsWeXLRWv9ej1CrXOmFK31XK11JBCOaWhDtPnxLVrrmzENk1iKqecY4EkgFOijtXbn32ETCkgDvJVSzjUc64UcixBCNGf1zuta65Va6+swdQbsw/R6c+Y2hGg0UvCKSi6YEk8GgFJqHKYe3kr+wGSllJ1S6nZM43FXVFl+t1Kqq7kwfAVYorUur2FfXwLDlFKDlVI2SilHpdSAqh8Uq0UGpktjHapbqJS6VCnVRyllBxQAxUC5Usre/CEJD611GZALVMbnhmncbo55XPJLldvTWidhuiw3zbyNy4FhDXQsQgjRnB3n31xcYy5USrVSSg03j+UtAfL5N/8eBwKVUvZNH764mEjBKwDQWu8B3gA2YEpA3YG/qzTZBFyCqdd2BnCb1jqryvIFwHxMQxEcgcm17OsIcDPwHKYC9gimXtg6z0fz0IIZwN/my2Z9z2jijqnnIBvT0IosYI552Wgg0Txs4QFMY84A3gKczMe2EfjljG3+F7jcvK3pwDeYkvYFHYsQQjRzM4HnzcMXRlJzLjRgupJ2FDiB6TMSD5m3sQbYDRxTSmUiRCNRpw/LFOJsSqmxwASt9VU1LP8D+FJr/XFTxmUpSqlvgH1a65fqbCyEEEIIi5NeKCHqYB4m0VEpZVBK3YCpF2OppeMSQgghRP3Uq+BVSj2qlNqllNqtlHqssYMSwsq0Bv7ANO5sLvCg1nqbRSMSohaSs4UQ4nR1DmlQplmqFmG6H2sppvGND2qtDzR+eEIIIc6F5GwhhDhbfXp4w4CN5pvyG4E/gVsbNywhhBDnSXK2EEKcwbYebXYBM5RSPphu3TQE022aTqOUuh+4H8DFxSWySxe5p7QQovmJi4vL1Fo356mhJWcLIS4a9c3Z9bpLg1JqPPAwpjGMe4AirfXjNbWPiorSsbFn5VchhLB6Sqk4rXWUpeO4EJKzhRAXi/rm7Hp9aE1r/YnWOkJr3R/TPfRkLJgQQlgpydlCCHG6+gxpQCnlr7VOV0oFAf/BdBN+IYQQVkhythBCnK5eBS/wnXk8WBnwsNY6uxFjEkIIcWEkZwshRBX1Kni11v0aOxAhhHUpKysjJSWF4uJiS4fSKBwdHQkMDMTOzs7SoTQ4ydlCXHwkZ9euvj28QoiLTEpKCm5uboSEhKCUsnQ4DUprTVZWFikpKbRv397S4QghxAWTnF07mVpYCFGt4uJifHx8WlziBFBK4ePj02J7QoQQFx/J2bWTglcIUaOWmDgrteRjE0JcnFpyXrvQY5OCVwghhBBCtGhS8AohmoVp06YxZ86cOttlZWVxzTXX4OrqyqRJk5ogMiGEENWxprwtH1oTQrQojo6OvPrqq+zatYtdu3ZZOhwhhBB1aIq8LT28QgirNWPGDEJDQxk0aBDx8fH1WsfFxYWrrroKR0fHRo5OCCHEmaw1b0sPrxCiTrM2z2LfiX0Nus0u3l145rJnalweFxfHokWL2LZtG0ajkYiICCIjI5k9ezYLFy48q33//v2ZO3dug8YohBDNkSVyNlh33paCVwhhldauXcutt96Ks7MzAMOHDwcgOjqa6OhoS4YmhBCiGtact6XgFULUqa539Y2lutvQWENPgRBCWDNL5Wyw3rwtBa8Qwir179+fsWPHMmXKFIxGIzExMUycONEqegqEEEKczZrzthS8QgirFBERwciRI+nVqxfBwcH069ev3uuGhISQm5tLaWkpS5cuZdWqVXTt2rURoxVCCGHNeVsKXiGE1Zo6dSpTp0495/USExMbPhghhBB1sta8LbclE0IIIYQQLZoUvEIIIYQQokWTglcIIYQQQrRoUvAKIYQQQogWTQpeIYQQQgjRoknBK4QQQgghWjQpeIUQzcK0adOYM2dOne1Wr15NZGQk3bt3JzIykjVr1jRBdEIIIc5kTXlb7sMrhGhRfH19iYmJISAggF27djF48GBSU1MtHZYQQogaNEXelh5eIYTVmjFjBqGhoQwaNIj4+Ph6rdO7d28CAgIACA8Pp7i4mJKSksYMUwghhJm15m3p4RVC1OnYa69Rsndfg27TIawLrZ97rsblcXFxLFq0iG3btmE0GomIiCAyMpLZs2ezcOHCs9r379+fuXPnnvbYd999R+/evXFwcGjQ2IUQwppZImeDdedtKXiFEFZp7dq13HrrrTg7OwMwfPhwAKKjo4mOjq5z/d27d/PMM8+watWqRo1TCCGEiTXnbSl4hRB1qutdfWNRSp31WH16ClJSUrj11lv54osv6NixY6PHKYQQ1sRSORusN29LwSuEsEr9+/dn7NixTJkyBaPRSExMDBMnTqyzpyAnJ4ehQ4cyc+ZMrrzyyiaMWAghLm7WnLflQ2tCCKsUERHByJEj6dWrFyNGjKBfv371Wu+dd97h4MGDvPrqq/Tq1YtevXqRnp7eyNEKIYSw5ryttNZ1N1LqcWACoIF/gHFa6+Ka2kdFRenY2NgGC1II0fT27t1LWFiYpcNoVNUdo1IqTmsdZaGQGoTkbCEuPpKza1dnD69Sqi0wGYjSWncDbIA7zzNWIYQQjUhythBCnK2+QxpsASellC3gDBxtvJCEEEJcIMnZQghRRZ0Fr9Y6FZgDJANpwEmttdznRwghrJDkbCGEOFt9hjR4ATcD7YEAwEUpdXc17e5XSsUqpWIzMjIaPlIhhBB1kpwthBBnq8+QhkFAgtY6Q2tdBnwPXHFmI631h1rrKK11lJ+fX0PHKYQQon4kZwshxBnqU/AmA32VUs7KdDfhgcDexg1LCCHEeZKcLYQQZ6jPGN5NwBJgK6bb2xiADxs5LiGEOM20adOYM2dOne02b9586j6OPXv25IcffmiC6KyH5GwhhLWwprxdr5nWtNYvAS81+N6FEKKBdevWjdjYWGxtbUlLS6Nnz54MGzYMW9uLZ2JJydlCiOakKfK2zLQmhLBaM2bMIDQ0lEGDBhEfH1+vdZydnU8lyeLi4mrndRdCCNE4rDVvXzxdHkKI87Z28X4yj+Q36DZ927nS747ONS6Pi4tj0aJFbNu2DaPRSEREBJGRkcyePZuFCxee1b5///7MnTsXgE2bNnHvvfeSlJTEggULLqreXSGEsETOBuvO2/IqIISwSmvXruXWW2/F2dkZgOHDhwMQHR1NdHR0rev26dOH3bt3s3fvXu655x5uvPFGHB0dGz1mIYS4mFlz3paCVwhRp7re1TeW6i5r1aenoFJYWBguLi7s2rWLqKg6p1oXQogWwVI5G6w3b8sYXiGEVerfvz8//PADRUVF5OXlERMTA5h6CrZv337WV2XSTEhIwGg0ApCUlER8fDwhISGWOgwhhLhoWHPelh5eIYRVioiIYOTIkfTq1Yvg4GD69etXr/XWrVvH66+/jp2dHQaDgffeew9fX99GjlYIIYQ1522ltW7QDQJERUXp2NjYBt+uEKLp7N27l7CwMEuH0aiqO0alVJzW+qIa/yA5W4jmT3J27WRIgxBCCCGEaNGk4G3mdFkZub+sRJvHvgghhBBCiNNJwdvM5a5cRepjj5H5ocwcKoQQQghRHSl4m7nCLVsAyHzvfYr37rVwNEIIIYQQ1kcK3mauMDYWp969sfHy5OiUZ9GlpZYOSQghhBDCqkjB24wZs7IoPXQI12uvoc3Lr1ASH0/G++9bOiwhhBBCCKsiBW8zVhgXB4BzVBRu116Dxy23kPXhRxT984+FIxOi4U2bNo05c+bUu31ycjKurq7ntI4QQoiGY015WwreZqwwNhbl6IhTeDgArZ57FltfX44++ywVJSUWjk4Iy3r88ce58cYbLR2GEEKIemrMvC0FbzNWGBuLU69eKHt7AGzc3Wkz/VVKDx4ic948C0cnxIWbMWMGoaGhDBo0iPj4+Hqvt3TpUjp06EC4+c2gEEKIpmGteVumFm6myvPyKNm7D9+HHz7tcdd+/fC8/XayPv0M14EDce7d20IRipbk9/kfkp50uEG36R/cgWvG3l/j8ri4OBYtWsS2bdswGo1EREQQGRnJ7NmzWbhw4Vnt+/fvz9y5cykoKGDWrFmsXr1ahjMIIS5KlsjZYN15WwreZqpo61bQGueos2fT83/mafL/XkfalGdpv/QHDE5OFohQiAuzdu1abr31VpydnQEYPnw4ANHR0URHR9e43ksvvcTjjz+Oq6trk8QphBDCxJrzthS8zVRhbCzY2eHUs8dZy2xcXQl47TWSx44j4623aPXssxaIULQkdb2rbyxKqbMeq6unYNOmTSxZsoSnn36anJwcDAYDjo6OTJo0qSlCFkIIi7NUzgbrzdtS8DZThVticerWrcbeW5e+ffEaNYoTXyzAbdAgnC+9tIkjFOLC9O/fn7FjxzJlyhSMRiMxMTFMnDixzp6CtWvXnvp52rRpuLq6SrErhBBNwJrztnxorRmqKCqiaNcunKMia23n/+QT2AUGcvS5qVQUFDRRdEI0jIiICEaOHEmvXr0YMWIE/fr1s3RIQgghamHNeVtprRt8o1FRUTo2NrbBtytMCjZuJHnsONp98D9cr7661raFsbEkjR6D11130vrFF5soQtES7N27l7CwMEuH0aiqO0alVJzW+uzB8S2Y5Gwhmj/J2bWTHt5mqHBLLCiFU0REnW2do6LwHjOG7K++pmDDhiaITgghhBDCukjB2wwVxsbiENYFGzc3juYf5ZHfHuFYwbEa2/s9/hj2ISEcnTqV8vz8JoxUCCGEEMLypOBtZnRpKUXbt5+6Hdn3B77nj5Q/mLlpZo3rGBwdCXh9JsZjx0mfNaupQhUtQGMMebIWLfnYhBAXp5ac1y702KTgbWaKdu1Gl5TgHBWF1pqViStxtHFkzZE1/J78e43rOfXqhc/4e8n5dgn5f/3VhBGL5srR0ZGsrKwWmUC11mRlZeHo6GjpUIQQokFIzq6d3JasmSmMM32wxDkqigM5B0jMTWTKZVNYsn8Jr21+jT5t+uBs51ztur6TJpH3+++kPf8CHWJ+xMbDoylDF81MYGAgKSkpZGRkWDqURuHo6EhgYKClwxBCiAYhObt2UvA2M4Wxsdh37Iittzertn2FQRkYHDKYcJ9wRv88mvd3vM+TUU9Wu67BwYGA12eROHIkx1+bScCs15s4etGc2NnZ0b59e0uHIYQQoh4kZ9euziENSqlQpdT2Kl+5SqnHmiI4cTpdXk5R3NbThjNEtYrC18mXXv69GHHJCBbsWUD8ifgat+HULRzfifdzctky8tasacLoG8DOxfBmN5jmafq+c7GlI/qXNccGDR+ftR/vRazZ5Ww5l0R9ybnSMjXR37XOgldrHa+17qW17gVEAoXAD40SjahVSXw8Ffn5pw1nuD74+lPLH498HA8HD17Z+AoVuqLG7fg+8AAOXbqQ9uJLGLOzmyL0C7dzMcRMhpNHAG36HjPZOhKeNccGDR+ftR/vRa5Z5Ww5l0R9ybnSMjXh3/VcP7Q2EDiktU5q8EhEnQpjK8fvRrIqcRUGZWBg8MBTyz0cPHgq6il2Zuxkyf4lNW5H2dsT8PpMyk+e5Pir0xs97gbx2ytQVnT6Y2VFpsctzZpjg4aPz9qPV1Rl3TlbziVRX3KutExN+Hc914L3TuDr6hYope5XSsUqpWJb6oBpSyvcEotdYCC2rVufNpyhqps63ESf1n14K+4tMosya9yWY5cu+D30ILkrVpD7y8rGDv3CnUw5t8ebkjXHBg0fn7Ufr6jKunO2nEuivuRcaZma8O9a74JXKWUPDAe+rW651vpDrXWU1jrKz8+voeITZlprCmNjaxzOUEkpxdS+UykuL2b2ltm1btNnwgQcw8M59vLLGLOyGiv0huFRwycza3q8KVlzbNDw8Vn78QqgmeRsOZdEfcm50jI14d/1XHp4bwS2aq2PN3gUok6lhw9Tnp2N86VR1Q5nqKq9R3smdJ/AioQVrD+6vsZtKjs7Al6fSUV+PsemvWzd9+4b+CLYOZ3+mJ2T6XFLs+bYoOHjs/bjFZWsP2fLuSTqS84Vq5KVksx7E0axYMqj/P75RxzYvJ7C3JPnvqEm/Luey23J7qKGS2Oi8RVuMY3fdYqMZOW2R6odzlDV+O7jWZGwghkbZ/D9zd/jYONQbTuHSy7B79HJpM95g9yfluMx7KZGif+C9bjD9P23V0yXOjwCTf8QlY9bkjXHBg0fn7Ufr6hk/TlbziVRX3KuWA2tNb9//hEV5eU4OLuwc/XPbF2xDACfwCACw7oR2LUbgV3CcfX2qX1jTfh3VfXp1VNKOQNHgA5a6zpL+KioKB1r/oCVaBipT0VTuGkTFcs+4raY23i+z/OM7DKy1nU2HN3A/avvZ2KPiUzqPanGdrq8nKRR/6UkMZEOMT9i5+/f0OEL0WwopeK01lGWjuNCSM4WQjSWw1u38MOslxkw5j4ih96MsayM44cOkLJ3Fyn7dpO6bw9lxaYPonm2bmMqgM1f7n7+KKUaNJ765ux69fBqrQuBOsp00Vi01hRu2YLzpVH8kLS61uEMVV0ecDlDOwzlk12fMKTDEDp4dKi2nbKxoc3MmSTceivHXnyJwPffa/ATUgjRdCRnCyEaQ7nRyB9ffIxXm7b0GjwEAFs7O9p26UrbLl3pA1SUl5OeeNhUAO/dzcHNG9j1+2oA3Hz8CAwLJ7BrN9p2Ccfdrw0VRk25UWMsK8fVyxGDoXHqD5lprRkoS03FePw4ThGRrExcVOdwhqqeinqKv1L+YvrG6Xxy/Sc1FrIOHdrj/8TjHJ/5Oid/WIrnf25tyEMQQgghRDO3Y9VystNSGf7kC+z47SgFOSWUGysoL6ug3FiBsayCcqOm3FhOeVkQ5cZAvIIHUVJwnJL8JIqLkti3fjN71/1h2qByxmDbFoNtIAbbQO6dczMuno6NErsUvM1A5fjdjFA/EvckcnfY3fVe19fJl8cjH+eVDa8QcziG4R2H19jWa/Roclev5vjMmbhdew02np4XHLuwHlpr6bkXQghxXgpzT7J+yVcE9+jNsSQvdv52CHsnW2zsDNjYKmxsDdjaGbCxNWBjZ8DW3oCDi63pd9uCpIISAAAgAElEQVRO2Nh1xtbWgMFGUVacSX7WYXIzD5Fz7CAlBQcAKM7vh4tnQKPELwVvM1AYuwWDhwerDfH1Hs5Q1YhLRrDs4DLmbJlD/7b98XSsvpBVBgOtX3iRhFtuIevTz/B/4vGGCF9YgaMHclj96W6Cwn0Y8N9QKXyFEEKckw1LvqK0sIguV97On4tS6HZ1W66+K/Q8t9YZuOLUb7kZ6Rw9sA+fwMYpduHcJ54QFlAYG4tzZASrjqw+p+EMlQzKwAt9XyC3NJc3t75Za1vH0M64DxnCiQULrP/evKJOWmu2/5rM0je3UVZazp51R9m47LClwxJCCNGMZB5JYsfqnwkfcD1bfs7Fq7UzV4zo1GDbd/fzp8sV/Rtse9WRgtfKlaWnU5aUTEF4CAknE6qdbKI+Qr1DGdN1DN8f+J6tx7fW2tZ30sPokhKyPvzovPYlrENpkZGVH+7i7yUH6dDTlzHTr6BrvwC2/pLEzt9ldiIhhBB101rzxxcfY+/kRGlpFMX5ZVx3bzh29jaWDu2cSMFr5Yri4gDY1CrvvIYzVPVAzwdo49KGVze+Sll5WY3tHNq3x+OWW8j++mvKjlvvPetFzbJS81k8cwuHd2Ry5W2duH5CVxJ3bKTnAFfa9/Rl7eL9HIxLt3SYQgghrFzCtliSdm6jfcRQkvcU0vfmjvgFuVk6rHMmBa+VK9wSi3J24ge1/byGM1TlbOfMc32e42DOQT7f83mtbX0fegitNZn/+995709YRvzGNJa8HktZSTm3PN6bkO72LJn+PDFvvs6iac8QeYMbrdt7sPqz3aTGZ1s6XCGEEFaq8jZkHv5tOBLflrahXvQa1K5h95FfQO7q1RybPgNdUdGg266qxRa8Vj1N7jkojI1Fdw/lUH7ieQ9nqGpAuwEMDBrIBzs+ICWv5sva9oFt8bxtBDlLvqM0RS5/NwflZRX88VU8v87fi3+IO7dPieTYwb/4PPphjiccot+osRgMBr6f+QJX/McHD18nVry/k8yUfEuHLoQQwgptX2m6DZm9ywBs7ewYNDYM1QD3yS1NSeHEgi9JHj+BA5dfTuojkzm5bBllqakNEHX1WmTBG7sigW+mb6bgZImlQ7kg5Tk5lOzfz6FghwsezlDVlMumYFAGZmyaUesbA98HHkApReZ77zfIfkXjyc0q4vs5cez+K5WIwUH0H+lHzJsv8fv8D2kX1o175rzLZTffxu0vvAbAsjkv0v/OVtg52vLTvO3knSi28BEIIYSwJoW5J9nw3Vd4tu5CbnZrBvy3C65e53ePXG00UhgbS/qcORy66SYODbqO4zNmUJaWhtfo0QR98Tmd1/+NfbuG7T2uqsUVvPnZxcSuSCIrtYCYeTsoKTJaOqTzVrjV9OGyVV6pFzycoarWLq2Z1HsS61LXsTppdY3t7Fq1wuuuuzi5dCklhxMaZN+i4SXtzmLxa1vIOV7IDfeHY2u7ky+nPEpWShI3PPQ4t06ZhruvHwA+ge24/YUZVBiNLJ/7MgNGtaastIKYudspzq95XLcQQoiLi+k2ZIUUFl5G2JUBdIr0P6f1y0+e5OTy5aRGP82BK68i6e7RZM3/HFs/P1o9O4WOK3+h44rltHo6GpfLLkPZ2TXSkZi0uII3dkUiWmsG/DeU7KMF/Pz+TsrLGm9MSGMq3BIL9nasc09rkOEMVd3V5S66eHdh1uZZ5JfWfEnb5/77UI6OZL7zToPuX1y4igrN5pjD/PTODlw9HRk0rg2bvvs//vryU4J79mbsnPcIv3rgWffc9W0XzG3PT8dYUsKqD6bT/87W5GYWs/y9HZSVllvoaIQQQliLytuQObj1xrN1O/rdcUmd62itKTl8mKxPPiVp9Bj2X3ElR598ioJ163C95hravvUWnTesJ/izz/C+5x7sg4Ob4Ej+1aIK3pz0Qvb+nUb4VQGE92vLtfeEkbo/h9Wf7aGiovmN6S2MjSW7gx/ldjYNNpyhkq3Blhf7vkhGUQbzts2ruZ2PD96jR5O7YgXF8fENGoM4f0X5pfz0zg62LE8k9DI/gsISWPZ/T5N97ChDHnmKm596HldvnxrX9w/pwG3PT6e0qJC1X87iyttacSwhl1Uf76aivHm+QRRCCHHhKm9DpgwOYNOH6+7tir1j9fOU6dJSCjZs4Nhrr3Fo8A0cHjKU9NmzKc/Lw+e+CYQs+ppL1q0l4PWZuN8wGBs3y93doUUVvFt+SsBgo4gcEgJAaJ/WXDGiE4e2prNu8YFm9UG28vwCivfsYWtAcYMOZ6iqu193RoaO5Ot9X7M7c3eN7XzuHYfBzY2MeTUXxqLpHE/IZfGMLRzdn0PUDa4c2/8JG75dQIfIyxj7xnuEXTWgXjOptWrfkdumTqc4P5/N3/8/LhvqR+LOTP78en+z+l8RQgjRcCpvQ2aw68Nlw7rSur3HWW2MJ06Q+uRT7L/8CpLH3UvON4txaN+e1tNeotOa3+iw9Af8H3sMp169UDbWcb/eFlPwZqXms3/LcXpcG4iLh8Opx3tfF0SvQe34548U4n5JsmCE56Zo+3YoL2e9f06DD2eoanLEZHydfHl5w8uUV1R/OdvGwwPvcWPJ//U3iv7Z1WixiNpprfnnjxS+nxMHqoJOvRJZv/g1cjMzGPb4FIY/8Rwunl5nrVdRWMiJLxdyaMhQjs98/bTbvrTueAkjnnuFwtwc/vl1Ht0HeLJn3VG2/CRjtoUQ4mJTbjSy5rMPMdh4ERjWj8gbzh52UHY8naTRY8j77Tfchw4l8L336LxxA+0++B9ed96JXUDjTQ98IVpMwbvpx8PYO9jQ+/pgytLTKYyNPbXsiv90ovNlrdi07DB7/j5qwSjrrzB2CxUGxcHAhh/OUJWbvRtPX/Y0e0/sZVH8ohrbeY8Zg42nJxlz5zZaLKJmZSXl/PrZHv5atB+/oGJ06TfsWL2Ezn2vZOwb79G571VnrWPMziZj3jscvHYgx6dPR5cbOfH55xx76SV0+b9vbtpcEsp/nn2F/OxsDm36kI6RrmxZnsjutY13exghhBDWZ9svMZxMT8PB/RquG98Dg83pZWJZaipJo0djTEsj6KMPafPKy7hdew0GJycLRVx/LaLgPZZwkoQdmfS+Pgh7m3I23z+emClPcPCFF9ClpSiD4toxYbTr6s0fC+NJ3Jlp6ZDrVLgllpS29nRvd2mjDGeoanDwYK4MuJJ52+ZxvKD6mdVsXF3xuW8CBWvXUmie/U00jexjBSyZFcv+zan4t9vDkR3/ozjvJDc/9TxDJ0fj7H765abSlBSOvTqdg9dcS+a77+IUEUHwVwvp+Msv+D70IDnfLiHtuefQxn/vYNI2NIz/THmJ3KwM0vZ+RtvOjvz5VTyHt2c09eEKIYSwgMLck/z9zVcYbIO5duyNuPueXsSWJiWRePdoynNyCPrsU5wvvdRCkZ6fFlHwblp2GCc3O3pc247UWbPYaigjxcedmH1xrBw9kqIjydjYGrjh/m74tXNl5Ue7OHb4pKXDrlFFSQlFO3ewPaC0UYczVFJKMbXvVIwVRmZtmVVjO69Ro7Dx8yXjrbdljGcTORiXzrczY8nNSMLe9juSd/5Clyuv5p433qPTpX1Pa1u8Zw+pTzzJoesHk714Me5DhtBh+U+0e+9dnCMiUErhN3kyfo89xsllP5L6VDS67N9bkQWGdeM/z7xEbkY6OUcX4t3WhlWf7CbtYE5TH7YQQogm9tun8zGWFtPx0v8Q2qf1actKDh4k6e7R6OJigj+fj1PPnhaK8vw1+4I3Zd8JUvZlE3lDCCUb1xH3+ypK7WwZ9sSzBIV0YjdlfP7oRPZ99SX2jrYMfbgnLp4O/PTuDk6kFVg6/GoV79wJZUbi2zXucIaq2rm1Y2KPiaxOWs1fKX9V28bg5ITvxAco3LKFwg0bmiSui9nmnxL45cPtKL2egowvqSgv5tZnXuLGh5/AydX0SVetNQXr15N873gS/jOC/D//xHvcWDr9upqA12bg0LHjWdv1fWAi/s88Q94vv5Dy2ONUlJaeWtYuvAe3RL/AyWNplOQswcVDs/y9nZw4ap3/K0IIIS5c2sHD7N/wK47uvRl8f//TPvhcvHcvSaPHoNEEL/gCx7AwC0Z6/pp1wau1ZuOyw7h6ORAaZs/hF54nwd+LLpf3o3OfKxkxey7Dxj4ANjYsX7aIJQ9PwFiSw7DJvTDYGIiZu538bOubYapgi2n8sVNE70YfzlDV2PCxdPDowKsbX+VYwbFq23jecTu2bdqQ/rb08jamPX8fZdPSDVC2iNzjfxM+YBD3zHmXDhGmS0jaaCR3xQoSR9xG8r3jKT6wH78nn6DT72toFR2NXatWtW7fZ9xYWr34Avm//UbKw5OoKP73/yC4Ry9ufmoq2WlHqCj5AWUoJWbedvKzm/fMhUIIIc6mtSbmzXmg7Bny8AQcnP+dAKJoxw6S7hmLcnQkZMECHDp1smCkF6ZZF7yJ/2RxPCGXqBuDSX/pBfa52KHsbLlq1NhTbTrfeBP3fvwl3Tz8OJJ+lE8njWffmu8Y8kA4JUVGYubtoLjAumaYytq0liQ/6N91aJPu187Gjtf6vUZ+aT7jV46vdjyvwd4e34cepHjHTvL/+KNJ4wM4GLuJ3Mz0Jt9vU0rdn82a+X9QVrAEe8cKRjz3CoMfmIyjiysVRUWcWLiQQzfcSOoTT1JRVESb6a/S6bff8L3vPmzc3eu9H+9Ro2gzYzoF69ZxZOIDVBQWnloW0iuS4U9N5URqMrbEUFxQQMy87ZQUWtf/ihBCiAuzfsmv5GUeIKTXENr3Cjr1eOGWLSSPuxcbDw9CvlyAfUiI5YJsAM224NUVmk3LDuHh50Srw2s4GruJFA8Xet84HA//03u37D08uf6DTxlxw3/wyStg3feLWPHGk0QOtiEnvZAV7+/EaCUzTGmjkbLt/7AvyNBkwxmqCvcJ5/1B75NVnMX4VeNJLzy7uPS85RbsgoLImDvvtFtcNbakndtZNvtVFr30DHknrP+Dh+cjJ72Qn+b9Smn+97j7ejNq+huE9Iww3XHh3XdNd1x4dTq2Pj4EvjOPDst/wvO22zDY25/X/jxHjCDg/2ZRGBtL8oT7KM//d9a9Dr0vZdgTz3LiaCIO9ivITsthxfv/YCyzjv8VIS5WukKTfayA+E3H+H3RHubPXcX6+C21zpopRHVOZuSxeekX2Dr4MOyx0acez//7b5Lvux/b1q0J/nIBdm3bWjDKhqEa47J0VFSUjq1yW7DGsH/LMVZ/socBN3pheP4eYntcQq6jPePnfoSji2uN6xX98w9x0U+w08mGIntb2ob1ITOtJx16hXDD/d3OugVHUyvcuZOkO0YSM7YzT09ZZrE4tqdvZ+Lqifg7+/Pp4E/xc/Y7bfnJH3/k6NPP0Patt3C/YXCjx2MsK+OL6EkYy0opKcjHzcePkS/POjWWtSUoLihj0au/cCLpC5w9nBn16mwcS8o4MX8+Od99hy4qwnXAAHwmjMcpMrJek0vUV+4vK0l96ikcu3Yl6KMPsfH4984PBzatJ+at1/EK6EhB/g10igjg+vu6YTA03P6tiVIqTmsdZek4mlJT5GxxfrTW5GYWkZ6UR3pSHhlJuaQn51FWbHrjWW4oowJNnsMJloW/jb+3D128u5z25efk16D5QrQMukLzxbPvkZn4M9dNnEKPa023t8xb8zupjz6KfceOBH3yMbY+Nc/aaQ3qm7OrnyvOypWXV7D5xwS82zjj8NELpLXyId1YwjW33VNrsQvg1L07fb9eTNunn2bn/t0cZjPKbjsHNvXF0dWGa+7uatHEcGTtLwB07D+sXu2NZeUc3p5BqxB3PPycGyyOXv69eH/Q+zzw6wOMXzWeTwd/etp4YvehQ8n84EMy5s3D7bpBjT6TSuyP35GdlsqI517BYGPD9zNf4odZL3P789Oxc3Bs1H03hfLyCmLm/klW4pc4uTpwx9RXKPzgI1K//hqUwuOmm/AZfy8Ol9Q9n/n5cL9hMMrentRHHyVp7DhTkvP2BuCSPlcwdHI0y9+ejWfrlRzcOhjnb+zpd2dneREVogFprck7UUyGubhNT8olIzmPkkLTLQQNtgrfQDcuucyfvTZb+fbEl9j7VPBowLMkft2ae1Ne5FDI7+w9sYfVSatPbdfb0fusIjjYPRiDal4XectKS4j/+y8CQrviHdD8exwtbfPyvWQmrcGnXVe6X3MlALk//0xq9NM4hoWZOj88PZssnmJjMY62jfd63iwL3vgNxziZUcTl3nsoOXiIA9f0wdPejp7X3Viv9W29vAj54APcPviAtv97n32XBHHc5k+2/7yb8tL/ct346xr5CGp2fP3vGL3h6t631Kv9usUH2L3WNJmGd4ALIT18ad/Dl1Yh7qgL7IGLaBXBewPf46HfHmLCygl8PPjjU0WvsrHB75FHSH3sMXJ/+gmPm2++oH3VJuf4MTb9sJjOl/cjpGcEAEMmR/PTm7OIefN1bn7qeWxsm+WpDJhe5H79dCPJOz7B3lEx4tFocqdMpSguDs+77sT3/vuxa9Om0eNwu/YaAt97j5RJk0i+5x6CPv0UWz9Tz37o5f2oKC/n53f+H+5+sPOPwbh4ORB5Q0ijxyVavqLdu8n+4gv8p0zB1uvs2QJbIq01BTmlp4ra9KRc0pPyKM43jZM3GBQ+ga50jPTHP8gN/2B3vANcOFqUynNrn2N7xnaGhA9hat+puNu7c9ApnZUf72LggTHMntidAmM++7P3s+/EvlNfX+z5AmOFqXh2snWis1fnUwVwmHcYnbw64WDjUFvYFlFWWsLO1b+w5cclFORkY+foxA0PPlrthDuifjKO5LFxyULQZQyd/BBKKXKWLiXtuak49e5Nuw/+h41r7R2IF6K0vJT92fvZlbmLXZm72J21m8STifx151+42TfOldtmN6TBWFbOwhc34mgooft3D3PipsFsOnKQ4U88xyV9rjjn7eX//TepT0WTZgs7g4MpK8sjILQvw594uNppWhtTRXk526J6cLCnLyPn/1ln+4Nx6az8aBfdrm6Lp78zCTszOXogB12hcXK3J6S7D+17+BIY5o2d/fn3wG45toWHf3uYtq5t+fj6j/FxMl3e0BUVJNx2GxV5+XRcsRxlZ1fHls6d1pofZr1Myt7djHvzfdy8/+1l3vnrL6z+6B3C+l3DjQ89jjI0r96KSpt+/Ie/v34dg00pI8bdT/H0mZSfOEGbGTPwuKlpP7gIULBpM0cefBA7f3+C5n+GXet/78e45681/Pzem7j6XEKZ8Qb63RFGz4HtmjzGxiRDGppWaUoKiSPvpDwrC7fBg2n71pst+spBWWk5fyzcR8rebApzTbcEVAaFd4CLubB1wy/YHZ+2Ltja/Zu3tdbEHI7htU2voVA83/d5hnY4PT/sWHOEdYsP0K1/W/rfdfYVmLLyMg6dPMTerL3EZ8ef+l5QZrrtoI2yob1He3r792Zk6EhCvUMb+dmoXVlpCf/8+gubl5kK3XbhPeh94zC2LFtC2oF4IofeQr9RY62iw6OstIR1X31OwvZY2ve+lPCrB+If0sHSYVWrrLScr15cTmbCR3S79gYGT3yY7EXfcGzaNFyuuJzAd97B4NxwV4zLK8pJOJnArqxdpwrc/dn7KaswvbnzdvQm3Cecbr7dGNVlFJ6O59ar3GKHNOz+6yj52SWEHvwE27Au/JObSUBoVzpddvl5bc/1yivp8P13ODz2OF5b/2HL5cM5Gr+ZTybvoN+oMfS8bgiGRr5cX+ng1jU4F1XgWY/CPTeziN8X7KVVe3euuuMSbGwM9BzYjuKCMpL3ZJGwI5NDcens/TsNWzsDgWHetO/pS0h3X5zdz+0DTpe2vpR3rn2Hh397mAmrJvDJ4E/wdvRGGQz4TZ5MygMPkvPDD3jdccf5HnqNDm7ZQMK2WAaMue+0Yhegx6AbzDPDLMDJzZ0BYyY0uxfKfRsO8ffX/4dSxQwdNJSCJ5/GxtOT4IULceoWbpGYXPpcRtDHH3Pk/vtJuns0QfPnYx9ounzYtf+1VJSXs/J/b+Pqo1i7GIrySulzc4dm99wLyys/eZIj909EG414jhxJzjffkLt8hUXe6DUFrTW/zd/LoW3phF7WGv8Qd/yD3fANdMW2lk6JkyUneXXjq6xMXEmEfwQz+80kwDXgrHY9r21HQU4J21Yl4+LpQNSQkNOW29nYnerRrVShK0jNS2Vf9r5TBfBPh3/i2/3f0rdNX8Z0HcOVba9s0uEPZxa6vkFd6BB1N2Wlrdm4rIDWHe7Bq82fxC1fyrFDB7jpsWdw9fJusvjOdPzwQVbMm8OJoykEdO7K9pXL2bpiGX5BIXS9eiBhVw1o8g602qxfcoATR1Zi5+hEv7vuJmv+fNJfn4XrgAG0ffstDA7n38uvtSYlP4XdmbtNxW3WLvZk7aHIWASAi50L4T7h3N31brr5dKObbzfauLRpktePZtXDW1ps5MvnN+CSm0zPzXNIf3A8W9b8wqjpb9Dmkgt7J6pLSzk+ew6ZCxcR1/dhTpTuoKIsGb+QDgwa/yABnRv/RstLZ00k9LO/8F6+mFYdu9fYrtxYwfdztpJzvJCRUy89a/q/qu2OHsghYWcmiTsyyTtRDApahbjTvqcv7Xv44dXGud4n2sa0jUz6bRLB7sF8fP3HeDl6obUm6c67KDt+nI4rf7mgf5QzlRYX8dkTD+Lk4srdr7+NwcaG8vIKbKp8sFBrze+ff8i2n2O46q576HPL7Q22/8aWuj+NxS8/iy4/ycDOPbFf/A1OkZEEvv0Wtr5Nd//lmhT98w/JE+7D4OxM8PzPsA8OPrVs52+/sPrDd/AJuoz83CsJvyqAq0eFWvxDnw1BenibRkVpKUfGT6Bo+3aCPv0Ep969Sfrv3ZQkJtLhxx+xa+XfpPE0hS3LE9gck8AV/+lE7+uD6l4B2Jy2mefWPUdWURYP9XqIe7vdi42h5uJYV2h+/XwP+zcd59oxXQi74uzC+EwVpaUUbd1K/tq1FPy1ltK0o+QEe7PJK5sd/oWUdgnh5r5jGdZhWKONsTSWlZORlM22lT9zcPPPlBXnYmPfDoN9X2zs2qEMCi8/B1wdy0lNNaI1tA45RvLOb3FwduGmx54hMKxbo8RWk4qKcrYs+4713y7E3tkNzzbDycnwxcamBMVBSgt3UVqYCsqAZ+sutA27nKDwSFy8XHB2s8fJzQ4nV3ts7JoubybuzOTHt5ZSVrCUa+65j6DkNDLeett0dWX2/6HO8Y4/GYUZpwrb3Zm72Z21m5wS0+yc9gZ7uvh0OVXYhvuGE+Ie0uBvnuqbs+tV8CqlPIGPgW6ABu7VWtc41VZjJc/YFQls+jGByLjZBD50B9+t+YmOkZdx02PP/Nto52I2//kKm8tzcbd3xzPsZjwvGYyngyeeDp54OHjgZu9W4xOeu2IFSdNmEhv2MIX2ORjK11BYUMiVfon07QgMfBF6NHxPptaab0ZeRvvkUi7bsL3WInT9dwfZtjqZwYNz6ZT4PJxMAY/AWmPTWpOVWkDizgwSdmSSnpQHgLufE+3N437bFK3G8PsrtW5vw9ENPLLmEULcQ/j4+o/xdPSkYMMGksfdS6upU/Eeffe/jXcuht9q315t/vzyU2JjvufOV2bTNjSMgyt3sOaHNFr5K258diD2TqYLFLqighXvvMG+v//kuvsfocfAxr9rxIXKPpbNF09HYyzJoI9LK3zWr8Xzjjto/fzUc044ZyqvKCc1P5VDOYc4dPKQ6XvOIZJyk+jp15Mno56s96XK4n37SB53L8rWlqD5n502c9u6RQvY9MM3dLz0dlIPtqNDLz+uG9/1tMuw9XKB50lDb68lFLzWkrNroisqOBr9NLnLlxMwZw77o/xZfng5E72GcfKuCThfeintPvygRV01qByC1iW0mCuzolEFx7EPbIW67qVqz8+y8jLmbZvH/N3zCXIPYla/WYT7nnHVZ+diCn6eTk5WNgGtPVCDTOd6ubGC5e/uICU+hyEPdiek+9lvoEtTUilY+xf5a9dRsHEjurAQ7OxwjozEPiiI4t27KY6PB6NpzG+mGyQF2uPeO5Koa0fROuKK877sXZRXSmZKPplH8slMySMjOYfMpI2UFW0GXYCNXSC+bfvR2tMH14KjOB/bh/3+WHR6GgAVHcJJGxrN/t2FQBa6dDklBVn0/+84Iofe0iTnzcn0Y6x45w2Oxu/F2TOcct0fV9sigvPXUuHgSqlvOCXObcnLP0Fu9m5K8nahK/JBOWBjF4qNQ1eUjaln097RBic3e/OXnel70UGcjvyMQ0kqti6u2PYagU3nq7G1s8HWzoCtvQEbO4Ppd3sDtnaG2jscdi6mcOWbfH34SfJPfoebh2JIpyiyP/oIj5uH02bGDFQ9h4acLDnJ4j9fYEnKGo7amJ5rGxSdvDqfKmy7+XSjk1cn7AwNP9TxTA1d8H4OrNVaf6yUsgectdY5NbVvjORZXFDGF8+uwyNtJ1f4H2B3t0vYt+53xr35Pzz8TWMMj8V+zJwts1jpXPs7UBtlg4eDBx4OHng5eOHh4GEqiB1NRbF/ehn+c1awo9U9YFuKe8X7pOa7MqbDVnxdNQyb2+BF7/4T+8m87maI6s4VHyyusV3Srix+emcH4eFFDMgdD2VF/y60c6p3bPnZJST+k0nizkxS9mVTbqzAwZBPsH0clziuJcQxrsbtrU9dzyNrHqGjZ0c+uv4j3O3dSb5nLCWHD9Np9SoMTk6moiNm8nnHl5mcyIIpjxJ+9UCuHTWOja99y44TQTgVZ1Hs6I2bo5Fhz/XHs5ULAOXGMpbOnk7Sjm0Me2IKl1x27uO5m0phXj6fPhpNSUEq3UtdaXfgH1o/PxWvO+88p+0YK4yk5KWcVdgmnEygtOLf6YJbObeik2cnAlwDWJW0itySXG7udDOTek2ilUvtM7KBee4U9GcAACAASURBVA71ceOgvIKgzz7FMdRULFdUlLNs9nQSd2yl99BH2b0WAi7xZMhDPXBwqudoqQs8T2raXl5hOYVGO1o5FZzz9lpIwWvxnF2b9P/3JlkffojHo5P4tHc238R/A0CASwDvZg+h7I3/0fqVlxtlmJQlZCTn8f3sOHy9S+if8BCZm01vapVtBY5eFThGXYlj/1twDO+KQ4cOJOQnM2XtFPae2MttnW8jOioaZztTcfn/2TvvuKbOtw9fCQkJeyN7CSqigAP33tU6qnVW6+xu3aN1VKu2tdZRZ7XWqrV1b+vee6IogiDKHrI32TnvH1T6s+LG1TcXHz4J5OTkfpKc53zP/dyjKCebpJs3SD61k6TIa+SqSlf4OrvepIpdcdl3Xa3Usn1u6Upgt1G1cXCRUXLxEsWnT1F08hTquDgApK6umDVrinnTppjWq4+RuVmZ3XqlEmXkTRTXrpF68Tgl169hmV3alVEvFiGu7Il1rXqYBAViEhiIceXKD+RSKArVpNzKIyu5sEzkFueVdm0UBC1GokhUhefQaYuxNLaiSpEOh5hIRNrS+E6RsTHGvpWRV6mKrGpVJLY2pP84B31JCeYTvyUi142Yy0nolYfQKG/hV68xHT4dgbFJxcWg/i+CIHDj2GGO/LYMvQ4k8pZY21eheu7vmF64gkhTvqbSi0RkOzmTbGtJukSLHgG5xAxbq8pY2gSA3AGV3hilRoyySIlSKULg6ZwHYrEIo7/F7z0hbCQVI9XkYpQbRbHWhuziVNSK07TQJGAaqS11tEyb+kQ5MMmFyfxx8w+2RW9CodfQqERBU4WSGioVVQUjTN6ueG30JFSY4BWJRJbANcBHeML4hxcxeZ7dFMXVIyk0jF2G87zJrPt2MnU6dqXF+8PQ6DSsvbmWZZfnoRcEhuUXMCi/ELUI8sVG5Fo6k9d9KfmqfHKVueSp8krvq3LJV+WTp8ojT5lHniqvTCjI1AIfHHQCq5Ho9BkUKbfgKi+gn3s4Imt3GHWjQse3cu9MGo3+E/PJ43DvP6TcbYrzVGyYeREzK2PeNf0YSWEsJVop57I8UOn+9nZKTaFap3/a/goCwt+3//wtgEDZNnqdHkVMKMUaU0p01ghiB2rYamhu+StSG8dyx3o65TTDjw7H19qXFe1WIL1xm4T3+uM4bix2Q4fC/BqQn/TgIKwe/94JgsDGaRPITkninfbvcnFLLCk2dXCWZfHW+ObcmreG83nVEBnLaP9pLTxrli59apRKNs+YREZCLD2++gb3gMDHv/EvGbVKyW8jJlCcG0vlQguq58bjtnABpiEhD32ORq8hqSCpTNTG5sVyO/828fnxZUH/UCoWfKx9qGxVmcrWpb8+Vj6YG/+TaVugLmDF9RX8efNPJGIJAwMGMjhgcNnJ9KF2x8eTMGgwgkKB+8qVZfHFyuIi1k0ajaqkhEa9J3J2azq2rmZ0/iL4yWLFn+N78rD9KbLT2JAQhFpvxNDKl5CIhafa35sueF+XOfth5G7cxN2pU1G93ZwJ9WNJK7lL/2oDqG/cnK+jx4Fez9K/nJBExeK9ayfGbm4vxa4XRXG+ii2zSt/btpmjyT9dgLmLEgs3Bcpcaelvngyh1JGK3lhCrIOeJBdjajXpgV9QazIURSTHRJEcGU5uWmlVHmMjPa7yPNzM8onIq4QIgfd9riD++/wkCAL5EXfYsSIWjVJLnbB5mOSnIDI2xrRePcybNcWsSVOMvb2eyiOaEH+dI/t/JvPyWbySNVS9K0b2d01gsZkZ8po1MQkMRB4YSArunN5zF7VCi0gM1tZGWBkXY1acQkHqBRKVqagkYmyKFFS5m4OjlQ0mfwtbedUqyKpWxdjT8wHPoyY9nZThI1Bcu4bdsKEI3Ydyfnc88VcPolWcxsymEj0mTsHBw7O8ITwzxbl57Jgzl7u3ryKSuGHv2Y2Qt6pjObcjBVE6TB1UuDTIQyQW0KrE6CRO6Fr9iDYnF11ODtrcHHQ5uSizs0ksyiVR0JAtL/WA2hYpcMspxCm/CIleQECEzkQOZjJEpkaITCSILC0QGg1BZGmDYGENZpboxFJ0GgGtRodWrS+91ejR/c997Z2z6DQCap2OzNwD2KsLCY7IwK6mCMdNEY/9/G9k3WB1xGoOJRxCLBLTUaFhYOZdqmj+1XnzWeft56QiBW8w8AsQCQQBocAIQRCK/7Xdh8CHAB4eHnUSEhKe0fQHKc5XsXbCSezTQ3lrfHP2Hd1L+p0YhixcwbXCCL678B1x+XG0LC5hfE4ubtp/d4ISwbSHOjfKEAQBhVZRKoBVeeT90pL8mEDiJGMRik+h1IZR2+MWLc0ynmh/T4ogCMyc3IJ3t2bgs3cPMp8HMzv1eoFdC8JIj8un51ch2C53BQR2J1fjdqEdFlLVvZGC3b0Eor+/xCJR6b2/v9Si/729dz89Aig9yLJUZhgZB+Bg4U8Hm3nYfXu9XLtPJp9k5LGR+Nn4saLdCvI+G4syPJzKhw9hNMeN0pXUf/P4zyLixBH2L51PsNSGu9rG5FlXIaiOCY2HNkAkFiEIAvGzl3Ii3JJicxcadvGk1luVEYlEKAoL2DB1AkU5WfSaOotK3pUf+VovE61Gw+/jJ5KbehPX4krUk+bivnjRAx1sCtWFnEs9x6mUU9zIukF8QXxZKSEAV3PXMkF7T9x6W3ljJjX790s+lOTCZBZcWcD++P3Ym9jzWfBnvOP7ziNjA9XJySQOHIQuPx/3Fb9gWqsWANnJifw5aQx2rm407D2eQyujMbWS0WV4MFYO5ceXlzHNmmf9npRr49e2bEmoQYbKnB7uN3A3y3/q/f0HBO8rn7MfRtHJkyR98impAY6MfisDD2tvptadTuo2SLqZS/W37fhBNQHSs5i7Uo+5fwAev695YyuwaDU6dsy7SnZKEW1rZqGaOwVzFyVujXMQ/c+hJuhFpPc+zMZd36OOuIVnkS2CyogcmZQSWakgkgrgYG6Fq48vniENcT3VB4mk9NiJKrBnT4o/HZ2icCsppNj9c4pOnUKTlESJiSOhdcdhbCymUxcz7JrVK12Fe07yVflsubWF9ZF/IknOoGGuPa2LPHBOKKQoNpXoyj3JdKyNtSKJqrnHkd8OBUFDkq0lsZVsUUqNcDAxp3ateng3boa8SpX7WqMLgkB6STpx+XHEF8QTnx9PfEE8yYXJ1HKsxYf+g5EuWkveho2YNWqIy9y53M2A42sPk357MyI01O08lKb9Oj13iINOq+f0xkNc2fMbep0Cy0otad6/D+62CtJGj0J16xZ21QtxqFHI/dGSj5938u6mEnn4AJFnT5CfnYVEIsFTl463NhcHVTF6hREahRHaEiO0CjFl5/V7r2BqipGTE+JKDuDgiMjOFsHGBqytECwswMIc7dahaAURtwvtiC20o0l0El6Vc3CoWYTom/Lt0wt6TiWfYlXEKkLTQ7GQWtCzak/6VetHpR+rUpHz9vNSkYK3LnAeaCwIwgWRSLQAKBAEYcrDnlPR3oLDPx7hVoye9l43MerYmK3fT6VOn97ssb3O/vj9uJm78VX9r2i2+dMK9xaRn0RUagOOCCNRF/1BkTSHRnWSaT0i/PkH9jcxuTEc+aArjZNMqHH+crkH57345ZYDqlG9sQvMr8Ht5GJ2JgfQxCGe+vZ/j/s5xwpwJtOT81keyE1qYWTSmGb9A/FvVH4W5YmkE4w8PpJqNtVY7DaKjD4Dsf/icxzUPz/TZ1GSkcFvoz5CVqRCbDMYtYktrQb6U7XBg4kX6WvXc2JHKpkOtfALtqHVkEAkxkYUZmexfso4dFoNfabPxsbp8UkbLxqdVsuGr6dy98417DS+tPYQ4frdt4hNTREEgVu5tzidcppTKacIywhDJ+iwMLagtmNtfK19S7211j54W3o/1hv7NFzLvMacS3MIywzD19qXMXXH0MT14bUtNWlpJA4ajDYzE/fly8o80zGXzrFrzrcENG9DUIdB7FlyHSMjMZ2HB2Hv9oiaihXo4dVpNWz/tAOJ+XI6u93EzyL7mfb3HxC8r3zOLg9lZCSx/fqRbKNn8nvQJ3gQgyt/wOFlUWQmFeHgbk5GYiGNB3kyM30Criej+fgvLY4TJmA3eNALte1FcK8iQ/SFuzQNyEe6ZCLmnuBaL5X/va7MV8s4YuTNCaU1tlkizEtKvZkyM3NcPL1xNLXArkSJSWwiqps30RcUlD5RDDJLDXIbNRIzHTv1/uh0YppFJyE2McGsfv3SUIUmTcjRWbFz/lVsnMzoNroWxvKKK9Ck0Ws4GH+QNRFruJlzk+qF9Wl2uycitZRAl1w8s06j1+tJtJARkZZIcXERLlX9adTzPTxqBKHQKsoEbVxBXJmwTShIKMvsh9Lsfi9LLxxNHTmbehatXktX364MinNH9cMiJPb2uC5aiLx6dSJPx3Bk5Tw0imQs7OvT/pOP8Kzx9EmQWo2OG8cTOLNpNcqCK0jkDjTr9znBbWtTsGcPaVOnIpbJcGmQj7lV8oM7eIp5RxAEUm9FEXniCNHH96LSiTGXqHAxKUAriNHqjdCI5KgtK6NRKNCqlGg0GnQ6LfqnKD7gk5FLU/t47AOKyrVPpVPx152/WBO5hrj8OJzMnBjgP4AeVXr841Sp6JW556QiBa8TcF4QBK+//24KfCkIwkPrxlTk5JkTlcSGeTdxU0XRafkHrJ08htyibDY2ikMr1jO05lCG1BhSWiz7BcUDolFwI7YhxyW9UBVvI8K7mGb9BzIwYGCFBMcvCVtCjY8W41a3OZWXLHvg8dSYPHbMu4Jv3Uq0HVLaCU596U9WLViDXKyhv/dVjERChY1VEGB/WhUi8yvh6NaaguIgqtSrRPN+VcudKI8nHWfU8VFUt63Ot3stUV24jO/CzzA69uUTfxaCIFB46BAHF88lQS7FxLIfJtZuvPVZEGauYs6knuF40nEupF2gpn1NRtQeQWXryuQfPMTZnw4Q69EReycZnUbUwdxGTnZKEhumTkBmYkKf6T++0pI1er2Ord98Q2LUFSxENena3Buzof24kHaBUymnOJVyioySDACq2VajqWtTmrg2IdAhEIn4xVcOFASBw4mHmR86n6TCJBq5NGJ0ndEPTWzTZGSQOGQImuQUbN9/H+tevTB2c+XMpj85v3U9rQZ/hGdQK3YvDEOt0NLps0Bc/B5SkqeCjllBr2fPojlEnz1JO7c4alr8z8nn/1kM76ues8sjJ+EWcb16UyIoWf65D+Pf+h4vkR+7F12jMEdJ+w9q4F7Nhh3zr5KdXES74VWZcWciLRdfoHaCGN/tO5D5+r4w+14EVw4kcG77HQK9irFfPb605NMHTRHvH82tbDPuFNmSWGJFkaY050QjA3f/mvgHN8TNvwYOHl4PeLYFQUCTnIwyIhLl8S0oLxxDmS1GpzYiy1nORUdXWtXyI2j0D4j/lfwaH57F3p/Dca9mQ8fPAu+rdlMRqBQadq49T+YVDdmmqZyssp4GNWrRRh1M9ObdFOVkY+7thqSJL2l2CuILEogriCub+wDEIjEuZi54WXnhZemFt5V32a29iX3Z+TazJJOVN1ayOXozekHPYKOmtPvlGuTl4zRtGtbvdEOj1rBr7mLiw44gMnLBM/g9mvSqRSUvy4cNoQyNWkfEyRQu/XWRwvSdCPpc/Oq3563PPsRIEEj/9jvyNm/GpE4dXOfOQZpxEnYP565eTbjMGKkAxkZS5I1GIKv6FnIjOTIjGXJJ6a3MSIZULH2oftBeWc+d9dOJzLEmV22CVKxDKgaJUxWkdp5IZXIkMhlSY9n9tzI5EokEsVIJRSWIiwohLx9iwyDmEpTocXQvxMav5IF5MU+Zx8bojayLWkeOMgd/W38GBQyirVfbBxPPKlprPScVnbR2ChgmCEK0SCSaBpgJgjDuYdtX1OQp6PXs+uhXUkWe9P7Mh1OxR7m9YTfHamXiXa8eE0Im4Gbxr/iuF5jxHRbVmuMSL3SaGP5qnEWTOq35usHXSI2ePQtREATeX9ORibPiqTRxIrbvD7jvcWWRho3fXkQsEdN7YkhZZYKjq5dzdd9u+gak4aK/U+Fj1Vm4sS2jAclJmdRo9TExoTKsHE1pNywAB/cHPXZHE48y5vgYWqh9+HDuTew++ADHNq5P9FmoExK4O/NbkkMvcs7PDSNZHez92iG0T+RU3jEupV9Cq9diLbMmxCmEc6nnKNGW0M23G58GfYpFdApXJizkhlcvjC1MeevTYJx9rUm7Hc3m6ZOwruREr2mzHtt2+kUg6PXs/OZr7kSFIZfUwamFmH0eSYRmhKLVazGTmtHIpRFNXZvS2LUxjqavrhSTRqdhQ/QGll1bRqG6kG6+3fi81ufl2qTNyeHu1KkUHjkKgoB5s2ZY9e7FkUuniL92hXcnz8TayY/dC8MoyFLSblgAPsEO5b/wcx6zgiBwdNUywg7soWm/QdTz1huqNLyiObs8TkXtR/3hOKzytYTN6MWAThMpuqth98IwtBo9nT4NxNm3tNB8SYGarbMvo1Hp6DIukPnnJtJl2lH0zvbU33HkARH3uhJ3PYu9P1/H00GFz6bRWDRvjuuihYiNjbn8y1ROHAnFWKIhxU7NHTsVISFtGd72S+TSpww1uL4J4fA36LNTEdu5sjauFlojUwbNXVpu/fjIM6kcWxtF1QZOtB7oX2HVDFJj8jiyJpLCbCW12nng1FzK+pg/ORC+i7eO2lJkpuNitRzS7EpLY1oYW+Bt6f2AsHW3dH+qTm93i+/ya/ivbI3ZimWxwPQD1jjeTMemXz8qfTkBkbExkSePc3D5IvR6CVLTjvjWq039Lj7YOj8YAqZWagk/nkzY4QSKMs+iVZ3DxMKKTl+MwTMwGFVcHCkjR6GKjsbugw9wGDEcFVqOJh5lZ9hyzhXcQXjC91QsEpeJ33+LYZmRDFlJLvLs21gri3GXmOER0BP3Gr1xt3B/qvC1Mh4yzyYVJrE2ci07bu9AoVXQxLUJgwIGUc+p3qO/HxWttZ6Diha8wZSWuDEGYoHBgiDkPmz7ipo8Yxf/zr5wF3xcCznW6DQWv0egNjei9ZfjaOHR4rn3/7QIWi3nPpzEueJbGMmd+bPpVQLcqjK/xXxs5M9WVDomN4a533djxC493tu2Iq9e/Z/XEwT2/hxOYkQ2PcbXwdGz9Mo07XY06yaPJbhdR1oP+aRCxlYeqpJiNkydQEFmOq2GTObSnkKUxVqa9PQloJnrAwfDkYQjjD0xlkn7TKkRVYLv4cNI7Oweun+9Ukn2LyvI/vVXdMZyjvpXRa1RU+DXjF3eK9EaqfG28qaFewtauLUgyCEII7ERucpcfrn+CxuiN2AkMqK/f38GmLUkceS3XHXqgcrUgWZ9qxLQ1JX461fZPusbnP2q0mPSdKTGL69tZrG6mO3jh5Oelo7EuC6n/E8T7ZqOr7UvTd2a0tS1KcGOwS+lbMvTkK/K55frv7Auah1SsZRBAYMYFDCo3FAKTVoaeZs3k7d5C9rMTARXZ854OKARi+n/wwKkcmv+WnydzIQCWvT/Oxyngjm7eR3ntqyjbufuNH9IwufT8B8RvK9kzv5f8lX5zDk3i5rf78Q/WUA8byoB7fuQejuPvUuvI5GK6Tw8GDvX+y9Ec9KK2fZjKKZWMrqNCWbjqi9osOQU17r60+P7ja/d8fJvslOK2Do7FAtjJTX3jMOqaaMysXvz9HH2LpqDtLorq70vYi23YWaTmTRyqZiqMjEXz7Jr7nd0+HQUAc1bl7vNvVrAtTt40rDb8+U4aDU6LuyKI+xwIpZ2ctoMql528QKwc+Esbp8/C0Pq4eXhXyZwbeW2FVo6LK0ojRXhK9gZvY2+x3R0uqBFElwTr4WLkTo6kpWUwM6535F3Nw2ZeVNEktpUa+RCvbe9sbCVoyzWEH48mWtHklAUZiEWDqMsTKRqo2a0GfopcnNz8vfs4e6UrxEZG+P8wyziqtuw4/YO9sftp1BTiLOZM10qd6Gle0sAlDolKp0KlVaFSqcq/VurKvu/Uvv34/d+H/JYjjKHHGXOfeO1ldviYeGBu4U77pbuuFu4l/1tLbN+ovc2PDOc1RGrOZx4GLFITCfvTgwMGIifjV+FfS4viwoVvE9LRUyeihsR7J62n2y7ANY3+AHvOD2B0Rb0mPYtXv5BFWTp06MrLGTH4I+JN8rHxLEzfwavw9LalMWtF1PZ+uknjyVhS9D9sJR2MSZUvXAB0f9cld9rE9mkp19Z+1adVsufX41EUVTIoLk/I6vA9n/lUZidxbrJYwDo/uX3nN+VQWJEDpVrO9JyQLUHyk8dSjjE3G1jmPuLBqv+7+E2aXL5+z1+nLszZ6JNTuFOg2qEy6pjnHeNJA838lopaenRkuZuzfGy8nqobcmFySwJW8Ke2D1YGFvwmVtfas89yVVZc3Js/KnR3JUmvfy4ffEMfy2YTeU69egyeuIL7ZwXnx/PqZRTnIk/TtXfb6EXrDCS1SGnuYiaTQNo6toUJzOnx+/oNSCpIImfrvzEwYSDOJg48Hmtz+lauWu5iW2CRkPh0WPkbdxAeuhlzvq5YSE3ocfHIzGp04D9KyJIisyh4TuVqdXOo8JOdmEH9nDkt58JaN6G9p+MqJD9/hcE79NS0YL3WOIxpp/7hr6bM2l6Q4/jdzOx696DuGuZHPg1AgtbOZ2HB2FpV75HMyU6l10Lw3Dxs6bTZ4Gc/rgHduei2TK2LhMGLK/QOPaKRFGoZvOsy2gLi6l9fAq2DYNwW7QIsbEx587u4cyiZWTaqthfJ42W3q2Z1nDaU7dRfRSCIPDHlyNRK0oYPH9ZuXOdIAgcXxdN5KlUmvauQmDLZ6uCkZlYyOHVkeSkFhPQ1IVGPXzvC3lLvRXF+iljqdetJ037DnzmMT0NKUUprLi+gvSdW/lorxbBzASn+XNxbtgSVUkJB5ct4NaFM9i4BqJUNkMkluEdaE9iZA5qhRbbSglkxu3FSGJE66Gf4N+kBXqVivTvvydvw0YkQTW48EljNucdIzY/FrmRnDaebejm240Qp5AX1o2uSF1EUmESiYWJJBUmlf0mFiSSXpJ+37YWUgvcLNzwsPQoE8JuFm54WHhgb2LPyeSTrI5YzZWMK1hILehVtRf9/Pu90tXF5+WNFrz6khIu9BzKFfcPuFZpH6Z1i/DenIFPcF06j/6qAi19NhRxcfw2bgQqqQVW3oPYVO0nCiQ5/Nj8x0cm/PwbQRDourMr4+Yn41ktBI/ly8sey0wsZMvsy3j429Lx08CyE/nFnVs4tW41XcZOwi/k2dopPy0Z8bFsnDYBK4dK9Jr6A5Fnsji/MxYLWxntP6hR5nm+x4H4A8SOH02TSAHP/XuxdPMqeyz9TjgJ06diceEmKfYi1rV0oWruQCRZ25E5V+K9777DxuTpvOVROVH8dOUnzqScwcvIkam7TEkrqkGiR1tc/Kxo/0FNbp07VCqMWrSh/ccVI4zuEZcfx/74/RyIO8Cd/DtYlAh8eMCObJkFRrJgmg4YTJ22r0+1iKclLCOMHy//yPXM6/jZ+DG2zlgauT7cI6WKi+PGL8s4HhuBa04hdY1MsezVm6uqIG5fyyW4jTuNuvsiEj/fZxB15gR7Fs2p8AsZg+B9dvKUeXx/8Xv2xu3lk0s2tDycif3wL3D49FMiz6Ry/I8oHDwsePuLIEzMHx2eEHUujSNrbuLfyJlmXZyJ7NiOu+Ii/hgXxML2P1eoUKwIdFo9uxaEcfdOLrUv/YhzbU+cFszj+N0z7DizFo892RSZ6hD6BNMrqB+B9oEVOg/d407oBXbMnkH7j0dQo2XbcrfR6/TsW36D+PAsOnxQg8q1n1zs6HV6rhxM5NJfccjNpbQa4I9njftX8gS9nvVTxlGQlcGQn5a/sJq4DyOpIImNe2dTZ8Fh7PMhakAjWo2cja3cltA9Ozj55yos7Z1wrtaXpCgRblXllOTuJ+nGZdwDAunw6Ugs7R1Rx8eTNGIk6uhoQtt4MK9OGhqxQLBDMN18u9HOqx0Wxo9Iyn0JqHQqUgpTysRwYkEiSUVJJBUkkVqUilb4p8KPkcgInaDD2cyZAdUH0N2v+7OFR7xmvLGCN704nXMj3keR/zYZDt7UHGWO9lg4EcePMGje0tci4x4gastG9mxei1TeECuPZhyouYII5TXGh4ynX7V+TzSRxeTGMGj9O/y6UIfDmNHYf/ABUBpHtOnbS2g1enpPDik7MeTdTWPNuM/xCqpN17GTXuj4/k1paMA03KrXpPuX08hIKObgrzcoKVDTqIcvgS3d7hvzofPrcBoyg/CGlQiavYTzCadQ/bGZRgdTEURwoJUVRg0HYH7WH23RfrSqGAbNXfJcn++FtAvMC51HdEYEXx40o1JqVaKqD8DUxpSOnwQSc2EX57asJ6RLD5q9N/i53o+kwiQOxB9gf9x+onOjESGidqXadNUHIl28j2gLGUbGNaj11kCa93vxbalfNIIgcDDhIPND55NSlEJj18aMrjMaH6vSEnpltZ3//gG4sHEdl3ZupSbGuF+7CTI5sc2+IF7jhU+IHQ37ed4neu/FsT0J8deusP2H6bhUqUb3id9UaKiKQfA+G4cTDjPj/AwKVAV8ndOEassOY9WjO04zZnD1YCLnd8TiXt2WDh/WeOIqARd2x3J5Tzz1u/pQzSyRpA8/4q8GEk5182Z5m+U4mzs/l80VhSAIHP8jisgzaVSPXIW7nxFHPqrDpvgdKLJyePu8C3K5Gb2++R435xd78SsIAn9OHI2yqIDB85dj9JDuWRq1jl0/XSUzsYguI4Jx8Xv8BUReegmHV0eSHleAbx1Hmvetitz8wRCTyJNH2bdk3iNDK14G8ck3iBn1OW7h6ZwKkqAcOZD3aw2jKDaZvxb8gEpRQt23u3P98D5UxUU06fM+dTp1A5GIyE2/ovt+ESqRlsWdRCQHOdGlche6Vu76NgwN+wAAIABJREFUyJXH1wmtXktacVqpR7ggiZSiFPzt/Gnr2falJES/LN44wavRa/gz8k8ubVhI/0MuXKk1mpBunvhUN+L38cOp1eFtWg76sMJtfR62jPmMpMQE5LaDMXd05mr9HRzK3kevKr34sv6Xj401WxK2hKubljFmmxbP9eswrVWrNGN+VSQxl9LpNrpWWXa7IAhs/e5r0mKiGDTvZyxsH2wV+aK5cfwwB37+qWz5WFWi5ciam8Rfz8I7yJ5W7/sjN/tnzBfGDMV031mWvyXmnXN6XHIgs74vDhPGU5LozvmdsVjaZJJxZy0N3+1Ho579nttGvaDnYMJBFl1eQLPdCbSIcCO8zhfojC1oOaAaieHbuXZwD836DyGkc/en2vfd4rtlIvdGdmnplSCHIDp4tqeVwgvJiYuE7t5OZCVrjIyr4V23H50/D350u8c3DLVOzfqo9Sy/vpxCdeGjNxag5RUH3DNMuOaXTsjtEppECKQ7dyDOuzPF4htsDVpFibzUAyEVS+nu151hNYc9MuwjLSaaTTMmYuPkQu9ps5CZVqyHwiB4n44cZQ7fXfiOA/EH8Lf1Z7q0B8LYmZjVq4fbzz9zdmcC144m4RdSidYD/TGSPPnxcG8+vHUxnbZDq2Ox+2dyN29m1kBzUipbsLzNcnxtXn31hnvhZ54J+zE2Pc/kTgUojHQ0s22A/34FIqWOPtNnY+fq/lLsib16ie2zvnlsq3VlkYZtc0IpKVDzztja2LmUn9grCAI3TqRwdtttjCRimvetil9I+V0a1UoFq0Z+hLmtHf1mzn3lNZQFvZ6YuTPRrVzPHScRS3qZ8VaDAfR07cKJpUtJiYrE3sOLjp+PQeRowd6onegW/kqDc3ncchUT9nlL2tTrSwPnBo+sU27g1fFGCd5rmdeYemYquUm3mb9KxM3g8ajtvek/syG7504n9VYUQxeuwMTi8eVEXib5GXf5bfiH2BbqUHqOwszOjMw2ofwWv5z6zvWZ23wuVjKrcp97L5yh575CQi7kUfXiBUTGxtw8m8bR329Sr7M3IZ28y7aPPHWMfYvn0nrIJwS3f2h1oRfOvQShhu/2pVHP9xAEgWtHkji3/Q5mVjLaDQvAyad0zJr0DGLatkGk1iB2d8X162nIGzTk+B/RRF+4S+XatqRELAUEBv64BEkFZmBr9Bq23tpK9LK5dDki4mrdT1DIPKnV1p3sxK3cOn/6ibwPmSWZHEw4yIH4A1xNv4pcLaaG3I8GFrWplmOO5sYt8mPvUKJSojSWUGAiQyL3w96rJ+9OCEFm+non2Dwreco8dt7ZSYm2BNHfP1DazESE6B9vv0pL4aoTCCUqLIe2wlgmxf7UTTgrJs6uI5ZFCbjanyGnQw1uWBWw885OAN7xfYdhNYfhYn6/xz87OZENUycgNzOnz/TZmFk/W7LoozAI3idDEAQOJBzgu/PfUagp5JOgT+gnbUxK/4FI3dxwW/M7x7cmEXMpncBWbjR51++Zwlh0Gj07F1wlPb6Azh/5oxw1AI1ew+jBkC9Rs6T1Emo51nrq/VYU0ddSOPxzFPZZ19GW/Movfa3oXK073T27cnb+UrKTE+k5ZSYuVV7eSo8gCKyfPJaivByGLvgFI8nD56GCbAVbZ4ciFovoPq4OFrb3r7IU5So5ujaKpMgcPKrb0nKAP+Y2D19ROb1hLRe2b6TvjB9f6pgfR+HRYySPG4tCpGV2Zy0Jfpa8V7UfTbUBZNhp2JXwF9E3TjJimxqfdMjs1ojAybOxNn944rWB14M3SvCGZYQx8cSXfL9FRlGGGWFVh9G8bxXMbTLZ+u2UZ/LGvSzObPyD89s2UCNNT0LQWEys5Jh2z2JmxFTczN1Y1GpRucsfMbkxdN/VnTUb7bFz9MJz9Spy7xaz6btLVPKypMvIWoj/PjmUFOSzevQnWDu70Peb2a/0ilkQBA78vICIE4fvixFLjyvgwK83KM5V0aBbZYLbuCMSi8jftQttZhY2/d9DoYR9y8JJjyugXmdvtMoLnN30Bz0mTscrqPYLsbdEU8LelZPwXXyIG/69yLdrQqUqcvSFe0iOCqfr2Mn41KqLoqiQopxsinNzyMhM5nrcZeJToynKzcZEaYSlRoZMKQL9g8eLTCRGbmGNib0zhXlWSEwa0fOr+o/vMvb/hJzUFNZNGo2VoxN9pv+AVCZHEARubrvIiUOFmBTfJfjaYqwDfTH6YihrhDNsi9mGgEDXyl0ZVnMYbhZuFGRlsP7r8Qg6HX2m/4h1pReT/GcQvI8nR5nDzPMzOZRwiAC7AGY0noGXyoL43n0AcP19HUd2ZpAYmUODbj7Ubu/5XPGqyiINW2ZfRlWspVNHGbmfD0LavROj6kaRVpzGnOZzaOHe4pn3/yzE5Maw+eJOzLf7Y1mUjXXRL/DtcDpU7YIUCTtmTychPIyuYydTuU69l2obQHxYKFu/n0qbYZ8S1LbjI7fNSi5k+5wrmNvK6T62NjJTKYIgEHMpnZMbbqHT6mn8rh8BTV0e+TnmZ6SzavTH+NVrRKfhD62C98pQxcWR/MUXqGPjON/Nj3lVbpd1G217x5xBu4uRSmW4zZqNRauWr9haA0/KGyV4ATKWLyNr/kLCOv+E1ticvlNDWD9pFCqFgsHzfq5Q719FolEpWT3qE8jKomaamPBaI5FZyPAdIOXLsDFoBS3zWsyjgXOD+563JGwJay8uZ9V8LfaffYbNRx+zZVYoxfkq+kyuh5n1P1fQ+5fO5+bp4wyYtQB7D6+XPMIH0Wm1bP/hG5IirvPOhKllYlVVouHo2ihir2biWdOONgOrl8V3ZSYVsnfpdZTFGtoMqo6dq541Yz7Fp049Oo/68oXbnHbqMBkjRhNfqT6Jnr3AogRz4TC5KQmIRGL0Ou0Dz9EYg7GlObZyW+RFOoSsQlCKESTWiOy80dt4oRbboSjSc+8wEktEdB1R64ni4f4/EXvlEttnT6dao2Z0/GJs2UkzKSqHfUuvI0VNcMRS5Km3sO7ZE+GjfqxK2sLWmK0IgkAX5w64/pWBuqCI3tNm4eDp/ZhXfHYMgvfxZCmy6L27N/38+5U24ClRkvBefzTJyVT6dQ2HDigqvBRdXkYJW38IRWYmoZn8LMW//4r10nmMLvmdmzk3mdpwKu/4vVMhr/UwNHoNRxKPsCFqA+EpkfS/PBozlQl1dLup9csSxDIZgiCwf+l8Ik8epd1Hw6nZqt0LtelhCILAhq/HU5CdydAFK5BIH73alByVw+5F13DysaLtkABOb47hzpUMnHwsaT2oOtaOj0882z1/FrFXLzFk/nIs7F5+2N2ToCsqJm3SJAoPHIDWjTnSqzKN9iZisvM48qBA3ObNe6Ddu4HXmzdK8CrCw4nv24/CVgO4pK5L60H+aJURHPj5JzoNH0e1xs0r3MaKJObCWXbN+44aBSrsBEeuVvsAY1MpDT5y4cuw0cTlxzGx/kR6VS0tynwvnCEk1oh3f4nCY/VqLsdZc+NECp0+C8Sr5j8TRUJ4GFtmTqb+O71p0mfAw0x46ahKStg4dTz5men0nvYDjl5/Jy/9Het1eksMphbGtB0agKJQzeFVkcjNpHT8JBB7d3O2//ANyTcjGDz/5cUjK2/dIn7YUDIEO0JrDkYQtIgcTpMvzuOusgBEZlhKXHAzqYqtwgltroBCLUH/r+B+I4kIc1s5FrZyzG1kmNuU3lrYyrF1MX/kct//Z85v28iZjWtpPmAodd/+R5hkJBTw1+JrqBVa5CgQ56ZjLKgw9/NA4utMRP4VSkL3IleoUDetzdvN++Pp6IbcTIrMRPLc1R7+jUHwPhlKrRK5RI6g0ZD08ScUnz+PzdxlHDknoTBHSbuhj2g2Ug4apRKp/NFJi2m389j5UxiOnubUPD4DivJx2rqBsVencjb1LCNqj2BojaEVWv1AoVWQUpjCgYQDbLm1hSxFFm6mbrx7fhCqQhsa6o8RtGQaYlnpcX9q3Wou7txC4179adCjT4XZ8SwkXA9jy7eTaTXkY2q1f/ux28dcSufgyojS1UUR1OvsTa22Hk+Uh5AUGc6mb76iUc/3aPhu34ow/4UhCAI5K1eSMW8+IokEQa3GdtAgHEePQvSaOtcMPJwnnbNfizQ9QaNFHhjEFasm2BiJ8Q6yZvWo33HyrULVRs1etXmPxbdeQzxqBhMTE41T5C1C7m7isksfzi9LZcnnK5h582tmnJ9BXH4cY+qOIS4/rvR+Vn2QSkkTuXLjxC2C27jfJ3Y1ahWHVyzBxtmFBt17v8IRPojM1JR3vprGuslj2T5rGn1nzsXS3gGRSETNFm44+Vixf8UNdsy7iqAXqORtyVsf18TMSkbMxbPEXb1Mi/eHvdTkO3mVKlTeuAnjDz/E/MJczjT+BHFGZxyAe6dlEQIyTQGUJGOmzsPJzhSbqp7Y1auBlbsd5rYy5GYPbwlp4OHUf6cXGXF3OPnHKhw8vPEMDAbA0dOS7uPqcON4CooiNSUZVhTdSSYjvhBNqg5xSSimWjVSs66YhlfmdHgyp/m7dbAIZKYS5KZS5OZSZKZS5Oalf1vYyQlu4/EKR/zfRi4pDU1JmzaN4jNnMPnqO/YfF6FRqeky/Mmy/gW9ntirl7m0ayspURHYOLvgHhCIR40g3KvXxNTq/n04+1rTeqA/B1dGYNJqPJ6/fUTB7Hks/n4xU85OYcGVBWQpshgfMv6Ja6IWqgtJLUot/S0uvU0rTiv7X66qtF+HCBFNXJvQp1ofVGuKuFUsJ0h//j6xe2XfLi7u3EJQ247Ufw3mbI+aQbhWC+Di9k3UbNnusSulfiGVUJVoiLmcQZNefuV21SwPvV7HsTUrsLB3oG7nF+tlrwhEIhF2w4Yhr16dzEWLsRs2FIvWr66ahIGXw2vh4QW4eTaVo79H0eGjGmTGHePMxrX0njYLN/8aFW7fiyA7OYnfx39OFT9/fDbtQt+uJ+clrZEYG9F5RBC/Ji1lbeRaGrs2xtvSm3VR69jyVxU0EhvOOvXHytGU7mNr35fBfGr9Gi7u2EzPKd/hUSPwFY7u4WQmxrPh6/FY2jvQZ/rs+zLm1Qotp7fEIDYS06SnLxKpEWqlgtWjP0VuZkb/WQteaBOIh6HLzyf5s88pvHKN3HeHI0dAFHoao+RbyHTFmDdqgGX7Dli0boWRtSE0oSJRK0pYN3ksxXm59P9+PlaO5cfhCoJA3s6d7Fu5lDRTY+o6VabG51+RqVWxJ3I/FxNCkahlBJgHUtMiGJnGBGWJFmWRBlWJBmWRBjMbOf2m1n9qGw0e3icn6+efyVywEAaN4UxGFYykYrqU0z3t3+i0Gm6ePsHl3dvITk7Ewt6Bao2bk52UQPLNG6gVCgDs3T1xDwjEvUYg7v41kZuX7vfyvngu7IylukMmTpun4bpwAeZt2zDn8hzWRq7lLa+3mNlkJlKxlFxVLmlFaaQUpdwnZFOLU0krSqNQc3+1EZmRDGczZ1zNXXE2d8bFzAUXcxeCHIJws3Dj0sI9XIw0wUd3k/aLhiL+2ysdfe4Ufy2YjV9IQ94eNQHxa5LRn3jjOptnTKTloA+p/VaXF/Ia14/s59Avi+k0YjzV3gAnlYH/Fm9USINOo+fPqeeRm0vp+Ellfhv5EZ41g196rdnn5fjalYTu2cHb9Zqj/2UlkiGjOJnhj5GRiG6ja3M4fy/fnv8WraCloW1tRkwMI7z1TAqxpNekevclOWUmxPHHVyPxb9qSDp+MfIWjejwJ4WFs+34qbv416P7VtEdmBJ/44zcu795Gn+k/4lr11WXw6lUqUsdPKI3jkkgwa9TQIHJfErl3U/lz4igs7R3pO/3HcpexBUHgyMqlXDu0j9rOnjgdPI6RlRWOY0Zj1b072aoc1kSsYWP0RpRaJR28O/Bx4Mf4WPuU7UOvF8oSP58Gg+B9MvJ37SJ1/ARKOg7lkqYu5jYyugwPxtL+4cmaqpISwo/sJ3TvTopysnHw9CakSw+qNGhSVi9Wr9ORHnubxIjrJEVcJyUqEq1aBSIRjl4+eNQIwq16TWLDJNy6mEtgwWEqxR3HZ/cujOzsWB2xmnmh87CT21GiLUGhVdxng5nUDBdzF1zMXMoVto9qexu9cjdHLsixJ53uP/VAYlo61sQb19n2/dc4+Vbl3UkzXruck03ffEVOajJDF/1a4e3VVSXFrBzxIbYurvSe9oNh9cvAS+eNErwRp1I4/mc0nYcHcevsBsKPHmTgnKXYurxZgeOqkhJWjfoICzsHWiCnYNt2TL/+kSNXrRCJRXQbVYsYIYIpZ6YwUdKZwuVJJHi2p92wAPzq/lPTUK/XsWHKePIy7jJ43s+vXTm28og4cYT9S+dTvVkrOnw6qtxJLysxnrVfjqB6s9a0/3j4K7DyfgSdDsW168h8vA0i9yUTFxbKtlnTqNqgCZ1GjH/g+3Jm0x+c37qBkK7v0qzfIJRRUdydPgPFlSuYBAVR6espmAQEkK3IZk3kGjZEbUCpVdLeqz0fBX70XLVZDYL38Shv3SKux7tkh/TkurwxDu4WvP15ECYW5Qu9otwcru7bxbVD+1CVFONRI5CQzj3wDKr9WIGk1Wi4ezuapIhwEiOukXYrCp1Wi0gsxtjUFZ3OGb+kq/hXc8Zr8RJEIhH74/ZzKOEQlcwqlQpb87+FrZkzlsaWzyTKkv7cyb6jImQSHb1mtcXEptTbXNqJ8kss7Ozp883sMi/060Ry5A02fvMlLd4fVtpYoQK55+jp/918Kvm8+prIBv7/8UYJXq1GR9y1LGwqKfl9/BcEte1I6yEfV7hdL4N7HWbaffAZlmvWowgLw2L+bxw4oABRqei1dTYj/PtVnEzwxL++A60G17xvH1f37+boquV0/GIs/k1avJqBPAPntq7n7KY/adCjD4179b/vMUEQ2DjtS7JTkhgyf9kbIeINvFgu7NjM6fVraPbeYEK69Cj7/5V9uzi2+hdqtGxHu4++KBMngiCQv3MnGT/OQZebi02fPjiMGI6RlRW5ylx+j/yddTfXUaItoa1nWz4O+pgqNlWe2i6D4H08er2esz/s5FqCFW7VbHjr45rldk/LSU3m8u5tRJ48il6nx69BY0I6d8epst8z26pRq0iNvklSRDgJ18O4eycG0CPSC1Sq5Ix381Z4BATi5Ff1sZUJHocgCKjv3CHn4DEOnDNBbWrHu5MbYOteWv85PyOd9V+PQyQW02/GnNe2MgHA5hmTyEpKYNjCXx+bHPik5KSmsGbsZ1Rv1uq1cGIY+P/JGyV473Evc3/owhWYWpbfsOF1514pmNy7qQyc9gNpQ4aiLyzCaula9q5LRRAE2g0NYN/8cxjrSui3tDtS439ivQqyMlk95lNcq/rT/atv3qjlIUEQOLh8ETeOHXygHM89D/CrLNNj4PVCEAT++ukHYi6cpftX0/AKqs3N08fZu2gOviEN6Dzqq3JjvHUFBWQuXETuunUYWVvjOHYsVt26IhKLyVPmlQrfqHWYSc040OPAU7fQNAjex5OdUsTGmRepXMeRNoOqP9A9LfXWTS7t2srtyxeQSKQEtGxL3U7dsHaq+FbAWck5bJ21C03+LWSFVymQS0AQMDYxoXLdBlRt2ATPwNpPJH4FtRplZCQloVfIuXyDu7H55EoqkW0bgMK0Em9/WgOPoNK485KCfDZMnUBJfi59vpmNvbtnhY+tIkmJimTD1PEPXGA+D6Xn7BsM+emXF9IExoCBJ+GNE7yJN66xecYkmvYbRL2u71a4TS+T9Njb/DFxFLU7dKZRs7bE9+mDtJIjVgt+Y9fyaEry1Yh1alq7RVFl6j/xuYIgsHPOTBKuhzFo7pKHJvS8zui02rKC690nTMUruA6KokJWjfoYGycX+nzzwytvNWng9UGtVLB+yjiKsrNo0vd9jq5ajmvV6nT/6pvHxkEqb97k7jfTUYSFYVKrFk5fT0HuXxoXnq/KJ74gniCHoKe2ySB4n4y023k4+ViVlYUrrbhw6e+KC5HIzS0Ibv82tTq8/cIdGOlxBWyfG4pZXjz1pKcRfzKUuKuXibl4DmVRIcYmpviGNKBqw6Z4BgaX5RnoiopRhIVRHBpKxpVY0u+qyTPxIN/KB6VJqbdWLBZwcJET1MG3LPRMo1SyecYkMhPi6DF5Bm7VAl7o+CqKLd9OISPuDsMWr8RY/nyNce41tqhIAW3AwLPwRgleQa/nj69GoSgqYPD8ZRUeVP8qOPzrEq4fOcD7PyzEJCWNxA8+xKxRQyymzeXgz1dwPLGSOlMGYNm2bdlz7tXzfZ07yz0JakUJG6Z9Sd7dNHpPm8X1w/sIP3qQAbMWvNCGAQbeTPLupvHnxFEoi4tw9KpMr6nfIzN9fJF7KJ078nfsJGPOHHR5edj064fD8C8wsnz2kBmD4H06yqu4UPftd6jRsu1zi6qnIfZqJvuWX8chM4zW3Z2xe68fOq2WpBvXiD5/mpiLZ1EVF2NsLMPFzBqzXDHaAksKLHwosPRCJym11UQm4FTZCpfqjjj5WOHgboGR9J+LdL1Ox845M4m7GkrnMV/hF9LwpY3xeUm9FcX6KWNp0ncg9bv1fOb96LRafh//BXqtloFzlz536IgBA8/DG1WH9+aZE2TE36Hj52P+E2IXoHHvAUSfO83RVcvp+fV3OH09hbtfT8V41SJaeziQmXEJ0zo/lW2vKinmyKplOHj5UKdj11do+fNjbGJK9wlTWTd5LFu/nYKiqJA6HbsYxK6BcrF2cqbLmImEHdhDqyEfP7HYBRCJxVh3fweL1q3IXLCQ3HXrKNi3D8dxY7Hq2vWNCgl601CVlHD9yH6u/E/FhY5fjL2v4sLLxKeWA416+HJ2q4hzG4/RrnE8AFa34/GOK0SW70KKDnJFGuJzUwE1WOVgai7Fxd+ZKg2DcK1ih4Wd/KHfG0EQOLRiCbFXLtFm2KdvlNgFcKlSDe/gOlzevY3gdp2e6lj7X64d2kdOShJdx00xiF0DbwyvhYdXUVhA+NGDhHTu/p9a7g47uJcjK5fy9sgJVG3YlPRZP5CzejVGNjYY2dpSec9fZdse/nUp1w/vp9+3c58roeN1IispgQ1fj0cqkzF4/jKMTZ5tcjVg4ElRRESQPn0GimvXMK1bF4/fVj515ySDh/fxZKcksW7SGNSKEjxqBBHSpQeegbVe+QWGIAgc/y2MyEu5uKSdQS0xI9/KB41xqcdfaqTH0UWGk78DInEKmQlXibt6EbWiBLm5BX71GlKlYVM8AgLLjR+/Vz2kQY++NO713sseXoVw9/Yt/pw0msa9BzxTQyNFYQG/jfgQRx9f3p0045V/5gYMvFEeXhMLyzc+brc8Atu05/qR/RxfuxKfWiE4jhuLOj6eouPHsWj3T+JWSlQk1w7tpU6nrv8ZsQulReP7z1qASIRB7Bp4KZgEBOC5fh3527ejjo83tAl9Qdg6u1KjZVuqN235WpWiEolENB8UREHKCZJpjLmJDi8fS1wCXXH2tcbG2ex/ajNXA1qjVauJv36VW+dOEXX2FOFHD2JiYYlfvUZUadgE9+o1ERsZEXZwL+e3bqBmq3Y06tnvVQ7zuXDyrYJPnXpc/msbtTq8fV+zoCfh7OY/USlKaPn+MIPYNfBG8Vp4eP/L3MuMrf9OL5r0eR9dUTHpM2di814/TGrWRKfVsHbCCNRKBYPmLn2pMW8GDBh4EIOH981Hr9OjVuqQmz3dcrtGrSL+2hVunTvNncsX0KiUmFha4VEjiOhzp/CpHULXMZNeSYfIiiQ99jZ/fDWSRj3fo+G7fZ/4eVmJ8fw+fjhB7d6i9ZBPXqCFBgw8OW+Uh/e/jGu16lRv2pLLu7cR0KINNk4uuMz6vuzxSzu3kp2cyDsTphrErgEDBgxUAGIjMXKzpw+PkxrL8AtpiF9Iw1LxezWU6HOnuBN6AZcq/rw9YvwbL3YBKvn44hvSgNA9O6jVofMTNcsQBIFjv/+KzNSURj3fzHAOA/+/+e8EzL7GNH1vMEZSKcfXrLjv/zmpyZzftoEqDZviUzvkFVlnwIABAwb+jdRYhl/9Rrw9cgKfrdxA76nfI5VVTMOG14GG7/ZDVVJM6N4dT7T9ndCLJIaH0bDne4bGQQbeSAyC9yVgbmNLgx59ib1yidgrl4B72b6LkRjLaDXow1dsoQEDBgwYeBgSqfQ/4dn9Xxy9fPCr34gre3eiKCp85LZajYYTv/+KnZsHQW3fekkWGjBQsRgE70ui9ludsXVx49jqX9Cq1dw4fojkyBs06z/Y0KHGgAEDBgy8dBq92w+1UknoX9sfud3VfbvIS0+jxfvDXknJOQMGKgKD4H1JGEmktBz8EXnpaZxav4aTa3/DtVoANVsa2uwaMGDAgIGXj72HF1UaNPm/9u4nxq6yDgPw+5WWtJIyJXVAZKpICCEGS8SR8MdUBDEqwkZDMEE3YmOi6M7EDYlE90hiYogb1BjSBYSUiNEgVVmgnYJtMEJssIZiLGNCC0hbpvVzMUOxLTL3Ts+dnvnmeZJmek/P3Pt+95788ubk9Nw89ejWvP7Kgbfd59/7X86TDz6Qiz5yZS68/IpFTgjdUXgX0YUbP5yLP3p1nvrFw5k5fCg3fvUbTd13GICl5ZovfDEzhw9l6v+c5X3igZ/myBszue5LX1nkZNAtbWuRXfflO7J67dm55tbbs35iw+mOA8Aytn7ifbn0mk15+pdb8/qB/cf9277nd+eZbb/OFZ+9Jeecf8FpSgjdUHgX2di55+VrP/pJk1+0AcDSc9Xnb8vRN2ayfeuDx7bVWvP4/fdlzdqzF/SNbNA3A119XkrZk+TVJEeTHOn0puy7tiSP3Z0c2JuMTSQ33JVsvLWzp++dXVtyxnJab5/1+djrczZ6b6QzO+n38dl1tj6vtSPrL9iQSz/28fzp0Ycz+eIPctbBv+e5mUvy4u5zc+PmO4f+NrZj+v7e9T0fnRrmv1u2eNsaAAAFSklEQVR+otb6r05ffdeWZOs3k5mDs48PvDD7OGnzoFtu6+2zPn8Wfc7GUtL9zE76fXx2na3Pa+3Y1ZeN5dnfH8n2PSXXjpf8bs/ZGV/9ei5b//LCnrDv713f89G503tJw2N3v3WwvWnm4Oz2Fi239fZZnz+LPmeDPh+fXWfr81o7ds7T9+SDY/uy8+Xz89t9H8irR1bn+vP+mhWPf29hT9j3967v+ejcoIW3JvlVKWVHKeVtvyWhlLK5lDJVSpmanp4e7FkP7B1u+1K33NbbZ33+LPqcjaViNDM76ffx2XW2Pq+1awf25qp3v5CjdUV27n9vLlk7nYl3vdLue9f3fHRu0MJ7ba31iiSfSfL1UsqmE3eotd5Xa52stU6Oj48P9qxjE8NtX+qW23r7rM+fRZ+zsVSMZmYn/T4+u87W57V2bWwi6848lA+t+2dWlqPZdO7fjm1f6PMNtX2x9T0fnRuo8NZa/zH386UkDyW5spNXv+GuZNWa47etWjO7vUXLbb191ufPos/ZWBJGNrOTfh+fXWfr81q7NrfWG96zO3dcvD1jZx5u+73rez46N2/hLaWcVUpZ++bfk3wqyTOdvPrGW5Ob703GNiQpsz9vvrfdC8aX23r7rM+fRZ+z0XsjndlJv4/PrrP1ea1dm1vrinUbctbKI+2/d33PR+dKrfWddyjlosyeIUhm7+rw81rr99/pdyYnJ+vU1FQ3CQEWUSllR+e38VpEZjawnAw6s+e9LVmt9fkkl3eSCoCRMrMBTuab1gAAaJrCCwBA0xReAACapvACANA0hRcAgKYpvAAANE3hBQCgaQovAABNU3gBAGiawgsAQNMUXgAAmqbwAgDQNIUXAICmKbwAADRN4QUAoGkKLwAATVN4AQBomsILAEDTFF4AAJqm8AIA0DSFFwCApim8AAA0TeEFAKBpCi8AAE1TeAEAaJrCCwBA0xReAACaNnDhLaWcUUp5upTyyCgDAXDqzGyAtwxzhvdbSf4yqiAAdMrMBpgzUOEtpUwkuSnJj0cbB4BTZWYDHG/QM7z3JPl2kv+MMAsA3TCzAf7HvIW3lPK5JC/VWnfMs9/mUspUKWVqenq6s4AADM7MBjjZIGd4r01ySyllT5IHklxfSvnZiTvVWu+rtU7WWifHx8c7jgnAgMxsgBPMW3hrrd+ptU7UWi9McluS39Rabx95MgCGZmYDnMx9eAEAaNrKYXautW5Lsm0kSQDolJkNMMsZXgAAmqbwAgDQNIUXAICmKbwAADRN4QUAoGkKLwAATVN4AQBomsILAEDTFF4AAJqm8AIA0DSFFwCApim8AAA0TeEFAKBpCi8AAE1TeAEAaJrCCwBA0xReAACapvACANA0hRcAgKYpvAAANE3hBQCgaQovAABNU3gBAGiawgsAQNMUXgAAmqbwAgDQNIUXAICmKbwAADRt3sJbSlldSvljKWVnKeXPpZTvLkYwAIZnZgOcbOUA+xxOcn2t9bVSyqokT5RSHq21PjnibAAMz8wGOMG8hbfWWpO8Nvdw1dyfOspQACyMmQ1wskHO8KaUckaSHUkuTvLDWusf3mafzUk2zz18rZTyXGcpARbP+093gFNlZgPLyEAzu8yeDBhMKWVdkoeS3FlrfWaBwQBYBGY2wKyh7tJQa92fZFuST48kDQCdMbMBZg1yl4bxubMEKaWsSfLJJM+OOhgAwzOzAU42yDW85ye5f+6asBVJttRaHxltLAAWyMwGOMFQ1/ACAMBS45vWAABomsILAEDTFF4AAJqm8AIA0DSFFwCApim8AAA0TeEFAKBp/wXIg39D/1/4LwAAAABJRU5ErkJggg==\n", "text/plain": ["<Figure size 864x288 with 2 Axes>"]}, "metadata": {}, "output_type": "display_data"}], "source": ["import matplotlib.pyplot as plt\n", "fig, ax = plt.subplots(1, 2, figsize=(12, 4))\n", "\n", "n = 15\n", "ax[0].plot(y_train[:n].reset_index(), 'o')\n", "ax[1].plot(y_test[:n].reset_index(), 'o')\n", "ax[0].set_title('Pr\u00e9dictions sur quelques valeurs\\napprentissage')\n", "ax[1].set_title('Pr\u00e9dictions sur quelques valeurs\\ntest')\n", "for x in ax:\n", "    x.set_ylim([3, 9])\n", "    x.get_xaxis().set_visible(False)\n", "\n", "for model in models:\n", "    d = model.get_params()['polynomialfeatures__degree']\n", "    tr = model.predict(X_train_norm[:n])\n", "    te = model.predict(X_test_norm[:n])\n", "    ax[0].plot(tr, label=\"d=%d\" % d)\n", "    ax[1].plot(te, label=\"d=%d\" % d)\n", "ax[0].legend()\n", "ax[1].legend();"]}, {"cell_type": "markdown", "metadata": {}, "source": ["Le mod\u00e8le de degr\u00e9 4 a l'air performant sur la base d'apprentissage mais s'\u00e9gare compl\u00e8tement sur la base de test comme s'il \u00e9tait surpris des valeurs rencontr\u00e9es sur la base de test. On dit que le mod\u00e8le fait du [sur-apprentissage](https://fr.wikipedia.org/wiki/Surapprentissage) ou [overfitting](https://en.wikipedia.org/wiki/Overfitting) en anglais. Le polyn\u00f4me de degr\u00e9 fonctionne mieux que la r\u00e9gression lin\u00e9aire simple. On peut se demander quelles sont les variables crois\u00e9es qui ont un impact sur la performance. On utilise le mod\u00e8le [statsmodels](http://www.statsmodels.org/stable/index.html)."]}, {"cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [], "source": ["poly = PolynomialFeatures(degree=2)\n", "poly_feat_train = poly.fit_transform(X_train_norm)\n", "poly_feat_test = poly.fit_transform(X_test_norm)"]}, {"cell_type": "code", "execution_count": 9, "metadata": {"scrolled": false}, "outputs": [{"data": {"text/html": ["<table class=\"simpletable\">\n", "<tr>\n", "        <td>Model:</td>               <td>OLS</td>         <td>Adj. R-squared:</td>      <td>0.302</td>  \n", "</tr>\n", "<tr>\n", "  <td>Dependent Variable:</td>      <td>quality</td>            <td>AIC:</td>         <td>10821.7528</td>\n", "</tr>\n", "<tr>\n", "         <td>Date:</td>        <td>2018-09-09 14:59</td>        <td>BIC:</td>         <td>11321.5798</td>\n", "</tr>\n", "<tr>\n", "   <td>No. Observations:</td>        <td>4872</td>         <td>Log-Likelihood:</td>     <td>-5333.9</td> \n", "</tr>\n", "<tr>\n", "       <td>Df Model:</td>             <td>76</td>           <td>F-statistic:</td>        <td>28.70</td>  \n", "</tr>\n", "<tr>\n", "     <td>Df Residuals:</td>          <td>4795</td>       <td>Prob (F-statistic):</td>    <td>0.00</td>   \n", "</tr>\n", "<tr>\n", "      <td>R-squared:</td>            <td>0.313</td>            <td>Scale:</td>          <td>0.53135</td> \n", "</tr>\n", "</table>\n", "<table class=\"simpletable\">\n", "<tr>\n", "    <td></td>       <th>Coef.</th>    <th>Std.Err.</th>     <th>t</th>     <th>P>|t|</th>   <th>[0.025</th>      <th>0.975]</th>  \n", "</tr>\n", "<tr>\n", "  <th>const</th> <td>-1062.0749</td>  <td>1873.8933</td> <td>-0.5668</td> <td>0.5709</td> <td>-4735.7657</td>   <td>2611.6159</td>\n", "</tr>\n", "<tr>\n", "  <th>x1</th>      <td>-2.0670</td>    <td>24.3092</td>  <td>-0.0850</td> <td>0.9322</td>  <td>-49.7241</td>     <td>45.5901</td> \n", "</tr>\n", "<tr>\n", "  <th>x2</th>     <td>-729.2270</td>  <td>157.0307</td>  <td>-4.6439</td> <td>0.0000</td> <td>-1037.0792</td>   <td>-421.3748</td>\n", "</tr>\n", "<tr>\n", "  <th>x3</th>      <td>8.7231</td>    <td>200.5623</td>  <td>0.0435</td>  <td>0.9653</td>  <td>-384.4709</td>   <td>401.9171</td> \n", "</tr>\n", "<tr>\n", "  <th>x4</th>      <td>3.4131</td>     <td>13.3672</td>  <td>0.2553</td>  <td>0.7985</td>  <td>-22.7927</td>     <td>29.6188</td> \n", "</tr>\n", "<tr>\n", "  <th>x5</th>    <td>-1171.6645</td>  <td>689.5425</td>  <td>-1.6992</td> <td>0.0893</td> <td>-2523.4842</td>   <td>180.1551</td> \n", "</tr>\n", "<tr>\n", "  <th>x6</th>      <td>40.1316</td>    <td>8.1636</td>   <td>4.9159</td>  <td>0.0000</td>   <td>24.1272</td>     <td>56.1360</td> \n", "</tr>\n", "<tr>\n", "  <th>x7</th>      <td>70.8825</td>    <td>22.2599</td>  <td>3.1843</td>  <td>0.0015</td>   <td>27.2429</td>    <td>114.5221</td> \n", "</tr>\n", "<tr>\n", "  <th>x8</th>     <td>-724.4624</td>  <td>724.6583</td>  <td>-0.9997</td> <td>0.3175</td> <td>-2145.1251</td>   <td>696.2003</td> \n", "</tr>\n", "<tr>\n", "  <th>x9</th>     <td>-251.7727</td>  <td>192.4538</td>  <td>-1.3082</td> <td>0.1909</td>  <td>-629.0704</td>   <td>125.5250</td> \n", "</tr>\n", "<tr>\n", "  <th>x10</th>    <td>-276.1104</td>  <td>163.5189</td>  <td>-1.6886</td> <td>0.0914</td>  <td>-596.6824</td>    <td>44.4616</td> \n", "</tr>\n", "<tr>\n", "  <th>x11</th>    <td>258.8220</td>    <td>24.9389</td>  <td>10.3782</td> <td>0.0000</td>  <td>209.9303</td>    <td>307.7138</td> \n", "</tr>\n", "<tr>\n", "  <th>x12</th>    <td>1021.3730</td>  <td>1866.5461</td> <td>0.5472</td>  <td>0.5843</td> <td>-2637.9138</td>   <td>4680.6598</td>\n", "</tr>\n", "<tr>\n", "  <th>x13</th>    <td>394.9815</td>   <td>155.7356</td>  <td>2.5362</td>  <td>0.0112</td>   <td>89.6682</td>    <td>700.2947</td> \n", "</tr>\n", "<tr>\n", "  <th>x14</th>    <td>250.5746</td>   <td>208.6039</td>  <td>1.2012</td>  <td>0.2297</td>  <td>-158.3848</td>   <td>659.5340</td> \n", "</tr>\n", "<tr>\n", "  <th>x15</th>     <td>-4.4734</td>    <td>21.4718</td>  <td>-0.2083</td> <td>0.8350</td>  <td>-46.5681</td>     <td>37.6213</td> \n", "</tr>\n", "<tr>\n", "  <th>x16</th>    <td>-829.1409</td>  <td>537.0924</td>  <td>-1.5438</td> <td>0.1227</td> <td>-1882.0886</td>   <td>223.8067</td> \n", "</tr>\n", "<tr>\n", "  <th>x17</th>     <td>-7.9049</td>    <td>9.5287</td>   <td>-0.8296</td> <td>0.4068</td>  <td>-26.5856</td>     <td>10.7758</td> \n", "</tr>\n", "<tr>\n", "  <th>x18</th>     <td>5.7595</td>     <td>21.3304</td>  <td>0.2700</td>  <td>0.7872</td>  <td>-36.0579</td>     <td>47.5770</td> \n", "</tr>\n", "<tr>\n", "  <th>x19</th>    <td>375.0969</td>   <td>1075.4182</td> <td>0.3488</td>  <td>0.7273</td> <td>-1733.2162</td>   <td>2483.4100</td>\n", "</tr>\n", "<tr>\n", "  <th>x20</th>    <td>114.3253</td>   <td>261.5978</td>  <td>0.4370</td>  <td>0.6621</td>  <td>-398.5264</td>   <td>627.1770</td> \n", "</tr>\n", "<tr>\n", "  <th>x21</th>    <td>-134.9056</td>  <td>151.9652</td>  <td>-0.8877</td> <td>0.3747</td>  <td>-432.8272</td>   <td>163.0159</td> \n", "</tr>\n", "<tr>\n", "  <th>x22</th>    <td>-72.6985</td>    <td>26.4607</td>  <td>-2.7474</td> <td>0.0060</td>  <td>-124.5735</td>   <td>-20.8234</td> \n", "</tr>\n", "<tr>\n", "  <th>x23</th>    <td>520.0041</td>   <td>1956.5000</td> <td>0.2658</td>  <td>0.7904</td> <td>-3315.6336</td>   <td>4355.6418</td>\n", "</tr>\n", "<tr>\n", "  <th>x24</th>    <td>-325.9049</td>  <td>1598.8675</td> <td>-0.2038</td> <td>0.8385</td> <td>-3460.4188</td>   <td>2808.6090</td>\n", "</tr>\n", "<tr>\n", "  <th>x25</th>    <td>-231.1519</td>  <td>129.0113</td>  <td>-1.7917</td> <td>0.0732</td>  <td>-484.0732</td>    <td>21.7694</td> \n", "</tr>\n", "<tr>\n", "  <th>x26</th>    <td>4345.4355</td>  <td>4193.6897</td> <td>1.0362</td>  <td>0.3002</td> <td>-3876.1206</td>  <td>12566.9917</td>\n", "</tr>\n", "<tr>\n", "  <th>x27</th>    <td>169.8998</td>    <td>63.4267</td>  <td>2.6787</td>  <td>0.0074</td>   <td>45.5543</td>    <td>294.2453</td> \n", "</tr>\n", "<tr>\n", "  <th>x28</th>    <td>542.4856</td>   <td>138.8860</td>  <td>3.9060</td>  <td>0.0001</td>  <td>270.2053</td>    <td>814.7658</td> \n", "</tr>\n", "<tr>\n", "  <th>x29</th>   <td>-8778.6169</td>  <td>5436.2565</td> <td>-1.6148</td> <td>0.1064</td> <td>-19436.1739</td>  <td>1878.9402</td>\n", "</tr>\n", "<tr>\n", "  <th>x30</th>    <td>1809.6247</td>  <td>1561.8380</td> <td>1.1587</td>  <td>0.2467</td> <td>-1252.2945</td>   <td>4871.5438</td>\n", "</tr>\n", "<tr>\n", "  <th>x31</th>   <td>-2415.5578</td>  <td>1236.4205</td> <td>-1.9537</td> <td>0.0508</td> <td>-4839.5094</td>    <td>8.3938</td>  \n", "</tr>\n", "<tr>\n", "  <th>x32</th>    <td>749.0360</td>   <td>191.8871</td>  <td>3.9035</td>  <td>0.0001</td>  <td>372.8492</td>    <td>1125.2228</td>\n", "</tr>\n", "<tr>\n", "  <th>x33</th>    <td>-566.9825</td>  <td>2058.9378</td> <td>-0.2754</td> <td>0.7830</td> <td>-4603.4454</td>   <td>3469.4803</td>\n", "</tr>\n", "<tr>\n", "  <th>x34</th>    <td>-107.1104</td>  <td>167.6736</td>  <td>-0.6388</td> <td>0.5230</td>  <td>-435.8275</td>   <td>221.6068</td> \n", "</tr>\n", "<tr>\n", "  <th>x35</th>    <td>3906.7743</td>  <td>5806.8281</td> <td>0.6728</td>  <td>0.5011</td> <td>-7477.2732</td>  <td>15290.8218</td>\n", "</tr>\n", "<tr>\n", "  <th>x36</th>     <td>10.0617</td>    <td>81.2204</td>  <td>0.1239</td>  <td>0.9014</td>  <td>-149.1677</td>   <td>169.2910</td> \n", "</tr>\n", "<tr>\n", "  <th>x37</th>    <td>-24.5923</td>   <td>177.8119</td>  <td>-0.1383</td> <td>0.8900</td>  <td>-373.1851</td>   <td>324.0006</td> \n", "</tr>\n", "<tr>\n", "  <th>x38</th>    <td>743.3110</td>   <td>8266.7074</td> <td>0.0899</td>  <td>0.9284</td> <td>-15463.2288</td> <td>16949.8507</td>\n", "</tr>\n", "<tr>\n", "  <th>x39</th>   <td>-2927.3589</td>  <td>2162.4656</td> <td>-1.3537</td> <td>0.1759</td> <td>-7166.7837</td>   <td>1312.0660</td>\n", "</tr>\n", "<tr>\n", "  <th>x40</th>    <td>1943.9211</td>  <td>1604.4419</td> <td>1.2116</td>  <td>0.2257</td> <td>-1201.5212</td>   <td>5089.3633</td>\n", "</tr>\n", "<tr>\n", "  <th>x41</th>    <td>604.3677</td>   <td>250.6121</td>  <td>2.4116</td>  <td>0.0159</td>  <td>113.0530</td>    <td>1095.6823</td>\n", "</tr>\n", "<tr>\n", "  <th>x42</th>    <td>1020.5273</td>  <td>1873.8778</td> <td>0.5446</td>  <td>0.5860</td> <td>-2653.1330</td>   <td>4694.1876</td>\n", "</tr>\n", "<tr>\n", "  <th>x43</th>    <td>1250.7078</td>  <td>760.6158</td>  <td>1.6443</td>  <td>0.1002</td>  <td>-240.4481</td>   <td>2741.8637</td>\n", "</tr>\n", "<tr>\n", "  <th>x44</th>    <td>-10.1449</td>    <td>4.8977</td>   <td>-2.0714</td> <td>0.0384</td>  <td>-19.7466</td>     <td>-0.5432</td> \n", "</tr>\n", "<tr>\n", "  <th>x45</th>     <td>2.2443</td>     <td>12.2455</td>  <td>0.1833</td>  <td>0.8546</td>  <td>-21.7624</td>     <td>26.2511</td> \n", "</tr>\n", "<tr>\n", "  <th>x46</th>    <td>616.4495</td>   <td>725.3393</td>  <td>0.8499</td>  <td>0.3954</td>  <td>-805.5484</td>   <td>2038.4474</td>\n", "</tr>\n", "<tr>\n", "  <th>x47</th>    <td>-134.1735</td>  <td>203.3448</td>  <td>-0.6598</td> <td>0.5094</td>  <td>-532.8226</td>   <td>264.4757</td> \n", "</tr>\n", "<tr>\n", "  <th>x48</th>    <td>-123.4359</td>  <td>167.3097</td>  <td>-0.7378</td> <td>0.4607</td>  <td>-451.4397</td>   <td>204.5678</td> \n", "</tr>\n", "<tr>\n", "  <th>x49</th>     <td>-4.4518</td>    <td>20.8884</td>  <td>-0.2131</td> <td>0.8312</td>  <td>-45.4026</td>     <td>36.4990</td> \n", "</tr>\n", "<tr>\n", "  <th>x50</th>    <td>4274.1441</td>  <td>6764.6180</td> <td>0.6318</td>  <td>0.5275</td> <td>-8987.6111</td>  <td>17535.8993</td>\n", "</tr>\n", "<tr>\n", "  <th>x51</th>    <td>-361.0029</td>  <td>283.6415</td>  <td>-1.2727</td> <td>0.2032</td>  <td>-917.0705</td>   <td>195.0647</td> \n", "</tr>\n", "<tr>\n", "  <th>x52</th>    <td>1132.4796</td>  <td>607.3146</td>  <td>1.8647</td>  <td>0.0623</td>  <td>-58.1356</td>    <td>2323.0948</td>\n", "</tr>\n", "<tr>\n", "  <th>x53</th>   <td>32386.1105</td> <td>22852.8882</td> <td>1.4172</td>  <td>0.1565</td> <td>-12416.0364</td> <td>77188.2575</td>\n", "</tr>\n", "<tr>\n", "  <th>x54</th>   <td>-9169.2287</td>  <td>6043.2328</td> <td>-1.5173</td> <td>0.1293</td> <td>-21016.7378</td>  <td>2678.2804</td>\n", "</tr>\n", "<tr>\n", "  <th>x55</th>   <td>-3935.3569</td>  <td>4533.8479</td> <td>-0.8680</td> <td>0.3854</td> <td>-12823.7790</td>  <td>4953.0653</td>\n", "</tr>\n", "<tr>\n", "  <th>x56</th>    <td>1044.0159</td>  <td>730.5207</td>  <td>1.4291</td>  <td>0.1530</td>  <td>-388.1398</td>   <td>2476.1716</td>\n", "</tr>\n", "<tr>\n", "  <th>x57</th>    <td>1011.9144</td>  <td>1874.0030</td> <td>0.5400</td>  <td>0.5892</td> <td>-2661.9913</td>   <td>4685.8201</td>\n", "</tr>\n", "<tr>\n", "  <th>x58</th>    <td>-31.4356</td>    <td>7.2561</td>   <td>-4.3323</td> <td>0.0000</td>  <td>-45.6610</td>    <td>-17.2103</td> \n", "</tr>\n", "<tr>\n", "  <th>x59</th>    <td>527.6354</td>   <td>299.1980</td>  <td>1.7635</td>  <td>0.0779</td>  <td>-58.9301</td>    <td>1114.2008</td>\n", "</tr>\n", "<tr>\n", "  <th>x60</th>    <td>-35.3657</td>    <td>79.2158</td>  <td>-0.4464</td> <td>0.6553</td>  <td>-190.6650</td>   <td>119.9336</td> \n", "</tr>\n", "<tr>\n", "  <th>x61</th>     <td>77.6789</td>    <td>65.5638</td>  <td>1.1848</td>  <td>0.2362</td>  <td>-50.8562</td>    <td>206.2139</td> \n", "</tr>\n", "<tr>\n", "  <th>x62</th>    <td>-57.8306</td>    <td>9.2836</td>   <td>-6.2293</td> <td>0.0000</td>  <td>-76.0308</td>    <td>-39.6304</td> \n", "</tr>\n", "<tr>\n", "  <th>x63</th>    <td>995.5783</td>   <td>1874.0603</td> <td>0.5312</td>  <td>0.5953</td> <td>-2678.4398</td>   <td>4669.5964</td>\n", "</tr>\n", "<tr>\n", "  <th>x64</th>     <td>28.6082</td>   <td>645.1442</td>  <td>0.0443</td>  <td>0.9646</td> <td>-1236.1706</td>   <td>1293.3869</td>\n", "</tr>\n", "<tr>\n", "  <th>x65</th>    <td>296.2682</td>   <td>172.0530</td>  <td>1.7220</td>  <td>0.0851</td>  <td>-41.0347</td>    <td>633.5711</td> \n", "</tr>\n", "<tr>\n", "  <th>x66</th>    <td>296.9037</td>   <td>146.6463</td>  <td>2.0246</td>  <td>0.0430</td>   <td>9.4096</td>     <td>584.3978</td> \n", "</tr>\n", "<tr>\n", "  <th>x67</th>    <td>-196.1275</td>   <td>22.5097</td>  <td>-8.7130</td> <td>0.0000</td>  <td>-240.2568</td>   <td>-151.9981</td>\n", "</tr>\n", "<tr>\n", "  <th>x68</th>   <td>-6153.6885</td> <td>18362.8686</td> <td>-0.3351</td> <td>0.7376</td> <td>-42153.3367</td> <td>29845.9598</td>\n", "</tr>\n", "<tr>\n", "  <th>x69</th>   <td>14642.0880</td>  <td>9841.6551</td> <td>1.4878</td>  <td>0.1369</td> <td>-4652.0717</td>  <td>33936.2477</td>\n", "</tr>\n", "<tr>\n", "  <th>x70</th>   <td>-1516.3079</td>  <td>5845.7221</td> <td>-0.2594</td> <td>0.7953</td> <td>-12976.6055</td>  <td>9943.9897</td>\n", "</tr>\n", "<tr>\n", "  <th>x71</th>   <td>-2197.1628</td>  <td>981.3622</td>  <td>-2.2389</td> <td>0.0252</td> <td>-4121.0830</td>   <td>-273.2427</td>\n", "</tr>\n", "<tr>\n", "  <th>x72</th>   <td>-2425.8554</td>  <td>1193.3038</td> <td>-2.0329</td> <td>0.0421</td> <td>-4765.2784</td>   <td>-86.4324</td> \n", "</tr>\n", "<tr>\n", "  <th>x73</th>    <td>1641.2450</td>  <td>1566.4224</td> <td>1.0478</td>  <td>0.2948</td> <td>-1429.6615</td>   <td>4712.1516</td>\n", "</tr>\n", "<tr>\n", "  <th>x74</th>    <td>725.5301</td>   <td>253.2564</td>  <td>2.8648</td>  <td>0.0042</td>  <td>229.0313</td>    <td>1222.0290</td>\n", "</tr>\n", "<tr>\n", "  <th>x75</th>   <td>-1657.2401</td>  <td>2016.9159</td> <td>-0.8217</td> <td>0.4113</td> <td>-5611.3208</td>   <td>2296.8406</td>\n", "</tr>\n", "<tr>\n", "  <th>x76</th>    <td>398.5591</td>   <td>200.1289</td>  <td>1.9915</td>  <td>0.0465</td>   <td>6.2146</td>     <td>790.9036</td> \n", "</tr>\n", "<tr>\n", "  <th>x77</th>    <td>898.1505</td>   <td>1871.4360</td> <td>0.4799</td>  <td>0.6313</td> <td>-2770.7228</td>   <td>4567.0237</td>\n", "</tr>\n", "</table>\n", "<table class=\"simpletable\">\n", "<tr>\n", "     <td>Omnibus:</td>    <td>85.458</td>  <td>Durbin-Watson:</td>         <td>1.991</td>     \n", "</tr>\n", "<tr>\n", "  <td>Prob(Omnibus):</td>  <td>0.000</td> <td>Jarque-Bera (JB):</td>      <td>148.619</td>    \n", "</tr>\n", "<tr>\n", "       <td>Skew:</td>      <td>0.132</td>     <td>Prob(JB):</td>           <td>0.000</td>     \n", "</tr>\n", "<tr>\n", "     <td>Kurtosis:</td>    <td>3.814</td>  <td>Condition No.:</td>   <td>4493323340279420</td>\n", "</tr>\n", "</table>"], "text/plain": ["<class 'statsmodels.iolib.summary2.Summary'>\n", "\"\"\"\n", "                  Results: Ordinary least squares\n", "===================================================================\n", "Model:              OLS              Adj. R-squared:     0.302     \n", "Dependent Variable: quality          AIC:                10821.7528\n", "Date:               2018-09-09 14:59 BIC:                11321.5798\n", "No. Observations:   4872             Log-Likelihood:     -5333.9   \n", "Df Model:           76               F-statistic:        28.70     \n", "Df Residuals:       4795             Prob (F-statistic): 0.00      \n", "R-squared:          0.313            Scale:              0.53135   \n", "-------------------------------------------------------------------\n", "          Coef.     Std.Err.     t    P>|t|     [0.025     0.975]  \n", "-------------------------------------------------------------------\n", "const   -1062.0749  1873.8933 -0.5668 0.5709  -4735.7657  2611.6159\n", "x1         -2.0670    24.3092 -0.0850 0.9322    -49.7241    45.5901\n", "x2       -729.2270   157.0307 -4.6439 0.0000  -1037.0792  -421.3748\n", "x3          8.7231   200.5623  0.0435 0.9653   -384.4709   401.9171\n", "x4          3.4131    13.3672  0.2553 0.7985    -22.7927    29.6188\n", "x5      -1171.6645   689.5425 -1.6992 0.0893  -2523.4842   180.1551\n", "x6         40.1316     8.1636  4.9159 0.0000     24.1272    56.1360\n", "x7         70.8825    22.2599  3.1843 0.0015     27.2429   114.5221\n", "x8       -724.4624   724.6583 -0.9997 0.3175  -2145.1251   696.2003\n", "x9       -251.7727   192.4538 -1.3082 0.1909   -629.0704   125.5250\n", "x10      -276.1104   163.5189 -1.6886 0.0914   -596.6824    44.4616\n", "x11       258.8220    24.9389 10.3782 0.0000    209.9303   307.7138\n", "x12      1021.3730  1866.5461  0.5472 0.5843  -2637.9138  4680.6598\n", "x13       394.9815   155.7356  2.5362 0.0112     89.6682   700.2947\n", "x14       250.5746   208.6039  1.2012 0.2297   -158.3848   659.5340\n", "x15        -4.4734    21.4718 -0.2083 0.8350    -46.5681    37.6213\n", "x16      -829.1409   537.0924 -1.5438 0.1227  -1882.0886   223.8067\n", "x17        -7.9049     9.5287 -0.8296 0.4068    -26.5856    10.7758\n", "x18         5.7595    21.3304  0.2700 0.7872    -36.0579    47.5770\n", "x19       375.0969  1075.4182  0.3488 0.7273  -1733.2162  2483.4100\n", "x20       114.3253   261.5978  0.4370 0.6621   -398.5264   627.1770\n", "x21      -134.9056   151.9652 -0.8877 0.3747   -432.8272   163.0159\n", "x22       -72.6985    26.4607 -2.7474 0.0060   -124.5735   -20.8234\n", "x23       520.0041  1956.5000  0.2658 0.7904  -3315.6336  4355.6418\n", "x24      -325.9049  1598.8675 -0.2038 0.8385  -3460.4188  2808.6090\n", "x25      -231.1519   129.0113 -1.7917 0.0732   -484.0732    21.7694\n", "x26      4345.4355  4193.6897  1.0362 0.3002  -3876.1206 12566.9917\n", "x27       169.8998    63.4267  2.6787 0.0074     45.5543   294.2453\n", "x28       542.4856   138.8860  3.9060 0.0001    270.2053   814.7658\n", "x29     -8778.6169  5436.2565 -1.6148 0.1064 -19436.1739  1878.9402\n", "x30      1809.6247  1561.8380  1.1587 0.2467  -1252.2945  4871.5438\n", "x31     -2415.5578  1236.4205 -1.9537 0.0508  -4839.5094     8.3938\n", "x32       749.0360   191.8871  3.9035 0.0001    372.8492  1125.2228\n", "x33      -566.9825  2058.9378 -0.2754 0.7830  -4603.4454  3469.4803\n", "x34      -107.1104   167.6736 -0.6388 0.5230   -435.8275   221.6068\n", "x35      3906.7743  5806.8281  0.6728 0.5011  -7477.2732 15290.8218\n", "x36        10.0617    81.2204  0.1239 0.9014   -149.1677   169.2910\n", "x37       -24.5923   177.8119 -0.1383 0.8900   -373.1851   324.0006\n", "x38       743.3110  8266.7074  0.0899 0.9284 -15463.2288 16949.8507\n", "x39     -2927.3589  2162.4656 -1.3537 0.1759  -7166.7837  1312.0660\n", "x40      1943.9211  1604.4419  1.2116 0.2257  -1201.5212  5089.3633\n", "x41       604.3677   250.6121  2.4116 0.0159    113.0530  1095.6823\n", "x42      1020.5273  1873.8778  0.5446 0.5860  -2653.1330  4694.1876\n", "x43      1250.7078   760.6158  1.6443 0.1002   -240.4481  2741.8637\n", "x44       -10.1449     4.8977 -2.0714 0.0384    -19.7466    -0.5432\n", "x45         2.2443    12.2455  0.1833 0.8546    -21.7624    26.2511\n", "x46       616.4495   725.3393  0.8499 0.3954   -805.5484  2038.4474\n", "x47      -134.1735   203.3448 -0.6598 0.5094   -532.8226   264.4757\n", "x48      -123.4359   167.3097 -0.7378 0.4607   -451.4397   204.5678\n", "x49        -4.4518    20.8884 -0.2131 0.8312    -45.4026    36.4990\n", "x50      4274.1441  6764.6180  0.6318 0.5275  -8987.6111 17535.8993\n", "x51      -361.0029   283.6415 -1.2727 0.2032   -917.0705   195.0647\n", "x52      1132.4796   607.3146  1.8647 0.0623    -58.1356  2323.0948\n", "x53     32386.1105 22852.8882  1.4172 0.1565 -12416.0364 77188.2575\n", "x54     -9169.2287  6043.2328 -1.5173 0.1293 -21016.7378  2678.2804\n", "x55     -3935.3569  4533.8479 -0.8680 0.3854 -12823.7790  4953.0653\n", "x56      1044.0159   730.5207  1.4291 0.1530   -388.1398  2476.1716\n", "x57      1011.9144  1874.0030  0.5400 0.5892  -2661.9913  4685.8201\n", "x58       -31.4356     7.2561 -4.3323 0.0000    -45.6610   -17.2103\n", "x59       527.6354   299.1980  1.7635 0.0779    -58.9301  1114.2008\n", "x60       -35.3657    79.2158 -0.4464 0.6553   -190.6650   119.9336\n", "x61        77.6789    65.5638  1.1848 0.2362    -50.8562   206.2139\n", "x62       -57.8306     9.2836 -6.2293 0.0000    -76.0308   -39.6304\n", "x63       995.5783  1874.0603  0.5312 0.5953  -2678.4398  4669.5964\n", "x64        28.6082   645.1442  0.0443 0.9646  -1236.1706  1293.3869\n", "x65       296.2682   172.0530  1.7220 0.0851    -41.0347   633.5711\n", "x66       296.9037   146.6463  2.0246 0.0430      9.4096   584.3978\n", "x67      -196.1275    22.5097 -8.7130 0.0000   -240.2568  -151.9981\n", "x68     -6153.6885 18362.8686 -0.3351 0.7376 -42153.3367 29845.9598\n", "x69     14642.0880  9841.6551  1.4878 0.1369  -4652.0717 33936.2477\n", "x70     -1516.3079  5845.7221 -0.2594 0.7953 -12976.6055  9943.9897\n", "x71     -2197.1628   981.3622 -2.2389 0.0252  -4121.0830  -273.2427\n", "x72     -2425.8554  1193.3038 -2.0329 0.0421  -4765.2784   -86.4324\n", "x73      1641.2450  1566.4224  1.0478 0.2948  -1429.6615  4712.1516\n", "x74       725.5301   253.2564  2.8648 0.0042    229.0313  1222.0290\n", "x75     -1657.2401  2016.9159 -0.8217 0.4113  -5611.3208  2296.8406\n", "x76       398.5591   200.1289  1.9915 0.0465      6.2146   790.9036\n", "x77       898.1505  1871.4360  0.4799 0.6313  -2770.7228  4567.0237\n", "-------------------------------------------------------------------\n", "Omnibus:            85.458    Durbin-Watson:       1.991           \n", "Prob(Omnibus):      0.000     Jarque-Bera (JB):    148.619         \n", "Skew:               0.132     Prob(JB):            0.000           \n", "Kurtosis:           3.814     Condition No.:       4493323340279420\n", "===================================================================\n", "* The condition number is large (4e+15). This might indicate\n", "strong multicollinearity or other numerical problems.\n", "\"\"\""]}, "execution_count": 10, "metadata": {}, "output_type": "execute_result"}], "source": ["from statsmodels.regression.linear_model import OLS\n", "model = OLS(y_train, poly_feat_train)\n", "results = model.fit()\n", "results.summary2()"]}, {"cell_type": "markdown", "metadata": {}, "source": ["Ce n'est pas tr\u00e8s lisible. Il faut ajouter le nom de chaque variable et recommencer."]}, {"cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [{"data": {"text/html": ["<div>\n", "<style scoped>\n", "    .dataframe tbody tr th:only-of-type {\n", "        vertical-align: middle;\n", "    }\n", "\n", "    .dataframe tbody tr th {\n", "        vertical-align: top;\n", "    }\n", "\n", "    .dataframe thead th {\n", "        text-align: right;\n", "    }\n", "</style>\n", "<table border=\"1\" class=\"dataframe\">\n", "  <thead>\n", "    <tr style=\"text-align: right;\">\n", "      <th></th>\n", "      <th>1</th>\n", "      <th>fixed_acidity</th>\n", "      <th>volatile_acidity</th>\n", "      <th>citric_acid</th>\n", "      <th>residual_sugar</th>\n", "      <th>chlorides</th>\n", "      <th>free_sulfur_dioxide</th>\n", "      <th>total_sulfur_dioxide</th>\n", "      <th>density</th>\n", "      <th>pH</th>\n", "      <th>...</th>\n", "      <th>density^2</th>\n", "      <th>density * pH</th>\n", "      <th>density * sulphates</th>\n", "      <th>density * alcohol</th>\n", "      <th>pH^2</th>\n", "      <th>pH * sulphates</th>\n", "      <th>pH * alcohol</th>\n", "      <th>sulphates^2</th>\n", "      <th>sulphates * alcohol</th>\n", "      <th>alcohol^2</th>\n", "    </tr>\n", "  </thead>\n", "  <tbody>\n", "    <tr>\n", "      <th>0</th>\n", "      <td>1.0</td>\n", "      <td>0.034535</td>\n", "      <td>0.001889</td>\n", "      <td>0.002752</td>\n", "      <td>0.042089</td>\n", "      <td>0.000297</td>\n", "      <td>0.285993</td>\n", "      <td>0.955108</td>\n", "      <td>0.005369</td>\n", "      <td>0.016836</td>\n", "      <td>...</td>\n", "      <td>0.000029</td>\n", "      <td>0.000090</td>\n", "      <td>0.000013</td>\n", "      <td>0.000278</td>\n", "      <td>0.000283</td>\n", "      <td>0.000041</td>\n", "      <td>0.000872</td>\n", "      <td>0.000006</td>\n", "      <td>0.000126</td>\n", "      <td>0.002683</td>\n", "    </tr>\n", "    <tr>\n", "      <th>1</th>\n", "      <td>1.0</td>\n", "      <td>0.056514</td>\n", "      <td>0.002868</td>\n", "      <td>0.003627</td>\n", "      <td>0.013496</td>\n", "      <td>0.000346</td>\n", "      <td>0.244614</td>\n", "      <td>0.961585</td>\n", "      <td>0.008352</td>\n", "      <td>0.027245</td>\n", "      <td>...</td>\n", "      <td>0.000070</td>\n", "      <td>0.000228</td>\n", "      <td>0.000031</td>\n", "      <td>0.000888</td>\n", "      <td>0.000742</td>\n", "      <td>0.000101</td>\n", "      <td>0.002896</td>\n", "      <td>0.000014</td>\n", "      <td>0.000394</td>\n", "      <td>0.011296</td>\n", "    </tr>\n", "    <tr>\n", "      <th>2</th>\n", "      <td>1.0</td>\n", "      <td>0.062938</td>\n", "      <td>0.001033</td>\n", "      <td>0.002442</td>\n", "      <td>0.139027</td>\n", "      <td>0.000498</td>\n", "      <td>0.413325</td>\n", "      <td>0.892406</td>\n", "      <td>0.009363</td>\n", "      <td>0.030060</td>\n", "      <td>...</td>\n", "      <td>0.000088</td>\n", "      <td>0.000281</td>\n", "      <td>0.000031</td>\n", "      <td>0.000862</td>\n", "      <td>0.000904</td>\n", "      <td>0.000099</td>\n", "      <td>0.002767</td>\n", "      <td>0.000011</td>\n", "      <td>0.000303</td>\n", "      <td>0.008475</td>\n", "    </tr>\n", "    <tr>\n", "      <th>3</th>\n", "      <td>1.0</td>\n", "      <td>0.120882</td>\n", "      <td>0.004638</td>\n", "      <td>0.005622</td>\n", "      <td>0.036546</td>\n", "      <td>0.001167</td>\n", "      <td>0.224896</td>\n", "      <td>0.955808</td>\n", "      <td>0.014025</td>\n", "      <td>0.046385</td>\n", "      <td>...</td>\n", "      <td>0.000197</td>\n", "      <td>0.000651</td>\n", "      <td>0.000095</td>\n", "      <td>0.001853</td>\n", "      <td>0.002152</td>\n", "      <td>0.000313</td>\n", "      <td>0.006129</td>\n", "      <td>0.000046</td>\n", "      <td>0.000891</td>\n", "      <td>0.017457</td>\n", "    </tr>\n", "    <tr>\n", "      <th>4</th>\n", "      <td>1.0</td>\n", "      <td>0.455253</td>\n", "      <td>0.018602</td>\n", "      <td>0.023497</td>\n", "      <td>0.186017</td>\n", "      <td>0.006462</td>\n", "      <td>0.146856</td>\n", "      <td>0.538471</td>\n", "      <td>0.048745</td>\n", "      <td>0.158115</td>\n", "      <td>...</td>\n", "      <td>0.002376</td>\n", "      <td>0.007707</td>\n", "      <td>0.001360</td>\n", "      <td>0.031497</td>\n", "      <td>0.025000</td>\n", "      <td>0.004412</td>\n", "      <td>0.102168</td>\n", "      <td>0.000779</td>\n", "      <td>0.018030</td>\n", "      <td>0.417530</td>\n", "    </tr>\n", "  </tbody>\n", "</table>\n", "<p>5 rows \u00d7 78 columns</p>\n", "</div>"], "text/plain": ["     1  fixed_acidity  volatile_acidity  citric_acid  residual_sugar  \\\n", "0  1.0       0.034535          0.001889     0.002752        0.042089   \n", "1  1.0       0.056514          0.002868     0.003627        0.013496   \n", "2  1.0       0.062938          0.001033     0.002442        0.139027   \n", "3  1.0       0.120882          0.004638     0.005622        0.036546   \n", "4  1.0       0.455253          0.018602     0.023497        0.186017   \n", "\n", "   chlorides  free_sulfur_dioxide  total_sulfur_dioxide   density        pH  \\\n", "0   0.000297             0.285993              0.955108  0.005369  0.016836   \n", "1   0.000346             0.244614              0.961585  0.008352  0.027245   \n", "2   0.000498             0.413325              0.892406  0.009363  0.030060   \n", "3   0.001167             0.224896              0.955808  0.014025  0.046385   \n", "4   0.006462             0.146856              0.538471  0.048745  0.158115   \n", "\n", "     ...      density^2  density * pH  density * sulphates  density * alcohol  \\\n", "0    ...       0.000029      0.000090             0.000013           0.000278   \n", "1    ...       0.000070      0.000228             0.000031           0.000888   \n", "2    ...       0.000088      0.000281             0.000031           0.000862   \n", "3    ...       0.000197      0.000651             0.000095           0.001853   \n", "4    ...       0.002376      0.007707             0.001360           0.031497   \n", "\n", "       pH^2  pH * sulphates  pH * alcohol  sulphates^2  sulphates * alcohol  \\\n", "0  0.000283        0.000041      0.000872     0.000006             0.000126   \n", "1  0.000742        0.000101      0.002896     0.000014             0.000394   \n", "2  0.000904        0.000099      0.002767     0.000011             0.000303   \n", "3  0.002152        0.000313      0.006129     0.000046             0.000891   \n", "4  0.025000        0.004412      0.102168     0.000779             0.018030   \n", "\n", "   alcohol^2  \n", "0   0.002683  \n", "1   0.011296  \n", "2   0.008475  \n", "3   0.017457  \n", "4   0.417530  \n", "\n", "[5 rows x 78 columns]"]}, "execution_count": 11, "metadata": {}, "output_type": "execute_result"}], "source": ["names = poly.get_feature_names(input_features=data.columns[:-2])\n", "names = [n.replace(\" \", \" * \") for n in names]\n", "pft = pandas.DataFrame(poly_feat_train, columns=names)\n", "pft.head()"]}, {"cell_type": "code", "execution_count": 11, "metadata": {"scrolled": false}, "outputs": [{"data": {"text/html": ["<table class=\"simpletable\">\n", "<tr>\n", "        <td>Model:</td>               <td>OLS</td>         <td>Adj. R-squared:</td>      <td>0.302</td>  \n", "</tr>\n", "<tr>\n", "  <td>Dependent Variable:</td>      <td>quality</td>            <td>AIC:</td>         <td>10821.7528</td>\n", "</tr>\n", "<tr>\n", "         <td>Date:</td>        <td>2018-09-09 15:02</td>        <td>BIC:</td>         <td>11321.5798</td>\n", "</tr>\n", "<tr>\n", "   <td>No. Observations:</td>        <td>4872</td>         <td>Log-Likelihood:</td>     <td>-5333.9</td> \n", "</tr>\n", "<tr>\n", "       <td>Df Model:</td>             <td>76</td>           <td>F-statistic:</td>        <td>28.70</td>  \n", "</tr>\n", "<tr>\n", "     <td>Df Residuals:</td>          <td>4795</td>       <td>Prob (F-statistic):</td>    <td>0.00</td>   \n", "</tr>\n", "<tr>\n", "      <td>R-squared:</td>            <td>0.313</td>            <td>Scale:</td>          <td>0.53135</td> \n", "</tr>\n", "</table>\n", "<table class=\"simpletable\">\n", "<tr>\n", "                       <td></td>                         <th>Coef.</th>    <th>Std.Err.</th>     <th>t</th>     <th>P>|t|</th>   <th>[0.025</th>      <th>0.975]</th>  \n", "</tr>\n", "<tr>\n", "  <th>1</th>                                          <td>-1062.0749</td>  <td>1873.8933</td> <td>-0.5668</td> <td>0.5709</td> <td>-4735.7657</td>   <td>2611.6159</td>\n", "</tr>\n", "<tr>\n", "  <th>fixed_acidity</th>                                <td>-2.0670</td>    <td>24.3092</td>  <td>-0.0850</td> <td>0.9322</td>  <td>-49.7241</td>     <td>45.5901</td> \n", "</tr>\n", "<tr>\n", "  <th>volatile_acidity</th>                            <td>-729.2270</td>  <td>157.0307</td>  <td>-4.6439</td> <td>0.0000</td> <td>-1037.0792</td>   <td>-421.3748</td>\n", "</tr>\n", "<tr>\n", "  <th>citric_acid</th>                                  <td>8.7231</td>    <td>200.5623</td>  <td>0.0435</td>  <td>0.9653</td>  <td>-384.4709</td>   <td>401.9171</td> \n", "</tr>\n", "<tr>\n", "  <th>residual_sugar</th>                               <td>3.4131</td>     <td>13.3672</td>  <td>0.2553</td>  <td>0.7985</td>  <td>-22.7927</td>     <td>29.6188</td> \n", "</tr>\n", "<tr>\n", "  <th>chlorides</th>                                  <td>-1171.6645</td>  <td>689.5425</td>  <td>-1.6992</td> <td>0.0893</td> <td>-2523.4842</td>   <td>180.1551</td> \n", "</tr>\n", "<tr>\n", "  <th>free_sulfur_dioxide</th>                          <td>40.1316</td>    <td>8.1636</td>   <td>4.9159</td>  <td>0.0000</td>   <td>24.1272</td>     <td>56.1360</td> \n", "</tr>\n", "<tr>\n", "  <th>total_sulfur_dioxide</th>                         <td>70.8825</td>    <td>22.2599</td>  <td>3.1843</td>  <td>0.0015</td>   <td>27.2429</td>    <td>114.5221</td> \n", "</tr>\n", "<tr>\n", "  <th>density</th>                                     <td>-724.4624</td>  <td>724.6583</td>  <td>-0.9997</td> <td>0.3175</td> <td>-2145.1251</td>   <td>696.2003</td> \n", "</tr>\n", "<tr>\n", "  <th>pH</th>                                          <td>-251.7727</td>  <td>192.4538</td>  <td>-1.3082</td> <td>0.1909</td>  <td>-629.0704</td>   <td>125.5250</td> \n", "</tr>\n", "<tr>\n", "  <th>sulphates</th>                                   <td>-276.1104</td>  <td>163.5189</td>  <td>-1.6886</td> <td>0.0914</td>  <td>-596.6824</td>    <td>44.4616</td> \n", "</tr>\n", "<tr>\n", "  <th>alcohol</th>                                     <td>258.8220</td>    <td>24.9389</td>  <td>10.3782</td> <td>0.0000</td>  <td>209.9303</td>    <td>307.7138</td> \n", "</tr>\n", "<tr>\n", "  <th>fixed_acidity^2</th>                             <td>1021.3730</td>  <td>1866.5461</td> <td>0.5472</td>  <td>0.5843</td> <td>-2637.9138</td>   <td>4680.6598</td>\n", "</tr>\n", "<tr>\n", "  <th>fixed_acidity * volatile_acidity</th>            <td>394.9815</td>   <td>155.7356</td>  <td>2.5362</td>  <td>0.0112</td>   <td>89.6682</td>    <td>700.2947</td> \n", "</tr>\n", "<tr>\n", "  <th>fixed_acidity * citric_acid</th>                 <td>250.5746</td>   <td>208.6039</td>  <td>1.2012</td>  <td>0.2297</td>  <td>-158.3848</td>   <td>659.5340</td> \n", "</tr>\n", "<tr>\n", "  <th>fixed_acidity * residual_sugar</th>               <td>-4.4734</td>    <td>21.4718</td>  <td>-0.2083</td> <td>0.8350</td>  <td>-46.5681</td>     <td>37.6213</td> \n", "</tr>\n", "<tr>\n", "  <th>fixed_acidity * chlorides</th>                   <td>-829.1409</td>  <td>537.0924</td>  <td>-1.5438</td> <td>0.1227</td> <td>-1882.0886</td>   <td>223.8067</td> \n", "</tr>\n", "<tr>\n", "  <th>fixed_acidity * free_sulfur_dioxide</th>          <td>-7.9049</td>    <td>9.5287</td>   <td>-0.8296</td> <td>0.4068</td>  <td>-26.5856</td>     <td>10.7758</td> \n", "</tr>\n", "<tr>\n", "  <th>fixed_acidity * total_sulfur_dioxide</th>         <td>5.7595</td>     <td>21.3304</td>  <td>0.2700</td>  <td>0.7872</td>  <td>-36.0579</td>     <td>47.5770</td> \n", "</tr>\n", "<tr>\n", "  <th>fixed_acidity * density</th>                     <td>375.0969</td>   <td>1075.4182</td> <td>0.3488</td>  <td>0.7273</td> <td>-1733.2162</td>   <td>2483.4100</td>\n", "</tr>\n", "<tr>\n", "  <th>fixed_acidity * pH</th>                          <td>114.3253</td>   <td>261.5978</td>  <td>0.4370</td>  <td>0.6621</td>  <td>-398.5264</td>   <td>627.1770</td> \n", "</tr>\n", "<tr>\n", "  <th>fixed_acidity * sulphates</th>                   <td>-134.9056</td>  <td>151.9652</td>  <td>-0.8877</td> <td>0.3747</td>  <td>-432.8272</td>   <td>163.0159</td> \n", "</tr>\n", "<tr>\n", "  <th>fixed_acidity * alcohol</th>                     <td>-72.6985</td>    <td>26.4607</td>  <td>-2.7474</td> <td>0.0060</td>  <td>-124.5735</td>   <td>-20.8234</td> \n", "</tr>\n", "<tr>\n", "  <th>volatile_acidity^2</th>                          <td>520.0041</td>   <td>1956.5000</td> <td>0.2658</td>  <td>0.7904</td> <td>-3315.6336</td>   <td>4355.6418</td>\n", "</tr>\n", "<tr>\n", "  <th>volatile_acidity * citric_acid</th>              <td>-325.9049</td>  <td>1598.8675</td> <td>-0.2038</td> <td>0.8385</td> <td>-3460.4188</td>   <td>2808.6090</td>\n", "</tr>\n", "<tr>\n", "  <th>volatile_acidity * residual_sugar</th>           <td>-231.1519</td>  <td>129.0113</td>  <td>-1.7917</td> <td>0.0732</td>  <td>-484.0732</td>    <td>21.7694</td> \n", "</tr>\n", "<tr>\n", "  <th>volatile_acidity * chlorides</th>                <td>4345.4355</td>  <td>4193.6897</td> <td>1.0362</td>  <td>0.3002</td> <td>-3876.1206</td>  <td>12566.9917</td>\n", "</tr>\n", "<tr>\n", "  <th>volatile_acidity * free_sulfur_dioxide</th>      <td>169.8998</td>    <td>63.4267</td>  <td>2.6787</td>  <td>0.0074</td>   <td>45.5543</td>    <td>294.2453</td> \n", "</tr>\n", "<tr>\n", "  <th>volatile_acidity * total_sulfur_dioxide</th>     <td>542.4856</td>   <td>138.8860</td>  <td>3.9060</td>  <td>0.0001</td>  <td>270.2053</td>    <td>814.7658</td> \n", "</tr>\n", "<tr>\n", "  <th>volatile_acidity * density</th>                 <td>-8778.6169</td>  <td>5436.2565</td> <td>-1.6148</td> <td>0.1064</td> <td>-19436.1739</td>  <td>1878.9402</td>\n", "</tr>\n", "<tr>\n", "  <th>volatile_acidity * pH</th>                       <td>1809.6247</td>  <td>1561.8380</td> <td>1.1587</td>  <td>0.2467</td> <td>-1252.2945</td>   <td>4871.5438</td>\n", "</tr>\n", "<tr>\n", "  <th>volatile_acidity * sulphates</th>               <td>-2415.5578</td>  <td>1236.4205</td> <td>-1.9537</td> <td>0.0508</td> <td>-4839.5094</td>    <td>8.3938</td>  \n", "</tr>\n", "<tr>\n", "  <th>volatile_acidity * alcohol</th>                  <td>749.0360</td>   <td>191.8871</td>  <td>3.9035</td>  <td>0.0001</td>  <td>372.8492</td>    <td>1125.2228</td>\n", "</tr>\n", "<tr>\n", "  <th>citric_acid^2</th>                               <td>-566.9825</td>  <td>2058.9378</td> <td>-0.2754</td> <td>0.7830</td> <td>-4603.4454</td>   <td>3469.4803</td>\n", "</tr>\n", "<tr>\n", "  <th>citric_acid * residual_sugar</th>                <td>-107.1104</td>  <td>167.6736</td>  <td>-0.6388</td> <td>0.5230</td>  <td>-435.8275</td>   <td>221.6068</td> \n", "</tr>\n", "<tr>\n", "  <th>citric_acid * chlorides</th>                     <td>3906.7743</td>  <td>5806.8281</td> <td>0.6728</td>  <td>0.5011</td> <td>-7477.2732</td>  <td>15290.8218</td>\n", "</tr>\n", "<tr>\n", "  <th>citric_acid * free_sulfur_dioxide</th>            <td>10.0617</td>    <td>81.2204</td>  <td>0.1239</td>  <td>0.9014</td>  <td>-149.1677</td>   <td>169.2910</td> \n", "</tr>\n", "<tr>\n", "  <th>citric_acid * total_sulfur_dioxide</th>          <td>-24.5923</td>   <td>177.8119</td>  <td>-0.1383</td> <td>0.8900</td>  <td>-373.1851</td>   <td>324.0006</td> \n", "</tr>\n", "<tr>\n", "  <th>citric_acid * density</th>                       <td>743.3110</td>   <td>8266.7074</td> <td>0.0899</td>  <td>0.9284</td> <td>-15463.2288</td> <td>16949.8507</td>\n", "</tr>\n", "<tr>\n", "  <th>citric_acid * pH</th>                           <td>-2927.3589</td>  <td>2162.4656</td> <td>-1.3537</td> <td>0.1759</td> <td>-7166.7837</td>   <td>1312.0660</td>\n", "</tr>\n", "<tr>\n", "  <th>citric_acid * sulphates</th>                     <td>1943.9211</td>  <td>1604.4419</td> <td>1.2116</td>  <td>0.2257</td> <td>-1201.5212</td>   <td>5089.3633</td>\n", "</tr>\n", "<tr>\n", "  <th>citric_acid * alcohol</th>                       <td>604.3677</td>   <td>250.6121</td>  <td>2.4116</td>  <td>0.0159</td>  <td>113.0530</td>    <td>1095.6823</td>\n", "</tr>\n", "<tr>\n", "  <th>residual_sugar^2</th>                            <td>1020.5273</td>  <td>1873.8778</td> <td>0.5446</td>  <td>0.5860</td> <td>-2653.1330</td>   <td>4694.1876</td>\n", "</tr>\n", "<tr>\n", "  <th>residual_sugar * chlorides</th>                  <td>1250.7078</td>  <td>760.6158</td>  <td>1.6443</td>  <td>0.1002</td>  <td>-240.4481</td>   <td>2741.8637</td>\n", "</tr>\n", "<tr>\n", "  <th>residual_sugar * free_sulfur_dioxide</th>        <td>-10.1449</td>    <td>4.8977</td>   <td>-2.0714</td> <td>0.0384</td>  <td>-19.7466</td>     <td>-0.5432</td> \n", "</tr>\n", "<tr>\n", "  <th>residual_sugar * total_sulfur_dioxide</th>        <td>2.2443</td>     <td>12.2455</td>  <td>0.1833</td>  <td>0.8546</td>  <td>-21.7624</td>     <td>26.2511</td> \n", "</tr>\n", "<tr>\n", "  <th>residual_sugar * density</th>                    <td>616.4495</td>   <td>725.3393</td>  <td>0.8499</td>  <td>0.3954</td>  <td>-805.5484</td>   <td>2038.4474</td>\n", "</tr>\n", "<tr>\n", "  <th>residual_sugar * pH</th>                         <td>-134.1735</td>  <td>203.3448</td>  <td>-0.6598</td> <td>0.5094</td>  <td>-532.8226</td>   <td>264.4757</td> \n", "</tr>\n", "<tr>\n", "  <th>residual_sugar * sulphates</th>                  <td>-123.4359</td>  <td>167.3097</td>  <td>-0.7378</td> <td>0.4607</td>  <td>-451.4397</td>   <td>204.5678</td> \n", "</tr>\n", "<tr>\n", "  <th>residual_sugar * alcohol</th>                     <td>-4.4518</td>    <td>20.8884</td>  <td>-0.2131</td> <td>0.8312</td>  <td>-45.4026</td>     <td>36.4990</td> \n", "</tr>\n", "<tr>\n", "  <th>chlorides^2</th>                                 <td>4274.1441</td>  <td>6764.6180</td> <td>0.6318</td>  <td>0.5275</td> <td>-8987.6111</td>  <td>17535.8993</td>\n", "</tr>\n", "<tr>\n", "  <th>chlorides * free_sulfur_dioxide</th>             <td>-361.0029</td>  <td>283.6415</td>  <td>-1.2727</td> <td>0.2032</td>  <td>-917.0705</td>   <td>195.0647</td> \n", "</tr>\n", "<tr>\n", "  <th>chlorides * total_sulfur_dioxide</th>            <td>1132.4796</td>  <td>607.3146</td>  <td>1.8647</td>  <td>0.0623</td>  <td>-58.1356</td>    <td>2323.0948</td>\n", "</tr>\n", "<tr>\n", "  <th>chlorides * density</th>                        <td>32386.1105</td> <td>22852.8882</td> <td>1.4172</td>  <td>0.1565</td> <td>-12416.0364</td> <td>77188.2575</td>\n", "</tr>\n", "<tr>\n", "  <th>chlorides * pH</th>                             <td>-9169.2287</td>  <td>6043.2328</td> <td>-1.5173</td> <td>0.1293</td> <td>-21016.7378</td>  <td>2678.2804</td>\n", "</tr>\n", "<tr>\n", "  <th>chlorides * sulphates</th>                      <td>-3935.3569</td>  <td>4533.8479</td> <td>-0.8680</td> <td>0.3854</td> <td>-12823.7790</td>  <td>4953.0653</td>\n", "</tr>\n", "<tr>\n", "  <th>chlorides * alcohol</th>                         <td>1044.0159</td>  <td>730.5207</td>  <td>1.4291</td>  <td>0.1530</td>  <td>-388.1398</td>   <td>2476.1716</td>\n", "</tr>\n", "<tr>\n", "  <th>free_sulfur_dioxide^2</th>                       <td>1011.9144</td>  <td>1874.0030</td> <td>0.5400</td>  <td>0.5892</td> <td>-2661.9913</td>   <td>4685.8201</td>\n", "</tr>\n", "<tr>\n", "  <th>free_sulfur_dioxide * total_sulfur_dioxide</th>  <td>-31.4356</td>    <td>7.2561</td>   <td>-4.3323</td> <td>0.0000</td>  <td>-45.6610</td>    <td>-17.2103</td> \n", "</tr>\n", "<tr>\n", "  <th>free_sulfur_dioxide * density</th>               <td>527.6354</td>   <td>299.1980</td>  <td>1.7635</td>  <td>0.0779</td>  <td>-58.9301</td>    <td>1114.2008</td>\n", "</tr>\n", "<tr>\n", "  <th>free_sulfur_dioxide * pH</th>                    <td>-35.3657</td>    <td>79.2158</td>  <td>-0.4464</td> <td>0.6553</td>  <td>-190.6650</td>   <td>119.9336</td> \n", "</tr>\n", "<tr>\n", "  <th>free_sulfur_dioxide * sulphates</th>              <td>77.6789</td>    <td>65.5638</td>  <td>1.1848</td>  <td>0.2362</td>  <td>-50.8562</td>    <td>206.2139</td> \n", "</tr>\n", "<tr>\n", "  <th>free_sulfur_dioxide * alcohol</th>               <td>-57.8306</td>    <td>9.2836</td>   <td>-6.2293</td> <td>0.0000</td>  <td>-76.0308</td>    <td>-39.6304</td> \n", "</tr>\n", "<tr>\n", "  <th>total_sulfur_dioxide^2</th>                      <td>995.5783</td>   <td>1874.0603</td> <td>0.5312</td>  <td>0.5953</td> <td>-2678.4398</td>   <td>4669.5964</td>\n", "</tr>\n", "<tr>\n", "  <th>total_sulfur_dioxide * density</th>               <td>28.6082</td>   <td>645.1442</td>  <td>0.0443</td>  <td>0.9646</td> <td>-1236.1706</td>   <td>1293.3869</td>\n", "</tr>\n", "<tr>\n", "  <th>total_sulfur_dioxide * pH</th>                   <td>296.2682</td>   <td>172.0530</td>  <td>1.7220</td>  <td>0.0851</td>  <td>-41.0347</td>    <td>633.5711</td> \n", "</tr>\n", "<tr>\n", "  <th>total_sulfur_dioxide * sulphates</th>            <td>296.9037</td>   <td>146.6463</td>  <td>2.0246</td>  <td>0.0430</td>   <td>9.4096</td>     <td>584.3978</td> \n", "</tr>\n", "<tr>\n", "  <th>total_sulfur_dioxide * alcohol</th>              <td>-196.1275</td>   <td>22.5097</td>  <td>-8.7130</td> <td>0.0000</td>  <td>-240.2568</td>   <td>-151.9981</td>\n", "</tr>\n", "<tr>\n", "  <th>density^2</th>                                  <td>-6153.6885</td> <td>18362.8686</td> <td>-0.3351</td> <td>0.7376</td> <td>-42153.3367</td> <td>29845.9598</td>\n", "</tr>\n", "<tr>\n", "  <th>density * pH</th>                               <td>14642.0880</td>  <td>9841.6551</td> <td>1.4878</td>  <td>0.1369</td> <td>-4652.0717</td>  <td>33936.2477</td>\n", "</tr>\n", "<tr>\n", "  <th>density * sulphates</th>                        <td>-1516.3079</td>  <td>5845.7221</td> <td>-0.2594</td> <td>0.7953</td> <td>-12976.6055</td>  <td>9943.9897</td>\n", "</tr>\n", "<tr>\n", "  <th>density * alcohol</th>                          <td>-2197.1628</td>  <td>981.3622</td>  <td>-2.2389</td> <td>0.0252</td> <td>-4121.0830</td>   <td>-273.2427</td>\n", "</tr>\n", "<tr>\n", "  <th>pH^2</th>                                       <td>-2425.8554</td>  <td>1193.3038</td> <td>-2.0329</td> <td>0.0421</td> <td>-4765.2784</td>   <td>-86.4324</td> \n", "</tr>\n", "<tr>\n", "  <th>pH * sulphates</th>                              <td>1641.2450</td>  <td>1566.4224</td> <td>1.0478</td>  <td>0.2948</td> <td>-1429.6615</td>   <td>4712.1516</td>\n", "</tr>\n", "<tr>\n", "  <th>pH * alcohol</th>                                <td>725.5301</td>   <td>253.2564</td>  <td>2.8648</td>  <td>0.0042</td>  <td>229.0313</td>    <td>1222.0290</td>\n", "</tr>\n", "<tr>\n", "  <th>sulphates^2</th>                                <td>-1657.2401</td>  <td>2016.9159</td> <td>-0.8217</td> <td>0.4113</td> <td>-5611.3208</td>   <td>2296.8406</td>\n", "</tr>\n", "<tr>\n", "  <th>sulphates * alcohol</th>                         <td>398.5591</td>   <td>200.1289</td>  <td>1.9915</td>  <td>0.0465</td>   <td>6.2146</td>     <td>790.9036</td> \n", "</tr>\n", "<tr>\n", "  <th>alcohol^2</th>                                   <td>898.1505</td>   <td>1871.4360</td> <td>0.4799</td>  <td>0.6313</td> <td>-2770.7228</td>   <td>4567.0237</td>\n", "</tr>\n", "</table>\n", "<table class=\"simpletable\">\n", "<tr>\n", "     <td>Omnibus:</td>    <td>85.458</td>  <td>Durbin-Watson:</td>         <td>1.991</td>     \n", "</tr>\n", "<tr>\n", "  <td>Prob(Omnibus):</td>  <td>0.000</td> <td>Jarque-Bera (JB):</td>      <td>148.619</td>    \n", "</tr>\n", "<tr>\n", "       <td>Skew:</td>      <td>0.132</td>     <td>Prob(JB):</td>           <td>0.000</td>     \n", "</tr>\n", "<tr>\n", "     <td>Kurtosis:</td>    <td>3.814</td>  <td>Condition No.:</td>   <td>4493323340279420</td>\n", "</tr>\n", "</table>"], "text/plain": ["<class 'statsmodels.iolib.summary2.Summary'>\n", "\"\"\"\n", "                                   Results: Ordinary least squares\n", "======================================================================================================\n", "Model:                           OLS                         Adj. R-squared:                0.302     \n", "Dependent Variable:              quality                     AIC:                           10821.7528\n", "Date:                            2018-09-09 15:02            BIC:                           11321.5798\n", "No. Observations:                4872                        Log-Likelihood:                -5333.9   \n", "Df Model:                        76                          F-statistic:                   28.70     \n", "Df Residuals:                    4795                        Prob (F-statistic):            0.00      \n", "R-squared:                       0.313                       Scale:                         0.53135   \n", "------------------------------------------------------------------------------------------------------\n", "                                             Coef.     Std.Err.     t    P>|t|     [0.025     0.975]  \n", "------------------------------------------------------------------------------------------------------\n", "1                                          -1062.0749  1873.8933 -0.5668 0.5709  -4735.7657  2611.6159\n", "fixed_acidity                                 -2.0670    24.3092 -0.0850 0.9322    -49.7241    45.5901\n", "volatile_acidity                            -729.2270   157.0307 -4.6439 0.0000  -1037.0792  -421.3748\n", "citric_acid                                    8.7231   200.5623  0.0435 0.9653   -384.4709   401.9171\n", "residual_sugar                                 3.4131    13.3672  0.2553 0.7985    -22.7927    29.6188\n", "chlorides                                  -1171.6645   689.5425 -1.6992 0.0893  -2523.4842   180.1551\n", "free_sulfur_dioxide                           40.1316     8.1636  4.9159 0.0000     24.1272    56.1360\n", "total_sulfur_dioxide                          70.8825    22.2599  3.1843 0.0015     27.2429   114.5221\n", "density                                     -724.4624   724.6583 -0.9997 0.3175  -2145.1251   696.2003\n", "pH                                          -251.7727   192.4538 -1.3082 0.1909   -629.0704   125.5250\n", "sulphates                                   -276.1104   163.5189 -1.6886 0.0914   -596.6824    44.4616\n", "alcohol                                      258.8220    24.9389 10.3782 0.0000    209.9303   307.7138\n", "fixed_acidity^2                             1021.3730  1866.5461  0.5472 0.5843  -2637.9138  4680.6598\n", "fixed_acidity * volatile_acidity             394.9815   155.7356  2.5362 0.0112     89.6682   700.2947\n", "fixed_acidity * citric_acid                  250.5746   208.6039  1.2012 0.2297   -158.3848   659.5340\n", "fixed_acidity * residual_sugar                -4.4734    21.4718 -0.2083 0.8350    -46.5681    37.6213\n", "fixed_acidity * chlorides                   -829.1409   537.0924 -1.5438 0.1227  -1882.0886   223.8067\n", "fixed_acidity * free_sulfur_dioxide           -7.9049     9.5287 -0.8296 0.4068    -26.5856    10.7758\n", "fixed_acidity * total_sulfur_dioxide           5.7595    21.3304  0.2700 0.7872    -36.0579    47.5770\n", "fixed_acidity * density                      375.0969  1075.4182  0.3488 0.7273  -1733.2162  2483.4100\n", "fixed_acidity * pH                           114.3253   261.5978  0.4370 0.6621   -398.5264   627.1770\n", "fixed_acidity * sulphates                   -134.9056   151.9652 -0.8877 0.3747   -432.8272   163.0159\n", "fixed_acidity * alcohol                      -72.6985    26.4607 -2.7474 0.0060   -124.5735   -20.8234\n", "volatile_acidity^2                           520.0041  1956.5000  0.2658 0.7904  -3315.6336  4355.6418\n", "volatile_acidity * citric_acid              -325.9049  1598.8675 -0.2038 0.8385  -3460.4188  2808.6090\n", "volatile_acidity * residual_sugar           -231.1519   129.0113 -1.7917 0.0732   -484.0732    21.7694\n", "volatile_acidity * chlorides                4345.4355  4193.6897  1.0362 0.3002  -3876.1206 12566.9917\n", "volatile_acidity * free_sulfur_dioxide       169.8998    63.4267  2.6787 0.0074     45.5543   294.2453\n", "volatile_acidity * total_sulfur_dioxide      542.4856   138.8860  3.9060 0.0001    270.2053   814.7658\n", "volatile_acidity * density                 -8778.6169  5436.2565 -1.6148 0.1064 -19436.1739  1878.9402\n", "volatile_acidity * pH                       1809.6247  1561.8380  1.1587 0.2467  -1252.2945  4871.5438\n", "volatile_acidity * sulphates               -2415.5578  1236.4205 -1.9537 0.0508  -4839.5094     8.3938\n", "volatile_acidity * alcohol                   749.0360   191.8871  3.9035 0.0001    372.8492  1125.2228\n", "citric_acid^2                               -566.9825  2058.9378 -0.2754 0.7830  -4603.4454  3469.4803\n", "citric_acid * residual_sugar                -107.1104   167.6736 -0.6388 0.5230   -435.8275   221.6068\n", "citric_acid * chlorides                     3906.7743  5806.8281  0.6728 0.5011  -7477.2732 15290.8218\n", "citric_acid * free_sulfur_dioxide             10.0617    81.2204  0.1239 0.9014   -149.1677   169.2910\n", "citric_acid * total_sulfur_dioxide           -24.5923   177.8119 -0.1383 0.8900   -373.1851   324.0006\n", "citric_acid * density                        743.3110  8266.7074  0.0899 0.9284 -15463.2288 16949.8507\n", "citric_acid * pH                           -2927.3589  2162.4656 -1.3537 0.1759  -7166.7837  1312.0660\n", "citric_acid * sulphates                     1943.9211  1604.4419  1.2116 0.2257  -1201.5212  5089.3633\n", "citric_acid * alcohol                        604.3677   250.6121  2.4116 0.0159    113.0530  1095.6823\n", "residual_sugar^2                            1020.5273  1873.8778  0.5446 0.5860  -2653.1330  4694.1876\n", "residual_sugar * chlorides                  1250.7078   760.6158  1.6443 0.1002   -240.4481  2741.8637\n", "residual_sugar * free_sulfur_dioxide         -10.1449     4.8977 -2.0714 0.0384    -19.7466    -0.5432\n", "residual_sugar * total_sulfur_dioxide          2.2443    12.2455  0.1833 0.8546    -21.7624    26.2511\n", "residual_sugar * density                     616.4495   725.3393  0.8499 0.3954   -805.5484  2038.4474\n", "residual_sugar * pH                         -134.1735   203.3448 -0.6598 0.5094   -532.8226   264.4757\n", "residual_sugar * sulphates                  -123.4359   167.3097 -0.7378 0.4607   -451.4397   204.5678\n", "residual_sugar * alcohol                      -4.4518    20.8884 -0.2131 0.8312    -45.4026    36.4990\n", "chlorides^2                                 4274.1441  6764.6180  0.6318 0.5275  -8987.6111 17535.8993\n", "chlorides * free_sulfur_dioxide             -361.0029   283.6415 -1.2727 0.2032   -917.0705   195.0647\n", "chlorides * total_sulfur_dioxide            1132.4796   607.3146  1.8647 0.0623    -58.1356  2323.0948\n", "chlorides * density                        32386.1105 22852.8882  1.4172 0.1565 -12416.0364 77188.2575\n", "chlorides * pH                             -9169.2287  6043.2328 -1.5173 0.1293 -21016.7378  2678.2804\n", "chlorides * sulphates                      -3935.3569  4533.8479 -0.8680 0.3854 -12823.7790  4953.0653\n", "chlorides * alcohol                         1044.0159   730.5207  1.4291 0.1530   -388.1398  2476.1716\n", "free_sulfur_dioxide^2                       1011.9144  1874.0030  0.5400 0.5892  -2661.9913  4685.8201\n", "free_sulfur_dioxide * total_sulfur_dioxide   -31.4356     7.2561 -4.3323 0.0000    -45.6610   -17.2103\n", "free_sulfur_dioxide * density                527.6354   299.1980  1.7635 0.0779    -58.9301  1114.2008\n", "free_sulfur_dioxide * pH                     -35.3657    79.2158 -0.4464 0.6553   -190.6650   119.9336\n", "free_sulfur_dioxide * sulphates               77.6789    65.5638  1.1848 0.2362    -50.8562   206.2139\n", "free_sulfur_dioxide * alcohol                -57.8306     9.2836 -6.2293 0.0000    -76.0308   -39.6304\n", "total_sulfur_dioxide^2                       995.5783  1874.0603  0.5312 0.5953  -2678.4398  4669.5964\n", "total_sulfur_dioxide * density                28.6082   645.1442  0.0443 0.9646  -1236.1706  1293.3869\n", "total_sulfur_dioxide * pH                    296.2682   172.0530  1.7220 0.0851    -41.0347   633.5711\n", "total_sulfur_dioxide * sulphates             296.9037   146.6463  2.0246 0.0430      9.4096   584.3978\n", "total_sulfur_dioxide * alcohol              -196.1275    22.5097 -8.7130 0.0000   -240.2568  -151.9981\n", "density^2                                  -6153.6885 18362.8686 -0.3351 0.7376 -42153.3367 29845.9598\n", "density * pH                               14642.0880  9841.6551  1.4878 0.1369  -4652.0717 33936.2477\n", "density * sulphates                        -1516.3079  5845.7221 -0.2594 0.7953 -12976.6055  9943.9897\n", "density * alcohol                          -2197.1628   981.3622 -2.2389 0.0252  -4121.0830  -273.2427\n", "pH^2                                       -2425.8554  1193.3038 -2.0329 0.0421  -4765.2784   -86.4324\n", "pH * sulphates                              1641.2450  1566.4224  1.0478 0.2948  -1429.6615  4712.1516\n", "pH * alcohol                                 725.5301   253.2564  2.8648 0.0042    229.0313  1222.0290\n", "sulphates^2                                -1657.2401  2016.9159 -0.8217 0.4113  -5611.3208  2296.8406\n", "sulphates * alcohol                          398.5591   200.1289  1.9915 0.0465      6.2146   790.9036\n", "alcohol^2                                    898.1505  1871.4360  0.4799 0.6313  -2770.7228  4567.0237\n", "------------------------------------------------------------------------------------------------------\n", "Omnibus:                       85.458                Durbin-Watson:                   1.991           \n", "Prob(Omnibus):                 0.000                 Jarque-Bera (JB):                148.619         \n", "Skew:                          0.132                 Prob(JB):                        0.000           \n", "Kurtosis:                      3.814                 Condition No.:                   4493323340279420\n", "======================================================================================================\n", "* The condition number is large (4e+15). This might indicate             strong multicollinearity or\n", "other numerical problems.\n", "\"\"\""]}, "execution_count": 12, "metadata": {}, "output_type": "execute_result"}], "source": ["results.summary2(xname=pft.columns)"]}, {"cell_type": "markdown", "metadata": {}, "source": ["On ne garde que celles dont la [p-value](http://www.xavierdupre.fr/app/mlstatpy/helpsphinx/c_metric/pvalues.html) est inf\u00e9rieur \u00e0 0.05."]}, {"cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [{"data": {"text/plain": ["x2     3.511159e-06\n", "x6     9.131393e-07\n", "x7     1.460269e-03\n", "x11    5.715290e-25\n", "x13    1.123675e-02\n", "x22    6.029122e-03\n", "x27    7.416579e-03\n", "x28    9.513630e-05\n", "x32    9.610320e-05\n", "x41    1.592150e-02\n", "x44    3.837865e-02\n", "x58    1.505845e-05\n", "x62    5.085920e-10\n", "x66    4.296131e-02\n", "x67    4.014502e-18\n", "x71    2.520870e-02\n", "x72    4.211861e-02\n", "x74    4.190811e-03\n", "x76    4.648129e-02\n", "dtype: float64"]}, "execution_count": 13, "metadata": {}, "output_type": "execute_result"}], "source": ["pval = results.pvalues.copy()\n", "pval[pval <= 0.05]"]}, {"cell_type": "code", "execution_count": 13, "metadata": {}, "outputs": [{"data": {"text/plain": ["volatile_acidity                              3.511159e-06\n", "free_sulfur_dioxide                           9.131393e-07\n", "total_sulfur_dioxide                          1.460269e-03\n", "alcohol                                       5.715290e-25\n", "fixed_acidity * volatile_acidity              1.123675e-02\n", "fixed_acidity * alcohol                       6.029122e-03\n", "volatile_acidity * free_sulfur_dioxide        7.416579e-03\n", "volatile_acidity * total_sulfur_dioxide       9.513630e-05\n", "volatile_acidity * alcohol                    9.610320e-05\n", "citric_acid * alcohol                         1.592150e-02\n", "residual_sugar * free_sulfur_dioxide          3.837865e-02\n", "free_sulfur_dioxide * total_sulfur_dioxide    1.505845e-05\n", "free_sulfur_dioxide * alcohol                 5.085920e-10\n", "total_sulfur_dioxide * sulphates              4.296131e-02\n", "total_sulfur_dioxide * alcohol                4.014502e-18\n", "density * alcohol                             2.520870e-02\n", "pH^2                                          4.211861e-02\n", "pH * alcohol                                  4.190811e-03\n", "sulphates * alcohol                           4.648129e-02\n", "dtype: float64"]}, "execution_count": 14, "metadata": {}, "output_type": "execute_result"}], "source": ["pval.index = pft.columns\n", "pval[pval <= 0.05]"]}, {"cell_type": "markdown", "metadata": {}, "source": ["Le mod\u00e8le fonctionne mieux mais il est plus compliqu\u00e9 de savoir si la contribution de l'alcool est corr\u00e9l\u00e9e positivement avec la qualit\u00e9 car l'alcool appara\u00eet dans plus d'une variable."]}, {"cell_type": "code", "execution_count": 14, "metadata": {}, "outputs": [], "source": []}], "metadata": {"kernelspec": {"display_name": "Python 3", "language": "python", "name": "python3"}, "language_info": {"codemirror_mode": {"name": "ipython", "version": 3}, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.0"}}, "nbformat": 4, "nbformat_minor": 2}