{"cells": [{"cell_type": "markdown", "metadata": {}, "source": ["# Cube multidimensionnel - correction\n", "\n", "Manipulation de tables de mortalit\u00e9s fa\u00e7on OLAP, correction des exercices."]}, {"cell_type": "code", "execution_count": 1, "metadata": {"collapsed": false}, "outputs": [{"name": "stdout", "output_type": "stream", "text": ["Populating the interactive namespace from numpy and matplotlib\n"]}, {"data": {"text/html": ["<b>Plan</b>\n", "<div id=\"my_menu_id\">run previous cell, wait for 2 seconds</div>\n", "<script>\n", "function repeat_indent_string(n){\n", "    var a = \"\" ;\n", "    for ( ; n > 0 ; --n) {\n", "        a += \"    \";\n", "    }\n", "    return a;\n", "}\n", "var update_menu = function() {\n", "    var anchors = document.getElementsByClassName(\"section\");\n", "    if (anchors.length == 0) {\n", "        anchors = document.getElementsByClassName(\"text_cell_render rendered_html\");\n", "    }\n", "    var i;\n", "    var text_menu = \"\";\n", "    var ind = \"\";\n", "    var memo_level = 1;\n", "    var href;\n", "    for (i = 0; i < anchors.length; i++) {\n", "        var child = anchors[i].children[0];\n", "        if (anchors[i].hasAttribute(\"id\")) {\n", "            href = anchors[i].id;\n", "        }\n", "        else if (child.hasAttribute(\"id\")) {\n", "            href = child.id;\n", "        }\n", "        else {\n", "            continue;\n", "        }\n", "        var title = child.textContent;\n", "        var level = parseInt(child.tagName.substring(1,2));\n", "        if ((level <= 2) || (level > 4)) {\n", "            continue ;\n", "        }\n", "        if (title.endsWith('\u00b6')) {\n", "            title = title.substring(0,title.length-1).replace(\"<\", \"&lt;\").replace(\">\", \"&gt;\").replace(\"&\", \"&amp;\")\n", "        }\n", "        if (title.length == 0) {\n", "            continue;\n", "        }\n", "        while (level > memo_level) {\n", "            text_menu += \"<ul>\\n\";\n", "            memo_level += 1;\n", "        }\n", "        while (level < memo_level) {\n", "            text_menu += \"</ul>\\n\";\n", "            memo_level -= 1;\n", "        }\n", "        text_menu += repeat_indent_string(level-2) + '<li><a href=\"#' + href + '\">' + title + '</a></li>';\n", "    }\n", "    while (1 < memo_level) {\n", "        text_menu += \"</ul>\\n\";\n", "        memo_level -= 1;\n", "    }\n", "    text_menu += \"\";\n", "    var menu = document.getElementById(\"my_menu_id\");\n", "    menu.innerHTML=text_menu;\n", "};\n", "window.setTimeout(update_menu,2000);\n", "            </script>"], "text/plain": ["<IPython.core.display.HTML object>"]}, "execution_count": 2, "metadata": {}, "output_type": "execute_result"}], "source": ["%matplotlib inline\n", "import matplotlib.pyplot as plt\n", "plt.style.use('ggplot')\n", "import pyensae\n", "from pyquickhelper.helpgen import NbImage\n", "from jyquickhelper import add_notebook_menu\n", "add_notebook_menu()"]}, {"cell_type": "markdown", "metadata": {}, "source": ["On lit les donn\u00e9es puis on recr\u00e9e un [DataSet](http://xarray.pydata.org/en/stable/data-structures.html#dataset) :"]}, {"cell_type": "code", "execution_count": 2, "metadata": {"collapsed": false}, "outputs": [{"data": {"text/plain": ["<xarray.Dataset>\n", "Dimensions:     (age_num: 84, annee: 54, genre: 3, pays: 54)\n", "Coordinates:\n", "  * annee       (annee) int64 1960 1961 1962 1963 1964 1965 1966 1967 1968 ...\n", "  * age_num     (age_num) float64 1.0 2.0 3.0 4.0 5.0 6.0 7.0 8.0 9.0 10.0 ...\n", "  * pays        (pays) object 'AM' 'AT' 'AZ' 'BE' 'BG' 'BY' 'CH' 'CY' 'CZ' ...\n", "  * genre       (genre) object 'F' 'M' 'T'\n", "Data variables:\n", "    DEATHRATE   (annee, age_num, pays, genre) float64 nan nan nan nan nan ...\n", "    LIFEXP      (annee, age_num, pays, genre) float64 nan nan nan nan nan ...\n", "    PROBDEATH   (annee, age_num, pays, genre) float64 nan nan nan nan nan ...\n", "    PROBSURV    (annee, age_num, pays, genre) float64 nan nan nan nan nan ...\n", "    PYLIVED     (annee, age_num, pays, genre) float64 nan nan nan nan nan ...\n", "    SURVIVORS   (annee, age_num, pays, genre) float64 nan nan nan nan nan ...\n", "    TOTPYLIVED  (annee, age_num, pays, genre) float64 nan nan nan nan nan ..."]}, "execution_count": 3, "metadata": {}, "output_type": "execute_result"}], "source": ["from actuariat_python.data import table_mortalite_euro_stat \n", "table_mortalite_euro_stat()\n", "import pandas\n", "df = pandas.read_csv(\"mortalite.txt\", sep=\"\\t\", encoding=\"utf8\", low_memory=False)\n", "df2 = df[[\"annee\", \"age_num\",\"indicateur\",\"pays\",\"genre\",\"valeur\"]].dropna().reset_index(drop=True)\n", "piv = df2.pivot_table(index=[\"annee\", \"age_num\",\"pays\",\"genre\"],\n", "                columns=[\"indicateur\"],\n", "               values=\"valeur\")\n", "import xarray\n", "ds = xarray.Dataset.from_dataframe(piv)\n", "ds"]}, {"cell_type": "markdown", "metadata": {}, "source": ["### Exercice 1 : que font les lignes suivantes ?\n", "\n", "Le programme suivant uilise les fonctions [align nad reindex](http://xarray.pydata.org/en/stable/data-structures.html?highlight=assign#transforming-datasets) pour faire une moyenne sur une des dimensions du DataSet (le pays) puis \u00e0 ajouter une variable *meanp* contenant le r\u00e9sultat."]}, {"cell_type": "code", "execution_count": 3, "metadata": {"collapsed": false}, "outputs": [{"data": {"text/plain": ["<xarray.Dataset>\n", "Dimensions:      (age_num: 84, annee: 54, genre: 3, pays: 54)\n", "Coordinates:\n", "  * annee        (annee) int64 1960 1961 1962 1963 1964 1965 1966 1967 1968 ...\n", "  * age_num      (age_num) float64 1.0 2.0 3.0 4.0 5.0 6.0 7.0 8.0 9.0 10.0 ...\n", "  * pays         (pays) object 'AM' 'AT' 'AZ' 'BE' 'BG' 'BY' 'CH' 'CY' 'CZ' ...\n", "  * genre        (genre) object 'F' 'M' 'T'\n", "Data variables:\n", "    DEATHRATE    (annee, age_num, pays, genre) float64 nan nan nan nan nan ...\n", "    LIFEXP       (annee, age_num, pays, genre) float64 nan nan nan nan nan ...\n", "    PROBDEATH    (annee, age_num, pays, genre) float64 nan nan nan nan nan ...\n", "    PROBSURV     (annee, age_num, pays, genre) float64 nan nan nan nan nan ...\n", "    PYLIVED      (annee, age_num, pays, genre) float64 nan nan nan nan nan ...\n", "    SURVIVORS    (annee, age_num, pays, genre) float64 nan nan nan nan nan ...\n", "    TOTPYLIVED   (annee, age_num, pays, genre) float64 nan nan nan nan nan ...\n", "    LIFEEXP_add  (annee, age_num, pays, genre) float64 nan nan nan nan nan ..."]}, "execution_count": 4, "metadata": {}, "output_type": "execute_result"}], "source": ["ds.assign(LIFEEXP_add = ds.LIFEXP-1)"]}, {"cell_type": "code", "execution_count": 4, "metadata": {"collapsed": false}, "outputs": [], "source": ["meanp = ds.mean(dim=\"pays\")\n", "ds1, ds2 = xarray.align(ds, meanp, join='outer')"]}, {"cell_type": "code", "execution_count": 5, "metadata": {"collapsed": false}, "outputs": [], "source": ["joined = ds1.assign(meanp = ds2[\"LIFEXP\"])"]}, {"cell_type": "code", "execution_count": 6, "metadata": {"collapsed": false}, "outputs": [{"data": {"text/html": ["<div>\n", "<table border=\"1\" class=\"dataframe\">\n", "  <thead>\n", "    <tr style=\"text-align: right;\">\n", "      <th></th>\n", "      <th></th>\n", "      <th></th>\n", "      <th></th>\n", "      <th>DEATHRATE</th>\n", "      <th>LIFEXP</th>\n", "      <th>PROBDEATH</th>\n", "      <th>PROBSURV</th>\n", "      <th>PYLIVED</th>\n", "      <th>SURVIVORS</th>\n", "      <th>TOTPYLIVED</th>\n", "      <th>meanp</th>\n", "    </tr>\n", "    <tr>\n", "      <th>age_num</th>\n", "      <th>annee</th>\n", "      <th>genre</th>\n", "      <th>pays</th>\n", "      <th></th>\n", "      <th></th>\n", "      <th></th>\n", "      <th></th>\n", "      <th></th>\n", "      <th></th>\n", "      <th></th>\n", "      <th></th>\n", "    </tr>\n", "  </thead>\n", "  <tbody>\n", "    <tr>\n", "      <th rowspan=\"5\" valign=\"top\">1</th>\n", "      <th rowspan=\"5\" valign=\"top\">1960</th>\n", "      <th rowspan=\"5\" valign=\"top\">F</th>\n", "      <th>AM</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "      <td>73.52</td>\n", "    </tr>\n", "    <tr>\n", "      <th>AT</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "      <td>73.52</td>\n", "    </tr>\n", "    <tr>\n", "      <th>AZ</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "      <td>73.52</td>\n", "    </tr>\n", "    <tr>\n", "      <th>BE</th>\n", "      <td>0.00159</td>\n", "      <td>73.7</td>\n", "      <td>0.00159</td>\n", "      <td>0.99841</td>\n", "      <td>97316</td>\n", "      <td>97393</td>\n", "      <td>7179465</td>\n", "      <td>73.52</td>\n", "    </tr>\n", "    <tr>\n", "      <th>BG</th>\n", "      <td>0.00652</td>\n", "      <td>73.2</td>\n", "      <td>0.00650</td>\n", "      <td>0.99350</td>\n", "      <td>95502</td>\n", "      <td>95813</td>\n", "      <td>7017023</td>\n", "      <td>73.52</td>\n", "    </tr>\n", "  </tbody>\n", "</table>\n", "</div>"], "text/plain": ["                          DEATHRATE  LIFEXP  PROBDEATH  PROBSURV  PYLIVED  \\\n", "age_num annee genre pays                                                    \n", "1       1960  F     AM          NaN     NaN        NaN       NaN      NaN   \n", "                    AT          NaN     NaN        NaN       NaN      NaN   \n", "                    AZ          NaN     NaN        NaN       NaN      NaN   \n", "                    BE      0.00159    73.7    0.00159   0.99841    97316   \n", "                    BG      0.00652    73.2    0.00650   0.99350    95502   \n", "\n", "                          SURVIVORS  TOTPYLIVED  meanp  \n", "age_num annee genre pays                                \n", "1       1960  F     AM          NaN         NaN  73.52  \n", "                    AT          NaN         NaN  73.52  \n", "                    AZ          NaN         NaN  73.52  \n", "                    BE        97393     7179465  73.52  \n", "                    BG        95813     7017023  73.52  "]}, "execution_count": 7, "metadata": {}, "output_type": "execute_result"}], "source": ["joined.to_dataframe().head()"]}, {"cell_type": "markdown", "metadata": {}, "source": ["Les valeurs *meanp* sont constantes quelque soient le pays \u00e0 *annee*, *age_num*, *genre* fix\u00e9s."]}, {"cell_type": "code", "execution_count": 7, "metadata": {"collapsed": false}, "outputs": [{"data": {"text/plain": ["<xarray.DataArray 'meanp' ()>\n", "array(23.83243243243243)\n", "Coordinates:\n", "    annee    int64 2000\n", "    genre    object 'F'\n", "    age_num  float64 59.0"]}, "execution_count": 8, "metadata": {}, "output_type": "execute_result"}], "source": ["joined.sel(annee=2000, age_num=59, genre='F')[\"meanp\"]"]}, {"cell_type": "code", "execution_count": 8, "metadata": {"collapsed": true}, "outputs": [], "source": []}], "metadata": {"kernelspec": {"display_name": "Python 3", "language": "python", "name": "python3"}, "language_info": {"codemirror_mode": {"name": "ipython", "version": 3}, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.5.2"}}, "nbformat": 4, "nbformat_minor": 2}