.. _winesmultirst:

===========================
Classification multi-classe
===========================


.. only:: html

    **Links:** :download:`notebook <wines_multi.ipynb>`, :downloadlink:`html <wines_multi2html.html>`, :download:`PDF <wines_multi.pdf>`, :download:`python <wines_multi.py>`, :downloadlink:`slides <wines_multi.slides.html>`, :githublink:`GitHub|_doc/notebooks/lectures/wines_multi.ipynb|*`


On cherche à prédire la note d’un vin avec un classifieur multi-classe.

.. code:: ipython3

    %matplotlib inline

.. code:: ipython3

    from papierstat.datasets import load_wines_dataset
    df = load_wines_dataset()
    X = df.drop(['quality', 'color'], axis=1)
    y = df['quality']

.. code:: ipython3

    from sklearn.model_selection import train_test_split
    X_train, X_test, y_train, y_test = train_test_split(X, y)

.. code:: ipython3

    from sklearn.linear_model import LogisticRegression
    clr = LogisticRegression()
    clr.fit(X_train, y_train)




.. parsed-literal::
    LogisticRegression(C=1.0, class_weight=None, dual=False, fit_intercept=True,
              intercept_scaling=1, max_iter=100, multi_class='ovr', n_jobs=1,
              penalty='l2', random_state=None, solver='liblinear', tol=0.0001,
              verbose=0, warm_start=False)



.. code:: ipython3

    import numpy
    numpy.mean(clr.predict(X_test).ravel() == y_test.ravel()) * 100




.. parsed-literal::
    53.84615384615385



On regarde la matrice de confusion.

.. code:: ipython3

    from sklearn.metrics import confusion_matrix
    import pandas
    pandas.DataFrame(confusion_matrix(y_test, clr.predict(X_test)))






.. raw:: html

    <div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }

        .dataframe tbody tr th {
            vertical-align: top;
        }

        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th></th>
          <th>0</th>
          <th>1</th>
          <th>2</th>
          <th>3</th>
          <th>4</th>
          <th>5</th>
          <th>6</th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th>0</th>
          <td>0</td>
          <td>0</td>
          <td>6</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
        </tr>
        <tr>
          <th>1</th>
          <td>0</td>
          <td>0</td>
          <td>39</td>
          <td>14</td>
          <td>1</td>
          <td>0</td>
          <td>0</td>
        </tr>
        <tr>
          <th>2</th>
          <td>0</td>
          <td>0</td>
          <td>338</td>
          <td>208</td>
          <td>2</td>
          <td>0</td>
          <td>0</td>
        </tr>
        <tr>
          <th>3</th>
          <td>0</td>
          <td>0</td>
          <td>195</td>
          <td>517</td>
          <td>17</td>
          <td>0</td>
          <td>0</td>
        </tr>
        <tr>
          <th>4</th>
          <td>0</td>
          <td>0</td>
          <td>19</td>
          <td>200</td>
          <td>20</td>
          <td>0</td>
          <td>0</td>
        </tr>
        <tr>
          <th>5</th>
          <td>0</td>
          <td>0</td>
          <td>2</td>
          <td>38</td>
          <td>8</td>
          <td>0</td>
          <td>0</td>
        </tr>
        <tr>
          <th>6</th>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>1</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
        </tr>
      </tbody>
    </table>
    </div>



On l’affiche différemment avec le nom des classes.

.. code:: ipython3

    conf = confusion_matrix(y_test, clr.predict(X_test))
    dfconf = pandas.DataFrame(conf)
    labels = list(clr.classes_)
    if len(labels) < dfconf.shape[1]:
        labels += [9]  # La classe 9 est très représentée, elle est parfois absente en train.
    elif len(labels) > dfconf.shape[1]:
        labels = labels[:dfconf.shape[1]] # ou l'inverse
    dfconf.columns = labels
    dfconf.index = labels
    dfconf






.. raw:: html

    <div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }

        .dataframe tbody tr th {
            vertical-align: top;
        }

        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th></th>
          <th>3</th>
          <th>4</th>
          <th>5</th>
          <th>6</th>
          <th>7</th>
          <th>8</th>
          <th>9</th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th>3</th>
          <td>0</td>
          <td>0</td>
          <td>6</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
        </tr>
        <tr>
          <th>4</th>
          <td>0</td>
          <td>0</td>
          <td>39</td>
          <td>14</td>
          <td>1</td>
          <td>0</td>
          <td>0</td>
        </tr>
        <tr>
          <th>5</th>
          <td>0</td>
          <td>0</td>
          <td>338</td>
          <td>208</td>
          <td>2</td>
          <td>0</td>
          <td>0</td>
        </tr>
        <tr>
          <th>6</th>
          <td>0</td>
          <td>0</td>
          <td>195</td>
          <td>517</td>
          <td>17</td>
          <td>0</td>
          <td>0</td>
        </tr>
        <tr>
          <th>7</th>
          <td>0</td>
          <td>0</td>
          <td>19</td>
          <td>200</td>
          <td>20</td>
          <td>0</td>
          <td>0</td>
        </tr>
        <tr>
          <th>8</th>
          <td>0</td>
          <td>0</td>
          <td>2</td>
          <td>38</td>
          <td>8</td>
          <td>0</td>
          <td>0</td>
        </tr>
        <tr>
          <th>9</th>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>1</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
        </tr>
      </tbody>
    </table>
    </div>



Pas extraordinaire. On applique la stratégie
`OneVsRestClassifier <http://scikit-learn.org/stable/modules/generated/sklearn.multiclass.OneVsRestClassifier.html>`__.

.. code:: ipython3

    from sklearn.multiclass import OneVsRestClassifier
    clr = OneVsRestClassifier(LogisticRegression())
    clr.fit(X_train, y_train)




.. parsed-literal::
    OneVsRestClassifier(estimator=LogisticRegression(C=1.0, class_weight=None, dual=False, fit_intercept=True,
              intercept_scaling=1, max_iter=100, multi_class='ovr', n_jobs=1,
              penalty='l2', random_state=None, solver='liblinear', tol=0.0001,
              verbose=0, warm_start=False),
              n_jobs=1)



.. code:: ipython3

    numpy.mean(clr.predict(X_test).ravel() == y_test.ravel()) * 100




.. parsed-literal::
    53.784615384615385



Le modèle logistique régression multi-classe est équivalent à la
stratégie *OneVsRest*. Voyons l’autre.

.. code:: ipython3

    from sklearn.multiclass import OneVsOneClassifier
    clr = OneVsOneClassifier(LogisticRegression())
    clr.fit(X_train, y_train)




.. parsed-literal::
    OneVsOneClassifier(estimator=LogisticRegression(C=1.0, class_weight=None, dual=False, fit_intercept=True,
              intercept_scaling=1, max_iter=100, multi_class='ovr', n_jobs=1,
              penalty='l2', random_state=None, solver='liblinear', tol=0.0001,
              verbose=0, warm_start=False),
              n_jobs=1)



.. code:: ipython3

    numpy.mean(clr.predict(X_test).ravel() == y_test.ravel()) * 100




.. parsed-literal::
    53.47692307692308



.. code:: ipython3

    conf = confusion_matrix(y_test, clr.predict(X_test))
    dfconf = pandas.DataFrame(conf)
    labels = list(clr.classes_)
    if len(labels) < dfconf.shape[1]:
        labels += [9]  # La classe 9 est très représentée, elle est parfois absente en train.
    elif len(labels) > dfconf.shape[1]:
        labels = labels[:dfconf.shape[1]] # ou l'inverse
    dfconf.columns = labels
    dfconf.index = labels
    dfconf






.. raw:: html

    <div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }

        .dataframe tbody tr th {
            vertical-align: top;
        }

        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th></th>
          <th>3</th>
          <th>4</th>
          <th>5</th>
          <th>6</th>
          <th>7</th>
          <th>8</th>
          <th>9</th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th>3</th>
          <td>0</td>
          <td>0</td>
          <td>6</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
        </tr>
        <tr>
          <th>4</th>
          <td>0</td>
          <td>0</td>
          <td>38</td>
          <td>15</td>
          <td>1</td>
          <td>0</td>
          <td>0</td>
        </tr>
        <tr>
          <th>5</th>
          <td>0</td>
          <td>0</td>
          <td>335</td>
          <td>208</td>
          <td>5</td>
          <td>0</td>
          <td>0</td>
        </tr>
        <tr>
          <th>6</th>
          <td>0</td>
          <td>0</td>
          <td>197</td>
          <td>491</td>
          <td>41</td>
          <td>0</td>
          <td>0</td>
        </tr>
        <tr>
          <th>7</th>
          <td>0</td>
          <td>0</td>
          <td>20</td>
          <td>176</td>
          <td>43</td>
          <td>0</td>
          <td>0</td>
        </tr>
        <tr>
          <th>8</th>
          <td>0</td>
          <td>0</td>
          <td>1</td>
          <td>34</td>
          <td>13</td>
          <td>0</td>
          <td>0</td>
        </tr>
        <tr>
          <th>9</th>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>1</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
        </tr>
      </tbody>
    </table>
    </div>



A peu près pareil mais sans doute pas de manière significative. Voyons
avec un arbre de décision.

.. code:: ipython3

    from sklearn.tree import DecisionTreeClassifier
    clr = DecisionTreeClassifier()
    clr.fit(X_train, y_train)




.. parsed-literal::
    DecisionTreeClassifier(class_weight=None, criterion='gini', max_depth=None,
                max_features=None, max_leaf_nodes=None,
                min_impurity_decrease=0.0, min_impurity_split=None,
                min_samples_leaf=1, min_samples_split=2,
                min_weight_fraction_leaf=0.0, presort=False, random_state=None,
                splitter='best')



.. code:: ipython3

    numpy.mean(clr.predict(X_test).ravel() == y_test.ravel()) * 100




.. parsed-literal::
    59.50769230769231



Et avec
`OneVsRestClassifier <http://scikit-learn.org/stable/modules/generated/sklearn.multiclass.OneVsRestClassifier.html>`__
:

.. code:: ipython3

    clr = OneVsRestClassifier(DecisionTreeClassifier())
    clr.fit(X_train, y_train)




.. parsed-literal::
    OneVsRestClassifier(estimator=DecisionTreeClassifier(class_weight=None, criterion='gini', max_depth=None,
                max_features=None, max_leaf_nodes=None,
                min_impurity_decrease=0.0, min_impurity_split=None,
                min_samples_leaf=1, min_samples_split=2,
                min_weight_fraction_leaf=0.0, presort=False, random_state=None,
                splitter='best'),
              n_jobs=1)



.. code:: ipython3

    numpy.mean(clr.predict(X_test).ravel() == y_test.ravel()) * 100




.. parsed-literal::
    52.92307692307693



Et avec
`OneVsOneClassifier <http://scikit-learn.org/stable/modules/generated/sklearn.multiclass.OneVsOneClassifier.html>`__

.. code:: ipython3

    clr = OneVsOneClassifier(DecisionTreeClassifier())
    clr.fit(X_train, y_train)




.. parsed-literal::
    OneVsOneClassifier(estimator=DecisionTreeClassifier(class_weight=None, criterion='gini', max_depth=None,
                max_features=None, max_leaf_nodes=None,
                min_impurity_decrease=0.0, min_impurity_split=None,
                min_samples_leaf=1, min_samples_split=2,
                min_weight_fraction_leaf=0.0, presort=False, random_state=None,
                splitter='best'),
              n_jobs=1)



.. code:: ipython3

    numpy.mean(clr.predict(X_test).ravel() == y_test.ravel()) * 100




.. parsed-literal::
    60.12307692307692



Mieux.

.. code:: ipython3

    from sklearn.ensemble import RandomForestClassifier
    clr = RandomForestClassifier()
    clr.fit(X_train, y_train)




.. parsed-literal::
    RandomForestClassifier(bootstrap=True, class_weight=None, criterion='gini',
                max_depth=None, max_features='auto', max_leaf_nodes=None,
                min_impurity_decrease=0.0, min_impurity_split=None,
                min_samples_leaf=1, min_samples_split=2,
                min_weight_fraction_leaf=0.0, n_estimators=10, n_jobs=1,
                oob_score=False, random_state=None, verbose=0,
                warm_start=False)



.. code:: ipython3

    numpy.mean(clr.predict(X_test).ravel() == y_test.ravel()) * 100




.. parsed-literal::
    66.46153846153847



.. code:: ipython3

    clr = OneVsRestClassifier(RandomForestClassifier())
    clr.fit(X_train, y_train)




.. parsed-literal::
    OneVsRestClassifier(estimator=RandomForestClassifier(bootstrap=True, class_weight=None, criterion='gini',
                max_depth=None, max_features='auto', max_leaf_nodes=None,
                min_impurity_decrease=0.0, min_impurity_split=None,
                min_samples_leaf=1, min_samples_split=2,
                min_weight_fraction_leaf=0.0, n_estimators=10, n_jobs=1,
                oob_score=False, random_state=None, verbose=0,
                warm_start=False),
              n_jobs=1)



.. code:: ipython3

    numpy.mean(clr.predict(X_test).ravel() == y_test.ravel()) * 100




.. parsed-literal::
    65.90769230769232



Proche, il faut affiner avec une validation croisée.

.. code:: ipython3

    from sklearn.neural_network import MLPClassifier
    clr = MLPClassifier(hidden_layer_sizes=30, max_iter=600)
    clr.fit(X_train, y_train)




.. parsed-literal::
    MLPClassifier(activation='relu', alpha=0.0001, batch_size='auto', beta_1=0.9,
           beta_2=0.999, early_stopping=False, epsilon=1e-08,
           hidden_layer_sizes=30, learning_rate='constant',
           learning_rate_init=0.001, max_iter=600, momentum=0.9,
           nesterovs_momentum=True, power_t=0.5, random_state=None,
           shuffle=True, solver='adam', tol=0.0001, validation_fraction=0.1,
           verbose=False, warm_start=False)



.. code:: ipython3

    numpy.mean(clr.predict(X_test).ravel() == y_test.ravel()) * 100




.. parsed-literal::
    51.323076923076925



.. code:: ipython3

    clr = OneVsRestClassifier(MLPClassifier(hidden_layer_sizes=30, max_iter=600))
    clr.fit(X_train, y_train)




.. parsed-literal::
    OneVsRestClassifier(estimator=MLPClassifier(activation='relu', alpha=0.0001, batch_size='auto', beta_1=0.9,
           beta_2=0.999, early_stopping=False, epsilon=1e-08,
           hidden_layer_sizes=30, learning_rate='constant',
           learning_rate_init=0.001, max_iter=600, momentum=0.9,
           nesterovs_momentum=True, power_t=0.5, random_state=None,
           shuffle=True, solver='adam', tol=0.0001, validation_fraction=0.1,
           verbose=False, warm_start=False),
              n_jobs=1)



.. code:: ipython3

    numpy.mean(clr.predict(X_test).ravel() == y_test.ravel()) * 100




.. parsed-literal::
    47.56923076923077



Pas foudroyant.