.. _correlationnonlineairerst:

==========================
Corrélations non linéaires
==========================


.. only:: html

    **Links:** :download:`notebook <correlation_non_lineaire.ipynb>`, :downloadlink:`html <correlation_non_lineaire2html.html>`, :download:`PDF <correlation_non_lineaire.pdf>`, :download:`python <correlation_non_lineaire.py>`, :downloadlink:`slides <correlation_non_lineaire.slides.html>`, :githublink:`GitHub|_doc/notebooks/dsgarden/correlation_non_lineaire.ipynb|*`


Les corrélations indiquent si deux variables sont linéairement
équivalentes. Comment étendre cette notion à des variables liées mais
pas de façon linéaire.

.. code:: ipython3

    from jyquickhelper import add_notebook_menu
    add_notebook_menu()






.. contents::
    :local:





.. code:: ipython3

    %matplotlib inline

Un exemple
----------

.. code:: ipython3

    from sklearn import datasets
    
    iris = datasets.load_iris()
    X = iris.data
    Y = iris.target
    import pandas
    df = pandas.DataFrame(X)
    df.columns = ["X1", "X2", "X3", "X4"]
    df.head()






.. raw:: html

    <div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }

        .dataframe tbody tr th {
            vertical-align: top;
        }

        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th></th>
          <th>X1</th>
          <th>X2</th>
          <th>X3</th>
          <th>X4</th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th>0</th>
          <td>5.1</td>
          <td>3.5</td>
          <td>1.4</td>
          <td>0.2</td>
        </tr>
        <tr>
          <th>1</th>
          <td>4.9</td>
          <td>3.0</td>
          <td>1.4</td>
          <td>0.2</td>
        </tr>
        <tr>
          <th>2</th>
          <td>4.7</td>
          <td>3.2</td>
          <td>1.3</td>
          <td>0.2</td>
        </tr>
        <tr>
          <th>3</th>
          <td>4.6</td>
          <td>3.1</td>
          <td>1.5</td>
          <td>0.2</td>
        </tr>
        <tr>
          <th>4</th>
          <td>5.0</td>
          <td>3.6</td>
          <td>1.4</td>
          <td>0.2</td>
        </tr>
      </tbody>
    </table>
    </div>



.. code:: ipython3

    import seaborn as sns
    sns.set()
    sns.pairplot(df);



.. image:: correlation_non_lineaire_5_0.png


Et les corrélations :

.. code:: ipython3

    df.corr()






.. raw:: html

    <div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }

        .dataframe tbody tr th {
            vertical-align: top;
        }

        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th></th>
          <th>X1</th>
          <th>X2</th>
          <th>X3</th>
          <th>X4</th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th>X1</th>
          <td>1.000000</td>
          <td>-0.117570</td>
          <td>0.871754</td>
          <td>0.817941</td>
        </tr>
        <tr>
          <th>X2</th>
          <td>-0.117570</td>
          <td>1.000000</td>
          <td>-0.428440</td>
          <td>-0.366126</td>
        </tr>
        <tr>
          <th>X3</th>
          <td>0.871754</td>
          <td>-0.428440</td>
          <td>1.000000</td>
          <td>0.962865</td>
        </tr>
        <tr>
          <th>X4</th>
          <td>0.817941</td>
          <td>-0.366126</td>
          <td>0.962865</td>
          <td>1.000000</td>
        </tr>
      </tbody>
    </table>
    </div>



Un peu de théorie
-----------------

Le coefficient de
`corrélation <https://fr.wikipedia.org/wiki/Corr%C3%A9lation_(statistiques)>`__
de Pearson est calculé comme suit :

.. math:: cor(X_i, X_j) = \frac{cov(X_i, Y_i)}{\sigma(X_i)\sigma(X_j)}

Lorsque les variables sont centrées
:math:`\mathbb{E}X_i=\mathbb{E}X_j=0`, cette formule devient :

.. math:: cor(X_i, X_j) = \frac{\mathbb{E}(X_i X_j)}{\sqrt{\mathbb{E}X_i^2 \mathbb{E}X_j^2}}

Lorsque les variables sont réduites
:math:`\mathbb{E}X_i^2=\mathbb{E}X_j^2=1`, cette formule devient
:math:`cor(X_i, X_j) = \mathbb{E}(X_i X_j)`. Admettons maintenant que
l’on cherche à trouver le coefficient :math:`\alpha_{ij}` qui minimise
la variance du bruit :math:`\epsilon_{ij}` :

.. math:: X_j = \alpha_{ij}X_i + \epsilon_{ij}

Le coefficient :math:`\alpha_{ij}` est le résultat d’une régression
linéaire qui minimise :math:`\mathbb{E}(X_j - \alpha_{ij}X_i)^2`. Si les
variables :math:`X_i`, :math:`X_j` sont centrées et réduites :
:math:`\alpha_{ij}^* = \mathbb{E}(X_i X_j) = cor(X_i, X_j)`. On étend
cette définition dans le cas d’une fonction paramétrable :math:`f` :
:math:`f(\omega, X) \rightarrow \mathbb{R}` et d’une régression non
linéaire. On suppose que les paramètres :math:`\omega^*` minimisent la
quantité :math:`\min_\omega (X_j - f(\omega, X_i))^2`. On écrit alors
:math:`X_j = \alpha_{ij} \frac{f(\omega^*, X_i)}{\alpha_{ij}} + \epsilon_{ij}`
et on choisit :math:`\alpha_{ij}` de telle sorte que
:math:`\mathbb{E}\left(\frac{f(\omega^*, X_i)^2}{\alpha_{ij}^2}\right) = 1`.
On définit la corrélation non linéaire au sens de :math:`f` :

.. math:: cor^f(X_i, X_j) = \sqrt{ \mathbb{E} (f(\omega, X_i)^2 )}

On vérifie que ce coefficient est compris entre [0, 1]. Il est positif
de manière évidente. Il est également inférieur à 1, si cela n’était pas
le cas, nous pourrions construire une fonction :math:`f(\omega^*, X)+c`
qui est une meilleur solution pour le programme de minimisation. Ce
nombre ressemble à une corrélation à ceci près qu’elle ne peut être
négative.

Vérifications
-------------

Tout d’abord le cas linéaire :

.. code:: ipython3

    from sklearn.preprocessing import scale
    import numpy
    
    def correlation_etendue(df, model, **params):
        cor = df.corr()
        df = scale(df)    
        for i in range(cor.shape[0]):
            xi = df[:, i:i+1]
            for j in range(cor.shape[1]):
                mod = model(**params)
                xj = df[:, j]
                mod.fit(xi, xj)
                v = mod.predict(xi)
                c = numpy.std(v)
                cor.iloc[i,j] = c
        return cor
    
    from sklearn.linear_model import LinearRegression
    cor = correlation_etendue(df, LinearRegression, fit_intercept=False)
    cor






.. raw:: html

    <div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }

        .dataframe tbody tr th {
            vertical-align: top;
        }

        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th></th>
          <th>X1</th>
          <th>X2</th>
          <th>X3</th>
          <th>X4</th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th>X1</th>
          <td>1.000000</td>
          <td>0.117570</td>
          <td>0.871754</td>
          <td>0.817941</td>
        </tr>
        <tr>
          <th>X2</th>
          <td>0.117570</td>
          <td>1.000000</td>
          <td>0.428440</td>
          <td>0.366126</td>
        </tr>
        <tr>
          <th>X3</th>
          <td>0.871754</td>
          <td>0.428440</td>
          <td>1.000000</td>
          <td>0.962865</td>
        </tr>
        <tr>
          <th>X4</th>
          <td>0.817941</td>
          <td>0.366126</td>
          <td>0.962865</td>
          <td>1.000000</td>
        </tr>
      </tbody>
    </table>
    </div>



On affiche à nouveau les corrélations qui sont identiques au signe près.

.. code:: ipython3

    df.corr()






.. raw:: html

    <div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }

        .dataframe tbody tr th {
            vertical-align: top;
        }

        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th></th>
          <th>X1</th>
          <th>X2</th>
          <th>X3</th>
          <th>X4</th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th>X1</th>
          <td>1.000000</td>
          <td>-0.117570</td>
          <td>0.871754</td>
          <td>0.817941</td>
        </tr>
        <tr>
          <th>X2</th>
          <td>-0.117570</td>
          <td>1.000000</td>
          <td>-0.428440</td>
          <td>-0.366126</td>
        </tr>
        <tr>
          <th>X3</th>
          <td>0.871754</td>
          <td>-0.428440</td>
          <td>1.000000</td>
          <td>0.962865</td>
        </tr>
        <tr>
          <th>X4</th>
          <td>0.817941</td>
          <td>-0.366126</td>
          <td>0.962865</td>
          <td>1.000000</td>
        </tr>
      </tbody>
    </table>
    </div>



Et le cas non linéaire :

.. code:: ipython3

    from sklearn.tree import DecisionTreeRegressor
    cor = correlation_etendue(df, DecisionTreeRegressor)
    cor






.. raw:: html

    <div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }

        .dataframe tbody tr th {
            vertical-align: top;
        }

        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th></th>
          <th>X1</th>
          <th>X2</th>
          <th>X3</th>
          <th>X4</th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th>X1</th>
          <td>1.000000</td>
          <td>0.544441</td>
          <td>0.915847</td>
          <td>0.879583</td>
        </tr>
        <tr>
          <th>X2</th>
          <td>0.418274</td>
          <td>1.000000</td>
          <td>0.591839</td>
          <td>0.539524</td>
        </tr>
        <tr>
          <th>X3</th>
          <td>0.937056</td>
          <td>0.789727</td>
          <td>1.000000</td>
          <td>0.978332</td>
        </tr>
        <tr>
          <th>X4</th>
          <td>0.846161</td>
          <td>0.761652</td>
          <td>0.980005</td>
          <td>1.000000</td>
        </tr>
      </tbody>
    </table>
    </div>



.. code:: ipython3

    from sklearn.ensemble import RandomForestRegressor
    cor = correlation_etendue(df, RandomForestRegressor, n_estimators=10)
    cor






.. raw:: html

    <div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }

        .dataframe tbody tr th {
            vertical-align: top;
        }

        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th></th>
          <th>X1</th>
          <th>X2</th>
          <th>X3</th>
          <th>X4</th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th>X1</th>
          <td>0.998300</td>
          <td>0.557325</td>
          <td>0.924134</td>
          <td>0.871019</td>
        </tr>
        <tr>
          <th>X2</th>
          <td>0.413502</td>
          <td>0.992939</td>
          <td>0.593398</td>
          <td>0.544801</td>
        </tr>
        <tr>
          <th>X3</th>
          <td>0.922292</td>
          <td>0.786128</td>
          <td>0.999064</td>
          <td>0.975211</td>
        </tr>
        <tr>
          <th>X4</th>
          <td>0.846106</td>
          <td>0.756621</td>
          <td>0.981181</td>
          <td>0.999269</td>
        </tr>
      </tbody>
    </table>
    </div>



Overfitting
-----------

Ces chiffres sont beaucoup trop optimistes. Les modèles de machine
learning peuvent tout à fait faire de l’overfitting. Il faut améliorer
la fonction en divisant en apprentissage et test plusieurs fois. Il faut
également tenir compte de l’erreur de prédiction. On rappelle que :

.. math:: X_j = \alpha_{ij} \frac{f(\omega^*, X_i)}{\alpha_{ij}} + \epsilon_{ij} = cor^f(X_i, X_j) \frac{f(\omega^*, X_i)}{\sqrt{ \mathbb{E} (f(\omega, X_i)^2 )}} + \epsilon_{ij}

Or :math:`\mathbb{E}(X_j^2)=1` et on suppose que les bruits ne sont pas
corrélées linéairement aux :math:`f(\omega^*, X_i)`. On en déduit que
:math:`cor^f(X_i, X_j) = \sqrt{ 1 - \mathbb{E}\epsilon_{ij}^2}`.

.. code:: ipython3

    from sklearn.model_selection import train_test_split
    import numpy
    
    def correlation_cross_val(df, model, draws=5, **params):
        cor = df.corr()
        df = scale(df)    
        for i in range(cor.shape[0]):
            xi = df[:, i:i+1]
            for j in range(cor.shape[1]):
                xj = df[:, j]
                mem = []
                for k in range(0, draws):
                    xi_train, xi_test, xj_train, xj_test = train_test_split(xi, xj, test_size=0.5)
                    mod = model(**params)            
                    mod.fit(xi_train, xj_train)
                    v = mod.predict(xi_test)
                    c = (1 - numpy.var(v - xj_test))
                    mem.append(max(c, 0) **0.5)
                cor.iloc[i,j] = sum(mem) / len(mem)
        return cor
    
    cor = correlation_cross_val(df, LinearRegression, fit_intercept=False, draws=20)
    cor






.. raw:: html

    <div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }

        .dataframe tbody tr th {
            vertical-align: top;
        }

        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th></th>
          <th>X1</th>
          <th>X2</th>
          <th>X3</th>
          <th>X4</th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th>X1</th>
          <td>1.000000</td>
          <td>0.067090</td>
          <td>0.878802</td>
          <td>0.817301</td>
        </tr>
        <tr>
          <th>X2</th>
          <td>0.156402</td>
          <td>1.000000</td>
          <td>0.355925</td>
          <td>0.347753</td>
        </tr>
        <tr>
          <th>X3</th>
          <td>0.878156</td>
          <td>0.421116</td>
          <td>1.000000</td>
          <td>0.963497</td>
        </tr>
        <tr>
          <th>X4</th>
          <td>0.808738</td>
          <td>0.378046</td>
          <td>0.961512</td>
          <td>1.000000</td>
        </tr>
      </tbody>
    </table>
    </div>



.. code:: ipython3

    cor = correlation_cross_val(df, DecisionTreeRegressor)
    cor






.. raw:: html

    <div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }

        .dataframe tbody tr th {
            vertical-align: top;
        }

        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th></th>
          <th>X1</th>
          <th>X2</th>
          <th>X3</th>
          <th>X4</th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th>X1</th>
          <td>0.999140</td>
          <td>0.000000</td>
          <td>0.840661</td>
          <td>0.792892</td>
        </tr>
        <tr>
          <th>X2</th>
          <td>0.000000</td>
          <td>0.995369</td>
          <td>0.268502</td>
          <td>0.202563</td>
        </tr>
        <tr>
          <th>X3</th>
          <td>0.864010</td>
          <td>0.492400</td>
          <td>0.999257</td>
          <td>0.955225</td>
        </tr>
        <tr>
          <th>X4</th>
          <td>0.715695</td>
          <td>0.539786</td>
          <td>0.968522</td>
          <td>0.999316</td>
        </tr>
      </tbody>
    </table>
    </div>



.. code:: ipython3

    cor = correlation_cross_val(df, RandomForestRegressor, n_estimators=10)
    cor






.. raw:: html

    <div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }

        .dataframe tbody tr th {
            vertical-align: top;
        }

        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th></th>
          <th>X1</th>
          <th>X2</th>
          <th>X3</th>
          <th>X4</th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th>X1</th>
          <td>0.998919</td>
          <td>0.048290</td>
          <td>0.872549</td>
          <td>0.794800</td>
        </tr>
        <tr>
          <th>X2</th>
          <td>0.064102</td>
          <td>0.996025</td>
          <td>0.209190</td>
          <td>0.201094</td>
        </tr>
        <tr>
          <th>X3</th>
          <td>0.875400</td>
          <td>0.481419</td>
          <td>0.998478</td>
          <td>0.958236</td>
        </tr>
        <tr>
          <th>X4</th>
          <td>0.733457</td>
          <td>0.656183</td>
          <td>0.967662</td>
          <td>0.999501</td>
        </tr>
      </tbody>
    </table>
    </div>



Les résultats sont assez fluctuants lorsque les données sont mal
corrélées. On remarque également que la matrice n’est plus
nécessairement symmétrique.

.. code:: ipython3

    import matplotlib.pyplot as plt
    
    def pairplot_cross_val(data, model=None, ax=None, **params):
        if ax is None:
            fig, ax = plt.subplots(data.shape[1], data.shape[1], figsize=params.get('figsize', (10,10)))
        if 'figsize' in params:
            del params["figsize"]
        if model is None:
            from sklearn.linear_model import LinearRegression
            model = LinearRegression
        
        df = scale(data)
        cor = numpy.corrcoef(df.T)
        for i in range(cor.shape[0]):
            xi = df[:, i:i+1]
            for j in range(cor.shape[1]):
                xj = df[:, j]
                mem = []
                xi_train, xi_test, xj_train, xj_test = train_test_split(xi, xj, test_size=0.5)
                mod = model(**params)            
                mod.fit(xi_train, xj_train)
                v = mod.predict(xi_test)
                mod = model(**params)            
                mod.fit(xi_test, xj_test)
                v2 = mod.predict(xi_train)
                ax[i,j].plot(xj_test, v, ".")
                ax[i,j].plot(xj_train, v2, ".")
                if j == 0:
                    ax[i,j].set_ylabel(data.columns[i])
                if i == data.shape[1]-1:
                    ax[i,j].set_xlabel(data.columns[j])
                mi = min(min(xj_test), min(v), min(xj_train), min(v2))
                ma = max(max(xj_test), max(v), max(xj_train), max(v2))
                ax[i,j].plot([mi, ma], [mi, ma], "--")
        return ax
                
    ax = pairplot_cross_val(df)
    ax;



.. image:: correlation_non_lineaire_22_0.png


.. code:: ipython3

    ax = pairplot_cross_val(df, model=DecisionTreeRegressor)
    ax;



.. image:: correlation_non_lineaire_23_0.png


.. code:: ipython3

    ax = pairplot_cross_val(df, model=RandomForestRegressor, n_estimators=10)
    ax;



.. image:: correlation_non_lineaire_24_0.png


.. code:: ipython3

    from sklearn.neighbors import KNeighborsRegressor
    ax = pairplot_cross_val(df, model=KNeighborsRegressor)
    ax;



.. image:: correlation_non_lineaire_25_0.png


Corrélations de variables catégorielles
---------------------------------------

C’est le problème épineux si on se restreint au linéaire. Cela n’a pas
trop de sens d’affecter une valeur à chaque catégorie et la corrélation
de deux variables binaires (des modalités) est toujours étrange car il
n’y a que deux valeurs possibles.

.. math:: cov(X,Y) = \mathbb{E}\left[(X - \mathbb{E}X)(Y - \mathbb{E}Y)\right] = \mathbb{E}(XY) - \mathbb{E}X\mathbb{E}Y = \mathbb{P}(X=1 \, et \, Y=1) - \mathbb{E}X\mathbb{E}Y

Dans le cas de variables binaires générées de modalités de la même
variables catégorielles, le premier terme est toujours nul puisque les
modalités sont exclusives et la corrélation est toujours négative.

.. code:: ipython3

    import random
    ex = numpy.zeros((100, 2))
    for i in range(0, ex.shape[0]):
        h = random.randint(0, ex.shape[1]-1)
        ex[i, h] = 1
    ex[:5]




.. parsed-literal::
    array([[0., 1.],
           [1., 0.],
           [1., 0.],
           [0., 1.],
           [0., 1.]])



.. code:: ipython3

    numpy.corrcoef(ex.T)




.. parsed-literal::
    array([[ 1., -1.],
           [-1.,  1.]])



.. code:: ipython3

    import random
    ex = numpy.zeros((100, 3))
    for i in range(0, ex.shape[0]):
        h = random.randint(0, ex.shape[1]-1)
        ex[i, h] = 1
    ex[:5]
    numpy.corrcoef(ex.T)




.. parsed-literal::
    array([[ 1.        , -0.45760432, -0.47790696],
           [-0.45760432,  1.        , -0.56235159],
           [-0.47790696, -0.56235159,  1.        ]])



Supposons maintenant que nous avons deux variables catégorielles très
proches :


-  :math:`X_1` est une couleur rouge, bleu, gris.
-  :math:`X_2` est une nuance rose, orange, cyan, magenta, blanc noir.

.. code:: ipython3

    c1 = ["rouge", "bleu", "gris"]
    c2 = ["rose" ,"orange" ,"cyan" ,"magenta", "blanc", "noir"]
    ind = [random.randint(0, 2) for i in range(0, 100)]
    x1 = [c1[i] for i in ind]
    x2 = [c2[i*2 + random.randint(0,1)] for i in ind]
    df = pandas.DataFrame(dict(X1=x1, X2=x2))
    df.head()






.. raw:: html

    <div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }

        .dataframe tbody tr th {
            vertical-align: top;
        }

        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th></th>
          <th>X1</th>
          <th>X2</th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th>0</th>
          <td>rouge</td>
          <td>rose</td>
        </tr>
        <tr>
          <th>1</th>
          <td>gris</td>
          <td>noir</td>
        </tr>
        <tr>
          <th>2</th>
          <td>bleu</td>
          <td>magenta</td>
        </tr>
        <tr>
          <th>3</th>
          <td>bleu</td>
          <td>magenta</td>
        </tr>
        <tr>
          <th>4</th>
          <td>bleu</td>
          <td>magenta</td>
        </tr>
      </tbody>
    </table>
    </div>



On peut évidemment transformer en entier.

.. code:: ipython3

    dummies = pandas.get_dummies(df)
    dummies.head()






.. raw:: html

    <div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }

        .dataframe tbody tr th {
            vertical-align: top;
        }

        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th></th>
          <th>X1_bleu</th>
          <th>X1_gris</th>
          <th>X1_rouge</th>
          <th>X2_blanc</th>
          <th>X2_cyan</th>
          <th>X2_magenta</th>
          <th>X2_noir</th>
          <th>X2_orange</th>
          <th>X2_rose</th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th>0</th>
          <td>0</td>
          <td>0</td>
          <td>1</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>1</td>
        </tr>
        <tr>
          <th>1</th>
          <td>0</td>
          <td>1</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>1</td>
          <td>0</td>
          <td>0</td>
        </tr>
        <tr>
          <th>2</th>
          <td>1</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>1</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
        </tr>
        <tr>
          <th>3</th>
          <td>1</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>1</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
        </tr>
        <tr>
          <th>4</th>
          <td>1</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>1</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
        </tr>
      </tbody>
    </table>
    </div>



.. code:: ipython3

    dummies.corr()






.. raw:: html

    <div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }

        .dataframe tbody tr th {
            vertical-align: top;
        }

        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th></th>
          <th>X1_bleu</th>
          <th>X1_gris</th>
          <th>X1_rouge</th>
          <th>X2_blanc</th>
          <th>X2_cyan</th>
          <th>X2_magenta</th>
          <th>X2_noir</th>
          <th>X2_orange</th>
          <th>X2_rose</th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th>X1_bleu</th>
          <td>1.000000</td>
          <td>-0.548514</td>
          <td>-0.511019</td>
          <td>-0.399795</td>
          <td>0.545824</td>
          <td>0.683520</td>
          <td>-0.295268</td>
          <td>-0.309086</td>
          <td>-0.348970</td>
        </tr>
        <tr>
          <th>X1_gris</th>
          <td>-0.548514</td>
          <td>1.000000</td>
          <td>-0.438420</td>
          <td>0.728869</td>
          <td>-0.299392</td>
          <td>-0.374920</td>
          <td>0.538305</td>
          <td>-0.265175</td>
          <td>-0.299392</td>
        </tr>
        <tr>
          <th>X1_rouge</th>
          <td>-0.511019</td>
          <td>-0.438420</td>
          <td>1.000000</td>
          <td>-0.319551</td>
          <td>-0.278927</td>
          <td>-0.349292</td>
          <td>-0.236004</td>
          <td>0.604842</td>
          <td>0.682890</td>
        </tr>
        <tr>
          <th>X2_blanc</th>
          <td>-0.399795</td>
          <td>0.728869</td>
          <td>-0.319551</td>
          <td>1.000000</td>
          <td>-0.218218</td>
          <td>-0.273268</td>
          <td>-0.184637</td>
          <td>-0.193278</td>
          <td>-0.218218</td>
        </tr>
        <tr>
          <th>X2_cyan</th>
          <td>0.545824</td>
          <td>-0.299392</td>
          <td>-0.278927</td>
          <td>-0.218218</td>
          <td>1.000000</td>
          <td>-0.238528</td>
          <td>-0.161165</td>
          <td>-0.168707</td>
          <td>-0.190476</td>
        </tr>
        <tr>
          <th>X2_magenta</th>
          <td>0.683520</td>
          <td>-0.374920</td>
          <td>-0.349292</td>
          <td>-0.273268</td>
          <td>-0.238528</td>
          <td>1.000000</td>
          <td>-0.201822</td>
          <td>-0.211266</td>
          <td>-0.238528</td>
        </tr>
        <tr>
          <th>X2_noir</th>
          <td>-0.295268</td>
          <td>0.538305</td>
          <td>-0.236004</td>
          <td>-0.184637</td>
          <td>-0.161165</td>
          <td>-0.201822</td>
          <td>1.000000</td>
          <td>-0.142745</td>
          <td>-0.161165</td>
        </tr>
        <tr>
          <th>X2_orange</th>
          <td>-0.309086</td>
          <td>-0.265175</td>
          <td>0.604842</td>
          <td>-0.193278</td>
          <td>-0.168707</td>
          <td>-0.211266</td>
          <td>-0.142745</td>
          <td>1.000000</td>
          <td>-0.168707</td>
        </tr>
        <tr>
          <th>X2_rose</th>
          <td>-0.348970</td>
          <td>-0.299392</td>
          <td>0.682890</td>
          <td>-0.218218</td>
          <td>-0.190476</td>
          <td>-0.238528</td>
          <td>-0.161165</td>
          <td>-0.168707</td>
          <td>1.000000</td>
        </tr>
      </tbody>
    </table>
    </div>



Ca ne dit pas grand-chose.

.. code:: ipython3

    from sklearn.preprocessing import LabelEncoder
    enc = LabelEncoder()
    df["X1e"] = enc.fit_transform(df["X1"])
    df["X2e"] = enc.fit_transform(df["X2"])
    df.head()






.. raw:: html

    <div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }

        .dataframe tbody tr th {
            vertical-align: top;
        }

        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th></th>
          <th>X1</th>
          <th>X2</th>
          <th>X1e</th>
          <th>X2e</th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th>0</th>
          <td>rouge</td>
          <td>rose</td>
          <td>2</td>
          <td>5</td>
        </tr>
        <tr>
          <th>1</th>
          <td>gris</td>
          <td>noir</td>
          <td>1</td>
          <td>3</td>
        </tr>
        <tr>
          <th>2</th>
          <td>bleu</td>
          <td>magenta</td>
          <td>0</td>
          <td>2</td>
        </tr>
        <tr>
          <th>3</th>
          <td>bleu</td>
          <td>magenta</td>
          <td>0</td>
          <td>2</td>
        </tr>
        <tr>
          <th>4</th>
          <td>bleu</td>
          <td>magenta</td>
          <td>0</td>
          <td>2</td>
        </tr>
      </tbody>
    </table>
    </div>



.. code:: ipython3

    df.corr()


.. parsed-literal::
    C:\Users\xavie\AppData\Local\Temp\ipykernel_148816\1134722465.py:1: FutureWarning: The default value of numeric_only in DataFrame.corr is deprecated. In a future version, it will default to False. Select only valid columns or specify the value of numeric_only to silence this warning.
      df.corr()






.. raw:: html

    <div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }

        .dataframe tbody tr th {
            vertical-align: top;
        }

        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th></th>
          <th>X1e</th>
          <th>X2e</th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th>X1e</th>
          <td>1.000000</td>
          <td>0.661507</td>
        </tr>
        <tr>
          <th>X2e</th>
          <td>0.661507</td>
          <td>1.000000</td>
        </tr>
      </tbody>
    </table>
    </div>



Ca ne veut toujours pas dire grand-chose. Et si on change la première
colonne en permutant les lables :

.. code:: ipython3

    df["X1e"] = df["X1e"].apply(lambda i: (i+1)%3)
    df.head()






.. raw:: html

    <div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }

        .dataframe tbody tr th {
            vertical-align: top;
        }

        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th></th>
          <th>X1</th>
          <th>X2</th>
          <th>X1e</th>
          <th>X2e</th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th>0</th>
          <td>rouge</td>
          <td>rose</td>
          <td>0</td>
          <td>5</td>
        </tr>
        <tr>
          <th>1</th>
          <td>gris</td>
          <td>noir</td>
          <td>2</td>
          <td>3</td>
        </tr>
        <tr>
          <th>2</th>
          <td>bleu</td>
          <td>magenta</td>
          <td>1</td>
          <td>2</td>
        </tr>
        <tr>
          <th>3</th>
          <td>bleu</td>
          <td>magenta</td>
          <td>1</td>
          <td>2</td>
        </tr>
        <tr>
          <th>4</th>
          <td>bleu</td>
          <td>magenta</td>
          <td>1</td>
          <td>2</td>
        </tr>
      </tbody>
    </table>
    </div>



.. code:: ipython3

    df.corr()


.. parsed-literal::
    C:\Users\xavie\AppData\Local\Temp\ipykernel_148816\1134722465.py:1: FutureWarning: The default value of numeric_only in DataFrame.corr is deprecated. In a future version, it will default to False. Select only valid columns or specify the value of numeric_only to silence this warning.
      df.corr()






.. raw:: html

    <div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }

        .dataframe tbody tr th {
            vertical-align: top;
        }

        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th></th>
          <th>X1e</th>
          <th>X2e</th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th>X1e</th>
          <td>1.000000</td>
          <td>-0.767644</td>
        </tr>
        <tr>
          <th>X2e</th>
          <td>-0.767644</td>
          <td>1.000000</td>
        </tr>
      </tbody>
    </table>
    </div>



La corrélation linéaire sur des variables catégorielles n’a pas de sens.
Essayons avec un arbre de décision. C’est le modèle adéquat pour ce type
de valeur discrètes :

.. code:: ipython3

    cor = correlation_cross_val(df[["X1e", "X2e"]], DecisionTreeRegressor)
    cor






.. raw:: html

    <div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }

        .dataframe tbody tr th {
            vertical-align: top;
        }

        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th></th>
          <th>X1e</th>
          <th>X2e</th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th>X1e</th>
          <td>1.0</td>
          <td>0.829865</td>
        </tr>
        <tr>
          <th>X2e</th>
          <td>1.0</td>
          <td>1.000000</td>
        </tr>
      </tbody>
    </table>
    </div>



Et si on permute le premier label :

.. code:: ipython3

    df["X1e"] = df["X1e"].apply(lambda i: (i+1)%3)
    correlation_cross_val(df[["X1e", "X2e"]], DecisionTreeRegressor)






.. raw:: html

    <div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }

        .dataframe tbody tr th {
            vertical-align: top;
        }

        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th></th>
          <th>X1e</th>
          <th>X2e</th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th>X1e</th>
          <td>1.0</td>
          <td>0.829759</td>
        </tr>
        <tr>
          <th>X2e</th>
          <td>1.0</td>
          <td>1.000000</td>
        </tr>
      </tbody>
    </table>
    </div>



Même résultat qui s’interprète de la sorte :


-  La variable *X1e* se déduit de *X2e* (car *cor(X2e, X1e) = 1*).
-  La variable *X2e* et fortement lié à *X2e*.

La valeur numérique choisie pour représente la variable catégorielle n’a
pas d’impact sur les résultats.

.. code:: ipython3

    ax = pairplot_cross_val(df[["X1e", "X2e"]], model=DecisionTreeRegressor)
    ax;



.. image:: correlation_non_lineaire_46_0.png


Et sur un jeu de données plus complet.

.. code:: ipython3

    from sklearn.datasets import load_diabetes
    df = load_diabetes()
    df = pandas.DataFrame(df.data, columns=df.feature_names)
    df.head()






.. raw:: html

    <div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }

        .dataframe tbody tr th {
            vertical-align: top;
        }

        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th></th>
          <th>age</th>
          <th>sex</th>
          <th>bmi</th>
          <th>bp</th>
          <th>s1</th>
          <th>s2</th>
          <th>s3</th>
          <th>s4</th>
          <th>s5</th>
          <th>s6</th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th>0</th>
          <td>0.038076</td>
          <td>0.050680</td>
          <td>0.061696</td>
          <td>0.021872</td>
          <td>-0.044223</td>
          <td>-0.034821</td>
          <td>-0.043401</td>
          <td>-0.002592</td>
          <td>0.019907</td>
          <td>-0.017646</td>
        </tr>
        <tr>
          <th>1</th>
          <td>-0.001882</td>
          <td>-0.044642</td>
          <td>-0.051474</td>
          <td>-0.026328</td>
          <td>-0.008449</td>
          <td>-0.019163</td>
          <td>0.074412</td>
          <td>-0.039493</td>
          <td>-0.068332</td>
          <td>-0.092204</td>
        </tr>
        <tr>
          <th>2</th>
          <td>0.085299</td>
          <td>0.050680</td>
          <td>0.044451</td>
          <td>-0.005670</td>
          <td>-0.045599</td>
          <td>-0.034194</td>
          <td>-0.032356</td>
          <td>-0.002592</td>
          <td>0.002861</td>
          <td>-0.025930</td>
        </tr>
        <tr>
          <th>3</th>
          <td>-0.089063</td>
          <td>-0.044642</td>
          <td>-0.011595</td>
          <td>-0.036656</td>
          <td>0.012191</td>
          <td>0.024991</td>
          <td>-0.036038</td>
          <td>0.034309</td>
          <td>0.022688</td>
          <td>-0.009362</td>
        </tr>
        <tr>
          <th>4</th>
          <td>0.005383</td>
          <td>-0.044642</td>
          <td>-0.036385</td>
          <td>0.021872</td>
          <td>0.003935</td>
          <td>0.015596</td>
          <td>0.008142</td>
          <td>-0.002592</td>
          <td>-0.031988</td>
          <td>-0.046641</td>
        </tr>
      </tbody>
    </table>
    </div>



.. code:: ipython3

    df.corr()






.. raw:: html

    <div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }

        .dataframe tbody tr th {
            vertical-align: top;
        }

        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th></th>
          <th>age</th>
          <th>sex</th>
          <th>bmi</th>
          <th>bp</th>
          <th>s1</th>
          <th>s2</th>
          <th>s3</th>
          <th>s4</th>
          <th>s5</th>
          <th>s6</th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th>age</th>
          <td>1.000000</td>
          <td>0.173737</td>
          <td>0.185085</td>
          <td>0.335428</td>
          <td>0.260061</td>
          <td>0.219243</td>
          <td>-0.075181</td>
          <td>0.203841</td>
          <td>0.270774</td>
          <td>0.301731</td>
        </tr>
        <tr>
          <th>sex</th>
          <td>0.173737</td>
          <td>1.000000</td>
          <td>0.088161</td>
          <td>0.241010</td>
          <td>0.035277</td>
          <td>0.142637</td>
          <td>-0.379090</td>
          <td>0.332115</td>
          <td>0.149916</td>
          <td>0.208133</td>
        </tr>
        <tr>
          <th>bmi</th>
          <td>0.185085</td>
          <td>0.088161</td>
          <td>1.000000</td>
          <td>0.395411</td>
          <td>0.249777</td>
          <td>0.261170</td>
          <td>-0.366811</td>
          <td>0.413807</td>
          <td>0.446157</td>
          <td>0.388680</td>
        </tr>
        <tr>
          <th>bp</th>
          <td>0.335428</td>
          <td>0.241010</td>
          <td>0.395411</td>
          <td>1.000000</td>
          <td>0.242464</td>
          <td>0.185548</td>
          <td>-0.178762</td>
          <td>0.257650</td>
          <td>0.393480</td>
          <td>0.390430</td>
        </tr>
        <tr>
          <th>s1</th>
          <td>0.260061</td>
          <td>0.035277</td>
          <td>0.249777</td>
          <td>0.242464</td>
          <td>1.000000</td>
          <td>0.896663</td>
          <td>0.051519</td>
          <td>0.542207</td>
          <td>0.515503</td>
          <td>0.325717</td>
        </tr>
        <tr>
          <th>s2</th>
          <td>0.219243</td>
          <td>0.142637</td>
          <td>0.261170</td>
          <td>0.185548</td>
          <td>0.896663</td>
          <td>1.000000</td>
          <td>-0.196455</td>
          <td>0.659817</td>
          <td>0.318357</td>
          <td>0.290600</td>
        </tr>
        <tr>
          <th>s3</th>
          <td>-0.075181</td>
          <td>-0.379090</td>
          <td>-0.366811</td>
          <td>-0.178762</td>
          <td>0.051519</td>
          <td>-0.196455</td>
          <td>1.000000</td>
          <td>-0.738493</td>
          <td>-0.398577</td>
          <td>-0.273697</td>
        </tr>
        <tr>
          <th>s4</th>
          <td>0.203841</td>
          <td>0.332115</td>
          <td>0.413807</td>
          <td>0.257650</td>
          <td>0.542207</td>
          <td>0.659817</td>
          <td>-0.738493</td>
          <td>1.000000</td>
          <td>0.617859</td>
          <td>0.417212</td>
        </tr>
        <tr>
          <th>s5</th>
          <td>0.270774</td>
          <td>0.149916</td>
          <td>0.446157</td>
          <td>0.393480</td>
          <td>0.515503</td>
          <td>0.318357</td>
          <td>-0.398577</td>
          <td>0.617859</td>
          <td>1.000000</td>
          <td>0.464669</td>
        </tr>
        <tr>
          <th>s6</th>
          <td>0.301731</td>
          <td>0.208133</td>
          <td>0.388680</td>
          <td>0.390430</td>
          <td>0.325717</td>
          <td>0.290600</td>
          <td>-0.273697</td>
          <td>0.417212</td>
          <td>0.464669</td>
          <td>1.000000</td>
        </tr>
      </tbody>
    </table>
    </div>



On dessine les 5 premières variables. On voit que la variable CHAS est
binaire.

.. code:: ipython3

    sns.pairplot(df[df.columns[:6]]);



.. image:: correlation_non_lineaire_51_0.png


.. code:: ipython3

    correlation_cross_val(df, DecisionTreeRegressor)






.. raw:: html

    <div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }

        .dataframe tbody tr th {
            vertical-align: top;
        }

        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th></th>
          <th>age</th>
          <th>sex</th>
          <th>bmi</th>
          <th>bp</th>
          <th>s1</th>
          <th>s2</th>
          <th>s3</th>
          <th>s4</th>
          <th>s5</th>
          <th>s6</th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th>age</th>
          <td>0.999796</td>
          <td>0.000000</td>
          <td>0.000000</td>
          <td>0.000000</td>
          <td>0.000000</td>
          <td>0.000000</td>
          <td>0.000000</td>
          <td>0.000000</td>
          <td>0.000000</td>
          <td>0.000000</td>
        </tr>
        <tr>
          <th>sex</th>
          <td>0.027919</td>
          <td>1.000000</td>
          <td>0.157867</td>
          <td>0.166623</td>
          <td>0.191943</td>
          <td>0.237330</td>
          <td>0.324815</td>
          <td>0.339291</td>
          <td>0.067600</td>
          <td>0.174746</td>
        </tr>
        <tr>
          <th>bmi</th>
          <td>0.000000</td>
          <td>0.000000</td>
          <td>0.998443</td>
          <td>0.000000</td>
          <td>0.000000</td>
          <td>0.000000</td>
          <td>0.000000</td>
          <td>0.000000</td>
          <td>0.000000</td>
          <td>0.000000</td>
        </tr>
        <tr>
          <th>bp</th>
          <td>0.000000</td>
          <td>0.000000</td>
          <td>0.067004</td>
          <td>0.999742</td>
          <td>0.034283</td>
          <td>0.000000</td>
          <td>0.000000</td>
          <td>0.000000</td>
          <td>0.093270</td>
          <td>0.000000</td>
        </tr>
        <tr>
          <th>s1</th>
          <td>0.000000</td>
          <td>0.000000</td>
          <td>0.000000</td>
          <td>0.000000</td>
          <td>0.999590</td>
          <td>0.818878</td>
          <td>0.000000</td>
          <td>0.000000</td>
          <td>0.015104</td>
          <td>0.000000</td>
        </tr>
        <tr>
          <th>s2</th>
          <td>0.000000</td>
          <td>0.000000</td>
          <td>0.000000</td>
          <td>0.000000</td>
          <td>0.799865</td>
          <td>0.999630</td>
          <td>0.000000</td>
          <td>0.050950</td>
          <td>0.000000</td>
          <td>0.000000</td>
        </tr>
        <tr>
          <th>s3</th>
          <td>0.000000</td>
          <td>0.022461</td>
          <td>0.000000</td>
          <td>0.000000</td>
          <td>0.000000</td>
          <td>0.000000</td>
          <td>0.999374</td>
          <td>0.715070</td>
          <td>0.059192</td>
          <td>0.000000</td>
        </tr>
        <tr>
          <th>s4</th>
          <td>0.123848</td>
          <td>0.045933</td>
          <td>0.280093</td>
          <td>0.000000</td>
          <td>0.425411</td>
          <td>0.593638</td>
          <td>0.718139</td>
          <td>0.999282</td>
          <td>0.542289</td>
          <td>0.215026</td>
        </tr>
        <tr>
          <th>s5</th>
          <td>0.000000</td>
          <td>0.000000</td>
          <td>0.000000</td>
          <td>0.000000</td>
          <td>0.000000</td>
          <td>0.000000</td>
          <td>0.000000</td>
          <td>0.055251</td>
          <td>0.999593</td>
          <td>0.000000</td>
        </tr>
        <tr>
          <th>s6</th>
          <td>0.000000</td>
          <td>0.000000</td>
          <td>0.000000</td>
          <td>0.074782</td>
          <td>0.026652</td>
          <td>0.000000</td>
          <td>0.000000</td>
          <td>0.000000</td>
          <td>0.115111</td>
          <td>0.999659</td>
        </tr>
      </tbody>
    </table>
    </div>



.. code:: ipython3

    pairplot_cross_val(df[df.columns[:6]], model=DecisionTreeRegressor, figsize=(16,16));



.. image:: correlation_non_lineaire_53_0.png


On regarde en pariculier les variables TAX, RAD, PTRATIO.

.. code:: ipython3

    sns.pairplot(df[["s1", "s2", "bmi"]]);



.. image:: correlation_non_lineaire_55_0.png


.. code:: ipython3

    df[["s1", "s2", "bmi"]].corr()






.. raw:: html

    <div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }

        .dataframe tbody tr th {
            vertical-align: top;
        }

        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th></th>
          <th>s1</th>
          <th>s2</th>
          <th>bmi</th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th>s1</th>
          <td>1.000000</td>
          <td>0.896663</td>
          <td>0.249777</td>
        </tr>
        <tr>
          <th>s2</th>
          <td>0.896663</td>
          <td>1.000000</td>
          <td>0.261170</td>
        </tr>
        <tr>
          <th>bmi</th>
          <td>0.249777</td>
          <td>0.261170</td>
          <td>1.000000</td>
        </tr>
      </tbody>
    </table>
    </div>



.. code:: ipython3

    pairplot_cross_val(df[["s1", "s2", "bmi"]], model=DecisionTreeRegressor);



.. image:: correlation_non_lineaire_57_0.png


.. code:: ipython3

    correlation_cross_val(df[["s1", "s2", "bmi"]], DecisionTreeRegressor)






.. raw:: html

    <div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }

        .dataframe tbody tr th {
            vertical-align: top;
        }

        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th></th>
          <th>s1</th>
          <th>s2</th>
          <th>bmi</th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th>s1</th>
          <td>0.999247</td>
          <td>0.806962</td>
          <td>0.000000</td>
        </tr>
        <tr>
          <th>s2</th>
          <td>0.788846</td>
          <td>0.997368</td>
          <td>0.000000</td>
        </tr>
        <tr>
          <th>bmi</th>
          <td>0.000000</td>
          <td>0.000000</td>
          <td>0.999158</td>
        </tr>
      </tbody>
    </table>
    </div>



Les variables sont toutes trois liées de façon non linéaire.

Maximal information coefficient
-------------------------------

Cette approche est plutôt pragmatique mais peut se révéler coûteuse en
terme de calculs. Elle permet aussi de comprendre qu’un coefficient de
corrélation dépend des hypothèses qu’on choisi pour les données. On peut
toujours construire un coefficient de corrélation qui soit égal à 1 mais
il correspond à toujours à un phénomène qu’on souhaite étudier. La
corrélation linéaire recherche des relations linéaires. On peut chercher
une relation polynomiale. Les arbres de décision recherche une
corrélation construite à partir de fonction en escalier. Plus la
relation a de degré de liberté, plus le coefficient a de chance de
tendre vers 1, moins il a de chance d’être aussi élevé sur de nouvelles
données.

Cela permet néanmoins de mieux comprendre les avantages et les
inconvénients de métriques du type
`MIC <https://en.wikipedia.org/wiki/Maximal_information_coefficient>`__
ou *Maximal information coefficient*. Plus de détails sont disponibles
dans cet article : `Equitability, mutual information, and the maximal
information coefficient <https://arxiv.org/abs/1301.7745v1>`__. Le
module `minepy <http://minepy.readthedocs.io/en/latest/python.html>`__
implémente cette métrique ainsi que d’autres qui poursuivent le même
objectif. L’information mutuelle est définie comme ceci pour deux
variables discrètes :

.. math:: MI(X,Y) = \sum_{x\in\mathcal{X}}\sum_{y\in\mathcal{Y}}p(x,y)\ln_2\frac{p(x,y)}{p(x)p(y)}

La fonction :math:`p(x,y)` définit la distribution conjointe des deux
variables, :math:`p(x)`, :math:`p(y)` les deux probabilités marginales.
Il existe une extension pour les variables continues :

.. math:: MIC(X,Y) = \int_{x\in\mathcal{X}}\in_{y\in\mathcal{Y}}p(x,y)\ln_2\frac{p(x,y)}{p(x)p(y)}dxdy

Une façon de calculer une approximation du coefficient :math:`MIC(x,y)`
est de discrétiser les deux variables :math:`X` et :math:`Y` ce qu’on
fait en appliquant un algorithme similaire à celui utilisé pour
construire un arbre de décision à ceci près que qu’il n’y a qu’une seule
variable et que la variable à prédire est elle-même.

L’information mutuelle est inspiré de la distance de
`Kullback-Leiber <https://fr.wikipedia.org/wiki/Divergence_de_Kullback-Leibler>`__
qui est une distance entre deux probabilités qui sont ici la disribution
du couple :math:`(X,Y)` et la distribution que ce couple aurait si les
deux variables étaient indépendantes, c’est à dire le produit de leur
distribution.

.. code:: ipython3

    %matplotlib inline

.. code:: ipython3

    import numpy as np
    import matplotlib.pyplot as plt
    
    
    rs = np.random.RandomState(seed=0)
    
    def mysubplot(x, y, numRows, numCols, plotNum,
                  xlim=(-4, 4), ylim=(-4, 4)):
    
        r = np.around(np.corrcoef(x, y)[0, 1], 1)
        
        # début ajout
        df = pandas.DataFrame(dict(x=x, y=y))
        cor = correlation_cross_val(df, DecisionTreeRegressor)
        dt = max(cor.iloc[1,0], cor.iloc[0,1])
        
        ax = plt.subplot(numRows, numCols, plotNum,
                         xlim=xlim, ylim=ylim)
        ax.set_title('Pearson r=%.1f\nDT=%.1f' % (r, dt),fontsize=10)
        ax.set_frame_on(False)
        ax.axes.get_xaxis().set_visible(False)
        ax.axes.get_yaxis().set_visible(False)
        ax.plot(x, y, ',')
        ax.set_xticks([])
        ax.set_yticks([])
        return ax
    
    def rotation(xy, t):
        return np.dot(xy, [[np.cos(t), -np.sin(t)], [np.sin(t), np.cos(t)]])
    
    def mvnormal(n=1000):
        cors = [1.0, 0.8, 0.4, 0.0, -0.4, -0.8, -1.0]
        for i, cor in enumerate(cors):
            cov = [[1, cor],[cor, 1]]
            xy = rs.multivariate_normal([0, 0], cov, n)
            mysubplot(xy[:, 0], xy[:, 1], 3, 7, i+1)
    
    def rotnormal(n=1000):
        ts = [0, np.pi/12, np.pi/6, np.pi/4, np.pi/2-np.pi/6,
              np.pi/2-np.pi/12, np.pi/2]
        cov = [[1, 1],[1, 1]]
        xy = rs.multivariate_normal([0, 0], cov, n)
        for i, t in enumerate(ts):
            xy_r = rotation(xy, t)
            mysubplot(xy_r[:, 0], xy_r[:, 1], 3, 7, i+8)
    
    def others(n=1000):
        x = rs.uniform(-1, 1, n)
        y = 4*(x**2-0.5)**2 + rs.uniform(-1, 1, n)/3
        mysubplot(x, y, 3, 7, 15, (-1, 1), (-1/3, 1+1/3))
    
        y = rs.uniform(-1, 1, n)
        xy = np.concatenate((x.reshape(-1, 1), y.reshape(-1, 1)), axis=1)
        xy = rotation(xy, -np.pi/8)
        lim = np.sqrt(2+np.sqrt(2)) / np.sqrt(2)
        mysubplot(xy[:, 0], xy[:, 1], 3, 7, 16, (-lim, lim), (-lim, lim))
    
        xy = rotation(xy, -np.pi/8)
        lim = np.sqrt(2)
        mysubplot(xy[:, 0], xy[:, 1], 3, 7, 17, (-lim, lim), (-lim, lim))
    
        y = 2*x**2 + rs.uniform(-1, 1, n)
        mysubplot(x, y, 3, 7, 18, (-1, 1), (-1, 3))
    
        y = (x**2 + rs.uniform(0, 0.5, n)) * \
            np.array([-1, 1])[rs.randint(0, 1, size=n)]
        mysubplot(x, y, 3, 7, 19, (-1.5, 1.5), (-1.5, 1.5))
    
        y = np.cos(x * np.pi) + rs.uniform(0, 1/8, n)
        x = np.sin(x * np.pi) + rs.uniform(0, 1/8, n)
        mysubplot(x, y, 3, 7, 20, (-1.5, 1.5), (-1.5, 1.5))
    
        xy1 = np.random.multivariate_normal([3, 3], [[1, 0], [0, 1]], int(n/4))
        xy2 = np.random.multivariate_normal([-3, 3], [[1, 0], [0, 1]], int(n/4))
        xy3 = np.random.multivariate_normal([-3, -3], [[1, 0], [0, 1]], int(n/4))
        xy4 = np.random.multivariate_normal([3, -3], [[1, 0], [0, 1]], int(n/4))
        xy = np.concatenate((xy1, xy2, xy3, xy4), axis=0)
        mysubplot(xy[:, 0], xy[:, 1], 3, 7, 21, (-7, 7), (-7, 7))
    
    plt.figure(figsize=(14,7))
    mvnormal(n=800)
    rotnormal(n=200)
    others(n=800)
    # plt.tight_layout()
    # plt.show()



.. image:: correlation_non_lineaire_62_0.png