.. _td2asentimentanalysiscorrectionrst:

==========================================
2A.ml - Analyse de sentiments - correction
==========================================


.. only:: html

    **Links:** :download:`notebook <td2a_sentiment_analysis_correction.ipynb>`, :downloadlink:`html <td2a_sentiment_analysis_correction2html.html>`, :download:`python <td2a_sentiment_analysis_correction.py>`, :downloadlink:`slides <td2a_sentiment_analysis_correction.slides.html>`, :githublink:`GitHub|_doc/notebooks/td2a_ml/td2a_sentiment_analysis_correction.ipynb|*`


C’est désormais un problème classique de machine learning. D’un côté, du
texte, de l’autre une appréciation, le plus souvent binaire, positive ou
négative mais qui pourrait être graduelle.

.. code:: ipython3

    %matplotlib inline

.. code:: ipython3

    from jyquickhelper import add_notebook_menu
    add_notebook_menu()






.. contents::
    :local:





Les données
-----------

On récupère les données depuis le site UCI `Sentiment Labelled Sentences
Data
Set <https://archive.ics.uci.edu/ml/datasets/Sentiment+Labelled+Sentences>`__
où on utilise la fonction ``load_sentiment_dataset``.

.. code:: ipython3

    from ensae_teaching_cs.data import load_sentiment_dataset
    df = load_sentiment_dataset()
    df.head()






.. raw:: html

    <div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }

        .dataframe tbody tr th {
            vertical-align: top;
        }

        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th></th>
          <th>sentance</th>
          <th>sentiment</th>
          <th>source</th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th>0</th>
          <td>So there is no way for me to plug it in here i...</td>
          <td>0</td>
          <td>amazon_cells_labelled</td>
        </tr>
        <tr>
          <th>1</th>
          <td>Good case, Excellent value.</td>
          <td>1</td>
          <td>amazon_cells_labelled</td>
        </tr>
        <tr>
          <th>2</th>
          <td>Great for the jawbone.</td>
          <td>1</td>
          <td>amazon_cells_labelled</td>
        </tr>
        <tr>
          <th>3</th>
          <td>Tied to charger for conversations lasting more...</td>
          <td>0</td>
          <td>amazon_cells_labelled</td>
        </tr>
        <tr>
          <th>4</th>
          <td>The mic is great.</td>
          <td>1</td>
          <td>amazon_cells_labelled</td>
        </tr>
      </tbody>
    </table>
    </div>



Exercice 1 : approche td-idf
----------------------------

La cible est la colonne *sentiment*, les deux autres colonnes sont les
features. Il faudra utiliser les prétraitements
`LabelEncoder <http://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.LabelEncoder.html>`__,
`OneHotEncoder <http://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.OneHotEncoder.html>`__,
`TF-IDF <http://scikit-learn.org/stable/modules/generated/sklearn.feature_extraction.text.TfidfVectorizer.html>`__.
L’un d’entre eux n’est pas nécessaire depuis la version
`0.20.0 <http://scikit-learn.org/stable/whats_new.html#sklearn-preprocessing>`__
de *scikit-learn*. On s’occupe des variables catégorielles.

La variable catégorielle
~~~~~~~~~~~~~~~~~~~~~~~~

Ce serait un peu plus simple avec le module `Category
Encoders <http://contrib.scikit-learn.org/categorical-encoding/>`__ ou
la dernière nouveauté de scikit-learn :
`ColumnTransformer <http://scikit-learn.org/stable/modules/generated/sklearn.compose.ColumnTransformer.html#sklearn.compose.ColumnTransformer>`__.

.. code:: ipython3

    from sklearn.model_selection import train_test_split
    X_train, X_test, y_train, y_test = train_test_split(
                    df.drop("sentiment", axis=1), df["sentiment"])

.. code:: ipython3

    from sklearn.preprocessing import LabelEncoder, OneHotEncoder
    le = LabelEncoder()
    le.fit(X_train["source"])
    X_le = le.transform(X_train["source"])
    X_le.shape




.. parsed-literal::
    (2250,)



.. code:: ipython3

    X_le_mat = X_le.reshape((X_le.shape[0], 1))

.. code:: ipython3

    ohe = OneHotEncoder(categories="auto")
    ohe.fit(X_le_mat)




.. parsed-literal::
    OneHotEncoder()



.. code:: ipython3

    X_le_encoded = ohe.transform(X_le_mat)
    train_cat = X_le_encoded.todense()
    test_cat = ohe.transform(le.transform(X_test["source"]).reshape((len(X_test), 1))).todense()

.. code:: ipython3

    import pandas
    X_train2 = pandas.concat([X_train.reset_index(drop=True),
                              pandas.DataFrame(train_cat, columns=le.classes_)],
                             sort=False, axis=1)
    X_train2.head(n=2)






.. raw:: html

    <div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }

        .dataframe tbody tr th {
            vertical-align: top;
        }

        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th></th>
          <th>sentance</th>
          <th>source</th>
          <th>amazon_cells_labelled</th>
          <th>imdb_labelled</th>
          <th>yelp_labelled</th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th>0</th>
          <td>Now we were chosen to be tortured with this di...</td>
          <td>imdb_labelled</td>
          <td>0.0</td>
          <td>1.0</td>
          <td>0.0</td>
        </tr>
        <tr>
          <th>1</th>
          <td>Woa, talk about awful.</td>
          <td>imdb_labelled</td>
          <td>0.0</td>
          <td>1.0</td>
          <td>0.0</td>
        </tr>
      </tbody>
    </table>
    </div>



.. code:: ipython3

    X_test2 = pandas.concat([X_test.reset_index(drop=True),
                             pandas.DataFrame(test_cat, columns=le.classes_)],
                             sort=False, axis=1)
    X_test2.head(n=2)






.. raw:: html

    <div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }

        .dataframe tbody tr th {
            vertical-align: top;
        }

        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th></th>
          <th>sentance</th>
          <th>source</th>
          <th>amazon_cells_labelled</th>
          <th>imdb_labelled</th>
          <th>yelp_labelled</th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th>0</th>
          <td>It looks very nice.</td>
          <td>amazon_cells_labelled</td>
          <td>1.0</td>
          <td>0.0</td>
          <td>0.0</td>
        </tr>
        <tr>
          <th>1</th>
          <td>As a European, the movie is a nice throwback t...</td>
          <td>imdb_labelled</td>
          <td>0.0</td>
          <td>1.0</td>
          <td>0.0</td>
        </tr>
      </tbody>
    </table>
    </div>



tokenisation
~~~~~~~~~~~~

On tokenise avec le module
`spacy <https://spacy.io/usage/spacy-101#annotations-token>`__ qui
requiert des données supplémentaires pour découper en mot avec
``pip install https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-2.0.0/en_core_web_sm-2.0.0.tar.gz``
selon les instructions dévoilées dans le `guide de
départ <https://spacy.io/usage/models>`__ ou encore
``python -m spacy download en``. Le module
`gensim <http://www.xavierdupre.fr/app/papierstat/helpsphinx/notebooks/artificiel_tokenize.html?highlight=tokenisation#gensim>`__
ne requiert pas d’installation. On peut aussi s’inspirer de l’example
`word2vec
pré-entraînés <http://www.xavierdupre.fr/app/papierstat/helpsphinx/notebooks/text_sentiment_wordvec.html#word2vec-pre-entraines>`__.

.. code:: ipython3

    import spacy
    nlp = spacy.load("en_core_web_sm")
    # Ca marche après avoir installé le corpus correspondant
    # python -m spacy download en_core_web_sm

.. code:: ipython3

    doc = nlp(X_train2.iloc[0,0])
    [token.text for token in doc]




.. parsed-literal::
    ['Now',
     'we',
     'were',
     'chosen',
     'to',
     'be',
     'tortured',
     'with',
     'this',
     'disgusting',
     'piece',
     'of',
     'blatant',
     'American',
     'propaganda',
     '.',
     ' ']



tf-idf
~~~~~~

Une fois que les mots sont tokenisé, on peut appliquer le *tf-idf*.

.. code:: ipython3

    from sklearn.feature_extraction.text import TfidfTransformer, CountVectorizer
    from sklearn.pipeline import make_pipeline
    tokenizer = lambda text: [token.text.lower() for token in nlp(text)]
    count = CountVectorizer(tokenizer=tokenizer, analyzer='word')
    tfidf = TfidfTransformer()
    pipe = make_pipeline(count, tfidf)

.. code:: ipython3

    pipe.fit(X_train["sentance"])




.. parsed-literal::
    Pipeline(steps=[('countvectorizer',
                     CountVectorizer(tokenizer=<function <lambda> at 0x000001DCC8835488>)),
                    ('tfidftransformer', TfidfTransformer())])



.. code:: ipython3

    train_feature = pipe.transform(X_train2["sentance"])
    train_feature




.. parsed-literal::
    <2250x4495 sparse matrix of type '<class 'numpy.float64'>'
    	with 29554 stored elements in Compressed Sparse Row format>



.. code:: ipython3

    test_feature = pipe.transform(X_test2["sentance"])

Combinaison de toutes les variables
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. code:: ipython3

    train_feature.shape, train_cat.shape




.. parsed-literal::
    ((2250, 4495), (2250, 3))



.. code:: ipython3

    import numpy
    np_train = numpy.hstack([train_feature.todense(), train_cat])
    np_test = numpy.hstack([test_feature.todense(), test_cat])

Calage d’un modèle
~~~~~~~~~~~~~~~~~~

.. code:: ipython3

    from sklearn.ensemble import RandomForestClassifier
    rf = RandomForestClassifier(n_estimators=50)
    rf.fit(np_train, y_train)




.. parsed-literal::
    RandomForestClassifier(n_estimators=50)



.. code:: ipython3

    rf.score(np_test, y_test)




.. parsed-literal::
    0.7533333333333333




Exercice 2 : word2vec
---------------------

On utilise l’approche
`word2vec <https://en.wikipedia.org/wiki/Word2vec>`__ du module
`gensim <https://radimrehurek.com/gensim/models/word2vec.html>`__ ou
`spacy <https://spacy.io/usage/vectors-similarity>`__. Avec
`spacy <https://spacy.io/usage/vectors-similarity>`__, c’est assez
simple :

.. code:: ipython3

    vv = nlp(X_train2.iloc[0, 0])
    list(vv)[0].vector[:10], vv.vector.shape




.. parsed-literal::
    (array([-0.21269655, -0.7653725 , -0.1316224 , -0.3766306 ,  0.5549566 ,
            -0.60907495,  5.3928123 ,  5.099738  ,  4.210167  ,  2.9974651 ],
           dtype=float32),
     (96,))



On fait la somme.

.. code:: ipython3

    sum([_.vector for _ in vv])[:10]




.. parsed-literal::
    array([-11.796999 ,  -8.17019  ,   3.1232045, -14.440253 ,  20.460987 ,
            -8.738287 ,  12.388309 ,  23.718775 ,  -9.392727 ,   1.9914403],
          dtype=float32)



.. code:: ipython3

    np_train_vect = numpy.zeros((X_train2.shape[0], vv.vector.shape[0]))
    for i, sentance in enumerate(X_train2["sentance"]):
        np_train_vect[i, :] = sum(v.vector for v in nlp(sentance.lower()))

.. code:: ipython3

    np_test_vect = numpy.zeros((X_test2.shape[0], vv.vector.shape[0]))
    for i, sentance in enumerate(X_test2["sentance"]):
        np_test_vect[i, :] = sum(v.vector for v in nlp(sentance.lower()))

.. code:: ipython3

    np_train_v = numpy.hstack([np_train_vect, train_cat])
    np_test_v = numpy.hstack([np_test_vect, test_cat])

.. code:: ipython3

    rfv = RandomForestClassifier(n_estimators=50)
    rfv.fit(np_train_v, y_train)




.. parsed-literal::
    RandomForestClassifier(n_estimators=50)



.. code:: ipython3

    rfv.score(np_test_v, y_test)




.. parsed-literal::
    0.6146666666666667



Moins bien…

Exercice 3 : comparer les deux approches
----------------------------------------

Avec une courbe
`ROC <http://scikit-learn.org/stable/modules/generated/sklearn.metrics.roc_curve.html>`__
par exemple.

.. code:: ipython3

    pmodel1 = rf.predict_proba(np_test)[:, 1]
    pmodel2 = rfv.predict_proba(np_test_v)[:, 1]

.. code:: ipython3

    from sklearn.metrics import roc_auc_score, roc_curve, auc
    fpr1, tpr1, th1 = roc_curve(y_test, pmodel1)
    fpr2, tpr2, th2 = roc_curve(y_test, pmodel2)

.. code:: ipython3

    import matplotlib.pyplot as plt
    fig, ax = plt.subplots(1,1, figsize=(4,4))
    ax.plot(fpr1, tpr1, label='tf-idf')
    ax.plot(fpr2, tpr2, label='word2vec')
    ax.legend();



.. image:: td2a_sentiment_analysis_correction_42_0.png


Petite analyse d’erreurs
------------------------

On combine les erreurs des modèles sur la base de test.

.. code:: ipython3

    final = X_test.copy()
    final["model1"] = pmodel1
    final["model2"] = pmodel2
    final["label"] = y_test
    final.head()






.. raw:: html

    <div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }

        .dataframe tbody tr th {
            vertical-align: top;
        }

        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th></th>
          <th>sentance</th>
          <th>source</th>
          <th>model1</th>
          <th>model2</th>
          <th>label</th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th>850</th>
          <td>It looks very nice.</td>
          <td>amazon_cells_labelled</td>
          <td>0.62</td>
          <td>0.62</td>
          <td>1</td>
        </tr>
        <tr>
          <th>415</th>
          <td>As a European, the movie is a nice throwback t...</td>
          <td>imdb_labelled</td>
          <td>0.68</td>
          <td>0.54</td>
          <td>1</td>
        </tr>
        <tr>
          <th>585</th>
          <td>Great food and great service in a clean and fr...</td>
          <td>yelp_labelled</td>
          <td>0.94</td>
          <td>0.78</td>
          <td>1</td>
        </tr>
        <tr>
          <th>785</th>
          <td>This allows the possibility of double booking ...</td>
          <td>amazon_cells_labelled</td>
          <td>0.48</td>
          <td>0.54</td>
          <td>0</td>
        </tr>
        <tr>
          <th>440</th>
          <td>Both do good jobs and are quite amusing.</td>
          <td>imdb_labelled</td>
          <td>0.64</td>
          <td>0.46</td>
          <td>1</td>
        </tr>
      </tbody>
    </table>
    </div>



On regarde des erreurs.

.. code:: ipython3

    erreurs = final[final["label"] == 1].sort_values("model2")
    erreurs.head()






.. raw:: html

    <div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }

        .dataframe tbody tr th {
            vertical-align: top;
        }

        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th></th>
          <th>sentance</th>
          <th>source</th>
          <th>model1</th>
          <th>model2</th>
          <th>label</th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th>527</th>
          <td>Be sure to order dessert, even if you need to ...</td>
          <td>yelp_labelled</td>
          <td>0.54</td>
          <td>0.20</td>
          <td>1</td>
        </tr>
        <tr>
          <th>707</th>
          <td>This is cool because most cases are just open ...</td>
          <td>amazon_cells_labelled</td>
          <td>0.38</td>
          <td>0.22</td>
          <td>1</td>
        </tr>
        <tr>
          <th>449</th>
          <td>I won't say any more - I don't like spoilers, ...</td>
          <td>imdb_labelled</td>
          <td>0.20</td>
          <td>0.22</td>
          <td>1</td>
        </tr>
        <tr>
          <th>676</th>
          <td>I can't wait to go back.</td>
          <td>yelp_labelled</td>
          <td>0.20</td>
          <td>0.22</td>
          <td>1</td>
        </tr>
        <tr>
          <th>908</th>
          <td>I can hear while I'm driving in the car, and u...</td>
          <td>amazon_cells_labelled</td>
          <td>0.34</td>
          <td>0.24</td>
          <td>1</td>
        </tr>
      </tbody>
    </table>
    </div>



.. code:: ipython3

    list(erreurs["sentance"])[:5]




.. parsed-literal::
    ['Be sure to order dessert, even if you need to pack it to-go - the tiramisu and cannoli are both to die for.',
     'This is cool because most cases are just open there allowing the screen to get all scratched up.',
     "I won't say any more - I don't like spoilers, so I don't want to be one, but I believe this film is worth your time.  ",
     "I can't wait to go back.",
     "I can hear while I'm driving in the car, and usually don't even have to put it on it's loudest setting."]



Le modèle 2 reconnaît mal les négations visiblement. On regarde le
modèle 1.

.. code:: ipython3

    erreurs = final[final["label"] == 1].sort_values("model1")
    erreurs.head()






.. raw:: html

    <div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }

        .dataframe tbody tr th {
            vertical-align: top;
        }

        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th></th>
          <th>sentance</th>
          <th>source</th>
          <th>model1</th>
          <th>model2</th>
          <th>label</th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th>436</th>
          <td>The soundtrack wasn't terrible, either.</td>
          <td>imdb_labelled</td>
          <td>0.06</td>
          <td>0.34</td>
          <td>1</td>
        </tr>
        <tr>
          <th>412</th>
          <td>Not too screamy not to masculine but just righ...</td>
          <td>imdb_labelled</td>
          <td>0.08</td>
          <td>0.26</td>
          <td>1</td>
        </tr>
        <tr>
          <th>161</th>
          <td>I was seated immediately.</td>
          <td>yelp_labelled</td>
          <td>0.10</td>
          <td>0.38</td>
          <td>1</td>
        </tr>
        <tr>
          <th>619</th>
          <td>Don't miss it.</td>
          <td>imdb_labelled</td>
          <td>0.12</td>
          <td>0.32</td>
          <td>1</td>
        </tr>
        <tr>
          <th>448</th>
          <td>My 8/10 score is mostly for the plot.</td>
          <td>imdb_labelled</td>
          <td>0.14</td>
          <td>0.48</td>
          <td>1</td>
        </tr>
      </tbody>
    </table>
    </div>



.. code:: ipython3

    list(erreurs["sentance"])[:5]




.. parsed-literal::
    ["The soundtrack wasn't terrible, either.  ",
     'Not too screamy not to masculine but just right.  ',
     'I was seated immediately.',
     "Don't miss it.  ",
     'My 8/10 score is mostly for the plot.  ']



Idem, voyons là où les modèles sont en désaccords.

.. code:: ipython3

    final["diff"] = final.model1 - final.model2

.. code:: ipython3

    erreurs = final[final["label"] == 1].sort_values("diff")
    erreurs.head()






.. raw:: html

    <div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }

        .dataframe tbody tr th {
            vertical-align: top;
        }

        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th></th>
          <th>sentance</th>
          <th>source</th>
          <th>model1</th>
          <th>model2</th>
          <th>label</th>
          <th>diff</th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th>390</th>
          <td>If you want healthy authentic or ethic food, t...</td>
          <td>yelp_labelled</td>
          <td>0.30</td>
          <td>0.72</td>
          <td>1</td>
          <td>-0.42</td>
        </tr>
        <tr>
          <th>797</th>
          <td>A good quality bargain.. I bought this after I...</td>
          <td>amazon_cells_labelled</td>
          <td>0.34</td>
          <td>0.68</td>
          <td>1</td>
          <td>-0.34</td>
        </tr>
        <tr>
          <th>53</th>
          <td>This phone is pretty sturdy and I've never had...</td>
          <td>amazon_cells_labelled</td>
          <td>0.38</td>
          <td>0.72</td>
          <td>1</td>
          <td>-0.34</td>
        </tr>
        <tr>
          <th>691</th>
          <td>Shot in the Southern California desert using h...</td>
          <td>imdb_labelled</td>
          <td>0.36</td>
          <td>0.70</td>
          <td>1</td>
          <td>-0.34</td>
        </tr>
        <tr>
          <th>448</th>
          <td>My 8/10 score is mostly for the plot.</td>
          <td>imdb_labelled</td>
          <td>0.14</td>
          <td>0.48</td>
          <td>1</td>
          <td>-0.34</td>
        </tr>
      </tbody>
    </table>
    </div>



.. code:: ipython3

    erreurs.tail()






.. raw:: html

    <div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }

        .dataframe tbody tr th {
            vertical-align: top;
        }

        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th></th>
          <th>sentance</th>
          <th>source</th>
          <th>model1</th>
          <th>model2</th>
          <th>label</th>
          <th>diff</th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th>464</th>
          <td>The inside is really quite nice and very clean.</td>
          <td>yelp_labelled</td>
          <td>0.94</td>
          <td>0.46</td>
          <td>1</td>
          <td>0.48</td>
        </tr>
        <tr>
          <th>4</th>
          <td>The mic is great.</td>
          <td>amazon_cells_labelled</td>
          <td>0.90</td>
          <td>0.42</td>
          <td>1</td>
          <td>0.48</td>
        </tr>
        <tr>
          <th>68</th>
          <td>Great for iPODs too.</td>
          <td>amazon_cells_labelled</td>
          <td>0.96</td>
          <td>0.46</td>
          <td>1</td>
          <td>0.50</td>
        </tr>
        <tr>
          <th>341</th>
          <td>It is a really good show to watch.</td>
          <td>imdb_labelled</td>
          <td>0.84</td>
          <td>0.32</td>
          <td>1</td>
          <td>0.52</td>
        </tr>
        <tr>
          <th>306</th>
          <td>Has been working great.</td>
          <td>amazon_cells_labelled</td>
          <td>0.90</td>
          <td>0.32</td>
          <td>1</td>
          <td>0.58</td>
        </tr>
      </tbody>
    </table>
    </div>



Le modèle 2 (word2vec) a l’air meilleur sur les phrases longues, le
modèle 1 (tf-idf) saisit mieux les mots positifs. A confirmer sur plus
de données.


-  Enlever les stop words, les signes de ponctuation.
-  Combiner les deux approches.
-  n-grammes
-  …

Dernière analyse en regardant le taux d’erreur par source.

.. code:: ipython3

    r1 = rf.predict(np_test)
    r2 = rfv.predict(np_test_v)
    final["rep1"] = r1
    final["rep2"] = r2
    final["err1"] = (final.label - final.rep1).abs()
    final["err2"] = (final.label - final.rep2).abs()
    final["total"] = 1
    final.head()






.. raw:: html

    <div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }

        .dataframe tbody tr th {
            vertical-align: top;
        }

        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th></th>
          <th>sentance</th>
          <th>source</th>
          <th>model1</th>
          <th>model2</th>
          <th>label</th>
          <th>diff</th>
          <th>rep1</th>
          <th>rep2</th>
          <th>err1</th>
          <th>err2</th>
          <th>total</th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th>850</th>
          <td>It looks very nice.</td>
          <td>amazon_cells_labelled</td>
          <td>0.62</td>
          <td>0.62</td>
          <td>1</td>
          <td>0.00</td>
          <td>1</td>
          <td>1</td>
          <td>0</td>
          <td>0</td>
          <td>1</td>
        </tr>
        <tr>
          <th>415</th>
          <td>As a European, the movie is a nice throwback t...</td>
          <td>imdb_labelled</td>
          <td>0.68</td>
          <td>0.54</td>
          <td>1</td>
          <td>0.14</td>
          <td>1</td>
          <td>1</td>
          <td>0</td>
          <td>0</td>
          <td>1</td>
        </tr>
        <tr>
          <th>585</th>
          <td>Great food and great service in a clean and fr...</td>
          <td>yelp_labelled</td>
          <td>0.94</td>
          <td>0.78</td>
          <td>1</td>
          <td>0.16</td>
          <td>1</td>
          <td>1</td>
          <td>0</td>
          <td>0</td>
          <td>1</td>
        </tr>
        <tr>
          <th>785</th>
          <td>This allows the possibility of double booking ...</td>
          <td>amazon_cells_labelled</td>
          <td>0.48</td>
          <td>0.54</td>
          <td>0</td>
          <td>-0.06</td>
          <td>0</td>
          <td>1</td>
          <td>0</td>
          <td>1</td>
          <td>1</td>
        </tr>
        <tr>
          <th>440</th>
          <td>Both do good jobs and are quite amusing.</td>
          <td>imdb_labelled</td>
          <td>0.64</td>
          <td>0.46</td>
          <td>1</td>
          <td>0.18</td>
          <td>1</td>
          <td>0</td>
          <td>0</td>
          <td>1</td>
          <td>1</td>
        </tr>
      </tbody>
    </table>
    </div>



.. code:: ipython3

    final[["source", "err1", "err2", "total"]].groupby("source").sum()






.. raw:: html

    <div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }

        .dataframe tbody tr th {
            vertical-align: top;
        }

        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th></th>
          <th>err1</th>
          <th>err2</th>
          <th>total</th>
        </tr>
        <tr>
          <th>source</th>
          <th></th>
          <th></th>
          <th></th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th>amazon_cells_labelled</th>
          <td>56</td>
          <td>94</td>
          <td>250</td>
        </tr>
        <tr>
          <th>imdb_labelled</th>
          <td>77</td>
          <td>107</td>
          <td>253</td>
        </tr>
        <tr>
          <th>yelp_labelled</th>
          <td>52</td>
          <td>88</td>
          <td>247</td>
        </tr>
      </tbody>
    </table>
    </div>



*imdb* paraît une source une peu plus difficile à saisir. Quoiqu’il en
soit, 2000 phrases pour apprendre est assez peu pour apprendre.

Versions utilisées pour ce notebook
-----------------------------------

`spacy <https://spacy.io/>`__ s’est montré quelque peu fantasques cette
année avec quelques erreurs notamment celle-ci : `ValueError:
cymem.cymem.Pool has the wrong size, try
recompiling <https://github.com/explosion/spaCy/issues/2852>`__. Voici
les versions utilisées…

.. code:: ipython3

    def version(module, sub=True):
        try:
            ver = getattr(module, '__version__', None)
            if ver is None:
                ver = [_ for _ in os.listdir(os.path.join(module.__file__, '..', '..' if sub else '')) \
                       if module.__name__ in _ and 'dist' in _][-1]
            return ver
        except Exception as e:
            return str(e)

.. code:: ipython3

    import os
    import thinc
    print("thinc", version(thinc))
    import preshed
    print("preshed", version(preshed))
    import cymem
    print("cymem", version(cymem))
    import murmurhash
    print("murmurhash", version(murmurhash))
    import spacy
    print("spacy", spacy.__version__)
    
    import msgpack
    print("msgpack", version(msgpack))
    import numpy
    print("numpy", numpy.__version__)


.. parsed-literal::
    thinc 7.4.1
    preshed preshed-3.0.2.dist-info
    cymem cymem-2.0.2.dist-info
    murmurhash murmurhash-1.0.2.dist-info
    spacy 2.3.2
    msgpack msgpack_numpy-0.4.4.3.dist-info
    numpy 1.18.1