.. _td2aclusteringrst:

==================
2A.ml - Clustering
==================


.. only:: html

    **Links:** :download:`notebook <td2a_clustering.ipynb>`, :downloadlink:`html <td2a_clustering2html.html>`, :download:`python <td2a_clustering.py>`, :downloadlink:`slides <td2a_clustering.slides.html>`, :githublink:`GitHub|_doc/notebooks/td2a_ml/td2a_clustering.ipynb|*`


Ce notebook utilise les données des vélos de Chicago `Divvy
Data <https://www.divvybikes.com/system-data>`__. Il s’inspire du
challenge créée pour découvrir les habitudes des habitantes de la ville
`City
Bike <http://www.xavierdupre.fr/app/ensae_projects/helpsphinx/challenges/city_bike.html>`__.
L’idée est d’explorer plusieurs algorithmes de clustering et de voire
comment trafiquer les données pour les faire marcher et en tirer
quelques apprentissages.

.. code:: ipython3

    from jyquickhelper import add_notebook_menu
    add_notebook_menu()






.. contents::
    :local:





.. code:: ipython3

    %matplotlib inline

Les données
-----------

Elles ont été prétraitées selon le notebook `Bike Pattern
2 <http://www.xavierdupre.fr/app/ensae_projects/helpsphinx/notebooks/city_bike_solution_cluster_start.html>`__.
Elles représentent la distribution du nombre de vélos partant
(*startdist*) et arrivant (*stopdist*). On utilise le clustering pour
découvrir les différents usages des habitants de Chicago avec pour
intuition le fait que les habitants de Chicago utilise majoritairement
les vélos pour aller et venir entre leur appartement et leur lieu de
travail. Cette même idée mais à Paris est illustrée par ce billet de
blog : `Busy areas in
Paris <http://www.xavierdupre.fr/blog/2013-09-26_nojs.html>`__.

.. code:: ipython3

    from pyensae.datasource import download_data
    file = download_data("features_bike_chicago.zip")
    file




.. parsed-literal::
    ['.\\features_bike_chicago.txt']



.. code:: ipython3

    import pandas
    features = pandas.read_csv("features_bike_chicago.txt", sep="\t", encoding="utf-8", low_memory=False, header=[0,1])
    features.columns = ["station_id", "station_name", "weekday"] + list(features.columns[3:])
    features.head()






.. raw:: html

    <div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }

        .dataframe tbody tr th {
            vertical-align: top;
        }

        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th></th>
          <th>station_id</th>
          <th>station_name</th>
          <th>weekday</th>
          <th>(startdist, 00:00:00)</th>
          <th>(startdist, 00:10:00)</th>
          <th>(startdist, 00:20:00)</th>
          <th>(startdist, 00:30:00)</th>
          <th>(startdist, 00:40:00)</th>
          <th>(startdist, 00:50:00)</th>
          <th>(startdist, 01:00:00)</th>
          <th>...</th>
          <th>(stopdist, 22:20:00)</th>
          <th>(stopdist, 22:30:00)</th>
          <th>(stopdist, 22:40:00)</th>
          <th>(stopdist, 22:50:00)</th>
          <th>(stopdist, 23:00:00)</th>
          <th>(stopdist, 23:10:00)</th>
          <th>(stopdist, 23:20:00)</th>
          <th>(stopdist, 23:30:00)</th>
          <th>(stopdist, 23:40:00)</th>
          <th>(stopdist, 23:50:00)</th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th>0</th>
          <td>2.0</td>
          <td>Michigan Ave &amp; Balbo Ave</td>
          <td>0.0</td>
          <td>0.003756</td>
          <td>0.000939</td>
          <td>0.002817</td>
          <td>0.000000</td>
          <td>0.000000</td>
          <td>0.003756</td>
          <td>0.000000</td>
          <td>...</td>
          <td>0.004381</td>
          <td>0.002191</td>
          <td>0.004381</td>
          <td>0.002191</td>
          <td>0.004381</td>
          <td>0.004381</td>
          <td>0.005476</td>
          <td>0.002191</td>
          <td>0.000000</td>
          <td>0.005476</td>
        </tr>
        <tr>
          <th>1</th>
          <td>2.0</td>
          <td>Michigan Ave &amp; Balbo Ave</td>
          <td>1.0</td>
          <td>0.000000</td>
          <td>0.000000</td>
          <td>0.001106</td>
          <td>0.001106</td>
          <td>0.001106</td>
          <td>0.002212</td>
          <td>0.000000</td>
          <td>...</td>
          <td>0.009371</td>
          <td>0.012048</td>
          <td>0.006693</td>
          <td>0.004016</td>
          <td>0.005355</td>
          <td>0.006693</td>
          <td>0.002677</td>
          <td>0.000000</td>
          <td>0.000000</td>
          <td>0.000000</td>
        </tr>
        <tr>
          <th>2</th>
          <td>2.0</td>
          <td>Michigan Ave &amp; Balbo Ave</td>
          <td>2.0</td>
          <td>0.001357</td>
          <td>0.002714</td>
          <td>0.000000</td>
          <td>0.001357</td>
          <td>0.000000</td>
          <td>0.005427</td>
          <td>0.000000</td>
          <td>...</td>
          <td>0.002907</td>
          <td>0.002907</td>
          <td>0.015988</td>
          <td>0.005814</td>
          <td>0.001453</td>
          <td>0.001453</td>
          <td>0.011628</td>
          <td>0.000000</td>
          <td>0.000000</td>
          <td>0.007267</td>
        </tr>
        <tr>
          <th>3</th>
          <td>2.0</td>
          <td>Michigan Ave &amp; Balbo Ave</td>
          <td>3.0</td>
          <td>0.000000</td>
          <td>0.004144</td>
          <td>0.000000</td>
          <td>0.000000</td>
          <td>0.002762</td>
          <td>0.004144</td>
          <td>0.000000</td>
          <td>...</td>
          <td>0.009274</td>
          <td>0.003091</td>
          <td>0.003091</td>
          <td>0.007728</td>
          <td>0.001546</td>
          <td>0.003091</td>
          <td>0.009274</td>
          <td>0.001546</td>
          <td>0.007728</td>
          <td>0.001546</td>
        </tr>
        <tr>
          <th>4</th>
          <td>2.0</td>
          <td>Michigan Ave &amp; Balbo Ave</td>
          <td>4.0</td>
          <td>0.000000</td>
          <td>0.000000</td>
          <td>0.000000</td>
          <td>0.002846</td>
          <td>0.000000</td>
          <td>0.000000</td>
          <td>0.000949</td>
          <td>...</td>
          <td>0.008214</td>
          <td>0.001027</td>
          <td>0.006160</td>
          <td>0.004107</td>
          <td>0.015400</td>
          <td>0.006160</td>
          <td>0.002053</td>
          <td>0.006160</td>
          <td>0.007187</td>
          <td>0.000000</td>
        </tr>
      </tbody>
    </table>
    <p>5 rows × 291 columns</p>
    </div>



Exercice 1 : petits clusters
----------------------------

Que faire des petits clusters ?


Exercice 2 : autres types de clustering
---------------------------------------

On essaye des algorithmes de clustering qui n’imposent pas de choisir un
nombre de clusters initial.

1. On essaye
   `DBScan <http://scikit-learn.org/stable/modules/generated/sklearn.cluster.DBSCAN.html>`__.
   Est-ce que cela fonctionne ? Si non pourquoi ?
2. Et si vous savez pourquoi, vous trouverez une solution d’y remédier.