.. image:: pystat.png
    :height: 20
    :alt: Statistique
    :target: http://www.xavierdupre.fr/app/ensae_teaching_cs/helpsphinx/td_2a_notions.html#pour-un-profil-plutot-data-scientist

Deep Learning en détail
+++++++++++++++++++++++

*Notebooks*

.. toctree::
    :maxdepth: 2

    ../notebooks/_gs2a_deep

(à venir foolbox)

*Cours*

* `Deep Learning course: lecture slides and lab notebooks <https://m2dsupsdlclass.github.io/lectures-labs/>`_
* `Artificial Intelligence, Revealed (1) <https://code.facebook.com/pages/1902086376686983>`_ :
  article de blog et vidéos expliquant les différents concepts du deep learning
* `colah's blog <http://colah.github.io/>`_ *(2016/08)* blog/cours sur le deep learning
* `Tutoriels avec CNTK <https://cntk.ai/pythondocs/tutorials.html>`_
* `Course notes for CS224N Winter17 <https://github.com/stanfordnlp/cs224n-winter17-notes>`_ (Stanford)

*Tutoriels*

* `Building Autoencoders in Keras <https://blog.keras.io/building-autoencoders-in-keras.html>`_
* `Image Similarity Ranking using Microsoft Cognitive Toolkit (CNTK) <https://github.com/azure/ImageSimilarityUsingCntk>`_
* `Tutoriels avec CNTK <https://github.com/Microsoft/CNTK/tree/v2.0.beta12.0/Tutorials>`_ :
  ces notebooks sont bien illlustrés
  (`GAN - Generative Models <https://github.com/Microsoft/CNTK/blob/v2.0.beta12.0/Tutorials/CNTK_206_Basic_GAN.ipynb>`_).
* `Tutoriels avec TensorFlow <https://github.com/Hvass-Labs/TensorFlow-Tutorials>`_ :
  ce ne sont pas les seuls mais ils ont l'avantage d'être bien illustrés
  (`Adversarial Training <https://github.com/Hvass-Labs/TensorFlow-Tutorials/blob/master/11_Adversarial_Examples.ipynb>`_).
* `Machine Learning is Fun! Part 3: Deep Learning and Convolutional Neural Networks <https://medium.com/@ageitgey/machine-learning-is-fun-part-3-deep-learning-and-convolutional-neural-networks-f40359318721>`_
* `Machine Learning is Fun! Part 4: Modern Face Recognition with Deep Learning <https://medium.com/@ageitgey/machine-learning-is-fun-part-4-modern-face-recognition-with-deep-learning-c3cffc121d78>`_
* `Object detection using Fast R-CNN <https://docs.microsoft.com/en-us/cognitive-toolkit/Object-Detection-using-Fast-R-CNN>`_
* `Deep Learning - The Straight Dope <http://gluon.mxnet.io/>`_ :
  séries de notebooks de difficulté graduelle

*Sites*

* `Tinker With a Neural Network Right Here in Your Browser <http://playground.tensorflow.org/>`_
* `ConvNetJS <http://cs.stanford.edu/people/karpathy/convnetjs/>`_
* `Databricks / Deep Learning <https://docs.databricks.com/applications/deep-learning/index.html>`_

*Liens*

* `Four deep learning trends from ACL 2017 (1) <http://www.abigailsee.com/2017/08/30/four-deep-learning-trends-from-acl-2017-part-1.html>`_
* `Four deep learning trends from ACL 2017 (2) <http://www.abigailsee.com/2017/08/30/four-deep-learning-trends-from-acl-2017-part-2.html>`_

*Articles scientifiques*

* `LightRNN: Memory and Computation-Efficient Recurrent Neural Networks <https://arxiv.org/abs/1610.09893>`_
* `Deep learning architecture diagrams <http://fastml.com/deep-learning-architecture-diagrams/>`_
* `Factorized Convolutional Neural Networks <https://arxiv.org/abs/1608.04337>`_
* `Deep Residual Learning for Image Recognition <https://arxiv.org/pdf/1512.03385v1.pdf>`_
* `Deep Learning <http://www-labs.iro.umontreal.ca/~bengioy/dlbook/>`_, Yoshua Bengio, Ian Goodfellow and Aaron Courville
* `LeNet5 <http://yann.lecun.com/exdb/lenet/>`_
* `mxnet <https://github.com/dmlc/mxnet>`_
* `Benchmarking State-of-the-Art Deep Learning Software Tools <http://arxiv.org/pdf/1608.07249v5.pdf>`_
* `Wide & Deep Learning: Better Together with TensorFlow <https://research.googleblog.com/2016/06/wide-deep-learning-better-together-with.html>`_,
  `Wide & Deep Learning for Recommender Systems <https://arxiv.org/pdf/1606.07792v1.pdf>`_
* `To go deep or wide in learning? <http://www.jmlr.org/proceedings/papers/v33/pandey14.pdf>`_
* `Three Classes of Deep Learning Architectures and Their Applications: A Tutorial Survey <https://www.microsoft.com/en-us/research/publication/three-classes-of-deep-learning-architectures-and-their-applications-a-tutorial-survey/>`_
* `Tutorial: Learning Deep Architectures <http://www.cs.toronto.edu/~rsalakhu/deeplearning/yoshua_icml2009.pdf>`_
* `Deep Learning <https://en.wikipedia.org/wiki/Deep_learning>`_ (wikipédia)
* `Fast R-CNN <https://arxiv.org/abs/1504.08083>`_ (voir `Object Detection using Fast R CNN <https://github.com/Microsoft/CNTK/wiki/Object-Detection-using-Fast-R-CNN>`_)
* `Evaluation of Deep Learning Toolkits <https://github.com/zer0n/deepframeworks/blob/master/README.md>`_ *(2015/12)*
* `Understanding Deep Learning Requires Rethinking Generalization <https://arxiv.org/pdf/1611.03530.pdf>`_
* `Training Deep Nets with Sublinear Memory Cost <https://arxiv.org/pdf/1604.06174.pdf>`_
* `On the importance of initialization and momentum in deep learning <http://www.cs.toronto.edu/~fritz/absps/momentum.pdf>`_
* `TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems <http://download.tensorflow.org/paper/whitepaper2015.pdf>`_
* `Foolbox v0.8.0: A Python toolbox to benchmark the robustness of machine learning models <https://arxiv.org/abs/1707.04131>`_

*Chiffres, Textes*

* `One weird trick for parallelizing convolutional neural networks <https://arxiv.org/pdf/1404.5997v2.pdf>`_
* `ImageNet Classification with Deep Convolutional Neural Networks <https://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks.pdf>`_
* `Very Deep Convolutional Networks for Large-Scale Image Recognition <https://arxiv.org/pdf/1409.1556v6.pdf>`_
* `Multi-Digit Recognition Using A Space Displacement Neural Network  <https://papers.nips.cc/paper/557-multi-digit-recognition-using-a-space-displacement-neural-network.pdf>`_
* `Space Displacement Localization Neural Networks to locate origin points of handwritten text lines in historical documents <http://liris.cnrs.fr/christian.wolf/papers/icdar-hip2015.pdf>`_
* `Neural Network Architectures <https://culurciello.github.io/tech/2016/06/04/nets.html>`_,
  `Convolutional Neural Networks (CNNs / ConvNets) <http://cs231n.github.io/convolutional-networks/#conv>`_
* `Transfer Learning <http://cs231n.github.io/transfer-learning/>`_

*Benchmarks*

* `Benchmarking CNTK on Keras: is it Better at Deep Learning than TensorFlow? <http://minimaxir.com/2017/06/keras-cntk/>`_
  (`code <https://github.com/minimaxir/keras-cntk-benchmark>`_)

*Plus théoriques*

* `Why Does Unsupervized Deep Learning Work? - A perspective from group theory <https://arxiv.org/pdf/1412.6621v3.pdf>`_
* `Deep Learning of Representations: Looking Forward <https://arxiv.org/pdf/1305.0445v2.pdf>`_
* `Why Does Unsupervised Pre-training Help Deep Learning? <http://jmlr.org/papers/volume11/erhan10a/erhan10a.pdf>`_

*Lectures deep text*

* `Efficient Estimation of Word Representations in Vector Space <http://arxiv.org/abs/1301.3781>`_, Tomas Mikolov, Kai Chen, Greg Corrado, Jeffrey Dean,
* `Distributed Representations of Words and Phrases and their Compositionality <http://papers.nips.cc/paper/5021-distributed-representations-of-words-and-phrases-and-their-compositionality.pdf>`_, Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, Jeff Dean,
* `word2vec Parameter Learning Explained <http://arxiv.org/abs/1411.2738>`_, Xin Rong,
* `Tutorial on Auto-Encoders <http://piotrmirowski.wordpress.com/2014/03/27/tutorial-on-auto-encoders/>`_, Piotr Mirowski
* `Pretrained Character Embeddings for Deep Learning and Automatic Text Generation <http://minimaxir.com/2017/04/char-embeddings/>`_

*Vus dans des conférences*

* `Fast R-CNN <Fast R-CNN>`_ *(dotAI)*
* `Mask R-CNN <https://arxiv.org/abs/1703.06870>`_ *(dotAI)*
* `Modèle Tenserflow <https://github.com/tensorflow/models>`_
  (modèle adaptés pour du transfer learning : ResNet, `Inception <http://nicolovaligi.com/history-inception-deep-learning-architecture.html>`_) *(dotAI)*
* `Domain-Adversarial Training of Neural Networks <http://jmlr.org/papers/volume17/15-239/15-239.pdf>`_ *(dotAI)*

*Deep learning embarqué*

* `TensorFlow sur Android <https://github.com/tensorflow/tensorflow/tree/master/tensorflow/examples/android>`_
* `TensorFlow sur RasberryPI <https://github.com/samjabrahams/tensorflow-on-raspberry-pi/blob/master/GUIDE.md>`_

*Modules - deep learning*

* `pytorch <http://pytorch.org/>`_ : design plus simple que tous les autres
* `keras <https://keras.io/>`_
* `mxnet <https://github.com/dmlc/mxnet>`_
* `caffe <http://caffe.berkeleyvision.org/>`_ (`installation <http://caffe.berkeleyvision.org/installation.html>`_)
* `climin <http://climin.readthedocs.io/en/latest/rmsprop.html>`_ (algorithme de back propagation)
* `tensorflow <https://www.tensorflow.org/>`_ (Google)
* `cntk <https://github.com/Microsoft/CNTK>`_

*Modules - à suivre*

* `chainer <https://github.com/pfnet/chainer>`_
* `Federated Learning: Collaborative Machine Learning without Centralized Training Data <https://research.googleblog.com/2017/04/federated-learning-collaborative.html>`_
* `foolbox <https://github.com/bethgelab/foolbox>`_ :
  trouver des petites perturbations des données qui trompent les réseaux de neurones