2021-05-03 Performance

Quelques articles sur la performance.

• How to optimize GEMM on CPU

• SLIDE : In Defense of Smart Algorithms over Hardware Acceleration for Large-Scale Deep Learning Systems

• Scalable and Sustainable Deep Learning via Randomized Hashing

• Accelerating SLIDE Deep Learning on Modern CPUs: Vectorization, Quantizations, Memory Optimizations, and More

Other papers:

• ZeRO: Memory Optimizations Toward Training Trillion Parameter Models

Fairness:

• How Facebook got addicted to spreading misinformation

• Fairness and machine learning

Neural networks:

• Neural Tangent Kernel: Convergence and Generalization in Neural Networks

Arbres :

• Unbiased Measurement of Feature Importance in Tree-Based Methods

Collèges de France

• Une vision mathématique du « Deep Learning »

• Stéphane Mallat, Représentation Parcimonieuse

• Algorithmes quantiques : quand la physique quantique défie la thèse de Church-Turing

Autres

• Invariant Risk Minimization (… Bottou …)

• Deep Neural Networks Motivated by Partial Differential Equations

• Lagrangian Neural Networks

ML

• A Hierarchical Model for Data-to-Text Generation

Apprentissage par renforcement

• Dynamic Programming in Distributional Reinforcement Learning

• A Distributional Perspective on Reinforcement Learning

Ethique

• Quelle responsabilité pour les algorithmes

Quelques autres directions :

• plaidml, Stripe: Tensor Compilation via the Nested Polyhedral Model

Une énième implémentation de pipeline de machine learning : kedro.

Quelques PR intéressantes (scikit-learn) :

• Improve near constant feature detection in scalers

• MNT Avoid catastrophic cancellation in mean_variance_axis