Python extensions to do machine learning

machine learning, python

2013-09-15 Python extensions to do machine learning

I started to compare the functionalities of some Python extensions (the list is not exhaustive) :

scikit-learn
mlpy
Modular toolkit for Data Processing (MDP)
PyBrain
Theano
MILK
Gensim
NLTK
Orange
statsmodels
pyMVPA

The first one (scikit-learn) covers many features and its documentation is quite clear. When a model is missing, you can look into PyBrain for Reinforcement Learning, in Gensim for Dirichlet Application (Latent, Hierarchical) and in NLTK for any text processing (tokenization for example). For those who do not want to code, Orange would be a good option. The module Theano does gradient optimization using GPU.

A couple of forums, kind of FAQ for machine learning:

It would be difficult to do machine learning without using visualization tools. matplotlib and ggplot would be a good way to start. We also manipulate tables: numpy and pandas. For a command line: ipython or bpython are two common options.

If you are looking for data UC Irvine Machine Learning Repository. If you work with Windows, many of the presented modules can be downloaded from Unofficial Windows Binaries for Python Extension Packages. It also gives a clear view of what package is available on which Python's version.

Next table summarizes where you can find which features (with some errors):

scikit-learn statsmodels mlpy MDP PyBrain Theano MILK pyMVPA NLTK Gensim Orange

AdaBoost yes yes

ANOVA yes yes

ARMA (Time Series) yes

C4.5 yes yes

Canonical Correlation Analysis yes yes

Cross Validation yes yes

DBSCAN yes

Decision Trees yes yes yes yes

Deep Belief Networks yes

Dictionary Learning yes

Dynamic Time Warping (yes) yes

Elastic Net yes yes yes yes

Evolution Strategies (ES) yes

Fast ICA yes yes

Fast/Partial PCA yes

Features Selection yes yes yes yes

Gaussian Mixture Model yes yes

Gaussian Naive Bayes yes yes

Genetic Algorithm yes

Golub Classifier yes

GPU computation yes

Gradient Based Optimization yes yes

Gradient Boosted Tree yes

Gradient Boosting Regression yes yes

Grid Search yes

Hidden Markov Model with Gaussian Mixture Emissions (HMM GMM) yes yes

Hierarchical Clustering (Ward…) yes yes yes yes

Hierarchical Dirichlet Application (HDP) yes

ICA yes yes

Isotonic Regression yes

KDTree yes

Kernal Density yes yes

Kernel Fisher Discriminant yes

Kernel PCA yes yes yes

Kernel Regression yes

Kernel Ridge Regression yes

k-Means yes yes yes yes yes

k-NN yes yes yes yes yes

Label Spreading yes yes

Largest Common Subsequence (LCS)

Lasso yes yes

Large Linear Classification yes

Latent Dirichlet Application (LDA) yes

Least Angle Regression (LARS) yes yes yes

Linear Discriminant Analysis (LDA) yes yes yes yes

Linear Regression yes yes yes yes yes yes yes

Logisitic Regression yes yes yes yes yes

Naive Bayesian Learner yes yes

Natural Language Processing (NLP) yes

Neural Network (NN) yes yes yes yes

Non-Negative matrix factorization by Projected Gradient (NMF) yes yes

Partial Least Square (PLS) yes yes

Partial Least Square (SVD) yes

Particle Swarm Optimization (PSO) yes

Passive Aggressive Classification yes

Passive Aggressive Regression yes

Pipeline yes yes

Principal Component Analysis (PCA) yes yes yes yes yes

Probabilistic Principal Component Analysis (pPCA) yes yes yes

p-Value yes yes

Quadratic Discriminant Analysis (QDA) yes yes

Random Forests yes yes yes yes

Recurrent Neural Network yes

Regression Tree yes yes yes

Reinforcement Learning yes

Ridge Regression yes yes yes yes

ROC / Precision / Recall yes yes

SARSA yes

Self Organizing Map (SOM - Kohonen) yes yes yes

Singular Value Decomposition (SVD) yes yes

Sparse PCA yes yes

Spectral BiClustering yes

Spectral Clustering yes

Spectral Coclustering yes

Spectral Regression Discriminant Analysis yes

Support Vector Classificiation (SVC) yes yes yes

Support Vector Machine (SVM) yes yes yes yes yes yes yes

Support Vector Regression (SVR) yes yes

TF-IDF yes yes

Wavelets yes yes

If the model you need is not in the previous list, you can use rpy2 to communicate with R where you will surely find a related package.

2014/09/03: you can also read Python Tools for Machine Learning.

<-- -->

Xavier Dupré

XD blog

2013-09-15 Python extensions to do machine learning