Links: github, documentation, mlprodict, blog

Build status Build Status Windows MIT License Requirements Status GitHub Issues Notebook Coverage Downloads Forks Stars size

mlprodict explores couple of ways to compute predictions faster than the library used to build the machine learned model, mostly scikit-learn which is optimized for training, which is equivalent to batch predictions. One way is to use ONNX. onnxruntime provides an efficient way to compute predictions. mlprodict implements a python/numpy runtime for ONNX which does not have any dependency on scikit-learn.


import numpy
from sklearn.linear_model import LinearRegression
from sklearn.datasets import load_iris
from mlprodict.onnxrt import OnnxInference
from mlprodict.onnxrt.validate.validate_difference import measure_relative_difference
from import get_ir_version_from_onnx

iris = load_iris()
X =[:, :2]
y =
lr = LinearRegression(), y)

# Predictions with scikit-learn.
expected = lr.predict(X[:5])

# Conversion into ONNX.
from mlprodict.onnx_conv import to_onnx
model_onnx = to_onnx(lr, X.astype(numpy.float32))
print("ONNX:", str(model_onnx)[:200] + "\n...")

# Predictions with onnxruntime
model_onnx.ir_version = get_ir_version_from_onnx()
oinf = OnnxInference(model_onnx, runtime='onnxruntime1')
ypred ={'X': X[:5].astype(numpy.float32)})
print("ONNX output:", ypred)

# Measuring the maximum difference.
print("max abs diff:", measure_relative_difference(
    expected, ypred['variable']))


    [0.172 0.343 0.069 0.059 0.034]
    ONNX: ir_version: 6
    producer_name: "skl2onnx"
    producer_version: "1.7.1100"
    domain: "ai.onnx"
    model_version: 0
    doc_string: ""
    graph {
      node {
        input: "X"
        output: "variable"
        name: "LinearRegressor
    ONNX output: {'variable': array([[0.172],
           [0.034]], dtype=float32)}
    max abs diff: 6.303014714402957e-06

These predictions are obtained with the following ONNX graph.

That concludes the example with ONNX. A similar way was introduced before switching to ONNX. It is far less advanced but aims at producing a C file replicating the predictions.


from sklearn.linear_model import LogisticRegression
from sklearn.datasets import load_iris

iris = load_iris()
X =[:, :2]
y =
y[y == 2] = 1
lr = LogisticRegression(), y)

# Conversion into a graph.
from mlprodict.grammar_sklearn import sklearn2graph
gr = sklearn2graph(lr, output_names=['Prediction', 'Score'])

# Conversion into C
ccode = gr.export(lang='c')
# We print after a little bit of cleaning (remove all comments)
print("\n".join(_ for _ in ccode['code'].split("\n") if "//" not in _))


    int LogisticRegression (float* pred, float* Features)
        float pred0c0c00c0[2] = {(float)3.3882975578308105, (float)-3.164527654647827};
        float* pred0c0c00c1 = Features;
        float pred0c0c00;
        adot_float_float(&pred0c0c00, pred0c0c00c0, pred0c0c00c1, 2);
        float pred0c0c01 = (float)-8.323304176330566;
        float pred0c0c0 = pred0c0c00 + pred0c0c01;
        float pred0c0;
        sign_float(&pred0c0, pred0c0c0);
        float pred0[2];
        concat_float_float(pred0, pred0c0, pred0c0c0);
        memcpy(pred, pred0, 2*sizeof(float));
        return 0;

Notebook ONNX visualization shows how to visualize an ONNX pipeline.





Static Methods


Module Index


Search Page






Notebook Gallery

Statistics on code

Unit Test Coverage