ONNX graph, single or double floats

The notebook shows discrepencies obtained by using double floats instead of single float in two cases. The second one involves GaussianProcessRegressor.

Simple case of a linear regression

A linear regression is simply a matrix multiplication followed by an addition: $Y=AX+B$. Let's train one with scikit-learn.

Let's predict with scikit-learn and python.


With ONNX, we would write this operation as follows... We still need to convert everything into single floats = float32.

The next line uses a python runtime to compute the prediction.

And here is the same with onnxruntime...

With double instead of single float

ONNX was originally designed for deep learning which usually uses floats but it does not mean cannot be used. Every number is converted into double floats.

And now the python runtime...

And the onnxruntime version of it.

And now the GaussianProcessRegressor

This shows a case

The differences between the predictions for single floats and double floats...

Who's right or wrong... The differences between the predictions with the original model...

Double predictions clearly wins.


Let's keep track of it.

Side by side

We may wonder where the discrepencies start. But for that, we need to do a side by side.

The differences really starts for output 'O0' after the matrix multiplication. This matrix melts different number with very different order of magnitudes and that alone explains the discrepencies with doubles and floats on that particular model.

Before going further, let's check how sensitive the trained model is about converting double into floats.

Having float or double inputs should not matter. We confirm that with the model converted into ONNX.

Last verification.