module onnxrt.validate.validate_benchmark#

Short summary#

module mlprodict.onnxrt.validate.validate_benchmark

Measures time processing for ONNX models.

source on GitHub

Functions#

function

truncated documentation

benchmark_fct

Benchmarks a function which takes an array as an input and changes the number of rows.

make_n_rows

Multiplies or reduces the rows of x to get exactly n rows.

Documentation#

Measures time processing for ONNX models.

source on GitHub

mlprodict.onnxrt.validate.validate_benchmark.benchmark_fct(fct, X, time_limit=4, obs=None, node_time=False, time_kwargs=None, skip_long_test=True)#

Benchmarks a function which takes an array as an input and changes the number of rows.

Parameters:

  • fct – function to benchmark, signature is fct(xo)

  • X – array

  • time_limit – above this time, measurement is stopped

  • obs – all information available in a dictionary

  • node_time – measure time execution for each node in the graph

  • time_kwargs – to define a more precise way to measure a model

  • skip_long_test – skips tests for high values of N if they seem too long

Returns:

dictionary with the results

The function uses obs to reduce the number of tries it does. sklearn.gaussian_process.GaussianProcessRegressor produces huge NxN if predict method is called with return_cov=True. The default for time_kwargs is the following:

<<<

from mlprodict.onnxrt.validate.validate_helper import default_time_kwargs
import pprint
pprint.pprint(default_time_kwargs())

>>>

    {1: {'number': 15, 'repeat': 20},
     10: {'number': 10, 'repeat': 20},
     100: {'number': 4, 'repeat': 10},
     1000: {'number': 4, 'repeat': 4},
     10000: {'number': 2, 'repeat': 2}}

See also notebook Time processing for every ONNX nodes in a graph to see how this function can be used to measure time spent in each node.

source on GitHub

mlprodict.onnxrt.validate.validate_benchmark.make_n_rows(x, n, y=None)#

Multiplies or reduces the rows of x to get exactly n rows.

Parameters:

  • x – matrix

  • n – number of rows

  • y – target (optional)

Returns:

new matrix or two new matrices if y is not None

source on GitHub