module cli.asv_bench

Short summary

module mlprodict.cli.asv_bench

Command line about validation of prediction runtime.

source on GitHub

Functions

function

truncated documentation

asv_bench

Creates an asv benchmark in a folder but does not run it.

Documentation

Command line about validation of prediction runtime.

source on GitHub

mlprodict.cli.asv_bench.asv_bench(location='asvsklonnx', opset_min=-1, opset_max=None, runtime='scikit-learn, python_compiled', models=None, skip_models=None, extended_list=True, dims='1, 10, 100, 1000, 10000', n_features='4, 20', dtype=None, verbose=1, fLOG=<built-in function print>, clean=True, flat=False, conf_params=None, build=None, add_pyspy=False, env=None, matrix=None)

Creates an asv benchmark in a folder but does not run it.

Parameters
  • location – location of the benchmark

  • n_features – number of features to try

  • dims – number of observations to try

  • verbose – integer from 0 (None) to 2 (full verbose)

  • opset_min – tries every conversion from this minimum opset, -1 to get the current opset defined by module onnx

  • opset_max – tries every conversion up to maximum opset, -1 to get the current opset defined by module onnx

  • runtime – runtime to check, scikit-learn, python, python_compiled compiles the graph structure and is more efficient when the number of observations is small, onnxruntime1 to check onnxruntime, onnxruntime2 to check every ONNX node independently with onnxruntime, many runtime can be checked at the same time if the value is a comma separated list

  • models – list of models to test or empty string to test them all

  • skip_models – models to skip

  • extended_list – extends the list of scikit-learn converters with converters implemented in this module

  • dtype – ‘32’ or ‘64’ or None for both, limits the test to one specific number types

  • fLOG – logging function

  • clean – clean the folder first, otherwise overwrites the content

  • conf_params – to overwrite some of the configuration parameters, format name,value;name2,value2

  • flat – one folder for all files or subfolders

  • build – location of the outputs (env, html, results)

  • add_pyspy – add an extra folder with code to profile each configuration

  • env – default environment or same to use the current one

  • matrix – specifies versions for a module as a json string, example: {'onnxruntime': ['1.1.1', '1.1.2']}, if a package name starts with ‘~’, the package is removed

Returns

created files

Automatically creates an asv benchmark

The command creates a benchmark based on asv module. It does not run it.

Example:

python -m mlprodict asv_bench --models LogisticRegression,LinearRegression

<<<

python -m mlprodict asv_bench --help

>>>

usage: asv_bench [-h] [-l LOCATION] [-o OPSET_MIN] [-op OPSET_MAX]
                 [-r RUNTIME] [-m MODELS] [-s SKIP_MODELS] [-e EXTENDED_LIST]
                 [-d DIMS] [-n N_FEATURES] [-dt DTYPE] [-v VERBOSE] [-c CLEAN]
                 [-f FLAT] [-co CONF_PARAMS] [-b BUILD] [-a ADD_PYSPY]
                 [--env ENV] [-ma MATRIX]

Creates an `asv` benchmark in a folder but does not run it.

optional arguments:
  -h, --help            show this help message and exit
  -l LOCATION, --location LOCATION
                        location of the benchmark (default: asvsklonnx)
  -o OPSET_MIN, --opset_min OPSET_MIN
                        tries every conversion from this minimum opset, `-1`
                        to get the current opset defined by module onnx
                        (default: -1)
  -op OPSET_MAX, --opset_max OPSET_MAX
                        tries every conversion up to maximum opset, `-1` to
                        get the current opset defined by module onnx (default:
                        )
  -r RUNTIME, --runtime RUNTIME
                        runtime to check, *scikit-learn*, *python*,
                        *python_compiled* compiles the graph structure and is
                        more efficient when the number of observations is
                        small, *onnxruntime1* to check `onnxruntime`,
                        *onnxruntime2* to check every ONNX node independently
                        with onnxruntime, many runtime can be checked at the
                        same time if the value is a comma separated list
                        (default: scikit-learn,python_compiled)
  -m MODELS, --models MODELS
                        list of models to test or empty string to test them
                        all (default: )
  -s SKIP_MODELS, --skip_models SKIP_MODELS
                        models to skip (default: )
  -e EXTENDED_LIST, --extended_list EXTENDED_LIST
                        extends the list of :epkg:`scikit-learn` converters
                        with converters implemented in this module (default:
                        True)
  -d DIMS, --dims DIMS  number of observations to try (default:
                        1,10,100,1000,10000)
  -n N_FEATURES, --n_features N_FEATURES
                        number of features to try (default: 4,20)
  -dt DTYPE, --dtype DTYPE
                        '32' or '64' or None for both, limits the test to one
                        specific number types (default: )
  -v VERBOSE, --verbose VERBOSE
                        integer from 0 (None) to 2 (full verbose) (default: 1)
  -c CLEAN, --clean CLEAN
                        clean the folder first, otherwise overwrites the
                        content (default: True)
  -f FLAT, --flat FLAT  one folder for all files or subfolders (default:
                        False)
  -co CONF_PARAMS, --conf_params CONF_PARAMS
                        to overwrite some of the configuration parameters,
                        format ``name,value;name2,value2`` (default: )
  -b BUILD, --build BUILD
                        location of the outputs (env, html, results) (default:
                        )
  -a ADD_PYSPY, --add_pyspy ADD_PYSPY
                        add an extra folder with code to profile each
                        configuration (default: False)
  --env ENV             default environment or ``same`` to use the current one
                        (default: )
  -ma MATRIX, --matrix MATRIX
                        specifies versions for a module as a json string,
                        example: ``{'onnxruntime': ['1.1.1', '1.1.2']}``, if a
                        package name starts with `'~'`, the package is removed
                        (default: )

source on GitHub