Training

Differences between onnxruntime and onnxruntime-training

onnxruntime-training is an extension of onnxruntime that supports training. Version 1.10 is obtained by compiling onnxruntime from the sources with different flags. One example:

python ./tools/ci_build/build.py --build_dir ./build/debian \\
       --config Release --build_wheel --numpy_version= \\
       --skip_tests --build_shared_lib --enable_training \\
       --enable_training_ops --enable_training_torch_interop \\
       --parallel

Python Wrapper for TrainingSession

class onnxruntime.TrainingSession(path_or_bytes, parameters, sess_options=None, providers=None, provider_options=None)

Parameters

• path_or_bytes – filename or serialized ONNX or ORT format model in a byte string

• sess_options – session options

• providers – Optional sequence of providers in order of decreasing precedence. Values can either be provider names or tuples of (provider name, options dict). If not provided, then all available providers are used with the default precedence.

• provider_options – Optional sequence of options dicts corresponding to the providers listed in ‘providers’.

The model type will be inferred unless explicitly set in the SessionOptions. To explicitly set:

so = onnxruntime.SessionOptions()
# so.add_session_config_entry('session.load_model_format', 'ONNX') or
so.add_session_config_entry('session.load_model_format', 'ORT')

A file extension of ‘.ort’ will be inferred as an ORT format model. All other filenames are assumed to be ONNX format models.

‘providers’ can contain either names or names and options. When any options are given in ‘providers’, ‘provider_options’ should not be used.

The list of providers is ordered by precedence. For example [‘CUDAExecutionProvider’, ‘CPUExecutionProvider’] means execute a node using CUDAExecutionProvider if capable, otherwise execute using CPUExecutionProvider.

__class__

alias of type

__del__()

__delattr__(name, /)

Implement delattr(self, name).

__dir__()

Default dir() implementation.

__eq__(value, /)

Return self==value.

__format__(format_spec, /)

Default object formatter.

__ge__(value, /)

Return self>=value.

__getattribute__(name, /)

Return getattr(self, name).

__gt__(value, /)

Return self>value.

__hash__()

Return hash(self).

__init__(path_or_bytes, parameters, sess_options=None, providers=None, provider_options=None)

Parameters

• path_or_bytes – filename or serialized ONNX or ORT format model in a byte string

• sess_options – session options

• providers – Optional sequence of providers in order of decreasing precedence. Values can either be provider names or tuples of (provider name, options dict). If not provided, then all available providers are used with the default precedence.

• provider_options – Optional sequence of options dicts corresponding to the providers listed in ‘providers’.

The model type will be inferred unless explicitly set in the SessionOptions. To explicitly set:

so = onnxruntime.SessionOptions()
# so.add_session_config_entry('session.load_model_format', 'ONNX') or
so.add_session_config_entry('session.load_model_format', 'ORT')

A file extension of ‘.ort’ will be inferred as an ORT format model. All other filenames are assumed to be ONNX format models.

‘providers’ can contain either names or names and options. When any options are given in ‘providers’, ‘provider_options’ should not be used.

The list of providers is ordered by precedence. For example [‘CUDAExecutionProvider’, ‘CPUExecutionProvider’] means execute a node using CUDAExecutionProvider if capable, otherwise execute using CPUExecutionProvider.

__init_subclass__()

This method is called when a class is subclassed.

The default implementation does nothing. It may be overridden to extend subclasses.

__le__(value, /)

Return self<=value.

__lt__(value, /)

Return self<value.

__ne__(value, /)

Return self!=value.

__new__(**kwargs)

__reduce__()

Helper for pickle.

__reduce_ex__(protocol, /)

Helper for pickle.

__repr__()

Return repr(self).

__setattr__(name, value, /)

Implement setattr(self, name, value).

__sizeof__()

Size of object in memory, in bytes.

__str__()

Return str(self).

__subclasshook__()

Abstract classes can override this to customize issubclass().

This is invoked early on by abc.ABCMeta.__subclasscheck__(). It should return True, False or NotImplemented. If it returns NotImplemented, the normal algorithm is used. Otherwise, it overrides the normal algorithm (and the outcome is cached).

_create_inference_session(providers, provider_options, disabled_optimizers=None)

_reset_session(providers, provider_options)

release underlying session object.

disable_fallback()

Disable session.run() fallback mechanism.

enable_fallback()

Enable session.Run() fallback mechanism. If session.Run() fails due to an internal Execution Provider failure, reset the Execution Providers enabled for this session. If GPU is enabled, fall back to CUDAExecutionProvider. otherwise fall back to CPUExecutionProvider.

end_profiling()

End profiling and return results in a file.

The results are stored in a filename if the option onnxruntime.SessionOptions.enable_profiling().

get_inputs()

Return the inputs metadata as a list of onnxruntime.NodeArg.

get_model_state(include_mixed_precision_weights=False)

get_modelmeta()

Return the metadata. See onnxruntime.ModelMetadata.

get_optimizer_state()

get_outputs()

Return the outputs metadata as a list of onnxruntime.NodeArg.

get_overridable_initializers()

Return the inputs (including initializers) metadata as a list of onnxruntime.NodeArg.

get_partition_info_map()

get_profiling_start_time_ns()

Return the nanoseconds of profiling’s start time Comparable to time.monotonic_ns() after Python 3.3 On some platforms, this timer may not be as precise as nanoseconds For instance, on Windows and MacOS, the precision will be ~100ns

get_provider_options()

Return registered execution providers’ configurations.

get_providers()

Return list of registered execution providers.

get_session_options()

Return the session options. See onnxruntime.SessionOptions.

get_state()

io_binding()

Return an onnxruntime.IOBinding object`.

is_output_fp32_node(output_name)

load_state(dict, strict=False)

run(output_names, input_feed, run_options=None)

Compute the predictions.

Parameters

• output_names – name of the outputs

• input_feed – dictionary { input_name: input_value }

• run_options – See onnxruntime.RunOptions.

sess.run([output_name], {input_name: x})

run_with_iobinding(iobinding, run_options=None)

Compute the predictions.

Parameters

• iobinding – the iobinding object that has graph inputs/outputs bind.

• run_options – See onnxruntime.RunOptions.

run_with_ort_values(output_names, input_dict_ort_values, run_options=None)

Compute the predictions.

Parameters

• output_names – name of the outputs

• input_feed – dictionary { input_name: input_ort_value } See OrtValue class how to create OrtValue from numpy array or SparseTensor

• run_options – See onnxruntime.RunOptions.

Returns

an array of OrtValue

sess.run([output_name], {input_name: x})

set_providers(providers=None, provider_options=None)

Register the input list of execution providers. The underlying session is re-created.

Parameters

• providers – Optional sequence of providers in order of decreasing precedence. Values can either be provider names or tuples of (provider name, options dict). If not provided, then all available providers are used with the default precedence.

• provider_options – Optional sequence of options dicts corresponding to the providers listed in ‘providers’.

‘providers’ can contain either names or names and options. When any options are given in ‘providers’, ‘provider_options’ should not be used.

The list of providers is ordered by precedence. For example [‘CUDAExecutionProvider’, ‘CPUExecutionProvider’] means execute a node using CUDAExecutionProvider if capable, otherwise execute using CPUExecutionProvider.

C Class TrainingSession

class onnxruntime.capi._pybind_state.OrtValue

__init__(*args, **kwargs)

as_sparse_tensor(self: onnxruntime.capi.onnxruntime_pybind11_state.OrtValue) → onnxruntime::python::PySparseTensor

data_ptr(self: onnxruntime.capi.onnxruntime_pybind11_state.OrtValue) → int

data_type(self: onnxruntime.capi.onnxruntime_pybind11_state.OrtValue) → str

device_name(self: onnxruntime.capi.onnxruntime_pybind11_state.OrtValue) → str

static from_dlpack(arg0: object, arg1: bool) → onnxruntime.capi.onnxruntime_pybind11_state.OrtValue

has_value(self: onnxruntime.capi.onnxruntime_pybind11_state.OrtValue) → bool

is_sparse_tensor(self: onnxruntime.capi.onnxruntime_pybind11_state.OrtValue) → bool

is_tensor(self: onnxruntime.capi.onnxruntime_pybind11_state.OrtValue) → bool

is_tensor_sequence(self: onnxruntime.capi.onnxruntime_pybind11_state.OrtValue) → bool

numpy(self: onnxruntime.capi.onnxruntime_pybind11_state.OrtValue) → object

static ort_value_from_sparse_tensor(arg0: onnxruntime::python::PySparseTensor) → onnxruntime.capi.onnxruntime_pybind11_state.OrtValue

static ortvalue_from_numpy(arg0: object, arg1: onnxruntime.capi.onnxruntime_pybind11_state.OrtDevice) → onnxruntime.capi.onnxruntime_pybind11_state.OrtValue

static ortvalue_from_shape_and_type(arg0: List[int], arg1: object, arg2: onnxruntime.capi.onnxruntime_pybind11_state.OrtDevice) → onnxruntime.capi.onnxruntime_pybind11_state.OrtValue

shape(self: onnxruntime.capi.onnxruntime_pybind11_state.OrtValue) → list

to_dlpack(self: onnxruntime.capi.onnxruntime_pybind11_state.OrtValue) → object

previous

Inference

next

Partial Training