.. _f-onnxinference: module ``onnxrt.onnx_inference`` ================================ .. inheritance-diagram:: mlprodict.onnxrt.onnx_inference Short summary +++++++++++++ module ``mlprodict.onnxrt.onnx_inference`` Implements a class able to compute the predictions from on an :epkg:`ONNX` model. :githublink:`%|py|7` Classes +++++++ +------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------+ | class | truncated documentation | +========================================================================+=====================================================================================================================+ | :class:`OnnxInference ` | Loads an :epkg:`ONNX` file or object or stream. Computes the output of the :epkg:`ONNX` graph. Several runtimes ... | +------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------+ Properties ++++++++++ +-------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------+ | property | truncated documentation | +=============================================================================================================+=================================================================================================================+ | :meth:`input_names ` | Returns the names of all inputs. It does not include the optional inputs. | +-------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------+ | :meth:`input_names_shapes ` | Returns the names and shapes of all inputs. This method assumes all inputs are tensors. It does not include ... | +-------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------+ | :meth:`input_names_shapes_types ` | Returns the names, shapes, types of all inputs. This method assumes all inputs are tensors. It does not ... | +-------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------+ | :meth:`optional_inputs ` | Returns the list of optional inputs (the model has an initalizer of the same name as one input). | +-------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------+ | :meth:`output_names ` | Returns the names of all outputs. | +-------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------+ | :meth:`output_names_shapes ` | Returns the names and shapes of all outputs. This method assumes all inputs are tensors. | +-------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------+ | :meth:`output_names_shapes_types ` | Returns the names, shapes, types of all outputs. This method assumes all inputs are tensors. It does not ... | +-------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------+ Static Methods ++++++++++++++ +--------------------------------------------------------------------------------------------------+-------------------------+ | staticmethod | truncated documentation | +==================================================================================================+=========================+ | :py:meth:`_get_type_property ` | | +--------------------------------------------------------------------------------------------------+-------------------------+ Methods +++++++ +--------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------+ | method | truncated documentation | +==========================================================================================================================+====================================================================================================================================+ | :py:meth:`__getitem__ ` | Returns the ONNX verions of a node. | +--------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------+ | :py:meth:`__getstate__ ` | To pickle the object. | +--------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------+ | :py:meth:`__init__ ` | | +--------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------+ | :py:meth:`__repr__ ` | usual | +--------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------+ | :py:meth:`__setstate__ ` | To unpickle the object. | +--------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------+ | :py:meth:`__str__ ` | usual | +--------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------+ | :py:meth:`_build_compile_run ` | Rewrite the run function in python, compiles it, and adds it as a method. | +--------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------+ | :py:meth:`_guess_inplace ` | Looks into every node of the graph to see if there is a way to do the computation inplace. By default (*input_inplace=False*), ... | +--------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------+ | :py:meth:`_guess_input_dtype ` | | +--------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------+ | :py:meth:`_init ` | Prepares the instance to deliver predictions. | +--------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------+ | :py:meth:`_run_sequence_runtime ` | | +--------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------+ | :py:meth:`_run_sequence_runtime_compiled ` | Executes a compiled version of :meth:`_run_sequence_runtime`, compiled with method :meth:`_build_compile_run`. ... | +--------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------+ | :py:meth:`_run_whole_runtime ` | | +--------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------+ | :py:meth:`_set_shape_inference_runtime ` | Set shapes based on shape inference relying on the runtime. The values are stored in every node. | +--------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------+ | :py:meth:`_validate_outputs ` | Checks the output have the expected type. The function returns the list of mismatches. | +--------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------+ | :meth:`build_intermediate ` | Builds every possible :epkg:`ONNX` file which computes one specific intermediate output from the inputs. ... | +--------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------+ | :meth:`check_onnx ` | Checks the model follow :epkg:`ONNX` conventions. | +--------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------+ | :meth:`display_sequence ` | Shows the sequence of nodes to run if ``runtime=='python'``. | +--------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------+ | :meth:`get_execution_order ` | This function returns a dictionary `{(kind, name): (order, op)}`, *name* can be a node name or a result name. In ... | +--------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------+ | :meth:`get_profiling ` | Returns the profiling after a couple of execution. | +--------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------+ | :meth:`global_index ` | Maps every name to one integer to avoid using dictionaries when running the predictions. | +--------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------+ | :meth:`infer_shapes ` | Computes expected shapes. | +--------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------+ | :meth:`reduce_size ` | Reduces the memory footprint as much as possible. | +--------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------+ | :meth:`run ` | Computes the predictions for this :epkg:`onnx` graph. | +--------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------+ | :meth:`run2onnx ` | Executes the graphs with the given inputs, then adds the intermediate results into ONNX nodes in the original graph. ... | +--------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------+ | :meth:`shape_inference ` | Infers the shape of the outputs with :epkg:`onnx` package. | +--------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------+ | :meth:`switch_initializers_dtype ` | Switches all initializers to ``numpy.float64``. If *model* is None, a simple cast is done. Otherwise, the function ... | +--------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------+ | :meth:`to_sequence ` | Produces a graph to facilitate the execution. One example: | +--------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------+ Documentation +++++++++++++ .. automodule:: mlprodict.onnxrt.onnx_inference :members: :special-members: __init__ :show-inheritance: