Python Runtime for ONNX operators#

The main function instantiates a runtime class which computes the outputs of a specific node.

mlprodict.onnxrt.ops.load_op (onnx_node, desc = None, options = None, variables = None, dtype = None, runtime = None)

Sets up a class for a specific ONNX operator.

Other sections documents available operators. This project was mostly started to show a way to implement a custom runtime, do some benchmarks, test, exepriment…

Python#

Abs#

mlprodict.onnxrt.ops_cpu.op_abs.Abs (self, onnx_node, desc = None, options)

Absolute takes one input data (Tensor<T>) and produces one output data (Tensor<T>) where the absolute is, y = abs(x), is applied to the tensor elementwise.

Inputs

  • X (heterogeneous)T: Input tensor

Outputs

  • Y (heterogeneous)T: Output tensor

Type Constraints

  • T tensor(uint8), tensor(uint16), tensor(uint32), tensor(uint64), tensor(int8), tensor(int16), tensor(int32), tensor(int64), tensor(float16), tensor(float), tensor(double), tensor(bfloat16): Constrain input and output types to all numeric tensors.

Version

Onnx name: Abs

This version of the operator has been available since version 13.

Runtime implementation: Abs

Acos#

mlprodict.onnxrt.ops_cpu.op_acos.Acos (self, onnx_node, desc = None, options)

Calculates the arccosine (inverse of cosine) of the given input tensor, element-wise.

Inputs

  • input (heterogeneous)T: Input tensor

Outputs

  • output (heterogeneous)T: The arccosine of the input tensor computed element-wise

Type Constraints

  • T tensor(float16), tensor(float), tensor(double): Constrain input and output types to float tensors.

Version

Onnx name: Acos

This version of the operator has been available since version 7.

Runtime implementation: Acos

Acosh#

mlprodict.onnxrt.ops_cpu.op_acosh.Acosh (self, onnx_node, desc = None, options)

Calculates the hyperbolic arccosine of the given input tensor element-wise.

Inputs

  • input (heterogeneous)T: Input tensor

Outputs

  • output (heterogeneous)T: The hyperbolic arccosine values of the input tensor computed element-wise

Type Constraints

  • T tensor(float16), tensor(float), tensor(double): Constrain input and output types to float tensors.

Version

Onnx name: Acosh

This version of the operator has been available since version 9.

Runtime implementation: Acosh

Adagrad#

mlprodict.onnxrt.ops_cpu.op_adagrad.Adagrad (self, onnx_node, desc = None, options)

Compute one iteration of ADAGRAD, a stochastic gradient based optimization algorithm. This operator can conduct the optimization of multiple tensor variables.

Let’s define the behavior of this operator. As you can imagine, ADAGRAD requires some parameters:

  • The initial learning-rate “R”.

  • The update count “T”. That is, the number of training iterations conducted.

  • A L2-norm regularization coefficient “norm_coefficient”.

  • A learning-rate decay factor “decay_factor”.

  • A small constant “epsilon” to avoid dividing-by-zero.

At each ADAGRAD iteration, the optimized tensors are moved along a direction computed based on their estimated gradient and accumulated squared gradient. Assume that only a single tensor “X” is updated by this operator. We need the value of “X”, its gradient “G”, and its accumulated squared gradient “H”. Therefore, variables in this operator’s input list are sequentially “R”, “T”, “X”, “G”, and “H”. Other parameters are given as attributes because they are usually constants. Also, the corresponding output tensors are the new value of “X” (called “X_new”), and then the new accumulated squared gradient (called “H_new”). Those outputs are computed from the given inputs following the pseudo code below.

Let “+”, “-”, “*”, and “/” are all element-wise arithmetic operations with numpy-style broadcasting support. The pseudo code to compute those outputs is:

// Compute a scalar learning-rate factor. At the first update of X, T is generally // 0 (0-based update index) or 1 (1-based update index). r = R / (1 + T * decay_factor);

// Add gradient of 0.5 * norm_coefficient * ||X||_2^2, where ||X||_2 is the 2-norm. G_regularized = norm_coefficient * X + G;

// Compute new accumulated squared gradient. H_new = H + G_regularized * G_regularized;

// Compute the adaptive part of per-coordinate learning rate. Note that Sqrt(…) // computes element-wise square-root. H_adaptive = Sqrt(H_new) + epsilon

// Compute the new value of “X”. X_new = X - r * G_regularized / H_adaptive;

If one assign this operators to optimize multiple inputs, for example, “X_1” and “X_2”, the same pseudo code may be extended to handle all tensors jointly. More specifically, we can view “X” as a concatenation of “X_1” and “X_2” (of course, their gradient and accumulate gradient should be concatenated too) and then just reuse the entire pseudo code.

Note that ADAGRAD was first proposed in http://jmlr.org/papers/volume12/duchi11a/duchi11a.pdf. In that reference paper, this operator is a special case of the Figure 1’s composite mirror descent update.

Attributes

  • decay_factor: The decay factor of learning rate after one update.The effective learning rate is computed by r = R / (1 + T * decay_factor). Default to 0 so that increasing update counts doesn’t reduce the learning rate. Default value is namedecayfactorf0.0typeFLOAT (FLOAT)

  • epsilon: Small scalar to avoid dividing by zero. Default value is nameepsilonf9.999999974752427e-07typeFLOAT (FLOAT)

  • norm_coefficient: Regularization coefficient in 0.5 * norm_coefficient * ||X||_2^2. Default to 0, which means no regularization. Default value is namenormcoefficientf0.0typeFLOAT (FLOAT)

Inputs

Between 3 and 2147483647 inputs.

  • R (heterogeneous)T1: The initial learning rate.

  • T (heterogeneous)T2: The update count of “X”. It should be a scalar.

  • inputs (variadic)T3: The current values of optimized tensors, followed by their respective gradients, followed by their respective accumulated squared gradients.For example, if two tensor “X_1” and “X_2” are optimized, The input list would be [“X_1”, “X_2”, gradient of “X_1”, gradient of “X_2”, accumulated squared gradient of “X_1”, accumulated squared gradient of “X_2”].

Outputs

Between 1 and 2147483647 outputs.

  • outputs (variadic)T3: Updated values of optimized tensors, followed by their updated values of accumulated squared gradients. For example, if two tensor “X_1” and “X_2” are optimized, the output list would be [new value of “X_1,” new value of “X_2” new accumulated squared gradient of “X_1”, new accumulated squared gradient of “X_2”].

Type Constraints

  • T1 tensor(float), tensor(double): Constrain input types to float scalars.

  • T2 tensor(int64): Constrain input types to 64-bit integer scalars.

  • T3 tensor(float), tensor(double): Constrain input and output types to float tensors.

Version

Onnx name: Adagrad

This version of the operator has been available since version 1 of domain ai.onnx.preview.training.

Runtime implementation: Adagrad

Adam#

mlprodict.onnxrt.ops_cpu.op_adam.Adam (self, onnx_node, desc = None, options)

Compute one iteration of Adam, a stochastic gradient based optimization algorithm. This operator can conduct the optimization of multiple tensor variables.

Let’s define the behavior of this operator. First of all, Adam requires some parameters:

  • The learning-rate “R”.

  • The update count “T”. That is, the number of training iterations conducted.

  • A L2-norm regularization coefficient “norm_coefficient”.

  • A small constant “epsilon” to avoid dividing-by-zero.

  • Two coefficients, “alpha” and “beta”.

At each Adam iteration, the optimized tensors are moved along a direction computed based on their exponentially-averaged historical gradient and exponentially-averaged historical squared gradient. Assume that only a tensor “X” is being optimized. The rest of required information is

  • the value of “X”,

  • “X“‘s gradient (denoted by “G”),

  • “X“‘s exponentially-averaged historical gradient (denoted by “V”), and

  • “X“‘s exponentially-averaged historical squared gradient (denoted by “H”).

Some of those parameters are passed into this operator as input tensors and others are stored as this operator’s attributes. Specifically, this operator’s input tensor list is [“R”, “T”, “X”, “G”, “V”, “H”]. That is, “R” is the first input, “T” is the second input, and so on. Other parameters are given as attributes because they are constants. Moreover, the corresponding output tensors are

  • the new value of “X” (called “X_new”),

  • the new exponentially-averaged historical gradient (denoted by “V_new”), and

  • the new exponentially-averaged historical squared gradient (denoted by “H_new”).

Those outputs are computed following the pseudo code below.

Let “+”, “-”, “*”, and “/” are all element-wise arithmetic operations with numpy-style broadcasting support. The pseudo code to compute those outputs is:

// Add gradient of 0.5 * norm_coefficient * ||X||_2^2, where ||X||_2 is the 2-norm. G_regularized = norm_coefficient * X + G

// Update exponentially-averaged historical gradient. V_new = alpha * V + (1 - alpha) * G_regularized

// Update exponentially-averaged historical squared gradient. H_new = beta * H + (1 - beta) * G_regularized * G_regularized

// Compute the element-wise square-root of H_new. V_new will be element-wisely // divided by H_sqrt for a better update direction. H_sqrt = Sqrt(H_new) + epsilon

// Compute learning-rate. Note that “alpha**T”/”beta**T” is alpha’s/beta’s T-th power. R_adjusted = T > 0 ? R * Sqrt(1 - beta**T) / (1 - alpha**T) : R

// Compute new value of “X”. X_new = X - R_adjusted * V_new / H_sqrt

// Post-update regularization. X_final = (1 - norm_coefficient_post) * X_new

If there are multiple inputs to be optimized, the pseudo code will be applied independently to each of them.

Attributes

  • alpha: Coefficient of previously accumulated gradient in running average. Default to 0.9. Default value is namealphaf0.8999999761581421typeFLOAT (FLOAT)

  • beta: Coefficient of previously accumulated squared-gradient in running average. Default to 0.999. Default value is namebetaf0.9990000128746033typeFLOAT (FLOAT)

  • epsilon: Small scalar to avoid dividing by zero. Default value is nameepsilonf9.999999974752427e-07typeFLOAT (FLOAT)

  • norm_coefficient: Regularization coefficient of 0.5 * norm_coefficient * ||X||_2^2. Default to 0, which means no regularization. Default value is namenormcoefficientf0.0typeFLOAT (FLOAT)

  • norm_coefficient_post: Regularization coefficient of 0.5 * norm_coefficient * ||X||_2^2. Default to 0, which means no regularization. Default value is namenormcoefficientpostf0.0typeFLOAT (FLOAT)

Inputs

Between 3 and 2147483647 inputs.

  • R (heterogeneous)T1: The initial learning rate.

  • T (heterogeneous)T2: The update count of “X”. It should be a scalar.

  • inputs (variadic)T3: The tensors to be optimized, followed by their respective gradients, followed by their respective accumulated gradients (aka momentum), followed by their respective accumulated squared gradients. For example, to optimize tensors “X_1” and “X_2,”, the input list would be [“X_1”, “X_2”, gradient of “X_1”, gradient of “X_2”, accumulated gradient of “X_1”, accumulated gradient of “X_2”, accumulated squared gradient of “X_1”, accumulated squared gradient of “X_2”].

Outputs

Between 1 and 2147483647 outputs.

  • outputs (variadic)T3: New values of optimized tensors, followed by their respective new accumulated gradients, followed by their respective new accumulated squared gradients. For example, if two tensors “X_1” and “X_2” are optimized, the outputs list would be [new value of “X_1”, new value of “X_2”, new accumulated gradient of “X_1”, new accumulated gradient of “X_2”, new accumulated squared gradient of “X_1”, new accumulated squared gradient of “X_2”].

Type Constraints

  • T1 tensor(float), tensor(double): Constrain input types to float scalars.

  • T2 tensor(int64): Constrain input types to 64-bit integer scalars.

  • T3 tensor(float), tensor(double): Constrain input and output types to float tensors.

Version

Onnx name: Adam

This version of the operator has been available since version 1 of domain ai.onnx.preview.training.

Runtime implementation: Adam

Add#

mlprodict.onnxrt.ops_cpu.op_add.Add (self, onnx_node, desc = None, options)

Performs element-wise binary addition (with Numpy-style broadcasting support).

This operator supports multidirectional (i.e., Numpy-style) broadcasting; for more details please check Broadcasting in ONNX.

(Opset 14 change): Extend supported types to include uint8, int8, uint16, and int16.

Inputs

  • A (heterogeneous)T: First operand.

  • B (heterogeneous)T: Second operand.

Outputs

  • C (heterogeneous)T: Result, has same element type as two inputs

Type Constraints

  • T tensor(uint8), tensor(uint16), tensor(uint32), tensor(uint64), tensor(int8), tensor(int16), tensor(int32), tensor(int64), tensor(float16), tensor(float), tensor(double), tensor(bfloat16): Constrain input and output types to all numeric tensors.

Version

Onnx name: Add

This version of the operator has been available since version 14.

Runtime implementation: Add

And#

mlprodict.onnxrt.ops_cpu.op_and.And (self, onnx_node, desc = None, options)

Returns the tensor resulted from performing the and logical operation elementwise on the input tensors A and B (with Numpy-style broadcasting support).

This operator supports multidirectional (i.e., Numpy-style) broadcasting; for more details please check Broadcasting in ONNX.

Inputs

  • A (heterogeneous)T: First input operand for the logical operator.

  • B (heterogeneous)T: Second input operand for the logical operator.

Outputs

  • C (heterogeneous)T1: Result tensor.

Type Constraints

  • T tensor(bool): Constrain input to boolean tensor.

  • T1 tensor(bool): Constrain output to boolean tensor.

Version

Onnx name: And

This version of the operator has been available since version 7.

Runtime implementation: And

ArgMax_12#

mlprodict.onnxrt.ops_cpu.op_argmax.ArgMax_12 (self, onnx_node, desc = None, options)

Computes the indices of the max elements of the input tensor’s element along the provided axis. The resulting tensor has the same rank as the input if keepdims equals 1. If keepdims equal 0, then the resulting tensor has the reduced dimension pruned. If select_last_index is True (default False), the index of the last occurrence of the max is selected if the max appears more than once in the input. Otherwise the index of the first occurrence is selected. The type of the output tensor is integer.

Attributes

  • axis: The axis in which to compute the arg indices. Accepted range is [-r, r-1] where r = rank(data). Default value is nameaxisi0typeINT (INT)

  • keepdims: Keep the reduced dimension or not, default 1 means keep reduced dimension. Default value is namekeepdimsi1typeINT (INT)

  • select_last_index: Whether to select the last index or the first index if the {name} appears in multiple indices, default is False (first index). Default value is nameselectlastindexi0typeINT (INT)

Inputs

  • data (heterogeneous)T: An input tensor.

Outputs

  • reduced (heterogeneous)tensor(int64): Reduced output tensor with integer data type.

Type Constraints

  • T tensor(uint8), tensor(uint16), tensor(uint32), tensor(uint64), tensor(int8), tensor(int16), tensor(int32), tensor(int64), tensor(float16), tensor(float), tensor(double): Constrain input and output types to all numeric tensors.

Version

Onnx name: ArgMax

This version of the operator has been available since version 12.

Runtime implementation: ArgMax

ArgMin_12#

mlprodict.onnxrt.ops_cpu.op_argmin.ArgMin_12 (self, onnx_node, desc = None, options)

Computes the indices of the min elements of the input tensor’s element along the provided axis. The resulting tensor has the same rank as the input if keepdims equals 1. If keepdims equal 0, then the resulting tensor has the reduced dimension pruned. If select_last_index is True (default False), the index of the last occurrence of the min is selected if the min appears more than once in the input. Otherwise the index of the first occurrence is selected. The type of the output tensor is integer.

Attributes

  • axis: The axis in which to compute the arg indices. Accepted range is [-r, r-1] where r = rank(data). Default value is nameaxisi0typeINT (INT)

  • keepdims: Keep the reduced dimension or not, default 1 means keep reduced dimension. Default value is namekeepdimsi1typeINT (INT)

  • select_last_index: Whether to select the last index or the first index if the {name} appears in multiple indices, default is False (first index). Default value is nameselectlastindexi0typeINT (INT)

Inputs

  • data (heterogeneous)T: An input tensor.

Outputs

  • reduced (heterogeneous)tensor(int64): Reduced output tensor with integer data type.

Type Constraints

  • T tensor(uint8), tensor(uint16), tensor(uint32), tensor(uint64), tensor(int8), tensor(int16), tensor(int32), tensor(int64), tensor(float16), tensor(float), tensor(double): Constrain input and output types to all numeric tensors.

Version

Onnx name: ArgMin

This version of the operator has been available since version 12.

Runtime implementation: ArgMin

ArrayFeatureExtractor#

mlprodict.onnxrt.ops_cpu.op_array_feature_extractor.ArrayFeatureExtractor (self, onnx_node, desc = None, options)

Select elements of the input tensor based on the indices passed.

The indices are applied to the last axes of the tensor.

Inputs

  • X (heterogeneous)T: Data to be selected

  • Y (heterogeneous)tensor(int64): The indices, based on 0 as the first index of any dimension.

Outputs

  • Z (heterogeneous)T: Selected output data as an array

Type Constraints

  • T tensor(float), tensor(double), tensor(int64), tensor(int32), tensor(string): The input must be a tensor of a numeric type or string. The output will be of the same tensor type.

Version

Onnx name: ArrayFeatureExtractor

This version of the operator has been available since version 1 of domain ai.onnx.ml.

Runtime implementation: ArrayFeatureExtractor

Asin#

mlprodict.onnxrt.ops_cpu.op_asin.Asin (self, onnx_node, desc = None, options)

Calculates the arcsine (inverse of sine) of the given input tensor, element-wise.

Inputs

  • input (heterogeneous)T: Input tensor

Outputs

  • output (heterogeneous)T: The arcsine of the input tensor computed element-wise

Type Constraints

  • T tensor(float16), tensor(float), tensor(double): Constrain input and output types to float tensors.

Version

Onnx name: Asin

This version of the operator has been available since version 7.

Runtime implementation: Asin

Asinh#

mlprodict.onnxrt.ops_cpu.op_asinh.Asinh (self, onnx_node, desc = None, options)

Calculates the hyperbolic arcsine of the given input tensor element-wise.

Inputs

  • input (heterogeneous)T: Input tensor

Outputs

  • output (heterogeneous)T: The hyperbolic arcsine values of the input tensor computed element-wise

Type Constraints

  • T tensor(float16), tensor(float), tensor(double): Constrain input and output types to float tensors.

Version

Onnx name: Asinh

This version of the operator has been available since version 9.

Runtime implementation: Asinh

Atan#

mlprodict.onnxrt.ops_cpu.op_atan.Atan (self, onnx_node, desc = None, options)

Calculates the arctangent (inverse of tangent) of the given input tensor, element-wise.

Inputs

  • input (heterogeneous)T: Input tensor

Outputs

  • output (heterogeneous)T: The arctangent of the input tensor computed element-wise

Type Constraints

  • T tensor(float16), tensor(float), tensor(double): Constrain input and output types to float tensors.

Version

Onnx name: Atan

This version of the operator has been available since version 7.

Runtime implementation: Atan

Atanh#

mlprodict.onnxrt.ops_cpu.op_atanh.Atanh (self, onnx_node, desc = None, options)

Calculates the hyperbolic arctangent of the given input tensor element-wise.

Inputs

  • input (heterogeneous)T: Input tensor

Outputs

  • output (heterogeneous)T: The hyperbolic arctangent values of the input tensor computed element-wise

Type Constraints

  • T tensor(float16), tensor(float), tensor(double): Constrain input and output types to float tensors.

Version

Onnx name: Atanh

This version of the operator has been available since version 9.

Runtime implementation: Atanh

AveragePool#

mlprodict.onnxrt.ops_cpu.op_average_pool.AveragePool (self, onnx_node, desc = None, options)

AveragePool consumes an input tensor X and applies average pooling across the tensor according to kernel sizes, stride sizes, and pad lengths. average pooling consisting of computing the average on all values of a subset of the input tensor according to the kernel size and downsampling the data into the output tensor Y for further processing. The output spatial shape will be following: `` output_spatial_shape[i] = floor((input_spatial_shape[i] + pad_shape[i] - kernel_spatial_shape[i]) / strides_spatial_shape[i] + 1) `` or `` output_spatial_shape[i] = ceil((input_spatial_shape[i] + pad_shape[i] - kernel_spatial_shape[i]) / strides_spatial_shape[i] + 1) `` if ceil_mode is enabled

`` * pad_shape[i] is sum of pads along axis i ``

auto_pad is a DEPRECATED attribute. If you are using them currently, the output spatial shape will be following: `` VALID: output_spatial_shape[i] = ceil((input_spatial_shape[i] - kernel_spatial_shape[i] + 1) / strides_spatial_shape[i]) SAME_UPPER or SAME_LOWER: output_spatial_shape[i] = ceil(input_spatial_shape[i] / strides_spatial_shape[i]) `` And pad shape will be following if SAME_UPPER or SAME_LOWER: `` pad_shape[i] = (output_spatial_shape[i] - 1) * strides_spatial_shape[i] + kernel_spatial_shape[i] - input_spatial_shape[i] `` The output of each pooling window is divided by the number of elements (exclude pad when attribute count_include_pad is zero).

Attributes

  • auto_pad: auto_pad must be either NOTSET, SAME_UPPER, SAME_LOWER or VALID. Where default value is NOTSET, which means explicit padding is used. SAME_UPPER or SAME_LOWER mean pad the input so that output_shape[i] = ceil(input_shape[i] / strides[i]) for each axis i. The padding is split between the two sides equally or almost equally (depending on whether it is even or odd). In case the padding is an odd number, the extra padding is added at the end for SAME_UPPER and at the beginning for SAME_LOWER. Default value is nameautopadsNOTSETtypeSTRING (STRING)

  • ceil_mode: Whether to use ceil or floor (default) to compute the output shape. Default value is nameceilmodei0typeINT (INT)

  • count_include_pad: Whether include pad pixels when calculating values for the edges. Default is 0, doesn’t count include pad. Default value is namecountincludepadi0typeINT (INT)

  • kernel_shape (required): The size of the kernel along each axis. default value cannot be automatically retrieved (INTS)

  • pads: Padding for the beginning and ending along each spatial axis, it can take any value greater than or equal to 0. The value represent the number of pixels added to the beginning and end part of the corresponding axis. pads format should be as follow [x1_begin, x2_begin…x1_end, x2_end,…], where xi_begin the number of pixels added at the beginning of axis i and xi_end, the number of pixels added at the end of axis i. This attribute cannot be used simultaneously with auto_pad attribute. If not present, the padding defaults to 0 along start and end of each spatial axis. default value cannot be automatically retrieved (INTS)

  • strides: Stride along each spatial axis. If not present, the stride defaults to 1 along each spatial axis. default value cannot be automatically retrieved (INTS)

Inputs

  • X (heterogeneous)T: Input data tensor from the previous operator; dimensions for image case are (N x C x H x W), where N is the batch size, C is the number of channels, and H and W are the height and the width of the data. For non image case, the dimensions are in the form of (N x C x D1 x D2 … Dn), where N is the batch size. Optionally, if dimension denotation is in effect, the operation expects the input data tensor to arrive with the dimension denotation of [DATA_BATCH, DATA_CHANNEL, DATA_FEATURE, DATA_FEATURE …].

Outputs

  • Y (heterogeneous)T: Output data tensor from average or max pooling across the input tensor. Dimensions will vary based on various kernel, stride, and pad sizes. Floor value of the dimension is used

Type Constraints

  • T tensor(float16), tensor(float), tensor(double): Constrain input and output types to float tensors.

Version

Onnx name: AveragePool

This version of the operator has been available since version 11.

Runtime implementation: AveragePool

BatchNormalization_14#

mlprodict.onnxrt.ops_cpu.op_batch_normalization.BatchNormalization_14 (self, onnx_node, desc = None, options)

Carries out batch normalization as described in the paper https://arxiv.org/abs/1502.03167. Depending on the mode it is being run, There are five required inputs ‘X’, ‘scale’, ‘B’, ‘input_mean’ and ‘input_var’. Note that ‘input_mean’ and ‘input_var’ are expected to be the estimated statistics in inference mode (training_mode=False, default), and the running statistics in training mode (training_mode=True). There are multiple cases for the number of outputs, which we list below:

Output case #1: Y, running_mean, running_var (training_mode=True) Output case #2: Y (training_mode=False)

When training_mode=False, extra outputs are invalid. The outputs are updated as follows when training_mode=True: `` running_mean = input_mean * momentum + current_mean * (1 - momentum) running_var = input_var * momentum + current_var * (1 - momentum)

Y = (X - current_mean) / sqrt(current_var + epsilon) * scale + B

where:

current_mean = ReduceMean(X, axis=all_except_channel_index) current_var = ReduceVar(X, axis=all_except_channel_index)

Notice that ReduceVar refers to the population variance, and it equals to sum(sqrd(x_i - x_avg)) / N where N is the population size (this formula does not use sample size N - 1).

``

When training_mode=False: `` Y = (X - input_mean) / sqrt(input_var + epsilon) * scale + B ``

For previous (depreciated) non-spatial cases, implementors are suggested to flatten the input shape to (N x C * D1 * D2 * … * Dn) before a BatchNormalization Op. This operator has optional inputs/outputs. See ONNX for more details about the representation of optional arguments. An empty string may be used in the place of an actual argument’s name to indicate a missing argument. Trailing optional arguments (those not followed by an argument that is present) may also be simply omitted.

Attributes

  • epsilon: The epsilon value to use to avoid division by zero. Default value is nameepsilonf9.999999747378752e-06typeFLOAT (FLOAT)

  • momentum: Factor used in computing the running mean and variance.e.g., running_mean = running_mean * momentum + mean * (1 - momentum). Default value is namemomentumf0.8999999761581421typeFLOAT (FLOAT)

  • training_mode: If set to true, it indicates BatchNormalization is being used for training, and outputs 1, 2, 3, and 4 would be populated. Default value is nametrainingmodei0typeINT (INT)

Inputs

  • X (heterogeneous)T: Input data tensor from the previous operator; dimensions are in the form of (N x C x D1 x D2 … Dn), where N is the batch size, C is the number of channels. Statistics are computed for every channel of C over N and D1 to Dn dimensions. For image data, input dimensions become (N x C x H x W). The op also accepts single dimension input of size N in which case C is assumed to be 1

  • scale (heterogeneous)T: Scale tensor of shape (C).

  • B (heterogeneous)T: Bias tensor of shape (C).

  • input_mean (heterogeneous)U: running (training) or estimated (testing) mean tensor of shape (C).

  • input_var (heterogeneous)U: running (training) or estimated (testing) variance tensor of shape (C).

Outputs

Between 1 and 3 outputs.

  • Y (heterogeneous)T: The output tensor of the same shape as X

  • running_mean (optional, heterogeneous)U: The running mean after the BatchNormalization operator.

  • running_var (optional, heterogeneous)U: The running variance after the BatchNormalization operator. This op uses the population size (N) for calculating variance, and not the sample size N-1.

Type Constraints

  • T tensor(float16), tensor(float), tensor(double), tensor(bfloat16): Constrain input and output types to float tensors.

  • U tensor(float16), tensor(float), tensor(double), tensor(bfloat16): Constrain mean and variance types to float tensors. It allows all float type for U.

Version

Onnx name: BatchNormalization

This version of the operator has been available since version 14.

Runtime implementation: BatchNormalization

Bernoulli#

mlprodict.onnxrt.ops_cpu.op_random.Bernoulli (self, onnx_node, desc = None, options)

Draws binary random numbers (0 or 1) from a Bernoulli distribution. The input tensor should be a tensor containing probabilities p (a value in the range [0,1]) to be used for drawing the binary random number, where an output of 1 is produced with probability p and an output of 0 is produced with probability (1-p).

This operator is non-deterministic and may not produce the same values in different implementations (even if a seed is specified).

Attributes

  • dtype: The data type for the elements of the output tensor. if not specified, we will use the data type of the input tensor. default value cannot be automatically retrieved (INT)

  • seed: (Optional) Seed to the random generator, if not specified we will auto generate one. default value cannot be automatically retrieved (FLOAT)

Inputs

  • input (heterogeneous)T1: All values in input have to be in the range:[0, 1].

Outputs

  • output (heterogeneous)T2: The returned output tensor only has values 0 or 1, same shape as input tensor.

Type Constraints

  • T1 tensor(float16), tensor(float), tensor(double): Constrain input types to float tensors.

  • T2 tensor(float16), tensor(float), tensor(double), tensor(bfloat16), tensor(uint8), tensor(uint16), tensor(uint32), tensor(uint64), tensor(int8), tensor(int16), tensor(int32), tensor(int64), tensor(bool): Constrain output types to all numeric tensors and bool tensors.

Version

Onnx name: Bernoulli

This version of the operator has been available since version 15.

Runtime implementation: Bernoulli

Binarizer#

mlprodict.onnxrt.ops_cpu.op_binarizer.Binarizer (self, onnx_node, desc = None, options)

Maps the values of the input tensor to either 0 or 1, element-wise, based on the outcome of a comparison against a threshold value.

Attributes

  • threshold: Values greater than this are mapped to 1, others to 0. Default value is namethresholdf0.0typeFLOAT (FLOAT)

Inputs

  • X (heterogeneous)T: Data to be binarized

Outputs

  • Y (heterogeneous)T: Binarized output data

Type Constraints

  • T tensor(float), tensor(double), tensor(int64), tensor(int32): The input must be a tensor of a numeric type. The output will be of the same tensor type.

Version

Onnx name: Binarizer

This version of the operator has been available since version 1 of domain ai.onnx.ml.

Runtime implementation: Binarizer

BitShift#

mlprodict.onnxrt.ops_cpu.op_bitshift.BitShift (self, onnx_node, desc = None, options)

Bitwise shift operator performs element-wise operation. For each input element, if the

attribute “direction” is “RIGHT”, this operator moves its binary representation toward the right side so that the input value is effectively decreased. If the attribute “direction” is “LEFT”, bits of binary representation moves toward the left side, which results the increase of its actual value. The input X is the tensor to be shifted and another input Y specifies the amounts of shifting. For example, if “direction” is “Right”, X is [1, 4], and S is [1, 1], the corresponding output Z would be [0, 2]. If “direction” is “LEFT” with X=[1, 2] and S=[1, 2], the corresponding output Y would be [2, 8].

Because this operator supports Numpy-style broadcasting, X’s and Y’s shapes are not necessarily identical.

This operator supports multidirectional (i.e., Numpy-style) broadcasting; for more details please check Broadcasting in ONNX.

Attributes

  • direction (required): Direction of moving bits. It can be either “RIGHT” (for right shift) or “LEFT” (for left shift). default value cannot be automatically retrieved (STRING)

Inputs

  • X (heterogeneous)T: First operand, input to be shifted.

  • Y (heterogeneous)T: Second operand, amounts of shift.

Outputs

  • Z (heterogeneous)T: Output tensor

Type Constraints

  • T tensor(uint8), tensor(uint16), tensor(uint32), tensor(uint64): Constrain input and output types to integer tensors.

Version

Onnx name: BitShift

This version of the operator has been available since version 11.

Runtime implementation: BitShift

BroadcastGradientArgs#

mlprodict.onnxrt.ops_cpu.op_broadcast_gradient_args.BroadcastGradientArgs (self, onnx_node, desc = None, options)

Version

Onnx name: BroadcastGradientArgs

This version of the operator has been available since version of domain mlprodict.

Runtime implementation: BroadcastGradientArgs

CDist#

mlprodict.onnxrt.ops_cpu.op_cdist.CDist (self, onnx_node, desc = None, options)

Version

Onnx name: CDist

This version of the operator has been available since version of domain mlprodict.

Runtime implementation: CDist

Cast#

mlprodict.onnxrt.ops_cpu.op_cast.Cast (self, onnx_node, desc = None, options)

The operator casts the elements of a given input tensor to a data type specified by the ‘to’ argument and returns an output tensor of the same size in the converted type. The ‘to’ argument must be one of the data types specified in the ‘DataType’ enum field in the TensorProto message.

Casting from string tensor in plain (e.g., “3.14” and “1000”) and scientific numeric representations (e.g., “1e-5” and “1E8”) to float types is supported. For example, converting string “100.5” to an integer may result 100. There are some string literals reserved for special floating-point values; “+INF” (and “INF”), “-INF”, and “NaN” are positive infinity, negative infinity, and not-a-number, respectively. Any string which can exactly match “+INF” in a case-insensitive way would be mapped to positive infinite. Similarly, this case-insensitive rule is applied to “INF” and “NaN”. When casting from numeric tensors to string tensors, plain floating-point representation (such as “314.15926”) would be used. Converting non-numerical-literal string such as “Hello World!” is an undefined behavior. Cases of converting string representing floating-point arithmetic value, such as “2.718”, to INT is an undefined behavior.

Conversion from a numerical type to any numerical type is always allowed. User must be aware of precision loss and value change caused by range difference between two types. For example, a 64-bit float 3.1415926459 may be round to a 32-bit float 3.141592. Similarly, converting an integer 36 to Boolean may produce 1 because we truncate bits which can’t be stored in the targeted type.

Attributes

  • to (required): The data type to which the elements of the input tensor are cast. Strictly must be one of the types from DataType enum in TensorProto default value cannot be automatically retrieved (INT)

Inputs

  • input (heterogeneous)T1: Input tensor to be cast.

Outputs

  • output (heterogeneous)T2: Output tensor with the same shape as input with type specified by the ‘to’ argument

Type Constraints

  • T1 tensor(float16), tensor(float), tensor(double), tensor(int8), tensor(int16), tensor(int32), tensor(int64), tensor(uint8), tensor(uint16), tensor(uint32), tensor(uint64), tensor(bool), tensor(string), tensor(bfloat16): Constrain input types. Casting from complex is not supported.

  • T2 tensor(float16), tensor(float), tensor(double), tensor(int8), tensor(int16), tensor(int32), tensor(int64), tensor(uint8), tensor(uint16), tensor(uint32), tensor(uint64), tensor(bool), tensor(string), tensor(bfloat16): Constrain output types. Casting to complex is not supported.

Version

Onnx name: Cast

This version of the operator has been available since version 13.

Runtime implementation: Cast

CastLike#

mlprodict.onnxrt.ops_cpu.op_cast.CastLike (self, onnx_node, desc = None, options)

The operator casts the elements of a given input tensor (the first input) to the same data type as the elements of the second input tensor. See documentation of the Cast operator for further details.

Inputs

  • input (heterogeneous)T1: Input tensor to be cast.

  • target_type (heterogeneous)T2: The (first) input tensor will be cast to produce a tensor of the same type as this (second input) tensor.

Outputs

  • output (heterogeneous)T2: Output tensor produced by casting the first input tensor to have the same type as the second input tensor.

Type Constraints

  • T1 tensor(float16), tensor(float), tensor(double), tensor(int8), tensor(int16), tensor(int32), tensor(int64), tensor(uint8), tensor(uint16), tensor(uint32), tensor(uint64), tensor(bool), tensor(string), tensor(bfloat16): Constrain input types. Casting from complex is not supported.

  • T2 tensor(float16), tensor(float), tensor(double), tensor(int8), tensor(int16), tensor(int32), tensor(int64), tensor(uint8), tensor(uint16), tensor(uint32), tensor(uint64), tensor(bool), tensor(string), tensor(bfloat16): Constrain output types. Casting to complex is not supported.

Version

Onnx name: CastLike

This version of the operator has been available since version 15.

Runtime implementation: CastLike

CategoryMapper#

mlprodict.onnxrt.ops_cpu.op_category_mapper.CategoryMapper (self, onnx_node, desc = None, options)

Converts strings to integers and vice versa.

Two sequences of equal length are used to map between integers and strings, with strings and integers at the same index detailing the mapping.

Each operator converts either integers to strings or strings to integers, depending on which default value attribute is provided. Only one default value attribute should be defined.

If the string default value is set, it will convert integers to strings. If the int default value is set, it will convert strings to integers.

Attributes

  • cats_int64s: The integers of the map. This sequence must be the same length as the ‘cats_strings’ sequence. default value cannot be automatically retrieved (INTS)

  • cats_strings: The strings of the map. This sequence must be the same length as the ‘cats_int64s’ sequence default value cannot be automatically retrieved (STRINGS)

  • default_int64: An integer to use when an input string value is not found in the map. One and only one of the ‘default_*’ attributes must be defined. Default value is namedefaultint64i-1typeINT (INT)

  • default_string: A string to use when an input integer value is not found in the map. One and only one of the ‘default_*’ attributes must be defined. Default value is namedefaultstringsUnusedtypeSTRING (STRING)

Inputs

  • X (heterogeneous)T1: Input data

Outputs

  • Y (heterogeneous)T2: Output data. If strings are input, the output values are integers, and vice versa.

Type Constraints

  • T1 tensor(string), tensor(int64): The input must be a tensor of strings or integers, either [N,C] or [C].

  • T2 tensor(string), tensor(int64): The output is a tensor of strings or integers. Its shape will be the same as the input shape.

Version

Onnx name: CategoryMapper

This version of the operator has been available since version 1 of domain ai.onnx.ml.

Runtime implementation: CategoryMapper

Ceil#

mlprodict.onnxrt.ops_cpu.op_ceil.Ceil (self, onnx_node, desc = None, options)

Ceil takes one input data (Tensor<T>) and produces one output data (Tensor<T>) where the ceil is, y = ceil(x), is applied to the tensor elementwise.

Inputs

  • X (heterogeneous)T: Input tensor

Outputs

  • Y (heterogeneous)T: Output tensor

Type Constraints

  • T tensor(float16), tensor(float), tensor(double), tensor(bfloat16): Constrain input and output types to float tensors.

Version

Onnx name: Ceil

This version of the operator has been available since version 13.

Runtime implementation: Ceil

Celu#

mlprodict.onnxrt.ops_cpu.op_celu.Celu (self, onnx_node, desc = None, options)

Continuously Differentiable Exponential Linear Units: Perform the linear unit element-wise on the input tensor X using formula:

`` max(0,x) + min(0,alpha*(exp(x/alpha)-1)) ``

Attributes

  • alpha: The Alpha value in Celu formula which control the shape of the unit. The default value is 1.0. Default value is namealphaf1.0typeFLOAT (FLOAT)

Inputs

  • X (heterogeneous)T: Input tensor

Outputs

  • Y (heterogeneous)T: Output tensor

Type Constraints

  • T tensor(float): Constrain input and output types to float32 tensors.

Version

Onnx name: Celu

This version of the operator has been available since version 12.

Runtime implementation: Celu

Clip_11#

mlprodict.onnxrt.ops_cpu.op_clip.Clip_11 (self, onnx_node, desc = None, options)

Clip operator limits the given input within an interval. The interval is specified by the inputs ‘min’ and ‘max’. They default to numeric_limits::lowest() and numeric_limits::max(), respectively.

Inputs

Between 1 and 3 inputs.

  • input (heterogeneous)T: Input tensor whose elements to be clipped

  • min (optional, heterogeneous)T: Minimum value, under which element is replaced by min. It must be a scalar(tensor of empty shape).

  • max (optional, heterogeneous)T: Maximum value, above which element is replaced by max. It must be a scalar(tensor of empty shape).

Outputs

  • output (heterogeneous)T: Output tensor with clipped input elements

Type Constraints

  • T tensor(float16), tensor(float), tensor(double): Constrain input and output types to float tensors.

Version

Onnx name: Clip

This version of the operator has been available since version 11.

Runtime implementation: Clip

ComplexAbs#

mlprodict.onnxrt.ops_cpu.op_complex_abs.ComplexAbs (self, onnx_node, desc = None, options)

Version

Onnx name: ComplexAbs

This version of the operator has been available since version of domain mlprodict.

Runtime implementation: ComplexAbs

Compress#

mlprodict.onnxrt.ops_cpu.op_compress.Compress (self, onnx_node, desc = None, options)

Selects slices from an input tensor along a given axis where condition evaluates to True for each axis index. In case axis is not provided, input is flattened before elements are selected. Compress behaves like numpy.compress: https://docs.scipy.org/doc/numpy/reference/generated/numpy.compress.html

Attributes

  • axis: (Optional) Axis along which to take slices. If not specified, input is flattened before elements being selected. Negative value means counting dimensions from the back. Accepted range is [-r, r-1] where r = rank(input). default value cannot be automatically retrieved (INT)

Inputs

  • input (heterogeneous)T: Tensor of rank r >= 1.

  • condition (heterogeneous)T1: Rank 1 tensor of booleans to indicate which slices or data elements to be selected. Its length can be less than the input length along the axis or the flattened input size if axis is not specified. In such cases data slices or elements exceeding the condition length are discarded.

Outputs

  • output (heterogeneous)T: Tensor of rank r if axis is specified. Otherwise output is a Tensor of rank 1.

Type Constraints

  • T tensor(uint8), tensor(uint16), tensor(uint32), tensor(uint64), tensor(int8), tensor(int16), tensor(int32), tensor(int64), tensor(float16), tensor(float), tensor(double), tensor(string), tensor(bool), tensor(complex64), tensor(complex128): Constrain input and output types to all tensor types.

  • T1 tensor(bool): Constrain to boolean tensors.

Version

Onnx name: Compress

This version of the operator has been available since version 11.

Runtime implementation: Compress

Concat#

mlprodict.onnxrt.ops_cpu.op_concat.Concat (self, onnx_node, desc = None, options)

Concatenate a list of tensors into a single tensor. All input tensors must have the same shape, except for the dimension size of the axis to concatenate on.

Attributes

  • axis (required): Which axis to concat on. A negative value means counting dimensions from the back. Accepted range is [-r, r-1] where r = rank(inputs).. default value cannot be automatically retrieved (INT)

Inputs

Between 1 and 2147483647 inputs.

  • inputs (variadic, heterogeneous)T: List of tensors for concatenation

Outputs

  • concat_result (heterogeneous)T: Concatenated tensor

Type Constraints

  • T tensor(uint8), tensor(uint16), tensor(uint32), tensor(uint64), tensor(int8), tensor(int16), tensor(int32), tensor(int64), tensor(bfloat16), tensor(float16), tensor(float), tensor(double), tensor(string), tensor(bool), tensor(complex64), tensor(complex128): Constrain output types to any tensor type.

Version

Onnx name: Concat

This version of the operator has been available since version 13.

Runtime implementation: Concat

ConcatFromSequence#

mlprodict.onnxrt.ops_cpu.op_concat_from_sequence.ConcatFromSequence (self, onnx_node, desc = None, options)

Concatenate a sequence of tensors into a single tensor. All input tensors must have the same shape, except for the dimension size of the axis to concatenate on. By default ‘new_axis’ is 0, the behavior is similar to numpy.concatenate. When ‘new_axis’ is 1, the behavior is similar to numpy.stack.

Attributes

  • axis (required): Which axis to concat on. Accepted range in [-r, r - 1], where r is the rank of input tensors. When new_axis is 1, accepted range is [-r - 1, r]. default value cannot be automatically retrieved (INT)

  • new_axis: Insert and concatenate on a new axis or not, default 0 means do not insert new axis. Default value is namenewaxisi0typeINT (INT)

Inputs

  • input_sequence (heterogeneous)S: Sequence of tensors for concatenation

Outputs

  • concat_result (heterogeneous)T: Concatenated tensor

Type Constraints

  • S seq(tensor(uint8)), seq(tensor(uint16)), seq(tensor(uint32)), seq(tensor(uint64)), seq(tensor(int8)), seq(tensor(int16)), seq(tensor(int32)), seq(tensor(int64)), seq(tensor(float16)), seq(tensor(float)), seq(tensor(double)), seq(tensor(string)), seq(tensor(bool)), seq(tensor(complex64)), seq(tensor(complex128)): Constrain input types to any tensor type.

  • T tensor(uint8), tensor(uint16), tensor(uint32), tensor(uint64), tensor(int8), tensor(int16), tensor(int32), tensor(int64), tensor(float16), tensor(float), tensor(double), tensor(string), tensor(bool), tensor(complex64), tensor(complex128): Constrain output types to any tensor type.

Version

Onnx name: ConcatFromSequence

This version of the operator has been available since version 11.

Runtime implementation: ConcatFromSequence

ConstantOfShape#

mlprodict.onnxrt.ops_cpu.op_constant_of_shape.ConstantOfShape (self, onnx_node, desc = None, options)

Generate a tensor with given value and shape.

Attributes

  • value: (Optional) The value of the output elements.Should be a one-element tensor. If not specified, it defaults to a tensor of value 0 and datatype float32 default value cannot be automatically retrieved (TENSOR)

Inputs

  • input (heterogeneous)T1: 1D tensor. The shape of the expected output tensor. If empty tensor is given, the output would be a scalar. All values must be >= 0.

Outputs

  • output (heterogeneous)T2: Output tensor of shape specified by ‘input’.If attribute ‘value’ is specified, the value and datatype of the output tensor is taken from ‘value’.If attribute ‘value’ is not specified, the value in the output defaults to 0, and the datatype defaults to float32.

Type Constraints

  • T1 tensor(int64): Constrain input types.

  • T2 tensor(float16), tensor(float), tensor(double), tensor(int8), tensor(int16), tensor(int32), tensor(int64), tensor(uint8), tensor(uint16), tensor(uint32), tensor(uint64), tensor(bool): Constrain output types to be numerics.

Version

Onnx name: ConstantOfShape

This version of the operator has been available since version 9.

Runtime implementation: ConstantOfShape

Constant_12#

mlprodict.onnxrt.ops_cpu.op_constant.Constant_12 (self, onnx_node, desc = None, options)

This operator produces a constant tensor. Exactly one of the provided attributes, either value, sparse_value, or value_* must be specified.

Attributes

  • sparse_value: The value for the elements of the output tensor in sparse format. default value cannot be automatically retrieved (SPARSE_TENSOR)

  • value: The value for the elements of the output tensor. default value cannot be automatically retrieved (TENSOR)

  • value_float: The value for the sole element for the scalar, float32, output tensor. default value cannot be automatically retrieved (FLOAT)

  • value_floats: The values for the elements for the 1D, float32, output tensor. default value cannot be automatically retrieved (FLOATS)

  • value_int: The value for the sole element for the scalar, int64, output tensor. default value cannot be automatically retrieved (INT)

  • value_ints: The values for the elements for the 1D, int64, output tensor. default value cannot be automatically retrieved (INTS)

  • value_string: The value for the sole element for the scalar, UTF-8 string, output tensor. default value cannot be automatically retrieved (STRING)

  • value_strings: The values for the elements for the 1D, UTF-8 string, output tensor. default value cannot be automatically retrieved (STRINGS)

Outputs

  • output (heterogeneous)T: Output tensor containing the same value of the provided tensor.

Type Constraints

  • T tensor(uint8), tensor(uint16), tensor(uint32), tensor(uint64), tensor(int8), tensor(int16), tensor(int32), tensor(int64), tensor(float16), tensor(float), tensor(double), tensor(string), tensor(bool), tensor(complex64), tensor(complex128): Constrain input and output types to all tensor types.

Version

Onnx name: Constant

This version of the operator has been available since version 12.

Runtime implementation: Constant

Conv#

mlprodict.onnxrt.ops_cpu.op_conv.Conv (self, onnx_node, desc = None, options)

The convolution operator consumes an input tensor and a filter, and computes the output.

Attributes

  • auto_pad: auto_pad must be either NOTSET, SAME_UPPER, SAME_LOWER or VALID. Where default value is NOTSET, which means explicit padding is used. SAME_UPPER or SAME_LOWER mean pad the input so that output_shape[i] = ceil(input_shape[i] / strides[i]) for each axis i. The padding is split between the two sides equally or almost equally (depending on whether it is even or odd). In case the padding is an odd number, the extra padding is added at the end for SAME_UPPER and at the beginning for SAME_LOWER. Default value is nameautopadsNOTSETtypeSTRING (STRING)

  • dilations: dilation value along each spatial axis of the filter. If not present, the dilation defaults is 1 along each spatial axis. default value cannot be automatically retrieved (INTS)

  • group: number of groups input channels and output channels are divided into. Default value is namegroupi1typeINT (INT)

  • kernel_shape: The shape of the convolution kernel. If not present, should be inferred from input W. default value cannot be automatically retrieved (INTS)

  • pads: Padding for the beginning and ending along each spatial axis, it can take any value greater than or equal to 0. The value represent the number of pixels added to the beginning and end part of the corresponding axis. pads format should be as follow [x1_begin, x2_begin…x1_end, x2_end,…], where xi_begin the number of pixels added at the beginning of axis i and xi_end, the number of pixels added at the end of axis i. This attribute cannot be used simultaneously with auto_pad attribute. If not present, the padding defaults to 0 along start and end of each spatial axis. default value cannot be automatically retrieved (INTS)

  • strides: Stride along each spatial axis. If not present, the stride defaults is 1 along each spatial axis. default value cannot be automatically retrieved (INTS)

Inputs

Between 2 and 3 inputs.

  • X (heterogeneous)T: Input data tensor from previous layer; has size (N x C x H x W), where N is the batch size, C is the number of channels, and H and W are the height and width. Note that this is for the 2D image. Otherwise the size is (N x C x D1 x D2 … x Dn). Optionally, if dimension denotation is in effect, the operation expects input data tensor to arrive with the dimension denotation of [DATA_BATCH, DATA_CHANNEL, DATA_FEATURE, DATA_FEATURE …].

  • W (heterogeneous)T: The weight tensor that will be used in the convolutions; has size (M x C/group x kH x kW), where C is the number of channels, and kH and kW are the height and width of the kernel, and M is the number of feature maps. For more than 2 dimensions, the kernel shape will be (M x C/group x k1 x k2 x … x kn), where (k1 x k2 x … kn) is the dimension of the kernel. Optionally, if dimension denotation is in effect, the operation expects the weight tensor to arrive with the dimension denotation of [FILTER_OUT_CHANNEL, FILTER_IN_CHANNEL, FILTER_SPATIAL, FILTER_SPATIAL …]. Assuming zero based indices for the shape array, X.shape[1] == (W.shape[1] * group) == C and W.shape[0] mod G == 0. Or in other words FILTER_IN_CHANNEL multiplied by the number of groups should be equal to DATA_CHANNEL and the number of feature maps M should be a multiple of the number of groups G.

  • B (optional, heterogeneous)T: Optional 1D bias to be added to the convolution, has size of M.

Outputs

  • Y (heterogeneous)T: Output data tensor that contains the result of the convolution. The output dimensions are functions of the kernel size, stride size, and pad lengths.

Type Constraints

  • T tensor(float16), tensor(float), tensor(double): Constrain input and output types to float tensors.

Version

Onnx name: Conv

This version of the operator has been available since version 11.

Runtime implementation: Conv

ConvTranspose#

mlprodict.onnxrt.ops_cpu.op_conv_transpose.ConvTranspose (self, onnx_node, desc = None, options)

The convolution transpose operator consumes an input tensor and a filter, and computes the output.

If the pads parameter is provided the shape of the output is calculated via the following equation:

output_shape[i] = stride[i] * (input_size[i] - 1) + output_padding[i] + ((kernel_shape[i] - 1) * dilations[i] + 1) - pads[start_i] - pads[end_i]

output_shape can also be explicitly specified in which case pads values are auto generated using these equations:

total_padding[i] = stride[i] * (input_size[i] - 1) + output_padding[i] + ((kernel_shape[i] - 1) * dilations[i] + 1) - output_shape[i] If (auto_pads == SAME_UPPER): pads[start_i] = total_padding[i]/2; pads[end_i] = total_padding[i] - (total_padding[i]/2) Else: pads[start_i] = total_padding[i] - (total_padding[i]/2); pads[end_i] = (total_padding[i]/2).

Attributes

  • auto_pad: auto_pad must be either NOTSET, SAME_UPPER, SAME_LOWER or VALID. Where default value is NOTSET, which means explicit padding is used. SAME_UPPER or SAME_LOWER mean pad the input so that output_shape[i] = input_shape[i] * strides[i] for each axis i. The padding is split between the two sides equally or almost equally (depending on whether it is even or odd). In case the padding is an odd number, the extra padding is added at the end for SAME_UPPER and at the beginning for SAME_LOWER. Default value is nameautopadsNOTSETtypeSTRING (STRING)

  • dilations: dilation value along each spatial axis of the filter. If not present, the dilation defaults to 1 along each spatial axis. default value cannot be automatically retrieved (INTS)

  • group: number of groups input channels and output channels are divided into. Default value is namegroupi1typeINT (INT)

  • kernel_shape: The shape of the convolution kernel. If not present, should be inferred from input W. default value cannot be automatically retrieved (INTS)

  • output_padding: Additional elements added to the side with higher coordinate indices in the output. Each padding value in “output_padding” must be less than the corresponding stride/dilation dimension. By default, this attribute is a zero vector. Note that this attribute doesn’t directly affect the computed output values. It only controls the selection of the computed values, so changing this attribute only adds or removes output elements. If “output_shape” is explicitly provided, “output_padding” does not contribute additional size to “output_shape” but participates in the computation of the needed padding amount. This is also called adjs or adjustment in some frameworks. default value cannot be automatically retrieved (INTS)

  • output_shape: The shape of the output can be explicitly set which will cause pads values to be auto generated. If output_shape is specified pads values are ignored. See doc for details for equations to generate pads default value cannot be automatically retrieved (INTS)

  • pads: Padding for the beginning and ending along each spatial axis, it can take any value greater than or equal to 0. The value represent the number of pixels added to the beginning and end part of the corresponding axis. pads format should be as follow [x1_begin, x2_begin…x1_end, x2_end,…], where xi_begin the number of pixels added at the beginning of axis i and xi_end, the number of pixels added at the end of axis i. This attribute cannot be used simultaneously with auto_pad attribute. If not present, the padding defaults to 0 along start and end of each spatial axis. default value cannot be automatically retrieved (INTS)

  • strides: Stride along each spatial axis. If not present, the stride defaults to 1 along each spatial axis. default value cannot be automatically retrieved (INTS)

Inputs

Between 2 and 3 inputs.

  • X (heterogeneous)T: Input data tensor from previous layer; has size (N x C x H x W), where N is the batch size, C is the number of channels, and H and W are the height and width. Note that this is for the 2D image. Otherwise the size is (N x C x D1 x D2 … x Dn)

  • W (heterogeneous)T: The weight tensor that will be used in the convolutions; has size (C x M/group x kH x kW), where C is the number of channels, and kH and kW are the height and width of the kernel, and M is the number of feature maps. For more than 2 dimensions, the weight shape will be (C x M/group x k1 x k2 x … x kn), where (k1 x k2 x … x kn) is the dimension of the kernel. The number of channels in the output should be equal to W.shape[1] * group (assuming zero based indices of the shape array)

  • B (optional, heterogeneous)T: Optional 1D bias to be added to the convolution, has size of M.

Outputs

  • Y (heterogeneous)T: Output data tensor that contains the result of the convolution. The output dimensions are functions of the kernel size, stride size, pad lengths and group count. The number of channels in the output should be equal to W.shape[1] * group (assuming zero based indices of the shape array)

Type Constraints

  • T tensor(float16), tensor(float), tensor(double): Constrain input and output types to float tensors.

Version

Onnx name: ConvTranspose

This version of the operator has been available since version 11.

Runtime implementation: ConvTranspose

Cos#

mlprodict.onnxrt.ops_cpu.op_cos.Cos (self, onnx_node, desc = None, options)

Calculates the cosine of the given input tensor, element-wise.

Inputs

  • input (heterogeneous)T: Input tensor

Outputs

  • output (heterogeneous)T: The cosine of the input tensor computed element-wise

Type Constraints

  • T tensor(float16), tensor(float), tensor(double): Constrain input and output types to float tensors.

Version

Onnx name: Cos

This version of the operator has been available since version 7.

Runtime implementation: Cos

Cosh#

mlprodict.onnxrt.ops_cpu.op_cosh.Cosh (self, onnx_node, desc = None, options)

Calculates the hyperbolic cosine of the given input tensor element-wise.

Inputs

  • input (heterogeneous)T: Input tensor

Outputs

  • output (heterogeneous)T: The hyperbolic cosine values of the input tensor computed element-wise

Type Constraints

  • T tensor(float16), tensor(float), tensor(double): Constrain input and output types to float tensors.

Version

Onnx name: Cosh

This version of the operator has been available since version 9.

Runtime implementation: Cosh

CumSum#

mlprodict.onnxrt.ops_cpu.op_cum_sum.CumSum (self, onnx_node, desc = None, options)

Performs cumulative sum of the input elements along the given axis. By default, it will do the sum inclusively meaning the first element is copied as is. Through an exclusive attribute, this behavior can change to exclude the first element. It can also perform summation in the opposite direction of the axis. For that, set reverse attribute to 1.

Example: `` input_x = [1, 2, 3] axis=0 output = [1, 3, 6] exclusive=1 output = [0, 1, 3] exclusive=0 reverse=1 output = [6, 5, 3] exclusive=1 reverse=1 output = [5, 3, 0] ``

Attributes

  • exclusive: If set to 1 will return exclusive sum in which the top element is not included. In other terms, if set to 1, the j-th output element would be the sum of the first (j-1) elements. Otherwise, it would be the sum of the first j elements. Default value is nameexclusivei0typeINT (INT)

  • reverse: If set to 1 will perform the sums in reverse direction. Default value is namereversei0typeINT (INT)

Inputs

  • x (heterogeneous)T: An input tensor that is to be processed.

  • axis (heterogeneous)T2: A 0-D tensor. Must be in the range [-rank(x), rank(x)-1]. Negative value means counting dimensions from the back.

Outputs

  • y (heterogeneous)T: Output tensor of the same type as ‘x’ with cumulative sums of the x’s elements

Type Constraints

  • T tensor(uint32), tensor(uint64), tensor(int32), tensor(int64), tensor(float16), tensor(float), tensor(double), tensor(bfloat16): Constrain input and output types to high-precision numeric tensors.

  • T2 tensor(int32), tensor(int64): axis tensor can be int32 or int64 only

Version

Onnx name: CumSum

This version of the operator has been available since version 14.

Runtime implementation: CumSum

DEBUG#

mlprodict.onnxrt.ops_cpu.op_debug.DEBUG (self, onnx_node, desc = None, options)

Version

Onnx name: DEBUG

This version of the operator has been available since version of domain mlprodict.

Runtime implementation: DEBUG

DepthToSpace#

mlprodict.onnxrt.ops_cpu.op_depth_to_space.DepthToSpace (self, onnx_node, desc = None, options)

DepthToSpace rearranges (permutes) data from depth into blocks of spatial data. This is the reverse transformation of SpaceToDepth. More specifically, this op outputs a copy of the input tensor where values from the depth dimension are moved in spatial blocks to the height and width dimensions. By default, mode = DCR. In the DCR mode, elements along the depth dimension from the input tensor are rearranged in the following order: depth, column, and then row. The output y is computed from the input x as below:

b, c, h, w = x.shape

tmp = np.reshape(x, [b, blocksize, blocksize, c // (blocksize**2), h, w])

tmp = np.transpose(tmp, [0, 3, 4, 1, 5, 2])

y = np.reshape(tmp, [b, c // (blocksize**2), h * blocksize, w * blocksize])

In the CRD mode, elements along the depth dimension from the input tensor are rearranged in the following order: column, row, and the depth. The output y is computed from the input x as below:

b, c, h, w = x.shape

tmp = np.reshape(x, [b, c // (blocksize ** 2), blocksize, blocksize, h, w])

tmp = np.transpose(tmp, [0, 1, 4, 2, 5, 3])

y = np.reshape(tmp, [b, c // (blocksize ** 2), h * blocksize, w * blocksize])

Attributes

  • blocksize (required): Blocks of [blocksize, blocksize] are moved. default value cannot be automatically retrieved (INT)

  • mode: DCR (default) for depth-column-row order re-arrangement. Use CRD for column-row-depth order. Default value is namemodesDCRtypeSTRING (STRING)

Inputs

  • input (heterogeneous)T: Input tensor of [N,C,H,W], where N is the batch axis, C is the channel or depth, H is the height and W is the width.

Outputs

  • output (heterogeneous)T: Output tensor of [N, C/(blocksize * blocksize), H * blocksize, W * blocksize].

Type Constraints

  • T tensor(uint8), tensor(uint16), tensor(uint32), tensor(uint64), tensor(int8), tensor(int16), tensor(int32), tensor(int64), tensor(bfloat16), tensor(float16), tensor(float), tensor(double), tensor(string), tensor(bool), tensor(complex64), tensor(complex128): Constrain input and output types to all tensor types.

Version

Onnx name: DepthToSpace

This version of the operator has been available since version 13.

Runtime implementation: DepthToSpace

DequantizeLinear#

mlprodict.onnxrt.ops_cpu.op_dequantize_linear.DequantizeLinear (self, onnx_node, desc = None, options)

The linear dequantization operator. It consumes a quantized tensor, a scale, and a zero point to compute the full precision tensor. The dequantization formula is y = (x - x_zero_point) * x_scale. ‘x_scale’ and ‘x_zero_point’ must have same shape, and can be either a scalar for per-tensor / per layer quantization, or a 1-D tensor for per-axis quantization. ‘x_zero_point’ and ‘x’ must have same type. ‘x’ and ‘y’ must have same shape. In the case of dequantizing int32, there’s no zero point (zero point is supposed to be 0).

Attributes

  • axis: (Optional) The axis of the dequantizing dimension of the input tensor. Ignored for per-tensor quantization. Negative value means counting dimensions from the back. Accepted range is [-r, r-1] where r = rank(input). Default value is nameaxisi1typeINT (INT)

Inputs

Between 2 and 3 inputs.

  • x (heterogeneous)T: N-D quantized input tensor to be de-quantized.

  • x_scale (heterogeneous)tensor(float): Scale for input ‘x’. It can be a scalar, which means a per-tensor/layer dequantization, or a 1-D tensor for per-axis dequantization.

  • x_zero_point (optional, heterogeneous)T: Zero point for input ‘x’. Shape must match x_scale. It’s optional. Zero point is 0 when it’s not specified.

Outputs

  • y (heterogeneous)tensor(float): N-D full precision output tensor. It has same shape as input ‘x’.

Type Constraints

  • T tensor(int8), tensor(uint8), tensor(int32): Constrain ‘x_zero_point’ and ‘x’ to 8-bit/32-bit integer tensor.

Version

Onnx name: DequantizeLinear

This version of the operator has been available since version 13.

Runtime implementation: DequantizeLinear

Det#

mlprodict.onnxrt.ops_cpu.op_det.Det (self, onnx_node, desc = None, options)

Det calculates determinant of a square matrix or batches of square matrices. Det takes one input tensor of shape [*, M, M], where * is zero or more batch dimensions, and the inner-most 2 dimensions form square matrices. The output is a tensor of shape [*], containing the determinants of all input submatrices. e.g., When the input is 2-D, the output is a scalar(shape is empty: []).

Inputs

  • X (heterogeneous)T: Input tensor

Outputs

  • Y (heterogeneous)T: Output tensor

Type Constraints

  • T tensor(float16), tensor(float), tensor(double): Constrain input and output types to floating-point tensors.

Version

Onnx name: Det

This version of the operator has been available since version 11.

Runtime implementation: Det

DictVectorizer#

mlprodict.onnxrt.ops_cpu.op_dict_vectorizer.DictVectorizer (self, onnx_node, desc = None, options)

Uses an index mapping to convert a dictionary to an array.

Given a dictionary, each key is looked up in the vocabulary attribute corresponding to the key type. The index into the vocabulary array at which the key is found is then used to index the output 1-D tensor ‘Y’ and insert into it the value found in the dictionary ‘X’.

The key type of the input map must correspond to the element type of the defined vocabulary attribute. Therefore, the output array will be equal in length to the index mapping vector parameter. All keys in the input dictionary must be present in the index mapping vector. For each item in the input dictionary, insert its value in the output array. Any keys not present in the input dictionary, will be zero in the output array.

For example: if the string_vocabulary parameter is set to ["a", "c", "b", "z"], then an input of {"a": 4, "c": 8} will produce an output of [4, 8, 0, 0].

Attributes

  • int64_vocabulary: An integer vocabulary array. One and only one of the vocabularies must be defined. default value cannot be automatically retrieved (INTS)

  • string_vocabulary: A string vocabulary array. One and only one of the vocabularies must be defined. default value cannot be automatically retrieved (STRINGS)

Inputs

  • X (heterogeneous)T1: A dictionary.

Outputs

  • Y (heterogeneous)T2: A 1-D tensor holding values from the input dictionary.

Type Constraints

  • T1 map(string, int64), map(int64, string), map(int64, float), map(int64, double), map(string, float), map(string, double): The input must be a map from strings or integers to either strings or a numeric type. The key and value types cannot be the same.

  • T2 tensor(int64), tensor(float), tensor(double), tensor(string): The output will be a tensor of the value type of the input map. It’s shape will be [1,C], where C is the length of the input dictionary.

Version

Onnx name: DictVectorizer

This version of the operator has been available since version 1 of domain ai.onnx.ml.

Runtime implementation: DictVectorizer

Div#

mlprodict.onnxrt.ops_cpu.op_div.Div (self, onnx_node, desc = None, options)

Performs element-wise binary division (with Numpy-style broadcasting support).

This operator supports multidirectional (i.e., Numpy-style) broadcasting; for more details please check Broadcasting in ONNX.

(Opset 14 change): Extend supported types to include uint8, int8, uint16, and int16.

Inputs

  • A (heterogeneous)T: First operand.

  • B (heterogeneous)T: Second operand.

Outputs

  • C (heterogeneous)T: Result, has same element type as two inputs

Type Constraints

  • T tensor(uint8), tensor(uint16), tensor(uint32), tensor(uint64), tensor(int8), tensor(int16), tensor(int32), tensor(int64), tensor(float16), tensor(float), tensor(double), tensor(bfloat16): Constrain input and output types to all numeric tensors.

Version

Onnx name: Div

This version of the operator has been available since version 14.

Runtime implementation: Div

Dropout_12#

mlprodict.onnxrt.ops_cpu.op_dropout.Dropout_12 (self, onnx_node, desc = None, options)

Dropout takes an input floating-point tensor, an optional input ratio (floating-point scalar) and an optional input training_mode (boolean scalar). It produces two tensor outputs, output (floating-point tensor) and mask (optional Tensor<bool>). If training_mode is true then the output Y will be a random dropout; Note that this Dropout scales the masked input data by the following equation, so to convert the trained model into inference mode, the user can simply not pass training_mode input or set it to false. `` output = scale * data * mask, `` where `` scale = 1. / (1. - ratio). `` This operator has optional inputs/outputs. See ONNX for more details about the representation of optional arguments. An empty string may be used in the place of an actual argument’s name to indicate a missing argument. Trailing optional arguments (those not followed by an argument that is present) may also be simply omitted.

Attributes

  • seed: (Optional) Seed to the random generator, if not specified we will auto generate one. default value cannot be automatically retrieved (INT)

Inputs

Between 1 and 3 inputs.

  • data (heterogeneous)T: The input data as Tensor.

  • ratio (optional, heterogeneous)T1: The ratio of random dropout, with value in [0, 1). If this input was not set, or if it was set to 0, the output would be a simple copy of the input. If it’s non-zero, output will be a random dropout of the scaled input, which is typically the case during training. It is an optional value, if not specified it will default to 0.5.

  • training_mode (optional, heterogeneous)T2: If set to true then it indicates dropout is being used for training. It is an optional value hence unless specified explicitly, it is false. If it is false, ratio is ignored and the operation mimics inference mode where nothing will be dropped from the input data and if mask is requested as output it will contain all ones.

Outputs

Between 1 and 2 outputs.

  • output (heterogeneous)T: The output.

  • mask (optional, heterogeneous)T2: The output mask.

Type Constraints

  • T tensor(float16), tensor(float), tensor(double): Constrain input and output types to float tensors.

  • T1 tensor(float16), tensor(float), tensor(double): Constrain input ‘ratio’ types to float tensors.

  • T2 tensor(bool): Constrain output ‘mask’ types to boolean tensors.

Version

Onnx name: Dropout

This version of the operator has been available since version 12.

Runtime implementation: Dropout

DynamicQuantizeLinear#

mlprodict.onnxrt.ops_cpu.op_quantize_linear.DynamicQuantizeLinear (self, onnx_node, desc = None, options)

A Function to fuse calculation for Scale, Zero Point and FP32->8Bit convertion of FP32 Input data. Outputs Scale, ZeroPoint and Quantized Input for a given FP32 Input. Scale is calculated as: ``

y_scale = (max(x) - min(x))/(qmax - qmin) * where qmax and qmin are max and min values for quantization range .i.e [0, 255] in case of uint8 * data range is adjusted to include 0.

`` Zero point is calculated as: `` intermediate_zero_point = qmin - min(x)/y_scale y_zero_point = cast(round(saturate(itermediate_zero_point))) * where qmax and qmin are max and min values for quantization range .i.e [0, 255] in case of uint8 * for saturation, it saturates to [0, 255] if it’s uint8, or [-127, 127] if it’s int8. Right now only uint8 is supported. * rounding to nearest ties to even. `` Data quantization formula is: `` y = saturate (round (x / y_scale) + y_zero_point) * for saturation, it saturates to [0, 255] if it’s uint8, or [-127, 127] if it’s int8. Right now only uint8 is supported. * rounding to nearest ties to even. ``

Inputs

  • x (heterogeneous)T1: Input tensor

Outputs

  • y (heterogeneous)T2: Quantized output tensor

  • y_scale (heterogeneous)tensor(float): Output scale. It’s a scalar, which means a per-tensor/layer quantization.

  • y_zero_point (heterogeneous)T2: Output zero point. It’s a scalar, which means a per-tensor/layer quantization.

Type Constraints

  • T1 tensor(float): Constrain ‘x’ to float tensor.

  • T2 tensor(uint8): Constrain ‘y_zero_point’ and ‘y’ to 8-bit unsigned integer tensor.

Version

Onnx name: DynamicQuantizeLinear

This version of the operator has been available since version 11.

Runtime implementation: DynamicQuantizeLinear

Einsum#

mlprodict.onnxrt.ops_cpu.op_einsum.Einsum (self, onnx_node, desc = None, options)

An einsum of the form term1, term2 -> output-term produces an output tensor using the following equation

output[output-term] = reduce-sum( input1[term1] * input2[term] )

where the reduce-sum performs a summation over all the indices occurring in the input terms (term1, term2) that do not occur in the output-term.

The Einsum operator evaluates algebraic tensor operations on a sequence of tensors, using the Einstein summation convention. The equation string contains a comma-separated sequence of lower case letters. Each term corresponds to an operand tensor, and the characters within the terms correspond to operands dimensions.

This sequence may be followed by “->” to separate the left and right hand side of the equation. If the equation contains “->” followed by the right-hand side, the explicit (not classical) form of the Einstein summation is performed, and the right-hand side indices indicate output tensor dimensions. In other cases, output indices are (implicitly) set to the alphabetically sorted sequence of indices appearing exactly once in the equation.

When a dimension character is repeated in the left-hand side, it represents summation along the dimension.

The equation may contain ellipsis (”…”) to enable broadcasting. Ellipsis must indicate a fixed number of dimensions. Specifically, every occurrence of ellipsis in the equation must represent the same number of dimensions. The right-hand side may contain exactly one ellipsis. In implicit mode, the ellipsis dimensions are set to the beginning of the output. The equation string may contain space (U+0020) character.

Attributes

  • equation (required): Einsum expression string. default value cannot be automatically retrieved (STRING)

Inputs

Between 1 and 2147483647 inputs.

  • Inputs (variadic, heterogeneous)T: Operands

Outputs

  • Output (heterogeneous)T: Output tensor

Type Constraints

  • T tensor(uint8), tensor(uint16), tensor(uint32), tensor(uint64), tensor(int8), tensor(int16), tensor(int32), tensor(int64), tensor(float16), tensor(float), tensor(double): Constrain input and output types to all numerical tensor types.

Version

Onnx name: Einsum

This version of the operator has been available since version 12.

Runtime implementation: Einsum

Elu#

mlprodict.onnxrt.ops_cpu.op_elu.Elu (self, onnx_node, desc = None, options)

Elu takes one input data (Tensor<T>) and produces one output data (Tensor<T>) where the function f(x) = alpha * (exp(x) - 1.) for x < 0, f(x) = x for x >= 0., is applied to the tensor elementwise.

Attributes

  • alpha: Coefficient of ELU. Default value is namealphaf1.0typeFLOAT (FLOAT)

Inputs

  • X (heterogeneous)T: 1D input tensor

Outputs

  • Y (heterogeneous)T: 1D output tensor

Type Constraints

  • T tensor(float16), tensor(float), tensor(double): Constrain input and output types to float tensors.

Version

Onnx name: Elu

This version of the operator has been available since version 6.

Runtime implementation: Elu

Equal#

mlprodict.onnxrt.ops_cpu.op_equal.Equal (self, onnx_node, desc = None, options)

Returns the tensor resulted from performing the equal logical operation elementwise on the input tensors A and B (with Numpy-style broadcasting support).

This operator supports multidirectional (i.e., Numpy-style) broadcasting; for more details please check Broadcasting in ONNX.

Inputs

  • A (heterogeneous)T: First input operand for the logical operator.

  • B (heterogeneous)T: Second input operand for the logical operator.

Outputs

  • C (heterogeneous)T1: Result tensor.

Type Constraints

  • T tensor(bool), tensor(uint8), tensor(uint16), tensor(uint32), tensor(uint64), tensor(int8), tensor(int16), tensor(int32), tensor(int64), tensor(float16), tensor(float), tensor(double), tensor(bfloat16): Constrain input types to all numeric tensors.

  • T1 tensor(bool): Constrain output to boolean tensor.

Version

Onnx name: Equal

This version of the operator has been available since version 13.

Runtime implementation: Equal

Erf#

mlprodict.onnxrt.ops_cpu.op_erf.Erf (self, onnx_node, desc = None, options)

Computes the error function of the given input tensor element-wise.

Inputs

  • input (heterogeneous)T: Input tensor

Outputs

  • output (heterogeneous)T: The error function of the input tensor computed element-wise. It has the same shape and type of the input.

Type Constraints

  • T tensor(uint8), tensor(uint16), tensor(uint32), tensor(uint64), tensor(int8), tensor(int16), tensor(int32), tensor(int64), tensor(float16), tensor(float), tensor(double), tensor(bfloat16): Constrain input and output types to all numeric tensors.

Version

Onnx name: Erf

This version of the operator has been available since version 13.

Runtime implementation: Erf

Exp#

mlprodict.onnxrt.ops_cpu.op_exp.Exp (self, onnx_node, desc = None, options)

Calculates the exponential of the given input tensor, element-wise.

Inputs

  • input (heterogeneous)T: Input tensor

Outputs

  • output (heterogeneous)T: The exponential of the input tensor computed element-wise

Type Constraints

  • T tensor(float16), tensor(float), tensor(double), tensor(bfloat16): Constrain input and output types to float tensors.

Version

Onnx name: Exp

This version of the operator has been available since version 13.

Runtime implementation: Exp

Expand_13#

mlprodict.onnxrt.ops_cpu.op_expand.Expand_13 (self, onnx_node, desc = None, options)

Broadcast the input tensor following the given shape and the broadcast rule. The broadcast rule is similar to numpy.array(input) * numpy.ones(shape): Dimensions are right alignment; Two corresponding dimensions must have the same value, or one of them is equal to 1. Also, this operator is similar to numpy.broadcast_to(input, shape), but the major difference is numpy.broadcast_to() does not allow shape to be smaller than input.size(). It is possible that the output.shape is not equal to shape, when some dimensions in shape is equal to 1, or the shape.ndim < input.shape.ndim.

Inputs

  • input (heterogeneous)T: Input tensor

  • shape (heterogeneous)tensor(int64): A 1-D tensor indicates the shape you want to expand to, following the broadcast rule

Outputs

  • output (heterogeneous)T: Output tensor

Type Constraints

  • T tensor(uint8), tensor(uint16), tensor(uint32), tensor(uint64), tensor(int8), tensor(int16), tensor(int32), tensor(int64), tensor(bfloat16), tensor(float16), tensor(float), tensor(double), tensor(string), tensor(bool), tensor(complex64), tensor(complex128): Constrain input and output types to all tensors.

Version

Onnx name: Expand

This version of the operator has been available since version 13.

Runtime implementation: Expand

EyeLike#

mlprodict.onnxrt.ops_cpu.op_eyelike.EyeLike (self, onnx_node, desc = None, options)

Generate a 2D tensor (matrix) with ones on the diagonal and zeros everywhere else. Only 2D tensors are supported, i.e. input T1 must be of rank 2. The shape of the output tensor is the same as the input tensor. The data type can be specified by the ‘dtype’ argument. If ‘dtype’ is not specified, then the type of input tensor is used. By default, the main diagonal is populated with ones, but attribute ‘k’ can be used to populate upper or lower diagonals. The ‘dtype’ argument must be one of the data types specified in the ‘DataType’ enum field in the TensorProto message and be valid as an output type.

Attributes

  • dtype: (Optional) The data type for the elements of the output tensor. If not specified,the data type of the input tensor T1 is used. If input tensor T1 is also notspecified, then type defaults to ‘float’. default value cannot be automatically retrieved (INT)

  • k: (Optional) Index of the diagonal to be populated with ones. Default is 0. If T2 is the output, this op sets T2[i, i+k] = 1. k = 0 populates the main diagonal, k > 0 populates an upper diagonal, and k < 0 populates a lower diagonal. Default value is nameki0typeINT (INT)

Inputs

  • input (heterogeneous)T1: 2D input tensor to copy shape, and optionally, type information from.

Outputs

  • output (heterogeneous)T2: Output tensor, same shape as input tensor T1.

Type Constraints

  • T1 tensor(float16), tensor(float), tensor(double), tensor(int8), tensor(int16), tensor(int32), tensor(int64), tensor(uint8), tensor(uint16), tensor(uint32), tensor(uint64), tensor(bool): Constrain input types. Strings and complex are not supported.

  • T2 tensor(float16), tensor(float), tensor(double), tensor(int8), tensor(int16), tensor(int32), tensor(int64), tensor(uint8), tensor(uint16), tensor(uint32), tensor(uint64), tensor(bool): Constrain output types. Strings and complex are not supported.

Version

Onnx name: EyeLike

This version of the operator has been available since version 9.

Runtime implementation: EyeLike

FFT#

mlprodict.onnxrt.ops_cpu.op_fft.FFT (self, onnx_node, desc = None, options)

Version

Onnx name: FFT

This version of the operator has been available since version of domain mlprodict.

Runtime implementation: FFT

FFT2D#

mlprodict.onnxrt.ops_cpu.op_fft2d.FFT2D (self, onnx_node, desc = None, options)

Version

Onnx name: FFT2D

This version of the operator has been available since version of domain mlprodict.

Runtime implementation: FFT2D

Flatten#

mlprodict.onnxrt.ops_cpu.op_flatten.Flatten (self, onnx_node, desc = None, options)

Flattens the input tensor into a 2D matrix. If input tensor has shape (d_0, d_1, … d_n) then the output will have shape (d_0 X d_1 … d_(axis-1), d_axis X d_(axis+1) … X dn).

Attributes

  • axis: Indicate up to which input dimensions (exclusive) should be flattened to the outer dimension of the output. The value for axis must be in the range [-r, r], where r is the rank of the input tensor. Negative value means counting dimensions from the back. When axis = 0, the shape of the output tensor is (1, (d_0 X d_1 … d_n), where the shape of the input tensor is (d_0, d_1, … d_n). Default value is nameaxisi1typeINT (INT)

Inputs

  • input (heterogeneous)T: A tensor of rank >= axis.

Outputs

  • output (heterogeneous)T: A 2D tensor with the contents of the input tensor, with input dimensions up to axis flattened to the outer dimension of the output and remaining input dimensions flattened into the inner dimension of the output.

Type Constraints

  • T tensor(uint8), tensor(uint16), tensor(uint32), tensor(uint64), tensor(int8), tensor(int16), tensor(int32), tensor(int64), tensor(bfloat16), tensor(float16), tensor(float), tensor(double), tensor(string), tensor(bool), tensor(complex64), tensor(complex128): Constrain input and output to all tensor types.

Version

Onnx name: Flatten

This version of the operator has been available since version 13.

Runtime implementation: Flatten

Floor#

mlprodict.onnxrt.ops_cpu.op_floor.Floor (self, onnx_node, desc = None, options)

Floor takes one input data (Tensor<T>) and produces one output data (Tensor<T>) where the floor is, y = floor(x), is applied to the tensor elementwise.

Inputs

  • X (heterogeneous)T: Input tensor

Outputs

  • Y (heterogeneous)T: Output tensor

Type Constraints

  • T tensor(float16), tensor(float), tensor(double), tensor(bfloat16): Constrain input and output types to float tensors.

Version

Onnx name: Floor

This version of the operator has been available since version 13.

Runtime implementation: Floor

FusedMatMul#

mlprodict.onnxrt.ops_cpu.op_fused_matmul.FusedMatMul (self, onnx_node, desc = None, options)

Version

Onnx name: FusedMatMul

This version of the operator has been available since version of domain mlprodict.

Runtime implementation: FusedMatMul

GRU#

mlprodict.onnxrt.ops_cpu.op_gru.GRU (self, onnx_node, desc = None, options)

Computes an one-layer GRU. This operator is usually supported via some custom implementation such as CuDNN.

Notations:

X - input tensor

z - update gate

r - reset gate

h - hidden gate

t - time step (t-1 means previous time step)

W[zrh] - W parameter weight matrix for update, reset, and hidden gates

R[zrh] - R recurrence weight matrix for update, reset, and hidden gates

Wb[zrh] - W bias vectors for update, reset, and hidden gates

Rb[zrh] - R bias vectors for update, reset, and hidden gates

WB[zrh] - W parameter weight matrix for backward update, reset, and hidden gates

RB[zrh] - R recurrence weight matrix for backward update, reset, and hidden gates

WBb[zrh] - W bias vectors for backward update, reset, and hidden gates

RBb[zrh] - R bias vectors for backward update, reset, and hidden gates

H - Hidden state

num_directions - 2 if direction == bidirectional else 1

Activation functions:

Relu(x) - max(0, x)

Tanh(x) - (1 - e^{-2x})/(1 + e^{-2x})

Sigmoid(x) - 1/(1 + e^{-x})

(NOTE: Below are optional)

Affine(x) - alpha*x + beta

LeakyRelu(x) - x if x >= 0 else alpha * x

ThresholdedRelu(x) - x if x >= alpha else 0

ScaledTanh(x) - alpha*Tanh(beta*x)

HardSigmoid(x) - min(max(alpha*x + beta, 0), 1)

Elu(x) - x if x >= 0 else alpha*(e^x - 1)

Softsign(x) - x/(1 + |x|)

Softplus(x) - log(1 + e^x)

Equations (Default: f=Sigmoid, g=Tanh):

  • zt = f(Xt*(Wz^T) + Ht-1*(Rz^T) + Wbz + Rbz)

  • rt = f(Xt*(Wr^T) + Ht-1*(Rr^T) + Wbr + Rbr)

  • ht = g(Xt*(Wh^T) + (rt (.) Ht-1)*(Rh^T) + Rbh + Wbh) # default, when linear_before_reset = 0

  • ht = g(Xt*(Wh^T) + (rt (.) (Ht-1*(Rh^T) + Rbh)) + Wbh) # when linear_before_reset != 0

  • Ht = (1 - zt) (.) ht + zt (.) Ht-1

This operator has optional inputs/outputs. See ONNX for more details about the representation of optional arguments. An empty string may be used in the place of an actual argument’s name to indicate a missing argument. Trailing optional arguments (those not followed by an argument that is present) may also be simply omitted.

Attributes

  • activation_alpha: Optional scaling values used by some activation functions. The values are consumed in the order of activation functions, for example (f, g, h) in LSTM. Default values are the same as of corresponding ONNX operators.For example with LeakyRelu, the default alpha is 0.01. default value cannot be automatically retrieved (FLOATS)

  • activation_beta: Optional scaling values used by some activation functions. The values are consumed in the order of activation functions, for example (f, g, h) in LSTM. Default values are the same as of corresponding ONNX operators. default value cannot be automatically retrieved (FLOATS)

  • activations: A list of 2 (or 4 if bidirectional) activation functions for update, reset, and hidden gates. The activation functions must be one of the activation functions specified above. Optional: See the equations for default if not specified. default value cannot be automatically retrieved (STRINGS)

  • clip: Cell clip threshold. Clipping bounds the elements of a tensor in the range of [-threshold, +threshold] and is applied to the input of activations. No clip if not specified. default value cannot be automatically retrieved (FLOAT)

  • direction: Specify if the RNN is forward, reverse, or bidirectional. Must be one of forward (default), reverse, or bidirectional. Default value is namedirectionsforwardtypeSTRING (STRING)

  • hidden_size: Number of neurons in the hidden layer default value cannot be automatically retrieved (INT)

  • layout: The shape format of inputs X, initial_h and outputs Y, Y_h. If 0, the following shapes are expected: X.shape = [seq_length, batch_size, input_size], Y.shape = [seq_length, num_directions, batch_size, hidden_size], initial_h.shape = Y_h.shape = [num_directions, batch_size, hidden_size]. If 1, the following shapes are expected: X.shape = [batch_size, seq_length, input_size], Y.shape = [batch_size, seq_length, num_directions, hidden_size], initial_h.shape = Y_h.shape = [batch_size, num_directions, hidden_size]. Default value is namelayouti0typeINT (INT)

  • linear_before_reset: When computing the output of the hidden gate, apply the linear transformation before multiplying by the output of the reset gate. Default value is namelinearbeforereseti0typeINT (INT)

Inputs

Between 3 and 6 inputs.

  • X (heterogeneous)T: The input sequences packed (and potentially padded) into one 3-D tensor with the shape of [seq_length, batch_size, input_size].

  • W (heterogeneous)T: The weight tensor for the gates. Concatenation of W[zrh] and WB[zrh] (if bidirectional) along dimension 0. This tensor has shape [num_directions, 3*hidden_size, input_size].

  • R (heterogeneous)T: The recurrence weight tensor. Concatenation of R[zrh] and RB[zrh] (if bidirectional) along dimension 0. This tensor has shape [num_directions, 3*hidden_size, hidden_size].

  • B (optional, heterogeneous)T: The bias tensor for the gates. Concatenation of [Wb[zrh], Rb[zrh]] and [WBb[zrh], RBb[zrh]] (if bidirectional) along dimension 0. This tensor has shape [num_directions, 6*hidden_size]. Optional: If not specified - assumed to be 0

  • sequence_lens (optional, heterogeneous)T1: Optional tensor specifying lengths of the sequences in a batch. If not specified - assumed all sequences in the batch to have length seq_length. It has shape [batch_size].

  • initial_h (optional, heterogeneous)T: Optional initial value of the hidden. If not specified - assumed to be 0. It has shape [num_directions, batch_size, hidden_size].

Outputs

Between 0 and 2 outputs.

  • Y (optional, heterogeneous)T: A tensor that concats all the intermediate output values of the hidden. It has shape [seq_length, num_directions, batch_size, hidden_size].

  • Y_h (optional, heterogeneous)T: The last output value of the hidden. It has shape [num_directions, batch_size, hidden_size].

Type Constraints

  • T tensor(float16), tensor(float), tensor(double): Constrain input and output types to float tensors.

  • T1 tensor(int32): Constrain seq_lens to integer tensor.

Version

Onnx name: GRU

This version of the operator has been available since version 14.

Runtime implementation: GRU

Gather#

mlprodict.onnxrt.ops_cpu.op_gather.Gather (self, onnx_node, desc = None, options)

Given data tensor of rank r >= 1, and indices tensor of rank q, gather entries of the axis dimension of data (by default outer-most one as axis=0) indexed by indices, and concatenates them in an output tensor of rank q + (r - 1).

axis = 0 :

Let k = indices[i_{0}, …, i_{q-1}] Then output[i_{0}, …, i_{q-1}, j_{0}, …, j_{r-2}] = input[k , j_{0}, …, j_{r-2}]

``
data = [

[1.0, 1.2], [2.3, 3.4], [4.5, 5.7],

] indices = [

[0, 1], [1, 2],

] output = [

[

[1.0, 1.2], [2.3, 3.4],

], [

[2.3, 3.4], [4.5, 5.7],

],

]

`` axis = 1 :

Let k = indices[i_{0}, …, i_{q-1}] Then output[j_{0}, i_{0}, …, i_{q-1}, j_{1}, …, j_{r-2}] = input[j_{0}, k, j_{1}, …, j_{r-2}]

``
data = [

[1.0, 1.2, 1.9], [2.3, 3.4, 3.9], [4.5, 5.7, 5.9],

] indices = [

[0, 2],

] axis = 1, output = [

[[1.0, 1.9]], [[2.3, 3.9]], [[4.5, 5.9]],

]

``

Attributes

  • axis: Which axis to gather on. Negative value means counting dimensions from the back. Accepted range is [-r, r-1] where r = rank(data). Default value is nameaxisi0typeINT (INT)

Inputs

  • data (heterogeneous)T: Tensor of rank r >= 1.

  • indices (heterogeneous)Tind: Tensor of int32/int64 indices, of any rank q. All index values are expected to be within bounds [-s, s-1] along axis of size s. It is an error if any of the index values are out of bounds.

Outputs

  • output (heterogeneous)T: Tensor of rank q + (r - 1).

Type Constraints

  • T tensor(uint8), tensor(uint16), tensor(uint32), tensor(uint64), tensor(int8), tensor(int16), tensor(int32), tensor(int64), tensor(bfloat16), tensor(float16), tensor(float), tensor(double), tensor(string), tensor(bool), tensor(complex64), tensor(complex128): Constrain input and output types to any tensor type.

  • Tind tensor(int32), tensor(int64): Constrain indices to integer types

Version

Onnx name: Gather

This version of the operator has been available since version 13.

Runtime implementation: Gather

GatherElements#

mlprodict.onnxrt.ops_cpu.op_gather_elements.GatherElements (self, onnx_node, desc = None, options)

GatherElements takes two inputs data and indices of the same rank r >= 1 and an optional attribute axis that identifies an axis of data (by default, the outer-most axis, that is axis 0). It is an indexing operation that produces its output by indexing into the input data tensor at index positions determined by elements of the indices tensor. Its output shape is the same as the shape of indices and consists of one value (gathered from the data) for each element in indices.

For instance, in the 3-D case (r = 3), the output produced is determined by the following equations: ``

out[i][j][k] = input[index[i][j][k]][j][k] if axis = 0, out[i][j][k] = input[i][index[i][j][k]][k] if axis = 1, out[i][j][k] = input[i][j][index[i][j][k]] if axis = 2,

``

This operator is also the inverse of ScatterElements. It is similar to Torch’s gather operation.

Example 1: ``

data = [

[1, 2], [3, 4],

] indices = [

[0, 0], [1, 0],

] axis = 1 output = [

[1, 1], [4, 3],

]

`` Example 2: ``

data = [

[1, 2, 3], [4, 5, 6], [7, 8, 9],

] indices = [

[1, 2, 0], [2, 0, 0],

] axis = 0 output = [

[4, 8, 3], [7, 2, 3],

]

``

Attributes

  • axis: Which axis to gather on. Negative value means counting dimensions from the back. Accepted range is [-r, r-1] where r = rank(data). Default value is nameaxisi0typeINT (INT)

Inputs

  • data (heterogeneous)T: Tensor of rank r >= 1.

  • indices (heterogeneous)Tind: Tensor of int32/int64 indices, with the same rank r as the input. All index values are expected to be within bounds [-s, s-1] along axis of size s. It is an error if any of the index values are out of bounds.

Outputs

  • output (heterogeneous)T: Tensor of the same shape as indices.

Type Constraints

  • T tensor(uint8), tensor(uint16), tensor(uint32), tensor(uint64), tensor(int8), tensor(int16), tensor(int32), tensor(int64), tensor(bfloat16), tensor(float16), tensor(float), tensor(double), tensor(string), tensor(bool), tensor(complex64), tensor(complex128): Constrain input and output types to any tensor type.

  • Tind tensor(int32), tensor(int64): Constrain indices to integer types

Version

Onnx name: GatherElements

This version of the operator has been available since version 13.

Runtime implementation: GatherElements

Gemm#

mlprodict.onnxrt.ops_cpu.op_gemm.Gemm (self, onnx_node, desc = None, options)

General Matrix multiplication: https://en.wikipedia.org/wiki/Basic_Linear_Algebra_Subprograms#Level_3

A’ = transpose(A) if transA else A

B’ = transpose(B) if transB else B

Compute Y = alpha * A’ * B’ + beta * C, where input tensor A has shape (M, K) or (K, M), input tensor B has shape (K, N) or (N, K), input tensor C is broadcastable to shape (M, N), and output tensor Y has shape (M, N). A will be transposed before doing the computation if attribute transA is non-zero, same for B and transB. This operator supports unidirectional broadcasting (tensor C should be unidirectional broadcastable to tensor A * B); for more details please check Broadcasting in ONNX. This operator has optional inputs/outputs. See ONNX for more details about the representation of optional arguments. An empty string may be used in the place of an actual argument’s name to indicate a missing argument. Trailing optional arguments (those not followed by an argument that is present) may also be simply omitted.

Attributes

  • alpha: Scalar multiplier for the product of input tensors A * B. Default value is namealphaf1.0typeFLOAT (FLOAT)

  • beta: Scalar multiplier for input tensor C. Default value is namebetaf1.0typeFLOAT (FLOAT)

  • transA: Whether A should be transposed Default value is nametransAi0typeINT (INT)

  • transB: Whether B should be transposed Default value is nametransBi0typeINT (INT)

Inputs

Between 2 and 3 inputs.

  • A (heterogeneous)T: Input tensor A. The shape of A should be (M, K) if transA is 0, or (K, M) if transA is non-zero.

  • B (heterogeneous)T: Input tensor B. The shape of B should be (K, N) if transB is 0, or (N, K) if transB is non-zero.

  • C (optional, heterogeneous)T: Optional input tensor C. If not specified, the computation is done as if C is a scalar 0. The shape of C should be unidirectional broadcastable to (M, N).

Outputs

  • Y (heterogeneous)T: Output tensor of shape (M, N).

Type Constraints

  • T tensor(float16), tensor(float), tensor(double), tensor(uint32), tensor(uint64), tensor(int32), tensor(int64), tensor(bfloat16): Constrain input and output types to float/int tensors.

Version

Onnx name: Gemm

This version of the operator has been available since version 13.

Runtime implementation: Gemm

GlobalAveragePool#

mlprodict.onnxrt.ops_cpu.op_global_average_pool.GlobalAveragePool (self, onnx_node, desc = None, options)

GlobalAveragePool consumes an input tensor X and applies average pooling across the values in the same channel. This is equivalent to AveragePool with kernel size equal to the spatial dimension of input tensor.

Inputs

  • X (heterogeneous)T: Input data tensor from the previous operator; dimensions for image case are (N x C x H x W), where N is the batch size, C is the number of channels, and H and W are the height and the width of the data. For non image case, the dimensions are in the form of (N x C x D1 x D2 … Dn), where N is the batch size.

Outputs

  • Y (heterogeneous)T: Output data tensor from pooling across the input tensor. The output tensor has the same rank as the input. The first two dimensions of output shape are the same as the input (N x C), while the other dimensions are all 1.

Type Constraints

  • T tensor(float16), tensor(float), tensor(double): Constrain input and output types to float tensors.

Version

Onnx name: GlobalAveragePool

This version of the operator has been available since version 1.

Runtime implementation: GlobalAveragePool

GlobalMaxPool#

mlprodict.onnxrt.ops_cpu.op_global_average_pool.GlobalMaxPool (self, onnx_node, desc = None, options)

GlobalMaxPool consumes an input tensor X and applies max pooling across the values in the same channel. This is equivalent to MaxPool with kernel size equal to the spatial dimension of input tensor.

Inputs

  • X (heterogeneous)T: Input data tensor from the previous operator; dimensions for image case are (N x C x H x W), where N is the batch size, C is the number of channels, and H and W are the height and the width of the data. For non image case, the dimensions are in the form of (N x C x D1 x D2 … Dn), where N is the batch size.

Outputs

  • Y (heterogeneous)T: Output data tensor from pooling across the input tensor. The output tensor has the same rank as the input. The first two dimensions of output shape are the same as the input (N x C), while the other dimensions are all 1.

Type Constraints

  • T tensor(float16), tensor(float), tensor(double): Constrain input and output types to float tensors.

Version

Onnx name: GlobalMaxPool

This version of the operator has been available since version 1.

Runtime implementation: GlobalMaxPool

Greater#

mlprodict.onnxrt.ops_cpu.op_greater.Greater (self, onnx_node, desc = None, options)

Returns the tensor resulted from performing the greater logical operation elementwise on the input tensors A and B (with Numpy-style broadcasting support).

This operator supports multidirectional (i.e., Numpy-style) broadcasting; for more details please check Broadcasting in ONNX.

Inputs

  • A (heterogeneous)T: First input operand for the logical operator.

  • B (heterogeneous)T: Second input operand for the logical operator.

Outputs

  • C (heterogeneous)T1: Result tensor.

Type Constraints

  • T tensor(uint8), tensor(uint16), tensor(uint32), tensor(uint64), tensor(int8), tensor(int16), tensor(int32), tensor(int64), tensor(float16), tensor(float), tensor(double), tensor(bfloat16): Constrain input types to all numeric tensors.

  • T1 tensor(bool): Constrain output to boolean tensor.

Version

Onnx name: Greater

This version of the operator has been available since version 13.

Runtime implementation: Greater

GreaterOrEqual#

mlprodict.onnxrt.ops_cpu.op_greater.GreaterOrEqual (self, onnx_node, desc = None, options)

Returns the tensor resulted from performing the greater_equal logical operation elementwise on the input tensors A and B (with Numpy-style broadcasting support).

This operator supports multidirectional (i.e., Numpy-style) broadcasting; for more details please check Broadcasting in ONNX.

Inputs

  • A (heterogeneous)T: First input operand for the logical operator.

  • B (heterogeneous)T: Second input operand for the logical operator.

Outputs

  • C (heterogeneous)T1: Result tensor.

Type Constraints

  • T tensor(uint8), tensor(uint16), tensor(uint32), tensor(uint64), tensor(int8), tensor(int16), tensor(int32), tensor(int64), tensor(float16), tensor(float), tensor(double), tensor(bfloat16): Constrain input types to all numeric tensors.

  • T1 tensor(bool): Constrain output to boolean tensor.

Version

Onnx name: GreaterOrEqual

This version of the operator has been available since version 16.

Runtime implementation: GreaterOrEqual

GridSample#

mlprodict.onnxrt.ops_cpu.op_grid_sample.GridSample (self, onnx_node, desc = None, options)

Given an input and a flow-field grid, computes the output using input values and pixel locations from grid. Currently, only spatial (4-D) inputs are supported. For input with shape (N, C, H, W) and grid with shape (N, H_out, W_out, 2), the output will have shape (N, C, H_out, W_out). For each output location output[N, C, H_out, W_out], the size-2 vector grid[N, H_out, W_out] specifies input pixel locations x and y, which are used to interpolate the output value output[N, C, H_out, W_out].

The GridSample operator is often used in doing grid generator and sampler in the [Spatial Transformer Networks](https://arxiv.org/abs/1506.02025). See also in [torch.nn.functional.grid_sample](https://pytorch.org/docs/master/generated/torch.nn.functional.grid_sample.html#torch-nn-functional-grid-sample).

Attributes

  • align_corners: If align_corners=1, the extrema (-1 and 1) are considered as referring to the center points of the input’s corner pixels. If align_corners=0, they are instead considered as referring to the corner points of the input’s corner pixels, making the sampling more resolution agnostic. Default value is namealigncornersi0typeINT (INT)

  • mode: Three interpolation modes: bilinear (default), nearest and bicubic. Default value is namemodesbilineartypeSTRING (STRING)

  • padding_mode: Support padding modes for outside grid values: zeros`(default), `border, reflection. zeros: use 0 for out-of-bound grid locations, border: use border values for out-of-bound grid locations, reflection: use values at locations reflected by the border for out-of-bound grid locations. If index 0 represents the margin pixel, the reflected value at index -1 will be the same as the value at index 1. For location far away from the border, it will keep being reflected until becoming in bound. If pixel location x = -3.5 reflects by border -1 and becomes x’ = 1.5, then reflects by border 1 and becomes x’’ = 0.5. Default value is namepaddingmodeszerostypeSTRING (STRING)

Inputs

  • X (heterogeneous)T1: 4-D tensor of shape (N, C, H, W), where N is the batch size, C is the numbers of channels, H and W are the height and width of the input data.

  • grid (heterogeneous)T1: Input offset, 4-D tensor of shape (N, H_out, W_out, 2), where H_out and W_out are the height and width of grid and output, Grid specifies the sampling pixel locations normalized by the input spatial dimensions. Therefore, it should have most values in the range of [-1, 1]. If grid has values outside the range of [-1, 1], the corresponding outputs will be handled as defined by padding_mode.

Outputs

  • Y (heterogeneous)T2: 4-D tensor of shape (N, C, H_out, W_out).

Type Constraints

  • T1 tensor(uint8), tensor(uint16), tensor(uint32), tensor(uint64), tensor(int8), tensor(int16), tensor(int32), tensor(int64), tensor(float16), tensor(float), tensor(double), tensor(string), tensor(bool), tensor(complex64), tensor(complex128): Constrain input types to all tensor types.

  • T2 tensor(float16), tensor(float), tensor(double): Constrain output types to float tensors.

Version

Onnx name: GridSample

This version of the operator has been available since version 16.

Runtime implementation: GridSample

HardSigmoid#

mlprodict.onnxrt.ops_cpu.op_hard_sigmoid.HardSigmoid (self, onnx_node, desc = None, options)

HardSigmoid takes one input data (Tensor<T>) and produces one output data (Tensor<T>) where the HardSigmoid function, y = max(0, min(1, alpha * x + beta)), is applied to the tensor elementwise.

Attributes

  • alpha: Value of alpha. Default value is namealphaf0.20000000298023224typeFLOAT (FLOAT)

  • beta: Value of beta. Default value is namebetaf0.5typeFLOAT (FLOAT)

Inputs

  • X (heterogeneous)T: Input tensor

Outputs

  • Y (heterogeneous)T: Output tensor

Type Constraints

  • T tensor(float16), tensor(float), tensor(double): Constrain input and output types to float tensors.

Version

Onnx name: HardSigmoid

This version of the operator has been available since version 6.

Runtime implementation: HardSigmoid

Hardmax#

mlprodict.onnxrt.ops_cpu.op_hardmax.Hardmax (self, onnx_node, desc = None, options)

The operator computes the hardmax values for the given input:

Hardmax(element in input, axis) = 1 if the element is the first maximum value along the specified axis, 0 otherwise

The “axis” attribute indicates the dimension along which Hardmax will be performed. The output tensor has the same shape and contains the Hardmax values of the corresponding input.

Attributes

  • axis:

Describes the dimension Hardmax will be performed on. Negative value means counting dimensions from the back. Accepted range is [-r, r-1] where r = rank(input).

Default value is nameaxisi-1typeINT (INT)

Inputs

  • input (heterogeneous)T: The input tensor of rank >= axis.

Outputs

  • output (heterogeneous)T: The output values with the same shape as the input tensor.

Type Constraints

  • T tensor(float16), tensor(float), tensor(double), tensor(bfloat16): Constrain input and output types to float tensors.

Version

Onnx name: Hardmax

This version of the operator has been available since version 13.

Runtime implementation: Hardmax

Identity#

mlprodict.onnxrt.ops_cpu.op_identity.Identity (self, onnx_node, desc = None, options)

Identity operator

Inputs

  • input (heterogeneous)V: Input tensor

Outputs

  • output (heterogeneous)V: Tensor to copy input into.

Type Constraints

  • V tensor(uint8), tensor(uint16), tensor(uint32), tensor(uint64), tensor(int8), tensor(int16), tensor(int32), tensor(int64), tensor(bfloat16), tensor(float16), tensor(float), tensor(double), tensor(string), tensor(bool), tensor(complex64), tensor(complex128), seq(tensor(uint8)), seq(tensor(uint16)), seq(tensor(uint32)), seq(tensor(uint64)), seq(tensor(int8)), seq(tensor(int16)), seq(tensor(int32)), seq(tensor(int64)), seq(tensor(float16)), seq(tensor(float)), seq(tensor(double)), seq(tensor(string)), seq(tensor(bool)), seq(tensor(complex64)), seq(tensor(complex128)), optional(seq(tensor(uint8))), optional(seq(tensor(uint16))), optional(seq(tensor(uint32))), optional(seq(tensor(uint64))), optional(seq(tensor(int8))), optional(seq(tensor(int16))), optional(seq(tensor(int32))), optional(seq(tensor(int64))), optional(seq(tensor(float16))), optional(seq(tensor(float))), optional(seq(tensor(double))), optional(seq(tensor(string))), optional(seq(tensor(bool))), optional(seq(tensor(complex64))), optional(seq(tensor(complex128))), optional(tensor(uint8)), optional(tensor(uint16)), optional(tensor(uint32)), optional(tensor(uint64)), optional(tensor(int8)), optional(tensor(int16)), optional(tensor(int32)), optional(tensor(int64)), optional(tensor(float16)), optional(tensor(float)), optional(tensor(double)), optional(tensor(string)), optional(tensor(bool)), optional(tensor(complex64)), optional(tensor(complex128)): Constrain input and output types to all tensor, sequence, and optional types.

Version

Onnx name: Identity

This version of the operator has been available since version 16.

Runtime implementation: Identity

If#

mlprodict.onnxrt.ops_cpu.op_if.If (self, onnx_node, desc = None, options)

If conditional

Attributes

  • else_branch (required): Graph to run if condition is false. Has N outputs: values you wish to be live-out to the enclosing scope. The number of outputs must match the number of outputs in the then_branch. default value cannot be automatically retrieved (GRAPH)

  • then_branch (required): Graph to run if condition is true. Has N outputs: values you wish to be live-out to the enclosing scope. The number of outputs must match the number of outputs in the else_branch. default value cannot be automatically retrieved (GRAPH)

Inputs

  • cond (heterogeneous)B: Condition for the if

Outputs

Between 1 and 2147483647 outputs.

  • outputs (variadic)V: Values that are live-out to the enclosing scope. The return values in the then_branch and else_branch must be of the same data type. The then_branch and else_branch may produce tensors with the same element type and different shapes. If corresponding outputs from the then-branch and the else-branch have static shapes S1 and S2, then the shape of the corresponding output variable of the if-node (if present) must be compatible with both S1 and S2 as it represents the union of both possible shapes.For example, if in a model file, the the first output of then_branch is typed float tensor with shape [2] and the first output of else_branch is another float tensor with shape [3], If’s first output should have (a) no shape set, or (b) a shape of rank 1 with neither dim_value nor dim_param set, or (c) a shape of rank 1 with a unique dim_param. In contrast, the first output cannot have the shape [2] since [2] and [3] are not compatible.

Type Constraints

  • V tensor(uint8), tensor(uint16), tensor(uint32), tensor(uint64), tensor(int8), tensor(int16), tensor(int32), tensor(int64), tensor(bfloat16), tensor(float16), tensor(float), tensor(double), tensor(string), tensor(bool), tensor(complex64), tensor(complex128), seq(tensor(uint8)), seq(tensor(uint16)), seq(tensor(uint32)), seq(tensor(uint64)), seq(tensor(int8)), seq(tensor(int16)), seq(tensor(int32)), seq(tensor(int64)), seq(tensor(bfloat16)), seq(tensor(float16)), seq(tensor(float)), seq(tensor(double)), seq(tensor(string)), seq(tensor(bool)), seq(tensor(complex64)), seq(tensor(complex128)), optional(seq(tensor(uint8))), optional(seq(tensor(uint16))), optional(seq(tensor(uint32))), optional(seq(tensor(uint64))), optional(seq(tensor(int8))), optional(seq(tensor(int16))), optional(seq(tensor(int32))), optional(seq(tensor(int64))), optional(seq(tensor(bfloat16))), optional(seq(tensor(float16))), optional(seq(tensor(float))), optional(seq(tensor(double))), optional(seq(tensor(string))), optional(seq(tensor(bool))), optional(seq(tensor(complex64))), optional(seq(tensor(complex128))), optional(tensor(uint8)), optional(tensor(uint16)), optional(tensor(uint32)), optional(tensor(uint64)), optional(tensor(int8)), optional(tensor(int16)), optional(tensor(int32)), optional(tensor(int64)), optional(tensor(bfloat16)), optional(tensor(float16)), optional(tensor(float)), optional(tensor(double)), optional(tensor(string)), optional(tensor(bool)), optional(tensor(complex64)), optional(tensor(complex128)): All Tensor, Sequence(Tensor), Optional(Tensor), and Optional(Sequence(Tensor)) types

  • B tensor(bool): Only bool

Version

Onnx name: If

This version of the operator has been available since version 16.

Runtime implementation: If

Imputer#

mlprodict.onnxrt.ops_cpu.op_imputer.Imputer (self, onnx_node, desc = None, options)

Replaces inputs that equal one value with another, leaving all other elements alone.

This operator is typically used to replace missing values in situations where they have a canonical representation, such as -1, 0, NaN, or some extreme value.

One and only one of imputed_value_floats or imputed_value_int64s should be defined – floats if the input tensor holds floats, integers if the input tensor holds integers. The imputed values must all fit within the width of the tensor element type. One and only one of the replaced_value_float or replaced_value_int64 should be defined, which one depends on whether floats or integers are being processed.

The imputed_value attribute length can be 1 element, or it can have one element per input feature. In other words, if the input tensor has the shape [*,F], then the length of the attribute array may be 1 or F. If it is 1, then it is broadcast along the last dimension and applied to each feature.

Attributes

  • imputed_value_floats: Value(s) to change to default value cannot be automatically retrieved (FLOATS)

  • imputed_value_int64s: Value(s) to change to. default value cannot be automatically retrieved (INTS)

  • replaced_value_float: A value that needs replacing. Default value is namereplacedvaluefloatf0.0typeFLOAT (FLOAT)

  • replaced_value_int64: A value that needs replacing. Default value is namereplacedvalueint64i0typeINT (INT)

Inputs

  • X (heterogeneous)T: Data to be processed.

Outputs

  • Y (heterogeneous)T: Imputed output data

Type Constraints

  • T tensor(float), tensor(double), tensor(int64), tensor(int32): The input type must be a tensor of a numeric type, either [N,C] or [C]. The output type will be of the same tensor type and shape.

Version

Onnx name: Imputer

This version of the operator has been available since version 1 of domain ai.onnx.ml.

Runtime implementation: Imputer

Inverse#

mlprodict.onnxrt.ops_cpu.op_inverse.Inverse (self, onnx_node, desc = None, options)

Version

Onnx name: Inverse

This version of the operator has been available since version of domain mlprodict.

Runtime implementation: Inverse

IsInf#

mlprodict.onnxrt.ops_cpu.op_isinf.IsInf (self, onnx_node, desc = None, options)

Map infinity to true and other values to false.

Attributes

  • detect_negative: (Optional) Whether map negative infinity to true. Default to 1 so that negative infinity induces true. Set this attribute to 0 if negative infinity should be mapped to false. Default value is namedetectnegativei1typeINT (INT)

  • detect_positive: (Optional) Whether map positive infinity to true. Default to 1 so that positive infinity induces true. Set this attribute to 0 if positive infinity should be mapped to false. Default value is namedetectpositivei1typeINT (INT)

Inputs

  • X (heterogeneous)T1: input

Outputs

  • Y (heterogeneous)T2: output

Type Constraints

  • T1 tensor(float), tensor(double): Constrain input types to float tensors.

  • T2 tensor(bool): Constrain output types to boolean tensors.

Version

Onnx name: IsInf

This version of the operator has been available since version 10.

Runtime implementation: IsInf

IsNaN#

mlprodict.onnxrt.ops_cpu.op_isnan.IsNaN (self, onnx_node, desc = None, options)

Returns which elements of the input are NaN.

Inputs

  • X (heterogeneous)T1: input

Outputs

  • Y (heterogeneous)T2: output

Type Constraints

  • T1 tensor(float16), tensor(float), tensor(double), tensor(bfloat16): Constrain input types to float tensors.

  • T2 tensor(bool): Constrain output types to boolean tensors.

Version

Onnx name: IsNaN

This version of the operator has been available since version 13.

Runtime implementation: IsNaN

LRN#

mlprodict.onnxrt.ops_cpu.op_lrn.LRN (self, onnx_node, desc = None, options)

Local Response Normalization proposed in the [AlexNet paper](https://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks.pdf). It normalizes over local input regions. The local region is defined across the channels. For an element X[n, c, d1, …, dk] in a tensor of shape (N x C x D1 x D2, …, Dk), its region is {X[n, i, d1, …, dk] | max(0, c - floor((size - 1) / 2)) <= i <= min(C - 1, c + ceil((size - 1) / 2))}.

square_sum[n, c, d1, …, dk] = sum(X[n, i, d1, …, dk] ^ 2), where max(0, c - floor((size - 1) / 2)) <= i <= min(C - 1, c + ceil((size - 1) / 2)).

Y[n, c, d1, …, dk] = X[n, c, d1, …, dk] / (bias + alpha / size * square_sum[n, c, d1, …, dk] ) ^ beta

Attributes

  • alpha: Scaling parameter. Default value is namealphaf9.999999747378752e-05typeFLOAT (FLOAT)

  • beta: The exponent. Default value is namebetaf0.75typeFLOAT (FLOAT)

  • bias: Default value is namebiasf1.0typeFLOAT (FLOAT)

  • size (required): The number of channels to sum over default value cannot be automatically retrieved (INT)

Inputs

  • X (heterogeneous)T: Input data tensor from the previous operator; dimensions for image case are (N x C x H x W), where N is the batch size, C is the number of channels, and H and W are the height and the width of the data. For non image case, the dimensions are in the form of (N x C x D1 x D2 … Dn), where N is the batch size. Optionally, if dimension denotation is in effect, the operation expects the input data tensor to arrive with the dimension denotation of [DATA_BATCH, DATA_CHANNEL, DATA_FEATURE, DATA_FEATURE …].

Outputs

  • Y (heterogeneous)T: Output tensor, which has the shape and type as input tensor

Type Constraints

  • T tensor(float16), tensor(float), tensor(double), tensor(bfloat16): Constrain input and output types to float tensors.

Version

Onnx name: LRN

This version of the operator has been available since version 13.

Runtime implementation: LRN

LSTM#

mlprodict.onnxrt.ops_cpu.op_lstm.LSTM (self, onnx_node, desc = None, options)

Computes an one-layer LSTM. This operator is usually supported via some custom implementation such as CuDNN.

Notations:

X - input tensor

i - input gate

o - output gate

f - forget gate

c - cell gate

t - time step (t-1 means previous time step)

W[iofc] - W parameter weight matrix for input, output, forget, and cell gates

R[iofc] - R recurrence weight matrix for input, output, forget, and cell gates

Wb[iofc] - W bias vectors for input, output, forget, and cell gates

Rb[iofc] - R bias vectors for input, output, forget, and cell gates

P[iof] - P peephole weight vector for input, output, and forget gates

WB[iofc] - W parameter weight matrix for backward input, output, forget, and cell gates

RB[iofc] - R recurrence weight matrix for backward input, output, forget, and cell gates

WBb[iofc] - W bias vectors for backward input, output, forget, and cell gates

RBb[iofc] - R bias vectors for backward input, output, forget, and cell gates

PB[iof] - P peephole weight vector for backward input, output, and forget gates

H - Hidden state

num_directions - 2 if direction == bidirectional else 1

Activation functions:

Relu(x) - max(0, x)

Tanh(x) - (1 - e^{-2x})/(1 + e^{-2x})

Sigmoid(x) - 1/(1 + e^{-x})

(NOTE: Below are optional)

Affine(x) - alpha*x + beta

LeakyRelu(x) - x if x >= 0 else alpha * x

ThresholdedRelu(x) - x if x >= alpha else 0

ScaledTanh(x) - alpha*Tanh(beta*x)

HardSigmoid(x) - min(max(alpha*x + beta, 0), 1)

Elu(x) - x if x >= 0 else alpha*(e^x - 1)

Softsign(x) - x/(1 + |x|)

Softplus(x) - log(1 + e^x)

Equations (Default: f=Sigmoid, g=Tanh, h=Tanh):

  • it = f(Xt*(Wi^T) + Ht-1*(Ri^T) + Pi (.) Ct-1 + Wbi + Rbi)

  • ft = f(Xt*(Wf^T) + Ht-1*(Rf^T) + Pf (.) Ct-1 + Wbf + Rbf)

  • ct = g(Xt*(Wc^T) + Ht-1*(Rc^T) + Wbc + Rbc)

  • Ct = ft (.) Ct-1 + it (.) ct

  • ot = f(Xt*(Wo^T) + Ht-1*(Ro^T) + Po (.) Ct + Wbo + Rbo)

  • Ht = ot (.) h(Ct)

This operator has optional inputs/outputs. See ONNX for more details about the representation of optional arguments. An empty string may be used in the place of an actual argument’s name to indicate a missing argument. Trailing optional arguments (those not followed by an argument that is present) may also be simply omitted.

Attributes

  • activation_alpha: Optional scaling values used by some activation functions. The values are consumed in the order of activation functions, for example (f, g, h) in LSTM. Default values are the same as of corresponding ONNX operators.For example with LeakyRelu, the default alpha is 0.01. default value cannot be automatically retrieved (FLOATS)

  • activation_beta: Optional scaling values used by some activation functions. The values are consumed in the order of activation functions, for example (f, g, h) in LSTM. Default values are the same as of corresponding ONNX operators. default value cannot be automatically retrieved (FLOATS)

  • activations: A list of 3 (or 6 if bidirectional) activation functions for input, output, forget, cell, and hidden. The activation functions must be one of the activation functions specified above. Optional: See the equations for default if not specified. default value cannot be automatically retrieved (STRINGS)

  • clip: Cell clip threshold. Clipping bounds the elements of a tensor in the range of [-threshold, +threshold] and is applied to the input of activations. No clip if not specified. default value cannot be automatically retrieved (FLOAT)

  • direction: Specify if the RNN is forward, reverse, or bidirectional. Must be one of forward (default), reverse, or bidirectional. Default value is namedirectionsforwardtypeSTRING (STRING)

  • hidden_size: Number of neurons in the hidden layer default value cannot be automatically retrieved (INT)

  • input_forget: Couple the input and forget gates if 1. Default value is nameinputforgeti0typeINT (INT)

  • layout: The shape format of inputs X, initial_h, initial_c and outputs Y, Y_h, Y_c. If 0, the following shapes are expected: X.shape = [seq_length, batch_size, input_size], Y.shape = [seq_length, num_directions, batch_size, hidden_size], initial_h.shape = Y_h.shape = initial_c.shape = Y_c.shape = [num_directions, batch_size, hidden_size]. If 1, the following shapes are expected: X.shape = [batch_size, seq_length, input_size], Y.shape = [batch_size, seq_length, num_directions, hidden_size], initial_h.shape = Y_h.shape = initial_c.shape = Y_c.shape = [batch_size, num_directions, hidden_size]. Default value is namelayouti0typeINT (INT)

Inputs

Between 3 and 8 inputs.

  • X (heterogeneous)T: The input sequences packed (and potentially padded) into one 3-D tensor with the shape of [seq_length, batch_size, input_size].

  • W (heterogeneous)T: The weight tensor for the gates. Concatenation of W[iofc] and WB[iofc] (if bidirectional) along dimension 0. The tensor has shape [num_directions, 4*hidden_size, input_size].

  • R (heterogeneous)T: The recurrence weight tensor. Concatenation of R[iofc] and RB[iofc] (if bidirectional) along dimension 0. This tensor has shape [num_directions, 4*hidden_size, hidden_size].

  • B (optional, heterogeneous)T: The bias tensor for input gate. Concatenation of [Wb[iofc], Rb[iofc]], and [WBb[iofc], RBb[iofc]] (if bidirectional) along dimension 0. This tensor has shape [num_directions, 8*hidden_size]. Optional: If not specified - assumed to be 0.

  • sequence_lens (optional, heterogeneous)T1: Optional tensor specifying lengths of the sequences in a batch. If not specified - assumed all sequences in the batch to have length seq_length. It has shape [batch_size].

  • initial_h (optional, heterogeneous)T: Optional initial value of the hidden. If not specified - assumed to be 0. It has shape [num_directions, batch_size, hidden_size].

  • initial_c (optional, heterogeneous)T: Optional initial value of the cell. If not specified - assumed to be 0. It has shape [num_directions, batch_size, hidden_size].

  • P (optional, heterogeneous)T: The weight tensor for peepholes. Concatenation of P[iof] and PB[iof] (if bidirectional) along dimension 0. It has shape [num_directions, 3*hidde_size]. Optional: If not specified - assumed to be 0.

Outputs

Between 0 and 3 outputs.

  • Y (optional, heterogeneous)T: A tensor that concats all the intermediate output values of the hidden. It has shape [seq_length, num_directions, batch_size, hidden_size].

  • Y_h (optional, heterogeneous)T: The last output value of the hidden. It has shape [num_directions, batch_size, hidden_size].

  • Y_c (optional, heterogeneous)T: The last output value of the cell. It has shape [num_directions, batch_size, hidden_size].

Type Constraints

  • T tensor(float16), tensor(float), tensor(double): Constrain input and output types to float tensors.

  • T1 tensor(int32): Constrain seq_lens to integer tensor.

Version

Onnx name: LSTM

This version of the operator has been available since version 14.

Runtime implementation: LSTM

LabelEncoder#

mlprodict.onnxrt.ops_cpu.op_label_encoder.LabelEncoder (self, onnx_node, desc = None, options)

Maps each element in the input tensor to another value.

The mapping is determined by the two parallel attributes, ‘keys_*’ and ‘values_*’ attribute. The i-th value in the specified ‘keys_*’ attribute would be mapped to the i-th value in the specified ‘values_*’ attribute. It implies that input’s element type and the element type of the specified ‘keys_*’ should be identical while the output type is identical to the specified ‘values_*’ attribute. If an input element can not be found in the specified ‘keys_*’ attribute, the ‘default_*’ that matches the specified ‘values_*’ attribute may be used as its output value.

Let’s consider an example which maps a string tensor to an integer tensor. Assume and ‘keys_strings’ is [“Amy”, “Sally”], ‘values_int64s’ is [5, 6], and ‘default_int64’ is ‘-1’. The input [“Dori”, “Amy”, “Amy”, “Sally”, “Sally”] would be mapped to [-1, 5, 5, 6, 6].

Since this operator is an one-to-one mapping, its input and output shapes are the same. Notice that only one of ‘keys_*’/’values_*’ can be set.

For key look-up, bit-wise comparison is used so even a float NaN can be mapped to a value in ‘values_*’ attribute.

Attributes

  • default_float: A float. Default value is namedefaultfloatf-0.0typeFLOAT (FLOAT)

  • default_int64: An integer. Default value is namedefaultint64i-1typeINT (INT)

  • default_string: A string. Default value is namedefaultstringsUnusedtypeSTRING (STRING)

  • keys_floats: A list of floats. default value cannot be automatically retrieved (FLOATS)

  • keys_int64s: A list of ints. default value cannot be automatically retrieved (INTS)

  • keys_strings: A list of strings. One and only one of ‘keys_*’s should be set. default value cannot be automatically retrieved (STRINGS)

  • values_floats: A list of floats. default value cannot be automatically retrieved (FLOATS)

  • values_int64s: A list of ints. default value cannot be automatically retrieved (INTS)

  • values_strings: A list of strings. One and only one of ‘value_*’s should be set. default value cannot be automatically retrieved (STRINGS)

Inputs

  • X (heterogeneous)T1: Input data. It can be either tensor or scalar.

Outputs

  • Y (heterogeneous)T2: Output data.

Type Constraints

  • T1 tensor(string), tensor(int64), tensor(float): The input type is a tensor of any shape.

  • T2 tensor(string), tensor(int64), tensor(float): Output type is determined by the specified ‘values_*’ attribute.

Version

Onnx name: LabelEncoder

This version of the operator has been available since version 2 of domain ai.onnx.ml.

Runtime implementation: LabelEncoder

LeakyRelu#

mlprodict.onnxrt.ops_cpu.op_leaky_relu.LeakyRelu (self, onnx_node, desc = None, options)

LeakyRelu takes input data (Tensor<T>) and an argument alpha, and produces one output data (Tensor<T>) where the function f(x) = alpha * x for x < 0, f(x) = x for x >= 0, is applied to the data tensor elementwise.

History - Version 16 adds bfloat16 to the types allowed.

Attributes

  • alpha: Coefficient of leakage. Default value is namealphaf0.009999999776482582typeFLOAT (FLOAT)

Inputs

  • X (heterogeneous)T: Input tensor

Outputs

  • Y (heterogeneous)T: Output tensor

Type Constraints

  • T tensor(bfloat16), tensor(float16), tensor(float), tensor(double): Constrain input and output types to float tensors.

Version

Onnx name: LeakyRelu

This version of the operator has been available since version 16.

Runtime implementation: LeakyRelu

Less#

mlprodict.onnxrt.ops_cpu.op_less.Less (self, onnx_node, desc = None, options)

Returns the tensor resulted from performing the less logical operation elementwise on the input tensors A and B (with Numpy-style broadcasting support).

This operator supports multidirectional (i.e., Numpy-style) broadcasting; for more details please check Broadcasting in ONNX.

Inputs

  • A (heterogeneous)T: First input operand for the logical operator.

  • B (heterogeneous)T: Second input operand for the logical operator.

Outputs

  • C (heterogeneous)T1: Result tensor.

Type Constraints

  • T tensor(uint8), tensor(uint16), tensor(uint32), tensor(uint64), tensor(int8), tensor(int16), tensor(int32), tensor(int64), tensor(float16), tensor(float), tensor(double), tensor(bfloat16): Constrain input types to all numeric tensors.

  • T1 tensor(bool): Constrain output to boolean tensor.

Version

Onnx name: Less

This version of the operator has been available since version 13.

Runtime implementation: Less

LessOrEqual#

mlprodict.onnxrt.ops_cpu.op_less.LessOrEqual (self, onnx_node, desc = None, options)

Returns the tensor resulted from performing the less_equal logical operation elementwise on the input tensors A and B (with Numpy-style broadcasting support).

This operator supports multidirectional (i.e., Numpy-style) broadcasting; for more details please check Broadcasting in ONNX.

Inputs

  • A (heterogeneous)T: First input operand for the logical operator.

  • B (heterogeneous)T: Second input operand for the logical operator.

Outputs

  • C (heterogeneous)T1: Result tensor.

Type Constraints

  • T tensor(uint8), tensor(uint16), tensor(uint32), tensor(uint64), tensor(int8), tensor(int16), tensor(int32), tensor(int64), tensor(float16), tensor(float), tensor(double), tensor(bfloat16): Constrain input types to all numeric tensors.

  • T1 tensor(bool): Constrain output to boolean tensor.

Version

Onnx name: LessOrEqual

This version of the operator has been available since version 16.

Runtime implementation: LessOrEqual

LinearClassifier#

mlprodict.onnxrt.ops_cpu.op_linear_classifier.LinearClassifier (self, onnx_node, desc = None, options)

Linear classifier

Attributes

  • classlabels_ints: Class labels when using integer labels. One and only one ‘classlabels’ attribute must be defined. default value cannot be automatically retrieved (INTS)

  • classlabels_strings: Class labels when using string labels. One and only one ‘classlabels’ attribute must be defined. default value cannot be automatically retrieved (STRINGS)

  • coefficients (required): A collection of weights of the model(s). default value cannot be automatically retrieved (FLOATS)

  • intercepts: A collection of intercepts. default value cannot be automatically retrieved (FLOATS)

  • multi_class: Indicates whether to do OvR or multinomial (0=OvR is the default). Default value is namemulticlassi0typeINT (INT)

  • post_transform: Indicates the transform to apply to the scores vector. One of ‘NONE,’ ‘SOFTMAX,’ ‘LOGISTIC,’ ‘SOFTMAX_ZERO,’ or ‘PROBIT’ Default value is nameposttransformsNONEtypeSTRING (STRING)

Inputs

  • X (heterogeneous)T1: Data to be classified.

Outputs

  • Y (heterogeneous)T2: Classification outputs (one class per example).

  • Z (heterogeneous)tensor(float): Classification scores ([N,E] - one score for each class and example

Type Constraints

  • T1 tensor(float), tensor(double), tensor(int64), tensor(int32): The input must be a tensor of a numeric type, and of of shape [N,C] or [C]. In the latter case, it will be treated as [1,C]

  • T2 tensor(string), tensor(int64): The output will be a tensor of strings or integers.

Version

Onnx name: LinearClassifier

This version of the operator has been available since version 1 of domain ai.onnx.ml.

Runtime implementation: LinearClassifier

LinearRegressor#

mlprodict.onnxrt.ops_cpu.op_linear_regressor.LinearRegressor (self, onnx_node, desc = None, options)

Generalized linear regression evaluation.

If targets is set to 1 (default) then univariate regression is performed.

If targets is set to M then M sets of coefficients must be passed in as a sequence and M results will be output for each input n in N.

The coefficients array is of length n, and the coefficients for each target are contiguous. Intercepts are optional but if provided must match the number of targets.

Attributes

  • coefficients: Weights of the model(s). default value cannot be automatically retrieved (FLOATS)

  • intercepts: Weights of the intercepts, if used. default value cannot be automatically retrieved (FLOATS)

  • post_transform: Indicates the transform to apply to the regression output vector. One of ‘NONE,’ ‘SOFTMAX,’ ‘LOGISTIC,’ ‘SOFTMAX_ZERO,’ or ‘PROBIT’ Default value is nameposttransformsNONEtypeSTRING (STRING)

  • targets: The total number of regression targets, 1 if not defined. Default value is nametargetsi1typeINT (INT)

Inputs

  • X (heterogeneous)T: Data to be regressed.

Outputs

  • Y (heterogeneous)tensor(float): Regression outputs (one per target, per example).

Type Constraints

  • T tensor(float), tensor(double), tensor(int64), tensor(int32): The input must be a tensor of a numeric type.

Version

Onnx name: LinearRegressor

This version of the operator has been available since version 1 of domain ai.onnx.ml.

Runtime implementation: LinearRegressor

Log#

mlprodict.onnxrt.ops_cpu.op_log.Log (self, onnx_node, desc = None, options)

Calculates the natural log of the given input tensor, element-wise.

Inputs

  • input (heterogeneous)T: Input tensor

Outputs

  • output (heterogeneous)T: The natural log of the input tensor computed element-wise

Type Constraints

  • T tensor(float16), tensor(float), tensor(double), tensor(bfloat16): Constrain input and output types to float tensors.

Version

Onnx name: Log

This version of the operator has been available since version 13.

Runtime implementation: Log

LogSoftmax#

mlprodict.onnxrt.ops_cpu.op_log_softmax.LogSoftmax (self, onnx_node, desc = None, options)

The operator computes the log of softmax values for the given input:

LogSoftmax(input, axis) = Log(Softmax(input, axis=axis))

The “axis” attribute indicates the dimension along which LogSoftmax will be performed. The output tensor has the same shape and contains the LogSoftmax values of the corresponding input.

Attributes

  • axis:

Describes the dimension LogSoftmax will be performed on. Negative value means counting dimensions from the back. Accepted range is [-r, r-1] where r = rank(input).

Default value is nameaxisi-1typeINT (INT)

Inputs

  • input (heterogeneous)T: The input tensor of rank >= axis.

Outputs

  • output (heterogeneous)T: The output values with the same shape as the input tensor.

Type Constraints

  • T tensor(float16), tensor(float), tensor(double), tensor(bfloat16): Constrain input and output types to float tensors.

Version

Onnx name: LogSoftmax

This version of the operator has been available since version 13.

Runtime implementation: LogSoftmax

Loop#

mlprodict.onnxrt.ops_cpu.op_loop.Loop (self, onnx_node, desc = None, options)

Generic Looping construct. This loop has multiple termination conditions:

  1. Trip count. Iteration count specified at runtime. Set by specifying the input M. Optional. Set to empty string to omit. Note that a static trip count (specified at graph construction time) can be specified by passing in a constant node for input M.

  2. Loop termination condition. This is an input to the op that determines whether to run the first iteration and also a loop-carried dependency for the body graph. The body graph must yield a value for the condition variable, whether this input is provided or not.

This table summarizes the operating modes of this operator with equivalent C-style code:

Operator inputs defined as (max_trip_count, condition_var).

input (“”, “”):
for (int i=0; ; ++i) {

cond = … // Note this value is ignored, but is required in the body

}

input (“”, cond) // Note this is analogous to a while loop

bool cond = …; for (int i=0; cond; ++i) {

cond = …;

}

input (“”, 1) // Note this is analogous to a do-while loop

bool cond = true for (int i=0; cond; ++i) {

cond = …;

}

input (trip_count, “”) // Note this is analogous to a for loop

int trip_count = … for (int i=0; i < trip_count; ++i) {

cond = …; // ignored

}

input (trip_count, cond)

int trip_count = …; bool cond = …; for (int i=0; i < trip_count && cond; ++i) {

cond = …;

}

Sample usage - cond as well as trip count

graph predict-net {

%a = Constant[value = <Scalar Tensor [3]>]() %b = Constant[value = <Scalar Tensor [6]>]() %keepgoing = Constant[value = <Scalar Tensor [1]>]() %max_trip_count = Constant[value = <Scalar Tensor [10]>]() %keepgoing_out, %b_out, %user_defined_vals = Loop[body = <graph body-net>](%max_trip_count, %keepgoing, %b) return

}

graph body-net (

%i[INT32, scalar] // iteration number %keepgoing_in[BOOL, scalar] // incoming loop-termination-condition; not used %b_in[INT32, scalar] // incoming value of loop-carried-dependency b

) {

%my_local = Add(%a, %b_in) %b_out = Sub(%a, %b_in) // outgoing value of loop-carried-dependency b %keepgoing_out = Greater(%my_local, %b_out) // outgoing loop-termination-condition %user_defined_val = Add(%b_in, %b_in) // scan-output value to be accumulated return %keepgoing_out, %b_out, %user_defined_val

}

Sample equivalent C code

{

/* User-defined code (enclosing scope) / int a = 3, b = 6; bool keepgoing = true; // Analogous to input cond / End user-defined code */

/* Implicitly-defined code / const int max_trip_count = 10; // Analogous to input M int user_defined_vals[]; // Imagine this is resizable / End implicitly-defined code / / initialize loop-carried variables and scan-output variables */ bool keepgoing_out = keepgoing int b_out = b

for (int i=0; i < max_trip_count && keepgoing_out; ++i) {
/* Implicitly-defined code: bind actual parameter values

to formal parameter variables of loop-body */

bool keepgoing_in = keepgoing_out; bool b_in = b_out;

/* User-defined code (loop body) / int my_local = a + b_in; // Reading value “a” from the enclosing scope is fine b_out = a - b_in; keepgoing_out = my_local > b_out; user_defined_val = b_in + b_in; // b_in and b_out are different variables / End user-defined code */

/* Implicitly defined-code */ user_defined_vals[i] = user_defined_val // accumulate scan-output values

} // int t = my_local; // Can’t do this. my_local is not accessible here.

// The values below are bound to the output variables of the loop and therefore accessible // b_out; user_defined_vals; keepgoing_out;

}

There are several things of note in this code snippet:

  1. Values from the enclosing scope (i.e. variable “a” here) are in scope and can be referenced in the inputs of the loop.

  2. Any values computed in the loop body that needs to be used in a subsequent iteration or after the loop are modelled using a pair of variables in the loop-body, consisting of an input variable (eg., b_in) and an output variable (eg., b_out). These are referred to as loop-carried dependences. The loop operation node supplies the input value of the input variable for the first iteration, and returns the output value of the output variable produced by the final iteration.

  3. Scan_output variables are used to implicitly concatenate values computed across all the iterations. In the above example, the value of user_defined_val computed over all iterations are concatenated and returned as the value of user_defined_vals after the loop.

  4. Values created in the body cannot be accessed in the enclosing scope, except using the mechanism described above.

Note that the semantics of this op support “diagonal” or “wavefront” execution. (See Step 3 here for an example: https://devblogs.nvidia.com/optimizing-recurrent-neural-networks-cudnn-5/). Frontends should emit multi-layer RNNs as a series of While operators (with time being the inner looping dimension), with each successive layer consuming the scan_outputs from the previous layer, possibly going through several point-wise operators (e.g. dropout, residual connections, linear layer).

The input/output of subgraph (produced by loop node) matching is based on order instead of name. The implementation will figure out the names based on this order.

Attributes

  • body (required): The graph run each iteration. It has 2+N inputs: (iteration_num, condition, loop carried dependencies…). It has 1+N+K outputs: (condition, loop carried dependencies…, scan_outputs…). Each scan_output is created by concatenating the value of the specified output value at the end of each iteration of the loop. It is an error if the dimensions or data type of these scan_outputs change across loop iterations. default value cannot be automatically retrieved (GRAPH)

Inputs

Between 2 and 2147483647 inputs.

  • M (optional, heterogeneous)I: A maximum trip-count for the loop specified at runtime. Optional. Pass empty string to skip.

  • cond (optional, heterogeneous)B: A boolean termination condition. Optional. Pass empty string to skip.

  • v_initial (variadic)V: The initial values of any loop-carried dependencies (values that change across loop iterations)

Outputs

Between 1 and 2147483647 outputs.

  • v_final_and_scan_outputs (variadic)V: Final N loop carried dependency values then K scan_outputs. Scan outputs must be Tensors.

Type Constraints

  • V tensor(uint8), tensor(uint16), tensor(uint32), tensor(uint64), tensor(int8), tensor(int16), tensor(int32), tensor(int64), tensor(bfloat16), tensor(float16), tensor(float), tensor(double), tensor(string), tensor(bool), tensor(complex64), tensor(complex128), seq(tensor(uint8)), seq(tensor(uint16)), seq(tensor(uint32)), seq(tensor(uint64)), seq(tensor(int8)), seq(tensor(int16)), seq(tensor(int32)), seq(tensor(int64)), seq(tensor(bfloat16)), seq(tensor(float16)), seq(tensor(float)), seq(tensor(double)), seq(tensor(string)), seq(tensor(bool)), seq(tensor(complex64)), seq(tensor(complex128)), optional(seq(tensor(uint8))), optional(seq(tensor(uint16))), optional(seq(tensor(uint32))), optional(seq(tensor(uint64))), optional(seq(tensor(int8))), optional(seq(tensor(int16))), optional(seq(tensor(int32))), optional(seq(tensor(int64))), optional(seq(tensor(bfloat16))), optional(seq(tensor(float16))), optional(seq(tensor(float))), optional(seq(tensor(double))), optional(seq(tensor(string))), optional(seq(tensor(bool))), optional(seq(tensor(complex64))), optional(seq(tensor(complex128))), optional(tensor(uint8)), optional(tensor(uint16)), optional(tensor(uint32)), optional(tensor(uint64)), optional(tensor(int8)), optional(tensor(int16)), optional(tensor(int32)), optional(tensor(int64)), optional(tensor(bfloat16)), optional(tensor(float16)), optional(tensor(float)), optional(tensor(double)), optional(tensor(string)), optional(tensor(bool)), optional(tensor(complex64)), optional(tensor(complex128)): All Tensor, Sequence(Tensor), Optional(Tensor), and Optional(Sequence(Tensor)) types

  • I tensor(int64): tensor of int64, which should be a scalar.

  • B tensor(bool): tensor of bool, which should be a scalar.

Version

Onnx name: Loop

This version of the operator has been available since version 16.

Runtime implementation: Loop

LpNormalization#

mlprodict.onnxrt.ops_cpu.op_lp_normalization.LpNormalization (self, onnx_node, desc = None, options)

Given a matrix, apply Lp-normalization along the provided axis.

Attributes

  • axis: The axis on which to apply normalization, -1 mean last axis. Default value is nameaxisi-1typeINT (INT)

  • p: The order of the normalization, only 1 or 2 are supported. Default value is namepi2typeINT (INT)

Inputs

  • input (heterogeneous)T: Input matrix

Outputs

  • output (heterogeneous)T: Matrix after normalization

Type Constraints

  • T tensor(float16), tensor(float), tensor(double): Constrain input and output types to float tensors.

Version

Onnx name: LpNormalization

This version of the operator has been available since version 1.

Runtime implementation: LpNormalization

MatMul#

mlprodict.onnxrt.ops_cpu.op_matmul.MatMul (self, onnx_node, desc = None, options)

Matrix product that behaves like numpy.matmul: https://docs.scipy.org/doc/numpy-1.13.0/reference/generated/numpy.matmul.html

Inputs

  • A (heterogeneous)T: N-dimensional matrix A

  • B (heterogeneous)T: N-dimensional matrix B

Outputs

  • Y (heterogeneous)T: Matrix multiply results from A * B

Type Constraints

  • T tensor(float16), tensor(float), tensor(double), tensor(uint32), tensor(uint64), tensor(int32), tensor(int64), tensor(bfloat16): Constrain input and output types to float/int tensors.

Version

Onnx name: MatMul

This version of the operator has been available since version 13.

Runtime implementation: MatMul

Max#

mlprodict.onnxrt.ops_cpu.op_max.Max (self, onnx_node, desc = None, options)

Element-wise max of each of the input tensors (with Numpy-style broadcasting support). All inputs and outputs must have the same data type. This operator supports multidirectional (i.e., Numpy-style) broadcasting; for more details please check Broadcasting in ONNX.

Inputs

Between 1 and 2147483647 inputs.

  • data_0 (variadic, heterogeneous)T: List of tensors for max.

Outputs

  • max (heterogeneous)T: Output tensor.

Type Constraints

  • T tensor(uint8), tensor(uint16), tensor(uint32), tensor(uint64), tensor(int8), tensor(int16), tensor(int32), tensor(int64), tensor(float16), tensor(float), tensor(double), tensor(bfloat16): Constrain input and output types to numeric tensors.

Version

Onnx name: Max

This version of the operator has been available since version 13.

Runtime implementation: Max

MaxPool#

mlprodict.onnxrt.ops_cpu.op_max_pool.MaxPool (self, onnx_node, desc = None, options)

MaxPool consumes an input tensor X and applies max pooling across the tensor according to kernel sizes, stride sizes, and pad lengths. max pooling consisting of computing the max on all values of a subset of the input tensor according to the kernel size and downsampling the data into the output tensor Y for further processing. The output spatial shape will be following: `` output_spatial_shape[i] = floor((input_spatial_shape[i] + pad_shape[i] - ((kernel_spatial_shape[i] - 1) * dilations[i] + 1)) / strides_spatial_shape[i] + 1) `` or `` output_spatial_shape[i] = ceil((input_spatial_shape[i] + pad_shape[i] - ((kernel_spatial_shape[i] - 1) * dilations[i] + 1)) / strides_spatial_shape[i] + 1) `` if ceil_mode is enabled

`` * pad_shape[i] is sum of pads along axis i ``

auto_pad is a DEPRECATED attribute. If you are using them currently, the output spatial shape will be following: `` VALID: output_spatial_shape[i] = ceil((input_spatial_shape[i] - ((kernel_spatial_shape[i] - 1) * dilations[i] + 1) + 1) / strides_spatial_shape[i]) SAME_UPPER or SAME_LOWER: output_spatial_shape[i] = ceil(input_spatial_shape[i] / strides_spatial_shape[i]) `` And pad shape will be following if SAME_UPPER or SAME_LOWER: `` pad_shape[i] = (output_spatial_shape[i] - 1) * strides_spatial_shape[i] + ((kernel_spatial_shape[i] - 1) * dilations[i] + 1) - input_spatial_shape[i] `` The output of each pooling window is maximum number of elements exclude pad.

Attributes

  • auto_pad: auto_pad must be either NOTSET, SAME_UPPER, SAME_LOWER or VALID. Where default value is NOTSET, which means explicit padding is used. SAME_UPPER or SAME_LOWER mean pad the input so that output_shape[i] = ceil(input_shape[i] / strides[i]) for each axis i. The padding is split between the two sides equally or almost equally (depending on whether it is even or odd). In case the padding is an odd number, the extra padding is added at the end for SAME_UPPER and at the beginning for SAME_LOWER. Default value is nameautopadsNOTSETtypeSTRING (STRING)

  • ceil_mode: Whether to use ceil or floor (default) to compute the output shape. Default value is nameceilmodei0typeINT (INT)

  • dilations: Dilation value along each spatial axis of filter. If not present, the dilation defaults to 1 along each spatial axis. default value cannot be automatically retrieved (INTS)

  • kernel_shape (required): The size of the kernel along each axis. default value cannot be automatically retrieved (INTS)

  • pads: Padding for the beginning and ending along each spatial axis, it can take any value greater than or equal to 0. The value represent the number of pixels added to the beginning and end part of the corresponding axis. pads format should be as follow [x1_begin, x2_begin…x1_end, x2_end,…], where xi_begin the number of pixels added at the beginning of axis i and xi_end, the number of pixels added at the end of axis i. This attribute cannot be used simultaneously with auto_pad attribute. If not present, the padding defaults to 0 along start and end of each spatial axis. default value cannot be automatically retrieved (INTS)

  • storage_order: The storage order of the tensor. 0 is row major, and 1 is column major. Default value is namestorageorderi0typeINT (INT)

  • strides: Stride along each spatial axis. If not present, the stride defaults to 1 along each spatial axis. default value cannot be automatically retrieved (INTS)

Inputs

  • X (heterogeneous)T: Input data tensor from the previous operator; dimensions for image case are (N x C x H x W), where N is the batch size, C is the number of channels, and H and W are the height and the width of the data. For non image case, the dimensions are in the form of (N x C x D1 x D2 … Dn), where N is the batch size. Optionally, if dimension denotation is in effect, the operation expects the input data tensor to arrive with the dimension denotation of [DATA_BATCH, DATA_CHANNEL, DATA_FEATURE, DATA_FEATURE …].

Outputs

Between 1 and 2 outputs.

  • Y (heterogeneous)T: Output data tensor from average or max pooling across the input tensor. Dimensions will vary based on various kernel, stride, and pad sizes. Floor value of the dimension is used

  • Indices (optional, heterogeneous)I: Indices tensor from max pooling across the input tensor. The dimensions of indices are the same as output tensor. The values in indices of are the indices of the selected values during pooling. The indices are computed as flatten 1-D tensor, and the indices do not consider padding. So the values in indices are in [0, N x C x D1 x … x Dn).

Type Constraints

  • T tensor(float16), tensor(float), tensor(double), tensor(int8), tensor(uint8): Constrain input and output types to float and 8 bit tensors.

  • I tensor(int64): Constrain index tensor to int64

Version

Onnx name: MaxPool

This version of the operator has been available since version 12.

Runtime implementation: MaxPool

Mean#

mlprodict.onnxrt.ops_cpu.op_mean.Mean (self, onnx_node, desc = None, options)

Element-wise mean of each of the input tensors (with Numpy-style broadcasting support). All inputs and outputs must have the same data type. This operator supports multidirectional (i.e., Numpy-style) broadcasting; for more details please check Broadcasting in ONNX.

Inputs

Between 1 and 2147483647 inputs.

  • data_0 (variadic, heterogeneous)T: List of tensors for mean.

Outputs

  • mean (heterogeneous)T: Output tensor.

Type Constraints

  • T tensor(float16), tensor(float), tensor(double), tensor(bfloat16): Constrain input and output types to float tensors.

Version

Onnx name: Mean

This version of the operator has been available since version 13.

Runtime implementation: Mean

Min#

mlprodict.onnxrt.ops_cpu.op_min.Min (self, onnx_node, desc = None, options)

Element-wise min of each of the input tensors (with Numpy-style broadcasting support). All inputs and outputs must have the same data type. This operator supports multidirectional (i.e., Numpy-style) broadcasting; for more details please check Broadcasting in ONNX.

Inputs

Between 1 and 2147483647 inputs.

  • data_0 (variadic, heterogeneous)T: List of tensors for min.

Outputs

  • min (heterogeneous)T: Output tensor.

Type Constraints

  • T tensor(uint8), tensor(uint16), tensor(uint32), tensor(uint64), tensor(int8), tensor(int16), tensor(int32), tensor(int64), tensor(float16), tensor(float), tensor(double), tensor(bfloat16): Constrain input and output types to numeric tensors.

Version

Onnx name: Min

This version of the operator has been available since version 13.

Runtime implementation: Min

Mod#

mlprodict.onnxrt.ops_cpu.op_mod.Mod (self, onnx_node, desc = None, options)

Performs element-wise binary modulus (with Numpy-style broadcasting support).

The sign of the remainder is the same as that of the Divisor.

Mod operator can also behave like C fmod() or numpy.fmod. In this case, the sign of the remainder however, will be the same as the Dividend (in contrast to integer mod). To force a behavior like numpy.fmod() an ‘fmod’ Attribute is provided. This attribute is set to 0 by default causing the behavior to be like integer mod. Setting this attribute to 1 causes the remainder to be calculated similar to that of numpy.fmod().

If the input type is floating point, then fmod attribute must be set to 1.

In case of dividend being zero, the results will be platform dependent.

This operator supports multidirectional (i.e., Numpy-style) broadcasting; for more details please check Broadcasting in ONNX.

Attributes

  • fmod: Whether the operator should behave like fmod (default=0 meaning it will do integer mods); Set this to 1 to force fmod treatment Default value is namefmodi0typeINT (INT)

Inputs

  • A (heterogeneous)T: Dividend tensor

  • B (heterogeneous)T: Divisor tensor

Outputs

  • C (heterogeneous)T: Remainder tensor

Type Constraints

  • T tensor(uint8), tensor(uint16), tensor(uint32), tensor(uint64), tensor(int8), tensor(int16), tensor(int32), tensor(int64), tensor(float16), tensor(float), tensor(double), tensor(bfloat16): Constrain input and output types to high-precision numeric tensors.

Version

Onnx name: Mod

This version of the operator has been available since version 13.

Runtime implementation: Mod

Momentum#

mlprodict.onnxrt.ops_cpu.op_momentum.Momentum (self, onnx_node, desc = None, options)

Compute one iteration of stochastic gradient update with momentum. This operator can conduct the optimization of multiple tensor variables.

Let’s define the behavior of this operator. As you can imagine, SG with momentum requires several parameters:

  • The learning-rate “R”.

  • The update count “T”. That is, the number of conducted training iterations. It should be zero in the first training iteration.

  • A L2-norm regularization coefficient “norm_coefficient”.

  • A decay coefficient of previous accumulated gradient (i.e., momentum) “alpha”.

  • The scaling coefficient of current gradient “beta”.

  • An attribute to choose either standard momentum or Nesterov’s momentum “mode” should be used.

For the sake of simplicity, assume that there is only one tensor (called “X”) to be optimized. Other necessary inputs are “X“‘s gradient (called “G”) and “X“‘s momentum (called “V”). This Momentum operator maps all these inputs to the new value of “X” (called “X_new”) and its new momentum (called “V_new”).

This operator supports two different momentum algorithms. Set the attribute “mode” to “nesterov” if Nesterov’s momentum is desired. Otherwise, set the attribute “model” to “standard” to use standard momentum. Computation details are described subsequently.

Let “+”, “-”, “*”, and “/” are all element-wise operations with numpy-style broadcasting.

Pseudo code for SG with standard momentum:

// Add gradient of 0.5 * norm_coefficient * ||X||^2, where ||X|| is the sum of squared // values of all elements in X. G_regularized = norm_coefficient * X + G

// In the first training iteration, beta should always be 1. beta_adjusted = T > 0 ? beta : 1

// Compute the current momentum based on previous momentum and the current gradient. V_new = alpha * V + beta_adjusted * G_regularized

// Update X. X_new = X - R * V_new

Pseudo code for SG with Nesterov’s momentum:

// Add gradient of 0.5 * norm_coefficient * ||X||^2, where ||X|| is the sum of squared // values of all elements in X. G_regularized = norm_coefficient * X + G;

// In the first training iteration, beta should always be 1. beta_adjusted = T > 0 ? beta : 1

// Compute the current momentum based on previous momentum and the current gradient. V_new = alpha * V + beta_adjusted * G_regularized;

// Compute final update direction and then update X. X_new = X - R * (G_regularized + alpha * V_new)

If one assign this operators to optimize multiple inputs, for example, “X_1” and “X_2”. The same pseudo code would be extended to handle all tensors jointly. More specifically, we can view “X” as a concatenation of “X_1” and “X_2” (of course, their gradient and accumulate gradient should be concatenated too) and then our pseudo code becomes applicable.

Attributes

  • alpha (required): The decay factor of momentum. It should be a scalar. default value cannot be automatically retrieved (FLOAT)

  • beta (required): The coefficient of gradient in computing new momentum. It should be a scalar. default value cannot be automatically retrieved (FLOAT)

  • mode (required): Its value should be either “nesterov” or “standard”. The value “nesterov” leads to the use of Nesterov’s momentum while “standard” invokes stochastic gradient method using standard momentum default value cannot be automatically retrieved (STRING)

  • norm_coefficient (required): Coefficient of 0.5 * norm_coefficient * ||X||^2. default value cannot be automatically retrieved (FLOAT)

Inputs

Between 3 and 2147483647 inputs.

  • R (heterogeneous)T1: The learning rate.

  • T (heterogeneous)T2: Update count of “X”. It should be a scalar.

  • inputs (variadic)T3: It sequentially contains the current values of optimized tensors, then their gradient tensors, and finally their momentum tensors. For example, if two tensors “X_1” and “X_2” are optimized, The expected input list would be [“X_1”, “X_2”, gradient of “X_1”, gradient of “X_2”, momentum of “X_1”, momentum of “X_2”].

Outputs

Between 1 and 2147483647 outputs.

  • outputs (variadic)T3: It sequentially contains the new values of optimized tensors and then the new values of their momentum tensors. For example, if two tensors “X_1” and “X_2” are optimized, the output list would be [new value of “X_1,” new value of “X_2” new momentum of “X_1”, new momentum of “X_2”].

Type Constraints

  • T1 tensor(float), tensor(double): Constrain input types to float scalars.

  • T2 tensor(int64): Constrain input types to 64-bit integer scalars.

  • T3 tensor(float), tensor(double): Constrain input types to float tensors.

Version

Onnx name: Momentum

This version of the operator has been available since version 1 of domain ai.onnx.preview.training.

Runtime implementation: Momentum

Mul#

mlprodict.onnxrt.ops_cpu.op_mul.Mul (self, onnx_node, desc = None, options)

Performs element-wise binary multiplication (with Numpy-style broadcasting support).

This operator supports multidirectional (i.e., Numpy-style) broadcasting; for more details please check Broadcasting in ONNX.

(Opset 14 change): Extend supported types to include uint8, int8, uint16, and int16.

Inputs

  • A (heterogeneous)T: First operand.

  • B (heterogeneous)T: Second operand.

Outputs

  • C (heterogeneous)T: Result, has same element type as two inputs

Type Constraints

  • T tensor(uint8), tensor(uint16), tensor(uint32), tensor(uint64), tensor(int8), tensor(int16), tensor(int32), tensor(int64), tensor(float16), tensor(float), tensor(double), tensor(bfloat16): Constrain input and output types to all numeric tensors.

Version

Onnx name: Mul

This version of the operator has been available since version 14.

Runtime implementation: Mul

Neg#

mlprodict.onnxrt.ops_cpu.op_neg.Neg (self, onnx_node, desc = None, options)

Neg takes one input data (Tensor<T>) and produces one output data (Tensor<T>) where each element flipped sign, y = -x, is applied to the tensor elementwise.

Inputs

  • X (heterogeneous)T: Input tensor

Outputs

  • Y (heterogeneous)T: Output tensor

Type Constraints

  • T tensor(float), tensor(int32), tensor(int8), tensor(int16), tensor(int64), tensor(float16), tensor(double), tensor(bfloat16): Constrain input and output types to signed numeric tensors.

Version

Onnx name: Neg

This version of the operator has been available since version 13.

Runtime implementation: Neg

NonMaxSuppression#

mlprodict.onnxrt.ops_cpu.op_non_max_suppression.NonMaxSuppression (self, onnx_node, desc = None, options)

Filter out boxes that have high intersection-over-union (IOU) overlap with previously selected boxes. Bounding boxes with score less than score_threshold are removed. Bounding box format is indicated by attribute center_point_box. Note that this algorithm is agnostic to where the origin is in the coordinate system and more generally is invariant to orthogonal transformations and translations of the coordinate system; thus translating or reflections of the coordinate system result in the same boxes being selected by the algorithm. The selected_indices output is a set of integers indexing into the input collection of bounding boxes representing the selected boxes. The bounding box coordinates corresponding to the selected indices can then be obtained using the Gather or GatherND operation.

Attributes

  • center_point_box: Integer indicate the format of the box data. The default is 0. 0 - the box data is supplied as [y1, x1, y2, x2] where (y1, x1) and (y2, x2) are the coordinates of any diagonal pair of box corners and the coordinates can be provided as normalized (i.e., lying in the interval [0, 1]) or absolute. Mostly used for TF models. 1 - the box data is supplied as [x_center, y_center, width, height]. Mostly used for Pytorch models. Default value is namecenterpointboxi0typeINT (INT)

Inputs

Between 2 and 5 inputs.

  • boxes (heterogeneous)tensor(float): An input tensor with shape [num_batches, spatial_dimension, 4]. The single box data format is indicated by center_point_box.

  • scores (heterogeneous)tensor(float): An input tensor with shape [num_batches, num_classes, spatial_dimension]

  • max_output_boxes_per_class (optional, heterogeneous)tensor(int64): Integer representing the maximum number of boxes to be selected per batch per class. It is a scalar. Default to 0, which means no output.

  • iou_threshold (optional, heterogeneous)tensor(float): Float representing the threshold for deciding whether boxes overlap too much with respect to IOU. It is scalar. Value range [0, 1]. Default to 0.

  • score_threshold (optional, heterogeneous)tensor(float): Float representing the threshold for deciding when to remove boxes based on score. It is a scalar.

Outputs

  • selected_indices (heterogeneous)tensor(int64): selected indices from the boxes tensor. [num_selected_indices, 3], the selected index format is [batch_index, class_index, box_index].

Version

Onnx name: NonMaxSuppression

This version of the operator has been available since version 11.

Runtime implementation: NonMaxSuppression

NonZero#

mlprodict.onnxrt.ops_cpu.op_non_zero.NonZero (self, onnx_node, desc = None, options)

Returns the indices of the elements that are non-zero (in row-major order - by dimension). NonZero behaves similar to numpy.nonzero: https://docs.scipy.org/doc/numpy/reference/generated/numpy.nonzero.html, but for scalar input, NonZero produces output shape (0, N) instead of (1, N), which is different from Numpy’s behavior.

Inputs

  • X (heterogeneous)T: input

Outputs

  • Y (heterogeneous)tensor(int64): output

Type Constraints

  • T tensor(uint8), tensor(uint16), tensor(uint32), tensor(uint64), tensor(int8), tensor(int16), tensor(int32), tensor(int64), tensor(bfloat16), tensor(float16), tensor(float), tensor(double), tensor(string), tensor(bool), tensor(complex64), tensor(complex128): Constrain to all tensor types.

Version

Onnx name: NonZero

This version of the operator has been available since version 13.

Runtime implementation: NonZero

Normalizer#

mlprodict.onnxrt.ops_cpu.op_normalizer.Normalizer (self, onnx_node, desc = None, options)

Normalize the input. There are three normalization modes, which have the corresponding formulas, defined using element-wise infix operators ‘/’ and ‘^’ and tensor-wide functions ‘max’ and ‘sum’:

Max: Y = X / max(X)

L1: Y = X / sum(X)

L2: Y = sqrt(X^2 / sum(X^2)}

In all modes, if the divisor is zero, Y == X.

For batches, that is, [N,C] tensors, normalization is done along the C axis. In other words, each row of the batch is normalized independently.

Attributes

  • norm: One of ‘MAX,’ ‘L1,’ ‘L2’ Default value is namenormsMAXtypeSTRING (STRING)

Inputs

  • X (heterogeneous)T: Data to be encoded, a tensor of shape [N,C] or [C]

Outputs

  • Y (heterogeneous)tensor(float): Encoded output data

Type Constraints

  • T tensor(float), tensor(double), tensor(int64), tensor(int32): The input must be a tensor of a numeric type.

Version

Onnx name: Normalizer

This version of the operator has been available since version 1 of domain ai.onnx.ml.

Runtime implementation: Normalizer

Not#

mlprodict.onnxrt.ops_cpu.op_not.Not (self, onnx_node, desc = None, options)

Returns the negation of the input tensor element-wise.

Inputs

  • X (heterogeneous)T: Input tensor

Outputs

  • Y (heterogeneous)T: Output tensor

Type Constraints

  • T tensor(bool): Constrain input/output to boolean tensors.

Version

Onnx name: Not

This version of the operator has been available since version 1.

Runtime implementation: Not

OneHot#

mlprodict.onnxrt.ops_cpu.op_one_hot.OneHot (self, onnx_node, desc = None, options)

Produces a one-hot tensor based on inputs. The locations represented by the index values in the ‘indices’ input tensor will have ‘on_value’ and the other locations will have ‘off_value’ in the output tensor, where ‘on_value’ and ‘off_value’ are specified as part of required input argument ‘values’, which is a two-element tensor of format [off_value, on_value]. The rank of the output tensor will be one greater than the rank of the input tensor. The additional dimension is for one-hot representation. The additional dimension will be inserted at the position specified by ‘axis’. If ‘axis’ is not specified then then additional dimension will be inserted as the innermost dimension, i.e. axis=-1. The size of the additional dimension is specified by required scalar input ‘depth’. The type of the output tensor is the same as the type of the ‘values’ input. Any entries in the ‘indices’ input tensor with values outside the range [-depth, depth-1] will result in one-hot representation with all ‘off_value’ values in the output tensor.

when axis = 0: output[input[i, j, k], i, j, k] = 1 for all i, j, k and 0 otherwise.

when axis = -1: output[i, j, k, input[i, j, k]] = 1 for all i, j, k and 0 otherwise.

Attributes

  • axis: (Optional) Axis along which one-hot representation in added. Default: axis=-1. axis=-1 means that the additional dimension will be inserted as the innermost/last dimension in the output tensor. Negative value means counting dimensions from the back. Accepted range is [-r-1, r] where r = rank(indices). Default value is nameaxisi-1typeINT (INT)

Inputs

  • indices (heterogeneous)T1: Input tensor containing indices. Any entries in the ‘indices’ input tensor with values outside the range [-depth, depth-1] will result in one-hot representation with all ‘off_value’ values in the output tensor.In case ‘indices’ is of non-integer type, the values will be casted to int64 before use.

  • depth (heterogeneous)T2: Scalar specifying the number of classes in one-hot tensor. This is also the size of the one-hot dimension (specified by ‘axis’ attribute) added on in the output tensor. The values in the ‘indices’ input tensor are expected to be in the range [-depth, depth-1]. In case ‘depth’ is of non-integer type, it will be casted to int64 before use.

  • values (heterogeneous)T3: Rank 1 tensor containing exactly two elements, in the format [off_value, on_value], where ‘on_value’ is the value used for filling locations specified in ‘indices’ input tensor, and ‘off_value’ is the value used for filling locations other than those specified in ‘indices’ input tensor.

Outputs

  • output (heterogeneous)T3: Tensor of rank one greater than input tensor ‘indices’, i.e. rank(output) = rank(indices) + 1. The data type for the elements of the output tensor is the same as the type of input ‘values’ is used.

Type Constraints

  • T1 tensor(uint8), tensor(uint16), tensor(uint32), tensor(uint64), tensor(int8), tensor(int16), tensor(int32), tensor(int64), tensor(float16), tensor(float), tensor(double): Constrain input to only numeric types.

  • T2 tensor(uint8), tensor(uint16), tensor(uint32), tensor(uint64), tensor(int8), tensor(int16), tensor(int32), tensor(int64), tensor(float16), tensor(float), tensor(double): Constrain input to only numeric types.

  • T3 tensor(uint8), tensor(uint16), tensor(uint32), tensor(uint64), tensor(int8), tensor(int16), tensor(int32), tensor(int64), tensor(float16), tensor(float), tensor(double), tensor(string), tensor(bool), tensor(complex64), tensor(complex128): Constrain to any tensor type.

Version

Onnx name: OneHot

This version of the operator has been available since version 11.

Runtime implementation: OneHot

Or#

mlprodict.onnxrt.ops_cpu.op_or.Or (self, onnx_node, desc = None, options)

Returns the tensor resulted from performing the or logical operation elementwise on the input tensors A and B (with Numpy-style broadcasting support).

This operator supports multidirectional (i.e., Numpy-style) broadcasting; for more details please check Broadcasting in ONNX.

Inputs

  • A (heterogeneous)T: First input operand for the logical operator.

  • B (heterogeneous)T: Second input operand for the logical operator.

Outputs

  • C (heterogeneous)T1: Result tensor.

Type Constraints

  • T tensor(bool): Constrain input to boolean tensor.

  • T1 tensor(bool): Constrain output to boolean tensor.

Version

Onnx name: Or

This version of the operator has been available since version 7.

Runtime implementation: Or

PRelu#

mlprodict.onnxrt.ops_cpu.op_prelu.PRelu (self, onnx_node, desc = None, options)

PRelu takes input data (Tensor<T>) and slope tensor as input, and produces one output data (Tensor<T>) where the function f(x) = slope * x for x < 0, f(x) = x for x >= 0., is applied to the data tensor elementwise.

History - Version 16 adds bfloat16 to the types allowed. This operator supports unidirectional broadcasting (tensor slope should be unidirectional broadcastable to input tensor X); for more details please check Broadcasting in ONNX.

Inputs

  • X (heterogeneous)T: Input tensor

  • slope (heterogeneous)T: Slope tensor. The shape of slope can be smaller then first input X; if so, its shape must be unidirectional broadcastable to X

Outputs

  • Y (heterogeneous)T: Output tensor (same size as X)

Type Constraints

  • T tensor(bfloat16), tensor(float16), tensor(float), tensor(double), tensor(uint32), tensor(uint64), tensor(int32), tensor(int64): Constrain input and output types to float/int tensors.

Version

Onnx name: PRelu

This version of the operator has been available since version 16.

Runtime implementation: PRelu

Pad#

mlprodict.onnxrt.ops_cpu.op_pad.Pad (self, onnx_node, desc = None, options)

Given a tensor containing the data to be padded (data), a tensor containing the number of start and end pad values for axis (pads), (optionally) a mode, and (optionally) constant_value, a padded tensor (output) is generated.

The three supported modes are (similar to corresponding modes supported by numpy.pad):

  1. constant`(default) - pads with a given constant value as specified by `constant_value (which defaults to 0, empty string, or False)

  2. reflect - pads with the reflection of the vector mirrored on the first and last values of the vector along each axis

  3. edge - pads with the edge values of array

Example 1 (constant mode):

Insert 0 pads to the beginning of the second dimension.

data = [

[1.0, 1.2], [2.3, 3.4], [4.5, 5.7],

]

pads = [0, 2, 0, 0]

mode = ‘constant’

constant_value = 0.0

output = [

[0.0, 0.0, 1.0, 1.2], [0.0, 0.0, 2.3, 3.4], [0.0, 0.0, 4.5, 5.7],

]

Example 2 (reflect mode):

data = [

[1.0, 1.2], [2.3, 3.4], [4.5, 5.7],

]

pads = [0, 2, 0, 0]

mode = ‘reflect’

output = [

[1.0, 1.2, 1.0, 1.2], [2.3, 3.4, 2.3, 3.4], [4.5, 5.7, 4.5, 5.7],

]

Example 3 (edge mode):

data = [

[1.0, 1.2], [2.3, 3.4], [4.5, 5.7],

]

pads = [0, 2, 0, 0]

mode = ‘edge’

output = [

[1.0, 1.0, 1.0, 1.2], [2.3, 2.3, 2.3, 3.4], [4.5, 4.5, 4.5, 5.7],

]

Attributes

  • mode: Supported modes: constant`(default), `reflect, edge Default value is namemodesconstanttypeSTRING (STRING)

Inputs

Between 2 and 3 inputs.

  • data (heterogeneous)T: Input tensor.

  • pads (heterogeneous)tensor(int64): Tensor of integers indicating the number of padding elements to add or remove (if negative) at the beginning and end of each axis. For 2D input tensor, it is the number of pixels. pads should be a 1D tensor of shape [2 * input_rank]. pads format should be: [x1_begin, x2_begin,…,x1_end, x2_end,…], where xi_begin is the number of pad values added at the beginning of axis i and xi_end, the number of pad values added at the end of axis i.

  • constant_value (optional, heterogeneous)T: (Optional) A scalar value to be used if the mode chosen is constant (by default it is 0, empty string or False).

Outputs

  • output (heterogeneous)T: Tensor after padding.

Type Constraints

  • T tensor(uint8), tensor(uint16), tensor(uint32), tensor(uint64), tensor(int8), tensor(int16), tensor(int32), tensor(int64), tensor(bfloat16), tensor(float16), tensor(float), tensor(double), tensor(string), tensor(bool), tensor(complex64), tensor(complex128): Constrain input and output types to all tensor types.

Version

Onnx name: Pad

This version of the operator has been available since version 13.

Runtime implementation: Pad

Pow#

mlprodict.onnxrt.ops_cpu.op_pow.Pow (self, onnx_node, desc = None, options)

Pow takes input data (Tensor<T>) and exponent Tensor, and produces one output data (Tensor<T>) where the function f(x) = x^exponent, is applied to the data tensor elementwise. This operator supports multidirectional (i.e., Numpy-style) broadcasting; for more details please check Broadcasting in ONNX.

Inputs

  • X (heterogeneous)T: First operand, base of the exponent.

  • Y (heterogeneous)T1: Second operand, power of the exponent.

Outputs

  • Z (heterogeneous)T: Output tensor

Type Constraints

  • T tensor(int32), tensor(int64), tensor(float16), tensor(float), tensor(double), tensor(bfloat16): Constrain input X and output types to float/int tensors.

  • T1 tensor(uint8), tensor(uint16), tensor(uint32), tensor(uint64), tensor(int8), tensor(int16), tensor(int32), tensor(int64), tensor(float16), tensor(float), tensor(double), tensor(bfloat16): Constrain input Y types to float/int tensors.

Version

Onnx name: Pow

This version of the operator has been available since version 15.

Runtime implementation: Pow

QLinearConv#

mlprodict.onnxrt.ops_cpu.op_qlinear_conv.QLinearConv (self, onnx_node, desc = None, options)

The convolution operator consumes a quantized input tensor, its scale and zero point, a quantized filter, its scale and zero point, and output’s scale and zero point, and computes the quantized output. Each scale and zero-point pair must have same shape. It means they must be either scalars (per tensor) or 1-D tensors (per output channel). Each input or output and its related zero point must have same type. When bias is present it must be quantized using scale = input scale * weight scale and zero point as 0.

Attributes

  • auto_pad: auto_pad must be either NOTSET, SAME_UPPER, SAME_LOWER or VALID. Where default value is NOTSET, which means explicit padding is used. SAME_UPPER or SAME_LOWER mean pad the input so that output_shape[i] = ceil(input_shape[i] / strides[i]) for each axis i. The padding is split between the two sides equally or almost equally (depending on whether it is even or odd). In case the padding is an odd number, the extra padding is added at the end for SAME_UPPER and at the beginning for SAME_LOWER. Default value is nameautopadsNOTSETtypeSTRING (STRING)

  • dilations: dilation value along each spatial axis of the filter. If not present, the dilation defaults to 1 along each spatial axis. default value cannot be automatically retrieved (INTS)

  • group: number of groups input channels and output channels are divided into. default is 1. Default value is namegroupi1typeINT (INT)

  • kernel_shape: The shape of the convolution kernel. If not present, should be inferred from input ‘w’. default value cannot be automatically retrieved (INTS)

  • pads: Padding for the beginning and ending along each spatial axis, it can take any value greater than or equal to 0.The value represent the number of pixels added to the beginning and end part of the corresponding axis.`pads` format should be as follow [x1_begin, x2_begin…x1_end, x2_end,…], where xi_begin the number ofpixels added at the beginning of axis i and xi_end, the number of pixels added at the end of axis i.This attribute cannot be used simultaneously with auto_pad attribute. If not present, the padding defaultsto 0 along start and end of each spatial axis. default value cannot be automatically retrieved (INTS)

  • strides: Stride along each spatial axis. If not present, the stride defaults to 1 along each spatial axis. default value cannot be automatically retrieved (INTS)

Inputs

Between 8 and 9 inputs.

  • x (heterogeneous)T1: Input data tensor from previous layer; has size (N x C x H x W), where N is the batch size, C is the number of channels, and H and W are the height and width. Note that this is for the 2D image. Otherwise the size is (N x C x D1 x D2 … x Dn). Optionally, if dimension denotation is in effect, the operation expects input data tensor to arrive with the dimension denotation of [DATA_BATCH, DATA_CHANNEL, DATA_FEATURE, DATA_FEATURE …].

  • x_scale (heterogeneous)tensor(float): Scale tensor for input ‘x’. It’s a scalar, which means a per-tensor/layer quantization.

  • x_zero_point (heterogeneous)T1: Zero point tensor for input ‘x’. It’s a scalar, which means a per-tensor/layer quantization.

  • w (heterogeneous)T2: The weight tensor that will be used in the convolutions; has size (M x C/group x kH x kW), where C is the number of channels, and kH and kW are the height and width of the kernel, and M is the number of feature maps. For more than 2 dimensions, the kernel shape will be (M x C/group x k1 x k2 x … x kn), where (k1 x k2 x … kn) is the dimension of the kernel. Optionally, if dimension denotation is in effect, the operation expects the weight tensor to arrive with the dimension denotation of [FILTER_OUT_CHANNEL, FILTER_IN_CHANNEL, FILTER_SPATIAL, FILTER_SPATIAL …]. X.shape[1] == (W.shape[1] * group) == C (assuming zero based indices for the shape array). Or in other words FILTER_IN_CHANNEL should be equal to DATA_CHANNEL.

  • w_scale (heterogeneous)tensor(float): Scale tensor for input ‘w’. It could be a scalar or a 1-D tensor, which means a per-tensor/layer or per output channel quantization. If it’s a 1-D tensor, its number of elements should be equal to the number of output channels (M).

  • w_zero_point (heterogeneous)T2: Zero point tensor for input ‘w’. It could be a scalar or a 1-D tensor, which means a per-tensor/layer or per output channel quantization. If it’s a 1-D tensor, its number of elements should be equal to the number of output channels (M).

  • y_scale (heterogeneous)tensor(float): Scale tensor for output ‘y’. It’s a scalar, which means a per-tensor/layer quantization.

  • y_zero_point (heterogeneous)T3: Zero point tensor for output ‘y’. It’s a scalar, which means a per-tensor/layer quantization.

  • B (optional, heterogeneous)T4: Optional 1D bias to be added to the convolution, has size of M. Bias must be quantized using scale = x_scale * w_scale and zero_point = 0

Outputs

  • y (heterogeneous)T3: Output data tensor that contains the result of the convolution. The output dimensions are functions of the kernel size, stride size, and pad lengths.

Type Constraints

  • T1 tensor(int8), tensor(uint8): Constrain input type to 8-bit integer tensor.

  • T2 tensor(int8), tensor(uint8): Constrain filter type to 8-bit integer tensor.

  • T3 tensor(int8), tensor(uint8): Constrain output type to 8-bit integer tensor.

  • T4 tensor(int32): Constrain bias type to 32-bit integer tensor.

Version

Onnx name: QLinearConv

This version of the operator has been available since version 10.

Runtime implementation: QLinearConv

QuantizeLinear#

mlprodict.onnxrt.ops_cpu.op_quantize_linear.QuantizeLinear (self, onnx_node, desc = None, options)

The linear quantization operator. It consumes a high precision tensor, a scale, and a zero point to compute the low precision / quantized tensor. The scale factor and zero point must have same shape, and can be either a scalar for per-tensor / per layer quantization, or a 1-D tensor for per-axis quantization. The quantization formula is y = saturate ((x / y_scale) + y_zero_point). For saturation, it saturates to [0, 255] if it’s uint8, or [-128, 127] if it’s int8. For (x / y_scale), it’s rounding to nearest ties to even. Refer to https://en.wikipedia.org/wiki/Rounding for details. ‘y_zero_point’ and ‘y’ must have same type.

Attributes

  • axis: (Optional) The axis of the quantization dimension of the input tensor. Ignored for per-tensor quantization. Negative value means counting dimensions from the back. Accepted range is [-r, r-1] where r = rank(input). Default value is nameaxisi1typeINT (INT)

Inputs

Between 2 and 3 inputs.

  • x (heterogeneous)T1: N-D full precision Input tensor to be quantized.

  • y_scale (heterogeneous)tensor(float): Scale for doing quantization to get ‘y’. It can be a scalar, which means per-tensor/layer quantization, or a 1-D Tensor for per-axis quantization.

  • y_zero_point (optional, heterogeneous)T2: Zero point for doing quantization to get ‘y’. Shape must match y_scale. Default is uint8 with zero point of 0 if it’s not specified.

Outputs

  • y (heterogeneous)T2: N-D quantized output tensor. It has same shape as input ‘x’.

Type Constraints

  • T1 tensor(float), tensor(int32): Constrain ‘x’ to float or int32 tensor.

  • T2 tensor(int8), tensor(uint8): Constrain ‘y_zero_point’ and ‘y’ to 8-bit integer tensor.

Version

Onnx name: QuantizeLinear

This version of the operator has been available since version 13.

Runtime implementation: QuantizeLinear

RFFT#

mlprodict.onnxrt.ops_cpu.op_rfft.RFFT (self, onnx_node, desc = None, options)

Version

Onnx name: RFFT

This version of the operator has been available since version of domain mlprodict.

Runtime implementation: RFFT

RNN_14#

mlprodict.onnxrt.ops_cpu.op_rnn.RNN_14 (self, onnx_node, desc = None, options)

Computes an one-layer simple RNN. This operator is usually supported via some custom implementation such as CuDNN.

Notations:

X - input tensor

i - input gate

t - time step (t-1 means previous time step)

Wi - W parameter weight matrix for input gate

Ri - R recurrence weight matrix for input gate

Wbi - W parameter bias vector for input gate

Rbi - R parameter bias vector for input gate

WBi - W parameter weight matrix for backward input gate

RBi - R recurrence weight matrix for backward input gate

WBbi - WR bias vectors for backward input gate

RBbi - RR bias vectors for backward input gate

H - Hidden state

num_directions - 2 if direction == bidirectional else 1

Activation functions:

Relu(x) - max(0, x)

Tanh(x) - (1 - e^{-2x})/(1 + e^{-2x})

Sigmoid(x) - 1/(1 + e^{-x})

(NOTE: Below are optional)

Affine(x) - alpha*x + beta

LeakyRelu(x) - x if x >= 0 else alpha * x

ThresholdedRelu(x) - x if x >= alpha else 0

ScaledTanh(x) - alpha*Tanh(beta*x)

HardSigmoid(x) - min(max(alpha*x + beta, 0), 1)

Elu(x) - x if x >= 0 else alpha*(e^x - 1)

Softsign(x) - x/(1 + |x|)

Softplus(x) - log(1 + e^x)

Equations (Default: f=Tanh):

  • Ht = f(Xt*(Wi^T) + Ht-1*(Ri^T) + Wbi + Rbi)

This operator has optional inputs/outputs. See ONNX for more details about the representation of optional arguments. An empty string may be used in the place of an actual argument’s name to indicate a missing argument. Trailing optional arguments (those not followed by an argument that is present) may also be simply omitted.

Attributes

  • activation_alpha: Optional scaling values used by some activation functions. The values are consumed in the order of activation functions, for example (f, g, h) in LSTM. Default values are the same as of corresponding ONNX operators.For example with LeakyRelu, the default alpha is 0.01. default value cannot be automatically retrieved (FLOATS)

  • activation_beta: Optional scaling values used by some activation functions. The values are consumed in the order of activation functions, for example (f, g, h) in LSTM. Default values are the same as of corresponding ONNX operators. default value cannot be automatically retrieved (FLOATS)

  • activations: One (or two if bidirectional) activation function for input gate. The activation function must be one of the activation functions specified above. Optional: Default Tanh if not specified. Default value is nameactivationsstringsTanhstringsTanhtypeSTRINGS (STRINGS)

  • clip: Cell clip threshold. Clipping bounds the elements of a tensor in the range of [-threshold, +threshold] and is applied to the input of activations. No clip if not specified. default value cannot be automatically retrieved (FLOAT)

  • direction: Specify if the RNN is forward, reverse, or bidirectional. Must be one of forward (default), reverse, or bidirectional. Default value is namedirectionsforwardtypeSTRING (STRING)

  • hidden_size: Number of neurons in the hidden layer default value cannot be automatically retrieved (INT)

  • layout: The shape format of inputs X, initial_h and outputs Y, Y_h. If 0, the following shapes are expected: X.shape = [seq_length, batch_size, input_size], Y.shape = [seq_length, num_directions, batch_size, hidden_size], initial_h.shape = Y_h.shape = [num_directions, batch_size, hidden_size]. If 1, the following shapes are expected: X.shape = [batch_size, seq_length, input_size], Y.shape = [batch_size, seq_length, num_directions, hidden_size], initial_h.shape = Y_h.shape = [batch_size, num_directions, hidden_size]. Default value is namelayouti0typeINT (INT)

Inputs

Between 3 and 6 inputs.

  • X (heterogeneous)T: The input sequences packed (and potentially padded) into one 3-D tensor with the shape of [seq_length, batch_size, input_size].

  • W (heterogeneous)T: The weight tensor for input gate. Concatenation of Wi and WBi (if bidirectional). The tensor has shape [num_directions, hidden_size, input_size].

  • R (heterogeneous)T: The recurrence weight tensor. Concatenation of Ri and RBi (if bidirectional). The tensor has shape [num_directions, hidden_size, hidden_size].

  • B (optional, heterogeneous)T: The bias tensor for input gate. Concatenation of [Wbi, Rbi] and [WBbi, RBbi] (if bidirectional). The tensor has shape [num_directions, 2*hidden_size]. Optional: If not specified - assumed to be 0.

  • sequence_lens (optional, heterogeneous)T1: Optional tensor specifying lengths of the sequences in a batch. If not specified - assumed all sequences in the batch to have length seq_length. It has shape [batch_size].

  • initial_h (optional, heterogeneous)T: Optional initial value of the hidden. If not specified - assumed to be 0. It has shape [num_directions, batch_size, hidden_size].

Outputs

Between 0 and 2 outputs.

  • Y (optional, heterogeneous)T: A tensor that concats all the intermediate output values of the hidden. It has shape [seq_length, num_directions, batch_size, hidden_size].

  • Y_h (optional, heterogeneous)T: The last output value of the hidden. It has shape [num_directions, batch_size, hidden_size].

Type Constraints

  • T tensor(float16), tensor(float), tensor(double): Constrain input and output types to float tensors.

  • T1 tensor(int32): Constrain seq_lens to integer tensor.

Version

Onnx name: RNN

This version of the operator has been available since version 14.

Runtime implementation: RNN

RandomNormal#

mlprodict.onnxrt.ops_cpu.op_random.RandomNormal (self, onnx_node, desc = None, options)

Generate a tensor with random values drawn from a normal distribution. The shape of the tensor is specified by the shape argument and the parameter of the normal distribution specified by mean and scale.

The data type is specified by the ‘dtype’ argument. The ‘dtype’ argument must be one of the data types specified in the ‘DataType’ enum field in the TensorProto message.

Attributes

  • dtype: The data type for the elements of the output tensor. Default is TensorProto::FLOAT. Default value is namedtypei1typeINT (INT)

  • mean: The mean of the normal distribution. Default value is namemeanf0.0typeFLOAT (FLOAT)

  • scale: The standard deviation of the normal distribution. Default value is namescalef1.0typeFLOAT (FLOAT)

  • seed: (Optional) Seed to the random generator, if not specified we will auto generate one. default value cannot be automatically retrieved (FLOAT)

  • shape (required): The shape of the output tensor. default value cannot be automatically retrieved (INTS)

Outputs

  • output (heterogeneous)T: Output tensor of random values drawn from normal distribution

Type Constraints

  • T tensor(float16), tensor(float), tensor(double): Constrain output types to float tensors.

Version

Onnx name: RandomNormal

This version of the operator has been available since version 1.

Runtime implementation: RandomNormal

RandomNormalLike#

mlprodict.onnxrt.ops_cpu.op_random.RandomNormalLike (self, onnx_node, desc = None, options)

Generate a tensor with random values drawn from a normal distribution. The shape of the output tensor is copied from the shape of the input tensor, and the parameters of the normal distribution are specified by mean and scale.

The data type is specified by the ‘dtype’ argument, or copied from the input tensor if not provided. The ‘dtype’ argument must be one of the data types specified in the ‘DataType’ enum field in the TensorProto message, and be valid as an output type.

Attributes

  • dtype: (Optional) The data type for the elements of the output tensor, if not specified, we will use the data type of the input tensor. default value cannot be automatically retrieved (INT)

  • mean: The mean of the normal distribution. Default value is namemeanf0.0typeFLOAT (FLOAT)

  • scale: The standard deviation of the normal distribution. Default value is namescalef1.0typeFLOAT (FLOAT)

  • seed: (Optional) Seed to the random generator, if not specified we will auto generate one. default value cannot be automatically retrieved (FLOAT)

Inputs

  • input (heterogeneous)T1: Input tensor to copy shape and optionally type information from.

Outputs

  • output (heterogeneous)T2: Output tensor of random values drawn from normal distribution

Type Constraints

  • T1 tensor(uint8), tensor(uint16), tensor(uint32), tensor(uint64), tensor(int8), tensor(int16), tensor(int32), tensor(int64), tensor(float16), tensor(float), tensor(double), tensor(string), tensor(bool), tensor(complex64), tensor(complex128): Constrain to any tensor type. If the dtype attribute is not provided this must be a valid output type.

  • T2 tensor(float16), tensor(float), tensor(double): Constrain output types to float tensors.

Version

Onnx name: RandomNormalLike

This version of the operator has been available since version 1.

Runtime implementation: RandomNormalLike

RandomUniform#

mlprodict.onnxrt.ops_cpu.op_random.RandomUniform (self, onnx_node, desc = None, options)

Generate a tensor with random values drawn from a uniform distribution. The shape of the tensor is specified by the shape argument and the range by low and high.

The data type is specified by the ‘dtype’ argument. The ‘dtype’ argument must be one of the data types specified in the ‘DataType’ enum field in the TensorProto message.

Attributes

  • dtype: The data type for the elements of the output tensor. If not specified, default is TensorProto::FLOAT. Default value is namedtypei1typeINT (INT)

  • high: Upper boundary of the output values. Default value is namehighf1.0typeFLOAT (FLOAT)

  • low: Lower boundary of the output values. Default value is namelowf0.0typeFLOAT (FLOAT)

  • seed: (Optional) Seed to the random generator, if not specified we will auto generate one. default value cannot be automatically retrieved (FLOAT)

  • shape (required): The shape of the output tensor. default value cannot be automatically retrieved (INTS)

Outputs

  • output (heterogeneous)T: Output tensor of random values drawn from uniform distribution

Type Constraints

  • T tensor(float16), tensor(float), tensor(double): Constrain output types to float tensors.

Version

Onnx name: RandomUniform

This version of the operator has been available since version 1.

Runtime implementation: RandomUniform

RandomUniformLike#

mlprodict.onnxrt.ops_cpu.op_random.RandomUniformLike (self, onnx_node, desc = None, options)

Generate a tensor with random values drawn from a uniform distribution. The shape of the output tensor is copied from the shape of the input tensor, and the parameters of the uniform distribution are specified by low and high.

The data type is specified by the ‘dtype’ argument, or copied from the input tensor if not provided. The ‘dtype’ argument must be one of the data types specified in the ‘DataType’ enum field in the TensorProto message and be valid as an output type.

Attributes

  • dtype: (Optional) The data type for the elements of the output tensor, if not specified, we will use the data type of the input tensor. default value cannot be automatically retrieved (INT)

  • high: Upper boundary of the output values. Default value is namehighf1.0typeFLOAT (FLOAT)

  • low: Lower boundary of the output values. Default value is namelowf0.0typeFLOAT (FLOAT)

  • seed: (Optional) Seed to the random generator, if not specified we will auto generate one. default value cannot be automatically retrieved (FLOAT)

Inputs

  • input (heterogeneous)T1: Input tensor to copy shape and optionally type information from.

Outputs

  • output (heterogeneous)T2: Output tensor of random values drawn from uniform distribution

Type Constraints

  • T1 tensor(uint8), tensor(uint16), tensor(uint32), tensor(uint64), tensor(int8), tensor(int16), tensor(int32), tensor(int64), tensor(float16), tensor(float), tensor(double), tensor(string), tensor(bool), tensor(complex64), tensor(complex128): Constrain to any tensor type. If the dtype attribute is not provided this must be a valid output type.

  • T2 tensor(float16), tensor(float), tensor(double): Constrain output types to float tensors.

Version

Onnx name: RandomUniformLike

This version of the operator has been available since version 1.

Runtime implementation: RandomUniformLike

Range#

mlprodict.onnxrt.ops_cpu.op_range.Range (self, onnx_node, desc = None, options)

Generate a tensor containing a sequence of numbers that begin at start and extends by increments of delta up to limit (exclusive).

The number of elements in the output of range is computed as below-

number_of_elements = max( ceil( (limit - start) / delta ) , 0 )

The pseudocode determining the contents of the output is shown below-

for(int i=0; i<number_of_elements; ++i)

{

` output[i] = start + (i * delta); `

}

Example 1 Inputs: start = 3, limit = 9, delta = 3 Output: [3, 6]

Example 2 Inputs: start = 10, limit = 4, delta = -2 Output: [10, 8, 6]

Inputs

  • start (heterogeneous)T: Scalar. First entry for the range of output values.

  • limit (heterogeneous)T: Scalar. Exclusive upper limit for the range of output values.

  • delta (heterogeneous)T: Scalar. Value to step by.

Outputs

  • output (heterogeneous)T: A 1-D tensor with same type as the inputs containing generated range of values.

Type Constraints

  • T tensor(float), tensor(double), tensor(int16), tensor(int32), tensor(int64): Constrain input types to common numeric type tensors.

Version

Onnx name: Range

This version of the operator has been available since version 11.

Runtime implementation: Range

Reciprocal#

mlprodict.onnxrt.ops_cpu.op_reciprocal.Reciprocal (self, onnx_node, desc = None, options)

Reciprocal takes one input data (Tensor<T>) and produces one output data (Tensor<T>) where the reciprocal is, y = 1/x, is applied to the tensor elementwise.

Inputs

  • X (heterogeneous)T: Input tensor

Outputs

  • Y (heterogeneous)T: Output tensor

Type Constraints

  • T tensor(float16), tensor(float), tensor(double), tensor(bfloat16): Constrain input and output types to float tensors.

Version

Onnx name: Reciprocal

This version of the operator has been available since version 13.

Runtime implementation: Reciprocal

ReduceL1#

mlprodict.onnxrt.ops_cpu.op_reduce_l1.ReduceL1 (self, onnx_node, desc = None, options)

Computes the L1 norm of the input tensor’s element along the provided axes. The resulting tensor has the same rank as the input if keepdims equals 1. If keepdims equals 0, then the resulting tensor has the reduced dimension pruned.

The above behavior is similar to numpy, with the exception that numpy defaults keepdims to False instead of True.

Attributes

  • axes: A list of integers, along which to reduce. The default is to reduce over all the dimensions of the input tensor. Accepted range is [-r, r-1] where r = rank(data). default value cannot be automatically retrieved (INTS)

  • keepdims: Keep the reduced dimension or not, default 1 means keep reduced dimension. Default value is namekeepdimsi1typeINT (INT)

Inputs

  • data (heterogeneous)T: An input tensor.

Outputs

  • reduced (heterogeneous)T: Reduced output tensor.

Type Constraints

  • T tensor(uint32), tensor(uint64), tensor(int32), tensor(int64), tensor(float16), tensor(float), tensor(double), tensor(bfloat16): Constrain input and output types to high-precision numeric tensors.

Version

Onnx name: ReduceL1

This version of the operator has been available since version 13.

Runtime implementation: ReduceL1

ReduceL2#

mlprodict.onnxrt.ops_cpu.op_reduce_l2.ReduceL2 (self, onnx_node, desc = None, options)

Computes the L2 norm of the input tensor’s element along the provided axes. The resulting tensor has the same rank as the input if keepdims equals 1. If keepdims equals 0, then the resulting tensor has the reduced dimension pruned.

The above behavior is similar to numpy, with the exception that numpy defaults keepdims to False instead of True.

Attributes

  • axes: A list of integers, along which to reduce. The default is to reduce over all the dimensions of the input tensor. Accepted range is [-r, r-1] where r = rank(data). default value cannot be automatically retrieved (INTS)

  • keepdims: Keep the reduced dimension or not, default 1 means keep reduced dimension. Default value is namekeepdimsi1typeINT (INT)

Inputs

  • data (heterogeneous)T: An input tensor.

Outputs

  • reduced (heterogeneous)T: Reduced output tensor.

Type Constraints

  • T tensor(uint32), tensor(uint64), tensor(int32), tensor(int64), tensor(float16), tensor(float), tensor(double), tensor(bfloat16): Constrain input and output types to high-precision numeric tensors.

Version

Onnx name: ReduceL2

This version of the operator has been available since version 13.

Runtime implementation: ReduceL2

ReduceLogSum#

mlprodict.onnxrt.ops_cpu.op_reduce_log_sum.ReduceLogSum (self, onnx_node, desc = None, options)

Computes the log sum of the input tensor’s element along the provided axes. The resulting tensor has the same rank as the input if keepdims equals 1. If keepdims equals 0, then the resulting tensor has the reduced dimension pruned.

The above behavior is similar to numpy, with the exception that numpy defaults keepdims to False instead of True.

Attributes

  • axes: A list of integers, along which to reduce. The default is to reduce over all the dimensions of the input tensor. Accepted range is [-r, r-1] where r = rank(data). default value cannot be automatically retrieved (INTS)

  • keepdims: Keep the reduced dimension or not, default 1 means keep reduced dimension. Default value is namekeepdimsi1typeINT (INT)

Inputs

  • data (heterogeneous)T: An input tensor.

Outputs

  • reduced (heterogeneous)T: Reduced output tensor.

Type Constraints

  • T tensor(uint32), tensor(uint64), tensor(int32), tensor(int64), tensor(float16), tensor(float), tensor(double), tensor(bfloat16): Constrain input and output types to high-precision numeric tensors.

Version

Onnx name: ReduceLogSum

This version of the operator has been available since version 13.

Runtime implementation: ReduceLogSum

ReduceLogSumExp#

mlprodict.onnxrt.ops_cpu.op_reduce_log_sum_exp.ReduceLogSumExp (self, onnx_node, desc = None, options)

Computes the log sum exponent of the input tensor’s element along the provided axes. The resulting tensor has the same rank as the input if keepdims equals 1. If keepdims equals 0, then the resulting tensor has the reduced dimension pruned.

The above behavior is similar to numpy, with the exception that numpy defaults keepdims to False instead of True.

Attributes

  • axes: A list of integers, along which to reduce. The default is to reduce over all the dimensions of the input tensor. Accepted range is [-r, r-1] where r = rank(data). default value cannot be automatically retrieved (INTS)

  • keepdims: Keep the reduced dimension or not, default 1 means keep reduced dimension. Default value is namekeepdimsi1typeINT (INT)

Inputs

  • data (heterogeneous)T: An input tensor.

Outputs

  • reduced (heterogeneous)T: Reduced output tensor.

Type Constraints

  • T tensor(uint32), tensor(uint64), tensor(int32), tensor(int64), tensor(float16), tensor(float), tensor(double), tensor(bfloat16): Constrain input and output types to high-precision numeric tensors.

Version

Onnx name: ReduceLogSumExp

This version of the operator has been available since version 13.

Runtime implementation: ReduceLogSumExp

ReduceMax#

mlprodict.onnxrt.ops_cpu.op_reduce_max.ReduceMax (self, onnx_node, desc = None, options)

Computes the max of the input tensor’s element along the provided axes. The resulting tensor has the same rank as the input if keepdims equals 1. If keepdims equals 0, then the resulting tensor has the reduced dimension pruned.

The above behavior is similar to numpy, with the exception that numpy defaults keepdims to False instead of True.

Attributes

  • axes: A list of integers, along which to reduce. The default is to reduce over all the dimensions of the input tensor. Accepted range is [-r, r-1] where r = rank(data). default value cannot be automatically retrieved (INTS)

  • keepdims: Keep the reduced dimension or not, default 1 means keep reduced dimension. Default value is namekeepdimsi1typeINT (INT)

Inputs

  • data (heterogeneous)T: An input tensor.

Outputs

  • reduced (heterogeneous)T: Reduced output tensor.

Type Constraints

  • T tensor(uint32), tensor(uint64), tensor(int32), tensor(int64), tensor(float16), tensor(float), tensor(double), tensor(bfloat16), tensor(uint8), tensor(int8): Constrain input and output types to high-precision and 8 bit numeric tensors.

Version

Onnx name: ReduceMax

This version of the operator has been available since version 13.

Runtime implementation: ReduceMax

ReduceMean#

mlprodict.onnxrt.ops_cpu.op_reduce_mean.ReduceMean (self, onnx_node, desc = None, options)

Computes the mean of the input tensor’s element along the provided axes. The resulting tensor has the same rank as the input if keepdims equals 1. If keepdims equals 0, then the resulting tensor has the reduced dimension pruned.

The above behavior is similar to numpy, with the exception that numpy defaults keepdims to False instead of True.

Attributes

  • axes: A list of integers, along which to reduce. The default is to reduce over all the dimensions of the input tensor. Accepted range is [-r, r-1] where r = rank(data). default value cannot be automatically retrieved (INTS)

  • keepdims: Keep the reduced dimension or not, default 1 means keep reduced dimension. Default value is namekeepdimsi1typeINT (INT)

Inputs

  • data (heterogeneous)T: An input tensor.

Outputs

  • reduced (heterogeneous)T: Reduced output tensor.

Type Constraints

  • T tensor(uint32), tensor(uint64), tensor(int32), tensor(int64), tensor(float16), tensor(float), tensor(double), tensor(bfloat16): Constrain input and output types to high-precision numeric tensors.

Version

Onnx name: ReduceMean

This version of the operator has been available since version 13.

Runtime implementation: ReduceMean

ReduceMin#

mlprodict.onnxrt.ops_cpu.op_reduce_min.ReduceMin (self, onnx_node, desc = None, options)

Computes the min of the input tensor’s element along the provided axes. The resulting tensor has the same rank as the input if keepdims equals 1. If keepdims equals 0, then the resulting tensor has the reduced dimension pruned.

The above behavior is similar to numpy, with the exception that numpy defaults keepdims to False instead of True.

Attributes

  • axes: A list of integers, along which to reduce. The default is to reduce over all the dimensions of the input tensor. Accepted range is [-r, r-1] where r = rank(data). default value cannot be automatically retrieved (INTS)

  • keepdims: Keep the reduced dimension or not, default 1 means keep reduced dimension. Default value is namekeepdimsi1typeINT (INT)

Inputs

  • data (heterogeneous)T: An input tensor.

Outputs

  • reduced (heterogeneous)T: Reduced output tensor.

Type Constraints

  • T tensor(uint32), tensor(uint64), tensor(int32), tensor(int64), tensor(float16), tensor(float), tensor(double), tensor(bfloat16), tensor(uint8), tensor(int8): Constrain input and output types to high-precision and 8 bit numeric tensors.

Version

Onnx name: ReduceMin

This version of the operator has been available since version 13.

Runtime implementation: ReduceMin

ReduceProd#

mlprodict.onnxrt.ops_cpu.op_reduce_prod.ReduceProd (self, onnx_node, desc = None, options)

Computes the product of the input tensor’s element along the provided axes. The resulting tensor has the same rank as the input if keepdims equals 1. If keepdims equals 0, then the resulting tensor has the reduced dimension pruned.

The above behavior is similar to numpy, with the exception that numpy defaults keepdims to False instead of True.

Attributes

  • axes: A list of integers, along which to reduce. The default is to reduce over all the dimensions of the input tensor. Accepted range is [-r, r-1] where r = rank(data). default value cannot be automatically retrieved (INTS)

  • keepdims: Keep the reduced dimension or not, default 1 means keep reduced dimension. Default value is namekeepdimsi1typeINT (INT)

Inputs

  • data (heterogeneous)T: An input tensor.

Outputs

  • reduced (heterogeneous)T: Reduced output tensor.

Type Constraints

  • T tensor(uint32), tensor(uint64), tensor(int32), tensor(int64), tensor(float16), tensor(float), tensor(double), tensor(bfloat16): Constrain input and output types to high-precision numeric tensors.

Version

Onnx name: ReduceProd

This version of the operator has been available since version 13.

Runtime implementation: ReduceProd

ReduceSumSquare#

mlprodict.onnxrt.ops_cpu.op_reduce_sum_square.ReduceSumSquare (self, onnx_node, desc = None, options)

Computes the sum square of the input tensor’s element along the provided axes. The resulting tensor has the same rank as the input if keepdims equals 1. If keepdims equals 0, then the resulting tensor has the reduced dimension pruned.

The above behavior is similar to numpy, with the exception that numpy defaults keepdims to False instead of True.

Attributes

  • axes: A list of integers, along which to reduce. The default is to reduce over all the dimensions of the input tensor. Accepted range is [-r, r-1] where r = rank(data). default value cannot be automatically retrieved (INTS)

  • keepdims: Keep the reduced dimension or not, default 1 means keep reduced dimension. Default value is namekeepdimsi1typeINT (INT)

Inputs

  • data (heterogeneous)T: An input tensor.

Outputs

  • reduced (heterogeneous)T: Reduced output tensor.

Type Constraints

  • T tensor(uint32), tensor(uint64), tensor(int32), tensor(int64), tensor(float16), tensor(float), tensor(double), tensor(bfloat16): Constrain input and output types to high-precision numeric tensors.

Version

Onnx name: ReduceSumSquare

This version of the operator has been available since version 13.

Runtime implementation: ReduceSumSquare

ReduceSum_13#

mlprodict.onnxrt.ops_cpu.op_reduce_sum.ReduceSum_13 (self, onnx_node, desc = None, options)

Computes the sum of the input tensor’s element along the provided axes. The resulting tensor has the same rank as the input if keepdims equals 1. If keepdims equals 0, then the resulting tensor has the reduced dimension pruned.

The above behavior is similar to numpy, with the exception that numpy defaults keepdims to False instead of True.

Attributes

  • keepdims: Keep the reduced dimension or not, default 1 means keep reduced dimension. Default value is namekeepdimsi1typeINT (INT)

  • noop_with_empty_axes: Defines behaviour if ‘axes’ is empty. Default behaviour with ‘false’ is to reduce all axes. When axes is empty and this attribute is set to true, input tensor will not be reduced,and the output tensor would be equivalent to input tensor. Default value is namenoopwithemptyaxesi0typeINT (INT)

Inputs

Between 1 and 2 inputs.

  • data (heterogeneous)T: An input tensor.

  • axes (optional, heterogeneous)tensor(int64): Optional input list of integers, along which to reduce. The default is to reduce over all the dimensions of the input tensor if ‘noop_with_empty_axes’ is false, else act as an Identity op when ‘noop_with_empty_axes’ is true. Accepted range is [-r, r-1] where r = rank(data).

Outputs

  • reduced (heterogeneous)T: Reduced output tensor.

Type Constraints

  • T tensor(uint32), tensor(uint64), tensor(int32), tensor(int64), tensor(float16), tensor(float), tensor(double), tensor(bfloat16): Constrain input and output types to high-precision numeric tensors.

Version

Onnx name: ReduceSum

This version of the operator has been available since version 13.

Runtime implementation: ReduceSum

Relu#

mlprodict.onnxrt.ops_cpu.op_relu.Relu (self, onnx_node, desc = None, options)

Relu takes one input data (Tensor<T>) and produces one output data (Tensor<T>) where the rectified linear function, y = max(0, x), is applied to the tensor elementwise.

Inputs

  • X (heterogeneous)T: Input tensor

Outputs

  • Y (heterogeneous)T: Output tensor

Type Constraints

  • T tensor(float), tensor(int32), tensor(int8), tensor(int16), tensor(int64), tensor(float16), tensor(double), tensor(bfloat16): Constrain input and output types to signed numeric tensors.

Version

Onnx name: Relu

This version of the operator has been available since version 14.

Runtime implementation: Relu

Reshape_14#

mlprodict.onnxrt.ops_cpu.op_reshape.Reshape_14 (self, onnx_node, desc = None, options)

Reshape the input tensor similar to numpy.reshape. First input is the data tensor, second input is a shape tensor which specifies the output shape. It outputs the reshaped tensor. At most one dimension of the new shape can be -1. In this case, the value is inferred from the size of the tensor and the remaining dimensions. A dimension could also be 0, in which case the actual dimension value is unchanged (i.e. taken from the input tensor). If ‘allowzero’ is set, and the new shape includes 0, the dimension will be set explicitly to zero (i.e. not taken from input tensor). Shape (second input) could be an empty shape, which means converting to a scalar. The input tensor’s shape and the output tensor’s shape are required to have the same number of elements.

Attributes

  • allowzero: (Optional) By default, when any value in the ‘shape’ input is equal to zero the corresponding dimension value is copied from the input tensor dynamically. allowzero=1 indicates that if any value in the ‘shape’ input is set to zero, the zero value is honored, similar to NumPy. Default value is nameallowzeroi0typeINT (INT)

Inputs

  • data (heterogeneous)T: An input tensor.

  • shape (heterogeneous)tensor(int64): Specified shape for output.

Outputs

  • reshaped (heterogeneous)T: Reshaped data.

Type Constraints

  • T tensor(uint8), tensor(uint16), tensor(uint32), tensor(uint64), tensor(int8), tensor(int16), tensor(int32), tensor(int64), tensor(bfloat16), tensor(float16), tensor(float), tensor(double), tensor(string), tensor(bool), tensor(complex64), tensor(complex128): Constrain input and output types to all tensor types.

Version

Onnx name: Reshape

This version of the operator has been available since version 14.

Runtime implementation: Reshape

Resize#

mlprodict.onnxrt.ops_cpu.op_resize.Resize (self, onnx_node, desc = None, expected_attributes = None, options)

Resize the input tensor. In general, it calculates every value in the output tensor as a weighted average of neighborhood (a.k.a. sampling locations) in the input tensor. Each dimension value of the output tensor is:

output_dimension = floor(input_dimension * (roi_end - roi_start) * scale) if input "sizes" is not specified.

Attributes

  • coordinate_transformation_mode:

This attribute describes how to transform the coordinate in the resized tensor to the coordinate in the original tensor. <br/>

The coordinate of each dimension is transformed individually. Let’s describe a case using axis x as an example. Denote x_resized as the coordinate of axis x in the resized tensor, x_original as the coordinate of axis x in the original tensor, length_original as the length of the original tensor in axis x, length_resized as the length of the resized tensor in axis x, roi_x = (start_x, end_x) of the axis x in input “roi”, scale = length_resized / length_original, <br/>

if coordinate_transformation_mode is “half_pixel”, <br/> x_original = (x_resized + 0.5) / scale - 0.5, <br/>

if coordinate_transformation_mode is “pytorch_half_pixel”, <br/> x_original = length_resized > 1 ? (x_resized + 0.5) / scale - 0.5 : 0, <br/>

if coordinate_transformation_mode is “align_corners”, <br/> x_original = x_resized * (length_original - 1) / (length_resized - 1), <br/>

if coordinate_transformation_mode is “asymmetric”, <br/> x_original = x_resized / scale, <br/>

if coordinate_transformation_mode is “tf_crop_and_resize”, <br/> x_original = length_resized > 1 ? start_x * (length_original - 1) + x_resized * (end_x - start_x) * (length_original - 1) / (length_resized - 1) : 0.5 * (start_x + end_x) * (length_original - 1). Default value is namecoordinatetransformationmodeshalfpixeltypeSTRING (STRING) * cubic_coeff_a: The coefficient ‘a’ used in cubic interpolation. Two common choice are -0.5 (in some cases of TensorFlow) and -0.75 (in PyTorch). Check out Equation (4) in https://ieeexplore.ieee.org/document/1163711 for the details. This attribute is valid only if “mode” is “cubic”. Default value is namecubiccoeffaf-0.75typeFLOAT (FLOAT) * exclude_outside: If set to 1, the weight of sampling locations outside the tensor will be set to 0 and the weight will be renormalized so that their sum is 1.0. The default value is 0. Default value is nameexcludeoutsidei0typeINT (INT) * extrapolation_value: When coordinate_transformation_mode is “tf_crop_and_resize” and x_original is outside the range [0, length_original - 1], this value is used as the corresponding output value. Default is 0.0f. Default value is nameextrapolationvaluef0.0typeFLOAT (FLOAT) * mode: Three interpolation modes: nearest (default), linear and cubic. The “linear” mode includes linear interpolation for 1D tensor and N-linear interpolation for N-D tensor (for example, bilinear interpolation for 2D tensor). The “cubic” mode includes cubic interpolation for 1D tensor and N-cubic interpolation for N-D tensor (for example, bicubic interpolation for 2D tensor). Default value is namemodesnearesttypeSTRING (STRING) * nearest_mode: Four modes: round_prefer_floor (default, as known as round half down), round_prefer_ceil (as known as round half up), floor, ceil. Only used by nearest interpolation. It indicates how to get “nearest” pixel in input tensor from x_original, so this attribute is valid only if “mode” is “nearest”. Default value is namenearestmodesroundpreferfloortypeSTRING (STRING)

Inputs

Between 1 and 4 inputs.

  • X (heterogeneous)T1: N-D tensor

  • roi (optional, heterogeneous)T2: 1-D tensor given as [start1, …, startN, end1, …, endN], where N is the rank of X. The RoIs’ coordinates are normalized in the coordinate system of the input image. It only takes effect when coordinate_transformation_mode is “tf_crop_and_resize”

  • scales (optional, heterogeneous)tensor(float): The scale array along each dimension. It takes value greater than 0. If it’s less than 1, it’s sampling down, otherwise, it’s upsampling. The number of elements of ‘scales’ should be the same as the rank of input ‘X’. One of ‘scales’ and ‘sizes’ MUST be specified and it is an error if both are specified. If ‘sizes’ is needed, the user can use an empty string as the name of ‘scales’ in this operator’s input list.

  • sizes (optional, heterogeneous)tensor(int64): The size of the output tensor. The number of elements of ‘sizes’ should be the same as the rank of input ‘X’. Only one of ‘scales’ and ‘sizes’ can be specified.

Outputs

  • Y (heterogeneous)T1: N-D tensor after resizing

Type Constraints

  • T1 tensor(uint8), tensor(uint16), tensor(uint32), tensor(uint64), tensor(int8), tensor(int16), tensor(int32), tensor(int64), tensor(bfloat16), tensor(float16), tensor(float), tensor(double), tensor(string), tensor(bool), tensor(complex64), tensor(complex128): Constrain input ‘X’ and output ‘Y’ to all tensor types.

  • T2 tensor(float16), tensor(float), tensor(double): Constrain roi type to float or double.

Version

Onnx name: Resize

This version of the operator has been available since version 13.

Runtime implementation: Resize

RoiAlign#

mlprodict.onnxrt.ops_cpu.op_roi_align.RoiAlign (self, onnx_node, desc = None, options)

Region of Interest (RoI) align operation described in the [Mask R-CNN paper](https://arxiv.org/abs/1703.06870). RoiAlign consumes an input tensor X and region of interests (rois) to apply pooling across each RoI; it produces a 4-D tensor of shape (num_rois, C, output_height, output_width).

RoiAlign is proposed to avoid the misalignment by removing quantizations while converting from original image into feature map and from feature map into RoI feature; in each ROI bin, the value of the sampled locations are computed directly through bilinear interpolation.

Attributes

  • coordinate_transformation_mode: Allowed values are ‘half_pixel’ and ‘output_half_pixel’. Use the value ‘half_pixel’ to pixel shift the input coordinates by -0.5 (the recommended behavior). Use the value ‘output_half_pixel’ to omit the pixel shift for the input (use this for a backward-compatible behavior). Default value is namecoordinatetransformationmodeshalfpixeltypeSTRING (STRING)

  • mode: The pooling method. Two modes are supported: ‘avg’ and ‘max’. Default is ‘avg’. Default value is namemodesavgtypeSTRING (STRING)

  • output_height: default 1; Pooled output Y’s height. Default value is nameoutputheighti1typeINT (INT)

  • output_width: default 1; Pooled output Y’s width. Default value is nameoutputwidthi1typeINT (INT)

  • sampling_ratio: Number of sampling points in the interpolation grid used to compute the output value of each pooled output bin. If > 0, then exactly sampling_ratio x sampling_ratio grid points are used. If == 0, then an adaptive number of grid points are used (computed as ceil(roi_width / output_width), and likewise for height). Default is 0. Default value is namesamplingratioi0typeINT (INT)

  • spatial_scale: Multiplicative spatial scale factor to translate ROI coordinates from their input spatial scale to the scale used when pooling, i.e., spatial scale of the input feature map X relative to the input image. E.g.; default is 1.0f. Default value is namespatialscalef1.0typeFLOAT (FLOAT)

Inputs

  • X (heterogeneous)T1: Input data tensor from the previous operator; 4-D feature map of shape (N, C, H, W), where N is the batch size, C is the number of channels, and H and W are the height and the width of the data.

  • rois (heterogeneous)T1: RoIs (Regions of Interest) to pool over; rois is 2-D input of shape (num_rois, 4) given as [[x1, y1, x2, y2], …]. The RoIs’ coordinates are in the coordinate system of the input image. Each coordinate set has a 1:1 correspondence with the ‘batch_indices’ input.

  • batch_indices (heterogeneous)T2: 1-D tensor of shape (num_rois,) with each element denoting the index of the corresponding image in the batch.

Outputs

  • Y (heterogeneous)T1: RoI pooled output, 4-D tensor of shape (num_rois, C, output_height, output_width). The r-th batch element Y[r-1] is a pooled feature map corresponding to the r-th RoI X[r-1].

Type Constraints

  • T1 tensor(float16), tensor(float), tensor(double): Constrain types to float tensors.

  • T2 tensor(int64): Constrain types to int tensors.

Version

Onnx name: RoiAlign

This version of the operator has been available since version 16.

Runtime implementation: RoiAlign

Round#

mlprodict.onnxrt.ops_cpu.op_round.Round (self, onnx_node, desc = None, options)

Round takes one input Tensor and rounds the values, element-wise, meaning it finds the nearest integer for each value. In case of halfs, the rule is to round them to the nearest even integer. The output tensor has the same shape and type as the input.

Examples: `` round([0.9]) = [1.0] round([2.5]) = [2.0] round([2.3]) = [2.0] round([1.5]) = [2.0] round([-4.5]) = [-4.0] ``

Inputs

  • X (heterogeneous)T: Input tensor

Outputs

  • Y (heterogeneous)T: Output tensor

Type Constraints

  • T tensor(float16), tensor(float), tensor(double): Constrain input and output types to float tensors.

Version

Onnx name: Round

This version of the operator has been available since version 11.

Runtime implementation: Round

SVMClassifier#

mlprodict.onnxrt.ops_cpu.op_svm_classifier.SVMClassifier (self, onnx_node, desc = None, options)

Support Vector Machine classifier

Attributes

  • classlabels_ints: Class labels if using integer labels. One and only one of the ‘classlabels_*’ attributes must be defined. default value cannot be automatically retrieved (INTS)

  • classlabels_strings: Class labels if using string labels. One and only one of the ‘classlabels_*’ attributes must be defined. default value cannot be automatically retrieved (STRINGS)

  • coefficients: default value cannot be automatically retrieved (FLOATS)

  • kernel_params: List of 3 elements containing gamma, coef0, and degree, in that order. Zero if unused for the kernel. default value cannot be automatically retrieved (FLOATS)

  • kernel_type: The kernel type, one of ‘LINEAR,’ ‘POLY,’ ‘RBF,’ ‘SIGMOID’. Default value is namekerneltypesLINEARtypeSTRING (STRING)

  • post_transform: Indicates the transform to apply to the score. One of ‘NONE,’ ‘SOFTMAX,’ ‘LOGISTIC,’ ‘SOFTMAX_ZERO,’ or ‘PROBIT’ Default value is nameposttransformsNONEtypeSTRING (STRING)

  • prob_a: First set of probability coefficients. default value cannot be automatically retrieved (FLOATS)

  • prob_b: Second set of probability coefficients. This array must be same size as prob_a. If these are provided then output Z are probability estimates, otherwise they are raw scores. default value cannot be automatically retrieved (FLOATS)

  • rho: default value cannot be automatically retrieved (FLOATS)

  • support_vectors: default value cannot be automatically retrieved (FLOATS)

  • vectors_per_class: default value cannot be automatically retrieved (INTS)

Inputs

  • X (heterogeneous)T1: Data to be classified.

Outputs

  • Y (heterogeneous)T2: Classification outputs (one class per example).

  • Z (heterogeneous)tensor(float): Class scores (one per class per example), if prob_a and prob_b are provided they are probabilities for each class, otherwise they are raw scores.

Type Constraints

  • T1 tensor(float), tensor(double), tensor(int64), tensor(int32): The input must be a tensor of a numeric type, either [C] or [N,C].

  • T2 tensor(string), tensor(int64): The output type will be a tensor of strings or integers, depending on which of the the classlabels_* attributes is used. Its size will match the bactch size of the input.

Version

Onnx name: SVMClassifier

This version of the operator has been available since version 1 of domain ai.onnx.ml.

Runtime implementation: SVMClassifier

SVMClassifierDouble#

mlprodict.onnxrt.ops_cpu.op_svm_classifier.SVMClassifierDouble (self, onnx_node, desc = None, options)

Version

Onnx name: SVMClassifierDouble

This version of the operator has been available since version of domain mlprodict.

Runtime implementation: SVMClassifierDouble

SVMRegressor#

mlprodict.onnxrt.ops_cpu.op_svm_regressor.SVMRegressor (self, onnx_node, desc = None, options)

Support Vector Machine regression prediction and one-class SVM anomaly detection.

Attributes

  • coefficients: Support vector coefficients. default value cannot be automatically retrieved (FLOATS)

  • kernel_params: List of 3 elements containing gamma, coef0, and degree, in that order. Zero if unused for the kernel. default value cannot be automatically retrieved (FLOATS)

  • kernel_type: The kernel type, one of ‘LINEAR,’ ‘POLY,’ ‘RBF,’ ‘SIGMOID’. Default value is namekerneltypesLINEARtypeSTRING (STRING)

  • n_supports: The number of support vectors. Default value is namensupportsi0typeINT (INT)

  • one_class: Flag indicating whether the regression is a one-class SVM or not. Default value is nameoneclassi0typeINT (INT)

  • post_transform: Indicates the transform to apply to the score. One of ‘NONE,’ ‘SOFTMAX,’ ‘LOGISTIC,’ ‘SOFTMAX_ZERO,’ or ‘PROBIT.’ Default value is nameposttransformsNONEtypeSTRING (STRING)

  • rho: default value cannot be automatically retrieved (FLOATS)

  • support_vectors: Chosen support vectors default value cannot be automatically retrieved (FLOATS)

Inputs

  • X (heterogeneous)T: Data to be regressed.

Outputs

  • Y (heterogeneous)tensor(float): Regression outputs (one score per target per example).

Type Constraints

  • T tensor(float), tensor(double), tensor(int64), tensor(int32): The input type must be a tensor of a numeric type, either [C] or [N,C].

Version

Onnx name: SVMRegressor

This version of the operator has been available since version 1 of domain ai.onnx.ml.

Runtime implementation: SVMRegressor

SVMRegressorDouble#

mlprodict.onnxrt.ops_cpu.op_svm_regressor.SVMRegressorDouble (self, onnx_node, desc = None, options)

Version

Onnx name: SVMRegressorDouble

This version of the operator has been available since version of domain mlprodict.

Runtime implementation: SVMRegressorDouble

Scaler#

mlprodict.onnxrt.ops_cpu.op_scaler.Scaler (self, onnx_node, desc = None, options)

Rescale input data, for example to standardize features by removing the mean and scaling to unit variance.

Attributes

  • offset: First, offset by this. Can be length of features in an [N,F] tensor or length 1, in which case it applies to all features, regardless of dimension count. default value cannot be automatically retrieved (FLOATS)

  • scale: Second, multiply by this. Can be length of features in an [N,F] tensor or length 1, in which case it applies to all features, regardless of dimension count. Must be same length as ‘offset’ default value cannot be automatically retrieved (FLOATS)

Inputs

  • X (heterogeneous)T: Data to be scaled.

Outputs

  • Y (heterogeneous)tensor(float): Scaled output data.

Type Constraints

  • T tensor(float), tensor(double), tensor(int64), tensor(int32): The input must be a tensor of a numeric type.

Version

Onnx name: Scaler

This version of the operator has been available since version 1 of domain ai.onnx.ml.

Runtime implementation: Scaler

Scan#

mlprodict.onnxrt.ops_cpu.op_scan.Scan (self, onnx_node, desc = None, options)

Scan can be used to iterate over one or more scan_input tensors, constructing zero or more scan_output tensors. It combines ideas from general recurrences, functional programming constructs such as scan, fold, map, and zip, and is intended to enable generalizations of RNN-like constructs for sequence-to-sequence processing. Other tensors (referred to as state_variables here) can be used to carry a state when iterating from one element to another (similar to hidden-state in RNNs, also referred to as loop-carried dependences in the context of loops). Many common usages involve a single scan_input tensor (where functionality similar to scan, fold and map can be obtained). When more than one scan_input is used, a behavior similar to zip is obtained.

The attribute body must be a graph, specifying the computation to be performed in every iteration. It takes as input the current values of the state_variables and the current iterated element of the scan_inputs. It must return the (updated) values of the state_variables and zero or more scan_output_element tensors. The values of the scan_output_element tensors are concatenated over all the iterations to produce the scan_output values of the scan construct (similar to the concatenated intermediate hidden-state values of RNN-like constructs). All the output tensors (state_variables as well as scan_output_element tensors) are required to have the same shape in each iteration of the loop (a restriction imposed to enable efficient memory allocation).

Note that the iterated element passed to the body subgraph does not have a sequence axis. It will have a rank one less than the rank of the corresponding scan_input.

The scan operation returns the final values of the state_variables as well as the scan_outputs.

The optional attribute scan_input_directions specifies the direction (forward or backward) for each scan input. If this attribute is omitted, all sequences are scanned in the forward direction. A bidirectional scan may be performed by specifying the same tensor input twice in the scan_inputs, once with a forward direction, and once with a backward direction.

The scan_output of the operation is produced by concatenating the scan_output_element values produced by the body in each iteration. The optional attribute scan_output_directions specifies the direction in which scan_output is constructed (by appending or prepending the scan_output_element to scan_output in each iteration) for each scan_output. If this attribute is omitted, the scan_output_element is appended to the scan_output in each iteration.

The optional attribute scan_input_axes specifies the axis to be scanned for each scan_input. If omitted, every scan_input will be scanned in axis 0. For example, if axis 0 is the batch axis and axis 1 is the time axis (to be scanned), specify an axis value of 1. Note that scanning a non-zero axis may be less efficient than scanning axis zero.

The optional attribute scan_output_axes specifies the axis along which the scan_outputs are accumulated for each scan_output. For example, if axis 1 is the time axis (to be scanned) for both inputs and outputs, specify a scan_input axis and scan_output axis value of 1.

Note that because of the ONNX restriction that only the last parameter of an operator can be variadic, the initial-states and scan-inputs are listed together as one input parameter. Similarly, the final-states and scan-outputs are listed together as one output parameter. The attribute num_scan_inputs indicates the number M of scan-inputs.

The behavior of

Scan <

num_scan_inputs = m, body = loop-body, scan_input_axes = [axis_1, …, axis_m]

> (init_1, …, init_n, scan_1, …, scan_m)

is equivalent to the following pseudo-code:

// scan_i.shape[axis_i] denotes the (max) sequence-length of scan_i // scan_i.shape[axis_i] is required to be equal to scan_j.shape[axis_j] for all i,j. sequence_length = scan_1.shape[axis_1];

// initialize state-variables st_1 = init_1; … st_n = init_n; // initialize scan-output variables: [] denotes an empty tensor scan_out_1 = []; …; scan_out_k = []; // identify number of iterations:

// execute loop for (int t = 0; t < sequence_length; ++t) {

// generate the scan-input elements: the notation T<axis=k>[t] indicates the sub-tensor // of rank one less than T obtained by indexing T at position t along axis k. si_1 = scan_1<axis=axis_1>[t]; … ; si_m = scan_m<axis=axis_m>[t]; // execute loop-body st_1, …, st_n, so_1, …, so_k = loop-body(st_1, …, st_n, si_1, …, si_m) // accumulate the scan-output elements scan_out_1 = Concat<axis=0>(scan_out_1, so_1); … ; scan_out_k = Concat<axis=0>(scan_out_k, so_k);

}

return st_1, …, st_n, scan_out_1, …, scan_out_k;

Sample usage: Encoding RNN using a Scan

The following example shows how a simple RNN over an input tensor %X, with weight tensor %Wi, recurrence weight tensor %Ri, bias tensors %Wbi and %Rbi, and initial hidden-state %H_0 can be encoded as a ScanLoop. Note that the loop-body is a nested graph, and it directly computes %Wi, %Ri, %Wbi, and %Rbi (typically constants or initializers in the body graph). If these values are computed in the outer graph, they need to be passed in as extra state_variables.

graph rnn-encoding {

%H_0 = … %X = … %Y_h, %Y = Scan[body = <graph rnn-cell-1>, num_scan_inputs=1](%H_0, %X) return %Y, %Y_h

}

graph rnn-cell-1 (

%H_tminus1[FLOAT, tensor] %X_t[FLOAT, tensor]

) {

%Wi = … %Ri = … %Wbi = … %Rbi = … %t1 = X_t * (Wi^T) %t2 = H_tminus1*(Ri^T) %t3 = Add(%t1, %t2) %t4 = Add(%t3, %Wbi) %t5 = Add(%t4, %Rbi) %Ht = Tanh(%t5) %Accumulate = Identity(%Ht) return %Ht, %Accumulate

}

Attributes

  • body (required): The graph run each iteration. It has N+M inputs: (loop state variables…, scan_input_elts…). It has N+K outputs: (loop state variables…, scan_output_elts…). Each scan_output is created by concatenating the value of the specified scan_output_elt value at the end of each iteration of the loop. It is an error if the dimensions of these values change across loop iterations. default value cannot be automatically retrieved (GRAPH)

  • num_scan_inputs (required): An attribute specifying the number of scan_inputs M. default value cannot be automatically retrieved (INT)

  • scan_input_axes: An optional list of M flags. The i-th element of the list specifies the axis to be scanned (the sequence axis) for the i-th scan_input. If omitted, 0 will be used as the scan axis for every scan_input. Negative value for an axis means counting dimensions from the back. Accepted range is [-r, r-1] where r = rank(input). default value cannot be automatically retrieved (INTS)

  • scan_input_directions: An optional list of M flags. The i-th element of the list specifies the direction to be scanned for the i-th scan_input tensor: 0 indicates forward direction and 1 indicates reverse direction. If omitted, all scan_input tensors will be scanned in the forward direction. default value cannot be automatically retrieved (INTS)

  • scan_output_axes: An optional list of K flags. The i-th element of the list specifies the axis for the i-th scan_output. The scan outputs are accumulated along the specified axis. If omitted, 0 will be used as the scan axis for every scan_output. Negative value for an axis means counting dimensions from the back. Accepted range is [-r, r-1]. default value cannot be automatically retrieved (INTS)

  • scan_output_directions: An optional list of K flags, one for each scan_output. The i-th element of the list specifies whether the i-th scan_output should be constructed by appending or prepending a new value in each iteration: 0 indicates appending and 1 indicates prepending. If omitted, all scan_output tensors will be produced by appending a value in each iteration. default value cannot be automatically retrieved (INTS)

Inputs

Between 1 and 2147483647 inputs.

  • initial_state_and_scan_inputs (variadic)V: Initial values of the loop’s N state variables followed by M scan_inputs

Outputs

Between 1 and 2147483647 outputs.

  • final_state_and_scan_outputs (variadic)V: Final values of the loop’s N state variables followed by K scan_outputs

Type Constraints

  • V tensor(uint8), tensor(uint16), tensor(uint32), tensor(uint64), tensor(int8), tensor(int16), tensor(int32), tensor(int64), tensor(bfloat16), tensor(float16), tensor(float), tensor(double), tensor(string), tensor(bool), tensor(complex64), tensor(complex128): All Tensor types

Version

Onnx name: Scan

This version of the operator has been available since version 16.

Runtime implementation: Scan

ScatterElements#

mlprodict.onnxrt.ops_cpu.op_scatter_elements.ScatterElements (self, onnx_node, desc = None, options)

ScatterElements takes three inputs data, updates, and indices of the same rank r >= 1 and an optional attribute axis that identifies an axis of data (by default, the outer-most axis, that is axis 0). The output of the operation is produced by creating a copy of the input data, and then updating its value to values specified by updates at specific index positions specified by indices. Its output shape is the same as the shape of data.

For each entry in updates, the target index in data is obtained by combining the corresponding entry in indices with the index of the entry itself: the index-value for dimension = axis is obtained from the value of the corresponding entry in indices and the index-value for dimension != axis is obtained from the index of the entry itself.

reduction allows specification of an optional reduction operation, which is applied to all values in updates tensor into output at the specified indices. In cases where reduction is set to “none”, indices should not have duplicate entries: that is, if idx1 != idx2, then indices[idx1] != indices[idx2]. For instance, in a 2-D tensor case, the update corresponding to the [i][j] entry is performed as below: ``

output[indices[i][j]][j] = updates[i][j] if axis = 0, output[i][indices[i][j]] = updates[i][j] if axis = 1,

`` When reduction is set to “add”, the update corresponding to the [i][j] entry is performed as below: ``

output[indices[i][j]][j] += updates[i][j] if axis = 0, output[i][indices[i][j]] += updates[i][j] if axis = 1,

`` When reduction is set to “mul”, the update corresponding to the [i][j] entry is performed as below: ``

output[indices[i][j]][j] *= updates[i][j] if axis = 0, output[i][indices[i][j]] *= updates[i][j] if axis = 1,

``

This operator is the inverse of GatherElements. It is similar to Torch’s Scatter operation.

Example 1: ``

data = [

[0.0, 0.0, 0.0], [0.0, 0.0, 0.0], [0.0, 0.0, 0.0],

] indices = [

[1, 0, 2], [0, 2, 1],

] updates = [

[1.0, 1.1, 1.2], [2.0, 2.1, 2.2],

] output = [

[2.0, 1.1, 0.0] [1.0, 0.0, 2.2] [0.0, 2.1, 1.2]

]

`` Example 2: ``

data = [[1.0, 2.0, 3.0, 4.0, 5.0]] indices = [[1, 3]] updates = [[1.1, 2.1]] axis = 1 output = [[1.0, 1.1, 3.0, 2.1, 5.0]]

``

Attributes

  • axis: Which axis to scatter on. Negative value means counting dimensions from the back. Accepted range is [-r, r-1] where r = rank(data). Default value is nameaxisi0typeINT (INT)

  • reduction: Type of reduction to apply: none (default), add, mul. ‘none’: no reduction applied. ‘add’: reduction using the addition operation. ‘mul’: reduction using the multiplication operation. Default value is namereductionsnonetypeSTRING (STRING)

Inputs

  • data (heterogeneous)T: Tensor of rank r >= 1.

  • indices (heterogeneous)Tind: Tensor of int32/int64 indices, of r >= 1 (same rank as input). All index values are expected to be within bounds [-s, s-1] along axis of size s. It is an error if any of the index values are out of bounds.

  • updates (heterogeneous)T: Tensor of rank r >=1 (same rank and shape as indices)

Outputs

  • output (heterogeneous)T: Tensor of rank r >= 1 (same rank as input).

Type Constraints

  • T tensor(uint8), tensor(uint16), tensor(uint32), tensor(uint64), tensor(int8), tensor(int16), tensor(int32), tensor(int64), tensor(bfloat16), tensor(float16), tensor(float), tensor(double), tensor(string), tensor(bool), tensor(complex64), tensor(complex128): Input and output types can be of any tensor type.

  • Tind tensor(int32), tensor(int64): Constrain indices to integer types

Version

Onnx name: ScatterElements

This version of the operator has been available since version 16.

Runtime implementation: ScatterElements

ScatterND#

mlprodict.onnxrt.ops_cpu.op_scatternd.ScatterND (self, onnx_node, desc = None, options)

ScatterND takes three inputs data tensor of rank r >= 1, indices tensor of rank q >= 1, and updates tensor of rank q + r - indices.shape[-1] - 1. The output of the operation is produced by creating a copy of the input data, and then updating its value to values specified by updates at specific index positions specified by indices. Its output shape is the same as the shape of data. Note that indices should not have duplicate entries. That is, two or more updates for the same index-location is not supported.

indices is an integer tensor. Let k denote indices.shape[-1], the last dimension in the shape of indices.

indices is treated as a (q-1)-dimensional tensor of k-tuples, where each k-tuple is a partial-index into data.

Hence, k can be a value at most the rank of data. When k equals rank(data), each update entry specifies an update to a single element of the tensor. When k is less than rank(data) each update entry specifies an update to a slice of the tensor.

updates is treated as a (q-1)-dimensional tensor of replacement-slice-values. Thus, the first (q-1) dimensions of updates.shape must match the first (q-1) dimensions of indices.shape. The remaining dimensions of updates correspond to the dimensions of the replacement-slice-values. Each replacement-slice-value is a (r-k) dimensional tensor, corresponding to the trailing (r-k) dimensions of data. Thus, the shape of updates must equal indices.shape[0:q-1] ++ data.shape[k:r-1], where ++ denotes the concatenation of shapes.

The output is calculated via the following equation:

output = np.copy(data) update_indices = indices.shape[:-1] for idx in np.ndindex(update_indices):

output[indices[idx]] = updates[idx]

The order of iteration in the above loop is not specified. In particular, indices should not have duplicate entries: that is, if idx1 != idx2, then indices[idx1] != indices[idx2]. This ensures that the output value does not depend on the iteration order.

reduction allows specification of an optional reduction operation, which is applied to all values in updates tensor into output at the specified indices. In cases where reduction is set to “none”, indices should not have duplicate entries: that is, if idx1 != idx2, then indices[idx1] != indices[idx2]. This ensures that the output value does not depend on the iteration order. When reduction is set to “add”, output is calculated as follows:

output = np.copy(data) update_indices = indices.shape[:-1] for idx in np.ndindex(update_indices):

output[indices[idx]] += updates[idx]

When reduction is set to “mul”, output is calculated as follows:

output = np.copy(data) update_indices = indices.shape[:-1] for idx in np.ndindex(update_indices):

output[indices[idx]] *= updates[idx]

This operator is the inverse of GatherND.

Example 1: ``

data = [1, 2, 3, 4, 5, 6, 7, 8] indices = [[4], [3], [1], [7]] updates = [9, 10, 11, 12] output = [1, 11, 3, 10, 9, 6, 7, 12]

``

Example 2: ``

data = [[[1, 2, 3, 4], [5, 6, 7, 8], [8, 7, 6, 5], [4, 3, 2, 1]],

[[1, 2, 3, 4], [5, 6, 7, 8], [8, 7, 6, 5], [4, 3, 2, 1]], [[8, 7, 6, 5], [4, 3, 2, 1], [1, 2, 3, 4], [5, 6, 7, 8]], [[8, 7, 6, 5], [4, 3, 2, 1], [1, 2, 3, 4], [5, 6, 7, 8]]]

indices = [[0], [2]] updates = [[[5, 5, 5, 5], [6, 6, 6, 6], [7, 7, 7, 7], [8, 8, 8, 8]],

[[1, 1, 1, 1], [2, 2, 2, 2], [3, 3, 3, 3], [4, 4, 4, 4]]]

output = [[[5, 5, 5, 5], [6, 6, 6, 6], [7, 7, 7, 7], [8, 8, 8, 8]],

[[1, 2, 3, 4], [5, 6, 7, 8], [8, 7, 6, 5], [4, 3, 2, 1]], [[1, 1, 1, 1], [2, 2, 2, 2], [3, 3, 3, 3], [4, 4, 4, 4]], [[8, 7, 6, 5], [4, 3, 2, 1], [1, 2, 3, 4], [5, 6, 7, 8]]]

``

Attributes

  • reduction: Type of reduction to apply: none (default), add, mul. ‘none’: no reduction applied. ‘add’: reduction using the addition operation. ‘mul’: reduction using the multiplication operation. Default value is namereductionsnonetypeSTRING (STRING)

Inputs

  • data (heterogeneous)T: Tensor of rank r >= 1.

  • indices (heterogeneous)tensor(int64): Tensor of rank q >= 1.

  • updates (heterogeneous)T: Tensor of rank q + r - indices_shape[-1] - 1.

Outputs

  • output (heterogeneous)T: Tensor of rank r >= 1.

Type Constraints

  • T tensor(uint8), tensor(uint16), tensor(uint32), tensor(uint64), tensor(int8), tensor(int16), tensor(int32), tensor(int64), tensor(bfloat16), tensor(float16), tensor(float), tensor(double), tensor(string), tensor(bool), tensor(complex64), tensor(complex128): Constrain input and output types to any tensor type.

Version

Onnx name: ScatterND

This version of the operator has been available since version 16.

Runtime implementation: ScatterND

Selu#

mlprodict.onnxrt.ops_cpu.op_selu.Selu (self, onnx_node, desc = None, options)

Selu takes one input data (Tensor<T>) and produces one output data (Tensor<T>) where the scaled exponential linear unit function, y = gamma * (alpha * e^x - alpha) for x <= 0, y = gamma * x for x > 0, is applied to the tensor elementwise.

Attributes

  • alpha: Coefficient of SELU default to 1.67326319217681884765625 (i.e., float32 approximation of 1.6732632423543772848170429916717). Default value is namealphaf1.6732631921768188typeFLOAT (FLOAT)

  • gamma: Coefficient of SELU default to 1.05070102214813232421875 (i.e., float32 approximation of 1.0507009873554804934193349852946). Default value is namegammaf1.0507010221481323typeFLOAT (FLOAT)

Inputs

  • X (heterogeneous)T: Input tensor

Outputs

  • Y (heterogeneous)T: Output tensor

Type Constraints

  • T tensor(float16), tensor(float), tensor(double): Constrain input and output types to float tensors.

Version

Onnx name: Selu

This version of the operator has been available since version 6.

Runtime implementation: Selu

SequenceAt#

mlprodict.onnxrt.ops_cpu.op_sequence_at.SequenceAt (self, onnx_node, desc = None, options)

Outputs a tensor copy from the tensor at ‘position’ in ‘input_sequence’. Accepted range for ‘position’ is in [-n, n - 1], where n is the number of tensors in ‘input_sequence’. Negative value means counting positions from the back.

Inputs

  • input_sequence (heterogeneous)S: Input sequence.

  • position (heterogeneous)I: Position of the tensor in the sequence. Negative value means counting positions from the back. Accepted range in [-n, n - 1], where n is the number of tensors in ‘input_sequence’. It is an error if any of the index values are out of bounds. It must be a scalar(tensor of empty shape).

Outputs

  • tensor (heterogeneous)T: Output tensor at the specified position in the input sequence.

Type Constraints

  • S seq(tensor(uint8)), seq(tensor(uint16)), seq(tensor(uint32)), seq(tensor(uint64)), seq(tensor(int8)), seq(tensor(int16)), seq(tensor(int32)), seq(tensor(int64)), seq(tensor(float16)), seq(tensor(float)), seq(tensor(double)), seq(tensor(string)), seq(tensor(bool)), seq(tensor(complex64)), seq(tensor(complex128)): Constrain to any tensor type.

  • T tensor(uint8), tensor(uint16), tensor(uint32), tensor(uint64), tensor(int8), tensor(int16), tensor(int32), tensor(int64), tensor(float16), tensor(float), tensor(double), tensor(string), tensor(bool), tensor(complex64), tensor(complex128): Constrain to any tensor type.

  • I tensor(int32), tensor(int64): Constrain position to integral tensor. It must be a scalar(tensor of empty shape).

Version

Onnx name: SequenceAt

This version of the operator has been available since version 11.

Runtime implementation: SequenceAt

SequenceConstruct#

mlprodict.onnxrt.ops_cpu.op_sequence_construct.SequenceConstruct (self, onnx_node, desc = None, options)

Construct a tensor sequence containing ‘inputs’ tensors. All tensors in ‘inputs’ must have the same data type.

Inputs

Between 1 and 2147483647 inputs.

  • inputs (variadic, heterogeneous)T: Tensors.

Outputs

  • output_sequence (heterogeneous)S: Sequence enclosing the input tensors.

Type Constraints

  • T tensor(uint8), tensor(uint16), tensor(uint32), tensor(uint64), tensor(int8), tensor(int16), tensor(int32), tensor(int64), tensor(float16), tensor(float), tensor(double), tensor(string), tensor(bool), tensor(complex64), tensor(complex128): Constrain input types to any tensor type.

  • S seq(tensor(uint8)), seq(tensor(uint16)), seq(tensor(uint32)), seq(tensor(uint64)), seq(tensor(int8)), seq(tensor(int16)), seq(tensor(int32)), seq(tensor(int64)), seq(tensor(float16)), seq(tensor(float)), seq(tensor(double)), seq(tensor(string)), seq(tensor(bool)), seq(tensor(complex64)), seq(tensor(complex128)): Constrain output types to any tensor type.

Version

Onnx name: SequenceConstruct

This version of the operator has been available since version 11.

Runtime implementation: SequenceConstruct

SequenceEmpty#

mlprodict.onnxrt.ops_cpu.op_sequence_empty.SequenceEmpty (self, onnx_node, desc = None, options)

Construct an empty tensor sequence, with given data type.

Attributes

  • dtype: (Optional) The data type of the tensors in the output sequence. The default type is ‘float’. default value cannot be automatically retrieved (INT)

Outputs

  • output (heterogeneous)S: Empty sequence.

Type Constraints

  • S seq(tensor(uint8)), seq(tensor(uint16)), seq(tensor(uint32)), seq(tensor(uint64)), seq(tensor(int8)), seq(tensor(int16)), seq(tensor(int32)), seq(tensor(int64)), seq(tensor(float16)), seq(tensor(float)), seq(tensor(double)), seq(tensor(string)), seq(tensor(bool)), seq(tensor(complex64)), seq(tensor(complex128)): Constrain output types to any tensor type.

Version

Onnx name: SequenceEmpty

This version of the operator has been available since version 11.

Runtime implementation: SequenceEmpty

SequenceInsert#

mlprodict.onnxrt.ops_cpu.op_sequence_insert.SequenceInsert (self, onnx_node, desc = None, options)

Outputs a tensor sequence that inserts ‘tensor’ into ‘input_sequence’ at ‘position’. ‘tensor’ must have the same data type as ‘input_sequence’. Accepted range for ‘position’ is in [-n, n], where n is the number of tensors in ‘input_sequence’. Negative value means counting positions from the back. ‘position’ is optional, by default it inserts ‘tensor’ to the back of ‘input_sequence’.

Inputs

Between 2 and 3 inputs.

  • input_sequence (heterogeneous)S: Input sequence.

  • tensor (heterogeneous)T: Input tensor to be inserted into the input sequence.

  • position (optional, heterogeneous)I: Position in the sequence where the new tensor is inserted. It is optional and default is to insert to the back of the sequence. Negative value means counting positions from the back. Accepted range in [-n, n], where n is the number of tensors in ‘input_sequence’. It is an error if any of the index values are out of bounds. It must be a scalar(tensor of empty shape).

Outputs

  • output_sequence (heterogeneous)S: Output sequence that contains the inserted tensor at given position.

Type Constraints

  • T tensor(uint8), tensor(uint16), tensor(uint32), tensor(uint64), tensor(int8), tensor(int16), tensor(int32), tensor(int64), tensor(float16), tensor(float), tensor(double), tensor(string), tensor(bool), tensor(complex64), tensor(complex128): Constrain to any tensor type.

  • S seq(tensor(uint8)), seq(tensor(uint16)), seq(tensor(uint32)), seq(tensor(uint64)), seq(tensor(int8)), seq(tensor(int16)), seq(tensor(int32)), seq(tensor(int64)), seq(tensor(float16)), seq(tensor(float)), seq(tensor(double)), seq(tensor(string)), seq(tensor(bool)), seq(tensor(complex64)), seq(tensor(complex128)): Constrain to any tensor type.

  • I tensor(int32), tensor(int64): Constrain position to integral tensor. It must be a scalar(tensor of empty shape).

Version

Onnx name: SequenceInsert

This version of the operator has been available since version 11.

Runtime implementation: SequenceInsert

Shape_15#

mlprodict.onnxrt.ops_cpu.op_shape.Shape_15 (self, onnx_node, desc = None, options)

Takes a tensor as input and outputs an 1D int64 tensor containing the shape of the input tensor. Optional attributes start and end can be used to compute a slice of the input tensor’s shape. If start axis is omitted, the slice starts from axis 0. The end axis, if specified, is exclusive (and the returned value will not include the size of that axis). If the end axis is omitted, the axes upto the last one will be included. Negative axes indicate counting back from the last axis. Note that axes will be clamped to the range [0, r-1], where r is the rank of the input tensor if they are out-of-range (after adding r in the case of negative axis). Thus, specifying any end value > r is equivalent to specifying an end value of r, and specifying any start value < -r is equivalent to specifying a start value of 0.

For example: Input tensor with shape: [2, 3, 4] No attributes specified. Output: [2, 3, 4]

Input tensor with shape: [2, 3, 4] start: -1 Output: [4]

Input tensor with shape: [2, 3, 4] end: -1 Output: [2, 3]

Input tensor with shape: [2, 3, 4] start: 1 end: 2 Output: [3]

Attributes

  • end: (Optional) Ending axis for slicing the shape. Negative value means counting dimensions from the back. If omitted, sizes of all axes upto (including) the last one will be included. default value cannot be automatically retrieved (INT)

  • start: (Optional) Starting axis for slicing the shape. Default value is 0.Negative value means counting dimensions from the back. Default value is namestarti0typeINT (INT)

Inputs

  • data (heterogeneous)T: An input tensor.

Outputs

  • shape (heterogeneous)T1: Shape of the input tensor

Type Constraints

  • T tensor(uint8), tensor(uint16), tensor(uint32), tensor(uint64), tensor(int8), tensor(int16), tensor(int32), tensor(int64), tensor(bfloat16), tensor(float16), tensor(float), tensor(double), tensor(string), tensor(bool), tensor(complex64), tensor(complex128): Input tensor can be of arbitrary type.

  • T1 tensor(int64): Constrain output to int64 tensor.

Version

Onnx name: Shape

This version of the operator has been available since version 15.

Runtime implementation: Shape

Shrink#

mlprodict.onnxrt.ops_cpu.op_shrink.Shrink (self, onnx_node, desc = None, options)

Shrink takes one input data (Tensor<numeric>) and produces one Tensor output, having same datatype and shape with input. It has two attributes, lambd and bias. The formula of this operator is: If x < -lambd, y = x + bias; If x > lambd, y = x - bias; Otherwise, y = 0.

Attributes

  • bias: The bias value added to output. Default is 0. Default value is namebiasf0.0typeFLOAT (FLOAT)

  • lambd: The lambd value for the Shrink formulation. Default is 0.5. Default value is namelambdf0.5typeFLOAT (FLOAT)

Inputs

  • input (heterogeneous)T: The input data as Tensor.

Outputs

  • output (heterogeneous)T: The output.

Type Constraints

  • T tensor(uint8), tensor(uint16), tensor(uint32), tensor(uint64), tensor(int8), tensor(int16), tensor(int32), tensor(int64), tensor(float16), tensor(float), tensor(double): Constrain input to only numeric types.

Version

Onnx name: Shrink

This version of the operator has been available since version 9.

Runtime implementation: Shrink

Sigmoid#

mlprodict.onnxrt.ops_cpu.op_sigmoid.Sigmoid (self, onnx_node, desc = None, options)

Sigmoid takes one input data (Tensor<T>) and produces one output data (Tensor<T>) where the sigmoid function, y = 1 / (1 + exp(-x)), is applied to the tensor elementwise.

Inputs

  • X (heterogeneous)T: Input tensor

Outputs

  • Y (heterogeneous)T: Output tensor

Type Constraints

  • T tensor(float16), tensor(float), tensor(double), tensor(bfloat16): Constrain input and output types to float tensors.

Version

Onnx name: Sigmoid

This version of the operator has been available since version 13.

Runtime implementation: Sigmoid

Sign#

mlprodict.onnxrt.ops_cpu.op_sign.Sign (self, onnx_node, desc = None, options)

Calculate the sign of the given input tensor element-wise. If input > 0, output 1. if input < 0, output -1. if input == 0, output 0.

Inputs

  • input (heterogeneous)T: Input tensor

Outputs

  • output (heterogeneous)T: The sign of the input tensor computed element-wise. It has the same shape and type of the input.

Type Constraints

  • T tensor(uint8), tensor(uint16), tensor(uint32), tensor(uint64), tensor(int8), tensor(int16), tensor(int32), tensor(int64), tensor(float16), tensor(float), tensor(double), tensor(bfloat16): Constrain input and output types to all numeric tensors.

Version

Onnx name: Sign

This version of the operator has been available since version 13.

Runtime implementation: Sign

Sin#

mlprodict.onnxrt.ops_cpu.op_sin.Sin (self, onnx_node, desc = None, options)

Calculates the sine of the given input tensor, element-wise.

Inputs

  • input (heterogeneous)T: Input tensor

Outputs

  • output (heterogeneous)T: The sine of the input tensor computed element-wise

Type Constraints

  • T tensor(float16), tensor(float), tensor(double): Constrain input and output types to float tensors.

Version

Onnx name: Sin

This version of the operator has been available since version 7.

Runtime implementation: Sin

Sinh#

mlprodict.onnxrt.ops_cpu.op_sinh.Sinh (self, onnx_node, desc = None, options)

Calculates the hyperbolic sine of the given input tensor element-wise.

Inputs

  • input (heterogeneous)T: Input tensor

Outputs

  • output (heterogeneous)T: The hyperbolic sine values of the input tensor computed element-wise

Type Constraints

  • T tensor(float16), tensor(float), tensor(double): Constrain input and output types to float tensors.

Version

Onnx name: Sinh

This version of the operator has been available since version 9.

Runtime implementation: Sinh

Size#

mlprodict.onnxrt.ops_cpu.op_size.Size (self, onnx_node, desc = None, options)

Takes a tensor as input and outputs a int64 scalar that equals to the total number of elements of the input tensor.

Inputs

  • data (heterogeneous)T: An input tensor.

Outputs

  • size (heterogeneous)T1: Total number of elements of the input tensor

Type Constraints

  • T tensor(uint8), tensor(uint16), tensor(uint32), tensor(uint64), tensor(int8), tensor(int16), tensor(int32), tensor(int64), tensor(bfloat16), tensor(float16), tensor(float), tensor(double), tensor(string), tensor(bool), tensor(complex64), tensor(complex128): Input tensor can be of arbitrary type.

  • T1 tensor(int64): Constrain output to int64 tensor, which should be a scalar though.

Version

Onnx name: Size

This version of the operator has been available since version 13.

Runtime implementation: Size

Slice_10#

mlprodict.onnxrt.ops_cpu.op_slice.Slice_10 (self, onnx_node, desc = None, options)

Produces a slice of the input tensor along multiple axes. Similar to numpy: https://docs.scipy.org/doc/numpy/reference/arrays.indexing.html Slices uses starts, ends, axes and steps inputs to specify the start and end dimension and step for each axis in the list of axes, it uses this information to slice the input data tensor. If a negative value is passed for any of the start or end indices, it represent number of elements before the end of that dimension. If the value passed to start or end is larger than the n (the number of elements in this dimension), it represents n. For slicing to the end of a dimension with unknown size, it is recommended to pass in INT_MAX. If a negative value is passed for step, it represents slicing backward. If axes are omitted, they are set to [0, …, ndim-1]. If steps are omitted, they are set to [1, …, 1] of length len(starts) Example 1:

data = [

[1, 2, 3, 4], [5, 6, 7, 8],

] axes = [0, 1] starts = [1, 0] ends = [2, 3] steps = [1, 2] result = [

[5, 7],

]

Example 2:
data = [

[1, 2, 3, 4], [5, 6, 7, 8],

] starts = [0, 1] ends = [-1, 1000] result = [

[2, 3, 4],

]

Inputs

Between 3 and 5 inputs.

  • data (heterogeneous)T: Tensor of data to extract slices from.

  • starts (heterogeneous)Tind: 1-D tensor of starting indices of corresponding axis in axes

  • ends (heterogeneous)Tind: 1-D tensor of ending indices (exclusive) of corresponding axis in axes

  • axes (optional, heterogeneous)Tind: 1-D tensor of axes that starts and ends apply to.

  • steps (optional, heterogeneous)Tind: 1-D tensor of slice step of corresponding axis in axes. Default to 1.

Outputs

  • output (heterogeneous)T: Sliced data tensor.

Type Constraints

  • T tensor(uint8), tensor(uint16), tensor(uint32), tensor(uint64), tensor(int8), tensor(int16), tensor(int32), tensor(int64), tensor(float16), tensor(float), tensor(double), tensor(string), tensor(bool), tensor(complex64), tensor(complex128): Constrain input and output types to all tensor types.

  • Tind tensor(int32), tensor(int64): Constrain indices to integer types

Version

Onnx name: Slice

This version of the operator has been available since version 10.

Runtime implementation: Slice

Softmax#

mlprodict.onnxrt.ops_cpu.op_softmax.Softmax (self, onnx_node, desc = None, options)

The operator computes the normalized exponential values for the given input:

Softmax(input, axis) = Exp(input) / ReduceSum(Exp(input), axis=axis, keepdims=1)

The “axis” attribute indicates the dimension along which Softmax will be performed. The output tensor has the same shape and contains the Softmax values of the corresponding input.

Attributes

  • axis:

Describes the dimension Softmax will be performed on. Negative value means counting dimensions from the back. Accepted range is [-r, r-1] where r = rank(input).

Default value is nameaxisi-1typeINT (INT)

Inputs

  • input (heterogeneous)T: The input tensor of rank >= axis.

Outputs

  • output (heterogeneous)T: The output values with the same shape as the input tensor.

Type Constraints

  • T tensor(float16), tensor(float), tensor(double), tensor(bfloat16): Constrain input and output types to float tensors.

Version

Onnx name: Softmax

This version of the operator has been available since version 13.

Runtime implementation: Softmax

Softplus#

mlprodict.onnxrt.ops_cpu.op_softplus.Softplus (self, onnx_node, desc = None, options)

Softplus takes one input data (Tensor<T>) and produces one output data (Tensor<T>) where the softplus function, y = ln(exp(x) + 1), is applied to the tensor elementwise.

Inputs

  • X (heterogeneous)T: 1D input tensor

Outputs

  • Y (heterogeneous)T: 1D input tensor

Type Constraints

  • T tensor(float16), tensor(float), tensor(double): Constrain input and output types to float tensors.

Version

Onnx name: Softplus

This version of the operator has been available since version 1.

Runtime implementation: Softplus

Softsign#

mlprodict.onnxrt.ops_cpu.op_softsign.Softsign (self, onnx_node, desc = None, options)

Calculates the softsign (x/(1+|x|)) of the given input tensor element-wise.

Inputs

  • input (heterogeneous)T: Input tensor

Outputs

  • output (heterogeneous)T: The softsign (x/(1+|x|)) values of the input tensor computed element-wise

Type Constraints

  • T tensor(float16), tensor(float), tensor(double): Constrain input and output types to float tensors.

Version

Onnx name: Softsign

This version of the operator has been available since version 1.

Runtime implementation: Softsign

Solve#

mlprodict.onnxrt.ops_cpu.op_solve.Solve (self, onnx_node, desc = None, options)

Version

Onnx name: Solve

This version of the operator has been available since version of domain mlprodict.

Runtime implementation: Solve

SpaceToDepth#

mlprodict.onnxrt.ops_cpu.op_depth_to_space.SpaceToDepth (self, onnx_node, desc = None, options)

SpaceToDepth rearranges blocks of spatial data into depth. More specifically, this op outputs a copy of the input tensor where values from the height and width dimensions are moved to the depth dimension.

Attributes

  • blocksize (required): Blocks of [blocksize, blocksize] are moved. default value cannot be automatically retrieved (INT)

Inputs

  • input (heterogeneous)T: Input tensor of [N,C,H,W], where N is the batch axis, C is the channel or depth, H is the height and W is the width.

Outputs

  • output (heterogeneous)T: Output tensor of [N, C * blocksize * blocksize, H/blocksize, W/blocksize].

Type Constraints

  • T tensor(uint8), tensor(uint16), tensor(uint32), tensor(uint64), tensor(int8), tensor(int16), tensor(int32), tensor(int64), tensor(bfloat16), tensor(float16), tensor(float), tensor(double), tensor(string), tensor(bool), tensor(complex64), tensor(complex128): Constrain input and output types to all tensor types.

Version

Onnx name: SpaceToDepth

This version of the operator has been available since version 13.

Runtime implementation: SpaceToDepth

Sqrt#

mlprodict.onnxrt.ops_cpu.op_sqrt.Sqrt (self, onnx_node, desc = None, options)

Square root takes one input data (Tensor<T>) and produces one output data (Tensor<T>) where the square root is, y = x^0.5, is applied to the tensor elementwise. If x is negative, then it will return NaN.

Inputs

  • X (heterogeneous)T: Input tensor

Outputs

  • Y (heterogeneous)T: Output tensor

Type Constraints

  • T tensor(float16), tensor(float), tensor(double), tensor(bfloat16): Constrain input and output types to float tensors.

Version

Onnx name: Sqrt

This version of the operator has been available since version 13.

Runtime implementation: Sqrt

Squeeze_13#

mlprodict.onnxrt.ops_cpu.op_squeeze.Squeeze_13 (self, onnx_node, desc = None, options)

Remove single-dimensional entries from the shape of a tensor. Takes an input axes with a list of axes to squeeze. If axes is not provided, all the single dimensions will be removed from the shape. If an axis is selected with shape entry not equal to one, an error is raised.

Inputs

Between 1 and 2 inputs.

  • data (heterogeneous)T: Tensors with at least max(dims) dimensions.

  • axes (optional, heterogeneous)tensor(int64): List of integers indicating the dimensions to squeeze. Negative value means counting dimensions from the back. Accepted range is [-r, r-1] where r = rank(data).

Outputs

  • squeezed (heterogeneous)T: Reshaped tensor with same data as input.

Type Constraints

  • T tensor(uint8), tensor(uint16), tensor(uint32), tensor(uint64), tensor(int8), tensor(int16), tensor(int32), tensor(int64), tensor(bfloat16), tensor(float16), tensor(float), tensor(double), tensor(string), tensor(bool), tensor(complex64), tensor(complex128): Constrain input and output types to all tensor types.

Version

Onnx name: Squeeze

This version of the operator has been available since version 13.

Runtime implementation: Squeeze

Sub#

mlprodict.onnxrt.ops_cpu.op_sub.Sub (self, onnx_node, desc = None, options)

Performs element-wise binary subtraction (with Numpy-style broadcasting support).

This operator supports multidirectional (i.e., Numpy-style) broadcasting; for more details please check Broadcasting in ONNX.

(Opset 14 change): Extend supported types to include uint8, int8, uint16, and int16.

Inputs

  • A (heterogeneous)T: First operand.

  • B (heterogeneous)T: Second operand.

Outputs

  • C (heterogeneous)T: Result, has same element type as two inputs

Type Constraints

  • T tensor(uint8), tensor(uint16), tensor(uint32), tensor(uint64), tensor(int8), tensor(int16), tensor(int32), tensor(int64), tensor(float16), tensor(float), tensor(double), tensor(bfloat16): Constrain input and output types to all numeric tensors.

Version

Onnx name: Sub

This version of the operator has been available since version 14.

Runtime implementation: Sub

Sum#

mlprodict.onnxrt.ops_cpu.op_sum.Sum (self, onnx_node, desc = None, options)

Element-wise sum of each of the input tensors (with Numpy-style broadcasting support). All inputs and outputs must have the same data type. This operator supports multidirectional (i.e., Numpy-style) broadcasting; for more details please check Broadcasting in ONNX.

Inputs

Between 1 and 2147483647 inputs.

  • data_0 (variadic, heterogeneous)T: List of tensors for sum.

Outputs

  • sum (heterogeneous)T: Output tensor.

Type Constraints

  • T tensor(float16), tensor(float), tensor(double), tensor(bfloat16): Constrain input and output types to float tensors.

Version

Onnx name: Sum

This version of the operator has been available since version 13.

Runtime implementation: Sum

Tan#

mlprodict.onnxrt.ops_cpu.op_tan.Tan (self, onnx_node, desc = None, options)

Calculates the tangent of the given input tensor, element-wise.

Inputs

  • input (heterogeneous)T: Input tensor

Outputs

  • output (heterogeneous)T: The tangent of the input tensor computed element-wise

Type Constraints

  • T tensor(float16), tensor(float), tensor(double): Constrain input and output types to float tensors.

Version

Onnx name: Tan

This version of the operator has been available since version 7.

Runtime implementation: Tan

Tanh#

mlprodict.onnxrt.ops_cpu.op_tanh.Tanh (self, onnx_node, desc = None, options)

Calculates the hyperbolic tangent of the given input tensor element-wise.

Inputs

  • input (heterogeneous)T: Input tensor

Outputs

  • output (heterogeneous)T: The hyperbolic tangent values of the input tensor computed element-wise

Type Constraints

  • T tensor(float16), tensor(float), tensor(double), tensor(bfloat16): Constrain input and output types to float tensors.

Version

Onnx name: Tanh

This version of the operator has been available since version 13.

Runtime implementation: Tanh

TfIdfVectorizer#

mlprodict.onnxrt.ops_cpu.op_tfidfvectorizer.TfIdfVectorizer (self, onnx_node, desc = None, options)

This transform extracts n-grams from the input sequence and save them as a vector. Input can be either a 1-D or 2-D tensor. For 1-D input, output is the n-gram representation of that input. For 2-D input, the output is also a 2-D tensor whose i-th row is the n-gram representation of the i-th input row. More specifically, if input shape is [C], the corresponding output shape would be [max(ngram_indexes) + 1]. If input shape is [N, C], this operator produces a [N, max(ngram_indexes) + 1]-tensor.

In contrast to standard n-gram extraction, here, the indexes of extracting an n-gram from the original sequence are not necessarily consecutive numbers. The discontinuity between indexes are controlled by the number of skips. If the number of skips is 2, we should skip two tokens when scanning through the original sequence. Let’s consider an example. Assume that input sequence is [94, 17, 36, 12, 28] and the number of skips is 2. The associated 2-grams are [94, 12] and [17, 28] respectively indexed by [0, 3] and [1, 4]. If the number of skips becomes 0, the 2-grams generated are [94, 17], [17, 36], [36, 12], [12, 28] indexed by [0, 1], [1, 2], [2, 3], [3, 4], respectively.

The output vector (denoted by Y) stores the count of each n-gram; Y[ngram_indexes[i]] indicates the times that the i-th n-gram is found. The attribute ngram_indexes is used to determine the mapping between index i and the corresponding n-gram’s output coordinate. If pool_int64s is [94, 17, 17, 36], ngram_indexes is [1, 0], ngram_counts=[0, 0], then the Y[0] (first element in Y) and Y[1] (second element in Y) are the counts of [17, 36] and [94, 17], respectively. An n-gram which cannot be found in pool_strings/pool_int64s should be ignored and has no effect on the output. Note that we may consider all skips up to S when generating the n-grams.

The examples used above are true if mode is “TF”. If mode is “IDF”, all the counts larger than 1 would be truncated to 1 and the i-th element in weights would be used to scale (by multiplication) the count of the i-th n-gram in pool. If mode is “TFIDF”, this operator first computes the counts of all n-grams and then scale them by the associated values in the weights attribute.

Only one of pool_strings and pool_int64s can be set. If pool_int64s is set, the input should be an integer tensor. If pool_strings is set, the input must be a string tensor.

Attributes

  • max_gram_length (required): Maximum n-gram length. If this value is 3, 3-grams will be used to generate the output. default value cannot be automatically retrieved (INT)

  • max_skip_count (required): Maximum number of items (integers/strings) to be skipped when constructing an n-gram from X. If max_skip_count=1, min_gram_length=2, max_gram_length=3, this operator may generate 2-grams with skip_count=0 and skip_count=1, and 3-grams with skip_count=0 and skip_count=1 default value cannot be automatically retrieved (INT)

  • min_gram_length (required): Minimum n-gram length. If this value is 2 and max_gram_length is 3, output may contain counts of 2-grams and 3-grams. default value cannot be automatically retrieved (INT)

  • mode (required): The weighting criteria. It can be one of “TF” (term frequency), “IDF” (inverse document frequency), and “TFIDF” (the combination of TF and IDF) default value cannot be automatically retrieved (STRING)

  • ngram_counts (required): The starting indexes of 1-grams, 2-grams, and so on in pool. It is useful when determining the boundary between two consecutive collections of n-grams. For example, if ngram_counts is [0, 17, 36], the first index (zero-based) of 1-gram/2-gram/3-gram in pool are 0/17/36. This format is essentially identical to CSR (or CSC) sparse matrix format, and we choose to use this due to its popularity. default value cannot be automatically retrieved (INTS)

  • ngram_indexes (required): list of int64s (type: AttributeProto::INTS). This list is parallel to the specified ‘pool_*’ attribute. The i-th element in ngram_indexes indicate the coordinate of the i-th n-gram in the output tensor. default value cannot be automatically retrieved (INTS)

  • pool_int64s: List of int64 n-grams learned from the training set. Either this or pool_strings attributes must be present but not both. It’s an 1-D tensor starting with the collections of all 1-grams and ending with the collections of n-grams. The i-th element in pool stores the n-gram that should be mapped to coordinate ngram_indexes[i] in the output vector. default value cannot be automatically retrieved (INTS)

  • pool_strings: List of strings n-grams learned from the training set. Either this or pool_int64s attributes must be present but not both. It’s an 1-D tensor starting with the collections of all 1-grams and ending with the collections of n-grams. The i-th element in pool stores the n-gram that should be mapped to coordinate ngram_indexes[i] in the output vector. default value cannot be automatically retrieved (STRINGS)

  • weights: list of floats. This attribute stores the weight of each n-gram in pool. The i-th element in weights is the weight of the i-th n-gram in pool. Its length equals to the size of ngram_indexes. By default, weights is an all-one tensor.This attribute is used when mode is “IDF” or “TFIDF” to scale the associated word counts. default value cannot be automatically retrieved (FLOATS)

Inputs

  • X (heterogeneous)T: Input for n-gram extraction

Outputs

  • Y (heterogeneous)T1: Ngram results

Type Constraints

  • T tensor(string), tensor(int32), tensor(int64): Input is ether string UTF-8 or int32/int64

  • T1 tensor(float): 1-D tensor of floats

Version

Onnx name: TfIdfVectorizer

This version of the operator has been available since version 9.

Runtime implementation: TfIdfVectorizer

ThresholdedRelu#

mlprodict.onnxrt.ops_cpu.op_relu.ThresholdedRelu (self, onnx_node, desc = None, options)

ThresholdedRelu takes one input data (Tensor<T>) and produces one output data (Tensor<T>) where the rectified linear function, y = x for x > alpha, y = 0 otherwise, is applied to the tensor elementwise.

Attributes

  • alpha: Threshold value Default value is namealphaf1.0typeFLOAT (FLOAT)

Inputs

  • X (heterogeneous)T: Input tensor

Outputs

  • Y (heterogeneous)T: Output tensor

Type Constraints

  • T tensor(float16), tensor(float), tensor(double): Constrain input and output types to float tensors.

Version

Onnx name: ThresholdedRelu

This version of the operator has been available since version 10.

Runtime implementation: ThresholdedRelu

TopK_11#

mlprodict.onnxrt.ops_cpu.op_topk.TopK_11 (self, onnx_node, desc = None, options)

Retrieve the top-K largest or smallest elements along a specified axis. Given an input tensor of shape [a_1, a_2, …, a_n, r] and integer argument k, return two outputs:

-Value tensor of shape [a_1, a_2, …, a_{axis-1}, k, a_{axis+1}, … a_n]

which contains the values of the top k elements along the specified axis

-Index tensor of shape [a_1, a_2, …, a_{axis-1}, k, a_{axis+1}, … a_n] which

contains the indices of the top k elements (original indices from the input tensor).

If “largest” is 1 (the default value) then the k largest elements are returned. If “sorted” is 1 (the default value) then the resulting k elements will be sorted. If “sorted” is 0, order of returned ‘Values’ and ‘Indices’ are undefined.

Given two equivalent values, this operator uses the indices along the axis as

a tiebreaker. That is, the element with the lower index will appear first.

Attributes

  • axis: Dimension on which to do the sort. Negative value means counting dimensions from the back. Accepted range is [-r, r-1] where r = rank(input). Default value is nameaxisi-1typeINT (INT)

  • largest: Whether to return the top-K largest or smallest elements. Default value is namelargesti1typeINT (INT)

  • sorted: Whether to return the elements in sorted order. Default value is namesortedi1typeINT (INT)

Inputs

  • X (heterogeneous)T: Tensor of shape [a_1, a_2, …, a_n, r]

  • K (heterogeneous)tensor(int64): A 1-D tensor containing a single positive value corresponding to the number of top elements to retrieve

Outputs

  • Values (heterogeneous)T: Tensor of shape [a_1, a_2, …, a_{axis-1}, k, a_{axis+1}, … a_n] containing top K values from the input tensor

  • Indices (heterogeneous)I: Tensor of shape [a_1, a_2, …, a_{axis-1}, k, a_{axis+1}, … a_n] containing the corresponding input tensor indices for the top K values.

Type Constraints

  • T tensor(uint8), tensor(uint16), tensor(uint32), tensor(uint64), tensor(int8), tensor(int16), tensor(int32), tensor(int64), tensor(float16), tensor(float), tensor(double): Constrain input and output types to numeric tensors.

  • I tensor(int64): Constrain index tensor to int64

Version

Onnx name: TopK

This version of the operator has been available since version 11.

Runtime implementation: TopK

Transpose#

mlprodict.onnxrt.ops_cpu.op_transpose.Transpose (self, onnx_node, desc = None, options)

Transpose the input tensor similar to numpy.transpose. For example, when perm=(1, 0, 2), given an input tensor of shape (1, 2, 3), the output shape will be (2, 1, 3).

Attributes

  • perm: A list of integers. By default, reverse the dimensions, otherwise permute the axes according to the values given. default value cannot be automatically retrieved (INTS)

Inputs

  • data (heterogeneous)T: An input tensor.

Outputs

  • transposed (heterogeneous)T: Transposed output.

Type Constraints

  • T tensor(uint8), tensor(uint16), tensor(uint32), tensor(uint64), tensor(int8), tensor(int16), tensor(int32), tensor(int64), tensor(bfloat16), tensor(float16), tensor(float), tensor(double), tensor(string), tensor(bool), tensor(complex64), tensor(complex128): Constrain input and output types to all tensor types.

Version

Onnx name: Transpose

This version of the operator has been available since version 13.

Runtime implementation: Transpose

TreeEnsembleClassifierDouble#

mlprodict.onnxrt.ops_cpu.op_tree_ensemble_classifier.TreeEnsembleClassifierDouble (self, onnx_node, desc = None, options)

Version

Onnx name: TreeEnsembleClassifierDouble

This version of the operator has been available since version of domain mlprodict.

Runtime implementation: TreeEnsembleClassifierDouble

TreeEnsembleClassifier_3#

mlprodict.onnxrt.ops_cpu.op_tree_ensemble_classifier.TreeEnsembleClassifier_3 (self, onnx_node, desc = None, options)

Tree Ensemble classifier. Returns the top class for each of N inputs.

The attributes named ‘nodes_X’ form a sequence of tuples, associated by index into the sequences, which must all be of equal length. These tuples define the nodes.

Similarly, all fields prefixed with ‘class_’ are tuples of votes at the leaves. A leaf may have multiple votes, where each vote is weighted by the associated class_weights index.

One and only one of classlabels_strings or classlabels_int64s will be defined. The class_ids are indices into this list. All fields ending with <i>_as_tensor</i> can be used instead of the same parameter without the suffix if the element type is double and not float.

Attributes

  • base_values: Base values for classification, added to final class score; the size must be the same as the classes or can be left unassigned (assumed 0) default value cannot be automatically retrieved (FLOATS)

  • base_values_as_tensor: Base values for classification, added to final class score; the size must be the same as the classes or can be left unassigned (assumed 0) default value cannot be automatically retrieved (TENSOR)

  • class_ids: The index of the class list that each weight is for. default value cannot be automatically retrieved (INTS)

  • class_nodeids: node id that this weight is for. default value cannot be automatically retrieved (INTS)

  • class_treeids: The id of the tree that this node is in. default value cannot be automatically retrieved (INTS)

  • class_weights: The weight for the class in class_id. default value cannot be automatically retrieved (FLOATS)

  • class_weights_as_tensor: The weight for the class in class_id. default value cannot be automatically retrieved (TENSOR)

  • classlabels_int64s: Class labels if using integer labels. One and only one of the ‘classlabels_*’ attributes must be defined. default value cannot be automatically retrieved (INTS)

  • classlabels_strings: Class labels if using string labels. One and only one of the ‘classlabels_*’ attributes must be defined. default value cannot be automatically retrieved (STRINGS)

  • nodes_falsenodeids: Child node if expression is false. default value cannot be automatically retrieved (INTS)

  • nodes_featureids: Feature id for each node. default value cannot be automatically retrieved (INTS)

  • nodes_hitrates: Popularity of each node, used for performance and may be omitted. default value cannot be automatically retrieved (FLOATS)

  • nodes_hitrates_as_tensor: Popularity of each node, used for performance and may be omitted. default value cannot be automatically retrieved (TENSOR)

  • nodes_missing_value_tracks_true: For each node, define what to do in the presence of a missing value: if a value is missing (NaN), use the ‘true’ or ‘false’ branch based on the value in this array. This attribute may be left undefined, and the defalt value is false (0) for all nodes. default value cannot be automatically retrieved (INTS)

  • nodes_modes: The node kind, that is, the comparison to make at the node. There is no comparison to make at a leaf node. One of ‘BRANCH_LEQ’, ‘BRANCH_LT’, ‘BRANCH_GTE’, ‘BRANCH_GT’, ‘BRANCH_EQ’, ‘BRANCH_NEQ’, ‘LEAF’ default value cannot be automatically retrieved (STRINGS)

  • nodes_nodeids: Node id for each node. Ids may restart at zero for each tree, but it not required to. default value cannot be automatically retrieved (INTS)

  • nodes_treeids: Tree id for each node. default value cannot be automatically retrieved (INTS)

  • nodes_truenodeids: Child node if expression is true. default value cannot be automatically retrieved (INTS)

  • nodes_values: Thresholds to do the splitting on for each node. default value cannot be automatically retrieved (FLOATS)

  • nodes_values_as_tensor: Thresholds to do the splitting on for each node. default value cannot be automatically retrieved (TENSOR)

  • post_transform: Indicates the transform to apply to the score. One of ‘NONE,’ ‘SOFTMAX,’ ‘LOGISTIC,’ ‘SOFTMAX_ZERO,’ or ‘PROBIT.’ Default value is nameposttransformsNONEtypeSTRING (STRING)

Inputs

  • X (heterogeneous)T1: Input of shape [N,F]

Outputs

  • Y (heterogeneous)T2: N, Top class for each point

  • Z (heterogeneous)tensor(float): The class score for each class, for each point, a tensor of shape [N,E].

Type Constraints

  • T1 tensor(float), tensor(double), tensor(int64), tensor(int32): The input type must be a tensor of a numeric type.

  • T2 tensor(string), tensor(int64): The output type will be a tensor of strings or integers, depending on which of the the classlabels_* attributes is used.

Version

Onnx name: TreeEnsembleClassifier

This version of the operator has been available since version 3 of domain ai.onnx.ml.

Runtime implementation: TreeEnsembleClassifier

TreeEnsembleRegressor_3#

mlprodict.onnxrt.ops_cpu.op_tree_ensemble_regressor.TreeEnsembleRegressor_3 (self, onnx_node, desc = None, runtime_version = 1, options)

Tree Ensemble regressor. Returns the regressed values for each input in N.

All args with nodes_ are fields of a tuple of tree nodes, and it is assumed they are the same length, and an index i will decode the tuple across these inputs. Each node id can appear only once for each tree id.

All fields prefixed with target_ are tuples of votes at the leaves.

A leaf may have multiple votes, where each vote is weighted by the associated target_weights index.

All fields ending with <i>_as_tensor</i> can be used instead of the same parameter without the suffix if the element type is double and not float. All trees must have their node ids start at 0 and increment by 1.

Mode enum is BRANCH_LEQ, BRANCH_LT, BRANCH_GTE, BRANCH_GT, BRANCH_EQ, BRANCH_NEQ, LEAF

Attributes

  • aggregate_function: Defines how to aggregate leaf values within a target. One of ‘AVERAGE,’ ‘SUM,’ ‘MIN,’ ‘MAX.’ Default value is nameaggregatefunctionsSUMtypeSTRING (STRING)

  • base_values: Base values for classification, added to final class score; the size must be the same as the classes or can be left unassigned (assumed 0) default value cannot be automatically retrieved (FLOATS)

  • base_values_as_tensor: Base values for classification, added to final class score; the size must be the same as the classes or can be left unassigned (assumed 0) default value cannot be automatically retrieved (TENSOR)

  • n_targets: The total number of targets. default value cannot be automatically retrieved (INT)

  • nodes_falsenodeids: Child node if expression is false default value cannot be automatically retrieved (INTS)

  • nodes_featureids: Feature id for each node. default value cannot be automatically retrieved (INTS)

  • nodes_hitrates: Popularity of each node, used for performance and may be omitted. default value cannot be automatically retrieved (FLOATS)

  • nodes_hitrates_as_tensor: Popularity of each node, used for performance and may be omitted. default value cannot be automatically retrieved (TENSOR)

  • nodes_missing_value_tracks_true: For each node, define what to do in the presence of a NaN: use the ‘true’ (if the attribute value is 1) or ‘false’ (if the attribute value is 0) branch based on the value in this array. This attribute may be left undefined and the defalt value is false (0) for all nodes. default value cannot be automatically retrieved (INTS)

  • nodes_modes: The node kind, that is, the comparison to make at the node. There is no comparison to make at a leaf node. One of ‘BRANCH_LEQ’, ‘BRANCH_LT’, ‘BRANCH_GTE’, ‘BRANCH_GT’, ‘BRANCH_EQ’, ‘BRANCH_NEQ’, ‘LEAF’ default value cannot be automatically retrieved (STRINGS)

  • nodes_nodeids: Node id for each node. Node ids must restart at zero for each tree and increase sequentially. default value cannot be automatically retrieved (INTS)

  • nodes_treeids: Tree id for each node. default value cannot be automatically retrieved (INTS)

  • nodes_truenodeids: Child node if expression is true default value cannot be automatically retrieved (INTS)

  • nodes_values: Thresholds to do the splitting on for each node. default value cannot be automatically retrieved (FLOATS)

  • nodes_values_as_tensor: Thresholds to do the splitting on for each node. default value cannot be automatically retrieved (TENSOR)

  • post_transform: Indicates the transform to apply to the score. One of ‘NONE,’ ‘SOFTMAX,’ ‘LOGISTIC,’ ‘SOFTMAX_ZERO,’ or ‘PROBIT’ Default value is nameposttransformsNONEtypeSTRING (STRING)

  • target_ids: The index of the target that each weight is for default value cannot be automatically retrieved (INTS)

  • target_nodeids: The node id of each weight default value cannot be automatically retrieved (INTS)

  • target_treeids: The id of the tree that each node is in. default value cannot be automatically retrieved (INTS)

  • target_weights: The weight for each target default value cannot be automatically retrieved (FLOATS)

  • target_weights_as_tensor: The weight for each target default value cannot be automatically retrieved (TENSOR)

Inputs

  • X (heterogeneous)T: Input of shape [N,F]

Outputs

  • Y (heterogeneous)tensor(float): N classes

Type Constraints

  • T tensor(float), tensor(double), tensor(int64), tensor(int32): The input type must be a tensor of a numeric type.

Version

Onnx name: TreeEnsembleRegressor

This version of the operator has been available since version 3 of domain ai.onnx.ml.

Runtime implementation: TreeEnsembleRegressor

Trilu#

mlprodict.onnxrt.ops_cpu.op_trilu.Trilu (self, onnx_node, desc = None, options)

Given a 2-D matrix or batches of 2-D matrices, returns the upper or lower triangular part of the tensor(s). The attribute “upper” determines whether the upper or lower part is retained. If set to true, the upper triangular matrix is retained. Lower triangular matrix is retained otherwise. Default value for the “upper” attribute is true. Trilu takes one input tensor of shape [*, N, M], where * is zero or more batch dimensions. The upper triangular part consists of the elements on and above the given diagonal (k). The lower triangular part consists of elements on and below the diagonal. All other elements in the matrix are set to zero. If k = 0, the triangular part on and above/below the main diagonal is retained. If upper is set to true, a positive k retains the upper triangular matrix excluding the main diagonal and (k-1) diagonals above it. A negative k value retains the main diagonal and |k| diagonals below it. If upper is set to false, a positive k retains the lower triangular matrix including the main diagonal and k diagonals above it. A negative k value excludes the main diagonal and (|k|-1) diagonals below it.

Attributes

  • upper: Boolean. Indicates whether upper or lower part of matrix is retained. Default is true. Default value is nameupperi1typeINT (INT)

Inputs

Between 1 and 2 inputs.

  • input (heterogeneous)T: Input tensor of rank 2 or higher.

  • k (optional, heterogeneous)tensor(int64): A 0-D tensor containing a single value corresponding to the number diagonals above or below the main diagonal to exclude or include. Default value is 0 if it’s not specified.

Outputs

  • output (heterogeneous)T: Output tensor of the same type and shape as the input tensor.

Type Constraints

  • T tensor(uint8), tensor(uint16), tensor(uint32), tensor(uint64), tensor(int8), tensor(int16), tensor(int32), tensor(int64), tensor(bfloat16), tensor(float16), tensor(float), tensor(double), tensor(string), tensor(bool), tensor(complex64), tensor(complex128): Constrain input and output types to all tensor types.

Version

Onnx name: Trilu

This version of the operator has been available since version 14.

Runtime implementation: Trilu

Unique#

mlprodict.onnxrt.ops_cpu.op_unique.Unique (self, onnx_node, desc = None, options)

Find the unique elements of a tensor. When an optional attribute ‘axis’ is provided, unique subtensors sliced along the ‘axis’ are returned. Otherwise the input tensor is flattened and unique values of the flattened tensor are returned.

This operator returns the unique values or sliced unique subtensors of the input tensor and three optional outputs. The first output tensor ‘Y’ contains all unique values or subtensors of the input. The second optional output tensor ‘indices’ contains indices of ‘Y’ elements’ first occurance in ‘X’.. The third optional output tensor ‘inverse_indices’ contains, for elements of ‘X’, its corresponding indices in ‘Y’. “. The fourth optional output tensor ‘counts’ contains the count of each element of ‘Y’ in the input.

Outputs are either sorted in ascending order or optionally in the order of the first occurrence of the values in the input.

https://docs.scipy.org/doc/numpy/reference/generated/numpy.unique.html

Example 1:

input_X = [2, 1, 1, 3, 4, 3] attribute_sorted = 0 attribute_axis = None output_Y = [2, 1, 3, 4] output_indices = [0, 1, 3, 4] output_inverse_indices = [0, 1, 1, 2, 3, 2] output_counts = [1, 2, 2, 1]

Example 2:

input_X = [[1, 3], [2, 3]] attribute_sorted = 1 attribute_axis = None output_Y = [1, 2, 3] output_indices = [0, 2, 1] output_inverse_indices = [0, 2, 1, 2] output_counts = [1, 1, 2]

Example 3:

input_X = [[1, 0, 0], [1, 0, 0], [2, 3, 4]] attribute_sorted = 1 attribute_axis = 0 output_Y = [[1, 0, 0], [2, 3, 4]] output_indices = [0, 2] output_inverse_indices = [0, 0, 1] output_counts = [2, 1]

Example 4:
input_x = [[[1., 1.], [0., 1.], [2., 1.], [0., 1.]],

[[1., 1.], [0., 1.], [2., 1.], [0., 1.]]]

attribute_sorted = 1 attribute_axis = 1

intermediate data are presented below for better understanding:

there are 4 subtensors sliced along axis 1 of input_x (shape = (2, 4, 2)): A: [[1, 1], [1, 1]],

[[0, 1], [0, 1]], [[2, 1], [2, 1]], [[0, 1], [0, 1]].

there are 3 unique subtensors: [[1, 1], [1, 1]], [[0, 1], [0, 1]], [[2, 1], [2, 1]].

sorted unique subtensors: B: [[0, 1], [0, 1]],

[[1, 1], [1, 1]], [[2, 1], [2, 1]].

output_Y is constructed from B: [[[0. 1.], [1. 1.], [2. 1.]],

[[0. 1.], [1. 1.], [2. 1.]]]

output_indices is to map from B to A: [1, 0, 2]

output_inverse_indices is to map from A to B: [1, 0, 2, 0]

output_counts = [2 1 1]

Attributes

  • axis: (Optional) The dimension to apply unique. If not specified, the unique elements of the flattened input are returned. Negative value means counting dimensions from the back. Accepted range is [-r, r-1] where r = rank(input). default value cannot be automatically retrieved (INT)

  • sorted: (Optional) Whether to sort the unique elements in ascending order before returning as output. Must be one of 0, or 1 (default). Default value is namesortedi1typeINT (INT)

Inputs

  • X (heterogeneous)T: A N-D input tensor that is to be processed.

Outputs

Between 1 and 4 outputs.

  • Y (heterogeneous)T: A tensor of the same type as ‘X’ containing all the unique values or subtensors sliced along a provided ‘axis’ in ‘X’, either sorted or maintained in the same order they occur in input ‘X’

  • indices (optional, heterogeneous)tensor(int64): A 1-D INT64 tensor containing indices of ‘Y’ elements’ first occurance in ‘X’. When ‘axis’ is provided, it contains indices to subtensors in input ‘X’ on the ‘axis’. When ‘axis’ is not provided, it contains indices to values in the flattened input tensor.

  • inverse_indices (optional, heterogeneous)tensor(int64): A 1-D INT64 tensor containing, for elements of ‘X’, its corresponding indices in ‘Y’. When ‘axis’ is provided, it contains indices to subtensors in output ‘Y’ on the ‘axis’. When ‘axis’ is not provided, it contains indices to values in output ‘Y’.

  • counts (optional, heterogeneous)tensor(int64): A 1-D INT64 tensor containing the count of each element of ‘Y’ in input ‘X’

Type Constraints

  • T tensor(uint8), tensor(uint16), tensor(uint32), tensor(uint64), tensor(int8), tensor(int16), tensor(int32), tensor(int64), tensor(float16), tensor(float), tensor(double), tensor(string), tensor(bool), tensor(complex64), tensor(complex128): Input can be of any tensor type.

Version

Onnx name: Unique

This version of the operator has been available since version 11.

Runtime implementation: Unique

Unsqueeze_13#

mlprodict.onnxrt.ops_cpu.op_unsqueeze.Unsqueeze_13 (self, onnx_node, desc = None, options)

Insert single-dimensional entries to the shape of an input tensor (data). Takes one required input axes - which contains a list of dimension indices and this operator will insert a dimension of value 1 into the corresponding index of the output tensor (expanded).

For example:

Given an input tensor (data) of shape [3, 4, 5], then Unsqueeze(data, axes=[0, 4]) outputs a tensor (expanded) containing same data as data but with shape [1, 3, 4, 5, 1].

The input axes should not contain any duplicate entries. It is an error if it contains duplicates. The rank of the output tensor (output_rank) is the rank of the input tensor (data) plus the number of values in axes. Each value in axes should be within the (inclusive) range [-output_rank , output_rank - 1]. The order of values in axes does not matter and can come in any order.

Inputs

  • data (heterogeneous)T: Original tensor

  • axes (heterogeneous)tensor(int64): List of integers indicating the dimensions to be inserted. Negative value means counting dimensions from the back. Accepted range is [-r, r-1] where r = rank(expanded).

Outputs

  • expanded (heterogeneous)T: Reshaped tensor with same data as input.

Type Constraints

  • T tensor(uint8), tensor(uint16), tensor(uint32), tensor(uint64), tensor(int8), tensor(int16), tensor(int32), tensor(int64), tensor(bfloat16), tensor(float16), tensor(float), tensor(double), tensor(string), tensor(bool), tensor(complex64), tensor(complex128): Constrain input and output types to all tensor types.

Version

Onnx name: Unsqueeze

This version of the operator has been available since version 13.

Runtime implementation: Unsqueeze

Where#

mlprodict.onnxrt.ops_cpu.op_where.Where (self, onnx_node, desc = None, options)

Return elements, either from X or Y, depending on condition. Where behaves like [numpy.where](https://docs.scipy.org/doc/numpy/reference/generated/numpy.where.html) with three parameters.

This operator supports multidirectional (i.e., Numpy-style) broadcasting; for more details please check Broadcasting in ONNX.

History - Version 16 adds bfloat16 to the types allowed (for the second and third parameter).

Inputs

  • condition (heterogeneous)B: When True (nonzero), yield X, otherwise yield Y

  • X (heterogeneous)T: values selected at indices where condition is True

  • Y (heterogeneous)T: values selected at indices where condition is False

Outputs

  • output (heterogeneous)T: Tensor of shape equal to the broadcasted shape of condition, X, and Y.

Type Constraints

  • B tensor(bool): Constrain to boolean tensors.

  • T tensor(uint8), tensor(uint16), tensor(uint32), tensor(uint64), tensor(int8), tensor(int16), tensor(int32), tensor(int64), tensor(bfloat16), tensor(float16), tensor(float), tensor(double), tensor(string), tensor(bool), tensor(complex64), tensor(complex128): Constrain input and output types to all tensor types (including bfloat).

Version

Onnx name: Where

This version of the operator has been available since version 16.

Runtime implementation: Where

Xor#

mlprodict.onnxrt.ops_cpu.op_xor.Xor (self, onnx_node, desc = None, options)

Returns the tensor resulted from performing the xor logical operation elementwise on the input tensors A and B (with Numpy-style broadcasting support).

This operator supports multidirectional (i.e., Numpy-style) broadcasting; for more details please check Broadcasting in ONNX.

Inputs

  • A (heterogeneous)T: First input operand for the logical operator.

  • B (heterogeneous)T: Second input operand for the logical operator.

Outputs

  • C (heterogeneous)T1: Result tensor.

Type Constraints

  • T tensor(bool): Constrain input to boolean tensor.

  • T1 tensor(bool): Constrain output to boolean tensor.

Version

Onnx name: Xor

This version of the operator has been available since version 7.

Runtime implementation: Xor

YieldOp#

mlprodict.onnxrt.ops_cpu.op_yield_op.YieldOp (self, onnx_node, desc = None, options)

Version

Onnx name: YieldOp

This version of the operator has been available since version of domain mlprodict.

Runtime implementation: YieldOp