LeakyRelu

LeakyRelu - 16

Version

• name: LeakyRelu (GitHub)

• domain: main

• since_version: 16

• function: True

• support_level: SupportType.COMMON

• shape inference: True

This version of the operator has been available since version 16.

Summary

LeakyRelu takes input data (Tensor<T>) and an argument alpha, and produces one output data (Tensor<T>) where the function f(x) = alpha * x for x < 0, f(x) = x for x >= 0, is applied to the data tensor elementwise.

History - Version 16 adds bfloat16 to the types allowed.

Attributes

• alpha: Coefficient of leakage. Default value is 0.009999999776482582.

Inputs

• X (heterogeneous) - T: Input tensor

Outputs

• Y (heterogeneous) - T: Output tensor

Type Constraints

• T in ( tensor(bfloat16), tensor(double), tensor(float), tensor(float16) ): Constrain input and output types to float tensors.

Examples

default

node = onnx.helper.make_node(
    "LeakyRelu", inputs=["x"], outputs=["y"], alpha=0.1
)

x = np.array([-1, 0, 1]).astype(np.float32)
# expected output [-0.1, 0., 1.]
y = np.clip(x, 0, np.inf) + np.clip(x, -np.inf, 0) * 0.1
expect(node, inputs=[x], outputs=[y], name="test_leakyrelu_example")

x = np.random.randn(3, 4, 5).astype(np.float32)
y = np.clip(x, 0, np.inf) + np.clip(x, -np.inf, 0) * 0.1
expect(node, inputs=[x], outputs=[y], name="test_leakyrelu")

_leakyrelu_default

default_alpha = 0.01
node = onnx.helper.make_node(
    "LeakyRelu",
    inputs=["x"],
    outputs=["y"],
)
x = np.random.randn(3, 4, 5).astype(np.float32)
y = np.clip(x, 0, np.inf) + np.clip(x, -np.inf, 0) * default_alpha
expect(node, inputs=[x], outputs=[y], name="test_leakyrelu_default")

Differences

0	0	LeakyRelu takes input data (Tensor) and an argument alpha, and produces one	LeakyRelu takes input data (Tensor) and an argument alpha, and produces one
1	1	output data (Tensor) where the function f(x) = alpha * x for x < 0,	output data (Tensor) where the function f(x) = alpha * x for x < 0,
2	2	f(x) = x for x >= 0, is applied to the data tensor elementwise.	f(x) = x for x >= 0, is applied to the data tensor elementwise.
3	3
	4		**History**
	5		- Version 16 adds bfloat16 to the types allowed.
	6
4	7	**Attributes**	**Attributes**
5	8
6	9	* **alpha**:	* **alpha**:
7	10	Coefficient of leakage. Default value is 0.009999999776482582.	Coefficient of leakage. Default value is 0.009999999776482582.
8	11
9	12	**Inputs**	**Inputs**
10	13
11	14	* **X** (heterogeneous) - **T**:	* **X** (heterogeneous) - **T**:
12	15	Input tensor	Input tensor
13	16
14	17	**Outputs**	**Outputs**
15	18
16	19	* **Y** (heterogeneous) - **T**:	* **Y** (heterogeneous) - **T**:
17	20	Output tensor	Output tensor
18	21
19	22	**Type Constraints**	**Type Constraints**
20	23
21	24	* **T** in (	* **T** in (
	25		tensor(bfloat16),
22	26	tensor(double),	tensor(double),
23	27	tensor(float),	tensor(float),
24	28	tensor(float16)	tensor(float16)
25	29	):	):
26	30	Constrain input and output types to float tensors.	Constrain input and output types to float tensors.

LeakyRelu - 6

Version

• name: LeakyRelu (GitHub)

• domain: main

• since_version: 6

• function: False

• support_level: SupportType.COMMON

• shape inference: True

This version of the operator has been available since version 6.

Summary

LeakyRelu takes input data (Tensor<T>) and an argument alpha, and produces one output data (Tensor<T>) where the function f(x) = alpha * x for x < 0, f(x) = x for x >= 0, is applied to the data tensor elementwise.

Attributes

• alpha: Coefficient of leakage. Default value is 0.009999999776482582.

Inputs

• X (heterogeneous) - T: Input tensor

Outputs

• Y (heterogeneous) - T: Output tensor

Type Constraints

• T in ( tensor(double), tensor(float), tensor(float16) ): Constrain input and output types to float tensors.

Differences

0	0	LeakyRelu takes input data (Tensor) and an argument alpha, and produces one	LeakyRelu takes input data (Tensor) and an argument alpha, and produces one
1	1	output data (Tensor) where the function f(x) = alpha * x for x < 0,	output data (Tensor) where the function f(x) = alpha * x for x < 0,
2	2	f(x) = x for x >= 0, is applied to the data tensor elementwise.	f(x) = x for x >= 0, is applied to the data tensor elementwise.
3	3
4	4	**Attributes**	**Attributes**
5	5
6	6	* **alpha**:	* **alpha**:
7	7	Coefficient of leakage default to 0.01. Default value is 0.009999999776482582.	Coefficient of leakage. Default value is 0.009999999776482582.
8		* **consumed_inputs**:
9		legacy optimization attribute.
10	8
11	9	**Inputs**	**Inputs**
12	10
13	11	* **X** (heterogeneous) - **T**:	* **X** (heterogeneous) - **T**:
14	12	Input tensor	Input tensor
15	13
16	14	**Outputs**	**Outputs**
17	15
18	16	* **Y** (heterogeneous) - **T**:	* **Y** (heterogeneous) - **T**:
19	17	Output tensor	Output tensor
20	18
21	19	**Type Constraints**	**Type Constraints**
22	20
23	21	* **T** in (	* **T** in (
24	22	tensor(double),	tensor(double),
25	23	tensor(float),	tensor(float),
26	24	tensor(float16)	tensor(float16)
27	25	):	):
28	26	Constrain input and output types to float tensors.	Constrain input and output types to float tensors.

LeakyRelu - 1

Version

• name: LeakyRelu (GitHub)

• domain: main

• since_version: 1

• function: False

• support_level: SupportType.COMMON

• shape inference: False

This version of the operator has been available since version 1.

Summary

LeakyRelu takes input data (Tensor<T>) and an argument alpha, and produces one output data (Tensor<T>) where the function f(x) = alpha * x for x < 0, f(x) = x for x >= 0, is applied to the data tensor elementwise.

Attributes

• alpha: Coefficient of leakage default to 0.01. Default value is 0.009999999776482582.

• consumed_inputs: legacy optimization attribute.

Inputs

• X (heterogeneous) - T: Input tensor

Outputs

• Y (heterogeneous) - T: Output tensor

Type Constraints

• T in ( tensor(double), tensor(float), tensor(float16) ): Constrain input and output types to float tensors.