# LeakyRelu#

## LeakyRelu - 16#

Version

• name: LeakyRelu (GitHub)

• domain: main

• since_version: 16

• function: True

• support_level: SupportType.COMMON

• shape inference: True

This version of the operator has been available since version 16.

Summary

LeakyRelu takes input data (Tensor<T>) and an argument alpha, and produces one output data (Tensor<T>) where the function f(x) = alpha * x for x < 0, f(x) = x for x >= 0, is applied to the data tensor elementwise.

History - Version 16 adds bfloat16 to the types allowed.

Attributes

• alpha: Coefficient of leakage. Default value is `0.009999999776482582`.

Inputs

• X (heterogeneous) - T: Input tensor

Outputs

• Y (heterogeneous) - T: Output tensor

Type Constraints

• T in ( tensor(bfloat16), tensor(double), tensor(float), tensor(float16) ): Constrain input and output types to float tensors.

Examples

default

```node = onnx.helper.make_node(
"LeakyRelu", inputs=["x"], outputs=["y"], alpha=0.1
)

x = np.array([-1, 0, 1]).astype(np.float32)
# expected output [-0.1, 0., 1.]
y = np.clip(x, 0, np.inf) + np.clip(x, -np.inf, 0) * 0.1
expect(node, inputs=[x], outputs=[y], name="test_leakyrelu_example")

x = np.random.randn(3, 4, 5).astype(np.float32)
y = np.clip(x, 0, np.inf) + np.clip(x, -np.inf, 0) * 0.1
expect(node, inputs=[x], outputs=[y], name="test_leakyrelu")
```

_leakyrelu_default

```default_alpha = 0.01
node = onnx.helper.make_node(
"LeakyRelu",
inputs=["x"],
outputs=["y"],
)
x = np.random.randn(3, 4, 5).astype(np.float32)
y = np.clip(x, 0, np.inf) + np.clip(x, -np.inf, 0) * default_alpha
expect(node, inputs=[x], outputs=[y], name="test_leakyrelu_default")
```

Differences

 `0` `0` `LeakyRelu takes input data (Tensor) and an argument alpha, and produces one` `LeakyRelu takes input data (Tensor) and an argument alpha, and produces one` `1` `1` `output data (Tensor) where the function f(x) = alpha * x for x < 0,` `output data (Tensor) where the function f(x) = alpha * x for x < 0,` `2` `2` `f(x) = x for x >= 0, is applied to the data tensor elementwise.` `f(x) = x for x >= 0, is applied to the data tensor elementwise.` `3` `3` `4` `**History**` `5` `- Version 16 adds bfloat16 to the types allowed.` `6` `4` `7` `**Attributes**` `**Attributes**` `5` `8` `6` `9` `* **alpha**:` `* **alpha**:` `7` `10` ` Coefficient of leakage. Default value is 0.009999999776482582.` ` Coefficient of leakage. Default value is 0.009999999776482582.` `8` `11` `9` `12` `**Inputs**` `**Inputs**` `10` `13` `11` `14` `* **X** (heterogeneous) - **T**:` `* **X** (heterogeneous) - **T**:` `12` `15` ` Input tensor` ` Input tensor` `13` `16` `14` `17` `**Outputs**` `**Outputs**` `15` `18` `16` `19` `* **Y** (heterogeneous) - **T**:` `* **Y** (heterogeneous) - **T**:` `17` `20` ` Output tensor` ` Output tensor` `18` `21` `19` `22` `**Type Constraints**` `**Type Constraints**` `20` `23` `21` `24` `* **T** in (` `* **T** in (` `25` ` tensor(bfloat16),` `22` `26` ` tensor(double),` ` tensor(double),` `23` `27` ` tensor(float),` ` tensor(float),` `24` `28` ` tensor(float16)` ` tensor(float16)` `25` `29` ` ):` ` ):` `26` `30` ` Constrain input and output types to float tensors.` ` Constrain input and output types to float tensors.`

## LeakyRelu - 6#

Version

• name: LeakyRelu (GitHub)

• domain: main

• since_version: 6

• function: False

• support_level: SupportType.COMMON

• shape inference: True

This version of the operator has been available since version 6.

Summary

LeakyRelu takes input data (Tensor<T>) and an argument alpha, and produces one output data (Tensor<T>) where the function f(x) = alpha * x for x < 0, f(x) = x for x >= 0, is applied to the data tensor elementwise.

Attributes

• alpha: Coefficient of leakage. Default value is `0.009999999776482582`.

Inputs

• X (heterogeneous) - T: Input tensor

Outputs

• Y (heterogeneous) - T: Output tensor

Type Constraints

• T in ( tensor(double), tensor(float), tensor(float16) ): Constrain input and output types to float tensors.

Differences

 `0` `0` `LeakyRelu takes input data (Tensor) and an argument alpha, and produces one` `LeakyRelu takes input data (Tensor) and an argument alpha, and produces one` `1` `1` `output data (Tensor) where the function f(x) = alpha * x for x < 0,` `output data (Tensor) where the function f(x) = alpha * x for x < 0,` `2` `2` `f(x) = x for x >= 0, is applied to the data tensor elementwise.` `f(x) = x for x >= 0, is applied to the data tensor elementwise.` `3` `3` `4` `4` `**Attributes**` `**Attributes**` `5` `5` `6` `6` `* **alpha**:` `* **alpha**:` `7` `7` ` Coefficient of leakage default to 0.01. Default value is 0.009999999776482582.` ` Coefficient of leakage. Default value is 0.009999999776482582.` `8` `* **consumed_inputs**:` `9` ` legacy optimization attribute.` `10` `8` `11` `9` `**Inputs**` `**Inputs**` `12` `10` `13` `11` `* **X** (heterogeneous) - **T**:` `* **X** (heterogeneous) - **T**:` `14` `12` ` Input tensor` ` Input tensor` `15` `13` `16` `14` `**Outputs**` `**Outputs**` `17` `15` `18` `16` `* **Y** (heterogeneous) - **T**:` `* **Y** (heterogeneous) - **T**:` `19` `17` ` Output tensor` ` Output tensor` `20` `18` `21` `19` `**Type Constraints**` `**Type Constraints**` `22` `20` `23` `21` `* **T** in (` `* **T** in (` `24` `22` ` tensor(double),` ` tensor(double),` `25` `23` ` tensor(float),` ` tensor(float),` `26` `24` ` tensor(float16)` ` tensor(float16)` `27` `25` ` ):` ` ):` `28` `26` ` Constrain input and output types to float tensors.` ` Constrain input and output types to float tensors.`

## LeakyRelu - 1#

Version

• name: LeakyRelu (GitHub)

• domain: main

• since_version: 1

• function: False

• support_level: SupportType.COMMON

• shape inference: False

This version of the operator has been available since version 1.

Summary

LeakyRelu takes input data (Tensor<T>) and an argument alpha, and produces one output data (Tensor<T>) where the function f(x) = alpha * x for x < 0, f(x) = x for x >= 0, is applied to the data tensor elementwise.

Attributes

• alpha: Coefficient of leakage default to 0.01. Default value is `0.009999999776482582`.

• consumed_inputs: legacy optimization attribute.

Inputs

• X (heterogeneous) - T: Input tensor

Outputs

• Y (heterogeneous) - T: Output tensor

Type Constraints

• T in ( tensor(double), tensor(float), tensor(float16) ): Constrain input and output types to float tensors.