.. _l-onnx-doccom.microsoft-LambOptimizer:

=============================
com.microsoft - LambOptimizer
=============================

.. contents::
    :local:


.. _l-onnx-opcom-microsoft-lamboptimizer-1:

LambOptimizer - 1 (com.microsoft)
=================================

**Version**

* **name**: `LambOptimizer (GitHub) <https://github.com/onnx/onnx/blob/main/docs/Operators.md#com.microsoft.LambOptimizer>`_
* **domain**: **com.microsoft**
* **since_version**: **1**
* **function**:
* **support_level**:
* **shape inference**:

This version of the operator has been available
**since version 1 of domain com.microsoft**.

**Summary**

**Attributes**

* **alpha**:
  Coefficient of previous gradient in running average. Default value is ``?``.
* **beta**:
  Coefficient of previous squared gradient in running average.The
  effective learning rate is computed by r = R / (1 + T *
  decay_factor). Default to 0 so that increasing update counts doesn't
  reduce the learning rate. Default value is ``?``.
* **do_bias_correction**:
  Compute unbiased 1st and 2nd momentums. Default value is ``?``.
* **epsilon**:
  Small scalar to avoid dividing by zero. Default value is ``?``.
* **lambda**:
  Regularization coefficient of 0.5 * lambda * ||X||_2^2. Default to
  0, which means no regularization. Default value is ``?``.
* **max_norm_clip**:
  clip threshold of gradients. Default value is ``?``.
* **ratio_max**:
  Upper bound on confidence ratio. Default value is ``?``.
* **ratio_min**:
  Lower bound on confidence ratio. Default value is ``?``.

**Inputs**

Between 0 and 5125 inputs.

* **update_signal** (optional, heterogeneous) - **T_BOOL**:
  This signal indicates if weight tensors should be updated.
* **loss_scale** (optional, heterogeneous) - **T2**:
  Loss scale for mixed precision training.
* **gradient_norm** (optional, heterogeneous) - **T_GRAD_NORM**:
  Norm of global gradient.
* **R** (optional, heterogeneous) - **T1**:
  The initial learning rate.
* **step** (optional, heterogeneous) - **TInt64**:
  One-based index of the current training iteration.
* **__group_0__weights** (optional) - **T2**:
  weights to optimize.
* **__group_0__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_0__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_0__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_0__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_1__weights** (optional) - **T2**:
  weights to optimize.
* **__group_1__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_1__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_1__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_1__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_2__weights** (optional) - **T2**:
  weights to optimize.
* **__group_2__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_2__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_2__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_2__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_3__weights** (optional) - **T2**:
  weights to optimize.
* **__group_3__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_3__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_3__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_3__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_4__weights** (optional) - **T2**:
  weights to optimize.
* **__group_4__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_4__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_4__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_4__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_5__weights** (optional) - **T2**:
  weights to optimize.
* **__group_5__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_5__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_5__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_5__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_6__weights** (optional) - **T2**:
  weights to optimize.
* **__group_6__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_6__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_6__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_6__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_7__weights** (optional) - **T2**:
  weights to optimize.
* **__group_7__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_7__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_7__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_7__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_8__weights** (optional) - **T2**:
  weights to optimize.
* **__group_8__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_8__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_8__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_8__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_9__weights** (optional) - **T2**:
  weights to optimize.
* **__group_9__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_9__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_9__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_9__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_10__weights** (optional) - **T2**:
  weights to optimize.
* **__group_10__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_10__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_10__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_10__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_11__weights** (optional) - **T2**:
  weights to optimize.
* **__group_11__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_11__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_11__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_11__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_12__weights** (optional) - **T2**:
  weights to optimize.
* **__group_12__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_12__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_12__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_12__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_13__weights** (optional) - **T2**:
  weights to optimize.
* **__group_13__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_13__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_13__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_13__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_14__weights** (optional) - **T2**:
  weights to optimize.
* **__group_14__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_14__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_14__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_14__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_15__weights** (optional) - **T2**:
  weights to optimize.
* **__group_15__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_15__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_15__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_15__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_16__weights** (optional) - **T2**:
  weights to optimize.
* **__group_16__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_16__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_16__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_16__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_17__weights** (optional) - **T2**:
  weights to optimize.
* **__group_17__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_17__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_17__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_17__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_18__weights** (optional) - **T2**:
  weights to optimize.
* **__group_18__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_18__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_18__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_18__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_19__weights** (optional) - **T2**:
  weights to optimize.
* **__group_19__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_19__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_19__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_19__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_20__weights** (optional) - **T2**:
  weights to optimize.
* **__group_20__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_20__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_20__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_20__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_21__weights** (optional) - **T2**:
  weights to optimize.
* **__group_21__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_21__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_21__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_21__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_22__weights** (optional) - **T2**:
  weights to optimize.
* **__group_22__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_22__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_22__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_22__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_23__weights** (optional) - **T2**:
  weights to optimize.
* **__group_23__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_23__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_23__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_23__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_24__weights** (optional) - **T2**:
  weights to optimize.
* **__group_24__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_24__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_24__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_24__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_25__weights** (optional) - **T2**:
  weights to optimize.
* **__group_25__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_25__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_25__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_25__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_26__weights** (optional) - **T2**:
  weights to optimize.
* **__group_26__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_26__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_26__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_26__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_27__weights** (optional) - **T2**:
  weights to optimize.
* **__group_27__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_27__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_27__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_27__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_28__weights** (optional) - **T2**:
  weights to optimize.
* **__group_28__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_28__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_28__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_28__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_29__weights** (optional) - **T2**:
  weights to optimize.
* **__group_29__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_29__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_29__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_29__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_30__weights** (optional) - **T2**:
  weights to optimize.
* **__group_30__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_30__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_30__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_30__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_31__weights** (optional) - **T2**:
  weights to optimize.
* **__group_31__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_31__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_31__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_31__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_32__weights** (optional) - **T2**:
  weights to optimize.
* **__group_32__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_32__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_32__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_32__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_33__weights** (optional) - **T2**:
  weights to optimize.
* **__group_33__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_33__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_33__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_33__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_34__weights** (optional) - **T2**:
  weights to optimize.
* **__group_34__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_34__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_34__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_34__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_35__weights** (optional) - **T2**:
  weights to optimize.
* **__group_35__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_35__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_35__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_35__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_36__weights** (optional) - **T2**:
  weights to optimize.
* **__group_36__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_36__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_36__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_36__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_37__weights** (optional) - **T2**:
  weights to optimize.
* **__group_37__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_37__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_37__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_37__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_38__weights** (optional) - **T2**:
  weights to optimize.
* **__group_38__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_38__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_38__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_38__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_39__weights** (optional) - **T2**:
  weights to optimize.
* **__group_39__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_39__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_39__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_39__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_40__weights** (optional) - **T2**:
  weights to optimize.
* **__group_40__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_40__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_40__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_40__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_41__weights** (optional) - **T2**:
  weights to optimize.
* **__group_41__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_41__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_41__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_41__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_42__weights** (optional) - **T2**:
  weights to optimize.
* **__group_42__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_42__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_42__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_42__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_43__weights** (optional) - **T2**:
  weights to optimize.
* **__group_43__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_43__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_43__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_43__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_44__weights** (optional) - **T2**:
  weights to optimize.
* **__group_44__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_44__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_44__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_44__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_45__weights** (optional) - **T2**:
  weights to optimize.
* **__group_45__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_45__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_45__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_45__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_46__weights** (optional) - **T2**:
  weights to optimize.
* **__group_46__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_46__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_46__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_46__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_47__weights** (optional) - **T2**:
  weights to optimize.
* **__group_47__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_47__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_47__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_47__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_48__weights** (optional) - **T2**:
  weights to optimize.
* **__group_48__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_48__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_48__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_48__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_49__weights** (optional) - **T2**:
  weights to optimize.
* **__group_49__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_49__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_49__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_49__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_50__weights** (optional) - **T2**:
  weights to optimize.
* **__group_50__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_50__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_50__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_50__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_51__weights** (optional) - **T2**:
  weights to optimize.
* **__group_51__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_51__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_51__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_51__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_52__weights** (optional) - **T2**:
  weights to optimize.
* **__group_52__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_52__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_52__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_52__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_53__weights** (optional) - **T2**:
  weights to optimize.
* **__group_53__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_53__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_53__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_53__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_54__weights** (optional) - **T2**:
  weights to optimize.
* **__group_54__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_54__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_54__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_54__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_55__weights** (optional) - **T2**:
  weights to optimize.
* **__group_55__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_55__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_55__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_55__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_56__weights** (optional) - **T2**:
  weights to optimize.
* **__group_56__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_56__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_56__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_56__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_57__weights** (optional) - **T2**:
  weights to optimize.
* **__group_57__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_57__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_57__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_57__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_58__weights** (optional) - **T2**:
  weights to optimize.
* **__group_58__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_58__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_58__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_58__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_59__weights** (optional) - **T2**:
  weights to optimize.
* **__group_59__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_59__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_59__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_59__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_60__weights** (optional) - **T2**:
  weights to optimize.
* **__group_60__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_60__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_60__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_60__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_61__weights** (optional) - **T2**:
  weights to optimize.
* **__group_61__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_61__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_61__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_61__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_62__weights** (optional) - **T2**:
  weights to optimize.
* **__group_62__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_62__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_62__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_62__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_63__weights** (optional) - **T2**:
  weights to optimize.
* **__group_63__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_63__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_63__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_63__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_64__weights** (optional) - **T2**:
  weights to optimize.
* **__group_64__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_64__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_64__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_64__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_65__weights** (optional) - **T2**:
  weights to optimize.
* **__group_65__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_65__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_65__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_65__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_66__weights** (optional) - **T2**:
  weights to optimize.
* **__group_66__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_66__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_66__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_66__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_67__weights** (optional) - **T2**:
  weights to optimize.
* **__group_67__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_67__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_67__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_67__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_68__weights** (optional) - **T2**:
  weights to optimize.
* **__group_68__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_68__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_68__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_68__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_69__weights** (optional) - **T2**:
  weights to optimize.
* **__group_69__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_69__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_69__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_69__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_70__weights** (optional) - **T2**:
  weights to optimize.
* **__group_70__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_70__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_70__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_70__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_71__weights** (optional) - **T2**:
  weights to optimize.
* **__group_71__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_71__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_71__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_71__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_72__weights** (optional) - **T2**:
  weights to optimize.
* **__group_72__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_72__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_72__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_72__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_73__weights** (optional) - **T2**:
  weights to optimize.
* **__group_73__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_73__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_73__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_73__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_74__weights** (optional) - **T2**:
  weights to optimize.
* **__group_74__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_74__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_74__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_74__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_75__weights** (optional) - **T2**:
  weights to optimize.
* **__group_75__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_75__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_75__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_75__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_76__weights** (optional) - **T2**:
  weights to optimize.
* **__group_76__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_76__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_76__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_76__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_77__weights** (optional) - **T2**:
  weights to optimize.
* **__group_77__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_77__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_77__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_77__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_78__weights** (optional) - **T2**:
  weights to optimize.
* **__group_78__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_78__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_78__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_78__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_79__weights** (optional) - **T2**:
  weights to optimize.
* **__group_79__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_79__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_79__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_79__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_80__weights** (optional) - **T2**:
  weights to optimize.
* **__group_80__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_80__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_80__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_80__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_81__weights** (optional) - **T2**:
  weights to optimize.
* **__group_81__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_81__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_81__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_81__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_82__weights** (optional) - **T2**:
  weights to optimize.
* **__group_82__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_82__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_82__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_82__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_83__weights** (optional) - **T2**:
  weights to optimize.
* **__group_83__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_83__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_83__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_83__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_84__weights** (optional) - **T2**:
  weights to optimize.
* **__group_84__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_84__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_84__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_84__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_85__weights** (optional) - **T2**:
  weights to optimize.
* **__group_85__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_85__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_85__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_85__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_86__weights** (optional) - **T2**:
  weights to optimize.
* **__group_86__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_86__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_86__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_86__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_87__weights** (optional) - **T2**:
  weights to optimize.
* **__group_87__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_87__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_87__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_87__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_88__weights** (optional) - **T2**:
  weights to optimize.
* **__group_88__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_88__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_88__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_88__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_89__weights** (optional) - **T2**:
  weights to optimize.
* **__group_89__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_89__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_89__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_89__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_90__weights** (optional) - **T2**:
  weights to optimize.
* **__group_90__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_90__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_90__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_90__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_91__weights** (optional) - **T2**:
  weights to optimize.
* **__group_91__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_91__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_91__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_91__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_92__weights** (optional) - **T2**:
  weights to optimize.
* **__group_92__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_92__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_92__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_92__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_93__weights** (optional) - **T2**:
  weights to optimize.
* **__group_93__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_93__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_93__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_93__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_94__weights** (optional) - **T2**:
  weights to optimize.
* **__group_94__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_94__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_94__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_94__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_95__weights** (optional) - **T2**:
  weights to optimize.
* **__group_95__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_95__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_95__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_95__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_96__weights** (optional) - **T2**:
  weights to optimize.
* **__group_96__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_96__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_96__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_96__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_97__weights** (optional) - **T2**:
  weights to optimize.
* **__group_97__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_97__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_97__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_97__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_98__weights** (optional) - **T2**:
  weights to optimize.
* **__group_98__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_98__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_98__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_98__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_99__weights** (optional) - **T2**:
  weights to optimize.
* **__group_99__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_99__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_99__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_99__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_100__weights** (optional) - **T2**:
  weights to optimize.
* **__group_100__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_100__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_100__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_100__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_101__weights** (optional) - **T2**:
  weights to optimize.
* **__group_101__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_101__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_101__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_101__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_102__weights** (optional) - **T2**:
  weights to optimize.
* **__group_102__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_102__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_102__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_102__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_103__weights** (optional) - **T2**:
  weights to optimize.
* **__group_103__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_103__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_103__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_103__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_104__weights** (optional) - **T2**:
  weights to optimize.
* **__group_104__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_104__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_104__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_104__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_105__weights** (optional) - **T2**:
  weights to optimize.
* **__group_105__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_105__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_105__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_105__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_106__weights** (optional) - **T2**:
  weights to optimize.
* **__group_106__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_106__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_106__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_106__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_107__weights** (optional) - **T2**:
  weights to optimize.
* **__group_107__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_107__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_107__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_107__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_108__weights** (optional) - **T2**:
  weights to optimize.
* **__group_108__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_108__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_108__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_108__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_109__weights** (optional) - **T2**:
  weights to optimize.
* **__group_109__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_109__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_109__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_109__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_110__weights** (optional) - **T2**:
  weights to optimize.
* **__group_110__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_110__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_110__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_110__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_111__weights** (optional) - **T2**:
  weights to optimize.
* **__group_111__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_111__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_111__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_111__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_112__weights** (optional) - **T2**:
  weights to optimize.
* **__group_112__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_112__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_112__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_112__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_113__weights** (optional) - **T2**:
  weights to optimize.
* **__group_113__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_113__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_113__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_113__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_114__weights** (optional) - **T2**:
  weights to optimize.
* **__group_114__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_114__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_114__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_114__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_115__weights** (optional) - **T2**:
  weights to optimize.
* **__group_115__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_115__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_115__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_115__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_116__weights** (optional) - **T2**:
  weights to optimize.
* **__group_116__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_116__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_116__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_116__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_117__weights** (optional) - **T2**:
  weights to optimize.
* **__group_117__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_117__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_117__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_117__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_118__weights** (optional) - **T2**:
  weights to optimize.
* **__group_118__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_118__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_118__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_118__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_119__weights** (optional) - **T2**:
  weights to optimize.
* **__group_119__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_119__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_119__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_119__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_120__weights** (optional) - **T2**:
  weights to optimize.
* **__group_120__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_120__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_120__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_120__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_121__weights** (optional) - **T2**:
  weights to optimize.
* **__group_121__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_121__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_121__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_121__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_122__weights** (optional) - **T2**:
  weights to optimize.
* **__group_122__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_122__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_122__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_122__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_123__weights** (optional) - **T2**:
  weights to optimize.
* **__group_123__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_123__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_123__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_123__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_124__weights** (optional) - **T2**:
  weights to optimize.
* **__group_124__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_124__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_124__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_124__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_125__weights** (optional) - **T2**:
  weights to optimize.
* **__group_125__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_125__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_125__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_125__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_126__weights** (optional) - **T2**:
  weights to optimize.
* **__group_126__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_126__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_126__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_126__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_127__weights** (optional) - **T2**:
  weights to optimize.
* **__group_127__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_127__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_127__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_127__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_128__weights** (optional) - **T2**:
  weights to optimize.
* **__group_128__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_128__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_128__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_128__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_129__weights** (optional) - **T2**:
  weights to optimize.
* **__group_129__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_129__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_129__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_129__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_130__weights** (optional) - **T2**:
  weights to optimize.
* **__group_130__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_130__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_130__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_130__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_131__weights** (optional) - **T2**:
  weights to optimize.
* **__group_131__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_131__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_131__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_131__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_132__weights** (optional) - **T2**:
  weights to optimize.
* **__group_132__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_132__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_132__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_132__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_133__weights** (optional) - **T2**:
  weights to optimize.
* **__group_133__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_133__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_133__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_133__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_134__weights** (optional) - **T2**:
  weights to optimize.
* **__group_134__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_134__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_134__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_134__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_135__weights** (optional) - **T2**:
  weights to optimize.
* **__group_135__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_135__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_135__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_135__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_136__weights** (optional) - **T2**:
  weights to optimize.
* **__group_136__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_136__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_136__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_136__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_137__weights** (optional) - **T2**:
  weights to optimize.
* **__group_137__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_137__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_137__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_137__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_138__weights** (optional) - **T2**:
  weights to optimize.
* **__group_138__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_138__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_138__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_138__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_139__weights** (optional) - **T2**:
  weights to optimize.
* **__group_139__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_139__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_139__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_139__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_140__weights** (optional) - **T2**:
  weights to optimize.
* **__group_140__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_140__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_140__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_140__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_141__weights** (optional) - **T2**:
  weights to optimize.
* **__group_141__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_141__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_141__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_141__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_142__weights** (optional) - **T2**:
  weights to optimize.
* **__group_142__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_142__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_142__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_142__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_143__weights** (optional) - **T2**:
  weights to optimize.
* **__group_143__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_143__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_143__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_143__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_144__weights** (optional) - **T2**:
  weights to optimize.
* **__group_144__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_144__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_144__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_144__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_145__weights** (optional) - **T2**:
  weights to optimize.
* **__group_145__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_145__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_145__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_145__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_146__weights** (optional) - **T2**:
  weights to optimize.
* **__group_146__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_146__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_146__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_146__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_147__weights** (optional) - **T2**:
  weights to optimize.
* **__group_147__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_147__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_147__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_147__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_148__weights** (optional) - **T2**:
  weights to optimize.
* **__group_148__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_148__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_148__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_148__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_149__weights** (optional) - **T2**:
  weights to optimize.
* **__group_149__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_149__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_149__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_149__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_150__weights** (optional) - **T2**:
  weights to optimize.
* **__group_150__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_150__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_150__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_150__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_151__weights** (optional) - **T2**:
  weights to optimize.
* **__group_151__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_151__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_151__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_151__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_152__weights** (optional) - **T2**:
  weights to optimize.
* **__group_152__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_152__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_152__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_152__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_153__weights** (optional) - **T2**:
  weights to optimize.
* **__group_153__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_153__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_153__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_153__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_154__weights** (optional) - **T2**:
  weights to optimize.
* **__group_154__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_154__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_154__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_154__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_155__weights** (optional) - **T2**:
  weights to optimize.
* **__group_155__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_155__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_155__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_155__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_156__weights** (optional) - **T2**:
  weights to optimize.
* **__group_156__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_156__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_156__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_156__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_157__weights** (optional) - **T2**:
  weights to optimize.
* **__group_157__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_157__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_157__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_157__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_158__weights** (optional) - **T2**:
  weights to optimize.
* **__group_158__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_158__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_158__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_158__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_159__weights** (optional) - **T2**:
  weights to optimize.
* **__group_159__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_159__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_159__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_159__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_160__weights** (optional) - **T2**:
  weights to optimize.
* **__group_160__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_160__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_160__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_160__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_161__weights** (optional) - **T2**:
  weights to optimize.
* **__group_161__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_161__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_161__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_161__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_162__weights** (optional) - **T2**:
  weights to optimize.
* **__group_162__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_162__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_162__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_162__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_163__weights** (optional) - **T2**:
  weights to optimize.
* **__group_163__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_163__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_163__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_163__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_164__weights** (optional) - **T2**:
  weights to optimize.
* **__group_164__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_164__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_164__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_164__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_165__weights** (optional) - **T2**:
  weights to optimize.
* **__group_165__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_165__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_165__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_165__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_166__weights** (optional) - **T2**:
  weights to optimize.
* **__group_166__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_166__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_166__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_166__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_167__weights** (optional) - **T2**:
  weights to optimize.
* **__group_167__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_167__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_167__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_167__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_168__weights** (optional) - **T2**:
  weights to optimize.
* **__group_168__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_168__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_168__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_168__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_169__weights** (optional) - **T2**:
  weights to optimize.
* **__group_169__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_169__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_169__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_169__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_170__weights** (optional) - **T2**:
  weights to optimize.
* **__group_170__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_170__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_170__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_170__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_171__weights** (optional) - **T2**:
  weights to optimize.
* **__group_171__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_171__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_171__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_171__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_172__weights** (optional) - **T2**:
  weights to optimize.
* **__group_172__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_172__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_172__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_172__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_173__weights** (optional) - **T2**:
  weights to optimize.
* **__group_173__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_173__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_173__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_173__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_174__weights** (optional) - **T2**:
  weights to optimize.
* **__group_174__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_174__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_174__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_174__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_175__weights** (optional) - **T2**:
  weights to optimize.
* **__group_175__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_175__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_175__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_175__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_176__weights** (optional) - **T2**:
  weights to optimize.
* **__group_176__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_176__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_176__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_176__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_177__weights** (optional) - **T2**:
  weights to optimize.
* **__group_177__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_177__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_177__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_177__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_178__weights** (optional) - **T2**:
  weights to optimize.
* **__group_178__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_178__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_178__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_178__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_179__weights** (optional) - **T2**:
  weights to optimize.
* **__group_179__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_179__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_179__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_179__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_180__weights** (optional) - **T2**:
  weights to optimize.
* **__group_180__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_180__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_180__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_180__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_181__weights** (optional) - **T2**:
  weights to optimize.
* **__group_181__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_181__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_181__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_181__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_182__weights** (optional) - **T2**:
  weights to optimize.
* **__group_182__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_182__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_182__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_182__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_183__weights** (optional) - **T2**:
  weights to optimize.
* **__group_183__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_183__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_183__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_183__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_184__weights** (optional) - **T2**:
  weights to optimize.
* **__group_184__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_184__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_184__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_184__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_185__weights** (optional) - **T2**:
  weights to optimize.
* **__group_185__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_185__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_185__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_185__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_186__weights** (optional) - **T2**:
  weights to optimize.
* **__group_186__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_186__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_186__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_186__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_187__weights** (optional) - **T2**:
  weights to optimize.
* **__group_187__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_187__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_187__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_187__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_188__weights** (optional) - **T2**:
  weights to optimize.
* **__group_188__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_188__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_188__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_188__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_189__weights** (optional) - **T2**:
  weights to optimize.
* **__group_189__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_189__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_189__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_189__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_190__weights** (optional) - **T2**:
  weights to optimize.
* **__group_190__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_190__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_190__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_190__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_191__weights** (optional) - **T2**:
  weights to optimize.
* **__group_191__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_191__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_191__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_191__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_192__weights** (optional) - **T2**:
  weights to optimize.
* **__group_192__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_192__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_192__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_192__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_193__weights** (optional) - **T2**:
  weights to optimize.
* **__group_193__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_193__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_193__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_193__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_194__weights** (optional) - **T2**:
  weights to optimize.
* **__group_194__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_194__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_194__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_194__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_195__weights** (optional) - **T2**:
  weights to optimize.
* **__group_195__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_195__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_195__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_195__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_196__weights** (optional) - **T2**:
  weights to optimize.
* **__group_196__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_196__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_196__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_196__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_197__weights** (optional) - **T2**:
  weights to optimize.
* **__group_197__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_197__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_197__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_197__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_198__weights** (optional) - **T2**:
  weights to optimize.
* **__group_198__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_198__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_198__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_198__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_199__weights** (optional) - **T2**:
  weights to optimize.
* **__group_199__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_199__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_199__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_199__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_200__weights** (optional) - **T2**:
  weights to optimize.
* **__group_200__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_200__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_200__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_200__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_201__weights** (optional) - **T2**:
  weights to optimize.
* **__group_201__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_201__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_201__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_201__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_202__weights** (optional) - **T2**:
  weights to optimize.
* **__group_202__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_202__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_202__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_202__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_203__weights** (optional) - **T2**:
  weights to optimize.
* **__group_203__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_203__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_203__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_203__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_204__weights** (optional) - **T2**:
  weights to optimize.
* **__group_204__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_204__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_204__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_204__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_205__weights** (optional) - **T2**:
  weights to optimize.
* **__group_205__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_205__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_205__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_205__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_206__weights** (optional) - **T2**:
  weights to optimize.
* **__group_206__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_206__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_206__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_206__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_207__weights** (optional) - **T2**:
  weights to optimize.
* **__group_207__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_207__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_207__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_207__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_208__weights** (optional) - **T2**:
  weights to optimize.
* **__group_208__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_208__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_208__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_208__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_209__weights** (optional) - **T2**:
  weights to optimize.
* **__group_209__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_209__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_209__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_209__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_210__weights** (optional) - **T2**:
  weights to optimize.
* **__group_210__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_210__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_210__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_210__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_211__weights** (optional) - **T2**:
  weights to optimize.
* **__group_211__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_211__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_211__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_211__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_212__weights** (optional) - **T2**:
  weights to optimize.
* **__group_212__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_212__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_212__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_212__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_213__weights** (optional) - **T2**:
  weights to optimize.
* **__group_213__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_213__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_213__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_213__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_214__weights** (optional) - **T2**:
  weights to optimize.
* **__group_214__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_214__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_214__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_214__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_215__weights** (optional) - **T2**:
  weights to optimize.
* **__group_215__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_215__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_215__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_215__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_216__weights** (optional) - **T2**:
  weights to optimize.
* **__group_216__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_216__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_216__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_216__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_217__weights** (optional) - **T2**:
  weights to optimize.
* **__group_217__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_217__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_217__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_217__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_218__weights** (optional) - **T2**:
  weights to optimize.
* **__group_218__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_218__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_218__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_218__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_219__weights** (optional) - **T2**:
  weights to optimize.
* **__group_219__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_219__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_219__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_219__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_220__weights** (optional) - **T2**:
  weights to optimize.
* **__group_220__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_220__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_220__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_220__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_221__weights** (optional) - **T2**:
  weights to optimize.
* **__group_221__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_221__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_221__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_221__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_222__weights** (optional) - **T2**:
  weights to optimize.
* **__group_222__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_222__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_222__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_222__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_223__weights** (optional) - **T2**:
  weights to optimize.
* **__group_223__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_223__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_223__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_223__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_224__weights** (optional) - **T2**:
  weights to optimize.
* **__group_224__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_224__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_224__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_224__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_225__weights** (optional) - **T2**:
  weights to optimize.
* **__group_225__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_225__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_225__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_225__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_226__weights** (optional) - **T2**:
  weights to optimize.
* **__group_226__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_226__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_226__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_226__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_227__weights** (optional) - **T2**:
  weights to optimize.
* **__group_227__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_227__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_227__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_227__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_228__weights** (optional) - **T2**:
  weights to optimize.
* **__group_228__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_228__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_228__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_228__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_229__weights** (optional) - **T2**:
  weights to optimize.
* **__group_229__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_229__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_229__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_229__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_230__weights** (optional) - **T2**:
  weights to optimize.
* **__group_230__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_230__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_230__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_230__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_231__weights** (optional) - **T2**:
  weights to optimize.
* **__group_231__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_231__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_231__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_231__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_232__weights** (optional) - **T2**:
  weights to optimize.
* **__group_232__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_232__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_232__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_232__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_233__weights** (optional) - **T2**:
  weights to optimize.
* **__group_233__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_233__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_233__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_233__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_234__weights** (optional) - **T2**:
  weights to optimize.
* **__group_234__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_234__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_234__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_234__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_235__weights** (optional) - **T2**:
  weights to optimize.
* **__group_235__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_235__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_235__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_235__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_236__weights** (optional) - **T2**:
  weights to optimize.
* **__group_236__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_236__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_236__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_236__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_237__weights** (optional) - **T2**:
  weights to optimize.
* **__group_237__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_237__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_237__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_237__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_238__weights** (optional) - **T2**:
  weights to optimize.
* **__group_238__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_238__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_238__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_238__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_239__weights** (optional) - **T2**:
  weights to optimize.
* **__group_239__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_239__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_239__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_239__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_240__weights** (optional) - **T2**:
  weights to optimize.
* **__group_240__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_240__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_240__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_240__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_241__weights** (optional) - **T2**:
  weights to optimize.
* **__group_241__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_241__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_241__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_241__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_242__weights** (optional) - **T2**:
  weights to optimize.
* **__group_242__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_242__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_242__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_242__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_243__weights** (optional) - **T2**:
  weights to optimize.
* **__group_243__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_243__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_243__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_243__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_244__weights** (optional) - **T2**:
  weights to optimize.
* **__group_244__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_244__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_244__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_244__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_245__weights** (optional) - **T2**:
  weights to optimize.
* **__group_245__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_245__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_245__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_245__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_246__weights** (optional) - **T2**:
  weights to optimize.
* **__group_246__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_246__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_246__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_246__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_247__weights** (optional) - **T2**:
  weights to optimize.
* **__group_247__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_247__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_247__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_247__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_248__weights** (optional) - **T2**:
  weights to optimize.
* **__group_248__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_248__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_248__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_248__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_249__weights** (optional) - **T2**:
  weights to optimize.
* **__group_249__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_249__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_249__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_249__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_250__weights** (optional) - **T2**:
  weights to optimize.
* **__group_250__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_250__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_250__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_250__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_251__weights** (optional) - **T2**:
  weights to optimize.
* **__group_251__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_251__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_251__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_251__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_252__weights** (optional) - **T2**:
  weights to optimize.
* **__group_252__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_252__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_252__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_252__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_253__weights** (optional) - **T2**:
  weights to optimize.
* **__group_253__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_253__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_253__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_253__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_254__weights** (optional) - **T2**:
  weights to optimize.
* **__group_254__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_254__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_254__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_254__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_255__weights** (optional) - **T2**:
  weights to optimize.
* **__group_255__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_255__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_255__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_255__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_256__weights** (optional) - **T2**:
  weights to optimize.
* **__group_256__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_256__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_256__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_256__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_257__weights** (optional) - **T2**:
  weights to optimize.
* **__group_257__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_257__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_257__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_257__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_258__weights** (optional) - **T2**:
  weights to optimize.
* **__group_258__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_258__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_258__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_258__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_259__weights** (optional) - **T2**:
  weights to optimize.
* **__group_259__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_259__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_259__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_259__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_260__weights** (optional) - **T2**:
  weights to optimize.
* **__group_260__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_260__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_260__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_260__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_261__weights** (optional) - **T2**:
  weights to optimize.
* **__group_261__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_261__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_261__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_261__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_262__weights** (optional) - **T2**:
  weights to optimize.
* **__group_262__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_262__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_262__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_262__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_263__weights** (optional) - **T2**:
  weights to optimize.
* **__group_263__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_263__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_263__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_263__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_264__weights** (optional) - **T2**:
  weights to optimize.
* **__group_264__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_264__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_264__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_264__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_265__weights** (optional) - **T2**:
  weights to optimize.
* **__group_265__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_265__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_265__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_265__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_266__weights** (optional) - **T2**:
  weights to optimize.
* **__group_266__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_266__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_266__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_266__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_267__weights** (optional) - **T2**:
  weights to optimize.
* **__group_267__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_267__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_267__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_267__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_268__weights** (optional) - **T2**:
  weights to optimize.
* **__group_268__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_268__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_268__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_268__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_269__weights** (optional) - **T2**:
  weights to optimize.
* **__group_269__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_269__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_269__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_269__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_270__weights** (optional) - **T2**:
  weights to optimize.
* **__group_270__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_270__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_270__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_270__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_271__weights** (optional) - **T2**:
  weights to optimize.
* **__group_271__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_271__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_271__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_271__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_272__weights** (optional) - **T2**:
  weights to optimize.
* **__group_272__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_272__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_272__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_272__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_273__weights** (optional) - **T2**:
  weights to optimize.
* **__group_273__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_273__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_273__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_273__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_274__weights** (optional) - **T2**:
  weights to optimize.
* **__group_274__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_274__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_274__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_274__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_275__weights** (optional) - **T2**:
  weights to optimize.
* **__group_275__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_275__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_275__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_275__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_276__weights** (optional) - **T2**:
  weights to optimize.
* **__group_276__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_276__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_276__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_276__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_277__weights** (optional) - **T2**:
  weights to optimize.
* **__group_277__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_277__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_277__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_277__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_278__weights** (optional) - **T2**:
  weights to optimize.
* **__group_278__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_278__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_278__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_278__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_279__weights** (optional) - **T2**:
  weights to optimize.
* **__group_279__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_279__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_279__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_279__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_280__weights** (optional) - **T2**:
  weights to optimize.
* **__group_280__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_280__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_280__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_280__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_281__weights** (optional) - **T2**:
  weights to optimize.
* **__group_281__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_281__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_281__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_281__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_282__weights** (optional) - **T2**:
  weights to optimize.
* **__group_282__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_282__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_282__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_282__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_283__weights** (optional) - **T2**:
  weights to optimize.
* **__group_283__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_283__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_283__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_283__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_284__weights** (optional) - **T2**:
  weights to optimize.
* **__group_284__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_284__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_284__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_284__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_285__weights** (optional) - **T2**:
  weights to optimize.
* **__group_285__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_285__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_285__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_285__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_286__weights** (optional) - **T2**:
  weights to optimize.
* **__group_286__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_286__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_286__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_286__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_287__weights** (optional) - **T2**:
  weights to optimize.
* **__group_287__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_287__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_287__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_287__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_288__weights** (optional) - **T2**:
  weights to optimize.
* **__group_288__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_288__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_288__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_288__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_289__weights** (optional) - **T2**:
  weights to optimize.
* **__group_289__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_289__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_289__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_289__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_290__weights** (optional) - **T2**:
  weights to optimize.
* **__group_290__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_290__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_290__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_290__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_291__weights** (optional) - **T2**:
  weights to optimize.
* **__group_291__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_291__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_291__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_291__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_292__weights** (optional) - **T2**:
  weights to optimize.
* **__group_292__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_292__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_292__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_292__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_293__weights** (optional) - **T2**:
  weights to optimize.
* **__group_293__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_293__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_293__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_293__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_294__weights** (optional) - **T2**:
  weights to optimize.
* **__group_294__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_294__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_294__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_294__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_295__weights** (optional) - **T2**:
  weights to optimize.
* **__group_295__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_295__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_295__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_295__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_296__weights** (optional) - **T2**:
  weights to optimize.
* **__group_296__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_296__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_296__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_296__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_297__weights** (optional) - **T2**:
  weights to optimize.
* **__group_297__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_297__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_297__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_297__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_298__weights** (optional) - **T2**:
  weights to optimize.
* **__group_298__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_298__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_298__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_298__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_299__weights** (optional) - **T2**:
  weights to optimize.
* **__group_299__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_299__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_299__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_299__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_300__weights** (optional) - **T2**:
  weights to optimize.
* **__group_300__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_300__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_300__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_300__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_301__weights** (optional) - **T2**:
  weights to optimize.
* **__group_301__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_301__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_301__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_301__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_302__weights** (optional) - **T2**:
  weights to optimize.
* **__group_302__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_302__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_302__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_302__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_303__weights** (optional) - **T2**:
  weights to optimize.
* **__group_303__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_303__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_303__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_303__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_304__weights** (optional) - **T2**:
  weights to optimize.
* **__group_304__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_304__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_304__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_304__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_305__weights** (optional) - **T2**:
  weights to optimize.
* **__group_305__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_305__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_305__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_305__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_306__weights** (optional) - **T2**:
  weights to optimize.
* **__group_306__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_306__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_306__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_306__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_307__weights** (optional) - **T2**:
  weights to optimize.
* **__group_307__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_307__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_307__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_307__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_308__weights** (optional) - **T2**:
  weights to optimize.
* **__group_308__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_308__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_308__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_308__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_309__weights** (optional) - **T2**:
  weights to optimize.
* **__group_309__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_309__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_309__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_309__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_310__weights** (optional) - **T2**:
  weights to optimize.
* **__group_310__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_310__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_310__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_310__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_311__weights** (optional) - **T2**:
  weights to optimize.
* **__group_311__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_311__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_311__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_311__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_312__weights** (optional) - **T2**:
  weights to optimize.
* **__group_312__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_312__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_312__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_312__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_313__weights** (optional) - **T2**:
  weights to optimize.
* **__group_313__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_313__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_313__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_313__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_314__weights** (optional) - **T2**:
  weights to optimize.
* **__group_314__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_314__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_314__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_314__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_315__weights** (optional) - **T2**:
  weights to optimize.
* **__group_315__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_315__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_315__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_315__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_316__weights** (optional) - **T2**:
  weights to optimize.
* **__group_316__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_316__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_316__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_316__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_317__weights** (optional) - **T2**:
  weights to optimize.
* **__group_317__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_317__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_317__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_317__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_318__weights** (optional) - **T2**:
  weights to optimize.
* **__group_318__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_318__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_318__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_318__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_319__weights** (optional) - **T2**:
  weights to optimize.
* **__group_319__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_319__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_319__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_319__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_320__weights** (optional) - **T2**:
  weights to optimize.
* **__group_320__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_320__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_320__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_320__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_321__weights** (optional) - **T2**:
  weights to optimize.
* **__group_321__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_321__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_321__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_321__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_322__weights** (optional) - **T2**:
  weights to optimize.
* **__group_322__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_322__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_322__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_322__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_323__weights** (optional) - **T2**:
  weights to optimize.
* **__group_323__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_323__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_323__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_323__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_324__weights** (optional) - **T2**:
  weights to optimize.
* **__group_324__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_324__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_324__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_324__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_325__weights** (optional) - **T2**:
  weights to optimize.
* **__group_325__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_325__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_325__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_325__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_326__weights** (optional) - **T2**:
  weights to optimize.
* **__group_326__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_326__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_326__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_326__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_327__weights** (optional) - **T2**:
  weights to optimize.
* **__group_327__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_327__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_327__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_327__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_328__weights** (optional) - **T2**:
  weights to optimize.
* **__group_328__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_328__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_328__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_328__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_329__weights** (optional) - **T2**:
  weights to optimize.
* **__group_329__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_329__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_329__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_329__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_330__weights** (optional) - **T2**:
  weights to optimize.
* **__group_330__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_330__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_330__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_330__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_331__weights** (optional) - **T2**:
  weights to optimize.
* **__group_331__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_331__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_331__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_331__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_332__weights** (optional) - **T2**:
  weights to optimize.
* **__group_332__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_332__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_332__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_332__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_333__weights** (optional) - **T2**:
  weights to optimize.
* **__group_333__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_333__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_333__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_333__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_334__weights** (optional) - **T2**:
  weights to optimize.
* **__group_334__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_334__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_334__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_334__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_335__weights** (optional) - **T2**:
  weights to optimize.
* **__group_335__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_335__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_335__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_335__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_336__weights** (optional) - **T2**:
  weights to optimize.
* **__group_336__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_336__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_336__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_336__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_337__weights** (optional) - **T2**:
  weights to optimize.
* **__group_337__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_337__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_337__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_337__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_338__weights** (optional) - **T2**:
  weights to optimize.
* **__group_338__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_338__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_338__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_338__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_339__weights** (optional) - **T2**:
  weights to optimize.
* **__group_339__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_339__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_339__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_339__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_340__weights** (optional) - **T2**:
  weights to optimize.
* **__group_340__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_340__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_340__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_340__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_341__weights** (optional) - **T2**:
  weights to optimize.
* **__group_341__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_341__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_341__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_341__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_342__weights** (optional) - **T2**:
  weights to optimize.
* **__group_342__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_342__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_342__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_342__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_343__weights** (optional) - **T2**:
  weights to optimize.
* **__group_343__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_343__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_343__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_343__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_344__weights** (optional) - **T2**:
  weights to optimize.
* **__group_344__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_344__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_344__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_344__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_345__weights** (optional) - **T2**:
  weights to optimize.
* **__group_345__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_345__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_345__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_345__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_346__weights** (optional) - **T2**:
  weights to optimize.
* **__group_346__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_346__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_346__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_346__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_347__weights** (optional) - **T2**:
  weights to optimize.
* **__group_347__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_347__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_347__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_347__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_348__weights** (optional) - **T2**:
  weights to optimize.
* **__group_348__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_348__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_348__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_348__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_349__weights** (optional) - **T2**:
  weights to optimize.
* **__group_349__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_349__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_349__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_349__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_350__weights** (optional) - **T2**:
  weights to optimize.
* **__group_350__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_350__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_350__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_350__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_351__weights** (optional) - **T2**:
  weights to optimize.
* **__group_351__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_351__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_351__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_351__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_352__weights** (optional) - **T2**:
  weights to optimize.
* **__group_352__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_352__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_352__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_352__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_353__weights** (optional) - **T2**:
  weights to optimize.
* **__group_353__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_353__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_353__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_353__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_354__weights** (optional) - **T2**:
  weights to optimize.
* **__group_354__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_354__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_354__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_354__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_355__weights** (optional) - **T2**:
  weights to optimize.
* **__group_355__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_355__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_355__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_355__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_356__weights** (optional) - **T2**:
  weights to optimize.
* **__group_356__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_356__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_356__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_356__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_357__weights** (optional) - **T2**:
  weights to optimize.
* **__group_357__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_357__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_357__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_357__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_358__weights** (optional) - **T2**:
  weights to optimize.
* **__group_358__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_358__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_358__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_358__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_359__weights** (optional) - **T2**:
  weights to optimize.
* **__group_359__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_359__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_359__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_359__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_360__weights** (optional) - **T2**:
  weights to optimize.
* **__group_360__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_360__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_360__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_360__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_361__weights** (optional) - **T2**:
  weights to optimize.
* **__group_361__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_361__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_361__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_361__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_362__weights** (optional) - **T2**:
  weights to optimize.
* **__group_362__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_362__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_362__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_362__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_363__weights** (optional) - **T2**:
  weights to optimize.
* **__group_363__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_363__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_363__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_363__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_364__weights** (optional) - **T2**:
  weights to optimize.
* **__group_364__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_364__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_364__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_364__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_365__weights** (optional) - **T2**:
  weights to optimize.
* **__group_365__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_365__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_365__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_365__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_366__weights** (optional) - **T2**:
  weights to optimize.
* **__group_366__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_366__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_366__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_366__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_367__weights** (optional) - **T2**:
  weights to optimize.
* **__group_367__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_367__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_367__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_367__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_368__weights** (optional) - **T2**:
  weights to optimize.
* **__group_368__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_368__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_368__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_368__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_369__weights** (optional) - **T2**:
  weights to optimize.
* **__group_369__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_369__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_369__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_369__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_370__weights** (optional) - **T2**:
  weights to optimize.
* **__group_370__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_370__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_370__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_370__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_371__weights** (optional) - **T2**:
  weights to optimize.
* **__group_371__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_371__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_371__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_371__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_372__weights** (optional) - **T2**:
  weights to optimize.
* **__group_372__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_372__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_372__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_372__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_373__weights** (optional) - **T2**:
  weights to optimize.
* **__group_373__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_373__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_373__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_373__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_374__weights** (optional) - **T2**:
  weights to optimize.
* **__group_374__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_374__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_374__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_374__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_375__weights** (optional) - **T2**:
  weights to optimize.
* **__group_375__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_375__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_375__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_375__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_376__weights** (optional) - **T2**:
  weights to optimize.
* **__group_376__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_376__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_376__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_376__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_377__weights** (optional) - **T2**:
  weights to optimize.
* **__group_377__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_377__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_377__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_377__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_378__weights** (optional) - **T2**:
  weights to optimize.
* **__group_378__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_378__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_378__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_378__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_379__weights** (optional) - **T2**:
  weights to optimize.
* **__group_379__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_379__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_379__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_379__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_380__weights** (optional) - **T2**:
  weights to optimize.
* **__group_380__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_380__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_380__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_380__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_381__weights** (optional) - **T2**:
  weights to optimize.
* **__group_381__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_381__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_381__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_381__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_382__weights** (optional) - **T2**:
  weights to optimize.
* **__group_382__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_382__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_382__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_382__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_383__weights** (optional) - **T2**:
  weights to optimize.
* **__group_383__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_383__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_383__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_383__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_384__weights** (optional) - **T2**:
  weights to optimize.
* **__group_384__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_384__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_384__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_384__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_385__weights** (optional) - **T2**:
  weights to optimize.
* **__group_385__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_385__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_385__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_385__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_386__weights** (optional) - **T2**:
  weights to optimize.
* **__group_386__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_386__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_386__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_386__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_387__weights** (optional) - **T2**:
  weights to optimize.
* **__group_387__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_387__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_387__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_387__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_388__weights** (optional) - **T2**:
  weights to optimize.
* **__group_388__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_388__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_388__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_388__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_389__weights** (optional) - **T2**:
  weights to optimize.
* **__group_389__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_389__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_389__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_389__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_390__weights** (optional) - **T2**:
  weights to optimize.
* **__group_390__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_390__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_390__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_390__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_391__weights** (optional) - **T2**:
  weights to optimize.
* **__group_391__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_391__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_391__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_391__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_392__weights** (optional) - **T2**:
  weights to optimize.
* **__group_392__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_392__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_392__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_392__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_393__weights** (optional) - **T2**:
  weights to optimize.
* **__group_393__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_393__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_393__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_393__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_394__weights** (optional) - **T2**:
  weights to optimize.
* **__group_394__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_394__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_394__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_394__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_395__weights** (optional) - **T2**:
  weights to optimize.
* **__group_395__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_395__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_395__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_395__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_396__weights** (optional) - **T2**:
  weights to optimize.
* **__group_396__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_396__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_396__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_396__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_397__weights** (optional) - **T2**:
  weights to optimize.
* **__group_397__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_397__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_397__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_397__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_398__weights** (optional) - **T2**:
  weights to optimize.
* **__group_398__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_398__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_398__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_398__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_399__weights** (optional) - **T2**:
  weights to optimize.
* **__group_399__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_399__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_399__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_399__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_400__weights** (optional) - **T2**:
  weights to optimize.
* **__group_400__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_400__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_400__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_400__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_401__weights** (optional) - **T2**:
  weights to optimize.
* **__group_401__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_401__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_401__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_401__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_402__weights** (optional) - **T2**:
  weights to optimize.
* **__group_402__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_402__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_402__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_402__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_403__weights** (optional) - **T2**:
  weights to optimize.
* **__group_403__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_403__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_403__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_403__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_404__weights** (optional) - **T2**:
  weights to optimize.
* **__group_404__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_404__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_404__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_404__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_405__weights** (optional) - **T2**:
  weights to optimize.
* **__group_405__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_405__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_405__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_405__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_406__weights** (optional) - **T2**:
  weights to optimize.
* **__group_406__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_406__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_406__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_406__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_407__weights** (optional) - **T2**:
  weights to optimize.
* **__group_407__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_407__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_407__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_407__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_408__weights** (optional) - **T2**:
  weights to optimize.
* **__group_408__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_408__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_408__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_408__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_409__weights** (optional) - **T2**:
  weights to optimize.
* **__group_409__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_409__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_409__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_409__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_410__weights** (optional) - **T2**:
  weights to optimize.
* **__group_410__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_410__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_410__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_410__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_411__weights** (optional) - **T2**:
  weights to optimize.
* **__group_411__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_411__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_411__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_411__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_412__weights** (optional) - **T2**:
  weights to optimize.
* **__group_412__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_412__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_412__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_412__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_413__weights** (optional) - **T2**:
  weights to optimize.
* **__group_413__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_413__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_413__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_413__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_414__weights** (optional) - **T2**:
  weights to optimize.
* **__group_414__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_414__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_414__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_414__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_415__weights** (optional) - **T2**:
  weights to optimize.
* **__group_415__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_415__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_415__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_415__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_416__weights** (optional) - **T2**:
  weights to optimize.
* **__group_416__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_416__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_416__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_416__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_417__weights** (optional) - **T2**:
  weights to optimize.
* **__group_417__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_417__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_417__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_417__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_418__weights** (optional) - **T2**:
  weights to optimize.
* **__group_418__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_418__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_418__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_418__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_419__weights** (optional) - **T2**:
  weights to optimize.
* **__group_419__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_419__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_419__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_419__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_420__weights** (optional) - **T2**:
  weights to optimize.
* **__group_420__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_420__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_420__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_420__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_421__weights** (optional) - **T2**:
  weights to optimize.
* **__group_421__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_421__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_421__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_421__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_422__weights** (optional) - **T2**:
  weights to optimize.
* **__group_422__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_422__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_422__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_422__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_423__weights** (optional) - **T2**:
  weights to optimize.
* **__group_423__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_423__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_423__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_423__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_424__weights** (optional) - **T2**:
  weights to optimize.
* **__group_424__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_424__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_424__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_424__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_425__weights** (optional) - **T2**:
  weights to optimize.
* **__group_425__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_425__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_425__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_425__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_426__weights** (optional) - **T2**:
  weights to optimize.
* **__group_426__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_426__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_426__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_426__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_427__weights** (optional) - **T2**:
  weights to optimize.
* **__group_427__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_427__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_427__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_427__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_428__weights** (optional) - **T2**:
  weights to optimize.
* **__group_428__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_428__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_428__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_428__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_429__weights** (optional) - **T2**:
  weights to optimize.
* **__group_429__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_429__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_429__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_429__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_430__weights** (optional) - **T2**:
  weights to optimize.
* **__group_430__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_430__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_430__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_430__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_431__weights** (optional) - **T2**:
  weights to optimize.
* **__group_431__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_431__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_431__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_431__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_432__weights** (optional) - **T2**:
  weights to optimize.
* **__group_432__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_432__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_432__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_432__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_433__weights** (optional) - **T2**:
  weights to optimize.
* **__group_433__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_433__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_433__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_433__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_434__weights** (optional) - **T2**:
  weights to optimize.
* **__group_434__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_434__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_434__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_434__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_435__weights** (optional) - **T2**:
  weights to optimize.
* **__group_435__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_435__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_435__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_435__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_436__weights** (optional) - **T2**:
  weights to optimize.
* **__group_436__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_436__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_436__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_436__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_437__weights** (optional) - **T2**:
  weights to optimize.
* **__group_437__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_437__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_437__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_437__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_438__weights** (optional) - **T2**:
  weights to optimize.
* **__group_438__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_438__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_438__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_438__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_439__weights** (optional) - **T2**:
  weights to optimize.
* **__group_439__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_439__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_439__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_439__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_440__weights** (optional) - **T2**:
  weights to optimize.
* **__group_440__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_440__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_440__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_440__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_441__weights** (optional) - **T2**:
  weights to optimize.
* **__group_441__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_441__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_441__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_441__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_442__weights** (optional) - **T2**:
  weights to optimize.
* **__group_442__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_442__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_442__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_442__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_443__weights** (optional) - **T2**:
  weights to optimize.
* **__group_443__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_443__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_443__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_443__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_444__weights** (optional) - **T2**:
  weights to optimize.
* **__group_444__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_444__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_444__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_444__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_445__weights** (optional) - **T2**:
  weights to optimize.
* **__group_445__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_445__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_445__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_445__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_446__weights** (optional) - **T2**:
  weights to optimize.
* **__group_446__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_446__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_446__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_446__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_447__weights** (optional) - **T2**:
  weights to optimize.
* **__group_447__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_447__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_447__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_447__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_448__weights** (optional) - **T2**:
  weights to optimize.
* **__group_448__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_448__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_448__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_448__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_449__weights** (optional) - **T2**:
  weights to optimize.
* **__group_449__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_449__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_449__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_449__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_450__weights** (optional) - **T2**:
  weights to optimize.
* **__group_450__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_450__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_450__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_450__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_451__weights** (optional) - **T2**:
  weights to optimize.
* **__group_451__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_451__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_451__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_451__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_452__weights** (optional) - **T2**:
  weights to optimize.
* **__group_452__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_452__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_452__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_452__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_453__weights** (optional) - **T2**:
  weights to optimize.
* **__group_453__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_453__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_453__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_453__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_454__weights** (optional) - **T2**:
  weights to optimize.
* **__group_454__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_454__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_454__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_454__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_455__weights** (optional) - **T2**:
  weights to optimize.
* **__group_455__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_455__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_455__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_455__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_456__weights** (optional) - **T2**:
  weights to optimize.
* **__group_456__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_456__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_456__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_456__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_457__weights** (optional) - **T2**:
  weights to optimize.
* **__group_457__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_457__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_457__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_457__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_458__weights** (optional) - **T2**:
  weights to optimize.
* **__group_458__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_458__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_458__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_458__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_459__weights** (optional) - **T2**:
  weights to optimize.
* **__group_459__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_459__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_459__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_459__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_460__weights** (optional) - **T2**:
  weights to optimize.
* **__group_460__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_460__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_460__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_460__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_461__weights** (optional) - **T2**:
  weights to optimize.
* **__group_461__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_461__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_461__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_461__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_462__weights** (optional) - **T2**:
  weights to optimize.
* **__group_462__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_462__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_462__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_462__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_463__weights** (optional) - **T2**:
  weights to optimize.
* **__group_463__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_463__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_463__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_463__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_464__weights** (optional) - **T2**:
  weights to optimize.
* **__group_464__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_464__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_464__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_464__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_465__weights** (optional) - **T2**:
  weights to optimize.
* **__group_465__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_465__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_465__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_465__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_466__weights** (optional) - **T2**:
  weights to optimize.
* **__group_466__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_466__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_466__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_466__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_467__weights** (optional) - **T2**:
  weights to optimize.
* **__group_467__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_467__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_467__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_467__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_468__weights** (optional) - **T2**:
  weights to optimize.
* **__group_468__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_468__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_468__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_468__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_469__weights** (optional) - **T2**:
  weights to optimize.
* **__group_469__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_469__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_469__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_469__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_470__weights** (optional) - **T2**:
  weights to optimize.
* **__group_470__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_470__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_470__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_470__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_471__weights** (optional) - **T2**:
  weights to optimize.
* **__group_471__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_471__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_471__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_471__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_472__weights** (optional) - **T2**:
  weights to optimize.
* **__group_472__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_472__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_472__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_472__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_473__weights** (optional) - **T2**:
  weights to optimize.
* **__group_473__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_473__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_473__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_473__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_474__weights** (optional) - **T2**:
  weights to optimize.
* **__group_474__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_474__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_474__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_474__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_475__weights** (optional) - **T2**:
  weights to optimize.
* **__group_475__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_475__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_475__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_475__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_476__weights** (optional) - **T2**:
  weights to optimize.
* **__group_476__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_476__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_476__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_476__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_477__weights** (optional) - **T2**:
  weights to optimize.
* **__group_477__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_477__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_477__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_477__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_478__weights** (optional) - **T2**:
  weights to optimize.
* **__group_478__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_478__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_478__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_478__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_479__weights** (optional) - **T2**:
  weights to optimize.
* **__group_479__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_479__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_479__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_479__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_480__weights** (optional) - **T2**:
  weights to optimize.
* **__group_480__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_480__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_480__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_480__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_481__weights** (optional) - **T2**:
  weights to optimize.
* **__group_481__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_481__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_481__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_481__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_482__weights** (optional) - **T2**:
  weights to optimize.
* **__group_482__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_482__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_482__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_482__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_483__weights** (optional) - **T2**:
  weights to optimize.
* **__group_483__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_483__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_483__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_483__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_484__weights** (optional) - **T2**:
  weights to optimize.
* **__group_484__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_484__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_484__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_484__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_485__weights** (optional) - **T2**:
  weights to optimize.
* **__group_485__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_485__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_485__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_485__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_486__weights** (optional) - **T2**:
  weights to optimize.
* **__group_486__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_486__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_486__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_486__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_487__weights** (optional) - **T2**:
  weights to optimize.
* **__group_487__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_487__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_487__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_487__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_488__weights** (optional) - **T2**:
  weights to optimize.
* **__group_488__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_488__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_488__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_488__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_489__weights** (optional) - **T2**:
  weights to optimize.
* **__group_489__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_489__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_489__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_489__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_490__weights** (optional) - **T2**:
  weights to optimize.
* **__group_490__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_490__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_490__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_490__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_491__weights** (optional) - **T2**:
  weights to optimize.
* **__group_491__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_491__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_491__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_491__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_492__weights** (optional) - **T2**:
  weights to optimize.
* **__group_492__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_492__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_492__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_492__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_493__weights** (optional) - **T2**:
  weights to optimize.
* **__group_493__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_493__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_493__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_493__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_494__weights** (optional) - **T2**:
  weights to optimize.
* **__group_494__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_494__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_494__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_494__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_495__weights** (optional) - **T2**:
  weights to optimize.
* **__group_495__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_495__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_495__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_495__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_496__weights** (optional) - **T2**:
  weights to optimize.
* **__group_496__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_496__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_496__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_496__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_497__weights** (optional) - **T2**:
  weights to optimize.
* **__group_497__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_497__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_497__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_497__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_498__weights** (optional) - **T2**:
  weights to optimize.
* **__group_498__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_498__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_498__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_498__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_499__weights** (optional) - **T2**:
  weights to optimize.
* **__group_499__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_499__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_499__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_499__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_500__weights** (optional) - **T2**:
  weights to optimize.
* **__group_500__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_500__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_500__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_500__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_501__weights** (optional) - **T2**:
  weights to optimize.
* **__group_501__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_501__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_501__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_501__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_502__weights** (optional) - **T2**:
  weights to optimize.
* **__group_502__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_502__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_502__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_502__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_503__weights** (optional) - **T2**:
  weights to optimize.
* **__group_503__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_503__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_503__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_503__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_504__weights** (optional) - **T2**:
  weights to optimize.
* **__group_504__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_504__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_504__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_504__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_505__weights** (optional) - **T2**:
  weights to optimize.
* **__group_505__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_505__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_505__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_505__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_506__weights** (optional) - **T2**:
  weights to optimize.
* **__group_506__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_506__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_506__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_506__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_507__weights** (optional) - **T2**:
  weights to optimize.
* **__group_507__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_507__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_507__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_507__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_508__weights** (optional) - **T2**:
  weights to optimize.
* **__group_508__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_508__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_508__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_508__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_509__weights** (optional) - **T2**:
  weights to optimize.
* **__group_509__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_509__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_509__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_509__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_510__weights** (optional) - **T2**:
  weights to optimize.
* **__group_510__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_510__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_510__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_510__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_511__weights** (optional) - **T2**:
  weights to optimize.
* **__group_511__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_511__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_511__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_511__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_512__weights** (optional) - **T2**:
  weights to optimize.
* **__group_512__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_512__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_512__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_512__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_513__weights** (optional) - **T2**:
  weights to optimize.
* **__group_513__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_513__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_513__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_513__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_514__weights** (optional) - **T2**:
  weights to optimize.
* **__group_514__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_514__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_514__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_514__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_515__weights** (optional) - **T2**:
  weights to optimize.
* **__group_515__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_515__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_515__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_515__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_516__weights** (optional) - **T2**:
  weights to optimize.
* **__group_516__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_516__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_516__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_516__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_517__weights** (optional) - **T2**:
  weights to optimize.
* **__group_517__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_517__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_517__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_517__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_518__weights** (optional) - **T2**:
  weights to optimize.
* **__group_518__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_518__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_518__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_518__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_519__weights** (optional) - **T2**:
  weights to optimize.
* **__group_519__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_519__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_519__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_519__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_520__weights** (optional) - **T2**:
  weights to optimize.
* **__group_520__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_520__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_520__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_520__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_521__weights** (optional) - **T2**:
  weights to optimize.
* **__group_521__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_521__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_521__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_521__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_522__weights** (optional) - **T2**:
  weights to optimize.
* **__group_522__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_522__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_522__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_522__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_523__weights** (optional) - **T2**:
  weights to optimize.
* **__group_523__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_523__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_523__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_523__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_524__weights** (optional) - **T2**:
  weights to optimize.
* **__group_524__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_524__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_524__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_524__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_525__weights** (optional) - **T2**:
  weights to optimize.
* **__group_525__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_525__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_525__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_525__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_526__weights** (optional) - **T2**:
  weights to optimize.
* **__group_526__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_526__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_526__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_526__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_527__weights** (optional) - **T2**:
  weights to optimize.
* **__group_527__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_527__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_527__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_527__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_528__weights** (optional) - **T2**:
  weights to optimize.
* **__group_528__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_528__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_528__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_528__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_529__weights** (optional) - **T2**:
  weights to optimize.
* **__group_529__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_529__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_529__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_529__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_530__weights** (optional) - **T2**:
  weights to optimize.
* **__group_530__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_530__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_530__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_530__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_531__weights** (optional) - **T2**:
  weights to optimize.
* **__group_531__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_531__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_531__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_531__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_532__weights** (optional) - **T2**:
  weights to optimize.
* **__group_532__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_532__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_532__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_532__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_533__weights** (optional) - **T2**:
  weights to optimize.
* **__group_533__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_533__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_533__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_533__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_534__weights** (optional) - **T2**:
  weights to optimize.
* **__group_534__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_534__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_534__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_534__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_535__weights** (optional) - **T2**:
  weights to optimize.
* **__group_535__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_535__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_535__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_535__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_536__weights** (optional) - **T2**:
  weights to optimize.
* **__group_536__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_536__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_536__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_536__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_537__weights** (optional) - **T2**:
  weights to optimize.
* **__group_537__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_537__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_537__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_537__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_538__weights** (optional) - **T2**:
  weights to optimize.
* **__group_538__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_538__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_538__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_538__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_539__weights** (optional) - **T2**:
  weights to optimize.
* **__group_539__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_539__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_539__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_539__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_540__weights** (optional) - **T2**:
  weights to optimize.
* **__group_540__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_540__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_540__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_540__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_541__weights** (optional) - **T2**:
  weights to optimize.
* **__group_541__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_541__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_541__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_541__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_542__weights** (optional) - **T2**:
  weights to optimize.
* **__group_542__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_542__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_542__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_542__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_543__weights** (optional) - **T2**:
  weights to optimize.
* **__group_543__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_543__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_543__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_543__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_544__weights** (optional) - **T2**:
  weights to optimize.
* **__group_544__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_544__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_544__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_544__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_545__weights** (optional) - **T2**:
  weights to optimize.
* **__group_545__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_545__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_545__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_545__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_546__weights** (optional) - **T2**:
  weights to optimize.
* **__group_546__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_546__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_546__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_546__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_547__weights** (optional) - **T2**:
  weights to optimize.
* **__group_547__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_547__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_547__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_547__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_548__weights** (optional) - **T2**:
  weights to optimize.
* **__group_548__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_548__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_548__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_548__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_549__weights** (optional) - **T2**:
  weights to optimize.
* **__group_549__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_549__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_549__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_549__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_550__weights** (optional) - **T2**:
  weights to optimize.
* **__group_550__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_550__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_550__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_550__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_551__weights** (optional) - **T2**:
  weights to optimize.
* **__group_551__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_551__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_551__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_551__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_552__weights** (optional) - **T2**:
  weights to optimize.
* **__group_552__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_552__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_552__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_552__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_553__weights** (optional) - **T2**:
  weights to optimize.
* **__group_553__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_553__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_553__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_553__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_554__weights** (optional) - **T2**:
  weights to optimize.
* **__group_554__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_554__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_554__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_554__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_555__weights** (optional) - **T2**:
  weights to optimize.
* **__group_555__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_555__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_555__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_555__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_556__weights** (optional) - **T2**:
  weights to optimize.
* **__group_556__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_556__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_556__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_556__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_557__weights** (optional) - **T2**:
  weights to optimize.
* **__group_557__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_557__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_557__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_557__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_558__weights** (optional) - **T2**:
  weights to optimize.
* **__group_558__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_558__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_558__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_558__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_559__weights** (optional) - **T2**:
  weights to optimize.
* **__group_559__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_559__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_559__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_559__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_560__weights** (optional) - **T2**:
  weights to optimize.
* **__group_560__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_560__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_560__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_560__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_561__weights** (optional) - **T2**:
  weights to optimize.
* **__group_561__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_561__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_561__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_561__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_562__weights** (optional) - **T2**:
  weights to optimize.
* **__group_562__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_562__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_562__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_562__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_563__weights** (optional) - **T2**:
  weights to optimize.
* **__group_563__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_563__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_563__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_563__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_564__weights** (optional) - **T2**:
  weights to optimize.
* **__group_564__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_564__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_564__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_564__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_565__weights** (optional) - **T2**:
  weights to optimize.
* **__group_565__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_565__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_565__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_565__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_566__weights** (optional) - **T2**:
  weights to optimize.
* **__group_566__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_566__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_566__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_566__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_567__weights** (optional) - **T2**:
  weights to optimize.
* **__group_567__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_567__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_567__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_567__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_568__weights** (optional) - **T2**:
  weights to optimize.
* **__group_568__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_568__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_568__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_568__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_569__weights** (optional) - **T2**:
  weights to optimize.
* **__group_569__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_569__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_569__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_569__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_570__weights** (optional) - **T2**:
  weights to optimize.
* **__group_570__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_570__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_570__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_570__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_571__weights** (optional) - **T2**:
  weights to optimize.
* **__group_571__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_571__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_571__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_571__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_572__weights** (optional) - **T2**:
  weights to optimize.
* **__group_572__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_572__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_572__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_572__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_573__weights** (optional) - **T2**:
  weights to optimize.
* **__group_573__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_573__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_573__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_573__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_574__weights** (optional) - **T2**:
  weights to optimize.
* **__group_574__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_574__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_574__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_574__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_575__weights** (optional) - **T2**:
  weights to optimize.
* **__group_575__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_575__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_575__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_575__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_576__weights** (optional) - **T2**:
  weights to optimize.
* **__group_576__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_576__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_576__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_576__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_577__weights** (optional) - **T2**:
  weights to optimize.
* **__group_577__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_577__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_577__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_577__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_578__weights** (optional) - **T2**:
  weights to optimize.
* **__group_578__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_578__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_578__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_578__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_579__weights** (optional) - **T2**:
  weights to optimize.
* **__group_579__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_579__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_579__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_579__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_580__weights** (optional) - **T2**:
  weights to optimize.
* **__group_580__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_580__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_580__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_580__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_581__weights** (optional) - **T2**:
  weights to optimize.
* **__group_581__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_581__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_581__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_581__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_582__weights** (optional) - **T2**:
  weights to optimize.
* **__group_582__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_582__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_582__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_582__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_583__weights** (optional) - **T2**:
  weights to optimize.
* **__group_583__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_583__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_583__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_583__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_584__weights** (optional) - **T2**:
  weights to optimize.
* **__group_584__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_584__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_584__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_584__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_585__weights** (optional) - **T2**:
  weights to optimize.
* **__group_585__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_585__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_585__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_585__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_586__weights** (optional) - **T2**:
  weights to optimize.
* **__group_586__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_586__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_586__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_586__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_587__weights** (optional) - **T2**:
  weights to optimize.
* **__group_587__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_587__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_587__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_587__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_588__weights** (optional) - **T2**:
  weights to optimize.
* **__group_588__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_588__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_588__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_588__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_589__weights** (optional) - **T2**:
  weights to optimize.
* **__group_589__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_589__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_589__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_589__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_590__weights** (optional) - **T2**:
  weights to optimize.
* **__group_590__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_590__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_590__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_590__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_591__weights** (optional) - **T2**:
  weights to optimize.
* **__group_591__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_591__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_591__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_591__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_592__weights** (optional) - **T2**:
  weights to optimize.
* **__group_592__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_592__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_592__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_592__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_593__weights** (optional) - **T2**:
  weights to optimize.
* **__group_593__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_593__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_593__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_593__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_594__weights** (optional) - **T2**:
  weights to optimize.
* **__group_594__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_594__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_594__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_594__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_595__weights** (optional) - **T2**:
  weights to optimize.
* **__group_595__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_595__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_595__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_595__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_596__weights** (optional) - **T2**:
  weights to optimize.
* **__group_596__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_596__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_596__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_596__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_597__weights** (optional) - **T2**:
  weights to optimize.
* **__group_597__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_597__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_597__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_597__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_598__weights** (optional) - **T2**:
  weights to optimize.
* **__group_598__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_598__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_598__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_598__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_599__weights** (optional) - **T2**:
  weights to optimize.
* **__group_599__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_599__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_599__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_599__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_600__weights** (optional) - **T2**:
  weights to optimize.
* **__group_600__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_600__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_600__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_600__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_601__weights** (optional) - **T2**:
  weights to optimize.
* **__group_601__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_601__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_601__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_601__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_602__weights** (optional) - **T2**:
  weights to optimize.
* **__group_602__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_602__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_602__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_602__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_603__weights** (optional) - **T2**:
  weights to optimize.
* **__group_603__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_603__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_603__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_603__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_604__weights** (optional) - **T2**:
  weights to optimize.
* **__group_604__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_604__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_604__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_604__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_605__weights** (optional) - **T2**:
  weights to optimize.
* **__group_605__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_605__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_605__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_605__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_606__weights** (optional) - **T2**:
  weights to optimize.
* **__group_606__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_606__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_606__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_606__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_607__weights** (optional) - **T2**:
  weights to optimize.
* **__group_607__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_607__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_607__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_607__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_608__weights** (optional) - **T2**:
  weights to optimize.
* **__group_608__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_608__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_608__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_608__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_609__weights** (optional) - **T2**:
  weights to optimize.
* **__group_609__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_609__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_609__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_609__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_610__weights** (optional) - **T2**:
  weights to optimize.
* **__group_610__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_610__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_610__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_610__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_611__weights** (optional) - **T2**:
  weights to optimize.
* **__group_611__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_611__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_611__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_611__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_612__weights** (optional) - **T2**:
  weights to optimize.
* **__group_612__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_612__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_612__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_612__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_613__weights** (optional) - **T2**:
  weights to optimize.
* **__group_613__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_613__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_613__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_613__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_614__weights** (optional) - **T2**:
  weights to optimize.
* **__group_614__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_614__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_614__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_614__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_615__weights** (optional) - **T2**:
  weights to optimize.
* **__group_615__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_615__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_615__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_615__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_616__weights** (optional) - **T2**:
  weights to optimize.
* **__group_616__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_616__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_616__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_616__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_617__weights** (optional) - **T2**:
  weights to optimize.
* **__group_617__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_617__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_617__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_617__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_618__weights** (optional) - **T2**:
  weights to optimize.
* **__group_618__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_618__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_618__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_618__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_619__weights** (optional) - **T2**:
  weights to optimize.
* **__group_619__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_619__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_619__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_619__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_620__weights** (optional) - **T2**:
  weights to optimize.
* **__group_620__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_620__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_620__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_620__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_621__weights** (optional) - **T2**:
  weights to optimize.
* **__group_621__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_621__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_621__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_621__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_622__weights** (optional) - **T2**:
  weights to optimize.
* **__group_622__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_622__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_622__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_622__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_623__weights** (optional) - **T2**:
  weights to optimize.
* **__group_623__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_623__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_623__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_623__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_624__weights** (optional) - **T2**:
  weights to optimize.
* **__group_624__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_624__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_624__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_624__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_625__weights** (optional) - **T2**:
  weights to optimize.
* **__group_625__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_625__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_625__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_625__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_626__weights** (optional) - **T2**:
  weights to optimize.
* **__group_626__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_626__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_626__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_626__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_627__weights** (optional) - **T2**:
  weights to optimize.
* **__group_627__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_627__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_627__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_627__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_628__weights** (optional) - **T2**:
  weights to optimize.
* **__group_628__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_628__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_628__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_628__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_629__weights** (optional) - **T2**:
  weights to optimize.
* **__group_629__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_629__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_629__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_629__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_630__weights** (optional) - **T2**:
  weights to optimize.
* **__group_630__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_630__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_630__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_630__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_631__weights** (optional) - **T2**:
  weights to optimize.
* **__group_631__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_631__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_631__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_631__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_632__weights** (optional) - **T2**:
  weights to optimize.
* **__group_632__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_632__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_632__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_632__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_633__weights** (optional) - **T2**:
  weights to optimize.
* **__group_633__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_633__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_633__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_633__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_634__weights** (optional) - **T2**:
  weights to optimize.
* **__group_634__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_634__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_634__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_634__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_635__weights** (optional) - **T2**:
  weights to optimize.
* **__group_635__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_635__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_635__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_635__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_636__weights** (optional) - **T2**:
  weights to optimize.
* **__group_636__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_636__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_636__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_636__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_637__weights** (optional) - **T2**:
  weights to optimize.
* **__group_637__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_637__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_637__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_637__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_638__weights** (optional) - **T2**:
  weights to optimize.
* **__group_638__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_638__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_638__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_638__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_639__weights** (optional) - **T2**:
  weights to optimize.
* **__group_639__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_639__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_639__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_639__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_640__weights** (optional) - **T2**:
  weights to optimize.
* **__group_640__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_640__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_640__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_640__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_641__weights** (optional) - **T2**:
  weights to optimize.
* **__group_641__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_641__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_641__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_641__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_642__weights** (optional) - **T2**:
  weights to optimize.
* **__group_642__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_642__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_642__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_642__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_643__weights** (optional) - **T2**:
  weights to optimize.
* **__group_643__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_643__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_643__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_643__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_644__weights** (optional) - **T2**:
  weights to optimize.
* **__group_644__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_644__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_644__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_644__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_645__weights** (optional) - **T2**:
  weights to optimize.
* **__group_645__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_645__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_645__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_645__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_646__weights** (optional) - **T2**:
  weights to optimize.
* **__group_646__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_646__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_646__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_646__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_647__weights** (optional) - **T2**:
  weights to optimize.
* **__group_647__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_647__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_647__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_647__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_648__weights** (optional) - **T2**:
  weights to optimize.
* **__group_648__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_648__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_648__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_648__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_649__weights** (optional) - **T2**:
  weights to optimize.
* **__group_649__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_649__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_649__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_649__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_650__weights** (optional) - **T2**:
  weights to optimize.
* **__group_650__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_650__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_650__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_650__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_651__weights** (optional) - **T2**:
  weights to optimize.
* **__group_651__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_651__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_651__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_651__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_652__weights** (optional) - **T2**:
  weights to optimize.
* **__group_652__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_652__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_652__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_652__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_653__weights** (optional) - **T2**:
  weights to optimize.
* **__group_653__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_653__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_653__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_653__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_654__weights** (optional) - **T2**:
  weights to optimize.
* **__group_654__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_654__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_654__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_654__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_655__weights** (optional) - **T2**:
  weights to optimize.
* **__group_655__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_655__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_655__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_655__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_656__weights** (optional) - **T2**:
  weights to optimize.
* **__group_656__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_656__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_656__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_656__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_657__weights** (optional) - **T2**:
  weights to optimize.
* **__group_657__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_657__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_657__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_657__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_658__weights** (optional) - **T2**:
  weights to optimize.
* **__group_658__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_658__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_658__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_658__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_659__weights** (optional) - **T2**:
  weights to optimize.
* **__group_659__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_659__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_659__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_659__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_660__weights** (optional) - **T2**:
  weights to optimize.
* **__group_660__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_660__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_660__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_660__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_661__weights** (optional) - **T2**:
  weights to optimize.
* **__group_661__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_661__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_661__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_661__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_662__weights** (optional) - **T2**:
  weights to optimize.
* **__group_662__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_662__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_662__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_662__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_663__weights** (optional) - **T2**:
  weights to optimize.
* **__group_663__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_663__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_663__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_663__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_664__weights** (optional) - **T2**:
  weights to optimize.
* **__group_664__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_664__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_664__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_664__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_665__weights** (optional) - **T2**:
  weights to optimize.
* **__group_665__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_665__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_665__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_665__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_666__weights** (optional) - **T2**:
  weights to optimize.
* **__group_666__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_666__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_666__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_666__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_667__weights** (optional) - **T2**:
  weights to optimize.
* **__group_667__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_667__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_667__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_667__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_668__weights** (optional) - **T2**:
  weights to optimize.
* **__group_668__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_668__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_668__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_668__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_669__weights** (optional) - **T2**:
  weights to optimize.
* **__group_669__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_669__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_669__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_669__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_670__weights** (optional) - **T2**:
  weights to optimize.
* **__group_670__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_670__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_670__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_670__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_671__weights** (optional) - **T2**:
  weights to optimize.
* **__group_671__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_671__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_671__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_671__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_672__weights** (optional) - **T2**:
  weights to optimize.
* **__group_672__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_672__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_672__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_672__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_673__weights** (optional) - **T2**:
  weights to optimize.
* **__group_673__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_673__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_673__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_673__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_674__weights** (optional) - **T2**:
  weights to optimize.
* **__group_674__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_674__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_674__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_674__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_675__weights** (optional) - **T2**:
  weights to optimize.
* **__group_675__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_675__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_675__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_675__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_676__weights** (optional) - **T2**:
  weights to optimize.
* **__group_676__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_676__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_676__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_676__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_677__weights** (optional) - **T2**:
  weights to optimize.
* **__group_677__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_677__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_677__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_677__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_678__weights** (optional) - **T2**:
  weights to optimize.
* **__group_678__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_678__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_678__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_678__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_679__weights** (optional) - **T2**:
  weights to optimize.
* **__group_679__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_679__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_679__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_679__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_680__weights** (optional) - **T2**:
  weights to optimize.
* **__group_680__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_680__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_680__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_680__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_681__weights** (optional) - **T2**:
  weights to optimize.
* **__group_681__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_681__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_681__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_681__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_682__weights** (optional) - **T2**:
  weights to optimize.
* **__group_682__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_682__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_682__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_682__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_683__weights** (optional) - **T2**:
  weights to optimize.
* **__group_683__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_683__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_683__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_683__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_684__weights** (optional) - **T2**:
  weights to optimize.
* **__group_684__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_684__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_684__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_684__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_685__weights** (optional) - **T2**:
  weights to optimize.
* **__group_685__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_685__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_685__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_685__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_686__weights** (optional) - **T2**:
  weights to optimize.
* **__group_686__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_686__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_686__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_686__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_687__weights** (optional) - **T2**:
  weights to optimize.
* **__group_687__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_687__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_687__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_687__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_688__weights** (optional) - **T2**:
  weights to optimize.
* **__group_688__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_688__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_688__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_688__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_689__weights** (optional) - **T2**:
  weights to optimize.
* **__group_689__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_689__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_689__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_689__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_690__weights** (optional) - **T2**:
  weights to optimize.
* **__group_690__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_690__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_690__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_690__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_691__weights** (optional) - **T2**:
  weights to optimize.
* **__group_691__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_691__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_691__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_691__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_692__weights** (optional) - **T2**:
  weights to optimize.
* **__group_692__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_692__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_692__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_692__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_693__weights** (optional) - **T2**:
  weights to optimize.
* **__group_693__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_693__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_693__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_693__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_694__weights** (optional) - **T2**:
  weights to optimize.
* **__group_694__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_694__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_694__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_694__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_695__weights** (optional) - **T2**:
  weights to optimize.
* **__group_695__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_695__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_695__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_695__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_696__weights** (optional) - **T2**:
  weights to optimize.
* **__group_696__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_696__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_696__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_696__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_697__weights** (optional) - **T2**:
  weights to optimize.
* **__group_697__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_697__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_697__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_697__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_698__weights** (optional) - **T2**:
  weights to optimize.
* **__group_698__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_698__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_698__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_698__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_699__weights** (optional) - **T2**:
  weights to optimize.
* **__group_699__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_699__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_699__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_699__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_700__weights** (optional) - **T2**:
  weights to optimize.
* **__group_700__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_700__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_700__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_700__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_701__weights** (optional) - **T2**:
  weights to optimize.
* **__group_701__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_701__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_701__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_701__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_702__weights** (optional) - **T2**:
  weights to optimize.
* **__group_702__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_702__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_702__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_702__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_703__weights** (optional) - **T2**:
  weights to optimize.
* **__group_703__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_703__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_703__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_703__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_704__weights** (optional) - **T2**:
  weights to optimize.
* **__group_704__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_704__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_704__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_704__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_705__weights** (optional) - **T2**:
  weights to optimize.
* **__group_705__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_705__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_705__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_705__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_706__weights** (optional) - **T2**:
  weights to optimize.
* **__group_706__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_706__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_706__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_706__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_707__weights** (optional) - **T2**:
  weights to optimize.
* **__group_707__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_707__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_707__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_707__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_708__weights** (optional) - **T2**:
  weights to optimize.
* **__group_708__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_708__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_708__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_708__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_709__weights** (optional) - **T2**:
  weights to optimize.
* **__group_709__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_709__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_709__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_709__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_710__weights** (optional) - **T2**:
  weights to optimize.
* **__group_710__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_710__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_710__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_710__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_711__weights** (optional) - **T2**:
  weights to optimize.
* **__group_711__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_711__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_711__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_711__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_712__weights** (optional) - **T2**:
  weights to optimize.
* **__group_712__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_712__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_712__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_712__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_713__weights** (optional) - **T2**:
  weights to optimize.
* **__group_713__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_713__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_713__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_713__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_714__weights** (optional) - **T2**:
  weights to optimize.
* **__group_714__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_714__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_714__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_714__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_715__weights** (optional) - **T2**:
  weights to optimize.
* **__group_715__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_715__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_715__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_715__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_716__weights** (optional) - **T2**:
  weights to optimize.
* **__group_716__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_716__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_716__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_716__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_717__weights** (optional) - **T2**:
  weights to optimize.
* **__group_717__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_717__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_717__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_717__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_718__weights** (optional) - **T2**:
  weights to optimize.
* **__group_718__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_718__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_718__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_718__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_719__weights** (optional) - **T2**:
  weights to optimize.
* **__group_719__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_719__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_719__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_719__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_720__weights** (optional) - **T2**:
  weights to optimize.
* **__group_720__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_720__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_720__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_720__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_721__weights** (optional) - **T2**:
  weights to optimize.
* **__group_721__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_721__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_721__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_721__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_722__weights** (optional) - **T2**:
  weights to optimize.
* **__group_722__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_722__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_722__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_722__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_723__weights** (optional) - **T2**:
  weights to optimize.
* **__group_723__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_723__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_723__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_723__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_724__weights** (optional) - **T2**:
  weights to optimize.
* **__group_724__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_724__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_724__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_724__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_725__weights** (optional) - **T2**:
  weights to optimize.
* **__group_725__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_725__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_725__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_725__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_726__weights** (optional) - **T2**:
  weights to optimize.
* **__group_726__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_726__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_726__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_726__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_727__weights** (optional) - **T2**:
  weights to optimize.
* **__group_727__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_727__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_727__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_727__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_728__weights** (optional) - **T2**:
  weights to optimize.
* **__group_728__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_728__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_728__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_728__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_729__weights** (optional) - **T2**:
  weights to optimize.
* **__group_729__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_729__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_729__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_729__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_730__weights** (optional) - **T2**:
  weights to optimize.
* **__group_730__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_730__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_730__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_730__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_731__weights** (optional) - **T2**:
  weights to optimize.
* **__group_731__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_731__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_731__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_731__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_732__weights** (optional) - **T2**:
  weights to optimize.
* **__group_732__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_732__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_732__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_732__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_733__weights** (optional) - **T2**:
  weights to optimize.
* **__group_733__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_733__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_733__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_733__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_734__weights** (optional) - **T2**:
  weights to optimize.
* **__group_734__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_734__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_734__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_734__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_735__weights** (optional) - **T2**:
  weights to optimize.
* **__group_735__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_735__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_735__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_735__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_736__weights** (optional) - **T2**:
  weights to optimize.
* **__group_736__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_736__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_736__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_736__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_737__weights** (optional) - **T2**:
  weights to optimize.
* **__group_737__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_737__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_737__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_737__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_738__weights** (optional) - **T2**:
  weights to optimize.
* **__group_738__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_738__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_738__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_738__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_739__weights** (optional) - **T2**:
  weights to optimize.
* **__group_739__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_739__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_739__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_739__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_740__weights** (optional) - **T2**:
  weights to optimize.
* **__group_740__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_740__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_740__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_740__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_741__weights** (optional) - **T2**:
  weights to optimize.
* **__group_741__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_741__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_741__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_741__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_742__weights** (optional) - **T2**:
  weights to optimize.
* **__group_742__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_742__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_742__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_742__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_743__weights** (optional) - **T2**:
  weights to optimize.
* **__group_743__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_743__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_743__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_743__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_744__weights** (optional) - **T2**:
  weights to optimize.
* **__group_744__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_744__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_744__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_744__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_745__weights** (optional) - **T2**:
  weights to optimize.
* **__group_745__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_745__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_745__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_745__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_746__weights** (optional) - **T2**:
  weights to optimize.
* **__group_746__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_746__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_746__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_746__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_747__weights** (optional) - **T2**:
  weights to optimize.
* **__group_747__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_747__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_747__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_747__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_748__weights** (optional) - **T2**:
  weights to optimize.
* **__group_748__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_748__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_748__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_748__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_749__weights** (optional) - **T2**:
  weights to optimize.
* **__group_749__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_749__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_749__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_749__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_750__weights** (optional) - **T2**:
  weights to optimize.
* **__group_750__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_750__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_750__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_750__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_751__weights** (optional) - **T2**:
  weights to optimize.
* **__group_751__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_751__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_751__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_751__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_752__weights** (optional) - **T2**:
  weights to optimize.
* **__group_752__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_752__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_752__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_752__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_753__weights** (optional) - **T2**:
  weights to optimize.
* **__group_753__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_753__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_753__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_753__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_754__weights** (optional) - **T2**:
  weights to optimize.
* **__group_754__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_754__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_754__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_754__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_755__weights** (optional) - **T2**:
  weights to optimize.
* **__group_755__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_755__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_755__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_755__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_756__weights** (optional) - **T2**:
  weights to optimize.
* **__group_756__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_756__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_756__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_756__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_757__weights** (optional) - **T2**:
  weights to optimize.
* **__group_757__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_757__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_757__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_757__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_758__weights** (optional) - **T2**:
  weights to optimize.
* **__group_758__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_758__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_758__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_758__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_759__weights** (optional) - **T2**:
  weights to optimize.
* **__group_759__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_759__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_759__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_759__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_760__weights** (optional) - **T2**:
  weights to optimize.
* **__group_760__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_760__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_760__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_760__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_761__weights** (optional) - **T2**:
  weights to optimize.
* **__group_761__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_761__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_761__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_761__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_762__weights** (optional) - **T2**:
  weights to optimize.
* **__group_762__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_762__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_762__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_762__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_763__weights** (optional) - **T2**:
  weights to optimize.
* **__group_763__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_763__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_763__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_763__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_764__weights** (optional) - **T2**:
  weights to optimize.
* **__group_764__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_764__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_764__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_764__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_765__weights** (optional) - **T2**:
  weights to optimize.
* **__group_765__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_765__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_765__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_765__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_766__weights** (optional) - **T2**:
  weights to optimize.
* **__group_766__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_766__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_766__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_766__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_767__weights** (optional) - **T2**:
  weights to optimize.
* **__group_767__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_767__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_767__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_767__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_768__weights** (optional) - **T2**:
  weights to optimize.
* **__group_768__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_768__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_768__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_768__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_769__weights** (optional) - **T2**:
  weights to optimize.
* **__group_769__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_769__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_769__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_769__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_770__weights** (optional) - **T2**:
  weights to optimize.
* **__group_770__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_770__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_770__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_770__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_771__weights** (optional) - **T2**:
  weights to optimize.
* **__group_771__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_771__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_771__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_771__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_772__weights** (optional) - **T2**:
  weights to optimize.
* **__group_772__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_772__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_772__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_772__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_773__weights** (optional) - **T2**:
  weights to optimize.
* **__group_773__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_773__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_773__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_773__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_774__weights** (optional) - **T2**:
  weights to optimize.
* **__group_774__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_774__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_774__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_774__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_775__weights** (optional) - **T2**:
  weights to optimize.
* **__group_775__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_775__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_775__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_775__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_776__weights** (optional) - **T2**:
  weights to optimize.
* **__group_776__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_776__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_776__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_776__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_777__weights** (optional) - **T2**:
  weights to optimize.
* **__group_777__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_777__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_777__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_777__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_778__weights** (optional) - **T2**:
  weights to optimize.
* **__group_778__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_778__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_778__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_778__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_779__weights** (optional) - **T2**:
  weights to optimize.
* **__group_779__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_779__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_779__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_779__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_780__weights** (optional) - **T2**:
  weights to optimize.
* **__group_780__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_780__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_780__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_780__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_781__weights** (optional) - **T2**:
  weights to optimize.
* **__group_781__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_781__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_781__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_781__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_782__weights** (optional) - **T2**:
  weights to optimize.
* **__group_782__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_782__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_782__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_782__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_783__weights** (optional) - **T2**:
  weights to optimize.
* **__group_783__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_783__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_783__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_783__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_784__weights** (optional) - **T2**:
  weights to optimize.
* **__group_784__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_784__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_784__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_784__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_785__weights** (optional) - **T2**:
  weights to optimize.
* **__group_785__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_785__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_785__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_785__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_786__weights** (optional) - **T2**:
  weights to optimize.
* **__group_786__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_786__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_786__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_786__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_787__weights** (optional) - **T2**:
  weights to optimize.
* **__group_787__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_787__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_787__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_787__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_788__weights** (optional) - **T2**:
  weights to optimize.
* **__group_788__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_788__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_788__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_788__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_789__weights** (optional) - **T2**:
  weights to optimize.
* **__group_789__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_789__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_789__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_789__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_790__weights** (optional) - **T2**:
  weights to optimize.
* **__group_790__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_790__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_790__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_790__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_791__weights** (optional) - **T2**:
  weights to optimize.
* **__group_791__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_791__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_791__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_791__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_792__weights** (optional) - **T2**:
  weights to optimize.
* **__group_792__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_792__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_792__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_792__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_793__weights** (optional) - **T2**:
  weights to optimize.
* **__group_793__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_793__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_793__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_793__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_794__weights** (optional) - **T2**:
  weights to optimize.
* **__group_794__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_794__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_794__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_794__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_795__weights** (optional) - **T2**:
  weights to optimize.
* **__group_795__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_795__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_795__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_795__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_796__weights** (optional) - **T2**:
  weights to optimize.
* **__group_796__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_796__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_796__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_796__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_797__weights** (optional) - **T2**:
  weights to optimize.
* **__group_797__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_797__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_797__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_797__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_798__weights** (optional) - **T2**:
  weights to optimize.
* **__group_798__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_798__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_798__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_798__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_799__weights** (optional) - **T2**:
  weights to optimize.
* **__group_799__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_799__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_799__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_799__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_800__weights** (optional) - **T2**:
  weights to optimize.
* **__group_800__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_800__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_800__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_800__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_801__weights** (optional) - **T2**:
  weights to optimize.
* **__group_801__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_801__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_801__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_801__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_802__weights** (optional) - **T2**:
  weights to optimize.
* **__group_802__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_802__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_802__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_802__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_803__weights** (optional) - **T2**:
  weights to optimize.
* **__group_803__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_803__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_803__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_803__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_804__weights** (optional) - **T2**:
  weights to optimize.
* **__group_804__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_804__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_804__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_804__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_805__weights** (optional) - **T2**:
  weights to optimize.
* **__group_805__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_805__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_805__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_805__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_806__weights** (optional) - **T2**:
  weights to optimize.
* **__group_806__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_806__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_806__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_806__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_807__weights** (optional) - **T2**:
  weights to optimize.
* **__group_807__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_807__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_807__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_807__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_808__weights** (optional) - **T2**:
  weights to optimize.
* **__group_808__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_808__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_808__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_808__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_809__weights** (optional) - **T2**:
  weights to optimize.
* **__group_809__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_809__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_809__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_809__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_810__weights** (optional) - **T2**:
  weights to optimize.
* **__group_810__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_810__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_810__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_810__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_811__weights** (optional) - **T2**:
  weights to optimize.
* **__group_811__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_811__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_811__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_811__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_812__weights** (optional) - **T2**:
  weights to optimize.
* **__group_812__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_812__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_812__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_812__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_813__weights** (optional) - **T2**:
  weights to optimize.
* **__group_813__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_813__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_813__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_813__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_814__weights** (optional) - **T2**:
  weights to optimize.
* **__group_814__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_814__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_814__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_814__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_815__weights** (optional) - **T2**:
  weights to optimize.
* **__group_815__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_815__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_815__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_815__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_816__weights** (optional) - **T2**:
  weights to optimize.
* **__group_816__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_816__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_816__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_816__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_817__weights** (optional) - **T2**:
  weights to optimize.
* **__group_817__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_817__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_817__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_817__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_818__weights** (optional) - **T2**:
  weights to optimize.
* **__group_818__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_818__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_818__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_818__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_819__weights** (optional) - **T2**:
  weights to optimize.
* **__group_819__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_819__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_819__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_819__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_820__weights** (optional) - **T2**:
  weights to optimize.
* **__group_820__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_820__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_820__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_820__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_821__weights** (optional) - **T2**:
  weights to optimize.
* **__group_821__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_821__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_821__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_821__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_822__weights** (optional) - **T2**:
  weights to optimize.
* **__group_822__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_822__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_822__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_822__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_823__weights** (optional) - **T2**:
  weights to optimize.
* **__group_823__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_823__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_823__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_823__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_824__weights** (optional) - **T2**:
  weights to optimize.
* **__group_824__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_824__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_824__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_824__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_825__weights** (optional) - **T2**:
  weights to optimize.
* **__group_825__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_825__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_825__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_825__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_826__weights** (optional) - **T2**:
  weights to optimize.
* **__group_826__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_826__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_826__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_826__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_827__weights** (optional) - **T2**:
  weights to optimize.
* **__group_827__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_827__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_827__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_827__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_828__weights** (optional) - **T2**:
  weights to optimize.
* **__group_828__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_828__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_828__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_828__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_829__weights** (optional) - **T2**:
  weights to optimize.
* **__group_829__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_829__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_829__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_829__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_830__weights** (optional) - **T2**:
  weights to optimize.
* **__group_830__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_830__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_830__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_830__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_831__weights** (optional) - **T2**:
  weights to optimize.
* **__group_831__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_831__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_831__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_831__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_832__weights** (optional) - **T2**:
  weights to optimize.
* **__group_832__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_832__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_832__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_832__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_833__weights** (optional) - **T2**:
  weights to optimize.
* **__group_833__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_833__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_833__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_833__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_834__weights** (optional) - **T2**:
  weights to optimize.
* **__group_834__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_834__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_834__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_834__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_835__weights** (optional) - **T2**:
  weights to optimize.
* **__group_835__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_835__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_835__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_835__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_836__weights** (optional) - **T2**:
  weights to optimize.
* **__group_836__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_836__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_836__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_836__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_837__weights** (optional) - **T2**:
  weights to optimize.
* **__group_837__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_837__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_837__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_837__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_838__weights** (optional) - **T2**:
  weights to optimize.
* **__group_838__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_838__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_838__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_838__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_839__weights** (optional) - **T2**:
  weights to optimize.
* **__group_839__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_839__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_839__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_839__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_840__weights** (optional) - **T2**:
  weights to optimize.
* **__group_840__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_840__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_840__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_840__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_841__weights** (optional) - **T2**:
  weights to optimize.
* **__group_841__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_841__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_841__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_841__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_842__weights** (optional) - **T2**:
  weights to optimize.
* **__group_842__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_842__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_842__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_842__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_843__weights** (optional) - **T2**:
  weights to optimize.
* **__group_843__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_843__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_843__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_843__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_844__weights** (optional) - **T2**:
  weights to optimize.
* **__group_844__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_844__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_844__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_844__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_845__weights** (optional) - **T2**:
  weights to optimize.
* **__group_845__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_845__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_845__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_845__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_846__weights** (optional) - **T2**:
  weights to optimize.
* **__group_846__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_846__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_846__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_846__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_847__weights** (optional) - **T2**:
  weights to optimize.
* **__group_847__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_847__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_847__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_847__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_848__weights** (optional) - **T2**:
  weights to optimize.
* **__group_848__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_848__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_848__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_848__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_849__weights** (optional) - **T2**:
  weights to optimize.
* **__group_849__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_849__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_849__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_849__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_850__weights** (optional) - **T2**:
  weights to optimize.
* **__group_850__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_850__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_850__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_850__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_851__weights** (optional) - **T2**:
  weights to optimize.
* **__group_851__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_851__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_851__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_851__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_852__weights** (optional) - **T2**:
  weights to optimize.
* **__group_852__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_852__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_852__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_852__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_853__weights** (optional) - **T2**:
  weights to optimize.
* **__group_853__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_853__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_853__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_853__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_854__weights** (optional) - **T2**:
  weights to optimize.
* **__group_854__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_854__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_854__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_854__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_855__weights** (optional) - **T2**:
  weights to optimize.
* **__group_855__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_855__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_855__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_855__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_856__weights** (optional) - **T2**:
  weights to optimize.
* **__group_856__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_856__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_856__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_856__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_857__weights** (optional) - **T2**:
  weights to optimize.
* **__group_857__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_857__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_857__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_857__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_858__weights** (optional) - **T2**:
  weights to optimize.
* **__group_858__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_858__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_858__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_858__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_859__weights** (optional) - **T2**:
  weights to optimize.
* **__group_859__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_859__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_859__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_859__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_860__weights** (optional) - **T2**:
  weights to optimize.
* **__group_860__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_860__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_860__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_860__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_861__weights** (optional) - **T2**:
  weights to optimize.
* **__group_861__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_861__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_861__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_861__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_862__weights** (optional) - **T2**:
  weights to optimize.
* **__group_862__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_862__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_862__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_862__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_863__weights** (optional) - **T2**:
  weights to optimize.
* **__group_863__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_863__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_863__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_863__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_864__weights** (optional) - **T2**:
  weights to optimize.
* **__group_864__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_864__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_864__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_864__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_865__weights** (optional) - **T2**:
  weights to optimize.
* **__group_865__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_865__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_865__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_865__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_866__weights** (optional) - **T2**:
  weights to optimize.
* **__group_866__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_866__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_866__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_866__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_867__weights** (optional) - **T2**:
  weights to optimize.
* **__group_867__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_867__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_867__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_867__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_868__weights** (optional) - **T2**:
  weights to optimize.
* **__group_868__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_868__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_868__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_868__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_869__weights** (optional) - **T2**:
  weights to optimize.
* **__group_869__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_869__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_869__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_869__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_870__weights** (optional) - **T2**:
  weights to optimize.
* **__group_870__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_870__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_870__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_870__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_871__weights** (optional) - **T2**:
  weights to optimize.
* **__group_871__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_871__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_871__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_871__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_872__weights** (optional) - **T2**:
  weights to optimize.
* **__group_872__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_872__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_872__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_872__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_873__weights** (optional) - **T2**:
  weights to optimize.
* **__group_873__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_873__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_873__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_873__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_874__weights** (optional) - **T2**:
  weights to optimize.
* **__group_874__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_874__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_874__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_874__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_875__weights** (optional) - **T2**:
  weights to optimize.
* **__group_875__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_875__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_875__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_875__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_876__weights** (optional) - **T2**:
  weights to optimize.
* **__group_876__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_876__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_876__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_876__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_877__weights** (optional) - **T2**:
  weights to optimize.
* **__group_877__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_877__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_877__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_877__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_878__weights** (optional) - **T2**:
  weights to optimize.
* **__group_878__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_878__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_878__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_878__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_879__weights** (optional) - **T2**:
  weights to optimize.
* **__group_879__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_879__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_879__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_879__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_880__weights** (optional) - **T2**:
  weights to optimize.
* **__group_880__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_880__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_880__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_880__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_881__weights** (optional) - **T2**:
  weights to optimize.
* **__group_881__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_881__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_881__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_881__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_882__weights** (optional) - **T2**:
  weights to optimize.
* **__group_882__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_882__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_882__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_882__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_883__weights** (optional) - **T2**:
  weights to optimize.
* **__group_883__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_883__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_883__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_883__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_884__weights** (optional) - **T2**:
  weights to optimize.
* **__group_884__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_884__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_884__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_884__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_885__weights** (optional) - **T2**:
  weights to optimize.
* **__group_885__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_885__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_885__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_885__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_886__weights** (optional) - **T2**:
  weights to optimize.
* **__group_886__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_886__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_886__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_886__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_887__weights** (optional) - **T2**:
  weights to optimize.
* **__group_887__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_887__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_887__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_887__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_888__weights** (optional) - **T2**:
  weights to optimize.
* **__group_888__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_888__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_888__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_888__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_889__weights** (optional) - **T2**:
  weights to optimize.
* **__group_889__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_889__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_889__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_889__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_890__weights** (optional) - **T2**:
  weights to optimize.
* **__group_890__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_890__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_890__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_890__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_891__weights** (optional) - **T2**:
  weights to optimize.
* **__group_891__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_891__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_891__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_891__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_892__weights** (optional) - **T2**:
  weights to optimize.
* **__group_892__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_892__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_892__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_892__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_893__weights** (optional) - **T2**:
  weights to optimize.
* **__group_893__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_893__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_893__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_893__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_894__weights** (optional) - **T2**:
  weights to optimize.
* **__group_894__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_894__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_894__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_894__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_895__weights** (optional) - **T2**:
  weights to optimize.
* **__group_895__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_895__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_895__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_895__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_896__weights** (optional) - **T2**:
  weights to optimize.
* **__group_896__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_896__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_896__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_896__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_897__weights** (optional) - **T2**:
  weights to optimize.
* **__group_897__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_897__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_897__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_897__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_898__weights** (optional) - **T2**:
  weights to optimize.
* **__group_898__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_898__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_898__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_898__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_899__weights** (optional) - **T2**:
  weights to optimize.
* **__group_899__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_899__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_899__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_899__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_900__weights** (optional) - **T2**:
  weights to optimize.
* **__group_900__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_900__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_900__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_900__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_901__weights** (optional) - **T2**:
  weights to optimize.
* **__group_901__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_901__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_901__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_901__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_902__weights** (optional) - **T2**:
  weights to optimize.
* **__group_902__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_902__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_902__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_902__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_903__weights** (optional) - **T2**:
  weights to optimize.
* **__group_903__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_903__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_903__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_903__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_904__weights** (optional) - **T2**:
  weights to optimize.
* **__group_904__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_904__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_904__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_904__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_905__weights** (optional) - **T2**:
  weights to optimize.
* **__group_905__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_905__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_905__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_905__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_906__weights** (optional) - **T2**:
  weights to optimize.
* **__group_906__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_906__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_906__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_906__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_907__weights** (optional) - **T2**:
  weights to optimize.
* **__group_907__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_907__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_907__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_907__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_908__weights** (optional) - **T2**:
  weights to optimize.
* **__group_908__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_908__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_908__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_908__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_909__weights** (optional) - **T2**:
  weights to optimize.
* **__group_909__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_909__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_909__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_909__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_910__weights** (optional) - **T2**:
  weights to optimize.
* **__group_910__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_910__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_910__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_910__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_911__weights** (optional) - **T2**:
  weights to optimize.
* **__group_911__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_911__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_911__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_911__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_912__weights** (optional) - **T2**:
  weights to optimize.
* **__group_912__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_912__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_912__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_912__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_913__weights** (optional) - **T2**:
  weights to optimize.
* **__group_913__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_913__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_913__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_913__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_914__weights** (optional) - **T2**:
  weights to optimize.
* **__group_914__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_914__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_914__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_914__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_915__weights** (optional) - **T2**:
  weights to optimize.
* **__group_915__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_915__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_915__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_915__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_916__weights** (optional) - **T2**:
  weights to optimize.
* **__group_916__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_916__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_916__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_916__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_917__weights** (optional) - **T2**:
  weights to optimize.
* **__group_917__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_917__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_917__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_917__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_918__weights** (optional) - **T2**:
  weights to optimize.
* **__group_918__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_918__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_918__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_918__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_919__weights** (optional) - **T2**:
  weights to optimize.
* **__group_919__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_919__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_919__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_919__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_920__weights** (optional) - **T2**:
  weights to optimize.
* **__group_920__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_920__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_920__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_920__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_921__weights** (optional) - **T2**:
  weights to optimize.
* **__group_921__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_921__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_921__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_921__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_922__weights** (optional) - **T2**:
  weights to optimize.
* **__group_922__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_922__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_922__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_922__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_923__weights** (optional) - **T2**:
  weights to optimize.
* **__group_923__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_923__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_923__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_923__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_924__weights** (optional) - **T2**:
  weights to optimize.
* **__group_924__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_924__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_924__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_924__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_925__weights** (optional) - **T2**:
  weights to optimize.
* **__group_925__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_925__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_925__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_925__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_926__weights** (optional) - **T2**:
  weights to optimize.
* **__group_926__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_926__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_926__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_926__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_927__weights** (optional) - **T2**:
  weights to optimize.
* **__group_927__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_927__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_927__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_927__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_928__weights** (optional) - **T2**:
  weights to optimize.
* **__group_928__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_928__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_928__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_928__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_929__weights** (optional) - **T2**:
  weights to optimize.
* **__group_929__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_929__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_929__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_929__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_930__weights** (optional) - **T2**:
  weights to optimize.
* **__group_930__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_930__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_930__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_930__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_931__weights** (optional) - **T2**:
  weights to optimize.
* **__group_931__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_931__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_931__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_931__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_932__weights** (optional) - **T2**:
  weights to optimize.
* **__group_932__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_932__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_932__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_932__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_933__weights** (optional) - **T2**:
  weights to optimize.
* **__group_933__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_933__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_933__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_933__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_934__weights** (optional) - **T2**:
  weights to optimize.
* **__group_934__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_934__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_934__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_934__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_935__weights** (optional) - **T2**:
  weights to optimize.
* **__group_935__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_935__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_935__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_935__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_936__weights** (optional) - **T2**:
  weights to optimize.
* **__group_936__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_936__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_936__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_936__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_937__weights** (optional) - **T2**:
  weights to optimize.
* **__group_937__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_937__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_937__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_937__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_938__weights** (optional) - **T2**:
  weights to optimize.
* **__group_938__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_938__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_938__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_938__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_939__weights** (optional) - **T2**:
  weights to optimize.
* **__group_939__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_939__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_939__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_939__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_940__weights** (optional) - **T2**:
  weights to optimize.
* **__group_940__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_940__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_940__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_940__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_941__weights** (optional) - **T2**:
  weights to optimize.
* **__group_941__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_941__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_941__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_941__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_942__weights** (optional) - **T2**:
  weights to optimize.
* **__group_942__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_942__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_942__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_942__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_943__weights** (optional) - **T2**:
  weights to optimize.
* **__group_943__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_943__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_943__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_943__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_944__weights** (optional) - **T2**:
  weights to optimize.
* **__group_944__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_944__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_944__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_944__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_945__weights** (optional) - **T2**:
  weights to optimize.
* **__group_945__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_945__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_945__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_945__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_946__weights** (optional) - **T2**:
  weights to optimize.
* **__group_946__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_946__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_946__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_946__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_947__weights** (optional) - **T2**:
  weights to optimize.
* **__group_947__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_947__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_947__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_947__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_948__weights** (optional) - **T2**:
  weights to optimize.
* **__group_948__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_948__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_948__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_948__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_949__weights** (optional) - **T2**:
  weights to optimize.
* **__group_949__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_949__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_949__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_949__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_950__weights** (optional) - **T2**:
  weights to optimize.
* **__group_950__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_950__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_950__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_950__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_951__weights** (optional) - **T2**:
  weights to optimize.
* **__group_951__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_951__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_951__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_951__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_952__weights** (optional) - **T2**:
  weights to optimize.
* **__group_952__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_952__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_952__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_952__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_953__weights** (optional) - **T2**:
  weights to optimize.
* **__group_953__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_953__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_953__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_953__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_954__weights** (optional) - **T2**:
  weights to optimize.
* **__group_954__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_954__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_954__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_954__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_955__weights** (optional) - **T2**:
  weights to optimize.
* **__group_955__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_955__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_955__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_955__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_956__weights** (optional) - **T2**:
  weights to optimize.
* **__group_956__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_956__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_956__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_956__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_957__weights** (optional) - **T2**:
  weights to optimize.
* **__group_957__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_957__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_957__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_957__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_958__weights** (optional) - **T2**:
  weights to optimize.
* **__group_958__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_958__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_958__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_958__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_959__weights** (optional) - **T2**:
  weights to optimize.
* **__group_959__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_959__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_959__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_959__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_960__weights** (optional) - **T2**:
  weights to optimize.
* **__group_960__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_960__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_960__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_960__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_961__weights** (optional) - **T2**:
  weights to optimize.
* **__group_961__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_961__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_961__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_961__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_962__weights** (optional) - **T2**:
  weights to optimize.
* **__group_962__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_962__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_962__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_962__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_963__weights** (optional) - **T2**:
  weights to optimize.
* **__group_963__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_963__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_963__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_963__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_964__weights** (optional) - **T2**:
  weights to optimize.
* **__group_964__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_964__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_964__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_964__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_965__weights** (optional) - **T2**:
  weights to optimize.
* **__group_965__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_965__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_965__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_965__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_966__weights** (optional) - **T2**:
  weights to optimize.
* **__group_966__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_966__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_966__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_966__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_967__weights** (optional) - **T2**:
  weights to optimize.
* **__group_967__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_967__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_967__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_967__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_968__weights** (optional) - **T2**:
  weights to optimize.
* **__group_968__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_968__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_968__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_968__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_969__weights** (optional) - **T2**:
  weights to optimize.
* **__group_969__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_969__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_969__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_969__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_970__weights** (optional) - **T2**:
  weights to optimize.
* **__group_970__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_970__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_970__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_970__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_971__weights** (optional) - **T2**:
  weights to optimize.
* **__group_971__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_971__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_971__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_971__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_972__weights** (optional) - **T2**:
  weights to optimize.
* **__group_972__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_972__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_972__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_972__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_973__weights** (optional) - **T2**:
  weights to optimize.
* **__group_973__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_973__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_973__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_973__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_974__weights** (optional) - **T2**:
  weights to optimize.
* **__group_974__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_974__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_974__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_974__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_975__weights** (optional) - **T2**:
  weights to optimize.
* **__group_975__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_975__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_975__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_975__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_976__weights** (optional) - **T2**:
  weights to optimize.
* **__group_976__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_976__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_976__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_976__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_977__weights** (optional) - **T2**:
  weights to optimize.
* **__group_977__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_977__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_977__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_977__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_978__weights** (optional) - **T2**:
  weights to optimize.
* **__group_978__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_978__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_978__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_978__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_979__weights** (optional) - **T2**:
  weights to optimize.
* **__group_979__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_979__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_979__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_979__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_980__weights** (optional) - **T2**:
  weights to optimize.
* **__group_980__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_980__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_980__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_980__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_981__weights** (optional) - **T2**:
  weights to optimize.
* **__group_981__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_981__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_981__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_981__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_982__weights** (optional) - **T2**:
  weights to optimize.
* **__group_982__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_982__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_982__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_982__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_983__weights** (optional) - **T2**:
  weights to optimize.
* **__group_983__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_983__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_983__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_983__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_984__weights** (optional) - **T2**:
  weights to optimize.
* **__group_984__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_984__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_984__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_984__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_985__weights** (optional) - **T2**:
  weights to optimize.
* **__group_985__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_985__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_985__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_985__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_986__weights** (optional) - **T2**:
  weights to optimize.
* **__group_986__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_986__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_986__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_986__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_987__weights** (optional) - **T2**:
  weights to optimize.
* **__group_987__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_987__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_987__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_987__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_988__weights** (optional) - **T2**:
  weights to optimize.
* **__group_988__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_988__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_988__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_988__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_989__weights** (optional) - **T2**:
  weights to optimize.
* **__group_989__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_989__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_989__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_989__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_990__weights** (optional) - **T2**:
  weights to optimize.
* **__group_990__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_990__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_990__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_990__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_991__weights** (optional) - **T2**:
  weights to optimize.
* **__group_991__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_991__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_991__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_991__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_992__weights** (optional) - **T2**:
  weights to optimize.
* **__group_992__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_992__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_992__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_992__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_993__weights** (optional) - **T2**:
  weights to optimize.
* **__group_993__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_993__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_993__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_993__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_994__weights** (optional) - **T2**:
  weights to optimize.
* **__group_994__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_994__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_994__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_994__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_995__weights** (optional) - **T2**:
  weights to optimize.
* **__group_995__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_995__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_995__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_995__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_996__weights** (optional) - **T2**:
  weights to optimize.
* **__group_996__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_996__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_996__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_996__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_997__weights** (optional) - **T2**:
  weights to optimize.
* **__group_997__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_997__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_997__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_997__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_998__weights** (optional) - **T2**:
  weights to optimize.
* **__group_998__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_998__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_998__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_998__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_999__weights** (optional) - **T2**:
  weights to optimize.
* **__group_999__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_999__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_999__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_999__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_1000__weights** (optional) - **T2**:
  weights to optimize.
* **__group_1000__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_1000__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_1000__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_1000__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_1001__weights** (optional) - **T2**:
  weights to optimize.
* **__group_1001__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_1001__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_1001__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_1001__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_1002__weights** (optional) - **T2**:
  weights to optimize.
* **__group_1002__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_1002__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_1002__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_1002__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_1003__weights** (optional) - **T2**:
  weights to optimize.
* **__group_1003__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_1003__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_1003__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_1003__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_1004__weights** (optional) - **T2**:
  weights to optimize.
* **__group_1004__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_1004__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_1004__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_1004__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_1005__weights** (optional) - **T2**:
  weights to optimize.
* **__group_1005__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_1005__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_1005__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_1005__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_1006__weights** (optional) - **T2**:
  weights to optimize.
* **__group_1006__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_1006__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_1006__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_1006__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_1007__weights** (optional) - **T2**:
  weights to optimize.
* **__group_1007__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_1007__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_1007__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_1007__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_1008__weights** (optional) - **T2**:
  weights to optimize.
* **__group_1008__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_1008__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_1008__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_1008__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_1009__weights** (optional) - **T2**:
  weights to optimize.
* **__group_1009__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_1009__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_1009__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_1009__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_1010__weights** (optional) - **T2**:
  weights to optimize.
* **__group_1010__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_1010__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_1010__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_1010__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_1011__weights** (optional) - **T2**:
  weights to optimize.
* **__group_1011__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_1011__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_1011__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_1011__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_1012__weights** (optional) - **T2**:
  weights to optimize.
* **__group_1012__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_1012__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_1012__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_1012__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_1013__weights** (optional) - **T2**:
  weights to optimize.
* **__group_1013__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_1013__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_1013__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_1013__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_1014__weights** (optional) - **T2**:
  weights to optimize.
* **__group_1014__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_1014__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_1014__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_1014__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_1015__weights** (optional) - **T2**:
  weights to optimize.
* **__group_1015__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_1015__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_1015__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_1015__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_1016__weights** (optional) - **T2**:
  weights to optimize.
* **__group_1016__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_1016__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_1016__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_1016__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_1017__weights** (optional) - **T2**:
  weights to optimize.
* **__group_1017__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_1017__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_1017__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_1017__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_1018__weights** (optional) - **T2**:
  weights to optimize.
* **__group_1018__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_1018__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_1018__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_1018__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_1019__weights** (optional) - **T2**:
  weights to optimize.
* **__group_1019__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_1019__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_1019__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_1019__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_1020__weights** (optional) - **T2**:
  weights to optimize.
* **__group_1020__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_1020__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_1020__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_1020__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_1021__weights** (optional) - **T2**:
  weights to optimize.
* **__group_1021__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_1021__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_1021__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_1021__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_1022__weights** (optional) - **T2**:
  weights to optimize.
* **__group_1022__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_1022__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_1022__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_1022__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.
* **__group_1023__weights** (optional) - **T2**:
  weights to optimize.
* **__group_1023__gradients** (optional) - **T3**:
  gradients computed in this iteration.
* **__group_1023__moment1** (optional) - **T4**:
  exponentially averaged historical gradients.
* **__group_1023__moment2** (optional) - **T4**:
  exponentially averaged historical squared gradients.
* **__group_1023__mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  FP16 or BF16 weights to optimize.

**Outputs**

Between 0 and 5121 outputs.

* **new_step** (optional, heterogeneous) - **TInt64**:
  One-based index of the next training iteration.
* **__group_0__new_weights** (optional) - **T2**:
  New weights
* **__group_0__new_gradients** (optional) - **T3**:
  New gradients
* **__group_0__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_0__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_0__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_1__new_weights** (optional) - **T2**:
  New weights
* **__group_1__new_gradients** (optional) - **T3**:
  New gradients
* **__group_1__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_1__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_1__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_2__new_weights** (optional) - **T2**:
  New weights
* **__group_2__new_gradients** (optional) - **T3**:
  New gradients
* **__group_2__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_2__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_2__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_3__new_weights** (optional) - **T2**:
  New weights
* **__group_3__new_gradients** (optional) - **T3**:
  New gradients
* **__group_3__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_3__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_3__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_4__new_weights** (optional) - **T2**:
  New weights
* **__group_4__new_gradients** (optional) - **T3**:
  New gradients
* **__group_4__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_4__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_4__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_5__new_weights** (optional) - **T2**:
  New weights
* **__group_5__new_gradients** (optional) - **T3**:
  New gradients
* **__group_5__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_5__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_5__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_6__new_weights** (optional) - **T2**:
  New weights
* **__group_6__new_gradients** (optional) - **T3**:
  New gradients
* **__group_6__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_6__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_6__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_7__new_weights** (optional) - **T2**:
  New weights
* **__group_7__new_gradients** (optional) - **T3**:
  New gradients
* **__group_7__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_7__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_7__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_8__new_weights** (optional) - **T2**:
  New weights
* **__group_8__new_gradients** (optional) - **T3**:
  New gradients
* **__group_8__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_8__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_8__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_9__new_weights** (optional) - **T2**:
  New weights
* **__group_9__new_gradients** (optional) - **T3**:
  New gradients
* **__group_9__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_9__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_9__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_10__new_weights** (optional) - **T2**:
  New weights
* **__group_10__new_gradients** (optional) - **T3**:
  New gradients
* **__group_10__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_10__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_10__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_11__new_weights** (optional) - **T2**:
  New weights
* **__group_11__new_gradients** (optional) - **T3**:
  New gradients
* **__group_11__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_11__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_11__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_12__new_weights** (optional) - **T2**:
  New weights
* **__group_12__new_gradients** (optional) - **T3**:
  New gradients
* **__group_12__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_12__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_12__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_13__new_weights** (optional) - **T2**:
  New weights
* **__group_13__new_gradients** (optional) - **T3**:
  New gradients
* **__group_13__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_13__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_13__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_14__new_weights** (optional) - **T2**:
  New weights
* **__group_14__new_gradients** (optional) - **T3**:
  New gradients
* **__group_14__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_14__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_14__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_15__new_weights** (optional) - **T2**:
  New weights
* **__group_15__new_gradients** (optional) - **T3**:
  New gradients
* **__group_15__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_15__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_15__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_16__new_weights** (optional) - **T2**:
  New weights
* **__group_16__new_gradients** (optional) - **T3**:
  New gradients
* **__group_16__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_16__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_16__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_17__new_weights** (optional) - **T2**:
  New weights
* **__group_17__new_gradients** (optional) - **T3**:
  New gradients
* **__group_17__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_17__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_17__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_18__new_weights** (optional) - **T2**:
  New weights
* **__group_18__new_gradients** (optional) - **T3**:
  New gradients
* **__group_18__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_18__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_18__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_19__new_weights** (optional) - **T2**:
  New weights
* **__group_19__new_gradients** (optional) - **T3**:
  New gradients
* **__group_19__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_19__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_19__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_20__new_weights** (optional) - **T2**:
  New weights
* **__group_20__new_gradients** (optional) - **T3**:
  New gradients
* **__group_20__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_20__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_20__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_21__new_weights** (optional) - **T2**:
  New weights
* **__group_21__new_gradients** (optional) - **T3**:
  New gradients
* **__group_21__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_21__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_21__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_22__new_weights** (optional) - **T2**:
  New weights
* **__group_22__new_gradients** (optional) - **T3**:
  New gradients
* **__group_22__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_22__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_22__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_23__new_weights** (optional) - **T2**:
  New weights
* **__group_23__new_gradients** (optional) - **T3**:
  New gradients
* **__group_23__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_23__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_23__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_24__new_weights** (optional) - **T2**:
  New weights
* **__group_24__new_gradients** (optional) - **T3**:
  New gradients
* **__group_24__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_24__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_24__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_25__new_weights** (optional) - **T2**:
  New weights
* **__group_25__new_gradients** (optional) - **T3**:
  New gradients
* **__group_25__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_25__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_25__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_26__new_weights** (optional) - **T2**:
  New weights
* **__group_26__new_gradients** (optional) - **T3**:
  New gradients
* **__group_26__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_26__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_26__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_27__new_weights** (optional) - **T2**:
  New weights
* **__group_27__new_gradients** (optional) - **T3**:
  New gradients
* **__group_27__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_27__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_27__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_28__new_weights** (optional) - **T2**:
  New weights
* **__group_28__new_gradients** (optional) - **T3**:
  New gradients
* **__group_28__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_28__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_28__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_29__new_weights** (optional) - **T2**:
  New weights
* **__group_29__new_gradients** (optional) - **T3**:
  New gradients
* **__group_29__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_29__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_29__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_30__new_weights** (optional) - **T2**:
  New weights
* **__group_30__new_gradients** (optional) - **T3**:
  New gradients
* **__group_30__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_30__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_30__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_31__new_weights** (optional) - **T2**:
  New weights
* **__group_31__new_gradients** (optional) - **T3**:
  New gradients
* **__group_31__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_31__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_31__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_32__new_weights** (optional) - **T2**:
  New weights
* **__group_32__new_gradients** (optional) - **T3**:
  New gradients
* **__group_32__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_32__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_32__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_33__new_weights** (optional) - **T2**:
  New weights
* **__group_33__new_gradients** (optional) - **T3**:
  New gradients
* **__group_33__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_33__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_33__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_34__new_weights** (optional) - **T2**:
  New weights
* **__group_34__new_gradients** (optional) - **T3**:
  New gradients
* **__group_34__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_34__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_34__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_35__new_weights** (optional) - **T2**:
  New weights
* **__group_35__new_gradients** (optional) - **T3**:
  New gradients
* **__group_35__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_35__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_35__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_36__new_weights** (optional) - **T2**:
  New weights
* **__group_36__new_gradients** (optional) - **T3**:
  New gradients
* **__group_36__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_36__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_36__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_37__new_weights** (optional) - **T2**:
  New weights
* **__group_37__new_gradients** (optional) - **T3**:
  New gradients
* **__group_37__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_37__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_37__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_38__new_weights** (optional) - **T2**:
  New weights
* **__group_38__new_gradients** (optional) - **T3**:
  New gradients
* **__group_38__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_38__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_38__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_39__new_weights** (optional) - **T2**:
  New weights
* **__group_39__new_gradients** (optional) - **T3**:
  New gradients
* **__group_39__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_39__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_39__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_40__new_weights** (optional) - **T2**:
  New weights
* **__group_40__new_gradients** (optional) - **T3**:
  New gradients
* **__group_40__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_40__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_40__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_41__new_weights** (optional) - **T2**:
  New weights
* **__group_41__new_gradients** (optional) - **T3**:
  New gradients
* **__group_41__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_41__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_41__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_42__new_weights** (optional) - **T2**:
  New weights
* **__group_42__new_gradients** (optional) - **T3**:
  New gradients
* **__group_42__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_42__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_42__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_43__new_weights** (optional) - **T2**:
  New weights
* **__group_43__new_gradients** (optional) - **T3**:
  New gradients
* **__group_43__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_43__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_43__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_44__new_weights** (optional) - **T2**:
  New weights
* **__group_44__new_gradients** (optional) - **T3**:
  New gradients
* **__group_44__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_44__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_44__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_45__new_weights** (optional) - **T2**:
  New weights
* **__group_45__new_gradients** (optional) - **T3**:
  New gradients
* **__group_45__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_45__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_45__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_46__new_weights** (optional) - **T2**:
  New weights
* **__group_46__new_gradients** (optional) - **T3**:
  New gradients
* **__group_46__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_46__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_46__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_47__new_weights** (optional) - **T2**:
  New weights
* **__group_47__new_gradients** (optional) - **T3**:
  New gradients
* **__group_47__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_47__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_47__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_48__new_weights** (optional) - **T2**:
  New weights
* **__group_48__new_gradients** (optional) - **T3**:
  New gradients
* **__group_48__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_48__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_48__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_49__new_weights** (optional) - **T2**:
  New weights
* **__group_49__new_gradients** (optional) - **T3**:
  New gradients
* **__group_49__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_49__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_49__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_50__new_weights** (optional) - **T2**:
  New weights
* **__group_50__new_gradients** (optional) - **T3**:
  New gradients
* **__group_50__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_50__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_50__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_51__new_weights** (optional) - **T2**:
  New weights
* **__group_51__new_gradients** (optional) - **T3**:
  New gradients
* **__group_51__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_51__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_51__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_52__new_weights** (optional) - **T2**:
  New weights
* **__group_52__new_gradients** (optional) - **T3**:
  New gradients
* **__group_52__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_52__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_52__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_53__new_weights** (optional) - **T2**:
  New weights
* **__group_53__new_gradients** (optional) - **T3**:
  New gradients
* **__group_53__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_53__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_53__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_54__new_weights** (optional) - **T2**:
  New weights
* **__group_54__new_gradients** (optional) - **T3**:
  New gradients
* **__group_54__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_54__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_54__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_55__new_weights** (optional) - **T2**:
  New weights
* **__group_55__new_gradients** (optional) - **T3**:
  New gradients
* **__group_55__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_55__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_55__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_56__new_weights** (optional) - **T2**:
  New weights
* **__group_56__new_gradients** (optional) - **T3**:
  New gradients
* **__group_56__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_56__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_56__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_57__new_weights** (optional) - **T2**:
  New weights
* **__group_57__new_gradients** (optional) - **T3**:
  New gradients
* **__group_57__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_57__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_57__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_58__new_weights** (optional) - **T2**:
  New weights
* **__group_58__new_gradients** (optional) - **T3**:
  New gradients
* **__group_58__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_58__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_58__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_59__new_weights** (optional) - **T2**:
  New weights
* **__group_59__new_gradients** (optional) - **T3**:
  New gradients
* **__group_59__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_59__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_59__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_60__new_weights** (optional) - **T2**:
  New weights
* **__group_60__new_gradients** (optional) - **T3**:
  New gradients
* **__group_60__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_60__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_60__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_61__new_weights** (optional) - **T2**:
  New weights
* **__group_61__new_gradients** (optional) - **T3**:
  New gradients
* **__group_61__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_61__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_61__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_62__new_weights** (optional) - **T2**:
  New weights
* **__group_62__new_gradients** (optional) - **T3**:
  New gradients
* **__group_62__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_62__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_62__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_63__new_weights** (optional) - **T2**:
  New weights
* **__group_63__new_gradients** (optional) - **T3**:
  New gradients
* **__group_63__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_63__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_63__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_64__new_weights** (optional) - **T2**:
  New weights
* **__group_64__new_gradients** (optional) - **T3**:
  New gradients
* **__group_64__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_64__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_64__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_65__new_weights** (optional) - **T2**:
  New weights
* **__group_65__new_gradients** (optional) - **T3**:
  New gradients
* **__group_65__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_65__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_65__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_66__new_weights** (optional) - **T2**:
  New weights
* **__group_66__new_gradients** (optional) - **T3**:
  New gradients
* **__group_66__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_66__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_66__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_67__new_weights** (optional) - **T2**:
  New weights
* **__group_67__new_gradients** (optional) - **T3**:
  New gradients
* **__group_67__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_67__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_67__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_68__new_weights** (optional) - **T2**:
  New weights
* **__group_68__new_gradients** (optional) - **T3**:
  New gradients
* **__group_68__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_68__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_68__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_69__new_weights** (optional) - **T2**:
  New weights
* **__group_69__new_gradients** (optional) - **T3**:
  New gradients
* **__group_69__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_69__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_69__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_70__new_weights** (optional) - **T2**:
  New weights
* **__group_70__new_gradients** (optional) - **T3**:
  New gradients
* **__group_70__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_70__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_70__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_71__new_weights** (optional) - **T2**:
  New weights
* **__group_71__new_gradients** (optional) - **T3**:
  New gradients
* **__group_71__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_71__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_71__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_72__new_weights** (optional) - **T2**:
  New weights
* **__group_72__new_gradients** (optional) - **T3**:
  New gradients
* **__group_72__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_72__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_72__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_73__new_weights** (optional) - **T2**:
  New weights
* **__group_73__new_gradients** (optional) - **T3**:
  New gradients
* **__group_73__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_73__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_73__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_74__new_weights** (optional) - **T2**:
  New weights
* **__group_74__new_gradients** (optional) - **T3**:
  New gradients
* **__group_74__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_74__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_74__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_75__new_weights** (optional) - **T2**:
  New weights
* **__group_75__new_gradients** (optional) - **T3**:
  New gradients
* **__group_75__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_75__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_75__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_76__new_weights** (optional) - **T2**:
  New weights
* **__group_76__new_gradients** (optional) - **T3**:
  New gradients
* **__group_76__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_76__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_76__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_77__new_weights** (optional) - **T2**:
  New weights
* **__group_77__new_gradients** (optional) - **T3**:
  New gradients
* **__group_77__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_77__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_77__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_78__new_weights** (optional) - **T2**:
  New weights
* **__group_78__new_gradients** (optional) - **T3**:
  New gradients
* **__group_78__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_78__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_78__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_79__new_weights** (optional) - **T2**:
  New weights
* **__group_79__new_gradients** (optional) - **T3**:
  New gradients
* **__group_79__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_79__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_79__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_80__new_weights** (optional) - **T2**:
  New weights
* **__group_80__new_gradients** (optional) - **T3**:
  New gradients
* **__group_80__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_80__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_80__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_81__new_weights** (optional) - **T2**:
  New weights
* **__group_81__new_gradients** (optional) - **T3**:
  New gradients
* **__group_81__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_81__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_81__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_82__new_weights** (optional) - **T2**:
  New weights
* **__group_82__new_gradients** (optional) - **T3**:
  New gradients
* **__group_82__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_82__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_82__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_83__new_weights** (optional) - **T2**:
  New weights
* **__group_83__new_gradients** (optional) - **T3**:
  New gradients
* **__group_83__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_83__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_83__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_84__new_weights** (optional) - **T2**:
  New weights
* **__group_84__new_gradients** (optional) - **T3**:
  New gradients
* **__group_84__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_84__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_84__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_85__new_weights** (optional) - **T2**:
  New weights
* **__group_85__new_gradients** (optional) - **T3**:
  New gradients
* **__group_85__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_85__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_85__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_86__new_weights** (optional) - **T2**:
  New weights
* **__group_86__new_gradients** (optional) - **T3**:
  New gradients
* **__group_86__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_86__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_86__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_87__new_weights** (optional) - **T2**:
  New weights
* **__group_87__new_gradients** (optional) - **T3**:
  New gradients
* **__group_87__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_87__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_87__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_88__new_weights** (optional) - **T2**:
  New weights
* **__group_88__new_gradients** (optional) - **T3**:
  New gradients
* **__group_88__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_88__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_88__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_89__new_weights** (optional) - **T2**:
  New weights
* **__group_89__new_gradients** (optional) - **T3**:
  New gradients
* **__group_89__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_89__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_89__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_90__new_weights** (optional) - **T2**:
  New weights
* **__group_90__new_gradients** (optional) - **T3**:
  New gradients
* **__group_90__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_90__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_90__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_91__new_weights** (optional) - **T2**:
  New weights
* **__group_91__new_gradients** (optional) - **T3**:
  New gradients
* **__group_91__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_91__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_91__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_92__new_weights** (optional) - **T2**:
  New weights
* **__group_92__new_gradients** (optional) - **T3**:
  New gradients
* **__group_92__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_92__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_92__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_93__new_weights** (optional) - **T2**:
  New weights
* **__group_93__new_gradients** (optional) - **T3**:
  New gradients
* **__group_93__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_93__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_93__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_94__new_weights** (optional) - **T2**:
  New weights
* **__group_94__new_gradients** (optional) - **T3**:
  New gradients
* **__group_94__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_94__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_94__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_95__new_weights** (optional) - **T2**:
  New weights
* **__group_95__new_gradients** (optional) - **T3**:
  New gradients
* **__group_95__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_95__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_95__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_96__new_weights** (optional) - **T2**:
  New weights
* **__group_96__new_gradients** (optional) - **T3**:
  New gradients
* **__group_96__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_96__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_96__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_97__new_weights** (optional) - **T2**:
  New weights
* **__group_97__new_gradients** (optional) - **T3**:
  New gradients
* **__group_97__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_97__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_97__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_98__new_weights** (optional) - **T2**:
  New weights
* **__group_98__new_gradients** (optional) - **T3**:
  New gradients
* **__group_98__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_98__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_98__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_99__new_weights** (optional) - **T2**:
  New weights
* **__group_99__new_gradients** (optional) - **T3**:
  New gradients
* **__group_99__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_99__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_99__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_100__new_weights** (optional) - **T2**:
  New weights
* **__group_100__new_gradients** (optional) - **T3**:
  New gradients
* **__group_100__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_100__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_100__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_101__new_weights** (optional) - **T2**:
  New weights
* **__group_101__new_gradients** (optional) - **T3**:
  New gradients
* **__group_101__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_101__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_101__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_102__new_weights** (optional) - **T2**:
  New weights
* **__group_102__new_gradients** (optional) - **T3**:
  New gradients
* **__group_102__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_102__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_102__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_103__new_weights** (optional) - **T2**:
  New weights
* **__group_103__new_gradients** (optional) - **T3**:
  New gradients
* **__group_103__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_103__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_103__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_104__new_weights** (optional) - **T2**:
  New weights
* **__group_104__new_gradients** (optional) - **T3**:
  New gradients
* **__group_104__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_104__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_104__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_105__new_weights** (optional) - **T2**:
  New weights
* **__group_105__new_gradients** (optional) - **T3**:
  New gradients
* **__group_105__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_105__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_105__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_106__new_weights** (optional) - **T2**:
  New weights
* **__group_106__new_gradients** (optional) - **T3**:
  New gradients
* **__group_106__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_106__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_106__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_107__new_weights** (optional) - **T2**:
  New weights
* **__group_107__new_gradients** (optional) - **T3**:
  New gradients
* **__group_107__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_107__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_107__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_108__new_weights** (optional) - **T2**:
  New weights
* **__group_108__new_gradients** (optional) - **T3**:
  New gradients
* **__group_108__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_108__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_108__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_109__new_weights** (optional) - **T2**:
  New weights
* **__group_109__new_gradients** (optional) - **T3**:
  New gradients
* **__group_109__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_109__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_109__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_110__new_weights** (optional) - **T2**:
  New weights
* **__group_110__new_gradients** (optional) - **T3**:
  New gradients
* **__group_110__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_110__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_110__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_111__new_weights** (optional) - **T2**:
  New weights
* **__group_111__new_gradients** (optional) - **T3**:
  New gradients
* **__group_111__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_111__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_111__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_112__new_weights** (optional) - **T2**:
  New weights
* **__group_112__new_gradients** (optional) - **T3**:
  New gradients
* **__group_112__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_112__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_112__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_113__new_weights** (optional) - **T2**:
  New weights
* **__group_113__new_gradients** (optional) - **T3**:
  New gradients
* **__group_113__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_113__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_113__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_114__new_weights** (optional) - **T2**:
  New weights
* **__group_114__new_gradients** (optional) - **T3**:
  New gradients
* **__group_114__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_114__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_114__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_115__new_weights** (optional) - **T2**:
  New weights
* **__group_115__new_gradients** (optional) - **T3**:
  New gradients
* **__group_115__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_115__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_115__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_116__new_weights** (optional) - **T2**:
  New weights
* **__group_116__new_gradients** (optional) - **T3**:
  New gradients
* **__group_116__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_116__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_116__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_117__new_weights** (optional) - **T2**:
  New weights
* **__group_117__new_gradients** (optional) - **T3**:
  New gradients
* **__group_117__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_117__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_117__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_118__new_weights** (optional) - **T2**:
  New weights
* **__group_118__new_gradients** (optional) - **T3**:
  New gradients
* **__group_118__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_118__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_118__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_119__new_weights** (optional) - **T2**:
  New weights
* **__group_119__new_gradients** (optional) - **T3**:
  New gradients
* **__group_119__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_119__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_119__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_120__new_weights** (optional) - **T2**:
  New weights
* **__group_120__new_gradients** (optional) - **T3**:
  New gradients
* **__group_120__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_120__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_120__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_121__new_weights** (optional) - **T2**:
  New weights
* **__group_121__new_gradients** (optional) - **T3**:
  New gradients
* **__group_121__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_121__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_121__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_122__new_weights** (optional) - **T2**:
  New weights
* **__group_122__new_gradients** (optional) - **T3**:
  New gradients
* **__group_122__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_122__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_122__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_123__new_weights** (optional) - **T2**:
  New weights
* **__group_123__new_gradients** (optional) - **T3**:
  New gradients
* **__group_123__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_123__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_123__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_124__new_weights** (optional) - **T2**:
  New weights
* **__group_124__new_gradients** (optional) - **T3**:
  New gradients
* **__group_124__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_124__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_124__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_125__new_weights** (optional) - **T2**:
  New weights
* **__group_125__new_gradients** (optional) - **T3**:
  New gradients
* **__group_125__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_125__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_125__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_126__new_weights** (optional) - **T2**:
  New weights
* **__group_126__new_gradients** (optional) - **T3**:
  New gradients
* **__group_126__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_126__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_126__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_127__new_weights** (optional) - **T2**:
  New weights
* **__group_127__new_gradients** (optional) - **T3**:
  New gradients
* **__group_127__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_127__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_127__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_128__new_weights** (optional) - **T2**:
  New weights
* **__group_128__new_gradients** (optional) - **T3**:
  New gradients
* **__group_128__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_128__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_128__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_129__new_weights** (optional) - **T2**:
  New weights
* **__group_129__new_gradients** (optional) - **T3**:
  New gradients
* **__group_129__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_129__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_129__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_130__new_weights** (optional) - **T2**:
  New weights
* **__group_130__new_gradients** (optional) - **T3**:
  New gradients
* **__group_130__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_130__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_130__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_131__new_weights** (optional) - **T2**:
  New weights
* **__group_131__new_gradients** (optional) - **T3**:
  New gradients
* **__group_131__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_131__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_131__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_132__new_weights** (optional) - **T2**:
  New weights
* **__group_132__new_gradients** (optional) - **T3**:
  New gradients
* **__group_132__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_132__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_132__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_133__new_weights** (optional) - **T2**:
  New weights
* **__group_133__new_gradients** (optional) - **T3**:
  New gradients
* **__group_133__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_133__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_133__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_134__new_weights** (optional) - **T2**:
  New weights
* **__group_134__new_gradients** (optional) - **T3**:
  New gradients
* **__group_134__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_134__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_134__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_135__new_weights** (optional) - **T2**:
  New weights
* **__group_135__new_gradients** (optional) - **T3**:
  New gradients
* **__group_135__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_135__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_135__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_136__new_weights** (optional) - **T2**:
  New weights
* **__group_136__new_gradients** (optional) - **T3**:
  New gradients
* **__group_136__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_136__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_136__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_137__new_weights** (optional) - **T2**:
  New weights
* **__group_137__new_gradients** (optional) - **T3**:
  New gradients
* **__group_137__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_137__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_137__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_138__new_weights** (optional) - **T2**:
  New weights
* **__group_138__new_gradients** (optional) - **T3**:
  New gradients
* **__group_138__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_138__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_138__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_139__new_weights** (optional) - **T2**:
  New weights
* **__group_139__new_gradients** (optional) - **T3**:
  New gradients
* **__group_139__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_139__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_139__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_140__new_weights** (optional) - **T2**:
  New weights
* **__group_140__new_gradients** (optional) - **T3**:
  New gradients
* **__group_140__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_140__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_140__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_141__new_weights** (optional) - **T2**:
  New weights
* **__group_141__new_gradients** (optional) - **T3**:
  New gradients
* **__group_141__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_141__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_141__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_142__new_weights** (optional) - **T2**:
  New weights
* **__group_142__new_gradients** (optional) - **T3**:
  New gradients
* **__group_142__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_142__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_142__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_143__new_weights** (optional) - **T2**:
  New weights
* **__group_143__new_gradients** (optional) - **T3**:
  New gradients
* **__group_143__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_143__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_143__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_144__new_weights** (optional) - **T2**:
  New weights
* **__group_144__new_gradients** (optional) - **T3**:
  New gradients
* **__group_144__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_144__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_144__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_145__new_weights** (optional) - **T2**:
  New weights
* **__group_145__new_gradients** (optional) - **T3**:
  New gradients
* **__group_145__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_145__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_145__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_146__new_weights** (optional) - **T2**:
  New weights
* **__group_146__new_gradients** (optional) - **T3**:
  New gradients
* **__group_146__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_146__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_146__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_147__new_weights** (optional) - **T2**:
  New weights
* **__group_147__new_gradients** (optional) - **T3**:
  New gradients
* **__group_147__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_147__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_147__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_148__new_weights** (optional) - **T2**:
  New weights
* **__group_148__new_gradients** (optional) - **T3**:
  New gradients
* **__group_148__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_148__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_148__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_149__new_weights** (optional) - **T2**:
  New weights
* **__group_149__new_gradients** (optional) - **T3**:
  New gradients
* **__group_149__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_149__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_149__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_150__new_weights** (optional) - **T2**:
  New weights
* **__group_150__new_gradients** (optional) - **T3**:
  New gradients
* **__group_150__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_150__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_150__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_151__new_weights** (optional) - **T2**:
  New weights
* **__group_151__new_gradients** (optional) - **T3**:
  New gradients
* **__group_151__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_151__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_151__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_152__new_weights** (optional) - **T2**:
  New weights
* **__group_152__new_gradients** (optional) - **T3**:
  New gradients
* **__group_152__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_152__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_152__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_153__new_weights** (optional) - **T2**:
  New weights
* **__group_153__new_gradients** (optional) - **T3**:
  New gradients
* **__group_153__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_153__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_153__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_154__new_weights** (optional) - **T2**:
  New weights
* **__group_154__new_gradients** (optional) - **T3**:
  New gradients
* **__group_154__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_154__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_154__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_155__new_weights** (optional) - **T2**:
  New weights
* **__group_155__new_gradients** (optional) - **T3**:
  New gradients
* **__group_155__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_155__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_155__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_156__new_weights** (optional) - **T2**:
  New weights
* **__group_156__new_gradients** (optional) - **T3**:
  New gradients
* **__group_156__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_156__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_156__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_157__new_weights** (optional) - **T2**:
  New weights
* **__group_157__new_gradients** (optional) - **T3**:
  New gradients
* **__group_157__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_157__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_157__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_158__new_weights** (optional) - **T2**:
  New weights
* **__group_158__new_gradients** (optional) - **T3**:
  New gradients
* **__group_158__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_158__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_158__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_159__new_weights** (optional) - **T2**:
  New weights
* **__group_159__new_gradients** (optional) - **T3**:
  New gradients
* **__group_159__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_159__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_159__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_160__new_weights** (optional) - **T2**:
  New weights
* **__group_160__new_gradients** (optional) - **T3**:
  New gradients
* **__group_160__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_160__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_160__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_161__new_weights** (optional) - **T2**:
  New weights
* **__group_161__new_gradients** (optional) - **T3**:
  New gradients
* **__group_161__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_161__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_161__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_162__new_weights** (optional) - **T2**:
  New weights
* **__group_162__new_gradients** (optional) - **T3**:
  New gradients
* **__group_162__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_162__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_162__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_163__new_weights** (optional) - **T2**:
  New weights
* **__group_163__new_gradients** (optional) - **T3**:
  New gradients
* **__group_163__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_163__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_163__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_164__new_weights** (optional) - **T2**:
  New weights
* **__group_164__new_gradients** (optional) - **T3**:
  New gradients
* **__group_164__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_164__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_164__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_165__new_weights** (optional) - **T2**:
  New weights
* **__group_165__new_gradients** (optional) - **T3**:
  New gradients
* **__group_165__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_165__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_165__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_166__new_weights** (optional) - **T2**:
  New weights
* **__group_166__new_gradients** (optional) - **T3**:
  New gradients
* **__group_166__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_166__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_166__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_167__new_weights** (optional) - **T2**:
  New weights
* **__group_167__new_gradients** (optional) - **T3**:
  New gradients
* **__group_167__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_167__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_167__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_168__new_weights** (optional) - **T2**:
  New weights
* **__group_168__new_gradients** (optional) - **T3**:
  New gradients
* **__group_168__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_168__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_168__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_169__new_weights** (optional) - **T2**:
  New weights
* **__group_169__new_gradients** (optional) - **T3**:
  New gradients
* **__group_169__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_169__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_169__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_170__new_weights** (optional) - **T2**:
  New weights
* **__group_170__new_gradients** (optional) - **T3**:
  New gradients
* **__group_170__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_170__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_170__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_171__new_weights** (optional) - **T2**:
  New weights
* **__group_171__new_gradients** (optional) - **T3**:
  New gradients
* **__group_171__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_171__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_171__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_172__new_weights** (optional) - **T2**:
  New weights
* **__group_172__new_gradients** (optional) - **T3**:
  New gradients
* **__group_172__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_172__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_172__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_173__new_weights** (optional) - **T2**:
  New weights
* **__group_173__new_gradients** (optional) - **T3**:
  New gradients
* **__group_173__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_173__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_173__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_174__new_weights** (optional) - **T2**:
  New weights
* **__group_174__new_gradients** (optional) - **T3**:
  New gradients
* **__group_174__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_174__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_174__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_175__new_weights** (optional) - **T2**:
  New weights
* **__group_175__new_gradients** (optional) - **T3**:
  New gradients
* **__group_175__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_175__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_175__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_176__new_weights** (optional) - **T2**:
  New weights
* **__group_176__new_gradients** (optional) - **T3**:
  New gradients
* **__group_176__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_176__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_176__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_177__new_weights** (optional) - **T2**:
  New weights
* **__group_177__new_gradients** (optional) - **T3**:
  New gradients
* **__group_177__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_177__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_177__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_178__new_weights** (optional) - **T2**:
  New weights
* **__group_178__new_gradients** (optional) - **T3**:
  New gradients
* **__group_178__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_178__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_178__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_179__new_weights** (optional) - **T2**:
  New weights
* **__group_179__new_gradients** (optional) - **T3**:
  New gradients
* **__group_179__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_179__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_179__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_180__new_weights** (optional) - **T2**:
  New weights
* **__group_180__new_gradients** (optional) - **T3**:
  New gradients
* **__group_180__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_180__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_180__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_181__new_weights** (optional) - **T2**:
  New weights
* **__group_181__new_gradients** (optional) - **T3**:
  New gradients
* **__group_181__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_181__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_181__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_182__new_weights** (optional) - **T2**:
  New weights
* **__group_182__new_gradients** (optional) - **T3**:
  New gradients
* **__group_182__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_182__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_182__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_183__new_weights** (optional) - **T2**:
  New weights
* **__group_183__new_gradients** (optional) - **T3**:
  New gradients
* **__group_183__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_183__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_183__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_184__new_weights** (optional) - **T2**:
  New weights
* **__group_184__new_gradients** (optional) - **T3**:
  New gradients
* **__group_184__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_184__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_184__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_185__new_weights** (optional) - **T2**:
  New weights
* **__group_185__new_gradients** (optional) - **T3**:
  New gradients
* **__group_185__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_185__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_185__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_186__new_weights** (optional) - **T2**:
  New weights
* **__group_186__new_gradients** (optional) - **T3**:
  New gradients
* **__group_186__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_186__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_186__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_187__new_weights** (optional) - **T2**:
  New weights
* **__group_187__new_gradients** (optional) - **T3**:
  New gradients
* **__group_187__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_187__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_187__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_188__new_weights** (optional) - **T2**:
  New weights
* **__group_188__new_gradients** (optional) - **T3**:
  New gradients
* **__group_188__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_188__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_188__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_189__new_weights** (optional) - **T2**:
  New weights
* **__group_189__new_gradients** (optional) - **T3**:
  New gradients
* **__group_189__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_189__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_189__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_190__new_weights** (optional) - **T2**:
  New weights
* **__group_190__new_gradients** (optional) - **T3**:
  New gradients
* **__group_190__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_190__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_190__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_191__new_weights** (optional) - **T2**:
  New weights
* **__group_191__new_gradients** (optional) - **T3**:
  New gradients
* **__group_191__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_191__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_191__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_192__new_weights** (optional) - **T2**:
  New weights
* **__group_192__new_gradients** (optional) - **T3**:
  New gradients
* **__group_192__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_192__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_192__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_193__new_weights** (optional) - **T2**:
  New weights
* **__group_193__new_gradients** (optional) - **T3**:
  New gradients
* **__group_193__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_193__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_193__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_194__new_weights** (optional) - **T2**:
  New weights
* **__group_194__new_gradients** (optional) - **T3**:
  New gradients
* **__group_194__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_194__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_194__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_195__new_weights** (optional) - **T2**:
  New weights
* **__group_195__new_gradients** (optional) - **T3**:
  New gradients
* **__group_195__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_195__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_195__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_196__new_weights** (optional) - **T2**:
  New weights
* **__group_196__new_gradients** (optional) - **T3**:
  New gradients
* **__group_196__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_196__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_196__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_197__new_weights** (optional) - **T2**:
  New weights
* **__group_197__new_gradients** (optional) - **T3**:
  New gradients
* **__group_197__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_197__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_197__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_198__new_weights** (optional) - **T2**:
  New weights
* **__group_198__new_gradients** (optional) - **T3**:
  New gradients
* **__group_198__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_198__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_198__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_199__new_weights** (optional) - **T2**:
  New weights
* **__group_199__new_gradients** (optional) - **T3**:
  New gradients
* **__group_199__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_199__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_199__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_200__new_weights** (optional) - **T2**:
  New weights
* **__group_200__new_gradients** (optional) - **T3**:
  New gradients
* **__group_200__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_200__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_200__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_201__new_weights** (optional) - **T2**:
  New weights
* **__group_201__new_gradients** (optional) - **T3**:
  New gradients
* **__group_201__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_201__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_201__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_202__new_weights** (optional) - **T2**:
  New weights
* **__group_202__new_gradients** (optional) - **T3**:
  New gradients
* **__group_202__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_202__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_202__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_203__new_weights** (optional) - **T2**:
  New weights
* **__group_203__new_gradients** (optional) - **T3**:
  New gradients
* **__group_203__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_203__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_203__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_204__new_weights** (optional) - **T2**:
  New weights
* **__group_204__new_gradients** (optional) - **T3**:
  New gradients
* **__group_204__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_204__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_204__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_205__new_weights** (optional) - **T2**:
  New weights
* **__group_205__new_gradients** (optional) - **T3**:
  New gradients
* **__group_205__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_205__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_205__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_206__new_weights** (optional) - **T2**:
  New weights
* **__group_206__new_gradients** (optional) - **T3**:
  New gradients
* **__group_206__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_206__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_206__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_207__new_weights** (optional) - **T2**:
  New weights
* **__group_207__new_gradients** (optional) - **T3**:
  New gradients
* **__group_207__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_207__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_207__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_208__new_weights** (optional) - **T2**:
  New weights
* **__group_208__new_gradients** (optional) - **T3**:
  New gradients
* **__group_208__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_208__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_208__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_209__new_weights** (optional) - **T2**:
  New weights
* **__group_209__new_gradients** (optional) - **T3**:
  New gradients
* **__group_209__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_209__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_209__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_210__new_weights** (optional) - **T2**:
  New weights
* **__group_210__new_gradients** (optional) - **T3**:
  New gradients
* **__group_210__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_210__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_210__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_211__new_weights** (optional) - **T2**:
  New weights
* **__group_211__new_gradients** (optional) - **T3**:
  New gradients
* **__group_211__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_211__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_211__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_212__new_weights** (optional) - **T2**:
  New weights
* **__group_212__new_gradients** (optional) - **T3**:
  New gradients
* **__group_212__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_212__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_212__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_213__new_weights** (optional) - **T2**:
  New weights
* **__group_213__new_gradients** (optional) - **T3**:
  New gradients
* **__group_213__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_213__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_213__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_214__new_weights** (optional) - **T2**:
  New weights
* **__group_214__new_gradients** (optional) - **T3**:
  New gradients
* **__group_214__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_214__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_214__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_215__new_weights** (optional) - **T2**:
  New weights
* **__group_215__new_gradients** (optional) - **T3**:
  New gradients
* **__group_215__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_215__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_215__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_216__new_weights** (optional) - **T2**:
  New weights
* **__group_216__new_gradients** (optional) - **T3**:
  New gradients
* **__group_216__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_216__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_216__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_217__new_weights** (optional) - **T2**:
  New weights
* **__group_217__new_gradients** (optional) - **T3**:
  New gradients
* **__group_217__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_217__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_217__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_218__new_weights** (optional) - **T2**:
  New weights
* **__group_218__new_gradients** (optional) - **T3**:
  New gradients
* **__group_218__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_218__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_218__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_219__new_weights** (optional) - **T2**:
  New weights
* **__group_219__new_gradients** (optional) - **T3**:
  New gradients
* **__group_219__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_219__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_219__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_220__new_weights** (optional) - **T2**:
  New weights
* **__group_220__new_gradients** (optional) - **T3**:
  New gradients
* **__group_220__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_220__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_220__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_221__new_weights** (optional) - **T2**:
  New weights
* **__group_221__new_gradients** (optional) - **T3**:
  New gradients
* **__group_221__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_221__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_221__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_222__new_weights** (optional) - **T2**:
  New weights
* **__group_222__new_gradients** (optional) - **T3**:
  New gradients
* **__group_222__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_222__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_222__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_223__new_weights** (optional) - **T2**:
  New weights
* **__group_223__new_gradients** (optional) - **T3**:
  New gradients
* **__group_223__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_223__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_223__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_224__new_weights** (optional) - **T2**:
  New weights
* **__group_224__new_gradients** (optional) - **T3**:
  New gradients
* **__group_224__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_224__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_224__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_225__new_weights** (optional) - **T2**:
  New weights
* **__group_225__new_gradients** (optional) - **T3**:
  New gradients
* **__group_225__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_225__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_225__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_226__new_weights** (optional) - **T2**:
  New weights
* **__group_226__new_gradients** (optional) - **T3**:
  New gradients
* **__group_226__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_226__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_226__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_227__new_weights** (optional) - **T2**:
  New weights
* **__group_227__new_gradients** (optional) - **T3**:
  New gradients
* **__group_227__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_227__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_227__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_228__new_weights** (optional) - **T2**:
  New weights
* **__group_228__new_gradients** (optional) - **T3**:
  New gradients
* **__group_228__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_228__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_228__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_229__new_weights** (optional) - **T2**:
  New weights
* **__group_229__new_gradients** (optional) - **T3**:
  New gradients
* **__group_229__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_229__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_229__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_230__new_weights** (optional) - **T2**:
  New weights
* **__group_230__new_gradients** (optional) - **T3**:
  New gradients
* **__group_230__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_230__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_230__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_231__new_weights** (optional) - **T2**:
  New weights
* **__group_231__new_gradients** (optional) - **T3**:
  New gradients
* **__group_231__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_231__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_231__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_232__new_weights** (optional) - **T2**:
  New weights
* **__group_232__new_gradients** (optional) - **T3**:
  New gradients
* **__group_232__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_232__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_232__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_233__new_weights** (optional) - **T2**:
  New weights
* **__group_233__new_gradients** (optional) - **T3**:
  New gradients
* **__group_233__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_233__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_233__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_234__new_weights** (optional) - **T2**:
  New weights
* **__group_234__new_gradients** (optional) - **T3**:
  New gradients
* **__group_234__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_234__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_234__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_235__new_weights** (optional) - **T2**:
  New weights
* **__group_235__new_gradients** (optional) - **T3**:
  New gradients
* **__group_235__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_235__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_235__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_236__new_weights** (optional) - **T2**:
  New weights
* **__group_236__new_gradients** (optional) - **T3**:
  New gradients
* **__group_236__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_236__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_236__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_237__new_weights** (optional) - **T2**:
  New weights
* **__group_237__new_gradients** (optional) - **T3**:
  New gradients
* **__group_237__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_237__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_237__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_238__new_weights** (optional) - **T2**:
  New weights
* **__group_238__new_gradients** (optional) - **T3**:
  New gradients
* **__group_238__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_238__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_238__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_239__new_weights** (optional) - **T2**:
  New weights
* **__group_239__new_gradients** (optional) - **T3**:
  New gradients
* **__group_239__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_239__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_239__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_240__new_weights** (optional) - **T2**:
  New weights
* **__group_240__new_gradients** (optional) - **T3**:
  New gradients
* **__group_240__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_240__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_240__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_241__new_weights** (optional) - **T2**:
  New weights
* **__group_241__new_gradients** (optional) - **T3**:
  New gradients
* **__group_241__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_241__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_241__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_242__new_weights** (optional) - **T2**:
  New weights
* **__group_242__new_gradients** (optional) - **T3**:
  New gradients
* **__group_242__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_242__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_242__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_243__new_weights** (optional) - **T2**:
  New weights
* **__group_243__new_gradients** (optional) - **T3**:
  New gradients
* **__group_243__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_243__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_243__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_244__new_weights** (optional) - **T2**:
  New weights
* **__group_244__new_gradients** (optional) - **T3**:
  New gradients
* **__group_244__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_244__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_244__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_245__new_weights** (optional) - **T2**:
  New weights
* **__group_245__new_gradients** (optional) - **T3**:
  New gradients
* **__group_245__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_245__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_245__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_246__new_weights** (optional) - **T2**:
  New weights
* **__group_246__new_gradients** (optional) - **T3**:
  New gradients
* **__group_246__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_246__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_246__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_247__new_weights** (optional) - **T2**:
  New weights
* **__group_247__new_gradients** (optional) - **T3**:
  New gradients
* **__group_247__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_247__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_247__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_248__new_weights** (optional) - **T2**:
  New weights
* **__group_248__new_gradients** (optional) - **T3**:
  New gradients
* **__group_248__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_248__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_248__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_249__new_weights** (optional) - **T2**:
  New weights
* **__group_249__new_gradients** (optional) - **T3**:
  New gradients
* **__group_249__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_249__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_249__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_250__new_weights** (optional) - **T2**:
  New weights
* **__group_250__new_gradients** (optional) - **T3**:
  New gradients
* **__group_250__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_250__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_250__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_251__new_weights** (optional) - **T2**:
  New weights
* **__group_251__new_gradients** (optional) - **T3**:
  New gradients
* **__group_251__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_251__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_251__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_252__new_weights** (optional) - **T2**:
  New weights
* **__group_252__new_gradients** (optional) - **T3**:
  New gradients
* **__group_252__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_252__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_252__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_253__new_weights** (optional) - **T2**:
  New weights
* **__group_253__new_gradients** (optional) - **T3**:
  New gradients
* **__group_253__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_253__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_253__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_254__new_weights** (optional) - **T2**:
  New weights
* **__group_254__new_gradients** (optional) - **T3**:
  New gradients
* **__group_254__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_254__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_254__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_255__new_weights** (optional) - **T2**:
  New weights
* **__group_255__new_gradients** (optional) - **T3**:
  New gradients
* **__group_255__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_255__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_255__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_256__new_weights** (optional) - **T2**:
  New weights
* **__group_256__new_gradients** (optional) - **T3**:
  New gradients
* **__group_256__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_256__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_256__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_257__new_weights** (optional) - **T2**:
  New weights
* **__group_257__new_gradients** (optional) - **T3**:
  New gradients
* **__group_257__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_257__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_257__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_258__new_weights** (optional) - **T2**:
  New weights
* **__group_258__new_gradients** (optional) - **T3**:
  New gradients
* **__group_258__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_258__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_258__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_259__new_weights** (optional) - **T2**:
  New weights
* **__group_259__new_gradients** (optional) - **T3**:
  New gradients
* **__group_259__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_259__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_259__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_260__new_weights** (optional) - **T2**:
  New weights
* **__group_260__new_gradients** (optional) - **T3**:
  New gradients
* **__group_260__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_260__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_260__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_261__new_weights** (optional) - **T2**:
  New weights
* **__group_261__new_gradients** (optional) - **T3**:
  New gradients
* **__group_261__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_261__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_261__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_262__new_weights** (optional) - **T2**:
  New weights
* **__group_262__new_gradients** (optional) - **T3**:
  New gradients
* **__group_262__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_262__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_262__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_263__new_weights** (optional) - **T2**:
  New weights
* **__group_263__new_gradients** (optional) - **T3**:
  New gradients
* **__group_263__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_263__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_263__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_264__new_weights** (optional) - **T2**:
  New weights
* **__group_264__new_gradients** (optional) - **T3**:
  New gradients
* **__group_264__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_264__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_264__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_265__new_weights** (optional) - **T2**:
  New weights
* **__group_265__new_gradients** (optional) - **T3**:
  New gradients
* **__group_265__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_265__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_265__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_266__new_weights** (optional) - **T2**:
  New weights
* **__group_266__new_gradients** (optional) - **T3**:
  New gradients
* **__group_266__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_266__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_266__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_267__new_weights** (optional) - **T2**:
  New weights
* **__group_267__new_gradients** (optional) - **T3**:
  New gradients
* **__group_267__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_267__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_267__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_268__new_weights** (optional) - **T2**:
  New weights
* **__group_268__new_gradients** (optional) - **T3**:
  New gradients
* **__group_268__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_268__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_268__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_269__new_weights** (optional) - **T2**:
  New weights
* **__group_269__new_gradients** (optional) - **T3**:
  New gradients
* **__group_269__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_269__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_269__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_270__new_weights** (optional) - **T2**:
  New weights
* **__group_270__new_gradients** (optional) - **T3**:
  New gradients
* **__group_270__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_270__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_270__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_271__new_weights** (optional) - **T2**:
  New weights
* **__group_271__new_gradients** (optional) - **T3**:
  New gradients
* **__group_271__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_271__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_271__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_272__new_weights** (optional) - **T2**:
  New weights
* **__group_272__new_gradients** (optional) - **T3**:
  New gradients
* **__group_272__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_272__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_272__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_273__new_weights** (optional) - **T2**:
  New weights
* **__group_273__new_gradients** (optional) - **T3**:
  New gradients
* **__group_273__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_273__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_273__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_274__new_weights** (optional) - **T2**:
  New weights
* **__group_274__new_gradients** (optional) - **T3**:
  New gradients
* **__group_274__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_274__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_274__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_275__new_weights** (optional) - **T2**:
  New weights
* **__group_275__new_gradients** (optional) - **T3**:
  New gradients
* **__group_275__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_275__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_275__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_276__new_weights** (optional) - **T2**:
  New weights
* **__group_276__new_gradients** (optional) - **T3**:
  New gradients
* **__group_276__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_276__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_276__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_277__new_weights** (optional) - **T2**:
  New weights
* **__group_277__new_gradients** (optional) - **T3**:
  New gradients
* **__group_277__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_277__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_277__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_278__new_weights** (optional) - **T2**:
  New weights
* **__group_278__new_gradients** (optional) - **T3**:
  New gradients
* **__group_278__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_278__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_278__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_279__new_weights** (optional) - **T2**:
  New weights
* **__group_279__new_gradients** (optional) - **T3**:
  New gradients
* **__group_279__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_279__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_279__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_280__new_weights** (optional) - **T2**:
  New weights
* **__group_280__new_gradients** (optional) - **T3**:
  New gradients
* **__group_280__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_280__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_280__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_281__new_weights** (optional) - **T2**:
  New weights
* **__group_281__new_gradients** (optional) - **T3**:
  New gradients
* **__group_281__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_281__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_281__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_282__new_weights** (optional) - **T2**:
  New weights
* **__group_282__new_gradients** (optional) - **T3**:
  New gradients
* **__group_282__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_282__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_282__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_283__new_weights** (optional) - **T2**:
  New weights
* **__group_283__new_gradients** (optional) - **T3**:
  New gradients
* **__group_283__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_283__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_283__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_284__new_weights** (optional) - **T2**:
  New weights
* **__group_284__new_gradients** (optional) - **T3**:
  New gradients
* **__group_284__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_284__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_284__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_285__new_weights** (optional) - **T2**:
  New weights
* **__group_285__new_gradients** (optional) - **T3**:
  New gradients
* **__group_285__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_285__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_285__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_286__new_weights** (optional) - **T2**:
  New weights
* **__group_286__new_gradients** (optional) - **T3**:
  New gradients
* **__group_286__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_286__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_286__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_287__new_weights** (optional) - **T2**:
  New weights
* **__group_287__new_gradients** (optional) - **T3**:
  New gradients
* **__group_287__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_287__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_287__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_288__new_weights** (optional) - **T2**:
  New weights
* **__group_288__new_gradients** (optional) - **T3**:
  New gradients
* **__group_288__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_288__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_288__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_289__new_weights** (optional) - **T2**:
  New weights
* **__group_289__new_gradients** (optional) - **T3**:
  New gradients
* **__group_289__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_289__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_289__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_290__new_weights** (optional) - **T2**:
  New weights
* **__group_290__new_gradients** (optional) - **T3**:
  New gradients
* **__group_290__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_290__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_290__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_291__new_weights** (optional) - **T2**:
  New weights
* **__group_291__new_gradients** (optional) - **T3**:
  New gradients
* **__group_291__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_291__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_291__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_292__new_weights** (optional) - **T2**:
  New weights
* **__group_292__new_gradients** (optional) - **T3**:
  New gradients
* **__group_292__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_292__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_292__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_293__new_weights** (optional) - **T2**:
  New weights
* **__group_293__new_gradients** (optional) - **T3**:
  New gradients
* **__group_293__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_293__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_293__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_294__new_weights** (optional) - **T2**:
  New weights
* **__group_294__new_gradients** (optional) - **T3**:
  New gradients
* **__group_294__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_294__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_294__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_295__new_weights** (optional) - **T2**:
  New weights
* **__group_295__new_gradients** (optional) - **T3**:
  New gradients
* **__group_295__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_295__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_295__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_296__new_weights** (optional) - **T2**:
  New weights
* **__group_296__new_gradients** (optional) - **T3**:
  New gradients
* **__group_296__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_296__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_296__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_297__new_weights** (optional) - **T2**:
  New weights
* **__group_297__new_gradients** (optional) - **T3**:
  New gradients
* **__group_297__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_297__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_297__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_298__new_weights** (optional) - **T2**:
  New weights
* **__group_298__new_gradients** (optional) - **T3**:
  New gradients
* **__group_298__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_298__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_298__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_299__new_weights** (optional) - **T2**:
  New weights
* **__group_299__new_gradients** (optional) - **T3**:
  New gradients
* **__group_299__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_299__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_299__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_300__new_weights** (optional) - **T2**:
  New weights
* **__group_300__new_gradients** (optional) - **T3**:
  New gradients
* **__group_300__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_300__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_300__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_301__new_weights** (optional) - **T2**:
  New weights
* **__group_301__new_gradients** (optional) - **T3**:
  New gradients
* **__group_301__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_301__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_301__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_302__new_weights** (optional) - **T2**:
  New weights
* **__group_302__new_gradients** (optional) - **T3**:
  New gradients
* **__group_302__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_302__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_302__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_303__new_weights** (optional) - **T2**:
  New weights
* **__group_303__new_gradients** (optional) - **T3**:
  New gradients
* **__group_303__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_303__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_303__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_304__new_weights** (optional) - **T2**:
  New weights
* **__group_304__new_gradients** (optional) - **T3**:
  New gradients
* **__group_304__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_304__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_304__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_305__new_weights** (optional) - **T2**:
  New weights
* **__group_305__new_gradients** (optional) - **T3**:
  New gradients
* **__group_305__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_305__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_305__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_306__new_weights** (optional) - **T2**:
  New weights
* **__group_306__new_gradients** (optional) - **T3**:
  New gradients
* **__group_306__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_306__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_306__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_307__new_weights** (optional) - **T2**:
  New weights
* **__group_307__new_gradients** (optional) - **T3**:
  New gradients
* **__group_307__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_307__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_307__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_308__new_weights** (optional) - **T2**:
  New weights
* **__group_308__new_gradients** (optional) - **T3**:
  New gradients
* **__group_308__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_308__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_308__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_309__new_weights** (optional) - **T2**:
  New weights
* **__group_309__new_gradients** (optional) - **T3**:
  New gradients
* **__group_309__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_309__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_309__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_310__new_weights** (optional) - **T2**:
  New weights
* **__group_310__new_gradients** (optional) - **T3**:
  New gradients
* **__group_310__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_310__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_310__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_311__new_weights** (optional) - **T2**:
  New weights
* **__group_311__new_gradients** (optional) - **T3**:
  New gradients
* **__group_311__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_311__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_311__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_312__new_weights** (optional) - **T2**:
  New weights
* **__group_312__new_gradients** (optional) - **T3**:
  New gradients
* **__group_312__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_312__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_312__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_313__new_weights** (optional) - **T2**:
  New weights
* **__group_313__new_gradients** (optional) - **T3**:
  New gradients
* **__group_313__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_313__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_313__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_314__new_weights** (optional) - **T2**:
  New weights
* **__group_314__new_gradients** (optional) - **T3**:
  New gradients
* **__group_314__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_314__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_314__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_315__new_weights** (optional) - **T2**:
  New weights
* **__group_315__new_gradients** (optional) - **T3**:
  New gradients
* **__group_315__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_315__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_315__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_316__new_weights** (optional) - **T2**:
  New weights
* **__group_316__new_gradients** (optional) - **T3**:
  New gradients
* **__group_316__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_316__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_316__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_317__new_weights** (optional) - **T2**:
  New weights
* **__group_317__new_gradients** (optional) - **T3**:
  New gradients
* **__group_317__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_317__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_317__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_318__new_weights** (optional) - **T2**:
  New weights
* **__group_318__new_gradients** (optional) - **T3**:
  New gradients
* **__group_318__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_318__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_318__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_319__new_weights** (optional) - **T2**:
  New weights
* **__group_319__new_gradients** (optional) - **T3**:
  New gradients
* **__group_319__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_319__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_319__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_320__new_weights** (optional) - **T2**:
  New weights
* **__group_320__new_gradients** (optional) - **T3**:
  New gradients
* **__group_320__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_320__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_320__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_321__new_weights** (optional) - **T2**:
  New weights
* **__group_321__new_gradients** (optional) - **T3**:
  New gradients
* **__group_321__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_321__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_321__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_322__new_weights** (optional) - **T2**:
  New weights
* **__group_322__new_gradients** (optional) - **T3**:
  New gradients
* **__group_322__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_322__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_322__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_323__new_weights** (optional) - **T2**:
  New weights
* **__group_323__new_gradients** (optional) - **T3**:
  New gradients
* **__group_323__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_323__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_323__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_324__new_weights** (optional) - **T2**:
  New weights
* **__group_324__new_gradients** (optional) - **T3**:
  New gradients
* **__group_324__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_324__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_324__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_325__new_weights** (optional) - **T2**:
  New weights
* **__group_325__new_gradients** (optional) - **T3**:
  New gradients
* **__group_325__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_325__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_325__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_326__new_weights** (optional) - **T2**:
  New weights
* **__group_326__new_gradients** (optional) - **T3**:
  New gradients
* **__group_326__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_326__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_326__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_327__new_weights** (optional) - **T2**:
  New weights
* **__group_327__new_gradients** (optional) - **T3**:
  New gradients
* **__group_327__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_327__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_327__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_328__new_weights** (optional) - **T2**:
  New weights
* **__group_328__new_gradients** (optional) - **T3**:
  New gradients
* **__group_328__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_328__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_328__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_329__new_weights** (optional) - **T2**:
  New weights
* **__group_329__new_gradients** (optional) - **T3**:
  New gradients
* **__group_329__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_329__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_329__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_330__new_weights** (optional) - **T2**:
  New weights
* **__group_330__new_gradients** (optional) - **T3**:
  New gradients
* **__group_330__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_330__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_330__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_331__new_weights** (optional) - **T2**:
  New weights
* **__group_331__new_gradients** (optional) - **T3**:
  New gradients
* **__group_331__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_331__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_331__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_332__new_weights** (optional) - **T2**:
  New weights
* **__group_332__new_gradients** (optional) - **T3**:
  New gradients
* **__group_332__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_332__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_332__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_333__new_weights** (optional) - **T2**:
  New weights
* **__group_333__new_gradients** (optional) - **T3**:
  New gradients
* **__group_333__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_333__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_333__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_334__new_weights** (optional) - **T2**:
  New weights
* **__group_334__new_gradients** (optional) - **T3**:
  New gradients
* **__group_334__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_334__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_334__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_335__new_weights** (optional) - **T2**:
  New weights
* **__group_335__new_gradients** (optional) - **T3**:
  New gradients
* **__group_335__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_335__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_335__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_336__new_weights** (optional) - **T2**:
  New weights
* **__group_336__new_gradients** (optional) - **T3**:
  New gradients
* **__group_336__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_336__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_336__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_337__new_weights** (optional) - **T2**:
  New weights
* **__group_337__new_gradients** (optional) - **T3**:
  New gradients
* **__group_337__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_337__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_337__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_338__new_weights** (optional) - **T2**:
  New weights
* **__group_338__new_gradients** (optional) - **T3**:
  New gradients
* **__group_338__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_338__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_338__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_339__new_weights** (optional) - **T2**:
  New weights
* **__group_339__new_gradients** (optional) - **T3**:
  New gradients
* **__group_339__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_339__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_339__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_340__new_weights** (optional) - **T2**:
  New weights
* **__group_340__new_gradients** (optional) - **T3**:
  New gradients
* **__group_340__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_340__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_340__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_341__new_weights** (optional) - **T2**:
  New weights
* **__group_341__new_gradients** (optional) - **T3**:
  New gradients
* **__group_341__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_341__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_341__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_342__new_weights** (optional) - **T2**:
  New weights
* **__group_342__new_gradients** (optional) - **T3**:
  New gradients
* **__group_342__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_342__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_342__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_343__new_weights** (optional) - **T2**:
  New weights
* **__group_343__new_gradients** (optional) - **T3**:
  New gradients
* **__group_343__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_343__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_343__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_344__new_weights** (optional) - **T2**:
  New weights
* **__group_344__new_gradients** (optional) - **T3**:
  New gradients
* **__group_344__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_344__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_344__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_345__new_weights** (optional) - **T2**:
  New weights
* **__group_345__new_gradients** (optional) - **T3**:
  New gradients
* **__group_345__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_345__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_345__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_346__new_weights** (optional) - **T2**:
  New weights
* **__group_346__new_gradients** (optional) - **T3**:
  New gradients
* **__group_346__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_346__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_346__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_347__new_weights** (optional) - **T2**:
  New weights
* **__group_347__new_gradients** (optional) - **T3**:
  New gradients
* **__group_347__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_347__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_347__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_348__new_weights** (optional) - **T2**:
  New weights
* **__group_348__new_gradients** (optional) - **T3**:
  New gradients
* **__group_348__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_348__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_348__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_349__new_weights** (optional) - **T2**:
  New weights
* **__group_349__new_gradients** (optional) - **T3**:
  New gradients
* **__group_349__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_349__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_349__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_350__new_weights** (optional) - **T2**:
  New weights
* **__group_350__new_gradients** (optional) - **T3**:
  New gradients
* **__group_350__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_350__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_350__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_351__new_weights** (optional) - **T2**:
  New weights
* **__group_351__new_gradients** (optional) - **T3**:
  New gradients
* **__group_351__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_351__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_351__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_352__new_weights** (optional) - **T2**:
  New weights
* **__group_352__new_gradients** (optional) - **T3**:
  New gradients
* **__group_352__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_352__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_352__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_353__new_weights** (optional) - **T2**:
  New weights
* **__group_353__new_gradients** (optional) - **T3**:
  New gradients
* **__group_353__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_353__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_353__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_354__new_weights** (optional) - **T2**:
  New weights
* **__group_354__new_gradients** (optional) - **T3**:
  New gradients
* **__group_354__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_354__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_354__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_355__new_weights** (optional) - **T2**:
  New weights
* **__group_355__new_gradients** (optional) - **T3**:
  New gradients
* **__group_355__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_355__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_355__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_356__new_weights** (optional) - **T2**:
  New weights
* **__group_356__new_gradients** (optional) - **T3**:
  New gradients
* **__group_356__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_356__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_356__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_357__new_weights** (optional) - **T2**:
  New weights
* **__group_357__new_gradients** (optional) - **T3**:
  New gradients
* **__group_357__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_357__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_357__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_358__new_weights** (optional) - **T2**:
  New weights
* **__group_358__new_gradients** (optional) - **T3**:
  New gradients
* **__group_358__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_358__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_358__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_359__new_weights** (optional) - **T2**:
  New weights
* **__group_359__new_gradients** (optional) - **T3**:
  New gradients
* **__group_359__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_359__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_359__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_360__new_weights** (optional) - **T2**:
  New weights
* **__group_360__new_gradients** (optional) - **T3**:
  New gradients
* **__group_360__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_360__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_360__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_361__new_weights** (optional) - **T2**:
  New weights
* **__group_361__new_gradients** (optional) - **T3**:
  New gradients
* **__group_361__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_361__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_361__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_362__new_weights** (optional) - **T2**:
  New weights
* **__group_362__new_gradients** (optional) - **T3**:
  New gradients
* **__group_362__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_362__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_362__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_363__new_weights** (optional) - **T2**:
  New weights
* **__group_363__new_gradients** (optional) - **T3**:
  New gradients
* **__group_363__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_363__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_363__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_364__new_weights** (optional) - **T2**:
  New weights
* **__group_364__new_gradients** (optional) - **T3**:
  New gradients
* **__group_364__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_364__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_364__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_365__new_weights** (optional) - **T2**:
  New weights
* **__group_365__new_gradients** (optional) - **T3**:
  New gradients
* **__group_365__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_365__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_365__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_366__new_weights** (optional) - **T2**:
  New weights
* **__group_366__new_gradients** (optional) - **T3**:
  New gradients
* **__group_366__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_366__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_366__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_367__new_weights** (optional) - **T2**:
  New weights
* **__group_367__new_gradients** (optional) - **T3**:
  New gradients
* **__group_367__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_367__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_367__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_368__new_weights** (optional) - **T2**:
  New weights
* **__group_368__new_gradients** (optional) - **T3**:
  New gradients
* **__group_368__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_368__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_368__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_369__new_weights** (optional) - **T2**:
  New weights
* **__group_369__new_gradients** (optional) - **T3**:
  New gradients
* **__group_369__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_369__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_369__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_370__new_weights** (optional) - **T2**:
  New weights
* **__group_370__new_gradients** (optional) - **T3**:
  New gradients
* **__group_370__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_370__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_370__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_371__new_weights** (optional) - **T2**:
  New weights
* **__group_371__new_gradients** (optional) - **T3**:
  New gradients
* **__group_371__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_371__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_371__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_372__new_weights** (optional) - **T2**:
  New weights
* **__group_372__new_gradients** (optional) - **T3**:
  New gradients
* **__group_372__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_372__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_372__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_373__new_weights** (optional) - **T2**:
  New weights
* **__group_373__new_gradients** (optional) - **T3**:
  New gradients
* **__group_373__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_373__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_373__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_374__new_weights** (optional) - **T2**:
  New weights
* **__group_374__new_gradients** (optional) - **T3**:
  New gradients
* **__group_374__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_374__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_374__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_375__new_weights** (optional) - **T2**:
  New weights
* **__group_375__new_gradients** (optional) - **T3**:
  New gradients
* **__group_375__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_375__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_375__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_376__new_weights** (optional) - **T2**:
  New weights
* **__group_376__new_gradients** (optional) - **T3**:
  New gradients
* **__group_376__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_376__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_376__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_377__new_weights** (optional) - **T2**:
  New weights
* **__group_377__new_gradients** (optional) - **T3**:
  New gradients
* **__group_377__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_377__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_377__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_378__new_weights** (optional) - **T2**:
  New weights
* **__group_378__new_gradients** (optional) - **T3**:
  New gradients
* **__group_378__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_378__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_378__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_379__new_weights** (optional) - **T2**:
  New weights
* **__group_379__new_gradients** (optional) - **T3**:
  New gradients
* **__group_379__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_379__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_379__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_380__new_weights** (optional) - **T2**:
  New weights
* **__group_380__new_gradients** (optional) - **T3**:
  New gradients
* **__group_380__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_380__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_380__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_381__new_weights** (optional) - **T2**:
  New weights
* **__group_381__new_gradients** (optional) - **T3**:
  New gradients
* **__group_381__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_381__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_381__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_382__new_weights** (optional) - **T2**:
  New weights
* **__group_382__new_gradients** (optional) - **T3**:
  New gradients
* **__group_382__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_382__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_382__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_383__new_weights** (optional) - **T2**:
  New weights
* **__group_383__new_gradients** (optional) - **T3**:
  New gradients
* **__group_383__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_383__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_383__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_384__new_weights** (optional) - **T2**:
  New weights
* **__group_384__new_gradients** (optional) - **T3**:
  New gradients
* **__group_384__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_384__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_384__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_385__new_weights** (optional) - **T2**:
  New weights
* **__group_385__new_gradients** (optional) - **T3**:
  New gradients
* **__group_385__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_385__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_385__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_386__new_weights** (optional) - **T2**:
  New weights
* **__group_386__new_gradients** (optional) - **T3**:
  New gradients
* **__group_386__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_386__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_386__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_387__new_weights** (optional) - **T2**:
  New weights
* **__group_387__new_gradients** (optional) - **T3**:
  New gradients
* **__group_387__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_387__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_387__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_388__new_weights** (optional) - **T2**:
  New weights
* **__group_388__new_gradients** (optional) - **T3**:
  New gradients
* **__group_388__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_388__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_388__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_389__new_weights** (optional) - **T2**:
  New weights
* **__group_389__new_gradients** (optional) - **T3**:
  New gradients
* **__group_389__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_389__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_389__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_390__new_weights** (optional) - **T2**:
  New weights
* **__group_390__new_gradients** (optional) - **T3**:
  New gradients
* **__group_390__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_390__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_390__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_391__new_weights** (optional) - **T2**:
  New weights
* **__group_391__new_gradients** (optional) - **T3**:
  New gradients
* **__group_391__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_391__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_391__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_392__new_weights** (optional) - **T2**:
  New weights
* **__group_392__new_gradients** (optional) - **T3**:
  New gradients
* **__group_392__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_392__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_392__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_393__new_weights** (optional) - **T2**:
  New weights
* **__group_393__new_gradients** (optional) - **T3**:
  New gradients
* **__group_393__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_393__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_393__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_394__new_weights** (optional) - **T2**:
  New weights
* **__group_394__new_gradients** (optional) - **T3**:
  New gradients
* **__group_394__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_394__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_394__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_395__new_weights** (optional) - **T2**:
  New weights
* **__group_395__new_gradients** (optional) - **T3**:
  New gradients
* **__group_395__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_395__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_395__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_396__new_weights** (optional) - **T2**:
  New weights
* **__group_396__new_gradients** (optional) - **T3**:
  New gradients
* **__group_396__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_396__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_396__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_397__new_weights** (optional) - **T2**:
  New weights
* **__group_397__new_gradients** (optional) - **T3**:
  New gradients
* **__group_397__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_397__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_397__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_398__new_weights** (optional) - **T2**:
  New weights
* **__group_398__new_gradients** (optional) - **T3**:
  New gradients
* **__group_398__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_398__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_398__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_399__new_weights** (optional) - **T2**:
  New weights
* **__group_399__new_gradients** (optional) - **T3**:
  New gradients
* **__group_399__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_399__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_399__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_400__new_weights** (optional) - **T2**:
  New weights
* **__group_400__new_gradients** (optional) - **T3**:
  New gradients
* **__group_400__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_400__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_400__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_401__new_weights** (optional) - **T2**:
  New weights
* **__group_401__new_gradients** (optional) - **T3**:
  New gradients
* **__group_401__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_401__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_401__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_402__new_weights** (optional) - **T2**:
  New weights
* **__group_402__new_gradients** (optional) - **T3**:
  New gradients
* **__group_402__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_402__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_402__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_403__new_weights** (optional) - **T2**:
  New weights
* **__group_403__new_gradients** (optional) - **T3**:
  New gradients
* **__group_403__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_403__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_403__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_404__new_weights** (optional) - **T2**:
  New weights
* **__group_404__new_gradients** (optional) - **T3**:
  New gradients
* **__group_404__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_404__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_404__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_405__new_weights** (optional) - **T2**:
  New weights
* **__group_405__new_gradients** (optional) - **T3**:
  New gradients
* **__group_405__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_405__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_405__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_406__new_weights** (optional) - **T2**:
  New weights
* **__group_406__new_gradients** (optional) - **T3**:
  New gradients
* **__group_406__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_406__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_406__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_407__new_weights** (optional) - **T2**:
  New weights
* **__group_407__new_gradients** (optional) - **T3**:
  New gradients
* **__group_407__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_407__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_407__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_408__new_weights** (optional) - **T2**:
  New weights
* **__group_408__new_gradients** (optional) - **T3**:
  New gradients
* **__group_408__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_408__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_408__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_409__new_weights** (optional) - **T2**:
  New weights
* **__group_409__new_gradients** (optional) - **T3**:
  New gradients
* **__group_409__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_409__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_409__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_410__new_weights** (optional) - **T2**:
  New weights
* **__group_410__new_gradients** (optional) - **T3**:
  New gradients
* **__group_410__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_410__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_410__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_411__new_weights** (optional) - **T2**:
  New weights
* **__group_411__new_gradients** (optional) - **T3**:
  New gradients
* **__group_411__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_411__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_411__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_412__new_weights** (optional) - **T2**:
  New weights
* **__group_412__new_gradients** (optional) - **T3**:
  New gradients
* **__group_412__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_412__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_412__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_413__new_weights** (optional) - **T2**:
  New weights
* **__group_413__new_gradients** (optional) - **T3**:
  New gradients
* **__group_413__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_413__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_413__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_414__new_weights** (optional) - **T2**:
  New weights
* **__group_414__new_gradients** (optional) - **T3**:
  New gradients
* **__group_414__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_414__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_414__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_415__new_weights** (optional) - **T2**:
  New weights
* **__group_415__new_gradients** (optional) - **T3**:
  New gradients
* **__group_415__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_415__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_415__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_416__new_weights** (optional) - **T2**:
  New weights
* **__group_416__new_gradients** (optional) - **T3**:
  New gradients
* **__group_416__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_416__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_416__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_417__new_weights** (optional) - **T2**:
  New weights
* **__group_417__new_gradients** (optional) - **T3**:
  New gradients
* **__group_417__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_417__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_417__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_418__new_weights** (optional) - **T2**:
  New weights
* **__group_418__new_gradients** (optional) - **T3**:
  New gradients
* **__group_418__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_418__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_418__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_419__new_weights** (optional) - **T2**:
  New weights
* **__group_419__new_gradients** (optional) - **T3**:
  New gradients
* **__group_419__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_419__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_419__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_420__new_weights** (optional) - **T2**:
  New weights
* **__group_420__new_gradients** (optional) - **T3**:
  New gradients
* **__group_420__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_420__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_420__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_421__new_weights** (optional) - **T2**:
  New weights
* **__group_421__new_gradients** (optional) - **T3**:
  New gradients
* **__group_421__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_421__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_421__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_422__new_weights** (optional) - **T2**:
  New weights
* **__group_422__new_gradients** (optional) - **T3**:
  New gradients
* **__group_422__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_422__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_422__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_423__new_weights** (optional) - **T2**:
  New weights
* **__group_423__new_gradients** (optional) - **T3**:
  New gradients
* **__group_423__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_423__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_423__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_424__new_weights** (optional) - **T2**:
  New weights
* **__group_424__new_gradients** (optional) - **T3**:
  New gradients
* **__group_424__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_424__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_424__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_425__new_weights** (optional) - **T2**:
  New weights
* **__group_425__new_gradients** (optional) - **T3**:
  New gradients
* **__group_425__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_425__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_425__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_426__new_weights** (optional) - **T2**:
  New weights
* **__group_426__new_gradients** (optional) - **T3**:
  New gradients
* **__group_426__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_426__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_426__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_427__new_weights** (optional) - **T2**:
  New weights
* **__group_427__new_gradients** (optional) - **T3**:
  New gradients
* **__group_427__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_427__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_427__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_428__new_weights** (optional) - **T2**:
  New weights
* **__group_428__new_gradients** (optional) - **T3**:
  New gradients
* **__group_428__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_428__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_428__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_429__new_weights** (optional) - **T2**:
  New weights
* **__group_429__new_gradients** (optional) - **T3**:
  New gradients
* **__group_429__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_429__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_429__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_430__new_weights** (optional) - **T2**:
  New weights
* **__group_430__new_gradients** (optional) - **T3**:
  New gradients
* **__group_430__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_430__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_430__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_431__new_weights** (optional) - **T2**:
  New weights
* **__group_431__new_gradients** (optional) - **T3**:
  New gradients
* **__group_431__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_431__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_431__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_432__new_weights** (optional) - **T2**:
  New weights
* **__group_432__new_gradients** (optional) - **T3**:
  New gradients
* **__group_432__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_432__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_432__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_433__new_weights** (optional) - **T2**:
  New weights
* **__group_433__new_gradients** (optional) - **T3**:
  New gradients
* **__group_433__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_433__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_433__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_434__new_weights** (optional) - **T2**:
  New weights
* **__group_434__new_gradients** (optional) - **T3**:
  New gradients
* **__group_434__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_434__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_434__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_435__new_weights** (optional) - **T2**:
  New weights
* **__group_435__new_gradients** (optional) - **T3**:
  New gradients
* **__group_435__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_435__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_435__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_436__new_weights** (optional) - **T2**:
  New weights
* **__group_436__new_gradients** (optional) - **T3**:
  New gradients
* **__group_436__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_436__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_436__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_437__new_weights** (optional) - **T2**:
  New weights
* **__group_437__new_gradients** (optional) - **T3**:
  New gradients
* **__group_437__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_437__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_437__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_438__new_weights** (optional) - **T2**:
  New weights
* **__group_438__new_gradients** (optional) - **T3**:
  New gradients
* **__group_438__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_438__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_438__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_439__new_weights** (optional) - **T2**:
  New weights
* **__group_439__new_gradients** (optional) - **T3**:
  New gradients
* **__group_439__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_439__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_439__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_440__new_weights** (optional) - **T2**:
  New weights
* **__group_440__new_gradients** (optional) - **T3**:
  New gradients
* **__group_440__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_440__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_440__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_441__new_weights** (optional) - **T2**:
  New weights
* **__group_441__new_gradients** (optional) - **T3**:
  New gradients
* **__group_441__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_441__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_441__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_442__new_weights** (optional) - **T2**:
  New weights
* **__group_442__new_gradients** (optional) - **T3**:
  New gradients
* **__group_442__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_442__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_442__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_443__new_weights** (optional) - **T2**:
  New weights
* **__group_443__new_gradients** (optional) - **T3**:
  New gradients
* **__group_443__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_443__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_443__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_444__new_weights** (optional) - **T2**:
  New weights
* **__group_444__new_gradients** (optional) - **T3**:
  New gradients
* **__group_444__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_444__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_444__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_445__new_weights** (optional) - **T2**:
  New weights
* **__group_445__new_gradients** (optional) - **T3**:
  New gradients
* **__group_445__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_445__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_445__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_446__new_weights** (optional) - **T2**:
  New weights
* **__group_446__new_gradients** (optional) - **T3**:
  New gradients
* **__group_446__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_446__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_446__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_447__new_weights** (optional) - **T2**:
  New weights
* **__group_447__new_gradients** (optional) - **T3**:
  New gradients
* **__group_447__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_447__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_447__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_448__new_weights** (optional) - **T2**:
  New weights
* **__group_448__new_gradients** (optional) - **T3**:
  New gradients
* **__group_448__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_448__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_448__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_449__new_weights** (optional) - **T2**:
  New weights
* **__group_449__new_gradients** (optional) - **T3**:
  New gradients
* **__group_449__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_449__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_449__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_450__new_weights** (optional) - **T2**:
  New weights
* **__group_450__new_gradients** (optional) - **T3**:
  New gradients
* **__group_450__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_450__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_450__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_451__new_weights** (optional) - **T2**:
  New weights
* **__group_451__new_gradients** (optional) - **T3**:
  New gradients
* **__group_451__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_451__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_451__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_452__new_weights** (optional) - **T2**:
  New weights
* **__group_452__new_gradients** (optional) - **T3**:
  New gradients
* **__group_452__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_452__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_452__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_453__new_weights** (optional) - **T2**:
  New weights
* **__group_453__new_gradients** (optional) - **T3**:
  New gradients
* **__group_453__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_453__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_453__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_454__new_weights** (optional) - **T2**:
  New weights
* **__group_454__new_gradients** (optional) - **T3**:
  New gradients
* **__group_454__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_454__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_454__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_455__new_weights** (optional) - **T2**:
  New weights
* **__group_455__new_gradients** (optional) - **T3**:
  New gradients
* **__group_455__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_455__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_455__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_456__new_weights** (optional) - **T2**:
  New weights
* **__group_456__new_gradients** (optional) - **T3**:
  New gradients
* **__group_456__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_456__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_456__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_457__new_weights** (optional) - **T2**:
  New weights
* **__group_457__new_gradients** (optional) - **T3**:
  New gradients
* **__group_457__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_457__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_457__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_458__new_weights** (optional) - **T2**:
  New weights
* **__group_458__new_gradients** (optional) - **T3**:
  New gradients
* **__group_458__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_458__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_458__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_459__new_weights** (optional) - **T2**:
  New weights
* **__group_459__new_gradients** (optional) - **T3**:
  New gradients
* **__group_459__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_459__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_459__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_460__new_weights** (optional) - **T2**:
  New weights
* **__group_460__new_gradients** (optional) - **T3**:
  New gradients
* **__group_460__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_460__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_460__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_461__new_weights** (optional) - **T2**:
  New weights
* **__group_461__new_gradients** (optional) - **T3**:
  New gradients
* **__group_461__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_461__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_461__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_462__new_weights** (optional) - **T2**:
  New weights
* **__group_462__new_gradients** (optional) - **T3**:
  New gradients
* **__group_462__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_462__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_462__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_463__new_weights** (optional) - **T2**:
  New weights
* **__group_463__new_gradients** (optional) - **T3**:
  New gradients
* **__group_463__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_463__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_463__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_464__new_weights** (optional) - **T2**:
  New weights
* **__group_464__new_gradients** (optional) - **T3**:
  New gradients
* **__group_464__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_464__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_464__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_465__new_weights** (optional) - **T2**:
  New weights
* **__group_465__new_gradients** (optional) - **T3**:
  New gradients
* **__group_465__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_465__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_465__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_466__new_weights** (optional) - **T2**:
  New weights
* **__group_466__new_gradients** (optional) - **T3**:
  New gradients
* **__group_466__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_466__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_466__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_467__new_weights** (optional) - **T2**:
  New weights
* **__group_467__new_gradients** (optional) - **T3**:
  New gradients
* **__group_467__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_467__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_467__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_468__new_weights** (optional) - **T2**:
  New weights
* **__group_468__new_gradients** (optional) - **T3**:
  New gradients
* **__group_468__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_468__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_468__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_469__new_weights** (optional) - **T2**:
  New weights
* **__group_469__new_gradients** (optional) - **T3**:
  New gradients
* **__group_469__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_469__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_469__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_470__new_weights** (optional) - **T2**:
  New weights
* **__group_470__new_gradients** (optional) - **T3**:
  New gradients
* **__group_470__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_470__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_470__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_471__new_weights** (optional) - **T2**:
  New weights
* **__group_471__new_gradients** (optional) - **T3**:
  New gradients
* **__group_471__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_471__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_471__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_472__new_weights** (optional) - **T2**:
  New weights
* **__group_472__new_gradients** (optional) - **T3**:
  New gradients
* **__group_472__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_472__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_472__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_473__new_weights** (optional) - **T2**:
  New weights
* **__group_473__new_gradients** (optional) - **T3**:
  New gradients
* **__group_473__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_473__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_473__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_474__new_weights** (optional) - **T2**:
  New weights
* **__group_474__new_gradients** (optional) - **T3**:
  New gradients
* **__group_474__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_474__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_474__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_475__new_weights** (optional) - **T2**:
  New weights
* **__group_475__new_gradients** (optional) - **T3**:
  New gradients
* **__group_475__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_475__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_475__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_476__new_weights** (optional) - **T2**:
  New weights
* **__group_476__new_gradients** (optional) - **T3**:
  New gradients
* **__group_476__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_476__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_476__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_477__new_weights** (optional) - **T2**:
  New weights
* **__group_477__new_gradients** (optional) - **T3**:
  New gradients
* **__group_477__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_477__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_477__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_478__new_weights** (optional) - **T2**:
  New weights
* **__group_478__new_gradients** (optional) - **T3**:
  New gradients
* **__group_478__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_478__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_478__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_479__new_weights** (optional) - **T2**:
  New weights
* **__group_479__new_gradients** (optional) - **T3**:
  New gradients
* **__group_479__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_479__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_479__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_480__new_weights** (optional) - **T2**:
  New weights
* **__group_480__new_gradients** (optional) - **T3**:
  New gradients
* **__group_480__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_480__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_480__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_481__new_weights** (optional) - **T2**:
  New weights
* **__group_481__new_gradients** (optional) - **T3**:
  New gradients
* **__group_481__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_481__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_481__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_482__new_weights** (optional) - **T2**:
  New weights
* **__group_482__new_gradients** (optional) - **T3**:
  New gradients
* **__group_482__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_482__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_482__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_483__new_weights** (optional) - **T2**:
  New weights
* **__group_483__new_gradients** (optional) - **T3**:
  New gradients
* **__group_483__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_483__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_483__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_484__new_weights** (optional) - **T2**:
  New weights
* **__group_484__new_gradients** (optional) - **T3**:
  New gradients
* **__group_484__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_484__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_484__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_485__new_weights** (optional) - **T2**:
  New weights
* **__group_485__new_gradients** (optional) - **T3**:
  New gradients
* **__group_485__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_485__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_485__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_486__new_weights** (optional) - **T2**:
  New weights
* **__group_486__new_gradients** (optional) - **T3**:
  New gradients
* **__group_486__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_486__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_486__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_487__new_weights** (optional) - **T2**:
  New weights
* **__group_487__new_gradients** (optional) - **T3**:
  New gradients
* **__group_487__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_487__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_487__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_488__new_weights** (optional) - **T2**:
  New weights
* **__group_488__new_gradients** (optional) - **T3**:
  New gradients
* **__group_488__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_488__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_488__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_489__new_weights** (optional) - **T2**:
  New weights
* **__group_489__new_gradients** (optional) - **T3**:
  New gradients
* **__group_489__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_489__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_489__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_490__new_weights** (optional) - **T2**:
  New weights
* **__group_490__new_gradients** (optional) - **T3**:
  New gradients
* **__group_490__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_490__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_490__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_491__new_weights** (optional) - **T2**:
  New weights
* **__group_491__new_gradients** (optional) - **T3**:
  New gradients
* **__group_491__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_491__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_491__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_492__new_weights** (optional) - **T2**:
  New weights
* **__group_492__new_gradients** (optional) - **T3**:
  New gradients
* **__group_492__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_492__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_492__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_493__new_weights** (optional) - **T2**:
  New weights
* **__group_493__new_gradients** (optional) - **T3**:
  New gradients
* **__group_493__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_493__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_493__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_494__new_weights** (optional) - **T2**:
  New weights
* **__group_494__new_gradients** (optional) - **T3**:
  New gradients
* **__group_494__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_494__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_494__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_495__new_weights** (optional) - **T2**:
  New weights
* **__group_495__new_gradients** (optional) - **T3**:
  New gradients
* **__group_495__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_495__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_495__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_496__new_weights** (optional) - **T2**:
  New weights
* **__group_496__new_gradients** (optional) - **T3**:
  New gradients
* **__group_496__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_496__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_496__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_497__new_weights** (optional) - **T2**:
  New weights
* **__group_497__new_gradients** (optional) - **T3**:
  New gradients
* **__group_497__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_497__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_497__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_498__new_weights** (optional) - **T2**:
  New weights
* **__group_498__new_gradients** (optional) - **T3**:
  New gradients
* **__group_498__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_498__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_498__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_499__new_weights** (optional) - **T2**:
  New weights
* **__group_499__new_gradients** (optional) - **T3**:
  New gradients
* **__group_499__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_499__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_499__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_500__new_weights** (optional) - **T2**:
  New weights
* **__group_500__new_gradients** (optional) - **T3**:
  New gradients
* **__group_500__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_500__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_500__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_501__new_weights** (optional) - **T2**:
  New weights
* **__group_501__new_gradients** (optional) - **T3**:
  New gradients
* **__group_501__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_501__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_501__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_502__new_weights** (optional) - **T2**:
  New weights
* **__group_502__new_gradients** (optional) - **T3**:
  New gradients
* **__group_502__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_502__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_502__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_503__new_weights** (optional) - **T2**:
  New weights
* **__group_503__new_gradients** (optional) - **T3**:
  New gradients
* **__group_503__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_503__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_503__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_504__new_weights** (optional) - **T2**:
  New weights
* **__group_504__new_gradients** (optional) - **T3**:
  New gradients
* **__group_504__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_504__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_504__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_505__new_weights** (optional) - **T2**:
  New weights
* **__group_505__new_gradients** (optional) - **T3**:
  New gradients
* **__group_505__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_505__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_505__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_506__new_weights** (optional) - **T2**:
  New weights
* **__group_506__new_gradients** (optional) - **T3**:
  New gradients
* **__group_506__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_506__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_506__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_507__new_weights** (optional) - **T2**:
  New weights
* **__group_507__new_gradients** (optional) - **T3**:
  New gradients
* **__group_507__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_507__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_507__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_508__new_weights** (optional) - **T2**:
  New weights
* **__group_508__new_gradients** (optional) - **T3**:
  New gradients
* **__group_508__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_508__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_508__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_509__new_weights** (optional) - **T2**:
  New weights
* **__group_509__new_gradients** (optional) - **T3**:
  New gradients
* **__group_509__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_509__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_509__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_510__new_weights** (optional) - **T2**:
  New weights
* **__group_510__new_gradients** (optional) - **T3**:
  New gradients
* **__group_510__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_510__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_510__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_511__new_weights** (optional) - **T2**:
  New weights
* **__group_511__new_gradients** (optional) - **T3**:
  New gradients
* **__group_511__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_511__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_511__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_512__new_weights** (optional) - **T2**:
  New weights
* **__group_512__new_gradients** (optional) - **T3**:
  New gradients
* **__group_512__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_512__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_512__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_513__new_weights** (optional) - **T2**:
  New weights
* **__group_513__new_gradients** (optional) - **T3**:
  New gradients
* **__group_513__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_513__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_513__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_514__new_weights** (optional) - **T2**:
  New weights
* **__group_514__new_gradients** (optional) - **T3**:
  New gradients
* **__group_514__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_514__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_514__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_515__new_weights** (optional) - **T2**:
  New weights
* **__group_515__new_gradients** (optional) - **T3**:
  New gradients
* **__group_515__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_515__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_515__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_516__new_weights** (optional) - **T2**:
  New weights
* **__group_516__new_gradients** (optional) - **T3**:
  New gradients
* **__group_516__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_516__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_516__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_517__new_weights** (optional) - **T2**:
  New weights
* **__group_517__new_gradients** (optional) - **T3**:
  New gradients
* **__group_517__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_517__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_517__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_518__new_weights** (optional) - **T2**:
  New weights
* **__group_518__new_gradients** (optional) - **T3**:
  New gradients
* **__group_518__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_518__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_518__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_519__new_weights** (optional) - **T2**:
  New weights
* **__group_519__new_gradients** (optional) - **T3**:
  New gradients
* **__group_519__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_519__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_519__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_520__new_weights** (optional) - **T2**:
  New weights
* **__group_520__new_gradients** (optional) - **T3**:
  New gradients
* **__group_520__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_520__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_520__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_521__new_weights** (optional) - **T2**:
  New weights
* **__group_521__new_gradients** (optional) - **T3**:
  New gradients
* **__group_521__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_521__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_521__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_522__new_weights** (optional) - **T2**:
  New weights
* **__group_522__new_gradients** (optional) - **T3**:
  New gradients
* **__group_522__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_522__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_522__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_523__new_weights** (optional) - **T2**:
  New weights
* **__group_523__new_gradients** (optional) - **T3**:
  New gradients
* **__group_523__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_523__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_523__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_524__new_weights** (optional) - **T2**:
  New weights
* **__group_524__new_gradients** (optional) - **T3**:
  New gradients
* **__group_524__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_524__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_524__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_525__new_weights** (optional) - **T2**:
  New weights
* **__group_525__new_gradients** (optional) - **T3**:
  New gradients
* **__group_525__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_525__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_525__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_526__new_weights** (optional) - **T2**:
  New weights
* **__group_526__new_gradients** (optional) - **T3**:
  New gradients
* **__group_526__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_526__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_526__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_527__new_weights** (optional) - **T2**:
  New weights
* **__group_527__new_gradients** (optional) - **T3**:
  New gradients
* **__group_527__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_527__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_527__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_528__new_weights** (optional) - **T2**:
  New weights
* **__group_528__new_gradients** (optional) - **T3**:
  New gradients
* **__group_528__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_528__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_528__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_529__new_weights** (optional) - **T2**:
  New weights
* **__group_529__new_gradients** (optional) - **T3**:
  New gradients
* **__group_529__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_529__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_529__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_530__new_weights** (optional) - **T2**:
  New weights
* **__group_530__new_gradients** (optional) - **T3**:
  New gradients
* **__group_530__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_530__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_530__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_531__new_weights** (optional) - **T2**:
  New weights
* **__group_531__new_gradients** (optional) - **T3**:
  New gradients
* **__group_531__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_531__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_531__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_532__new_weights** (optional) - **T2**:
  New weights
* **__group_532__new_gradients** (optional) - **T3**:
  New gradients
* **__group_532__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_532__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_532__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_533__new_weights** (optional) - **T2**:
  New weights
* **__group_533__new_gradients** (optional) - **T3**:
  New gradients
* **__group_533__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_533__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_533__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_534__new_weights** (optional) - **T2**:
  New weights
* **__group_534__new_gradients** (optional) - **T3**:
  New gradients
* **__group_534__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_534__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_534__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_535__new_weights** (optional) - **T2**:
  New weights
* **__group_535__new_gradients** (optional) - **T3**:
  New gradients
* **__group_535__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_535__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_535__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_536__new_weights** (optional) - **T2**:
  New weights
* **__group_536__new_gradients** (optional) - **T3**:
  New gradients
* **__group_536__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_536__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_536__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_537__new_weights** (optional) - **T2**:
  New weights
* **__group_537__new_gradients** (optional) - **T3**:
  New gradients
* **__group_537__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_537__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_537__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_538__new_weights** (optional) - **T2**:
  New weights
* **__group_538__new_gradients** (optional) - **T3**:
  New gradients
* **__group_538__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_538__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_538__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_539__new_weights** (optional) - **T2**:
  New weights
* **__group_539__new_gradients** (optional) - **T3**:
  New gradients
* **__group_539__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_539__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_539__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_540__new_weights** (optional) - **T2**:
  New weights
* **__group_540__new_gradients** (optional) - **T3**:
  New gradients
* **__group_540__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_540__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_540__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_541__new_weights** (optional) - **T2**:
  New weights
* **__group_541__new_gradients** (optional) - **T3**:
  New gradients
* **__group_541__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_541__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_541__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_542__new_weights** (optional) - **T2**:
  New weights
* **__group_542__new_gradients** (optional) - **T3**:
  New gradients
* **__group_542__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_542__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_542__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_543__new_weights** (optional) - **T2**:
  New weights
* **__group_543__new_gradients** (optional) - **T3**:
  New gradients
* **__group_543__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_543__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_543__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_544__new_weights** (optional) - **T2**:
  New weights
* **__group_544__new_gradients** (optional) - **T3**:
  New gradients
* **__group_544__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_544__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_544__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_545__new_weights** (optional) - **T2**:
  New weights
* **__group_545__new_gradients** (optional) - **T3**:
  New gradients
* **__group_545__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_545__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_545__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_546__new_weights** (optional) - **T2**:
  New weights
* **__group_546__new_gradients** (optional) - **T3**:
  New gradients
* **__group_546__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_546__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_546__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_547__new_weights** (optional) - **T2**:
  New weights
* **__group_547__new_gradients** (optional) - **T3**:
  New gradients
* **__group_547__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_547__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_547__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_548__new_weights** (optional) - **T2**:
  New weights
* **__group_548__new_gradients** (optional) - **T3**:
  New gradients
* **__group_548__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_548__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_548__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_549__new_weights** (optional) - **T2**:
  New weights
* **__group_549__new_gradients** (optional) - **T3**:
  New gradients
* **__group_549__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_549__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_549__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_550__new_weights** (optional) - **T2**:
  New weights
* **__group_550__new_gradients** (optional) - **T3**:
  New gradients
* **__group_550__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_550__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_550__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_551__new_weights** (optional) - **T2**:
  New weights
* **__group_551__new_gradients** (optional) - **T3**:
  New gradients
* **__group_551__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_551__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_551__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_552__new_weights** (optional) - **T2**:
  New weights
* **__group_552__new_gradients** (optional) - **T3**:
  New gradients
* **__group_552__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_552__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_552__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_553__new_weights** (optional) - **T2**:
  New weights
* **__group_553__new_gradients** (optional) - **T3**:
  New gradients
* **__group_553__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_553__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_553__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_554__new_weights** (optional) - **T2**:
  New weights
* **__group_554__new_gradients** (optional) - **T3**:
  New gradients
* **__group_554__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_554__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_554__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_555__new_weights** (optional) - **T2**:
  New weights
* **__group_555__new_gradients** (optional) - **T3**:
  New gradients
* **__group_555__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_555__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_555__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_556__new_weights** (optional) - **T2**:
  New weights
* **__group_556__new_gradients** (optional) - **T3**:
  New gradients
* **__group_556__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_556__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_556__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_557__new_weights** (optional) - **T2**:
  New weights
* **__group_557__new_gradients** (optional) - **T3**:
  New gradients
* **__group_557__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_557__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_557__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_558__new_weights** (optional) - **T2**:
  New weights
* **__group_558__new_gradients** (optional) - **T3**:
  New gradients
* **__group_558__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_558__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_558__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_559__new_weights** (optional) - **T2**:
  New weights
* **__group_559__new_gradients** (optional) - **T3**:
  New gradients
* **__group_559__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_559__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_559__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_560__new_weights** (optional) - **T2**:
  New weights
* **__group_560__new_gradients** (optional) - **T3**:
  New gradients
* **__group_560__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_560__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_560__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_561__new_weights** (optional) - **T2**:
  New weights
* **__group_561__new_gradients** (optional) - **T3**:
  New gradients
* **__group_561__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_561__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_561__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_562__new_weights** (optional) - **T2**:
  New weights
* **__group_562__new_gradients** (optional) - **T3**:
  New gradients
* **__group_562__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_562__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_562__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_563__new_weights** (optional) - **T2**:
  New weights
* **__group_563__new_gradients** (optional) - **T3**:
  New gradients
* **__group_563__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_563__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_563__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_564__new_weights** (optional) - **T2**:
  New weights
* **__group_564__new_gradients** (optional) - **T3**:
  New gradients
* **__group_564__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_564__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_564__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_565__new_weights** (optional) - **T2**:
  New weights
* **__group_565__new_gradients** (optional) - **T3**:
  New gradients
* **__group_565__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_565__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_565__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_566__new_weights** (optional) - **T2**:
  New weights
* **__group_566__new_gradients** (optional) - **T3**:
  New gradients
* **__group_566__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_566__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_566__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_567__new_weights** (optional) - **T2**:
  New weights
* **__group_567__new_gradients** (optional) - **T3**:
  New gradients
* **__group_567__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_567__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_567__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_568__new_weights** (optional) - **T2**:
  New weights
* **__group_568__new_gradients** (optional) - **T3**:
  New gradients
* **__group_568__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_568__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_568__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_569__new_weights** (optional) - **T2**:
  New weights
* **__group_569__new_gradients** (optional) - **T3**:
  New gradients
* **__group_569__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_569__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_569__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_570__new_weights** (optional) - **T2**:
  New weights
* **__group_570__new_gradients** (optional) - **T3**:
  New gradients
* **__group_570__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_570__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_570__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_571__new_weights** (optional) - **T2**:
  New weights
* **__group_571__new_gradients** (optional) - **T3**:
  New gradients
* **__group_571__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_571__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_571__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_572__new_weights** (optional) - **T2**:
  New weights
* **__group_572__new_gradients** (optional) - **T3**:
  New gradients
* **__group_572__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_572__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_572__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_573__new_weights** (optional) - **T2**:
  New weights
* **__group_573__new_gradients** (optional) - **T3**:
  New gradients
* **__group_573__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_573__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_573__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_574__new_weights** (optional) - **T2**:
  New weights
* **__group_574__new_gradients** (optional) - **T3**:
  New gradients
* **__group_574__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_574__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_574__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_575__new_weights** (optional) - **T2**:
  New weights
* **__group_575__new_gradients** (optional) - **T3**:
  New gradients
* **__group_575__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_575__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_575__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_576__new_weights** (optional) - **T2**:
  New weights
* **__group_576__new_gradients** (optional) - **T3**:
  New gradients
* **__group_576__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_576__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_576__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_577__new_weights** (optional) - **T2**:
  New weights
* **__group_577__new_gradients** (optional) - **T3**:
  New gradients
* **__group_577__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_577__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_577__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_578__new_weights** (optional) - **T2**:
  New weights
* **__group_578__new_gradients** (optional) - **T3**:
  New gradients
* **__group_578__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_578__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_578__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_579__new_weights** (optional) - **T2**:
  New weights
* **__group_579__new_gradients** (optional) - **T3**:
  New gradients
* **__group_579__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_579__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_579__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_580__new_weights** (optional) - **T2**:
  New weights
* **__group_580__new_gradients** (optional) - **T3**:
  New gradients
* **__group_580__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_580__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_580__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_581__new_weights** (optional) - **T2**:
  New weights
* **__group_581__new_gradients** (optional) - **T3**:
  New gradients
* **__group_581__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_581__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_581__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_582__new_weights** (optional) - **T2**:
  New weights
* **__group_582__new_gradients** (optional) - **T3**:
  New gradients
* **__group_582__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_582__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_582__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_583__new_weights** (optional) - **T2**:
  New weights
* **__group_583__new_gradients** (optional) - **T3**:
  New gradients
* **__group_583__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_583__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_583__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_584__new_weights** (optional) - **T2**:
  New weights
* **__group_584__new_gradients** (optional) - **T3**:
  New gradients
* **__group_584__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_584__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_584__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_585__new_weights** (optional) - **T2**:
  New weights
* **__group_585__new_gradients** (optional) - **T3**:
  New gradients
* **__group_585__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_585__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_585__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_586__new_weights** (optional) - **T2**:
  New weights
* **__group_586__new_gradients** (optional) - **T3**:
  New gradients
* **__group_586__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_586__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_586__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_587__new_weights** (optional) - **T2**:
  New weights
* **__group_587__new_gradients** (optional) - **T3**:
  New gradients
* **__group_587__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_587__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_587__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_588__new_weights** (optional) - **T2**:
  New weights
* **__group_588__new_gradients** (optional) - **T3**:
  New gradients
* **__group_588__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_588__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_588__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_589__new_weights** (optional) - **T2**:
  New weights
* **__group_589__new_gradients** (optional) - **T3**:
  New gradients
* **__group_589__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_589__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_589__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_590__new_weights** (optional) - **T2**:
  New weights
* **__group_590__new_gradients** (optional) - **T3**:
  New gradients
* **__group_590__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_590__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_590__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_591__new_weights** (optional) - **T2**:
  New weights
* **__group_591__new_gradients** (optional) - **T3**:
  New gradients
* **__group_591__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_591__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_591__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_592__new_weights** (optional) - **T2**:
  New weights
* **__group_592__new_gradients** (optional) - **T3**:
  New gradients
* **__group_592__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_592__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_592__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_593__new_weights** (optional) - **T2**:
  New weights
* **__group_593__new_gradients** (optional) - **T3**:
  New gradients
* **__group_593__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_593__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_593__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_594__new_weights** (optional) - **T2**:
  New weights
* **__group_594__new_gradients** (optional) - **T3**:
  New gradients
* **__group_594__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_594__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_594__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_595__new_weights** (optional) - **T2**:
  New weights
* **__group_595__new_gradients** (optional) - **T3**:
  New gradients
* **__group_595__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_595__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_595__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_596__new_weights** (optional) - **T2**:
  New weights
* **__group_596__new_gradients** (optional) - **T3**:
  New gradients
* **__group_596__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_596__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_596__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_597__new_weights** (optional) - **T2**:
  New weights
* **__group_597__new_gradients** (optional) - **T3**:
  New gradients
* **__group_597__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_597__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_597__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_598__new_weights** (optional) - **T2**:
  New weights
* **__group_598__new_gradients** (optional) - **T3**:
  New gradients
* **__group_598__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_598__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_598__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_599__new_weights** (optional) - **T2**:
  New weights
* **__group_599__new_gradients** (optional) - **T3**:
  New gradients
* **__group_599__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_599__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_599__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_600__new_weights** (optional) - **T2**:
  New weights
* **__group_600__new_gradients** (optional) - **T3**:
  New gradients
* **__group_600__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_600__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_600__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_601__new_weights** (optional) - **T2**:
  New weights
* **__group_601__new_gradients** (optional) - **T3**:
  New gradients
* **__group_601__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_601__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_601__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_602__new_weights** (optional) - **T2**:
  New weights
* **__group_602__new_gradients** (optional) - **T3**:
  New gradients
* **__group_602__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_602__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_602__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_603__new_weights** (optional) - **T2**:
  New weights
* **__group_603__new_gradients** (optional) - **T3**:
  New gradients
* **__group_603__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_603__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_603__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_604__new_weights** (optional) - **T2**:
  New weights
* **__group_604__new_gradients** (optional) - **T3**:
  New gradients
* **__group_604__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_604__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_604__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_605__new_weights** (optional) - **T2**:
  New weights
* **__group_605__new_gradients** (optional) - **T3**:
  New gradients
* **__group_605__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_605__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_605__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_606__new_weights** (optional) - **T2**:
  New weights
* **__group_606__new_gradients** (optional) - **T3**:
  New gradients
* **__group_606__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_606__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_606__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_607__new_weights** (optional) - **T2**:
  New weights
* **__group_607__new_gradients** (optional) - **T3**:
  New gradients
* **__group_607__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_607__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_607__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_608__new_weights** (optional) - **T2**:
  New weights
* **__group_608__new_gradients** (optional) - **T3**:
  New gradients
* **__group_608__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_608__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_608__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_609__new_weights** (optional) - **T2**:
  New weights
* **__group_609__new_gradients** (optional) - **T3**:
  New gradients
* **__group_609__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_609__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_609__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_610__new_weights** (optional) - **T2**:
  New weights
* **__group_610__new_gradients** (optional) - **T3**:
  New gradients
* **__group_610__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_610__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_610__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_611__new_weights** (optional) - **T2**:
  New weights
* **__group_611__new_gradients** (optional) - **T3**:
  New gradients
* **__group_611__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_611__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_611__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_612__new_weights** (optional) - **T2**:
  New weights
* **__group_612__new_gradients** (optional) - **T3**:
  New gradients
* **__group_612__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_612__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_612__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_613__new_weights** (optional) - **T2**:
  New weights
* **__group_613__new_gradients** (optional) - **T3**:
  New gradients
* **__group_613__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_613__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_613__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_614__new_weights** (optional) - **T2**:
  New weights
* **__group_614__new_gradients** (optional) - **T3**:
  New gradients
* **__group_614__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_614__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_614__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_615__new_weights** (optional) - **T2**:
  New weights
* **__group_615__new_gradients** (optional) - **T3**:
  New gradients
* **__group_615__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_615__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_615__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_616__new_weights** (optional) - **T2**:
  New weights
* **__group_616__new_gradients** (optional) - **T3**:
  New gradients
* **__group_616__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_616__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_616__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_617__new_weights** (optional) - **T2**:
  New weights
* **__group_617__new_gradients** (optional) - **T3**:
  New gradients
* **__group_617__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_617__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_617__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_618__new_weights** (optional) - **T2**:
  New weights
* **__group_618__new_gradients** (optional) - **T3**:
  New gradients
* **__group_618__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_618__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_618__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_619__new_weights** (optional) - **T2**:
  New weights
* **__group_619__new_gradients** (optional) - **T3**:
  New gradients
* **__group_619__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_619__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_619__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_620__new_weights** (optional) - **T2**:
  New weights
* **__group_620__new_gradients** (optional) - **T3**:
  New gradients
* **__group_620__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_620__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_620__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_621__new_weights** (optional) - **T2**:
  New weights
* **__group_621__new_gradients** (optional) - **T3**:
  New gradients
* **__group_621__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_621__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_621__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_622__new_weights** (optional) - **T2**:
  New weights
* **__group_622__new_gradients** (optional) - **T3**:
  New gradients
* **__group_622__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_622__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_622__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_623__new_weights** (optional) - **T2**:
  New weights
* **__group_623__new_gradients** (optional) - **T3**:
  New gradients
* **__group_623__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_623__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_623__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_624__new_weights** (optional) - **T2**:
  New weights
* **__group_624__new_gradients** (optional) - **T3**:
  New gradients
* **__group_624__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_624__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_624__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_625__new_weights** (optional) - **T2**:
  New weights
* **__group_625__new_gradients** (optional) - **T3**:
  New gradients
* **__group_625__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_625__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_625__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_626__new_weights** (optional) - **T2**:
  New weights
* **__group_626__new_gradients** (optional) - **T3**:
  New gradients
* **__group_626__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_626__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_626__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_627__new_weights** (optional) - **T2**:
  New weights
* **__group_627__new_gradients** (optional) - **T3**:
  New gradients
* **__group_627__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_627__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_627__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_628__new_weights** (optional) - **T2**:
  New weights
* **__group_628__new_gradients** (optional) - **T3**:
  New gradients
* **__group_628__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_628__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_628__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_629__new_weights** (optional) - **T2**:
  New weights
* **__group_629__new_gradients** (optional) - **T3**:
  New gradients
* **__group_629__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_629__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_629__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_630__new_weights** (optional) - **T2**:
  New weights
* **__group_630__new_gradients** (optional) - **T3**:
  New gradients
* **__group_630__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_630__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_630__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_631__new_weights** (optional) - **T2**:
  New weights
* **__group_631__new_gradients** (optional) - **T3**:
  New gradients
* **__group_631__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_631__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_631__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_632__new_weights** (optional) - **T2**:
  New weights
* **__group_632__new_gradients** (optional) - **T3**:
  New gradients
* **__group_632__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_632__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_632__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_633__new_weights** (optional) - **T2**:
  New weights
* **__group_633__new_gradients** (optional) - **T3**:
  New gradients
* **__group_633__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_633__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_633__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_634__new_weights** (optional) - **T2**:
  New weights
* **__group_634__new_gradients** (optional) - **T3**:
  New gradients
* **__group_634__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_634__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_634__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_635__new_weights** (optional) - **T2**:
  New weights
* **__group_635__new_gradients** (optional) - **T3**:
  New gradients
* **__group_635__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_635__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_635__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_636__new_weights** (optional) - **T2**:
  New weights
* **__group_636__new_gradients** (optional) - **T3**:
  New gradients
* **__group_636__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_636__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_636__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_637__new_weights** (optional) - **T2**:
  New weights
* **__group_637__new_gradients** (optional) - **T3**:
  New gradients
* **__group_637__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_637__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_637__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_638__new_weights** (optional) - **T2**:
  New weights
* **__group_638__new_gradients** (optional) - **T3**:
  New gradients
* **__group_638__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_638__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_638__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_639__new_weights** (optional) - **T2**:
  New weights
* **__group_639__new_gradients** (optional) - **T3**:
  New gradients
* **__group_639__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_639__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_639__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_640__new_weights** (optional) - **T2**:
  New weights
* **__group_640__new_gradients** (optional) - **T3**:
  New gradients
* **__group_640__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_640__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_640__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_641__new_weights** (optional) - **T2**:
  New weights
* **__group_641__new_gradients** (optional) - **T3**:
  New gradients
* **__group_641__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_641__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_641__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_642__new_weights** (optional) - **T2**:
  New weights
* **__group_642__new_gradients** (optional) - **T3**:
  New gradients
* **__group_642__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_642__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_642__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_643__new_weights** (optional) - **T2**:
  New weights
* **__group_643__new_gradients** (optional) - **T3**:
  New gradients
* **__group_643__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_643__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_643__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_644__new_weights** (optional) - **T2**:
  New weights
* **__group_644__new_gradients** (optional) - **T3**:
  New gradients
* **__group_644__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_644__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_644__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_645__new_weights** (optional) - **T2**:
  New weights
* **__group_645__new_gradients** (optional) - **T3**:
  New gradients
* **__group_645__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_645__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_645__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_646__new_weights** (optional) - **T2**:
  New weights
* **__group_646__new_gradients** (optional) - **T3**:
  New gradients
* **__group_646__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_646__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_646__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_647__new_weights** (optional) - **T2**:
  New weights
* **__group_647__new_gradients** (optional) - **T3**:
  New gradients
* **__group_647__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_647__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_647__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_648__new_weights** (optional) - **T2**:
  New weights
* **__group_648__new_gradients** (optional) - **T3**:
  New gradients
* **__group_648__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_648__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_648__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_649__new_weights** (optional) - **T2**:
  New weights
* **__group_649__new_gradients** (optional) - **T3**:
  New gradients
* **__group_649__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_649__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_649__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_650__new_weights** (optional) - **T2**:
  New weights
* **__group_650__new_gradients** (optional) - **T3**:
  New gradients
* **__group_650__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_650__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_650__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_651__new_weights** (optional) - **T2**:
  New weights
* **__group_651__new_gradients** (optional) - **T3**:
  New gradients
* **__group_651__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_651__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_651__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_652__new_weights** (optional) - **T2**:
  New weights
* **__group_652__new_gradients** (optional) - **T3**:
  New gradients
* **__group_652__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_652__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_652__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_653__new_weights** (optional) - **T2**:
  New weights
* **__group_653__new_gradients** (optional) - **T3**:
  New gradients
* **__group_653__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_653__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_653__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_654__new_weights** (optional) - **T2**:
  New weights
* **__group_654__new_gradients** (optional) - **T3**:
  New gradients
* **__group_654__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_654__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_654__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_655__new_weights** (optional) - **T2**:
  New weights
* **__group_655__new_gradients** (optional) - **T3**:
  New gradients
* **__group_655__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_655__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_655__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_656__new_weights** (optional) - **T2**:
  New weights
* **__group_656__new_gradients** (optional) - **T3**:
  New gradients
* **__group_656__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_656__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_656__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_657__new_weights** (optional) - **T2**:
  New weights
* **__group_657__new_gradients** (optional) - **T3**:
  New gradients
* **__group_657__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_657__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_657__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_658__new_weights** (optional) - **T2**:
  New weights
* **__group_658__new_gradients** (optional) - **T3**:
  New gradients
* **__group_658__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_658__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_658__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_659__new_weights** (optional) - **T2**:
  New weights
* **__group_659__new_gradients** (optional) - **T3**:
  New gradients
* **__group_659__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_659__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_659__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_660__new_weights** (optional) - **T2**:
  New weights
* **__group_660__new_gradients** (optional) - **T3**:
  New gradients
* **__group_660__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_660__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_660__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_661__new_weights** (optional) - **T2**:
  New weights
* **__group_661__new_gradients** (optional) - **T3**:
  New gradients
* **__group_661__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_661__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_661__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_662__new_weights** (optional) - **T2**:
  New weights
* **__group_662__new_gradients** (optional) - **T3**:
  New gradients
* **__group_662__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_662__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_662__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_663__new_weights** (optional) - **T2**:
  New weights
* **__group_663__new_gradients** (optional) - **T3**:
  New gradients
* **__group_663__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_663__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_663__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_664__new_weights** (optional) - **T2**:
  New weights
* **__group_664__new_gradients** (optional) - **T3**:
  New gradients
* **__group_664__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_664__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_664__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_665__new_weights** (optional) - **T2**:
  New weights
* **__group_665__new_gradients** (optional) - **T3**:
  New gradients
* **__group_665__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_665__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_665__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_666__new_weights** (optional) - **T2**:
  New weights
* **__group_666__new_gradients** (optional) - **T3**:
  New gradients
* **__group_666__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_666__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_666__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_667__new_weights** (optional) - **T2**:
  New weights
* **__group_667__new_gradients** (optional) - **T3**:
  New gradients
* **__group_667__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_667__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_667__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_668__new_weights** (optional) - **T2**:
  New weights
* **__group_668__new_gradients** (optional) - **T3**:
  New gradients
* **__group_668__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_668__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_668__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_669__new_weights** (optional) - **T2**:
  New weights
* **__group_669__new_gradients** (optional) - **T3**:
  New gradients
* **__group_669__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_669__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_669__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_670__new_weights** (optional) - **T2**:
  New weights
* **__group_670__new_gradients** (optional) - **T3**:
  New gradients
* **__group_670__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_670__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_670__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_671__new_weights** (optional) - **T2**:
  New weights
* **__group_671__new_gradients** (optional) - **T3**:
  New gradients
* **__group_671__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_671__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_671__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_672__new_weights** (optional) - **T2**:
  New weights
* **__group_672__new_gradients** (optional) - **T3**:
  New gradients
* **__group_672__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_672__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_672__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_673__new_weights** (optional) - **T2**:
  New weights
* **__group_673__new_gradients** (optional) - **T3**:
  New gradients
* **__group_673__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_673__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_673__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_674__new_weights** (optional) - **T2**:
  New weights
* **__group_674__new_gradients** (optional) - **T3**:
  New gradients
* **__group_674__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_674__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_674__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_675__new_weights** (optional) - **T2**:
  New weights
* **__group_675__new_gradients** (optional) - **T3**:
  New gradients
* **__group_675__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_675__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_675__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_676__new_weights** (optional) - **T2**:
  New weights
* **__group_676__new_gradients** (optional) - **T3**:
  New gradients
* **__group_676__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_676__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_676__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_677__new_weights** (optional) - **T2**:
  New weights
* **__group_677__new_gradients** (optional) - **T3**:
  New gradients
* **__group_677__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_677__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_677__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_678__new_weights** (optional) - **T2**:
  New weights
* **__group_678__new_gradients** (optional) - **T3**:
  New gradients
* **__group_678__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_678__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_678__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_679__new_weights** (optional) - **T2**:
  New weights
* **__group_679__new_gradients** (optional) - **T3**:
  New gradients
* **__group_679__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_679__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_679__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_680__new_weights** (optional) - **T2**:
  New weights
* **__group_680__new_gradients** (optional) - **T3**:
  New gradients
* **__group_680__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_680__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_680__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_681__new_weights** (optional) - **T2**:
  New weights
* **__group_681__new_gradients** (optional) - **T3**:
  New gradients
* **__group_681__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_681__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_681__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_682__new_weights** (optional) - **T2**:
  New weights
* **__group_682__new_gradients** (optional) - **T3**:
  New gradients
* **__group_682__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_682__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_682__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_683__new_weights** (optional) - **T2**:
  New weights
* **__group_683__new_gradients** (optional) - **T3**:
  New gradients
* **__group_683__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_683__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_683__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_684__new_weights** (optional) - **T2**:
  New weights
* **__group_684__new_gradients** (optional) - **T3**:
  New gradients
* **__group_684__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_684__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_684__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_685__new_weights** (optional) - **T2**:
  New weights
* **__group_685__new_gradients** (optional) - **T3**:
  New gradients
* **__group_685__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_685__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_685__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_686__new_weights** (optional) - **T2**:
  New weights
* **__group_686__new_gradients** (optional) - **T3**:
  New gradients
* **__group_686__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_686__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_686__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_687__new_weights** (optional) - **T2**:
  New weights
* **__group_687__new_gradients** (optional) - **T3**:
  New gradients
* **__group_687__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_687__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_687__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_688__new_weights** (optional) - **T2**:
  New weights
* **__group_688__new_gradients** (optional) - **T3**:
  New gradients
* **__group_688__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_688__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_688__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_689__new_weights** (optional) - **T2**:
  New weights
* **__group_689__new_gradients** (optional) - **T3**:
  New gradients
* **__group_689__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_689__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_689__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_690__new_weights** (optional) - **T2**:
  New weights
* **__group_690__new_gradients** (optional) - **T3**:
  New gradients
* **__group_690__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_690__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_690__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_691__new_weights** (optional) - **T2**:
  New weights
* **__group_691__new_gradients** (optional) - **T3**:
  New gradients
* **__group_691__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_691__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_691__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_692__new_weights** (optional) - **T2**:
  New weights
* **__group_692__new_gradients** (optional) - **T3**:
  New gradients
* **__group_692__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_692__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_692__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_693__new_weights** (optional) - **T2**:
  New weights
* **__group_693__new_gradients** (optional) - **T3**:
  New gradients
* **__group_693__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_693__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_693__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_694__new_weights** (optional) - **T2**:
  New weights
* **__group_694__new_gradients** (optional) - **T3**:
  New gradients
* **__group_694__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_694__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_694__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_695__new_weights** (optional) - **T2**:
  New weights
* **__group_695__new_gradients** (optional) - **T3**:
  New gradients
* **__group_695__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_695__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_695__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_696__new_weights** (optional) - **T2**:
  New weights
* **__group_696__new_gradients** (optional) - **T3**:
  New gradients
* **__group_696__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_696__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_696__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_697__new_weights** (optional) - **T2**:
  New weights
* **__group_697__new_gradients** (optional) - **T3**:
  New gradients
* **__group_697__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_697__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_697__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_698__new_weights** (optional) - **T2**:
  New weights
* **__group_698__new_gradients** (optional) - **T3**:
  New gradients
* **__group_698__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_698__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_698__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_699__new_weights** (optional) - **T2**:
  New weights
* **__group_699__new_gradients** (optional) - **T3**:
  New gradients
* **__group_699__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_699__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_699__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_700__new_weights** (optional) - **T2**:
  New weights
* **__group_700__new_gradients** (optional) - **T3**:
  New gradients
* **__group_700__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_700__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_700__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_701__new_weights** (optional) - **T2**:
  New weights
* **__group_701__new_gradients** (optional) - **T3**:
  New gradients
* **__group_701__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_701__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_701__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_702__new_weights** (optional) - **T2**:
  New weights
* **__group_702__new_gradients** (optional) - **T3**:
  New gradients
* **__group_702__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_702__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_702__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_703__new_weights** (optional) - **T2**:
  New weights
* **__group_703__new_gradients** (optional) - **T3**:
  New gradients
* **__group_703__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_703__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_703__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_704__new_weights** (optional) - **T2**:
  New weights
* **__group_704__new_gradients** (optional) - **T3**:
  New gradients
* **__group_704__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_704__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_704__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_705__new_weights** (optional) - **T2**:
  New weights
* **__group_705__new_gradients** (optional) - **T3**:
  New gradients
* **__group_705__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_705__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_705__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_706__new_weights** (optional) - **T2**:
  New weights
* **__group_706__new_gradients** (optional) - **T3**:
  New gradients
* **__group_706__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_706__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_706__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_707__new_weights** (optional) - **T2**:
  New weights
* **__group_707__new_gradients** (optional) - **T3**:
  New gradients
* **__group_707__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_707__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_707__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_708__new_weights** (optional) - **T2**:
  New weights
* **__group_708__new_gradients** (optional) - **T3**:
  New gradients
* **__group_708__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_708__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_708__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_709__new_weights** (optional) - **T2**:
  New weights
* **__group_709__new_gradients** (optional) - **T3**:
  New gradients
* **__group_709__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_709__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_709__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_710__new_weights** (optional) - **T2**:
  New weights
* **__group_710__new_gradients** (optional) - **T3**:
  New gradients
* **__group_710__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_710__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_710__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_711__new_weights** (optional) - **T2**:
  New weights
* **__group_711__new_gradients** (optional) - **T3**:
  New gradients
* **__group_711__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_711__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_711__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_712__new_weights** (optional) - **T2**:
  New weights
* **__group_712__new_gradients** (optional) - **T3**:
  New gradients
* **__group_712__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_712__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_712__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_713__new_weights** (optional) - **T2**:
  New weights
* **__group_713__new_gradients** (optional) - **T3**:
  New gradients
* **__group_713__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_713__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_713__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_714__new_weights** (optional) - **T2**:
  New weights
* **__group_714__new_gradients** (optional) - **T3**:
  New gradients
* **__group_714__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_714__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_714__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_715__new_weights** (optional) - **T2**:
  New weights
* **__group_715__new_gradients** (optional) - **T3**:
  New gradients
* **__group_715__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_715__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_715__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_716__new_weights** (optional) - **T2**:
  New weights
* **__group_716__new_gradients** (optional) - **T3**:
  New gradients
* **__group_716__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_716__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_716__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_717__new_weights** (optional) - **T2**:
  New weights
* **__group_717__new_gradients** (optional) - **T3**:
  New gradients
* **__group_717__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_717__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_717__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_718__new_weights** (optional) - **T2**:
  New weights
* **__group_718__new_gradients** (optional) - **T3**:
  New gradients
* **__group_718__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_718__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_718__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_719__new_weights** (optional) - **T2**:
  New weights
* **__group_719__new_gradients** (optional) - **T3**:
  New gradients
* **__group_719__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_719__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_719__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_720__new_weights** (optional) - **T2**:
  New weights
* **__group_720__new_gradients** (optional) - **T3**:
  New gradients
* **__group_720__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_720__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_720__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_721__new_weights** (optional) - **T2**:
  New weights
* **__group_721__new_gradients** (optional) - **T3**:
  New gradients
* **__group_721__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_721__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_721__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_722__new_weights** (optional) - **T2**:
  New weights
* **__group_722__new_gradients** (optional) - **T3**:
  New gradients
* **__group_722__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_722__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_722__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_723__new_weights** (optional) - **T2**:
  New weights
* **__group_723__new_gradients** (optional) - **T3**:
  New gradients
* **__group_723__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_723__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_723__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_724__new_weights** (optional) - **T2**:
  New weights
* **__group_724__new_gradients** (optional) - **T3**:
  New gradients
* **__group_724__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_724__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_724__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_725__new_weights** (optional) - **T2**:
  New weights
* **__group_725__new_gradients** (optional) - **T3**:
  New gradients
* **__group_725__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_725__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_725__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_726__new_weights** (optional) - **T2**:
  New weights
* **__group_726__new_gradients** (optional) - **T3**:
  New gradients
* **__group_726__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_726__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_726__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_727__new_weights** (optional) - **T2**:
  New weights
* **__group_727__new_gradients** (optional) - **T3**:
  New gradients
* **__group_727__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_727__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_727__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_728__new_weights** (optional) - **T2**:
  New weights
* **__group_728__new_gradients** (optional) - **T3**:
  New gradients
* **__group_728__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_728__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_728__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_729__new_weights** (optional) - **T2**:
  New weights
* **__group_729__new_gradients** (optional) - **T3**:
  New gradients
* **__group_729__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_729__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_729__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_730__new_weights** (optional) - **T2**:
  New weights
* **__group_730__new_gradients** (optional) - **T3**:
  New gradients
* **__group_730__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_730__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_730__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_731__new_weights** (optional) - **T2**:
  New weights
* **__group_731__new_gradients** (optional) - **T3**:
  New gradients
* **__group_731__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_731__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_731__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_732__new_weights** (optional) - **T2**:
  New weights
* **__group_732__new_gradients** (optional) - **T3**:
  New gradients
* **__group_732__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_732__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_732__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_733__new_weights** (optional) - **T2**:
  New weights
* **__group_733__new_gradients** (optional) - **T3**:
  New gradients
* **__group_733__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_733__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_733__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_734__new_weights** (optional) - **T2**:
  New weights
* **__group_734__new_gradients** (optional) - **T3**:
  New gradients
* **__group_734__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_734__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_734__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_735__new_weights** (optional) - **T2**:
  New weights
* **__group_735__new_gradients** (optional) - **T3**:
  New gradients
* **__group_735__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_735__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_735__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_736__new_weights** (optional) - **T2**:
  New weights
* **__group_736__new_gradients** (optional) - **T3**:
  New gradients
* **__group_736__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_736__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_736__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_737__new_weights** (optional) - **T2**:
  New weights
* **__group_737__new_gradients** (optional) - **T3**:
  New gradients
* **__group_737__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_737__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_737__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_738__new_weights** (optional) - **T2**:
  New weights
* **__group_738__new_gradients** (optional) - **T3**:
  New gradients
* **__group_738__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_738__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_738__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_739__new_weights** (optional) - **T2**:
  New weights
* **__group_739__new_gradients** (optional) - **T3**:
  New gradients
* **__group_739__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_739__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_739__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_740__new_weights** (optional) - **T2**:
  New weights
* **__group_740__new_gradients** (optional) - **T3**:
  New gradients
* **__group_740__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_740__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_740__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_741__new_weights** (optional) - **T2**:
  New weights
* **__group_741__new_gradients** (optional) - **T3**:
  New gradients
* **__group_741__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_741__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_741__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_742__new_weights** (optional) - **T2**:
  New weights
* **__group_742__new_gradients** (optional) - **T3**:
  New gradients
* **__group_742__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_742__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_742__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_743__new_weights** (optional) - **T2**:
  New weights
* **__group_743__new_gradients** (optional) - **T3**:
  New gradients
* **__group_743__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_743__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_743__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_744__new_weights** (optional) - **T2**:
  New weights
* **__group_744__new_gradients** (optional) - **T3**:
  New gradients
* **__group_744__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_744__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_744__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_745__new_weights** (optional) - **T2**:
  New weights
* **__group_745__new_gradients** (optional) - **T3**:
  New gradients
* **__group_745__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_745__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_745__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_746__new_weights** (optional) - **T2**:
  New weights
* **__group_746__new_gradients** (optional) - **T3**:
  New gradients
* **__group_746__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_746__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_746__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_747__new_weights** (optional) - **T2**:
  New weights
* **__group_747__new_gradients** (optional) - **T3**:
  New gradients
* **__group_747__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_747__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_747__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_748__new_weights** (optional) - **T2**:
  New weights
* **__group_748__new_gradients** (optional) - **T3**:
  New gradients
* **__group_748__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_748__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_748__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_749__new_weights** (optional) - **T2**:
  New weights
* **__group_749__new_gradients** (optional) - **T3**:
  New gradients
* **__group_749__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_749__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_749__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_750__new_weights** (optional) - **T2**:
  New weights
* **__group_750__new_gradients** (optional) - **T3**:
  New gradients
* **__group_750__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_750__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_750__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_751__new_weights** (optional) - **T2**:
  New weights
* **__group_751__new_gradients** (optional) - **T3**:
  New gradients
* **__group_751__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_751__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_751__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_752__new_weights** (optional) - **T2**:
  New weights
* **__group_752__new_gradients** (optional) - **T3**:
  New gradients
* **__group_752__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_752__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_752__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_753__new_weights** (optional) - **T2**:
  New weights
* **__group_753__new_gradients** (optional) - **T3**:
  New gradients
* **__group_753__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_753__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_753__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_754__new_weights** (optional) - **T2**:
  New weights
* **__group_754__new_gradients** (optional) - **T3**:
  New gradients
* **__group_754__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_754__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_754__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_755__new_weights** (optional) - **T2**:
  New weights
* **__group_755__new_gradients** (optional) - **T3**:
  New gradients
* **__group_755__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_755__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_755__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_756__new_weights** (optional) - **T2**:
  New weights
* **__group_756__new_gradients** (optional) - **T3**:
  New gradients
* **__group_756__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_756__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_756__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_757__new_weights** (optional) - **T2**:
  New weights
* **__group_757__new_gradients** (optional) - **T3**:
  New gradients
* **__group_757__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_757__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_757__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_758__new_weights** (optional) - **T2**:
  New weights
* **__group_758__new_gradients** (optional) - **T3**:
  New gradients
* **__group_758__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_758__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_758__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_759__new_weights** (optional) - **T2**:
  New weights
* **__group_759__new_gradients** (optional) - **T3**:
  New gradients
* **__group_759__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_759__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_759__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_760__new_weights** (optional) - **T2**:
  New weights
* **__group_760__new_gradients** (optional) - **T3**:
  New gradients
* **__group_760__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_760__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_760__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_761__new_weights** (optional) - **T2**:
  New weights
* **__group_761__new_gradients** (optional) - **T3**:
  New gradients
* **__group_761__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_761__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_761__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_762__new_weights** (optional) - **T2**:
  New weights
* **__group_762__new_gradients** (optional) - **T3**:
  New gradients
* **__group_762__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_762__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_762__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_763__new_weights** (optional) - **T2**:
  New weights
* **__group_763__new_gradients** (optional) - **T3**:
  New gradients
* **__group_763__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_763__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_763__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_764__new_weights** (optional) - **T2**:
  New weights
* **__group_764__new_gradients** (optional) - **T3**:
  New gradients
* **__group_764__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_764__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_764__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_765__new_weights** (optional) - **T2**:
  New weights
* **__group_765__new_gradients** (optional) - **T3**:
  New gradients
* **__group_765__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_765__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_765__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_766__new_weights** (optional) - **T2**:
  New weights
* **__group_766__new_gradients** (optional) - **T3**:
  New gradients
* **__group_766__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_766__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_766__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_767__new_weights** (optional) - **T2**:
  New weights
* **__group_767__new_gradients** (optional) - **T3**:
  New gradients
* **__group_767__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_767__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_767__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_768__new_weights** (optional) - **T2**:
  New weights
* **__group_768__new_gradients** (optional) - **T3**:
  New gradients
* **__group_768__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_768__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_768__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_769__new_weights** (optional) - **T2**:
  New weights
* **__group_769__new_gradients** (optional) - **T3**:
  New gradients
* **__group_769__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_769__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_769__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_770__new_weights** (optional) - **T2**:
  New weights
* **__group_770__new_gradients** (optional) - **T3**:
  New gradients
* **__group_770__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_770__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_770__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_771__new_weights** (optional) - **T2**:
  New weights
* **__group_771__new_gradients** (optional) - **T3**:
  New gradients
* **__group_771__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_771__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_771__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_772__new_weights** (optional) - **T2**:
  New weights
* **__group_772__new_gradients** (optional) - **T3**:
  New gradients
* **__group_772__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_772__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_772__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_773__new_weights** (optional) - **T2**:
  New weights
* **__group_773__new_gradients** (optional) - **T3**:
  New gradients
* **__group_773__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_773__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_773__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_774__new_weights** (optional) - **T2**:
  New weights
* **__group_774__new_gradients** (optional) - **T3**:
  New gradients
* **__group_774__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_774__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_774__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_775__new_weights** (optional) - **T2**:
  New weights
* **__group_775__new_gradients** (optional) - **T3**:
  New gradients
* **__group_775__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_775__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_775__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_776__new_weights** (optional) - **T2**:
  New weights
* **__group_776__new_gradients** (optional) - **T3**:
  New gradients
* **__group_776__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_776__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_776__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_777__new_weights** (optional) - **T2**:
  New weights
* **__group_777__new_gradients** (optional) - **T3**:
  New gradients
* **__group_777__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_777__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_777__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_778__new_weights** (optional) - **T2**:
  New weights
* **__group_778__new_gradients** (optional) - **T3**:
  New gradients
* **__group_778__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_778__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_778__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_779__new_weights** (optional) - **T2**:
  New weights
* **__group_779__new_gradients** (optional) - **T3**:
  New gradients
* **__group_779__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_779__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_779__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_780__new_weights** (optional) - **T2**:
  New weights
* **__group_780__new_gradients** (optional) - **T3**:
  New gradients
* **__group_780__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_780__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_780__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_781__new_weights** (optional) - **T2**:
  New weights
* **__group_781__new_gradients** (optional) - **T3**:
  New gradients
* **__group_781__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_781__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_781__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_782__new_weights** (optional) - **T2**:
  New weights
* **__group_782__new_gradients** (optional) - **T3**:
  New gradients
* **__group_782__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_782__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_782__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_783__new_weights** (optional) - **T2**:
  New weights
* **__group_783__new_gradients** (optional) - **T3**:
  New gradients
* **__group_783__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_783__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_783__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_784__new_weights** (optional) - **T2**:
  New weights
* **__group_784__new_gradients** (optional) - **T3**:
  New gradients
* **__group_784__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_784__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_784__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_785__new_weights** (optional) - **T2**:
  New weights
* **__group_785__new_gradients** (optional) - **T3**:
  New gradients
* **__group_785__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_785__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_785__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_786__new_weights** (optional) - **T2**:
  New weights
* **__group_786__new_gradients** (optional) - **T3**:
  New gradients
* **__group_786__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_786__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_786__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_787__new_weights** (optional) - **T2**:
  New weights
* **__group_787__new_gradients** (optional) - **T3**:
  New gradients
* **__group_787__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_787__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_787__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_788__new_weights** (optional) - **T2**:
  New weights
* **__group_788__new_gradients** (optional) - **T3**:
  New gradients
* **__group_788__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_788__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_788__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_789__new_weights** (optional) - **T2**:
  New weights
* **__group_789__new_gradients** (optional) - **T3**:
  New gradients
* **__group_789__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_789__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_789__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_790__new_weights** (optional) - **T2**:
  New weights
* **__group_790__new_gradients** (optional) - **T3**:
  New gradients
* **__group_790__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_790__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_790__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_791__new_weights** (optional) - **T2**:
  New weights
* **__group_791__new_gradients** (optional) - **T3**:
  New gradients
* **__group_791__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_791__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_791__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_792__new_weights** (optional) - **T2**:
  New weights
* **__group_792__new_gradients** (optional) - **T3**:
  New gradients
* **__group_792__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_792__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_792__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_793__new_weights** (optional) - **T2**:
  New weights
* **__group_793__new_gradients** (optional) - **T3**:
  New gradients
* **__group_793__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_793__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_793__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_794__new_weights** (optional) - **T2**:
  New weights
* **__group_794__new_gradients** (optional) - **T3**:
  New gradients
* **__group_794__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_794__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_794__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_795__new_weights** (optional) - **T2**:
  New weights
* **__group_795__new_gradients** (optional) - **T3**:
  New gradients
* **__group_795__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_795__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_795__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_796__new_weights** (optional) - **T2**:
  New weights
* **__group_796__new_gradients** (optional) - **T3**:
  New gradients
* **__group_796__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_796__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_796__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_797__new_weights** (optional) - **T2**:
  New weights
* **__group_797__new_gradients** (optional) - **T3**:
  New gradients
* **__group_797__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_797__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_797__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_798__new_weights** (optional) - **T2**:
  New weights
* **__group_798__new_gradients** (optional) - **T3**:
  New gradients
* **__group_798__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_798__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_798__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_799__new_weights** (optional) - **T2**:
  New weights
* **__group_799__new_gradients** (optional) - **T3**:
  New gradients
* **__group_799__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_799__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_799__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_800__new_weights** (optional) - **T2**:
  New weights
* **__group_800__new_gradients** (optional) - **T3**:
  New gradients
* **__group_800__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_800__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_800__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_801__new_weights** (optional) - **T2**:
  New weights
* **__group_801__new_gradients** (optional) - **T3**:
  New gradients
* **__group_801__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_801__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_801__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_802__new_weights** (optional) - **T2**:
  New weights
* **__group_802__new_gradients** (optional) - **T3**:
  New gradients
* **__group_802__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_802__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_802__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_803__new_weights** (optional) - **T2**:
  New weights
* **__group_803__new_gradients** (optional) - **T3**:
  New gradients
* **__group_803__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_803__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_803__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_804__new_weights** (optional) - **T2**:
  New weights
* **__group_804__new_gradients** (optional) - **T3**:
  New gradients
* **__group_804__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_804__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_804__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_805__new_weights** (optional) - **T2**:
  New weights
* **__group_805__new_gradients** (optional) - **T3**:
  New gradients
* **__group_805__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_805__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_805__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_806__new_weights** (optional) - **T2**:
  New weights
* **__group_806__new_gradients** (optional) - **T3**:
  New gradients
* **__group_806__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_806__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_806__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_807__new_weights** (optional) - **T2**:
  New weights
* **__group_807__new_gradients** (optional) - **T3**:
  New gradients
* **__group_807__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_807__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_807__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_808__new_weights** (optional) - **T2**:
  New weights
* **__group_808__new_gradients** (optional) - **T3**:
  New gradients
* **__group_808__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_808__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_808__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_809__new_weights** (optional) - **T2**:
  New weights
* **__group_809__new_gradients** (optional) - **T3**:
  New gradients
* **__group_809__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_809__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_809__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_810__new_weights** (optional) - **T2**:
  New weights
* **__group_810__new_gradients** (optional) - **T3**:
  New gradients
* **__group_810__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_810__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_810__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_811__new_weights** (optional) - **T2**:
  New weights
* **__group_811__new_gradients** (optional) - **T3**:
  New gradients
* **__group_811__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_811__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_811__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_812__new_weights** (optional) - **T2**:
  New weights
* **__group_812__new_gradients** (optional) - **T3**:
  New gradients
* **__group_812__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_812__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_812__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_813__new_weights** (optional) - **T2**:
  New weights
* **__group_813__new_gradients** (optional) - **T3**:
  New gradients
* **__group_813__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_813__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_813__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_814__new_weights** (optional) - **T2**:
  New weights
* **__group_814__new_gradients** (optional) - **T3**:
  New gradients
* **__group_814__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_814__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_814__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_815__new_weights** (optional) - **T2**:
  New weights
* **__group_815__new_gradients** (optional) - **T3**:
  New gradients
* **__group_815__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_815__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_815__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_816__new_weights** (optional) - **T2**:
  New weights
* **__group_816__new_gradients** (optional) - **T3**:
  New gradients
* **__group_816__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_816__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_816__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_817__new_weights** (optional) - **T2**:
  New weights
* **__group_817__new_gradients** (optional) - **T3**:
  New gradients
* **__group_817__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_817__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_817__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_818__new_weights** (optional) - **T2**:
  New weights
* **__group_818__new_gradients** (optional) - **T3**:
  New gradients
* **__group_818__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_818__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_818__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_819__new_weights** (optional) - **T2**:
  New weights
* **__group_819__new_gradients** (optional) - **T3**:
  New gradients
* **__group_819__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_819__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_819__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_820__new_weights** (optional) - **T2**:
  New weights
* **__group_820__new_gradients** (optional) - **T3**:
  New gradients
* **__group_820__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_820__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_820__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_821__new_weights** (optional) - **T2**:
  New weights
* **__group_821__new_gradients** (optional) - **T3**:
  New gradients
* **__group_821__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_821__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_821__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_822__new_weights** (optional) - **T2**:
  New weights
* **__group_822__new_gradients** (optional) - **T3**:
  New gradients
* **__group_822__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_822__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_822__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_823__new_weights** (optional) - **T2**:
  New weights
* **__group_823__new_gradients** (optional) - **T3**:
  New gradients
* **__group_823__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_823__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_823__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_824__new_weights** (optional) - **T2**:
  New weights
* **__group_824__new_gradients** (optional) - **T3**:
  New gradients
* **__group_824__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_824__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_824__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_825__new_weights** (optional) - **T2**:
  New weights
* **__group_825__new_gradients** (optional) - **T3**:
  New gradients
* **__group_825__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_825__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_825__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_826__new_weights** (optional) - **T2**:
  New weights
* **__group_826__new_gradients** (optional) - **T3**:
  New gradients
* **__group_826__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_826__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_826__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_827__new_weights** (optional) - **T2**:
  New weights
* **__group_827__new_gradients** (optional) - **T3**:
  New gradients
* **__group_827__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_827__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_827__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_828__new_weights** (optional) - **T2**:
  New weights
* **__group_828__new_gradients** (optional) - **T3**:
  New gradients
* **__group_828__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_828__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_828__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_829__new_weights** (optional) - **T2**:
  New weights
* **__group_829__new_gradients** (optional) - **T3**:
  New gradients
* **__group_829__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_829__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_829__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_830__new_weights** (optional) - **T2**:
  New weights
* **__group_830__new_gradients** (optional) - **T3**:
  New gradients
* **__group_830__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_830__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_830__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_831__new_weights** (optional) - **T2**:
  New weights
* **__group_831__new_gradients** (optional) - **T3**:
  New gradients
* **__group_831__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_831__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_831__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_832__new_weights** (optional) - **T2**:
  New weights
* **__group_832__new_gradients** (optional) - **T3**:
  New gradients
* **__group_832__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_832__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_832__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_833__new_weights** (optional) - **T2**:
  New weights
* **__group_833__new_gradients** (optional) - **T3**:
  New gradients
* **__group_833__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_833__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_833__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_834__new_weights** (optional) - **T2**:
  New weights
* **__group_834__new_gradients** (optional) - **T3**:
  New gradients
* **__group_834__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_834__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_834__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_835__new_weights** (optional) - **T2**:
  New weights
* **__group_835__new_gradients** (optional) - **T3**:
  New gradients
* **__group_835__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_835__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_835__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_836__new_weights** (optional) - **T2**:
  New weights
* **__group_836__new_gradients** (optional) - **T3**:
  New gradients
* **__group_836__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_836__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_836__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_837__new_weights** (optional) - **T2**:
  New weights
* **__group_837__new_gradients** (optional) - **T3**:
  New gradients
* **__group_837__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_837__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_837__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_838__new_weights** (optional) - **T2**:
  New weights
* **__group_838__new_gradients** (optional) - **T3**:
  New gradients
* **__group_838__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_838__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_838__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_839__new_weights** (optional) - **T2**:
  New weights
* **__group_839__new_gradients** (optional) - **T3**:
  New gradients
* **__group_839__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_839__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_839__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_840__new_weights** (optional) - **T2**:
  New weights
* **__group_840__new_gradients** (optional) - **T3**:
  New gradients
* **__group_840__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_840__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_840__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_841__new_weights** (optional) - **T2**:
  New weights
* **__group_841__new_gradients** (optional) - **T3**:
  New gradients
* **__group_841__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_841__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_841__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_842__new_weights** (optional) - **T2**:
  New weights
* **__group_842__new_gradients** (optional) - **T3**:
  New gradients
* **__group_842__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_842__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_842__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_843__new_weights** (optional) - **T2**:
  New weights
* **__group_843__new_gradients** (optional) - **T3**:
  New gradients
* **__group_843__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_843__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_843__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_844__new_weights** (optional) - **T2**:
  New weights
* **__group_844__new_gradients** (optional) - **T3**:
  New gradients
* **__group_844__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_844__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_844__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_845__new_weights** (optional) - **T2**:
  New weights
* **__group_845__new_gradients** (optional) - **T3**:
  New gradients
* **__group_845__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_845__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_845__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_846__new_weights** (optional) - **T2**:
  New weights
* **__group_846__new_gradients** (optional) - **T3**:
  New gradients
* **__group_846__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_846__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_846__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_847__new_weights** (optional) - **T2**:
  New weights
* **__group_847__new_gradients** (optional) - **T3**:
  New gradients
* **__group_847__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_847__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_847__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_848__new_weights** (optional) - **T2**:
  New weights
* **__group_848__new_gradients** (optional) - **T3**:
  New gradients
* **__group_848__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_848__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_848__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_849__new_weights** (optional) - **T2**:
  New weights
* **__group_849__new_gradients** (optional) - **T3**:
  New gradients
* **__group_849__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_849__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_849__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_850__new_weights** (optional) - **T2**:
  New weights
* **__group_850__new_gradients** (optional) - **T3**:
  New gradients
* **__group_850__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_850__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_850__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_851__new_weights** (optional) - **T2**:
  New weights
* **__group_851__new_gradients** (optional) - **T3**:
  New gradients
* **__group_851__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_851__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_851__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_852__new_weights** (optional) - **T2**:
  New weights
* **__group_852__new_gradients** (optional) - **T3**:
  New gradients
* **__group_852__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_852__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_852__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_853__new_weights** (optional) - **T2**:
  New weights
* **__group_853__new_gradients** (optional) - **T3**:
  New gradients
* **__group_853__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_853__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_853__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_854__new_weights** (optional) - **T2**:
  New weights
* **__group_854__new_gradients** (optional) - **T3**:
  New gradients
* **__group_854__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_854__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_854__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_855__new_weights** (optional) - **T2**:
  New weights
* **__group_855__new_gradients** (optional) - **T3**:
  New gradients
* **__group_855__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_855__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_855__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_856__new_weights** (optional) - **T2**:
  New weights
* **__group_856__new_gradients** (optional) - **T3**:
  New gradients
* **__group_856__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_856__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_856__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_857__new_weights** (optional) - **T2**:
  New weights
* **__group_857__new_gradients** (optional) - **T3**:
  New gradients
* **__group_857__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_857__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_857__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_858__new_weights** (optional) - **T2**:
  New weights
* **__group_858__new_gradients** (optional) - **T3**:
  New gradients
* **__group_858__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_858__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_858__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_859__new_weights** (optional) - **T2**:
  New weights
* **__group_859__new_gradients** (optional) - **T3**:
  New gradients
* **__group_859__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_859__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_859__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_860__new_weights** (optional) - **T2**:
  New weights
* **__group_860__new_gradients** (optional) - **T3**:
  New gradients
* **__group_860__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_860__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_860__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_861__new_weights** (optional) - **T2**:
  New weights
* **__group_861__new_gradients** (optional) - **T3**:
  New gradients
* **__group_861__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_861__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_861__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_862__new_weights** (optional) - **T2**:
  New weights
* **__group_862__new_gradients** (optional) - **T3**:
  New gradients
* **__group_862__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_862__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_862__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_863__new_weights** (optional) - **T2**:
  New weights
* **__group_863__new_gradients** (optional) - **T3**:
  New gradients
* **__group_863__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_863__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_863__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_864__new_weights** (optional) - **T2**:
  New weights
* **__group_864__new_gradients** (optional) - **T3**:
  New gradients
* **__group_864__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_864__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_864__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_865__new_weights** (optional) - **T2**:
  New weights
* **__group_865__new_gradients** (optional) - **T3**:
  New gradients
* **__group_865__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_865__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_865__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_866__new_weights** (optional) - **T2**:
  New weights
* **__group_866__new_gradients** (optional) - **T3**:
  New gradients
* **__group_866__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_866__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_866__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_867__new_weights** (optional) - **T2**:
  New weights
* **__group_867__new_gradients** (optional) - **T3**:
  New gradients
* **__group_867__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_867__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_867__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_868__new_weights** (optional) - **T2**:
  New weights
* **__group_868__new_gradients** (optional) - **T3**:
  New gradients
* **__group_868__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_868__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_868__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_869__new_weights** (optional) - **T2**:
  New weights
* **__group_869__new_gradients** (optional) - **T3**:
  New gradients
* **__group_869__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_869__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_869__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_870__new_weights** (optional) - **T2**:
  New weights
* **__group_870__new_gradients** (optional) - **T3**:
  New gradients
* **__group_870__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_870__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_870__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_871__new_weights** (optional) - **T2**:
  New weights
* **__group_871__new_gradients** (optional) - **T3**:
  New gradients
* **__group_871__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_871__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_871__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_872__new_weights** (optional) - **T2**:
  New weights
* **__group_872__new_gradients** (optional) - **T3**:
  New gradients
* **__group_872__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_872__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_872__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_873__new_weights** (optional) - **T2**:
  New weights
* **__group_873__new_gradients** (optional) - **T3**:
  New gradients
* **__group_873__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_873__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_873__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_874__new_weights** (optional) - **T2**:
  New weights
* **__group_874__new_gradients** (optional) - **T3**:
  New gradients
* **__group_874__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_874__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_874__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_875__new_weights** (optional) - **T2**:
  New weights
* **__group_875__new_gradients** (optional) - **T3**:
  New gradients
* **__group_875__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_875__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_875__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_876__new_weights** (optional) - **T2**:
  New weights
* **__group_876__new_gradients** (optional) - **T3**:
  New gradients
* **__group_876__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_876__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_876__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_877__new_weights** (optional) - **T2**:
  New weights
* **__group_877__new_gradients** (optional) - **T3**:
  New gradients
* **__group_877__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_877__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_877__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_878__new_weights** (optional) - **T2**:
  New weights
* **__group_878__new_gradients** (optional) - **T3**:
  New gradients
* **__group_878__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_878__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_878__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_879__new_weights** (optional) - **T2**:
  New weights
* **__group_879__new_gradients** (optional) - **T3**:
  New gradients
* **__group_879__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_879__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_879__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_880__new_weights** (optional) - **T2**:
  New weights
* **__group_880__new_gradients** (optional) - **T3**:
  New gradients
* **__group_880__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_880__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_880__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_881__new_weights** (optional) - **T2**:
  New weights
* **__group_881__new_gradients** (optional) - **T3**:
  New gradients
* **__group_881__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_881__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_881__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_882__new_weights** (optional) - **T2**:
  New weights
* **__group_882__new_gradients** (optional) - **T3**:
  New gradients
* **__group_882__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_882__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_882__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_883__new_weights** (optional) - **T2**:
  New weights
* **__group_883__new_gradients** (optional) - **T3**:
  New gradients
* **__group_883__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_883__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_883__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_884__new_weights** (optional) - **T2**:
  New weights
* **__group_884__new_gradients** (optional) - **T3**:
  New gradients
* **__group_884__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_884__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_884__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_885__new_weights** (optional) - **T2**:
  New weights
* **__group_885__new_gradients** (optional) - **T3**:
  New gradients
* **__group_885__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_885__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_885__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_886__new_weights** (optional) - **T2**:
  New weights
* **__group_886__new_gradients** (optional) - **T3**:
  New gradients
* **__group_886__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_886__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_886__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_887__new_weights** (optional) - **T2**:
  New weights
* **__group_887__new_gradients** (optional) - **T3**:
  New gradients
* **__group_887__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_887__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_887__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_888__new_weights** (optional) - **T2**:
  New weights
* **__group_888__new_gradients** (optional) - **T3**:
  New gradients
* **__group_888__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_888__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_888__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_889__new_weights** (optional) - **T2**:
  New weights
* **__group_889__new_gradients** (optional) - **T3**:
  New gradients
* **__group_889__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_889__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_889__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_890__new_weights** (optional) - **T2**:
  New weights
* **__group_890__new_gradients** (optional) - **T3**:
  New gradients
* **__group_890__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_890__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_890__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_891__new_weights** (optional) - **T2**:
  New weights
* **__group_891__new_gradients** (optional) - **T3**:
  New gradients
* **__group_891__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_891__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_891__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_892__new_weights** (optional) - **T2**:
  New weights
* **__group_892__new_gradients** (optional) - **T3**:
  New gradients
* **__group_892__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_892__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_892__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_893__new_weights** (optional) - **T2**:
  New weights
* **__group_893__new_gradients** (optional) - **T3**:
  New gradients
* **__group_893__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_893__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_893__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_894__new_weights** (optional) - **T2**:
  New weights
* **__group_894__new_gradients** (optional) - **T3**:
  New gradients
* **__group_894__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_894__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_894__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_895__new_weights** (optional) - **T2**:
  New weights
* **__group_895__new_gradients** (optional) - **T3**:
  New gradients
* **__group_895__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_895__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_895__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_896__new_weights** (optional) - **T2**:
  New weights
* **__group_896__new_gradients** (optional) - **T3**:
  New gradients
* **__group_896__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_896__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_896__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_897__new_weights** (optional) - **T2**:
  New weights
* **__group_897__new_gradients** (optional) - **T3**:
  New gradients
* **__group_897__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_897__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_897__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_898__new_weights** (optional) - **T2**:
  New weights
* **__group_898__new_gradients** (optional) - **T3**:
  New gradients
* **__group_898__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_898__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_898__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_899__new_weights** (optional) - **T2**:
  New weights
* **__group_899__new_gradients** (optional) - **T3**:
  New gradients
* **__group_899__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_899__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_899__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_900__new_weights** (optional) - **T2**:
  New weights
* **__group_900__new_gradients** (optional) - **T3**:
  New gradients
* **__group_900__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_900__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_900__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_901__new_weights** (optional) - **T2**:
  New weights
* **__group_901__new_gradients** (optional) - **T3**:
  New gradients
* **__group_901__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_901__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_901__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_902__new_weights** (optional) - **T2**:
  New weights
* **__group_902__new_gradients** (optional) - **T3**:
  New gradients
* **__group_902__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_902__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_902__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_903__new_weights** (optional) - **T2**:
  New weights
* **__group_903__new_gradients** (optional) - **T3**:
  New gradients
* **__group_903__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_903__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_903__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_904__new_weights** (optional) - **T2**:
  New weights
* **__group_904__new_gradients** (optional) - **T3**:
  New gradients
* **__group_904__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_904__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_904__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_905__new_weights** (optional) - **T2**:
  New weights
* **__group_905__new_gradients** (optional) - **T3**:
  New gradients
* **__group_905__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_905__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_905__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_906__new_weights** (optional) - **T2**:
  New weights
* **__group_906__new_gradients** (optional) - **T3**:
  New gradients
* **__group_906__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_906__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_906__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_907__new_weights** (optional) - **T2**:
  New weights
* **__group_907__new_gradients** (optional) - **T3**:
  New gradients
* **__group_907__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_907__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_907__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_908__new_weights** (optional) - **T2**:
  New weights
* **__group_908__new_gradients** (optional) - **T3**:
  New gradients
* **__group_908__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_908__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_908__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_909__new_weights** (optional) - **T2**:
  New weights
* **__group_909__new_gradients** (optional) - **T3**:
  New gradients
* **__group_909__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_909__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_909__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_910__new_weights** (optional) - **T2**:
  New weights
* **__group_910__new_gradients** (optional) - **T3**:
  New gradients
* **__group_910__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_910__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_910__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_911__new_weights** (optional) - **T2**:
  New weights
* **__group_911__new_gradients** (optional) - **T3**:
  New gradients
* **__group_911__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_911__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_911__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_912__new_weights** (optional) - **T2**:
  New weights
* **__group_912__new_gradients** (optional) - **T3**:
  New gradients
* **__group_912__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_912__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_912__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_913__new_weights** (optional) - **T2**:
  New weights
* **__group_913__new_gradients** (optional) - **T3**:
  New gradients
* **__group_913__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_913__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_913__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_914__new_weights** (optional) - **T2**:
  New weights
* **__group_914__new_gradients** (optional) - **T3**:
  New gradients
* **__group_914__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_914__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_914__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_915__new_weights** (optional) - **T2**:
  New weights
* **__group_915__new_gradients** (optional) - **T3**:
  New gradients
* **__group_915__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_915__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_915__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_916__new_weights** (optional) - **T2**:
  New weights
* **__group_916__new_gradients** (optional) - **T3**:
  New gradients
* **__group_916__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_916__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_916__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_917__new_weights** (optional) - **T2**:
  New weights
* **__group_917__new_gradients** (optional) - **T3**:
  New gradients
* **__group_917__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_917__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_917__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_918__new_weights** (optional) - **T2**:
  New weights
* **__group_918__new_gradients** (optional) - **T3**:
  New gradients
* **__group_918__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_918__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_918__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_919__new_weights** (optional) - **T2**:
  New weights
* **__group_919__new_gradients** (optional) - **T3**:
  New gradients
* **__group_919__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_919__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_919__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_920__new_weights** (optional) - **T2**:
  New weights
* **__group_920__new_gradients** (optional) - **T3**:
  New gradients
* **__group_920__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_920__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_920__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_921__new_weights** (optional) - **T2**:
  New weights
* **__group_921__new_gradients** (optional) - **T3**:
  New gradients
* **__group_921__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_921__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_921__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_922__new_weights** (optional) - **T2**:
  New weights
* **__group_922__new_gradients** (optional) - **T3**:
  New gradients
* **__group_922__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_922__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_922__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_923__new_weights** (optional) - **T2**:
  New weights
* **__group_923__new_gradients** (optional) - **T3**:
  New gradients
* **__group_923__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_923__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_923__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_924__new_weights** (optional) - **T2**:
  New weights
* **__group_924__new_gradients** (optional) - **T3**:
  New gradients
* **__group_924__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_924__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_924__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_925__new_weights** (optional) - **T2**:
  New weights
* **__group_925__new_gradients** (optional) - **T3**:
  New gradients
* **__group_925__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_925__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_925__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_926__new_weights** (optional) - **T2**:
  New weights
* **__group_926__new_gradients** (optional) - **T3**:
  New gradients
* **__group_926__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_926__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_926__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_927__new_weights** (optional) - **T2**:
  New weights
* **__group_927__new_gradients** (optional) - **T3**:
  New gradients
* **__group_927__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_927__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_927__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_928__new_weights** (optional) - **T2**:
  New weights
* **__group_928__new_gradients** (optional) - **T3**:
  New gradients
* **__group_928__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_928__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_928__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_929__new_weights** (optional) - **T2**:
  New weights
* **__group_929__new_gradients** (optional) - **T3**:
  New gradients
* **__group_929__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_929__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_929__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_930__new_weights** (optional) - **T2**:
  New weights
* **__group_930__new_gradients** (optional) - **T3**:
  New gradients
* **__group_930__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_930__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_930__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_931__new_weights** (optional) - **T2**:
  New weights
* **__group_931__new_gradients** (optional) - **T3**:
  New gradients
* **__group_931__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_931__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_931__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_932__new_weights** (optional) - **T2**:
  New weights
* **__group_932__new_gradients** (optional) - **T3**:
  New gradients
* **__group_932__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_932__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_932__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_933__new_weights** (optional) - **T2**:
  New weights
* **__group_933__new_gradients** (optional) - **T3**:
  New gradients
* **__group_933__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_933__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_933__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_934__new_weights** (optional) - **T2**:
  New weights
* **__group_934__new_gradients** (optional) - **T3**:
  New gradients
* **__group_934__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_934__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_934__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_935__new_weights** (optional) - **T2**:
  New weights
* **__group_935__new_gradients** (optional) - **T3**:
  New gradients
* **__group_935__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_935__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_935__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_936__new_weights** (optional) - **T2**:
  New weights
* **__group_936__new_gradients** (optional) - **T3**:
  New gradients
* **__group_936__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_936__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_936__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_937__new_weights** (optional) - **T2**:
  New weights
* **__group_937__new_gradients** (optional) - **T3**:
  New gradients
* **__group_937__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_937__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_937__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_938__new_weights** (optional) - **T2**:
  New weights
* **__group_938__new_gradients** (optional) - **T3**:
  New gradients
* **__group_938__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_938__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_938__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_939__new_weights** (optional) - **T2**:
  New weights
* **__group_939__new_gradients** (optional) - **T3**:
  New gradients
* **__group_939__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_939__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_939__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_940__new_weights** (optional) - **T2**:
  New weights
* **__group_940__new_gradients** (optional) - **T3**:
  New gradients
* **__group_940__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_940__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_940__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_941__new_weights** (optional) - **T2**:
  New weights
* **__group_941__new_gradients** (optional) - **T3**:
  New gradients
* **__group_941__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_941__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_941__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_942__new_weights** (optional) - **T2**:
  New weights
* **__group_942__new_gradients** (optional) - **T3**:
  New gradients
* **__group_942__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_942__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_942__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_943__new_weights** (optional) - **T2**:
  New weights
* **__group_943__new_gradients** (optional) - **T3**:
  New gradients
* **__group_943__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_943__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_943__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_944__new_weights** (optional) - **T2**:
  New weights
* **__group_944__new_gradients** (optional) - **T3**:
  New gradients
* **__group_944__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_944__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_944__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_945__new_weights** (optional) - **T2**:
  New weights
* **__group_945__new_gradients** (optional) - **T3**:
  New gradients
* **__group_945__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_945__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_945__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_946__new_weights** (optional) - **T2**:
  New weights
* **__group_946__new_gradients** (optional) - **T3**:
  New gradients
* **__group_946__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_946__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_946__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_947__new_weights** (optional) - **T2**:
  New weights
* **__group_947__new_gradients** (optional) - **T3**:
  New gradients
* **__group_947__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_947__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_947__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_948__new_weights** (optional) - **T2**:
  New weights
* **__group_948__new_gradients** (optional) - **T3**:
  New gradients
* **__group_948__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_948__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_948__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_949__new_weights** (optional) - **T2**:
  New weights
* **__group_949__new_gradients** (optional) - **T3**:
  New gradients
* **__group_949__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_949__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_949__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_950__new_weights** (optional) - **T2**:
  New weights
* **__group_950__new_gradients** (optional) - **T3**:
  New gradients
* **__group_950__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_950__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_950__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_951__new_weights** (optional) - **T2**:
  New weights
* **__group_951__new_gradients** (optional) - **T3**:
  New gradients
* **__group_951__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_951__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_951__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_952__new_weights** (optional) - **T2**:
  New weights
* **__group_952__new_gradients** (optional) - **T3**:
  New gradients
* **__group_952__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_952__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_952__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_953__new_weights** (optional) - **T2**:
  New weights
* **__group_953__new_gradients** (optional) - **T3**:
  New gradients
* **__group_953__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_953__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_953__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_954__new_weights** (optional) - **T2**:
  New weights
* **__group_954__new_gradients** (optional) - **T3**:
  New gradients
* **__group_954__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_954__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_954__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_955__new_weights** (optional) - **T2**:
  New weights
* **__group_955__new_gradients** (optional) - **T3**:
  New gradients
* **__group_955__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_955__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_955__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_956__new_weights** (optional) - **T2**:
  New weights
* **__group_956__new_gradients** (optional) - **T3**:
  New gradients
* **__group_956__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_956__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_956__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_957__new_weights** (optional) - **T2**:
  New weights
* **__group_957__new_gradients** (optional) - **T3**:
  New gradients
* **__group_957__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_957__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_957__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_958__new_weights** (optional) - **T2**:
  New weights
* **__group_958__new_gradients** (optional) - **T3**:
  New gradients
* **__group_958__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_958__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_958__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_959__new_weights** (optional) - **T2**:
  New weights
* **__group_959__new_gradients** (optional) - **T3**:
  New gradients
* **__group_959__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_959__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_959__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_960__new_weights** (optional) - **T2**:
  New weights
* **__group_960__new_gradients** (optional) - **T3**:
  New gradients
* **__group_960__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_960__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_960__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_961__new_weights** (optional) - **T2**:
  New weights
* **__group_961__new_gradients** (optional) - **T3**:
  New gradients
* **__group_961__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_961__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_961__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_962__new_weights** (optional) - **T2**:
  New weights
* **__group_962__new_gradients** (optional) - **T3**:
  New gradients
* **__group_962__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_962__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_962__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_963__new_weights** (optional) - **T2**:
  New weights
* **__group_963__new_gradients** (optional) - **T3**:
  New gradients
* **__group_963__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_963__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_963__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_964__new_weights** (optional) - **T2**:
  New weights
* **__group_964__new_gradients** (optional) - **T3**:
  New gradients
* **__group_964__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_964__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_964__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_965__new_weights** (optional) - **T2**:
  New weights
* **__group_965__new_gradients** (optional) - **T3**:
  New gradients
* **__group_965__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_965__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_965__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_966__new_weights** (optional) - **T2**:
  New weights
* **__group_966__new_gradients** (optional) - **T3**:
  New gradients
* **__group_966__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_966__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_966__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_967__new_weights** (optional) - **T2**:
  New weights
* **__group_967__new_gradients** (optional) - **T3**:
  New gradients
* **__group_967__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_967__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_967__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_968__new_weights** (optional) - **T2**:
  New weights
* **__group_968__new_gradients** (optional) - **T3**:
  New gradients
* **__group_968__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_968__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_968__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_969__new_weights** (optional) - **T2**:
  New weights
* **__group_969__new_gradients** (optional) - **T3**:
  New gradients
* **__group_969__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_969__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_969__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_970__new_weights** (optional) - **T2**:
  New weights
* **__group_970__new_gradients** (optional) - **T3**:
  New gradients
* **__group_970__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_970__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_970__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_971__new_weights** (optional) - **T2**:
  New weights
* **__group_971__new_gradients** (optional) - **T3**:
  New gradients
* **__group_971__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_971__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_971__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_972__new_weights** (optional) - **T2**:
  New weights
* **__group_972__new_gradients** (optional) - **T3**:
  New gradients
* **__group_972__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_972__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_972__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_973__new_weights** (optional) - **T2**:
  New weights
* **__group_973__new_gradients** (optional) - **T3**:
  New gradients
* **__group_973__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_973__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_973__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_974__new_weights** (optional) - **T2**:
  New weights
* **__group_974__new_gradients** (optional) - **T3**:
  New gradients
* **__group_974__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_974__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_974__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_975__new_weights** (optional) - **T2**:
  New weights
* **__group_975__new_gradients** (optional) - **T3**:
  New gradients
* **__group_975__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_975__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_975__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_976__new_weights** (optional) - **T2**:
  New weights
* **__group_976__new_gradients** (optional) - **T3**:
  New gradients
* **__group_976__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_976__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_976__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_977__new_weights** (optional) - **T2**:
  New weights
* **__group_977__new_gradients** (optional) - **T3**:
  New gradients
* **__group_977__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_977__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_977__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_978__new_weights** (optional) - **T2**:
  New weights
* **__group_978__new_gradients** (optional) - **T3**:
  New gradients
* **__group_978__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_978__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_978__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_979__new_weights** (optional) - **T2**:
  New weights
* **__group_979__new_gradients** (optional) - **T3**:
  New gradients
* **__group_979__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_979__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_979__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_980__new_weights** (optional) - **T2**:
  New weights
* **__group_980__new_gradients** (optional) - **T3**:
  New gradients
* **__group_980__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_980__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_980__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_981__new_weights** (optional) - **T2**:
  New weights
* **__group_981__new_gradients** (optional) - **T3**:
  New gradients
* **__group_981__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_981__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_981__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_982__new_weights** (optional) - **T2**:
  New weights
* **__group_982__new_gradients** (optional) - **T3**:
  New gradients
* **__group_982__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_982__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_982__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_983__new_weights** (optional) - **T2**:
  New weights
* **__group_983__new_gradients** (optional) - **T3**:
  New gradients
* **__group_983__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_983__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_983__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_984__new_weights** (optional) - **T2**:
  New weights
* **__group_984__new_gradients** (optional) - **T3**:
  New gradients
* **__group_984__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_984__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_984__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_985__new_weights** (optional) - **T2**:
  New weights
* **__group_985__new_gradients** (optional) - **T3**:
  New gradients
* **__group_985__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_985__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_985__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_986__new_weights** (optional) - **T2**:
  New weights
* **__group_986__new_gradients** (optional) - **T3**:
  New gradients
* **__group_986__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_986__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_986__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_987__new_weights** (optional) - **T2**:
  New weights
* **__group_987__new_gradients** (optional) - **T3**:
  New gradients
* **__group_987__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_987__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_987__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_988__new_weights** (optional) - **T2**:
  New weights
* **__group_988__new_gradients** (optional) - **T3**:
  New gradients
* **__group_988__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_988__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_988__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_989__new_weights** (optional) - **T2**:
  New weights
* **__group_989__new_gradients** (optional) - **T3**:
  New gradients
* **__group_989__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_989__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_989__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_990__new_weights** (optional) - **T2**:
  New weights
* **__group_990__new_gradients** (optional) - **T3**:
  New gradients
* **__group_990__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_990__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_990__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_991__new_weights** (optional) - **T2**:
  New weights
* **__group_991__new_gradients** (optional) - **T3**:
  New gradients
* **__group_991__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_991__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_991__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_992__new_weights** (optional) - **T2**:
  New weights
* **__group_992__new_gradients** (optional) - **T3**:
  New gradients
* **__group_992__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_992__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_992__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_993__new_weights** (optional) - **T2**:
  New weights
* **__group_993__new_gradients** (optional) - **T3**:
  New gradients
* **__group_993__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_993__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_993__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_994__new_weights** (optional) - **T2**:
  New weights
* **__group_994__new_gradients** (optional) - **T3**:
  New gradients
* **__group_994__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_994__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_994__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_995__new_weights** (optional) - **T2**:
  New weights
* **__group_995__new_gradients** (optional) - **T3**:
  New gradients
* **__group_995__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_995__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_995__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_996__new_weights** (optional) - **T2**:
  New weights
* **__group_996__new_gradients** (optional) - **T3**:
  New gradients
* **__group_996__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_996__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_996__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_997__new_weights** (optional) - **T2**:
  New weights
* **__group_997__new_gradients** (optional) - **T3**:
  New gradients
* **__group_997__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_997__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_997__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_998__new_weights** (optional) - **T2**:
  New weights
* **__group_998__new_gradients** (optional) - **T3**:
  New gradients
* **__group_998__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_998__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_998__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_999__new_weights** (optional) - **T2**:
  New weights
* **__group_999__new_gradients** (optional) - **T3**:
  New gradients
* **__group_999__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_999__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_999__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_1000__new_weights** (optional) - **T2**:
  New weights
* **__group_1000__new_gradients** (optional) - **T3**:
  New gradients
* **__group_1000__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_1000__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_1000__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_1001__new_weights** (optional) - **T2**:
  New weights
* **__group_1001__new_gradients** (optional) - **T3**:
  New gradients
* **__group_1001__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_1001__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_1001__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_1002__new_weights** (optional) - **T2**:
  New weights
* **__group_1002__new_gradients** (optional) - **T3**:
  New gradients
* **__group_1002__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_1002__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_1002__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_1003__new_weights** (optional) - **T2**:
  New weights
* **__group_1003__new_gradients** (optional) - **T3**:
  New gradients
* **__group_1003__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_1003__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_1003__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_1004__new_weights** (optional) - **T2**:
  New weights
* **__group_1004__new_gradients** (optional) - **T3**:
  New gradients
* **__group_1004__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_1004__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_1004__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_1005__new_weights** (optional) - **T2**:
  New weights
* **__group_1005__new_gradients** (optional) - **T3**:
  New gradients
* **__group_1005__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_1005__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_1005__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_1006__new_weights** (optional) - **T2**:
  New weights
* **__group_1006__new_gradients** (optional) - **T3**:
  New gradients
* **__group_1006__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_1006__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_1006__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_1007__new_weights** (optional) - **T2**:
  New weights
* **__group_1007__new_gradients** (optional) - **T3**:
  New gradients
* **__group_1007__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_1007__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_1007__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_1008__new_weights** (optional) - **T2**:
  New weights
* **__group_1008__new_gradients** (optional) - **T3**:
  New gradients
* **__group_1008__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_1008__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_1008__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_1009__new_weights** (optional) - **T2**:
  New weights
* **__group_1009__new_gradients** (optional) - **T3**:
  New gradients
* **__group_1009__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_1009__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_1009__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_1010__new_weights** (optional) - **T2**:
  New weights
* **__group_1010__new_gradients** (optional) - **T3**:
  New gradients
* **__group_1010__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_1010__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_1010__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_1011__new_weights** (optional) - **T2**:
  New weights
* **__group_1011__new_gradients** (optional) - **T3**:
  New gradients
* **__group_1011__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_1011__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_1011__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_1012__new_weights** (optional) - **T2**:
  New weights
* **__group_1012__new_gradients** (optional) - **T3**:
  New gradients
* **__group_1012__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_1012__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_1012__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_1013__new_weights** (optional) - **T2**:
  New weights
* **__group_1013__new_gradients** (optional) - **T3**:
  New gradients
* **__group_1013__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_1013__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_1013__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_1014__new_weights** (optional) - **T2**:
  New weights
* **__group_1014__new_gradients** (optional) - **T3**:
  New gradients
* **__group_1014__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_1014__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_1014__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_1015__new_weights** (optional) - **T2**:
  New weights
* **__group_1015__new_gradients** (optional) - **T3**:
  New gradients
* **__group_1015__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_1015__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_1015__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_1016__new_weights** (optional) - **T2**:
  New weights
* **__group_1016__new_gradients** (optional) - **T3**:
  New gradients
* **__group_1016__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_1016__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_1016__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_1017__new_weights** (optional) - **T2**:
  New weights
* **__group_1017__new_gradients** (optional) - **T3**:
  New gradients
* **__group_1017__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_1017__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_1017__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_1018__new_weights** (optional) - **T2**:
  New weights
* **__group_1018__new_gradients** (optional) - **T3**:
  New gradients
* **__group_1018__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_1018__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_1018__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_1019__new_weights** (optional) - **T2**:
  New weights
* **__group_1019__new_gradients** (optional) - **T3**:
  New gradients
* **__group_1019__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_1019__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_1019__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_1020__new_weights** (optional) - **T2**:
  New weights
* **__group_1020__new_gradients** (optional) - **T3**:
  New gradients
* **__group_1020__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_1020__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_1020__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_1021__new_weights** (optional) - **T2**:
  New weights
* **__group_1021__new_gradients** (optional) - **T3**:
  New gradients
* **__group_1021__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_1021__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_1021__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_1022__new_weights** (optional) - **T2**:
  New weights
* **__group_1022__new_gradients** (optional) - **T3**:
  New gradients
* **__group_1022__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_1022__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_1022__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights
* **__group_1023__new_weights** (optional) - **T2**:
  New weights
* **__group_1023__new_gradients** (optional) - **T3**:
  New gradients
* **__group_1023__new_moment_1** (optional) - **T4**:
  New averaged gradients
* **__group_1023__new_moment_2** (optional) - **T4**:
  New averaged squared gradients
* **__group_1023__new_mixed_precision_weights** (optional) - **T_MIXED_PRECISION_FP**:
  New FP16 or BF16 weights

**Examples**