com.microsoft - GreedySearch#

GreedySearch - 1 (com.microsoft)#

Version

  • name: GreedySearch (GitHub)

  • domain: com.microsoft

  • since_version: 1

  • function:

  • support_level:

  • shape inference:

This version of the operator has been available since version 1 of domain com.microsoft.

Summary

Greedy Search for text generation.

Attributes

  • decoder (required): Decoder subgraph to execute in a loop. Default value is ?.

  • decoder_start_token_id: The id of the token that indicates decoding starts. Default value is ?.

  • encoder: The subgraph for initialization of encoder and decoder. It will be called once before decoder subgraph. Default value is ?.

  • eos_token_id (required): The id of the end-of-sequence token Default value is ?.

  • model_type: model type: 0 for decoder only like GPT-2; 1 for encoder decoder like Bart Default value is ?.

  • no_repeat_ngram_size: no repeat ngrams size Default value is ?.

  • pad_token_id (required): The id of the padding token Default value is ?.

Inputs

Between 2 and 7 inputs.

  • input_ids (heterogeneous) - I: The sequence used as a prompt for the generation. Shape is (batch_size, sequence_length)

  • max_length (heterogeneous) - I: The maximum length of the sequence to be generated. Shape is (1)

  • min_length (optional, heterogeneous) - I: The minimum length below which the score of eos_token_id is set to -Inf. Shape is (1)

  • repetition_penalty (optional, heterogeneous) - T: The parameter for repetition penalty. Default value 1.0 means no penalty. Accepts value > 0.0. Shape is (1)

  • vocab_mask (optional, heterogeneous) - I: Mask of vocabulary. Words that masked with 0 are not allowed to be generated, and 1 is allowed. Shape is (vacab_size)

  • prefix_vocab_mask (optional, heterogeneous) - I: Mask of vocabulary for first step. Words that masked with 0 are not allowed to be generated, and 1 is allowed. Shape is (batch_size, vocab_size)

  • attention_mask (optional, heterogeneous) - I: Custom attention mask. Shape is (batch_size, sequence_length)

Outputs

  • sequences (heterogeneous) - I: Word IDs of generated sequences. Shape is (batch_size, max_sequence_length)

Examples