EventStream.transformer.generation.generation_utils module¶

class EventStream.transformer.generation.generation_utils.SampleDecoderOnlyOutput(scores: tuple[GenerativeSequenceModelPredictions] | None = None, batch: PytorchBatch | None = None, attentions: tuple[tuple[FloatTensor]] | None = None, hidden_states: tuple[tuple[FloatTensor]] | None = None)[source]¶

Bases: ModelOutput

Base class for outputs of decoder-only generation models using sampling.

Parameters:¶

batch: PytorchBatch | None = None¶

The generated sequences.

( : hidden_states

tuple(GenerativeSequenceModelPredictions) optional, returned when output_scores=True is passed or when config.output_scores=True

)

Processed predictions of the generative sequence modeling head, as torch distributions at each generation step.

(

tuple(tuple(torch.FloatTensor)), optional, returned when output_attentions=True is passed or config.output_attentions=True

)

Tuple (one element for each generated token) of tuples (one element for each layer of the decoder) of torch.FloatTensor of shape (batch_size, num_heads, generated_length, sequence_length).

(

tuple(tuple(torch.FloatTensor)), optional, returned when output_hidden_states=True is passed or when config.output_hidden_states=True

)

Tuple (one element for each generated token) of tuples (one element for each layer of the decoder) of torch.FloatTensor of shape (batch_size, generated_length, dependency_graph_len, hidden_size).

attentions : tuple[tuple[FloatTensor]] | None = None¶

batch : PytorchBatch | None = None¶

hidden_states : tuple[tuple[FloatTensor]] | None = None¶

scores : tuple[GenerativeSequenceModelPredictions] | None = None¶

class EventStream.transformer.generation.generation_utils.StructuredGenerationMixin[source]¶

Bases: object

A class containing all functions for auto-regressive structured event stream generation, to be used as a mixin in [PreTrainedModel].

The class exposes [generate], which can be used for:

• sampling by calling [sample] if do_sample=True.

generate(batch: PytorchBatch, max_length: int | None = None, do_sample: bool | None = True, num_return_sequences: int | None = None, max_new_events: int | None = None, use_cache: bool | None = None, stopping_criteria: StoppingCriteriaList | None = [], output_attentions: bool | None = None, output_hidden_states: bool | None = None, output_scores: bool | None = None, return_dict_in_generate: bool | None = None, synced_gpus: bool | None = False, **model_kwargs) → SampleDecoderOnlyOutput | PytorchBatch[source]¶

prepare_inputs_for_generation(batch: PytorchBatch, **kwargs) → dict[str, Any][source]¶