TransformerDecoder

class torch.nn.TransformerDecoder(decoder_layer, num_layers, norm=None) [source]

TransformerDecoder is a stack of N decoder layers

Parameters

• decoder_layer – an instance of the TransformerDecoderLayer() class (required).
• num_layers – the number of sub-decoder-layers in the decoder (required).
• norm – the layer normalization component (optional).

Examples::

>>> decoder_layer = nn.TransformerDecoderLayer(d_model=512, nhead=8)
>>> transformer_decoder = nn.TransformerDecoder(decoder_layer, num_layers=6)
>>> memory = torch.rand(10, 32, 512)
>>> tgt = torch.rand(20, 32, 512)
>>> out = transformer_decoder(tgt, memory)

forward(tgt: torch.Tensor, memory: torch.Tensor, tgt_mask: Optional[torch.Tensor] = None, memory_mask: Optional[torch.Tensor] = None, tgt_key_padding_mask: Optional[torch.Tensor] = None, memory_key_padding_mask: Optional[torch.Tensor] = None) → torch.Tensor [source]

Pass the inputs (and mask) through the decoder layer in turn.

Parameters

• tgt – the sequence to the decoder (required).
• memory – the sequence from the last layer of the encoder (required).
• tgt_mask – the mask for the tgt sequence (optional).
• memory_mask – the mask for the memory sequence (optional).
• tgt_key_padding_mask – the mask for the tgt keys per batch (optional).
• memory_key_padding_mask – the mask for the memory keys per batch (optional).

Shape:

see the docs in Transformer class.