AICurious Logo

What is: Transformer Decoder?

SourceGenerating Wikipedia by Summarizing Long Sequences
Year2000
Data SourceCC BY-SA - https://paperswithcode.com

Transformer-Decoder is a modification to Transformer-Encoder-Decoder for long sequences that drops the encoder module, combines the input and output sequences into a single ”sentence” and is trained as a standard language model. It is used in GPT and later revisions.