T5: Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Reframed many NLP tasks as text-to-text generation under one unified training format.
T5 turns many different NLP tasks into one format: text goes in, text comes out.
It made task unification a major training and product design idea.
T5 makes many NLP tasks share one interface: tell the model the task, give text, get text back.
The quick digest
Before T5, NLP tasks often had custom formats: classification labels here, spans there, translations somewhere else. T5 says: make everything text-to-text. If the task is sentiment, output text. If it is translation, output text. If it is summarization, output text.
That unification simplifies transfer learning. The same model and training interface can cover many tasks, with task prefixes telling the model what kind of transformation to perform.
The modern echo is obvious: prompts are text instructions that turn different workflows into a shared interface. T5 is one of the bridges from classic NLP task engineering to instruction-style model interaction.
What to remember
Read it like this
- First pass: Focus on the task formatting examples.
- Second pass: Then inspect the C4 data and transfer experiments.
- Then build taste: Connect T5 to instruction tuning and prompt templates.
Turn three different tasks into text-to-text examples and train or prompt one model through the same interface.