Module 6 — Week 6

Assignment A06: Exploring T5

Assignment Type	Assignment — Technical Analysis
Topic	Text-to-Text Transfer Transformer (T5) model and framework
Date Submitted	8 March 2026

What is T5?

Text-to-Text Transfer Transformer (T5) is a model and framework used to train machine learning systems to perform natural language processing (NLP). In simple terms, it takes language problems and treats them as text going in and text coming out, so the same general setup can be used for tasks like summarizing, classifying, answering questions, and translating (Raffel et al.).

T5 text-to-text unified framework diagram

Why T5 is Important

Scalability: The model can be trained with a wide range of labeled data and computational resources.
Efficient knowledge extraction: Uses pre-training and “denoising” spans of text to improve downstream tasks.
C4 Dataset: Introduced the “Colossal Clean Crawled Corpus” (C4), a dataset consisting of hundreds of gigabytes of cleaned English text, establishing a new standard for large-scale training data.
Limitations: Because it was pre-trained with English-only data, it did not achieve expected state-of-the-art results for translation tasks (Raffel et al.).

T5 vs. GPT-3: Key Differences

Feature	T5	GPT-3
Architecture	Encoder–decoder transformer	Decoder-only
Approach	Text-in, text-out with task prefixes	Prompted without explicit task prefix
Training	Pre-trained then fine-tuned for specific tasks	Broadly usable through prompting without fine-tuning
Attention	Bidirectional — looks at entire input at once	Unidirectional (left-to-right) — ideal for open-ended generation

What I Learned

T5’s unified text-to-text approach is elegant because it standardizes how different NLP tasks are handled. Instead of designing separate architectures for translation, summarization, and classification, T5 treats them all as the same kind of problem with a task prefix as the only distinction.

Comparing T5 to GPT-3 clarified the fundamental differences between encoder-decoder and decoder-only architectures. T5’s bidirectional attention gives it a complete view of the input before generating output, which is ideal for tasks where full context matters. GPT-3’s unidirectional attention is better suited for fluent, open-ended text generation. Neither is universally superior — the right architecture depends on the task.

Citation

Raffel, Colin, et al. “Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer.” Journal of Machine Learning Research, vol. 21, 2020, pp. 1–67, http://jmlr.org/papers/v21/20-074.html.