Module 4 Lesson 1: Transformer Architecture Overview
The engine under the hood. A non-math guide to the Transformer architecture that powers all modern LLMs.
Transformer Architecture: The Brain of the LLM
Every model you run in Ollama—Llama, Mistral, Gemma—shares the same fundamental ancestry: the Transformer. Published by Google researchers in 2017 in the famous paper "Attention Is All You Need," the Transformer structure is what allowed AI to finally understand human language context.
The Problem Before Transformers
Before 2017, AI processed language like a human reading through a straw: one word at a time. By the time it got to the end of a long sentence, it often "forgot" how the sentence started. This made long-form writing and complex logic impossible.
2. The Solution: "Attention"
The magic of the Transformer is a mechanism called Self-Attention.
Imagine you are reading this sentence: "The animal didn't cross the street because it was too tired." How do you know what "it" refers to?
- Is it the animal?
- Is it the street?
A Transformer uses Self-Attention to look at every word in the sentence simultaneously and calculate which words are "related." The model "pays attention" to the word "animal" when it processes the word "it."
3. The Layers of the Brain
An LLM is composed of many layers of these Transformers stacked on top of each other.
- Bottom Layers: Understand grammar, spelling, and basic syntax.
- Middle Layers: Understand themes, sentiment, and specialized knowledge (like coding).
- Top Layers: Make the final decision on which word comes next.
When you see a llama3:8b model, it means there are 8 billion parameters (numbers) spread across these layers, acting as the "weights" or "memories" of the network.
4. Why This Matters for Local AI
Because Transformers are composed of massive "Matrix Multiplications" (essentially huge grids of numbers being multiplied), they are incredibly hungry for memory bandwidth.
This is why we spent so much time in Module 1 talking about GPUs. A GPU is a specialized tool designed specifically for these "Matrix" operations. Without a GPU, your CPU has to do these billions of multiplications one-by-one, which is why non-GPU inference is so slow.
The "Token" Prediction
A Transformer never "thinks" a whole sentence. It is a Next-Token Predictor.
- It looks at the words you typed.
- It calculates the probability of every word in its vocabulary being the next word.
- It picks one.
- It adds that word to the sentence and starts the process again.
This "Loop" is why you see the text streaming out word-by-word in Ollama.
Key Takeaways
- The Transformer is the blueprint for all modern LLMs.
- Self-Attention allows the model to understand the relationship between far-apart words.
- The architecture is optimized for parallel math, which is why GPUs/Unified Memory are required for performance.
- Models "think" by constantly predicting the single next word in a sequence.