INPUT ↔ ↔ ↔ REPRESENTATION
Encoder-only
Looks both left and right. Best at understanding.
QUICK RECOLLECTION REFERENCE · AUGUST 2026
See the whole AI stack.
Remember how it connects.
A compact map from text → tokens → tensors → Transformer layers → output, and from architecture → parameters → file format → framework → inference engine → provider.
Your inputtext · image · audio
MODELarchitecture + parameters
ENGINEexecutes tensor operations
Outputtokens · embeddings · labels
Anchor idea: Architecture is the blueprint. Parameters are what it learned. A checkpoint stores those parameters. An engine executes them on hardware.
01
THE CORE LOOP
What inference actually does
A · PREFILL — read the prompt once
PromptExplain tensors
Token IDs[849, 312, 991…]
Vectorsnumbers + positions
Every decoder layerattention → FFN → residual/norm
KV cache + logitsmemory + scores
First output tokenselected probability
B · REPEATED DECODING — one new token per iteration
Newest tokenonly this is new
Every decoder layerconsult KV cache
Update cacheadd new K + V
Logits → samplingchoose next token
Append tokenstream to user
EOS / stop?NO ↺ repeat
YES ✓ finish
YES ✓ finish
Is inference only steps 1–3?Almost. It is the whole forward-running process: input preparation + prefill + first-token choice + repeated decoding until stopping.
02
ENCODING
Words become model-readable numbers
TextTokenizerIDsEmbeddingsContextual hidden states
Encoding is not encryption or instant “understanding.” It is the transformations turning raw input into increasingly context-aware tensors. In decoder-only LLMs, this occurs during prefill.
03
BLUEPRINTS
Predominant model architectures
PROMPT → → → NEXT TOKEN ↺
Decoder-only
Looks backward. Best at open-ended generation.
ENCODE ⇢ MEANING ⇢ DECODE ↺
Encoder–decoder
Reads a source, then generates a target.
| Architecture / trait | Information flow | Where it shines | Examples | Remember |
|---|---|---|---|---|
| Encoder-only | Whole input, bidirectionally | Understanding, classification, embeddings | BERT, RoBERTa, DeBERTa | Usually not long-form generation |
| Decoder-only | Causal; predicts next token | Chat, writing, code, agents | GPT, Claude, Llama, Qwen, Mistral, DeepSeek | Dominant general-purpose LLM |
| Encoder–decoder | Encoder reads; decoder writes | Translation, summarization, transformations | T5, FLAN-T5, BART | Distinct source and target sequences |
| Mixture of Experts | Router activates select expert FFNs | More capacity per active compute | Mixtral, DeepSeek-V3, Qwen3-MoE, Llama 4 | A Transformer routing design |
| Multimodal | Adds modality encoders or unified tokens | Images, audio, video and text | GPT-4o, Gemini, Claude, Gemma 3n | Capability—not one architecture |
| Diffusion / parallel text | Iteratively denoises positions | Experimental parallel generation | LLaDA and research systems | Not the usual next-token loop |
No universal “best” model: task, data, latency, context, hardware, license and cost determine the best fit. Benchmark on your actual workload.
04
MODEL SPECS
What the important numbers mean
Learned-number count—not layers. More capacity can help, but training and design matter.
Repeated Transformer blocks. Every token passes through all of them.
Maximum working token sequence; longer context increases cache and compute.
Width of each token’s internal vector.
Parallel relevance subspaces; GQA/MQA share KV heads to save memory.
Number precision. Fewer bits shrink memory, sometimes with quality loss.
Dense activates all weights; MoE routes tokens through selected experts.
Decode speed; time-to-first-token is strongly affected by prefill.
Approximate raw weight memoryparameters × bits ÷ 87B × 4-bit ÷ 8 ≈ 3.5 GB, plus metadata, KV cache and runtime overhead.
05
DO NOT MIX THESE UP
The complete model software stack
1Architecture
↓ implemented withTransformer · encoder · decoder · MoE · multimodal
blueprint2Framework
↓ exports / storesPyTorch · JAX · TensorFlow/Keras · MLX
tensor toolkit3Model artifact
↓ loaded or converted forSafeTensors · PyTorch checkpoint · ONNX · GGUF · MLX
stored learning4Inference engine
↓ runs onTransformers · vLLM · llama.cpp · TensorRT-LLM · ONNX Runtime
execution runtime5Hardware / provider
CPU · GPU · Apple silicon · TPU · cloud API
compute + operationsFrameworks: where tensor code is created and executed
| Framework | Best for | Why Python is not the bottleneck | Where it shines |
|---|---|---|---|
| PyTorch | Research, training, fine-tuning, flexible inference | Python orchestrates; compiled C++/CUDA/Triton kernels do heavy math | Default for experimentation and Hugging Face |
| JAX | Accelerator-scale research | XLA compilation + functional transformations | TPUs and highly vectorized research |
| TensorFlow / Keras | Established production ML | Graph/eager execution with compiled kernels | Mature enterprise/deployment ecosystem |
| ONNX + ONNX Runtime | Portable inference and cross-framework deployment | ONNX stores the computation graph; ONNX Runtime executes it with optimized hardware backends | Moving trained models between PyTorch/TensorFlow and server, desktop, browser or edge |
| MLX | Apple-silicon work | Lazy arrays + unified memory | Local Mac fine-tuning and inference |
Why isn’t Python too slow?Python coordinates the work; large matrix operations execute in compiled C++/CUDA/Metal/XLA kernels. The expensive loop runs on the accelerator.
Checkpoint and deployment formats
| Format | Contains | Main purpose | Preferred situation | Important nuance |
|---|---|---|---|---|
| SafeTensors | Weights only | Safe, fast checkpoint | Training, fine-tuning, Transformers inference | No arbitrary pickle execution |
| PyTorch .bin/.pt/.pth | Weights or Python objects | General/legacy serialization | Older checkpoints and research code | Supported—not universally deprecated; pickle can be unsafe |
| GGUF | Quantized weights + metadata | Portable local inference | llama.cpp ecosystem; CPU/GPU local use | Not PyTorch-native training format |
| ONNX | Portable graph + weights | Framework-neutral deployment | ONNX Runtime: server, browser, edge | Operator export support varies |
| TensorRT engine | Compiled optimized graph | Maximum NVIDIA performance | Specific production NVIDIA stack | Fast but less portable |
| MLX | Apple-oriented arrays/weights | Unified-memory execution | Apple-silicon training and inference | macOS-focused ecosystem |
Why PyTorch cannot directly run GGUF: GGUF is arranged for GGML/llama.cpp with its own quantization, naming, layout and metadata. PyTorch expects its own tensors and operators. Same learned values, different native packaging and kernels—convert it or use a GGUF-aware engine.
Popular inference engines
| Engine | Typical input | Hardware | Where it shines |
|---|---|---|---|
| llama.cpp | GGUF | CPU, CUDA, Metal, Vulkan | Local/offline and quantized models |
| vLLM | Usually SafeTensors/Hugging Face | GPU servers | High throughput, continuous batching, PagedAttention |
| TensorRT-LLM | Converted/compiled artifacts | NVIDIA GPUs | Maximum optimized production performance |
| Transformers + PyTorch | SafeTensors / PyTorch | CPU/GPU/accelerators | Flexible reference inference and debugging |
| ONNX Runtime | ONNX | CPU, GPU, NPU, browser/edge | Portable deployment |
| MLX-LM | MLX weights | Apple silicon | Convenient Mac inference/fine-tuning |
| Ollama / LM Studio | Often GGUF | Desktop/local | Friendly model management and chat; wrappers, not formats |
06
CHANGE KNOWLEDGE OR SUPPLY KNOWLEDGE?
Training, LoRA and RAG
Full fine-tuning
Update most or all parameters. Maximum control, compute and storage.
dataset → loss → backprop → updated baseLoRA / QLoRA
Freeze the base and train small low-rank adapters. QLoRA holds a quantized base while training.
base + small adapter → specialized behaviorRAG
Retrieve current/private information at inference. No weight update.
question → retrieve → augmented prompt → answerCurrent/private facts?Use RAG.
Repeatable behavior/style?Use LoRA/fine-tuning.
Need both?Fine-tune behavior; retrieve changing knowledge.
Documents → Chunks → Embedding model → Vector DB → Similarity search → Passages + question → LLM answer
07
HOSTED INFERENCE
What an inference provider provides
You send an API request. It handles model loading, GPUs, the engine, batching, autoscaling, caching, security, observability and billing. Model creator, model host and provider can be different companies.
Your appAPI →PROVIDERengine · GPUs · scaling · metering→ tokensYour app
08
FAMILY TREE
How major model families evolved
Simplified lineage. Arrows show family progression; shared Transformer ancestry does not imply inherited proprietary weights.
20172018–222023202420252026
OpenAITransformer→GPT-1 → 3 / ChatGPT→GPT-4→GPT-4o / o1→GPT-5→GPT-5.4 → 5.6
GoogleTransformer→BERT / T5 / PaLM→Gemini 1→Gemini 1.5 / Gemma→Gemini 2.5→Gemini 3 → 3.5
MetaTransformer→RoBERTa / OPT→LLaMA / Llama 2→Llama 3.x→Llama 4→Muse Spark
AnthropicTransformer→—Claude 1 / 2→Claude 3.x→Claude 4→Claude 5 family
AlibabaTransformer→—Qwen 1 / 1.5→Qwen 2 / 2.5→Qwen 3→Qwen 3 variants
DeepSeekTransformer→—DeepSeek LLM→V2→V3 / R1→V4
MistralTransformer→—7B / Mixtral→Large / Pixtral→Magistral / Mistral 3→Mistral 3 family
Updated from official family announcements through August 2026.
09
FAST LOOKUP
Searchable concept cards
THE 20-SECOND RECALL
Architecture defines the path. Parameters carry learning.
Formats store it. Engines run it. Providers operate it.
Prompt → Tokens → Embeddings → Prefill → First token → Decode ↺ → Answer
HIGH-RESOLUTION MEMORY MAP
Transformer architecture and inference mechanics
