Skip to main content
ZICQ

Wiki AI Concepts

In-Context Learning (ICL)

AI Concepts
Aliases: In-Context Learning ICL Few-Shot Learning Prompt-Based Learning ·2026-09-19

In-Context Learning (ICL)

In-context learning (ICL) is one of the most important capabilities of LLMs: putting a few "input-output examples" or "instructions" in the prompt lets the model perform a task without updating weights. It blurs the line between "training" and "inference" — the same model can do countless tasks just by swapping the prompt.

Three typical forms

  • Zero-shot: instruction only, no examples. "Translate the following Chinese to English: …"
  • One-shot: 1 example + task
  • Few-shot: 3-10 examples + task (the classic paradigm from the GPT-3 paper)

Why it matters

  • No fine-tuning: traditional deep learning requires retraining per task; ICL turns "swap prompt" into "swap capability" at near-zero cost
  • Instant: new requirements work immediately; cross-language / cross-task switching is free
  • Composable: multiple examples can demonstrate complex reasoning / tool calls / agent behavior
  • Interpretable: humans see every example in the prompt and know what the model is "thinking"
  • Personalization: per-user / per-scene prompt adjustments don't affect the underlying model

How it works (rough theories)

Three complementary explanations for ICL's success:

  1. Implicit Bayesian inference: the model infers latent concepts from examples, similar to learning the conditional distribution of the task
  2. Task location: the model contains many latent task vectors in its activations; few-shot examples "surface" the right one
  3. Meta-learning: trained on a vast task distribution; inference is "just picking a task in distribution" — this is the central finding of Anthropic / DeepMind transformer-circuits work

Advanced uses

  • Chain-of-Thought (CoT): examples include reasoning steps; model learns to write out intermediate steps
  • Self-consistency: sample multiple times and majority-vote
  • Tool-use prompting: examples include tool calls; model learns to invoke structured tools
  • Retrieval-augmented ICL: dynamically pull examples from an external database
  • Many-shot ICL: Claude 2.1 / Gemini 1.5 use 1000+ token example blocks to lift long-tail tasks (the "many-shot is still zero-shot" finding)

Boundaries and failure modes

  • Sensitive to example order and noise: permuting or corrupting examples can swing results
  • Complex reasoning still needs CoT: pure ICL fails on multi-step math / logic
  • Long-prompt cost: more tokens = more cost, but the benefit isn't linear
  • Can't learn new capabilities: only "surfaces" latent tasks the model already has