Few-shot prompting

TL;DR

• Few-shot prompting teaches the model a task by showing 3-10 input/output examples inside the context window, with no training required.
• Example quality matters more than example count. Five diverse, edge-case-covering examples beat twenty examples of the same type.
• Positional effects can matter: test whether putting a relevant example near the query helps on the target model. Published studies have reported sizable accuracy changes from example ordering, but the size and direction vary by model and task.
• Chain-of-thought-style few-shot examples can help some reasoning tasks, but there is no universal parameter threshold. Compare concise observable work summaries or structured checks with direct answers on the target model.
• Few-shot is instant and free to set up. Fine-tuning is permanent but costs data, GPU time, and a deployment pipeline. Know when to use each.
• In production, dynamic few-shot retrieval (embedding an example bank and pulling close matches per query) can outperform static example lists on diverse task distributions, but the gain depends on retrieval quality, example coverage, and task.

30-Second Explanation

Mental model: few-shot prompting is temporary task conditioning through examples in the context window, not a weight update. Select examples for coverage and similarity, fit them within the budget, and evaluate ordering and retrieval because the model may imitate both the desired pattern and mistakes.

The problem it solves

Suppose you ask a model such as GPT-4 to classify customer support tickets into one of six categories: billing, shipping, product_defect, account_access, feature_request, and spam. In an illustrative zero-shot evaluation, the model gets about 70% accuracy. It may invent new categories ("general_inquiry"), merge categories you want separate ("billing" and "account_access"), and format outputs inconsistently.

You could fine-tune. That requires collecting 2,000+ labeled tickets, formatting them into training data, running a fine-tuning job for a few hours, and deploying a new model endpoint. Three days of work before you see results.

Or you paste five correctly labeled tickets directly into the prompt. One billing example, one shipping example, and one ambiguous ticket that could be billing or account_access (labeled "billing" with a concise explanation). The examples can clarify the format, category boundaries, and edge-case policy. In this illustrative scenario, accuracy rises to 92%; an actual result needs a held-out evaluation. The prompt may take minutes to assemble, but selecting and validating examples is ongoing work.

Few-shot prompting is a fast way to condition a general-purpose model for a specific task. It is a normal use of how language models process context, but it does not guarantee a reliable specialization.

What is it?

Few-shot prompting means including a small number of labeled input/output examples in the prompt before the actual query. The model sees the pattern and applies it to the new input. No weights change. No training happens. The model is pattern-matching against the structure you showed it.

Think of it like training a new hire by showing them completed examples. You don't rewrite their brain (fine-tuning). You show them three finished reports and say "do it like this." They pick up the format, the tone, and the edge-case handling from your examples. If your examples are sloppy, their work will be sloppy too.

There are three variants at the highest level:

• Zero-shot: instruction only, no examples. "Classify this ticket into one of these categories."
• One-shot: a single example to establish format. Enough for simple formatting tasks.
• Few-shot: typically 3-10 examples, enough to convey task nuance, category boundaries, and edge-case handling. This is the most broadly useful variant.

The number of examples is not the only important variable. Quality, diversity, ordering, and task coverage all affect performance. Three well-chosen examples can outperform 15 randomly selected ones on some benchmarks, so compare selection strategies on the target distribution.

How it works

In-context learning: pattern matching, not weight updates

The model does not "learn" from your examples in the gradient-descent sense. Its weights do not change. What happens is closer to pattern completion. The model's pretraining exposed it to billions of structured input/output pairs across web text, code, and documentation. When you provide examples in context, you activate the most relevant patterns from that pretraining and tell the model which ones apply right now.

This is called in-context learning (ICL), and it was first characterized in the GPT-3 paper (Brown et al., 2020). The paper showed that scaling model size dramatically improved few-shot performance, suggesting that larger models store more patterns and activate them more precisely.

The practical implication: your examples do not teach new knowledge. They select which existing knowledge to apply.

Example selection strategies

Taking the first N examples from a dataset is a tempting baseline, but those examples may cluster around common cases and leave edge cases uncovered.

Diversity over quantity. If you are classifying into six categories, include at least one example per category. If you only have room for five examples, cover five of the six categories and add a note in the system prompt about the sixth. Five diverse examples beat ten examples from two categories.

Include at least one edge case. Every task has ambiguous inputs. Show the model how you want those resolved. An example labeled "This could be billing or account_access, but we classify it as billing because the user mentions charges" teaches more than three straightforward examples.

Match the input distribution. If 40% of your real inputs are long-form paragraphs and 60% are short one-liners, your examples should reflect that ratio. Models can be sensitive to format mismatch between examples and actual queries.

Dynamic few-shot retrieval goes one step further. Embed your example bank into a vector store, and at query time retrieve the K most semantically similar examples to the current input. This can help when a task has many subtypes, provided the embedding and retrieval method surface useful examples. The model is intended to see examples relevant to the specific query, but retrieval can be wrong or incomplete.

In a design discussion, dynamic few-shot is worth mentioning when example coverage varies by query. Explain the retrieval, latency, token, privacy, and evaluation trade-offs rather than treating it as a default.

Example ordering and positional bias

Order matters more than most people realize. Models have positional recency bias: they weight content toward the end of the context window more heavily than content at the beginning. This is well-documented in the "Lost in the Middle" paper by Liu et al. (2023), which showed that both GPT-4 and Claude retrieve information best from the beginning and end of context, with a significant accuracy drop for information buried in the middle.

The practical hypothesis is to place a relevant example near the actual query, then verify the effect. The model may use nearby context differently depending on architecture and prompt layout, so do not assume that “last” is universally best.

For chain-of-thought reasoning tasks, order examples from simplest to most complex. The model ramps up through the reasoning format gradually. Putting a complex example first can confuse the pattern.

Token budget math

Few-shot examples consume context tokens. This is a real engineering cost at scale.

A typical example (input + output) runs 100-300 tokens. Five examples at 200 tokens each means 1,000 tokens per call just for examples. At GPT-4o pricing ($2.50 per million input tokens), that is $2.50 per million calls in example overhead alone. For a system handling 10 million calls per month, that is $25/month just for in-context examples.

The math gets worse with chain-of-thought examples because the reasoning steps add 100-400 tokens per example. Five CoT examples can easily consume 2,000-3,000 tokens.

Budget your examples deliberately:

Chain-of-thought + few-shot

For math, logic, and multi-step reasoning, few-shot examples with concise intermediate work can be useful on some models and tasks. Compare them with direct-answer examples and structured checks, because generated reasoning-like text is not guaranteed to be faithful.

Instead of showing only input and output:

Input: A store has 15 red and 27 blue shirts. How many total?
Output: 42

You show the reasoning trace:

Input: A store has 15 red and 27 blue shirts. How many total?
Reasoning: I need to add 15 and 27. 15 + 20 = 35, then 35 + 7 = 42.
Output: 42

Wei et al. (2022) showed that chain-of-thought few-shot prompting improved accuracy on GSM8K (grade school math) from 17.7% to 57.1% for PaLM 540B. On commonsense reasoning benchmarks, improvements ranged from 10-25 percentage points. The model is not just copying format. It is copying the reasoning pattern and applying it to novel problems.

Few-shot prompt assembly pipeline

In production, you don't manually paste examples. Your code assembles the prompt automatically: loading the system instruction, selecting or retrieving examples, ordering them by relevance, and inserting the user query at the end.

Key variants and types

The term "few-shot" covers a spectrum of techniques. Here is how they compare:

A practical sequence is to start with static few-shot. If accuracy on diverse inputs is below the threshold, test dynamic retrieval. If reasoning errors dominate, compare concise intermediate summaries or structured checks. Layer techniques based on measured performance, not assumptions.

When to use / when to avoid

When to use few-shot

• When you need to establish a specific output format the model keeps getting wrong (JSON schema, category labels, structured tables).
• When you have fewer than 500 labeled examples. Few-shot avoids a training pipeline and can be a useful baseline, but its quality relative to fine-tuning depends on the task and model.
• When your task definition is still changing. Few-shot examples can be swapped in seconds. Fine-tuned models take hours to retrain.
• When you need to teach edge-case handling explicitly. Showing the model how to resolve ambiguous cases is more reliable than describing the rules in prose.
• When prototyping a new feature. Often start with few-shot to validate the task. Consider fine-tuning after the task is stable and you have enough representative labeled data.

When to avoid few-shot

• When the task is already well within the model's default behavior. Adding examples for "summarize this text" to GPT-4o wastes context tokens. The model already knows how to summarize.
• When you have 5,000+ labeled examples and a stable task. Fine-tuning a smaller model (where the provider supports it) may be cheaper per call and faster at inference, but compare training, hosting, and maintenance costs.
• When latency is critical and every token counts. Extra input tokens add latency and cost; the amount depends on the model and serving stack. For real-time applications, measure the effect against the latency budget.
• When the base model has no pretraining signal for the task. Few-shot activates existing patterns. If the pattern does not exist (a completely proprietary format the model has never seen), fine-tuning or a different approach is needed.

The decision framework

The bottom line: few-shot is a low-friction baseline, not a universal first or last step. Many production tasks can remain in the few-shot regime, while stable high-volume tasks may justify fine-tuning or another approach.

Real-world examples

Fraud classification with dynamic few-shot

An illustrative fraud-detection pipeline can classify edge-case transactions that rule-based systems flag as uncertain. It might embed a bank of 10,000+ labeled fraud examples and retrieve the 5 most similar to each flagged transaction at query time. Compared with static few-shot, a reported 18% reduction in false positives would be a workload-specific result that needs a controlled comparison. Each classification call might use about 1,200 tokens of examples; the dollar cost depends on current pricing.

Provider documentation example (2024)

Anthropic documentation has recommended 3-5 diverse examples as a baseline for some format-consistency tasks. Results from any provider's internal testing are model- and benchmark-specific; the broader lesson is to compare example count, placement, and edge-case coverage on your own data.

Notion AI: template generation

A writing assistant can use few-shot prompting to generate content matching specific templates (meeting notes, PRDs, sprint retros). Each template type might have 3 curated examples stored in a prompt management system. When a user selects "meeting notes," the system injects those examples before the user's raw notes. Swapping examples lets a team iterate without model retraining, although template coverage and quality still need evaluation.

OpenAI Cookbook benchmarks (2023)

OpenAI's published classification experiments reported 5-shot prompting on GPT-4 at 94.2% accuracy versus 93.8% for a fine-tuned GPT-3.5 setup on that sentiment task. The few-shot approach avoided training time and GPU cost for that comparison and could be updated by changing examples. The takeaway is narrower: compare a capable base model plus examples with fine-tuning on the target task and cost profile.

Limitations and tradeoffs

The fundamental tension: few-shot gives you flexibility at the cost of per-call token overhead. Fine-tuning can reduce per-call context cost at the cost of flexibility, training data, and setup time. Choose between them using task quality, volume, latency, and maintenance measurements.

How this shows up in interviews

When to bring it up

Few-shot prompting comes up in AI system design and prompt-engineering discussions where you need to explain how an LLM should perform a specific task beyond "just give it instructions."

Bring it up proactively whenever the design involves classification, extraction, formatting, or any task where output consistency matters. Say: "I would start with few-shot prompting to validate the task, then evaluate whether fine-tuning is needed based on accuracy and cost at production volume."

Depth by level

• Junior: Knows that few-shot means putting examples in the prompt. Can explain zero-shot vs. few-shot.
• Senior: Understands example selection (diversity, edge cases), ordering (recency bias), and the tradeoff with fine-tuning. Can articulate when to use each.
• Staff: Designs dynamic few-shot retrieval pipelines, reasons about token cost at scale, knows when CoT helps vs. hurts, and integrates few-shot into broader evaluation and deployment systems.

Q&A table

Common interview mistakes

Test your understanding

Quick recap

1. Few-shot prompting places a small set of labeled examples in the context window to activate in-context learning. No weights change, so a training pipeline is not required for the prompt itself.
2. Example quality beats quantity. Five diverse examples covering edge cases outperform twenty clustered examples from common cases.
3. Positional effects can matter, so compare example placement and ordering on the target model and task rather than assuming the most relevant example always goes last.
4. Chain-of-thought-style few-shot can help some reasoning tasks, but generated intermediate work may be wrong or unfaithful; use concise observable summaries and structured checks where possible.
5. Dynamic few-shot can outperform static few-shot on some diverse task distributions, but retrieval quality, token cost, and latency determine whether it is worthwhile.
6. Use few-shot as a low-friction baseline. Consider fine-tuning when the task is stable and measured quality, volume, latency, and maintenance costs justify it.
7. Treat examples as code: version-control them, run evaluations after changes, and audit for distribution mismatch.

Related concepts

• Context engineering - Few-shot examples are one zone in the context window. Context engineering is the broader discipline of assembling optimal prompts.
• Fine-tuning - The permanent alternative to few-shot. Use when task is stable and volume justifies training infrastructure.
• LLM evaluations - You cannot know if your few-shot examples are working without systematic evaluation. Evaluation suites are the testing framework for prompt engineering.
• Large language models - Understanding transformer architecture and pretraining explains why in-context learning works mechanically.