Few-shot prompting is often seen as just a fancier version of zero-shot, but it’s actually a fundamentally different strategy for guiding LLMs that hinges on demonstrating desired output formats rather than just describing them.
Let’s see this in action. Imagine we want an LLM to extract the company name and the product from a marketing blurb.
First, zero-shot. We just tell it what we want:
Extract the company name and product from the following text:
"Acme Corp is proud to announce the release of our revolutionary new widget, the WidgetMaster 3000."
The LLM might respond with:
Company: Acme Corp
Product: WidgetMaster 3000
This works, but what if the input format changes, or we want a specific JSON output? It can get messy.
Now, few-shot. We give it a couple of examples of the input and the exact output format we expect:
Text: "Globex Corporation's latest innovation, the SuperGizmo, is set to redefine the industry."
Company: Globex Corporation
Product: SuperGizmo
Text: "Cyberdyne Systems is launching the T-800, a groundbreaking personal assistant."
Company: Cyberdyne Systems
Product: T-800
Text: "Acme Corp is proud to announce the release of our revolutionary new widget, the WidgetMaster 3000."
Company:
The LLM will then complete it like this, perfectly matching the format:
Company: Acme Corp
Product: WidgetMaster 3000
This few-shot approach, by providing concrete examples, implicitly teaches the model the structure and style of the desired output. It’s like showing a chef a finished dish versus just describing it. The chef can replicate the finished dish much more reliably.
The core problem few-shot prompting solves is ambiguity. LLMs are incredibly good at pattern matching. When you provide a few examples, you’re giving them a very strong, unambiguous pattern to follow for the specific task. Zero-shot relies on the LLM’s general understanding of language and instructions, which can be powerful but also prone to misinterpretation, especially for nuanced formatting or complex extraction rules.
Internally, when an LLM processes a few-shot prompt, it’s not just reading the instructions. It’s conditioning its internal state based on the provided examples. The weights and attention mechanisms adjust dynamically to prioritize generating output that mirrors the patterns seen in the examples. It’s learning a micro-task on the fly. The "context window" acts as a temporary, highly specific fine-tuning space.
The exact levers you control are the quality and quantity of your examples.
- Number of shots: Too few, and the pattern might not be clear. Too many, and you might hit context window limits or dilute the core instruction. Typically, 2-5 examples are a good starting point.
- Example quality: Each example must be accurate and representative of the desired input-output relationship. Mismatched examples will confuse the model.
- Consistency: The format of the examples must be perfectly consistent. If one example uses "Company:" and another uses "Company Name:", the model will struggle.
- Order: While less critical than consistency, presenting examples in a logical flow can sometimes help.
A common mistake is to think that if the LLM can do something in zero-shot, few-shot won’t add much. But for tasks requiring specific output structures, like JSON, XML, or even just consistently formatted key-value pairs, few-shot excels. It’s not about what the LLM can do, but how it should present its answer. The prompt becomes a template.
The most surprising thing is how sensitive LLMs are to the exact phrasing of the completion in the few-shot examples. If your examples end with a newline character before the final desired output, the model will likely include that newline. The model doesn’t just learn the mapping; it learns the complete sequence, including trailing whitespace.
Once you’ve mastered few-shot prompting for structured output, you’ll likely start thinking about how to make these prompts more dynamic, perhaps by retrieving relevant examples from a database based on the input query.