The most surprising true thing about LLM system prompts is that they often act less like direct instructions and more like subtle nudges that shape the LLM’s entire worldview for a given interaction.
Let’s see how this plays out. Imagine we want an LLM to act as a helpful, concise summarizer of technical articles.
import openai
openai.api_key = "YOUR_API_KEY"
def summarize_article(article_text):
response = openai.ChatCompletion.create(
model="gpt-3.5-turbo",
messages=[
{"role": "system", "content": "You are a helpful assistant that summarizes technical articles concisely."},
{"role": "user", "content": f"Summarize this article: {article_text}"}
]
)
return response.choices[0].message.content
article = """
The recent advancements in quantum computing have opened up new avenues for solving complex problems that are intractable for classical computers.
Specifically, algorithms like Shor's algorithm for factoring large numbers and Grover's algorithm for searching unsorted databases demonstrate the potential for exponential speedups.
However, current quantum computers are still in their nascent stages, facing challenges such as qubit decoherence, error correction, and scalability.
Researchers are exploring various hardware implementations, including superconducting circuits, trapped ions, and photonic systems, each with its own set of advantages and disadvantages.
The development of fault-tolerant quantum computers remains a significant long-term goal, requiring breakthroughs in error mitigation and correction techniques.
Despite these hurdles, the theoretical promise of quantum computing continues to drive innovation and investment in the field.
"""
summary = summarize_article(article)
print(summary)
Output:
This article discusses the progress and challenges in quantum computing. It highlights algorithms like Shor's and Grover's for their potential speedups but notes current limitations such as qubit decoherence, error correction, and scalability. Various hardware implementations are being explored, with the ultimate goal of achieving fault-tolerant quantum computers through advanced error correction. Despite obstacles, the field sees continued innovation.
This is a decent start, but what if we want it to be even more concise, or perhaps focus on the implications? We can tweak the system prompt.
Consider this variation:
def summarize_article_implications(article_text):
response = openai.ChatCompletion.create(
model="gpt-3.5-turbo",
messages=[
{"role": "system", "content": "You are an expert AI that distills the core implications of technical articles into a single, impactful sentence. Focus on what this means for the future."},
{"role": "user", "content": f"Summarize this article: {article_text}"}
]
)
return response.choices[0].message.content
summary_implications = summarize_article_implications(article)
print(summary_implications)
Output:
Quantum computing's theoretical promise, despite current hardware limitations and the quest for fault tolerance, is poised to revolutionize fields by enabling solutions to previously intractable problems.
See how the "expert AI," "core implications," and "single, impactful sentence" directives shifted the output significantly? The system prompt isn’t just telling the LLM what to do, but how to think about the task, setting its persona and priorities.
The mental model for system prompts revolves around establishing a context window and persona for the LLM. The system message is the first piece of information the LLM receives for a given conversation, acting as a foundational instruction. It primes the model for subsequent user messages. Think of it as setting the stage before the play begins. You’re not just giving a command; you’re defining the character, their motivations, and the general tone of the performance.
The key levers you control with system prompts are:
- Persona: "You are a helpful assistant," "You are a sarcastic critic," "You are a Shakespearean actor." This dictates the tone, vocabulary, and style.
- Task Definition: "Summarize this text," "Translate this to French," "Write a poem about…" This clearly states the objective.
- Constraints & Formatting: "Respond in under 50 words," "Use bullet points," "Never mention the word 'banana'," "Output in JSON format." These guide the structure and content of the output.
- Knowledge Priming: "You are an expert in astrophysics," "Focus on the historical context." This can subtly steer the model toward recalling and prioritizing certain types of information.
The most powerful aspect of system prompts is their ability to imbue the LLM with implicit knowledge or a specific interpretive lens without explicitly stating every single fact. For instance, telling an LLM "You are a seasoned diplomat negotiating a peace treaty" implicitly instructs it to be cautious, diplomatic, find common ground, and avoid inflammatory language, even if you don’t explicitly say "avoid inflammatory language." The LLM draws upon its training data related to "seasoned diplomats" and "peace treaties" to infer these behaviors.
If you’re struggling with an LLM that’s too verbose, try adding a constraint like "Your responses must be no more than two sentences long." If it’s consistently missing a crucial piece of information, explicitly state in the system prompt, "You must always include the current market price and its 24-hour change in your stock analysis."
The next concept you’ll likely grapple with is managing long-term memory and context windows within conversations, where the initial system prompt’s influence can become diluted by many subsequent turns.