LangChain’s Guardrails are a clever way to ensure LLM outputs are actually useful, not just random text.
Let’s see it in action. Imagine we’re building a simple chatbot that only answers questions about cats. We’ll use a basic prompt and then wrap it with a guardrail to make sure the LLM sticks to the topic.
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser
from langchain_openai import ChatOpenAI
from langchain_core.runnables import RunnablePassthrough
from langchain_core.runnables.base import Runnable
# --- The LLM Chain ---
prompt = ChatPromptTemplate.from_messages([
("system", "You are a helpful assistant that ONLY answers questions about cats. If the question is not about cats, say 'I can only answer questions about cats.'"),
("user", "{question}")
])
llm = ChatOpenAI(model="gpt-3.5-turbo", temperature=0)
output_parser = StrOutputParser()
chain = prompt | llm | output_parser
# --- The Guardrail ---
# This guardrail checks if the output contains the forbidden phrase
def is_not_about_cats(text: str) -> bool:
forbidden_phrases = ["I can only answer questions about cats."]
for phrase in forbidden_phrases:
if phrase in text:
return True
return False
# We'll wrap our existing chain with a guardrail that re-runs the chain
# if the output is deemed "bad".
# For this simple example, "bad" means it contains the forbidden phrase.
# In a real-world scenario, you'd have a more sophisticated check.
def guardrail(output: str) -> Runnable:
if is_not_about_cats(output):
# If the output is bad, we re-run the original chain
# In a more complex scenario, you might have a fallback chain
# or prompt here.
print("Output was not about cats, re-running...")
return RunnablePassthrough() # This will re-trigger the previous step in the chain
else:
# If the output is good, we just return it
return RunnablePassthrough.assign(output=output)
# --- Putting it all together ---
# We'll use RunnablePassthrough to inject the output of the LLM chain
# into the guardrail's check.
full_chain = (
{"question": RunnablePassthrough()}
| prompt
| llm
| RunnablePassthrough.assign(output=output_parser) # Assign output to a key named 'output'
| guardrail # Pass the output to the guardrail
)
# --- Test Cases ---
print("--- Test Case 1: Question about cats ---")
result1 = full_chain.invoke("What is the average lifespan of a domestic cat?")
print(f"Final Output: {result1['output']}\n")
print("--- Test Case 2: Question NOT about cats ---")
result2 = full_chain.invoke("What is the capital of France?")
print(f"Final Output: {result2['output']}\n")
print("--- Test Case 3: Another question NOT about cats ---")
result3 = full_chain.invoke("Tell me about the weather today.")
print(f"Final Output: {result3['output']}\n")
The surprising thing about LangChain Guardrails is that they don’t just validate output, they actively enforce it by re-running the LLM or invoking fallback logic until the output meets your criteria. It’s less about a post-hoc check and more about a continuous refinement loop.
When you run the code above, you’ll see the "Output was not about cats, re-running…" message appear for the second and third test cases. This is the guardrail kicking in. The guardrail function checks the output of the LLM. If it detects that the LLM has strayed from the topic (by checking for the specific phrase "I can only answer questions about cats."), it signals the RunnablePassthrough() to essentially go back and re-execute the preceding steps in the chain. The LLM then gets another chance to answer, guided by the same prompt. This continues until a satisfactory (cat-related) answer is produced or a maximum retry count is reached (which we haven’t configured here, but is a common feature in more robust guardrail implementations).
The mental model is a feedback loop:
- Prompting: You give the LLM a task and instructions.
- Generation: The LLM produces an output.
- Validation: A separate mechanism (your guardrail logic) inspects this output.
- Correction: If the output fails validation, the process repeats from step 1 (or a modified step 1, like a different prompt or a fallback LLM call).
In our example, the guardrail function is the validator. It receives the LLM’s raw output. The is_not_about_cats function is the specific validation rule. If is_not_about_cats returns True, the guardrail function returns RunnablePassthrough(). In LangChain Expression Language (LCEL), when a step returns a Runnable, the chain will execute that runnable. RunnablePassthrough() effectively means "pass through the current context and re-run the previous steps." This creates the retry loop. The RunnablePassthrough.assign(output=output_parser) is crucial because it ensures the LLM’s generated string is available under a specific key (output) for the guardrail function to inspect.
The actual "guardrails" can be much more sophisticated than a simple string check. You can use other LLMs to evaluate the output for safety, factual accuracy, tone, or adherence to a specific format. You can define rules based on regular expressions, Pydantic models, or even custom Python functions. The key is that the guardrail mechanism orchestrates the retry or fallback behavior based on these validation rules.
A common pattern for implementing more complex guardrails involves defining a Runnable that always returns a dictionary. When the guardrail detects an invalid output, instead of returning RunnablePassthrough(), it might return a different runnable, perhaps a fallback LLM call with a prompt like "The previous answer was rejected for [reason]. Please try again, ensuring you adhere to the following: [original instructions]." This fallback runnable would then produce its own output, which would again be fed back into the guardrail.
The next step after implementing basic output validation is handling structured output and more complex validation schemas.