When you need to switch between OpenAI and Anthropic models in LangChain, it’s not just about changing an API key; it’s about understanding how LangChain abstracts and mediates access to these different LLM providers.
Let’s see this in action. Imagine you have a simple prompt you want to send to an LLM.
from langchain_openai import ChatOpenAI
from langchain_anthropic import ChatAnthropic
from langchain_core.prompts import ChatPromptTemplate
# Define a simple prompt template
prompt = ChatPromptTemplate.from_messages([
("system", "You are a helpful assistant."),
("user", "{input}")
])
# Initialize OpenAI model
openai_llm = ChatOpenAI(model="gpt-4o", temperature=0.7)
# Initialize Anthropic model
anthropic_llm = ChatAnthropic(model="claude-3-opus-20240229", temperature=0.7)
# Create a chain for OpenAI
openai_chain = prompt | openai_llm
# Create a chain for Anthropic
anthropic_chain = prompt | anthropic_llm
# Run the chain with OpenAI
response_openai = openai_chain.invoke({"input": "What is the capital of France?"})
print(f"OpenAI Response: {response_openai.content}")
# Run the chain with Anthropic
response_anthropic = anthropic_chain.invoke({"input": "What is the capital of France?"})
print(f"Anthropic Response: {response_anthropic.content}")
This code directly demonstrates switching by instantiating two different LLM classes, ChatOpenAI and ChatAnthropic, each configured with their respective provider’s models and API access. The core concept is that LangChain provides a unified interface (Runnable) for these disparate LLM APIs. You construct your prompt and then pipe it to the LLM object. The invoke method handles sending the request to the correct backend and parsing the response.
The problem LangChain solves here is abstracting away the vendor-specific API calls, request/response formats, and authentication mechanisms. Instead of writing separate code blocks for OpenAI and Anthropic, you write one block that uses LangChain’s common interface. The ChatOpenAI and ChatAnthropic classes are wrappers that translate the generic LangChain BaseChatModel interface into the specific API calls required by OpenAI and Anthropic, respectively.
Internally, when you invoke openai_llm.invoke(...), LangChain looks at the configuration of that specific LLM instance. It sees it’s an OpenAI model, so it constructs an HTTP request to OpenAI’s API endpoint (e.g., https://api.openai.com/v1/chat/completions), populates it with your prompt, any specified parameters like temperature, and includes the necessary API key from your environment variables or configuration. The same process happens for Anthropic, but it targets Anthropic’s API endpoints (e.g., https://api.anthropic.com/v1/messages) with Anthropic’s specific payload structure.
The key levers you control are the model parameter (e.g., "gpt-4o", "claude-3-opus-20240229"), temperature for creativity, max_tokens for output length, and any provider-specific parameters that LangChain exposes through its LLM wrappers. You can also configure the API key and base URL if you’re not using environment variables.
A crucial, often overlooked, detail is how streaming is handled. Both OpenAI and Anthropic support streaming responses. LangChain’s LLM wrappers abstract this too. When you use the .stream() method on a LangChain Runnable that ends in an LLM, LangChain will internally manage the connection for streaming. For OpenAI, this involves a stream=True parameter in the API call and processing Server-Sent Events (SSE). For Anthropic, it’s a similar SSE mechanism. The LLM wrappers ensure that the output from these different streaming protocols is presented as a unified AsyncIterator or Iterator of AIMessageChunk objects, allowing you to write code that can handle streaming from any supported LLM provider without modification.
The next step you’ll likely encounter is dynamically choosing which LLM to use based on runtime conditions or user input.