OpenAI’s function calling feature in LangChain isn’t just about letting an LLM pick a tool; it’s a sophisticated negotiation where the LLM proposes a plan and the system validates it before execution.
Let’s see it in action with a simple calculator tool.
import os
from langchain_openai import ChatOpenAI
from langchain.agents import AgentExecutor, create_react_agent
from langchain_core.prompts import PromptTemplate
from langchain.tools import tool
@tool
def add(a: int, b: int) -> int:
"""Adds two integers together."""
return a + b
@tool
def subtract(a: int, b: int) -> int:
"""Subtracts the second integer from the first."""
return a - b
llm = ChatOpenAI(model="gpt-4o-mini", temperature=0)
tools = [add, subtract]
# This prompt is key: it tells the LLM *how* to use the tools
# by providing the tool definitions and instructing it to output
# a JSON object matching the OpenAI function call schema.
prompt = PromptTemplate.from_template("""
You are a helpful assistant. Use the available tools to answer the user's question.
You are given a list of tools in the following format:
name: {tool_name}
description: {tool_description}
parameters: {tool_parameters}
Respond to the user in the following format:
{format_instructions}
User: {input}
Assistant:""")
# The create_react_agent function is where the magic happens.
# It understands how to parse the LLM's output and decide
# whether to call a tool or respond to the user.
agent = create_react_agent(llm, tools, prompt)
agent_executor = AgentExecutor(agent=agent, tools=tools, verbose=True)
# Example 1: Simple addition
result_add = agent_executor.invoke({"input": "What is 5 + 3?"})
print(f"Result of addition: {result_add['output']}")
# Example 2: Chained calls (LLM decides to use subtract after add)
# For this to work, the prompt needs to be more sophisticated to handle intermediate steps.
# The default ReAct prompt handles this implicitly by expecting a sequence of thoughts/actions.
result_chain = agent_executor.invoke({"input": "What is 10 - (5 + 3)?"})
print(f"Result of chained operations: {result_chain['output']}")
When you run this, you’ll see the LLM outputting JSON that describes a function call, not the result of the function call itself. The AgentExecutor then parses this JSON, calls the actual Python function (add or subtract), and feeds the result back to the LLM for a final answer.
The core problem LangChain’s OpenAI function integration solves is bridging the gap between the LLM’s natural language understanding and deterministic, structured tool execution. LLMs are probabilistic and can "hallucinate" tool outputs. Function calling provides a strict schema: the LLM doesn’t run the add function; it generates a JSON object that says, "To answer this, I need to call add with a=5 and b=3." The AgentExecutor then takes that JSON, finds the add function in your tools list, executes it with those arguments, and returns the actual result (8, in this case) to the LLM. This ensures reliable, verifiable execution.
The magic is in the create_react_agent and AgentExecutor. The agent is trained to output a specific JSON structure that maps to OpenAI’s function calling API. This JSON includes tool_calls (an array of tool invocations) or a final answer. The AgentExecutor’s job is to:
- Send the prompt and conversation history to the LLM.
- Parse the LLM’s output.
- If it’s a function call, extract the tool name and arguments.
- Look up the corresponding Python function in the
toolslist. - Execute the Python function with the extracted arguments.
- Append the function’s return value to the conversation history.
- Repeat until the LLM produces a final answer.
The PromptTemplate is crucial because it explicitly instructs the LLM on how to format its output, including the format_instructions which are dynamically generated by LangChain to match the expected OpenAI function call schema. This makes the LLM’s output predictable and parsable.
A common point of confusion is how the LLM knows which tool to use and with what arguments. This comes directly from the prompt. The prompt includes the names, descriptions, and parameter schemas of all available tools. The LLM uses its understanding of the user’s input and the tool descriptions to "select" the most appropriate tool and "fill in" its parameters. The description field of your @tool decorated functions is incredibly important here; it’s what the LLM reads to understand what a tool does.
What most people miss is that the LLM isn’t truly "calling" the function; it’s generating structured data that represents a function call. The AgentExecutor is the actual component that translates this data into a real function invocation. If the LLM generates a JSON for a tool that doesn’t exist in the tools list, or if the arguments don’t match the schema, the AgentExecutor will likely error out or ask the LLM to retry.
The next step is exploring how to handle more complex tool signatures, custom output parsers, and strategies for when the LLM needs to make multiple tool calls in sequence.