LangChain’s Graph RAG allows LLMs to query knowledge graphs, but the real magic is how it turns unstructured text into structured graph data on the fly for precise, contextual answers.
Let’s see it in action. Imagine we have a collection of documents about different companies and their funding rounds.
from langchain_community.document_loaders import PyPDFLoader
from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain_experimental.graph_rag.schema import GraphRAGRetriever
from langchain_experimental.graph_rag.graphs.neo4j_graph import Neo4jGraph
from langchain_community.vectorstores import Chroma
from langchain_community.embeddings import OpenAIEmbeddings
from langchain.chains import LLMChain
from langchain.prompts import PromptTemplate
from langchain_openai import ChatOpenAI
import os
# Assuming you have API keys set as environment variables
# os.environ["OPENAI_API_KEY"] = "YOUR_OPENAI_API_KEY"
# os.environ["NEO4J_URI"] = "bolt://localhost:7687"
# os.environ["NEO4J_USER"] = "neo4j"
# os.environ["NEO4J_PASSWORD"] = "your_neo4j_password"
# Load documents
loader = PyPDFLoader("company_funding.pdf") # Replace with your PDF file
documents = loader.load()
# Split documents into chunks
text_splitter = RecursiveCharacterTextSplitter(chunk_size=1000, chunk_overlap=100)
chunks = text_splitter.split_documents(documents)
# Initialize embeddings and vector store
embeddings = OpenAIEmbeddings()
vectorstore = Chroma.from_documents(chunks, embeddings)
# Initialize Neo4j graph
graph = Neo4jGraph(
url=os.environ["NEO4J_URI"],
username=os.environ["NEO4J_USER"],
password=os.environ["NEO4J_PASSWORD"]
)
# Initialize Graph RAG Retriever
retriever = GraphRAGRetriever(
graph=graph,
vectorstore=vectorstore,
llm=ChatOpenAI(model="gpt-4o-mini", temperature=0),
verbose=True,
)
# Initialize LLM and prompt for question answering
llm = ChatOpenAI(model="gpt-4o-mini", temperature=0)
prompt = PromptTemplate(
template="""
Use the following pieces of context to answer the question at the end.
If you don't know the answer, just say that you don't know, don't try to make up an answer.
Context:
{context}
Question: {question}
""",
input_variables=["context", "question"],
)
chain = LLMChain(llm=llm, prompt=prompt)
# Example query
question = "What companies have received Series A funding from Sequoia Capital?"
result = retriever.get_relevant_documents(question)
context = "\n".join([doc.page_content for doc in result])
answer = chain.invoke({"context": context, "question": question})
print(f"Question: {question}")
print(f"Answer: {answer['text']}")
This setup involves several components working in concert. First, your unstructured documents (like PDFs) are loaded and split into manageable chunks. These chunks are then embedded into vectors and stored in a vector database (Chroma in this case). Simultaneously, the system analyzes these text chunks to extract entities (like company names, funding rounds, investors) and relationships between them, populating a knowledge graph (Neo4j).
When you ask a question, the GraphRAGRetriever does a few things. It first uses the vector store to find semantically similar text chunks that might contain relevant information. Crucially, it also queries the knowledge graph for structured data that directly answers parts of the question. It then combines the information from both sources – the unstructured text chunks and the structured graph data – to provide a rich context to the LLM. The LLM then uses this combined context to generate a precise answer.
The key levers you control are the quality of your source documents, how you split them (chunk size and overlap), the choice of embedding model, the vector store, and the knowledge graph database. The prompt engineering for the LLM is also vital for guiding its interpretation of the combined context.
The truly surprising part is how the system dynamically builds and queries the graph. It doesn’t require a pre-built, perfectly structured graph. Instead, it extracts potential nodes and relationships from your documents as needed based on your query, enriching the graph on the fly or querying an existing one to find answers that would be impossible to surface from plain text search alone. This hybrid approach allows for both broad semantic searching and deep, precise relational querying.
The next step is to explore how to define custom node and relationship extraction schemas to improve graph accuracy.