GraphRAG Toolkit Builds Graph Enhanced GenAI Applications

Posted on by
Contents show
GraphRAG Toolkit

Ask a standard RAG chatbot what the return policy says and it performs flawlessly. Ask it how the return policy interacts with warranty terms for international orders and it wobbles, because the answer is not sitting in any single chunk of text. Two pieces of the document that were never near each other must be connected, and a vector search that only measures how similar each chunk sounds to the question has no mechanism to make that connection. That gap is exactly what graph enhanced retrieval closes, and the GraphRAG Toolkit from AWS Labs is the open source implementation worth building on: an Apache 2.0 licensed Python framework that constructs a hierarchical knowledge graph from unstructured documents and queries it with retrieval that follows real relationships instead of surface similarity. The GraphRAG Toolkit ships in a 3.18 release series, runs against graph and vector stores you can host locally in Docker, and powers a production deployment inside Deloitte's security intelligence practice. This article explains the architecture, walks through a working build, and shows where the GraphRAG Toolkit beats plain vector search in measurable terms.

Why vector only RAG fails at multi-hop questions

The failure mechanism is precise, and worth understanding rather than gesturing at. A standard vector store retrieves chunks based on semantic similarity between the embedded question and each embedded chunk, and it evaluates every chunk in isolation. Nothing in that process represents how one chunk relates to another: whether they discuss the same entity, whether one provides context the other depends on, or whether they sit in a cause and effect chain. The similarity computation has no memory of document structure, entity co-occurrence, or any relationship that crosses a chunk boundary. The GraphRAG Toolkit exists because that memory has to live somewhere, and a flat vector index is the wrong place to put it.

Multi-hop questions, the kind that need two or three connected facts assembled before an answer makes sense, are exactly where this breaks. It is a structural limitation of the retrieval mechanism itself, not a prompting problem or a context window problem. A larger context window does not help if the retrieval step never surfaces the second and third relevant pieces; the model cannot reason over passages it was never given. This is why teams that improve their RAG stack by swapping models often see no improvement on multi-hop benchmarks while seeing large gains on single fact recall.

The distinction has commercial weight. Users do not consistently ask polite single fact questions. Support teams ask how two policies interact. Analysts ask how one supplier's problem propagates to a product line. Research assistants ask how three related programs connect. Every one of those questions needs retrieval that follows structure, and every one of them is where the GraphRAG Toolkit was designed to operate. Teams already running conventional stacks can see the contrast in production terms through the RAG explainer with a real project, which covers the single hop case that vector search handles well.

What the GraphRAG Toolkit contains

The toolkit is two components with distinct jobs. The first, and the centerpiece, is lexical-graph: a framework for automatically constructing a hierarchical knowledge graph from unstructured documents and composing question answering engines over it. The second is BYOKG-RAG, short for Bring Your Own Knowledge Graph, a multi-strategy question answering layer for teams that already maintain a graph and want retrieval over it without re-extracting anything from raw documents. Together they cover both entry points into graph retrieval: build the graph from your documents with the GraphRAG Toolkit, or bring the graph you already have.

The lexical graph organizes extracted content into six connected levels: source, chunk, topic, statement, fact, and entity. That hierarchy is the structural feature that makes multi-hop questions answerable, because retrieval can hop between levels instead of being stuck at chunk level similarity. A question about a relationship between programs anchors on relevant statements, then follows entity and topic links across sources that a vector index would never have connected.

The project is Apache 2.0 licensed, developed in the open by AWS Labs, with the design documented in the research paper on hierarchical lexical graphs for enhanced multi-hop retrieval. The GraphRAG Toolkit is not purely theoretical either: Deloitte runs the lexical-graph inside its Amazon Neptune based GenAI Security Intelligence Center, a production deployment covered in a re:Invent technical session. The storage layer is pluggable in both directions: graph stores include Amazon Neptune Database, Neptune Analytics, Neo4j, FalkorDB through a separate contrib package, and RDF/SPARQL stores, while vector stores include Neptune, Amazon OpenSearch Serverless, Postgres with pgvector, and Amazon S3 Vectors. The model provider layer is the one fixed point: Amazon Bedrock is the built-in option, so an AWS account with Bedrock model access is required even when the graph and vector stores run entirely on your own hardware.

The extraction pipeline in detail

Building the graph in the GraphRAG Toolkit happens in two deliberately separate stages. The extract stage ingests raw source documents, chunks them, and uses an LLM to pull out topics, statements, facts, and entities. The build stage takes those extracted elements and populates the graph and vector indexes. Running them as separate micro-batched, resumable stages rather than one monolithic process means a large ingestion job can be paused, inspected, or filtered partway through without losing progress, which matters at production document volumes.

One design detail explains why extraction quality holds up: the pipeline makes two LLM calls per chunk rather than one. The first call cleans the raw chunked text into a set of well-formed, self-contained propositions. The second call extracts topics, statements, facts, and entities from those cleaned propositions rather than from messy raw text. The cleanup pass is optional, but the documentation is direct that extraction measurably improves when it runs first, because the extraction model works from unambiguous atomic claims instead of fragmented prose.

The hierarchy does the rest. Statements become nodes connected to their source chunks, facts connect to the statements that assert them, and entities become the connective tissue that links facts across documents. When the same entity appears in multiple sources, the graph records that co-occurrence, and the two connectivity metrics in the build stats, local and global, quantify how densely the resulting graph is linked. Teams grounding agent SQL in business semantics will find the same philosophy in NVIDIA's Auto Ontology semantic layer: encode the relationships once, and every downstream consumer inherits them.

Building a working assistant step by step

The fastest way to evaluate the GraphRAG Toolkit is a working build on free infrastructure: Neo4j and Postgres with pgvector both run locally in Docker, and the only paid dependency is Bedrock's per-call pricing. Prerequisites are Python 3.10 or later, Docker running, and an AWS account with access to at least one Bedrock model. The whole build below takes an afternoon, and every step maps to a documented method on the GraphRAG Toolkit classes rather than to hidden internals.

Step one spins up the stores with a two-service Docker Compose file: a neo4j:5 image exposing ports 7474 and 7687, and the pgvector/pgvector pg16 image exposing 5432, each with credentials set through environment variables. After docker compose up, both databases are live, with the pgvector extension already included in the image.

Step two installs the GraphRAG Toolkit and dependencies. The PyPI package for the lexical graph is graphrag-lexical-graph, currently in the 3.18.x series. The Postgres vector store requires psycopg2-binary and pgvector as separate installs, and llama-index-readers-web supplies the SimpleWebPageReader used to pull source documents, with html2text as its companion dependency.

Step three builds the index. A with block opens the graph store through GraphStoreFactory.for_graph_store with a neo4j:// connection string and the vector store through VectorStoreFactory.for_vector_store with a postgresql:// string and index_names set to chunk. The factories detect the backend from the connection scheme. LexicalGraphIndex wraps both stores, SimpleWebPageReader loads three deliberately related Wikipedia articles covering human spaceflight programs, and extract_and_build runs both pipeline stages in one pass with progress display. The context manager matters: the factory methods return context managers that keep the database connections open, so indexing and querying need to run inside the same with block rather than as disconnected scripts.

Step four inspects what got built. get_stats returns exact node counts across all six levels, for example three sources, 47 chunks, 62 topics, 210 statements, 178 facts, and 94 entities from the three-article GraphRAG Toolkit build, plus the connectivity metrics. Running it before querying confirms the extraction found and connected real content rather than producing an empty shell.

Step five queries with a genuine multi-hop question spanning all three sources: how the Commercial Crew Program relates to crew transport for the International Space Station and connects to Artemis program logistics. No single chunk from any single article contains the complete picture, which is the point. LexicalGraphQueryEngine.for_traversal_based_search is the recommended engine in the GraphRAG Toolkit: the question gets embedded, top-K similarity finds anchor statements, those anchors seed graph traversal queries that pull connected content across the graph, the retrieved material is grouped by topic and source, and the LLM generates the final answer from that assembled context. Passing context_format as json returns retrieved statements as structured output for inspection, and verbose set to false produces concise answers for production chat interfaces.

Two tuning notes complete the GraphRAG Toolkit build. The older semantic-guided search strategy still ships with the toolkit, but the documentation states it is likely to be retired in a future release, so traversal-based search is the right default to build against. And versioned updates handle the real-world case the one-shot build ignores: re-ingesting a document that has changed archives the old version automatically instead of leaving stale contradictions in the graph.

GraphRAG versus vector RAG on the same question

The clearest way to see what the GraphRAG Toolkit buys is to run the same multi-hop question against both retrieval stacks and compare the answers. On the vector side, the search returns the chunks whose embeddings sit closest to the question, which typically means passages from one or two of the three source documents, ranked by surface similarity. The model then answers from whatever made it into the top-K, and the missing program relationships simply do not appear. The answer sounds confident and is incomplete, which is the most dangerous failure mode an assistant can have.

On the graph side, the traversal engine anchors on statements from all three documents, because the entities linking them, the crew vehicles, the station, and the program milestones, exist as nodes the query can walk. The retrieved context arrives grouped by topic and source, so the model can attribute each claim to where it came from, and the answer covers the full relationship chain rather than the fragment with the highest similarity score. The difference is not model quality; it is what retrieval hands the model to think with.

This comparison is exactly what the RAG Explorer application from AWS demonstrates on your own documents, and teams replicating it should use questions they actually receive, because the advantage appears precisely and only when a question crosses document or section boundaries. Single fact recall shows no measurable difference, and pretending otherwise undersells the honest scope of graph retrieval.

Where the toolkit fits in production

Beyond the single index, the GraphRAG Toolkit exposes features that map to real deployment constraints. Multi-tenancy hosts several separate graphs, different document collections for different users, inside the same underlying stores rather than provisioning new infrastructure per tenant. Metadata filtering constrains both indexing and querying to document subsets based on tags. The graph can be exposed as a callable tool through a Model Context Protocol server, letting an agent browse a catalog of domain-specific graphs and pick the right one per question. Those three features together turn the GraphRAG Toolkit from a demo into a platform layer for organizations with many document domains.

The cost profile of the GraphRAG Toolkit is unusual in a good way: graph and vector stores run on infrastructure you own, from a laptop Docker setup to a managed Neo4j, and the per-call Bedrock pricing is the only metered component. The extraction stage is where the cost concentrates, since every chunk consumes two LLM calls, which makes the resumable pipeline more than a convenience; re-running extraction on a large corpus is the expensive path, so inspect and filter mid-pipeline instead.

For teams starting from an existing knowledge graph rather than raw documents, BYOKG-RAG skips extraction entirely and layers multi-strategy retrieval over what is already there. AWS also provides a self-guided workshop of Jupyter notebooks and a RAG Explorer application that compares graph-based and plain vector responses side by side on the same questions, which is the fastest way to see the difference on your own data. Agents that need deep retrieval across many sources, such as the pattern in Exa's ultra deep research agent, are natural consumers of a GraphRAG Toolkit deployment served through MCP.

Questions readers are asking

What is the GraphRAG Toolkit and who maintains it? The GraphRAG Toolkit is an Apache 2.0 licensed open source Python toolkit from AWS Labs for building graph enhanced GenAI applications. It contains two components: lexical-graph, which constructs a hierarchical knowledge graph from unstructured documents and queries it with traversal-based retrieval, and BYOKG-RAG, which layers multi-strategy question answering over an existing knowledge graph without re-extraction. How does GraphRAG differ from plain vector RAG? Plain vector RAG retrieves chunks based on similarity to the question, evaluating each chunk in isolation, so it cannot follow relationships between pieces of information. The GraphRAG Toolkit retrieves over a knowledge graph that records entities, facts, and their connections, so a question can anchor on relevant statements and follow real relationships across documents. Multi-hop questions that span two or three connected facts are where the difference is decisive.

Can the GraphRAG Toolkit run entirely on my own hardware? The graph and vector stores can: Neo4j and Postgres with pgvector both run locally in Docker with zero infrastructure cost, and FalkorDB and RDF stores are also options. The one cloud dependency is Amazon Bedrock, which performs the LLM extraction and answer generation calls, so an AWS account with Bedrock model access is required even for a fully local GraphRAG Toolkit build.

Why does the GraphRAG Toolkit run two LLM calls per chunk? The first call cleans raw chunk text into well-formed, self-contained propositions, and the second extracts topics, statements, facts, and entities from those propositions. Working from cleaned atomic claims instead of fragmented raw text measurably improves extraction quality. The cleanup pass is optional, but the GraphRAG Toolkit documentation recommends it, and the resumable two-stage pipeline means you can inspect extraction output before committing it to the graph.

When should I use BYOKG-RAG instead of the lexical graph? Use BYOKG-RAG when your team already maintains a knowledge graph and wants multi-strategy question answering over it without re-extracting content from raw documents. Use the lexical graph when you are starting from unstructured documents and need the hierarchical source, chunk, topic, statement, fact, and entity structure built automatically.

How do I serve the graph to an agent application? The lexical-graph can expose a tenant's graph as a callable tool through a Model Context Protocol server, so an agent can browse a catalog of domain graphs and select the right one for a question. Combined with multi-tenancy and metadata filtering, one deployment can serve many document domains with scoped access per user or application.

Sources
[fluentform id="2"]