Most teams that need full-text search eventually make an uncomfortable choice: keep everything in Postgres and accept the limits of its built-in text search, or stand up a separate engine like Elasticsearch and figure out how to keep two copies of the data consistent. ParadeDB, the repository behind the paradedb/paradedb GitHub organization, makes a third bet — embed a real search engine inside Postgres itself. The project’s tagline is blunt about it: “Just use Postgres. One Postgres for your application data, full-text search, vector retrieval, and aggregations.”

The implementation lives in the pg_search extension, a Rust crate compiled against pgrx that embeds Tantivy — the same Lucene-inspired library that powers Meilisearch-style engines — directly into the Postgres backend process. On top of that engine sits a custom index access method, a SQL-visible @@@ search operator, a planner hook that reroutes queries into a parallel custom scan, and MVCC-aware segment storage that survives Postgres crashes. Apache DataFusion joins the picture for aggregate workloads.

This post is a guided tour of that source tree as of October 2026, based on the main branch. Rather than describe the project from its documentation, we traced the actual code paths: what happens at CREATE EXTENSION, what runs when you write CREATE INDEX ... USING paradedb, and how a WHERE description @@@ 'running shoes' clause becomes a Tantivy query and then rows in your result set. Two diagrams map the moving parts — a high-level overview first, then a deeper walkthrough of the reader, writer, and storage layers.

Overview of the pg_search extension: the extension entry point registers GUCs and the index access method; SQL-level predicates compile into executable query inputs; the access method's build, insert, and scan callbacks drive the embedded Tantivy engine.

Reading the overview from left to right:

• _PG_init, L127 is the extension’s entry point in pg_search/src/lib.rs. It runs when Postgres loads the shared library and wires up the planner machinery before any SQL is spoken.
• gucs::init, L374 registers the paradedb.* session settings in gucs.rs — dozens of toggles covering custom scans, segment counts, parallelism thresholds, and cost heuristics.
• bm25_handler, L111 is the index access method handler. It fills an IndexAmRoutine with function pointers for build, insert, scan, delete, and vacuum — this is the moment USING bm25 (and its newer alias USING paradedb) becomes a first-class Postgres index type.
• ambuild, L38 implements CREATE INDEX: it reads every heap tuple, converts rows into Tantivy documents, and commits them as segments, with parallel builds supported on Postgres 17+.
• aminsert, L227 keeps the index fresh on ordinary INSERTs. Each indexed row becomes a Tantivy document keyed by its heap ctid.
• amgettuple, L293 and its bitmap sibling amgetbitmap, L346 are the two ways Postgres pulls matching rows out of the index during an index or bitmap scan.
• register_planner_hook, L268 intercepts planning so that a search predicate is executed by a custom scan node instead of a generic index scan — the key to top-N ordering and parallel workers.
• SerialIndexWriter, L211 is the single-writer wrapper around Tantivy that every mutation funnels through; commit, L522 makes written segments durable and visible.
• SearchIndexReader, L588 opens an MVCC-consistent view of the committed segments and exposes the search, top-K, and estimation entry points used by both scan paths.

The next diagram zooms into the internals: the SQL surface with its pdb helper types, every access-method callback, the planner and custom-scan layer, the Tantivy reader and writer, and the segment storage they share.

Detailed view of pg_search internals. The access-method group shows the build, insert, scan, delete, and vacuum callbacks that `bm25_handler` wires into Postgres; every query shape funnels through `SearchQueryInput` before hitting the Tantivy reader; the writer commits segments that `MVCCDirectory` later exposes to snapshot-consistent readers.

From Install to Your First Model

The fastest way to run ParadeDB is the bundled Docker image. The README’s one-liner is:

curl -fsSL https://paradedb.com/install.sh | sh

That script starts a container with the extension preloaded and drops you into a psql session. If you build from source instead, the repo is a standard pgrx workspace: cargo pgrx init to provision a Postgres install, then cargo pgrx install from the pg_search/ directory. Either way, the first SQL statement is the usual extension dance:

CREATE EXTENSION pg_search CASCADE;

The CASCADE matters — pg_search depends on companion extension objects for its pdb schema of helper types. Next, create a table and index it. Following the docs’ mock_items example:

CREATE TABLE mock_items (
    id INT PRIMARY KEY,
    description TEXT,
    category TEXT,
    rating INTEGER
);

CREATE INDEX search_idx ON mock_items
USING paradedb (id, description, category);

A note on naming: version 0.25.0 renamed the index from “BM25 index” to “ParadeDB index” and added USING paradedb as the canonical access method name. USING bm25 still works as a backwards-compatible alias — both names resolve to the same bm25_handler C function in pg_search/src/postgres/mod.rs. Listing the columns inside the CREATE INDEX statement is how you select which fields become searchable; per-field tokenizers and filters are configured with the pdb.field(...) and pdb.tokenizer(...) option helpers you can see parsed in api/config.rs.

Then query it:

SELECT id, description, rating
FROM mock_items
WHERE description @@@ 'running shoes'
ORDER BY rating DESC
LIMIT 10;

The @@@ operator is the whole query language in one glyph. Its right-hand side can be a bare string, or a typed expression built from the pdb schema — pdb.bool_and, pdb.fuzzy, phrase queries with slop, boosts via pdb.boost, and so on. Scoring is BM25 by default, and you can select the score with paradedb.score(id) or get highlighted snippets back. Beyond plain search, the same index serves filters, facets, term aggregations, vector similarity, and even cross-table joins — which is why the 0.25.0 rename makes sense in context.

Inside the Index Access Method

The cleverest structural decision in the codebase is that ParadeDB is not a separate server talking to Postgres over a socket. It is a bona fide Postgres index access method — the same extension point that btree, hash, and GIN use. That is what bm25_handler registers, and it explains why CREATE INDEX ... USING paradedb looks exactly like any other Postgres DDL.

Each callback has a non-trivial story. ambuild calls into create_index at pg_search/src/postgres/build.rs L303, which reads BM25IndexOptions (L409 in postgres/options.rs) to turn index options into a Tantivy schema, then streams every heap tuple through the writer. Inserts go through aminsert and are batched; Postgres 17+ gets an aminsertcleanup callback (L397 in insert.rs) that finalizes the batch when the insert series ends, which also keeps logical replication correct. Deletions during VACUUM land in ambulkdelete (L96 in delete.rs), and amvacuumcleanup (L24 in vacuum.rs) schedules segment merges afterward.

The storage layer underneath is worth appreciating. Tantivy segments are stored on Postgres relation forks through a custom directory implementation — MVCCDirectory at L372 in index/directory/mvcc.rs gives each snapshot of the database a consistent view of which segments exist, so a long-running query never sees a segment that a concurrent transaction hasn’t committed, and a rollback can’t leave phantom segments behind. Segment merges are serialized by a MergeLock (L43 in postgres/storage/merge.rs) so two vacuum workers can’t merge the same segments twice.

Queries: From @@@ to Tantivy

When the planner sees a @@@ predicate, three layers cooperate. First, the SQL surface: the operator’s support functions in pg_search/src/api/operator.rs parse the right-hand side into a pdb::Query value (the typed DSL at L362 in query/pdb_query.rs) and rewrite the predicate into a SearchQueryInput — the central enum at L76 in pg_search/src/query/mod.rs that represents every executable query shape. Second, estimation: estimate_selectivity_and_cost (L780 in operator.rs) asks the reader for a fast doc-count estimate via estimate_docs (L2697 in index/reader/index.rs) so Postgres can cost the plan sanely. Third, execution: the planner hook replaces the plan with a custom scan that compiles the SearchQueryInput into Tantivy queries and runs them over the MVCC-visible segments.

Custom Scans and Top-K

Why not just answer from amgettuple? Because search workloads care about ordering. A web-style query is almost always “top 10 by relevance” or “top 10 by relevance with filters,” and a generic index scan would fetch every match and then sort. The BaseScan custom scan (L112 in postgres/customscan/basescan/mod.rs) instead runs a TopKScanExecState execution method (L63 in top_k.rs) that keeps a bounded heap of the best documents seen so far, often using “fast fields” (stored numeric columns inside the index) to avoid heap fetches entirely. The planner hook also drives aggregate custom scans built on DataFusion, parallel scans that split segments across workers, and a join custom scan for cross-table search predicates. All of these are individually switchable through the GUCs registered at startup.

Honest Limits

Some grounded observations from the docs and code, without invented benchmarks:

• Not a standalone server. Tantivy runs inside each Postgres backend. You get ACID transactions and one system to operate, but heavy indexing work consumes the Postgres memory and CPU you give it; there are GUCs for writer memory budgets and background merging, but capacity planning is still your job.
• Index the fields you query. Like any inverted index, only columns listed in the CREATE INDEX statement are searchable. The deprecated key_field option is accepted for compatibility but ignored — new indexes don’t need a designated key column.
• Renaming churn is real. This repository moves fast: the 0.25.0 access-method rename, per-version migration files from 0.15.x through 0.26.x, and frequent option changes mean pin versions and read the changelog before upgrading.
• Version floor matters. Current examples in the docs assume ParadeDB 0.26.0 or newer, and some parallel features require Postgres 17 or 18.
• License. The community edition is AGPL-3.0, with enterprise features commercially licensed — relevant if you embed it in a service.

For context on where this design sits in the ecosystem, ParadeDB is best read alongside the tools it borrows from and competes with: the embedded Tantivy engine we covered in our Tantivy source tour, the classic standalone architecture of Elasticsearch, and the vector-only Postgres extension pgvector, which ParadeDB complements with hybrid search — BM25 and vector similarity from the same index.

Enjoyed this post? Never miss out on future posts by following us