Welcome to The State of Serene, our monthly note on what we shipped, what we learned and where SereneDB is heading. This is the first issue, so before we get to June, let's answer the obvious question: what is SereneDB and what can you actually do with it today?

Instead of moving your data to the model, point the model at the data. Imagine an agent gets asked: "Which enterprise accounts are unhappy about latency and how much revenue is at risk?" It can't grep 100GB of tickets and it won't fit them in a context window, so it asks SereneDB instead. Search pulls the tickets that are really about slowness, even the ones that never use the word "latency". The same query joins them to accounts and sums the revenue at risk by plan. The agent gets back a handful of numbers it can act on, not a stack of tickets to read.

So an agent needs two things: the most relevant data and the statistics over it. Hybrid search gives the relevant part. Analytics gives the numbers on top. SereneDB does both in one query and hands back a small answer, so the agent spends its tokens on the decision, not on scanning files.

That's SereneDB: lakehouse search and analytics for your agents. One Apache-2.0 database that speaks Postgres and reads your data where it already lives, whether that's Parquet and Iceberg in the lake, your Postgres or your warehouse. It indexes that data in place and hands back the slice that matters, so nothing gets copied into another system and nothing goes stale. Think of it as a compute and delivery machine for your agents.

What you can do today​

It's Postgres-compatible​

Compatible on two levels, the wire protocol and the SQL dialect, so most of what you already use just works.

• Connect with psql, any Postgres driver or your favorite ORM. Nothing new to learn.
• Standard types plus pg_catalog and system tables, so BI tools and ORMs introspect it like any Postgres.
• See what's supported and the few quirks in the compatibility docs.

Search and analytics in one engine​

One inverted index holds full-text and vectors, sitting right next to columnar, vectorized analytics. Same data, same query.

• Full-text search: BM25, phrase, fuzzy, wildcard and regex, plus a stack of tokenizers.
• Vector search and hybrid that fuses both with tunable ranking.
• Geospatial search in the same index.
• Real analytical SQL right beside it: joins, CTEs, window functions and GROUP BY over the same rows. Filter by relevance, then aggregate, in one query.
• It all runs on iresearch, our own open-source engine and a fast one.

Index your data where it lives​

Point SereneDB at your data and query it like a normal table, no ETL and no copies.

• Remote files: Parquet, JSON, CSV and Iceberg on S3, HTTP or Hugging Face.
• ATTACH a live Postgres database and join it against your indexed data.
• Bring it in for one source of truth or leave it where it is and index it in place. Same SQL either way.

See it end to end in the code search demo: hybrid search over 11.4M rows read straight from Hugging Face. More on that below.

What happened in June​

A few of June's highlights worth your time.

Search queries prepare in parallel​

Setting up a search query used to walk the index segments one at a time. Now the prepare phase and the scan both fan out across segments, so query setup scales with your cores instead of pinning a single one. Ranked queries also skip documents that can't reach the top n. Under real concurrency it adds up. Here are Top-K queries at 32 threads, before and after this month's search work:

filter	before	after	speedup
ngram	162 ms	65 ms	2.48x
and_not	325 ms	177 ms	1.83x
any_minmatch	871 ms	518 ms	1.68x
like	7,421 ms	4,831 ms	1.54x
phrase	1,171 ms	964 ms	1.22x
regexp	5,467 ms	4,887 ms	1.12x

Every one of the 26 filter types came out ahead, from 1.06x (gt) to 2.48x (ngram).

And it scales with cores. BM25 scorer streaming runs 1.90x faster at 32 threads and 2.08x at 64. Top-K gets most of its win by 32 threads.

Smarter search filters​

Search filters now run through a set of optimizer rules that lower each one to the cheapest form that still matches, done once up front instead of ad-hoc code in the prepare path. Some of the rewrites:

you write	it becomes
ts_starts_with('abc') AND ts_levenshtein('abcdef')	one fuzzy match with the abc prefix fused into its args
nested booleans, a OR (b OR (c OR d))	one flat a OR b OR c OR d
nested negations, NOT (NOT a)	one flat a
NOT (a OR b)	NOT a AND NOT b

Cheaper filters and the lowering runs once at optimize time instead of per query.

A rebuilt server that speaks two protocols​

We rebuilt the server from the ground up as a sans-io network server with zero-copy framing. Two protocols talk to it:

• Postgres wire protocol: a clean implementation with auth, a real CLI and a psql-style shell out of the box, plus background schedulers for maintenance.
• Elasticsearch HTTP API: a drop-in _bulk, _search and scroll, so existing Elasticsearch clients point straight at it. There's a migration guide for when you want to move queries to SQL.

Underneath, we also started collapsing the storage layer, so fewer moving parts and a simpler single binary.

And it's faster. A wire bench (trivial queries, pgbench-style, 16 clients), so it measures the server and protocol layer, not query execution, since the engine underneath is unchanged. select1 is a bare SELECT 1 (pure transport cost), wide is a parse-heavy SELECT 1 WHERE 1=1 AND ... AND 10=10 (isolates parse cost) and simple/extended/prepared are the Postgres protocol modes. New server against the old one:

The demo that ties it together​

We put the whole thing in a live demo: code search over ~11.4M Codeforces solutions and their problem statements, read straight from Hugging Face with no ingestion job. Solutions get GitHub-style code search (exact and fuzzy), matched with a new sparse ngram tokenizer. Tasks get hybrid keyword plus semantic. And there's an MCP endpoint so an agent can search it directly.

The fun part: the backend took about 15 minutes. We handed Claude our docs and it wrote the whole thing, pointing the database at the remote data, building the indexes and writing the queries.

Go try it: codesearch.serenedb.com · repo: github.com/serenedb/codesearch · writeup: Code search over a data lake.

Also shipped​

• a new on-disk index format, the base layer a lot of the above sits on
• column statistics from the index, so aggregates like SUM hit faster paths (6-7 ms, was 23-27 on 25M rows)
• VARIANT support (store, load, shredding) and array indexing for semi-structured data
• the inet type
• nanosecond timestamptz

What to expect in July​

On the engineering side​

• landing ts_dict (analytics from the search index)
• a new vector-search design we've been cooking
• more efficient columnar storage and execution, with dynamic filters
• indexing of complex types, including schema-agnostic indexing
• foreign data sources via CREATE SERVER, including a ClickHouse connector
• ongoing work on logical replication (publications and subscriptions)
• PG-compatible authorization and RBAC

Benchmark season​

Summer's here, so we're calling it benchmark season. Over the coming weeks we're putting SereneDB up against the best-in-class databases in the open, with methodology and raw numbers published like we always do. First up is a billion-record benchmark on search-heavy workloads, our biggest test so far. Stay tuned.

Kudos​

A lot of SereneDB's search surface comes from external contributors. A few we want to shout out:

• romanpovol built the path-hierarchy and pattern tokenizers
• afigor2701 worked on inverted-index internals (WAND data handling and integer encoding)
• aksel2904 added the regexp filter
• abhishek593 added the union tokenizer
• k-morozov did the synonym tokenizer with WordNet support, plus the CREATE PUBLICATION / CREATE SUBSCRIPTION grammar
• hinata34 co-authored the filter optimizer

Thanks folks, keep the PRs coming.

Want to be in the next one? We love outside contributions and we're happy to help you land your first PR. We tag beginner-friendly work with good first issue, so grab one, ask questions in the issue and we'll get you going. There's plenty of things to dig into.

---

If "search and analytics on the data where it lives" sounds useful, the fastest path is the code search demo. And if you like what you see, ⭐ star us on GitHub. Hit a rough edge or have a use case? We'd love to hear it: open an issue or just say hi.

See you in the next State of Serene.