SereneDB is a database that runs search and analytics in the same query engine. The search half is a C++ library called IResearch that we started in 2016 and still maintain and if that sounds familiar it is because it is what ArangoSearch, a search engine integrated into ArangoDB, runs on.

A billion OpenTelemetry logs, 92 queries and on the other side of the table a ten year old fork of our own code. ArangoDB still ships IResearch, wired into a real database and answering real queries, so for once we get to benchmark ourselves against ourselves. Back in March we published that IResearch won "Search Benchmark, The Game", but that was a microbenchmark and it said nothing about the database around it. Ten years of work either shows up in the numbers or it doesn't and this is how you find out.

Everything below is open so you can rerun it yourself. The head-to-head runs at 100M because that is as far as ArangoDB's Community Edition is licensed to go and past that it stops being a performance question and turns into a licensing one.

TL;DR​

If you only want the numbers.

Results are available at serenedb.com/searchbench and the benchmark itself lives at github.com/serenedb/searchbench.

What SearchBench is​

That March benchmark is the search-benchmark-game, built by the Tantivy team and forked by us to add an IResearch engine. One process, one index, term and phrase and boolean queries, count and top-100 by score. It measures the search library on its own: decode the posting lists, intersect them, score the top 100. There is no SQL layer, no query planner, no columnar storage involved and no joins. So a win there tells you the postings format and the scoring loop are fast and almost nothing about what a database built on top of them does with a real query.

So we built SearchBench, an open benchmark for search and analytics, which we announced at Berlin Buzzwords. Same idea as Search Benchmark, The Game, moved end to end: how fast does the whole database answer a real question.

The methodology is heavily inspired by ClickBench. We run two scales, 100M and 1B log records and measure three things at each: time to ingest and index, size on disk and the median latency across the 92 queries. Result caching is off on both engines. Each engine implements the same handful of shell scripts (install, start, stop, check, load, query, data-size) and a shared driver orchestrates them, so adding an engine is an afternoon.

One rule sits above all of that: every engine gets our best attempt at the best configuration we can build for it. Without that none of these numbers are worth publishing.

The data​

The corpus is generated OpenTelemetry logs. We use the set published by TextBench instead of rolling our own. Every record has 15 columns describing one log message:

The three *Attributes columns are JSON maps. The database has to store all 15 columns, including the ones no query touches. That rule matters more than it sounds, because an engine that only keeps the searchable text will report a beautiful index size and then be useless for actually reading your logs.

The queries​

92 of them, in five groups:

Two of those five are top-100 problems and they differ only in the ordering key: top_k (bm25) ranks by relevance, top_k (time) ranks by timestamp. Keeping them apart matters, because the two get answered by completely different machinery.

Each group is then crossed with how you are asking: single term, conjunction, disjunction, minimum-should-match, phrase, phrase with proximity, prefix, regexp, wildcard, fuzzy, negation and time windows. Plus a term-frequency dimension, because a word in 40% of your logs and a word in 0.001% of them are different problems.

That grid is what produces 92. Every query carries its tags in the result file, so you can slice by group or by filter or by term frequency instead of staring at one aggregate. In the raw files the two top-100 groups are tagged top_k and recent.

Why ArangoDB is the right mirror​

The comparison is worth running because of where the other side comes from. ArangoDB is where an older version of our own code still runs in production and that makes it the closest thing we have to a time machine.

And we do mean our own code. The lineage is traceable in public:

• October 2016. We open iresearch-toolkit/iresearch. That repo is archived today.
• December 2023. ArangoDB forks it into their own org.
• January 2026. They move it out of 3rdParty and copy it into the main tree at lib/iresearch. A little over a million lines in one commit.

What happened in between is what makes this a fair mirror. 51 commits in the two and a half years since the fork and a few of those are merges pulling our upstream work back in. The rest is maintenance: new consolidation defaults, a clang compatibility pass, a memory over-allocation fix, an iterator seek edge case. The same codebase took 1,052 commits in 2017 alone.

So that is a stable dependency doing its job. It also means the engine on the other side of this benchmark is close to the one we handed over, which is what makes it a mirror. Most of what the numbers show is what changed on our side.

How we set it up​

The machine​

Both engines got 100M OpenTelemetry log records on the same machine: a single GCP n2-standard-32, which is 32 vCPUs of Intel Ice Lake at 2.6 GHz and 128 GB of RAM, running Ubuntu 24.04.4 LTS. Storage is a 3.9 TB pd-ssd Google Persistent Disk. One instance each, no cluster and no sharding on either side. SereneDB is v26.07.5 and ArangoDB is version 3.12.9-4, Community Edition.

Community Edition needs a note, because "we benchmarked the free tier" usually means "we benchmarked a crippled build". Not here. Since 3.12.5 the Community Edition ships every Enterprise feature with no time limit. So feature for feature, this is the best ArangoDB there is. What it costs you is the right to run it commercially in production and a 100 GiB ceiling on total database size. That ceiling matters a lot later on.

The data model​

None of the numbers mean anything unless both indexes have the same information to work with. The ArangoSearch view below is the fastest one we managed to build for this workload and every field the queries touch is indexed the same way on both sides. Here is exactly what each engine got.

Tokenization. SereneDB indexes the expression ts_split_by_non_alpha(Body, true), which lowercases and splits on runs of non-alphanumeric characters, over a keyword dictionary with frequency, norm and position enabled. ArangoSearch gets a segmentation analyzer with break=alpha and case=lower and the same three features. Those two are the closest analogs the two engines have. They agree on ordinary log text and they can disagree at the margins, on things like v1.2.3 or user_id, so the term dictionaries are not identical down to the byte. That difference is worth saying out loud and it is nowhere near the size of the gaps below.

Stored columns. SereneDB's index carries all 15 OTel columns as INCLUDE columns, so retrieval, filtering and aggregation run index-only. The ArangoSearch view gets storedValues covering every column the queries return, which is the same deal. Neither engine has to go back to a row store to answer a query.

Scored top-K, meaning the top_k (bm25) family and not top_k (time). SereneDB's index is built with optimize_top_k = 'bm25(1.2, 0.75)', our own WAND implementation inside IResearch. The ArangoSearch view is created with optimizeTopK enabled and the matching BM25 parameters. Both sides get to skip blocks they can prove cannot make the top of the list.

Query translation. Every SereneDB query has a hand-written AQL twin with the same filters, the same windows, the same grouping, the same sort and the same limit. The mapping is mechanical and both sides are documented: SereneDB full-text functions against ArangoSearch AQL functions.

That last row is the one place we had to improvise. ArangoDB has no regular-expression matcher inside SEARCH at all, so the nine queries that use one are hand-translated into whichever primitive comes closest: STARTS_WITH(d.Body, 'charg') where the pattern is a prefix followed by .*, LIKE(d.Body, 'c_che') where a . sits inside the word. We could have marked those queries unsupported and moved on. Giving ArangoDB the nearest primitive instead hands it an advantage, because STARTS_WITH walks one contiguous range of the term dictionary while the regex it stands in for compiles to an automaton that can walk all of it. Nine of the 92 queries are tilted ArangoDB's way and we would rather say so up front.

Both query files are in the repo. If a translation looks unfair the diff is right there and we want to hear about it.

How we measure​

Timing. Three runs per query and we report the best of the last two. Between queries we stop the engine, drop the OS page cache, start it again and wait for it to report healthy. Each measurement is the client round-trip for that one query, taken from psql's \timing on the Postgres-wire side and curl's %{time_total} on the HTTP side, so the cost of spawning and connecting a client stays out of the number.

The 60 second rule. Any single query gets 60 seconds. If it does not finish, we record that it did not finish. There are several of those below and they are all on the ArangoDB side, so we want to be precise about what it means: it is not a latency of 60 seconds, it is a query that never came back.

What this does not measure. The driver runs one query at a time from a single client, so everything here is latency and none of it is throughput. There is no concurrent load, no ingest running alongside queries, no updates or deletes and nothing distributed. Memory footprint is not recorded either. Those are all fair questions about a log store and none of them are answered here.

Load and size​

Before any query runs, there is the part where you wait.

Ingest is 20.5x faster and the index is 5.3x smaller. Both numbers cover the same work: read the corpus, tokenize the body, build the inverted index, store every column for retrieval.

The ingest number needs one note, because the two sides do not get the data exactly the same way. Both engines run in Docker with --network host. SereneDB builds the index straight off the parquet: the table is a view over read_parquet and CREATE INDEX consumes it in process, so nothing gets serialized and nothing crosses a socket. ArangoDB cannot read parquet at all, so the corpus goes through a throwaway serened container that emits NDJSON on stdout, arangoimport reads that over a pipe at its own default parallelism and inserts the documents and only then does the view link index them. Some of those 18.6 minutes is JSON serialization and JSONL parsing rather than index building. We report the pipeline because it is the pipeline you would actually have to run, but if you want the pure indexing cost on the ArangoDB side it might actually be lower.

Query latency​

Median query latency per task family.

2 of the 9 join queries never came back. They hit our 60 second ceiling and we record them as 60 seconds, which understates ArangoDB's real time and therefore understates the gap. Both sit above the join median of 32.5 s, so neither is what that bar is measuring. The other seven finish between 13 and 45 seconds. Drop the two capped queries and the median across the remaining 90 is 15.6x, against 16.1x for all 92.

16.1x at the median, 17.2x geometric mean, across all 92 queries. Our best single query is 460x: a time-windowed top-100 that takes us 8 ms and ArangoDB 3.7 s. We quote the median instead because that is the number that tells you what the database feels like to use. It is also the number that survives an argument.

SereneDB is faster on 89 of the 92 queries. On the three it loses, both engines land somewhere between 1 ms and 15 ms, close enough to the floor that run-to-run variance decides the winner rather than anything about the engines.

The billion logs benchmark​

We ran the full billion on our side. SereneDB answers all 92 queries at that scale with nothing capped and nothing failed.

Ingest and index took 8.5 minutes and the index is 120 GiB on disk.

None of it scales linearly. Ten times the data costs a lot less than ten times as much:

Latency is the one that matters. Take each query on its own and the median slowdown is 2.8x for ten times the corpus, with a geometric mean of 2.9x. Build time came in slightly under linear at 9.4x. Size went the other way at 11.5x, so the index gets proportionally fatter as the corpus grows. That one is moving against us and we would rather say so than let someone find it.

Why we can't run ArangoDB at a billion​

ArangoDB's Community Edition enforces a 100 GiB total database size limit. Go over it and you get warnings for two days, then the deployment drops into read-only for two days, then it shuts down. At 100M logs the ArangoDB instance is already 55.3 GiB, so the licensed room runs out somewhere around 181M logs. A billion would have put us far past that, so we did not run it.

That leaves us extrapolating the two things that scale predictably. Straight-line from the measured 100M run, a 1B ArangoSearch index comes to roughly 553 GiB and at least 3.1 hours to build. Both are floors if ArangoSearch scales the way ours does: our own index grew 11.5x for ten times the data. Read the same figures as capacity and one 100 GiB instance holds around 181M logs on ArangoDB against around 830M on SereneDB. If you want measurements instead of our arithmetic, SearchBench is open and reproducible: the ArangoDB adapter is in the repo, so anyone holding a license that permits it can run the billion and publish what they get.

Query latency is the one thing we cannot extrapolate, so we did the next best thing and lined our 1B numbers up against ArangoDB's 100M ones, with only one side carrying ten times the data.

So those are the numbers. The rest of this post is about where they come from.

Where the speed comes from​

Query categories explained​

Inside the search engine​

We reworked most of IResearch: the retrieval path, the scoring loop and the postings format itself. The details are published as the Search optimization journey series and each part maps onto something in the table above.

• Collecting top-K candidates, part 1. The pruning threshold has to be exact after every insert, because a stale bound makes every later block do more work than it needs to. Tantivy hit the same thing and swapped in a binary heap for exactly that reason.
• Block scoring, part 2. Score a whole block of postings at a time instead of a document at a time. Tantivy is now exploring the same approach, citing that post.
• Optimize norm gathering, part 3. Norms sit on the hot path of every scored query, so how you fetch them shows up everywhere in top_k (bm25).
• How to efficiently execute two phase queries, part 4. Matching separated from scoring, so the expensive half only runs on the documents that survive the cheap half.
• Adaptive posting list format, part 5. The encoding adapts to the density of each term rather than forcing one layout on everything, which is most of why the index is 5.3x smaller.
• A new columnar store. This is the one that never got a post and it matters twice over. The store holds the storedValues that retrieval reads and the norms that BM25 scoring reads and in the version ArangoSearch runs it does no compression at all: norms are created with compression::none hard-coded. So the same bytes are bigger on disk and more expensive to pull through the scoring loop. Ours compresses, which is a good part of both the 5.3x size difference and the top_k (bm25) numbers.
• BlockMax MaxScore, not written up yet. For scored disjunctions, per-block score bounds let the iterator skip whole blocks that cannot reach the current top-K threshold. That is what carries the or entries in top_k (bm25).
• A filter optimizer. Query rewriting used to happen inline while a filter was being prepared. It is now a rule registry with separate rule families for booleans, negation, ranges, terms, lowering and levenshtein prefixes and prepare does nothing but look up cookies. Wildcards and regexes lower to an AutomatonFilter and fuzzy matches to a LevenshteinAutomatonFilter, so regexp, like and fuzzy all end up as automaton intersections against the term dictionary instead of separate hand-written code paths.
• Parallel per-segment prepare. Preparing a filter is not free and for a fuzzy query it is most of the query: before anything can match, the engine has to walk the term dictionary of every segment collecting the terms within edit distance. That work now runs per segment in parallel.
• A dedicated path for counting. Counting is a first-class operation on the iterator tree rather than something layered on top. count() is pure virtual on DocIterator, so every iterator implements it and a conjunction or a disjunction can report how many documents it matches without ever scoring or collecting them. Thirteen iterators implement it today. On top of that the planner pushes an empty projection into the scan for count(*) shapes, so the column store is never opened either. The plan says so out loud:

╭─ IRESEARCH_SCAN ─────────────────────────────╮
│ Index: logs_idx                              │
│ Index Filter:                                │
│ ╭─ Term ───────────────────────────────────╮ │
│ │ Field: ts_split_by_non_alpha(body, true) │ │
│ │ Value: payment                           │ │
│ ╰──────────────────────────────────────────╯ │
│ Output: row-count only                       │
╰──────────────────────────────────────────────╯

Around the search engine​

• Parallel execution. ArangoSearch exposes a parallelism option on SEARCH that spreads index segments across threads, so we tried it. The workload came out around 1.5x slower, so the published run leaves it at the default. On our side both halves run in parallel: preparation per segment inside the search engine, then execution out here in the query engine.
• Vectorized execution. Operators run over batches of values. Aggregating a few million matched rows by service or by minute is a columnar problem where the index plays no part and the group_by family is essentially a measurement of that.
• Join strategies with a cost-based optimizer. Correlating two services through a trace id is a join. For us it becomes a hash join inside the vectorized engine, running over columns the search index already holds, with a planner picking between several join algorithms. In ArangoDB the same question becomes a nested loop: for every log on the left, go look up the matching logs on the right. At 100M records that difference is architectural and no amount of postings tuning closes it.
• Regular expressions. ArangoSearch has no regex matcher inside SEARCH at all, so the nine regexp queries had to be hand-translated into STARTS_WITH and LIKE. We have one thanks to aksel2904, who contributed the regexp filter.

Don't trust us, run it yourself​

Everything behind every number above is in github.com/serenedb/searchbench: the adapters, the 92 tagged queries in each dialect, the shared driver and the raw result JSONs. Each engine directory has a README listing its env vars.

git clone https://github.com/serenedb/searchbench
cd searchbench/arangodb   # or serenedb
SEARCHBENCH_DATASET=otel_logs_100m SEARCHBENCH_DATA_DIR=/data ./benchmark.sh --index

--index fetches the corpus, loads it and builds the index. Drop it to re-run queries against an engine that is already loaded. Omit SEARCHBENCH_DATASET and you get the billion. There is a 1M smoke scale that streams about 50 MB over HTTPS if you want to check an adapter before committing to a real run.

Full per-query results for every engine we have wired up, including the ones absent from this post, are on the results page.

Send us a better run​

If you think we got something wrong or that we left an engine undertuned, the fix is a pull request. We mean that literally. Every adapter in that repo is somebody's best attempt at configuring a database they did not write and ours are no exception. If you know ArangoDB better than we do, we would rather learn it from your PR than defend a number we got by guessing.

The same goes for the workload. If a translation looks unfair, if a query family is missing, if the corpus looks nothing like your logs, open an issue and say so. New engines are welcome too and cost about seven small scripts.

A benchmark is only worth anything when the people being measured can push back on it. That is why this one is Apache 2.0 with no conditions attached to publishing results and why the queries and the adapters and the raw JSONs are all sitting there in the open. Everyone in this field is guessing about everyone else's engine and the only way out of that is to compare notes properly. We would like SearchBench to be where that happens.

Conclusion​

Ten years on, the engine we started in 2016 as a pet project is unrecognizable from the one still shipping inside ArangoDB and this is the first time we have put numbers on that.

Here is what a billion OpenTelemetry logs look like on a single machine. Ingest and index takes eight and a half minutes and lands at 120 GiB on disk. Of the 92 queries, the 83 non-join ones come back at a median of 29 ms, with just over seven in ten under 100 ms. The nine expensive joins that correlate two services through a trace id take just a few seconds.

If you are running search workloads on something that makes you choose between full-text and analytics, SereneDB is worth twenty minutes of your evening. It speaks the Postgres wire protocol, so whatever client you already have will do and every number in this post is a CREATE INDEX away from being reproducible on your own dataset.

This is the first of these and not the last. Several engines are already wired up with results in the repo at both scales, so the next comparisons are mostly a matter of writing them up.

If you find this interesting, we'd be grateful if you support SereneDB with a star on GitHub. For an early-stage project, it means more than you might think.