auto (4,096 dimensions).
Two configurations of the retrieval engine were measured:
- Hybrid store (“Text Store”): zero-configuration storage with automatic hybrid retrieval — vector similarity combined with full-text search.
- Typed table (“Tables”): an explicit schema (document ID, content, 4,096-dimension vector) with pure vector search under cosine distance.
Results summary
A retrieval query goes through two stages: embedding inference (converting the query text to a vector) and database search (finding the nearest vectors in the store). The total response time is the sum of both.
The database search latency is reported by the server itself and excludes network overhead and embedding inference. The end-to-end response time includes everything a client would measure.
Methodology
Dataset. The full SciFact corpus and test split: 5,183 scientific abstracts, 300 scientific-claim queries, and the official binary relevance judgments (every one of the 300 queries carries labels). Embeddings. Every document (title + abstract) and every query was embedded with theauto embedding model (4,096 dimensions) through the platform embeddings API, in batches of 64. Corpus embedding averaged 8.2 ms per document; query embedding inference measured 12 ms per query.
Indexing. The same corpus and the same pre-computed embeddings were indexed into both engine configurations in batches of 100 documents. Indexing throughput was measured over the full 5,183-document run: 182 docs/sec for the hybrid store and 166 docs/sec for the typed table.
Query execution. All 300 queries ran against each configuration, retrieving the top 10 results. Two latencies were recorded per query: the server-reported database search time (pure engine time, no network) and the client-observed end-to-end response time. The tables above report p50/p95 across all 300 queries.
Scoring. Retrieved rankings were scored against the relevance judgments with the standard BEIR metrics — NDCG@10, Recall@10, and MRR@10 — computed per query and averaged over all queries with labels.
Interpreting the results
Accuracy context
The BEIR leaderboard reports NDCG@10 on SciFact for reference:
NDCG@10 scores in the 70–76 range indicate strong retrieval quality, competitive with leading embedding models.
What to look for
- NDCG@10 is the primary metric. It penalizes relevant documents that appear at lower ranks.
- Recall@10 measures how many relevant documents appear in the top 10 at all — important for RAG pipelines where downstream generation depends on retrieval completeness.
- MRR@10 measures how quickly the first relevant result appears — important for user-facing search.
- Hybrid vs pure vector: the hybrid configuration combines vector similarity with full-text keyword matching automatically; the pure-vector configuration trades that for slightly lower search latency. Their accuracy on this dataset is nearly identical — hybrid retrieval’s advantage grows on corpora where exact terms matter (names, codes, rare tokens).
Reading the latency breakdown
A retrieval query has three latency components:- Embedding inference — the time to convert the query text into a vector. This is model inference and is the same regardless of storage configuration.
- Database search — the time the engine spends finding nearest vectors, as reported by the server. This is pure search time with no network overhead.
- End-to-end response — what the client observes: network round-trip + gateway routing + database search.
See also
- Files and vector stores — the platform retrieval surface applications build on, including file ingestion, chunking, and search.
- GraphRAG evaluation — graph-aware retrieval benchmarked against flat RAG on multi-hop questions.