Why YoloSearch

YoloSearch keeps published segments, schemas, and catalog generations in object storage. Query workers read immutable segments and retain disposable local caches. A worker can execute any assigned segment without owning it permanently.

This separates durable storage from query capacity. Adding a worker adds execution capacity; its cache fills as it reads or prewarms segments. Removing a worker leaves published data in object storage, but can interrupt active queries and reduce cache hit rates.

A query pins one catalog generation. Its segment set and document liveness remain fixed for the query's lifetime.

Workers return ranked runs with score bounds. The merge tree uses those bounds and a deterministic tie order to establish the exact prefix. Exactness depends on the retrieval path: lexical search can be globally exact, while ANN vector search is exact within the candidates it retrieves. Each response reports the applicable class. See ranking and exactness.

Placement aggregators supply expiring capacity and cache observations. Workers perform their own admission checks. If aggregators are unavailable, selection uses the configured fallback workers. Operators must keep that list current.

Failed work can be retried before emission. A failure after emission can end the public stream with an error; clients must check stream completion before treating the response as complete.

• Lexical, positional, vector, and hybrid search over document collections.
• Interactive top-K requests alongside longer streams of document IDs.
• Deployments that scale query workers independently of durable storage.
• Corpora where remote reads and local caching are useful alternatives to keeping the full corpus on each worker.

The current server caps a request at 100,000 results. top_k accepts values from 1 up to the configured result limit. Approximate tails require explicit opt-in and compatible segment data.

Billion-document corpora, aggregate ingest in the hundreds of thousands of documents per second, and p99 publication lag below 60 seconds are design targets. They are not capacity guarantees established by the implementation. Measure your corpus, query mix, hardware, and object store before sizing a production deployment. The sizing guide describes the resource budgets.

A local process is enough to evaluate schemas and queries. A distributed deployment adds worker discovery, cache sizing, object-store traffic, and catalog propagation to the operational work.

Area	Current behavior
Write visibility	A flushed push to serve or yolosearch-local waits for publication and local visibility; other nodes follow the catalog independently
Ingest acknowledgment	ACCEPTED_EPHEMERAL does not establish durable publication; callers must handle replay and duplicate submission
Unsealed ingest buffers	The spool is synced at seal, so a crash can lose unsealed data
Document updates	A new segment carries the new version; generation liveness selects the winning version
Document deletion	A tombstone removes the winning key from search once the generation is loaded; physical reclamation requires compaction and garbage collection
APIs	Client and inter-node APIs use gRPC; health, metrics, and the admin console also expose HTTP endpoints
Transactions and joins	There is no transactional multi-document update or relational join API

Stored fields let applications retrieve values with search hits. The --ids output mode skips stored-field reads when downstream processing needs only document identifiers.

• Architecture — components and coordination
• Ranking and exactness — score bounds and result ordering
• Ingest — acknowledgments, publication, and deletion