Reference
Glossary
The vocabulary the rest of the documentation assumes — segments, generations, identities, storage modes, roles, and the exactness terms.
Terms are grouped by what they describe. Where a term has a precise mechanical meaning in YoloSearch that differs from its general use in search systems, the difference is stated.
Segment — The unit of immutability, publication, caching, placement, and compaction. One immutable WavesDB checkpoint plus its sidecars, in one object prefix. A segment is never modified after it is committed.
Commit marker — The segment.commit object. It is written only after the
checkpoint, metadata, and sidecars are durable, which is what makes a partially
uploaded segment invisible rather than corrupt.
Checkpoint — The WavesDB object set a segment's data lives in: a MANIFEST
plus the table and blob objects for nine column families.
Column family — One of the nine WavesDB families a segment is built from:
meta, terms, postings, positions, impact, filters, docvalues,
vectors, stored. Each has its own table files, even though all nine share
one checkpoint and one commit.
Sidecar — A per-segment object outside WavesDB that supports routing and
filtering: ids.bloom, ids.winners, ids.ordinals, terms.bloom,
filters.postings.
Catalog — The authoritative record of what is visible. It holds an
immutable generation lineage plus a compare-and-swap latest hint.
Generation — An immutable catalog record naming the complete active segment set. A query pins exactly one generation for its whole life.
latest — A hint object naming the current generation, updated by
compare-and-swap. It is a hint, not the authority; the lineage under
catalogs/generations/ is what a reader trusts.
Lineage — The parent-child chain of generations. When conflicting children of one parent appear, the lexicographically lowest generation ID wins and the losers' segments are carried onto it, which is recorded as a lineage repair rather than losing a document.
Announce — The step that makes a published segment visible, by writing a new generation that includes it.
Publication lag — The time from commit-marker durability to
catalog-generation publication, measured by
yolosearch_publisher_publication_lag_seconds. The builder's own
yolosearch_builder_publication_lag_seconds measures from the first document
spooled instead, so the two answer different questions.
Compaction — Physically merging several active segments into one and atomically replacing them. It writes a new segment and a new generation and retires its inputs; it never modifies a visible segment in place.
Retired — A segment no longer referenced by the active generation. Retired is not deleted: reclaiming the bytes is garbage collection's job.
GC dry run — A mark-only pass that proposes unreferenced objects against explicit retention horizons and publishes an immutable proposal record. It removes nothing.
Quarantine — Two different things. In garbage collection, the waiting
period an immutable proposal must survive before a fresh mark may authorize
deletion (gc.quarantine_age). In the cache, the state an entry enters when
its content-addressed identity fails verification.
Logical key — The schema field marked key: true. It defines upsert
identity: pushing the same key again stores a new version rather than replacing
the old one.
Public ID — The deterministic 128-bit document identity used in result
frames and tie ordering. Streamed as lowercase hex by search --ids.
Ordinal — A segment-local dense document number. Ordinals are how postings and docvalues address documents inside one segment; they mean nothing outside it.
Mutation version — The version number distinguishing copies of the same
logical key. With query.collapse_key_versions on — the default — a query
returns only the copy with the greatest mutation version.
Tombstone — The delete primitive. A key's current version is superseded by nothing, so it stops appearing in results. The bytes are not reclaimed until a later compaction merges away the segment holding the document.
Liveness bitmap — The per-generation record of which documents are the current winners, used to resolve versions and deletes at query time.
Storage mode — Where an immutable reader gets checkpoint bytes:
HYDRATE_FULL, REMOTE_BLOCKS, or AUTO. It never changes query semantics or
the segment format.
HYDRATE_FULL — Requires a complete, verified local copy of every selected
segment before executing against it.
REMOTE_BLOCKS — Opens stable readers over immutable object-store sources.
Opens read table metadata; data blocks are fetched with bounded range requests
on demand.
AUTO — Decides per segment. A verified full-cache hit executes locally;
otherwise predicted request count, byte count, and scan fraction are weighed
against measured remote latency and throughput.
Hydration — Downloading every object a segment's checkpoint names, verifying it against the commit, and atomically admitting the completed directory.
Full-segment cache — The verified content-addressed disk cache of complete
segments, bounded by cache.full_bytes and cache.full_entries.
Block cache — The persistent decoded-block disk cache, bounded by
cache.block_bytes and cache.block_entries. Consulted only for remote-block
execution.
Read resources — The WavesDB in-process caches: decoded blocks, table
readers, and file handles, bounded by the cache.read_* settings. Distinct
from the disk caches and discarded on restart.
Scan bypass — A scan or readahead read may consult existing block-cache entries but does not admit new ones, so a one-off scan cannot evict a selective hot set.
Scrubber — The background pass that re-reads cached full segments and checks them against their commits. It is the only thing that notices a cached segment rotting on disk, because verifying costs the SHA-256 of every file in the segment and doing that per request would make a many-segment query spend all its time hashing.
Pinned — A cache entry currently in use, which eviction must not take.
Role — One of eight behaviors a yolosearch node process can serve. One
binary; the --roles value decides what it does.
| Role | Does |
|---|---|
coordinator |
Plans segment work units against the published catalog, dispatches them, and merges |
worker |
Executes work units against segments; owns the disposable caches |
merger |
A merge-tier node the coordinator delegates fan-in to |
aggregator |
Holds soft cache-residency and capacity state for placement |
router |
Admits ingest batches and routes them to builders |
builder |
Spools, seals, builds, and publishes segments |
publisher |
Holds the catalog lease and announces generations |
compactor |
Maintenance only: physical merges and garbage collection |
Role mix — The value of the role metric label. A process serving several
roles reports them joined: role="builder-publisher".
Zone — The placement zone a node advertises in (server.zone).
Aggregators are zone-local.
Work unit — One segment's share of a query, dispatched by a coordinator to a worker.
Placement — Choosing which worker executes a work unit, using cache-residency and capacity hints. Placement is never a correctness dependency: losing every aggregator degrades placement to a cache-oblivious tier and nothing else.
Hierarchical merge — Combining worker results through a tree of merge nodes rather than all at once, so fan-in and buffering stay bounded as segment count rises.
Bound propagation — Passing score bounds through the merge tree so subtrees can stop early without affecting the result.
Fan-in — Children per merge node in a coordinator's plan
(fleet.fan_in).
Lane — The scheduling class of a query: interactive, streaming, or
background. Worker slot pools are per lane (fleet.lane_slots).
Follower — The component in every node that watches latest and installs
new generations. The follower.* settings govern its polling, overlap, and
retirement behavior.
Generation window — The bounded period in which a worker may resolve a
generation it has not seen through latest, controlled by
follower.generation_overlap. Older unseen generations are refused.
Spool — The on-disk accumulation of accepted documents before a build is
sealed. Lives under ingest.dir.
Seal — Closing an open build so it can be built and published. Triggered by age, bytes, documents, an explicit flush, a schema change, or replay.
Admission — The router's decision to accept or refuse a batch, against
ingest.global_queue_bytes, ingest.index_queue_bytes, and the optional rate
quota.
Acknowledgement status — What a batch was told: accepted_ephemeral,
throttled, rejected, or published. Only published means the documents
are in a served generation.
Ephemeral acceptance — A batch accepted into the spool but not yet published. It is durable enough to be retried, not yet visible to a query.
Dialect — Which grammar a query string is parsed with: lucene or cqp.
Lowering — Turning a parsed query into the structured request the server executes. It happens client-side, so a typo lands a caret before any round trip.
BM25F — The ranking function, with per-field weights and b values
declared in the schema.
Exactness — The scope of the result-ordering guarantee.
| Value | Meaning |
|---|---|
GLOBAL_EXACT |
Exact ordering over the pinned generation |
EXACT_WITHIN_CANDIDATES |
Exact ordering within the retrieved candidate set |
APPROXIMATE |
Approximate ordering with a reported error bound |
Exact prefix — The leading results ordered under the query's exactness
class. top_k sets its requested size, from 1 up to the server limit of
100,000. A lower configured limit can apply.
Tail — The part of the result stream after the exact prefix.
--tail exact requests exact ordering; --tail banded permits approximate
ordering within a reported error bound.
Impact bands — Conservative per-term score-impact directories that make a
bounded approximate tail possible. Persisted only when tail.impact_enabled is
on; exact fallback does not require them.
Ordering error — The maximum score inversion a banded tail is allowed to contain. Zero requires exact fallback.
Collapse — Returning one hit per logical key, the copy with the greatest
mutation version. On by default; --no-collapse returns every hit in rank
order.
Projection — Asking for stored fields back with --fields. --keys
projects only the schema key; --ids asks for no stored documents and no
scores at all.
Vector field — A schema field declared to carry an embedding. An index may have several.
IVF-PQ — The approximate vector index: an inverted file of coarse centroids over product-quantized codes.
Probe — One coarse IVF list examined during candidate generation
(vector.query_default_probes, --probes).
Candidate — A vector the ANN stage produced for exact reranking
(vector.query_candidate_multiplier, --candidates).
Exact rerank — Scoring the candidate set against full-precision float32
vectors, which is what makes EXACT_WITHIN_CANDIDATES exact within its
candidates.
Exhaustive — Scanning every covered vector instead of generating candidates
(--exhaustive).
Fusion — Combining lexical and vector scores in a hybrid query: weighted
or rrf, reciprocal rank fusion.
Embedding mode — Which provider produced a vector: internal (pure-Go,
in-process), external (an OpenAI-compatible HTTP endpoint), or gRPC. The mode
metric label carries the schema-selected one.
Truncation — A text longer than a model's token window being cut to fit. It
is a real quality loss — the vector then represents only a prefix while BM25
still indexes the whole document — so it is counted by
yolosearch_embedding_truncated_texts_total rather than being silent.
Schema — A protobuf message carrying yolosearch.v1.document and
yolosearch.v1.field options. Field numbers are the field IDs the index keeps
forever.
Additive change — A schema change that only adds. It mints the next version. Anything else is refused with a diff.
Indexed, stored, filterable — What the index does with a field: analyze it into the term dictionary, keep it for projection, or make it available to equality and range filters.
Setting — One entry in the configuration catalog, with a dotted key, a kind, a scope, a default, an environment variable, and usually a flag.
Scope — What may change at runtime: startup, node-runtime, or
compile-time. Only node-runtime settings accept config set.
Provenance — The layer a setting's current value came from: default, file,
environment, flag, or the runtime API. Reported by config get, config list,
and config export.
Format version — The segment format a binary writes
(format.segment_version) and the oldest it reads
(format.minimum_reader_version). The Kubernetes operator gates upgrades on a
declared read range and write format.
WavesDB — The immutable segment storage engine YoloSearch builds on.
Receipt — A durable record of what a run measured, with the revision and inputs it was measured on. Receipts record milestone evidence, benchmark results, and vendored dependency manifests.
Milestone — One numbered phase of the implementation roadmap, with an entry gate, deliverables, and an exit gate. A milestone is not closed until its exit gate has a receipt.
- Concepts — these terms in context
- Metrics reference — where the label vocabularies are used
- Configuration reference — every setting the terms above name