Ranking and exactness

YoloSearch reports the scope of its ordering guarantee in the response. Globally exact retrieval covers the pinned generation; candidate-based retrieval orders the selected candidates. An approximate tail reports its ordering error bound.

The lexical scorer is BM25F. Each indexed field carries two parameters set in the schema:

• weight (default 1.0) — the field's contribution multiplier.
• b (default 0.75) — the length-normalization factor for that field.

Both are accepted on indexed fields only. In the sample schema, title is weighted 2.0 with b: 0.6, increasing its contribution relative to a field with weight 1.0. The final score also depends on term frequency and field length.

A bare term in either grammar lowers to the pseudo-attribute word, which means every indexed field: an OR of one term per indexed field, rendered field=*. That is the exact form under BM25F, and it collapses to a single term when only one field is indexed.

The search service supports lexical and vector weights and weighted or reciprocal-rank fusion. Static-feature and query-boost fields exist in the protocol, but the current service rejects nonempty values for them.

Internally, scoring contracts track evaluation, bounds, versions, and whether candidate generation is exhaustive. Global exactness requires exhaustive retrieval and valid bounds. The response reports the class supported by the execution path.

The class appears in the SearchHeader and again in the SearchTrailer, and the ordering class appears on every individual ResultFrame.

Within a segment, a worker uses block-max WAND and MaxScore to skip blocks that cannot contain a competitive document. It returns four things to its parent:

• an ordered local run;
• the highest remaining unseen score bound;
• cost counters; and
• the score-bound version it used.

Those runs meet in a bounded-fan-in merge tree (default fan-in 32) with one lookahead frame per child. A node's advertised bound is the maximum over its children's current bounds and its own buffered-but-unemitted candidates.

                     root
                      │  advertised bound = max(child bounds, own buffer)
        ┌─────────────┼─────────────┐
     merge          merge         merge      each: bound = max over children
     ┌──┴──┐        ┌──┴──┐       ┌──┴──┐
    w    w        w     w       w     w      each worker returns:
                                               ordered run + highest unseen bound

  Before emitting the next candidate:
     choose the best buffered head using the full tie comparator
     fetch any unbuffered, unfinished child with no bound
       or an unseen score bound >= the candidate's score

Equality still blocks emission: an unseen result with the same score may have a lower public ID and belong earlier. Buffered child heads are compared using the full tie order. The merge repeats these checks as it emits the prefix.

For large K the coordinator requests bounded pages with bounds, raises the global threshold as results accumulate, and asks still-competitive segments for more. top_k accepts values from 1 through 100,000, subject to query.max_results. An explicit exact_prefix must equal top_k.

Equal scores use a deterministic tie order. The comparator is:

score descending
  → public ID ascending
    → catalog segment ID
      → ordinal

Public IDs can occur in multiple active segments. Result frames omit per-candidate segment identity to reduce frame size.

The topology solves it structurally. Segments are assigned to subtrees as disjoint contiguous ranges in ascending catalog segment-ID order, and children are ordered by range. A tie on (score, public ID) between two children therefore resolves by child order — which is exactly segment-ID order. The process-local multi-segment operator orders its children by the same key, so single-process and distributed execution use the same tie order.

The distributed qualification tests compare exact prefixes with an independent oracle: per-segment exhaustive gathering sorted by the canonical comparator, including tie order, at K = 1,000 and 10,000 on one-worker and two-worker shapes, and at 1,000 / 10,000 / 100,000 on a tiered four-worker, 64-segment shape. The distributed stream, the process-local multi-segment service, and the single-process serve path emit identical packed bytes over the same generation.

Past the proven prefix, a caller may accept an approximately ordered tail — but only with a declared error bound.

This requires the impact lane, which is off by default. When tail.impact_enabled is set, the builder transposes each term's ordinal-ordered postings into a score-ordered impact directory: by default 64 fixed BM25F saturation bands targeting 4,096 ordinals per run. Every directory entry records a closed, outward-rounded minimum and maximum contribution plus its run count. Entries are stored highest-score first, while ordinals within a run stay ascending for compact delta coding and forward-only seeks.

The lane writes two extra WavesDB records per unique term. Enable it for namespaces that serve banded requests.

A banded query reads the term dictionary, its impact directory, and then high-score runs until it has proved the requested exact prefix. Proof requires completing a band whose next band's maximum is strictly below the exact prefix threshold. After that boundary, the final tail band may stop early — and the outward-rounded width of that band is the advertised maximum ordering error.

Banded execution preserves these constraints:

• Higher bands are drained completely before lower bands.
• Exact BM25F scoring, liveness, and filters still apply to every selected ordinal.
• IDs are never fabricated.

Execution falls back to the exact path when the impact lane is absent, when the requested error is tighter than the stored bands, or when a query overrides a schema field weight. Exhausting all matches can also upgrade an approximate request to exact. Either way, the header and trailer report the ordering used, not the one requested.

Operators may set tail.impact_maximum_bands from 1 through 4,096 and tail.impact_target_documents to trade directory precision against block count.

Building the lane does not retain a hot term's complete postings in RAM: each band keeps at most 256 ordinals in memory, spills larger bands to reusable local files, and emits final runs at term completion through one shared spill descriptor. Peak impact RAM is bounded by band count rather than by document frequency.

An index may declare several vector fields, each pinning its own dimension, similarity, normalization rule, source-field recipe, and embedding profile. A query selects exactly one.

Retrieval is IVF-PQ with exact reranking: only the probed list blocks are read, and survivors are reranked from exact float32 rows stored beside each IVF list. IVF-PQ candidate generation is not globally exhaustive, so that path reports EXACT_WITHIN_CANDIDATES. --exhaustive scans every covered vector. Small segments can also use exhaustive retrieval automatically. Check the response for the effective candidate mode and exactness class.

Two fusion methods combine the two signals:

• weighted (the default) — a weighted sum, with --lexical-weight and --vector-weight.
• rrf — reciprocal-rank fusion.

--probes controls how many IVF lists are examined and --candidates how many survivors are exact-reranked; both default to the server's setting when zero. The effective values are reported in the header and trailer, so clients can inspect the retrieval parameters used.

Provider failures and model-identity mismatches return errors. The schema pins model identity, and the embedding fingerprint prevents a changed model package from reusing vectors or query embeddings from a different space.

The CLI table groups results by logical key in first-appearance order, showing the version with the greatest mutation_version; the version cell then reads … (n versions).

For generations with liveness metadata, grouping affects presentation only. Query selection already excludes superseded ordinals before top-K using the generation's liveness object, so a full requested prefix is returned. A generation written by an older publisher may have no liveness object; for that compatibility case the server retains a post-selection collapse, which can return fewer than top_k because it cannot pull replacements from the unselected tail.

--no-collapse prints every hit in rank order.

• The query lifecycle — where this proof sits in a request
• Segments and objects — the impact and vectors lanes
• Vectors and hybrid search — embedding profiles and provider modes
• Configuration reference — the tail.* and query.* keys