Local development

There are two local setups, and the choice between them is about what you need to observe.

One binary. yolosearch-local runs the complete data plane plus the admin UI in a single process, with no flags required. Use it for schema work, ingest, queries, and anything that does not depend on more than one node.

A Docker Swarm fleet. just swarm-up deploys a coordinator, two workers with separate caches, a writable server, an aggregator, a compactor, MinIO, and the admin console. Use it to exercise worker placement, remote execution, and cross-node failures.

Testing placement, remote workers, and cross-node failure requires separate node processes. Swarm and Kubernetes provide deployment setups for these tests.

Go 1.27 or newer, Node.js and npm, just, and zsh. The Swarm path adds Docker Engine or Docker Desktop with a running Linux-container daemon.

From the repository root:

just build
./bin/yolosearch-local

Or compile and run in one step with just local. Once the ready block appears, open http://127.0.0.1:8787:

yolosearch-local: ready
  data:  127.0.0.1:9500
  admin: http://127.0.0.1:8787
  root:  <cwd>/yolosearch-local-data
  objects: <cwd>/yolosearch-local-data/objects
  cache:   <cwd>/yolosearch-local-data/cache
  ingest:  <cwd>/yolosearch-local-data/ingest
  models:  <cwd>/yolosearch-local-data/models

The native gRPC API is 127.0.0.1:9500, which is the address the ordinary yolosearch client already dials. In another terminal:

./bin/yolosearch schema apply deploy/swarm/articles.proto
./bin/yolosearch push articles deploy/swarm/articles.jsonl
./bin/yolosearch search articles search --fields title,url
./bin/yolosearch stats articles

One OS process containing the production direct multi-namespace data path, the builder, the publisher, the follower and cache, the idle-aware compactor, the optional GC scheduler, the embedding runtime, the native gRPC server, and the admin HTTP/Connect server. The admin bridge still calls the public gRPC API; there is no local-only search adapter, so what the console does here is what it does everywhere.

The default persistent layout:

yolosearch-local-data/
├── config.yaml   optional configuration
├── objects/      authoritative schemas, segments, commits, and catalogs
├── ingest/       durable spool and build scratch
├── cache/        disposable segment/block cache
└── models/       checksum-addressed internal embedding models

Data persists across restarts; restarting the same command reopens the same schemas and documents. Ctrl-C or SIGTERM drains both endpoints and joins every background goroutine. Logs go to the inherited stdout and stderr only; the process creates no hidden log file.

Settings work exactly as configuration describes — flags, YOLOSEARCH_* variables, yolosearch-local-data/config.yaml, or the runtime config API. Only server.data_dir has a local command default, so every explicit setting wins. Secrets remain environment-only.

./bin/yolosearch-local --data-dir ./scratch/search \
  --listen '[::1]:9500' --admin-listen '[::1]:8787'

Local mode refuses S3 as authoritative storage. Internal model packages live under models/; external HTTP and gRPC embedders use the same embedding.* settings as yolosearch serve, so vector search works here. A provider being unavailable affects the requests that need it and does not prevent lexical-only startup.

The idle-aware compaction scheduler is available in-process: set compaction.enabled=true when the local process should run it. Safe GC stays off unless gc.sweep_enabled=true, and enabling it retains the existing dry-run, quarantine, and fresh reachability proof.

Stop the process and run this guarded, path-resolved reset from the repository root:

local_root="$(realpath ./yolosearch-local-data)"
test "$local_root" = "$(pwd -P)/yolosearch-local-data" && rm -rf -- "$local_root"

--dir turns a directory into a self-contained installation and runs it:

./bin/yolosearch-local --dir ~/search/articles
# yolosearch-local: created installation /home/you/search/articles
# config: loaded /home/you/search/articles/yolosearch.yml (--config)
# yolosearch-local: ready

It creates the directory and objects/, cache/, ingest/, and models/ beneath it — the same names --data-dir derives — so the layout exists and is inspectable before anything starts. When the directory holds no configuration file yet (yolosearch.yml, yolosearch.yaml, or config.yaml), it writes yolosearch.yml with the object backend, the four directories, the listen address, and the log level. A file that is already there is never rewritten: --dir means "run this installation", not "reset it".

The generated file spells its directories relative to itself, and a relative path in a configuration file resolves against that file's directory. So the installation moves:

mv ~/search/articles /media/stick/articles
./bin/yolosearch-local --dir /media/stick/articles   # nothing to edit

--dir and --data-dir both name a root, so giving both is an error. An explicit --config still wins over the installation's own file, and every flag still outranks both.

--help does not print every catalog flag. It opens on an ESSENTIALS block — the dozen flags that decide where the data lives, what is listening, and what is switched on — followed by an index of configuration topics and how many flags each holds:

./bin/yolosearch-local --help            # essentials and the topic index
./bin/yolosearch-local --help cache      # one topic, every flag in it
./bin/yolosearch-local --help all        # every topic

Each topic screen shows the flag, its first sentence of documentation, its constraint, its default, and the catalog key to write in yolosearch.yml. The full paragraph for every setting is in yolosearch manual and the CLI reference. The screen wraps to the terminal's width between 60 and 120 columns and is colored when it is going to one — NO_COLOR, TERM=dumb, a pipe, or --color never each turn that off, and --color always forces it on through a pager.

Six flags are specific to this binary:

If port 8787 or 9500 is taken, the process reports the conflicting address and exits. Choose others with --admin-listen and --listen, using bracketed IPv6 literals where needed.

just swarm-up

That is the whole bootstrap. The first run downloads dependencies and container base images, builds the current checkout, initializes a single-manager Swarm if Docker is not already in Swarm mode, deploys the fleet, creates an articles namespace, indexes six sample documents, and verifies a distributed query.

When it returns:

./bin/yolosearch --server 127.0.0.1:9600 \
  search articles 'title:search' --fields title,url

The MinIO login is yolosearchdev / yolosearch-dev-only-secret. These are public credentials for local development.

The single-process write path is on 9500 and the distributed read path is on 9600. All query workers read the same authoritative object catalog and do not own shards permanently. The aggregator improves placement but is not a correctness dependency — the coordinator can fall back to its explicit worker list. See architecture.

Application roots run read-only as UID/GID 65532. MinIO data uses a normal named volume; builder scratch and query caches use bounded tmpfs-backed named volumes, so they may be cold after a stack stop or a Docker restart.

just swarm-status           # services, task placement, endpoints
just swarm-logs             # last 200 timestamped lines from every service
just swarm-logs coordinator # one allowlisted service
just swarm-down             # stop services; preserve MinIO object data
just swarm-up               # rebuild current source and resume or reseed

swarm-up is idempotent. Reapplying the same schema is a no-op and pushing the same six keys creates newer versions that normal searches collapse by key. Existing services are force-refreshed, so rebuilding a local image under the same development tag rolls the tasks.

To delete the local stack and its data:

just swarm-reset

swarm-reset removes the yolosearch-dev stack and only its six exact named volumes. It does not remove images, unrelated stacks, networks, or other volumes, and it never runs docker swarm leave. Run just swarm-up afterward for a fresh namespace and cold caches.

No lifecycle driver detaches a helper shell, log tail, or find process. Every readiness loop is finite and attached to the command. If startup fails, the stack is left in place for diagnosis instead of being erased.

./bin/yolosearch --server 127.0.0.1:9500 schema apply deploy/swarm/articles.proto
./bin/yolosearch --server 127.0.0.1:9500 push articles deploy/swarm/articles.jsonl

./bin/yolosearch --server 127.0.0.1:9600 \
  search articles 'search AND body:cache' --fields title,site,url
./bin/yolosearch --server 127.0.0.1:9600 \
  search articles '[word="cold"] []{0,3} [word="cache"]' --dialect cqp --fields title,url

The workers and coordinator follow the latest catalog generation every second, so a published write becomes searchable through port 9600 without restarting the fleet.

Corpus and fleet statistics:

./bin/yolosearch --server 127.0.0.1:9600 stats articles
./bin/yolosearch --server 127.0.0.1:9600 stats --node
./bin/yolosearch --server 127.0.0.1:9600 stats --fleet

The sandbox's non-secret settings are in deploy/swarm/stack.yml. Change cache budgets, builder memory and concurrency, seal thresholds, follower cadence, storage mode, or compaction thresholds there, then rerun just swarm-up. The stack disables configuration discovery so a host's files cannot leak into containers, and its flags override shared environment settings.

Startup-scoped settings require editing the stack and rolling the service; node-runtime settings can also use the ephemeral overlay:

./bin/yolosearch --server 127.0.0.1:9500 config get ingest.seal_age
./bin/yolosearch --server 127.0.0.1:9500 config set ingest.seal_age 5s

For the direct-server split, compile and run serve on its own:

just build
./bin/yolosearch serve --data-dir ./ys --listen 127.0.0.1:9500

Then, in another terminal, the same schema, push, and search commands. This form has no co-located admin UI. Prefer yolosearch-local unless you specifically need to exercise yolosearch serve alone. Stop it with Ctrl-C; nothing is detached. ./ys holds the filesystem object store, ingest scratch, cache, and derived config path.

Common contributor commands:

just test-unit       # deterministic Go tests
just ui-test         # React tests and TypeScript checking
just bench-short     # quick diagnostic microbenchmark
just ci              # infrastructure-free CI gate
just gate            # CI plus compatibility and integration proofs

Laptop benchmark numbers are diagnostic. Reportable resource and performance receipts run through the repository's remote benchmark commands.

failed to connect to the docker API. The CLI exists but the daemon is not running, or the active context is wrong:

docker context show
docker info

Swarm is locked, pending, or belongs to another cluster. swarm-up initializes only an inactive daemon and reuses only an active one. It will not repair or leave another Swarm:

docker info --format '{{.Swarm.LocalNodeState}}'
docker node ls

Neither swarm-down nor swarm-reset runs docker swarm leave.

A public port is in use. Ports 8787, 9001, 9500, and 9600 must be available to Swarm. Change the corresponding published value in deploy/swarm/stack.yml and use the new address manually — the lifecycle acceptance probes use the default ports, so changing them also means changing the constants in tools/dev/swarm.go.

A service keeps restarting.

just swarm-status
docker service ps --no-trunc yolosearch-dev_server
just swarm-logs server
just swarm-logs minio-init

Catalog-bound workers, the coordinator, and the compactor can restart briefly before MinIO and the sample namespace exist. A persistent restart loop usually means the bucket initializer failed, object credentials disagree, a cache mount is not writable, or the local image architecture is wrong.

Search says the index or generation is unavailable. Confirm seed publication on the write endpoint, then allow one follower poll:

./bin/yolosearch --server 127.0.0.1:9500 stats articles
./bin/yolosearch --server 127.0.0.1:9600 stats articles
just swarm-logs coordinator

A rebuilt service still runs old code. Use just swarm-up, not a raw docker stack deploy. The lifecycle command force-refreshes existing application services after rebuilding the fixed local tags; confirm task creation times with just swarm-status.

Images fail on another Swarm node. Local image tags exist only in one Docker daemon. Push immutable multi-platform images to a registry, authenticate every node, set YOLOSEARCH_SWARM_IMAGE and YOLOSEARCH_SWARM_ADMIN_IMAGE, and deploy with --with-registry-auth. Also replace the node-local volume strategy before treating the deployment as durable — the default placement constrains tasks to manager nodes and every named volume uses node-local storage, which is what makes this a developer topology rather than a highly available one.

Full clean start.

just swarm-reset
just swarm-up

• Quick start — the shortest path to a served query.
• The admin console — what the UI on 8787 can do.
• Configuration — precedence, reload, and scope.
• Containers and Kubernetes — the deployment forms beyond a sandbox.