Explanations

Deeper dives into how chops-search works under the hood, and why it's built the way it is.

  • The model is a file you read offsets from — Why a model2vec lookup table can be range-fetched row by row, and the four loading disciplines that keep a query under a kilobyte.
  • How ranking works — BM25F over pre-WordPiece tokens, cosine over embedded chunks, confidence-weighted rank fusion, and the two gates that make empty results possible.
  • One tokenizer, enforced structurally — Why the engine is one Rust core compiled twice, and the parity tests that pin the tokenizer and embeddings to the reference implementation.
  • When it breaks, it says so — The failure modes partial loading creates, and why chops-search degrades to keyword-only or eager loading instead of returning plausible garbage.