Explanations
Deeper dives into how chops-search works under the hood, and why it's built the way it is.
- The model is a file you read offsets from — Why a model2vec lookup table can be range-fetched row by row, and the four loading disciplines that keep a query under a kilobyte.
- How ranking works — BM25F over pre-WordPiece tokens, cosine over embedded chunks, confidence-weighted rank fusion, and the two gates that make empty results possible.
- One tokenizer, enforced structurally — Why the engine is one Rust core compiled twice, and the parity tests that pin the tokenizer and embeddings to the reference implementation.
- When it breaks, it says so — The failure modes partial loading creates, and why chops-search degrades to keyword-only or eager loading instead of returning plausible garbage.