v3.10.19

## What ships

**Multi-field BM25 (subject 3× over body)** and an **opt-in type penalty** for
meta-commits. Both improvements come with a full ablation table — and the
ablation drove a real decision (type penalty defaults to OFF because it
*hurts* top-1 when multi-field BM25 is doing the heavy lifting).

## The cumulative win (3.10.17 cosine → 3.10.19)

| Metric (N=385, 10 queries) | 3.10.17 cosine | 3.10.18 hybrid | **3.10.19** | Δ since cosine |
|---|---:|---:|---:|---:|
| **Top-1 hit rate** | 0% | 50% | **80%** | **+80pp** |
| **Top-3 hit rate** | 0% | 70% | **80%** | **+80pp** |
| **MRR@3** | 0.000 | 0.583 | **0.800** | **+0.800** |
| Top-1 diversity | 100% | 80% | **100%** | 0pp (recovered) |
| Avg query latency | 28.7 ms | 40.6 ms | 39.0 ms | +10 ms |

## The ablation that drove the decisions

| Configuration | Top-1 | Top-3 | MRR@3 |
|---|:---:|:---:|:---:|
| Cosine baseline | 0/10 | 0/10 | 0.000 |
| Single-field BM25, no penalty (~3.10.18) | 5/10 | 7/10 | 0.583 |
| Single-field BM25 + type penalty 0.5 | 7/10 | 7/10 | 0.700 |
| **Multi-field BM25 3:1, no penalty (3.10.19 default)** | **8/10** | **8/10** | **0.800** |
| Multi-field BM25 3:1 + type penalty 0.5 | 7/10 | 8/10 | 0.750 |

Multi-field BM25 alone wins. Adding the type penalty *hurts* top-1 because
some real work commits start with `Merge feat/...` and get falsely demoted.
The penalty stays in the codebase as an opt-in for callers with different
commit conventions.

## What changed in code

1. **`multiFieldBM25()`** in `src/memory/hybrid-retrieval.ts` — treats
   pattern subject (`name`) and body (`content`) as separate fields with
   independent IDF distributions. Defaults: `subjectWeight=3.0, bodyWeight=1.0`.

2. **`typePenalty()` + `META_COMMIT_REGEX`** — exported but defaults to no-op
   (`typePenaltyFactor=1.0`). Callers can pass `0.5` for aggressive
   meta-commit suppression.

3. **`neural_patterns` MCP tool** — new params: `subjectWeight`,
   `bodyWeight`, `typePenaltyFactor`. Response shape unchanged.

4. **39 unit tests** in `__tests__/hybrid-retrieval.test.ts` (was 21 in
   3.10.18); covers ranking, weight collapse cases, regex coverage, factor
   bounds, undefined-name safety.

## Reproduce

```bash
git clone https://github.com/ruvnet/ruflo && cd ruflo
npm install && ( cd v3/@claude-flow/cli && npx tsc )

# Unit tests
( cd v3/@claude-flow/cli && npx vitest run __tests__/hybrid-retrieval.test.ts __tests__/pretrain-from-github.test.ts )

# Live A/B
cd v3/@claude-flow/cli
node scripts/pretrain-from-github.mjs
node scripts/benchmark-pretrained-retrieval.mjs              # 3.10.19 default → 80% top-1
HYBRID=0 node scripts/benchmark-pretrained-retrieval.mjs     # cosine baseline → 0% top-1
```

## Honest limits

- N=385, 10 queries is small. Relevance metric is regex-over-commit-subject —
  a labelled held-out corpus would tighten confidence intervals. Direction
  (0% → 80% top-1) is robust to noise.
- Subject:body 3:1 weight chosen by inspection, not grid-search. Future ADR
  could grid-search on a wider corpus.
- Type penalty regex is hand-curated for ruflo's conventions. Other repos
  with different conventions need their own regex — the function takes one
  as a parameter.

## What's next

- **Cross-encoder reranker** (3.11.0, MINOR — new dep): paper-proven path
  for closing the remaining 80% → 100% top-1 gap
- **Learned distiller** (paper's 11× compression): #2241 round-D
- **Grid-search for subject/body weights** on a wider held-out corpus

## Install

```bash
npx ruflo@3.10.19    # latest / alpha / v3alpha all aligned
```

Full ADR: [v3/docs/adr/ADR-079-multifield-bm25-and-type-penalty.md](v3/docs/adr/ADR-079-multifield-bm25-and-type-penalty.md)