diff --git a/docs/backends.md b/docs/backends.md index 3326036..2669d85 100644 --- a/docs/backends.md +++ b/docs/backends.md @@ -390,7 +390,10 @@ analysis at k1=0.9/b=0.4. Under the reference's own conventions — `--analyzer k1=0.9/b=0.4, query-term frequency, reference index statistics, score rounding, omitted saturation constant, single precision, depth 1,000 — the in-memory row reads **0.236** against 0.2361 and the paper's 0.236: the figure is reproduced. This engine cannot use `uax29`; with its closest -configuration (`english`, k1=0.9/b=0.4) it reads **0.2291**. The pattern is SciFact's rather than a scientific-corpus accident: +configuration (`english`, k1=0.9/b=0.4) it reads **0.2291**. The gap was decomposed: a PostgreSQL +configuration with the reference's 33-word stop list and Snowball reads 0.2196, so the stop list is +not the cause (the longer stock list helps) and the remainder is the stemmer and the parser — not +replaceable on stock PostgreSQL. The pattern is SciFact's rather than a scientific-corpus accident: the ungated pool truncates a coverage-ranked candidate set, and the gate's idf-selected one does not. diff --git a/docs/evaluation.md b/docs/evaluation.md index 3633fb5..8e5add0 100644 --- a/docs/evaluation.md +++ b/docs/evaluation.md @@ -531,7 +531,10 @@ same qrels (harness defaults, exponential gains, 10 retrieved at 10): the published **0.2361** (the paper's 0.236): the figure is reproduced at the precision it was published at. The PostgreSQL row cannot use `uax29`; with its closest configuration (`english`, k1=0.9/b=0.4) it reads **0.2291**, the same ~0.007 analysis gap the other corpora show, not a gate - cost. The gate's direction is SciFact's: + cost. Where the gap sits was measured rather than assumed: a PostgreSQL configuration carrying the + reference's 33-word stop list and its Snowball stemmer reads **0.2196**, so the stop list is not + the cause — the longer stock list helps here — and what remains is the stemmer and the parser, + which stock PostgreSQL cannot replace without an extension. The gate's direction is SciFact's: the ungated candidates are chosen by coverage and truncated, the gated ones by idf. - **`ts_rank_cd`'s and ParadeDB's scores are PostgreSQL-native and not numerically comparable** with `Bm25Scorer`; the stock-PostgreSQL BM25 row's are `Bm25Scorer`'s and are. The metric