[manykarim]

Promotes the first OpenRouter baselines (deepseek/deepseek-v4.1-flash) for rf-api, rf-core, rf-mobile and rf-web, under plugins/<plugin>/evals/baselines/openrouter/deepseek-v4.1-flash/ (openrouter-ci-eval-tier tasks 6.1–6.2). Stacked on #2; merge #2 first.

Source: run 42993062842: all tiers, both arms, N=6, turn-limit factor 1.5, all trigger sets on both splits, on 42be259. Files are the candidate-baseline-* artifacts, unchanged.

rf-desktop is not in this PR. Its PlatynUI tasks stopped at the 1.5 limit in 3–5 of 6 sessions and swung between 0.33 and 0.83 across two runs, so #2 now gives it factor 2.0 (007a7ab); its baseline is being recorded in run 43096210688 and will be added here.

Review

Pass rate with the skill (T) vs. without (B), N=6:

Plugin	Notable tasks
rf-core	skill lift on adv-language-force-tags (T 1.00 / B 0.00), adv-language-rf71-typed (0.83 / 0.17), adv-setup-system-pip (0.83 / 0.33)
rf-web	lift on adv-browser-deprecated-wait (0.83 / 0.33)
rf-mobile	adv-appium-deprecated-visibility 0.67 / 0.50
rf-api	every task 1.00 in both arms (no measurable lift on DeepSeek)

• Flaky: adv-legacy-syntax and adv-selenium-nonexistent-kw have T 0.83 (one failed session of 6) against B 1.00; expect noise in the gate there.
• Turn-limit stops: common in the treatment arm (for example 6/6 on narrow-language-env-varfiles-01, 4/6 on adv-restinstance-expectation-leak), but those sessions still pass, so 1.5 stays for these four plugins.
• Triggers: recall about 1.0, precision about 0.5–0.8; DeepSeek loads skills on many negative queries. The baseline records that level.
• No incomplete runs.

Spend (key usage before/after each job)

Plugin	Key spend	Harness estimate (list prices)
rf-api	$1.41	$2.23
rf-core	$3.47	$5.25
rf-desktop	$0.78	$1.18
rf-mobile	$0.94	$1.49
rf-web	$1.73	$2.43
Total	$8.34	$12.58

The list-price estimate is 37–60% above the charged amount, so the cost caps err on the safe side.

Verification

After #2 merges, the next PR eval run should report the gate as pass (not rebaseline-needed) for these four plugins.

🤖 Generated with Claude Code

https://claude.ai/code/session_01FHEmzL3ErAhGQykKWDx2wP