openrouter-stream

A small streaming client for OpenRouter chat completions, built around the things that actually broke while running an LLM product on it.

Zero dependencies (global fetch, Node ≥ 18), TypeScript types included.

Problem we hit	What this does
Some models refuse to run with reasoning disabled (Reasoning is mandatory for this endpoint and cannot be disabled – seen on several GLM, Qwen and Gemini endpoints)	Sends reasoning: { enabled: false } by default; on that exact 400 it retries once with { effort: "low" } instead of failing
You need the real cost of each call	onUsage gives model, prompt / completion / cached tokens and dollars. Cost is null when the provider reports none — never an invented 0
The same large system prompt is re-sent on every call	Optional Anthropic prompt caching (cache_control), "auto" = only for anthropic/* (other providers cache by themselves and would not understand the marker)
A client that disconnects should stop paying for the upstream call	signal is forwarded to fetch
A provider error is not safe to show users (a 402 contains a console URL with your key id)	classifyError(err) → "credits" | "rate_limit" | "unavailable" | "other". Show your own message; alert yourself on "credits"
One provider is rate-limited	Several models → OpenRouter's models fallback list

import { streamChat, complete, classifyError, type Usage } from "openrouter-stream";

let usage: Usage | undefined;
try {
  for await (const chunk of streamChat({
    apiKey: process.env.OPENROUTER_API_KEY!,
    models: ["deepseek/deepseek-v4.1-flash", "openai/gpt-5.6-luna"], // first, then fallbacks
    system: "You edit code.",
    user: "Change the price to 6500.",
    maxTokens: 2000,
    onUsage: (u) => (usage = u), // { model, promptTokens, completionTokens, cachedTokens, costUsd, durationMs }
  })) {
    process.stdout.write(chunk);
  }
} catch (err) {
  switch (classifyError(err)) {
    case "credits":     alertOperator(err); showUser("Service temporarily unavailable."); break;
    case "rate_limit":  showUser("Busy, try again in a moment."); break;
    case "unavailable": showUser("The engine is not responding."); break;
    default:            showUser("Something went wrong.");
  }
}

npm install github:hypesecret/openrouter-stream

Options worth knowing

reasoning (override the default), cacheSystem ("auto" | true | false), title / referer (dashboard attribution), extraBody (temperature, provider preferences…), baseUrl and fetch (proxies, tests).

What this is not

• Not a router: it does not choose models for you. (Choosing a model per task by measured cost per successful task is a separate problem.)
• Not a retry framework: the only automatic retry is the reasoning one. Retrying on 429/5xx, backoff, and budget limits are yours to add — classifyError tells you what happened.
• Only the chat-completions streaming path is covered.
• Model ids and prices change constantly; nothing here hard-codes either.

Tested

17 unit tests with a simulated fetch (no network). Each behavior in the table above has a test whose failure was verified by deliberately breaking the code (mutation check). It has not been run against every OpenRouter model: the reasoning retry, for instance, was validated on the endpoints where we saw the error.

Résumé en français

Client de streaming OpenRouter sans dépendance : gère les modèles qui imposent la réflexion (nouvel essai automatique), relève le coût réel par appel (jamais un faux 0), met en cache le prompt système (Anthropic), transmet l'annulation, et classe les erreurs sans jamais exposer les détails du fournisseur (une erreur 402 contient l'URL de la console et l'identifiant de la clé).

MIT © hypesecret