docs(sdk): add decision model guides

docs.json

@@ -279,7 +279,9 @@
                   "sdk/guides/llm-error-handling",
                   "sdk/guides/llm-fallback",
                   "sdk/guides/llm-profile-store",
-                  "sdk/guides/llm-task-routing"
+                  "sdk/guides/llm-task-routing",
+                  "sdk/guides/llm-decision-models",
+                  "sdk/guides/llm-decision-model-providers"
                 ]
               },
               {

sdk/guides/llm-decision-model-providers.mdx

@@ -0,0 +1,292 @@
+---
+title: Decision Model Providers
+description: Use the decision models LiteLLM supports, or add your own provider by subclassing DecisionModel.
+---
+
+> A ready-to-run example is available [here](#ready-to-run-example)!
+
+[`DecisionModel`](/sdk/guides/llm-decision-models) takes the SDK's question types and returns the SDK's answer types. It can reach a model in two ways:
+
+1. **Through LiteLLM**, the default. Set `model` to a LiteLLM model string.
+2. **Through your own code.** Subclass `DecisionModel` and override `send()` and `asend()`.
+
+Retries, error mapping, metrics and the `decision.<model>` span work the same way for both routes.
+
+## Models Supported Through LiteLLM
+
+The SDK uses LiteLLM `>=1.104.3,<1.106`. With those releases, these decision models work through `DecisionModel`:
+
+| Provider | Example `model` | Credentials |
+|---|---|---|
+| TypeSafe Jev | `typesafe/jev-latest` | `TYPESAFE_API_KEY` |
+| OpenAI | `openai/gpt-6-luna` | `OPENAI_API_KEY` |
+| OpenRouter | `openrouter/typesafe/jev-1.14` | `OPENROUTER_API_KEY` |
+| Perplexity | `perplexity/pplx-decider-v1-27b` | `PERPLEXITYAI_API_KEY` or `PERPLEXITY_API_KEY` |
+| Cloudflare Clef | `cloudflare/clef` or `cloudflare/clef-flash` | `CLOUDFLARE_API_KEY` and `CLOUDFLARE_ACCOUNT_ID` |
+| Microsoft Foundry | `azure_ai/<deployment>` | `AZURE_AI_API_KEY` and `AZURE_AI_API_BASE` |
+| Strands Decider (self-hosted) | `strands_decider/<model>` | `STRANDS_DECIDER_API_BASE`, plus AWS credentials when it is a Bedrock AgentCore runtime ARN |
+| vLLM (self-hosted) | `hosted_vllm/<model>` | `HOSTED_VLLM_API_BASE` |
+
+Pass `api_key` and `base_url` to `DecisionModel`, or set the environment variables above and LiteLLM picks them up. `DecisionModel` also forwards these settings to LiteLLM:
+
+- **`custom_llm_provider`** - The provider, when the model string has no provider prefix.
+- **`extra_headers`** - HTTP headers to send to the provider.
+- **`safety_identifier`** - A stable end-user identifier for abuse monitoring. OpenAI supports it.
+- **`aws_access_key_id`**, **`aws_secret_access_key`**, **`aws_session_token`**, **`aws_region_name`**, **`aws_profile_name`**, **`aws_role_name`** and **`aws_session_name`** - AWS credentials for Strands Decider on Bedrock AgentCore. The secret ones are stored as secrets, like `api_key`.
+
+Databricks, Fireworks AI, Laya, Bespoke Nimble, image input and `Box` questions need LiteLLM 1.106 or later. They become available when the SDK moves to that release. For the current list and provider details, see LiteLLM's [decisions documentation](https://docs.litellm.ai/docs/decisions).
+
+## Add Your Own Provider
+
+To use a model that LiteLLM does not support, subclass `DecisionModel` and override `send()`. Override `asend()` too if you call `adecide()`.
+
+```python icon="python" wrap
+from openhands.sdk import DecisionModel
+from openhands.sdk.llm import DecisionReply, DecisionUsage, NoulAnswer
+
+
+class AcmeDecisionModel(DecisionModel):
+    def send(self, state, questions) -> DecisionReply:
+        payload = call_acme_api(state, questions)  # your provider's own API
+        return DecisionReply(
+            answers={
+                name: NoulAnswer(probability=p) for name, p in payload["yes"].items()
+            },
+            model=payload["model"],
+            usage=DecisionUsage(input_tokens=payload["tokens"]),
+        )
+```
+
+Each call to `send()` makes one provider request. It receives the SDK question objects, so `question.model_dump(exclude_none=True)` gives the same JSON that LiteLLM sends. It returns a `DecisionReply` with:
+
+- **`answers`** - One answer per question name, of the matching type: `NoulAnswer` for a `Noul`, `ChoiceAnswer` for a `Choice`, `ScoreAnswer` for a `Score`. If a question has no answer, an answer has the wrong type, or there is an answer to a question nobody asked, `decide()` raises `LLMResponseError` without retrying.
+- **`model`** - The versioned model that answered. It becomes `result.model` and the span's `gen_ai.response.model`.
+- **`usage`** and **`cost`** - Leave them as `None` if the provider does not report them.
+- **`raw_response`** - The provider's own payload, kept on the result for debugging.
+
+`decide()` calls `send()` once per attempt and handles the rest:
+
+- **Retries.** Raise `LLMRateLimitError`, `LLMTimeoutError` or `LLMServiceUnavailableError` for failures worth retrying. Any other exception is raised to the caller unchanged, without a retry.
+- **Metrics.** Tokens, cost and latency go into `result.metrics` and `decision_model.metrics`.
+- **Tracing.** Each attempt produces a `decision.<model>` span, as on the LiteLLM route.
+
+LiteLLM's limits of 128 questions per call and 255 options per `Choice` do not apply to a custom `send()`.
+
+## Prefer Contributing to LiteLLM
+
+Write a subclass for a model that only you can reach, such as one you host privately or an internal service.
+
+For a generally available model, contribute a decision provider to [LiteLLM](https://github.com/BerriAI/litellm) instead. Existing providers subclass `BaseDecisionsConfig` in `litellm/llms/<provider>/decisions/transformation.py`. Once LiteLLM ships the provider, every LiteLLM user can use it. SDK users get it through the default route when the SDK takes that LiteLLM release, without any SDK change.
+
+## Ready-to-run Example
+
+This example lists the supported models. Then it adds TypeSafe's HTTP API as a custom provider and asks it the same questions as the LiteLLM route. LiteLLM already supports TypeSafe, so the two routes can be compared: the code that reads the answers is the same for both.
+
+<Note>
+This example is available on GitHub: [examples/01_standalone_sdk/62_decision_model_custom_provider.py](https://github.com/OpenHands/software-agent-sdk/blob/main/examples/01_standalone_sdk/62_decision_model_custom_provider.py)
+</Note>
+
+```python icon="python" expandable examples/01_standalone_sdk/62_decision_model_custom_provider.py
+"""Reach a decision model through LiteLLM, or through your own code.
+
+``DecisionModel`` takes the SDK's question types (``Noul``, ``Choice``,
+``Score``) and returns the SDK's answer types. Where it sends the questions is
+up to you:
+
+1. Through LiteLLM, the default. Set ``model`` to a LiteLLM model string. With
+   the LiteLLM release the SDK uses (1.105), that covers TypeSafe Jev
+   (``typesafe/jev-latest``), OpenAI (``openai/gpt-6-luna``), OpenRouter,
+   Perplexity, Cloudflare Clef, Microsoft Foundry (``azure_ai/<deployment>``),
+   Strands Decider and self-hosted vLLM (``hosted_vllm/<model>``). Databricks,
+   Fireworks AI, Laya, Bespoke Nimble and ``Box`` questions need LiteLLM 1.106
+   or later. See https://docs.litellm.ai/docs/decisions for the current list.
+
+2. Through your own code. Subclass ``DecisionModel`` and override ``send()``
+   (and ``asend()`` if you call ``adecide()``). Each method makes one request
+   and returns a ``DecisionReply`` built from SDK types. ``decide()`` still
+   adds retries, error mapping, metrics and the ``decision.<model>`` span, and
+   checks that every question got one answer of the right type.
+
+Use route 2 for a model only you can reach, such as one you host privately or
+an internal service. For a generally available model, contribute a provider to
+LiteLLM (https://github.com/BerriAI/litellm) instead. Every LiteLLM user gets
+it, and SDK users get it through route 1 when the SDK takes that LiteLLM
+release.
+
+This example writes route 2 for TypeSafe's HTTP API. LiteLLM already supports
+TypeSafe, so the example can ask both routes the same questions and show that
+the code that reads the answers does not change.
+"""
+
+import asyncio
+import os
+from collections.abc import Mapping
+from typing import Any
+
+import httpx
+from pydantic import SecretStr
+
+from openhands.sdk import DecisionModel, DecisionResult
+from openhands.sdk.llm import (
+    Choice,
+    ChoiceAnswer,
+    DecisionReply,
+    DecisionUsage,
+    Noul,
+    NoulAnswer,
+    Question,
+    Score,
+    ScoreAnswer,
+)
+from openhands.sdk.llm.exceptions import (
+    LLMRateLimitError,
+    LLMResponseError,
+    LLMServiceUnavailableError,
+    LLMTimeoutError,
+)
+
+
+class TypeSafeHTTPDecisionModel(DecisionModel):
+    """Call TypeSafe's ``/v1/systemone`` API directly, without LiteLLM."""
+
+    def send(self, state: Any, questions: Mapping[str, Question]) -> DecisionReply:
+        try:
+            response = httpx.post(**self._request(state, questions))
+        except httpx.TimeoutException as e:
+            raise LLMTimeoutError(str(e)) from e
+        return _reply(response)
+
+    async def asend(
+        self, state: Any, questions: Mapping[str, Question]
+    ) -> DecisionReply:
+        try:
+            async with httpx.AsyncClient() as client:
+                response = await client.post(**self._request(state, questions))
+        except httpx.TimeoutException as e:
+            raise LLMTimeoutError(str(e)) from e
+        return _reply(response)
+
+    def _request(self, state: Any, questions: Mapping[str, Question]) -> dict:
+        api_key = self.api_key
+        assert isinstance(api_key, SecretStr), "Set api_key."
+        return {
+            "url": f"{self.base_url or 'https://api.typesafe.ai'}/v1/systemone",
+            "headers": {"Authorization": f"Bearer {api_key.get_secret_value()}"},
+            "json": {
+                "model": self.model,
+                "state": state,
+                # exclude_none drops optional fields left unset, such as
+                # Noul.criteria.
+                "questions": {
+                    name: q.model_dump(exclude_none=True)
+                    for name, q in questions.items()
+                },
+            },
+            "timeout": self.timeout,
+        }
+
+
+def _reply(response: httpx.Response) -> DecisionReply:
+    # DecisionModel retries these SDK errors; any other error is raised as is.
+    if response.status_code == 429:
+        raise LLMRateLimitError(response.text)
+    if response.status_code >= 500:
+        raise LLMServiceUnavailableError(response.text)
+    response.raise_for_status()
+    body = response.json()
+    return DecisionReply(
+        answers={name: _answer(a) for name, a in body["answers"].items()},
+        model=body["model"],
+        usage=DecisionUsage(
+            input_tokens=body["usage"]["input_tokens"],
+            output_tokens=body["usage"]["output_tokens"],
+        ),
+        # TypeSafe's response has no price, so cost stays unset. Set it if
+        # your provider reports one or you keep a price table.
+        raw_response=body,
+    )
+
+
+def _answer(answer: dict) -> NoulAnswer | ChoiceAnswer | ScoreAnswer:
+    match answer["type"]:
+        case "noul":
+            return NoulAnswer(probability=answer["noul"])
+        case "choice":
+            return ChoiceAnswer(
+                choice=answer["choice"],
+                confidence=answer["confidence"],
+                probabilities=answer["probabilities"],
+            )
+        case "score":
+            return ScoreAnswer(
+                score=answer["score"],
+                confidence=answer["confidence"],
+                probabilities=answer["probabilities"],
+                legend=answer["legend"],
+            )
+    raise LLMResponseError(f"Unsupported answer type: {answer['type']!r}")
+
+
+api_key = os.getenv("JEV_API_KEY")
+assert api_key, "Set JEV_API_KEY to a TypeSafe API key."
+
+state = {
+    "issue": {
+        "title": "Export to CSV drops the header row",
+        "body": "Since 2.3, exported CSV files have no header row. "
+        "Steps: open a table, click Export, choose CSV. Expected a header row.",
+    }
+}
+questions: dict[str, Question] = {
+    "is_bug": Noul(instructions="Does `issue` report broken behavior?"),
+    "area": Choice(
+        instructions="Which part of the product does `issue` concern?",
+        criteria={
+            "import_export": "Reading or writing files",
+            "ui": "Screens, buttons and layout",
+            "other": "Anything else",
+        },
+    ),
+    "severity": Score(
+        instructions="How badly does `issue` affect the user's work?",
+        criteria=["Cosmetic", "Workaround available", "Work blocked"],
+    ),
+}
+
+
+def summarize(result: DecisionResult) -> str:
+    """Read the answers. This code does not know which route answered."""
+    severity = result.score("severity")
+    return (
+        f"is_bug={result.noul('is_bug').probability:.2f} "
+        f"area={result.choice('area').choice} "
+        f"severity={severity.score:.2f} (confidence {severity.confidence:.2f}) "
+        f"model={result.model}"
+    )
+
+
+direct = TypeSafeHTTPDecisionModel(model="jev-latest", api_key=api_key)
+via_litellm = DecisionModel(model="typesafe/jev-latest", api_key=api_key)
+
+print("direct HTTP:", summarize(direct.decide(state, questions)))
+print("direct, async:", summarize(asyncio.run(direct.adecide(state, questions))))
+print("LiteLLM:", summarize(via_litellm.decide(state, questions)))
+
+usage = direct.metrics.accumulated_token_usage
+assert usage is not None
+print(f"\ndirect route tokens over 2 calls: {usage.prompt_tokens} in")
+print(f"LiteLLM route cost: ${via_litellm.metrics.accumulated_cost:.6f}")
+print(f"EXAMPLE_COST: {via_litellm.metrics.accumulated_cost}")
+```
+
+```bash Running the Example
+export JEV_API_KEY="your-typesafe-api-key"
+cd software-agent-sdk
+uv run python examples/01_standalone_sdk/62_decision_model_custom_provider.py
+```
+
+## Next Steps
+
+- **[Decision Models](/sdk/guides/llm-decision-models)** - Question types, usage and best practices
+- **[Observability & Tracing](/sdk/guides/observability)** - Send decision spans to Laminar or any OTLP backend
+- **[Exception Handling](/sdk/guides/llm-error-handling)** - The SDK's exception types