Skip to content

docs(sdk): add decision model guides - #1054

Draft
jpshackelford wants to merge 2 commits into
mainfrom
feat/decision-model-examples
Draft

jpshackelford wants to merge 2 commits into
mainfrom
feat/decision-model-examples

Conversation

@jpshackelford

@jpshackelford jpshackelford commented Oct 30, 2026 •

Copy link
Copy Markdown
Member
  • I have read and reviewed the documentation changes to the best of my ability.
  • If the change is significant, I have run the documentation site locally and confirmed it renders as expected.

Summary of changes

This PR documents DecisionModel (OpenHands/software-agent-sdk#6068) and its examples (OpenHands/software-agent-sdk#6339). The branch name matches the SDK branch, so the SDK's check-examples job reads these pages.

  • New sdk/guides/llm-decision-models.mdx (Decision Models):
    • the Noul, Choice and Score question types, and basic decide() / adecide() usage;
    • best practices from TypeSafe's guide: keep control flow in code, send structured state, ask many questions in one call, structure options that are easy to confuse, and route on uncertainty;
    • optional instructions (a Noul needs instructions or criteria) and Choice options without a description;
    • limits: 128 questions per call and 255 options per Choice through LiteLLM (going over raises LLMBadRequestError), 10 Score levels, and no Box or image input yet;
    • metrics (result.metrics is a snapshot of running totals) and the decision.<model> span;
    • examples 60 (triage an issue in one call) and 61 (gate an agent).
  • New sdk/guides/llm-decision-model-providers.mdx (Decision Model Providers):
    • the decision models available through the LiteLLM version the SDK uses, and the ones that need LiteLLM 1.106 or later;
    • the extra settings DecisionModel forwards to LiteLLM: custom_llm_provider, extra_headers, safety_identifier, and AWS credentials for Strands Decider on Bedrock AgentCore;
    • how to add a provider by subclassing DecisionModel and overriding send() / asend(), and what decide() still handles, including its answer checks;
    • why a generally available model should be contributed to LiteLLM instead;
    • example 62.
  • sdk/guides/observability.mdx: "What Gets Traced" now lists decision model calls. The embedded copy of example 27 picks up that example's new docstring.
  • docs.json: adds both pages to SDK → Guides → LLM Features.

Needs a decision before merge: the Decision Models page says DecisionModel is available in SDK v1.60.0 and later. That assumes #6068 merges before v1.60.0 is cut, since main is at 1.59.0. Please adjust if the release is different.

Validation

  • Example code blocks: I filled them with .github/scripts/sync_code_blocks.py, using AGENT_SDK_PATH pointed at the #6339 branch. I kept only the changes for the pages this PR owns. The script also rewrote 46 other pages because the docs and SDK main branches have drifted apart, and I reverted those changes.
  • SDK check-examples: I ran check_documented_examples.py against this branch in pull-request mode, as CI does. It reports "All examples are documented" for the four example files #6339 changes.
  • Links and rendering: npx mint broken-links reports no broken links. With mint dev, all three pages render, and both new pages appear in the LLM Features sidebar.
  • Snippets: I ran the basic-usage snippet and the structured Choice / NoulCriteria snippet against live Jev. The Basic Usage comments show the values the live run returned.
  • After SDK commit 70245fdc8 (review fixes): I updated both pages for the behavior it changed and re-synced examples 60 and 62. I took both snippets from the pages and ran them live again. I also checked each new statement against the SDK code and live calls. That covers optional fields on both routes, the limits raising LLMBadRequestError, wrong or extra answers raising LLMResponseError without a retry, and unmapped errors passing through unchanged. check-examples, mint broken-links and a mint dev render all pass.

This PR was created by an AI agent (OpenHands) on behalf of @jpshackelford.

@jpshackelford can click here to continue refining the PR

Add a Decision Models guide (question types, usage, best practices, limits,
metrics and tracing) and a Decision Model Providers guide (models supported
through LiteLLM, custom providers via send()/asend(), contributing upstream).
They embed examples 60, 61 and 62 from software-agent-sdk#6339. The
observability guide now lists decision model spans.

Co-authored-by: openhands <openhands@all-hands.dev>
@mintlify

mintlify Bot commented Oct 30, 2026 •

Copy link
Copy Markdown

Preview deployment for your docs. Learn more about Mintlify Previews.

Project Status Preview Updated
all-hands-ai 🟢 Ready View Preview Oct 30, 2026, 12:44 PM

💡 Tip: Enable Automations to automatically generate PRs for you.

- Decision Models: instructions are optional, Choice options may have no
  description, LiteLLM's 255-option limit, LLMBadRequestError for limits, and
  result.metrics as a snapshot of running totals.
- Decision Model Providers: the new LiteLLM settings (custom_llm_provider,
  extra_headers, safety_identifier, AWS credentials for Strands Decider), and
  stricter answer checks in decide().
- Sync examples 60 and 62.

Co-authored-by: openhands <openhands@all-hands.dev>

This branch was successfully deployed

1 active deployment
staging — ccc863ff Deployed Oct 30, 2026 by mintlify[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants