Repository navigation
docs(sdk): add decision model guides - #1054
Draft
jpshackelford wants to merge 2 commits into
Draft
jpshackelford wants to merge 2 commits into
jpshackelford wants to merge 2 commits into
Conversation
Add a Decision Models guide (question types, usage, best practices, limits, metrics and tracing) and a Decision Model Providers guide (models supported through LiteLLM, custom providers via send()/asend(), contributing upstream). They embed examples 60, 61 and 62 from software-agent-sdk#6339. The observability guide now lists decision model spans. Co-authored-by: openhands <openhands@all-hands.dev>
|
Preview deployment for your docs. Learn more about Mintlify Previews.
💡 Tip: Enable Automations to automatically generate PRs for you. |
1 of 5 tasks
1 of 5 tasks
- Decision Models: instructions are optional, Choice options may have no description, LiteLLM's 255-option limit, LLMBadRequestError for limits, and result.metrics as a snapshot of running totals. - Decision Model Providers: the new LiteLLM settings (custom_llm_provider, extra_headers, safety_identifier, AWS credentials for Strands Decider), and stricter answer checks in decide(). - Sync examples 60 and 62. Co-authored-by: openhands <openhands@all-hands.dev>
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary of changes
This PR documents
DecisionModel(OpenHands/software-agent-sdk#6068) and its examples (OpenHands/software-agent-sdk#6339). The branch name matches the SDK branch, so the SDK'scheck-examplesjob reads these pages.sdk/guides/llm-decision-models.mdx(Decision Models):Noul,ChoiceandScorequestion types, and basicdecide()/adecide()usage;instructions(aNoulneeds instructions or criteria) andChoiceoptions without a description;Choicethrough LiteLLM (going over raisesLLMBadRequestError), 10Scorelevels, and noBoxor image input yet;result.metricsis a snapshot of running totals) and thedecision.<model>span;sdk/guides/llm-decision-model-providers.mdx(Decision Model Providers):DecisionModelforwards to LiteLLM:custom_llm_provider,extra_headers,safety_identifier, and AWS credentials for Strands Decider on Bedrock AgentCore;DecisionModeland overridingsend()/asend(), and whatdecide()still handles, including its answer checks;sdk/guides/observability.mdx: "What Gets Traced" now lists decision model calls. The embedded copy of example 27 picks up that example's new docstring.docs.json: adds both pages to SDK → Guides → LLM Features.Needs a decision before merge: the Decision Models page says
DecisionModelis available in SDKv1.60.0and later. That assumes #6068 merges beforev1.60.0is cut, sincemainis at1.59.0. Please adjust if the release is different.Validation
.github/scripts/sync_code_blocks.py, usingAGENT_SDK_PATHpointed at the #6339 branch. I kept only the changes for the pages this PR owns. The script also rewrote 46 other pages because the docs and SDKmainbranches have drifted apart, and I reverted those changes.check-examples: I rancheck_documented_examples.pyagainst this branch in pull-request mode, as CI does. It reports "All examples are documented" for the four example files #6339 changes.npx mint broken-linksreports no broken links. Withmint dev, all three pages render, and both new pages appear in the LLM Features sidebar.Choice/NoulCriteriasnippet against live Jev. The Basic Usage comments show the values the live run returned.70245fdc8(review fixes): I updated both pages for the behavior it changed and re-synced examples 60 and 62. I took both snippets from the pages and ran them live again. I also checked each new statement against the SDK code and live calls. That covers optional fields on both routes, the limits raisingLLMBadRequestError, wrong or extra answers raisingLLMResponseErrorwithout a retry, and unmapped errors passing through unchanged.check-examples,mint broken-linksand amint devrender all pass.This PR was created by an AI agent (OpenHands) on behalf of @jpshackelford.
@jpshackelford can click here to continue refining the PR