Repository navigation
Feat/1588 display ai credits in model list - #1589
Open
rakotomandimby wants to merge 16 commits into
Open
rakotomandimby wants to merge 16 commits into
rakotomandimby wants to merge 16 commits into
Conversation
Updates input preparation functions (`prepare_chat_input` and `prepare_responses_input`) to include reasoning configuration blocks when `opts.model.reasoning_effort` is specified. Notable behavior changes: - Outgoing requests to providers now transmit the selected reasoning effort parameter in the expected format. Notable risks: - Providers that do not support reasoning payloads might throw validation errors. Things to test: - Monitor outgoing debug logs during chat requests to confirm reasoning parameters are correctly included in the request body.
Refactor model picker formatting logic to handle indentation and item selection indicators more consistently, moving effort names to a global module-level constant table `EFFORT_NAMES`. Notable behavior changes: - Adjusted spacing and bullet points for models and reasoning efforts inside the selection menu. Notable risks: - Visual regressions in the Neovim UI picker formatting if terminal fonts or spacing expectations differ. Things to test: - Open the model selection menu and visually inspect the spacing for standard models and nested reasoning effort choices.
This fix updates the model selection mechanism to handle default `vim.ui.select` implementations gracefully. The built-in implementation uses `inputlist()`, which only supports selection by flat list position and fails when rendering complex hierarchical indexes like `3b` or `10c`. Now, the plugin checks if `vim.ui.select` is using the default built-in implementation. If it is, it falls back to rendering a custom text-based list using `vim.api.nvim_echo` and reads user choices via `vim.ui.input`. Otherwise, custom pickers (such as Telescope, fzf-lua, or Snacks) continue to use standard `vim.ui.select` workflows. Notable behavior changes: - Users with the default Neovim UI select will see a manual grouped text layout and prompt. - Selection codes (like `3` or `10c`) must be typed explicitly when using the default fallback. Notable risks: - Custom pickers might misbehave if they override `vim.ui.select` without updating its source path signature to something other than `vim/ui.lua`. Things to test to avoid regression: - Open model selection with the default Neovim UI and verify that inputting codes functions properly. - Open model selection with Telescope or other picker plugins to ensure custom pickers still use standard selection lists.
for more information, see https://pre-commit.ci
Add support for specifying reasoning effort values using a colon suffix on
model names (e.g., 'gpt-5.4:high'). When a model is requested in this
format, the client parses the effort level, looks up the base model configuration,
and passes the appropriate parameters down to the respective API requests.
Changes include:
- Updating `Client:ask` to parse model strings containing a colon and extract
the target reasoning effort.
- Enhancing error response formatting to handle table bodies gracefully via
`vim.inspect`.
- Adding comprehensive specs covering serialization and validation of reasoning
effort variants.
Risks:
- Incorrect parsing could lead to failures when model IDs themselves contain
colons or unconventional formatting.
To test against regression:
- Invoke `ask` with various valid and invalid reasoning effort suffixes and
verify parameters are correctly propagated to requests.
local ok, err = pcall(function()
client:ask({ model = 'gpt-5.4:high', history = {} })
end)
for more information, see https://pre-commit.ci
Add support for fetching, normalizing, and displaying AI Credit token pricing rates for models in the CopilotChat model picker. What changed and why: - Updated the copilot provider request headers to target the newer model-list API version (`2026-08-01`) required for retrieving token pricing metadata. - Implemented `model_credit_rates` in `lua/CopilotChat/config/providers.lua` to extract, validate, and scale default input and output token prices based on billing metadata and batch sizes. - Updated model choice rendering in `lua/CopilotChat/init.lua` to display credit rates as `input/output` per million tokens, replacing the previous multiplier badge. - Handled missing or malformed pricing gracefully by falling back to a question mark (`?`) and logging warnings for invalid metadata. Notable behavior changes: - The model picker interface now displays token credit rates alongside provider and capability indicators instead of the old multipliers. - Models without pricing info show `?` for their credit rates. Notable risks: - Malformed pricing structures or unexpected API payload changes could trigger log warnings or improper credit rate displays. - Network calls to the `/models` endpoint rely on the `2026-08-01` API version, which must be fully compatible with the backend provider. Things to test to avoid regression: - Verify that the model picker correctly renders valid, partial, and missing credit rates for various models. - Ensure reasoning-effort model variants preserve their parent model's credit pricing correctly. - Test graceful handling of invalid pricing metadata (non-numeric prices, zero or negative batch sizes) to ensure fallback values are assigned properly.
Remove the credit rates display string from reasoning effort items within the model selection picker. This simplifies the UI by ensuring credit rates are only displayed at the parent model level rather than repeating them for each individual reasoning effort variant. Notable Behavior Changes: - Model picker labels for reasoning efforts (e.g., High, Low) no longer append credit rate details like [100/500]. Notable Risks: - Users relying on credit rate visibility per reasoning effort variant might find it slightly harder to compare exact costs directly from the sub-menu without looking at the parent model item. Things to Test to Avoid Regression: - Open the model selection picker and verify that reasoning effort items (such as 2a. Low and 2b. High) display correctly without credit rates. - Ensure selecting a reasoning effort updates the model configuration properly and renders the selection indicator (*) accurately.
Refactor the model picker sorting and grouping logic in `lua/CopilotChat/init.lua` to properly handle base models and their reasoning effort variations. Auto models are now guaranteed to be listed first, followed by other models ordered by output rate from lowest to highest, and unavailable rates placed last. Provider and request model fields are also taken into account during sorting. Notable behavior changes include updated ordering in the model picker UI and new handling for double-digit codes and duplicate effort indexes. Risks involve potential misordering if model credit rates are malformed or missing, though fallbacks have been implemented. To avoid regression, test the model picker UI options to ensure correct sorting order, unique code generation for efforts, and appropriate selection persistence.
![pullfrog[bot]](https://avatars.githubusercontent.com/in/1768019?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
ℹ️ No critical issues — minor suggestions and one open question below.
Reviewed changes
Reviewed the full PR (9 files, 10 commits) which adds AI Credit pricing to the model picker and, on the same branch, a reasoning-effort selection feature.
- Credit pricing display —
providers.luaderivesinput_credits/output_credits(AI Credits per 1M tokens) frombilling.token_prices.defaultand drops the oldmultiplier;init.luarendersinput/outputrates and sorts entries by output rate (unavailable last,autofirst). - Reasoning-effort variants — synthetic
<model>:<effort>picker entries fromcapabilities.supports.reasoning_effort, keeping the API id inrequest_model;client.luaresolves the colon syntax and forwards the effort;prepare_inputemitsreasoning.effort(Responses) /reasoning_effort(Chat Completions). - Model picker rewrite — grouped model+effort entries with generated display codes and a built-in
vim.ui.selectworkaround, withM.open/close/togglemoved without behavior change. - Plumbing + tests —
prompts.resolve_modelnow accepts colon suffixes, table response bodies no longer crash the error path, and substantial new specs intests/init_spec.lua/tests/models_spec.lua.
ℹ️ doc/CopilotChat.txt was hand-regenerated and is out of sync
The generated vimdoc is included in this PR but does not reflect the README change it accompanies, and per AGENTS.md it is meant to be produced by CI from README.md.
Technical details
# doc/CopilotChat.txt is stale relative to README.md
## Affected sites
- `doc/CopilotChat.txt:2` — header date changed by hand ("2026 October 08")
- `doc/CopilotChat.txt` — widespread trailing-whitespace normalisation (e.g. around the VIM-PLUG and function tables)
- `README.md:121-123` — the new credits note is present here but absent from the generated doc (see around `doc/CopilotChat.txt:156`, the `:CopilotChatModels` entry)
## Required outcome
- Drop `doc/CopilotChat.txt` from the PR, or regenerate it with the pinned panvimdoc version from the updated README so the two stay consistent.
## Open questions for the human
- Was the doc change an intentional local regeneration or an accidental commit?ℹ️ Two features are bundled under a credits-only title
The PR title and linked issue (#1588) cover the credit display, but the same branch also introduces a full reasoning-effort feature and a model-picker rewrite. That makes the change hard to review and review notes harder to attribute.
Technical details
# Scope: reasoning effort bundled with credits
## Affected sites
- `lua/CopilotChat/config/providers.lua` — variant generation + reasoning serialisation
- `lua/CopilotChat/client.lua` — colon-syntax resolution / `reasoning_effort` forwarding
- `lua/CopilotChat/init.lua` — `M.select_model` group/code rewrite
- `lua/CopilotChat/prompts.lua` — `$model:effort` resolution
## Required outcome
- Split the reasoning-effort work into its own PR, or update the PR title/description so reviewers and the release notes capture both features.
## Open questions for the human
- Is the reasoning-effort work intended to be stacked separately on top of the credits change?deepseek-v4.1-flash (free via Pullfrog for OSS) ｜ 𝕏
Comment on lines
+321
to
+323
| if opts.model.reasoning_effort then | ||
| out.reasoning_effort = opts.model.reasoning_effort | ||
| end |
Contributor
There was a problem hiding this comment.
Now that a Chat Completions request can carry reasoning_effort, note this path still emits top_p = 1 and n = 1 (plus temperature when set) for every model not matched by the o1/codex name check above, which does not cover gpt-5.x. OpenAI documents reasoning models as not supporting those sampling parameters alongside reasoning, so a request that combines them may be rejected — worth confirming whether any reasoning-capable model in the Copilot catalog lacks /responses and therefore reaches this branch. The Responses path (prepare_responses_input) sends none of these, so it is unaffected.
Technical details
# reasoning_effort coexists with sampling params on Chat Completions
## Affected site
- `lua/CopilotChat/config/providers.lua:321-342` — `prepare_chat_input` sets `out.reasoning_effort` here, then unconditionally sets `out.n`/`out.top_p`/`out.temperature` under the `not is_o1 and not is_codex` guard.
## Required outcome
- Confirm reachability: does any model exposing `capabilities.supports.reasoning_effort` route to `/chat/completions` (i.e. not advertise `/responses`)? If none do, no change is needed; if some do, the guard at `:338` likely needs to also skip these params when `opts.model.reasoning_effort` is present.
## Open questions for the human
- Have you seen or tested a reasoning-capable model that Copilot serves only via `/chat/completions`?
- Does the Copilot proxy enforce the OpenAI/Azure restriction, or does it clamp/ignore the extra params?Refactor model resolution logic in client to extract effort variant resolution into a dedicated helper function, improving code organization and maintainability. Changes: - Extracted `resolve_effort_variant` from `Client:ask`. - Cleaned up inline checks for reasoning efforts. Risks: - Potential misidentification of models if model IDs contain colons and match prefixes incorrectly. Things to test: - Verify asking queries with models specifying effort levels like `model:high` works as expected without errors.
Refactor the initialization and model picker tests to utilize a unified patching helper mechanism and centralized cleanup routines. Changes include: - Introduce a `patch` utility function to safely mock table properties and automatically restore them during `after_each` execution. - Add specialized test helper functions such as `capture_warnings`, `capture_select`, `choose_id`, and `use_builtin_picker` to reduce duplication across test cases. Risks: - Incorrect restoration order could lead to test leakage if global state fails to reset properly. Test to avoid regression: - Verify that running the full test suite repeatedly does not result in pollution of global `vim`, `client`, or `log` states between runs.
for more information, see https://pre-commit.ci
Contributor
There was a problem hiding this comment.
✅ No new issues found.
Reviewed changes
Reviewed the incremental delta since the prior pullfrog review at b852e8e (commits 377564d, c1a4a75), which refactors the credit-pricing / reasoning-effort code from the earlier review without adding new features or changing observable behavior.
- Extracted effort resolution —
Client:ask's inline<model>:<effort>longest-prefix scan moved intoresolve_effort_variantinclient.lua, and theautoguard now usesutils.is_auto_model. - Extracted pricing helpers —
providers.luareplacesmodel_credit_rateswithpricing_table/parse_credit_rates, addsunique_effortsandsplit_copilot_headers, and hoists the base URL / API-version / credit-unit constants. - Rewrote the model picker into helpers —
init.luasplitsM.select_modelintogroup_models,sort_key/compare_keys,build_choices/make_choice, theformat_*family, andpick_with_codes/pick_with_select; invalid-rate validation is hoisted intowith_sanitized_rates/sanitize_rate. - Added shared predicates —
utils.luagainsis_rate,is_missing,is_default_select,is_auto_model;health.luareusesis_default_select. - Test refactor —
tests/init_spec.luaandtests/models_spec.luaadopt apatch/restoreshelper withcapture_warnings/capture_select/use_builtin_picker/serve_models/by_id, andtests/utils_spec.luacovers the new predicates.
The delta is a behavior-preserving extraction: choice order, generated codes, effort grouping, selection/cancellation, and formatted labels are unchanged from b852e8e, and the only concrete value expression that changed (picker = model.model_picker_enabled == true) is value-identical to the previous not not form for the boolean/nil the API can produce.
deepseek-v4.1-flash (free via Pullfrog for OSS) ｜ 𝕏
Transition the Copilot provider statistics display from the legacy "Premium requests" terminology to the updated "AI Credits (AIC)" nomenclature provided by the backend API. Notable Behavior Changes: - User info command output now prints headers as `**AI Credits (AIC)**`. Notable Risks: - Scripts or automated parsing dependent on the old "Premium requests" string might experience parsing breakages. Things to Test: - Check integration tests in `tests/models_spec.lua` to ensure regex and string matching handles the new header labels correctly.
for more information, see https://pre-commit.ci
Contributor
There was a problem hiding this comment.
ℹ️ No critical issues — one minor suggestion inline.
Reviewed changes
Reviewed the incremental delta since the prior pullfrog review (commits 0e2fab9, 54ca3d3), which reworks the Copilot stats output and adds get_info tests.
- Renamed the stats header —
get_infonow printsAI Credits (AIC)instead ofPremium requests, preferring the newai_creditssnapshot and falling back to the legacypremium_interactionskey. - Fractional credit amounts — a new
format_amounthelper renders integers bare and fractions with up to 2 decimals, and the usage block is now guarded against non-numericentitlement/remaining. - Dropped the
Chatquota line — the stats panel no longer lists chat usage (see the inline note). - Added
get_infospecs —tests/models_spec.luacovers the rename, key preference, fractional amounts, and the missing-stats path. - Pre-commit autofixes —
54ca3d3only removes two trailing blank lines.
deepseek-v4.1-flash (free via Pullfrog for OSS) ｜ 𝕏
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.