Skip to content

feat(llm): add 88API as a native LLM provider - #327

Open
blackdm666 wants to merge 1 commit into
OpenByteInc:mainfrom
blackdm666:feat/88api-provider
Open

blackdm666 wants to merge 1 commit into
OpenByteInc:mainfrom
blackdm666:feat/88api-provider

Conversation

@blackdm666

Copy link
Copy Markdown

Summary

Adds 88API as a native LLM provider, next to OpenRouter and AtlasCloud. 88API is an OpenAI-compatible gateway (https://88api.ai/v1): one key can call GPT, Claude, Gemini, DeepSeek, Grok, MiniMax and other models. Until now users had to go through the Custom provider and fill in the URL, key and model by hand. With this change they pick 88API in the provider list and paste their key.

It also fills a gap in the presets: none of the existing presets serves Claude models directly. The 88API default model is claude-sonnet-5-5, with claude-haiku-5-5 as the static fallback.

Changes

The provider follows the existing provider pattern. It goes through _call_openai_compatible like DeepSeek, Grok and AtlasCloud, and adds no new request code path.

  • app/services/llm.py
    • LLMProvider.API88 = "api88". Enum member names and shell env var names can't start with a digit, so the identifier is api88 everywhere: LLM_PROVIDER=api88, API88_* env vars, the api88 config section and the settings group. Because of that, the generic {value.upper()}_BASE_URL / _MODEL lookups and the "please set API88_API_KEY" error work without special cases.
    • PROVIDER_CONFIGS entry: base URL https://88api.ai/v1, default claude-sonnet-5-5, fallback claude-haiku-5-5.
    • Added to the auto-detect and alternative-provider priority lists just before OpenRouter, so existing setups with several keys keep their current choice.
    • api88/ and 88api/ model prefixes resolve to this provider and get stripped (88API model ids have no vendor prefix).
    • HTTP errors are labelled 88API API <status>, like the existing AtlasCloud/OpenRouter labels.
  • app/config/api_keys.py: API88_API_KEY (env first, then addon config, same as the other keys).
  • app/utils/config_loader.py: API88_API_KEY / API88_BASE_URL / API88_MODEL mappings.
  • app/routes/settings.py: 88API (Multi-model gateway) option in the LLM_PROVIDER select, plus key / model / base-URL fields in the api88 group. API88_BASE_URL goes in the Advanced tab with the other base URLs.
  • .env.example, backend_api_python/env.example: commented 88API block.
  • README.md, README_CN.md: 88API added to the AI providers row.

No frontend change should be needed: the provider dropdown and its fields come from the settings schema. The default provider is still openrouter.

How to test

cd backend_api_python
python -m pytest tests/test_llm_litellm_provider.py tests/test_settings_llm_connection.py -q

New unit tests in tests/test_llm_litellm_provider.py cover:

  • the env → config mapping
  • explicit selection and defaults
  • the base-URL override
  • model prefix normalization
  • auto-detection when 88API is the only key
  • OpenRouter staying the default when no key is set
  • the OpenAI-compatible request shape
  • error labelling

Manual check against the live endpoint:

LLM_PROVIDER=api88 API88_API_KEY=<your key> python -c "from app.services.llm import LLMService; s=LLMService(); print(s.test_connection()); print(s.call_llm_api([{'role':'user','content':'Return {\"ok\": true} as JSON'}]))"

Results

  • python -m pytest -m "not integration and not stress": 3867 passed, 37 skipped.
  • ruff check on all changed Python files passes.
  • Live check against https://88api.ai/v1 through LLMService.call_llm_api (JSON mode on, the default for analysis calls). All 56 text models on 88API returned valid JSON:
    • Claude: Opus, Sonnet, Haiku and Fable
    • GPT 5.5 / 5.6 / 6 / 6.1
    • Gemini 2.5 – 4
    • DeepSeek V4
    • Grok 4.5 – 4.7
    • MiniMax M2.1 – M3
    • Qwen, Kimi, GLM, Hunyuan and MiMo
  • stream_llm_api streams correctly for one model from each of the 9 vendor families.
  • test_connection() passes with a valid key.

One finding from the live run shaped the code: Anthropic rejects response_format: {"type": "json_object"} with a 400 ("use text or json_schema"). So when the request goes to 88API and the model id starts with claude, _call_openai_compatible leaves that field out. This is the same approach already used for AtlasCloud. The prompts still ask for JSON, and all Claude models returned valid JSON in the live run. Other 88API models keep JSON mode.

Backward compatibility

The change only adds things. No existing provider, default or env var changes, and no trading, execution or risk code is touched.

🤖 Generated with Claude Code

88API (https://88api.ai/v1) is an OpenAI-compatible multi-model gateway:
one key calls GPT, Claude, Gemini, DeepSeek, Grok, MiniMax and more.
Users previously had to configure it through the Custom provider.

- LLMProvider.API88 ("api88"), API88_API_KEY / API88_BASE_URL /
  API88_MODEL, default claude-sonnet-5-5 with claude-haiku-5-5 fallback
- settings schema option and fields, env examples, README provider row
- skip response_format json_object for claude-* models on 88API, which
  Anthropic rejects (same approach as AtlasCloud)
- unit tests for config mapping, selection, normalization, request shape

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: blackdm666 <blackdm666@users.noreply.github.com>
@sonarqubecloud

Copy link
Copy Markdown

@codecov

codecov Bot commented Oct 29, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 86.66667% with 2 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
backend_api_python/app/services/llm.py 71.42% 1 Missing and 1 partial ⚠️

📢 Thoughts on this report? Let us know!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant