Skip to content

Fix client retrying a 429 response forever - #2459

Open
anikeya9 wants to merge 1 commit into
Materials-Consortia:mainfrom
anikeya9:fix/client-retry-limit
Open

anikeya9 wants to merge 1 commit into
Materials-Consortia:mainfrom
anikeya9:fix/client-retry-limit

Conversation

@anikeya9

Copy link
Copy Markdown
Contributor

Closes #2458

The retry counter in the client's pagination loops was reset at the start of every pass of while next_url:, including the pass that follows a retry. As a result, a provider that keeps returning 429 Too Many Requests was retried forever and max_attempts was never reached (see the issue for a reproduction).

Changes

  • optimade/client/client.py: in both _get_one_async and _get_one, attempts is now initialised once before the loop and reset only after a page has been fetched successfully, so each page keeps its own retry budget of max_attempts.
  • tests/server/test_client.py: new test_retry_limit_async and test_retry_limit_sync, using test HTTP clients that always return 429 (with the wait between retries patched out). They check that exactly max_attempts + 1 requests are made and that the "Exceeded maximum number of retries" error is reported for the provider. Without the fix, both tests fail (the test clients stop the loop after 50 requests instead of hanging).

Note

Retries still count towards number_of_requests, and therefore towards the max_requests_per_provider guard rail. I left this unchanged to keep the PR minimal, but I'm happy to change it if you prefer.

🤖 Generated with Claude Code

The retry counter in the client's pagination loops was reset at the
start of every pass of `while next_url:`, so a provider that kept
returning 429 Too Many Requests was retried forever and `max_attempts`
was never reached. Initialise the counter once before the loop and
reset it only after a page succeeds, in both the sync and async paths.

Add sync and async tests with an HTTP client that always returns 429.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@codecov

codecov Bot commented Nov 12, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 90.71%. Comparing base (f90d814) to head (46a33d4).

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2459      +/-   ##
==========================================
+ Coverage   90.37%   90.71%   +0.33%     
==========================================
  Files          78       78              
  Lines        5122     5124       +2     
==========================================
+ Hits         4629     4648      +19     
+ Misses        493      476      -17     
Flag Coverage Δ
project 90.71% <100.00%> (+0.33%) ⬆️
validator 90.71% <100.00%> (+0.33%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Client retries forever when a provider keeps returning 429 (max_attempts is never reached)

1 participant