Skip to content

Let DCR upstreams choose the token endpoint auth method - #6986

Merged
jhrozek merged 2 commits into
stacklok:mainfrom
aron-muon:dcr-honor-auth-method
Oct 30, 2026
Merged

jhrozek merged 2 commits into
stacklok:mainfrom
aron-muon:dcr-honor-auth-method

Conversation

@aron-muon

@aron-muon aron-muon commented Oct 29, 2026 •

Copy link
Copy Markdown
Contributor

Summary

An OAuth2 upstream with dcrConfig always registers with the token endpoint auth method the resolver prefers. That is client_secret_basic when the upstream advertises it, and also by default on the registrationEndpoint branch, where no metadata is read. An upstream whose token endpoint does not accept that method fails every code exchange and refresh, and the operator has no way out: oauth2Config.tokenEndpointAuthMethod fails validation next to dcrConfig, and the negotiated method would replace it anyway. We hit this with an upstream that advertises client_secret_basic but reads client credentials only from the request body (invalid_client: Missing client_id).

  • With dcrConfig, oauth2Config.tokenEndpointAuthMethod now selects the method the client registers with (dcr.Request.TokenEndpointAuthMethod). Run-config validation no longer asks for a client secret source in that case, since the secret comes from the registration. The resolved run config hands the method to the built provider config through the resolution, together with the registered secret.
  • The resolver refuses a configured method the upstream does not advertise, none without S256 PKCE, and a confidential method on a public client. It also refuses a registration response that names a method other than the configured one, and a final resolution that still names one. That happens when a cache write returns a row a replica on an older configuration stored first, or when a concurrent caller configured differently shares the leader's singleflight result.
  • On a cache hit, a registration made with another method is replaced at the next resolve through the same ReplaceIfUnchanged path a rejected registration takes, so changing the field takes effect at the next start without a rotation key.

Fixes #6985

Type of change

  • Bug fix
  • New feature
  • Refactoring (no behavior change)
  • Dependency update
  • Documentation
  • Other (describe):

Test plan

  • Unit tests (task test)
  • E2E tests (task test-e2e)
  • Linting (task lint-fix)
  • Manual testing (describe below)

New tests: TestResolveDCRCredentials_ConfiguredAuthMethod (selection and every refusal), TestResolveCredentials_CacheHitConfiguredAuthMethod (replace on mismatch without probing, keep on match, error from a store without Replacer), TestResolveDCRCredentials_AuthoritativeWinnerWithOtherAuthMethodErrors (a stored winner with another method), TestBuildUpstreamConfigs_DCRConfiguredAuthMethod (both confidential methods through registration and a cache hit, asserting the built config's method and registered secret), TestConsumeResolution_ClearsDCRFields, plus cases in TestNewDCRRequest and TestOAuth2UpstreamRunConfigValidate. task test passes except three packages outside this change on this machine: pkg/skills/identitytoken fails the same way on main, pkg/container/docker needs real Squid and Envoy containers, and pkg/lockfile is timing-sensitive under full-suite load.

API Compatibility

  • This PR does not break the v1beta1 API, OR the api-break-allowed label is applied and the migration guidance is described above.

The field already exists; only its documented meaning with dcrConfig changes.

Changes

File Change
pkg/auth/dcr/request.go Request.TokenEndpointAuthMethod
pkg/auth/dcr/resolver.go Configured method in selection, response check, mismatch replacement on a cache hit, final method check
pkg/authserver/runner/dcr_adapter.go Copies the OAuth2 upstream's method onto the request; the resolved run config takes it from the resolution
pkg/authserver/config.go No secret source required with dcr_config; field doc
cmd/thv-operator/api/v1beta1/mcpexternalauthconfig_types.go Field doc
docs/arch/11-auth-server-storage.md Registration method and the replacement rule
CRDs, docs/operator/crd-api.md, swagger, SDK Regenerated for the field docs

Does this introduce a user-facing change?

Yes, for OAuth2 upstreams with dcrConfig:

  • Before: setting tokenEndpointAuthMethod failed at start with requires client_secret_file or client_secret_env_var, and the client always registered with the preferred method.
  • After: the client registers with the configured method. A cached registration made with another method is replaced at the next start, whatever registrationProbe says. A configured method the upstream does not advertise, or a registration the upstream switched to another method, fails at start with an error naming both methods. A stored row with another method that a replica on an older configuration wrote during a rollout fails that start or recheck with an auth_method_mismatch error, and the next one replaces the row.

Upstreams that leave the field unset behave as before.

Implementation plan

Approved implementation plan
  1. Reuse OAuth2UpstreamConfig.TokenEndpointAuthMethod rather than add a dcrConfig field, since it already means "how this client authenticates at the token endpoint".
  2. Carry it on dcr.Request and let selectTokenEndpointAuthMethod return it ahead of the preference order, checked against the advertised set and the S256 gate.
  3. Refuse a registration response that names another method, so a configured method is never silently replaced.
  4. On a cache hit, treat a method mismatch like a probe rejection and replace the row with ReplaceIfUnchanged, instead of folding the method into the cache key.
  5. Check the final resolution against the configured method, so a stored winner or a shared flight result with another method is refused.
  6. Relax the run-config secret-source check when dcr_config is set, and clear the method on the resolved run config so the resolution supplies it.

Special notes for reviewers

  • The method is not part of the cache key, unlike rotationKey. A row registered with the configured method keeps being used, and only a mismatching row is replaced, so turning the field on does not orphan a working registration.
  • Refusing a response that names another method deviates from RFC 7591 §3.2.1, which lets the server replace requested metadata and expects the client to use what it returned. Here the operator configured the method because the server's own choice does not work. Accepting the replacement would also re-register at every start, since the stored row would never match.
  • A stored winner with another method fails the resolution instead of being replaced in the same call. The next resolution replaces it through the cache-hit path, which keeps a single replacement path and registrationProbe: none still runs the method check.
  • A store without Replacer returns an error on a mismatch, where the probe path hands back the cached row unchecked. Using that row would present credentials with the method the operator configured away from.
  • OIDC upstreams have no tokenEndpointAuthMethod field, so this covers OAuth2 upstreams only; Basic auth for OIDC upstreams is Support Basic auth for OIDC upstreams #6552.

🤖 Generated with Claude Code

An OAuth2 upstream with dcrConfig always registered with the token
endpoint auth method the resolver prefers: client_secret_basic when
advertised, and by default on the registrationEndpoint branch, where no
metadata is read. An upstream whose token endpoint does not accept that
method, such as one that advertises client_secret_basic but reads client
credentials only from the request body, failed every code exchange and
refresh, and tokenEndpointAuthMethod could not help: with dcrConfig it
failed validation for want of a secret source, and the negotiated method
replaced it anyway.

With dcrConfig, tokenEndpointAuthMethod now selects the method the client
registers with. The resolver refuses a configured method the upstream
does not advertise, and a registration response that names another one.
A cached registration made with another method is replaced at the next
resolve through the same compare-and-set as a rejected one.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Aron <aron@muonspace.com>
@codecov

codecov Bot commented Oct 29, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 98.38710% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 80.35%. Comparing base (20bc1a1) to head (175807b).
⚠️ Report is 10 commits behind head on main.

Files with missing lines Patch % Lines
pkg/authserver/config.go 50.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #6986      +/-   ##
==========================================
+ Coverage   80.29%   80.35%   +0.06%     
==========================================
  Files         847      849       +2     
  Lines       86142    86679     +537     
==========================================
+ Hits        69165    69654     +489     
- Misses      16972    17020      +48     
  Partials        5        5              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Comment thread pkg/authserver/config.go Resolved
Comment thread pkg/auth/dcr/resolver.go Resolved
Comment thread docs/arch/11-auth-server-storage.md Resolved
A DCR upstream with a configured client_secret_post or
client_secret_basic method failed at start: the resolved run config
dropped DCRConfig but kept the method, and validating that copy
demanded a secret file. The resolution carries the method to the built
config, so the resolved copy no longer keeps it.

The resolver checked the method only on its own registration response.
A cache write can return a row a replica on an older configuration
stored first, and a concurrent caller shares the leader's result, so
the final resolution is now refused when its method differs from the
configured one. The next start or recheck replaces the row.

The registrationProbe none docs now say it turns off the upstream
probe, not the method check.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Aron <aron@muonspace.com>

@jhrozek jhrozek left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, all three are addressed in 175807b. LGTM.

@jhrozek
jhrozek merged commit d752cf5 into stacklok:main Oct 30, 2026
51 checks passed
@github-actions github-actions Bot mentioned this pull request Nov 2, 2026
2 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

DCR upstreams cannot choose the token endpoint auth method

2 participants