Skip to content

Add operator TLS listener field and validation - #7024

Open
jhrozek wants to merge 1 commit into
mainfrom
spiffe-v2-05-operator-tls-listener
Open

jhrozek wants to merge 1 commit into
mainfrom
spiffe-v2-05-operator-tls-listener

Conversation

@jhrozek

@jhrozek jhrozek commented Nov 2, 2026

Copy link
Copy Markdown
Contributor

Summary

Operator-managed workloads had no way to turn on the embedded auth server's TLS listener. That listener, on fixed port 8443, is what SPIFFE X.509 clients need to authenticate, and until now it could only be set up by hand-writing a RunConfig.

The same gap also meant that changing the signing key, HMAC, SPIFFE bundle or Redis CA Secret reference did not restart MCPServer or MCPRemoteProxy pods. The mount paths are fixed, so the RunConfig checksum stayed the same and old pods kept mounting the old Secret.

What changed:

  • Adds tlsListener to the embedded auth server config. It takes a certificate Secret ref and a private key Secret ref, both required. The two files are projected as one read-only directory, so kubelet Secret rotation reaches the pod.
  • Rejects an X.509 SPIFFE client association with no listener, and an incomplete listener. This runs at reconcile time and again in RunConfig.Validate, so CLI and standalone vMCP users get a startup error.
  • MCPServer and MCPRemoteProxy reject proxyPort: 8443 when the referenced config sets a listener. The error goes on the auth server reference's validated condition.
  • MCPServer and MCPRemoteProxy roll their pods when the listener, signing-key, HMAC, SPIFFE bundle or Redis CA volumes change. VirtualMCPServer already rolls on any volume change.
  • Moves the repeated status-setting code in VirtualMCPServer auth config validation into a helper. No behavior change.

Part of "Add SPIRE and operator deployment integration for SPIFFE client auth" (#6205).

Type of change

  • Bug fix
  • New feature
  • Refactoring (no behavior change)
  • Dependency update
  • Documentation
  • Other (describe):

Test plan

  • Unit tests (task test)
  • E2E tests (task test-e2e)
  • Linting (task lint-fix)
  • Manual testing (describe below)

The pod-roll behavior is covered by unit tests only: per-volume-type comparison tests, plus controller tests that check MCPServer and MCPRemoteProxy (with and without a podTemplateSpec) roll on one listener change and one non-listener change. A kind-cluster run is still outstanding.

API Compatibility

  • This PR does not break the v1beta1 API. It adds one optional field (tlsListener) and changes no existing fields.

Changes

File Change
cmd/thv-operator/api/v1beta1/mcpexternalauthconfig_types.go TLSListenerConfig type, tlsListener field, ValidateTLSListener
cmd/thv-operator/pkg/controllerutil/authserver.go Listener volume projection, volume comparison helper, port 8443 clash check, RunConfig wiring
cmd/thv-operator/controllers/mcpserver_controller.go, mcpremoteproxy_controller.go Roll pods when auth server volumes change; MCPRemoteProxy podTemplateSpec path
cmd/thv-operator/controllers/*_runconfig.go Port 8443 clash check before RunConfig build
cmd/thv-operator/controllers/virtualmcpserver_controller.go Auth config validation helpers (refactor), listener validation
pkg/authserver/config.go RunConfig.Validate rejects X.509 association without a listener
CRDs under deploy/charts/operator-crds/ and deploy/charts/operator/ Generated schema for tlsListener
docs/operator/crd-api.md, docs/redis-storage.md, docs/arch/* Field docs, Redis CA note, SPIFFE association doc

Does this introduce a user-facing change?

Before: tlsListener does not exist. Changing a signing-key, HMAC, SPIFFE bundle or Redis CA Secret reference did not restart MCPServer or MCPRemoteProxy pods, and proxyPort: 8443 was accepted.

After: tlsListener configures the listener. An X.509 SPIFFE association without one, or a listener with a missing or empty Secret reference, fails validation with a status condition (operator) or a startup error (RunConfig). proxyPort: 8443 fails with a terminal condition when a listener is set. Pods roll when the listener, signing-key, HMAC, SPIFFE bundle or Redis CA reference changes.

Special notes for reviewers

  • Not deployable yet. The listener is reachable on the pod only. The operator does not add port 8443 to the Service or container ports, and SPIFFE trust configuration is still rejected at startup.
  • Pod-roll fix is wider than the listener. The signing-key, HMAC, bundle and Redis CA volumes had the same gap and share the comparison helper, so they are fixed here too. It could be split into its own PR if reviewers prefer.
  • Go validation, not CEL. The "tlsListener required for spiffe_x509" rule is enforced in ValidateTLSListener. A nested exists() over spiffeClientAuth[].methods pushed the schema over the apiserver's CEL cost budget in a dry-run apply.
  • Still open: upstream and trusted-issuer CA bundle references roll pods only when the bundle content changes. That is tracked in "Auth server CA bundle reference change does not roll MCPServer/MCPRemoteProxy pods" (Auth server CA bundle reference change does not roll MCPServer/MCPRemoteProxy pods #6928).
  • VirtualMCPServer refactor is a pure extraction. The HTTP issuer check now lives in insecureHTTPIssuerMessage with the same logic.

The auth server's TLS listener (fixed port 8443) could only be set up
by hand-writing a RunConfig, so operator-managed workloads had no way
to serve SPIFFE X.509 clients. This adds the operator side.

- A tlsListener field on the embedded auth server config names the
  certificate and private key Secrets (both required). They are
  projected as one read-only directory at /etc/toolhive/authserver/tls,
  so kubelet-driven Secret rotation reaches the pod (subPath mounts are
  never updated after the first mount). BuildAuthServerRunConfig points
  the runtime config at the projected files.
- A SPIFFE X.509 client association without a listener is rejected at
  reconcile time (the operator has no admission webhook) and again by
  RunConfig.Validate, so CLI and standalone vMCP users get a startup
  error instead of clients that fail to authenticate with no signal. A
  listener with a missing Secret reference, or one with an empty name
  or key, is rejected the same way.
- MCPServer and MCPRemoteProxy reject a proxyPort of 8443 with a
  terminal condition instead of a pod crash loop.
- MCPServer and MCPRemoteProxy now roll their pods when the listener,
  signing-key, HMAC, SPIFFE bundle or Redis CA reference changes. The
  mount paths are fixed, so changing only the reference left the
  RunConfig unchanged and old pods kept mounting the old Secret.
  VirtualMCPServer already rolls on any volume change through its
  volumes hash.

User-facing change:

Before: tlsListener does not exist. Changing a signing-key, HMAC,
SPIFFE bundle or Redis CA Secret reference did not restart MCPServer or
MCPRemoteProxy pods, and proxyPort 8443 was accepted.
After: tlsListener configures the listener. An X.509 association
without one, or a listener with a missing or empty Secret reference,
fails validation with a status condition (operator) or a startup error
(RunConfig). proxyPort 8443 fails with a terminal condition when a
listener is set. Pods roll when the listener, signing-key, HMAC, SPIFFE
bundle or Redis CA reference changes.

Notes for reviewers:

- The "tlsListener required for spiffe_x509" rule is Go, not CEL. The
  nested exists() over spiffeClientAuth[].methods pushed the schema
  over the apiserver's CEL cost budget in a real dry-run apply. It is
  enforced in ValidateTLSListener.
- The listener is reachable on the pod only. Adding port 8443 to the
  Service and container ports is a separate change, and SPIFFE trust
  configuration is still rejected at startup, so the feature is not
  deployable yet.
- The pod-roll fix covers more than the listener because the signing
  key, HMAC, bundle and Redis CA volumes had the same gap and share the
  comparison helper. Upstream and trusted-issuer CA bundle references
  still roll pods only when the bundle content changes (#6928).
- The proxyPort 8443 clash is reported through the auth server
  reference's validated condition rather than a new one. The clash only
  exists because the referenced config enables tlsListener, and the
  message tells the user to change proxyPort.
- The volume comparison is tested per volume type in
  TestAuthServerVolumesNeedUpdate. The controller tests check that each
  of MCPServer and MCPRemoteProxy (with and without a podTemplateSpec)
  rolls on one listener change and one non-listener change. Tested with
  unit tests only. A kind-cluster run of the pod-roll behaviour is still
  outstanding.

Part of #6205.

Co-authored-by: Mecatl <noreply@mecatl.dev>
@github-actions github-actions Bot added the size/XL Extra large PR: 1000+ lines changed label Nov 2, 2026
@codecov

codecov Bot commented Nov 2, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 96.47059% with 6 lines in your changes missing coverage. Please review.
✅ Project coverage is 80.54%. Comparing base (b0fcff9) to head (0555039).

Files with missing lines Patch % Lines
...perator/controllers/virtualmcpserver_controller.go 89.65% 3 Missing ⚠️
...perator/api/v1beta1/mcpexternalauthconfig_types.go 96.66% 1 Missing ⚠️
...-operator/controllers/mcpremoteproxy_controller.go 94.73% 1 Missing ⚠️
cmd/thv-operator/pkg/controllerutil/authserver.go 98.70% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #7024      +/-   ##
==========================================
+ Coverage   80.45%   80.54%   +0.09%     
==========================================
  Files         852      852              
  Lines       87035    87119      +84     
==========================================
+ Hits        70021    70173     +152     
+ Misses      17009    16941      -68     
  Partials        5        5              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/XL Extra large PR: 1000+ lines changed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant