Skip to content

ConsulDiscoveryClient.getServices() throws NPE when Consul has no leader, and the gateway's discovery routes stay empty afterwards #1040

Closed

Description

@mraible

After a Consul leader election, a Spring Cloud Gateway app that uses discovery-based routes loses all of its routes and keeps returning 404 until the Consul catalog changes again.

While Consul has no leader, GET /v1/catalog/services returns 500 "No cluster leader". ConsulDiscoveryClient.getServices() then calls keySet() on a null body:

Map<String, List<String>> catalogServices = client
	.getCatalogServices(properties.getAclToken(), QueryParams.DEFAULT)
	.getBody();
return new ArrayList<>(catalogServices.keySet());

ConsulReactiveDiscoveryClient.getServices() logs the exception and returns Flux.empty(), so the discovery route locator rebuilds its routes from an empty service list:

ERROR ConsulReactiveDiscoveryClient : Error getting services from Consul.
java.lang.NullPointerException: Cannot invoke "java.util.Map.keySet()" because "catalogServices" is null
	at org.springframework.cloud.consul.discovery.ConsulDiscoveryClient.getServices(ConsulDiscoveryClient.java:104)
	at org.springframework.cloud.consul.discovery.reactive.ConsulReactiveDiscoveryClient.lambda$getServices$2(ConsulReactiveDiscoveryClient.java:74)

Once a leader is elected, nothing rebuilds the routes, because the catalog index doesn't change. In our case the gateway returned 404 for every /services/<app>/** route for more than 5 minutes after the election. Registering any service (which changes the catalog index) brought the routes back about 3 seconds later.

How we hit it: a JHipster 9.4.0 microservices app (Spring Boot 4.1.1, Spring Cloud 2025.1.3, spring-cloud-consul 5.0.3) on a kind cluster, with a 3-server Consul 2.1.0 StatefulSet and a reactive gateway using discovery routes.

  1. Deploy the gateway and two services, all registered with Consul, and check that /services/<app>/** routes work.
  2. Run kubectl rollout restart statefulset/consul so Consul elects a new leader.
  3. While there's no leader, the gateway logs the NPE above. After the rollout, every discovery route returns 404 until something changes the catalog.

The code is the same in v5.0.4 and on main (getAllInstances() has the same unchecked getBody()).

I'd expect getServices() to throw an exception that says Consul couldn't answer, rather than an NPE. I'd also expect the reactive client not to turn that failure into an empty service list, since an empty list looks like a valid answer and wipes every route. Either keeping the last known services or propagating the error so the route locator keeps its current routes would avoid the outage.

Activity

  1. added a commit that references this issue on Oct 30, 2026
    adf81bb
  2. ryanjbaxter commented on Oct 30, 2026

    @ryanjbaxter
    Contributor

    Hi @mraible, I believe I have a fix for this in #1041, could you take a look?

  3. added this to the 5.0.5 milestone on Oct 30, 2026
  4. moved this from Todo to In Progress in 2026.0.0-RC2on Oct 30, 2026
  5. moved this from Todo to In Progress in 2025.1.5on Oct 30, 2026
  6. mraible commented on Oct 30, 2026

    @mraible
    Author

    @ryanjbaxter I added a comment to the PR. LGTM!

  7. added a commit that references this issue on Oct 30, 2026
    a431f19
  8. moved this from In Progress to Done in 2026.0.0-RC2on Oct 30, 2026
  9. moved this from In Progress to Done in 2025.1.5on Oct 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions