Skip to content

Add a ConfigResourceClient for plain text and binary files - #3355

Open
sharanggupta wants to merge 5 commits into
spring-cloud:mainfrom
sharanggupta:gh-789-config-resource-client
Open

sharanggupta wants to merge 5 commits into
spring-cloud:mainfrom
sharanggupta:gh-789-config-resource-client

Conversation

@sharanggupta

Copy link
Copy Markdown
Contributor

Fixes gh-789

Ports the ConfigResourceClient from the commercial config client (spring-cloud-services-starters, Apache 2.0; original authors kept in the Javadoc), as suggested on the issue, so that an application can read the plain text and binary files the config server serves with the same settings that loaded its configuration.

  • PlainTextConfigClient, BinaryResourceConfigClient and ConfigResourceClient are the ported API; DefaultConfigResourceClient is the RestClient based implementation, with the application name, profile and label coming from ConfigClientProperties.
  • ConfigServerConfigDataLocationResolver exposes the RestTemplate that loaded the configuration as configClientRestTemplate and a RestClient built from it as configClientRestClient when the bootstrap context closes, as the commercial resolver does, so the client carries the URI, credentials, headers, TLS settings, timeouts and token of the import.
  • ConfigResourceClientAutoConfiguration creates the ConfigResourceClient when that RestClient bean exists and no custom client is defined.

Differences from the commercial code, all because the open source client has no default label: without a label the request uses ?useDefaultLabel, a slash in a label is escaped as (_) as ConfigServerConfigDataLoader does, a nested path is sent segment by segment, and the profile falls back to default when none is configured.

Tests cover the request paths, headers, defaults and error cases against a MockRestServiceServer, the auto-configuration conditions, and that a spring.config.import=configserver: import exposes the beans; the client module suite passes. The client docs gain a "Reading Plain Text and Binary Files" section.

Port the ConfigResourceClient of the commercial config client into the
client module, so that an application can read the plain text and
binary files the config server serves with the same settings that
loaded its configuration: the config data resolver exposes the
RestTemplate that loaded the configuration and a RestClient built from
it as beans, and the auto-configuration creates the client from them.

Without a label the request asks the server for its default label, a
slash in a label is escaped as the config data loader does, and a
nested path is sent segment by segment.

Fixes spring-cloudgh-789

Signed-off-by: Sharang Gupta <sharang@sharanggupta.dev>
A context can hold two ConfigClientProperties beans, the one resolved by
the config server import and the one bound by ConfigClientAutoConfiguration,
which made the configResourceClient bean fail to resolve its dependency
in the sample application. Prefer the imported properties, which
describe the server the RestClient talks to.

Signed-off-by: Sharang Gupta <sharang@sharanggupta.dev>
if (label == null) {
label = this.properties.getLabel();
}
UriComponentsBuilder builder = UriComponentsBuilder.fromUriString(uris[0]).pathSegment(name, profile);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This only uses the first URI, what about the rest?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Every URI is now tried in order with the rules of ConfigServerConfigDataLoader: an unreachable server (ResourceAccessException) is skipped, an error response moves on to the next URI with MultipleUriStrategy.ALWAYS and is thrown with CONNECTION_TIMEOUT_ONLY, and the last URI's error is thrown.

}
}
if (label == null) {
label = this.properties.getLabel();

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The label could be a comma separated list

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The label is now split like the loader does and tried one label at a time, a 404 moving on to the next; the last 404 is thrown.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A list like a,,b yields an empty element, which buildUri treats as "no label" and turns into ?useDefaultLabel. Should empty elements be skipped?

try {
beanFactory.registerSingleton(REST_CLIENT_BEAN_NAME, RestClient.create(restTemplate));
}
catch (RuntimeException ex) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you explain what this is for?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The resolver adds the close listener every time resolveProfileSpecific runs, which is once per import location and per profile pass, and addCloseListener does not deduplicate, so the listener runs several times when the bootstrap context closes; registerSingleton refuses a name that already exists, so the guard makes the registration idempotent. I have put that in a comment.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In production it turns any failure into a debug-level log and a missing ConfigResourceClient bean, which then shows up as an unexplained NoSuchBeanDefinitionException. I'd either let the exception propagate and fix the test double

if (binary) {
request = request.accept(MediaType.APPLICATION_OCTET_STREAM);
}
return request.retrieve().body(Resource.class);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if we should have a configurable limit to the size here so that we dont get into trouble for large files

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added spring.cloud.config.resource-client.max-size (1MB by default). The body is read with exchange into memory up to the limit plus one byte, so a larger file is rejected before it is loaded; a Content-Length above the limit is rejected without reading. Error responses still surface as HttpClientErrorException/HttpServerErrorException.

* {@link RestClient} built from it, as beans for components that call the config
* server at runtime, such as the {@link ConfigResourceClient}.
*/
private void registerRestClient(BootstrapContextClosedEvent event) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Registering the RestTemplate and RestClient as context beans means that an application with its own RestTemplate or RestClient bean gets NoUniqueBeanDefinitionException when it injects by type. These instances also carry the config server's custom headers and OAuth2 interceptor, so an app that autowires one by accident could send those to other hosts.

Also ConfigResourceClient only needs RestClient

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed. The RestTemplate and RestClient are no longer registered as beans. The resolver now registers a package-private ConfigServerRestClientHolder at bootstrap close, which only ConfigResourceClientAutoConfiguration consumes, so an application's own RestTemplate or RestClient injection is untouched and the config server's headers and interceptors cannot be autowired into anything else; the integration test asserts that no RestClient or RestTemplate bean exists after a spring.config.import=configserver: import. The docs no longer mention the beans.

…nfigResourceClient

The client now behaves like the config server import when it reads a
file: the credentials of each URI (explicit, from the URI's user info,
or the configured authorization header) are sent, every URI is tried
according to the multiple-uri-strategy, and a comma-separated label is
tried one label at a time, a 404 moving on to the next.

A file is read into memory up to spring.cloud.config.resource-client.max-size
(1MB by default) and rejected beyond it, so a large file cannot exhaust
the heap.

The RestTemplate and RestClient of the import are no longer registered
as beans, which would clash with an application's own by-type injection
and expose the config server's headers and interceptors. The resolver
registers a package-private ConfigServerRestClientHolder instead, which
only the auto-configuration consumes.

See spring-cloudgh-789

Signed-off-by: Sharang Gupta <sharang@sharanggupta.dev>
@sharanggupta

Copy link
Copy Markdown
Contributor Author

Thanks for the review, @ryanjbaxter. Pushed as a follow-up commit and replied to each point inline.

return this.maxSize;
}

public void setMaxSize(DataSize maxSize) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should validate this in some way. A negative number does not really make sense in this case. Also I wonder if setting it to 0 would be away to disable the limit?

return getResource(profile, label, path, true);
}

private Resource getResource(String profile, String label, String path, boolean binary) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When a URI is skipped nothing is logged. The loader logs "Will try the next url if available". If every attempt fails, only the last exception surfaces, so someone debugging a failed read can't see which servers or labels were tried.

Could we log the skipped URI and label at info or debug level?

}
}
if (label == null) {
label = this.properties.getLabel();

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A list like a,,b yields an empty element, which buildUri treats as "no label" and turns into ?useDefaultLabel. Should empty elements be skipped?

try {
beanFactory.registerSingleton(REST_CLIENT_BEAN_NAME, RestClient.create(restTemplate));
}
catch (RuntimeException ex) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In production it turns any failure into a debug-level log and a missing ConfigResourceClient bean, which then shows up as an unexplained NoSuchBeanDefinitionException. I'd either let the exception propagate and fix the test double

A failed read now shows which servers and labels were tried, and a
RestClient that cannot be built fails instead of hiding the client bean.

Signed-off-by: Sharang Gupta <sharang@sharanggupta.dev>
@Test
void context() {
RestTemplate rest = mock(RestTemplate.class);
when(rest.getErrorHandler()).thenReturn(new DefaultResponseErrorHandler());

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now that I see this, I understand better why the catch was there originally. Lets add it back, an optional component must not prevent startup. Sorry about that!

The ConfigResourceClient is optional, so a RestTemplate that cannot back
a RestClient must not prevent startup; log it at warn to explain the bean.

Signed-off-by: Sharang Gupta <sharang@sharanggupta.dev>

@kvmw kvmw left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved with a minor comment on API visibility.

* @author Sharang Gupta
* @since 5.1.0
*/
public class DefaultConfigResourceClient implements ConfigResourceClient {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This class and it's constructor can stay package private, unless there is a good reason to expose it.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Enhance client side resolution of plain-text files

4 participants