Repository navigation
server: configurable gRPC max concurrent streams - #1272
Merged
Merged
Conversation
While Redis was unresponsive, every rate limit call blocked inside the service, and with no limit on concurrent streams the blocked calls, and the goroutines and memory serving them, kept piling up for as long as Redis stayed away. Add GRPC_MAX_CONCURRENT_STREAMS (default 0, the gRPC default) to bound the concurrent calls per gRPC connection, so excess calls wait on the client side instead. Signed-off-by: Ignasi Barrera <ignasi@tetrate.io>
Clarified timeout settings for Redis connection establishment. Signed-off-by: Prabhjot Singh <86856448+psbrar99@users.noreply.github.com>
Contributor
|
Overlaps with #1241, which also adds GRPC_MAX_CONCURRENT_STREAMS |
psbrar99
previously approved these changes
Oct 15, 2026
Signed-off-by: Ignasi Barrera <ignasi@tetrate.io>
psbrar99
approved these changes
Oct 21, 2026
nacx
added a commit
to nacx/ratelimit
that referenced
this pull request
Oct 22, 2026
radix v4 hard-codes a 10s TCP keepalive on the connections it dials, and nothing sets TCP_USER_TIMEOUT. Keepalive probes are only sent on idle connections, so they cannot detect a dead peer while there is unacknowledged data in flight, which is exactly the state of a busy connection to a Redis server that vanished. The kernel then only closes it after net.ipv4.tcp_retries2 retransmissions, about 15 minutes by default. Add REDIS_TCP_USER_TIMEOUT / REDIS_PERSECOND_TCP_USER_TIMEOUT (Linux only, off by default) to bound how long sent data may stay unacknowledged, and REDIS_TCP_KEEPALIVE / REDIS_PERSECOND_TCP_KEEPALIVE (default 10s, as radix did; negative disables it). These complement the I/O timeout: TCP_USER_TIMEOUT catches a peer that stopped acknowledging, the I/O timeout catches one that acknowledges but never replies. golang.org/x/sys becomes a direct dependency. Also restore the "Connection Timeout" README intro, which envoyproxy#1272 reverted to "connection establishment" only while REDIS_TIMEOUT also bounds reads and writes since envoyproxy#1271. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Ignasi Barrera <ignasi@tetrate.io>
psbrar99
pushed a commit
that referenced
this pull request
Oct 22, 2026
radix v4 hard-codes a 10s TCP keepalive on the connections it dials, and nothing sets TCP_USER_TIMEOUT. Keepalive probes are only sent on idle connections, so they cannot detect a dead peer while there is unacknowledged data in flight, which is exactly the state of a busy connection to a Redis server that vanished. The kernel then only closes it after net.ipv4.tcp_retries2 retransmissions, about 15 minutes by default. Add REDIS_TCP_USER_TIMEOUT / REDIS_PERSECOND_TCP_USER_TIMEOUT (Linux only, off by default) to bound how long sent data may stay unacknowledged, and REDIS_TCP_KEEPALIVE / REDIS_PERSECOND_TCP_KEEPALIVE (default 10s, as radix did; negative disables it). These complement the I/O timeout: TCP_USER_TIMEOUT catches a peer that stopped acknowledging, the I/O timeout catches one that acknowledges but never replies. golang.org/x/sys becomes a direct dependency. Also restore the "Connection Timeout" README intro, which #1272 reverted to "connection establishment" only while REDIS_TIMEOUT also bounds reads and writes since #1271. Signed-off-by: Ignasi Barrera <ignasi@tetrate.io> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
While Redis is unresponsive, every rate limit call is blocked inside the service, and with no limit on concurrent streams, the blocked calls, and the goroutines and memory serving them, keep piling up for as long as Redis stays away.
This PR adds
GRPC_MAX_CONCURRENT_STREAMS(defaults to 0, the gRPC default) to bound the concurrent calls per gRPC connection, so excess calls wait on the client side instead.