Skip to content

Cache the regions that the server sends with a leave - #1709

Open
lukasIO wants to merge 5 commits into
lukas/region-fetch-earlyfrom
lukas/region-server-reported
Open

lukasIO wants to merge 5 commits into
lukas/region-fetch-earlyfrom
lukas/region-server-reported

Conversation

@lukasIO

@lukasIO lukasIO commented Nov 21, 2026 •

Copy link
Copy Markdown
Contributor

The server sends its region list in LeaveRequest.regions before it asks the client to reconnect. The JS SDK puts that list in its region cache. The Rust SDK dropped it, so the reconnect that follows had to fetch the list again before a failover.

A full reconnect could not have used the list anyway. It passed SignalClient::url to the new session, which is the region URL after a region failover, so the new session looked up regions under the region host.

Change

  • signal_task stores LeaveRequest.regions in the region cache, with the default 5 s TTL as in JS, keyed by the host of the URL that the caller connected with. SignalInner keeps that project URL next to the URL it dialed.
  • Only a wss session is trusted, and only its wss region URLs. The cache is keyed by host, so a list planted on a plaintext ws session would reach a later wss connect to the same host, which sends its token to the region it dials.
  • New SignalOptions::region_url: connect dials that region instead of its url, while the url keys the region list. A failed region is marked failed, and the failover skips it.
  • New SignalClient::project_url returns the url passed to connect. The doc comment of SignalClient::url now says that it is the URL the session dialed.
  • A full reconnect connects with the project URL, and dials the region the session ran in first. As in JS, the failover then goes through the project's regions.

Verified

  • cargo test -p livekit-signaling: 62 tests pass. cargo check -p livekit -p livekit-ffi --tests passes.
  • leave_regions_reach_the_region_cache: the signal task gets one leave with a region list, and a fetch without a token, which the mock answers with a 401, still returns the list. Without the change, the test fails with the 401.
  • server_regions_need_tls: a ws session adds nothing, and a wss session adds only its wss URLs.
  • connect_dials_the_region_url_first: the region is dialed, the project URL is not, and the failover dials the project's regions.
  • The new tests fail without their change.
  • The full reconnect change in reconnect_task has no test of its own: a full reconnect needs a WebRTC session.

🤖 Generated with Claude Code

The server sends LeaveRequest.regions before it asks the client to
reconnect, and the JS SDK already uses them. The signal task now stores
them in the region cache under the project host. SignalInner keeps that
project URL, because after a region failover its own URL is the region URL.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@devin-ai-integration devin-ai-integration Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note

Newer findings are available below. Devin Review posted a newer report on this PR, in addition to the findings presented here.

Devin Review found 2 potential issues.

1 flag not posted on this PR by your GitHub settings — view it in Devin Review. (Configure)

Devin Review

Comment thread livekit-signaling/src/lib.rs Resolved
Comment on lines +153 to +160
let urls: Vec<String> = settings
.regions
.iter()
.filter(|region| !region.url.is_empty())
.map(|region| region.url.clone())
.collect();
if is_cloud_host(&host) && !urls.is_empty() {
region_cache().insert(host, urls, None);

@devin-ai-integration devin-ai-integration Bot Nov 21, 2026 •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟥 Plaintext region overrides poison secure failover

When region_url uses plaintext, set_server_regions still trusts its leave hints if the project URL uses TLS. An on-path attacker can redirect later secure failover and receive another session’s token.

Devin Review


Was this helpful? React with 👍 or 👎 to provide feedback.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in f6464c6: set_server_regions accepts a list only from a wss session, and only wss region URLs. The same exposure exists through region discovery over http for a ws URL. #1601 validates region URLs on that path.

lukasIO and others added 4 commits November 21, 2026 12:58
The region cache is keyed by host. A region list planted on a plaintext
ws:// session would reach a later wss:// connect to the same host, which
sends its token to the first region it dials. set_server_regions now
ignores a session that is not wss, and keeps only wss region URLs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A full reconnect passed SignalClient::url to the new session. After a
region failover, that is the region URL, so the new session keyed its
region lookup by the region host. It never saw the regions cached under
the project host, from a leave or a refresh.

SignalOptions::region_url lets SignalClient::connect dial a region while
the project URL keys the region list, and SignalClient::project_url
returns that URL. A full reconnect now connects with the project URL and
dials the region the session ran in first. When that region fails, it is
marked failed and the failover skips it, as in the JS SDK.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown
Contributor

Changeset ✓

This PR includes a changeset covering all affected packages:

Package Bump
livekit patch
livekit-api patch
livekit-capture patch
livekit-ffi patch
livekit-signaling minor

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 1 new potential issue.

2 flags not posted on this PR by your GitHub settings — view them in Devin Review. (Configure)

Devin Review

Comment on lines +156 to +158
if !url.starts_with("wss://") {
return;
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 HTTPS joins discard leave regions

When a room joins through an https:// URL, set_server_regions discards the leave’s region list despite the secure WebSocket. Reconnect falls back to an older list or fetches one again.

Learn more

get_livekit_url converts an HTTPS join URL to a secure WebSocket URL before opening signaling. signal_task passes the original project URL to the cache when a server asks the client to leave, so the prefix check rejects that secure session. SignalClient::connect then looks up the project’s cached list during the full reconnect.

Example: Join https://my-project.livekit.cloud, then receive a leave naming region B. The SDK discards the list, so reconnect cannot use that hint and must rely on an older entry or fetch the list.

Recommended fix: Keep the project URL as the cache key, but pass the actual signaling endpoint separately and check its normalized scheme for TLS. Add a regression test for a secure join made through an HTTPS project URL.

Devin Review


Was this helpful? React with 👍 or 👎 to provide feedback.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant