Skip to content

Overlap ordinary track registration and SDP negotiation - #1704

Open
xianshijing-lk wants to merge 2 commits into
mainfrom
sxian/fast-track-publication
Open

xianshijing-lk wants to merge 2 commits into
mainfrom
sxian/fast-track-publication

Conversation

@xianshijing-lk

@xianshijing-lk xianshijing-lk commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor

Ordinary publish_track() currently waits for TrackPublishedResponse before attaching its sender and requesting SDP negotiation. This ports the registration/negotiation overlap from livekit/client-sdk-js#1228 into Rust: send AddTrack, then request negotiation without waiting for its acknowledgment. Publication still completes only after server acceptance.

The existing public API and RoomOptions::publish_tracks join flow from #1050 remain intact. Both paths reuse preparation, sender configuration, and publication completion helpers.

Implementation

  • Overlap when the requested codec is advertised, the actual prepared sender preferences use that codec, and publication does not require encryption or packet trailers. Other configurations retain sequential publication.
  • JoinResponse.fast_publish continues to control publisher warm-up and the existing negotiation scheduling policy. It does not gate registration/SDP overlap; codecs advertised with fast_publish = false still permit overlap, including when the server suppresses warm-up for ICE fallback.
  • Prepare native video codec preferences once. Inspect that exact vector for eligibility and move it into sender configuration. Empty preferences or a native fallback to a different codec force the sequential path.
  • Register the response waiter before sending AddTrack, and send AddTrack before requesting an offer through the existing negotiation coordinator. No new production tasks or negotiation scheduler.
  • Keep the transaction and completion within one connection generation. Sequential encrypted publication installs its cryptor before requesting negotiation.
  • A scoped guard claims each track and rolls back pending registration, speculative sender, and transceiver on rejection, timeout, disconnect, or cancellation. A duplicate cannot remove the original sender. Also remove a newly created native sender if codec configuration fails.

Deliberate scope

Unlike JS's server-codec clamping, this change does not automatically replace the application's chosen video codec with a server-supported alternative. A mismatch takes the sequential path, preserving Rust's existing codec-selection behavior. Server-driven codec substitution would change publication options and encoding selection and belongs in a separate change.

Rollback continues requesting reconciliation after removing an abandoned sender. Normal sequential setup and completion have no intervening await; the attached-sender failure window is session closure, where the existing coordinator suppresses negotiation. A simple fast check is also insufficient to prove a sender was never included in another concurrent offer. has_published retains its existing transport-history semantics.

Validation

Tested on macOS with livekit-server 1.13.10 on isolated ports with loopback candidates: 95 relevant tests passed.

  • 32 fast-publication integration cases, including a 16-case audio/video × single/dual PC × warm-up on/off × codecs present/absent ordering matrix. Withheld acknowledgments prove overlap structurally; completion still requires acknowledgment.
  • Connect with/without initial tracks, concurrent muted audio/video, missing-codec/encryption/packet-trailer fallback, accepted/in-flight duplicates, rejection/cancellation/disconnect cleanup, and full reconnect.
  • A session unit test rejects actual fallback/mixed/empty codec preferences even when the requested codec is advertised. The exact validated preference vector is consumed by sender setup.
  • 18 existing publish-with-join tests, 28 peer-connection signaling tests, the engine lifecycle test, and 16 session unit tests passed.
  • Changeset detection now covers the full dependent closure: livekit, livekit-capture, and livekit-ffi, with no missing bumps.
  • cargo check -p livekit --features __lk-e2e-test,native-tls, minimal-feature check, formatting, and diff checks.

The proxy uses bounded queues and explicitly owned forwarding tasks. Cleanup tests check sender removal and observable Rust track destruction.

Measured improvement

Five samples per arm on the final revision, alternating arm order, with 50 ms added per signaling direction (100 ms RTT), unchanged media transport, and dual PC with the publisher data channel verified ready before timing. Both arms retain eager negotiation; clearing advertised codecs selects the sequential baseline to isolate overlap from warm-up and debounce.

Median Sequential fallback Parallel publication
AddTrack → track offer 109 ms 1 ms
Publish start → first received video frame 405 ms 305 ms

That is 100 ms less time to first video in this controlled run. The publish acknowledgment still takes approximately one signaling RTT. Raw samples print FAST_PUBLISH_SAMPLE; latency thresholds are intentionally not asserted.

SPC runs emit libwebrtc codec-collision diagnostics during renegotiation in both policies; the functional assertions pass. The benchmark uses dual PC to isolate publisher publication from SPC subscriber renegotiation; these numbers are not a general SPC latency result.

Reproduction instructions and assumptions are in livekit/tests/README.md:

LIVEKIT_TEST_SERVER_URL=ws://127.0.0.1:17880 \
FAST_PUBLISH_BENCH_ITERATIONS=10 \
cargo test -p livekit --features '__lk-e2e-test,native-tls' \
  --test fast_publish_test -- --nocapture --test-threads=1

Includes patch changesets for the affected crate and its managed dependents. The proxy adds only a dev dependency on the existing resolved tokio-tungstenite version, with no new transitive packages.

@github-actions

github-actions Bot commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor

Changeset ✓

This PR includes a changeset covering all affected packages:

Package Bump
livekit patch
livekit-capture patch
livekit-ffi patch

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 1 potential issue.

Devin Review

Comment on lines +560 to +563
if track.transceiver().is_some() || !publishing.insert(cid.to_owned()) {
return Err(EngineError::Internal(
"track is already publishing or published".into(),
));

@devin-ai-integration devin-ai-integration Bot Nov 19, 2026 •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Join recovery blocked by obsolete senders

After a full reconnect during join, PendingTrackPublish::new rejects tracks still awaiting join completion. complete_pre_publish retries them normally, but session teardown leaves their old transceivers attached to the Rust tracks. Room::connect then fails instead of recovering.

Learn more

Join-time publication creates and stores a transceiver before connecting, in create_early_publisher. A full reconnect can replace that session before all join-time tracks reach publication completion. complete_join_track detects the replaced session and returns false, so complete_pre_publish falls back to ordinary publication. SessionInner::close removes the native senders but does not clear those tracks' stored transceivers. The new duplicate check therefore rejects the fallback, and Room::connect closes the recovered room and returns an error.

Example: Connect in single-PC mode with two initial tracks. A server-requested full reconnect replaces the session before the second track's completion acquires the reconnection lock. The second track enters ordinary publication with its original transceiver still stored and receives an already-published error instead of a new sender.

Recommended fix: Clear the obsolete join-time transceiver when complete_join_track reports that the session was replaced, before calling ordinary publication. Keep duplicate rejection for transceivers belonging to live publications. Add a regression test that replaces the initial session before all join-time publications complete.

Devin Review


Was this helpful? React with 👍 or 👎 to provide feedback.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant