Skip to content

fix(client): write websocket frames from a writer thread, fixing the JRuby deadlock - #680

Merged
route merged 5 commits into
mainfrom
fix/websocket-writer-thread
Oct 26, 2026
Merged

route merged 5 commits into
mainfrom
fix/websocket-writer-thread

Conversation

@route

@route route commented Oct 26, 2026 •

Copy link
Copy Markdown
Member

Fixes #676.

Problem

#644 replaced the blocking websocket write with write_nonblock + IO.select, so a browser that stops reading couldn't block the sending thread forever while it held the send lock. On JRuby that deadlocks: IO#readpartial waits in a NIO select holding the channel's blocking lock, and write_nonblock/IO.select need the same lock to switch the channel to nonblocking mode. The reader thread is parked in readpartial before the handshake is written, so Ferrum::Browser.new hangs forever. It's not only the handshake: any later write waits until the browser happens to send something.

Change

  • Revert fix(client): give up on a websocket write the browser doesn't take #644.
  • Frames are written by a dedicated writer thread. Callers and the driver only encode frames under the lock and queue them; the writer is the only code writing the socket and the reader the only code reading it, both with plain blocking calls. No lock is held across socket I/O, and the reader keeps delivering responses and events while a write is stuck.
  • Each queued frame carries its command's deadline and is dropped if that passes before it's written, so a command queued behind a large one fails after its own timeout:, not the large one's.
  • Connection health is separate from command timeouts: frames go out in 64KB chunks, and a chunk the browser takes no data of for protocol_timeout closes the connection. Commands still waiting get DeadBrowserError with the reason. A shorter pause doesn't kill the connection anymore.
  • Restores the rest of fix(client): give up on a websocket write the browser doesn't take #644: the command timeout is passed down to the websocket, and a pending command whose send raised is forgotten.

Behavior changes

  • A command whose own timeout expires while its frame is being written raises TimeoutError at that timeout. The frame can't be cut off midway, so it still goes out if the browser resumes, and the late response is ignored, as for any timed-out command.
  • Client::WebSocket#send_message returns once the frame is queued.

JRuby in CI

  • The test matrix runs on jruby with Java 21. rbs is installed on MRI only, it doesn't build on JRuby.
  • The test server is stopped after the suite: on JRuby Puma's reactor isn't interrupted by the kill at exit, so the process never exited.
  • The websocket test proxy pipes with readpartial/write instead of IO.copy_stream from two threads, which hit the same JRuby lock.
  • browser.pages, windows(:last) and workers came back in random order on JRuby: targets were kept in a Concurrent::Map, which is an insertion-ordered Hash on CRuby but a hash-ordered Java map on JRuby. They're kept in a Concurrent::Hash now.

route added 5 commits October 26, 2026 13:21
This reverts commit 080aadc (#644).

On JRuby IO#readpartial waits in a NIO select while holding the channel's blocking lock, and write_nonblock
and IO.select need that same lock to switch the channel to nonblocking mode. With the reader thread parked
in readpartial, the handshake write never returns and Ferrum::Browser.new hangs forever. Any write after
that waits for the browser to send something first, so the approach can't work on JRuby at all.

Conflicts with the logger rework (#658) are resolved by keeping its logging. The 0.19.0 changelog entry
stays, the history of that release doesn't change.

Fixes #676
Brings back the protection from #644 in a way that works on JRuby: a browser that stops reading its socket
blocked the sending thread forever while it held the lock every other command needs to be sent.

Callers and the driver now only encode frames and queue them; a writer thread is the only code writing the
socket, with blocking writes, and the reader the only code reading it. No lock is held across socket I/O.
Each queued frame carries its command's deadline and is dropped if that passes before it's written, so a
command behind a large one fails after its own timeout. A chunk the browser takes no data of for
protocol_timeout closes the connection, and the commands still waiting get DeadBrowserError with the reason.

Restores the rest of #644 as well: the command timeout is passed down to the websocket, and a pending
command whose send raised is forgotten.
… hand

On JRuby the process never exited after the suite: Puma was only killed at exit, and killing doesn't
interrupt its reactor's select. The websocket test proxy used IO.copy_stream from two threads on the same
sockets, which hits the same JRuby lock as #676, so browsers behind it timed out.
Concurrent::Map is a Hash on CRuby but a hash-ordered Java map on JRuby, so pages, windows(:last) and
workers came back in a different random order per process. Targets are kept in a Concurrent::Hash and
iterated over a snapshot, so a target added by the event thread meanwhile can't break the iteration.
@route
route merged commit 3fb1a00 into main Oct 26, 2026
7 checks passed
@route
route deleted the fix/websocket-writer-thread branch October 26, 2026 12:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

0.19 deadlocks on JRuby: the websocket handshake write never returns

1 participant