[guybedford]

Holds a blocking pipe writer to the pipe's capacity where it can wait for the reader: from a pthread, on the JS FS. Builds on #28082.

With #28082 a pipe holds 64 KiB and a non-blocking writer gets EAGAIN when it is full, but a blocking writer is still let through without bound, since fd_write has no way to wait. A pthread's write is proxied to the main thread and the pthread sits on a futex until the call completes, so on a pthread's behalf fd_write holds the call open where the stream would block, as a blocking write does on Linux:

• PIPEFS holds a writer to the capacity (with the PIPE_BUF all-or-nothing rule) when fd_write can wait for the reader, which it marks on the stream around each write it issues, as it already does for a non-blocking writer.
• fd_write becomes an __async: 'auto' function taking the sync argument of Pass sync to __async: 'auto' library functions #28022. When proxied it writes through doWritevBlocking: on EAGAIN from a blocking stream it parks on the pipe's wait-queue for POLLOUT and continues once the reader frees room, completing the whole request. It stays synchronous when nothing blocks, so ordinary pthread writes are unchanged. An error after some data went gives the short count, and recurs on the next call, so a writer blocked on a pipe whose reader closes gets the count that fit, then EPIPE.

The wait completes synchronously rather than through a Promise. The room is freed by the reader's own proxied read, which the main thread services even while it is itself blocked (main() in pthread_join, or a mutex, or sleep) and never returning to its event loop, so a microtask could never deliver the result and the writer would hang. Waits instead hand back a SyncDeferred, a thenable whose then fires on settle: the proxy completion in libpthread.js calls it directly, delivering a plain value or a settled deferred on the spot, while a real Promise still defers. Where the stack suspends under ASYNCIFY/JSPI the result is awaited, which adopts the thenable, so the one return serves every waiting stack. poll() and epoll_wait() move to the same form, since a pipe wakes them from the same source; a blocking poll() on a pipe from a pthread hung in exactly the same way. Under JSPI the main-thread wrapper of an 'auto' body now also dispatches a proxied call directly instead of through the async helper, as it already did under ASYNCIFY, so the result is not re-wrapped in a Promise.

The sync argument is bound by the caller, not read from PThread.currentProxiedOperationCallerThread. That global is set for the whole proxied call, including any wasm the call re-enters, so a blocking write the main thread made inside a MAIN_THREAD_EM_ASM from a pthread looked like the pthread's own. A sync-proxied 'auto' body in a pthread build now keeps sync as its own trailing parameter, declared as <name>$sync, and its two callers bind it: the symbol's own function, which is what wasm imports, passes true, and the proxied table entry passes false. So only the directly proxied call acts on the pthread's behalf.

function _fd_write$sync(fd, iov, iovcnt, pnum, sync) { ... }
function _fd_write(fd, iov, iovcnt, pnum) {
  if (ENVIRONMENT_IS_PTHREAD) return proxyToMainThread(4, 0, 2, fd, iov, iovcnt, pnum);
  return _fd_write$sync(fd, iov, iovcnt, pnum, true);
}
// proxiedFunctionTable
(fd, iov, iovcnt, pnum) => _fd_write$sync(fd, iov, iovcnt, pnum, false),

Since the symbol keeps its own name and the import's arity, the import object, MAIN_MODULE's by-name sends and exports all reach it unchanged. This shape measured at parity with main per write(), where giving the import itself an extra parameter, defaulted or not, measured about 7% slower. Only functions with a trailing sync under __proxy: 'sync' in a pthread build take this form; every other function is emitted as before.

fd_write is 'auto' only under pthreads without ASYNCIFY, as 'auto' would otherwise make the import itself suspend, which every printf from a callback cannot afford under JSPI. So on the main thread, and in ASYNCIFY builds, a blocking write to a full pipe remains unbounded for now; WasmFS is unchanged. Builds without pthreads, and ASYNCIFY builds, emit the same fd_write and PIPEFS as before. A pthread (including main() under PROXY_TO_PTHREAD) that writes more than 64 KiB to a pipe nothing reads until after it is joined now blocks, as on Linux, where it previously completed; the ChangeLog notes this.

Tests: other.test_pipe_write_blocking streams 1 MiB through a 64 KiB pipe with a slow reader thread blocking in poll(), including a single 256 KiB write, and checks a writer blocked on a full pipe gets EPIPE once the reader closes. It also writes 256 KiB from the main thread inside a MAIN_THREAD_EM_ASM issued by a pthread, expecting the full count at once, and has the main thread drain a pthread's 256 KiB write, checking the write completed only once all but a capacity's worth had been read. The scenario runs on a pthread that main() joins, under PROXY_TO_PTHREAD, as a plain -pthread build where main() spins in pthread_join on the main thread, and under MAIN_MODULE.

Made with AI assistance under my review

[gemini-code-assist]

Code Review

This pull request introduces a 64 KiB capacity limit for pipes, aligning Emscripten's pipe behavior with Linux. It implements non-blocking write failures with EAGAIN when a pipe is full, and blocking write support for pthreads. The changes span JS library files (libpipefs.js, libpthread.js, libsyscall.js, and libwasi.js) and WasmFS C++ files, along with new test cases. Feedback on the changes highlights a bug in __syscall_poll where O_RDWR pipes are not polled correctly for both reading and writing. Additionally, using instanceof Promise in libpthread.js and libwasi.js should be replaced with a more robust thenable check to avoid cross-realm issues.

[PedroRosalba]

Tried this branch (fd1b763) against main (05afea5) on node 24 with the 6.0.13 LLVM. With plain -pthread (no PROXY_TO_PTHREAD), a blocking write that has to wait never finishes if main() is in pthread_join on the writer:

// emcc -pthread -sEXIT_RUNTIME join_writer.c
#include <pthread.h>
#include <stdio.h>
#include <string.h>
#include <unistd.h>

#define TOTAL (256 * 1024)
int p[2];

void* writer(void* arg) {
  static char buf[TOTAL];
  memset(buf, 'x', sizeof buf);
  write(p[1], buf, sizeof buf);
  close(p[1]);
  return NULL;
}

void* reader(void* arg) {
  char buf[4096];
  size_t got = 0;
  while (got < TOTAL) {
    ssize_t n = read(p[0], buf, sizeof buf);
    if (n > 0) got += n;
    else usleep(1000);
  }
  return NULL;
}

int main() {
  pipe(p);
  pthread_t w, r;
  pthread_create(&r, NULL, reader, NULL);
  pthread_create(&w, NULL, writer, NULL);
  pthread_join(w, NULL);
  pthread_join(r, NULL);
  printf("done\n");
}

On main it prints done. On this branch it hangs once the reader has drained the first 64 KiB. With a console.error added around doWritevBlocking's wait, the run shows:

pipe full, writer waits
room freed, Promise resolved

and never reaches the .then(step) continuation. The reader's proxied read runs while the main thread spins in pthread_join, so the wait's Promise resolves there, but the continuation is a microtask and the main thread doesn't get back to its event loop. That's the same thing test_stdio_locking ran into, only on the path where the write really waits. With PROXY_TO_PTHREAD the same program passes, and so does a 32 KiB version where the pipe never fills.

Separately, the ChangeLog line reads as if it applies everywhere, but WasmFS blocking writes are still unbounded (the same program under -sWASMFS passes on both trees, with no cap). Saying "(JS FS)" there would match the change.

A direction that avoids the event loop entirely (I haven't prototyped it): let the blocked pthread do the waiting rather than the main thread. When the pipe is full, fd_write returns at once with what fit, and the pthread then Atomics.waits on a shared word that the reader's side bumps and notifies when it frees room, then re-issues the rest. Notify is synchronous, so it still works while the main thread spins in pthread_join, and fd_write stays a plain synchronous import in every build, ASYNCIFY/JSPI included. Does that fit how you'd want this to work, or is there a reason the wait has to live on the main thread?

[guybedford]

Thanks for the feedback on this @PedroRosalba I'm pushing a change now to support the non-proxied pthread wait scenario.

[PedroRosalba]

Thanks, the plain -pthread case from my earlier comment passes now, and so does the PROXY_TO_PTHREAD one.

One regression against main: a blocking write() made by the main thread while it is servicing any proxied call goes down the pthread path. For example, a pthread runs MAIN_THREAD_EM_ASM_INT({ return _main_writer($0); }, 100000), and main_writer does write(p[1], buf, 100000) on a pipe nobody is reading yet. On main that returns 100000. On this PR it returns whatever count was left on the stack (249848 in one of my builds, 0 in the repro below), and a later 4-byte write() from main() lands at offset 65536, ahead of the last 34464 bytes. Repro:

#include <emscripten.h>
#include <fcntl.h>
#include <pthread.h>
#include <stdio.h>
#include <string.h>
#include <unistd.h>
#define N 100000
int p[2];
EMSCRIPTEN_KEEPALIVE int main_writer(int n) {
  static char buf[N];
  memset(buf, 'x', sizeof buf);
  return write(p[1], buf, n);
}
void* thr(void* arg) {
  printf("write returned %d\n", MAIN_THREAD_EM_ASM_INT({ return _main_writer($0); }, N));
  return NULL;
}
int main() {
  pipe(p);
  pthread_t t;
  pthread_create(&t, NULL, thr, NULL);
  pthread_join(t, NULL);
}

Built with -pthread -sEXIT_RUNTIME, it prints write returned 0 on this PR and write returned 100000 on main.

The cause: the sync argument (!PThread.currentProxiedOperationCallerThread in handleSyncArg) and the capacity check in libpipefs.js both read a global that is set for the whole time a proxied call is being serviced, including any wasm that the call re-enters. So the nested write() looks like a pthread's. When I wrap the fd_write import in the generated JS, the nested call returns a SyncDeferred while the global holds the pthread's address. Wasm converts the object to 0 (success), so *pnum is never stored, and the listener writes the rest later. If the EM_ASM saves the global, sets it to 0 around _main_writer and restores it afterwards, the same program returns 100000 on this PR.

One option is for the proxying path to hand the "on behalf of a pthread" bit to the function it calls, and to nothing that function calls in turn. Do you see a reason the global has to stay visible to nested calls?

Two tradeoffs that may be worth a line in the ChangeLog or the PR body:

• A pthread that writes more than 64 KiB to a pipe that the main thread only reads after pthread_join now deadlocks, where it completed before. That matches Linux, but it's a new hang for code that used to work. Whether a program hangs can also depend on the build: main() writing 100 KB to a pipe and then reading it back passes with plain -pthread and hangs with -sPROXY_TO_PTHREAD, since main() is then a pthread.
• The cap holds only while the reader is a pthread. When the main thread reads, the wait resumes from a call that isn't proxied, so the rest goes in without a cap. A 1 MiB write() from a pthread returned once the main thread had read 4 KiB. With [FS] Limit pipes to 64 KiB in all cases and use a single buffer #28087 that tail would get EAGAIN instead, so this may resolve itself once both land.

Cost: a hello world with -O2 -pthread -sFORCE_FILESYSTEM grows by 1916 B of JS (558 B gzipped) even when it never uses a pipe, and no codesize test covers that combination.

AI-generated with Claude, reviewed by me before posting.

[guybedford]

Thanks, all four points were right.

Nested write. Reproduced and fixed. Both reads of PThread.currentProxiedOperationCallerThread are gone. The sync argument is now bound by the caller: a sync-proxied 'auto' body in a pthread build keeps sync as its own trailing parameter, the wasm import entry calls it with true, and the proxied table entry calls it with false. So only the directly proxied call acts on the pthread's behalf, and anything it re-enters on the main thread takes the main-thread path. PIPEFS likewise no longer looks at the global: doWritevBlocking marks the stream as able to wait around each write it issues, and the capacity hold keys off that. Your repro now returns 100000 and the sequence reads back in order. It's added to test_pipe_write_blocking in both variants.

The import entry stays a function of the import's exact arity rather than taking sync as a defaulted parameter, since an extra parameter on the import, defaulted or not, measured about 7% slower per write(); this shape is at parity with main.

Deadlock. Documented in the ChangeLog and PR body, including that main() under PROXY_TO_PTHREAD is a pthread.

Cap when main reads. Fixed by the same stream mark, since the resumed write sets it regardless of which thread's read woke it. Added a test where the main thread drains a pthread's 256 KiB write and the write is checked to complete only once all but a capacity's worth was read.

Size. Confirmed at +1915 B for that combination. The bulk is doWritevBlocking and SyncDeferred; a one-shot SyncDeferred without chaining saved nothing after minification, so I left it. No codesize test covers -pthread -sFORCE_FILESYSTEM, and I'd rather not add a baseline for this PR alone.

AI generated under my review

[sbc100]

What is "this thread" you are referring to?

This function (_emscripten_receive_on_main_thread_js) can only ever run on the main thread right?

[guybedford]

Yeah it's the main thread, so if the main thread is itself blocked on the calling thread it can still be handling proxied calls just not suspending to the event loop and a microtask can't run.

[guybedford]

(comment updated)

[sbc100]

Is there a reason to move this assertion up like this? Maybe this can be split out and landed separately?

[guybedford]

This PR makes a proxied call able to complete synchronously which wakes the caller and it can return before we get back here. f->owned becomes a use-after-return, which was caught by the lsam wasm64 job. While this case doesn't affect the main branch, it is very much associated with this PR change.