pipe() doesn't create a blocking pipe

[emaxx-google]

On Emscripten, when reading from a pipe created via pipe(), the read() call returns immediately with EAGAIN when there's no data in the pipe. This contradicts the standard behavior specified by POSIX:

  EAGAIN The file descriptor fd refers to a file other than a
         socket and has been marked nonblocking (O_NONBLOCK), and
         the read would block.  See open(2) for further details on
         the O_NONBLOCK flag.
  EAGAIN or EWOULDBLOCK
         The file descriptor fd refers to a socket and has been
         marked nonblocking (O_NONBLOCK), and the read would block.

https://man7.org/linux/man-pages/man2/read.2.html

  If a process attempts to read from an empty pipe, then read(2)
  will block until data is available. <...>
  Nonblocking I/O is possible by using the fcntl(2)
  F_SETFL operation to enable the O_NONBLOCK open file status flag.

https://man7.org/linux/man-pages/man7/pipe.7.html

Given that pipe() doesn't set the O_NONBLOCK flag, I believe the Emscripten's behavior with returning EAGAIN is wrong.

Sample program:

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
int main(int argc, char** argv) {
  int pipefd[2];
  if (pipe(pipefd)) {
    perror("pipe() failed");
    exit(-1);
  }
  if (argc > 1) {
    char c = 0;
    if (write(pipefd[1], &c, 1) != 1) {
      perror("write() failed");
      exit(-1);
    }
  }
  char c;
  if (read(pipefd[0], &c, 1) != 1) {
    perror("read() failed");
    exit(-1);
  }
  exit(0);
}

This program, when run under Emscripten, fails with "Resource temporarily unavailable". When the same program is run on Linux using standard g++ and libc, it correctly blocks in read().
P.S. The program also demonstrates that, in case write() was previously called, the data is successfully read from it under Emscripten.

[sbc100]

Blocking is very tricky on the web platform. Unless you are running on the main thread you can't really block. You have to condtant yield to the event loop. In emscripten we can do this using ASYNCIFY but it has a cost and is not on by default. If you can think of a reasonable way simulate/emulate a blocking pipe we would welcome it. Or a better way to report that we don't support such things?

[sbc100]

Sorry I mean to write "Unless you are running off the main thread".

[emaxx-google]

I also tested the same snippet in a background thread, and the result was the same - it returns EAGAIN instead of blocking.

Also if a primitive like that isn't supported on a main thread, then probably the following docs need to be updated (they're written to suggest that all such blocking operation work on the main thread, even at a cost of runtime penalty): https://emscripten.org/docs/porting/pthreads.html#blocking-on-the-main-browser-thread

[sbc100]

If you can think of a way to discuss that issue in that document that could be good yes. That document mostly talks about threading APIs though. Perhaps https://emscripten.org/docs/porting/asyncify.html could be another possible place to mention this since its one way it could be possible to create a blocking pipe.

Since emscripten is single process and doesn't support any kind of fork() i'm not sure how useful pipe really is for our users. Maybe for multi-threaded application one might want to pipe data between threads? I'm curious what you use case for them is?

[emaxx-google]

It's true that pipe() is intended for multi-process communication, which doesn't make sense in Emscripten. But there's some amount of existing code that uses it as a threading primitive. In the codebase I'm migrating (which is a cross-platform *nix code that also used to work under NaCl correctly) the pipe() call is there: https://github.com/LudovicRousseau/PCSC/blob/113f74a1d7189ced85038b890253f2f3b477c3f0/src/hotplug_libusb.c#L561

And, yeah, it's confusing to see that the pipe() function returns success in Emscripten, but subsequent operations don't really work as expected. If pipes aren't supported at all, it'll be much less surprises if pipe() would return an error straight away. Application authors like me would have to find a workaround, but the problematic piece would be much easier to discover than in the current situation. Of course, it'll be even cooler if Emscripten would be fixed to support pipes fully, so that no workarounds are needed :)

[sbc100]

I agree that maybe failing to create the pipe at all might be more clear. I'd be happy with that approach if the user is building in a configuration where blocking pipes are clearly not possible (e.g. without ASYNCIFY (which could one day allow for actual blocking pipes although I don't think it does today)).

Alternatively we could change the error return into a fatal abort and report "attempt to read from empty pipe without NONBLOCK.. this is not supported on emscripten because we can't block". Then certain programs that don't need blocking would continue to work and developers who rely on blocking would see a fatal error. Either way I think is fine given that there are probably very few uses for pipe in emscripten today.

[emaxx-google]

Thanks, these options will be a good way to tell developer about the problem. Also would be great to mention it in docs - it helps when you know which unsupported functions you have to look for in the codebase.

[femski]

I had exact same question here:
#10556

and answer then was that its not possible.

But now that we have support for atomic.waitAsync in browser:
https://www.chromestatus.com/feature/6243382101803008

couldn't this be done using native browser feature (instead of asyncify which could also use same browser feature)?
Instead of waiting for a pipe, threads would wait for a piece of memory protected by atomics.

Is support for atomics coming in emscripten/webassembly also?
This will enable a broader range of event driven applications ported to WASM then.

attn: @kripken @sbc100

[sbc100]

Sure if you want to do blocking pipes with multiple threads it could be possible to do with atomics. We do a lot of inter-thread communication within emscripten using atomics already.

What won't work (at least not without ASYNICIFY) is a single threaded app where the pipe is fed, for example, from some kind of JS event (like a fetch request).

[stale]

This issue has been automatically marked as stale because there has been no activity in the past year. It will be closed automatically if no further activity occurs in the next 30 days. Feel free to re-open at any time if this issue is still relevant.

[emaxx-google]

Stale bot intends to close this issue, but I believe it'll still be a valuable addition: POSIX doesn't forbid in-process usages of pipe(), and it's not unusual for Linux code to use it as a simple equivalent to mutexes/conditionvariables/queues. Silent deviation from the POSIX behavior makes this harder to discover.

[sbc100]

I guess we could error on pipe creation if O_NONBLOCK is not set? (either return an error or just abort). If thats sounds a like a reasonable solution, perhaps you would like to make a PR? Otherwise, we can leave this open an mark as help wanted and good first bug and somebody should pick it up.

[emaxx-google]

I'll probably not have cycles to work on this anymore soon. Erroring and/or logging would definitely be much more developer-friendly! Implementing POSIX behavior would be even better, as this would mean developers don't have to rewrite pipe() calls when porting to Emscripten, but I can't judge how difficult would implementing it be.

[sbc100]

Any kind of blocking behaviour on the web is on the more complex/hard problems we face. So failing on the creation of blocking pipes seems like that most reasonable solution for now.