[GHC] #13296: stat() calls can block Haskell runtime

[GHC]

#13296: stat() calls can block Haskell runtime
-------------------------------------+-------------------------------------
           Reporter:  nh2            |             Owner:  (none)
               Type:  bug            |            Status:  new
           Priority:  normal         |         Milestone:
          Component:  Runtime        |           Version:  8.0.2
  System                             |
           Keywords:                 |  Operating System:  Unknown/Multiple
       Architecture:                 |   Type of failure:  Runtime
  Unknown/Multiple                   |  performance bug
          Test Case:                 |        Blocked By:
           Blocking:                 |   Related Tickets:
Differential Rev(s):                 |         Wiki Page:
-------------------------------------+-------------------------------------
 `getFileStatus` (`stat()` syscall) is marked as `unsafe`, which means that
 if I have e.g. `+RTS -N4`, I can't stat more than 4 files at the same time
 without completely stopping the Haskell world.

 This is an issue on network file systems, where a single `stat()` can
 easily take 2 milliseconds, so you typically want to do them in parallel
 (but due to the above you can't).

 The underlying problem is that there are some Linux syscalls you typically
 need on networked file systems that have no asynchronous equivalent;
 according to http://blog.libtorrent.org/2012/10/asynchronous-disk-io/
 these are at least:

 * `stat()`
 * `open()`
 * `fallocate()`
 * `rename()`

 A quick skim through `libraries/base/System/Posix/Internals.hs` reveals
 the situation:

 * `stat()` only exists as `unsafe`
 * `open()` has both `safe` and `unsafe` variants (I haven't checked which
 one is used in practice)

 The remaining ones are in the `unix` package

 * `fallocate()` is `safe`
 * `rename()` is `unsafe`

 It seems to me that there are two issues here:

 1) None of these calls should be `unsafe` because they may block for a
 very long time (e.g. > 0.5 ms even on the fastest LANs).

 2) We need to answer the question: If we marked them as `safe`, how many
 of them would the RTS execute in parallel? To my current knowledge (thanks
 `rwbarton` and `slyfox` on #ghc), there's a pool of Haskell executing
 threads (the usual `-RTS -N`), and a pool of FFI threads. Are there any
 restrictions on the size of that latter pool? The docs
 https://downloads.haskell.org/~ghc/latest/docs/html/users_guide/ffi-
 chap.html#foreign-imports-and-multi-threading are not specific on that
 topic, simply mentioning "but there may be an arbitrary number of foreign
 calls in progress at any one time, regardless of the +RTS -N value". Does
 that mean the amount of FFI threads is truly unbounded?

--
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/13296>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler