buildFdSets: file descriptor out of range
marlowsd at gmail.com
Wed Jul 15 08:55:25 EDT 2009
On 15/07/2009 05:16, Brandon S. Allbery KF8NH wrote:
> On Jul 14, 2009, at 21:48 , Kazu Yamamoto (山本和彦) wrote:
>> running well at the beginning. But after several hours, it receives an
>> error, "buildFdSets: file descriptor out of range".
> I believe the runtime uses select(), which has a hard limit (enforced by
> the kernel) that the maximum file descriptor id be 1023. (select() uses
> bitmasks and there is a limit on the size of a bitmask; see FD_SETSIZE.)
Strictly speaking it's the IO library, not the runtime, that calls
select() when you're using -threaded. But otherwise that's all correct.
>> - pushes the limit of file descriptors to 65536 with setResourceLimit.
> Reduce this to 1024, otherwise the runtime will eventually find itself
> dealing with file descriptors beyond the select() limit mentioned above.
> Someone with more knowledge of the Haskell runtime will have to advise
> as to possible ways around it if you really need more than 1024 file
There's no easy workaround. We could have the IO library switch to
using blocking read() calls for the out-of-range FDs to avoid the error
from the IO manager, but that is likely to lead to a different problem:
too many OS threads.
The right fix is to move to using epoll() instead. I understand it is
being worked on, but I don't know the current status (Johan?).
More information about the Glasgow-haskell-users