buildFdSets: file descriptor out of range

[Simon Marlow]

On 15/07/2009 05:16, Brandon S. Allbery KF8NH wrote:
> On Jul 14, 2009, at 21:48 , Kazu Yamamoto (山本和彦) wrote:
>> running well at the beginning. But after several hours, it receives an
>> error, "buildFdSets: file descriptor out of range".
>
> I believe the runtime uses select(), which has a hard limit (enforced by
> the kernel) that the maximum file descriptor id be 1023. (select() uses
> bitmasks and there is a limit on the size of a bitmask; see FD_SETSIZE.)

Strictly speaking it's the IO library, not the runtime, that calls 
select() when you're using -threaded.  But otherwise that's all correct.

>> - pushes the limit of file descriptors to 65536 with setResourceLimit.
>
>
> Reduce this to 1024, otherwise the runtime will eventually find itself
> dealing with file descriptors beyond the select() limit mentioned above.
> Someone with more knowledge of the Haskell runtime will have to advise
> as to possible ways around it if you really need more than 1024 file
> descriptors.

There's no easy workaround.  We could have the IO library switch to 
using blocking read() calls for the out-of-range FDs to avoid the error 
from the IO manager, but that is likely to lead to a different problem: 
too many OS threads.

The right fix is to move to using epoll() instead.  I understand it is 
being worked on, but I don't know the current status (Johan?).

Cheers,
	Simon