[Haskell-cafe] Re: Hugsvs GHC (again)was: Re: Somerandomnewbiequestions

[Marcin 'Qrczak' Kowalczyk]

Glynn Clements <glynn at gclements.plus.com> writes:

>> We do use a thread pool.  But you still need as many OS threads as there
>> are blocked read() calls, unless you have a single thread doing select()
>> as I described.
>
> How does the select() help? AFAIK, select() on a regular file or block
> device will always indicate that it is readable, even if a subsequent
> read() would have to read the data from disk.

It doesn't help if we don't want I/O requests to delay one another,
and not only avoiding delay of execution of pure Haskell code.

BTW, poll is generally preferred to select. The maximum fd supported
by select may be lower than the maximum fd supported by the system.
And the interface of poll allows the cost to be proportional to the
number of descriptors rather than to the highest descriptor.

The timeout is specified in microseconds for select and in milliseconds
for poll, but on Linux the actual resolution is the clock tick in both
cases anyway (usually 1ms or 10ms).

It's probably yet better to use epoll than poll. The difference is
that with epoll you register fds using separate calls, and you don't
have to provide them each time you wait (and the kernel doesn't have
to scan the array each time). So it scales better to a large number of
threads which perform I/O. It's available in Linux 2.6.

Caveat: before Linux 2.6.8 epoll had a memory leak in the kernel
because of a reference counting bug (0.5kB per epoll_create call,
which means 0.5kB of physical memory lost per starting a program
which waits for I/O using epoll).

poll is in Single Unix Spec, epoll is Linux-specific.

poll and epoll both take the timeout in the same format, but they
interpret it differently: poll sleeps at least the given time (unless
a fd is ready or a signal arrives), while epoll rounds it up to a
whole number of clock ticks and then sleeps between this time and one
tick shorter. I was told that this is intentional because it allows
to sleep until the next clock tick by specifying the timeout of 1ms
(a timeout of 0ms means to not sleep at all).

Accurate sleeping requires to measure the time by which poll/epoll can
make the timeout longer (it's 1 tick for epoll and 2 ticks for poll),
subtract this time from the timeout passed to them, add 1ms, and sleep
the remaining time by busy waiting calling gettimeofday interspersed
with poll/epoll with no timeout. gettimeofday() is accurate to
microseconds, it asks some clock chip instead of relying on the timer
interrupt only.

-- 
   __("<         Marcin Kowalczyk
   \__/       qrczak at knm.org.pl
    ^^     http://qrnik.knm.org.pl/~qrczak/