[Haskell-cafe] Killer pickler combinators (was Time leak)

[Cale Gibbard]

Well, yes. I think that part of the problem you're having is that the
threads compete very highly, and some eventually get ignored for
longer than the timeouts. The reason why the (withMVar lock $ const $
putStrLn s) helped in the first place was not so much because of any
evaluation of s (we tested that assumption), but rather because during
the time that withMVar is active in one thread, any other threads
can't pass that same point. So a bunch of the threads which were
stealing too much of the time before, now end up blocking, which
reduces the competition so the other threads which were being ignored
by the scheduler can get some work done.

Your problem is partly in the fact that the pickler/unpickler is a bit
slow for the workload/machine you have it on, and partly in the fact
that it's hard to get the processes to advance smoothly. (The
pickling/unpickling doesn't really take very long in any one thread,
but if the thread isn't scheduled for a long time in the first
place...) A bit of locking is sort of what you want for that.
Unfortunately, the way we have it here still affords no guarantees, it
just serves as a mechanism to help keep things from getting too bad.
It's unclear to me that you really don't want any kind of locking. I
think that you might do quite well to decide which threads get to run
yourself at least to some extent. After all, you know more about their
priorities and how long they have to complete their tasks than the GHC
scheduler does.

 - Cale

On 21/12/05, Joel Reymont <joelr1 at gmail.com> wrote:
> The other thing worth noting is that by inserting a lock with a
> thread delay we are fooling ourselves. While the individual pickling
> time goes down, the threads are slowed down overall. Assuming that an
> external source was waiting for the unpickled packet _that_ source
> would get a timeout!
>
> --
> http://wagerlabs.com/
>
>
>
>
>
>