[Haskell-cafe] binary IO

[Lennart Augustsson]

Joel Reymont wrote:
> You are right in that I spent the first few weeks learning. By now I  
> know that pickling is the bottleneck, though. The timeleak code is  very 
> simple. It forks X threads where each thread opens a file for  reading. 
Why on earth do you want each tread to open the file and unpickle?
Why not unpickle once and reuse it?
Or, if this is just a test and in the future they will all read
from different files (or sockets), then maybe you are hitting
on a different bottleneck than you think.



> This discussion is a bit of a dead-end unless you are willing to take  
> the next step and apply your solution to my timeleak repro case. If  you 
> or someone else is willing to do that than I can continue my set  of 
> profiler reports and eventually get some closure. It will happen  once 
> either 1) the last bit of performance is squeezed out of  pickling and 
> it's determined that threading or the scheduler is the  bottleneck or 2) 
> things work out nicely.

Instead of starting optimizing a particular pickling library perhaps you
should have tried different libraries and picked the best one.
Since this is your project I don't think your project I don't think
you can expect others to test things for you.  Well, maybe if you post
your code as a challange. ;)

	-- Lennart