[Haskell-cafe] Re: Battling time leaks
bulatz at HotPOP.com
Fri Dec 23 03:25:23 EST 2005
Wednesday, December 21, 2005, 9:47:19 PM, you wrote:
>> can you say what it exactly means? we are not mastered in your code.
>> some common explanation like "my program takes 6 seconds to
>> deserialize 50kb of data on Pentium4/3ghz" will be more understabdable
JR> That's why I posted the code at http://wagerlabs.com/timeleak.tgz
JR> The alerts are issued when the time of unstuffing exceeds 3 seconds.
this says nothing to me. you must say
1) your end goal - say, "run 1000 threads each 3 seconds"
2) why you can't share result of one unpickling among all threads?
3) your computer
4) unpickling time for one 50kb record
>> if your current problem is deserialization speed, i can give you my
>> own library. it now runs about 500kb/sec on 1ghz processor
JR> My issue is that I 1) have about 250 records and 2) my wire format is
JR> different from the Haskell representation. Everything arrives to me
JR> little-endian, for example.
1) you can use TH (may be, Einar will help you. he already have TH
support in its own SerTH library). even without TH, writing Binary
instances, imho, is an order easier than writing pickler functions -
especially when you need only deserialiation :)
2) Binary library use network format (little-endian) on all platforms
but i'm absolutely not sure that your real problem is raw unpickling speed
JR> It looks like pickling is the bottleneck so I'm converting all the
JR> structures to Storable :(. In the meantime, I'm looking for suggestions.
JR> Einar offered BinSer which lets me use a single spec for the record
JR> format, conversions included. See http://cs.helsinki.fi/u/ekarttun/
JR> haskell/test.hs. I still can't figure out how I would go from a :+: b
JR> to Foo a b, though.
interesting thing, i will look. i'm rewrote from scratch my serialization
library 2 times, and still want to fully rewrite it again :)
Bulat mailto:bulatz at HotPOP.com
More information about the Haskell-Cafe