[Haskell-cafe] newbie question about list performance

[Don Stewart]

jwlato:
> Hello,
> I've been following the list optimization thread with great interest,
> as it pertains to something I'm working on at the moment.  I'm working
> with moderate-sized files (tens to hundreds of MBs) that have some
> ascii header data followed by a bunch of 32-bit ints.  I can read the
> files into a lazy ByteString (and parse the header), but I'd like some
> advice as to the best data type to convert the ints into.  Ideally,
> I'd have some functions like this:
> decode :: ByteString -> (FileFormat, [Int32])
> encode :: FileFormat -> [Int32] -> ByteString
> 
> but I don't know if Int32 is actually the best choice.  It seems to me
> that something like a lazy list of strict arrays (analogous to a lazy
> bytestring) would be better.  Is there a library like this already?
> Or is this a case of premature optimization, and I should just try the
> list and see if it's good enough?  Any suggestions would be
> appreciated.

could you use Data.Binary.encode/decode (with custom put and get
instances)? They read fomr lazy bytestrings into custom structures, such
as arrays.

    http://hackage.haskell.org/cgi-bin/hackage-scripts/package/binary-0.4.1

> Also, I'd like to let the maintainers and implementors know that I
> really appreciate the work that's been done on optimizing Haskell.  I
> haven't used Haskell much yet, but I've fallen in love with the
> language and it's great to see that performance even for heavy I/O
> tasks can be comparable to or exceed C.

Yay!

-- Don