[Haskell-cafe] Performance: MD5

[Thomas M. DuBuisson]

Andrew,
I spent a reasonable amount of time making pureMD5 (available on
hackage) faster, which mainly ment strictness annoitations and unboxing
strict fields, but I also spent a good deal of time with the profiler.
One of my early versions was fairly slow due to the converting of the
LPS to blocks (an Isabelle proven 'blocks' function) caused O(n) space
requirement.  I've been meaning to revisit this to optimize more and
look closly at GHC core.

I'm on vacation now, but when I get back I'll try to make time to look
at your code (unless its moot by then).  Finally, I encourage anyone
interested to make reasonably fast bytestring implementations of SHA
algorithms as well - Haskell/Hackage has a number of pure and FFI
implementations of MD5 but is a bit lacking in any other cryptographic
algorithm.

TomMD

On Sat, 2008-05-17 at 15:51 +0100, Andrew Coppin wrote:
> Hi folks.
> 
> OK, try this:
> 
>   darcs get http://darcs.orphi.me.uk/MyMD5
>   cd MyMD5
>   ghc -O2 --make md5sum
>   md5sum <some large filename>
> 
> On my PC, it takes 3.5 seconds to compute the MD5 hash of a 10 MB file. 
> For comparison, the md5.exe I downloaded from the Internet does it 
> instantaneously. It's literally so fast I can't even time it. As far as 
> I know, my program always produces the *correct* answer, it just takes 
> far too long to do it.
> 
> If anybody has any interesting insights as to why my version is so 
> damned slow, I'd like to hear them. (Indeed, a description of the 
> process for tracking the problem down would also be useful!)
> 
> [Much to my bemusement, when I had a look at the code, I discovered that 
> tucked away in a corner somewhere, it reads a ByteString from disk and 
> instantly converts it to [Word8]. Uh... oops? Anyway, eliminating this 
> changed the numbers I get from the profiler, but it's still far too slow.]
> 
> _______________________________________________
> Haskell-Cafe mailing list
> Haskell-Cafe at haskell.org
> http://www.haskell.org/mailman/listinfo/haskell-cafe