[Haskell-cafe] A very edgy language

[Andrew Coppin]

Bulat Ziganshin wrote:
> i've improved memory usage of my program 3 times one month after i've
> started to use Haskell, and 4 times more 1.5 years later (the last
> improvement included development of ByteString-alike library and
> strictifying some computations). i think that for programming-in-large
> experienced haskeller may reach C-like level of efficiency, unlike for
> programming-in-small (i.e. implementation of raw computations)
>   

Yeah, I spent yesterday building a whole toolbox of compression 
algorithms. However, it turns out that just one algorithm - BWT - is too 
absurdly slow. You may recall I implemented that a while back, and 
discovered that making it use a lazy ByteString make it many orders of 
magnitude faster.

Trouble is... the whole of the rest of my toolbox uses normal lists. So 
I'm going to have to do some horribly ugly hack just to make BWT work 
properly. (Like, I've built this whole nice abstract framework for all 
the other algorithms, and I'm going to have to punch a massive hole 
through the middle of it to make a ByteString BWT fit.) It's a real 
shame. (OTOH, I waited over 5 minutes for my program to try to take the 
BWT of a 12 KB file. It used in excess of 2 GB of RAM. That's clearly 
absurd...)

Does anybody have any clue why ByteStrings are actually faster? (And why 
this information isn't easily findable anywhere - must shorly be a VFAQ.)