[Haskell-cafe] How on Earth Do You Reason about Space?

Edward Z. Yang ezyang at MIT.EDU
Wed Jun 1 12:54:43 CEST 2011

That sounds like a plausible reason why naive copying explodes space.
Something like string interning would be good here... and since you're
hashing already...


Excerpts from Daniel Fischer's message of Wed Jun 01 06:46:24 -0400 2011:
> On Wednesday 01 June 2011 12:28:28, John Lato wrote:
> > There are a few solutions to this.  The first is to make a copy of the
> > bytestring so only the required data is retained.  In my experiments
> > this wasn't helpful, but it would depend on your corpus.  The second is
> > to start with smaller chunks.
> The third, check whether the word is already known, and *make a copy if 
> not*. That should only keep the required parts (including the currently 
> processed chunk) in memory.

More information about the Haskell-Cafe mailing list