[Haskell-cafe] How on Earth Do You Reason about Space?
Edward Z. Yang
ezyang at MIT.EDU
Wed Jun 1 12:54:43 CEST 2011
That sounds like a plausible reason why naive copying explodes space.
Something like string interning would be good here... and since you're
Excerpts from Daniel Fischer's message of Wed Jun 01 06:46:24 -0400 2011:
> On Wednesday 01 June 2011 12:28:28, John Lato wrote:
> > There are a few solutions to this. The first is to make a copy of the
> > bytestring so only the required data is retained. In my experiments
> > this wasn't helpful, but it would depend on your corpus. The second is
> > to start with smaller chunks.
> The third, check whether the word is already known, and *make a copy if
> not*. That should only keep the required parts (including the currently
> processed chunk) in memory.
More information about the Haskell-Cafe