[Haskell-cafe] Slow IO
daniel.is.fischer at web.de
Wed Sep 13 10:08:14 EDT 2006
Am Mittwoch, 13. September 2006 11:07 schrieben Sie:
> Daniel Fischer wrote:
> > > Most certainly not. I'm pretty sure this is to a bug in your code.
> > > Something retains a data structure which is actually unneeded.
> > > Probably
> > Apparently. And my money is on a load of lines from the file (of which I
> > need only the first and last Char).
> Then you're doing it wrong[TM]. You shouldn't need to keep any part of
Yes, I did it wrong, but I didn't keep anything (but the first and last Char
of each line) in memory on purpose. I hoped for the lines to be read one
after the other, head and last extracted - possibly immediately passed to
accumArray, but I wouldn't necessarily expect that - and the already used
lines thrown in the dustbin on next GC. Maybe the compiler couldn't figure
out that it wouldn't access these lines anymore.
> the input in memory. Whatever it is, nobody can tell you without seeing
> the code. Try heap profiling, should you have no idea where to look for
Profiling (hy,hc) shows that the IO part of the programme holds on to tons of
lists - that couldn't be anything but parts of the file-contents, I believe.
> > How could I solve the problem without representing the graph in some way?
> By using an advanced tool called "brains". Sorry for not being more
> specific, but that's actually the fun part of the challenge and I'm not
> going to spoil it for you. ;-)
> > Forgive the stupid question, but where if not RAM would the chunk
> > currently processed reside?
> Oh, I overlooked "chunk". Well, yes, the "chunk" currently processed
> needs to fit into RAM. But how much of a problem could a single Char
Well, if it's the last straw...
But not much, I presume and even though it might be that we must have a few
thousand Chars inmemory, that shouldn't do much harm either.
> Donald Bruce Stewart wrote:
> > I agree. Some problems simply require you to hold large strings in
> > memory. And for those, [Char] conks out around 5-10M (try reversing a
> > 10M [Char]).
> Sure, this one just isn't of that kind.
Yes, but I didn't tell the compiler :-(
"In My Egotistical Opinion, most people's C programs should be
indented six feet downward and covered with dirt."
-- Blair P. Houghton
More information about the Haskell-Cafe