[Haskell-cafe] Slow IO

[Daniel Fischer]

Am Mittwoch, 13. September 2006 11:07 schrieben Sie:
> Daniel Fischer wrote:
> > > Most certainly not.  I'm pretty sure this is to a bug in your code.
> > > Something retains a data structure which is actually unneeded. 
> > > Probably
> >
> > Apparently. And my money is on a load of lines from the file (of which I
> > need only the first and last Char).
>
> Then you're doing it wrong[TM].  You shouldn't need to keep any part of

Yes, I did it wrong, but I didn't keep anything (but the first and last Char 
of each line) in memory on purpose. I hoped for the lines to be read one 
after the other, head and last extracted - possibly immediately passed to 
accumArray, but I wouldn't necessarily expect that - and the already used 
lines thrown in the dustbin on next GC. Maybe the compiler couldn't figure 
out that it wouldn't access these lines anymore.

> the input in memory.  Whatever it is, nobody can tell you without seeing
> the code.  Try heap profiling, should you have no idea where to look for
> leaks.

Profiling (hy,hc) shows that the IO part of the programme holds on to tons of 
lists - that couldn't be anything but parts of the file-contents, I believe.
>
> > How could I solve the problem without representing the graph in some way?
>
> By using an advanced tool called "brains".  Sorry for not being more
> specific, but that's actually the fun part of the challenge and I'm not
> going to spoil it for you.  ;-)
>
> > Forgive the stupid question, but where if not RAM would the chunk
> > currently processed reside?
>
> Oh, I overlooked "chunk".  Well, yes, the "chunk" currently processed
> needs to fit into RAM.  But how much of a problem could a single Char
> pose?

Well, if it's the last straw...
But not much, I presume and even though it might be that we must have a few 
thousand Chars inmemory, that shouldn't do much harm either.

>
> Donald Bruce Stewart wrote:
> > I agree. Some problems simply require you to hold large strings in
> > memory. And for those, [Char] conks out around 5-10M (try reversing a
> > 10M [Char]).
>
> Sure, this one just isn't of that kind.

Yes, but I didn't tell the compiler :-(

>
>
> Udo.

Cheers,
Daniel

-- 

"In My Egotistical Opinion, most people's C programs should be
indented six feet downward and covered with dirt."
	-- Blair P. Houghton