[Haskell-cafe] stack overflow pain

[Leon Smith]

On Wed, Sep 21, 2011 at 3:39 AM, Heinrich Apfelmus
<apfelmus at quantentunnel.de> wrote:
> Of course, a list of 1 million items is going to take a lot of memory,
> unless you generate it lazily. Unfortunately  mapM  cannot generate its
> result lazily because it has to execute all IO actions before returning the
> list of results.

That's oversimplifying a bit.  The outer list cannot be generated
lazily,  but the inner values (in this case inner lists) can be
generated lazily.

On Wed, Sep 21, 2011 at 7:00 PM, Tim Docker <tim at dockerz.net> wrote:
> I believe the error is happening in the concat because there are subsequent
> IO actions that fail to execute. ie the code is equivalent to:
>
>        vs <- fmap concat $ mapM applyAction sas
>        someOtherAction
>        consume vs
>
> and someOtherAction seems not to be run. However, to be sure, I'll confirm
> with code akin to what you suggest above.

The error shouldn't be happening in either concat or mapM.   Are you
sure that someOtherAction isn't being run?  Might it be writing to a
file and the result isn't getting flushed?

GHC has no inherent limit on the stack size, though using extremely
large amounts of stack is usually indicative of an error.   You can up
the stack limit with the -Ksize RTS option, and I think there is a way
it can be disabled entirely.    You might try upping your stack size
and profiling your program to see if that's helpful:  the -xt
profiling option might be useful, but I haven't played with it much.

I suspect the issue is that one of your applyAction is creating a
thunk that blows the stack when it's evaluated,  and "return []"
ensures that the thunk is never evaluated.    Though it's not clear to
me why it'd be getting evaluated in this new scenario with the
information you've provided,  assuming you really truly aren't running
someOtherAction.

Best,
Leon