[Haskell-cafe] Stack space overflow in HaskellNet

[Manfred Lotz]

On Tue, 26 Jul 2011 14:17:08 -0700 (PDT)
Donn Cave <donn at avvanta.com> wrote:

> Quoth Manfred Lotz <manfred.lotz at arcor.de>,
> ...
> > I'm not quite sure I understand what you mean. Stack overflow comes
> > from this:
> >         forM_ msgs (\x -> fetch con x >>= print)
> >
> > If I change it to:
> >         mapM_  (\x -> fetch con x >>= print) msgs
> >
> > there is the same stack overflow.
> 
> I didn't understand that myself, but neither do I know what might
> be wrong.  One thing to consider is that email messages can be very
> large.  Looking at messages received in the last 10 days I see I
> have one that exceeds your reported stack size, and that isn't
> counting the extra space required for text representation of non
> printing characters etc.  There may be messages that you simply
> can't "print".
> 
> The HaskellNet IMAP "fetch" is actually FETCH ... BODY[], i.e.,
> the whole contents of the message.  Normal practice for giant data
> files is to send them as part of a MIME multipart/mixed message,
> and something like the above can proceed with a reasonable chance
> of success if it avoids these attachments by fetching BODY[1]
> (or BODY[1.1], etc. depending on actual structure.)  I just fetched
> the 10Mb message I mentioned above to check the structure, and it
> happened in the blink of an eye - BODY[1] is smaller than the header.
> 
> I don't see any support for fetch by part, you might have to hack
> that up yourself.  You may ideally also want to fetch BODYSTRUCTURE,
> but practically I might go out on a limb and predict that you won't
> run into messages where the first part is a multipart/mixed with a
> large attachment - so if the object is just a survivable first part,
> you could live without BODYSTRUCTURE analysis and optimistically
> ask for BODY[1].
> 
> Moving on to practical use of email via IMAP, you'd also want to
> be able to fetch and decode the attachments.  At this point, it's
> interesting to return to the question of space requirements.
> 
> 	Donn

The problem seems to lie in the HaskellNet package. If for example I
only fetch a specific message 
   m <- fetch con 2092
having a size of some 1.2m then I get the same stack overflow.

If at runtime I specify +RTS -K40M -RTS it works but takes over 40
seconds to fetch the message. 



-- 
Manfred