[Haskell-cafe] Space leak in WAI 3.0 application

[Bardur Arantsson]

On 2014-11-20 08:29, Thomas Koster wrote:
> On Tue, Nov 18, 2014 at 4:59 PM, Thomas Koster <tkoster at gmail.com> wrote:
>> -- | This version has a space leak.
>> zerosAppA :: Application
>> zerosAppA _req respond =
>>   withZeros 100000000 $ \ largeLBS ->
>>     respond $ responseStream status200 [] $ \ write _flush ->
>>       write $ fromLazyByteString largeLBS
>>
>> -- | This version does not have a space leak.
>> zerosAppB :: Application
>> zerosAppB _req respond =
>>  respond $ responseStream status200 [] $ \ write _flush ->
>>    withZeros 100000000 $ \ largeLBS ->
>>      write $ fromLazyByteString largeLBS
> 
> On 20 November 2014 16:59, Bardur Arantsson <spam at scientician.net> wrote:
>> I would recommend avoiding lazy I/O altogether and using
>> "responseStream" instead. This will let you decide exactly what to write
>> and when.
> 
> I am already using responseStream. It's just that in my examples and
> my real application, the 'octets' to be used for the response entity
> ("largeLBS" from my examples) are provided to the callback as a lazy
> bytestring built from lazy I/O, until I get my head around pipes and
> conduit.
> 

Lol, sorry, I guess I shouldn't be posting when I'm sick. :)

I guess I got confused by the fact what you're using a huge LBS when you
could just write it in pieces by providing the "write" and "flush"
callbacks directly to the code which is actually generating the response
(and let it be resposible for avoiding generating huge blobs).

> built from lazy I/O,

I would suggest *never* *ever* using lazy I/O. Even though it looks like
the easy route, it's much harder to reason about than any of the
alternatives. Btw, there are a couple of alternatives which you haven't
mentioned (and thus may not be familiar with:)

  - Strict/direct I/O: Just use write/flush directly, do your
    own looping "until done" instead of hGetContents, etc.
    (I realize I've already mentioned it, I just repeat it here for
    completeness.)
  - io-streams: A very thin layer of stream abstractions directly
    on top of I/O. I found this much easier to get a handle on
    than either conduit or pipes, but then it's also less powerful
    and gives you very few abstractions independent of IO. Works
    very well in practice, though, if you just want to "get the job
    done". See https://hackage.haskell.org/package/io-streams

Anyway, sorry about the dud reply before -- hopefully this'll be more
helpful!

Regards,