[Haskell-cafe] streaming with logs / metadata

[Evan Laforge]

On Tue, Apr 9, 2019 at 12:53 PM Olaf Klinke <olf at aatal-apotheke.de> wrote:
> When you say interleaving, do you think of lists like the following?
> [Left warning1,Right value 1,Left warning 2,Right value2,...]
> The above is isomorphic to
> [(warning1,value1),(warning2,value2),...]
>
> I don't think that the writer monad destroys streaming. Below is a somewhat contrived, but ordinary use case.

Ah, I see what you mean.  I think I'm using a stricter definition of
"streaming", what I mean is that I can process without a memory leak,
in more-or-less constant memory (i.e. what packages like streams and
pipes mean by it).  With the Writer example, once I do:

> (ys,warnings) = runWriter (mapM half stream)

If I print warnings, I'm going to force all of 'ys' into memory.
Taking one thing off of 'warnings' will force an unknown amount of
'ys'.

Another way to look at it is that I can always go from [Either a b] to
([a], [b]) just by throwing away the interleaving, but then I can't go
back since I lost that information.

My instinct is that monads are too general for this, because they
leave the output type polymorphic.  It's more similar to 'streams',
but focusing on interleaving "transparent" metadata rather than
interleaving effects.  And since 'streams' reimplements the whole list
panopoly I guess that bodes ill for my attempt to avoid doing that
too.  In fact it could be a layer on top of 'streams', and maybe
should be, to avoid having to do it all over again if I want effects
some day...

I still can't think of a good name though.