[Haskell-cafe] Resumable applicative parsing from tail end?

[Ruben Astudillo]

On 20/06/16 06:42, martin wrote:
> as the simulation progresses, I collect "primary" log entries, each one associated with a timestamp. These entries carry
> the information I am interested in (and not so much the final state). Currently I am using a simple list to collect log
> entries. Since (:) is so much easier than (++) I prepend new entries to the head of the list. So my log runs backwards
> in time with the oldest entries at the tail end.
>
> Because the log-entries are typically too fine-grained to be useful, I want to aggregate several of them into a single
> log entry in a secondary log. I want to do this "as I go" and discard primary log entries which are already aggregated.
> Thus I can keep the primary log reasonably small.
>
> I thought that aggregation is just a special case of parsing. And indeed, I could write an applicative  parser
>
> PrimaryLog -> (PrimaryLog, SecondaryLog)
>
> which does the trick, except I had to reverse the ordering of the primary log. This is because the parser/aggregator
> must parse from past to future, because there may be new log entries coming up in the future. I need to be able to
> resume parsing where I left off whenever new primary log entries are entered, becuase I may then have just enough
> unconsumed primary log entries to create another secondary entry.

You are dealing with the classic "streaming problem" when you keep a
growing "Log" that you can't reduce until the end of the process; 
instead of it being returned incrementally as when you `map a list`.

Luckily this problem has two standards solution namely pipes/conduit. 
They let you recover the streaming behaviour you want and will make your 
parsing go left-to-right.

--
-- Ruben Astudillo