[Haskell-cafe] Resumable applicative parsing from tail end?
ruben.astud at gmail.com
Thu Jul 7 03:53:40 UTC 2016
On 20/06/16 06:42, martin wrote:
> as the simulation progresses, I collect "primary" log entries, each one associated with a timestamp. These entries carry
> the information I am interested in (and not so much the final state). Currently I am using a simple list to collect log
> entries. Since (:) is so much easier than (++) I prepend new entries to the head of the list. So my log runs backwards
> in time with the oldest entries at the tail end.
> Because the log-entries are typically too fine-grained to be useful, I want to aggregate several of them into a single
> log entry in a secondary log. I want to do this "as I go" and discard primary log entries which are already aggregated.
> Thus I can keep the primary log reasonably small.
> I thought that aggregation is just a special case of parsing. And indeed, I could write an applicative parser
> PrimaryLog -> (PrimaryLog, SecondaryLog)
> which does the trick, except I had to reverse the ordering of the primary log. This is because the parser/aggregator
> must parse from past to future, because there may be new log entries coming up in the future. I need to be able to
> resume parsing where I left off whenever new primary log entries are entered, becuase I may then have just enough
> unconsumed primary log entries to create another secondary entry.
You are dealing with the classic "streaming problem" when you keep a
growing "Log" that you can't reduce until the end of the process;
instead of it being returned incrementally as when you `map a list`.
Luckily this problem has two standards solution namely pipes/conduit.
They let you recover the streaming behaviour you want and will make your
parsing go left-to-right.
-- Ruben Astudillo
More information about the Haskell-Cafe