Incrementally consuming the eventlog

[Duncan Coutts]

On Thu, 2011-04-28 at 23:31 +0200, Johan Tibell wrote:

> The RTS would invoke listeners every time a new event is written. This
> design has many benefits:
> 
> - We don't need to introduce the serialization, deserialization, and
> I/O overhead of first writing the eventlog to file and then parsing it
> again.

The events are basically generated in serialised form (via C code that
writes them directly into the event buffer). They never exist as Haskell
data structures, or even C structures.

> - Programs could monitor themselves and provide debug output (e.g. via
> some UI component).
> - Users could write code that redirects the output elsewhere e.g. to a
> socket for remote monitoring.
> 
> Would invoking a callback on each event add too big of an overhead?

Yes, by orders of magnitude. In fact it's impossible because the act of
invoking the callback would generate more events... :-)

> How about invoking the callback once every time the event buffer is
> full?

That's much more realistic. Still, do we need the generality of pushing
the event buffers through the Haskell code? For some reason it makes me
slightly nervous. How about just setting which output FD the event
buffers get written to.

Turning all events or various classes of events on/off at runtime should
be doable. The design already supports multiple classes, though
currently it just has one class (the 'scheduler' class). The current
design does not support fine grained filtering at the point of event
generation.

Those two features combined (plus control over the frequency of event
buffer flushing) would be enough to implement a monitoring socket
interface (web http or local unix domain socket).

Making the parser in the ghc-events package incremental would be
sensible and quite doable as people have already demonstrated.

Duncan