[GHC] #12582: HSOC Eventlog live profiling

[GHC]

#12582: HSOC Eventlog live profiling
-------------------------------------+-------------------------------------
           Reporter:  NCrashed       |             Owner:  NCrashed
               Type:  feature        |            Status:  new
  request                            |
           Priority:  normal         |         Milestone:
          Component:  Runtime        |           Version:  8.0.1
  System                             |
           Keywords:  eventlog       |  Operating System:  Unknown/Multiple
  profile hsoc                       |
       Architecture:                 |   Type of failure:  None/Unknown
  Unknown/Multiple                   |
          Test Case:                 |        Blocked By:
           Blocking:                 |   Related Tickets:
Differential Rev(s):                 |         Wiki Page:
-------------------------------------+-------------------------------------
 Hi, it is ticket for results of my Haskell Summer of Code 2016 project
 [1].

 The goal is provide feature for redirection of events to external
 profilers without an application termination.

 Proposed changes:

 * Changing eventlog file descriptor in runtime.

 {{{#!c
 /*
  * Set custom file stream for global event log sink.
  *
  * The function overwrites previous event log file pointer. Previouss
  * sink is closed only if closePrev flag is on.
  *
  * Writing to the sink is protected by global mutex.
  *
  * The function puts header to the new sink only when emitHeader flag
  * is on. User might not want the header if it is switching to
  * already existed eventlog handle that was switched away recently.
  */
 void rts_setEventLogSink(FILE *sink,
                          StgBool closePrev,
                          StgBool emitHeader);

 /*
  * Get current file stream that is used for global event log sink.
  *
  * You shouldn't do anything with the pointer until
  * rts_setEventLogSink(otherFileOrNull, false) is called. After that
  * you can do anything with the file stream.
  */
 FILE* rts_getEventLogSink(void);
 }}}

 * Special flag `-lm` for RTS to store eventlog in memory chunks. User
 space functions to retrieve the chunks from memory.

 {{{#!c
 /*
  * If RTS started with '-lm' flag then eventlog is stored in memory
 buffer.
  *
  * The function allows to pop chunks of the buffer. Return value of 0
 means
  * that there is no any filled chunk of data.
  *
  * If the function returns nonzero value the parameter contains full chunk
  * of eventlog data with size of the returned value. Caller must free the
  * buffer, the buffer isn't referenced anywhere anymore.
  *
  * If nobody calls the function with '-lm' flag then the memory is kinda
  * to be exhausted.
  */
 StgWord64 rts_getEventLogChunk(StgInt8** ptr);
 }}}

 * Feature for dynamic resize of eventlog buffers.

 {{{#!c
 /*
  * Reallocate inner buffers to match the new size. The size should be not
  * too small to contain at least one event.
  *
  * If RTS started with '-lm' the chunks of memory buffer is also resized.
  */
 void rts_resizeEventLog(StgWord64 size);

 /*
  * Return current size of eventlog buffers.
  */
 StgWord64 rts_getEventLogBuffersSize(void);
 }}}

 * Bindings to new features in `Debug.Trace` in `base` package.

 {{{#!hs
 -- | The 'setEventLogCFile' function changes current sink of the eventlog,
 if eventlog
 -- profiling is available and enabled at runtime.
 --
 -- The second parameter defines whether old sink should be finalized and
 closed or not.
 -- Preserving it could be helpful for temporal redirection of eventlog
 data into not
 -- standard sink and then restoring to the default file sink.
 --
 -- The third parameter defines whether new header section should be
 emitted to the new
 -- sink. Emitting header to already started eventlog streams will corrupt
 the structure
 -- of eventlog format.
 --
 -- The function is more low-level than 'setEventLogHandle' but doesn't
 recreate underlying
 -- file descriptor and is intended to use with 'getEventLogCFile' to save
 and restore
 -- current sink of the eventlog.
 --
 -- @since 4.10.0.0
 setEventLogCFile :: Ptr CFile -> Bool -> Bool -> IO ()

 -- | The 'getEventLogCFile' function returns current sink of the eventlog,
 if eventlog
 -- profiling is available and enabled at runtime.
 --
 -- The function is intented to be used with 'setEventLogCFile' to save and
 restore
 -- current sink of the eventlog.
 --
 -- @since 4.10.0.0
 getEventLogCFile :: IO (Ptr CFile)

 -- | Setting size of internal eventlog buffers. The size should be not
 -- too small to contain at least one event.
 --
 -- If RTS started with '-lm' the chunks of memory buffer is also resized.
 --
 -- The larger the buffers the lesser overhead from event logging, but
 -- larger delays between data dumps.
 --
 -- See also: 'getEventLogChunk', 'getEventLogBufferSize'
 setEventLogBufferSize :: Word -> IO ()

 -- | Getting size of internal eventlog buffers.
 --
 -- See also: 'setEventLogBufferSize', 'getEventLogChunk'
 getEventLogBufferSize :: IO Word

 -- | Get next portion of the eventlog data.
 --
 -- If RTS started with '-lm' flag then eventlog is stored in memory
 buffer.
 --
 -- The function allows to pop chunks out of the buffer. Return value of
 Nothing
 -- means that there is no any filled chunk of data.
 --
 -- If the function returns nonzero value the parameter contains full chunk
 -- of eventlog data with size of the returned value. Caller must free the
 -- buffer with 'free' from 'Foreign.Marshal.Alloc', the buffer isn't
 referenced
 -- anywhere anymore.
 --
 -- If nobody calls the function with '-lm' flag on then the memory is
 kinda
 -- to be exhausted.
 --
 -- If '-lm' flag is off, the function returns always 'Nothing'.
 --
 -- See also: 'setEventLogBufferSize'
 getEventLogChunk :: IO (Maybe CStringLen)
 }}}

 References:

 1. [http://ncrashed.github.io/blog/posts/2016-06-12-hsoc-acceptance.html
 HSOC project description]

 2. [http://ncrashed.github.io/blog/posts/2016-06-22-hsoc-rts.html Design
 choices for implementation]

--
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/12582>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler