[Git][ghc/ghc][wip/ghc-dynamic-census] Profiling: Allow heap profiling to be controlled dynamically.

Matthew Pickering gitlab at gitlab.haskell.org
Tue Dec 8 14:24:07 UTC 2020



Matthew Pickering pushed to branch wip/ghc-dynamic-census at Glasgow Haskell Compiler / GHC


Commits:
3687de21 by Matthew Pickering at 2020-12-08T14:23:57+00:00
Profiling: Allow heap profiling to be controlled dynamically.

This patch exposes three new functions in `GHC.Profiling` which allow
heap profiling to be enabled and disabled dynamically.

1. startHeapProfTimer - Starts heap profiling with the given RTS options
2. stopHeapProfTimer  - Stops heap profiling
3. requestHeapCensus  - Perform a heap census on the next context
                        switch, regardless of whether the timer is enabled or not.

- - - - -


12 changed files:

- docs/users_guide/9.2.1-notes.rst
- docs/users_guide/profiling.rst
- includes/Rts.h
- includes/rts/Flags.h
- + includes/rts/prof/Heap.h
- libraries/base/GHC/Profiling.hs
- libraries/base/GHC/RTS/Flags.hsc
- rts/Proftimer.c
- rts/Proftimer.h
- rts/RtsFlags.c
- rts/Schedule.c
- rts/rts.cabal.in


Changes:

=====================================
docs/users_guide/9.2.1-notes.rst
=====================================
@@ -47,7 +47,13 @@ Compiler
 - There is a significant refactoring in the solver; any type-checker plugins
   will have to be updated, as GHC no longer uses flattening skolems or
   flattening metavariables.
-  
+
+- The heap profiler can now be controlled from within a Haskell program using
+  functions in ``GHC.Profiling``. Profiling can be started and stopped or a heap
+  census requested at a specific point in the program.
+  There is a new RTS flag :rts-flag:`--no-automatic-heap-samples` which can be
+  used to stop heap profiling starting when a program starts.
+
 ``ghc-prim`` library
 ~~~~~~~~~~~~~~~~~~~~
 


=====================================
docs/users_guide/profiling.rst
=====================================
@@ -459,7 +459,7 @@ compiled program.
     :type: dynamic
 
     Deprecated alias for :ghc-flag:`-fprof-auto-exported`
-    
+
 .. ghc-flag:: -caf-all
     :shortdesc: *(deprecated)* Alias for :ghc-flag:`-fprof-cafs`
     :type: dynamic
@@ -885,6 +885,13 @@ There are three more options which relate to heap profiling:
     profiles are always sampled with the frequency of the RTS clock. See
     :ref:`prof-time-options` for changing that.
 
+.. rts-flag:: --no-automatic-heap-samples
+    :since: 9.2.1
+
+    Don't start heap profiling from the start of program executation. If this
+    option is enabled, it's expected that the user will manually start heap
+    profiling or request specific samples using functions from ``GHC.Profiling``.
+
 .. rts-flag:: -xt
 
     Include the memory occupied by threads in a heap profile. Each


=====================================
includes/Rts.h
=====================================
@@ -194,6 +194,7 @@ void _assertFail(const char *filename, unsigned int linenum)
 
 /* Profiling information */
 #include "rts/prof/CCS.h"
+#include "rts/prof/Heap.h"
 #include "rts/prof/LDV.h"
 
 /* Parallel information */


=====================================
includes/rts/Flags.h
=====================================
@@ -145,6 +145,7 @@ typedef struct _PROFILING_FLAGS {
     Time        heapProfileInterval; /* time between samples */
     uint32_t    heapProfileIntervalTicks; /* ticks between samples (derived) */
     bool        includeTSOs;
+    bool        startHeapProfileAtStartup; /* true if we start profiling from program startup */
 
 
     bool		showCCSOnException;


=====================================
includes/rts/prof/Heap.h
=====================================
@@ -0,0 +1,24 @@
+/* -----------------------------------------------------------------------------
+ *
+ * (c) The University of Glasgow, 2009
+ *
+ * Heap Census Profiling
+ *
+ * Do not #include this file directly: #include "Rts.h" instead.
+ *
+ * To understand the structure of the RTS headers, see the wiki:
+ *   https://gitlab.haskell.org/ghc/ghc/wikis/commentary/source-tree/includes
+ *
+ * ---------------------------------------------------------------------------*/
+
+#pragma once
+
+/* -----------------------------------------------------------------------------
+ * Fine-grained control over heap census profiling which can be called from
+ * Haskell to restrict the profile to portion(s) of the execution.
+ * See the module GHC.Profiling.
+ * ---------------------------------------------------------------------------*/
+
+void requestHeapCensus ( void );
+void startHeapProfTimer ( void );
+void stopHeapProfTimer ( void );


=====================================
libraries/base/GHC/Profiling.hs
=====================================
@@ -2,7 +2,14 @@
 {-# LANGUAGE NoImplicitPrelude #-}
 
 -- | @since 4.7.0.0
-module GHC.Profiling where
+module GHC.Profiling ( -- * Cost Centre Profiling
+                       startProfTimer
+                     , stopProfTimer
+                       -- * Heap Profiling
+                     , startHeapProfTimer
+                     , stopHeapProfTimer
+                     , requestHeapCensus
+                     )where
 
 import GHC.Base
 
@@ -17,3 +24,19 @@ foreign import ccall stopProfTimer :: IO ()
 --
 -- @since 4.7.0.0
 foreign import ccall startProfTimer :: IO ()
+
+-- | Request a heap census on the next context switch.
+--
+-- @since 4.16.0.0
+foreign import ccall requestHeapCensus :: IO ()
+
+-- | Start heap profiling. This is called normally by the RTS on start-up,
+-- but can be disabled using the rts flag `--no-automatic-gc-intervals`
+--
+-- @since 4.16.0.0
+foreign import ccall startHeapProfTimer :: IO ()
+
+-- | Stop heap profiling.
+--
+-- @since 4.16.0.0
+foreign import ccall stopHeapProfTimer :: IO ()


=====================================
libraries/base/GHC/RTS/Flags.hsc
=====================================
@@ -289,6 +289,7 @@ data ProfFlags = ProfFlags
     , heapProfileInterval      :: RtsTime -- ^ time between samples
     , heapProfileIntervalTicks :: Word    -- ^ ticks between samples (derived)
     , includeTSOs              :: Bool
+    , startHeapProfileAtStartup :: Bool
     , showCCSOnException       :: Bool
     , maxRetainerSetSize       :: Word
     , ccsLength                :: Word
@@ -586,6 +587,8 @@ getProfFlags = do
             <*> #{peek PROFILING_FLAGS, heapProfileIntervalTicks} ptr
             <*> (toBool <$>
                   (#{peek PROFILING_FLAGS, includeTSOs} ptr :: IO CBool))
+            <*> (toBool <$>
+                  (#{peek PROFILING_FLAGS, startHeapProfileAtStartup} ptr :: IO CBool))
             <*> (toBool <$>
                   (#{peek PROFILING_FLAGS, showCCSOnException} ptr :: IO CBool))
             <*> #{peek PROFILING_FLAGS, maxRetainerSetSize} ptr


=====================================
rts/Proftimer.c
=====================================
@@ -18,7 +18,12 @@
 static bool do_prof_ticks = false;       // enable profiling ticks
 #endif
 
-static bool do_heap_prof_ticks = false;  // enable heap profiling ticks
+static bool do_heap_prof_ticks = false;  // Whether the timer is currently ticking down
+static bool heap_prof_timer_active = false;  // Whether the timer is enabled at all
+
+/* The heap_prof_timer_active flag controls whether heap profiling is enabled
+at all, once it is enabled, the `do_heap_prof_ticks` flag controls whether the
+counter is currently counting down. This is paused, for example, in Schedule.c. */
 
 // Sampling of Ticky-Ticky profiler to eventlog
 #if defined(TICKY_TICKY) && defined(TRACING)
@@ -51,18 +56,36 @@ startProfTimer( void )
 void
 stopHeapProfTimer( void )
 {
-    RELAXED_STORE(&do_heap_prof_ticks, false);
+    RELAXED_STORE(&heap_prof_timer_active, false);
+    pauseHeapProfTimer();
 }
 
 void
 startHeapProfTimer( void )
 {
+    RELAXED_STORE(&heap_prof_timer_active, true);
+    resumeHeapProfTimer();
+}
+
+void
+pauseHeapProfTimer ( void ) {
+    RELAXED_STORE(&do_heap_prof_ticks, false);
+}
+
+
+void
+resumeHeapProfTimer ( void ) {
     if (RtsFlags.ProfFlags.doHeapProfile &&
         RtsFlags.ProfFlags.heapProfileIntervalTicks > 0) {
-        do_heap_prof_ticks = true;
+        RELAXED_STORE(&do_heap_prof_ticks, true);
     }
 }
 
+void
+requestHeapCensus( void ){
+  RELAXED_STORE(&performHeapProfile, true);
+}
+
 void
 initProfTimer( void )
 {
@@ -70,7 +93,12 @@ initProfTimer( void )
 
     ticks_to_heap_profile = RtsFlags.ProfFlags.heapProfileIntervalTicks;
 
-    startHeapProfTimer();
+    /* This might look a bit strange but the heap profile timer can
+      be toggled on/off from within Haskell by calling the startHeapProf
+      function from within Haskell */
+    if (RtsFlags.ProfFlags.startHeapProfileAtStartup){
+      startHeapProfTimer();
+    }
 }
 
 uint32_t total_ticks = 0;
@@ -99,7 +127,7 @@ handleProfTick(void)
     }
 #endif
 
-    if (RELAXED_LOAD(&do_heap_prof_ticks)) {
+    if (RELAXED_LOAD(&do_heap_prof_ticks) && RELAXED_LOAD(&heap_prof_timer_active))  {
         ticks_to_heap_profile--;
         if (ticks_to_heap_profile <= 0) {
             ticks_to_heap_profile = RtsFlags.ProfFlags.heapProfileIntervalTicks;


=====================================
rts/Proftimer.h
=====================================
@@ -12,9 +12,8 @@
 
 void initProfTimer      ( void );
 void handleProfTick     ( void );
-
-void stopHeapProfTimer  ( void );
-void startHeapProfTimer ( void );
+void pauseHeapProfTimer  ( void );
+void resumeHeapProfTimer ( void );
 
 extern bool performHeapProfile;
 extern bool performTickySample;


=====================================
rts/RtsFlags.c
=====================================
@@ -211,6 +211,7 @@ void initRtsFlagsDefaults(void)
 
     RtsFlags.ProfFlags.doHeapProfile      = false;
     RtsFlags.ProfFlags.heapProfileInterval = USToTime(100000); // 100ms
+    RtsFlags.ProfFlags.startHeapProfileAtStartup = true;
 
 #if defined(PROFILING)
     RtsFlags.ProfFlags.includeTSOs        = false;
@@ -390,6 +391,10 @@ usage_text[] = {
 "  -hT      Produce a heap profile grouped by closure type",
 #endif /* PROFILING */
 
+"  -i<sec>  Time between heap profile samples (seconds, default: 0.1)",
+"  --no-automatic-heap-samples  Do not start the heap profile interval time",
+"                               rely on the user to trigger samples from their application",
+
 #if defined(TRACING)
 "",
 "  -ol<file>  Send binary eventlog to <file> (default: <program>.eventlog)",
@@ -415,7 +420,6 @@ usage_text[] = {
 "             the initial enabled event classes are 'sgpu'",
 #endif
 
-"  -i<sec>  Time between heap profile samples (seconds, default: 0.1)",
 "",
 #if defined(TICKY_TICKY)
 "  -r<file>  Produce ticky-ticky statistics (with -rstderr for stderr)",
@@ -1080,6 +1084,12 @@ error = true;
                       }
                       break;
                   }
+                  else if (strequal("no-automatic-heap-samples",
+                               &rts_argv[arg][2])) {
+                      OPTION_SAFE;
+                      RtsFlags.ProfFlags.startHeapProfileAtStartup = false;
+                      break;
+                  }
                   else {
                       OPTION_SAFE;
                       errorBelch("unknown RTS option: %s",rts_argv[arg]);


=====================================
rts/Schedule.c
=====================================
@@ -415,7 +415,7 @@ run_thread:
     // that.
     cap->r.rCurrentTSO = t;
 
-    startHeapProfTimer();
+    resumeHeapProfTimer();
 
     // ----------------------------------------------------------------------
     // Run the current thread
@@ -533,7 +533,7 @@ run_thread:
     // ----------------------------------------------------------------------
 
     // Costs for the scheduler are assigned to CCS_SYSTEM
-    stopHeapProfTimer();
+    pauseHeapProfTimer();
 #if defined(PROFILING)
     cap->r.rCCCS = CCS_SYSTEM;
 #endif


=====================================
rts/rts.cabal.in
=====================================
@@ -180,6 +180,7 @@ library
                       rts/Types.h
                       rts/Utils.h
                       rts/prof/CCS.h
+                      rts/prof/Heap.h
                       rts/prof/LDV.h
                       rts/storage/Block.h
                       rts/storage/ClosureMacros.h



View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/commit/3687de21f77d5bea938ee328a1b29855cffc8122

-- 
View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/commit/3687de21f77d5bea938ee328a1b29855cffc8122
You're receiving this email because of your account on gitlab.haskell.org.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-commits/attachments/20201208/d992a8da/attachment-0001.html>


More information about the ghc-commits mailing list