[Git][ghc/ghc][master] 3 commits: rts: ensure we are below maxHeapSize after returning megablocks

[Marge Bot (@marge-bot)]

Marge Bot pushed to branch master at Glasgow Haskell Compiler / GHC


Commits:
54e41b16 by Teo Camarasu at 2022-10-15T18:09:24+01:00
rts: ensure we are below maxHeapSize after returning megablocks

When the heap is heavily block fragmented the live byte size might be
low while the memory usage is high. We want to ensure that heap overflow
triggers in these cases.

We do so by checking that we can return enough megablocks to
under maxHeapSize at the end of GC.

- - - - -
29bb90db by Teo Camarasu at 2022-10-15T18:09:24+01:00
rts: trigger a major collection if megablock usage exceeds maxHeapSize

When the heap is suffering from block fragmentation, live bytes might be
low while megablock usage is high.

If megablock usage exceeds maxHeapSize, we want to trigger a major GC to
try to recover some memory otherwise we will die from a heapOverflow at
the end of the GC.

Fixes #21927

- - - - -
4a4641ca by Teo Camarasu at 2022-10-15T18:11:29+01:00
Add realease note for #21927

- - - - -


3 changed files:

- docs/users_guide/9.6.1-notes.rst
- rts/Schedule.c
- rts/sm/GC.c


Changes:

=====================================
docs/users_guide/9.6.1-notes.rst
=====================================
@@ -121,6 +121,12 @@ Runtime system
   by library authors directly, who may wrap them a safe API that maintains the
   necessary invariants. See the documentation in ``GHC.Prim`` for more details.
 
+- The behaviour of the ``-M`` flag has been made more strict. It will now trigger
+  a heap overflow if the total amount of memory used by the Haskell heap exceeds the limit.
+  Previously only live blocks were taken into account.
+  This makes it more likely to trigger promptly when the heap is highly fragmented.
+
+
 ``base`` library
 ~~~~~~~~~~~~~~~~
 


=====================================
rts/Schedule.c
=====================================
@@ -1592,9 +1592,13 @@ scheduleDoGC (Capability **pcap, Task *task USED_IF_THREADS,
 
     heap_census = scheduleNeedHeapProfile(true);
 
+    // We force a major collection if the size of the heap exceeds maxHeapSize.
+    // We will either return memory until we are below maxHeapSize or trigger heapOverflow.
+    bool mblock_overflow = RtsFlags.GcFlags.maxHeapSize != 0 && mblocks_allocated > BLOCKS_TO_MBLOCKS(RtsFlags.GcFlags.maxHeapSize);
+
     // Figure out which generation we are collecting, so that we can
     // decide whether this is a parallel GC or not.
-    collect_gen = calcNeeded(force_major || heap_census, NULL);
+    collect_gen = calcNeeded(force_major || heap_census || mblock_overflow , NULL);
     major_gc = (collect_gen == RtsFlags.GcFlags.generations-1);
 
 #if defined(THREADED_RTS)


=====================================
rts/sm/GC.c
=====================================
@@ -1061,6 +1061,13 @@ GarbageCollect (uint32_t collect_gen,
           returned = returnMemoryToOS(got - need);
       }
       traceEventMemReturn(cap, got, need, returned);
+
+      // Ensure that we've returned enough mblocks to place us under maxHeapSize.
+      // This may fail for instance due to block fragmentation.
+      W_ after = got - returned;
+      if (RtsFlags.GcFlags.maxHeapSize != 0 && after > BLOCKS_TO_MBLOCKS(RtsFlags.GcFlags.maxHeapSize)) {
+        heapOverflow();
+      }
   }
 
   // extra GC trace info



View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/62a550010ed94e1969c96150f2781854a0802766...4a4641ca2f2a157dab0fe2df5316f79ffb52c047

-- 
View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/62a550010ed94e1969c96150f2781854a0802766...4a4641ca2f2a157dab0fe2df5316f79ffb52c047
You're receiving this email because of your account on gitlab.haskell.org.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-commits/attachments/20221017/c70abfc8/attachment-0001.html>