[commit: ghc] master: Overhaul of Compact Regions (#12455) (7036fde)

git at git.haskell.org git at git.haskell.org
Wed Dec 7 10:59:49 UTC 2016


Repository : ssh://git@git.haskell.org/ghc

On branch  : master
Link       : http://ghc.haskell.org/trac/ghc/changeset/7036fde9df61b6eae9719c7f6c656778c756bec9/ghc

>---------------------------------------------------------------

commit 7036fde9df61b6eae9719c7f6c656778c756bec9
Author: Simon Marlow <marlowsd at gmail.com>
Date:   Fri Jul 29 14:11:03 2016 +0100

    Overhaul of Compact Regions (#12455)
    
    Summary:
    This commit makes various improvements and addresses some issues with
    Compact Regions (aka Compact Normal Forms).
    
    This was the most important thing I wanted to fix.  Compaction
    previously prevented GC from running until it was complete, which
    would be a problem in a multicore setting.  Now, we compact using a
    hand-written Cmm routine that can be interrupted at any point.  When a
    GC is triggered during a sharing-enabled compaction, the GC has to
    traverse and update the hash table, so this hash table is now stored
    in the StgCompactNFData object.
    
    Previously, compaction consisted of a deepseq using the NFData class,
    followed by a traversal in C code to copy the data.  This is now done
    in a single pass with hand-written Cmm (see rts/Compact.cmm). We no
    longer use the NFData instances, instead the Cmm routine evaluates
    components directly as it compacts.
    
    The new compaction is about 50% faster than the old one with no
    sharing, and a little faster on average with sharing (the cost of the
    hash table dominates when we're doing sharing).
    
    Static objects that don't (transitively) refer to any CAFs don't need
    to be copied into the compact region.  In particular this means we
    often avoid copying Char values and small Int values, because these
    are static closures in the runtime.
    
    Each Compact# object can support a single compactAdd# operation at any
    given time, so the Data.Compact library now enforces mutual exclusion
    using an MVar stored in the Compact object.
    
    We now get exceptions rather than killing everything with a barf()
    when we encounter an object that cannot be compacted (a function, or a
    mutable object).  We now also detect pinned objects, which can't be
    compacted either.
    
    The Data.Compact API has been refactored and cleaned up.  A new
    compactSize operation returns the size (in bytes) of the compact
    object.
    
    Most of the documentation is in the Haddock docs for the compact
    library, which I've expanded and improved here.
    
    Various comments in the code have been improved, especially the main
    Note [Compact Normal Forms] in rts/sm/CNF.c.
    
    I've added a few tests, and expanded a few of the tests that were
    there.  We now also run the tests with GHCi, and in a new test way
    that enables sanity checking (+RTS -DS).
    
    There's a benchmark in libraries/compact/tests/compact_bench.hs for
    measuring compaction speed and comparing sharing vs. no sharing.
    
    The field totalDataW in StgCompactNFData was unnecessary.
    
    Test Plan:
    * new unit tests
    * validate
    * tested manually that we can compact Data.Aeson data
    
    Reviewers: gcampax, bgamari, ezyang, austin, niteria, hvr, erikd
    
    Subscribers: thomie, simonpj
    
    Differential Revision: https://phabricator.haskell.org/D2751
    
    GHC Trac Issues: #12455


>---------------------------------------------------------------

7036fde9df61b6eae9719c7f6c656778c756bec9
 compiler/prelude/primops.txt.pp                    |  36 +-
 docs/users_guide/sooner.rst                        |   7 +
 includes/rts/Flags.h                               |   1 +
 includes/rts/storage/ClosureMacros.h               |   6 -
 includes/rts/storage/Closures.h                    |  86 +-
 includes/stg/MiscClosures.h                        |   6 +-
 libraries/base/Control/Exception.hs                |   1 +
 libraries/base/Control/Exception/Base.hs           |   1 +
 libraries/base/GHC/IO/Exception.hs                 |  31 +
 libraries/compact/Data/Compact.hs                  | 151 ++--
 libraries/compact/Data/Compact/Internal.hs         | 128 +--
 libraries/compact/Data/Compact/Serialized.hs       |  48 +-
 libraries/compact/tests/.gitignore                 |   3 -
 libraries/compact/tests/all.T                      |  25 +-
 libraries/compact/tests/compact_append.hs          |   4 +-
 libraries/compact/tests/compact_autoexpand.hs      |   3 +-
 libraries/compact/tests/compact_bench.hs           |  28 +
 libraries/compact/tests/compact_bytestring.hs      |   8 +
 libraries/compact/tests/compact_cycle.hs           |  10 +
 libraries/compact/tests/compact_cycle.stdout       |   2 +
 libraries/compact/tests/compact_function.hs        |  10 +
 libraries/compact/tests/compact_function.stderr    |   1 +
 libraries/compact/tests/compact_gc.hs              |  12 +
 libraries/compact/tests/compact_gc.stdout          |  13 +
 ...mpact_simple_array.hs => compact_huge_array.hs} |   9 +-
 libraries/compact/tests/compact_largemap.hs        |  10 +
 libraries/compact/tests/compact_largemap.stdout    |   2 +
 libraries/compact/tests/compact_loop.hs            |   3 +-
 libraries/compact/tests/compact_mutable.hs         |  13 +
 libraries/compact/tests/compact_mutable.stderr     |   1 +
 libraries/compact/tests/compact_pinned.hs          |   6 +
 libraries/compact/tests/compact_pinned.stderr      |   1 +
 libraries/compact/tests/compact_serialize.hs       |   3 +-
 libraries/compact/tests/compact_share.hs           |  14 +
 libraries/compact/tests/compact_share.stdout       |   4 +
 libraries/compact/tests/compact_simple.hs          |   8 +-
 libraries/compact/tests/compact_simple.stdout      |   2 +
 libraries/compact/tests/compact_simple_array.hs    |   7 +-
 libraries/compact/tests/compact_threads.hs         |  21 +
 libraries/compact/tests/compact_threads.stdout     |   1 +
 rts/Compact.cmm                                    | 437 +++++++++++
 rts/Hash.c                                         |  33 +-
 rts/Hash.h                                         |   4 +
 rts/Prelude.h                                      |   6 +
 rts/PrimOps.cmm                                    | 131 ---
 rts/Printer.c                                      |   2 +-
 rts/RtsFlags.c                                     |   5 +
 rts/RtsStartup.c                                   |   3 +
 rts/RtsSymbols.c                                   |   4 +-
 rts/Stats.c                                        |  22 +-
 rts/StgMiscClosures.cmm                            |  15 +-
 rts/Trace.h                                        |   1 +
 rts/package.conf.in                                |   6 +
 rts/sm/CNF.c                                       | 874 +++++++++------------
 rts/sm/CNF.h                                       |  22 +-
 rts/sm/Evac.c                                      |  32 +-
 rts/sm/Sanity.c                                    |   4 +-
 rts/sm/Scav.c                                      |  64 +-
 rts/sm/ShouldCompact.h                             |  26 +
 rts/sm/Storage.c                                   |   2 +
 rts/win32/libHSbase.def                            |   3 +
 testsuite/config/ghc                               |   3 +
 utils/deriveConstants/Main.hs                      |   4 +
 63 files changed, 1515 insertions(+), 914 deletions(-)

Diff suppressed because of size. To see it, use:

    git diff-tree --root --patch-with-stat --no-color --find-copies-harder --ignore-space-at-eol --cc 7036fde9df61b6eae9719c7f6c656778c756bec9


More information about the ghc-commits mailing list