[GHC] #14797: High-residency modules during GHC build

Mon Feb 12 16:56:56 UTC 2018

#14797: High-residency modules during GHC build
-------------------------------------+-------------------------------------
           Reporter:  tdammers       |             Owner:  (none)
               Type:  bug            |            Status:  new
           Priority:  normal         |         Milestone:
          Component:  Compiler       |           Version:  8.2.1
           Keywords:                 |  Operating System:  Linux
       Architecture:  x86_64         |   Type of failure:  Runtime
  (amd64)                            |  performance bug
          Test Case:                 |        Blocked By:
           Blocking:                 |   Related Tickets:
Differential Rev(s):                 |         Wiki Page:
-------------------------------------+-------------------------------------
 While trying to figure out memory allocation failures on the CI server, I
 tracked allocations and residency on a per-module basis, and noticed that
 some modules require a lot more memory to compile than others.

 I've compiled a digest of the most memory-hungry compilation steps:

 {{{
 "/usr/local/bin/ghc" ... compiler/stage1/build/OptCoercion.o: Max.
 residency: 991356584 / In use: 2094M
 "inplace/bin/ghc-stage1" ... compiler/stage2/build/OptCoercion.dyn_o: Max.
 residency: 714940528 / In use: 1471M
 "/usr/local/bin/ghc" ... compiler/stage1/build/X86/CodeGen.o: Max.
 residency: 679947656 / In use: 1497M
 "inplace/bin/ghc-stage1" ... libraries/Cabal/Cabal/dist-
 install/build/Distribution/SPDX/LicenseId.dyn_o: Max. residency: 653177496
 / In use: 1355M
 "inplace/bin/ghc-stage1" ... compiler/stage2/build/X86/CodeGen.dyn_o: Max.
 residency: 641515344 / In use: 1425M
 "/usr/local/bin/ghc" ... -ilibraries/parsec: Max. residency: 593335568 /
 In use: 1355M
 "/usr/local/bin/ghc" ... libraries/Cabal/Cabal/dist-
 boot/build/Distribution/SPDX/LicenseId.o: Max. residency: 575034968 / In
 use: 1137M
 "inplace/bin/ghc-stage1" ... compiler/stage2/build/DynFlags.dyn_o: Max.
 residency: 402734408 / In use: 1059M
 "/usr/local/bin/ghc" ... compiler/stage1/build/DynFlags.o: Max. residency:
 343154784 / In use: 892M
 "inplace/bin/ghc-stage1" ... libraries/template-haskell/dist-
 install/build/Language/Haskell/TH/Syntax.dyn_o: Max. residency: 266744312
 / In use: 720M
 "/usr/local/bin/ghc" ... compiler/stage1/build/Parser.o: Max. residency:
 258613584 / In use: 679M
 "inplace/bin/ghc-stage1" ... compiler/stage2/build/CmmOpt.dyn_o: Max.
 residency: 254639624 / In use: 568M
 "inplace/bin/ghc-stage1" ... compiler/stage2/build/Parser.dyn_o: Max.
 residency: 236774592 / In use: 635M
 "inplace/bin/ghc-stage1" ... ghc/stage2/build/GHCi/UI.dyn_o: Max.
 residency: 234055792 / In use: 609M
 "/usr/local/bin/ghc" ... compiler/stage1/build/HsExpr.o: Max. residency:
 233724400 / In use: 657M
 "/usr/local/bin/ghc" ... compiler/stage1/build/CmmOpt.o: Max. residency:
 208378888 / In use: 436M
 "/usr/local/bin/ghc" ... compiler/stage1/build/HscMain.o: Max. residency:
 197177248 / In use: 500M
 "inplace/bin/ghc-stage1" ... compiler/stage2/build/HsExpr.dyn_o: Max.
 residency: 192780496 / In use: 521M
 "inplace/bin/ghc-stage1" ... libraries/Cabal/Cabal/dist-
 install/build/Distribution/Simple/Setup.dyn_o: Max. residency: 191649952 /
 In use: 521M
 "/usr/local/bin/ghc" ... compiler/stage1/build/HsDecls.o: Max. residency:
 183441120 / In use: 508M
 "inplace/bin/ghc-stage1" ... compiler/stage2/build/TcSplice.dyn_o: Max.
 residency: 176802000 / In use: 445M
 "inplace/bin/ghc-stage1" ... compiler/stage2/build/TcRnDriver.dyn_o: Max.
 residency: 172892128 / In use: 458M
 "inplace/bin/ghc-stage1" ... compiler/stage2/build/HscMain.dyn_o: Max.
 residency: 172802688 / In use: 470M
 "inplace/bin/ghc-stage1" ... libraries/ghci/dist-
 install/build/GHCi/TH/Binary.dyn_o: Max. residency: 169775368 / In use:
 455M
 "inplace/bin/ghc-stage1" ... compiler/stage2/build/GhcMake.dyn_o: Max.
 residency: 167305408 / In use: 431M
 "inplace/bin/ghc-stage1" ... compiler/stage2/build/PrimOp.dyn_o: Max.
 residency: 162532408 / In use: 455M
 "/usr/local/bin/ghc" ... compiler/stage1/build/HscTypes.o: Max. residency:
 160814736 / In use: 421M
 "inplace/bin/ghc-stage1" ... compiler/stage2/build/HsDecls.dyn_o: Max.
 residency: 160445552 / In use: 426M
 "inplace/bin/ghc-stage1" ... libraries/Cabal/Cabal/dist-
 install/build/Language/Haskell/Extension.dyn_o: Max. residency: 159817712
 / In use: 363M
 "/usr/local/bin/ghc" ... compiler/stage1/build/TcSplice.o: Max. residency:
 156738656 / In use: 375M
 "/usr/local/bin/ghc" ... compiler/stage1/build/TcRnDriver.o: Max.
 residency: 153910768 / In use: 421M
 "inplace/bin/ghc-stage1" ... libraries/containers/dist-
 install/build/Data/Sequence/Internal.dyn_o: Max. residency: 151318224 / In
 use: 405M
 "inplace/bin/ghc-stage1" ... compiler/stage2/build/MatchLit.dyn_o: Max.
 residency: 150681176 / In use: 382M
 "/usr/local/bin/ghc" ... compiler/stage1/build/ByteCodeGen.o: Max.
 residency: 144579176 / In use: 389M
 "/usr/local/bin/ghc" ... compiler/stage1/build/GhcMake.o: Max. residency:
 144411312 / In use: 379M
 "/usr/local/bin/ghc" ... compiler/stage1/build/MatchLit.o: Max. residency:
 142596736 / In use: 382M
 "inplace/bin/ghc-stage1" ... compiler/stage2/build/StgCmmPrim.dyn_o: Max.
 residency: 141484616 / In use: 340M
 "inplace/bin/ghc-stage1" ... libraries/ghci/dist-
 install/build/GHCi/Message.dyn_o: Max. residency: 139224136 / In use: 363M
 "/usr/local/bin/ghc" ... compiler/stage1/build/StgCmmPrim.o: Max.
 residency: 137390744 / In use: 346M
 "inplace/bin/ghc-stage1" ... compiler/stage2/build/TcExpr.dyn_o: Max.
 residency: 136387128 / In use: 370M
 "inplace/bin/ghc-stage1" ... compiler/stage2/build/Lexer.dyn_o: Max.
 residency: 135228120 / In use: 337M
 "inplace/bin/ghc-stage1" ... compiler/stage2/build/TcTyClsDecls.dyn_o:
 Max. residency: 134817024 / In use: 369M
 "/usr/local/bin/ghc" ... compiler/stage1/build/MkIface.o: Max. residency:
 134039144 / In use: 339M
 "inplace/bin/ghc-stage1" ... compiler/stage2/build/RnExpr.dyn_o: Max.
 residency: 133947600 / In use: 345M
 "inplace/bin/ghc-stage1" ... compiler/stage2/build/HscTypes.dyn_o: Max.
 residency: 133938840 / In use: 366M
 "inplace/bin/ghc-stage1" ... compiler/stage2/build/TcGenDeriv.dyn_o: Max.
 residency: 133744056 / In use: 320M
 "/usr/local/bin/ghc" ... compiler/stage1/build/TcExpr.o: Max. residency:
 131766896 / In use: 336M
 "inplace/bin/ghc-stage1" ... compiler/stage2/build/GHC.dyn_o: Max.
 residency: 130998912 / In use: 277M
 "inplace/bin/ghc-stage1" ... compiler/stage2/build/ByteCodeGen.dyn_o: Max.
 residency: 129499464 / In use: 348M
 "inplace/bin/ghc-stage1" ... compiler/stage2/build/InteractiveEval.dyn_o:
 Max. residency: 128902944 / In use: 286M
 }}}

-- 
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/14797>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler