[GHC] #14490: TTG Snags

[GHC]

#14490: TTG Snags
-------------------------------------+-------------------------------------
           Reporter:  alanz          |             Owner:  (none)
               Type:  bug            |            Status:  new
           Priority:  normal         |         Milestone:
          Component:  Compiler       |           Version:  8.3
           Keywords:                 |  Operating System:  Unknown/Multiple
       Architecture:                 |   Type of failure:  None/Unknown
  Unknown/Multiple                   |
          Test Case:                 |        Blocked By:
           Blocking:                 |   Related Tickets:  #14482
Differential Rev(s):                 |         Wiki Page:
                                     |  ImplementingTreesThatGrow
-------------------------------------+-------------------------------------
 == Context

 ImplementingTreesThatGrow

 The intention is to get the first-pass implementation of Trees that Grow
 into GHC 8.4.1.

 So far the following commits are on master
   -
 https://phabricator.haskell.org/rGHC438dd1cbba13d35f3452b4dcef3f94ce9a216905
   -
 https://phabricator.haskell.org/rGHCe3ec2e7ae94524ebd111963faf34b84d942265b4
   -
 https://phabricator.haskell.org/rGHC47ad6578ea460999b53eb4293c3a3b3017a56d65

 Unfortunately we have picked up compile-time performance regressions in
 compiling GHC itself, with these patches.  The perf.haskell.org results
 are
   -
 https://perf.haskell.org/ghc/#revision/438dd1cbba13d35f3452b4dcef3f94ce9a216905
   -
 https://perf.haskell.org/ghc/#revision/e3ec2e7ae94524ebd111963faf34b84d942265b4
   -
 https://perf.haskell.org/ghc/#revision/47ad6578ea460999b53eb4293c3a3b3017a56d65

 Summary: each of the three patches caused GHC validate time to worsen by
 another 5% or so, cumulatively pushing compile time from 2011 to 2400
 seconds.

 It seems the major culprit is deriving the `Data` instances, probably
 related to the complex constraint sets required to ensure that all the
 extension points have `Data` instances too.

 Some discussion around this is captured in
 ImplementingTreesThatGrow/Instances

 == Things tried so far

 PLAN E (ImplementingTreesThatGrow/Instances#PLANE), in
 https://github.com/ghc/ghc/tree/wip/ttg5-data-one-file-2017-11-17.

 This puts all the `Data` derivations for the hsSyn AST into a single file,
 `hsSyn/HsInstances.hs`.

 This causes compile-time memory usage to spike up to around 7GB when
 compiling this file.

 Gipedia reports a compilation time of 2468 secs for this version, which
 perhaps indicates that the instance derivation is not the main culprit.
 The module itself only takes in the order of 45secs to compile, on `home
 .smart-cactus.org` (http://lpaste.net/7779809210564345856)

 Splitting the `HsInstances.hs` file in 2, as per
 https://github.com/ghc/ghc/tree/wip/ttg5-data-2017-11-17 reduces the
 memory requirement for each, but shows up #14482, where there is a linker
 failure due to incorrectly processing boot files.

 == Next Steps

 I guess the first thing to do is to try to isolate what precisely is
 causing the slowdown. It may not be instance generation at all.

 Also, it is probably better to revert the three patches from master, when
 cutting the 8.4 initial freeze.

 Open to suggestions ...

-- 
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/14490>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler