[GHC] #14741: High-memory usage during compilation using Template Haskell

[GHC]

#14741: High-memory usage during compilation using Template Haskell
-------------------------------------+-------------------------------------
        Reporter:  donatello         |                Owner:  sighingnow
            Type:  bug               |               Status:  new
        Priority:  normal            |            Milestone:
       Component:  Compiler          |              Version:  8.2.2
      Resolution:                    |             Keywords:
Operating System:  Unknown/Multiple  |         Architecture:
 Type of failure:  Compile-time      |  Unknown/Multiple
  performance bug                    |            Test Case:
      Blocked By:                    |             Blocking:
 Related Tickets:  #16190            |  Differential Rev(s):  Phab:D4384
       Wiki Page:                    |
-------------------------------------+-------------------------------------

Comment (by hsyl20):

 I have made a patch to add an helper to TH to create "bytes" primitives:
 https://gitlab.haskell.org/hsyl20/ghc/tree/hsyl20-T14741

 Using it we can patch `file-embed` like this:

 {{{
 19c19
 < module Data.FileEmbed
 ---
 > module FileEmbed
 49,53c49
 < #if MIN_VERSION_template_haskell(2,5,0)
 <     , Lit (StringL, StringPrimL, IntegerL)
 < #else
 <     , Lit (StringL, IntegerL)
 < #endif
 ---
 >     , Lit (..)
 60a57
 > import Language.Haskell.TH
 65a63
 > import qualified Data.ByteString.Internal as B
 154c152,156
 < #if MIN_VERSION_template_haskell(2, 8, 0)
 ---
 > #if MIN_VERSION_template_haskell(2, 15, 0)
 >       `AppE` LitE (bytesPrimL (
 >                 let B.PS ptr off sz = bs
 >                 in  mkBytes ptr (fromIntegral off) (fromIntegral sz))))
 > #elif MIN_VERSION_template_haskell(2, 8, 0)
 }}}

 Using previous patches for #16198 and #16190, we get the following results
 when we embed a file of the given size:

 * V1: HEAD + patch for #16198
 * V2: V1 + patch for #16190 (default threshold set to 500K)
 * V3: V2 + this patch

 || Size || 8.6.3   || V1       || V2      || V3       || Gain (V3 over v2)
 ||
 || 128  || 2.650   || 2.331    || 2.346   || 2.291    ||  +2%  ||
 || 3K   || 2.651   || 2.289    || 2.290   || 2.310    ||  -1%  ||
 || 30K  || 2.590   || 2.353    || 2.307   || 2.299    ||  +0%  ||
 || 100K || 2.717   || 2.379    || 2.389   || 2.298    ||  +4%  ||
 || 500K || 3.621   || 2.814    || 2.331   || 2.315    ||  +1%  ||
 || 1M   || 4.694   || 3.526    || 2.654   || 2.320    ||  +12%  ||
 || 2M   || 6.784   || 4.668    || 2.650   || 2.350    ||  +11%  ||
 || 3M   || 8.851   || 5.616    || 3.073   || 2.400    ||  +22%  ||
 || 30M  || 63.181  || 34.318   || 8.517   || 3.390    ||  +60%  ||

-- 
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/14741#comment:18>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler