[GHC] #13535: vector test suite uses excessive memory on GHC 8.2

[GHC]

#13535: vector test suite uses excessive memory on GHC 8.2
-------------------------------------+-------------------------------------
        Reporter:  RyanGlScott       |                Owner:  (none)
            Type:  bug               |               Status:  new
        Priority:  highest           |            Milestone:  8.2.1
       Component:  Compiler          |              Version:  8.1
      Resolution:                    |             Keywords:
Operating System:  Unknown/Multiple  |         Architecture:
 Type of failure:  Compile-time      |  Unknown/Multiple
  performance bug                    |            Test Case:
      Blocked By:                    |             Blocking:
 Related Tickets:  #10800            |  Differential Rev(s):
       Wiki Page:                    |
-------------------------------------+-------------------------------------

Comment (by bgamari):

 > Could we CSE those calls before inlining the foldlM?

 Perhaps, but

 It seems like what is happening is this,
 1. The specialiser creates an `INLINE` specialisation,
     {{{ $sunstream :: Bundle Vector Int -> New Vector Int }}} which has
 `foldlM` (and consequently `foldlM_loop`) let-bound within it
 2. FloatOut gets run, but since we are in an `InlineCtxt` we don't float
 the `foldlM_loop`s out to top-level where CSE might be able to consolidate
 them (incidentally, is `Note [FloatOut inside INLINE]` in `SetLevels`
 still correct? It doesn't seem to be referenced from anywhere)
 3. The specialisation is inlined into dozens of callsites
 4. We do lots of expensive, redundant simplification
 5. Eventually we do CSE, but even then we don't eliminate the redundant
 top-level `foldlM_loop`s for the reason discussed in comment:16

--
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/13535#comment:19>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler