[GHC] #13861: Take more advantage of STG representation invariance (follows up #9291)

[GHC]

#13861: Take more advantage of STG representation invariance (follows up #9291)
-------------------------------------+-------------------------------------
        Reporter:  heisenbug         |                Owner:  (none)
            Type:  feature request   |               Status:  new
        Priority:  normal            |            Milestone:
       Component:  Compiler          |              Version:  8.0.2
      Resolution:                    |             Keywords:
Operating System:  Unknown/Multiple  |         Architecture:
                                     |  Unknown/Multiple
 Type of failure:  None/Unknown      |            Test Case:
      Blocked By:                    |             Blocking:
 Related Tickets:                    |  Differential Rev(s):
       Wiki Page:                    |
-------------------------------------+-------------------------------------

Comment (by heisenbug):

 Brain dump of conversation with SPJ at Haskell eXchange, London.

 Currently pointer tagging only effective for "small constructor families".
 If not small, the tag is 1 on pointers to evaluated constructors.

 See `isSmallFamily` in `compiler/codeGen/StgCmmClosure.hs`.

 Of course this penalizes big families.

 I suggest to set tags 1..6 for non-small families' lower constructors, 7
 for all other (overflowing) constructors. This would allow more precise
 branching for big families too (in a significant number of cases), as the
 ''former'' constructors are usually the more common ones (keeping fingers
 crossed).

 Also the coercion between small and big families would be straightforward,
 with following ranges directly castable:
 || || |||| from ||
 ||  ||        ||= small =||= big =||
 ||= to =||= small =||  1..7  || 1..6 ||
 ||  ||= big =||  1..7  || 1..7 ||

 Conservatively in the beginning one could only allow 1..6.

 Note: `(-1 :: Int) .&. 7 == 7` so that would lead to all-ones too.

 It is not immediately clear how to find out whether the constructor is n a
 big family. We could add the family size as an additional piece of
 information.

 A (future) wiki page should explain the new conventions. Many references
 to pinter tagging in the code should be updated.

-- 
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/13861#comment:6>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler