[GHC] #16059: checkValidType is defeated by a type synonym

[GHC]

#16059: checkValidType is defeated by a type synonym
-------------------------------------+-------------------------------------
        Reporter:  RyanGlScott       |                Owner:  (none)
            Type:  bug               |               Status:  closed
        Priority:  normal            |            Milestone:  8.8.1
       Component:  Compiler (Type    |              Version:  8.7
  checker)                           |
      Resolution:  fixed             |             Keywords:
Operating System:  Unknown/Multiple  |         Architecture:
                                     |  Unknown/Multiple
 Type of failure:  GHC accepts       |            Test Case:
  invalid program                    |  typecheck/should_fail/T16059{a,c,d,e}
      Blocked By:                    |             Blocking:
 Related Tickets:                    |  Differential Rev(s):
       Wiki Page:                    |  https://gitlab.haskell.org/ghc/ghc/merge_requests/112
-------------------------------------+-------------------------------------

Comment (by Ryan Scott <ryan.gl.scott@…>):

 In [changeset:"9dc56b61ae10cb26ec8b2bc132ae6c9b620707a8/ghc" 9dc56b6/ghc]:
 {{{
 #!CommitTicketReference repository="ghc"
 revision="9dc56b61ae10cb26ec8b2bc132ae6c9b620707a8"
 Control validity-checking of type synonym applications more carefully

 Trac #16059 shows that when validity checking applications of type
 synonyms, GHC sometimes wasn't checking the expanded type enough.
 We must be careful, however, since checking both the expanded type as
 well as the arguments to the type synonym can lead to exponential
 blowup (see https://ghc.haskell.org/trac/ghc/ticket/16059#comment:4).
 Nor can we omit checking either the expanded type or the argument for
 correctness reasons.

 The solution here is to introduce a new `ExpandMode` data type that
 is plumbed through all of the type-validity-checking functions in
 `TcValidity`. `ExpandMode` dictates whether we only check the
 expanded type (`Expand`), only check the arguments (`NoExpand), or
 both (`Both`). Importantly, if we check `Both` in the function for
 validity checking type synonym applications, then we switch to
 `NoExpand` when checking the arguments so as to avoid exponential
 blowup. See `Note [Correctness and performance of type synonym validity
 checking]` for the full story.
 }}}

-- 
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/16059#comment:18>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler