Trying to refactor DeriveAnyClass, stuck on typechecker-related code

[Ryan Scott]

Simon,

I'm currently working on a solution to #12144, which changes the way
inferred contexts for DeriveAnyClass works, as per your suggestion
here [1]. I've made some good progress in that instances derived using
DeriveAnyClass now emit contexts that are gathered from the default
signatures of the derived class itself.

However, I've come to an impasse. This approach currently fails to
work with most GHC Generics-style classes, for reasons that I'll
explain below. Consider this example (abridged from here [2]):

    data StrictMaybe a = StrictNothing | StrictJust !a
      deriving (FullyStrict, Generic)

    class FullyStrict a where
      fullyStrict :: proxy a -> Bool
      default fullyStrict :: (GFullyStrict (Rep a)) => proxy a -> Bool
      fullyStrict _ = gfullyStrict (Proxy :: Proxy (Rep a p))

With the new approach, the derived FullyStrict instance for
StrictMaybe a will emit the context (GFullyStrict (Rep (StrictMaybe
a)), but then fail with the error:

    No instance for (GFullyStrict (Rep (StrictMaybe a))

I think I understand why this is happening: if you look at the
definition of tcDeriving in TcDeriv [3], it calls
simplifyInstanceContexts, which would normally reduce type families
like Rep. But the problem is that the Rep (StrictMaybe a) instance is
also being derived at the same time (via the derived Generic
instance)! What's worse, the call to simplifyInstanceContexts happens
before the code that adds Rep (StrictMaybe a) to the type family
instance environment (tcExtendLocalFamInstEnv (bagToList famInsts)).
So simplifyInstanceContexts is unable to reduce Rep (StrictMaybe a),
and as a result the context fails to simplify, resulting in the above
error.

This leads me to think we need to be adding Rep (StrictMaybe a) to the
instance environment before calling simplifyInstanceContexts. But with
the way the code is structured currently, I don't see an easy way to
do this. The problem is that the genInst function [4] generates all of
the following at the same time in one large bundle:

* The instance code itself (InstInfo RdrName)
* The associated type family instances (BagDerivStuff)
* Auxiliary top-level bindings (BagDerivStuff)

And within tcDeriving, the call to genInst takes as input the output
of simplifyInstanceContexts, making it impossible to get the type
family instances before calling simplifyInstanceContexts.

Here are my questions: is it possible to take type family instances
out of BagDerivStuff and instead add them to the instance environment
before simplifyInstanceContexts? Would there be any typechecking
issues involved with having the associated family instances in the
local environment before the actual class instances themselves?

Ryan S.
-----
[1] https://mail.haskell.org/pipermail/ghc-devs/2016-June/012276.html
[2] http://git.haskell.org/ghc.git/blob/0050aff22ba04baca732bf5124002417ab667f8a:/testsuite/tests/generics/GFullyStrict.hs
[3] http://git.haskell.org/ghc.git/blob/0050aff22ba04baca732bf5124002417ab667f8a:/compiler/typecheck/TcDeriv.hs#l364
[4] http://git.haskell.org/ghc.git/blob/0050aff22ba04baca732bf5124002417ab667f8a:/compiler/typecheck/TcDeriv.hs#l2269