small improvement to roles mechanism

[Austin Seipp]

(Sigh, I used the wrong email again. Here it is for the list, and
apologies to recipients. My client has gotten quite annoying with its
defaults. Verbatim message below.)

I'm not actively involved in the design of Roles or Coercible, but
here are my opinions:

1. We're marking Coercible as experimental anyway. Several people seem
to be quite excited about it, but clearly the design is up for change.
We're leaving it in because we want feedback on the design - we
shouldn't try to cram everything. The expectation here should clearly
be "it will change and break your programs, so proceed with caution."
I think that's OK: the people who are willing to do so will know what
they're dealing with.

2. Following that - if roles are pointing out legitimate type errors
in programs, I think point #2 Richard mentioned has merit: we can
liberalize the GND check a bit, and let the minor failures fix
themselves and get cleaned up - and disallow Coercible.

Richard, Joachim - If the design of Coercible is going to change, we
can just bar that, can't we? Do we expect the *core* of role inference
and checking to change much beyond this, or does most of the flux
still lie between the intersection of these two things? As I said, I
haven't kept up too much, but as an outsider familiar with the GND
problem, it seems that Roles on-the-whole don't look like they're
really in that much of a flux still.

2.1. As a minor point, the amount of build failures due to roles seems
minor enough that I also don't think it will be anywhere close to the
biggest adoption hurdle. Most of the ones mentioned in Bryan's email
actually have a fairly small number of dependencies, almost all of
them negligable modulo language-c which is required by c2hs. But I
still think this is a fairly minor problem given the numbers and can
be fixed with less hassle than it's made out to be.

On a separate note:

7.8 is way, way, *way* too early for an HP release if it's scheduled
for November. There is no way we can push it out so early - as Mark
said, it's essentially impossible at this point.

If OS X is going to break, this is perhaps indication that instead we
should seriously re-evaluate a potential 7.6.4 release. Several people
have asked me about it, and I had up-until-now not really considered
it. But perhaps it is the right thing to do instead of forcing an
upgrade on users with a bleeding-edge compiler, the tension seems
quite high.

I wrote the original patches to Clang and GHC to fix the OS X issues.
So this one lies on my head and I'm willing to bite the bullet here
and prepare a concurrent release - but I'm not going to have open
season on "everyone gets their favorite 7.6.x bug into the 7.6.4
release." The major hurdle I have is that I don't have a
Mavericks/XCode 5 machine on hand (only a Lion machine,) so I'll need
access to one.

(The general problem here, really, is that Clang is a bit annoying as
a preprocessor for Haskell code and far stricter than GCC, so
generally-speaking the tweaks required are tiresome. For the 7.10
timeframe, we'll be looking into just writing our own traditional-mode
preprocessor.)

On Fri, Oct 11, 2013 at 9:19 PM, Richard Eisenberg <eir at cis.upenn.edu> wrote:
> I tend to agree with the above remarks and have been silencing the little
> voice in the back of my head that has suggested we pull roles out of 7.8.
> Ever since the debate about default roles vis-a-vis abstract datatypes
> started, I've been wondering if we're being a little hasty here.
>
> As we think about how to proceed, I think it's worth teasing apart the two
> different strains in all of this: enforcing type safety and enforcing
> abstraction.
>
> Roles, as they are, do a quite good job of enforcing type safety. When
> compiling all of Hackage, the majority of the breakage from roles was due to
> real type goofs in programs. A total of 18 packages (out of the 3,234 that
> compile with 7.6.3) fail to compile because of deficiencies in the role
> system, including transitive breakage (i.e., packages that depend on broken
> packages). A small change in the GND check discussed in this forum would fix
> 11 of these, leaving only 7. Given that type systems are necessarily
> conservative, I'm not too displeased with this result -- the 3 packages that
> would need to be changed would require only a few lines of code. Given this
> result, I disagree with Edward's claim that roles will provide the biggest
> pain in upgrading from 7.6 to 7.8.
>
> That said, I of course would like to do better and would prefer to 0 broken
> packages due to deficiencies in the role system.
>
> On the other hand, roles as they are do a poor job of enforcing abstraction.
> But, before roles came along, there wasn't anything resembling a way of
> enforcing abstraction in the context of GND. So, this isn't exactly a new
> problem. The only thing (I think) exacerbating the problem is that Coercible
> now makes it easier than ever to break abstraction.
>
> That said, I of course would like to fix this problem, too.
>
> So, how to proceed? There are a few options:
>
> 1. I'll change the GND check to be a little more liberal, and we release
> roles in 7.8. I could notify the authors of the three packages that need to
> be updated because of the lack of role abstraction.
>
> 2. Like #1, but disallow Coercible. This way, the abstraction problem is no
> worse than it was before. (Apologies to Joachim if he minds this
> suggestion.)
>
> 3. Pull roles out of 7.8, giving us a little more time to Get It Right.
>
> It's a little hard for me to choose between these options, but I think it's
> good to be conservative in language design of a language in real use, so I
> lean slightly toward option #3. Doing this is very feasible from a technical
> standpoint -- it's easy to turn off the checks.
>
> If we go with #3, given the flux in the design, I'm even uncertain about the
> idea of warnings in 7.8. It would all feel a little silly if we issue these
> warnings and then change the design between 7.8 and 7.10 just enough to make
> the warnings wrong. I think warnings in 7.8.2 is a better idea.
>
> As for Johan's remark that the feature should have been vetted more
> thoroughly -- I completely agree, and I'll take responsibility for that
> decision. I did do some ad-hoc testing against packages on Hackage known to
> use GND, but the testing was not as thorough as it could have been. That
> said, I'm not sure anything would have played out too differently had I done
> more extensive testing; the results were about as I expected.
>
> I do greatly appreciate everyone's feedback and interest in this!
>
> Richard
>
> On Oct 11, 2013, at 7:55 PM, Edward Kmett wrote:
>
> I have to agree that I'm somewhat disturbed by the fact that we're pushing
> this out and we're still finding issues with it this close to release. =(
>
> It strikes me that the role machinery is going to be the cause of the
> majority of the pain users have upgrading to 7.8, and if I try to pretend to
> be Mark Lentczner for a bit, it makes it seem highly likely that it'd be the
> kind of thing that keeps 7.8 from going into a Haskell Platform, causing the
> groundhog to see his shadow, leaving us with another year of 7.4 or 7.6.x.
>
> I know we're at the 5 yard line, but to metaphorically throw a bunch of
> metaphors in a blender, if we had to make the uncomfortable decision to
> perform triage and ask if it should be put off (at least enforcing) the
> roles machinery to 7.10, so we can know we have it right, how much fallout
> would there be? Off the top of my head, of course Joachim's work on
> Coercible would be affected. What else?
>
> One option might be to pull the teeth of role inference for 7.8 with regards
> to GND, and turn bad roles use into a warning for a release cycle. That
> would give the community a year to get role annotations in place before
> generalized newtype deriving for their code just stopped working.
>
> If we ship with this the way it stands, I don't foresee the community
> reaction being good.
>
> With its teeth pulled, then GND could proceed as before, but with the added
> detailed warnings from the dictionary coercions helping to guide folks to
> make the change. By the time we'd be enforcing correct role annotations most
> folks would have them in place to silence the warnings.
>
> -Edward
>
>
> On Fri, Oct 11, 2013 at 7:15 PM, Johan Tibell <johan.tibell at gmail.com>
> wrote:
>>
>> Oh and let me add: it would have been nice to have the people actually
>> making these change to have done an impact analysis on Hackage, instead of
>> discovering potential issues a week or two before the release. Lets try to
>> do that next time.
>>
>>
>> On Fri, Oct 11, 2013 at 4:14 PM, Johan Tibell <johan.tibell at gmail.com>
>> wrote:
>>>
>>> Let me start by saying that I'm happy we're trying to fix the GND
>>> problem. Thanks for working on that.
>>>
>>> That being said: is this ready for mainstream consumption? We're forcing
>>> this on everyone without any language pragma or flags to opt-in/out. That is
>>> bad if we're not sure we're doing the right thing in some cases or if we're
>>> causing spurious failures. At ICFP I got the impression that very few people
>>> will be affected, but Bryan's result suggests there are more people than I
>>> thought.
>>>
>>>
>>>
>>> On Thu, Oct 10, 2013 at 8:26 PM, Richard Eisenberg <eir at cis.upenn.edu>
>>> wrote:
>>>>
>>>> In Bryan's recent test of GHC 7.8 against all of Hackage, there were
>>>> three spurious errors caused by lack of role abstraction. Here are the class
>>>> definitions where a nominal parameter is inferred, probably against the
>>>> wishes of the author:
>>>>
>>>> from logict-0.2.3/Control.Monad.Logic.Class:
>>>> > class (MonadPlus m) => MonadLogic m where
>>>> >     msplit     :: m a -> m (Maybe (a, m a))
>>>>
>>>> from monadLib-3.5.2/MonadLib:
>>>> > class (Monad m) => ReaderM m i | m -> i where
>>>> >   ask :: m i
>>>>
>>>> from base/Control.Arrow:
>>>> > class Arrow a => ArrowApply a where
>>>> >     app :: a (a b c, b) c
>>>>
>>>> In each of these, the last parameter of the class is given a nominal
>>>> role because it appears as the parameter of a type variable. However, in
>>>> each case, it appears as the parameter of a *class* type variable. This
>>>> means that, if we somehow knew that the class author wanted the class to be
>>>> usable with GND, we could simply check every instance declaration for that
>>>> class to make sure that the relevant concrete instantiation has the right
>>>> role. For example, when the user writes, for example
>>>>
>>>> > instance ArrowApply Foo where ?
>>>>
>>>> we check that Foo's first parameter has a representational role. If it
>>>> doesn't, then the instance is rejected.
>>>>
>>>> An alternative, somewhat heavier idea would be to represent roles as
>>>> class constraints. We could have
>>>>
>>>> > class NextParamNominal (c :: k)
>>>> > class NextParamRepresentational (c :: k)
>>>>
>>>> GHC could "generate" instances for every datatype definition. For
>>>> example:
>>>>
>>>> > type role Map nominal representational
>>>> > data Map k v = ?
>>>>
>>>> would induce
>>>>
>>>> > instance NextParamNominal Map
>>>> > instance NextParamRepresentational (Map k)
>>>>
>>>> Users would not be able to write these instances -- they would have to
>>>> be generated by GHC. (Alternatively, there could be no instances, just a
>>>> little magic in the constraint solver. Somewhat like Coercible.)
>>>>
>>>> Then, the classes above would just have to add a superclass, like this:
>>>>
>>>> > class (Arrow a, NextParamRepresentational a) => ArrowApply a where
>>>> >   app :: a (a b c, b) c
>>>>
>>>> The role inference mechanism would be made aware of role constraints and
>>>> use this one to derive that ArrowApply is OK for GND.
>>>>
>>>> This "heavier" approach has a similar upshot to the first idea of just
>>>> checking at instance declarations, but it is more customizable and
>>>> transparent to users (I think).
>>>>
>>>>
>>>> I'm not sure I'm advocating for this change (or volunteering to
>>>> implement before the release candidate), but I wanted to document the idea
>>>> and get any feedback that is out there. This would fix the breakage we've
>>>> seen without totally changing the kind system.
>>>>
>>>> Thanks,
>>>> Richard
>>>>
>>>> PS: Due credit is to migmit for suggesting the type-class idea on
>>>> glasgow-haskell-users.
>>>> _______________________________________________
>>>> ghc-devs mailing list
>>>> ghc-devs at haskell.org
>>>> http://www.haskell.org/mailman/listinfo/ghc-devs
>>>
>>>
>>
>>
>> _______________________________________________
>> ghc-devs mailing list
>> ghc-devs at haskell.org
>> http://www.haskell.org/mailman/listinfo/ghc-devs
>>
>
>
>
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://www.haskell.org/mailman/listinfo/ghc-devs
>



-- 
Regards,

Austin Seipp, Haskell Consultant
Well-Typed LLP, http://www.well-typed.com/