[Haskell-cafe] MRP, 3-year-support-window, and the non-requirement of CPP

[Gershom B]

Dear all,

I think this discussion has gotten quite heated for reasons not related to the concrete MRP proposal, which, to be honest, I considered quite modest in terms of both scope and impact.

Instead, I think it is a proxy for lots of remaining frustration and anxiety over the poor handling over the Foldable Traversable Proposal. I would like to remind everyone that due to the broad discussions and concerns over the proposal, a very rare, careful poll of Haskell users was taken, announced broadly in many channels. [1] The poll, overwhelmingly, revealed a mandate for the FTP. The breakdown of that mandate was 87% in favor among hobbyists and 79% in favor among non-hobbyists (who constituted a majority of those polled). 

I. Generalities

That said, even the _best_ poll was not a substitute for a better earlier discussion. The handling of the AMP and FTP, which I think was heroic in terms of minimizing breakage while accomplishing long-desired change also still could have been better. As a whole, the work accomplished the mandate of allowing code to be written backwards-compatible without requiring CPP. However, it did not also seek to prevent warnings. This in itself was an enormous step forward from changes in the past which have _not_ even managed to prevent the need for CPP. At the time, I think it was not recognized how much desire there would be for things that were _both_ CPP free and _also_ warning-free for 3 releases.

I think one of the great elements of progress in the current discussion is that there is now a proposal on the table which recognizes this, and seeks to accomplish this change in accordance with this desire. It is not the world’s most important change, but the recognition that change should seek to be both CPP _and_ warning free is a good recognition, and I’m sure it will be taken into account in future proposals as well.

I don’t think it is useful to continue to have abstract discussions on the conflict between desire for incremental improvement versus the need to minimize pain on maintainers. We might as well continue to argue about the need for purely functional programming versus the need to print “hello world” to the console. Rather, we should put our collective minds together as collaborators and colleagues to accomplish _both_, and to come up with solutions that should work for everyone. To the extent this discussion has been about that, I think it has been useful and positive. However, to the extent this discussion insists, on either side, on the shallow idea that we must treat “improvement” versus “stability” as irreconcilable factions in necessary conflict, then I fear it will be a missed opportunity.

II. Particulars

With that in mind, I think the _concrete_ voices of concern have been the most useful. Gregory Collins’ list of issues requiring CPP should be very sobering. Of note, I think they point to areas where the core libraries committee has not paid _enough_ attention (or perhaps has not been sufficiently empowered: recall that not all core libraries fall under its maintenance [2]). Things like the newtype FFI issue, the changes to prim functions, the splitup of old-time and the changes to exception code were _not_ vetted as closely as the AMP and FTP were, or as the MRP is currently being. I don’t know all the reasons for this, but I suspect they just somewhat slipped under the radar. In any case, if all those changes were as carefully engineered as the MRP proposal has been, then imho things would have been much smoother. So, while this discussion may be frustrating, it nonetheless in some ways provides a model of how people have sought to do better and be more proactive with careful discussion of changes. This is much appreciated.

Personally, since the big switch to extensible exceptions back prior in 6.10, and since the split-base nonsense prior to that, very few changes to the core libraries have really caused too much disruption in my code. Since then, the old-time cleanup was the worst, and the big sin there was that time-locale-compat was only written some time after the fact by a helpful third-party contributor and not engineered from the start. (I will note that the time library is one of the core libraries that is _not_ maintained by the core libraries committee). 

Outside of that, the most disruptive changes to my code that I can recall have been from changes to the aeson library over the years — particularly but not only regarding its handling of doubles. I don’t begrudge these changes — they iteratively arrived at a _much_ better library than had they not been made. [3] After than, I made a few changes regarding Happstack and Snap API changes if I recall. Additionally, the addition of “die” to System.Exit caused a few name clashes. My point is simply that there are many packages outside of base that also move, and “real” users with “real” code will these days often have quite a chain of dependencies, and will encounter movement and change from across many of them. So if we say “base never changes” that does not mean “packages will never break” — it just means that base will not have the same opportunity to improve that other packages do, which will eventually lead to frustration, just as it did in the past and in the leadup to the BBP.

III. Discussions

Further, since there has been much discussion of a window of opportunity, I would like to offer a counterpoint to the (sound) advice that we take into consideration voices with long experience in Haskell. The window of opportunity is, by definition, regarding takeup of Haskell by new users. And so if newer users favor certain changes, then it is good evidence that those changes will help with uptake among other new users. So, if they are good changes on their own, then the fact that they are appealing to newer users should be seen as a point in their favor, rather than a reason to dismiss those opinions. But if we are in a situation where we see generations of adopters pitted against one another, then we already have deeper problems that need to be sorted out.

Regarding where and how to have these discussions — the decision was made some time ago (I believe at the start of the initial Haskell Prime process if not sooner, so circa 2009?) that the prime committee would focus on language extensions and not library changes, and that those changes would be delegated to the libraries@ list. The lack of structure to the libraries@ list is what prompted the creation of the libraries committee, whose ultimately responsibility it is to decide on and shepherd through these changes, in consultation with others and ideally driven by broad consensus. Prior to this structure, things broke even more, imho, and simultaneously the things that were widely desired were still not implemented. So I thank the libraries committee for their good work so far.

So, it may be that the process of community discussion on core libraries changes is not best suited for the libraries@ list. But if not there, Where? I worry that the proliferation of lists will not improve things here. Those involved with Haskell have multiplied (this is good). The voices to take into account have multiplied (this is good). Necessarily, this means that there will just be _more_ stuff, and making sure that everyone can filter to just that part they want to is difficult. Here, perhaps, occasional libraries-related summary addenda to the ghc newsletter could be appropriate? Or is there another venue we should look towards? “Chair’s reports” to the Haskell Weekly News maybe?

IV. Summing up

We should bear in mind after all that this is just about cleaning up a redundant typeclass method (albeit one in a very prominent place) and hardly the hill anyone would want to die on [4]. Nonetheless, I think it would be a good sign of progress and collaboration if we can find a way to implement a modest change like this in a way that everyone finds acceptable vis a vis a sufficiently slow pace, the lack of a need for CPP and the lack of any induced warnings. On the other hand, other opportunities will doubtless present themselves in the future.

Best,
Gershom

[1] https://mail.haskell.org/pipermail/libraries/2015-February/025009.html
[2] https://wiki.haskell.org/Library_submissions#The_Core_Libraries
[3] and in any case I am sure Bryan would be the last to want us to treat him as some sort of “guru” on these matters. 
[4] for those in search of better hills to die on, this is a list of some good ones: http://www.theawl.com/2015/07/hills-to-die-on-ranked 

P.S. In case there is any question, this email, as all emails I write that do not state otherwise, is not being written in any particular capacity regarding the various infra-related hats I wear, but is just an expression of my own personal views.