API changes and HP inclusion [was: Re: Adding binary to the Haskell Platform]

[wren ng thornton]

Ian Lynagh wrote:
> Donald Bruce Stewart wrote:
>> So two things:
>>
>>     1. We can do API-breaking changes in the platform on major releases.
>>
>>     2. When new packages come in, I'm wary of breaking their APIs as
>>         established prior to them being added, as then the platform
>>         release that includes them is not usable as a platform for those
>>         existing packages.
> 
> I don't understand how
>     adding newapi in HPv2
> is worse than 
>     adding oldapi in HPv2
>     replacing it with newapi in HPv3
> 
> Either way, when you put newapi in the HP there will be a period where
> other packages need to catch up with the changes.

The difference is this: If we follow the former (API before HP) then 
people who need to debug the upgrade process will have to switch back 
and forth between HP vs raw Cabal, which could be a nightmare. If we 
follow the latter (HP before API) then developers will only need to 
toggle between different HP versions, which would be a lot easier to 
setup in most environments.

I think the HP-before-API path is more likely to keep us agile since it 
makes the API breaking changes easier to fix. However, it does have the 
downside of including known "broken" packages in the HP, even if only 
for a little while. The question is, how much do we want the HP to serve 
as a migration path? If expected breakage is small, then the API changes 
should probably come before HP inclusion. If the expected breakage is 
large ---as it will be for any popular package--- then it may be good to 
have an initial HP which solidifies the "classic" interface so that 
unknown and one-off projects, which may not be maintained, can still 
make use of the old API as well as the benefits of the HP.

That is, there's some benefit in providing snapshots of the community as 
it is, rather than the ideal of what we would like. The question is the 
extent to which this is the HP's job. We do already have Cabal and 
Hackage which would allow users to install old versions of packages, so 
the benefits would be in the one-click install as well as capturing the 
in-time correlations between different packages (which Hackage doesn't 
currently provide a nice interface for). Providing semi-blessed packages 
in the HP before changing APIs could help improve buy in, and will 
reduce the disturbance of adding new packages, but it also increases the 
instability for end users since there will be more API changes within 
the HP track.


>> Maybe 2 is not a valid concern. Maybe the HP inclusion is the time to
>> break things. But maybe we won't then add popular packages -- because
>> changing their APIs as requested would break too many things :/
> 
> I think we have to be willing to break things, or we will become
> obsolete. There is a balance between stability and progress, of course.

I'd say that stability is actually the balance between legacy and 
progress. Ironically, Hackage serves both legacy and progress, since it 
gives access to old versions and GHC is able to switch between different 
versions of packages. From that perspective, the HP serves as the 
balance, providing a stable subset of Hackage. This is a rather unique 
arrangement since most package trees conflate the recording of history 
and the stable image of it. The question remains, how closely should the 
fulcrum follow the growing edge?

-- 
Live well,
~wren