Proposal: better library management ideas (was: how to checkout proper submodules)

[Austin Seipp]

On Sun, Jun 9, 2013 at 3:47 AM, Jan Stolarek <jan.stolarek at p.lodz.pl> wrote:
> I admire your talent for writing emails ;-)

You can be honest and just call them what they are: horribly written novellas.

> As you wrote in your email I'm totally for including testsuite into GHC, because it is essentially
> part of GHC and it doesn't make sense to have a version of testsuite not corresponding to a
> version of GHC. As you pointed out the same argument can be used for other packages, but still
> there is one thing I don't like about that idea. What if an average haskeller wants to improve
> one of the libraries e.g. by adding comments or fixing a minor bug? If we have a super-repo that
> person would need to check out everything, which is discouraging.

This is a good point I hadn't considered, but it's less of a worry for
some packages than others. For example, base, ghc-prim and
template-haskell are so intimately tied into GHC that reinstalling
them is either impossible or a bad idea. To change them, you must
build your own GHC anyway (either from source, or HEAD.) And if you're
using a Haskell Platform compiler, clearly you'd have no luck with the
git repository anyway (due to their strong interdependence.)

But again, I'm totally OK with a lot of these other repositories being
submodules. For example, process, unix, deepseq, filepath, directory.
Those don't need to be folded in. Lots of them could have their own
maintainers with separate upstreams. They're touched infrequently
enough traffic concerns aren't as much of a deal. I just want the most
high-traffic'd repositories dealt with, because in practice these are
the *most* critical and the most interdependent. That in turn leads to
the most problems.

> Another, separate issue here is
> that such a person needs to either register to ghc-devs or trac to send a patch. Using github
> would be helpful here, though I agree with Geoffrey about merge commits - we'd have to think of
> sth here. Also, the fact that GHC HQ is maintaining all of the mentioned packages doesn't mean
> that they need to be stored in one repo, at least not in git (this would make more sense to me
> with SVN where you can checkout a subdirectory).

Not necessarily, the 'owners' of the packages are still the libraries
committee. People can propose changes there as they have always done.
It just so happens most of the 'libraries' maintained packages are
de-facto maintained by GHC people.

You're right not all of them need to be folded in. But I think several
of them should be, and these are the ones that hurt the most.

(Plus, my radical proposal can't be considered totally, completely
radical unless I propose something which would - of course - be shot
down.)

> Still, I strongly agree that sth should be done about current setup. I'm not a git guru so I
> cannot fully foresee what would be the consequences of turning everything into submodules, but I
> think that it cannot be worse than it is now, right?

For some submodules, it could certainly be worse. Please see Ian's
link in the prior discussion concerning submodules - for high-traffic
repositories, some of the concerns are disconcerning.

> Jan
>
> Dnia niedziela, 9 czerwca 2013, Roman Cheplyaka napisał:
>> Hi Austin,
>>
>> I apologize for not having read the full email yet (I'm in a hurry right
>> now), but...
>>
>> * Austin Seipp <aseipp at pobox.com> [2013-06-09 00:23:22-0500]
>>
>> > --> Let's just put base and testsuite inside the GHC repository
>> > directly. No submodules, no floating repos. Just put it directly
>> > inside and make a super commit, I guess. GHC becomes the de facto
>> > repository. And hey, why not nofib?
>> >
>> > I know, I know. People really want to split the maintenance burdens I
>> > guess, and ideologically the Haskell community is all about clean
>> > separation but, please? All of GHC HQ are the de facto maintainers of
>> > this stuff anyway. And as Jan mentioned, testsuite is really *so*
>> > crucial GHC should have it inline. The testsuite is perhaps the most
>> > important of al
>> >
>> > There are other candidates for this treatment too, really. For
>> > example, why is template-haskell, ghc-prim, and hpc split out? GHC is
>> > the only thing that supports them. template-haskell is especially
>> > super-intrusive of an extension to support, and arguably hpc as well.
>> > integer-simple and integer-gmp follow the exact same story. Same with
>> > hoopl and dph. They're all ours. We own them. Just put them all inside
>> > GHC and be done with it. Having active fragmentation in the VCS is not
>> > necessary when there need be none. These packages de-facto ship with
>> > GHC and are very tied to it.
>>
>> I'm a strong -1 on this. As one example, we have forks of base and
>> ghc-prim for Haskell suite:
>>
>>   https://github.com/haskell-suite/base
>>   https://github.com/haskell-suite/ghc-prim
>>
>> which would be much more complicated if these were not independent
>> repositories.
>>
>> But more generally, I think there's still hope that the core packages
>> will be made portable — I'm referring to Joachim Breitner's work on
>> splitting the base.
>>
>> Roman
>>
>> _______________________________________________
>> ghc-devs mailing list
>> ghc-devs at haskell.org
>> http://www.haskell.org/mailman/listinfo/ghc-devs
>
>



-- 
Regards,
Austin - PGP: 4096R/0x91384671