[Haskell-cafe] Improvements to package hosting and security

[Mathieu Boespflug]

Hi Tilman,

we should certainly have all Haskell users in mind in this discussion,
including beginners, and of course including Windows users.

At the end of the day, MinGHC is the recommended way to get Haskell on
Windows. It is what haskell.org points to. It includes msys, which is
also the main prereq for git on windows. Adding git would grow the
archive size by about 5MB (adding Perl not required) to an archive
that is 125MB in size. So I don't see Git being a problem on Windows.

The more general point here is whether leveraging (arguably standard)
third-party commands and/or C code in order to keep our maintainance
burden low, and pick up many robust features for free to boot, is a
good approach. I believe that it is. Our infrastructure and tooling is
cracking at the seams as it is (cabal-install mysteriously dropping
HTTP connections and corrupting .cabal files when behind a corporate
firewall, updates to hackage-server inadvertently reversing the order
of revisions, low availability of Hackage, ...). Leveraging Git would
solve all mentioned problems, plus give us incremental updates for
free, plus give us package index signing for little effort.

To me that sounds like a pretty big win for 99% of users (including
Windows + OS X), at a *lower* extra maintenance cost for the community
today since we would need to maintain less code. The added reliability
and availability should ultimately benefit beginner users, who can
find it pretty confounding when following instructions just don't
work, because temp glitch, when they've never seen it work before.

Now, we are hearing of use cases where an extra 57MB (including perl,
which should actually be optional) unpacked is a liability for
minimalistic server images. I fully expect other niche use cases that
would prefer a different technical solution. But if we use a de facto
standard format for histories and for distributing signatures, then we
can support multiple ways of accessing and manipulating it, including
via a custom haskell (or existing C) git downloader if users of the
niche use cases deem the cost worth it.

Also, one can envision specialized mirrors where appropriate for
certain niche use cases. Why should the "canonical" (or upstream)
source for the package index be served via Git then? Because I believe
it makes the common case code path on Hackage simpler, and the end
user tools people use in the common case simpler, including for
advanced features like index signing, which we get nearly for free
once we switch to Git. Keep the common case simple, make niche cases
possible without complicating the common case.

That's what the Julia and the Ocaml folks have been betting on, and
for having tried out their tools just recently, they've ended up with
tooling that's arguably quite a bit more user friendly than we have.



On 2 May 2015 at 12:43, Tillmann Rendel
<rendel at informatik.uni-tuebingen.de> wrote:
> Hi,
>
> [I decided to drop haskell-infrastructure at community.galois.com from the CC
> list because for my last message in this thread, I got some noise about
> moderation].
>
> amindfv at gmail.com wrote:
>>
>> I think the idea is that package signing is not a requirement, but
>> that git is a requirement for package signing. So users can still get
>> the behavior that they get today, without git.
>
>
> So there would be `cabal update --unsigned` and `cabal update --signed`
> and the former doesn't need git?
>
> I skimmed the the proposal at
>
> https://github.com/commercialhaskell/commercialhaskell/wiki/Git-backed-Hackage-index-signing-and-distribution
>
> and did not find this information there. Instead, I found this snippet:
>
>> Especially in developing countries, it would be a real liability for
>> Haskell if the first step before doing anything is having to download
>> a 1GB Git archive. Especially considering that given the current
>> growth curve, the Git repository with all content imported will
>> likely be hitting 2GB by this time next year, and so on.
>
>
> This sounds as if for all Haskell users, "the first step before doing
> anything" would have to be to use git.
>
>   Tillmann
>
> PS. BTW, check out this stack overflow question to understand why installing
> and configuring git will be hard for some Haskell users on Windows:
>
> http://stackoverflow.com/questions/30000688/windows-loading-haskell-source-code-into-ghci