[Haskell-cafe] Improvements to package hosting and security

Tue Apr 28 21:07:21 UTC 2015

This is a valid concern. One that I should have addressed explicitly
in the proposal. Git is fairly well supported on Windows these days
and installs easily. It could conceivably be included as part of
MinGHC. There are many alternatives, but I doubt we'll need them:
statically linking a C implementation (libgit2 or another), or a
simple native implementation of the git protocol (the protocol is
quite straightforward and is documented) and basic disk format.

The same is true about GnuPG, via gpg4win, though note that under this
proposal GnuPG wouldn't be a requirement for `cabal update` to work.
Just an additional optional dependency which you'll want to have
installed if you want to protect yourself from the attacks listed in
the proposal.

By the way, one side note about this Git proposal: it sides steps the
discussion around how to add SSL support to cabal-install entirely.
Since Git understands (among others) HTTPS natively, so we can
outsource our support for that to Git. In any case SSL no longer
becomes a necessity for protecting against MITM (the commit signing
takes care of that), only a nice-to-have for privacy.

On 28 April 2015 at 19:46,  <radix at twistedmatrix.com> wrote:
> To me, the elephant in the room is how the dependency on Git will be
> handled. I'm not a Windows user, but how much more painful will it be to set
> up a Haskell environment on Windows with a new dependency on Git? Will users
> need to install it separately, or do you suggest embedding Git into the
> relevant tools? Should the Haskell Platform bundle it? What about MinGHC?
> Oh, and I guess the same question can be asked about GnuPG.
>
> I personally use a Mac and Homebrew, so it's pretty easy for me to install
> those dependencies, and I'm sure the same is true on Linux. But also, not
> everyone uses Homebrew (in fact, I'm sure most programmers on Macs don't use
> it), so it's also worth considering whether the requisite tools should be
> embedded in the "GHC for Mac OS X" distribution.
>
> On Linux this probably isn't an issue because pretty much everyone has a
> decent dependency-tracking package manager.
>
> I don't know if you care personally about these issues, but I think any
> proposal which introduces new dependencies to the core development
> environment of Haskell should take it into consideration. Very few people
> have Git and GPG already installed, and I think the new-user experience
> should be considered, and I'm surprised nobody has mentioned it in this
> entire thread (unless I missed it).
>
> -- radix (Christopher Armstrong)
>
> P.S. I'm very excited to see this work, including the emphasis on using the
> well-researched TUF. Thanks to you and other people working on this. :)
>
> On Tuesday, April 28, 2015 at 5:07:56 AM UTC-5, Mathieu Boespflug wrote:
>>
>> Hi all,
>>
>> last week, I found some time to write up a very simple proposal that
>> addresses the following goals simultaneously:
>>
>> - maintain a difficult to forge public audit log of Hackage updates;
>> - make downloads from Hackage mirrors just as trustworthy as
>> downloading from Hackage itself;
>> - guarantee that `cabal update` is always pulling the freshest package
>> index (called "snapshots" in the proposal), and detect when this might
>> not be the case;
>> - implement the first half of TUF (namely the index signing part
>> discussed in Duncan's blog post, not the author package signing part)
>> with fewer metadata files and in a way that reuses existing tooling;
>> -  get low-implementation-cost, straightforward and incremental `cabal
>> update`.
>>
>> After a preliminary review from a few colleagues and friends in the
>> community, here is the proposal, in the form of Commercial Haskell
>> wiki page:
>>
>>
>> https://github.com/commercialhaskell/commercialhaskell/wiki/Git-backed-Hackage-index-signing-and-distribution
>>
>> The design constraints here are:
>>
>> - stay backwards compatible where the cost for doing so is low.
>> - reuse existing tooling and mechanisms, especially when it comes to
>> key management, snapshot identity, and distributing signatures.
>> - Focus on the above 5 goals only, because they happen to all be
>> solvable by changing a single piece of mechanism. But strive to reuse
>> whatever mechanism others are proposing to solve other goals (e.g.
>> certification of provenance using author package signing, as Chris
>> Done has already proposed).
>>
>> To that effect, the tl;dr is that I'm proposing that we just use Git
>> for maintaining the Hackage package index, that we use Git for
>> synchronizing this locally, and that we use Git commit signatures for
>> implementing the first half of TUF. The Git tooling currently assumes
>> GnuPG keys for signatures, so I'm proposing that we use GnuPG keys for
>> signing, and that we manage key revocation and any trust delegation
>> between keys using GnuPG and its existing infrasture.
>>
>> I estimate the total effort necessary here to be the equivalent of 5-6
>> full time days overall. However, I have not pooled the necessary
>> resources to carry that out yet. I'd like to get feedback first before
>> going ahead with this, but in meantime,
>>
>> ** if there are any volunteers that would like to signal their intent
>> to help with the implementation effort then please add your name at
>> the bottom of the wiki page. **
>>
>> Best,
>>
>> Mathieu
>>
>> On 18 April 2015 at 20:11, Michael Snoyman <mic... at snoyman.com> wrote:
>> >
>> >
>> > On Sat, Apr 18, 2015 at 12:20 AM Bardur Arantsson
>> > <sp... at scientician.net>
>> > wrote:
>> >>
>> >> On 17-04-2015 10:17, Michael Snoyman wrote:
>> >> > This is a great idea, thank you both for raising it. I was discussing
>> >> > something similar with others in a text chat earlier this morning.
>> >> > I've
>> >> > gone ahead and put together a page to cover this discussion:
>> >> >
>> >> >
>> >> >
>> >> > https://github.com/commercialhaskell/commercialhaskell/blob/master/proposal/improved-hackage-security.md
>> >> >
>> >> > The document definitely needs more work, this is just meant to get
>> >> > the
>> >> > ball
>> >> > rolling. As usual with the commercialhaskell repo, if anyone wants
>> >> > edit
>> >> > access, just request it on the issue tracker. Or most likely, send a
>> >> > PR
>> >> > and
>> >> > you'll get a commit bit almost magically ;)
>> >>
>> >> Thank you. Just to make sure that I understand -- is this page only
>> >> meant to cover the original "strawman proposal" at the start of this
>> >> thread, or...?
>> >>
>> >> Maybe you intend for this to be extended in a detailed way under the
>> >> "Long-term solutions" heading?
>> >>
>> >> I was imagining a wiki page which could perhaps start out by collecting
>> >> all the currently identified possible threats in a table, and then all
>> >> "participants" could perhaps fill in how their suggestion addresses
>> >> those threats (or tell us why we shouldn't care about this particular
>> >> threat). Of course other relevent non-threat considerations might be
>> >> relevant to add to such a table, such as: how prevalent is the
>> >> software/idea we're basing this on? does this have any prior
>> >> implementation (e.g. the append-to-tar and expect that web servers will
>> >> behave sanely thing)? etc.
>> >>
>> >> (I realize that I'm asking for a lot of work, but I think it's going to
>> >> be necessary, at least if there's going to be consensus and not just a
>> >> de-facto "winner".)
>> >>
>> >>
>> >
>> > Hi Bardur,
>> >
>> >
>> > I don't think I have any different intention for this page than you've
>> > identified. In fact, I thought that I had clearly said exactly what you
>> > described when I said:
>> >
>> >> There are various ideas at play already. The bullets are not intended
>> >> to
>> >> be full representations of the proposals, but rather high level
>> >> summaries.
>> >> We should continue to expand this page with more details going forward.
>> >
>> > If this is unclear somehow, please tell me. But my intention absolutely
>> > is
>> > that many people can edit this page to add their ideas and we can flesh
>> > out
>> > a complete solution.
>> >
>> > Michael
>> >
>> > _______________________________________________
>> > Haskell-Cafe mailing list
>> > Haskel... at haskell.org
>> > http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
>> >
>> _______________________________________________
>> Haskell-Cafe mailing list
>> Haskel... at haskell.org
>> http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe