[Haskell-cafe] Improvements to package hosting and security

Tue Apr 28 21:09:21 UTC 2015

[removing erroneous haskell-cafe at googlegroups.com from To list.]

On 28 April 2015 at 23:07, Mathieu Boespflug <mboes at tweag.net> wrote:
> This is a valid concern. One that I should have addressed explicitly
> in the proposal. Git is fairly well supported on Windows these days
> and installs easily. It could conceivably be included as part of
> MinGHC. There are many alternatives, but I doubt we'll need them:
> statically linking a C implementation (libgit2 or another), or a
> simple native implementation of the git protocol (the protocol is
> quite straightforward and is documented) and basic disk format.
>
> The same is true about GnuPG, via gpg4win, though note that under this
> proposal GnuPG wouldn't be a requirement for `cabal update` to work.
> Just an additional optional dependency which you'll want to have
> installed if you want to protect yourself from the attacks listed in
> the proposal.
>
> By the way, one side note about this Git proposal: it sides steps the
> discussion around how to add SSL support to cabal-install entirely.
> Since Git understands (among others) HTTPS natively, so we can
> outsource our support for that to Git. In any case SSL no longer
> becomes a necessity for protecting against MITM (the commit signing
> takes care of that), only a nice-to-have for privacy.
>
> On 28 April 2015 at 19:46,  <radix at twistedmatrix.com> wrote:
>> To me, the elephant in the room is how the dependency on Git will be
>> handled. I'm not a Windows user, but how much more painful will it be to set
>> up a Haskell environment on Windows with a new dependency on Git? Will users
>> need to install it separately, or do you suggest embedding Git into the
>> relevant tools? Should the Haskell Platform bundle it? What about MinGHC?
>> Oh, and I guess the same question can be asked about GnuPG.
>>
>> I personally use a Mac and Homebrew, so it's pretty easy for me to install
>> those dependencies, and I'm sure the same is true on Linux. But also, not
>> everyone uses Homebrew (in fact, I'm sure most programmers on Macs don't use
>> it), so it's also worth considering whether the requisite tools should be
>> embedded in the "GHC for Mac OS X" distribution.
>>
>> On Linux this probably isn't an issue because pretty much everyone has a
>> decent dependency-tracking package manager.
>>
>> I don't know if you care personally about these issues, but I think any
>> proposal which introduces new dependencies to the core development
>> environment of Haskell should take it into consideration. Very few people
>> have Git and GPG already installed, and I think the new-user experience
>> should be considered, and I'm surprised nobody has mentioned it in this
>> entire thread (unless I missed it).
>>
>> -- radix (Christopher Armstrong)
>>
>> P.S. I'm very excited to see this work, including the emphasis on using the
>> well-researched TUF. Thanks to you and other people working on this. :)
>>
>> On Tuesday, April 28, 2015 at 5:07:56 AM UTC-5, Mathieu Boespflug wrote:
>>>
>>> Hi all,
>>>
>>> last week, I found some time to write up a very simple proposal that
>>> addresses the following goals simultaneously:
>>>
>>> - maintain a difficult to forge public audit log of Hackage updates;
>>> - make downloads from Hackage mirrors just as trustworthy as
>>> downloading from Hackage itself;
>>> - guarantee that `cabal update` is always pulling the freshest package
>>> index (called "snapshots" in the proposal), and detect when this might
>>> not be the case;
>>> - implement the first half of TUF (namely the index signing part
>>> discussed in Duncan's blog post, not the author package signing part)
>>> with fewer metadata files and in a way that reuses existing tooling;
>>> -  get low-implementation-cost, straightforward and incremental `cabal
>>> update`.
>>>
>>> After a preliminary review from a few colleagues and friends in the
>>> community, here is the proposal, in the form of Commercial Haskell
>>> wiki page:
>>>
>>>
>>> https://github.com/commercialhaskell/commercialhaskell/wiki/Git-backed-Hackage-index-signing-and-distribution
>>>
>>> The design constraints here are:
>>>
>>> - stay backwards compatible where the cost for doing so is low.
>>> - reuse existing tooling and mechanisms, especially when it comes to
>>> key management, snapshot identity, and distributing signatures.
>>> - Focus on the above 5 goals only, because they happen to all be
>>> solvable by changing a single piece of mechanism. But strive to reuse
>>> whatever mechanism others are proposing to solve other goals (e.g.
>>> certification of provenance using author package signing, as Chris
>>> Done has already proposed).
>>>
>>> To that effect, the tl;dr is that I'm proposing that we just use Git
>>> for maintaining the Hackage package index, that we use Git for
>>> synchronizing this locally, and that we use Git commit signatures for
>>> implementing the first half of TUF. The Git tooling currently assumes
>>> GnuPG keys for signatures, so I'm proposing that we use GnuPG keys for
>>> signing, and that we manage key revocation and any trust delegation
>>> between keys using GnuPG and its existing infrasture.
>>>
>>> I estimate the total effort necessary here to be the equivalent of 5-6
>>> full time days overall. However, I have not pooled the necessary
>>> resources to carry that out yet. I'd like to get feedback first before
>>> going ahead with this, but in meantime,
>>>
>>> ** if there are any volunteers that would like to signal their intent
>>> to help with the implementation effort then please add your name at
>>> the bottom of the wiki page. **
>>>
>>> Best,
>>>
>>> Mathieu
>>>
>>> On 18 April 2015 at 20:11, Michael Snoyman <mic... at snoyman.com> wrote:
>>> >
>>> >
>>> > On Sat, Apr 18, 2015 at 12:20 AM Bardur Arantsson
>>> > <sp... at scientician.net>
>>> > wrote:
>>> >>
>>> >> On 17-04-2015 10:17, Michael Snoyman wrote:
>>> >> > This is a great idea, thank you both for raising it. I was discussing
>>> >> > something similar with others in a text chat earlier this morning.
>>> >> > I've
>>> >> > gone ahead and put together a page to cover this discussion:
>>> >> >
>>> >> >
>>> >> >
>>> >> > https://github.com/commercialhaskell/commercialhaskell/blob/master/proposal/improved-hackage-security.md
>>> >> >
>>> >> > The document definitely needs more work, this is just meant to get
>>> >> > the
>>> >> > ball
>>> >> > rolling. As usual with the commercialhaskell repo, if anyone wants
>>> >> > edit
>>> >> > access, just request it on the issue tracker. Or most likely, send a
>>> >> > PR
>>> >> > and
>>> >> > you'll get a commit bit almost magically ;)
>>> >>
>>> >> Thank you. Just to make sure that I understand -- is this page only
>>> >> meant to cover the original "strawman proposal" at the start of this
>>> >> thread, or...?
>>> >>
>>> >> Maybe you intend for this to be extended in a detailed way under the
>>> >> "Long-term solutions" heading?
>>> >>
>>> >> I was imagining a wiki page which could perhaps start out by collecting
>>> >> all the currently identified possible threats in a table, and then all
>>> >> "participants" could perhaps fill in how their suggestion addresses
>>> >> those threats (or tell us why we shouldn't care about this particular
>>> >> threat). Of course other relevent non-threat considerations might be
>>> >> relevant to add to such a table, such as: how prevalent is the
>>> >> software/idea we're basing this on? does this have any prior
>>> >> implementation (e.g. the append-to-tar and expect that web servers will
>>> >> behave sanely thing)? etc.
>>> >>
>>> >> (I realize that I'm asking for a lot of work, but I think it's going to
>>> >> be necessary, at least if there's going to be consensus and not just a
>>> >> de-facto "winner".)
>>> >>
>>> >>
>>> >
>>> > Hi Bardur,
>>> >
>>> >
>>> > I don't think I have any different intention for this page than you've
>>> > identified. In fact, I thought that I had clearly said exactly what you
>>> > described when I said:
>>> >
>>> >> There are various ideas at play already. The bullets are not intended
>>> >> to
>>> >> be full representations of the proposals, but rather high level
>>> >> summaries.
>>> >> We should continue to expand this page with more details going forward.
>>> >
>>> > If this is unclear somehow, please tell me. But my intention absolutely
>>> > is
>>> > that many people can edit this page to add their ideas and we can flesh
>>> > out
>>> > a complete solution.
>>> >
>>> > Michael
>>> >
>>> > _______________________________________________
>>> > Haskell-Cafe mailing list
>>> > Haskel... at haskell.org
>>> > http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
>>> >
>>> _______________________________________________
>>> Haskell-Cafe mailing list
>>> Haskel... at haskell.org
>>> http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe