[Haskell-cafe] Improvements to package hosting and security

[Mathieu Boespflug]

Hi all,

last week, I found some time to write up a very simple proposal that
addresses the following goals simultaneously:

- maintain a difficult to forge public audit log of Hackage updates;
- make downloads from Hackage mirrors just as trustworthy as
downloading from Hackage itself;
- guarantee that `cabal update` is always pulling the freshest package
index (called "snapshots" in the proposal), and detect when this might
not be the case;
- implement the first half of TUF (namely the index signing part
discussed in Duncan's blog post, not the author package signing part)
with fewer metadata files and in a way that reuses existing tooling;
-  get low-implementation-cost, straightforward and incremental `cabal update`.

After a preliminary review from a few colleagues and friends in the
community, here is the proposal, in the form of Commercial Haskell
wiki page:

https://github.com/commercialhaskell/commercialhaskell/wiki/Git-backed-Hackage-index-signing-and-distribution

The design constraints here are:

- stay backwards compatible where the cost for doing so is low.
- reuse existing tooling and mechanisms, especially when it comes to
key management, snapshot identity, and distributing signatures.
- Focus on the above 5 goals only, because they happen to all be
solvable by changing a single piece of mechanism. But strive to reuse
whatever mechanism others are proposing to solve other goals (e.g.
certification of provenance using author package signing, as Chris
Done has already proposed).

To that effect, the tl;dr is that I'm proposing that we just use Git
for maintaining the Hackage package index, that we use Git for
synchronizing this locally, and that we use Git commit signatures for
implementing the first half of TUF. The Git tooling currently assumes
GnuPG keys for signatures, so I'm proposing that we use GnuPG keys for
signing, and that we manage key revocation and any trust delegation
between keys using GnuPG and its existing infrasture.

I estimate the total effort necessary here to be the equivalent of 5-6
full time days overall. However, I have not pooled the necessary
resources to carry that out yet. I'd like to get feedback first before
going ahead with this, but in meantime,

** if there are any volunteers that would like to signal their intent
to help with the implementation effort then please add your name at
the bottom of the wiki page. **

Best,

Mathieu

On 18 April 2015 at 20:11, Michael Snoyman <michael at snoyman.com> wrote:
>
>
> On Sat, Apr 18, 2015 at 12:20 AM Bardur Arantsson <spam at scientician.net>
> wrote:
>>
>> On 17-04-2015 10:17, Michael Snoyman wrote:
>> > This is a great idea, thank you both for raising it. I was discussing
>> > something similar with others in a text chat earlier this morning. I've
>> > gone ahead and put together a page to cover this discussion:
>> >
>> >
>> > https://github.com/commercialhaskell/commercialhaskell/blob/master/proposal/improved-hackage-security.md
>> >
>> > The document definitely needs more work, this is just meant to get the
>> > ball
>> > rolling. As usual with the commercialhaskell repo, if anyone wants edit
>> > access, just request it on the issue tracker. Or most likely, send a PR
>> > and
>> > you'll get a commit bit almost magically ;)
>>
>> Thank you. Just to make sure that I understand -- is this page only
>> meant to cover the original "strawman proposal" at the start of this
>> thread, or...?
>>
>> Maybe you intend for this to be extended in a detailed way under the
>> "Long-term solutions" heading?
>>
>> I was imagining a wiki page which could perhaps start out by collecting
>> all the currently identified possible threats in a table, and then all
>> "participants" could perhaps fill in how their suggestion addresses
>> those threats (or tell us why we shouldn't care about this particular
>> threat). Of course other relevent non-threat considerations might be
>> relevant to add to such a table, such as: how prevalent is the
>> software/idea we're basing this on? does this have any prior
>> implementation (e.g. the append-to-tar and expect that web servers will
>> behave sanely thing)? etc.
>>
>> (I realize that I'm asking for a lot of work, but I think it's going to
>> be necessary, at least if there's going to be consensus and not just a
>> de-facto "winner".)
>>
>>
>
> Hi Bardur,
>
>
> I don't think I have any different intention for this page than you've
> identified. In fact, I thought that I had clearly said exactly what you
> described when I said:
>
>> There are various ideas at play already. The bullets are not intended to
>> be full representations of the proposals, but rather high level summaries.
>> We should continue to expand this page with more details going forward.
>
> If this is unclear somehow, please tell me. But my intention absolutely is
> that many people can edit this page to add their ideas and we can flesh out
> a complete solution.
>
> Michael
>
> _______________________________________________
> Haskell-Cafe mailing list
> Haskell-Cafe at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
>