[Haskell-cafe] Improvements to package hosting and security

[Michael Snoyman]

Thanks for responding, I intend to go read up on TUF and your blog post
now. One question:

      * We're incorporating an existing design for incremental updates
        of the package index to significantly improve "cabal update"
        times.

Can you give any details about what you're planning here? I put together a
Git repo already that has all of the cabal files from Hackage and which
updates every 30 minutes, and it seems that, instead of reinventing
anything, simply using `git pull` would be the right solution here:

https://github.com/commercialhaskell/all-cabal-files

On Thu, Apr 16, 2015 at 12:34 PM Duncan Coutts <duncan at well-typed.com>
wrote:

> Hi folks,
>
> As I mentioned previously on the commercialhaskell list, we're working
> on Hackage security for the IHG at the moment.
>
> We've finally written up the design for that as a blog post:
>
> http://www.well-typed.com/blog/2015/04/improving-hackage-security
>
> It includes a section at the end comparing in general terms to this
> proposal (specifically Chris's part on package signing).
>
> The design is basically "The Update Framework" for Hackage. Our current
> implementation effort for the IHG covers the first part of that design.
>
> http://theupdateframework.com/
>
> I think TUF addresses many of the concerns that have been raised in this
> thread, e.g. about threat models, what signatures actually mean etc.
>
> It also covers the question of making the "who's allowed to upload what"
> information transparent, with proper cryptographic evidence (albeit
> that's in the second part of the design).
>
> So if collectively we can also implement the second part of TUF for
> Hackage then I think we can address these issues properly.
>
> Other things worth noting:
>       * This will finally allow us to have untrusted public mirrors,
>         which is the traditional approach to improving repository
>         reliability.
>       * We're incorporating an existing design for incremental updates
>         of the package index to significantly improve "cabal update"
>         times.
>
> I'll chip in elsewhere in this thread with more details about how TUF
> (or our adaptation of it for hackage) solves some of the problems raised
> here.
>
> Duncan
>
> On Mon, 2015-04-13 at 10:02 +0000, Michael Snoyman wrote:
> > Many of you saw the blog post Mathieu wrote[1] about having more
> composable
> > community infrastructure, which in particular focused on improvements to
> > Hackage. I've been discussing some of these ideas with both Mathieu and
> > others in the community working on some similar thoughts. I've also
> > separately spent some time speaking with Chris about package signing[2].
> > Through those discussions, it's become apparent to me that there are in
> > fact two core pieces of functionality we're relying on Hackage for today:
> >
> > * A centralized location for accessing package metadata (i.e., the cabal
> > files) and the package contents themselves (i.e., the sdist tarballs)
> > * A central authority for deciding who is allowed to make releases of
> > packages, and make revisions to cabal files
> >
> > In my opinion, fixing the first problem is in fact very straightforward
> to
> > do today using existing tools. FP Complete already hosts a full Hackage
> > mirror[3] backed by S3, for instance, and having the metadata mirrored
> to a
> > Git repository as well is not a difficult technical challenge. This is
> the
> > core of what Mathieu was proposing as far as composable infrastructure,
> > corresponding to next actions 1 and 3 at the end of his blog post (step
> 2,
> > modifying Hackage, is not a prerequesite). In my opinion, such a system
> > would far surpass in usability, reliability, and extensibility our
> current
> > infrastructure, and could be rolled out in a few days at most.
> >
> > However, that second point- the central authority- is the more
> interesting
> > one. As it stands, our entire package ecosystem is placing a huge level
> of
> > trust in Hackage, without any serious way to vet what's going on there.
> > Attack vectors abound, e.g.:
> >
> > * Man in the middle attacks: as we are all painfully aware, cabal-install
> > does not support HTTPS, so a MITM attack on downloads from Hackage is
> > trivial
> > * A breach of the Hackage Server codebase would allow anyone to upload
> > nefarious code[4]
> > * Any kind of system level vulnerability could allow an attacker to
> > compromise the server in the same way
> >
> > Chris's package signing work addresses most of these vulnerabilities, by
> > adding a layer of cryptographic signatures on top of Hackage as the
> central
> > authority. I'd like to propose taking this a step further: removing
> Hackage
> > as the central authority, and instead relying entirely on cryptographic
> > signatures to release new packages.
> >
> > I wrote up a strawman proposal last week[5] which clearly needs work to
> be
> > a realistic option. My question is: are people interested in moving
> forward
> > on this? If there's no interest, and everyone is satisfied with
> continuing
> > with the current Hackage-central-authority, then we can proceed with
> having
> > reliable and secure services built around Hackage. But if others- like
> me-
> > would like to see a more secure system built from the ground up, please
> say
> > so and let's continue that conversation.
> >
> > [1]
> >
> https://www.fpcomplete.com/blog/2015/03/composable-community-infrastructure
> >
> > [2]
> >
> https://github.com/commercialhaskell/commercialhaskell/wiki/Package-signing-detailed-propsal
> >
> > [3] https://www.fpcomplete.com/blog/2015/03/hackage-mirror
> > [4] I don't think this is just a theoretical possibility for some point
> in
> > the future. I have reported an easily trigerrable DoS attack on the
> current
> > Hackage Server codebase, which has been unresolved for 1.5 months now
> > [5] https://gist.github.com/snoyberg/732aa47a5dd3864051b9
> >
>
>
> --
> Duncan Coutts, Haskell Consultant
> Well-Typed LLP, http://www.well-typed.com/
>
> --
> You received this message because you are subscribed to the Google Groups
> "Commercial Haskell" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to commercialhaskell+unsubscribe at googlegroups.com.
> To post to this group, send email to commercialhaskell at googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/commercialhaskell/1429176843.25663.31.camel%40dunky.localdomain
> .
> For more options, visit https://groups.google.com/d/optout.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/haskell-cafe/attachments/20150416/c96d5cf2/attachment-0001.html>