[Haskell-cafe] Improvements to package hosting and security

Thu Apr 16 15:28:10 UTC 2015

Minor update. Some of your points about checking signatures before
unpacking made me curious about what Git had to offer in these
circumstances. For those like me who were unaware of the functionality, it
turns out that Git has the option to reject non-signed commits, just run:

git pull --verify-signatures

I've set up the Travis job that pulls from Hackage to sign its commits with
the GPG key I've attached to this email (fingerprint E595 AD42 14AF A6BB
1552  0B23 E40D 74D6 D6CF 60FD).

On Thu, Apr 16, 2015 at 2:58 PM Duncan Coutts <duncan at well-typed.com> wrote:

> On Thu, 2015-04-16 at 11:18 +0000, Michael Snoyman wrote:
> > On Thu, Apr 16, 2015 at 1:57 PM Duncan Coutts <duncan at well-typed.com>
> wrote:
>
> > > I was not proposing to change the repository format significantly (and
> > > only in a backwards compatible way). The existing format is pretty
> > > simple, using standard old well understood formats and protocols with
> > > wide tool support.
> > >
> > > The incremental update is fairly unobtrusive. Passive http servers
> don't
> > > need to know about it, and clients that don't know about it can just
> > > download the whole index as they do now.
> > >
> > > The security extensions for TUF are also compatible with the existing
> > > format and clients.
> > >
> > The theme you seem to be creating here is "compatible with current
> format."
> > You didn't say it directly, but you've strongly implied that, somehow,
> Git
> > isn't compatible with existing tooling. Let me make clear that that is,
> in
> > fact, false[1]:
>
> Sure, one can use git or rsync or other methods to transfer the set of
> files that makes up a repository or repository index. The point is,
> existing clients expect both this format and this (http) protocol.
>
> There's a number of other minor arguments to be made here about what's
> simpler and more backwards compatible, but here are two more significant
> and positive arguments:
>
>      1. This incremental update approach works well with the TUF
>         security design
>      2. This approach to transferring the repository index and files has
>         a much lower security attack surface
>
> For 1, the basic TUF approach is based on a simple http server serving a
> set of files. Because we are implementing TUF for Hackage we picked this
> update method to go with it. It's really not exotic, the HTTP spec says
> about byte range requests: "Range supports efficient recovery from
> partially failed transfers, and supports efficient partial retrieval of
> large entities." We're doing an efficient partial retrieval of a large
> entity.
>
> For 2, Mathieu elsewhere in this thread pointed to an academic paper
> about attacks on package repositories and update systems. A surprising
> number of these are attacks on the download mechanism itself, before you
> even get to trying to verify individual package signatures. If you read
> the TUF papers you see that they also list these attacks and address
> them in various ways. One of them is that the download mechanism needs
> to know in advance the size (and content hash) of entities it is going
> to download. Also, we should strive to minimise the amount of complex
> unaudited code that has to run before we get to checking the signature
> of the package index (or individual package tarballs). In the TUF
> design, the only code that runs before verification is downloading two
> files over HTTP (one that's known to be very small, and the other we
> already know the length and signed content hash). If we're being
> paranoid we shouldn't even run any decompression before signature
> verification. With our implementation the C code that runs before
> signature verification is either none, or just zlib decompression if we
> want to do on-the-fly http transport compression, but that's optional if
> we don't want to trust zlib's security record (though it's extremely
> widely used). By contrast, if we use rsync or git then there's a massive
> amount of unaudited C code that is running with your user credentials
> prior to signature verification. In addition it is likely vulnerable to
> endless data and slow download attacks (see the papers).
>
> --
> Duncan Coutts, Haskell Consultant
> Well-Typed LLP, http://www.well-typed.com/
>
> --
> You received this message because you are subscribed to the Google Groups
> "Commercial Haskell" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to commercialhaskell+unsubscribe at googlegroups.com.
> To post to this group, send email to commercialhaskell at googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/commercialhaskell/1429185521.25663.103.camel%40dunky.localdomain
> .
> For more options, visit https://groups.google.com/d/optout.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/haskell-cafe/attachments/20150416/272bb538/attachment.html>
-------------- next part --------------
-----BEGIN PGP PUBLIC KEY BLOCK-----
Version: GnuPG v1

mQENBFUvs4EBCADOl99iNq7EijkoKfsPjOC5Dq2tLWe5jdWh/slJPhEK1oadenDB
mKDU1BO8ysbQxJwxkCRw5/iEPIwAFBjjVnKTRuH6jf9LjPIsfunCQjIARTTFGIX4
RyhchbIRv4qCCyceDfKxLtuy9NU7GYnJSijITX39X2al6BhAMcf7fNW1ztsQMQbC
O4EoT5isnHS+wQFs8APuAt5xatRAnkF8fMCdDPDKAUm4oCnojBXLAr2Z03B6sL5B
XFygA71CMqOrkQIecLj86YKM4DebluDG24NjQv3NKdX2b/mTSzr3v0nCwaDR07Kj
GNmRRNi0W6HJqbHjAyTcwpUHIuvPUpJqryDlABEBAAG0n0NvbW1lcmNpYWwgSGFz
a2VsbCBhbGwtY2FiYWwtZmlsZXMgVHJhdmlzIGpvYiAoVXNlZCBleGNsdXNpdmVs
eSBvbjogaHR0cHM6Ly9naXRodWIuY29tL2NvbW1lcmNpYWxoYXNrZWxsL2FsbC1j
YWJhbC1maWxlcykgPG1pY2hhZWwrYWxsLWNhYmFsLWZpbGVzQHNub3ltYW4uY29t
PokBOAQTAQIAIgUCVS+zgQIbAwYLCQgHAwIGFQgCCQoLBBYCAwECHgECF4AACgkQ
5A101tbPYP1kNgf+O/cVwcsE7sgpAuqtg/MItwiN+uz5wjizfWP+oEFaVrjL0r+H
2Zh4Hrj/ztPww1LPp/v0nYhkPqTl36dQKpniUHwLt2fNtjBpmF0ccCcHJI7ZcZVk
D28vPnPRPVHl2gj67dP/jcR5tVmXtYHFzwc4Y5hwdbvov+UXbG1rG0Kh40v5Wtmg
nqSWVH2QMpYD6KDlxkDVOKhbFOeWP2OEsjM57LWJvQ467CCI9dW3Hx0enXAlazeS
ocwbbxfFzDJKbZJ5ZwNVBcoNFOBtrksptJ2GkXuq7WHLQ63akRxwEx9Q85yVv5it
TNOUd+wgykyxLdKydefpfpzX//vFqbAccixAZYkBHAQQAQIABgUCVS+60AAKCRCg
SOjAV+hodicLB/9mwfV4QI2lOjU+eTONwnKIePSNcYxD4rZmdzRXx3eiN57LtqKd
xm7f49IyByIErzJaUXwZdag+InEWJmKgY2a0p1y+bLmnMzbM0GsFwzEFzC3ysQ5u
Os8J/mT0jFb1W69panXSlPs5e9MeOQ5fRL82JEn5ymNa81+5mmDqOxu2CffK5rqr
eFln+GjmXCrXCFA7SrKSNr8RH+luumhE5Lk3fOVrUOp6aY5l7nMEhLmUnOLH7jZK
HgBxxbxm0gi5dlH+CeDK6PChORsZm5aytub87xFUqyXHOOihPQTTBDZVVpBbkyJ+
K1YJOIX5hKoM2vti0BJmeWh15KDD6PyasHstuQENBFUvs4EBCAC3Tz9BhlTWD8ge
N3UDDG1mYqoXF+u5AvtVIKG20nsCCTjM3PX/I4JjkFJyV6oqGo5J1CwlnMO9kZ/r
MT3aSUipJdx5vGcOophLB/+1HPzCd52mCOW+kpaD/GSKqKRgcxM4tjz1bBPXpTbb
LjC1KJEAn/q5Rcv0SmYpc9n6fAsoyas1f4vTOhNL8NXzjhsJBoPImb58/ce0o12D
zHhRklueydQauwY/wlglN2XaC6B85rv8JteFYwF6SNmGG0ghRBAFG8ymKdh56xpy
0I+9kZ4DAnLJbFNzNfTSZxwq1+8jU1jApe+XdLEXr/lcrM8LJ1tqKej+ZdYj4O/p
MC16KWkLABEBAAGJAR8EGAECAAkFAlUvs4ECGwwACgkQ5A101tbPYP07GwgAo9xg
VZeNt1ft80c8KfW0+7Xvs0LtkRSVEJFgcNb2MWettFf/JEFr8CRM/Y6eeeaD/UwQ
zYNqB3EPI/sCMZ1rrrfF8zYTxN6MuOamoL0L9fmd7/m0xS4VCDmj1X7mm6BaLtT6
ecLqVvjQ/ni8YiBPX963owdDHvuk7n+UVpR8W7mBS5O8sQ7X9DEy8FVDVdy7SLjs
3CpV93eQh9L9spxcB/ODx7FauUKuid5CboFyJVonlcFiL4MCTu/EA4Qk62E4yV6U
dpq1y+1lBUcv/2GSIrTwA8sFge8AT5UIoCgIDF9yO3mldYsU4mzsiM9ulWvSLJBM
2VhVqxW9Mh1gYUkz5g==
=O7Sx
-----END PGP PUBLIC KEY BLOCK-----