<div dir="ltr">Minor update. Some of your points about checking signatures before unpacking made me curious about what Git had to offer in these circumstances. For those like me who were unaware of the functionality, it turns out that Git has the option to reject non-signed commits, just run:<div><br></div><div>git pull --verify-signatures</div><div><br></div><div>I've set up the Travis job that pulls from Hackage to sign its commits with the GPG key I've attached to this email (fingerprint E595 AD42 14AF A6BB 1552 0B23 E40D 74D6 D6CF 60FD).</div><div dir="ltr"><div><br><div class="gmail_quote">On Thu, Apr 16, 2015 at 2:58 PM Duncan Coutts <<a href="mailto:duncan@well-typed.com" target="_blank">duncan@well-typed.com</a>> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">On Thu, 2015-04-16 at 11:18 +0000, Michael Snoyman wrote:<br>
> On Thu, Apr 16, 2015 at 1:57 PM Duncan Coutts <<a href="mailto:duncan@well-typed.com" target="_blank">duncan@well-typed.com</a>> wrote:<br>
<br>
> > I was not proposing to change the repository format significantly (and<br>
> > only in a backwards compatible way). The existing format is pretty<br>
> > simple, using standard old well understood formats and protocols with<br>
> > wide tool support.<br>
> ><br>
> > The incremental update is fairly unobtrusive. Passive http servers don't<br>
> > need to know about it, and clients that don't know about it can just<br>
> > download the whole index as they do now.<br>
> ><br>
> > The security extensions for TUF are also compatible with the existing<br>
> > format and clients.<br>
> ><br>
> The theme you seem to be creating here is "compatible with current format."<br>
> You didn't say it directly, but you've strongly implied that, somehow, Git<br>
> isn't compatible with existing tooling. Let me make clear that that is, in<br>
> fact, false[1]:<br>
<br>
Sure, one can use git or rsync or other methods to transfer the set of<br>
files that makes up a repository or repository index. The point is,<br>
existing clients expect both this format and this (http) protocol.<br>
<br>
There's a number of other minor arguments to be made here about what's<br>
simpler and more backwards compatible, but here are two more significant<br>
and positive arguments:<br>
<br>
1. This incremental update approach works well with the TUF<br>
security design<br>
2. This approach to transferring the repository index and files has<br>
a much lower security attack surface<br>
<br>
For 1, the basic TUF approach is based on a simple http server serving a<br>
set of files. Because we are implementing TUF for Hackage we picked this<br>
update method to go with it. It's really not exotic, the HTTP spec says<br>
about byte range requests: "Range supports efficient recovery from<br>
partially failed transfers, and supports efficient partial retrieval of<br>
large entities." We're doing an efficient partial retrieval of a large<br>
entity.<br>
<br>
For 2, Mathieu elsewhere in this thread pointed to an academic paper<br>
about attacks on package repositories and update systems. A surprising<br>
number of these are attacks on the download mechanism itself, before you<br>
even get to trying to verify individual package signatures. If you read<br>
the TUF papers you see that they also list these attacks and address<br>
them in various ways. One of them is that the download mechanism needs<br>
to know in advance the size (and content hash) of entities it is going<br>
to download. Also, we should strive to minimise the amount of complex<br>
unaudited code that has to run before we get to checking the signature<br>
of the package index (or individual package tarballs). In the TUF<br>
design, the only code that runs before verification is downloading two<br>
files over HTTP (one that's known to be very small, and the other we<br>
already know the length and signed content hash). If we're being<br>
paranoid we shouldn't even run any decompression before signature<br>
verification. With our implementation the C code that runs before<br>
signature verification is either none, or just zlib decompression if we<br>
want to do on-the-fly http transport compression, but that's optional if<br>
we don't want to trust zlib's security record (though it's extremely<br>
widely used). By contrast, if we use rsync or git then there's a massive<br>
amount of unaudited C code that is running with your user credentials<br>
prior to signature verification. In addition it is likely vulnerable to<br>
endless data and slow download attacks (see the papers).<br>
<br>
--<br>
Duncan Coutts, Haskell Consultant<br>
Well-Typed LLP, <a href="http://www.well-typed.com/" target="_blank">http://www.well-typed.com/</a><br>
<br>
--<br>
You received this message because you are subscribed to the Google Groups "Commercial Haskell" group.<br>
To unsubscribe from this group and stop receiving emails from it, send an email to <a href="mailto:commercialhaskell%2Bunsubscribe@googlegroups.com" target="_blank">commercialhaskell+unsubscribe@googlegroups.com</a>.<br>
To post to this group, send email to <a href="mailto:commercialhaskell@googlegroups.com" target="_blank">commercialhaskell@googlegroups.com</a>.<br>
To view this discussion on the web visit <a href="https://groups.google.com/d/msgid/commercialhaskell/1429185521.25663.103.camel%40dunky.localdomain" target="_blank">https://groups.google.com/d/msgid/commercialhaskell/1429185521.25663.103.camel%40dunky.localdomain</a>.<br>
For more options, visit <a href="https://groups.google.com/d/optout" target="_blank">https://groups.google.com/d/optout</a>.<br>
</blockquote></div></div></div></div>