[Haskell-cafe] Ongoing IHG work to improve Hackage security

[Duncan Coutts]

On Thu, 2015-04-16 at 16:08 +0200, Herbert Valerio Riedel wrote:
> On 2015-04-16 at 15:14:47 +0200, Mikhail Glushenkov wrote:
> 
> [...]
> 
> > Thank you, this is very exciting. But won't the post-release .cabal
> > update feature interfere with "package index as an append-only log"
> > concept? IIUC, right now it is implemented as a destructive update of
> > the corresponding package index entry, so making the package index
> > immutable will break backwards compatibility.
> 
> Being historically a tape-originating archive format, `tar` does support
> such destructive updates as a standard-feature by appended content. See
> also the '--update' operation in the tar(1) man-page:
> 
> | -u, --update
> |        Append files which are newer than the corresponding copy in
> |        the archive.  Arguments have the same meaning as with -c and
> |        -r options.
> 
> Using this, we can actually have *all* historic .cabal revisions in the
> 00-index.tar file as hidden entries. This would in theory give
> `cabal-install` access to all cabal revisions (if we'd ever want to have
> a feature in cabal that requires access to previous cabal revisions)

Yes, we can choose to do that. That's sort-of what I was getting at when
I mentioned that you could then view the tar file as a transaction log.

> I actually find Duncan's approach rather elegant, as it simply builds on
> existing standard features of both .tar files and the HTTP protocol
> (resuming HTTP downloads). And they don't even require any custom HTTP
> server implementation to work, and there's a graceful fallback. And we
> don't need to introduce a completely new protocol/tool into the mix
> (such as Git, which has dump-http(s), smart-http(s), git, as
> transport-layers to chose from, and has various other moving parts such
> as a ~/.gitconfig that can affect how Git operates, like redirecting
> clone-urls transparently, also it avoids having to handle subtle
> platform-specific issues with shelling out to an external command; from
> this POV the HTTP/tar approach is the more KISS-compliant one, IMHO)
> 
> Cheers,
>   hvr
>