ETag support broken on hackage.haskell.org

[Gintautas Miliauskas]

Hi,

I spent some time looking into this and was able to get etags checking to
work, see [1]

[1] https://github.com/haskell/hackage-server/pull/236

However, my approach only deals with etags only for the index tarball. I
poked around for quite a while seeing if there is a more general solution,
but could not find anything useful. The thing is, support for computing
etags is hand-coded for the index tarball, so a more generic mechanism does
not make much sense. Plus, servePackagesIndex is the only logical place to
handle etags in the request, since further down we do not have access to
the request and further up we do not have access to the tarball object
which embeds the md5sum.


On Tue, Jun 10, 2014 at 11:38 AM, Duncan Coutts <
duncan.coutts at googlemail.com> wrote:

> I'm pretty sure it's the hackage-server. I think we simply don't have
> code yet to react to the if-none-match. Happstack has some util for
> the time-based conditionals, but not the etag one. So it should just
> be a matter of us adding a combinator that both serves the etag and
> reacts to the if-none-match header. Then we'd go and use that in all
> the places where we currently serve etags. At the same time, it's
> probably worth making that caching combinator also take an optional
> timestamp and "valid for" time, so we can do all our caching things in
> one call in each resource handler.
>
> On 10 June 2014 08:54, Johan Tibell <johan.tibell at gmail.com> wrote:
> > We need to figure out if nginx or the hackage-server is to blame here.
> >
> >
> > On Tue, Jun 10, 2014 at 1:34 AM, Gintautas Miliauskas
> > <gintautas.miliauskas at gmail.com> wrote:
> >>
> >> Hi,
> >>
> >> first of all, a shout out to all ZuriHac2014 folks, thanks for the
> gentle
> >> introduction to the deeper trenches of Haskell infrastructure!
> >>
> >> I spent a bit of time looking into why "cabal update" always redownloads
> >> the entire package list even though the code has explicit support for
> etags,
> >> and it looks like etags are broken on the server side. The server does
> serve
> >> "Etag" headers, but does not react to If-None-Match:
> >>
> >> $ curl -I --header 'If-None-Match: "1ea6c9a3168b9c186d5bbafd065ce5ec"'
> >> hackage.haskell.org/packages/index.tar.gz
> >> HTTP/1.1 200 OK
> >> Server: nginx/1.6.0
> >> Date: Mon, 09 Jun 2014 23:22:49 GMT
> >> Content-Type: application/x-gzip
> >> Content-Length: 7193838
> >> Connection: keep-alive
> >> Content-MD5: 1ea6c9a3168b9c186d5bbafd065ce5ec
> >> ETag: "1ea6c9a3168b9c186d5bbafd065ce5ec"
> >> Last-modified: Mon, 09 Jun 2014 23:10:44 GMT
> >>
> >> In this case the server is supposed to return a 304 code because the
> ETag
> >> matches, but it returns the usual 200.
> >>
> >> Here's a well-behaved server for comparison:
> >>
> >> $ curl -I --header 'If-None-Match: f89ef04efbcb1bbd413c9170523beeb1'
> >> http://upload.wikimedia.org/
> >> HTTP/1.1 304 Not Modified
> >> Last-Modified: Thu, 03 Oct 2013 22:29:34 GMT
> >> Etag: f89ef04efbcb1bbd413c9170523beeb1
> >> X-Timestamp: 1380839374.33426
> >> X-Object-Meta-Mtime: 1366745378.0
> >> Content-Type: text/html
> >> X-Varnish: 2552484147 2541352860, 2937266545 2853346814, 523731362
> >> Via: 1.1 varnish, 1.1 varnish, 1.1 varnish
> >> Accept-Ranges: bytes
> >> Date: Mon, 09 Jun 2014 23:29:36 GMT
> >> Age: 38954
> >> Connection: keep-alive
> >> X-Cache: cp1064 hit (336), cp3010 hit (1791), cp3007 frontend miss (0)
> >> Access-Control-Allow-Origin: *
> >> Access-Control-Expose-Headers: Age, Content-Length, Date, X-Cache,
> >> X-Varnish
> >>
> >> Note the 304 return code.
> >>
> >> I do not know about the configuration in use on the server, but
> something
> >> seems to be wrong there. There are some nginx bug reports [1] about
> similar
> >> issues, not sure if they are relevant here.
> >>
> >> [1] https://github.com/rgrove/rawgit/issues/13
> >>
> >> Best regards,
> >> --
> >> Gintautas Miliauskas
> >>
> >> _______________________________________________
> >> cabal-devel mailing list
> >> cabal-devel at haskell.org
> >> http://www.haskell.org/mailman/listinfo/cabal-devel
> >>
> >
> >
> > _______________________________________________
> > cabal-devel mailing list
> > cabal-devel at haskell.org
> > http://www.haskell.org/mailman/listinfo/cabal-devel
> >
>



-- 
Gintautas Miliauskas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.haskell.org/pipermail/cabal-devel/attachments/20140616/96016870/attachment.html>