[Haskell-cafe] is it normal for docs to be pending for 24 hours?

[Ben Gamari]

Patrick Pelletier <code at funwithsoftware.org> writes:

> This is only my second time uploading a package to Hackage, so I don't 
> yet have a feel for how it's supposed to go.  I uploaded 
> normalization-insensitive-2.0.0.1 about 24 hours ago:
>
> https://hackage.haskell.org/package/normalization-insensitive-2.0.0.1
>
> Under "Status", it says "Docs pending".  (And the module names are all 
> non-clickable.)  Is it normal to take this long to build the docs?  Is 
> there some way to find out where in the queue my job is? Is this an 
> indication that something has gone wrong?  How do I fix it?
>
Should it be normal? No. Has it been normal in the past? Sadly yes. In
fact, due to some unfortunately interactions with our CDN it was not
uncommon for builds to silently fail and never be reattempted.

However, we have been slowly plugging away at the builder to try to sort
out the common failure modes, including this one. One of the challenges
has been working out how to retry the builds that have previously failed
(of which there are many) without harming build latencies for new
uploads. The problem is that there are two classes of failed
documentation builds,

 a) Expected, reproducible failures (e.g. the builder environment
    missing native library dependencies).
 b) Various sporatic failures such as the CDN issue noted above

While ideally we would like to rebuild only those packages falling in
class (b), it's not so easy to accomplish this. Consequently I
instead opted to retry all failed builds (over 17000 packages) and
queue the work in such a way to prioritize new uploads. Unfortunately,
my previous attempt at fair scheduling wasn't quite behaving as
expected. However, this morning I took another look at this and things
should now be back to normal. Let me know if you still see issues.

Cheers,

- Ben

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 454 bytes
Desc: not available
URL: <http://mail.haskell.org/pipermail/haskell-cafe/attachments/20161114/574a01dc/attachment.sig>