Hackage2 server ready for testing

[Matthew Gruen]

Reverse dependencies can mean different things. The current implementation
answers these questions:
1. What are the packages for which the latest version depends directly on a
given version of foo?
2. What are the packages for which only previous or deprecated versions
depend directly on a given version of foo?
3. For which packages do any versions depend either directly and indirectly
on any version of bar? (this is what takes 2 seconds to regenerate, as
incremental transitive closure wasn't implemented)
4. Which packages are most depended-upon (only direct, or both direct and
indirect)?

It seems to me as though 1 is the most important question to answer. This
involves knowing, for any package and version, the list of package-version
pairs which depend on it. This involves using a fair bit of memory, it
seems. Additionally, when a package is uploaded, removed, or changed, the
packages which now depend on it also need to be updated.

Another question answered by packdeps is:
5. Which packages depend on a previous or deprecated version of foo but not
the latest version? (and vice versa)

This can also be answered by the above data structure. It is currently
implemented using nested Data.Maps, which is serialized in the database.
There are some opportunities for trimming and optimization.

Best,
Matt
On Fri, Aug 24, 2012 at 3:36 PM, Erik Hesselink <hesselink at gmail.com> wrote:

> Hi Ian,
>
> Looks great, thanks for the work! One question: I see the reverse
> dependencies have been disabled, darcs says for 'performance
> problems'. Do you have some details? Do you think it is hard to fix?
>
> I wouldn't mind doing some permission granting for the testing period,
> by the way.
>
> Erik
>
> On Fri, Aug 24, 2012 at 5:17 PM, Ian Lynagh <ian at well-typed.com> wrote:
> >
> > Hi all,
> >
> > All the bugs identified in the Hackage 2 server as blockers have now
> > been fixed, and I have a mirror of Hackage running here:
> >     http://new-hackage.haskell.org/
> >
> > I suggest that we proceed with:
> >
> > * A 3 week testing period, to be announced on haskell@ and libraries@
> > * Assuming no new blocking bugs are found which justify additional
> >   testing time, disable uploads to the Hackage 1 server
> > * Throw the testing Hackage 2 instance away, and make a new mirror from
> >   the Hackage 1 server
> > * Point hackage.haskell.org at the new Hackage 2 instance
> >
> >
> > I think that a 3 week testing period will be sufficient as I expect most
> > testing will happen in the first week or two, or the last few days, so
> > any extra time is essentially wasted. I was actually initially going to
> > propose 2 weeks, but given the timing of ICFP et al I think an extra
> > week is justified.
> >
> > The reason I propose throwing away this test instance is that I think
> > it's important that people are able to test uploading, and therefore
> > uploading dummy packages and dummy versions should be encouraged.
> >
> >
> > Does that plan sound reasonable to everyone?
> >
> > Is anyone willing to volunteer to be an admin for the new Hackage server
> > please? (people need to have an admin add their account to the uploaders
> > group before they can upload).
> >
> >
> > A couple more notes:
> >
> > The mirror is a week or two old, so the latest package uploads won't be
> > on it. I don't plan to set up continuous mirroring, as I think it may be
> > more useful for people to be able to upload new versions to both servers
> > themselves, for testing.
> >
> > The documentation builder has built (or failed to build) docs for the
> > latest versions of all packages, but it's still working through older
> > versions. It will therefore take up to 35mins before it notices that a
> > new package has been uploaded, and builds docs for it.
> >
> >
> > Finally, I would like to thank the Industrial Haskell Group
> > (http://industry.haskell.org/) for funding this work.
> >
> >
> > Thanks
> > Ian
> > --
> > Ian Lynagh, Haskell Consultant
> > Well-Typed LLP, http://www.well-typed.com/
> >
> >
> > _______________________________________________
> > cabal-devel mailing list
> > cabal-devel at haskell.org
> > http://www.haskell.org/mailman/listinfo/cabal-devel
>
> _______________________________________________
> cabal-devel mailing list
> cabal-devel at haskell.org
> http://www.haskell.org/mailman/listinfo/cabal-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.haskell.org/pipermail/cabal-devel/attachments/20120831/9968621d/attachment.htm>