[Hackage] #288: the package indexes are very slow

[Hackage]

#288: the package indexes are very slow
---------------------------------+------------------------------------------
  Reporter:  duncan              |        Owner:         
      Type:  defect              |       Status:  new    
  Priority:  normal              |    Milestone:         
 Component:  cabal-install tool  |      Version:  1.2.3.0
  Severity:  normal              |     Keywords:         
Difficulty:  hard (< 1 day)      |   Ghcversion:  6.8.2  
  Platform:                      |  
---------------------------------+------------------------------------------
 In a large run, eg trying to make a plan to install 560 packages from
 hackage:

 {{{
 $ cabal install --dry-run $(cat pkgs)
 }}}

 it turns out (according to the ghc profile), 91% of the time is spent
 reading the index of installed and available packages.

 The ghc package index is a couple of massive text files in `Read/Show`
 format so that takes for ever to read. The available package index is the
 tarball of .cabal files and our .cabal file parser is really slow.

 For smaller runs it's not so bad:

 {{{
 $ cabal install --dry-run xmonad
 }}}

 since we only have to inspect the subset of the available package index
 that make up xmonad's transitive deps (any versions thereof), so that
 allows us to avoid forcing most of the index.

 http://hackage.haskell.org/trac/ghc/ticket/2089

 might help us, but then again maybe not if we still have to parse the
 result of calling ghc-pkg since that will give us another text format.

 For our own package index, perhaps we should be generating a cache in some
 other format when we download the package index.

-- 
Ticket URL: <http://hackage.haskell.org/trac/hackage/ticket/288>
Hackage <http://haskell.org/cabal/>
Hackage: Cabal and related projects