[Hackage] #338: use of HTTP needs overhauling

Hackage trac at galois.com
Fri Aug 22 19:27:18 EDT 2008

#338: use of HTTP needs overhauling
  Reporter:  duncan              |        Owner:         
      Type:  enhancement         |       Status:  new    
  Priority:  normal              |    Milestone:         
 Component:  cabal-install tool  |      Version:
  Severity:  normal              |     Keywords:         
Difficulty:  normal              |   Ghcversion:  6.8.2  
  Platform:                      |  
 We're not using HTTP the protocol or HTTP the package effectively

  * As I understand it, the HTTP lib buffers the entire http download in
 memory rather than returning it lazily, so we cannot stream it to disk.
 It's also all done using `String`.
  * We do not check (and nor does the HTTP lib) for truncated downloads.
 That is where the `Content-Length` header differs from the actual length.
  * We do not cache the ETag so we cannot optimise the case of re-
 downloading an unchanged index.
  * We make a new tcp connection for each download, rather than doing all
 downloads within a single browse session.
  * We are not doing downloads concurrent with package builds.

 We should probably use a dedicated thread for downloads. It should
 serialise downloads and use a single browse session. Even without support
 for streaming the body directly we could at least write files atomically,
 checking length header. And also cache the ETag.

Ticket URL: <http://hackage.haskell.org/trac/hackage/ticket/338>
Hackage <http://haskell.org/cabal/>
Hackage: Cabal and related projects

More information about the cabal-devel mailing list