[Hackage] #338: use of HTTP needs overhauling

[Hackage]

#338: use of HTTP needs overhauling
---------------------------------+------------------------------------------
  Reporter:  duncan              |        Owner:         
      Type:  enhancement         |       Status:  new    
  Priority:  normal              |    Milestone:         
 Component:  cabal-install tool  |      Version:  1.2.3.0
  Severity:  normal              |     Keywords:         
Difficulty:  normal              |   Ghcversion:  6.8.2  
  Platform:                      |  
---------------------------------+------------------------------------------
 We're not using HTTP the protocol or HTTP the package effectively

  * As I understand it, the HTTP lib buffers the entire http download in
 memory rather than returning it lazily, so we cannot stream it to disk.
 It's also all done using `String`.
  * We do not check (and nor does the HTTP lib) for truncated downloads.
 That is where the `Content-Length` header differs from the actual length.
  * We do not cache the ETag so we cannot optimise the case of re-
 downloading an unchanged index.
  * We make a new tcp connection for each download, rather than doing all
 downloads within a single browse session.
  * We are not doing downloads concurrent with package builds.

 We should probably use a dedicated thread for downloads. It should
 serialise downloads and use a single browse session. Even without support
 for streaming the body directly we could at least write files atomically,
 checking length header. And also cache the ETag.

-- 
Ticket URL: <http://hackage.haskell.org/trac/hackage/ticket/338>
Hackage <http://haskell.org/cabal/>
Hackage: Cabal and related projects