[Haskell-cafe] Contributing to http-conduit

[Michael Snoyman]

Hi Myles,

These sound like two solid features, and I'd be happy to merge in code to
support it. Some comments below.

On Sat, Jan 21, 2012 at 8:38 AM, Myles C. Maxfield <myles.maxfield at gmail.com
> wrote:

> To: Michael Snoyman, author and maintainer of http-conduit
> CC: haskell-cafe
>
> Hello!
>
> I am interested in contributing to the http-conduit library. I've been
> using it for a little while and reading through its source, but have felt
> that it could be improved with two features:
>
>    - Allowing the caller to know the final URL that ultimately resulted
>    in the HTTP Source. Because httpRaw is not exported, the caller can't even
>    re-implement the redirect-following code themselves. Ideally, the caller
>    would be able to know not only the final URL, but also the entire chain of
>    URLs that led to the final request. I was thinking that it would be even
>    cooler if the caller could be notified of these redirects as they happen in
>    another thread. There are a couple ways to implement this that I have been
>    thinking about:
>       - A straightforward way would be to add a [W.Ascii] to the type of
>       Response, and getResponse can fill in this extra field. getResponse already
>       knows about the Request so it can tell if the response should be gunzipped.
>
> What would be in the [W.Ascii], a list of all paths redirected to? Also,
I'm not sure what gunzipping has to do with here, can you clarify?


>
>    - It would be nice for the caller to be able to know in real time what
>       URLs the request is being redirected to. A possible way to do this would be
>       for the 'http' function to take an extra argument of type (Maybe
>       (Control.Concurrent.Chan W.Ascii)) which httpRaw can push URLs into. If the
>       caller doesn't want to use this variable, they can simply pass Nothing.
>       Otherwise, the caller can create an IO thread which reads the Chan until
>       some termination condition is met (Perhaps this will change the type of the
>       extra argument to (Maybe (Chan (Maybe W.Ascii)))). I like this solution,
>       though I can see how it could be considered too heavyweight.
>
>
I do think it's too heavyweight. I think if people really want lower-level
control of the redirects, they should turn off automatic redirect and allow
3xx responses.

>
>    - Making the redirection aware of cookies. There are redirects around
>    the web where the first URL returns a Set-Cookie header and a 3xx code
>    which redirects to another site that expects the cookie that the first HTTP
>    transaction set. I propose to add an (IORef to a Data.Set of Cookies) to
>    the Manager datatype, letting the Manager act as a cookie store as well as
>    a repository of available TCP connections. httpRaw could deal with the
>    cookie store. Network.HTTP.Types does not declare a Cookie datatype, so I
>    would probably be adding one. I would probably take it directly from
>    Network.HTTP.Cookie.
>
> Actually, we already have the cookie package for this. I'm not sure if
putting the cookie store in the manager is necessarily the right approach,
since I can imagine wanting to have separate sessions while reusing the
same connections. A different approach could be adding a list of Cookies to
both the Request and Response.

>
>
> I'd be happy to do both of these things, but I'm hoping for your input on
> how to go about this endeavor. Are these features even good to be pursuing?
> Should I be going about this entirely differently?
>
> Thanks,
> Myles C. Maxfield
>
> P.S. I'm curious about the lack of Network.URI throughout
> Network.HTTP.Conduit. Is there a particular design decision that led you to
> use raw ascii strings?
>

Because there are plenty of URIs that are valid that we don't handle at
all, e.g., ftp.

Michael
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.haskell.org/pipermail/haskell-cafe/attachments/20120121/751ea8f8/attachment.htm>