[web-devel] [Haskell-cafe] http-enumerator: redirects, streaming and keep-alive

[Michael Snoyman]

On Thu, Feb 3, 2011 at 12:00 AM, Antoine Latter <aslatter at gmail.com> wrote:
> On Wed, Feb 2, 2011 at 3:01 PM, Felipe Almeida Lessa
> <felipe.lessa at gmail.com> wrote:
>> On Wed, Feb 2, 2011 at 6:30 PM, Antoine Latter <aslatter at gmail.com> wrote:
>>> Or you could remove the socket from the map while it's in use.
>>
>> And what about connection limits?  We shouldn't create a thousand
>> connections to the same host =).
>>
>
> Not a bad idea, but that may be creeping outside the scope of the bug
> as reported.
>
> If you're writing a web-scraper/spider, it is true you might need some
> sort of higher-level manager to handle concurrent access. I'm not sure
> what that would look like, though
>
>> --
>> Felipe.
>>
>

I think Felipe's point is that using the approach I outlined, we will
never spawn more than one connection to a single host. By simply
taking the socket out of the map, there's no limit to the number of
sockets that will be created.

Michael