[Haskell-beginners] HTTP Download -> Save File - non-strict

[Dan Krol]

Ah yes, the actual docs. Somehow didn't think to check that, sorry.

Alright, I'll try to figure that one out, thanks. Any particular reason
nobody just offers http over lazy I/O? Is it just because lazy I/O is
generally discouraged? Or just particularly bad over a network?

And is this an area where Conduit is better than Pipes? There doesn't seem
to be a similar http for Pipes.


On Sat, Aug 10, 2013 at 11:30 PM, Magnus Therning <magnus at therning.org>wrote:

> On Sat, Aug 10, 2013 at 05:16:58PM -0700, Dan Krol wrote:
> > Hi,
> >
> > I'm working on an rss file getter. I was wondering if I could get
> > some help getting files to download and save without holding the
> > entire file in memory in between. I chose Conduit's version of
> > SimpleHttp only because it was recommended, and it was the quickest
> > thing I could get to work correctly because I was eager to get
> > started on this project, so I'd be happy to switch.
> >
> > Here's where I define the download and save functions:
> >
> > https://github.com/orblivion/feedGetter/blob/master/rss.hs#L107
> >
> > And here's where I use them, getting multiple at a time with async:
> >
> > https://github.com/orblivion/feedGetter/blob/master/rss.hs#L208
> >
> > What happens when I run this is that it outputs that it's "Getting"
> > the file, waits a while (presumably to download the whole thing),
> > then says it's "Saving". And I checked the file system, it's not
> > there during the pause. I'm not entirely sure why. Is it my choice
> > of libraries, or the way I'm using them? Perhaps something to do
> > with async? I just tried content <- simpleHttp "http://google.com"
> > in ghci, and it does pause for a second, so I'm guessing this is
> > strict from the getgo. But I've done almost no I/O before.
> >
> > Is there a straightforward, canonical option? It seems like there
> > perhaps should be. But if it comes down to using pipes or conduit,
> > what the heck I'll try it out, I'd like to learn pipes eventually.
>
> Michael is very good with documenting his packages, this is what I
> found in the docs for http-conduit (http://is.gd/WkDb7G):
>
>   Note: Even though this function returns a lazy bytestring, it does
>   not utilize lazy I/O, and therefore the entire response body will
>   live in memory. If you want constant memory usage, you'll need to
>   use the conduit package and http directly.
>
> /M
>
> --
> Magnus Therning                      OpenPGP: 0xAB4DFBA4
> email: magnus at therning.org   jabber: magnus at therning.org
> twitter: magthe               http://therning.org/magnus
>
> I invented the term Object-Oriented, and I can tell you I did not have
> C++ in mind.
>      -- Alan Kay
>
> _______________________________________________
> Beginners mailing list
> Beginners at haskell.org
> http://www.haskell.org/mailman/listinfo/beginners
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.haskell.org/pipermail/beginners/attachments/20130811/91b2ebc3/attachment.htm>