[web-devel] Generating, serving, and deleting temporary files with Wai/Yesod

[Michael Snoyman]

On Fri, Oct 22, 2010 at 8:21 PM, Matt Brown <matt at softmechanics.net> wrote:
> Hello all,
>
> I've been experimenting with generating graphs from log files using
> timeplot and serving them with yesod.  Each generated graph is only
> served once, so I need a way to clean them up.  A cron job would be
> ok, but it's one more thing I'd have to remember to do, and I'm lazy.
>
> Instead, I'd like to pass a "clean up action" to the enumerator
> sending the file, to be performed after EOF is reached.  For most
> files, this would just close the handle.  For temp files, it would
> also delete the file.  Here's the gist of it:
> http://gist.github.com/640902
>
> The first revision defines a monolithic fromTempFile :: FilePath ->
> Enumerator.  The second imports a factored version from
> Network.Wai.Handler (see my wai commit:
> http://github.com/softmechanics/wai/commit/2cb5ea47ca458f4b91a73e01a1dd0a29e84e001e).
> Not sure wai is the place for it.  Perhaps wai-extra would be a better?
>
> Does this seem like a good approach?

This does seem like the right approach. Just as fair warning: using an
enumerator for serving files loses out on the possible optimization of
a sendfile system call. Another possibility may be to keep a pool of
file names to use as temporary files and keep rotating their usage.
But I *do* think your approach is the correct one.

Regarding the API changes to wai: they seem reasonable, and I'd
probably accept them. Some concerns about the implementation:

* Instead of fromHandle' and fromFile', perhaps fromHandleFinally and
fromFileFinally?
* In fromFile', why are you calling hClose? withBinaryFile
automatically closes the file handle. Are you worried about the
temporary file getting deleted while the file handle is still open?
* In fromHandle', onEOF won't get called if the iteratee returns Left.
Is this intended behavior?
* Your code won't work in the presence of exceptions. Ideally the body
of fromHandle' would be wrapped with a call to finally. This point is
related to the previous one.

Michael