[Haskell-cafe] [ANNOUNCE] (and request for review): directory-tree v0.9.0

[Jason Dagit]

On Mon, Aug 9, 2010 at 10:48 PM, Brandon Simmons <
brandon.m.simmons at gmail.com> wrote:

> Greetings Haskellers!
>
> directory-tree is a module providing a directory-tree-like datatype
> along with Foldable and Traversable instances, along with a simple,
> high-level IO interface. You can see the package along with some
> examples here (apologies if the haddock docs haven't been generated
> yet) :
>
>    http://hackage.haskell.org/package/directory-tree


If I understand what you're saying, then your library is very similar to an
abstraction that darcs had for years knows as "Slurpy".  The experience in
the darcs project was that it lead to performance issues and correctness
issues that were hard to find/fix.

>
>
> This primary change in this release is the addition of two
> experimental "lazy" functions: `readDirectoryWithL` and `buildL`.
> These functions use `unsafePerformIO` behind the scenes to traverse
> the filesystem as required by pure computations consuming the returned
> DirTree data structure. I believe I am doing this safely and sanely
> but would love if some more experienced folks could comment on the
> code.
>

unsafePerformIO or unsafeInterleaveIO?

Either way, to me it seems a bit dangerous to be doing this sort of lazy IO.
 If the directory structure is large will I run out of file handles?  How
will IO errors be handled?  Will I receive the exceptions in pure code or
inside my IO actions?  Will I run into space leaks if something holds on to
1 file and then references it "after" the directory traversal?  I might have
my history wrong, but as I recall darcs started with lazy slurpies and moved
to doing things strictly due to space leaks, running out of file
descriptors, file descriptor leaks (not running out, but having the file be
locked long after darcs should have been 'done' with it), and exception
delivery.

It's a seductive path but one that does not seem to have a good ending.

I'm not sure what darcs uses these days.  Perhaps that's what hashed-storage
provides, although I haven't been able to find any documentation on
hashed-storage other than the haddocks (which only document the api with no
overview or explanation of the problem hashed-storage solves).

Jason
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.haskell.org/pipermail/haskell-cafe/attachments/20100810/70ae9195/attachment.html