[GHC] #9266: getDirectoryContents blow its stack in a huge directory

[GHC]

#9266: getDirectoryContents blow its stack in a huge directory
----------------------------------------+----------------------------------
        Reporter:  joeyhess             |            Owner:
            Type:  bug                  |           Status:  new
        Priority:  normal               |        Milestone:
       Component:  libraries/directory  |          Version:  7.6.2
      Resolution:                       |         Keywords:
Operating System:  Linux                |     Architecture:
 Type of failure:  None/Unknown         |  Unknown/Multiple
       Test Case:                       |       Difficulty:  Unknown
        Blocking:                       |       Blocked By:
                                        |  Related Tickets:
----------------------------------------+----------------------------------

Comment (by joeyhess):

 Sorry, had wrong ghc version entered before.

 It might indeed save some memory to deepseq the accumulator as described
 in the blog post. I have not tried, since in my use case I want to avoid
 buffering the whole list in memory at all.

 I'm currently using a getDirectoryContents' that uses unsafeInterleaveIO.
 To avoid exceptions being thrown after the call has succeeded, when the
 return list is traversed, I made it catch and discard exceptions from
 Posix.readDirStream etc, so in an exceptional condition the list may not
 contain all items in the directory.

 That was ok in my use case, but I dunno if it would be acceptable for the
 real getDirectoryContents. It would probably be fine to just fix it to not
 blow the stack, and perhaps add a note to its documentation that the list
 of directory contents is not streamed lazily. (Although note that eg,
 removeDirectoryRecursive uses getDirectoryContents and so can also
 unexpectedly use large amounts of memory..)

 I do wonder if conduit has a better way to handle this.

--
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/9266#comment:5>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler