Reading multiple files fails

[Henk-Jan.van.Tuyl]

L.S.,

As my e-mail program mangled the copy/pasted source code a bit, I resend it as an attachment.

Best regards,
Henk-Jan van Tuyl


On Fri, 10 Oct 2003 01:33:48 +0200, Henk-Jan.van.Tuyl <Henk-Jan.van.Tuyl at hotpop.com> wrote:

>
> L.S.,
>
> After reading through 60 or 61 files (about 220 kB), Hugs fails with the message:
>      (1426170 reductions, 2976893 cells, 68 garbage collections)
>      ERROR - Garbage collection fails to reclaim sufficient space
>
> This is done with the program below (ReadALot.lhs), which reads a series of files (these files are listed in dir.dat).
> The program reads the first and the last (non-empty) line of each file and writes them, together with the filename in
> the output file result.dat.
>
> The program works fine when compiled with GHC, it then reads 2946 files without a problem.
>
> More info.:
> System:       Windows 98
> Hugs version: Nov 2002
>
> Best regards,
> Henk-Jan van Tuyl-------------- next part --------------
{-
   ReadALot.lhs
   Read a lot of files

   2003-10-06

   History:
   031009  Added dummy parameter to "main"
-}

> import IO
> import Monad

> inputSet   :: String
> inputSet   = "dir.dat"

> outputFile :: String
> outputFile = "result.dat"

===============================================================================
main
===============================================================================

> main :: a -> IO ()
> main _ = 
>   do 
>     hInputSet   <- openFile inputSet   ReadMode
>     hOutputFile <- openFile outputFile WriteMode
>     fileNames   <- hGetContents hInputSet
>     allData     <- collectAllData $ lines fileNames 
>     hPutStrLn hOutputFile $ unlines allData   
>     hClose hInputSet
>     hClose hOutputFile


===============================================================================
collectAllData 
Read a list of file names and return file name, first line and last line
of each file in a list
===============================================================================

> collectAllData :: [String] ->      -- A list of filenames
>                   IO [String]      -- A list of results
> 
> collectAllData []                     = return []
> collectAllData (fileName : fileNames) = 
>   do
>     putStrLn $ "Reading: " ++ fileName
>     h <- openFile fileName ReadMode
> 
>     contents <- hGetContents h
>
>     dataFromFile <- 
>       return           $! 
>         parseFile      $ 
>         filter (/= "") $ 
>         lines contents 
>
>     let (firstLine, lastLine) = dataFromFile 

      To force reading of the file before closing, we add a "when" (from 
      the Monad library), with a boolean expression that uses lastLine
      and will always be true:

>     when (length lastLine >= 0) $
>       hClose h
>
>     rest  <- collectAllData fileNames
>
>     return $ fileName : firstLine : lastLine : "" : rest


===============================================================================
parseFile 
Returns the first and the last string of a list of strings, as a tuple
===============================================================================

> parseFile :: [String] -> (String, String)
> parseFile []       = ("", "")
> parseFile [x]      = (x,  "")
> parseFile (x : xs) = (x,  last xs)