swap leak problem

[David Roundy]

Hello.  I've got a memory problem that is stumping me, and could use advice
in trying to fix it.

In brief, the heap of my application keeps increasing with time, until
after a few hours it uses up all my swap and gets killed.  There is no
indication of a true memory leak.  Heap profiling indicates that the
haskell memory usage (including stacks, via -xt) remains below its initial
peak value.  Similarly, I've attached a sort of debugging counting
finalizer to my foreignPtrs (a static int in C land), and it's clear that
all the foreignPtrs I allocate are getting freed.  Most of them are
allocated with mallocForeignPtrBytes anyways, which should be about as safe
an allocation method as one could use.

The allocation pattern of my code is as follows.  It reads a gzipped patch
file into memory, parses it, then reads in a bunch of other files, and
writes modified versions of them.  Then it reads in the next patch and
repeats.  So it does large amounts of allocation and deallocation on a
periodic basis.

About half of the memory used is stored in foreignPtrs.  With the exception
of the patch files (which I'll get to later), all foreignPtrs are generated
by mallocForeignPtrBytes or mallocForeignPtrArray.

The patch files are gzip compressed, so I don't know their size until I've
read them.  I read the patch file one chunk at a time, and use reallocBytes
to allocate a bit more memory each time.  I don't expect in general to be
calling reallocBytes more than four times per patch file.  Of course, once
I'm done I convert to a foreignPtr via newForeignPtr finalizerFree.

My only idea as to the source of the problem is that it could be either a
result of memory fragmentation or of not cleaning up before allocating.
Neither theory is particularly convincing.  Unfortunately, I'm at a loss as
to how to track this problem down, so any advise would be appreciated.
-- 
David Roundy
http://www.abridgegame.org/darcs