[Haskell-cafe] debugging memory corruption

[Alexander Kjeldaas]

What I've mostly done in similar circumstances (jni)

1. Create an interface (virtual functions or template) for the FFI in C++
that covers everything you use. Then create one test implementation and one
real implementation. The test implementation must allocate resources
whenever the real FFI does so. Doing memory allocation works. This makes it
possible to test all your FFI in C++ using valgrind.

2. Add tracing support to the real implementation and replay support to the
test implementation.

3. Upload to Hackage.

Alexander
On Dec 1, 2012 5:06 PM, "Evan Laforge" <qdunkan at gmail.com> wrote:

> Ever since upgrading to 7.6.1 I regularly get panics like this:
>
> seq: internal error: evacuate: strange closure type -1958168540
>     (GHC version 7.6.1 for x86_64_apple_darwin)
>     Please report this as a GHC bug:
> http://www.haskell.org/ghc/reportabug
>
> I've seen some variations, but basically I think it just means someone
> is corrupting memory and it goes unnoticed until the GC trips over it.
>
> This happens infrequently (maybe once in 15m, very roughly, it's
> extremely inconsistent) in non-optimized code, and frequently (maybe
> once in 5m) in optimized code.  This only happens during interactive
> use, not during testing or profiling.
>
> I had a problem like this once before, and it took a very long time to
> track down.  And in fact, I never really tracked it down, I just got a
> test that could semi-reliably reproduce it, and then by trial and
> error discovered that if I changed the alignment of a particular
> Storable instance from 1 to 4, the problem stopped happening (and 1
> should have been correct, it was a struct of chars).  Not exactly a
> satisfying solution, and now I'm thinking all I did was make ghc 6
> stop manifesting it, and with 7 it's back again.
>
> I'm most suspicious of the FFI usage since it's easy to corrupt memory
> in C++ and even easier to write a bogus Storable instance that does
> the same, but I really have no idea what or where.  My first thought
> was to start cutting out portions (especially FFI-using portions) to
> try to isolate it, but it's slow going because it can sometimes take
> quite a while to the bug to come out again.  My second thought was
> that I need a more automated way to reproduce it, but it's nontrivial
> because it only comes up when I'm using the interactive GUI parts,
> which are also incidentally a big chunk of FFI.  And even if I do get
> a repro, as I did before, it just means I can more quickly poke
> randomly at things hoping they change it, but even if I can get it to
> stop happening it doesn't mean I understood it, or even really fixed
> it.  This is also the kind of bug (well, it was last time), which is
> highly dependent on the code, so add one print and it stops happening.
>  I have a sort of complicated scheme where I pass a wrapped haskell
> function callback along with a wrapped freeHaskellFunPtr to free the
> last one, along with itself, maybe it's something to do with that.
>
> Anyone out there with ideas or advice on how to track down this kind
> of bug?  My next thought is to try to automate the GUI parts, or maybe
> just the FFI part, so I can write a program to randomly fuss with it
> until it crashes.  I've also tried valgrind, but it doesn't report
> anything suspicious.  But it also doesn't seem to work on FFI Storable
> corruption, I've tried intentionally inserting a bad poke and valgrind
> still won't report it.
>
> Thanks in advance for any insight!
>
>
> Actually, there's a whole other discussion which has been nagging at
> me for a while, though another thread would be more appropriate.  But
> in short it's that it feels like hsc2hs is just too low level, and too
> error-prone.  It's tempting to use because it comes with ghc, but it
> seems bad to tell people "haskell is a safe language, but as soon as
> you want to talk to C you're writing totally unchecked pokes and
> peeks".  Maybe I should go evaluate the alternatives like c2hs, or
> maybe safety features can added to hsc2hs.  Wouldn't it be nice to
> have ghc come with a high level and safe FFI language?
>
> _______________________________________________
> Haskell-Cafe mailing list
> Haskell-Cafe at haskell.org
> http://www.haskell.org/mailman/listinfo/haskell-cafe
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.haskell.org/pipermail/haskell-cafe/attachments/20121201/7486224a/attachment.htm>