heap corruption?

[Frederik Eaton]

Hi Alberto,

No, nothing new. I talked to John Meacham and he said that using
unsafePerformIO with ForeignPtr is asking for trouble; but then when I
showed him the code he was like "oh that's probably OK".

So I guess you've actually managed to reproduce it then?

Since you use withForeignPtr everywhere, it should be impossible for
any C function to get an unallocated piece of memory, right? I don't
ever allocate my own memory in my code. I imagine it's just exposing a
race condition in GHC.

Frederik

On Mon, Jun 05, 2006 at 09:54:27PM +0200, Alberto Ruiz wrote:
> Hi Frederik, have you discovered something new? Curiously, when I print the 
> foreign pointer in createV, the problem disappears (in ghc6.4.1, tomorrow I 
> will try this in other machine with 6.4.2)...  
> 
> Alberto
> 
> createV s n f = unsafePerformIO $ do
>     p <- mallocForeignPtrArray n
>     --print p
>     prot s $ withForeignPtr p (f n)
>     --print p
>     return (V n p)
> 
> Loading package base-1.0 ... linking ... done.
> Prelude> :m Vector
> Prelude Vector> dot (useFast $ $(dim 1000000) (2*vconst 2)) (vconst 4)
> Loading package bla bla
> ap=0x2202008, bp=0x2202008, rp=0x6446c8
> an=1000000, bn=1000000, rn=1
> res=16000000.000000
> 1.6e7
> Prelude Vector> dot (useFast $ $(dim 1000000) (2.*vconst 2)) (vconst 4)
> ap=0x3602008, bp=0x1502008, rp=0x404770
> an=1000000, bn=1000000, rn=1
> res=15971552.000417
> 1.597155200041718e7
> 
> createV s n f = unsafePerformIO $ do
>     p <- mallocForeignPtrArray n
>     print p
>     prot s $ withForeignPtr p (f n)
>     print p
>     return (V n p)
> 
> Prelude> :m Vector
> Prelude Vector> dot (useFast $ $(dim 1000000) (2*vconst 2)) (vconst 4)
> Loading package bla bla
> 0x007066c8
> 0x04602008
> 0x04602008
> 0x04e02008
> 0x04e02008
> 0x04602008
> 0x01402008
> 0x01402008
> 0x04602008
> ap=0x4602008, bp=0x4602008, rp=0x7066c8
> an=1000000, bn=1000000, rn=1
> res=16000000.000000
> 0x007066c8
> 1.6e7
> Prelude Vector> dot (useFast $ $(dim 1000000) (2.*vconst 2)) (vconst 4)
> 0x00412770
> 0x02102008
> 0x02102008
> 0x02902008
> 0x02902008
> 0x02102008
> 0x02102008
> ap=0x2102008, bp=0x2102008, rp=0x412770
> an=1000000, bn=1000000, rn=1
> res=16000000.000000
> 0x00412770
> 1.6e7
> 
> 
> 
> On Friday 02 June 2006 22:00, Frederik Eaton wrote:
> > I'm now using 6.4.2. The bug still persists, but I can't immediately
> > reproduce it with basic GSLHaskell operations, nor with raw memory
> > operations. It is even difficult to reproduce as it is, the examples I
> > gave don't trigger it anymore (I was running them under 6.4.1). For
> >
> > instance, this gives a correct result:
> > > dot (useFast $ $(dim 1000000) (2*vconst 2)) (vconst 4)
> >
> > and this gives an incorrect result:
> > > dot (useFast $ $(dim 1000000) (2.*vconst 2)) (vconst 4)
> >
> > (the first one creates a new matrix of 2's and does element-wise
> > multiplication; the second just scales by 2)
> >
> > It could be a bug in our code, but I'd be surprised. It only triggers
> > for vectors of length 130056 or greater (that's 130056 8-byte doubles,
> > or 2^20-8128 bytes). If something were being prematurely finalized, one
> > would imagine that shorter vectors would also expose the problem. Most
> > of all, it seems very strange that in certain situations two different
> > objects would get the same address when they're large, but not when
> > they're small. But I'll keep looking for the source of the problem.
> >
> > Frederik
> >
> > On Fri, Jun 02, 2006 at 05:11:24PM +0200, Alberto Ruiz wrote:
> > > Hi Frederik, I will be out of town this weekend (I really need it!), I
> > > will take a look at the problem on Monday. By now my recommendation is:
> > >
> > > 1) try to reproduce the problem without GSLHaskell or any other library,
> > > using just mallocArray, newArray, etc. with those big sizes. (Use ghc
> > > 6.4.2 or better..)
> > >
> > > 2) If this is not possible, try to reproduce the problem with the
> > > simplest possible example using the unmodified GSLHaskell.
> > >
> > > Alberto
> > >
> > > On Friday 02 June 2006 16:15, Frederik Eaton wrote:
> > > > Hi Alberto,
> > > >
> > > > I'm experiencing a problem which may be a bug in GHC. Here I take the
> > > > dot product of two vectors. I put debugging traces in the 'dotR' C
> > > > function (which I call in 'dot'), so one can see the addresses of the
> > > > memory blocks. When the vectors are below a certain size, the function
> > > > seems to give the correct result. However, when they are above a
> > > > certain size, the same pointers are sometimes used for both arguments
> > > > (ap and bp), and the result is of course incorrect. The incorrect
> > > > result is consistent with both pointers referring to the first vector
> > > > (vconst 2). Also, with other arguments I sometimes get heap
> > > > corruption. Any idea what might be the problem?
> > > >
> > > > Prelude Vector> print $ (dot $ useFast $ $(dim 1000) $ (vconst 2))
> > > > (ones) ap=0xb1b0a008, bp=0xb1b0c008, rp=0xb1b036c8
> > > > an=1000, bn=1000, rn=1
> > > > res=2000.000000
> > > > 2000.0
> > > > Prelude Vector> print $ (dot $ useFast $ $(dim 1000000) $ (vconst 2))
> > > > (ones) ap=0xb3402008, bp=0xb3402008, rp=0xb1b0c770
> > > > an=1000000, bn=1000000, rn=1
> > > > res=4000000.000000
> > > > 4000000.0
> > > > Prelude Vector> print $ (dot $ useFast $ $(dim 1000000) $ (vconst 2))
> > > > (vconst 2) ap=0xb2c02008, bp=0xb2302008, rp=0xb1b08780
> > > > an=1000000, bn=1000000, rn=1
> > > > ap[130055]=2.000000 bp[130055]=-0.000000
> > > > ap[130056]=2.000000 bp[130056]=2.000001
> > > > ap[130058]=2.000000 bp[130058]=2.000000
> > > > ap[130059]=2.000000 bp[130059]=0.000000
> > > > ap[130060]=2.000000 bp[130060]=2.000001
> > > > ...
> > > > res=3992888.004728
> > > > 3992888.004727577
> > > >
> > > > Here is dotR:
> > > >
> > > > int dotR(DVEC(a), DVEC(b), DVEC(r)) {
> > > >     REQUIRES(an == bn && rn == 1, BAD_SIZE);
> > > >     DEBUGMSG("dotR");
> > > >     fprintf(stderr,"ap=%p, bp=%p, rp=%p\n",ap,bp,rp);
> > > >     fprintf(stderr,"an=%d, bn=%d, rn=%d\n",an,bn,rn);
> > > >     double res=0.0;
> > > >     int i;
> > > >     for(i=0; i<an; i++) {
> > > >       if(ap[i] != floor(ap[i]) || bp[i] != floor(bp[i]))
> > > >           fprintf(stderr,"ap[%d]=%lf bp[%d]=%lf\n",i,ap[i],i,bp[i]);
> > > >       res += ap[i]*bp[i];
> > > >     }
> > > >     fprintf(stderr,"res=%lf\n",res);
> > > >     rp[0] = res;
> > > >     OK
> > > > }
> > > >
> > > > I switched to not using gsl here, so that I could be sure the bug
> > > > wasn't in gsl. However, when using gsl the same problems occur. So I
> > > > think it may be heap corruption in ghc. But I don't know how to debug
> > > > that. I tried running in valgrind but it doesn't show anything
> > > > obvious. If you want to run what I have, here is my code:
> > > >
> > > > http://ofb.net/~frederik/GSLHaskell2.tar.gz
> > > >
> > > > The problem is exposed by building and running:
> > > >
> > > > print -l ':m Vector' 'print $ (dot $ useFast $ $(dim 1000000) $ (vconst
> > > > 2)) (vconst 2)' | =ghci -package GSL -fglasgow-exts
> > > >
> > > > Thanks,
> > > >
> > > > Frederik
> 

-- 
http://ofb.net/~frederik/