[Haskell-cafe] Use Haskell shared library with foreign exports with dlopen/dlsym

[Albert Y. C. Lai]

This is going to be very cunning.

hs_init is not needed in this case because the executable comes from

     ghc --make -dynamic test.hs

The "-dynamic" causes the exemption. This is similar to the case of ghci 
7.8 on Linux.

(I agree that there are differences too. For now, the similarity 
dominates. The executable has already brought in a dynamic RTS, and it 
has already been hs_inited and running. Subsequently, any dynamic 
library that looks for a dynamic RTS will simply find the running one.)

Here is the story of the segfault:

1. "foreign export" causes the following dynamic library constructor to 
be included in the dynamic library:

static void stginit_export_Test_zdfstableZZC0ZZCmainZZCTestZZCtest() 
__attribute__((constructor));
static void stginit_export_Test_zdfstableZZC0ZZCmainZZCTestZZCtest()
{
   foreignExportStablePtr((StgPtr) 
&Test_zdfstableZZC0ZZCmainZZCTestZZCtest_closure);
}

foreignExportStablePtr is in the RTS (and we do have a dynamic RTS 
already running). Test_zdfstableZZC0ZZCmainZZCTestZZCtest_closure is in 
the dynamic library representing the test::Int in module Test.

I found out by -keep-tmp-files and examining the tmp files.

2. dlopen causes a mmap, and the closure's address is in that mmap region.

3. dlopen also causes the constructor to be invoked. I suppose it means 
a StablePtr is created, therefore the closure's address is registered in 
the global table of StablePtrs.

4. dlclose causes a munmap. The closure's address is now invalid.

5. Garbage collection visits everything registered in the global table 
of StablePtrs (among other roots). But one of the registered addresses 
is invalid. Segfault.

I found out by asking gdb for backtrace. So in fact I can specifically 
point at "evacuate()".


At this point, my recommendation:

Don't use "-dynamic" on the executable. Do it static.

This implies you will have two RTSes, the static one used by the 
executable for itself, and the dynamic one used by the dynamic library 
for itself.

This is a good thing because the static GCtor will never visit the 
closure in the dynamic library, hell it will never hear of it.

The dynamic library will find and use foreignExportStablePtr from the 
dynamic RTS only. It doesn't see the static RTS, or any static library 
for that matter.

The dynamic GCtor is the only one who knows of the closure, but this 
GCtor no longer runs after dlclose, hell it no longer exists.

The down side is that you now have to do the full thing to the dynamic 
library:
http://www.vex.net/~trebla/haskell/so.xhtml
section "Making The Library"


P.S. I believe that those things about hs_add_root and __stginit_* are 
outdated.


On 14-09-08 12:47 PM, Carter Schonwald wrote:
> what OS and other things?
> what does ghc --info say?
> Also, when calling haskell code as if it were C code, you need to init
> the RTS its using, its not going to magically know you're linking to it
> from haskell.(though maybe you can arrange things to use the pre inited
> hs runtime, but i'm not familiar with how to do so)
> http://www.haskell.org/haskellwiki/Calling_Haskell_from_C
>
> On Mon, Sep 8, 2014 at 10:05 AM, Tobias Neumann <mail at tobias-neumann.eu
> <mailto:mail at tobias-neumann.eu>> wrote:
>
>     Hello Cafe,
>
>     I am trying to use a Haskell shared library with foreign exports from
>     Haskell again via dlopen/dlsym. Sadly it segfaults, and the segfaults
>     happen on dlclose during garbage collection points (as figured out by
>     monochrom in #haskell). So right now I can only open a library once and
>     may not dlclose it. Can someone point me to a mistake I made, or is this
>     rather a ghc (7.8.3) bug? Please see attached minimal example.
>
>     Regards, Tobias
>
>
>     test.hs:
>     module Main where
>
>     import qualified System.Posix.DynamicLinker as DL
>     import Foreign
>
>     foreign import ccall "dynamic"
>          mkTest :: FunPtr Int -> Int
>
>     main = do
>          DL.withDL ("./libtest.so") [DL.RTLD_NOW] $ \dl -> do
>              dimPtr <- DL.dlsym dl "test"
>              let test = mkTest dimPtr
>              print test
>
>     libtest.hs:
>     module Test() where
>
>     import Foreign
>
>     foreign export ccall test :: Int
>
>     test :: Int
>     test = 124
>
>     build with:
>              ghc --make -shared -dynamic -fPIC libtest.hs -o libtest.so
>              ghc --make -dynamic test.hs
>
>     _______________________________________________
>     Haskell-Cafe mailing list
>     Haskell-Cafe at haskell.org <mailto:Haskell-Cafe at haskell.org>
>     http://www.haskell.org/mailman/listinfo/haskell-cafe
>
>
>
>
> _______________________________________________
> Haskell-Cafe mailing list
> Haskell-Cafe at haskell.org
> http://www.haskell.org/mailman/listinfo/haskell-cafe
>