ghci linker and circular dependencies: bug or feature?

[Simon Marlow]

Andrea Rossato wrote:
> Hello,
> 
> this is a followup of this:
> 
>     http://article.gmane.org/gmane.comp.lang.haskell.cafe/48644
> 
> which didn't have replays. In order to make the review a bit easier I
> prepared a minimal test case which, I believe, could prove there's a
> bug in the ghci linker. I'm not submitting a bug report because I'm
> not familiar with these problems and, in order not to waste other
> people's time, I'd rather have some preliminary review.
> 
> All the code examples can be found here:
> http://gorgias.mine.nu/ffi-test/
> 
> Here a tarball of everything:
> http://gorgias.mine.nu/ffi-test.tar.gz
> 
> Take a simple C library like this:
> 
>     #include <stdio.h>
>     extern char my_var[];
>     void my_fun()
>     {
>         fprintf (stderr, "%s\n", my_var);
>     }
> 
> 
> Since my_var is defined as external, the dynamic library compiled
> and linked with:
> 
>     gcc -Wall -fPIC -c -o mylib.o mylib.c
>     gcc -shared -Wl,-soname,libmylib.so.1 -o libmylib.so.1.0 mylib.o
> 
> would have my_var as undefined.
> 
> If I write some Haskell bindings (Test.hsc) like this:
> 
>     module Test where
> 
>     import Foreign.C
>     import Foreign
> 
>     #include <mylib.h>
> 
>     foreign import ccall unsafe "my_fun" my_cfun  :: IO ()
> 
>     my_fun :: IO ()
>     my_fun = my_cfun
> 
> I'll need to include a stub.c file to initialize my_var:
> 
>     char my_var[] = "Hello World!!";
> 
> And now come my problems:
> 
> 1. First I have a Cabal problem. If I set:
> 
>     extra-libraries: mylib
> 
> this will be used also when compiling dist/build/Test_hsc_make, and,
> since libmylib.so.1.0 has an undefined reference to my_var, which will
> be initialized only later, by stub.c, the compilation of the bindings
> will fail with:
> 
>     Configuring mylib-0.1...
>     Preprocessing library mylib-0.1...
>     /tmp/ffi-test/libmylib.so: undefined reference to `my_var'
>     collect2: ld returned 1 exit status
>     linking dist/build/Test_hsc_make.o failed
>     command was: /usr/bin/gcc -L/tmp/ffi-test -lmylib -L/usr/lib/ghc-6.10.1/base-4.0.0.0 [...]
> 
> I can find a work-around by not setting cabal extra-libraries and
> instead setting
> 
>     ghc-options: -lmylib
> 
> But when compiling a test file:
> 
>     import Test
>     main = my_fun
> 
> I'll need to pass -lmylib to ghc --make.
> 
> 2. I also have ghci problem. If I try with:
> 
>     ghci test_lib.hs
> 
>     GHCi, version 6.10.1: http://www.haskell.org/ghc/  :? for help
>     Loading package ghc-prim ... linking ... done.
>     Loading package integer ... linking ... done.
>     Loading package base ... linking ... done.
>     Ok, modules loaded: Main.
> 
>     Prelude Main> main
>     Loading package mylib-0.1 ... linking ... <interactive>: /home/andrea/.cabal/lib/mylib-0.1/ghc-6.10.1/HSmylib-0.1.o: unknown symbol `my_fun'
>     ghc: unable to load package `mylib-0.1'
> 
> Now, I have to remember to pass -lmylib:
> 
>     ghci -lmylib test_lib.hs 
> 
>     GHCi, version 6.10.1: http://www.haskell.org/ghc/  :? for help
> 
>     Loading object (dynamic) mylib ... failed.
>     <command line>: user specified .o/.so/.DLL could not be loaded (/tmp/ffi-test/libmylib.so: undefined symbol: my_var)
>     Whilst trying to load:  (dynamic) mylib
>     Additional directories searched: (none)
> 
> Now, the my_var symbol is included in:
> 
>     /home/andrea/.cabal/lib/mylib-0.1/ghc-6.10.1/HSmylib-0.1.o
> 
> and indeed:
> 
>     nm --print-arma /home/andrea/.cabal/lib/mylib-0.1/ghc-6.10.1/HSmylib-0.1.o | grep my_var
>     0000001c D my_var
> 
> So I desperately try:
> 
>     ghci /home/andrea/.cabal/lib/mylib-0.1/ghc-6.10.1/HSmylib-0.1.o -lmylib  /tmp/ffi-test/bindings/test_lib.hs 
> 
>     GHCi, version 6.10.1: http://www.haskell.org/ghc/  :? for help
> 
>     Loading object (static) /home/andrea/.cabal/lib/mylib-0.1/ghc-6.10.1/HSmylib-0.1.o ... done
>     Loading object (dynamic) mylib ... failed.
>     <command line>: user specified .o/.so/.DLL could not be loaded (/tmp/ffi-test/libmylib.so: undefined symbol: my_var)
>     Whilst trying to load:  (dynamic) mylib
>     Additional directories searched: (none)
> 
> Is this a bug? An intended behaviour? Is there a way out? What am I
> missing?

It's not a bug - or rather, it's a consequence of the fact that GHC does 
its own  runtime linking.  The dynamic library libmylib.so is being linked 
by the system linker, which has its own symbol table and knows nothing 
about GHC's linker and its symbol table.  The file HSmylib-0.1.o is loaded 
by GHC's linker, so the my_var symbol is in GHC's symbol table, and hence 
can't be found by the system linker when loading libmylib.so.

Some good news is that we should be able to make this work when we start 
using shared libraries, because HSmylib will be a shared library and it 
will be linked by the system linker (what I'm not sure about is how you 
load mutually recursive shared objects at runtime, but presumably there's a 
way to do that).

Perhaps you could work around the Cabal/hsc2hs problem by making a dummy 
library containing my_var, and passing it to Cabal using 
--hsc2hs-option=-ldummy.

Cheers,
	Simon


> Obviously the problem goes away if I link mylib.o with stub.o into
> libmylib.so.1.0.
> 
> Thanks for your help.
> 
> Andrea
> 
> ps: here you can find a small shell script to automate the proposed
> test (included in the tarball linked above):
> http://gorgias.mine.nu/ffi-test/test_dynamic.sh
> _______________________________________________
> Glasgow-haskell-users mailing list
> Glasgow-haskell-users at haskell.org
> http://www.haskell.org/mailman/listinfo/glasgow-haskell-users