[Haskell-cafe] Haskell lib in non-Haskell program

[Eric Y. Kow]

Hi all,

I'd like to share my experiences packaging a Haskell program so that it
can be used as a library by a non-Haskell application.  I hope somebody
might be able to (A) confirm that I'm doing the right thing or better
still (B) suggest some corrections and/or simplifications.

My objectives are:

1. For the non-Haskell application to use my library
2. ... without any Haskell infrastructure on the machine

I think this means I want a shared library [1,2] although I'm a little
bit fuzzy on the issue. I am not particularly concerned about the size
of my executable or libraries.  Including the Haskell RTS and my package
deps is acceptable if that's what it takes to meet my primary and
secondary objectives.

I am targeting Windows and MacOS X.  For Windows, my approach is to
build a giant DLL with all the Haskell stuff statically linked in.  For
MacOS X I am using the `-dynamic` flag to compile all my dependencies
(actually, I just set shared: True in cabal config) as well as the
`-shared` flag.

I am also now using GHC 7.0.3 with the latest (at the time of this writing)
Haskell Platform (2011.2.0.1)

[1] http://hackage.haskell.org/trac/ghc/wiki/SharedLibraries
[2] http://www.haskell.org/ghc/docs/latest/html/users_guide/using-shared-libs.html#id555945

The whole thing
----------------------------------------------------------------------
You can see my efforts so far with

     darcs get --lazy http://code.haskell.org/GenI
     cd GenI
     cabal install
     cd geniwrapper
     make

MacOS X users should be able to download

     http://erickow.com/tmp/MinimalGenI-OSX.tar.gz

and run a test-mac.sh script which compiles a program with GCC
using GenI, runs it and outputs some JSON string.

Exposing some C functions and using them
----------------------------------------------------------------------
So far, I think I know how to expose some of library as C functions and to
compile a little program written in C that uses this library.

    ghc --make -fvia-C MinimalGenI # provides some C functions via FFI exports
    ghc -c StartEnd.c              # http://www.haskell.org/ghc/docs/latest/html/users_guide/win32-dlls.html

    ghc test-c.c MinimalGenI.o MinimalGenI_stub.o StartEnd.o\
        -package GenI -package utf8-string\
        -o test-c.o

I notice that -fvia-C is going away, but I'm going to ignore this fact
for now and assume/hope that doing without it will be easy

Building a Windows shared library
----------------------------------------------------------------------
As I understand it, it is possible to build a Windows DLL, albeit one
containing the Haskell RTS and all my Haskell libraries statically
linked in.  Users can download the DLL and link it to their Visual Basic
programs without having to touch GHC.  The extra work involved looks
like this:

   ghc MinimalGenI.o MinimalGenI_stub.o StartEnd.o\
         -package GenI -package utf8-string\
         -shared -o MinimalGenI.dll
   ghc test-c.c MinimalGenI.dll -o test-c2

Note that building test-c.c with ghc does not meet my second
objective of being able to combine Haskell lib with non-Haskell
program sans Haskell infrastructure.  However, I think I know
how to do it for MacOS X and that the procedure would be more or
less the same. I have not looked into it.

[3] http://www.haskell.org/ghc/docs/latest/html/users_guide/win32-dlls.html

Building a MacOS X shared library
----------------------------------------------------------------------
For MacOS X, I would almost like to generate a gigantic file like the
Windows DLL. But maybe that's the wrong thing to want.

I do almost exactly the same thing, except that instead of distributing
a single giant file, I track down a whole bunch of dependencies and copy
over the dylib files for them.  Note that as a prerequisite, this
requires reinstalling a lot of packages with --enable-shared (I just
edit ~/.cabal/config and set shared to True).

   ghc MinimalGenI.o MinimalGenI_stub.o StartEnd.o\
         -package GenI -package utf8-string\
         -dynamic -shared -o MinimalGenI.dylib
   ghc test-c.c MinimalGenI.dylib -o test-c2

As an alternative to building with GHC, I can compile test-c.c and
link with MinimalGenI.dylib directly with GCC.  It's a little bit
hairy.  What I did was to run ghc with -v3 to see what gcc commands
it was running, clean them up and package them in a script
(test-mac.sh attached).

It's a little bit voodoo-ish, a lot of flags that I don't really
understand the significance of, and some libraries I'm not 100%
clear on why I need the static versions for.  Also I find it slightly
odd that I can't seem to just merge the first two steps of the process,
producing/assembling assembly code for test.c.  In any case, it seems
to work...

Packaging the MacOS X shared library
----------------------------------------------------------------------
I'm trying to distribute my Haskell library in a way that does not
require the user to install anything beyond XCode.  To do this, I
track down the dynamic libraries for all the packages I'm using
and their dependencies.  Unfortunately, the way I did it was
basically trial and error (find all dylib and hand remove the ones
that seem totally irrelevant; get a friend to test it on a raw machine).
Rather than the dumb approach used in the attached dist-mac.sh, it
should be relatively easy to write a small Haskell script to recurse
down the package dependency tree and tell me exactly which files I need
to copy.  Maybe such a tool exists already.

If this procedure is right, and if somebody should write such a script
(maybe already part of the cabal-macosx package?) then I bet creating
app bundles for your MacOS X Haskell GUI tools should be quite
straightforward as well.

Thanks for reading!

-- 
Eric Kow <http://erickow.com>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: test-mac.sh
Type: application/x-sh
Size: 2384 bytes
Desc: not available
URL: <http://www.haskell.org/pipermail/haskell-cafe/attachments/20110518/eb76736c/attachment.sh>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: dist-mac.sh
Type: application/x-sh
Size: 1858 bytes
Desc: not available
URL: <http://www.haskell.org/pipermail/haskell-cafe/attachments/20110518/eb76736c/attachment-0001.sh>
-------------- next part --------------
OS=$(shell uname)

ifeq ($(OS),Darwin)
  SHARED_LIB_EXT=dylib
  SHARED_LIB_FLAGS=-dynamic -shared
else
  SHARED_LIB_EXT=dll
  SHARED_LIB_FLAGS=-shared
endif

PACKAGES=$(patsubst %, -package %,\
	 GenI utf8-string)

all: test-hs test-c test-c2 test-hs-c

windows: all test-c-win

clean:
	rm -f *.dll *.dll.a *.dylib
	rm -f *.o *.hi
	rm -f MinimalGenI_stub.*
	rm -f test-hs test-c
	rm -f *~

# Exporting a C API
# -----------------
MinimalGenI.o: MinimalGenI.hs
	ghc --make -fvia-C $<

StartEnd.o:
	ghc -c StartEnd.c

test-hs: test-hs.hs MinimalGenI.o
	ghc --make $<

test-hs-c: test-hs-c.hs MinimalGenI.o
	ghc --make $<

test-c: test-c.c MinimalGenI.o StartEnd.o
	ghc $^ -o $@ MinimalGenI_stub.o $(PACKAGES)

# Bundling into a single shared library
# -------------------------------------
MinimalGenI.$(SHARED_LIB_EXT): MinimalGenI.o StartEnd.o
	ghc $(SHARED_LIB_FLAGS) -o $@ $^ MinimalGenI_stub.o $(PACKAGES)

test-c2: test-c.c MinimalGenI.$(SHARED_LIB_EXT)
	ghc $^ -o $@
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 195 bytes
Desc: not available
URL: <http://www.haskell.org/pipermail/haskell-cafe/attachments/20110518/eb76736c/attachment.pgp>