GHC status report

Simon Marlow marlowsd at gmail.com
Fri May 2 19:44:35 UTC 2014


On 02/05/14 18:19, Edward Kmett wrote:
> I may have to dig to find an example, but when I last checked it seemed
> that c++ libraries would load fine, but there was a problem with static
> initializers not getting called, when loading from ghci, so if your c++
> library needed them then you'd have problems.

I think we're getting confused here.  External C++ libraries are, and 
always have been, loaded by the system linker, regardless of whether 
GHCi itself is dynamically linked.  So there's no difference in 7.8 with 
respect to how external C++ code is loaded.

However, if you link the C++ code into the same library as the Haskell 
code as you're doing with MPFR, then of course there is a difference 
(let's call that *internal* C++ code, for want of a better name).  I 
think that's a rare case (I understand why you need to do it), and most 
people should be linking their C++ code via separate shared libraries.

> An earlier version of the MPFR hackery used a static initializer to
> replace the GHC garbage collection hook for GMP with one that played
> evil games to figure out if it was being called from the MPFR constant
> cache.
>
> That initialized and loaded fine from ghc, but not from ghci. Our static
> initializer was never being called, despite the library being loaded.
>
> If your C++ practices there forbid static initializers -- some places do
> -- that may be why you aren't seeing the issue.

Absolutely not, we're drowning in static initialisers :)

Cheers,
Simon


> -Edward
>
>
> On Fri, May 2, 2014 at 9:47 AM, Simon Marlow <marlowsd at gmail.com
> <mailto:marlowsd at gmail.com>> wrote:
>
>     On 02/05/2014 14:28, Edward Kmett wrote:
>      > Perhaps. We actually tried that originally, but had issues about
>      > where and how to get cabal to place it. We'd need it to go somewhere
>      > installed rather than the local build dir lest it not be there when
>      > we go to use the lib, but IIRC, cabal couldn't/wouldn't tell me where
>      > it was putting the final installed version and then there is the
>      > issue of the local in place runs vs. post cabal install runs and
>      > referencing that dir from subsequent projects. If we want a
>      > transparent ’cabal install rounded` It isn't clear to me how to get
>      > there down this path, but it is entirely possible I just missed
>      > something obvious.
>      >
>      > MPFR/rounded was just an off the cuff example. Another place where
>      > the dynamic linker really helps is external c++ libraries which
>      > should now actually get all of their initializers called in the right
>      > order when launched from ghci.
>
>     External C++ libraries work just fine with the statically linked GHCi -
>     it uses the system linker to load them, and all the initalizers get
>     called, in the right order, as they should.  So this is where I'm
>     confused about what the problems actually are - there seems to be this
>     perception that GHCi didn't work with external C++ code, but as far as I
>     know it works just fine (indeed we're doing it a lot at Facebook, so
>     there's at least an existence proof that depending on a non-trivial
>     amount of external C++ with a statically-linked GHCi can work).
>
>     Cheers,
>     Simon
>
>
>      >
>      >
>      >> Cheers,
>      >> Simon
>      >>
>      >>> Switching to the system dynamic linker fo ghci seems to have
>     resolved
>      >>> all of that effortlessly.
>      >>>
>      >>> Dan Peebles has been talking to the MPFR folks to see if we can
>     get them
>      >>> to expose enough information about the 'hidden' allocations
>     they use
>      >>> that we can make them visible to GHC or have them do what our
>     local fix
>      >>> does and avoid using the MPFR allocator for their hidden
>     constant cache.
>      >>>
>      >>> If they do that then we can actually link to the library like
>     normal
>      >>> rather than link it in directly, but it isn't clear to me what
>     would
>      >>> happen even with those hooks if we rolled back to something
>     like the old
>      >>> custom linker.
>      >>>
>      >>> -Edward
>      >>>
>      >>>
>      >>> On Thu, May 1, 2014 at 5:30 PM, Simon Marlow
>     <marlowsd at gmail.com <mailto:marlowsd at gmail.com>
>      >>> <mailto:marlowsd at gmail.com <mailto:marlowsd at gmail.com>>> wrote:
>      >>>
>      >>>     On 01/05/14 15:27, Edward Kmett wrote:
>      >>>
>      >>>         Figured I'd make one case for dynamic linking:
>      >>>
>      >>> https://github.com/ekmett/__rounded
>      >>>         <https://github.com/ekmett/rounded>
>      >>>
>      >>>         Dynamic linking is finally enabling us to build a
>     version of MPFR
>      >>>         bindings for Haskell for scientific/high precision
>     computing
>      >>>         with 7.8. I
>      >>>         would really hate to lose it after all of these years
>     trying to
>      >>>         get it
>      >>>         work, as I have a rather large edifice being built atop
>     that
>      >>>         platform.
>      >>>         We tried and failed due to limitations of the old
>     linker for
>      >>>         almost 3 years.
>      >>>
>      >>>
>      >>>     I understand the issues with MPFR.  But how is dynamic
>     linking helping?
>      >>>
>      >>>
>      >>>         That said, -dynamic-too seems to cause me all sorts of
>     problems
>      >>>         elsewhere. ^C'ing out of a build and restarting it will
>     often
>      >>>         make a .o
>      >>>         but lose the .dyn_o, leading to GHC + cabal getting
>     confused and
>      >>>         refusing to build until I clean. This hits me several
>     times a day.
>      >>>
>      >>>
>      >>>     We should fix this (or at least make it a lot less likely).  Is
>      >>>     there a ticket?
>      >>>
>      >>>     Cheers,
>      >>>     Simon
>      >>>
>      >>>
>      >>>         -Edward
>      >>>
>      >>>
>      >>>         On Thu, May 1, 2014 at 3:29 AM, Simon Peyton Jones
>      >>>         <simonpj at microsoft.com <mailto:simonpj at microsoft.com>
>     <mailto:simonpj at microsoft.com <mailto:simonpj at microsoft.com>>
>      >>>         <mailto:simonpj at microsoft.com
>     <mailto:simonpj at microsoft.com> <mailto:simonpj at microsoft.com
>     <mailto:simonpj at microsoft.com>>>__>
>      >>>         wrote:
>      >>>
>      >>>              | Dynamic linking has been a huge headache in GHC,
>     and it's not
>      >>>              clear that
>      >>>              | it's an overall improvement compared with the static
>      >>>         linker.  Now that
>      >>>              | 7.8 is out of the way, it's time to have a
>     conversation about
>      >>>              whether we
>      >>>              | want to do dynamic linking again for 7.10, or
>     revert to
>      >>>         static
>      >>>              linking.
>      >>>
>      >>>              I echo this. Dynamic linking has had many
>     un-anticipated
>      >>>         costs and
>      >>>              it is still very far from sorted out.  It
>     originally felt
>      >>>         like a
>      >>>              Fantastic Idea to give up our own linker and adopt
>     the system
>      >>>              linker, but it now feels to me like a black hole,
>     endlessly
>      >>>         sucking
>      >>>              effort and increasing complexity.
>      >>>
>      >>>              My viewpoint is highly un-informed about details;
>     I just
>      >>>         watch the
>      >>>              traffic going by.  And of course it does have
>     benefits that
>      >>>              doubtless generate less traffic.
>      >>>
>      >>>              Simon
>      >>>
>      >>>              |
>      >>>              |
>      >>>              |
>      >>>              | >
>      >>>              | > On Tue, Apr 29, 2014 at 6:13 PM, Simon Peyton
>     Jones
>      >>>              | > <simonpj at microsoft.com
>     <mailto:simonpj at microsoft.com> <mailto:simonpj at microsoft.com
>     <mailto:simonpj at microsoft.com>>
>      >>>         <mailto:simonpj at microsoft.com
>     <mailto:simonpj at microsoft.com> <mailto:simonpj at microsoft.com
>     <mailto:simonpj at microsoft.com>>>
>      >>>              <mailto:simonpj at microsoft.com
>     <mailto:simonpj at microsoft.com>
>      >>>         <mailto:simonpj at microsoft.com
>     <mailto:simonpj at microsoft.com>> <mailto:simonpj at microsoft.com
>     <mailto:simonpj at microsoft.com>
>      >>>         <mailto:simonpj at microsoft.com
>     <mailto:simonpj at microsoft.com>>>__>> wrote:
>      >>>              | >
>      >>>              | >     As Austin has told us, there's a draft of
>     the *GHC
>      >>>         Status Report
>      >>>              | for
>      >>>              | >     the HCAR*, here:____
>      >>>              | >
>      >>>              | >
>      >>> https://ghc.haskell.org/trac/__ghc/wiki/Status/May14____
>      >>>         <https://ghc.haskell.org/trac/ghc/wiki/Status/May14____>
>      >>>              | >
>      >>>              | >     Have we missed out something  you have
>     been working
>      >>>         hard on?  Do
>      >>>              | >     take a moment to add a bullet in an
>     appropriate
>      >>>         place (it's a
>      >>>              | >     wiki).  I'd like to be sure that we are giving
>      >>>         credit to all the
>      >>>              | >     appropriate people, so please help us fix
>     that too.
>      >>>           GHC is
>      >>>              a team
>      >>>              | >     effort.____
>      >>>              | >
>      >>>              | >     Deadline is 1 May I think.____
>      >>>              | >
>      >>>              | >     Thanks____
>      >>>              | >
>      >>>              | >     Simon____
>      >>>              | >
>      >>>              | >     __ __
>      >>>              | >
>      >>>              | >
>      >>>              | >
>     _________________________________________________
>      >>>              | >     ghc-devs mailing list
>      >>>              | > ghc-devs at haskell.org
>     <mailto:ghc-devs at haskell.org> <mailto:ghc-devs at haskell.org
>     <mailto:ghc-devs at haskell.org>>
>      >>>         <mailto:ghc-devs at haskell.org
>     <mailto:ghc-devs at haskell.org> <mailto:ghc-devs at haskell.org
>     <mailto:ghc-devs at haskell.org>>>
>      >>>              <mailto:ghc-devs at haskell.org
>     <mailto:ghc-devs at haskell.org> <mailto:ghc-devs at haskell.org
>     <mailto:ghc-devs at haskell.org>>
>      >>>         <mailto:ghc-devs at haskell.org
>     <mailto:ghc-devs at haskell.org> <mailto:ghc-devs at haskell.org
>     <mailto:ghc-devs at haskell.org>>>>
>      >>>
>      >>>              | > http://www.haskell.org/__mailman/listinfo/ghc-devs
>      >>>         <http://www.haskell.org/mailman/listinfo/ghc-devs>
>      >>>              | >
>      >>>              | >
>      >>>              | >
>      >>>              | >
>      >>>              | > _________________________________________________
>      >>>              | > ghc-devs mailing list
>      >>>              | > ghc-devs at haskell.org
>     <mailto:ghc-devs at haskell.org> <mailto:ghc-devs at haskell.org
>     <mailto:ghc-devs at haskell.org>>
>      >>>         <mailto:ghc-devs at haskell.org
>     <mailto:ghc-devs at haskell.org> <mailto:ghc-devs at haskell.org
>     <mailto:ghc-devs at haskell.org>>>
>      >>>              | > http://www.haskell.org/__mailman/listinfo/ghc-devs
>      >>>         <http://www.haskell.org/mailman/listinfo/ghc-devs>
>      >>>              | >
>      >>>              _________________________________________________
>      >>>              ghc-devs mailing list
>      >>> ghc-devs at haskell.org <mailto:ghc-devs at haskell.org>
>     <mailto:ghc-devs at haskell.org <mailto:ghc-devs at haskell.org>>
>      >>>         <mailto:ghc-devs at haskell.org
>     <mailto:ghc-devs at haskell.org> <mailto:ghc-devs at haskell.org
>     <mailto:ghc-devs at haskell.org>>>
>      >>> http://www.haskell.org/__mailman/listinfo/ghc-devs
>      >>>         <http://www.haskell.org/mailman/listinfo/ghc-devs>
>      >>>
>      >>>
>      >>>
>      >>>
>
>



More information about the ghc-devs mailing list