GHC status report
Edward Kmett
ekmett at gmail.com
Fri May 2 22:39:27 UTC 2014
Come to think of it the static initializer we were trying _was_ being glued
into the haskell package, not the external lib where it might have actually
worked so that may be where my confusion came in.
-Edward
On Fri, May 2, 2014 at 3:44 PM, Simon Marlow <marlowsd at gmail.com> wrote:
> On 02/05/14 18:19, Edward Kmett wrote:
>
>> I may have to dig to find an example, but when I last checked it seemed
>> that c++ libraries would load fine, but there was a problem with static
>> initializers not getting called, when loading from ghci, so if your c++
>> library needed them then you'd have problems.
>>
>
> I think we're getting confused here. External C++ libraries are, and
> always have been, loaded by the system linker, regardless of whether GHCi
> itself is dynamically linked. So there's no difference in 7.8 with respect
> to how external C++ code is loaded.
>
> However, if you link the C++ code into the same library as the Haskell
> code as you're doing with MPFR, then of course there is a difference (let's
> call that *internal* C++ code, for want of a better name). I think that's
> a rare case (I understand why you need to do it), and most people should be
> linking their C++ code via separate shared libraries.
>
>
> An earlier version of the MPFR hackery used a static initializer to
>> replace the GHC garbage collection hook for GMP with one that played
>> evil games to figure out if it was being called from the MPFR constant
>> cache.
>>
>> That initialized and loaded fine from ghc, but not from ghci. Our static
>> initializer was never being called, despite the library being loaded.
>>
>> If your C++ practices there forbid static initializers -- some places do
>> -- that may be why you aren't seeing the issue.
>>
>
> Absolutely not, we're drowning in static initialisers :)
>
> Cheers,
> Simon
>
>
> -Edward
>>
>>
>> On Fri, May 2, 2014 at 9:47 AM, Simon Marlow <marlowsd at gmail.com
>> <mailto:marlowsd at gmail.com>> wrote:
>>
>> On 02/05/2014 14:28, Edward Kmett wrote:
>> > Perhaps. We actually tried that originally, but had issues about
>> > where and how to get cabal to place it. We'd need it to go
>> somewhere
>> > installed rather than the local build dir lest it not be there when
>> > we go to use the lib, but IIRC, cabal couldn't/wouldn't tell me
>> where
>> > it was putting the final installed version and then there is the
>> > issue of the local in place runs vs. post cabal install runs and
>> > referencing that dir from subsequent projects. If we want a
>> > transparent ’cabal install rounded` It isn't clear to me how to get
>> > there down this path, but it is entirely possible I just missed
>> > something obvious.
>> >
>> > MPFR/rounded was just an off the cuff example. Another place where
>> > the dynamic linker really helps is external c++ libraries which
>> > should now actually get all of their initializers called in the
>> right
>> > order when launched from ghci.
>>
>> External C++ libraries work just fine with the statically linked GHCi
>> -
>> it uses the system linker to load them, and all the initalizers get
>> called, in the right order, as they should. So this is where I'm
>> confused about what the problems actually are - there seems to be this
>> perception that GHCi didn't work with external C++ code, but as far
>> as I
>> know it works just fine (indeed we're doing it a lot at Facebook, so
>> there's at least an existence proof that depending on a non-trivial
>> amount of external C++ with a statically-linked GHCi can work).
>>
>> Cheers,
>> Simon
>>
>>
>> >
>> >
>> >> Cheers,
>> >> Simon
>> >>
>> >>> Switching to the system dynamic linker fo ghci seems to have
>> resolved
>> >>> all of that effortlessly.
>> >>>
>> >>> Dan Peebles has been talking to the MPFR folks to see if we can
>> get them
>> >>> to expose enough information about the 'hidden' allocations
>> they use
>> >>> that we can make them visible to GHC or have them do what our
>> local fix
>> >>> does and avoid using the MPFR allocator for their hidden
>> constant cache.
>> >>>
>> >>> If they do that then we can actually link to the library like
>> normal
>> >>> rather than link it in directly, but it isn't clear to me what
>> would
>> >>> happen even with those hooks if we rolled back to something
>> like the old
>> >>> custom linker.
>> >>>
>> >>> -Edward
>> >>>
>> >>>
>> >>> On Thu, May 1, 2014 at 5:30 PM, Simon Marlow
>> <marlowsd at gmail.com <mailto:marlowsd at gmail.com>
>> >>> <mailto:marlowsd at gmail.com <mailto:marlowsd at gmail.com>>> wrote:
>> >>>
>> >>> On 01/05/14 15:27, Edward Kmett wrote:
>> >>>
>> >>> Figured I'd make one case for dynamic linking:
>> >>>
>> >>> https://github.com/ekmett/__rounded
>> >>> <https://github.com/ekmett/rounded>
>> >>>
>> >>> Dynamic linking is finally enabling us to build a
>> version of MPFR
>> >>> bindings for Haskell for scientific/high precision
>> computing
>> >>> with 7.8. I
>> >>> would really hate to lose it after all of these years
>> trying to
>> >>> get it
>> >>> work, as I have a rather large edifice being built atop
>> that
>> >>> platform.
>> >>> We tried and failed due to limitations of the old
>> linker for
>> >>> almost 3 years.
>> >>>
>> >>>
>> >>> I understand the issues with MPFR. But how is dynamic
>> linking helping?
>> >>>
>> >>>
>> >>> That said, -dynamic-too seems to cause me all sorts of
>> problems
>> >>> elsewhere. ^C'ing out of a build and restarting it will
>> often
>> >>> make a .o
>> >>> but lose the .dyn_o, leading to GHC + cabal getting
>> confused and
>> >>> refusing to build until I clean. This hits me several
>> times a day.
>> >>>
>> >>>
>> >>> We should fix this (or at least make it a lot less likely).
>> Is
>> >>> there a ticket?
>> >>>
>> >>> Cheers,
>> >>> Simon
>> >>>
>> >>>
>> >>> -Edward
>> >>>
>> >>>
>> >>> On Thu, May 1, 2014 at 3:29 AM, Simon Peyton Jones
>> >>> <simonpj at microsoft.com <mailto:simonpj at microsoft.com>
>> <mailto:simonpj at microsoft.com <mailto:simonpj at microsoft.com>>
>> >>> <mailto:simonpj at microsoft.com
>> <mailto:simonpj at microsoft.com> <mailto:simonpj at microsoft.com
>> <mailto:simonpj at microsoft.com>>>__>
>> >>> wrote:
>> >>>
>> >>> | Dynamic linking has been a huge headache in GHC,
>> and it's not
>> >>> clear that
>> >>> | it's an overall improvement compared with the
>> static
>> >>> linker. Now that
>> >>> | 7.8 is out of the way, it's time to have a
>> conversation about
>> >>> whether we
>> >>> | want to do dynamic linking again for 7.10, or
>> revert to
>> >>> static
>> >>> linking.
>> >>>
>> >>> I echo this. Dynamic linking has had many
>> un-anticipated
>> >>> costs and
>> >>> it is still very far from sorted out. It
>> originally felt
>> >>> like a
>> >>> Fantastic Idea to give up our own linker and adopt
>> the system
>> >>> linker, but it now feels to me like a black hole,
>> endlessly
>> >>> sucking
>> >>> effort and increasing complexity.
>> >>>
>> >>> My viewpoint is highly un-informed about details;
>> I just
>> >>> watch the
>> >>> traffic going by. And of course it does have
>> benefits that
>> >>> doubtless generate less traffic.
>> >>>
>> >>> Simon
>> >>>
>> >>> |
>> >>> |
>> >>> |
>> >>> | >
>> >>> | > On Tue, Apr 29, 2014 at 6:13 PM, Simon Peyton
>> Jones
>> >>> | > <simonpj at microsoft.com
>> <mailto:simonpj at microsoft.com> <mailto:simonpj at microsoft.com
>> <mailto:simonpj at microsoft.com>>
>> >>> <mailto:simonpj at microsoft.com
>> <mailto:simonpj at microsoft.com> <mailto:simonpj at microsoft.com
>> <mailto:simonpj at microsoft.com>>>
>> >>> <mailto:simonpj at microsoft.com
>> <mailto:simonpj at microsoft.com>
>> >>> <mailto:simonpj at microsoft.com
>> <mailto:simonpj at microsoft.com>> <mailto:simonpj at microsoft.com
>> <mailto:simonpj at microsoft.com>
>> >>> <mailto:simonpj at microsoft.com
>> <mailto:simonpj at microsoft.com>>>__>> wrote:
>> >>> | >
>> >>> | > As Austin has told us, there's a draft of
>> the *GHC
>> >>> Status Report
>> >>> | for
>> >>> | > the HCAR*, here:____
>> >>> | >
>> >>> | >
>> >>> https://ghc.haskell.org/trac/__ghc/wiki/Status/May14____
>> >>> <https://ghc.haskell.org/trac/ghc/wiki/Status/May14____>
>> >>> | >
>> >>> | > Have we missed out something you have
>> been working
>> >>> hard on? Do
>> >>> | > take a moment to add a bullet in an
>> appropriate
>> >>> place (it's a
>> >>> | > wiki). I'd like to be sure that we are
>> giving
>> >>> credit to all the
>> >>> | > appropriate people, so please help us fix
>> that too.
>> >>> GHC is
>> >>> a team
>> >>> | > effort.____
>> >>> | >
>> >>> | > Deadline is 1 May I think.____
>> >>> | >
>> >>> | > Thanks____
>> >>> | >
>> >>> | > Simon____
>> >>> | >
>> >>> | > __ __
>> >>> | >
>> >>> | >
>> >>> | >
>> _________________________________________________
>> >>> | > ghc-devs mailing list
>> >>> | > ghc-devs at haskell.org
>> <mailto:ghc-devs at haskell.org> <mailto:ghc-devs at haskell.org
>> <mailto:ghc-devs at haskell.org>>
>> >>> <mailto:ghc-devs at haskell.org
>> <mailto:ghc-devs at haskell.org> <mailto:ghc-devs at haskell.org
>> <mailto:ghc-devs at haskell.org>>>
>> >>> <mailto:ghc-devs at haskell.org
>> <mailto:ghc-devs at haskell.org> <mailto:ghc-devs at haskell.org
>> <mailto:ghc-devs at haskell.org>>
>> >>> <mailto:ghc-devs at haskell.org
>> <mailto:ghc-devs at haskell.org> <mailto:ghc-devs at haskell.org
>> <mailto:ghc-devs at haskell.org>>>>
>> >>>
>> >>> | > http://www.haskell.org/__
>> mailman/listinfo/ghc-devs
>> >>> <http://www.haskell.org/mailman/listinfo/ghc-devs>
>> >>> | >
>> >>> | >
>> >>> | >
>> >>> | >
>> >>> | > ______________________________
>> ___________________
>> >>> | > ghc-devs mailing list
>> >>> | > ghc-devs at haskell.org
>> <mailto:ghc-devs at haskell.org> <mailto:ghc-devs at haskell.org
>> <mailto:ghc-devs at haskell.org>>
>> >>> <mailto:ghc-devs at haskell.org
>> <mailto:ghc-devs at haskell.org> <mailto:ghc-devs at haskell.org
>> <mailto:ghc-devs at haskell.org>>>
>> >>> | > http://www.haskell.org/__
>> mailman/listinfo/ghc-devs
>> >>> <http://www.haskell.org/mailman/listinfo/ghc-devs>
>> >>> | >
>> >>> _________________________________________________
>> >>> ghc-devs mailing list
>> >>> ghc-devs at haskell.org <mailto:ghc-devs at haskell.org>
>> <mailto:ghc-devs at haskell.org <mailto:ghc-devs at haskell.org>>
>> >>> <mailto:ghc-devs at haskell.org
>> <mailto:ghc-devs at haskell.org> <mailto:ghc-devs at haskell.org
>> <mailto:ghc-devs at haskell.org>>>
>> >>> http://www.haskell.org/__mailman/listinfo/ghc-devs
>> >>> <http://www.haskell.org/mailman/listinfo/ghc-devs>
>> >>>
>> >>>
>> >>>
>> >>>
>>
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.haskell.org/pipermail/ghc-devs/attachments/20140502/0ba4ecd9/attachment-0001.html>
More information about the ghc-devs
mailing list