GHC status report
Simon Marlow
marlowsd at gmail.com
Fri May 9 15:45:42 UTC 2014
On 07/05/2014 05:11, Dominick Samperi wrote:
> OK, so if I understand correctly, there is no point trying to
> narrow down my issue further as it is not supposed to work
> with DYNAMIC_GHC_PROGRAMS=NO (because this disables
> use of the system linker).
On the contrary, if the external C++ code that you depend on is in a
shared library or DLL then DYNAMIC_GHC_PROGRAMS=NO should work fine.
> Furthermore, if the plan to disable
> some form of GHC dynamic linkage is carried out, this should
> not affect how external C/C++ libs are linked to (using the
> system linker).
Correct. (but there's no plan at the moment)
> From other comments in this thread it
> appears that there may be problems linking to C++ libs
> that require static initialization.
Only when both
(a) the C++ code is in the same library as the Haskell code
(b) DYNAMIC_GHC_PROGRAMS=NO
And even then, GHC 7.8 has support for running static initializers in
the linker so it might work now.
Cheers,
Simon
>
> On Mon, May 5, 2014 at 4:58 PM, Simon Marlow <marlowsd at gmail.com> wrote:
>> On 03/05/14 04:15, Dominick Samperi wrote:
>>>
>>> I'm trying to understand the dynamic linking situation with the help of
>>> https://ghc.haskell.org/trac/ghc/wiki/Status/GHC-7.8, and according
>>> to this information I need to specify DYNAMIC_GHC_PROGRAMS=YES
>>> for ghci to use the system linker. I don't understand why you suggested
>>> I test with DYNAMIC_GHC_PROGRAMS=NO?
>>
>>
>> The question I'm trying to answer is "what stops working if we use
>> DYNAMIC_GHC_PROGRAMS=NO?". So that's why I'm interested in what happens if
>> you set this option.
>>
>> Thanks for your help!
>>
>> Cheers,
>> Simon
>>
>>
>>
>>> Another interesting twist is that my experience under Windows
>>> is the opposite of the negative experience described on this
>>> page. What I mean by this is more versions of ghci seem to
>>> work under Windows (correctly linking to R.dll) than work
>>> under Linux!
>>>
>>>
>>> On Fri, May 2, 2014 at 3:45 PM, Simon Marlow <marlowsd at gmail.com> wrote:
>>>>
>>>> Can you give me a quick summary of how to reproduce the problem? (not
>>>> including the GHC build steps)
>>>>
>>>> Cheers,
>>>> Simon
>>>>
>>>>
>>>> On 02/05/14 18:18, Dominick Samperi wrote:
>>>>>
>>>>>
>>>>> I downloaded HEAD and placed DYNAMIC_GHC_PROGRAMS=NO in the "quick"
>>>>> section of mk/build.mk (with BuildFlavour = quick), and set
>>>>> DYNAMIC_GHC_PROGRAMS=NO
>>>>> in my environment before running configure (just to be sure!). Near
>>>>> the end of the build
>>>>> I saw some messages like "Warning: vectorization failure," but the
>>>>> build completed.
>>>>>
>>>>> The status at the end of configure doesn't say that dynamic linking
>>>>> via RTS has been
>>>>> turned off, and I don't know how to check that this is so.
>>>>> Nevertheless, I checked
>>>>> the linking issue and it is NOT fixed with this build, so
>>>>> DYNAMIC_GHC_PROGRAMS=YES is required to prevent the problems
>>>>> reported by me and others.
>>>>>
>>>>>
>>>>> On Fri, May 2, 2014 at 9:09 AM, Simon Marlow <marlowsd at gmail.com> wrote:
>>>>>>
>>>>>>
>>>>>> On 02/05/2014 01:09, Dominick Samperi wrote:
>>>>>>
>>>>>>> If I understand your last comment correctly linking to libR should
>>>>>>> continue to work, even if you revert to static linking of Haskell
>>>>>>> compiled
>>>>>>> code via RTS linker. Since you say this is the way things have always
>>>>>>> been, perhaps the real explanation for why 7.8 fixed the issue is that
>>>>>>> some bug was fixed along the way...
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> Indeed. To know for sure we would have to test 7.8 with
>>>>>> DYNAMIC_GHC_PROGRAMS=NO with your setup - is there a way to do that?
>>>>>>
>>>>>> Cheers,
>>>>>> Simon
>>>>>>
>>>>>>
>>>>>>
>>>>>>> Note that R is a C library, so the C++ issues that Carter mentions are
>>>>>>> not a factor here.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Dominick
>>>>>>>
>>>>>>> On Thu, May 1, 2014 at 5:27 PM, Simon Marlow <marlowsd at gmail.com>
>>>>>>> wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On 01/05/14 14:48, Dominick Samperi wrote:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> The problem with some graphics libraries used via FFI (and other
>>>>>>>>> libraries
>>>>>>>>> that are not thread-safe), if I understand the situation correctly,
>>>>>>>>> is
>>>>>>>>> that ghci
>>>>>>>>> forks a thread when it shouldn't, causing some programs to
>>>>>>>>> miscalculate
>>>>>>>>> the available stack space (because they think there is only one
>>>>>>>>> thread).
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> The new dynamic linking support and the flag -fno-ghci-sandbox fixes
>>>>>>>>> this problem, and I would not vote for the removal of these
>>>>>>>>> features.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> So I understand how -fno-ghci-sandbox avoids problems with GUI
>>>>>>>> libraries
>>>>>>>> that use thread-local state. But how is dynamic linking involved
>>>>>>>> here?
>>>>>>>> What improved in GHC 7.8 relative to 7.6 for you? And could you
>>>>>>>> clarify
>>>>>>>> "miscalculate the available stack space"? What needs to calculate
>>>>>>>> stack
>>>>>>>> space? Why? C stack space?
>>>>>>>>
>>>>>>>> We can certainly make a smoother experience around -fno-ghci-sandbox
>>>>>>>> for
>>>>>>>> using GUI libraries.
>>>>>>>>
>>>>>>>>
>>>>>>>>> It is not clear to me from Simon's original post how linking to all
>>>>>>>>> of
>>>>>>>>> those C++ libs can continue to work if dynamic linking is removed
>>>>>>>>> in 7.10? Perhaps I misunderstand what you mean by "revert to
>>>>>>>>> static linking"?
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> When I say "revert to static linking" I mean make GHCi static linked,
>>>>>>>> and
>>>>>>>> have it load Haskell code compiled for static linking using the RTS
>>>>>>>> linker
>>>>>>>> (like it did in 7.6). Foreign libraries would still be loaded using
>>>>>>>> the
>>>>>>>> system linker, as they always have been.
>>>>>>>>
>>>>>>>> To be clear, I'm not officially proposing that we drop dynamic
>>>>>>>> linking,
>>>>>>>> but
>>>>>>>> I think it's worthwhile exploring the design space again, given that
>>>>>>>> we
>>>>>>>> know
>>>>>>>> dynamic linking has been tougher than we expected.
>>>>>>>>
>>>>>>>> Cheers,
>>>>>>>> Simon
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Dominick
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Wed, Apr 30, 2014 at 9:26 PM, George Colpitts
>>>>>>>>> <george.colpitts at gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> To elaborate, in the past, I had a lot of problems using libraries
>>>>>>>>>> from
>>>>>>>>>> the
>>>>>>>>>> ghci prompt on the Mac but I haven't tried recently.
>>>>>>>>>>
>>>>>>>>>> As an example, on the web page for the book the Haskell School of
>>>>>>>>>> Expression
>>>>>>>>>> it says:
>>>>>>>>>>
>>>>>>>>>> Note for OS X users: running graphics applications from GHCi is no
>>>>>>>>>> longer
>>>>>>>>>> supported. Instead, one has to compile a graphics program using GHC
>>>>>>>>>> in
>>>>>>>>>> order
>>>>>>>>>> to run it (see example/GMIExamples.lhs for an example).
>>>>>>>>>>
>>>>>>>>>> I had similar problems using the Yale Euterpea music program from
>>>>>>>>>> ghci.
>>>>>>>>>> When
>>>>>>>>>> I inquired I was referred to
>>>>>>>>>> https://ghc.haskell.org/trac/ghc/ticket/4244
>>>>>>>>>> and https://ghc.haskell.org/trac/ghc/ticket/781. I see that the
>>>>>>>>>> latter
>>>>>>>>>> is
>>>>>>>>>> now scheduled for 7.10.1
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Wed, Apr 30, 2014 at 1:45 PM, Simon Marlow <marlowsd at gmail.com>
>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On 30/04/2014 01:35, George Colpitts wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> It doesn't have anything about the dynamic linking changes made
>>>>>>>>>>>> for
>>>>>>>>>>>> 7.8.
>>>>>>>>>>>> I think it's worth mentioning the improvements we expect to get
>>>>>>>>>>>> from
>>>>>>>>>>>> that. The highlights of the release notes do mention it, so maybe
>>>>>>>>>>>> that
>>>>>>>>>>>> suffices.
>>>>>>>>>>>>
>>>>>>>>>>>> In particular, I'm hoping that it is going to fix a lot of
>>>>>>>>>>>> problems
>>>>>>>>>>>> with
>>>>>>>>>>>> using foreign libraries such as OpenGL from ghci. I could be
>>>>>>>>>>>> wrong
>>>>>>>>>>>> about
>>>>>>>>>>>> that though.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> I'd like to understand more about what those problems are. As a
>>>>>>>>>>> data
>>>>>>>>>>> point, at Facebook we're using static linking (I compiled GHC with
>>>>>>>>>>> DYNAMIC_GHC_PROGRAMS=NO), we're loading upwards of 50 3rd-party
>>>>>>>>>>> C++
>>>>>>>>>>> libraries and one gigantic shared library consisting of a ton of
>>>>>>>>>>> in-house
>>>>>>>>>>> C++ code, together with all our Haskell code into GHCi, and it
>>>>>>>>>>> works
>>>>>>>>>>> perfectly. The key to using the static linker is to not use it
>>>>>>>>>>> for
>>>>>>>>>>> C++
>>>>>>>>>>> code
>>>>>>>>>>> - you want all your external C++ code in shared libraries and load
>>>>>>>>>>> those
>>>>>>>>>>> using the system linker.
>>>>>>>>>>>
>>>>>>>>>>> Dynamic linking has been a huge headache in GHC, and it's not
>>>>>>>>>>> clear
>>>>>>>>>>> that
>>>>>>>>>>> it's an overall improvement compared with the static linker. Now
>>>>>>>>>>> that
>>>>>>>>>>> 7.8
>>>>>>>>>>> is out of the way, it's time to have a conversation about whether
>>>>>>>>>>> we
>>>>>>>>>>> want to
>>>>>>>>>>> do dynamic linking again for 7.10, or revert to static linking. I
>>>>>>>>>>> think
>>>>>>>>>>> Austin is going to update
>>>>>>>>>>> https://ghc.haskell.org/trac/ghc/wiki/DynamicGhcPrograms, and then
>>>>>>>>>>> we'll
>>>>>>>>>>> see
>>>>>>>>>>> where we stand.
>>>>>>>>>>>
>>>>>>>>>>> Cheers,
>>>>>>>>>>> Simon
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Tue, Apr 29, 2014 at 6:13 PM, Simon Peyton Jones
>>>>>>>>>>>> <simonpj at microsoft.com <mailto:simonpj at microsoft.com>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> As Austin has told us, there's a draft of the *GHC Status
>>>>>>>>>>>> Report
>>>>>>>>>>>> for
>>>>>>>>>>>> the HCAR*, here:____
>>>>>>>>>>>>
>>>>>>>>>>>> https://ghc.haskell.org/trac/ghc/wiki/Status/May14____
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Have we missed out something you have been working hard
>>>>>>>>>>>> on?
>>>>>>>>>>>> Do
>>>>>>>>>>>> take a moment to add a bullet in an appropriate place
>>>>>>>>>>>> (it's
>>>>>>>>>>>> a
>>>>>>>>>>>> wiki). I'd like to be sure that we are giving credit to
>>>>>>>>>>>> all
>>>>>>>>>>>> the
>>>>>>>>>>>> appropriate people, so please help us fix that too. GHC
>>>>>>>>>>>> is
>>>>>>>>>>>> a
>>>>>>>>>>>> team
>>>>>>>>>>>> effort.____
>>>>>>>>>>>>
>>>>>>>>>>>> Deadline is 1 May I think.____
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks____
>>>>>>>>>>>>
>>>>>>>>>>>> Simon____
>>>>>>>>>>>>
>>>>>>>>>>>> __ __
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>> ghc-devs mailing list
>>>>>>>>>>>> ghc-devs at haskell.org <mailto:ghc-devs at haskell.org>
>>>>>>>>>>>> http://www.haskell.org/mailman/listinfo/ghc-devs
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>> ghc-devs mailing list
>>>>>>>>>>>> ghc-devs at haskell.org
>>>>>>>>>>>> http://www.haskell.org/mailman/listinfo/ghc-devs
>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> _______________________________________________
>>>>>>>>>> ghc-devs mailing list
>>>>>>>>>> ghc-devs at haskell.org
>>>>>>>>>> http://www.haskell.org/mailman/listinfo/ghc-devs
>>>>>>>>>>
>>>>>>>>
>>>>>>
>>>>
>>
More information about the ghc-devs
mailing list