GHC status report

Dominick Samperi djsamperi at gmail.com
Wed May 7 04:11:48 UTC 2014 

OK, so if I understand correctly, there is no point trying to
narrow down my issue further as it is not supposed to work
with DYNAMIC_GHC_PROGRAMS=NO (because this disables
use of the system linker). Furthermore, if the plan to disable
some form of GHC dynamic linkage is carried out, this should
not affect how external C/C++ libs are linked to (using the
system linker). From other comments in this thread it
appears that there may be problems linking to C++ libs
that require static initialization.


On Mon, May 5, 2014 at 4:58 PM, Simon Marlow <marlowsd at gmail.com> wrote:
> On 03/05/14 04:15, Dominick Samperi wrote:
>>
>> I'm trying to understand the dynamic linking situation with the help of
>> https://ghc.haskell.org/trac/ghc/wiki/Status/GHC-7.8, and according
>> to this information I need to specify DYNAMIC_GHC_PROGRAMS=YES
>> for ghci to use the system linker. I don't understand why you suggested
>> I test with DYNAMIC_GHC_PROGRAMS=NO?
>
>
> The question I'm trying to answer is "what stops working if we use
> DYNAMIC_GHC_PROGRAMS=NO?".  So that's why I'm interested in what happens if
> you set this option.
>
> Thanks for your help!
>
> Cheers,
> Simon
>
>
>
>> Another interesting twist is that my experience under Windows
>> is the opposite of the negative experience described on this
>> page. What I mean by this is more versions of ghci seem to
>> work under Windows (correctly linking to R.dll) than work
>> under Linux!
>>
>>
>> On Fri, May 2, 2014 at 3:45 PM, Simon Marlow <marlowsd at gmail.com> wrote:
>>>
>>> Can you give me a quick summary of how to reproduce the problem? (not
>>> including the GHC build steps)
>>>
>>> Cheers,
>>> Simon
>>>
>>>
>>> On 02/05/14 18:18, Dominick Samperi wrote:
>>>>
>>>>
>>>> I downloaded HEAD and placed DYNAMIC_GHC_PROGRAMS=NO in the "quick"
>>>> section of mk/build.mk (with BuildFlavour = quick), and set
>>>> DYNAMIC_GHC_PROGRAMS=NO
>>>> in my environment before running configure (just to be sure!). Near
>>>> the end of the build
>>>> I saw some messages like "Warning: vectorization failure," but the
>>>> build completed.
>>>>
>>>> The status at the end of configure doesn't say that dynamic linking
>>>> via RTS has been
>>>> turned off, and I don't know how to check that this is so.
>>>> Nevertheless, I checked
>>>> the linking issue and it is NOT fixed with this build, so
>>>> DYNAMIC_GHC_PROGRAMS=YES is required to prevent the problems
>>>> reported by me and others.
>>>>
>>>>
>>>> On Fri, May 2, 2014 at 9:09 AM, Simon Marlow <marlowsd at gmail.com> wrote:
>>>>>
>>>>>
>>>>> On 02/05/2014 01:09, Dominick Samperi wrote:
>>>>>
>>>>>> If I understand your last comment correctly linking to libR should
>>>>>> continue to work, even if you revert to static linking of Haskell
>>>>>> compiled
>>>>>> code via RTS linker. Since you say this is the way things have always
>>>>>> been, perhaps the real explanation for why 7.8 fixed the issue is that
>>>>>> some bug was fixed along the way...
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> Indeed.  To know for sure we would have to test 7.8 with
>>>>> DYNAMIC_GHC_PROGRAMS=NO with your setup - is there a way to do that?
>>>>>
>>>>> Cheers,
>>>>> Simon
>>>>>
>>>>>
>>>>>
>>>>>> Note that R is a C library, so the C++ issues that Carter mentions are
>>>>>> not a factor here.
>>>>>>
>>>>>> Thanks,
>>>>>> Dominick
>>>>>>
>>>>>> On Thu, May 1, 2014 at 5:27 PM, Simon Marlow <marlowsd at gmail.com>
>>>>>> wrote:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On 01/05/14 14:48, Dominick Samperi wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> The problem with some graphics libraries used via FFI (and other
>>>>>>>> libraries
>>>>>>>> that are not thread-safe), if I understand the situation correctly,
>>>>>>>> is
>>>>>>>> that ghci
>>>>>>>> forks a thread when it shouldn't, causing some programs to
>>>>>>>> miscalculate
>>>>>>>> the available stack space (because they think there is only one
>>>>>>>> thread).
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> The new dynamic linking support and the flag -fno-ghci-sandbox fixes
>>>>>>>> this problem, and I would not vote for the removal of these
>>>>>>>> features.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> So I understand how -fno-ghci-sandbox avoids problems with GUI
>>>>>>> libraries
>>>>>>> that use thread-local state.  But how is dynamic linking involved
>>>>>>> here?
>>>>>>> What improved in GHC 7.8 relative to 7.6 for you? And could you
>>>>>>> clarify
>>>>>>> "miscalculate the available stack space"? What needs to calculate
>>>>>>> stack
>>>>>>> space? Why? C stack space?
>>>>>>>
>>>>>>> We can certainly make a smoother experience around -fno-ghci-sandbox
>>>>>>> for
>>>>>>> using GUI libraries.
>>>>>>>
>>>>>>>
>>>>>>>> It is not clear to me from Simon's original post how linking to all
>>>>>>>> of
>>>>>>>> those C++ libs can continue to work if dynamic linking is removed
>>>>>>>> in 7.10? Perhaps I misunderstand what you mean by "revert to
>>>>>>>> static linking"?
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> When I say "revert to static linking" I mean make GHCi static linked,
>>>>>>> and
>>>>>>> have it load Haskell code compiled for static linking using the RTS
>>>>>>> linker
>>>>>>> (like it did in 7.6).  Foreign libraries would still be loaded using
>>>>>>> the
>>>>>>> system linker, as they always have been.
>>>>>>>
>>>>>>> To be clear, I'm not officially proposing that we drop dynamic
>>>>>>> linking,
>>>>>>> but
>>>>>>> I think it's worthwhile exploring the design space again, given that
>>>>>>> we
>>>>>>> know
>>>>>>> dynamic linking has been tougher than we expected.
>>>>>>>
>>>>>>> Cheers,
>>>>>>> Simon
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Dominick
>>>>>>>>
>>>>>>>>
>>>>>>>> On Wed, Apr 30, 2014 at 9:26 PM, George Colpitts
>>>>>>>> <george.colpitts at gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> To elaborate, in the past, I had a lot of problems using libraries
>>>>>>>>> from
>>>>>>>>> the
>>>>>>>>> ghci prompt on the Mac but I haven't tried recently.
>>>>>>>>>
>>>>>>>>> As an example, on the web page for the book the Haskell School of
>>>>>>>>> Expression
>>>>>>>>> it says:
>>>>>>>>>
>>>>>>>>> Note for OS X users: running graphics applications from GHCi is no
>>>>>>>>> longer
>>>>>>>>> supported. Instead, one has to compile a graphics program using GHC
>>>>>>>>> in
>>>>>>>>> order
>>>>>>>>> to run it (see example/GMIExamples.lhs for an example).
>>>>>>>>>
>>>>>>>>> I had similar problems using the Yale Euterpea music program from
>>>>>>>>> ghci.
>>>>>>>>> When
>>>>>>>>> I inquired I was referred to
>>>>>>>>> https://ghc.haskell.org/trac/ghc/ticket/4244
>>>>>>>>> and https://ghc.haskell.org/trac/ghc/ticket/781. I see that the
>>>>>>>>> latter
>>>>>>>>> is
>>>>>>>>> now scheduled for 7.10.1
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Wed, Apr 30, 2014 at 1:45 PM, Simon Marlow <marlowsd at gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On 30/04/2014 01:35, George Colpitts wrote:
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> It doesn't have anything about the dynamic linking changes made
>>>>>>>>>>> for
>>>>>>>>>>> 7.8.
>>>>>>>>>>> I think it's worth mentioning the improvements we expect to get
>>>>>>>>>>> from
>>>>>>>>>>> that. The highlights of the release notes do mention it, so maybe
>>>>>>>>>>> that
>>>>>>>>>>> suffices.
>>>>>>>>>>>
>>>>>>>>>>> In particular, I'm hoping that it is going to fix a lot of
>>>>>>>>>>> problems
>>>>>>>>>>> with
>>>>>>>>>>> using foreign libraries such as OpenGL from ghci. I could be
>>>>>>>>>>> wrong
>>>>>>>>>>> about
>>>>>>>>>>> that though.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> I'd like to understand more about what those problems are.  As a
>>>>>>>>>> data
>>>>>>>>>> point, at Facebook we're using static linking (I compiled GHC with
>>>>>>>>>> DYNAMIC_GHC_PROGRAMS=NO), we're loading upwards of 50 3rd-party
>>>>>>>>>> C++
>>>>>>>>>> libraries and one gigantic shared library consisting of a ton of
>>>>>>>>>> in-house
>>>>>>>>>> C++ code, together with all our Haskell code into GHCi, and it
>>>>>>>>>> works
>>>>>>>>>> perfectly.  The key to using the static linker is to not use it
>>>>>>>>>> for
>>>>>>>>>> C++
>>>>>>>>>> code
>>>>>>>>>> - you want all your external C++ code in shared libraries and load
>>>>>>>>>> those
>>>>>>>>>> using the system linker.
>>>>>>>>>>
>>>>>>>>>> Dynamic linking has been a huge headache in GHC, and it's not
>>>>>>>>>> clear
>>>>>>>>>> that
>>>>>>>>>> it's an overall improvement compared with the static linker.  Now
>>>>>>>>>> that
>>>>>>>>>> 7.8
>>>>>>>>>> is out of the way, it's time to have a conversation about whether
>>>>>>>>>> we
>>>>>>>>>> want to
>>>>>>>>>> do dynamic linking again for 7.10, or revert to static linking.  I
>>>>>>>>>> think
>>>>>>>>>> Austin is going to update
>>>>>>>>>> https://ghc.haskell.org/trac/ghc/wiki/DynamicGhcPrograms, and then
>>>>>>>>>> we'll
>>>>>>>>>> see
>>>>>>>>>> where we stand.
>>>>>>>>>>
>>>>>>>>>> Cheers,
>>>>>>>>>> Simon
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Tue, Apr 29, 2014 at 6:13 PM, Simon Peyton Jones
>>>>>>>>>>> <simonpj at microsoft.com <mailto:simonpj at microsoft.com>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>         As Austin has told us, there's a draft of the *GHC Status
>>>>>>>>>>> Report
>>>>>>>>>>> for
>>>>>>>>>>>         the HCAR*, here:____
>>>>>>>>>>>
>>>>>>>>>>>         https://ghc.haskell.org/trac/ghc/wiki/Status/May14____
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>         Have we missed out something  you have been working hard
>>>>>>>>>>> on?
>>>>>>>>>>> Do
>>>>>>>>>>>         take a moment to add a bullet in an appropriate place
>>>>>>>>>>> (it's
>>>>>>>>>>> a
>>>>>>>>>>>         wiki).  I'd like to be sure that we are giving credit to
>>>>>>>>>>> all
>>>>>>>>>>> the
>>>>>>>>>>>         appropriate people, so please help us fix that too.  GHC
>>>>>>>>>>> is
>>>>>>>>>>> a
>>>>>>>>>>> team
>>>>>>>>>>>         effort.____
>>>>>>>>>>>
>>>>>>>>>>>         Deadline is 1 May I think.____
>>>>>>>>>>>
>>>>>>>>>>>         Thanks____
>>>>>>>>>>>
>>>>>>>>>>>         Simon____
>>>>>>>>>>>
>>>>>>>>>>>         __ __
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>         _______________________________________________
>>>>>>>>>>>         ghc-devs mailing list
>>>>>>>>>>>         ghc-devs at haskell.org <mailto:ghc-devs at haskell.org>
>>>>>>>>>>>         http://www.haskell.org/mailman/listinfo/ghc-devs
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> _______________________________________________
>>>>>>>>>>> ghc-devs mailing list
>>>>>>>>>>> ghc-devs at haskell.org
>>>>>>>>>>> http://www.haskell.org/mailman/listinfo/ghc-devs
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> ghc-devs mailing list
>>>>>>>>> ghc-devs at haskell.org
>>>>>>>>> http://www.haskell.org/mailman/listinfo/ghc-devs
>>>>>>>>>
>>>>>>>
>>>>>
>>>
>

More information about the ghc-devs mailing list