GHC status report

Dominick Samperi djsamperi at gmail.com
Sat May 3 00:52:52 UTC 2014 

I posted a ticket related to this (#8371), but the example provided there
has no problems today for all versions of ghci that I tested (including
7.6.3), provided -fno-ghci-sandbox is specified. So this problem was
clearly related to the threads issue.

Today there are problems when DYNAMIC_GHC_PROGRAMS=NO, but they
happen later, and I have not yet narrowed this down to a small example.
Basically, I have an Haskell app that embeds R (as in the sample code
attached to the above ticket), but when it tries to do some calculations
it seg faults (works fine with 7.8.2).

On Fri, May 2, 2014 at 3:45 PM, Simon Marlow <marlowsd at gmail.com> wrote:
> Can you give me a quick summary of how to reproduce the problem? (not
> including the GHC build steps)
>
> Cheers,
> Simon
>
>
> On 02/05/14 18:18, Dominick Samperi wrote:
>>
>> I downloaded HEAD and placed DYNAMIC_GHC_PROGRAMS=NO in the "quick"
>> section of mk/build.mk (with BuildFlavour = quick), and set
>> DYNAMIC_GHC_PROGRAMS=NO
>> in my environment before running configure (just to be sure!). Near
>> the end of the build
>> I saw some messages like "Warning: vectorization failure," but the
>> build completed.
>>
>> The status at the end of configure doesn't say that dynamic linking
>> via RTS has been
>> turned off, and I don't know how to check that this is so.
>> Nevertheless, I checked
>> the linking issue and it is NOT fixed with this build, so
>> DYNAMIC_GHC_PROGRAMS=YES is required to prevent the problems
>> reported by me and others.
>>
>>
>> On Fri, May 2, 2014 at 9:09 AM, Simon Marlow <marlowsd at gmail.com> wrote:
>>>
>>> On 02/05/2014 01:09, Dominick Samperi wrote:
>>>
>>>> If I understand your last comment correctly linking to libR should
>>>> continue to work, even if you revert to static linking of Haskell
>>>> compiled
>>>> code via RTS linker. Since you say this is the way things have always
>>>> been, perhaps the real explanation for why 7.8 fixed the issue is that
>>>> some bug was fixed along the way...
>>>
>>>
>>>
>>> Indeed.  To know for sure we would have to test 7.8 with
>>> DYNAMIC_GHC_PROGRAMS=NO with your setup - is there a way to do that?
>>>
>>> Cheers,
>>> Simon
>>>
>>>
>>>
>>>> Note that R is a C library, so the C++ issues that Carter mentions are
>>>> not a factor here.
>>>>
>>>> Thanks,
>>>> Dominick
>>>>
>>>> On Thu, May 1, 2014 at 5:27 PM, Simon Marlow <marlowsd at gmail.com> wrote:
>>>>>
>>>>>
>>>>> On 01/05/14 14:48, Dominick Samperi wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>> The problem with some graphics libraries used via FFI (and other
>>>>>> libraries
>>>>>> that are not thread-safe), if I understand the situation correctly, is
>>>>>> that ghci
>>>>>> forks a thread when it shouldn't, causing some programs to
>>>>>> miscalculate
>>>>>> the available stack space (because they think there is only one
>>>>>> thread).
>>>>>
>>>>>
>>>>>
>>>>>>
>>>>>>
>>>>>> The new dynamic linking support and the flag -fno-ghci-sandbox fixes
>>>>>> this problem, and I would not vote for the removal of these features.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> So I understand how -fno-ghci-sandbox avoids problems with GUI
>>>>> libraries
>>>>> that use thread-local state.  But how is dynamic linking involved here?
>>>>> What improved in GHC 7.8 relative to 7.6 for you? And could you clarify
>>>>> "miscalculate the available stack space"? What needs to calculate stack
>>>>> space? Why? C stack space?
>>>>>
>>>>> We can certainly make a smoother experience around -fno-ghci-sandbox
>>>>> for
>>>>> using GUI libraries.
>>>>>
>>>>>
>>>>>> It is not clear to me from Simon's original post how linking to all of
>>>>>> those C++ libs can continue to work if dynamic linking is removed
>>>>>> in 7.10? Perhaps I misunderstand what you mean by "revert to
>>>>>> static linking"?
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> When I say "revert to static linking" I mean make GHCi static linked,
>>>>> and
>>>>> have it load Haskell code compiled for static linking using the RTS
>>>>> linker
>>>>> (like it did in 7.6).  Foreign libraries would still be loaded using
>>>>> the
>>>>> system linker, as they always have been.
>>>>>
>>>>> To be clear, I'm not officially proposing that we drop dynamic linking,
>>>>> but
>>>>> I think it's worthwhile exploring the design space again, given that we
>>>>> know
>>>>> dynamic linking has been tougher than we expected.
>>>>>
>>>>> Cheers,
>>>>> Simon
>>>>>
>>>>>
>>>>>
>>>>>> Thanks,
>>>>>> Dominick
>>>>>>
>>>>>>
>>>>>> On Wed, Apr 30, 2014 at 9:26 PM, George Colpitts
>>>>>> <george.colpitts at gmail.com> wrote:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> To elaborate, in the past, I had a lot of problems using libraries
>>>>>>> from
>>>>>>> the
>>>>>>> ghci prompt on the Mac but I haven't tried recently.
>>>>>>>
>>>>>>> As an example, on the web page for the book the Haskell School of
>>>>>>> Expression
>>>>>>> it says:
>>>>>>>
>>>>>>> Note for OS X users: running graphics applications from GHCi is no
>>>>>>> longer
>>>>>>> supported. Instead, one has to compile a graphics program using GHC
>>>>>>> in
>>>>>>> order
>>>>>>> to run it (see example/GMIExamples.lhs for an example).
>>>>>>>
>>>>>>> I had similar problems using the Yale Euterpea music program from
>>>>>>> ghci.
>>>>>>> When
>>>>>>> I inquired I was referred to
>>>>>>> https://ghc.haskell.org/trac/ghc/ticket/4244
>>>>>>> and https://ghc.haskell.org/trac/ghc/ticket/781. I see that the
>>>>>>> latter
>>>>>>> is
>>>>>>> now scheduled for 7.10.1
>>>>>>>
>>>>>>>
>>>>>>> On Wed, Apr 30, 2014 at 1:45 PM, Simon Marlow <marlowsd at gmail.com>
>>>>>>> wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On 30/04/2014 01:35, George Colpitts wrote:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> It doesn't have anything about the dynamic linking changes made for
>>>>>>>>> 7.8.
>>>>>>>>> I think it's worth mentioning the improvements we expect to get
>>>>>>>>> from
>>>>>>>>> that. The highlights of the release notes do mention it, so maybe
>>>>>>>>> that
>>>>>>>>> suffices.
>>>>>>>>>
>>>>>>>>> In particular, I'm hoping that it is going to fix a lot of problems
>>>>>>>>> with
>>>>>>>>> using foreign libraries such as OpenGL from ghci. I could be wrong
>>>>>>>>> about
>>>>>>>>> that though.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> I'd like to understand more about what those problems are.  As a
>>>>>>>> data
>>>>>>>> point, at Facebook we're using static linking (I compiled GHC with
>>>>>>>> DYNAMIC_GHC_PROGRAMS=NO), we're loading upwards of 50 3rd-party C++
>>>>>>>> libraries and one gigantic shared library consisting of a ton of
>>>>>>>> in-house
>>>>>>>> C++ code, together with all our Haskell code into GHCi, and it works
>>>>>>>> perfectly.  The key to using the static linker is to not use it for
>>>>>>>> C++
>>>>>>>> code
>>>>>>>> - you want all your external C++ code in shared libraries and load
>>>>>>>> those
>>>>>>>> using the system linker.
>>>>>>>>
>>>>>>>> Dynamic linking has been a huge headache in GHC, and it's not clear
>>>>>>>> that
>>>>>>>> it's an overall improvement compared with the static linker.  Now
>>>>>>>> that
>>>>>>>> 7.8
>>>>>>>> is out of the way, it's time to have a conversation about whether we
>>>>>>>> want to
>>>>>>>> do dynamic linking again for 7.10, or revert to static linking.  I
>>>>>>>> think
>>>>>>>> Austin is going to update
>>>>>>>> https://ghc.haskell.org/trac/ghc/wiki/DynamicGhcPrograms, and then
>>>>>>>> we'll
>>>>>>>> see
>>>>>>>> where we stand.
>>>>>>>>
>>>>>>>> Cheers,
>>>>>>>> Simon
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Tue, Apr 29, 2014 at 6:13 PM, Simon Peyton Jones
>>>>>>>>> <simonpj at microsoft.com <mailto:simonpj at microsoft.com>> wrote:
>>>>>>>>>
>>>>>>>>>        As Austin has told us, there's a draft of the *GHC Status
>>>>>>>>> Report
>>>>>>>>> for
>>>>>>>>>        the HCAR*, here:____
>>>>>>>>>
>>>>>>>>>        https://ghc.haskell.org/trac/ghc/wiki/Status/May14____
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>        Have we missed out something  you have been working hard on?
>>>>>>>>> Do
>>>>>>>>>        take a moment to add a bullet in an appropriate place (it's
>>>>>>>>> a
>>>>>>>>>        wiki).  I'd like to be sure that we are giving credit to all
>>>>>>>>> the
>>>>>>>>>        appropriate people, so please help us fix that too.  GHC is
>>>>>>>>> a
>>>>>>>>> team
>>>>>>>>>        effort.____
>>>>>>>>>
>>>>>>>>>        Deadline is 1 May I think.____
>>>>>>>>>
>>>>>>>>>        Thanks____
>>>>>>>>>
>>>>>>>>>        Simon____
>>>>>>>>>
>>>>>>>>>        __ __
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>        _______________________________________________
>>>>>>>>>        ghc-devs mailing list
>>>>>>>>>        ghc-devs at haskell.org <mailto:ghc-devs at haskell.org>
>>>>>>>>>        http://www.haskell.org/mailman/listinfo/ghc-devs
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> ghc-devs mailing list
>>>>>>>>> ghc-devs at haskell.org
>>>>>>>>> http://www.haskell.org/mailman/listinfo/ghc-devs
>>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> ghc-devs mailing list
>>>>>>> ghc-devs at haskell.org
>>>>>>> http://www.haskell.org/mailman/listinfo/ghc-devs
>>>>>>>
>>>>>
>>>
>

More information about the ghc-devs mailing list