GHC status report
Simon Marlow
marlowsd at gmail.com
Mon May 5 21:13:10 UTC 2014
I think the issue is really "loading static C++ libs into GHCi doesn't
work properly", and that's true of all versions of GHC, including 7.8.
On 05/05/14 22:08, Carter Schonwald wrote:
> i don't have 7.6 setup anymore, i'll see about reproducing the issue
> later. it falls under the umbrella of "linking not working in ghci", But
> it could be that didn't try the -fshared-llvm + ghci in 7.6
>
>
> On Mon, May 5, 2014 at 5:01 PM, Simon Marlow <marlowsd at gmail.com
> <mailto:marlowsd at gmail.com>> wrote:
>
> On 05/05/14 21:58, Carter Schonwald wrote:
>
> no, i was saying LLVM-General when static linked to llvm doesnt
> work. I
> wasnt talking about ghc being dynamic or static
>
>
> I believe you that it doesn't work. But I think that if you use a
> dynamically-linked llvm (i.e. the shared-llvm flag), it *should*
> work, even with 7.6.3. What goes wrong in that case?
>
> Cheers,
> Simon
>
> merely that theres no way to get llvm-general to work on ghci in
> 7.6 afaik
>
>
> On Mon, May 5, 2014 at 4:54 PM, Simon Marlow <marlowsd at gmail.com
> <mailto:marlowsd at gmail.com>
> <mailto:marlowsd at gmail.com <mailto:marlowsd at gmail.com>>> wrote:
>
> I don't see any reason why llvm-general with the
> shared-llvm flag
> shouldn't work with a statically linked GHCi (7.6.3 or 7.8 with
> DYNAMIC_GHC_PROGRAMS=NO). Doesn't it work?
>
>
> On 03/05/14 02:12, Carter Schonwald wrote:
>
> if theres a way i can patch llvm-general so that i can
> use it
> with llvm
> static linked into rather than dylinked, i'm all ears!
>
> llvm-general has to use some C++ wrappers (that in turn use
> extern "C"
> sections) to make parts of the llvm api accessible from
> hasskell.
>
> I had trouble following some of the thread earlier
> today, but is the
> suggestion to try building those wrappers with -fPIC
> would make them
> play nice?
>
>
> On Fri, May 2, 2014 at 8:52 PM, Dominick Samperi
> <djsamperi at gmail.com <mailto:djsamperi at gmail.com>
> <mailto:djsamperi at gmail.com <mailto:djsamperi at gmail.com>>
> <mailto:djsamperi at gmail.com
> <mailto:djsamperi at gmail.com> <mailto:djsamperi at gmail.com
> <mailto:djsamperi at gmail.com>>>> wrote:
>
> I posted a ticket related to this (#8371), but the
> example
> provided
> there
> has no problems today for all versions of ghci that I
> tested (including
> 7.6.3), provided -fno-ghci-sandbox is specified.
> So this
> problem was
> clearly related to the threads issue.
>
> Today there are problems when
> DYNAMIC_GHC_PROGRAMS=NO, but they
> happen later, and I have not yet narrowed this
> down to a
> small example.
> Basically, I have an Haskell app that embeds R (as
> in the
> sample code
> attached to the above ticket), but when it tries
> to do some
> calculations
> it seg faults (works fine with 7.8.2).
>
> On Fri, May 2, 2014 at 3:45 PM, Simon Marlow
> <marlowsd at gmail.com <mailto:marlowsd at gmail.com>
> <mailto:marlowsd at gmail.com <mailto:marlowsd at gmail.com>>
> <mailto:marlowsd at gmail.com
> <mailto:marlowsd at gmail.com> <mailto:marlowsd at gmail.com
> <mailto:marlowsd at gmail.com>>>> wrote:
> > Can you give me a quick summary of how to
> reproduce the
> problem? (not
> > including the GHC build steps)
> >
> > Cheers,
> > Simon
> >
> >
> > On 02/05/14 18:18, Dominick Samperi wrote:
> >>
> >> I downloaded HEAD and placed
> DYNAMIC_GHC_PROGRAMS=NO in
> the "quick"
> >> section of mk/build.mk <http://build.mk>
> <http://build.mk>
> <http://build.mk> (with BuildFlavour =
>
> quick), and set
> >> DYNAMIC_GHC_PROGRAMS=NO
> >> in my environment before running configure
> (just to be
> sure!). Near
> >> the end of the build
> >> I saw some messages like "Warning: vectorization
> failure," but the
> >> build completed.
> >>
> >> The status at the end of configure doesn't say
> that
> dynamic linking
> >> via RTS has been
> >> turned off, and I don't know how to check that
> this is so.
> >> Nevertheless, I checked
> >> the linking issue and it is NOT fixed with
> this build, so
> >> DYNAMIC_GHC_PROGRAMS=YES is required to
> prevent the
> problems
> >> reported by me and others.
> >>
> >>
> >> On Fri, May 2, 2014 at 9:09 AM, Simon Marlow
> <marlowsd at gmail.com <mailto:marlowsd at gmail.com>
> <mailto:marlowsd at gmail.com <mailto:marlowsd at gmail.com>>
> <mailto:marlowsd at gmail.com
> <mailto:marlowsd at gmail.com> <mailto:marlowsd at gmail.com
> <mailto:marlowsd at gmail.com>>>> wrote:
> >>>
> >>> On 02/05/2014 01:09, Dominick Samperi wrote:
> >>>
> >>>> If I understand your last comment correctly
> linking
> to libR should
> >>>> continue to work, even if you revert to static
> linking of Haskell
> >>>> compiled
> >>>> code via RTS linker. Since you say this is
> the way
> things have
> always
> >>>> been, perhaps the real explanation for why
> 7.8 fixed
> the issue
> is that
> >>>> some bug was fixed along the way...
> >>>
> >>>
> >>>
> >>> Indeed. To know for sure we would have to
> test 7.8 with
> >>> DYNAMIC_GHC_PROGRAMS=NO with your setup - is
> there a
> way to do
> that?
> >>>
> >>> Cheers,
> >>> Simon
> >>>
> >>>
> >>>
> >>>> Note that R is a C library, so the C++
> issues that Carter
> mentions are
> >>>> not a factor here.
> >>>>
> >>>> Thanks,
> >>>> Dominick
> >>>>
> >>>> On Thu, May 1, 2014 at 5:27 PM, Simon Marlow
> <marlowsd at gmail.com <mailto:marlowsd at gmail.com>
> <mailto:marlowsd at gmail.com <mailto:marlowsd at gmail.com>>
> <mailto:marlowsd at gmail.com <mailto:marlowsd at gmail.com>
> <mailto:marlowsd at gmail.com <mailto:marlowsd at gmail.com>>>> wrote:
> >>>>>
> >>>>>
> >>>>> On 01/05/14 14:48, Dominick Samperi wrote:
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> The problem with some graphics libraries
> used via
> FFI (and other
> >>>>>> libraries
> >>>>>> that are not thread-safe), if I understand the
> situation
> correctly, is
> >>>>>> that ghci
> >>>>>> forks a thread when it shouldn't, causing some
> programs to
> >>>>>> miscalculate
> >>>>>> the available stack space (because they
> think there
> is only one
> >>>>>> thread).
> >>>>>
> >>>>>
> >>>>>
> >>>>>>
> >>>>>>
> >>>>>> The new dynamic linking support and the flag
> -fno-ghci-sandbox fixes
> >>>>>> this problem, and I would not vote for the
> removal
> of these
> features.
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>> So I understand how -fno-ghci-sandbox avoids
> problems with GUI
> >>>>> libraries
> >>>>> that use thread-local state. But how is
> dynamic linking
> involved here?
> >>>>> What improved in GHC 7.8 relative to 7.6
> for you?
> And could
> you clarify
> >>>>> "miscalculate the available stack space"?
> What needs to
> calculate stack
> >>>>> space? Why? C stack space?
> >>>>>
> >>>>> We can certainly make a smoother experience
> around
> -fno-ghci-sandbox
> >>>>> for
> >>>>> using GUI libraries.
> >>>>>
> >>>>>
> >>>>>> It is not clear to me from Simon's
> original post
> how linking
> to all of
> >>>>>> those C++ libs can continue to work if dynamic
> linking is
> removed
> >>>>>> in 7.10? Perhaps I misunderstand what you
> mean by
> "revert to
> >>>>>> static linking"?
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>> When I say "revert to static linking" I
> mean make
> GHCi static
> linked,
> >>>>> and
> >>>>> have it load Haskell code compiled for static
> linking using
> the RTS
> >>>>> linker
> >>>>> (like it did in 7.6). Foreign libraries
> would still be
> loaded using
> >>>>> the
> >>>>> system linker, as they always have been.
> >>>>>
> >>>>> To be clear, I'm not officially proposing
> that we drop
> dynamic linking,
> >>>>> but
> >>>>> I think it's worthwhile exploring the
> design space
> again,
> given that we
> >>>>> know
> >>>>> dynamic linking has been tougher than we
> expected.
> >>>>>
> >>>>> Cheers,
> >>>>> Simon
> >>>>>
> >>>>>
> >>>>>
> >>>>>> Thanks,
> >>>>>> Dominick
> >>>>>>
> >>>>>>
> >>>>>> On Wed, Apr 30, 2014 at 9:26 PM, George
> Colpitts
> >>>>>> <george.colpitts at gmail.com
> <mailto:george.colpitts at gmail.com>
> <mailto:george.colpitts at gmail.__com
> <mailto:george.colpitts at gmail.com>>
> <mailto:george.colpitts at gmail.
> <mailto:george.colpitts at gmail.>____com
>
> <mailto:george.colpitts at gmail.__com
> <mailto:george.colpitts at gmail.com>>>> wrote:
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> To elaborate, in the past, I had a lot of
> problems
> using
> libraries
> >>>>>>> from
> >>>>>>> the
> >>>>>>> ghci prompt on the Mac but I haven't
> tried recently.
> >>>>>>>
> >>>>>>> As an example, on the web page for the
> book the
> Haskell
> School of
> >>>>>>> Expression
> >>>>>>> it says:
> >>>>>>>
> >>>>>>> Note for OS X users: running graphics
> applications
> from
> GHCi is no
> >>>>>>> longer
> >>>>>>> supported. Instead, one has to compile a
> graphics
> program
> using GHC
> >>>>>>> in
> >>>>>>> order
> >>>>>>> to run it (see example/GMIExamples.lhs for an
> example).
> >>>>>>>
> >>>>>>> I had similar problems using the Yale
> Euterpea music
> program from
> >>>>>>> ghci.
> >>>>>>> When
> >>>>>>> I inquired I was referred to
> >>>>>>>
> https://ghc.haskell.org/trac/____ghc/ticket/4244
> <https://ghc.haskell.org/trac/__ghc/ticket/4244>
> <https://ghc.haskell.org/trac/__ghc/ticket/4244
> <https://ghc.haskell.org/trac/ghc/ticket/4244>>
> >>>>>>> and
> https://ghc.haskell.org/trac/____ghc/ticket/781
> <https://ghc.haskell.org/trac/__ghc/ticket/781>
> <https://ghc.haskell.org/trac/__ghc/ticket/781
> <https://ghc.haskell.org/trac/ghc/ticket/781>>. I see that the
>
> >>>>>>> latter
> >>>>>>> is
> >>>>>>> now scheduled for 7.10.1
> >>>>>>>
> >>>>>>>
> >>>>>>> On Wed, Apr 30, 2014 at 1:45 PM, Simon Marlow
> <marlowsd at gmail.com <mailto:marlowsd at gmail.com>
> <mailto:marlowsd at gmail.com <mailto:marlowsd at gmail.com>>
> <mailto:marlowsd at gmail.com <mailto:marlowsd at gmail.com>
> <mailto:marlowsd at gmail.com <mailto:marlowsd at gmail.com>>>>
>
>
> >>>>>>> wrote:
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> On 30/04/2014 01:35, George Colpitts wrote:
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> It doesn't have anything about the
> dynamic linking
> changes made for
> >>>>>>>>> 7.8.
> >>>>>>>>> I think it's worth mentioning the
> improvements
> we expect
> to get
> >>>>>>>>> from
> >>>>>>>>> that. The highlights of the release
> notes do
> mention it,
> so maybe
> >>>>>>>>> that
> >>>>>>>>> suffices.
> >>>>>>>>>
> >>>>>>>>> In particular, I'm hoping that it is
> going to
> fix a lot
> of problems
> >>>>>>>>> with
> >>>>>>>>> using foreign libraries such as OpenGL from
> ghci. I could
> be wrong
> >>>>>>>>> about
> >>>>>>>>> that though.
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> I'd like to understand more about what those
> problems are.
> As a
> >>>>>>>> data
> >>>>>>>> point, at Facebook we're using static
> linking (I
> compiled
> GHC with
> >>>>>>>> DYNAMIC_GHC_PROGRAMS=NO), we're loading
> upwards of 50
> 3rd-party C++
> >>>>>>>> libraries and one gigantic shared library
> consisting of a
> ton of
> >>>>>>>> in-house
> >>>>>>>> C++ code, together with all our Haskell
> code into
> GHCi,
> and it works
> >>>>>>>> perfectly. The key to using the static
> linker is
> to not
> use it for
> >>>>>>>> C++
> >>>>>>>> code
> >>>>>>>> - you want all your external C++ code in
> shared
> libraries
> and load
> >>>>>>>> those
> >>>>>>>> using the system linker.
> >>>>>>>>
> >>>>>>>> Dynamic linking has been a huge headache
> in GHC,
> and it's
> not clear
> >>>>>>>> that
> >>>>>>>> it's an overall improvement compared
> with the static
> linker. Now
> >>>>>>>> that
> >>>>>>>> 7.8
> >>>>>>>> is out of the way, it's time to have a
> conversation about
> whether we
> >>>>>>>> want to
> >>>>>>>> do dynamic linking again for 7.10, or
> revert to
> static
> linking. I
> >>>>>>>> think
> >>>>>>>> Austin is going to update
> >>>>>>>>
> https://ghc.haskell.org/trac/____ghc/wiki/DynamicGhcPrograms
> <https://ghc.haskell.org/trac/__ghc/wiki/DynamicGhcPrograms>
>
> <https://ghc.haskell.org/trac/__ghc/wiki/DynamicGhcPrograms
> <https://ghc.haskell.org/trac/ghc/wiki/DynamicGhcPrograms>>,
>
> and then
> >>>>>>>> we'll
> >>>>>>>> see
> >>>>>>>> where we stand.
> >>>>>>>>
> >>>>>>>> Cheers,
> >>>>>>>> Simon
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>>
> >>>>>>>>> On Tue, Apr 29, 2014 at 6:13 PM, Simon
> Peyton Jones
> >>>>>>>>> <simonpj at microsoft.com
> <mailto:simonpj at microsoft.com>
> <mailto:simonpj at microsoft.com
> <mailto:simonpj at microsoft.com>> <mailto:simonpj at microsoft.com
> <mailto:simonpj at microsoft.com>
> <mailto:simonpj at microsoft.com
> <mailto:simonpj at microsoft.com>>__>
> <mailto:simonpj at microsoft.com
> <mailto:simonpj at microsoft.com>
> <mailto:simonpj at microsoft.com
> <mailto:simonpj at microsoft.com>> <mailto:simonpj at microsoft.com
> <mailto:simonpj at microsoft.com>
> <mailto:simonpj at microsoft.com
> <mailto:simonpj at microsoft.com>>__>__>> wrote:
> >>>>>>>>>
> >>>>>>>>> As Austin has told us, there's a
> draft of
> the *GHC
> Status
> >>>>>>>>> Report
> >>>>>>>>> for
> >>>>>>>>> the HCAR*, here:____
> >>>>>>>>>
> >>>>>>>>>
> https://ghc.haskell.org/trac/____ghc/wiki/Status/May14____
> <https://ghc.haskell.org/trac/__ghc/wiki/Status/May14____>
>
>
> <https://ghc.haskell.org/trac/__ghc/wiki/Status/May14____
> <https://ghc.haskell.org/trac/ghc/wiki/Status/May14____>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> Have we missed out something
> you have been
> working hard on?
> >>>>>>>>> Do
> >>>>>>>>> take a moment to add a bullet in an
> appropriate
> place (it's
> >>>>>>>>> a
> >>>>>>>>> wiki). I'd like to be sure that
> we are
> giving
> credit to all
> >>>>>>>>> the
> >>>>>>>>> appropriate people, so please
> help us fix
> that
> too. GHC is
> >>>>>>>>> a
> >>>>>>>>> team
> >>>>>>>>> effort.____
> >>>>>>>>>
> >>>>>>>>> Deadline is 1 May I think.____
> >>>>>>>>>
> >>>>>>>>> Thanks____
> >>>>>>>>>
> >>>>>>>>> Simon____
> >>>>>>>>>
> >>>>>>>>> __ __
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> ___________________________________________________
>
> >>>>>>>>> ghc-devs mailing list
> >>>>>>>>> ghc-devs at haskell.org
> <mailto:ghc-devs at haskell.org>
> <mailto:ghc-devs at haskell.org
> <mailto:ghc-devs at haskell.org>> <mailto:ghc-devs at haskell.org
> <mailto:ghc-devs at haskell.org>
> <mailto:ghc-devs at haskell.org
> <mailto:ghc-devs at haskell.org>>>
> <mailto:ghc-devs at haskell.org
> <mailto:ghc-devs at haskell.org> <mailto:ghc-devs at haskell.org
> <mailto:ghc-devs at haskell.org>>
> <mailto:ghc-devs at haskell.org
> <mailto:ghc-devs at haskell.org> <mailto:ghc-devs at haskell.org
> <mailto:ghc-devs at haskell.org>>>__>
>
>
> >>>>>>>>>
> http://www.haskell.org/____mailman/listinfo/ghc-devs
> <http://www.haskell.org/__mailman/listinfo/ghc-devs>
> <http://www.haskell.org/__mailman/listinfo/ghc-devs
> <http://www.haskell.org/mailman/listinfo/ghc-devs>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> ___________________________________________________
> >>>>>>>>> ghc-devs mailing list
> >>>>>>>>> ghc-devs at haskell.org
> <mailto:ghc-devs at haskell.org>
> <mailto:ghc-devs at haskell.org
> <mailto:ghc-devs at haskell.org>> <mailto:ghc-devs at haskell.org
> <mailto:ghc-devs at haskell.org>
> <mailto:ghc-devs at haskell.org
> <mailto:ghc-devs at haskell.org>>>
> >>>>>>>>>
> http://www.haskell.org/____mailman/listinfo/ghc-devs
> <http://www.haskell.org/__mailman/listinfo/ghc-devs>
> <http://www.haskell.org/__mailman/listinfo/ghc-devs
> <http://www.haskell.org/mailman/listinfo/ghc-devs>>
> >>>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> ___________________________________________________
>
> >>>>>>> ghc-devs mailing list
> >>>>>>> ghc-devs at haskell.org
> <mailto:ghc-devs at haskell.org> <mailto:ghc-devs at haskell.org
> <mailto:ghc-devs at haskell.org>>
> <mailto:ghc-devs at haskell.org
> <mailto:ghc-devs at haskell.org> <mailto:ghc-devs at haskell.org
> <mailto:ghc-devs at haskell.org>>>
> >>>>>>>
> http://www.haskell.org/____mailman/listinfo/ghc-devs
> <http://www.haskell.org/__mailman/listinfo/ghc-devs>
> <http://www.haskell.org/__mailman/listinfo/ghc-devs
> <http://www.haskell.org/mailman/listinfo/ghc-devs>>
> >>>>>>>
> >>>>>
> >>>
> >
> ___________________________________________________
>
> ghc-devs mailing list
> ghc-devs at haskell.org <mailto:ghc-devs at haskell.org>
> <mailto:ghc-devs at haskell.org <mailto:ghc-devs at haskell.org>>
> <mailto:ghc-devs at haskell.org
> <mailto:ghc-devs at haskell.org> <mailto:ghc-devs at haskell.org
> <mailto:ghc-devs at haskell.org>>>
> http://www.haskell.org/____mailman/listinfo/ghc-devs
> <http://www.haskell.org/__mailman/listinfo/ghc-devs>
> <http://www.haskell.org/__mailman/listinfo/ghc-devs
> <http://www.haskell.org/mailman/listinfo/ghc-devs>>
>
>
>
>
>
>
More information about the ghc-devs
mailing list