GHC status report

Simon Marlow marlowsd at gmail.com
Mon May 5 21:13:10 UTC 2014


I think the issue is really "loading static C++ libs into GHCi doesn't 
work properly", and that's true of all versions of GHC, including 7.8.

On 05/05/14 22:08, Carter Schonwald wrote:
> i don't have 7.6 setup anymore, i'll see about reproducing the issue
> later. it falls under the umbrella of "linking not working in ghci", But
> it could be that didn't try the -fshared-llvm + ghci in 7.6
>
>
> On Mon, May 5, 2014 at 5:01 PM, Simon Marlow <marlowsd at gmail.com
> <mailto:marlowsd at gmail.com>> wrote:
>
>     On 05/05/14 21:58, Carter Schonwald wrote:
>
>         no, i was saying LLVM-General when static linked to llvm doesnt
>         work. I
>         wasnt talking about ghc being dynamic or static
>
>
>     I believe you that it doesn't work.  But I think that if you use a
>     dynamically-linked llvm (i.e. the shared-llvm flag), it *should*
>     work, even with 7.6.3.  What goes wrong in that case?
>
>     Cheers,
>     Simon
>
>         merely that theres no way to get llvm-general to work on ghci in
>         7.6 afaik
>
>
>         On Mon, May 5, 2014 at 4:54 PM, Simon Marlow <marlowsd at gmail.com
>         <mailto:marlowsd at gmail.com>
>         <mailto:marlowsd at gmail.com <mailto:marlowsd at gmail.com>>> wrote:
>
>              I don't see any reason why llvm-general with the
>         shared-llvm flag
>              shouldn't work with a statically linked GHCi (7.6.3 or 7.8 with
>              DYNAMIC_GHC_PROGRAMS=NO).  Doesn't it work?
>
>
>              On 03/05/14 02:12, Carter Schonwald wrote:
>
>                  if theres a way i can patch llvm-general so that i can
>         use it
>                  with llvm
>                  static linked into rather than dylinked, i'm all ears!
>
>                  llvm-general has to use some C++ wrappers (that in turn use
>                  extern "C"
>                  sections) to make parts of the llvm api accessible from
>         hasskell.
>
>                  I had trouble following some of the thread earlier
>         today, but is the
>                  suggestion to try building those wrappers with -fPIC
>         would make them
>                  play nice?
>
>
>                  On Fri, May 2, 2014 at 8:52 PM, Dominick Samperi
>                  <djsamperi at gmail.com <mailto:djsamperi at gmail.com>
>         <mailto:djsamperi at gmail.com <mailto:djsamperi at gmail.com>>
>                  <mailto:djsamperi at gmail.com
>         <mailto:djsamperi at gmail.com> <mailto:djsamperi at gmail.com
>         <mailto:djsamperi at gmail.com>>>> wrote:
>
>                       I posted a ticket related to this (#8371), but the
>         example
>                  provided
>                       there
>                       has no problems today for all versions of ghci that I
>                  tested (including
>                       7.6.3), provided -fno-ghci-sandbox is specified.
>         So this
>                  problem was
>                       clearly related to the threads issue.
>
>                       Today there are problems when
>         DYNAMIC_GHC_PROGRAMS=NO, but they
>                       happen later, and I have not yet narrowed this
>         down to a
>                  small example.
>                       Basically, I have an Haskell app that embeds R (as
>         in the
>                  sample code
>                       attached to the above ticket), but when it tries
>         to do some
>                  calculations
>                       it seg faults (works fine with 7.8.2).
>
>                       On Fri, May 2, 2014 at 3:45 PM, Simon Marlow
>                  <marlowsd at gmail.com <mailto:marlowsd at gmail.com>
>         <mailto:marlowsd at gmail.com <mailto:marlowsd at gmail.com>>
>                       <mailto:marlowsd at gmail.com
>         <mailto:marlowsd at gmail.com> <mailto:marlowsd at gmail.com
>         <mailto:marlowsd at gmail.com>>>> wrote:
>                        > Can you give me a quick summary of how to
>         reproduce the
>                  problem? (not
>                        > including the GHC build steps)
>                        >
>                        > Cheers,
>                        > Simon
>                        >
>                        >
>                        > On 02/05/14 18:18, Dominick Samperi wrote:
>                        >>
>                        >> I downloaded HEAD and placed
>         DYNAMIC_GHC_PROGRAMS=NO in
>                  the "quick"
>                        >> section of mk/build.mk <http://build.mk>
>         <http://build.mk>
>                  <http://build.mk> (with BuildFlavour =
>
>                       quick), and set
>                        >> DYNAMIC_GHC_PROGRAMS=NO
>                        >> in my environment before running configure
>         (just to be
>                  sure!). Near
>                        >> the end of the build
>                        >> I saw some messages like "Warning: vectorization
>                  failure," but the
>                        >> build completed.
>                        >>
>                        >> The status at the end of configure doesn't say
>         that
>                  dynamic linking
>                        >> via RTS has been
>                        >> turned off, and I don't know how to check that
>         this is so.
>                        >> Nevertheless, I checked
>                        >> the linking issue and it is NOT fixed with
>         this build, so
>                        >> DYNAMIC_GHC_PROGRAMS=YES is required to
>         prevent the
>                  problems
>                        >> reported by me and others.
>                        >>
>                        >>
>                        >> On Fri, May 2, 2014 at 9:09 AM, Simon Marlow
>                  <marlowsd at gmail.com <mailto:marlowsd at gmail.com>
>         <mailto:marlowsd at gmail.com <mailto:marlowsd at gmail.com>>
>                       <mailto:marlowsd at gmail.com
>         <mailto:marlowsd at gmail.com> <mailto:marlowsd at gmail.com
>         <mailto:marlowsd at gmail.com>>>> wrote:
>                        >>>
>                        >>> On 02/05/2014 01:09, Dominick Samperi wrote:
>                        >>>
>                        >>>> If I understand your last comment correctly
>         linking
>                  to libR should
>                        >>>> continue to work, even if you revert to static
>                  linking of Haskell
>                        >>>> compiled
>                        >>>> code via RTS linker. Since you say this is
>         the way
>                  things have
>                       always
>                        >>>> been, perhaps the real explanation for why
>         7.8 fixed
>                  the issue
>                       is that
>                        >>>> some bug was fixed along the way...
>                        >>>
>                        >>>
>                        >>>
>                        >>> Indeed.  To know for sure we would have to
>         test 7.8 with
>                        >>> DYNAMIC_GHC_PROGRAMS=NO with your setup - is
>         there a
>                  way to do
>                       that?
>                        >>>
>                        >>> Cheers,
>                        >>> Simon
>                        >>>
>                        >>>
>                        >>>
>                        >>>> Note that R is a C library, so the C++
>         issues that Carter
>                       mentions are
>                        >>>> not a factor here.
>                        >>>>
>                        >>>> Thanks,
>                        >>>> Dominick
>                        >>>>
>                        >>>> On Thu, May 1, 2014 at 5:27 PM, Simon Marlow
>                       <marlowsd at gmail.com <mailto:marlowsd at gmail.com>
>         <mailto:marlowsd at gmail.com <mailto:marlowsd at gmail.com>>
>                  <mailto:marlowsd at gmail.com <mailto:marlowsd at gmail.com>
>         <mailto:marlowsd at gmail.com <mailto:marlowsd at gmail.com>>>> wrote:
>                        >>>>>
>                        >>>>>
>                        >>>>> On 01/05/14 14:48, Dominick Samperi wrote:
>                        >>>>>>
>                        >>>>>>
>                        >>>>>>
>                        >>>>>> The problem with some graphics libraries
>         used via
>                  FFI (and other
>                        >>>>>> libraries
>                        >>>>>> that are not thread-safe), if I understand the
>                  situation
>                       correctly, is
>                        >>>>>> that ghci
>                        >>>>>> forks a thread when it shouldn't, causing some
>                  programs to
>                        >>>>>> miscalculate
>                        >>>>>> the available stack space (because they
>         think there
>                  is only one
>                        >>>>>> thread).
>                        >>>>>
>                        >>>>>
>                        >>>>>
>                        >>>>>>
>                        >>>>>>
>                        >>>>>> The new dynamic linking support and the flag
>                       -fno-ghci-sandbox fixes
>                        >>>>>> this problem, and I would not vote for the
>         removal
>                  of these
>                       features.
>                        >>>>>
>                        >>>>>
>                        >>>>>
>                        >>>>>
>                        >>>>> So I understand how -fno-ghci-sandbox avoids
>                  problems with GUI
>                        >>>>> libraries
>                        >>>>> that use thread-local state.  But how is
>         dynamic linking
>                       involved here?
>                        >>>>> What improved in GHC 7.8 relative to 7.6
>         for you?
>                  And could
>                       you clarify
>                        >>>>> "miscalculate the available stack space"?
>         What needs to
>                       calculate stack
>                        >>>>> space? Why? C stack space?
>                        >>>>>
>                        >>>>> We can certainly make a smoother experience
>         around
>                       -fno-ghci-sandbox
>                        >>>>> for
>                        >>>>> using GUI libraries.
>                        >>>>>
>                        >>>>>
>                        >>>>>> It is not clear to me from Simon's
>         original post
>                  how linking
>                       to all of
>                        >>>>>> those C++ libs can continue to work if dynamic
>                  linking is
>                       removed
>                        >>>>>> in 7.10? Perhaps I misunderstand what you
>         mean by
>                  "revert to
>                        >>>>>> static linking"?
>                        >>>>>
>                        >>>>>
>                        >>>>>
>                        >>>>>
>                        >>>>> When I say "revert to static linking" I
>         mean make
>                  GHCi static
>                       linked,
>                        >>>>> and
>                        >>>>> have it load Haskell code compiled for static
>                  linking using
>                       the RTS
>                        >>>>> linker
>                        >>>>> (like it did in 7.6).  Foreign libraries
>         would still be
>                       loaded using
>                        >>>>> the
>                        >>>>> system linker, as they always have been.
>                        >>>>>
>                        >>>>> To be clear, I'm not officially proposing
>         that we drop
>                       dynamic linking,
>                        >>>>> but
>                        >>>>> I think it's worthwhile exploring the
>         design space
>                  again,
>                       given that we
>                        >>>>> know
>                        >>>>> dynamic linking has been tougher than we
>         expected.
>                        >>>>>
>                        >>>>> Cheers,
>                        >>>>> Simon
>                        >>>>>
>                        >>>>>
>                        >>>>>
>                        >>>>>> Thanks,
>                        >>>>>> Dominick
>                        >>>>>>
>                        >>>>>>
>                        >>>>>> On Wed, Apr 30, 2014 at 9:26 PM, George
>         Colpitts
>                        >>>>>> <george.colpitts at gmail.com
>         <mailto:george.colpitts at gmail.com>
>                  <mailto:george.colpitts at gmail.__com
>         <mailto:george.colpitts at gmail.com>>
>                       <mailto:george.colpitts at gmail.
>         <mailto:george.colpitts at gmail.>____com
>
>                  <mailto:george.colpitts at gmail.__com
>         <mailto:george.colpitts at gmail.com>>>> wrote:
>                        >>>>>>>
>                        >>>>>>>
>                        >>>>>>>
>                        >>>>>>> To elaborate, in the past, I had a lot of
>         problems
>                  using
>                       libraries
>                        >>>>>>> from
>                        >>>>>>> the
>                        >>>>>>> ghci prompt on the Mac but I haven't
>         tried recently.
>                        >>>>>>>
>                        >>>>>>> As an example, on the web page for the
>         book the
>                  Haskell
>                       School of
>                        >>>>>>> Expression
>                        >>>>>>> it says:
>                        >>>>>>>
>                        >>>>>>> Note for OS X users: running graphics
>         applications
>                  from
>                       GHCi is no
>                        >>>>>>> longer
>                        >>>>>>> supported. Instead, one has to compile a
>         graphics
>                  program
>                       using GHC
>                        >>>>>>> in
>                        >>>>>>> order
>                        >>>>>>> to run it (see example/GMIExamples.lhs for an
>                  example).
>                        >>>>>>>
>                        >>>>>>> I had similar problems using the Yale
>         Euterpea music
>                       program from
>                        >>>>>>> ghci.
>                        >>>>>>> When
>                        >>>>>>> I inquired I was referred to
>                        >>>>>>>
>         https://ghc.haskell.org/trac/____ghc/ticket/4244
>         <https://ghc.haskell.org/trac/__ghc/ticket/4244>
>                  <https://ghc.haskell.org/trac/__ghc/ticket/4244
>         <https://ghc.haskell.org/trac/ghc/ticket/4244>>
>                        >>>>>>> and
>         https://ghc.haskell.org/trac/____ghc/ticket/781
>         <https://ghc.haskell.org/trac/__ghc/ticket/781>
>                  <https://ghc.haskell.org/trac/__ghc/ticket/781
>         <https://ghc.haskell.org/trac/ghc/ticket/781>>. I see that the
>
>                        >>>>>>> latter
>                        >>>>>>> is
>                        >>>>>>> now scheduled for 7.10.1
>                        >>>>>>>
>                        >>>>>>>
>                        >>>>>>> On Wed, Apr 30, 2014 at 1:45 PM, Simon Marlow
>                       <marlowsd at gmail.com <mailto:marlowsd at gmail.com>
>         <mailto:marlowsd at gmail.com <mailto:marlowsd at gmail.com>>
>                  <mailto:marlowsd at gmail.com <mailto:marlowsd at gmail.com>
>         <mailto:marlowsd at gmail.com <mailto:marlowsd at gmail.com>>>>
>
>
>                        >>>>>>> wrote:
>                        >>>>>>>>
>                        >>>>>>>>
>                        >>>>>>>>
>                        >>>>>>>>
>                        >>>>>>>> On 30/04/2014 01:35, George Colpitts wrote:
>                        >>>>>>>>>
>                        >>>>>>>>>
>                        >>>>>>>>>
>                        >>>>>>>>>
>                        >>>>>>>>> It doesn't have anything about the
>         dynamic linking
>                       changes made for
>                        >>>>>>>>> 7.8.
>                        >>>>>>>>> I think it's worth mentioning the
>         improvements
>                  we expect
>                       to get
>                        >>>>>>>>> from
>                        >>>>>>>>> that. The highlights of the release
>         notes do
>                  mention it,
>                       so maybe
>                        >>>>>>>>> that
>                        >>>>>>>>> suffices.
>                        >>>>>>>>>
>                        >>>>>>>>> In particular, I'm hoping that it is
>         going to
>                  fix a lot
>                       of problems
>                        >>>>>>>>> with
>                        >>>>>>>>> using foreign libraries such as OpenGL from
>                  ghci. I could
>                       be wrong
>                        >>>>>>>>> about
>                        >>>>>>>>> that though.
>                        >>>>>>>>
>                        >>>>>>>>
>                        >>>>>>>>
>                        >>>>>>>>
>                        >>>>>>>>
>                        >>>>>>>> I'd like to understand more about what those
>                  problems are.
>                         As a
>                        >>>>>>>> data
>                        >>>>>>>> point, at Facebook we're using static
>         linking (I
>                  compiled
>                       GHC with
>                        >>>>>>>> DYNAMIC_GHC_PROGRAMS=NO), we're loading
>         upwards of 50
>                       3rd-party C++
>                        >>>>>>>> libraries and one gigantic shared library
>                  consisting of a
>                       ton of
>                        >>>>>>>> in-house
>                        >>>>>>>> C++ code, together with all our Haskell
>         code into
>                  GHCi,
>                       and it works
>                        >>>>>>>> perfectly.  The key to using the static
>         linker is
>                  to not
>                       use it for
>                        >>>>>>>> C++
>                        >>>>>>>> code
>                        >>>>>>>> - you want all your external C++ code in
>         shared
>                  libraries
>                       and load
>                        >>>>>>>> those
>                        >>>>>>>> using the system linker.
>                        >>>>>>>>
>                        >>>>>>>> Dynamic linking has been a huge headache
>         in GHC,
>                  and it's
>                       not clear
>                        >>>>>>>> that
>                        >>>>>>>> it's an overall improvement compared
>         with the static
>                       linker.  Now
>                        >>>>>>>> that
>                        >>>>>>>> 7.8
>                        >>>>>>>> is out of the way, it's time to have a
>                  conversation about
>                       whether we
>                        >>>>>>>> want to
>                        >>>>>>>> do dynamic linking again for 7.10, or
>         revert to
>                  static
>                       linking.  I
>                        >>>>>>>> think
>                        >>>>>>>> Austin is going to update
>                        >>>>>>>>
>         https://ghc.haskell.org/trac/____ghc/wiki/DynamicGhcPrograms
>         <https://ghc.haskell.org/trac/__ghc/wiki/DynamicGhcPrograms>
>
>         <https://ghc.haskell.org/trac/__ghc/wiki/DynamicGhcPrograms
>         <https://ghc.haskell.org/trac/ghc/wiki/DynamicGhcPrograms>>,
>
>                       and then
>                        >>>>>>>> we'll
>                        >>>>>>>> see
>                        >>>>>>>> where we stand.
>                        >>>>>>>>
>                        >>>>>>>> Cheers,
>                        >>>>>>>> Simon
>                        >>>>>>>>
>                        >>>>>>>>
>                        >>>>>>>>
>                        >>>>>>>>>
>                        >>>>>>>>> On Tue, Apr 29, 2014 at 6:13 PM, Simon
>         Peyton Jones
>                        >>>>>>>>> <simonpj at microsoft.com
>         <mailto:simonpj at microsoft.com>
>                  <mailto:simonpj at microsoft.com
>         <mailto:simonpj at microsoft.com>> <mailto:simonpj at microsoft.com
>         <mailto:simonpj at microsoft.com>
>                  <mailto:simonpj at microsoft.com
>         <mailto:simonpj at microsoft.com>>__>
>                       <mailto:simonpj at microsoft.com
>         <mailto:simonpj at microsoft.com>
>                  <mailto:simonpj at microsoft.com
>         <mailto:simonpj at microsoft.com>> <mailto:simonpj at microsoft.com
>         <mailto:simonpj at microsoft.com>
>                  <mailto:simonpj at microsoft.com
>         <mailto:simonpj at microsoft.com>>__>__>> wrote:
>                        >>>>>>>>>
>                        >>>>>>>>>        As Austin has told us, there's a
>         draft of
>                  the *GHC
>                       Status
>                        >>>>>>>>> Report
>                        >>>>>>>>> for
>                        >>>>>>>>>        the HCAR*, here:____
>                        >>>>>>>>>
>                        >>>>>>>>>
>         https://ghc.haskell.org/trac/____ghc/wiki/Status/May14____
>         <https://ghc.haskell.org/trac/__ghc/wiki/Status/May14____>
>
>
>         <https://ghc.haskell.org/trac/__ghc/wiki/Status/May14____
>         <https://ghc.haskell.org/trac/ghc/wiki/Status/May14____>>
>                        >>>>>>>>>
>                        >>>>>>>>>
>                        >>>>>>>>>        Have we missed out something
>           you have been
>                       working hard on?
>                        >>>>>>>>> Do
>                        >>>>>>>>>        take a moment to add a bullet in an
>                  appropriate
>                       place (it's
>                        >>>>>>>>> a
>                        >>>>>>>>>        wiki).  I'd like to be sure that
>         we are
>                  giving
>                       credit to all
>                        >>>>>>>>> the
>                        >>>>>>>>>        appropriate people, so please
>         help us fix
>                  that
>                       too.  GHC is
>                        >>>>>>>>> a
>                        >>>>>>>>> team
>                        >>>>>>>>>        effort.____
>                        >>>>>>>>>
>                        >>>>>>>>>        Deadline is 1 May I think.____
>                        >>>>>>>>>
>                        >>>>>>>>>        Thanks____
>                        >>>>>>>>>
>                        >>>>>>>>>        Simon____
>                        >>>>>>>>>
>                        >>>>>>>>>        __ __
>                        >>>>>>>>>
>                        >>>>>>>>>
>                        >>>>>>>>>
>                    ___________________________________________________
>
>                        >>>>>>>>>        ghc-devs mailing list
>                        >>>>>>>>> ghc-devs at haskell.org
>         <mailto:ghc-devs at haskell.org>
>                  <mailto:ghc-devs at haskell.org
>         <mailto:ghc-devs at haskell.org>> <mailto:ghc-devs at haskell.org
>         <mailto:ghc-devs at haskell.org>
>                  <mailto:ghc-devs at haskell.org
>         <mailto:ghc-devs at haskell.org>>>
>                       <mailto:ghc-devs at haskell.org
>         <mailto:ghc-devs at haskell.org> <mailto:ghc-devs at haskell.org
>         <mailto:ghc-devs at haskell.org>>
>                  <mailto:ghc-devs at haskell.org
>         <mailto:ghc-devs at haskell.org> <mailto:ghc-devs at haskell.org
>         <mailto:ghc-devs at haskell.org>>>__>
>
>
>                        >>>>>>>>>
>         http://www.haskell.org/____mailman/listinfo/ghc-devs
>         <http://www.haskell.org/__mailman/listinfo/ghc-devs>
>                  <http://www.haskell.org/__mailman/listinfo/ghc-devs
>         <http://www.haskell.org/mailman/listinfo/ghc-devs>>
>                        >>>>>>>>>
>                        >>>>>>>>>
>                        >>>>>>>>>
>                        >>>>>>>>>
>                        >>>>>>>>>
>                        >>>>>>>>>
>         ___________________________________________________
>                        >>>>>>>>> ghc-devs mailing list
>                        >>>>>>>>> ghc-devs at haskell.org
>         <mailto:ghc-devs at haskell.org>
>                  <mailto:ghc-devs at haskell.org
>         <mailto:ghc-devs at haskell.org>> <mailto:ghc-devs at haskell.org
>         <mailto:ghc-devs at haskell.org>
>                  <mailto:ghc-devs at haskell.org
>         <mailto:ghc-devs at haskell.org>>>
>                        >>>>>>>>>
>         http://www.haskell.org/____mailman/listinfo/ghc-devs
>         <http://www.haskell.org/__mailman/listinfo/ghc-devs>
>                  <http://www.haskell.org/__mailman/listinfo/ghc-devs
>         <http://www.haskell.org/mailman/listinfo/ghc-devs>>
>                        >>>>>>>>>
>                        >>>>>>>
>                        >>>>>>>
>                        >>>>>>>
>         ___________________________________________________
>
>                        >>>>>>> ghc-devs mailing list
>                        >>>>>>> ghc-devs at haskell.org
>         <mailto:ghc-devs at haskell.org> <mailto:ghc-devs at haskell.org
>         <mailto:ghc-devs at haskell.org>>
>                  <mailto:ghc-devs at haskell.org
>         <mailto:ghc-devs at haskell.org> <mailto:ghc-devs at haskell.org
>         <mailto:ghc-devs at haskell.org>>>
>                        >>>>>>>
>         http://www.haskell.org/____mailman/listinfo/ghc-devs
>         <http://www.haskell.org/__mailman/listinfo/ghc-devs>
>                  <http://www.haskell.org/__mailman/listinfo/ghc-devs
>         <http://www.haskell.org/mailman/listinfo/ghc-devs>>
>                        >>>>>>>
>                        >>>>>
>                        >>>
>                        >
>                       ___________________________________________________
>
>                       ghc-devs mailing list
>         ghc-devs at haskell.org <mailto:ghc-devs at haskell.org>
>         <mailto:ghc-devs at haskell.org <mailto:ghc-devs at haskell.org>>
>                  <mailto:ghc-devs at haskell.org
>         <mailto:ghc-devs at haskell.org> <mailto:ghc-devs at haskell.org
>         <mailto:ghc-devs at haskell.org>>>
>         http://www.haskell.org/____mailman/listinfo/ghc-devs
>         <http://www.haskell.org/__mailman/listinfo/ghc-devs>
>                  <http://www.haskell.org/__mailman/listinfo/ghc-devs
>         <http://www.haskell.org/mailman/listinfo/ghc-devs>>
>
>
>
>
>
>



More information about the ghc-devs mailing list