[GHC] #14444: Linker limit on OS X Sierra breaks builds for big projects

[GHC]

#14444: Linker limit on OS X Sierra breaks builds for big projects
-------------------------------------+-------------------------------------
        Reporter:  dredozubov        |                Owner:  angerman
            Type:  bug               |               Status:  patch
        Priority:  normal            |            Milestone:
       Component:  Compiler          |              Version:  8.2.1
  (Linking)                          |
      Resolution:                    |             Keywords:
Operating System:  MacOS X           |         Architecture:
                                     |  Unknown/Multiple
 Type of failure:  None/Unknown      |            Test Case:
      Blocked By:                    |             Blocking:
 Related Tickets:                    |  Differential Rev(s):  Phab:D4717
       Wiki Page:                    |
-------------------------------------+-------------------------------------

Comment (by Ben Gamari <ben@…>):

 In [changeset:"b592bd98ff25730bbe3c13d6f62a427df8c78e28/ghc"
 b592bd98/ghc]:
 {{{
 #!CommitTicketReference repository="ghc"
 revision="b592bd98ff25730bbe3c13d6f62a427df8c78e28"
 dead strip dylibs on macOS

 When linking dynamic libraries or executables, we compute the full
 transitive closure over the dependencies, and instruct the linker
 to link all dependencies.  With deep dependency trees the number
 of transitive dependencies can grow quickly.

 macOS since the Sierra release has an upper limit on the load
 command sizes the linker parses when loading dynamic lirbaries.
 As such it is mandatory to keep the number of load commands (and
 their size) small on recent macOS releases.

 An approach that would just link direct dependencies as specified
 by the -package-id flag is insufficient, because GHC can inline
 across packages and the library or executable being linked could
 refer to symbols deep in the dependency tree.

 If we just recursively linked librarys and re-exported their
 symbols, this increases the number of symbols in libraries with
 many dependencies and ultimately puts excessive strain on the
 linker to the point where linking takes a lot longer than even
 the compilation of the modules.

 We can however build a list of symbols from the obejcts we want
 to link, and try to compute the libraries we need to link that
 contain those symbols from the transitive dependency closure.
 Luckily, we don't need to write this ourselves, but can use
 the ld64 `-dead_strip_dylibs` linker flag on macOS to achive
 the same result.  This will link only the libraries that are
 actually referenced, which is usually a small subset of the
 full transitive dependency closure.  As such we should stay
 within the load command size limit for almost all but pathological
 cases.

 Reviewers: bgamari

 Reviewed By: bgamari

 Subscribers: lelf, rwbarton, thomie, carter

 GHC Trac Issues: #14444

 Differential Revision: https://phabricator.haskell.org/D4714
 }}}

-- 
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/14444#comment:23>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler