[GHC] #13828: Re-linking is not avoided when -dynamic is used
GHC
ghc-devs at haskell.org
Thu Jun 15 00:46:05 UTC 2017
#13828: Re-linking is not avoided when -dynamic is used
-------------------------------------+-------------------------------------
Reporter: nh2 | Owner: (none)
Type: bug | Status: new
Priority: normal | Milestone:
Component: Compiler | Version: 8.0.2
(Linking) |
Resolution: | Keywords:
Operating System: Unknown/Multiple | Architecture:
| Unknown/Multiple
Type of failure: None/Unknown | Test Case:
Blocked By: | Blocking:
Related Tickets: | Differential Rev(s):
Wiki Page: |
-------------------------------------+-------------------------------------
Comment (by nh2):
When using `cabal build -v`, we see these ghc invocations:
{{{
Component build order: library, executable 'cabal-relink-test'
creating dist/build
creating dist/build/autogen
Building cabal-relink-test-0.1.0.0...
/path/to/ghc-8.0.2-with-packages/bin/ghc-pkg init
dist/package.conf.inplace
Preprocessing library cabal-relink-test-0.1.0.0...
Building library...
creating dist/build
/path/to/ghc-8.0.2-with-packages/bin/ghc --make -fbuilding-cabal-package
-O -j4 -static -dynamic-too -dynosuf dyn_o -dynhisuf dyn_hi -outputdir
dist/build -odir dist/build -hidir dist/build -stubdir dist/build -i
-idist/build -isrc -idist/build/autogen -Idist/build/autogen -Idist/build
-optP-include -optPdist/build/autogen/cabal_macros.h -this-unit-id cabal-
relink-test-0.1.0.0-9KTeoqstB5uArQE6VcRCGn -hide-all-packages -package-db
dist/package.conf.inplace -package-id base-4.9.1.0 -XHaskell2010 Mymodule
Linking...
[(SimpleUnitId (ComponentId "base-4.9.1.0"),PackageIdentifier {pkgName =
PackageName {unPackageName = "base"}, pkgVersion = Version {versionBranch
=
[4,9,1,0], versionTags = []}},ModuleRenaming True [])]
/path/to/binutils-2.27/bin/ar -r dist/build/objs-18165/libHScabal-relink-
test-0.1.0.0-9KTeoqstB5uArQE6VcRCGn.a dist/build/Mymodule.o
/path/to/binutils-2.27/bin/ar: creating dist/build/objs-18165/libHScabal-
relink-test-0.1.0.0-9KTeoqstB5uArQE6VcRCGn.a
/path/to/ghc-8.0.2-with-packages/bin/ghc -shared -dynamic '-dynload
deploy' -optl-Wl,-rpath,/path/to/ghc-8.0.2/lib/ghc-8.0.2/base-4.9.1.0
-optl-Wl,-rpath,/path/to/ghc-8.0.2/lib/ghc-8.0.2/ghc-prim-0.5.0.0 -optl-
Wl,-rpath,/path/to/ghc-8.0.2/lib/ghc-8.0.2/integer-gmp-1.0.0.1 -optl-
Wl,-rpath,/path/to/gmp-6.1.1/lib -optl-
Wl,-rpath,/path/to/ghc-8.0.2/lib/ghc-8.0.2/rts -hide-all-packages -no-
auto-link-packages -package-db dist/package.conf.inplace -package-id
base-4.9.1.0 dist/build/Mymodule.dyn_o -o dist/build/libHScabal-relink-
test-0.1.0.0-9KTeoqstB5uArQE6VcRCGn-ghc8.0.2.so
/path/to/ghc-8.0.2-with-packages/bin/ghc-pkg update - --global --user
'--package-db=dist/package.conf.inplace'
Preprocessing executable 'cabal-relink-test' for cabal-relink-
test-0.1.0.0...
Building executable cabal-relink-test...
creating dist/build/cabal-relink-test
creating dist/build/cabal-relink-test/cabal-relink-test-tmp
/path/to/ghc-8.0.2-with-packages/bin/ghc --make -no-link -fbuilding-cabal-
package -O -j4 -static -outputdir dist/build/cabal-relink-test/cabal-
relink-test-tmp -odir dist/build/cabal-relink-test/cabal-relink-test-tmp
-hidir dist/build/cabal-relink-test/cabal-relink-test-tmp -stubdir
dist/build/cabal-relink-test/cabal-relink-test-tmp -i -idist/build/cabal-
relink-test/cabal-relink-test-tmp -iapp -idist/build/autogen
-Idist/build/autogen -Idist/build/cabal-relink-test/cabal-relink-test-tmp
-optP-include -optPdist/build/autogen/cabal_macros.h -hide-all-packages
-package-db dist/package.conf.inplace -package-id base-4.9.1.0 -package-id
cabal-relink-test-0.1.0.0-9KTeoqstB5uArQE6VcRCGn -XHaskell2010 app/Main.hs
-dynamic
Linking...
/path/to/ghc-8.0.2-with-packages/bin/ghc --make -fbuilding-cabal-package
-O -static -outputdir dist/build/cabal-relink-test/cabal-relink-test-tmp
-odir dist/build/cabal-relink-test/cabal-relink-test-tmp -hidir dist/build
/cabal-relink-test/cabal-relink-test-tmp -stubdir dist/build/cabal-relink-
test/cabal-relink-test-tmp -i -idist/build/cabal-relink-test/cabal-relink-
test-tmp -iapp -idist/build/autogen -Idist/build/autogen -Idist/build
/cabal-relink-test/cabal-relink-test-tmp -optP-include
-optPdist/build/autogen/cabal_macros.h -hide-all-packages -package-db
dist/package.conf.inplace -package-id base-4.9.1.0 -package-id cabal-
relink-test-0.1.0.0-9KTeoqstB5uArQE6VcRCGn -XHaskell2010 app/Main.hs -o
dist/build/cabal-relink-test/cabal-relink-test -dynamic
Linking dist/build/cabal-relink-test/cabal-relink-test ...
}}}
The following 2 GHC invocations are of interest:
1. Building the shared library `.so` file for the cabal `library`:
`ghc -shared -dynamic '-dynload deploy' -optl-
Wl,-rpath,/path/to/lib/ghc-8.0.2/base-4.9.1.0 -optl-
Wl,-rpath,/path/to/lib/ghc-8.0.2/ghc-prim-0.5.0.0 -optl-
Wl,-rpath,/path/to/lib/ghc-8.0.2/integer-gmp-1.0.0.1 -optl-
Wl,-rpath,/path/to/gmp-6.1.1/lib -optl-
Wl,-rpath,/path/to/lib/ghc-8.0.2/rts -hide-all-packages -no-auto-link-
packages -package-db dist/package.conf.inplace -package-id base-4.9.1.0
dist/build/Mymodule.dyn_o -o dist/build/libHScabal-relink-test-0.1.0.0
-9KTeoqstB5uArQE6VcRCGn-ghc8.0.2.so`
2. Building the dynamically linked executable:
`ghc --make -fbuilding-cabal-package -O -static -outputdir dist/build
/cabal-relink-test/cabal-relink-test-tmp -odir dist/build/cabal-relink-
test/cabal-relink-test-tmp -hidir dist/build/cabal-relink-test/cabal-
relink-test-tmp -stubdir dist/build/cabal-relink-test/cabal-relink-test-
tmp -i -idist/build/cabal-relink-test/cabal-relink-test-tmp -iapp
-idist/build/autogen -Idist/build/autogen -Idist/build/cabal-relink-test
/cabal-relink-test-tmp -optP-include
-optPdist/build/autogen/cabal_macros.h -hide-all-packages -package-db
dist/package.conf.inplace -package-id base-4.9.1.0 -package-id cabal-
relink-test-0.1.0.0-9KTeoqstB5uArQE6VcRCGn -XHaskell2010 app/Main.hs -o
dist/build/cabal-relink-test/cabal-relink-test -rtsopts -dynamic`
The first invocation always touches (updates the mtime of)
`dist/build/libHScabal-relink-test-0.1.0.0-9KTeoqstB5uArQE6VcRCGn-
ghc8.0.2.so`
and when the second one sees the mtime updated, it relinks.
From a quick look at the code, I think the reason is this:
In `linkingNeeded` in
https://github.com/ghc/ghc/blob/ghc-8.0.2-release/compiler/main/DriverPipeline.hs#L420-L427
{{{
let pkg_hslibs = [ (collectLibraryPaths dflags [c], lib)
| Just c <- map (lookupPackage dflags) pkg_deps,
lib <- packageHsLibs dflags c ]
pkg_libfiles <- mapM (uncurry (findHSLib dflags)) pkg_hslibs
if any isNothing pkg_libfiles then return True else do
e_lib_times <- mapM (tryIO . getModificationUTCTime) (catMaybes
pkg_libfiles)
}}}
I suspect that when linking statically, `pkg_libfiles` is empty, and when
linking dynamically, it contains the above mentioned `.so` file; then, if
the `.so` file was touched `linkingNeeded` returns `True`, and it relinks.
Now the question is: Why does the mtime of the `.so` file change at all?
It's simply because ghc unconditionally invokes `ld` (through `gcc`, or
more precisely, whatever is configured as the linker program `pgm_l`) here
in `linkDynLib`:
https://github.com/ghc/ghc/blob/ghc-8.0.2-release/compiler/main/SysTools.hs#L1720-L1724
{{{
runLink dflags (
map Option verbFlags
++ [ Option "-o"
, FileOption "" output_fn
, Option "-shared"
] ++ ...
...
}}}
And `ld` will always touch the `-o` output file.
What's not clear to me is where the chain should have ended:
* Should `linkDynLib` have not been called in the first place?
* Or should `linkDynLib` only touch the file when the contents change
(e.g. by writing it somewhere else, and doing a `mv` after a contents
comparison; cabal does that in a couple places)?
--
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/13828#comment:1>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler
More information about the ghc-tickets
mailing list