Observation on Hadrian's relative performance re current buildsystem
Simon Peyton Jones
simonpj at microsoft.com
Fri Nov 17 13:23:37 UTC 2017
Urk! I expected Hadrian to be faster because it has more accurate dependencies.
Simon
| -----Original Message-----
| From: ghc-devs [mailto:ghc-devs-bounces at haskell.org] On Behalf Of
| Herbert Valerio Riedel
| Sent: 17 November 2017 13:08
| To: ghc-devs at haskell.org
| Subject: Observation on Hadrian's relative performance re current
| buildsystem
|
| Hello GHC devs,
|
| I took the opportunity to give Hadrian a test-run to see whether it
| could live up to the big promise of delivering a "more scalable,
| faster"
| system than the current GNU Make based system. Unfortunately, my
| preliminary results don't back this claim, and actually make Hadrian
| appear to be significantly slower.
|
| ----
|
| Here's the summary of the results:
|
| | Hadrian | GNU Make |
| +------------------------+-----------+-----------+
| | Compiling `hadrian` | 5m25s | 0 |
| | (one-time setup) | | |
| +------------------------+-----------+-----------+
| | build "all" at -j8 | 38m | 33m |
| +------------------------+-----------+-----------+
| | no-op build at -j8 | 10.977s | 3.258s |
| +------------------------+-----------+-----------+
| | "clean" | 21s | 51s |
| +------------------------+-----------+-----------+
|
|
| So, Hadrian is ~5 minutes than GNU Make (or even ~10 minutes if you
| also count the one-time setup cost).
|
| And what I personally consider a bit annoying is that it's ~3 times
| slower detecting; i.e. you have to wait 11s for Hadrian to detect
| there's nothing to be done which compared to GNU Make (which currently
| needlessly re-runs Sphinx; so it could be even faster!) is very
| noticeable to me.
|
| There's a silver-lining though, deleting files is the part which is a
| lot more costly in the GNU Make system currently since artifacts are
| spread over several (scroll to the end of this email) subfolders
| there. Whereas Hadrian did something we should have done for the GNU
| Make system as well (and probably would have done sooner or later
| anyway in order to support the srcdir!=buildir scheme that people are
| used to from GNU Autotools projects); Hadrian places build-artifacts
| into a few top-level folders, and so cleaning up is trivial and
| requires to unlink only a few folders from the filesystem.
|
| ----
|
| At the very least, I'd expect Hadrian to be as fast as the GNU Make
| system (and ideally beat it, not the least as this was besides
| maintainability its big promise), but so far it doesn't seem to
| deliver that promise for me.
|
| It could easily be that I'm comparing apples to oranges here or that
| I've otherwise overlooked something, so let me describe how I came to
| this conclusion:
|
| I tried this on an reasonably idle Linux workstation with an Intel(R)
| Core(TM) i7-3770 CPU @ 3.40GHz CPU, and with 32GiB RAM (i.e. the
| filesystem content was well-cached into memory; NB: a ghc source tree
| + compiled artifacts takes up about 4GiB on the filesystem).
|
| I've started from a fresh Git clone, i.e.
|
| git clone --recursive git://git.haskell.org/ghc.git
|
| followed by
|
| ./boot
| ./configure
|
| At this point, we're at the common point from which both the Hadrian
| and the GNU Make build-system would start diverging:
|
| For the Hadrian build-system, we need to pay for a one-time setup,
| since we need to build the `hadrian` executable (which requires to
| build the in-tree lib:Cabal as an in-place library):
|
|
| $ time ./hadrian/build.sh --help
|
| ...
|
| real 5m25.992s
| user 6m19.196s
| sys 0m6.079s
|
|
| I'm not too worried about this part, as there's a few tricks by which
| we could likely bring that down to about 2 minutes or so, and we
| mostly pay this setup-cost, when lib:Cabal and/or `hadrian` changes
| and requires to be recompiled.
|
|
| Now, after having made sure that the `hadrian` executable is fresh, I
| started the actual build:
|
| $ time ./hadrian/build.sh -j8
|
| ...
|
| shakeArgsWith 0.000s 0%
| Function shake 0.178s 0%
| Database read 0.000s 0%
| With database 0.000s 0%
| Running rules 2338.398s 99%
| =========================
| Pool finished (5261 threads, 8 max) 0.002s 0%
| Lint checking 0.111s 0%
| Total 2338.690s 100%
| Build completed in 38:59m
|
| real 38m59.626s
| user 219m7.421s
| sys 11m7.584s
|
|
| then I immediately re-issued the same command to test how long it
| takes to perform a no-op build:
|
|
| $ time ./hadrian/build.sh -j8
|
| Up to date
| Up to date
| shakeArgsWith 0.000s 0%
| Function shake 0.183s 1%
| Database read 0.144s 1%
| With database 0.241s 2%
| Running rules 9.379s 93%
| =========================
| Pool finished (4165 threads, 8 max) 0.004s 0%
| Lint checking 0.101s 1%
| Total 10.051s 100%
| Build completed in 0:11m
|
|
| real 0m10.977s
| user 0m19.399s
| sys 0m2.443s
|
|
| Same without -j8:
|
| $ time ./hadrian/build.sh
| Up to date
| Up to date
| shakeArgsWith 0.000s 0%
| Function shake 0.175s 2%
| Database read 0.123s 1%
| With database 0.197s 2%
| Running rules 7.710s 92%
| =========================
| Pool finished (1 threads, 1 max) 0.003s 0%
| Lint checking 0.084s 1%
| Total 8.293s 100%
| Build completed in 0:09m
|
|
| real 0m9.196s
| user 0m8.656s
| sys 0m0.724s
|
|
| And finally clean it again:
|
| $ sync
| $ time ./hadrian/build.sh clean
| Up to date
| Up to date
| | Remove directory _build/stage0
| | Remove directory _build/stage1
| | Remove directory _build/stage2
| | Remove directory _build/stage3
| | Remove directory inplace/bin
| | Remove directory inplace/lib
| | Remove directory sdistprep
| | Remove Hadrian files...
| | Remove directory _build/generated
| | Done.
| shakeArgsWith 0.000s 0%
| Function shake 0.175s 0%
| Database read 0.114s 0%
| With database 0.205s 0%
| Running rules 20.037s 97%
| ========================
| Pool finished (1 threads, 1 max) 0.002s 0%
| Lint checking 0.006s 0%
| Total 20.540s 100%
| Build completed in 0:21m
|
|
| real 0m21.426s
| user 0m1.415s
| sys 0m1.045s
|
|
|
| ----
|
| Running the full build via our rusty GNU Make system:
|
| $ time make V=0 -j8
| real 33m30.801s
| user 157m49.520s
| sys 6m49.289s
|
| A no-op build:
|
| $ time make V=0 -j8
| ===--- building phase 0
| make --no-print-directory -f ghc.mk phase=0 phase_0_builds
| make[1]: Nothing to be done for 'phase_0_builds'.
| ===--- building phase 1
| make --no-print-directory -f ghc.mk phase=1 phase_1_builds
| make[1]: Nothing to be done for 'phase_1_builds'.
| ===--- building final phase
| make --no-print-directory -f ghc.mk phase=final all
| make -C utils/haddock/doc html SPHINX_BUILD=/usr/bin/sphinx-build
| /usr/bin/sphinx-build -b html . .build-html
| Running Sphinx v1.3.6
| loading translations [en]... done
| loading pickled environment... done
| building [mo]: targets for 0 po files that are out of date
| building [html]: targets for 0 source files that are out of date
| updating environment: 0 added, 0 changed, 0 removed
| looking for now-outdated files... none found
| no targets are out of date.
| build succeeded.
| cp -R utils/haddock/doc/.build-html utils/haddock/doc/haddock
|
| real 0m3.258s
| user 0m3.134s
| sys 0m0.283s
|
| And finally `make clean`:
|
| $ sync
| $ time make clean
| make --no-print-directory -f ghc.mk clean CLEANING=YES
| "rm" -rf inplace/bin inplace/lib
| "rm" -rf utils/touchy/dist
| "rm" -rf inplace/lib/bin/touchy
| "rm" -rf utils/unlit/dist
| "rm" -rf inplace/lib/bin/unlit
| "rm" -rf utils/unlit/dist-install
| "rm" -rf utils/hp2ps/dist
| "rm" -rf inplace/bin/hp2ps
| "rm" -rf inplace/lib/bin/hp2ps
| "rm" -rf utils/hp2ps/dist-install
| "rm" -rf driver/split/dist inplace/lib/bin/ghc-split
| "rm" -rf utils/genprimopcode/dist
| "rm" -rf inplace/bin/genprimopcode
| "rm" -rf libffi/build libffi/stamp.ffi.static-shared.configure
| libffi/stamp.ffi.static-shared.build libffi/stamp.ffi.static-
| shared.install libffi/dist-install
| "rm" -rf utils/deriveConstants/dist
| "rm" -rf inplace/bin/deriveConstants
| "rm" -rf includes/ghcautoconf.h includes/ghcplatform.h
| includes/ghcversion.h
| "rm" -rf rts/dist
| "rm" -rf bindisttest/"install dir" bindisttest/HelloWorld
| bindisttest/HelloWorld.o bindisttest/HelloWorld.hi bindisttest/output
| "rm" -rf utils/genapply/dist
| "rm" -rf inplace/bin/genapply
| "rm" -rf libraries/integer-gmp/include/ghc-gmp.h
| libraries/integer-gmp/gmp/config.mk libraries/integer-gmp/gmp/libgmp.a
| libraries/integer-gmp/gmp/gmp.h libraries/integer-gmp/gmp/gmpbuild
| libraries/integer-gmp/gmp/gmp-6.1.2
| "rm" -rf libraries/integer-gmp/gmp/objs
| "rm" -rf libraries/integer-gmp/gmp/gmpbuild
| "rm" -rf utils/haddock/dist
| "rm" -rf inplace/bin/haddock
| "rm" -rf inplace/lib/bin/haddock
| "rm" -rf compiler/stage1
| "rm" -rf compiler/stage2
| "rm" -rf compiler/stage3
| "rm" -rf utils/hsc2hs/dist
| "rm" -rf inplace/bin/hsc2hs
| "rm" -rf inplace/lib/bin/hsc2hs
| "rm" -rf utils/hsc2hs/dist-install
| "rm" -rf utils/ghc-pkg/dist
| "rm" -rf inplace/bin/ghc-pkg
| "rm" -rf inplace/lib/bin/ghc-pkg
| "rm" -rf utils/ghc-pkg/dist-install
| "rm" -rf utils/ghctags/dist-install
| "rm" -rf inplace/bin/ghctags
| "rm" -rf utils/check-api-annotations/dist-install
| "rm" -rf inplace/bin/check-api-annotations
| "rm" -rf utils/check-ppr/dist-install
| "rm" -rf inplace/bin/check-ppr
| "rm" -rf utils/ghc-cabal/dist bootstrapping
| "rm" -rf utils/ghc-cabal/dist-install
| "rm" -rf utils/hpc/dist-install
| "rm" -rf inplace/bin/hpc
| "rm" -rf inplace/lib/bin/hpc
| "rm" -rf utils/runghc/dist-install
| "rm" -rf inplace/bin/runghc
| "rm" -rf inplace/lib/bin/runghc
| "rm" -rf ghc/stage1
| "rm" -rf inplace/bin/ghc-stage1
| "rm" -rf inplace/lib/bin/ghc-stage1
| "rm" -rf ghc/stage2
| "rm" -rf inplace/bin/ghc-stage2
| "rm" -rf inplace/lib/bin/ghc-stage2
| "rm" -rf ghc/stage3
| "rm" -rf docs/users_guide/.doctrees-html/
| docs/users_guide/.doctrees-pdf/ docs/users_guide/build-html/
| docs/users_guide/build-pdf/ docs/users_guide/users_guide.pdf
| "rm" -rf docs/users_guide/.doctrees-man/ docs/users_guide/build-
| man/
| "rm" -rf utils/count_lines/dist inplace/bin/count_lines
| "rm" -rf utils/compare_sizes/dist-install
| "rm" -rf iserv/stage2
| "rm" -rf inplace/lib/bin/ghc-iserv
| "rm" -rf iserv/stage2_p
| "rm" -rf inplace/lib/bin/ghc-iserv-prof
| "rm" -rf iserv/stage2_dyn
| "rm" -rf inplace/lib/bin/ghc-iserv-dyn
| "rm" -f libraries/integer-gmp/include/HsIntegerGmp.h
| libraries/base/include/EventConfig.h mk/config.mk.old
| mk/project.mk.old compiler/ghc.cabal.old includes/GHCConstants.h
| includes/DerivedConstants.h includes/ghcautoconf.h
| includes/ghcplatform.h includes/ghcversion.h utils/ghc-pkg/Version.hs
| compiler/prelude/primops.txt
| "rm" -rf includes/dist-derivedconstants
| "rm" -rf inplace/bin
| "rm" -rf inplace/lib
| "rm" -rf libraries/bootstrapping.conf
| "rm" -f mk/are-validating.mk
| "rm" -rf libraries/ghc-boot-th/dist-install
| "rm" -rf libraries/ghc-boot/dist-install
| "rm" -rf libraries/ghci/dist-install
| "rm" -rf libraries/base/dist-install
| "rm" -rf libraries/ghc-prim/dist-install
| "rm" -rf libraries/integer-gmp/dist-install
| "rm" -rf libraries/integer-simple/dist-install
| "rm" -rf libraries/template-haskell/dist-install
| "rm" -rf libraries/array/dist-install
| "rm" -rf libraries/binary/dist-install
| "rm" -rf libraries/bytestring/dist-install
| "rm" -rf libraries/Cabal/Cabal/dist-install
| "rm" -rf libraries/ghc-compact/dist-install
| "rm" -rf libraries/containers/dist-install
| "rm" -rf libraries/deepseq/dist-install
| "rm" -rf libraries/directory/dist-install
| "rm" -rf libraries/filepath/dist-install
| "rm" -rf libraries/haskeline/dist-install
| "rm" -rf libraries/hpc/dist-install
| "rm" -rf libraries/mtl/dist-install
| "rm" -rf libraries/parsec/dist-install
| "rm" -rf libraries/pretty/dist-install
| "rm" -rf libraries/process/dist-install
| "rm" -rf libraries/terminfo/dist-install
| "rm" -rf libraries/text/dist-install
| "rm" -rf libraries/time/dist-install
| "rm" -rf libraries/transformers/dist-install
| "rm" -rf libraries/unix/dist-install
| "rm" -rf libraries/Win32/dist-install
| "rm" -rf libraries/xhtml/dist-install
| "rm" -rf libraries/parallel/dist-install
| "rm" -rf libraries/stm/dist-install
| "rm" -rf libraries/random/dist-install
| "rm" -rf libraries/primitive/dist-install
| "rm" -rf libraries/vector/dist-install
| "rm" -rf libraries/dph/dph-base/dist-install
| "rm" -rf libraries/dph/dph-prim-interface/dist-install
| "rm" -rf libraries/dph/dph-prim-seq/dist-install
| "rm" -rf libraries/dph/dph-prim-par/dist-install
| "rm" -rf libraries/dph/dph-lifted-base/dist-install
| "rm" -rf libraries/dph/dph-lifted-boxed/dist-install
| "rm" -rf libraries/dph/dph-lifted-copy/dist-install
| "rm" -rf libraries/dph/dph-lifted-vseg/dist-install
| "rm" -rf libraries/ghc-boot-th/dist-boot
| "rm" -rf libraries/ghc-boot/dist-boot
| "rm" -rf libraries/ghci/dist-boot
| "rm" -rf libraries/base/dist-boot
| "rm" -rf libraries/ghc-prim/dist-boot
| "rm" -rf libraries/integer-gmp/dist-boot
| "rm" -rf libraries/integer-simple/dist-boot
| "rm" -rf libraries/template-haskell/dist-boot
| "rm" -rf libraries/array/dist-boot
| "rm" -rf libraries/binary/dist-boot
| "rm" -rf libraries/bytestring/dist-boot
| "rm" -rf libraries/Cabal/Cabal/dist-boot
| "rm" -rf libraries/ghc-compact/dist-boot
| "rm" -rf libraries/containers/dist-boot
| "rm" -rf libraries/deepseq/dist-boot
| "rm" -rf libraries/directory/dist-boot
| "rm" -rf libraries/filepath/dist-boot
| "rm" -rf libraries/haskeline/dist-boot
| "rm" -rf libraries/hpc/dist-boot
| "rm" -rf libraries/mtl/dist-boot
| "rm" -rf libraries/parsec/dist-boot
| "rm" -rf libraries/pretty/dist-boot
| "rm" -rf libraries/process/dist-boot
| "rm" -rf libraries/terminfo/dist-boot
| "rm" -rf libraries/text/dist-boot
| "rm" -rf libraries/time/dist-boot
| "rm" -rf libraries/transformers/dist-boot
| "rm" -rf libraries/unix/dist-boot
| "rm" -rf libraries/Win32/dist-boot
| "rm" -rf libraries/xhtml/dist-boot
| "rm" -rf libraries/parallel/dist-boot
| "rm" -rf libraries/stm/dist-boot
| "rm" -rf libraries/random/dist-boot
| "rm" -rf libraries/primitive/dist-boot
| "rm" -rf libraries/vector/dist-boot
| "rm" -rf libraries/dph/dph-base/dist-boot
| "rm" -rf libraries/dph/dph-prim-interface/dist-boot
| "rm" -rf libraries/dph/dph-prim-seq/dist-boot
| "rm" -rf libraries/dph/dph-prim-par/dist-boot
| "rm" -rf libraries/dph/dph-lifted-base/dist-boot
| "rm" -rf libraries/dph/dph-lifted-boxed/dist-boot
| "rm" -rf libraries/dph/dph-lifted-copy/dist-boot
| "rm" -rf libraries/dph/dph-lifted-vseg/dist-boot
| "rm" -rf libraries/ghc-boot-th/dist libraries/ghc-boot/dist
| libraries/ghci/dist libraries/base/dist libraries/ghc-prim/dist
| libraries/integer-gmp/dist libraries/integer-simple/dist
| libraries/template-haskell/dist libraries/array/dist
| libraries/binary/dist libraries/bytestring/dist
| libraries/Cabal/Cabal/dist libraries/ghc-compact/dist
| libraries/containers/dist libraries/deepseq/dist
| libraries/directory/dist libraries/filepath/dist
| libraries/haskeline/dist libraries/hpc/dist libraries/mtl/dist
| libraries/parsec/dist libraries/pretty/dist libraries/process/dist
| libraries/terminfo/dist libraries/text/dist libraries/time/dist
| libraries/transformers/dist libraries/unix/dist libraries/Win32/dist
| libraries/xhtml/dist libraries/parallel/dist libraries/stm/dist
| libraries/random/dist libraries/primitive/dist libraries/vector/dist
| libraries/dph/dph-base/dist libraries/dph/dph-prim-interface/dist
| libraries/dph/dph-prim-seq/dist libraries/dph/dph-prim-par/dist
| libraries/dph/dph-lifted-base/dist libraries/dph/dph-lifted-
| boxed/dist libraries/dph/dph-lifted-copy/dist libraries/dph/dph-
| lifted-vseg/dist
| "rm" -f libraries/base/base.buildinfo libraries/integer-
| gmp/integer-gmp.buildinfo libraries/terminfo/terminfo.buildinfo
| libraries/unix/unix.buildinfo
| "rm" -f libraries/ghc-boot-th/config.log libraries/ghc-
| boot/config.log libraries/ghci/config.log libraries/base/config.log
| libraries/ghc-prim/config.log libraries/integer-gmp/config.log
| libraries/integer-simple/config.log libraries/template-
| haskell/config.log libraries/array/config.log
| libraries/binary/config.log libraries/bytestring/config.log
| libraries/Cabal/Cabal/config.log libraries/ghc-compact/config.log
| libraries/containers/config.log libraries/deepseq/config.log
| libraries/directory/config.log libraries/filepath/config.log
| libraries/haskeline/config.log libraries/hpc/config.log
| libraries/mtl/config.log libraries/parsec/config.log
| libraries/pretty/config.log libraries/process/config.log
| libraries/terminfo/config.log libraries/text/config.log
| libraries/time/config.log libraries/transformers/config.log
| libraries/unix/config.log libraries/Win32/config.log
| libraries/xhtml/config.log libraries/parallel/config.log
| libraries/stm/config.log libraries/random/config.log
| libraries/primitive/config.log libraries/vector/config.log
| libraries/dph/dph-base/config.log libraries/dph/dph-prim-
| interface/config.log libraries/dph/dph-prim-seq/config.log
| libraries/dph/dph-prim-par/config.log libraries/dph/dph-lifted-
| base/config.log libraries/dph/dph-lifted-boxed/config.log
| libraries/dph/dph-lifted-copy/config.log libraries/dph/dph-lifted-
| vseg/config.log
| "rm" -f libraries/ghc-boot-th/config.status libraries/ghc-
| boot/config.status libraries/ghci/config.status
| libraries/base/config.status libraries/ghc-prim/config.status
| libraries/integer-gmp/config.status libraries/integer-
| simple/config.status libraries/template-haskell/config.status
| libraries/array/config.status libraries/binary/config.status
| libraries/bytestring/config.status
| libraries/Cabal/Cabal/config.status libraries/ghc-
| compact/config.status libraries/containers/config.status
| libraries/deepseq/config.status libraries/directory/config.status
| libraries/filepath/config.status libraries/haskeline/config.status
| libraries/hpc/config.status libraries/mtl/config.status
| libraries/parsec/config.status libraries/pretty/config.status
| libraries/process/config.status libraries/terminfo/config.status
| libraries/text/config.status libraries/time/config.status
| libraries/transformers/config.status libraries/unix/config.status
| libraries/Win32/config.status libraries/xhtml/config.status
| libraries/parallel/config.status libraries/stm/config.status
| libraries/random/config.status libraries/primitive/config.status
| libraries/vector/config.status libraries/dph/dph-base/config.status
| libraries/dph/dph-prim-interface/config.status libraries/dph/dph-
| prim-seq/config.status libraries/dph/dph-prim-par/config.status
| libraries/dph/dph-lifted-base/config.status libraries/dph/dph-lifted-
| boxed/config.status libraries/dph/dph-lifted-copy/config.status
| libraries/dph/dph-lifted-vseg/config.status
| "rm" -f libraries/base/include/HsBaseConfig.h
| libraries/process/include/HsProcessConfig.h
| libraries/unix/include/HsUnixConfig.h
| "rm" -rf libraries/dist-haddock
| "rm" -rf bindistprep/
| test ! -d testsuite || make -C testsuite clean
| make[1]: Entering directory '/stuff3/work/GHC2/ghc/testsuite'
| make -C ./timeout clean
| make[2]: Entering directory
| '/stuff3/work/GHC2/ghc/testsuite/timeout'
| test ! -f Setup || ./Setup clean
| rm -f -rf install-inplace
| rm -f -f calibrate.out
| rm -f -f Setup Setup.exe Setup.hi Setup.o
| make[2]: Leaving directory
| '/stuff3/work/GHC2/ghc/testsuite/timeout'
| rm -f -f mk/*.o
| rm -f -f mk/*.hi
| rm -f -f mk/ghcconfig*.mk
| rm -f -f mk/ghc-config mk/ghc-config.exe
| rm -f -f driver/*.pyc
| make[1]: Leaving directory '/stuff3/work/GHC2/ghc/testsuite'
|
| real 0m50.990s
| user 0m0.496s
| sys 0m1.582s
|
|
|
| --
| _______________________________________________
| ghc-devs mailing list
| ghc-devs at haskell.org
| https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmail.h
| askell.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fghc-
| devs&data=02%7C01%7Csimonpj%40microsoft.com%7Cec692486529544de389e08d5
| 2dbc7432%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C6364652097984492
| 46&sdata=I1TY6c2l59jou87OVjqScDvct%2FaygyM4HRuSirCSp5w%3D&reserved=0
More information about the ghc-devs
mailing list