[ANNOUNCE] Shaking up GHC

Sun Jan 31 00:44:27 UTC 2016

Simon & all,

> Why not just make the 'validate' script invoke the shake build system
> (instead of 'make') when you say 'sh validate --shake'?  That would be
> simple, wouldn't it? So shake builds GHC, and then 'validate' invokes
> 'make test' (or whatever) to run the testsuite.
> 	
> Little things like this would significantly increase uptake I think!

We can now run:

shake-build/build.sh validate

This will rebuild stage2 GHC, ghc-pkg and hpc if need be, and will then run GHC tests by executing `make fast` in `testsuite/tests` folder (https://github.com/snowleopard/shaking-up-ghc#testing). 

There is still a lot of work to add proper support for various useful command line settings like `TEST=abc` or `TEST_HC=xyz`.

It would be great if someone could add support for `--shake` command line flag to the `validate` script, as Simon suggested. This would be a better solution, but I'm not confident enough to edit the validate script myself.

Thanks to Thomas Miedema for his patient guidance on how GHC test suite works.

Cheers,
Andrey

> -----Original Message-----
> From: Simon Peyton Jones [mailto:simonpj at microsoft.com]
> Sent: 26 January 2016 08:55
> To: Andrey Mokhov; Ben Gamari
> Cc: ghc-devs at haskell.org
> Subject: RE: [ANNOUNCE] Shaking up GHC
> 
> |  That's correct. Note though that the two build systems put (some)
> |  build results in the same directories, e.g. inplace/bin/ghc-stage1,
> so
> |  there is some interaction between them. In future it would be
> possible
> |  to decouple them completely (if need be).
> 
> Documenting this side-by-side possibility on the home page would be
> very helpful to up-take.
> 
> |  > How can I do that using Shake to build?  Maybe
> |  >	sh validate --shake --fast --no-clean
> |  > or something?  Just modifying the validate script to make it
> |  >convenient to use shake for the build part would be great.
> |
> |  This may be possible precisely because binaries are where validate
> |  expects them to be. I tried to pull this off, but unsuccessfully so
> |  far (validate starts rebuilding everything from scratch with make).
> I
> |  think the reason is that we changed some naming conventions for
> |  directories (using stageN instead of dist, dist-boot, dist-install),
> |  so validate can't find some utils. I'll let you know if I find a
> |  workaround.
> 
> Why not just make the 'validate' script invoke the shake build system
> (instead of 'make') when you say 'sh validate --shake'?  That would be
> simple, wouldn't it?  So shake builds GHC, and then 'validate' invokes
> 'make test' (or whatever) to run the testsuite.
> 	
> Little things like this would significantly increase uptake I think!
> 
> Simon
> 
> 
> 
> |
> |  Ben,
> |
> |  > > But User.hs is a source file, which we shouldn't normally modify
> |  > > lest we accidentally commit it. Could we have a non-source file
> to
> |  > > modify (rather like
> |  > >
> |  https://na01.safelinks.protection.outlook.com/?url=build.mk&data=01%
> |  > >
> |  7c01%7csimonpj%40064d.mgd.microsoft.com%7c7f0edcff3c78490bd0e808d325
> |  > >
> |  be4774%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=8L%2b7QWlhbAwUnY
> |  > > fAfCEwTXTC2PUMd17m1ceCcD2TE%2fU%3d being based on
> build.mk.sample)
> |  > >
> |  > One way around this would be to add User.hs to .gitignore. That
> way
> |  > git will ignore changes to this files when you `git commit -a`.
> You
> |  > can still, however, commit changes to it if you ask git
> explicitly.
> |
> |  Yes, I was thinking about the same idea.
> |
> |  Cheers,
> |  Andrey
> |
> |  -----Original Message-----
> |  From: Simon Peyton Jones [mailto:simonpj at microsoft.com]
> |  Sent: 25 January 2016 12:14
> |  To: Andrey Mokhov <andrey.mokhov at newcastle.ac.uk>; ghc-
> |  devs at haskell.org
> |  Cc: Neil Mitchell <ndmitchell at gmail.com>; Simon Marlow
> |  <marlowsd at gmail.com>
> |  Subject: RE: [ANNOUNCE] Shaking up GHC
> |
> |  Very good.
> |
> |  As I understand it, it can work side-by-side with the existing build
> |  system, correct?  That means we don't need to make an either/or
> |  choice, which is very helpful.
> |
> |  Every day I do
> |  	sh validate --fast --no-clean
> |  How can I do that using Shake to build?  Maybe
> |  	sh validate --shake --fast --no-clean
> |  or something?  Just modifying the validate script to make it
> |  convenient to use shake for the build part would be great.
> |
> |  You say: The make-based build system uses mk/build.mk to specify
> user
> |  build settings. We use src/Settings/User.hs for the same purpose.
> Feel
> |  free to experiment following the Haddock comments.
> |
> |  But User.hs is a source file, which we shouldn't normally modify
> lest
> |  we accidentally commit it.  Could we have a non-source file to
> modify
> |  (rather like
> |
> https://na01.safelinks.protection.outlook.com/?url=build.mk&data=01%7c
> |
> 01%7csimonpj%40064d.mgd.microsoft.com%7c7f0edcff3c78490bd0e808d325be47
> |
> 74%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=8L%2b7QWlhbAwUnYfAfCEw
> |  TXTC2PUMd17m1ceCcD2TE%2fU%3d being based on build.mk.sample)
> |
> |  Simon
> |
> |  |  -----Original Message-----
> |  |  From: Andrey Mokhov [mailto:andrey.mokhov at newcastle.ac.uk]
> |  |  Sent: 22 January 2016 14:27
> |  |  To: ghc-devs at haskell.org
> |  |  Cc: Simon Peyton Jones <simonpj at microsoft.com>; Neil Mitchell
> |  | <ndmitchell at gmail.com>; Simon Marlow <marlowsd at gmail.com>
> |  |  Subject: [ANNOUNCE] Shaking up GHC
> |  |
> |  |  Dear GHC developers,
> |  |
> |  |  I am happy to announce that the Shaking up GHC project has
> finally
> |  | reached the first milestone. The goal of the project is to design
> a
> |  | new GHC build system based on Shake that will eventually replace
> the
> |  | current make-based one. See the project page for more details:
> |  |  https://github.com/snowleopard/shaking-up-ghc.
> |  |
> |  |  There is still a long way until we can match the capabilities of
> |  the
> |  | current build system. At the moment we only build vanilla way.
> |  |  Validation, documentation, build flavours and cross-compilation
> are
> |  | not yet implemented. Known limitations are listed here:
> |  |  https://github.com/snowleopard/shaking-up-ghc#current-
> limitations.
> |  |
> |  |  The purpose of this announcement is to attract alpha testers and
> |  | collect early feedback across multiple platforms and build
> |  | configurations. We already have several success reports on Linux,
> OS
> |  | X, Solaris and Windows. However, things will inevitably break and
> we
> |  | hope to catch and fix as many of these cases as possible with your
> |  | help. The instructions on how to try the new build system can be
> |  found
> |  |  here: https://github.com/snowleopard/shaking-up-ghc#your-first-
> |  build.
> |  |
> |  |  We plan to be ready to become a part of the GHC tree around 1
> March
> |  | 2016, and catch up with the make build system around 1 June 2016.
> |  The
> |  | dates are tentative and depend on how much time it takes us to
> |  resolve
> |  | the remaining issues: https://github.com/snowleopard/shaking-up-
> |  |  ghc/milestones.
> |  |
> |  |  I would like thank everyone who contributed to this project so
> far:
> |  |  Simon Peyton Jones, Neil Mitchell and Simon Marlow came up with
> the
> |  | idea and guided me throughout the project; Moritz Angermann, Ben
> |  | Gamari, Karel Gardas, David Luposchainsky, and Neil Mitchell
> |  | contributed to the codebase. Thank you all!
> |  |
> |  |  Kind regards,
> |  |  Andrey