More failure

[Andrey Mokhov]

Hi Tamar,

I think the difficulty here is more with dynamic *outputs* rather than dynamic inputs/dependencies.

We do not statically know which of the following three alternatives holds:

  *   `*.dyn_o/hi` files are not built at all.
  *   `*.dyn_o/hi` files are built via a separate execution of GHC.
  *   `*.dyn_o/hi` files are built together with `*.o/hi` files, in a single execution of GHC with `-dynamic-too`.

Here is the current implementation:

https://gitlab.haskell.org/ghc/ghc/blob/master/hadrian/src/Rules/Compile.hs#L51-69

I believe the last person looking into this was James Foster, so CC-ing to him in case he has any insights.

Cheers,
Andrey

________________________________
From: Phyx <lonetiger at gmail.com>
Sent: 10 December 2019 07:47
To: Andrey Mokhov <andrey.mokhov at newcastle.ac.uk>
Cc: Simon Peyton-Jones (simonpj at microsoft.com) <simonpj at microsoft.com>; Ben Gamari <ben at well-typed.com>; ghc-devs at haskell.org <ghc-devs at haskell.org>
Subject: Re: More failure

Hi Andrey,

I'm not sure what the original issue here is (should probably find the original message) but

> The Make build system happens to do the right thing, somehow. I believe we should be able to express the same logic in Shake, but it's not easy.

I believe this typically works because GCC and GHC support dumping the dependencies that a command would have caused to a file. So your dynamic dependencies don't matter as their static to the build system after this invocation.

See
https://downloads.haskell.org/~ghc/latest/docs/html/users_guide/separate_compilation.html#dependency-generation

These Compilers are able to dump out make rules which enabled better dependency handling..

Kind regards,
Tamar

Sent from my Mobile

On Tue, Dec 10, 2019, 00:58 Andrey Mokhov <andrey.mokhov at newcastle.ac.uk<mailto:andrey.mokhov at newcastle.ac.uk>> wrote:

Hi Simon,



(Re-sending from the email address that’s allowed on the mailing list.)



> Ugh.  That's not a very happy state of affairs, is it?  It didn't happen with 'make'.

> Is it a fundamental problem, or just not yet fixed?



I think this is not a fundamental problem, but the problem of getting dependencies right.



In this case, the complexity comes from the fact that a single invocation of GHC produces a set of files, and which set depends on the command line flags, which are in turn determined dynamically by reading environment settings (specifically, `platformSupportsSharedLibs`).



Such rules are hard to describe precisely, because build systems are tuned to the typical case where we statically know, for every output file, which rule produces it -- recall the Tasks = k -> Maybe Task function from our paper. In this case, we deal with something like k -> f (Maybe Task) instead, i.e. with `f` around the Maybe.



The Make build system happens to do the right thing, somehow. I believe we should be able to express the same logic in Shake, but it's not easy.



(I never really had a chance to look at dynamic builds, since they are not supported on Windows. I guess I should finally find a Linux box for Hadrian.)



Cheers,

Andrey

_______________________________________________
ghc-devs mailing list
ghc-devs at haskell.org<mailto:ghc-devs at haskell.org>
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191210/0f242bf4/attachment.html>