[ghc-steering-committee] Plugin recompilation avoidance interface (#108)

[Ben Gamari]

Joachim Breitner <mail at joachim-breitner.de> writes:

> Hi Ben,
>
> thanks for kicking this off.
>
> Am Montag, den 19.02.2018, 22:04 -0500 schrieb Ben Gamari:
>> The proposal addresses a long-standing limitation (#7414) of the GHC
>> plugin interface which has become increasingly visible to users in
>> recent years. While the particular approach proposed breaks existing
>> plugin users, it is expressive enough to allow GHC to perform more
>> aggressive recompilation avoidance in the future [1]. Moreover, the
>> smart constructors proposed should make for a reasonably smooth
>> migration path.
>> 
>> Given that the plugins appears to have buy-in from a few notable plugin
>> authors, I recommend that we accept this proposal.
>
> How realistic is it to do the partial module recompilation thing that
> warrants multiple PluginRecompile values per module? For this to be
> useful the compiler would have to
>
>  * calculate the typechecker plugin’s hash for module M
>  * notice that it has changed, and start compiling M
>  * but then, when we have core, magically notice that the output of the
>    desugarer has not changed, and consider to abort calculation
>  * but then notice that a core2core plugin was involved, and ask that 
>    for the new hash
>  * and then this hash must be unchanged
>  * and then really the compilation has ended.
>
> Is that realistic? Can someone fill this example with concrete life?
>
> Maybe I am just not seeing what people have in mind – but I feel unless
> this really is more than a very vague idea, we should consider the
> simpler variant where each plugin gets to calculate a single hash per
> module – and not possibly many in various stages.
>
> Another way of putting it: The compiler pipeline is (simplified)
>
>  1. decide what to compiler
>  2. parse
>  3. typecheck
>  4. desguar
>  5. optimize
>  6. code gen
>
> Plugins so far hook into step 3 and 5. Every one of these steps
> probably eventually deserves a plugin hook. This proposal is about
> adding one to step 1. With this view, I find the “add a single function
> to Plugin”, as described in
> https://github.com/ghc-proposals/ghc-proposals/pull/108#issuecomment-362671438
> much more convincing.
>
I understand the concern; your description indeed matches how GHC
currently handles recompilation, which makes it quite difficult to bail
out after deciding to recompile.

Simon PJ has mentioned in the past that one of the goals that would be
worth pursuing is more aggressive recompilation avoidance (e.g. perhaps
even some day on a sub-module granularity), especially given the recent
complaints surrounding compilation times. I don't believe there is any
motion on this and it would certainly be a significant undertaking.

Given that we have very little idea of what such a design
would look like, I agree that we shouldn't try to design the plugin
interface too carefully around this (likely quite far-off) goal.
However, I personally feel like the proposed design is a reasonable
compromise. Lumping together all of the recompilation checks into a
single hash would expose a relatively ugly facet of GHC's implementation
to plugin authors and force authors to tie together logic from
conceptually-distinct compilation phases. Furthermore, we would tie
ourselves to GHC's current design.

The one thing that gives me pause is the proposal's treatment of
core-to-core plugins, which return a list of CoreTodos. Tying
recompilation to this function is admittedly a bit odd.

Cheers,

- Ben
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 487 bytes
Desc: not available
URL: <http://mail.haskell.org/pipermail/ghc-steering-committee/attachments/20180220/845163bf/attachment.sig>