Plugins: Accessing unexported bindings
Levent Erkok
erkokl at gmail.com
Tue Dec 8 03:32:04 UTC 2015
Thanks for the explanation, Simon. I think Eric is spot on. Looking at some
of the "ghc -v3" output, it appears GHC always runs the following passes
before any plugins are run:
*** Simplify:
*** CorePrep:
*** ByteCodeGen:
*** Desugar:
Just judging by the name, it could either be the first "Simplify" or the
"Desugar" that gets rid of the dead-code at this point, I'm not sure which.
It would help if this stage didn't remove dead-bindings. If doing so
automatically is not the best option, requiring a user given pragma like
'KeepAlive' wouldn't be too onerous either.
-Levent.
On Mon, Dec 7, 2015 at 8:22 AM, Simon Peyton Jones <simonpj at microsoft.com>
wrote:
> It would not be hard to stop the desugarer dropping dead bindings, if that
> was helpful.
>
> S
>
> | -----Original Message-----
> | From: Eric Seidel [mailto:eric at seidel.io]
> | Sent: 07 December 2015 15:44
> | To: Simon Peyton Jones <simonpj at microsoft.com>
> | Cc: Levent Erkok <erkokl at gmail.com>; omeragacan at gmail.com;
> | ezyang at mit.edu; ghc-devs at haskell.org
> | Subject: Re: Plugins: Accessing unexported bindings
> |
> | The problem, as I recall, is that GHC does an initial bit of dead-code
> | elimination in the desugarer, before the plugins have a chance to run.
> | (I believe this is part of simpleOptPgm, but may be mistaken)
> |
> | I'm not sure why this is done in the desugarer, it seems to be out of
> | place there.
> |
> | On Mon, Dec 7, 2015, at 05:14, Simon Peyton Jones wrote:
> | > Plugins get to edit the entire core-to-core pipeline! There is no
> | magic.
> | > At least I don’t think so
> | >
> | > file:///Z:/tmp/users_guide/compiler-plugins.html
> | >
> | > S
> | >
> | > From: Levent Erkok [mailto:erkokl at gmail.com]
> | > Sent: 07 December 2015 13:11
> | > To: Simon Peyton Jones <simonpj at microsoft.com>
> | > Cc: Eric Seidel <eric at seidel.io>; omeragacan at gmail.com;
> | > ezyang at mit.edu; ghc-devs at haskell.org
> | > Subject: Re: Plugins: Accessing unexported bindings
> | >
> | > That's a good point; keeping all annotated bindings alive seems to
> | be
> | > an overkill..
> | >
> | > Regarding implementing "pass at the start." I'm not sure if plugin
> | > authors have any freedom as to decide when their plugin actually
> | runs.
> | > It seems GHC magically determines the order and runs them. Can you
> | > point me to some code/docs that tells me how to go "first" in that
> | > sense? (Or at least before the pass that drops dead code.)
> | >
> | > On Dec 7, 2015, at 4:45 AM, Simon Peyton Jones
> | > <simonpj at microsoft.com<mailto:simonpj at microsoft.com>> wrote:
> | > Indeed. How about this: if there's an ANN on a binder (any ANN),
> | then
> | > GHC should keep it alive.
> | >
> | > Really? It might be something like “don’t give warnings for this
> | > binding” or “don’t inline me” or something. To say *any*
> | annotation
> | > seems a bit brutal doesn’t it? Mind you I don’t have a better
> | idea.
> | >
> | > One thought: your plugin could add a pass right at the start, which
> | > marks everything you want as keep-alive.
> | >
> | > S
> | >
> | > From: Levent Erkok [mailto:erkokl at gmail.com]
> | > Sent: 07 December 2015 12:42
> | > To: Simon Peyton Jones
> | > <simonpj at microsoft.com<mailto:simonpj at microsoft.com>>
> | > Cc: Eric Seidel <eric at seidel.io<mailto:eric at seidel.io>>;
> | > omeragacan at gmail.com<mailto:omeragacan at gmail.com>;
> | > ezyang at mit.edu<mailto:ezyang at mit.edu>;
> | > ghc-devs at haskell.org<mailto:ghc-devs at haskell.org>
> | > Subject: Re: Plugins: Accessing unexported bindings
> | >
> | > Indeed. How about this: if there's an ANN on a binder (any ANN),
> | then
> | > GHC should keep it alive.
> | >
> | > Is that something one of the core-developers can implement? Happy to
> | > open a ticket if that helps.
> | >
> | > On Dec 7, 2015, at 4:14 AM, Simon Peyton Jones
> | > <simonpj at microsoft.com<mailto:simonpj at microsoft.com>> wrote:
> | > If it's "dead" in this sense, it's already removed from ModGuts, no?
> | >
> | > Yes, if it’s dead it’s gone. That’s not too surprising, is it?
> | >
> | > So you need a way to keep it alive. Maybe we need a pragma for that.
> | Or
> | > how would you like to signal it in the source code?
> | >
> | > Simon
> | >
> | > From: Levent Erkok [mailto:erkokl at gmail.com]
> | > Sent: 07 December 2015 12:05
> | > To: Simon Peyton Jones
> | > <simonpj at microsoft.com<mailto:simonpj at microsoft.com>>
> | > Cc: Eric Seidel <eric at seidel.io<mailto:eric at seidel.io>>;
> | > omeragacan at gmail.com<mailto:omeragacan at gmail.com>;
> | > ezyang at mit.edu<mailto:ezyang at mit.edu>;
> | > ghc-devs at haskell.org<mailto:ghc-devs at haskell.org>
> | > Subject: Re: Plugins: Accessing unexported bindings
> | >
> | > Thanks Simon.. But I remain utterly confused. As a "plugin" author,
> | > how do I get my hands on the Id associated with a top-level binder?
> | If
> | > it's "dead" in this sense, it's already removed from ModGuts, no?
> | >
> | > That is, by the time GHC runs my plugin, the Id has already
> | > disappeared for me to mark it "Local Exported." Is that not correct?
> | >
> | > On Dec 7, 2015, at 2:28 AM, Simon Peyton Jones
> | > <simonpj at microsoft.com<mailto:simonpj at microsoft.com>> wrote:
> | > In the mean time, I'm still looking for a solution that doesn't
> | > involve exporting such identifiers from modules. As Eric pointed
> | out,
> | > that seems to be the only current work-around for the time being.
> | >
> | > “Exported” in this context only means “keep alive”. It does not mean
> | > exported in the Haskell source code sense. I’ve just added this
> | > comment to Var.hs.
> | >
> | > So I think it does just what you want.
> | >
> | > Simon
> | >
> | > data ExportFlag -- See Note [ExportFlag on binders]
> | > = NotExported -- ^ Not exported: may be discarded as dead code.
> | > | Exported -- ^ Exported: kept alive
> | >
> | > {- Note [ExportFlag on binders]
> | > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> | > An ExportFlag of "Exported" on a top-level binder says "keep this
> | > binding alive; do not drop it as dead code". This transititively
> | > keeps alive all the other top-level bindings that this binding
> | refers
> | > to. This property is persisted all the way down the pipeline, so
> | that
> | > the binding will be compiled all the way to object code, and its
> | > symbols will appear in the linker symbol table.
> | >
> | > However, note that this use of "exported" is quite different to the
> | > export list on a Haskell module. Setting the ExportFlag on an Id
> | does
> | > /not/ mean that if you import the module (in Haskell source code you
> | > will see this Id. Of course, things that appear in the export list
> | of
> | > the source Haskell module do indeed have their ExportFlag set.
> | > But many other things, such as dictionary functions, are kept alive
> | by
> | > having their ExportFlag set, even though they are not exported in
> | the
> | > source-code sense.
> | >
> | > We should probably use a different term for ExportFlag, like
> | > KeepAlive.
> | >
> | > From: ghc-devs [mailto:ghc-devs-bounces at haskell.org] On Behalf Of
> | > Levent Erkok
> | > Sent: 06 December 2015 20:32
> | > To: Eric Seidel <eric at seidel.io<mailto:eric at seidel.io>>;
> | > omeragacan at gmail.com<mailto:omeragacan at gmail.com>;
> | > ezyang at mit.edu<mailto:ezyang at mit.edu>
> | > Cc: ghc-devs at haskell.org<mailto:ghc-devs at haskell.org>
> | > Subject: Re: Plugins: Accessing unexported bindings
> | >
> | > Omer, Eric, Ed: Thanks for the comments.
> | >
> | > Omer: I think Eric's observation is at play here. We're talking
> | about
> | > "dead-code," i.e., a binding that is neither exported, nor used by
> | any
> | > binding inside the module. Those seem to be getting dropped by the
> | > time user-plugins are run. Unfortunately, this is precisely what one
> | > would do with "properties" embedded in code. They serve as
> | > documentation perhaps, but are otherwise not needed by any other
> | > binding nor it makes sense to export them.
> | >
> | > Edward: Can you provide some more info into your solution? Sounds
> | like
> | > a chicken-egg issue to me: As a plugin author, I need the bindings
> | to
> | > access the Ids, and looks like I need the Ids to access the binders?
> | >
> | > A simple solution would be to simply keep all top-level bindings
> | > around while the plugin are running, but that obviously can lead to
> | > unnecessary work if the code is truly dead. A compromise could be
> | that
> | > the annotations can serve as entry points as well: I.e., if there's
> | an
> | > annotation on a top-level binder, then it should *not* be considered
> | > dead-code at least until after all the plugins are run. That would
> | > definitely simplify life. Would that be an acceptable alternative?
> | >
> | > In the mean time, I'm still looking for a solution that doesn't
> | > involve exporting such identifiers from modules. As Eric pointed
> | out,
> | > that seems to be the only current work-around for the time being.
> | >
> | > Thanks,
> | >
> | > -Levent.
> | >
> | > On Sun, Dec 6, 2015 at 11:08 AM, Eric Seidel
> | > <eric at seidel.io<mailto:eric at seidel.io>> wrote:
> | > GHC should only drop un-exported bindings from the ModGuts if
> | they're
> | > also unused, ie *dead code*.
> | >
> | > The only way I know to get around this is to use the bindings
> | > somewhere, or just export them.
> | >
> | > On Sat, Dec 5, 2015, at 23:01, Levent Erkok wrote:
> | > > Hello,
> | > >
> | > > The mg_binds field of the ModGuts seem to only contain the
> | bindings
> | > > that are exported from the module being compiled.
> | > >
> | > > I guess GHC must be running user-plugins after it drops the
> | bindings
> | > > that are not exported, which makes perfect sense for most use
> | cases.
> | > > However, I'm working on a plugin where the end-programmer embeds
> | > > "properties" in the form of functions inside his/her code, which
> | are
> | > > not necessarily exported from the module under consideration.
> | > >
> | > > Is there a way to access all top-level bindings in a module from a
> | > > plugin, even if those bindings are not exported?
> | > >
> | > > Thanks,
> | > >
> | > > -Levent.
> | > > _______________________________________________
> | > > ghc-devs mailing list
> | > > ghc-devs at haskell.org<mailto:ghc-devs at haskell.org>
> | > >
> | https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2fmail
> | > > .haskell.org%2fcgi-bin%2fmailman%2flistinfo%2fghc-
> | devs&data=01%7c01%
> | > >
> | 7csimonpj%40064d.mgd.microsoft.com%7cf6e3a9d4ad9f4e53a3ab08d2ff1d493
> | > >
> | 4%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=Bv7lpsB%2fD88nSMuB7NY
> | > >
> | fBqR90%2bBq%2fwpJJ0JU9%2b6E4RI%3d<https://na01.safelinks.protection.
> | > > outlook.com/?url=http%3a%2f%2fmail.haskell.org%2fcgi-
> | bin%2fmailman%2
> | > > flistinfo%2fghc-
> | devs&data=01%7c01%7csimonpj%40064d.mgd.microsoft.com
> | > >
> | %7cac4cbfe22e314080909908d2fe7c4ed8%7c72f988bf86f141af91ab2d7cd011db
> | > > 47%7c1&sdata=1z6DcZxjIAKj0PcsLeALphRLWJ3i%2fxvyaPtq0qo6elY%3d>
> | > _______________________________________________
> | > ghc-devs mailing list
> | > ghc-devs at haskell.org<mailto:ghc-devs at haskell.org>
> | >
> | https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2fmail.h
> | > askell.org%2fcgi-bin%2fmailman%2flistinfo%2fghc-
> | devs&data=01%7c01%7csi
> | >
> | monpj%40064d.mgd.microsoft.com%7cf6e3a9d4ad9f4e53a3ab08d2ff1d4934%7c72
> | >
> | f988bf86f141af91ab2d7cd011db47%7c1&sdata=Bv7lpsB%2fD88nSMuB7NYfBqR90%2
> | >
> | bBq%2fwpJJ0JU9%2b6E4RI%3d<https://na01.safelinks.protection.outlook.co
> | > m/?url=http%3a%2f%2fmail.haskell.org%2fcgi-
> | bin%2fmailman%2flistinfo%2f
> | > ghc-
> | devs&data=01%7c01%7csimonpj%40064d.mgd.microsoft.com%7cac4cbfe22e3
> | >
> | 14080909908d2fe7c4ed8%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=1z6
> | > DcZxjIAKj0PcsLeALphRLWJ3i%2fxvyaPtq0qo6elY%3d>
> | >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20151207/833edbc5/attachment-0001.html>
More information about the ghc-devs
mailing list