Plugins: Accessing unexported bindings
Simon Peyton Jones
simonpj at microsoft.com
Tue Dec 8 12:33:36 UTC 2015
I think it’s probably the desugarer, and the plugin can’t do anything before desugaring!
By all means open a ticket. I can advise if anyone wants to take it up.
Simon
From: Levent Erkok [mailto:erkokl at gmail.com]
Sent: 08 December 2015 03:32
To: Simon Peyton Jones <simonpj at microsoft.com>
Cc: Eric Seidel <eric at seidel.io>; omeragacan at gmail.com; ezyang at mit.edu; ghc-devs at haskell.org
Subject: Re: Plugins: Accessing unexported bindings
Thanks for the explanation, Simon. I think Eric is spot on. Looking at some of the "ghc -v3" output, it appears GHC always runs the following passes before any plugins are run:
*** Simplify:
*** CorePrep:
*** ByteCodeGen:
*** Desugar:
Just judging by the name, it could either be the first "Simplify" or the "Desugar" that gets rid of the dead-code at this point, I'm not sure which.
It would help if this stage didn't remove dead-bindings. If doing so automatically is not the best option, requiring a user given pragma like 'KeepAlive' wouldn't be too onerous either.
-Levent.
On Mon, Dec 7, 2015 at 8:22 AM, Simon Peyton Jones <simonpj at microsoft.com<mailto:simonpj at microsoft.com>> wrote:
It would not be hard to stop the desugarer dropping dead bindings, if that was helpful.
S
| -----Original Message-----
| From: Eric Seidel [mailto:eric at seidel.io<mailto:eric at seidel.io>]
| Sent: 07 December 2015 15:44
| To: Simon Peyton Jones <simonpj at microsoft.com<mailto:simonpj at microsoft.com>>
| Cc: Levent Erkok <erkokl at gmail.com<mailto:erkokl at gmail.com>>; omeragacan at gmail.com<mailto:omeragacan at gmail.com>;
| ezyang at mit.edu<mailto:ezyang at mit.edu>; ghc-devs at haskell.org<mailto:ghc-devs at haskell.org>
| Subject: Re: Plugins: Accessing unexported bindings
|
| The problem, as I recall, is that GHC does an initial bit of dead-code
| elimination in the desugarer, before the plugins have a chance to run.
| (I believe this is part of simpleOptPgm, but may be mistaken)
|
| I'm not sure why this is done in the desugarer, it seems to be out of
| place there.
|
| On Mon, Dec 7, 2015, at 05:14, Simon Peyton Jones wrote:
| > Plugins get to edit the entire core-to-core pipeline! There is no
| magic.
| > At least I don’t think so
| >
| > file:///Z:/tmp/users_guide/compiler-plugins.html
| >
| > S
| >
| > From: Levent Erkok [mailto:erkokl at gmail.com<mailto:erkokl at gmail.com>]
| > Sent: 07 December 2015 13:11
| > To: Simon Peyton Jones <simonpj at microsoft.com<mailto:simonpj at microsoft.com>>
| > Cc: Eric Seidel <eric at seidel.io<mailto:eric at seidel.io>>; omeragacan at gmail.com<mailto:omeragacan at gmail.com>;
| > ezyang at mit.edu<mailto:ezyang at mit.edu>; ghc-devs at haskell.org<mailto:ghc-devs at haskell.org>
| > Subject: Re: Plugins: Accessing unexported bindings
| >
| > That's a good point; keeping all annotated bindings alive seems to
| be
| > an overkill..
| >
| > Regarding implementing "pass at the start." I'm not sure if plugin
| > authors have any freedom as to decide when their plugin actually
| runs.
| > It seems GHC magically determines the order and runs them. Can you
| > point me to some code/docs that tells me how to go "first" in that
| > sense? (Or at least before the pass that drops dead code.)
| >
| > On Dec 7, 2015, at 4:45 AM, Simon Peyton Jones
| > <simonpj at microsoft.com<mailto:simonpj at microsoft.com><mailto:simonpj at microsoft.com<mailto:simonpj at microsoft.com>>> wrote:
| > Indeed. How about this: if there's an ANN on a binder (any ANN),
| then
| > GHC should keep it alive.
| >
| > Really? It might be something like “don’t give warnings for this
| > binding” or “don’t inline me” or something. To say *any*
| annotation
| > seems a bit brutal doesn’t it? Mind you I don’t have a better
| idea.
| >
| > One thought: your plugin could add a pass right at the start, which
| > marks everything you want as keep-alive.
| >
| > S
| >
| > From: Levent Erkok [mailto:erkokl at gmail.com<mailto:erkokl at gmail.com>]
| > Sent: 07 December 2015 12:42
| > To: Simon Peyton Jones
| > <simonpj at microsoft.com<mailto:simonpj at microsoft.com><mailto:simonpj at microsoft.com<mailto:simonpj at microsoft.com>>>
| > Cc: Eric Seidel <eric at seidel.io<mailto:eric at seidel.io><mailto:eric at seidel.io<mailto:eric at seidel.io>>>;
| > omeragacan at gmail.com<mailto:omeragacan at gmail.com><mailto:omeragacan at gmail.com<mailto:omeragacan at gmail.com>>;
| > ezyang at mit.edu<mailto:ezyang at mit.edu><mailto:ezyang at mit.edu<mailto:ezyang at mit.edu>>;
| > ghc-devs at haskell.org<mailto:ghc-devs at haskell.org><mailto:ghc-devs at haskell.org<mailto:ghc-devs at haskell.org>>
| > Subject: Re: Plugins: Accessing unexported bindings
| >
| > Indeed. How about this: if there's an ANN on a binder (any ANN),
| then
| > GHC should keep it alive.
| >
| > Is that something one of the core-developers can implement? Happy to
| > open a ticket if that helps.
| >
| > On Dec 7, 2015, at 4:14 AM, Simon Peyton Jones
| > <simonpj at microsoft.com<mailto:simonpj at microsoft.com><mailto:simonpj at microsoft.com<mailto:simonpj at microsoft.com>>> wrote:
| > If it's "dead" in this sense, it's already removed from ModGuts, no?
| >
| > Yes, if it’s dead it’s gone. That’s not too surprising, is it?
| >
| > So you need a way to keep it alive. Maybe we need a pragma for that.
| Or
| > how would you like to signal it in the source code?
| >
| > Simon
| >
| > From: Levent Erkok [mailto:erkokl at gmail.com<mailto:erkokl at gmail.com>]
| > Sent: 07 December 2015 12:05
| > To: Simon Peyton Jones
| > <simonpj at microsoft.com<mailto:simonpj at microsoft.com><mailto:simonpj at microsoft.com<mailto:simonpj at microsoft.com>>>
| > Cc: Eric Seidel <eric at seidel.io<mailto:eric at seidel.io><mailto:eric at seidel.io<mailto:eric at seidel.io>>>;
| > omeragacan at gmail.com<mailto:omeragacan at gmail.com><mailto:omeragacan at gmail.com<mailto:omeragacan at gmail.com>>;
| > ezyang at mit.edu<mailto:ezyang at mit.edu><mailto:ezyang at mit.edu<mailto:ezyang at mit.edu>>;
| > ghc-devs at haskell.org<mailto:ghc-devs at haskell.org><mailto:ghc-devs at haskell.org<mailto:ghc-devs at haskell.org>>
| > Subject: Re: Plugins: Accessing unexported bindings
| >
| > Thanks Simon.. But I remain utterly confused. As a "plugin" author,
| > how do I get my hands on the Id associated with a top-level binder?
| If
| > it's "dead" in this sense, it's already removed from ModGuts, no?
| >
| > That is, by the time GHC runs my plugin, the Id has already
| > disappeared for me to mark it "Local Exported." Is that not correct?
| >
| > On Dec 7, 2015, at 2:28 AM, Simon Peyton Jones
| > <simonpj at microsoft.com<mailto:simonpj at microsoft.com><mailto:simonpj at microsoft.com<mailto:simonpj at microsoft.com>>> wrote:
| > In the mean time, I'm still looking for a solution that doesn't
| > involve exporting such identifiers from modules. As Eric pointed
| out,
| > that seems to be the only current work-around for the time being.
| >
| > “Exported” in this context only means “keep alive”. It does not mean
| > exported in the Haskell source code sense. I’ve just added this
| > comment to Var.hs.
| >
| > So I think it does just what you want.
| >
| > Simon
| >
| > data ExportFlag -- See Note [ExportFlag on binders]
| > = NotExported -- ^ Not exported: may be discarded as dead code.
| > | Exported -- ^ Exported: kept alive
| >
| > {- Note [ExportFlag on binders]
| > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
| > An ExportFlag of "Exported" on a top-level binder says "keep this
| > binding alive; do not drop it as dead code". This transititively
| > keeps alive all the other top-level bindings that this binding
| refers
| > to. This property is persisted all the way down the pipeline, so
| that
| > the binding will be compiled all the way to object code, and its
| > symbols will appear in the linker symbol table.
| >
| > However, note that this use of "exported" is quite different to the
| > export list on a Haskell module. Setting the ExportFlag on an Id
| does
| > /not/ mean that if you import the module (in Haskell source code you
| > will see this Id. Of course, things that appear in the export list
| of
| > the source Haskell module do indeed have their ExportFlag set.
| > But many other things, such as dictionary functions, are kept alive
| by
| > having their ExportFlag set, even though they are not exported in
| the
| > source-code sense.
| >
| > We should probably use a different term for ExportFlag, like
| > KeepAlive.
| >
| > From: ghc-devs [mailto:ghc-devs-bounces at haskell.org<mailto:ghc-devs-bounces at haskell.org>] On Behalf Of
| > Levent Erkok
| > Sent: 06 December 2015 20:32
| > To: Eric Seidel <eric at seidel.io<mailto:eric at seidel.io><mailto:eric at seidel.io<mailto:eric at seidel.io>>>;
| > omeragacan at gmail.com<mailto:omeragacan at gmail.com><mailto:omeragacan at gmail.com<mailto:omeragacan at gmail.com>>;
| > ezyang at mit.edu<mailto:ezyang at mit.edu><mailto:ezyang at mit.edu<mailto:ezyang at mit.edu>>
| > Cc: ghc-devs at haskell.org<mailto:ghc-devs at haskell.org><mailto:ghc-devs at haskell.org<mailto:ghc-devs at haskell.org>>
| > Subject: Re: Plugins: Accessing unexported bindings
| >
| > Omer, Eric, Ed: Thanks for the comments.
| >
| > Omer: I think Eric's observation is at play here. We're talking
| about
| > "dead-code," i.e., a binding that is neither exported, nor used by
| any
| > binding inside the module. Those seem to be getting dropped by the
| > time user-plugins are run. Unfortunately, this is precisely what one
| > would do with "properties" embedded in code. They serve as
| > documentation perhaps, but are otherwise not needed by any other
| > binding nor it makes sense to export them.
| >
| > Edward: Can you provide some more info into your solution? Sounds
| like
| > a chicken-egg issue to me: As a plugin author, I need the bindings
| to
| > access the Ids, and looks like I need the Ids to access the binders?
| >
| > A simple solution would be to simply keep all top-level bindings
| > around while the plugin are running, but that obviously can lead to
| > unnecessary work if the code is truly dead. A compromise could be
| that
| > the annotations can serve as entry points as well: I.e., if there's
| an
| > annotation on a top-level binder, then it should *not* be considered
| > dead-code at least until after all the plugins are run. That would
| > definitely simplify life. Would that be an acceptable alternative?
| >
| > In the mean time, I'm still looking for a solution that doesn't
| > involve exporting such identifiers from modules. As Eric pointed
| out,
| > that seems to be the only current work-around for the time being.
| >
| > Thanks,
| >
| > -Levent.
| >
| > On Sun, Dec 6, 2015 at 11:08 AM, Eric Seidel
| > <eric at seidel.io<mailto:eric at seidel.io><mailto:eric at seidel.io<mailto:eric at seidel.io>>> wrote:
| > GHC should only drop un-exported bindings from the ModGuts if
| they're
| > also unused, ie *dead code*.
| >
| > The only way I know to get around this is to use the bindings
| > somewhere, or just export them.
| >
| > On Sat, Dec 5, 2015, at 23:01, Levent Erkok wrote:
| > > Hello,
| > >
| > > The mg_binds field of the ModGuts seem to only contain the
| bindings
| > > that are exported from the module being compiled.
| > >
| > > I guess GHC must be running user-plugins after it drops the
| bindings
| > > that are not exported, which makes perfect sense for most use
| cases.
| > > However, I'm working on a plugin where the end-programmer embeds
| > > "properties" in the form of functions inside his/her code, which
| are
| > > not necessarily exported from the module under consideration.
| > >
| > > Is there a way to access all top-level bindings in a module from a
| > > plugin, even if those bindings are not exported?
| > >
| > > Thanks,
| > >
| > > -Levent.
| > > _______________________________________________
| > > ghc-devs mailing list
| > > ghc-devs at haskell.org<mailto:ghc-devs at haskell.org><mailto:ghc-devs at haskell.org<mailto:ghc-devs at haskell.org>>
| > >
| https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2fmail
| > > .haskell.org<https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2fhaskell.org&data=01%7c01%7csimonpj%40064d.mgd.microsoft.com%7ce3d9fc32f78e4a42bc8108d2ff80251a%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=qk2h3pMEX2WFCux6QLm4rXk%2faqydq4W13J5U%2bOPZKWo%3d>%2fcgi-bin%2fmailman%2flistinfo%2fghc-
| devs&data=01%7c01%
| > >
| 7csimonpj%40064d.mgd.microsoft.com<https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2f40064d.mgd.microsoft.com&data=01%7c01%7csimonpj%40064d.mgd.microsoft.com%7ce3d9fc32f78e4a42bc8108d2ff80251a%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=9om7aGmNLkTw%2bvUYADKPk7040LjeLAwT2DQz511Wi5M%3d>%7cf6e3a9d4ad9f4e53a3ab08d2ff1d493
| > >
| 4%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=Bv7lpsB%2fD88nSMuB7NY
| > >
| fBqR90%2bBq%2fwpJJ0JU9%2b6E4RI%3d<https://na01.safelinks.protection.
| > > outlook.com/?url=http%3a%2f%2fmail.haskell.org%2fcgi-<https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2foutlook.com%2f%3furl%3dhttp%253a%252f%252fmail.haskell.org%252fcgi-&data=01%7c01%7csimonpj%40064d.mgd.microsoft.com%7ce3d9fc32f78e4a42bc8108d2ff80251a%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=VxOGYDnOtuRJ0X%2fgNtZoxIA5lzCe9vIGSZNbq2Nj0Tc%3d>
| bin%2fmailman%2
| > > flistinfo%2fghc-
| devs&data=01%7c01%7csimonpj%40064d.mgd.microsoft.com<https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2f40064d.mgd.microsoft.com&data=01%7c01%7csimonpj%40064d.mgd.microsoft.com%7ce3d9fc32f78e4a42bc8108d2ff80251a%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=9om7aGmNLkTw%2bvUYADKPk7040LjeLAwT2DQz511Wi5M%3d>
| > >
| %7cac4cbfe22e314080909908d2fe7c4ed8%7c72f988bf86f141af91ab2d7cd011db
| > > 47%7c1&sdata=1z6DcZxjIAKj0PcsLeALphRLWJ3i%2fxvyaPtq0qo6elY%3d>
| > _______________________________________________
| > ghc-devs mailing list
| > ghc-devs at haskell.org<mailto:ghc-devs at haskell.org><mailto:ghc-devs at haskell.org<mailto:ghc-devs at haskell.org>>
| >
| https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2fmail.h
| > askell.org<https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2faskell.org&data=01%7c01%7csimonpj%40064d.mgd.microsoft.com%7ce3d9fc32f78e4a42bc8108d2ff80251a%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=GX53BhjqknGuX4oZVWqLVbz%2b7Up6NY9YLKjWTkwoeck%3d>%2fcgi-bin%2fmailman%2flistinfo%2fghc-
| devs&data=01%7c01%7csi
| >
| monpj%40064d.mgd.microsoft.com<https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2f40064d.mgd.microsoft.com&data=01%7c01%7csimonpj%40064d.mgd.microsoft.com%7ce3d9fc32f78e4a42bc8108d2ff80251a%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=9om7aGmNLkTw%2bvUYADKPk7040LjeLAwT2DQz511Wi5M%3d>%7cf6e3a9d4ad9f4e53a3ab08d2ff1d4934%7c72
| >
| f988bf86f141af91ab2d7cd011db47%7c1&sdata=Bv7lpsB%2fD88nSMuB7NYfBqR90%2
| >
| bBq%2fwpJJ0JU9%2b6E4RI%3d<https://na01.safelinks.protection.outlook.co
| > m/?url=http%3a%2f%2fmail.haskell.org<https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2f2fmail.haskell.org&data=01%7c01%7csimonpj%40064d.mgd.microsoft.com%7ce3d9fc32f78e4a42bc8108d2ff80251a%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=BiZsEZyyTCvpit0cBcPceGv2E6hQ172kSrf6kcDp0so%3d>%2fcgi-
| bin%2fmailman%2flistinfo%2f
| > ghc-
| devs&data=01%7c01%7csimonpj%40064d.mgd.microsoft.com<https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2f40064d.mgd.microsoft.com&data=01%7c01%7csimonpj%40064d.mgd.microsoft.com%7ce3d9fc32f78e4a42bc8108d2ff80251a%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=9om7aGmNLkTw%2bvUYADKPk7040LjeLAwT2DQz511Wi5M%3d>%7cac4cbfe22e3
| >
| 14080909908d2fe7c4ed8%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=1z6
| > DcZxjIAKj0PcsLeALphRLWJ3i%2fxvyaPtq0qo6elY%3d>
| >
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20151208/95eb60e4/attachment-0001.html>
More information about the ghc-devs
mailing list