Plugins: Accessing unexported bindings

Simon Peyton Jones simonpj at microsoft.com
Mon Dec 7 16:22:55 UTC 2015


It would not be hard to stop the desugarer dropping dead bindings, if that was helpful.

S

|  -----Original Message-----
|  From: Eric Seidel [mailto:eric at seidel.io]
|  Sent: 07 December 2015 15:44
|  To: Simon Peyton Jones <simonpj at microsoft.com>
|  Cc: Levent Erkok <erkokl at gmail.com>; omeragacan at gmail.com;
|  ezyang at mit.edu; ghc-devs at haskell.org
|  Subject: Re: Plugins: Accessing unexported bindings
|  
|  The problem, as I recall, is that GHC does an initial bit of dead-code
|  elimination in the desugarer, before the plugins have a chance to run.
|  (I believe this is part of simpleOptPgm, but may be mistaken)
|  
|  I'm not sure why this is done in the desugarer, it seems to be out of
|  place there.
|  
|  On Mon, Dec 7, 2015, at 05:14, Simon Peyton Jones wrote:
|  > Plugins get to edit the entire core-to-core pipeline!  There is no
|  magic.
|  >  At least I don’t think so
|  >
|  > file:///Z:/tmp/users_guide/compiler-plugins.html
|  >
|  > S
|  >
|  > From: Levent Erkok [mailto:erkokl at gmail.com]
|  > Sent: 07 December 2015 13:11
|  > To: Simon Peyton Jones <simonpj at microsoft.com>
|  > Cc: Eric Seidel <eric at seidel.io>; omeragacan at gmail.com;
|  > ezyang at mit.edu; ghc-devs at haskell.org
|  > Subject: Re: Plugins: Accessing unexported bindings
|  >
|  > That's a good point; keeping all annotated bindings alive seems to
|  be
|  > an overkill..
|  >
|  > Regarding implementing "pass at the start." I'm not sure if plugin
|  > authors have any freedom as to decide when their plugin actually
|  runs.
|  > It seems GHC magically determines the order and runs them. Can you
|  > point me to some code/docs that tells me how to go "first" in that
|  > sense? (Or at least before the pass that drops dead code.)
|  >
|  > On Dec 7, 2015, at 4:45 AM, Simon Peyton Jones
|  > <simonpj at microsoft.com<mailto:simonpj at microsoft.com>> wrote:
|  > Indeed. How about this: if there's an ANN on a binder (any ANN),
|  then
|  > GHC should keep it alive.
|  >
|  > Really?  It might be something like “don’t give warnings for this
|  > binding” or “don’t inline me” or something.   To say *any*
|  annotation
|  > seems a bit brutal doesn’t it?    Mind you I don’t have a better
|  idea.
|  >
|  > One thought: your plugin could add a pass right at the start, which
|  > marks everything you want as keep-alive.
|  >
|  > S
|  >
|  > From: Levent Erkok [mailto:erkokl at gmail.com]
|  > Sent: 07 December 2015 12:42
|  > To: Simon Peyton Jones
|  > <simonpj at microsoft.com<mailto:simonpj at microsoft.com>>
|  > Cc: Eric Seidel <eric at seidel.io<mailto:eric at seidel.io>>;
|  > omeragacan at gmail.com<mailto:omeragacan at gmail.com>;
|  > ezyang at mit.edu<mailto:ezyang at mit.edu>;
|  > ghc-devs at haskell.org<mailto:ghc-devs at haskell.org>
|  > Subject: Re: Plugins: Accessing unexported bindings
|  >
|  > Indeed. How about this: if there's an ANN on a binder (any ANN),
|  then
|  > GHC should keep it alive.
|  >
|  > Is that something one of the core-developers can implement? Happy to
|  > open a ticket if that helps.
|  >
|  > On Dec 7, 2015, at 4:14 AM, Simon Peyton Jones
|  > <simonpj at microsoft.com<mailto:simonpj at microsoft.com>> wrote:
|  > If it's "dead" in this sense, it's already removed from ModGuts, no?
|  >
|  > Yes, if it’s dead it’s gone.   That’s not too surprising, is it?
|  >
|  > So you need a way to keep it alive. Maybe we need a pragma for that.
|  Or
|  > how would you like to signal it in the source code?
|  >
|  > Simon
|  >
|  > From: Levent Erkok [mailto:erkokl at gmail.com]
|  > Sent: 07 December 2015 12:05
|  > To: Simon Peyton Jones
|  > <simonpj at microsoft.com<mailto:simonpj at microsoft.com>>
|  > Cc: Eric Seidel <eric at seidel.io<mailto:eric at seidel.io>>;
|  > omeragacan at gmail.com<mailto:omeragacan at gmail.com>;
|  > ezyang at mit.edu<mailto:ezyang at mit.edu>;
|  > ghc-devs at haskell.org<mailto:ghc-devs at haskell.org>
|  > Subject: Re: Plugins: Accessing unexported bindings
|  >
|  > Thanks Simon.. But I remain utterly confused. As a "plugin" author,
|  > how do I get my hands on the Id associated with a top-level binder?
|  If
|  > it's "dead" in this sense, it's already removed from ModGuts, no?
|  >
|  > That is, by the time GHC runs my plugin, the Id has already
|  > disappeared for me to mark it "Local Exported." Is that not correct?
|  >
|  > On Dec 7, 2015, at 2:28 AM, Simon Peyton Jones
|  > <simonpj at microsoft.com<mailto:simonpj at microsoft.com>> wrote:
|  > In the mean time, I'm still looking for a solution that doesn't
|  > involve exporting such identifiers from modules. As Eric pointed
|  out,
|  > that seems to be the only current work-around for the time being.
|  >
|  > “Exported” in this context only means “keep alive”. It does not mean
|  > exported in the Haskell source code sense. I’ve just added this
|  > comment to Var.hs.
|  >
|  > So I think it does just what you want.
|  >
|  > Simon
|  >
|  > data ExportFlag   -- See Note [ExportFlag on binders]
|  >   = NotExported   -- ^ Not exported: may be discarded as dead code.
|  >   | Exported      -- ^ Exported: kept alive
|  >
|  > {- Note [ExportFlag on binders]
|  > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|  > An ExportFlag of "Exported" on a top-level binder says "keep this
|  > binding alive; do not drop it as dead code".  This transititively
|  > keeps alive all the other top-level bindings that this binding
|  refers
|  > to.  This property is persisted all the way down the pipeline, so
|  that
|  > the binding will be compiled all the way to object code, and its
|  > symbols will appear in the linker symbol table.
|  >
|  > However, note that this use of "exported" is quite different to the
|  > export list on a Haskell module.  Setting the ExportFlag on an Id
|  does
|  > /not/ mean that if you import the module (in Haskell source code you
|  > will see this Id.  Of course, things that appear in the export list
|  of
|  > the source Haskell module do indeed have their ExportFlag set.
|  > But many other things, such as dictionary functions, are kept alive
|  by
|  > having their ExportFlag set, even though they are not exported in
|  the
|  > source-code sense.
|  >
|  > We should probably use a different term for ExportFlag, like
|  > KeepAlive.
|  >
|  > From: ghc-devs [mailto:ghc-devs-bounces at haskell.org] On Behalf Of
|  > Levent Erkok
|  > Sent: 06 December 2015 20:32
|  > To: Eric Seidel <eric at seidel.io<mailto:eric at seidel.io>>;
|  > omeragacan at gmail.com<mailto:omeragacan at gmail.com>;
|  > ezyang at mit.edu<mailto:ezyang at mit.edu>
|  > Cc: ghc-devs at haskell.org<mailto:ghc-devs at haskell.org>
|  > Subject: Re: Plugins: Accessing unexported bindings
|  >
|  > Omer, Eric, Ed: Thanks for the comments.
|  >
|  > Omer: I think Eric's observation is at play here. We're talking
|  about
|  > "dead-code," i.e., a binding that is neither exported, nor used by
|  any
|  > binding inside the module. Those seem to be getting dropped by the
|  > time user-plugins are run. Unfortunately, this is precisely what one
|  > would do with "properties" embedded in code. They serve as
|  > documentation perhaps, but are otherwise not needed by any other
|  > binding nor it makes sense to export them.
|  >
|  > Edward: Can you provide some more info into your solution? Sounds
|  like
|  > a chicken-egg issue to me: As a plugin author, I need the bindings
|  to
|  > access the Ids, and looks like I need the Ids to access the binders?
|  >
|  > A simple solution would be to simply keep all top-level bindings
|  > around while the plugin are running, but that obviously can lead to
|  > unnecessary work if the code is truly dead. A compromise could be
|  that
|  > the annotations can serve as entry points as well: I.e., if there's
|  an
|  > annotation on a top-level binder, then it should *not* be considered
|  > dead-code at least until after all the plugins are run. That would
|  > definitely simplify life. Would that be an acceptable alternative?
|  >
|  > In the mean time, I'm still looking for a solution that doesn't
|  > involve exporting such identifiers from modules. As Eric pointed
|  out,
|  > that seems to be the only current work-around for the time being.
|  >
|  > Thanks,
|  >
|  > -Levent.
|  >
|  > On Sun, Dec 6, 2015 at 11:08 AM, Eric Seidel
|  > <eric at seidel.io<mailto:eric at seidel.io>> wrote:
|  > GHC should only drop un-exported bindings from the ModGuts if
|  they're
|  > also unused, ie *dead code*.
|  >
|  > The only way I know to get around this is to use the bindings
|  > somewhere, or just export them.
|  >
|  > On Sat, Dec 5, 2015, at 23:01, Levent Erkok wrote:
|  > > Hello,
|  > >
|  > > The mg_binds field of the ModGuts seem to only contain the
|  bindings
|  > > that are exported from the module being compiled.
|  > >
|  > > I guess GHC must be running user-plugins after it drops the
|  bindings
|  > > that are not exported, which makes perfect sense for most use
|  cases.
|  > > However, I'm working on a plugin where the end-programmer embeds
|  > > "properties" in the form of functions inside his/her code, which
|  are
|  > > not necessarily exported from the module under consideration.
|  > >
|  > > Is there a way to access all top-level bindings in a module from a
|  > > plugin, even if those bindings are not exported?
|  > >
|  > > Thanks,
|  > >
|  > > -Levent.
|  > > _______________________________________________
|  > > ghc-devs mailing list
|  > > ghc-devs at haskell.org<mailto:ghc-devs at haskell.org>
|  > >
|  https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2fmail
|  > > .haskell.org%2fcgi-bin%2fmailman%2flistinfo%2fghc-
|  devs&data=01%7c01%
|  > >
|  7csimonpj%40064d.mgd.microsoft.com%7cf6e3a9d4ad9f4e53a3ab08d2ff1d493
|  > >
|  4%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=Bv7lpsB%2fD88nSMuB7NY
|  > >
|  fBqR90%2bBq%2fwpJJ0JU9%2b6E4RI%3d<https://na01.safelinks.protection.
|  > > outlook.com/?url=http%3a%2f%2fmail.haskell.org%2fcgi-
|  bin%2fmailman%2
|  > > flistinfo%2fghc-
|  devs&data=01%7c01%7csimonpj%40064d.mgd.microsoft.com
|  > >
|  %7cac4cbfe22e314080909908d2fe7c4ed8%7c72f988bf86f141af91ab2d7cd011db
|  > > 47%7c1&sdata=1z6DcZxjIAKj0PcsLeALphRLWJ3i%2fxvyaPtq0qo6elY%3d>
|  > _______________________________________________
|  > ghc-devs mailing list
|  > ghc-devs at haskell.org<mailto:ghc-devs at haskell.org>
|  >
|  https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2fmail.h
|  > askell.org%2fcgi-bin%2fmailman%2flistinfo%2fghc-
|  devs&data=01%7c01%7csi
|  >
|  monpj%40064d.mgd.microsoft.com%7cf6e3a9d4ad9f4e53a3ab08d2ff1d4934%7c72
|  >
|  f988bf86f141af91ab2d7cd011db47%7c1&sdata=Bv7lpsB%2fD88nSMuB7NYfBqR90%2
|  >
|  bBq%2fwpJJ0JU9%2b6E4RI%3d<https://na01.safelinks.protection.outlook.co
|  > m/?url=http%3a%2f%2fmail.haskell.org%2fcgi-
|  bin%2fmailman%2flistinfo%2f
|  > ghc-
|  devs&data=01%7c01%7csimonpj%40064d.mgd.microsoft.com%7cac4cbfe22e3
|  >
|  14080909908d2fe7c4ed8%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=1z6
|  > DcZxjIAKj0PcsLeALphRLWJ3i%2fxvyaPtq0qo6elY%3d>
|  >


More information about the ghc-devs mailing list