Plugins: Accessing unexported bindings

Levent Erkok erkokl at gmail.com
Tue Dec 8 17:38:07 UTC 2015


http://ghc.haskell.org/trac/ghc/ticket/11179#ticket

Thanks!

-Levent.

On Tue, Dec 8, 2015 at 4:33 AM, Simon Peyton Jones <simonpj at microsoft.com>
wrote:

> I think it’s probably the desugarer, and the plugin can’t do anything
> before desugaring!
>
>
>
> By all means open a ticket.  I can advise if anyone wants to take it up.
>
>
>
> Simon
>
>
>
> *From:* Levent Erkok [mailto:erkokl at gmail.com]
> *Sent:* 08 December 2015 03:32
>
> *To:* Simon Peyton Jones <simonpj at microsoft.com>
> *Cc:* Eric Seidel <eric at seidel.io>; omeragacan at gmail.com; ezyang at mit.edu;
> ghc-devs at haskell.org
> *Subject:* Re: Plugins: Accessing unexported bindings
>
>
>
> Thanks for the explanation, Simon. I think Eric is spot on. Looking at
> some of the "ghc -v3" output, it appears GHC always runs the following
> passes before any plugins are run:
>
>
>
>    *** Simplify:
>
>    *** CorePrep:
>
>    *** ByteCodeGen:
>
>    *** Desugar:
>
>
>
> Just judging by the name, it could either be the first "Simplify" or the
> "Desugar" that gets rid of the dead-code at this point, I'm not sure which.
>
>
>
> It would help if this stage didn't remove dead-bindings. If doing so
> automatically is not the best option, requiring a user given pragma like
> 'KeepAlive' wouldn't be too onerous either.
>
>
>
> -Levent.
>
>
>
> On Mon, Dec 7, 2015 at 8:22 AM, Simon Peyton Jones <simonpj at microsoft.com>
> wrote:
>
> It would not be hard to stop the desugarer dropping dead bindings, if that
> was helpful.
>
> S
>
> |  -----Original Message-----
> |  From: Eric Seidel [mailto:eric at seidel.io]
> |  Sent: 07 December 2015 15:44
> |  To: Simon Peyton Jones <simonpj at microsoft.com>
>
> |  Cc: Levent Erkok <erkokl at gmail.com>; omeragacan at gmail.com;
> |  ezyang at mit.edu; ghc-devs at haskell.org
> |  Subject: Re: Plugins: Accessing unexported bindings
> |
> |  The problem, as I recall, is that GHC does an initial bit of dead-code
> |  elimination in the desugarer, before the plugins have a chance to run.
> |  (I believe this is part of simpleOptPgm, but may be mistaken)
> |
> |  I'm not sure why this is done in the desugarer, it seems to be out of
> |  place there.
> |
> |  On Mon, Dec 7, 2015, at 05:14, Simon Peyton Jones wrote:
> |  > Plugins get to edit the entire core-to-core pipeline!  There is no
> |  magic.
> |  >  At least I don’t think so
> |  >
> |  > file:///Z:/tmp/users_guide/compiler-plugins.html
> |  >
> |  > S
> |  >
> |  > From: Levent Erkok [mailto:erkokl at gmail.com]
> |  > Sent: 07 December 2015 13:11
> |  > To: Simon Peyton Jones <simonpj at microsoft.com>
> |  > Cc: Eric Seidel <eric at seidel.io>; omeragacan at gmail.com;
> |  > ezyang at mit.edu; ghc-devs at haskell.org
> |  > Subject: Re: Plugins: Accessing unexported bindings
> |  >
> |  > That's a good point; keeping all annotated bindings alive seems to
> |  be
> |  > an overkill..
> |  >
> |  > Regarding implementing "pass at the start." I'm not sure if plugin
> |  > authors have any freedom as to decide when their plugin actually
> |  runs.
> |  > It seems GHC magically determines the order and runs them. Can you
> |  > point me to some code/docs that tells me how to go "first" in that
> |  > sense? (Or at least before the pass that drops dead code.)
> |  >
> |  > On Dec 7, 2015, at 4:45 AM, Simon Peyton Jones
> |  > <simonpj at microsoft.com<mailto:simonpj at microsoft.com>> wrote:
> |  > Indeed. How about this: if there's an ANN on a binder (any ANN),
> |  then
> |  > GHC should keep it alive.
> |  >
> |  > Really?  It might be something like “don’t give warnings for this
> |  > binding” or “don’t inline me” or something.   To say *any*
> |  annotation
> |  > seems a bit brutal doesn’t it?    Mind you I don’t have a better
> |  idea.
> |  >
> |  > One thought: your plugin could add a pass right at the start, which
> |  > marks everything you want as keep-alive.
> |  >
> |  > S
> |  >
> |  > From: Levent Erkok [mailto:erkokl at gmail.com]
> |  > Sent: 07 December 2015 12:42
> |  > To: Simon Peyton Jones
> |  > <simonpj at microsoft.com<mailto:simonpj at microsoft.com>>
> |  > Cc: Eric Seidel <eric at seidel.io<mailto:eric at seidel.io>>;
> |  > omeragacan at gmail.com<mailto:omeragacan at gmail.com>;
> |  > ezyang at mit.edu<mailto:ezyang at mit.edu>;
> |  > ghc-devs at haskell.org<mailto:ghc-devs at haskell.org>
> |  > Subject: Re: Plugins: Accessing unexported bindings
> |  >
> |  > Indeed. How about this: if there's an ANN on a binder (any ANN),
> |  then
> |  > GHC should keep it alive.
> |  >
> |  > Is that something one of the core-developers can implement? Happy to
> |  > open a ticket if that helps.
> |  >
> |  > On Dec 7, 2015, at 4:14 AM, Simon Peyton Jones
> |  > <simonpj at microsoft.com<mailto:simonpj at microsoft.com>> wrote:
> |  > If it's "dead" in this sense, it's already removed from ModGuts, no?
> |  >
> |  > Yes, if it’s dead it’s gone.   That’s not too surprising, is it?
> |  >
> |  > So you need a way to keep it alive. Maybe we need a pragma for that.
> |  Or
> |  > how would you like to signal it in the source code?
> |  >
> |  > Simon
> |  >
> |  > From: Levent Erkok [mailto:erkokl at gmail.com]
> |  > Sent: 07 December 2015 12:05
> |  > To: Simon Peyton Jones
> |  > <simonpj at microsoft.com<mailto:simonpj at microsoft.com>>
> |  > Cc: Eric Seidel <eric at seidel.io<mailto:eric at seidel.io>>;
> |  > omeragacan at gmail.com<mailto:omeragacan at gmail.com>;
> |  > ezyang at mit.edu<mailto:ezyang at mit.edu>;
> |  > ghc-devs at haskell.org<mailto:ghc-devs at haskell.org>
> |  > Subject: Re: Plugins: Accessing unexported bindings
> |  >
> |  > Thanks Simon.. But I remain utterly confused. As a "plugin" author,
> |  > how do I get my hands on the Id associated with a top-level binder?
> |  If
> |  > it's "dead" in this sense, it's already removed from ModGuts, no?
> |  >
> |  > That is, by the time GHC runs my plugin, the Id has already
> |  > disappeared for me to mark it "Local Exported." Is that not correct?
> |  >
> |  > On Dec 7, 2015, at 2:28 AM, Simon Peyton Jones
> |  > <simonpj at microsoft.com<mailto:simonpj at microsoft.com>> wrote:
> |  > In the mean time, I'm still looking for a solution that doesn't
> |  > involve exporting such identifiers from modules. As Eric pointed
> |  out,
> |  > that seems to be the only current work-around for the time being.
> |  >
> |  > “Exported” in this context only means “keep alive”. It does not mean
> |  > exported in the Haskell source code sense. I’ve just added this
> |  > comment to Var.hs.
> |  >
> |  > So I think it does just what you want.
> |  >
> |  > Simon
> |  >
> |  > data ExportFlag   -- See Note [ExportFlag on binders]
> |  >   = NotExported   -- ^ Not exported: may be discarded as dead code.
> |  >   | Exported      -- ^ Exported: kept alive
> |  >
> |  > {- Note [ExportFlag on binders]
> |  > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> |  > An ExportFlag of "Exported" on a top-level binder says "keep this
> |  > binding alive; do not drop it as dead code".  This transititively
> |  > keeps alive all the other top-level bindings that this binding
> |  refers
> |  > to.  This property is persisted all the way down the pipeline, so
> |  that
> |  > the binding will be compiled all the way to object code, and its
> |  > symbols will appear in the linker symbol table.
> |  >
> |  > However, note that this use of "exported" is quite different to the
> |  > export list on a Haskell module.  Setting the ExportFlag on an Id
> |  does
> |  > /not/ mean that if you import the module (in Haskell source code you
> |  > will see this Id.  Of course, things that appear in the export list
> |  of
> |  > the source Haskell module do indeed have their ExportFlag set.
> |  > But many other things, such as dictionary functions, are kept alive
> |  by
> |  > having their ExportFlag set, even though they are not exported in
> |  the
> |  > source-code sense.
> |  >
> |  > We should probably use a different term for ExportFlag, like
> |  > KeepAlive.
> |  >
> |  > From: ghc-devs [mailto:ghc-devs-bounces at haskell.org] On Behalf Of
> |  > Levent Erkok
> |  > Sent: 06 December 2015 20:32
> |  > To: Eric Seidel <eric at seidel.io<mailto:eric at seidel.io>>;
> |  > omeragacan at gmail.com<mailto:omeragacan at gmail.com>;
> |  > ezyang at mit.edu<mailto:ezyang at mit.edu>
> |  > Cc: ghc-devs at haskell.org<mailto:ghc-devs at haskell.org>
> |  > Subject: Re: Plugins: Accessing unexported bindings
> |  >
> |  > Omer, Eric, Ed: Thanks for the comments.
> |  >
> |  > Omer: I think Eric's observation is at play here. We're talking
> |  about
> |  > "dead-code," i.e., a binding that is neither exported, nor used by
> |  any
> |  > binding inside the module. Those seem to be getting dropped by the
> |  > time user-plugins are run. Unfortunately, this is precisely what one
> |  > would do with "properties" embedded in code. They serve as
> |  > documentation perhaps, but are otherwise not needed by any other
> |  > binding nor it makes sense to export them.
> |  >
> |  > Edward: Can you provide some more info into your solution? Sounds
> |  like
> |  > a chicken-egg issue to me: As a plugin author, I need the bindings
> |  to
> |  > access the Ids, and looks like I need the Ids to access the binders?
> |  >
> |  > A simple solution would be to simply keep all top-level bindings
> |  > around while the plugin are running, but that obviously can lead to
> |  > unnecessary work if the code is truly dead. A compromise could be
> |  that
> |  > the annotations can serve as entry points as well: I.e., if there's
> |  an
> |  > annotation on a top-level binder, then it should *not* be considered
> |  > dead-code at least until after all the plugins are run. That would
> |  > definitely simplify life. Would that be an acceptable alternative?
> |  >
> |  > In the mean time, I'm still looking for a solution that doesn't
> |  > involve exporting such identifiers from modules. As Eric pointed
> |  out,
> |  > that seems to be the only current work-around for the time being.
> |  >
> |  > Thanks,
> |  >
> |  > -Levent.
> |  >
> |  > On Sun, Dec 6, 2015 at 11:08 AM, Eric Seidel
> |  > <eric at seidel.io<mailto:eric at seidel.io>> wrote:
> |  > GHC should only drop un-exported bindings from the ModGuts if
> |  they're
> |  > also unused, ie *dead code*.
> |  >
> |  > The only way I know to get around this is to use the bindings
> |  > somewhere, or just export them.
> |  >
> |  > On Sat, Dec 5, 2015, at 23:01, Levent Erkok wrote:
> |  > > Hello,
> |  > >
> |  > > The mg_binds field of the ModGuts seem to only contain the
> |  bindings
> |  > > that are exported from the module being compiled.
> |  > >
> |  > > I guess GHC must be running user-plugins after it drops the
> |  bindings
> |  > > that are not exported, which makes perfect sense for most use
> |  cases.
> |  > > However, I'm working on a plugin where the end-programmer embeds
> |  > > "properties" in the form of functions inside his/her code, which
> |  are
> |  > > not necessarily exported from the module under consideration.
> |  > >
> |  > > Is there a way to access all top-level bindings in a module from a
> |  > > plugin, even if those bindings are not exported?
> |  > >
> |  > > Thanks,
> |  > >
> |  > > -Levent.
> |  > > _______________________________________________
> |  > > ghc-devs mailing list
> |  > > ghc-devs at haskell.org<mailto:ghc-devs at haskell.org>
> |  > >
>
> |  https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2fmail
> |  > > .haskell.org
> <https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2fhaskell.org&data=01%7c01%7csimonpj%40064d.mgd.microsoft.com%7ce3d9fc32f78e4a42bc8108d2ff80251a%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=qk2h3pMEX2WFCux6QLm4rXk%2faqydq4W13J5U%2bOPZKWo%3d>
> %2fcgi-bin%2fmailman%2flistinfo%2fghc-
> |  devs&data=01%7c01%
> |  > >
> |  7csimonpj%40064d.mgd.microsoft.com
> <https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2f40064d.mgd.microsoft.com&data=01%7c01%7csimonpj%40064d.mgd.microsoft.com%7ce3d9fc32f78e4a42bc8108d2ff80251a%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=9om7aGmNLkTw%2bvUYADKPk7040LjeLAwT2DQz511Wi5M%3d>
> %7cf6e3a9d4ad9f4e53a3ab08d2ff1d493
> |  > >
> |  4%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=Bv7lpsB%2fD88nSMuB7NY
> |  > >
> |  fBqR90%2bBq%2fwpJJ0JU9%2b6E4RI%3d<https://na01.safelinks.protection.
> |  > > outlook.com/?url=http%3a%2f%2fmail.haskell.org%2fcgi-
> <https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2foutlook.com%2f%3furl%3dhttp%253a%252f%252fmail.haskell.org%252fcgi-&data=01%7c01%7csimonpj%40064d.mgd.microsoft.com%7ce3d9fc32f78e4a42bc8108d2ff80251a%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=VxOGYDnOtuRJ0X%2fgNtZoxIA5lzCe9vIGSZNbq2Nj0Tc%3d>
> |  bin%2fmailman%2
> |  > > flistinfo%2fghc-
> |  devs&data=01%7c01%7csimonpj%40064d.mgd.microsoft.com
> <https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2f40064d.mgd.microsoft.com&data=01%7c01%7csimonpj%40064d.mgd.microsoft.com%7ce3d9fc32f78e4a42bc8108d2ff80251a%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=9om7aGmNLkTw%2bvUYADKPk7040LjeLAwT2DQz511Wi5M%3d>
> |  > >
> |  %7cac4cbfe22e314080909908d2fe7c4ed8%7c72f988bf86f141af91ab2d7cd011db
> |  > > 47%7c1&sdata=1z6DcZxjIAKj0PcsLeALphRLWJ3i%2fxvyaPtq0qo6elY%3d>
> |  > _______________________________________________
> |  > ghc-devs mailing list
> |  > ghc-devs at haskell.org<mailto:ghc-devs at haskell.org>
> |  >
> |  https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2fmail.h
> |  > askell.org
> <https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2faskell.org&data=01%7c01%7csimonpj%40064d.mgd.microsoft.com%7ce3d9fc32f78e4a42bc8108d2ff80251a%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=GX53BhjqknGuX4oZVWqLVbz%2b7Up6NY9YLKjWTkwoeck%3d>
> %2fcgi-bin%2fmailman%2flistinfo%2fghc-
> |  devs&data=01%7c01%7csi
> |  >
> |  monpj%40064d.mgd.microsoft.com
> <https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2f40064d.mgd.microsoft.com&data=01%7c01%7csimonpj%40064d.mgd.microsoft.com%7ce3d9fc32f78e4a42bc8108d2ff80251a%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=9om7aGmNLkTw%2bvUYADKPk7040LjeLAwT2DQz511Wi5M%3d>
> %7cf6e3a9d4ad9f4e53a3ab08d2ff1d4934%7c72
> |  >
> |  f988bf86f141af91ab2d7cd011db47%7c1&sdata=Bv7lpsB%2fD88nSMuB7NYfBqR90%2
> |  >
> |  bBq%2fwpJJ0JU9%2b6E4RI%3d<https://na01.safelinks.protection.outlook.co
> |  > m/?url=http%3a%2f%2fmail.haskell.org
> <https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2f2fmail.haskell.org&data=01%7c01%7csimonpj%40064d.mgd.microsoft.com%7ce3d9fc32f78e4a42bc8108d2ff80251a%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=BiZsEZyyTCvpit0cBcPceGv2E6hQ172kSrf6kcDp0so%3d>
> %2fcgi-
> |  bin%2fmailman%2flistinfo%2f
> |  > ghc-
> |  devs&data=01%7c01%7csimonpj%40064d.mgd.microsoft.com
> <https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2f40064d.mgd.microsoft.com&data=01%7c01%7csimonpj%40064d.mgd.microsoft.com%7ce3d9fc32f78e4a42bc8108d2ff80251a%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=9om7aGmNLkTw%2bvUYADKPk7040LjeLAwT2DQz511Wi5M%3d>
> %7cac4cbfe22e3
> |  >
> |  14080909908d2fe7c4ed8%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=1z6
> |  > DcZxjIAKj0PcsLeALphRLWJ3i%2fxvyaPtq0qo6elY%3d>
> |  >
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20151208/5ce8c8b2/attachment-0001.html>


More information about the ghc-devs mailing list