<div dir="ltr">Thanks for the explanation, Simon. I think Eric is spot on. Looking at some of the "ghc -v3" output, it appears GHC always runs the following passes before any plugins are run:<div> </div><div><div> *** Simplify:</div><div> *** CorePrep:</div><div> *** ByteCodeGen:</div><div> *** Desugar:</div></div><div><br></div><div>Just judging by the name, it could either be the first "Simplify" or the "Desugar" that gets rid of the dead-code at this point, I'm not sure which.</div><div><br></div><div>It would help if this stage didn't remove dead-bindings. If doing so automatically is not the best option, requiring a user given pragma like 'KeepAlive' wouldn't be too onerous either.</div><div><br></div><div>-Levent.</div></div><div class="gmail_extra"><br><div class="gmail_quote">On Mon, Dec 7, 2015 at 8:22 AM, Simon Peyton Jones <span dir="ltr"><<a href="mailto:simonpj@microsoft.com" target="_blank">simonpj@microsoft.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">It would not be hard to stop the desugarer dropping dead bindings, if that was helpful.<br>
<br>
S<br>
<span class=""><br>
| -----Original Message-----<br>
| From: Eric Seidel [mailto:<a href="mailto:eric@seidel.io">eric@seidel.io</a>]<br>
| Sent: 07 December 2015 15:44<br>
| To: Simon Peyton Jones <<a href="mailto:simonpj@microsoft.com">simonpj@microsoft.com</a>><br>
</span><div><div class="h5">| Cc: Levent Erkok <<a href="mailto:erkokl@gmail.com">erkokl@gmail.com</a>>; <a href="mailto:omeragacan@gmail.com">omeragacan@gmail.com</a>;<br>
| <a href="mailto:ezyang@mit.edu">ezyang@mit.edu</a>; <a href="mailto:ghc-devs@haskell.org">ghc-devs@haskell.org</a><br>
| Subject: Re: Plugins: Accessing unexported bindings<br>
|<br>
| The problem, as I recall, is that GHC does an initial bit of dead-code<br>
| elimination in the desugarer, before the plugins have a chance to run.<br>
| (I believe this is part of simpleOptPgm, but may be mistaken)<br>
|<br>
| I'm not sure why this is done in the desugarer, it seems to be out of<br>
| place there.<br>
|<br>
| On Mon, Dec 7, 2015, at 05:14, Simon Peyton Jones wrote:<br>
| > Plugins get to edit the entire core-to-core pipeline! There is no<br>
| magic.<br>
| > At least I don’t think so<br>
| ><br>
| > file:///Z:/tmp/users_guide/compiler-plugins.html<br>
| ><br>
| > S<br>
| ><br>
| > From: Levent Erkok [mailto:<a href="mailto:erkokl@gmail.com">erkokl@gmail.com</a>]<br>
| > Sent: 07 December 2015 13:11<br>
| > To: Simon Peyton Jones <<a href="mailto:simonpj@microsoft.com">simonpj@microsoft.com</a>><br>
| > Cc: Eric Seidel <<a href="mailto:eric@seidel.io">eric@seidel.io</a>>; <a href="mailto:omeragacan@gmail.com">omeragacan@gmail.com</a>;<br>
| > <a href="mailto:ezyang@mit.edu">ezyang@mit.edu</a>; <a href="mailto:ghc-devs@haskell.org">ghc-devs@haskell.org</a><br>
| > Subject: Re: Plugins: Accessing unexported bindings<br>
| ><br>
| > That's a good point; keeping all annotated bindings alive seems to<br>
| be<br>
| > an overkill..<br>
| ><br>
| > Regarding implementing "pass at the start." I'm not sure if plugin<br>
| > authors have any freedom as to decide when their plugin actually<br>
| runs.<br>
| > It seems GHC magically determines the order and runs them. Can you<br>
| > point me to some code/docs that tells me how to go "first" in that<br>
| > sense? (Or at least before the pass that drops dead code.)<br>
| ><br>
| > On Dec 7, 2015, at 4:45 AM, Simon Peyton Jones<br>
| > <<a href="mailto:simonpj@microsoft.com">simonpj@microsoft.com</a><mailto:<a href="mailto:simonpj@microsoft.com">simonpj@microsoft.com</a>>> wrote:<br>
| > Indeed. How about this: if there's an ANN on a binder (any ANN),<br>
| then<br>
| > GHC should keep it alive.<br>
| ><br>
| > Really? It might be something like “don’t give warnings for this<br>
| > binding” or “don’t inline me” or something. To say *any*<br>
| annotation<br>
| > seems a bit brutal doesn’t it? Mind you I don’t have a better<br>
| idea.<br>
| ><br>
| > One thought: your plugin could add a pass right at the start, which<br>
| > marks everything you want as keep-alive.<br>
| ><br>
| > S<br>
| ><br>
| > From: Levent Erkok [mailto:<a href="mailto:erkokl@gmail.com">erkokl@gmail.com</a>]<br>
| > Sent: 07 December 2015 12:42<br>
| > To: Simon Peyton Jones<br>
| > <<a href="mailto:simonpj@microsoft.com">simonpj@microsoft.com</a><mailto:<a href="mailto:simonpj@microsoft.com">simonpj@microsoft.com</a>>><br>
| > Cc: Eric Seidel <<a href="mailto:eric@seidel.io">eric@seidel.io</a><mailto:<a href="mailto:eric@seidel.io">eric@seidel.io</a>>>;<br>
| > <a href="mailto:omeragacan@gmail.com">omeragacan@gmail.com</a><mailto:<a href="mailto:omeragacan@gmail.com">omeragacan@gmail.com</a>>;<br>
| > <a href="mailto:ezyang@mit.edu">ezyang@mit.edu</a><mailto:<a href="mailto:ezyang@mit.edu">ezyang@mit.edu</a>>;<br>
| > <a href="mailto:ghc-devs@haskell.org">ghc-devs@haskell.org</a><mailto:<a href="mailto:ghc-devs@haskell.org">ghc-devs@haskell.org</a>><br>
| > Subject: Re: Plugins: Accessing unexported bindings<br>
| ><br>
| > Indeed. How about this: if there's an ANN on a binder (any ANN),<br>
| then<br>
| > GHC should keep it alive.<br>
| ><br>
| > Is that something one of the core-developers can implement? Happy to<br>
| > open a ticket if that helps.<br>
| ><br>
| > On Dec 7, 2015, at 4:14 AM, Simon Peyton Jones<br>
| > <<a href="mailto:simonpj@microsoft.com">simonpj@microsoft.com</a><mailto:<a href="mailto:simonpj@microsoft.com">simonpj@microsoft.com</a>>> wrote:<br>
| > If it's "dead" in this sense, it's already removed from ModGuts, no?<br>
| ><br>
| > Yes, if it’s dead it’s gone. That’s not too surprising, is it?<br>
| ><br>
| > So you need a way to keep it alive. Maybe we need a pragma for that.<br>
| Or<br>
| > how would you like to signal it in the source code?<br>
| ><br>
| > Simon<br>
| ><br>
| > From: Levent Erkok [mailto:<a href="mailto:erkokl@gmail.com">erkokl@gmail.com</a>]<br>
| > Sent: 07 December 2015 12:05<br>
| > To: Simon Peyton Jones<br>
| > <<a href="mailto:simonpj@microsoft.com">simonpj@microsoft.com</a><mailto:<a href="mailto:simonpj@microsoft.com">simonpj@microsoft.com</a>>><br>
| > Cc: Eric Seidel <<a href="mailto:eric@seidel.io">eric@seidel.io</a><mailto:<a href="mailto:eric@seidel.io">eric@seidel.io</a>>>;<br>
| > <a href="mailto:omeragacan@gmail.com">omeragacan@gmail.com</a><mailto:<a href="mailto:omeragacan@gmail.com">omeragacan@gmail.com</a>>;<br>
| > <a href="mailto:ezyang@mit.edu">ezyang@mit.edu</a><mailto:<a href="mailto:ezyang@mit.edu">ezyang@mit.edu</a>>;<br>
| > <a href="mailto:ghc-devs@haskell.org">ghc-devs@haskell.org</a><mailto:<a href="mailto:ghc-devs@haskell.org">ghc-devs@haskell.org</a>><br>
| > Subject: Re: Plugins: Accessing unexported bindings<br>
| ><br>
| > Thanks Simon.. But I remain utterly confused. As a "plugin" author,<br>
| > how do I get my hands on the Id associated with a top-level binder?<br>
| If<br>
| > it's "dead" in this sense, it's already removed from ModGuts, no?<br>
| ><br>
| > That is, by the time GHC runs my plugin, the Id has already<br>
| > disappeared for me to mark it "Local Exported." Is that not correct?<br>
| ><br>
| > On Dec 7, 2015, at 2:28 AM, Simon Peyton Jones<br>
| > <<a href="mailto:simonpj@microsoft.com">simonpj@microsoft.com</a><mailto:<a href="mailto:simonpj@microsoft.com">simonpj@microsoft.com</a>>> wrote:<br>
| > In the mean time, I'm still looking for a solution that doesn't<br>
| > involve exporting such identifiers from modules. As Eric pointed<br>
| out,<br>
| > that seems to be the only current work-around for the time being.<br>
| ><br>
| > “Exported” in this context only means “keep alive”. It does not mean<br>
| > exported in the Haskell source code sense. I’ve just added this<br>
| > comment to Var.hs.<br>
| ><br>
| > So I think it does just what you want.<br>
| ><br>
| > Simon<br>
| ><br>
| > data ExportFlag -- See Note [ExportFlag on binders]<br>
| > = NotExported -- ^ Not exported: may be discarded as dead code.<br>
| > | Exported -- ^ Exported: kept alive<br>
| ><br>
| > {- Note [ExportFlag on binders]<br>
| > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~<br>
| > An ExportFlag of "Exported" on a top-level binder says "keep this<br>
| > binding alive; do not drop it as dead code". This transititively<br>
| > keeps alive all the other top-level bindings that this binding<br>
| refers<br>
| > to. This property is persisted all the way down the pipeline, so<br>
| that<br>
| > the binding will be compiled all the way to object code, and its<br>
| > symbols will appear in the linker symbol table.<br>
| ><br>
| > However, note that this use of "exported" is quite different to the<br>
| > export list on a Haskell module. Setting the ExportFlag on an Id<br>
| does<br>
| > /not/ mean that if you import the module (in Haskell source code you<br>
| > will see this Id. Of course, things that appear in the export list<br>
| of<br>
| > the source Haskell module do indeed have their ExportFlag set.<br>
| > But many other things, such as dictionary functions, are kept alive<br>
| by<br>
| > having their ExportFlag set, even though they are not exported in<br>
| the<br>
| > source-code sense.<br>
| ><br>
| > We should probably use a different term for ExportFlag, like<br>
| > KeepAlive.<br>
| ><br>
| > From: ghc-devs [mailto:<a href="mailto:ghc-devs-bounces@haskell.org">ghc-devs-bounces@haskell.org</a>] On Behalf Of<br>
| > Levent Erkok<br>
| > Sent: 06 December 2015 20:32<br>
| > To: Eric Seidel <<a href="mailto:eric@seidel.io">eric@seidel.io</a><mailto:<a href="mailto:eric@seidel.io">eric@seidel.io</a>>>;<br>
| > <a href="mailto:omeragacan@gmail.com">omeragacan@gmail.com</a><mailto:<a href="mailto:omeragacan@gmail.com">omeragacan@gmail.com</a>>;<br>
| > <a href="mailto:ezyang@mit.edu">ezyang@mit.edu</a><mailto:<a href="mailto:ezyang@mit.edu">ezyang@mit.edu</a>><br>
| > Cc: <a href="mailto:ghc-devs@haskell.org">ghc-devs@haskell.org</a><mailto:<a href="mailto:ghc-devs@haskell.org">ghc-devs@haskell.org</a>><br>
| > Subject: Re: Plugins: Accessing unexported bindings<br>
| ><br>
| > Omer, Eric, Ed: Thanks for the comments.<br>
| ><br>
| > Omer: I think Eric's observation is at play here. We're talking<br>
| about<br>
| > "dead-code," i.e., a binding that is neither exported, nor used by<br>
| any<br>
| > binding inside the module. Those seem to be getting dropped by the<br>
| > time user-plugins are run. Unfortunately, this is precisely what one<br>
| > would do with "properties" embedded in code. They serve as<br>
| > documentation perhaps, but are otherwise not needed by any other<br>
| > binding nor it makes sense to export them.<br>
| ><br>
| > Edward: Can you provide some more info into your solution? Sounds<br>
| like<br>
| > a chicken-egg issue to me: As a plugin author, I need the bindings<br>
| to<br>
| > access the Ids, and looks like I need the Ids to access the binders?<br>
| ><br>
| > A simple solution would be to simply keep all top-level bindings<br>
| > around while the plugin are running, but that obviously can lead to<br>
| > unnecessary work if the code is truly dead. A compromise could be<br>
| that<br>
| > the annotations can serve as entry points as well: I.e., if there's<br>
| an<br>
| > annotation on a top-level binder, then it should *not* be considered<br>
| > dead-code at least until after all the plugins are run. That would<br>
| > definitely simplify life. Would that be an acceptable alternative?<br>
| ><br>
| > In the mean time, I'm still looking for a solution that doesn't<br>
| > involve exporting such identifiers from modules. As Eric pointed<br>
| out,<br>
| > that seems to be the only current work-around for the time being.<br>
| ><br>
| > Thanks,<br>
| ><br>
| > -Levent.<br>
| ><br>
| > On Sun, Dec 6, 2015 at 11:08 AM, Eric Seidel<br>
| > <<a href="mailto:eric@seidel.io">eric@seidel.io</a><mailto:<a href="mailto:eric@seidel.io">eric@seidel.io</a>>> wrote:<br>
| > GHC should only drop un-exported bindings from the ModGuts if<br>
| they're<br>
| > also unused, ie *dead code*.<br>
| ><br>
| > The only way I know to get around this is to use the bindings<br>
| > somewhere, or just export them.<br>
| ><br>
| > On Sat, Dec 5, 2015, at 23:01, Levent Erkok wrote:<br>
| > > Hello,<br>
| > ><br>
| > > The mg_binds field of the ModGuts seem to only contain the<br>
| bindings<br>
| > > that are exported from the module being compiled.<br>
| > ><br>
| > > I guess GHC must be running user-plugins after it drops the<br>
| bindings<br>
| > > that are not exported, which makes perfect sense for most use<br>
| cases.<br>
| > > However, I'm working on a plugin where the end-programmer embeds<br>
| > > "properties" in the form of functions inside his/her code, which<br>
| are<br>
| > > not necessarily exported from the module under consideration.<br>
| > ><br>
| > > Is there a way to access all top-level bindings in a module from a<br>
| > > plugin, even if those bindings are not exported?<br>
| > ><br>
| > > Thanks,<br>
| > ><br>
| > > -Levent.<br>
| > > _______________________________________________<br>
| > > ghc-devs mailing list<br>
| > > <a href="mailto:ghc-devs@haskell.org">ghc-devs@haskell.org</a><mailto:<a href="mailto:ghc-devs@haskell.org">ghc-devs@haskell.org</a>><br>
| > ><br>
</div></div>| <a href="https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2fmail" rel="noreferrer" target="_blank">https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2fmail</a><br>
| > > .<a href="http://haskell.org" rel="noreferrer" target="_blank">haskell.org</a>%2fcgi-bin%2fmailman%2flistinfo%2fghc-<br>
| devs&data=01%7c01%<br>
| > ><br>
| 7csimonpj%<a href="http://40064d.mgd.microsoft.com" rel="noreferrer" target="_blank">40064d.mgd.microsoft.com</a>%7cf6e3a9d4ad9f4e53a3ab08d2ff1d493<br>
| > ><br>
| 4%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=Bv7lpsB%2fD88nSMuB7NY<br>
| > ><br>
| fBqR90%2bBq%2fwpJJ0JU9%2b6E4RI%3d<<a href="https://na01.safelinks.protection" rel="noreferrer" target="_blank">https://na01.safelinks.protection</a>.<br>
| > > <a href="http://outlook.com/?url=http%3a%2f%2fmail.haskell.org%2fcgi-" rel="noreferrer" target="_blank">outlook.com/?url=http%3a%2f%2fmail.haskell.org%2fcgi-</a><br>
| bin%2fmailman%2<br>
| > > flistinfo%2fghc-<br>
| devs&data=01%7c01%7csimonpj%<a href="http://40064d.mgd.microsoft.com" rel="noreferrer" target="_blank">40064d.mgd.microsoft.com</a><br>
| > ><br>
| %7cac4cbfe22e314080909908d2fe7c4ed8%7c72f988bf86f141af91ab2d7cd011db<br>
| > > 47%7c1&sdata=1z6DcZxjIAKj0PcsLeALphRLWJ3i%2fxvyaPtq0qo6elY%3d><br>
<span class="">| > _______________________________________________<br>
| > ghc-devs mailing list<br>
| > <a href="mailto:ghc-devs@haskell.org">ghc-devs@haskell.org</a><mailto:<a href="mailto:ghc-devs@haskell.org">ghc-devs@haskell.org</a>><br>
| ><br>
</span>| <a href="https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2fmail.h" rel="noreferrer" target="_blank">https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2fmail.h</a><br>
| > <a href="http://askell.org" rel="noreferrer" target="_blank">askell.org</a>%2fcgi-bin%2fmailman%2flistinfo%2fghc-<br>
| devs&data=01%7c01%7csi<br>
| ><br>
| monpj%<a href="http://40064d.mgd.microsoft.com" rel="noreferrer" target="_blank">40064d.mgd.microsoft.com</a>%7cf6e3a9d4ad9f4e53a3ab08d2ff1d4934%7c72<br>
| ><br>
| f988bf86f141af91ab2d7cd011db47%7c1&sdata=Bv7lpsB%2fD88nSMuB7NYfBqR90%2<br>
| ><br>
| bBq%2fwpJJ0JU9%2b6E4RI%3d<<a href="https://na01.safelinks.protection.outlook.co" rel="noreferrer" target="_blank">https://na01.safelinks.protection.outlook.co</a><br>
| > m/?url=http%3a%2f%<a href="http://2fmail.haskell.org" rel="noreferrer" target="_blank">2fmail.haskell.org</a>%2fcgi-<br>
| bin%2fmailman%2flistinfo%2f<br>
| > ghc-<br>
| devs&data=01%7c01%7csimonpj%<a href="http://40064d.mgd.microsoft.com" rel="noreferrer" target="_blank">40064d.mgd.microsoft.com</a>%7cac4cbfe22e3<br>
| ><br>
| 14080909908d2fe7c4ed8%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=1z6<br>
| > DcZxjIAKj0PcsLeALphRLWJ3i%2fxvyaPtq0qo6elY%3d><br>
| ><br>
</blockquote></div><br></div>