<div dir="ltr"><div class="gmail_quote"><div dir="ltr">On Sun, May 28, 2017 at 11:30 PM Simon Peyton Jones <<a href="mailto:simonpj@microsoft.com">simonpj@microsoft.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div lang="EN-GB" link="blue" vlink="purple">
<div class="m_5429673471515783493WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">Is there really a compelling case for forking Hoopl? I was talking to Kavon last week about doing exactly the opposite: using Hoopl more wholeheartedly!<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">Before going ahead with this, let’s remember the downsides<u></u><u></u></span></p>
<p class="m_5429673471515783493MsoListParagraph"><u></u><span style="font-size:11.0pt;font-family:Symbol"><span>·<span style="font:7.0pt "Times New Roman"">
</span></span></span><u></u><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">If we fork Hoopl, improvements in one place will not be seen in the other. GHC originally used its own containers library but now uses
‘containers’, most of which is irrelevant to GHC, just to pick up the work that has been done to make ‘containers’ fast. Similarly, GHC has a clone of ‘pretty’, but someone is working (I think) to make GHC use ‘pretty’.<u></u><u></u></span></p>
<p class="m_5429673471515783493MsoListParagraph"><u></u><span style="font-size:11.0pt;font-family:Symbol"><span>·<span style="font:7.0pt "Times New Roman"">
</span></span></span><u></u><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">It’s not clear to me why GHC has a clone of parts of Hoopl. Would it not be better just to make Hoopl faster?<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">If anything I ‘d like to use Hoopl more in Cmm optimisation passes in GHC, so we may want to use more of Hoopl’s facilities.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">The main reason you suggest for forking is that there are some awkward name clashes. Surely we could resolve these? e.g we could change CLabel in
GHC; or agree with Hoopl maintainers that BlockId would be more helpful than Label.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">You mention that Hoopl uses Unique set/map. Why not use ‘containers’ for that? (Like GHC!)<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">Let’s discuss this a bit more before executing<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">I’m also interested to know:<u></u><u></u></span></p>
<p class="m_5429673471515783493MsoListParagraph"><u></u><span style="font-size:11.0pt;font-family:Symbol"><span>·<span style="font:7.0pt "Times New Roman"">
</span></span></span><u></u><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">who is actively working on Hoopl (Michael, Sophie, …)?<u></u><u></u></span></p>
<p class="m_5429673471515783493MsoListParagraph"><u></u><span style="font-size:11.0pt;font-family:Symbol"><span>·<span style="font:7.0pt "Times New Roman"">
</span></span></span><u></u><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">how are you using it (within GHC, or somewhere else)?<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">It’d be good to review and update
<a href="https://ghc.haskell.org/trac/ghc/wiki/Hoopl/Cleanup" target="_blank">https://ghc.haskell.org/trac/ghc/wiki/Hoopl/Cleanup</a>. Are there any other improvements planned?<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">Simon</span></p></div></div></blockquote><div><br></div><div>Hi Simon,</div><div> </div><div>Thanks for chiming in! Let me try to clarify the current situation and</div><div>the motivation for my changes.</div><div> </div><div>1) Initial fork of Hoopl</div><div> </div><div>Note that what I’m actually advocating is to *finish* forking Hoopl. The</div><div>fork really started in ~2012 when the “new Cmm backend” was being</div><div>finished.</div><div>IIRC the main reason was the unacceptable performance and it seems that</div><div>even Simon Marlow had trouble making it run fast enough:</div><div><a href="https://plus.google.com/107890464054636586545/posts/dBbewpRfw6R">https://plus.google.com/107890464054636586545/posts/dBbewpRfw6R</a></div><div><a href="https://ghc.haskell.org/trac/ghc/wiki/Commentary/Compiler/HooplPerformance">https://ghc.haskell.org/trac/ghc/wiki/Commentary/Compiler/HooplPerformance</a></div><div>The end result is pretty sad: GHC has its own forked/specialized</div><div>`Hoopl.Dataflow` module and is using Hoopl only for definitions of</div><div>`Block`/`Graph` and maps/sets (if you look at my commit, it’s pretty</div><div>clear what I’m copying). In particular it’s not using *any* of dataflow</div><div>analysis or rewriting capabilities of the Hoopl package.</div><div> </div><div>2) Reasons to finish forking</div><div> </div><div>The reasons I listed in my previous email already assumed the we have</div><div>the forked `Hoopl.Dataflow` module in GHC. But if we want to discuss</div><div>what are reasons for forking in general, then apart from the performance</div><div>(as noted above), there’s the issue of Hoopl’s interface. IMHO the</div><div>node-oriented approach taken by Hoopl is both not flexible enough and it</div><div>makes it harder to optimize it. That’s why I’ve already changed GHC’s</div><div>`Hoopl.Dataflow` module to operate “block-at-a-time”</div><div>(<a href="https://github.com/ghc/ghc/commit/679ccd1c8860f1ef4b589c9593b74d04c97ae836">https://github.com/ghc/ghc/commit/679ccd1c8860f1ef4b589c9593b74d04c97ae836</a>)</div><div>Some concrete examples:</div><div>- For proc-point analysis it was necessary to introduce a hack to GHC’s</div><div> `Dataflow` module to expose a separate analysis function that</div><div> *ignores* the middle nodes (since for proc-points they’re irrelevant).</div><div> My change to go “block-at-a-time” allowed us to remove that hack.</div><div>- I’m trying to fix non-linearity of `CmmLayoutStack` in</div><div> (<a href="https://phabricator.haskell.org/D3586">https://phabricator.haskell.org/D3586</a>) and again the block-oriented</div><div> interface is useful - I want to do different rewrites based on</div><div> which block is being considered (whether it’s a proc-point or not).</div><div> This is not easily possible if I don’t know which block I’m in (which</div><div> is the case for the node-oriented interface).</div><div> </div><div>I also don’t think that name clashes and the tension between Hoopl’s</div><div>interface and GHC are easy to solve. Hoopl is a public, stand-alone</div><div>package, so we can’t just change things without considering</div><div>compatibility. For instance, we can’t use GHC’s `Unique` in Hoopl. But</div><div>should we switch all of GHC to use Hoopl’s? Also having closely related</div><div>concepts spread around GHC and Hoopl is not helping when trying to</div><div>understand what’s happening. Finally, any changes to both GHC & Hoopl</div><div>have much higher overhead than just changing GHC.</div><div> </div><div>In general, it really seems to me that Hoopl has been released simply</div><div>too early, with not enough real-world usage and testing. When you say</div><div>that we should “just fix Hoopl”, it sounds to me that we’d really need</div><div>to rewrite it from scratch. And it’s much easier to do that if we can</div><div>just experiment within GHC without worrying about breaking other</div><div>existing Hoopl users. Only once we’re happy with the result, we should</div><div>be considering separating it into a stand-alone package.</div><div> </div><div>3) Difference between pretty/containers and Hoopl</div><div> </div><div>I also think that the situation with pretty/containers is quite</div><div>different than Hoopl. They are much more general-purpose libraries,</div><div>*far* more widely used and with more contributors. Take containers - the</div><div>package is still very actively developed and constantly improved.</div><div>Whereas Hoopl hasn’t really seen much activity in the last 5 years. So</div><div>the benefit-cost ratio is much better - yes there is some cost in having</div><div>containers as a dependency, but the benefits from the regular stream of</div><div>improvements easily outweigh it. I don’t think that’s the case for</div><div>Hoopl.</div><div> </div><div>Does this help understand my motivation? Let me know if anything is</div><div>still unclear!</div><div> </div><div>Thanks,</div><div>Michal</div><div> </div></div></div>