Specialisation doesn't kick in (RE: Instantiation of overloaded definition in Core)

Mon Oct 11 07:32:56 UTC 2021

It's incredibly hard to debug this sort of thing remotely, without the ability to reproduce it.  First, you are using a variant of GHC, with changes that we can only guess at. Second, even with unmodified GHC I often find that unexpected things happen - until I dig deeper and it becomes obvious!

There is one odd thing about your dump: it seems to be in reverse dependency order, with functions being defined before they are used, rather than after.  That would certainly stop the specialiser from working.  The occurrence analyser would sort this out (literally).   But that's a total guess.

Simon

PS: I am leaving Microsoft at the end of November 2021, at which point simonpj at microsoft.com<mailto:simonpj at microsoft.com> will cease to work.  Use simon.peytonjones at gmail.com<mailto:simon.peytonjones at gmail.com> instead.  (For now, it just forwards to simonpj at microsoft.com.)

From: Erdi, Gergo <Gergo.Erdi at sc.com>
Sent: 11 October 2021 03:58
To: Simon Peyton Jones <simonpj at microsoft.com>; Matthew Pickering <matthewtpickering at gmail.com>
Cc: Montelatici, Raphael Laurent <Raphael.Montelatici at sc.com>; 'GHC' <ghc-devs at haskell.org>
Subject: RE: Specialisation doesn't kick in (RE: Instantiation of overloaded definition *in Core*)

PUBLIC

PUBLIC

Hi Simon, Matt & others,

It took me until now to be able to try out GHC HEAD, mostly because I had to adapt to all the GHC.Unit.* refactorings. However, now I am on a466b02492f73a43c6cb9ce69491fc85234b9559 which includes the commit Simon pointed out. My original plan was to expose the first half of specProgram, i.e. the part that calls `go binds`. I did that because I want to apply specialisation on collected whole-program Core, not just whatever would be in scope for a Core-to-Core plugin pass, so I am not writing a CoreM and I don't even have a ModGuts at hand.

However, I found out from Matt's email on this thread that this is not going to be enough and eventually I'll need to figure out how I intend to apply the rewrite rules that come out of this. So for now, I am just turning on specialization in the normal pipeline by setting Opt_Specialise, Opt_SpecialiseAggressively, and Opt_CrossModuleSpecialise. And I'm still not seeing $dm>> being specialized.

Is this because I define each of "class Monad", "data IO a", "instance Monad IO", and "main", in four distinct modules? In other words, is this something I will not be able to try out until I figure out how to make a fake ModGuts and run a CoreM from outside the normal compilation pipeline, feeding it the whole-program collected CoreBinds? But if so, why is it that when I feed my whole program to just specBinds (which I can try easily since it has no ModGuts/CoreM dependency other than a uniq supply and the CoreProgram), I get back an empty UsageDetails?

Thanks,
Gergo

For reference, the relevant definitions dumped from GHC with specialization (supposedly) turned on:

main = $fMonadIO_$c>> @() @() sat_sJg xmain

$fMonadIO_$c>> :: forall a b. IO a -> IO b -> IO b
$fMonadIO_$c>> = \ (@a_aF9) (@b_aFa) -> $dm>> @IO $fMonadIO @a_aF9 @b_aFa;

$dm>> :: forall (m :: Type -> Type) a b. Monad m => m a -> m b -> m b
$dm>>
  = \ (@(m_ani :: Type -> Type))
      ($dMonad_sIi [Occ=Once1] :: Monad m_ani)
      (@a_ar4)
      (@b_ar5)
      (ma_sIj [Occ=Once1] :: m_ani a_ar4)
      (mb_sIk [Occ=OnceL1] :: m_ani b_ar5) ->
      let {
        sat_sIm [Occ=Once1] :: a_ar4 -> m_ani b_ar5
        [LclId]
        sat_sIm = \ _ [Occ=Dead] -> mb_sIk } in
      >>= @m_ani $dMonad_sIi @a_ar4 @b_ar5 ma_sIj sat_sIm

From: Erdi, Gergo
Sent: Thursday, October 7, 2021 9:30 AM
To: Simon Peyton Jones <simonpj at microsoft.com<mailto:simonpj at microsoft.com>>
Cc: Montelatici, Raphael Laurent <Raphael.Montelatici at sc.com<mailto:Raphael.Montelatici at sc.com>>; GHC <ghc-devs at haskell.org<mailto:ghc-devs at haskell.org>>
Subject: RE: Specialisation doesn't kick in (RE: Instantiation of overloaded definition *in Core*)

PUBLIC

Indeed, I am using 9.0.1. I'll try upgrading. Thanks!

From: Simon Peyton Jones <simonpj at microsoft.com<mailto:simonpj at microsoft.com>>
Sent: Wednesday, October 6, 2021 6:12 PM
To: Erdi, Gergo <Gergo.Erdi at sc.com<mailto:Gergo.Erdi at sc.com>>
Cc: Montelatici, Raphael Laurent <Raphael.Montelatici at sc.com<mailto:Raphael.Montelatici at sc.com>>; GHC <ghc-devs at haskell.org<mailto:ghc-devs at haskell.org>>
Subject: [External] RE: Specialisation doesn't kick in (RE: Instantiation of overloaded definition *in Core*)

Grego,

Yes I think that should auto-specialise.

Which version of GHC are you using?   Do you have this patch?

commit ef0135934fe32da5b5bb730dbce74262e23e72e8

Author: Simon Peyton Jones simonpj at microsoft.com<mailto:simonpj at microsoft.com>

Date:   Thu Apr 8 22:42:31 2021 +0100

    Make the specialiser handle polymorphic specialisation

Here's why I ask.  The call

$fMonadIO_$c>> = \ (@a) (@b) -> $dm>> @IO $fMonadIO @a @b

indeed applies $dm>> to $fMonadIO, but it also applies it to a and b.  In the version of GHC you have, maybe that stops the call from floating up to the definition site, and being used to specialise it.

Can you make a repro case without your plugin?

Simon

PS: I am leaving Microsoft at the end of November 2021, at which point simonpj at microsoft.com<mailto:simonpj at microsoft.com> will cease to work.  Use simon.peytonjones at gmail.com<mailto:simon.peytonjones at gmail.com> instead.  (For now, it just forwards to simonpj at microsoft.com<mailto:simonpj at microsoft.com>.)

This email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please delete all copies and notify the sender immediately. You may wish to refer to the incorporation details of Standard Chartered PLC, Standard Chartered Bank and their subsidiaries at https: //www.sc.com/en/our-locations

Where you have a Financial Markets relationship with Standard Chartered PLC, Standard Chartered Bank and their subsidiaries (the "Group"), information on the regulatory standards we adhere to and how it may affect you can be found in our Regulatory Compliance Statement at https: //www.sc.com/rcs/ and Regulatory Compliance Disclosures at http: //www.sc.com/rcs/fm

Insofar as this communication is not sent by the Global Research team and contains any market commentary, the market commentary has been prepared by the sales and/or trading desk of Standard Chartered Bank or its affiliate. It is not and does not constitute research material, independent research, recommendation or financial advice. Any market commentary is for information purpose only and shall not be relied on for any other purpose and is subject to the relevant disclaimers available at https: //www.sc.com/en/regulatory-disclosures/#market-disclaimer.

Insofar as this communication is sent by the Global Research team and contains any research materials prepared by members of the team, the research material is for information purpose only and shall not be relied on for any other purpose, and is subject to the relevant disclaimers available at https: //research.sc.com/research/api/application/static/terms-and-conditions.

Insofar as this e-mail contains the term sheet for a proposed transaction, by responding affirmatively to this e-mail, you agree that you have understood the terms and conditions in the attached term sheet and evaluated the merits and risks of the transaction. We may at times also request you to sign the term sheet to acknowledge the same.

Please visit https: //www.sc.com/en/regulatory-disclosures/dodd-frank/ for important information with respect to derivative products.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20211011/a90654ae/attachment-0001.html>