From Gergo.Erdi at sc.com Wed Dec 1 05:22:24 2021 From: Gergo.Erdi at sc.com (Erdi, Gergo) Date: Wed, 1 Dec 2021 05:22:24 +0000 Subject: Recognizing default method implementations Message-ID: PUBLIC Hi, Is there a way to recognize that an Id / Var refers to a default method implementation, e.g. $dm/= in the following? $dm/= :: forall a. Eq a => a -> a -> Bool [GblId, Arity=3, Unf=OtherCon []] $dm/= = \ (@a_ahz) ($dEq_sI6 [Occ=Once1] :: Eq a_ahz) (x_sI7 [Occ=Once1] :: a_ahz) (y_sI8 [Occ=Once1] :: a_ahz) -> case == @a_ahz $dEq_sI6 x_sI7 y_sI8 of { False -> True; True -> False } Thanks, Gergo This email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please delete all copies and notify the sender immediately. You may wish to refer to the incorporation details of Standard Chartered PLC, Standard Chartered Bank and their subsidiaries at https: //www.sc.com/en/our-locations Where you have a Financial Markets relationship with Standard Chartered PLC, Standard Chartered Bank and their subsidiaries (the "Group"), information on the regulatory standards we adhere to and how it may affect you can be found in our Regulatory Compliance Statement at https: //www.sc.com/rcs/ and Regulatory Compliance Disclosures at http: //www.sc.com/rcs/fm Insofar as this communication is not sent by the Global Research team and contains any market commentary, the market commentary has been prepared by the sales and/or trading desk of Standard Chartered Bank or its affiliate. It is not and does not constitute research material, independent research, recommendation or financial advice. Any market commentary is for information purpose only and shall not be relied on for any other purpose and is subject to the relevant disclaimers available at https: //www.sc.com/en/regulatory-disclosures/#market-disclaimer. Insofar as this communication is sent by the Global Research team and contains any research materials prepared by members of the team, the research material is for information purpose only and shall not be relied on for any other purpose, and is subject to the relevant disclaimers available at https: //research.sc.com/research/api/application/static/terms-and-conditions. Insofar as this e-mail contains the term sheet for a proposed transaction, by responding affirmatively to this e-mail, you agree that you have understood the terms and conditions in the attached term sheet and evaluated the merits and risks of the transaction. We may at times also request you to sign the term sheet to acknowledge the same. Please visit https: //www.sc.com/en/regulatory-disclosures/dodd-frank/ for important information with respect to derivative products. -------------- next part -------------- An HTML attachment was scrubbed... URL: From simonpj at microsoft.com Wed Dec 1 08:56:40 2021 From: simonpj at microsoft.com (Simon Peyton Jones) Date: Wed, 1 Dec 2021 08:56:40 +0000 Subject: Recognizing default method implementations In-Reply-To: References: Message-ID: Is there a way to recognize that an Id / Var refers to a default method implementation, e.g. $dm/= in the following? I'm afraid not. There has never been a reason to do so. And it might not be robust e.g. it might be inlined. What is the problem you are trying to solve? Simon PS: I am leaving Microsoft at the end of November 2021, at which point simonpj at microsoft.com will cease to work. Use simon.peytonjones at gmail.com instead. (For now, it just forwards to simonpj at microsoft.com.) From: ghc-devs On Behalf Of Erdi, Gergo via ghc-devs Sent: 01 December 2021 05:22 To: 'GHC' Subject: [EXTERNAL] Recognizing default method implementations PUBLIC Hi, Is there a way to recognize that an Id / Var refers to a default method implementation, e.g. $dm/= in the following? $dm/= :: forall a. Eq a => a -> a -> Bool [GblId, Arity=3, Unf=OtherCon []] $dm/= = \ (@a_ahz) ($dEq_sI6 [Occ=Once1] :: Eq a_ahz) (x_sI7 [Occ=Once1] :: a_ahz) (y_sI8 [Occ=Once1] :: a_ahz) -> case == @a_ahz $dEq_sI6 x_sI7 y_sI8 of { False -> True; True -> False } Thanks, Gergo This email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please delete all copies and notify the sender immediately. You may wish to refer to the incorporation details of Standard Chartered PLC, Standard Chartered Bank and their subsidiaries at https: //www.sc.com/en/our-locations Where you have a Financial Markets relationship with Standard Chartered PLC, Standard Chartered Bank and their subsidiaries (the "Group"), information on the regulatory standards we adhere to and how it may affect you can be found in our Regulatory Compliance Statement at https: //www.sc.com/rcs/ and Regulatory Compliance Disclosures at http: //www.sc.com/rcs/fm Insofar as this communication is not sent by the Global Research team and contains any market commentary, the market commentary has been prepared by the sales and/or trading desk of Standard Chartered Bank or its affiliate. It is not and does not constitute research material, independent research, recommendation or financial advice. Any market commentary is for information purpose only and shall not be relied on for any other purpose and is subject to the relevant disclaimers available at https: //www.sc.com/en/regulatory-disclosures/#market-disclaimer. Insofar as this communication is sent by the Global Research team and contains any research materials prepared by members of the team, the research material is for information purpose only and shall not be relied on for any other purpose, and is subject to the relevant disclaimers available at https: //research.sc.com/research/api/application/static/terms-and-conditions. Insofar as this e-mail contains the term sheet for a proposed transaction, by responding affirmatively to this e-mail, you agree that you have understood the terms and conditions in the attached term sheet and evaluated the merits and risks of the transaction. We may at times also request you to sign the term sheet to acknowledge the same. Please visit https: //www.sc.com/en/regulatory-disclosures/dodd-frank/ for important information with respect to derivative products. -------------- next part -------------- An HTML attachment was scrubbed... URL: From Gergo.Erdi at sc.com Thu Dec 2 07:51:37 2021 From: Gergo.Erdi at sc.com (Erdi, Gergo) Date: Thu, 2 Dec 2021 07:51:37 +0000 Subject: Recognizing default method implementations In-Reply-To: References: Message-ID: PUBLIC PUBLIC Basically I want to skip everything that can't possibly needed after full specialization, i.e. dictionary and default method definitions and method accesors. But I realize now that I should just look at the types, and skip anything with a non-empty context, since after full specialization I should, transitively, also have no need for normal top-level function definitions that have a context. From: Simon Peyton Jones Sent: Wednesday, December 1, 2021 4:57 PM To: Erdi, Gergo Cc: GHC Subject: [External] RE: Recognizing default method implementations PUBLIC ATTENTION: This email came from an external source. Do not open attachments or click on links from unknown senders or unexpected emails. Always report suspicious emails using the Report As Phishing button in Outlook to protect the Bank and our clients. Is there a way to recognize that an Id / Var refers to a default method implementation, e.g. $dm/= in the following? I'm afraid not. There has never been a reason to do so. And it might not be robust e.g. it might be inlined. What is the problem you are trying to solve? Simon PS: I am leaving Microsoft at the end of November 2021, at which point simonpj at microsoft.com will cease to work. Use simon.peytonjones at gmail.com instead. (For now, it just forwards to simonpj at microsoft.com.) From: ghc-devs > On Behalf Of Erdi, Gergo via ghc-devs Sent: 01 December 2021 05:22 To: 'GHC' > Subject: [EXTERNAL] Recognizing default method implementations PUBLIC Hi, Is there a way to recognize that an Id / Var refers to a default method implementation, e.g. $dm/= in the following? $dm/= :: forall a. Eq a => a -> a -> Bool [GblId, Arity=3, Unf=OtherCon []] $dm/= = \ (@a_ahz) ($dEq_sI6 [Occ=Once1] :: Eq a_ahz) (x_sI7 [Occ=Once1] :: a_ahz) (y_sI8 [Occ=Once1] :: a_ahz) -> case == @a_ahz $dEq_sI6 x_sI7 y_sI8 of { False -> True; True -> False } Thanks, Gergo This email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please delete all copies and notify the sender immediately. You may wish to refer to the incorporation details of Standard Chartered PLC, Standard Chartered Bank and their subsidiaries at https: //www.sc.com/en/our-locations Where you have a Financial Markets relationship with Standard Chartered PLC, Standard Chartered Bank and their subsidiaries (the "Group"), information on the regulatory standards we adhere to and how it may affect you can be found in our Regulatory Compliance Statement at https: //www.sc.com/rcs/ and Regulatory Compliance Disclosures at http: //www.sc.com/rcs/fm Insofar as this communication is not sent by the Global Research team and contains any market commentary, the market commentary has been prepared by the sales and/or trading desk of Standard Chartered Bank or its affiliate. It is not and does not constitute research material, independent research, recommendation or financial advice. Any market commentary is for information purpose only and shall not be relied on for any other purpose and is subject to the relevant disclaimers available at https: //www.sc.com/en/regulatory-disclosures/#market-disclaimer. Insofar as this communication is sent by the Global Research team and contains any research materials prepared by members of the team, the research material is for information purpose only and shall not be relied on for any other purpose, and is subject to the relevant disclaimers available at https: //research.sc.com/research/api/application/static/terms-and-conditions. Insofar as this e-mail contains the term sheet for a proposed transaction, by responding affirmatively to this e-mail, you agree that you have understood the terms and conditions in the attached term sheet and evaluated the merits and risks of the transaction. We may at times also request you to sign the term sheet to acknowledge the same. Please visit https: //www.sc.com/en/regulatory-disclosures/dodd-frank/ for important information with respect to derivative products. This email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please delete all copies and notify the sender immediately. You may wish to refer to the incorporation details of Standard Chartered PLC, Standard Chartered Bank and their subsidiaries at https: //www.sc.com/en/our-locations Where you have a Financial Markets relationship with Standard Chartered PLC, Standard Chartered Bank and their subsidiaries (the "Group"), information on the regulatory standards we adhere to and how it may affect you can be found in our Regulatory Compliance Statement at https: //www.sc.com/rcs/ and Regulatory Compliance Disclosures at http: //www.sc.com/rcs/fm Insofar as this communication is not sent by the Global Research team and contains any market commentary, the market commentary has been prepared by the sales and/or trading desk of Standard Chartered Bank or its affiliate. It is not and does not constitute research material, independent research, recommendation or financial advice. Any market commentary is for information purpose only and shall not be relied on for any other purpose and is subject to the relevant disclaimers available at https: //www.sc.com/en/regulatory-disclosures/#market-disclaimer. Insofar as this communication is sent by the Global Research team and contains any research materials prepared by members of the team, the research material is for information purpose only and shall not be relied on for any other purpose, and is subject to the relevant disclaimers available at https: //research.sc.com/research/api/application/static/terms-and-conditions. Insofar as this e-mail contains the term sheet for a proposed transaction, by responding affirmatively to this e-mail, you agree that you have understood the terms and conditions in the attached term sheet and evaluated the merits and risks of the transaction. We may at times also request you to sign the term sheet to acknowledge the same. Please visit https: //www.sc.com/en/regulatory-disclosures/dodd-frank/ for important information with respect to derivative products. -------------- next part -------------- An HTML attachment was scrubbed... URL: From csaba.hruska at gmail.com Thu Dec 2 14:52:10 2021 From: csaba.hruska at gmail.com (Csaba Hruska) Date: Thu, 2 Dec 2021 15:52:10 +0100 Subject: Why and How the External STG interpreter is Useful (Online presentation, Dec 2, Friday, 17:00 UTC) Message-ID: Hello, Today I'll do a presentation about the external stg interpreter. If you are interested please join and ask questions. https://skillsmatter.com/meetups/13654-haskell-stg-interp Regards, Csaba Hruska Abstract: Haskell: Why and How the External STG Interpreter is Useful The external STG interpreter is a from scratch implementation of the STG machine in Haskell. Currently it supports almost all GHC primops and RTS features. It can run real world Haskell programs that were compiled with GHC Whole Program Compiler (GHC-WPC). GHC-WPC is a GHC fork that exports the whole program STG IR. The external STG interpreter is an excellent tool to study the runtime behaviour of Haskell programs, i.e. it can run/interpret GHC or Pandoc. The implementation of the interpreter is in plain simple Haskell, so it makes compiler backend and tooling development approachable for everyone. It already has a programmable debugger which supports step-by-step evaluation, breakpoints and execution region based inspection. It also can export the whole program memory state and call-graphs to files for further investigation. These features make it easy to find a memory leak or to identify a performance bottleneck in a large real world Haskell application. https://github.com/grin-compiler/ghc-whole-program-compiler-project -------------- next part -------------- An HTML attachment was scrubbed... URL: From csaba.hruska at gmail.com Thu Dec 2 14:54:37 2021 From: csaba.hruska at gmail.com (Csaba Hruska) Date: Thu, 2 Dec 2021 15:54:37 +0100 Subject: Why and How the External STG interpreter is Useful (Online presentation, Dec 2, Friday, 17:00 UTC) In-Reply-To: References: Message-ID: It's on Thursday Dec 2 17:00 UTC. (Today) Sorry for the confusion. On Thu, Dec 2, 2021 at 3:52 PM Csaba Hruska wrote: > Hello, > > Today I'll do a presentation about the external stg interpreter. > If you are interested please join and ask questions. > https://skillsmatter.com/meetups/13654-haskell-stg-interp > > Regards, > Csaba Hruska > > Abstract: > Haskell: Why and How the External STG Interpreter is Useful > > The external STG interpreter is a from scratch implementation of the STG > machine in Haskell. Currently it supports almost all GHC primops and RTS > features. It can run real world Haskell programs that were compiled with > GHC Whole Program Compiler (GHC-WPC). GHC-WPC is a GHC fork that exports > the whole program STG IR. > > The external STG interpreter is an excellent tool to study the runtime > behaviour of Haskell programs, i.e. it can run/interpret GHC or Pandoc. The > implementation of the interpreter is in plain simple Haskell, so it makes > compiler backend and tooling development approachable for everyone. It > already has a programmable debugger which supports step-by-step evaluation, > breakpoints and execution region based inspection. It also can export the > whole program memory state and call-graphs to files for further > investigation. These features make it easy to find a memory leak or to > identify a performance bottleneck in a large real world Haskell application. > > https://github.com/grin-compiler/ghc-whole-program-compiler-project > -------------- next part -------------- An HTML attachment was scrubbed... URL: From chessai1996 at gmail.com Thu Dec 2 15:04:40 2021 From: chessai1996 at gmail.com (chessai) Date: Thu, 2 Dec 2021 07:04:40 -0800 Subject: Why and How the External STG interpreter is Useful (Online presentation, Dec 2, Friday, 17:00 UTC) In-Reply-To: References: Message-ID: Hi, Will there be a recording? Thanks On Thu, Dec 2, 2021, 06:55 Csaba Hruska wrote: > It's on Thursday Dec 2 17:00 UTC. (Today) > Sorry for the confusion. > > On Thu, Dec 2, 2021 at 3:52 PM Csaba Hruska > wrote: > >> Hello, >> >> Today I'll do a presentation about the external stg interpreter. >> If you are interested please join and ask questions. >> https://skillsmatter.com/meetups/13654-haskell-stg-interp >> >> Regards, >> Csaba Hruska >> >> Abstract: >> Haskell: Why and How the External STG Interpreter is Useful >> >> The external STG interpreter is a from scratch implementation of the STG >> machine in Haskell. Currently it supports almost all GHC primops and RTS >> features. It can run real world Haskell programs that were compiled with >> GHC Whole Program Compiler (GHC-WPC). GHC-WPC is a GHC fork that exports >> the whole program STG IR. >> >> The external STG interpreter is an excellent tool to study the runtime >> behaviour of Haskell programs, i.e. it can run/interpret GHC or Pandoc. The >> implementation of the interpreter is in plain simple Haskell, so it makes >> compiler backend and tooling development approachable for everyone. It >> already has a programmable debugger which supports step-by-step evaluation, >> breakpoints and execution region based inspection. It also can export the >> whole program memory state and call-graphs to files for further >> investigation. These features make it easy to find a memory leak or to >> identify a performance bottleneck in a large real world Haskell application. >> >> https://github.com/grin-compiler/ghc-whole-program-compiler-project >> > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > -------------- next part -------------- An HTML attachment was scrubbed... URL: From csaba.hruska at gmail.com Thu Dec 2 15:07:25 2021 From: csaba.hruska at gmail.com (Csaba Hruska) Date: Thu, 2 Dec 2021 16:07:25 +0100 Subject: Why and How the External STG interpreter is Useful (Online presentation, Dec 2, Friday, 17:00 UTC) In-Reply-To: References: Message-ID: Yes! On Thu, Dec 2, 2021 at 4:04 PM chessai wrote: > Hi, > > Will there be a recording? > > Thanks > > On Thu, Dec 2, 2021, 06:55 Csaba Hruska wrote: > >> It's on Thursday Dec 2 17:00 UTC. (Today) >> Sorry for the confusion. >> >> On Thu, Dec 2, 2021 at 3:52 PM Csaba Hruska >> wrote: >> >>> Hello, >>> >>> Today I'll do a presentation about the external stg interpreter. >>> If you are interested please join and ask questions. >>> https://skillsmatter.com/meetups/13654-haskell-stg-interp >>> >>> Regards, >>> Csaba Hruska >>> >>> Abstract: >>> Haskell: Why and How the External STG Interpreter is Useful >>> >>> The external STG interpreter is a from scratch implementation of the STG >>> machine in Haskell. Currently it supports almost all GHC primops and RTS >>> features. It can run real world Haskell programs that were compiled with >>> GHC Whole Program Compiler (GHC-WPC). GHC-WPC is a GHC fork that exports >>> the whole program STG IR. >>> >>> The external STG interpreter is an excellent tool to study the runtime >>> behaviour of Haskell programs, i.e. it can run/interpret GHC or Pandoc. The >>> implementation of the interpreter is in plain simple Haskell, so it makes >>> compiler backend and tooling development approachable for everyone. It >>> already has a programmable debugger which supports step-by-step evaluation, >>> breakpoints and execution region based inspection. It also can export the >>> whole program memory state and call-graphs to files for further >>> investigation. These features make it easy to find a memory leak or to >>> identify a performance bottleneck in a large real world Haskell application. >>> >>> https://github.com/grin-compiler/ghc-whole-program-compiler-project >>> >> _______________________________________________ >> ghc-devs mailing list >> ghc-devs at haskell.org >> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From compl.yue at icloud.com Thu Dec 2 15:07:58 2021 From: compl.yue at icloud.com (YueCompl) Date: Thu, 2 Dec 2021 23:07:58 +0800 Subject: Why and How the External STG interpreter is Useful (Online presentation, Dec 2, Friday, 17:00 UTC) In-Reply-To: References: Message-ID: <7C4CEA7F-8C26-4EAF-8748-48B961DEFBA4@icloud.com> This sounds pretty exciting! Can we expect a full fledged stepping debugger integrated with IDEs via https://github.com/phoityne/haskell-debug-adapter ? https://github.com/phoityne/ghci-dap is still quite limited feature-wise. > On 2021-12-02, at 22:52, Csaba Hruska wrote: > > Hello, > > Today I'll do a presentation about the external stg interpreter. > If you are interested please join and ask questions. > https://skillsmatter.com/meetups/13654-haskell-stg-interp > > Regards, > Csaba Hruska > > Abstract: > Haskell: Why and How the External STG Interpreter is Useful > > The external STG interpreter is a from scratch implementation of the STG machine in Haskell. Currently it supports almost all GHC primops and RTS features. It can run real world Haskell programs that were compiled with GHC Whole Program Compiler (GHC-WPC). GHC-WPC is a GHC fork that exports the whole program STG IR. > > The external STG interpreter is an excellent tool to study the runtime behaviour of Haskell programs, i.e. it can run/interpret GHC or Pandoc. The implementation of the interpreter is in plain simple Haskell, so it makes compiler backend and tooling development approachable for everyone. It already has a programmable debugger which supports step-by-step evaluation, breakpoints and execution region based inspection. It also can export the whole program memory state and call-graphs to files for further investigation. These features make it easy to find a memory leak or to identify a performance bottleneck in a large real world Haskell application. > > https://github.com/grin-compiler/ghc-whole-program-compiler-project _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs -------------- next part -------------- An HTML attachment was scrubbed... URL: From csaba.hruska at gmail.com Fri Dec 3 09:30:30 2021 From: csaba.hruska at gmail.com (Csaba Hruska) Date: Fri, 3 Dec 2021 10:30:30 +0100 Subject: Why and How the External STG interpreter is Useful (Online presentation, Dec 2, Friday, 17:00 UTC) In-Reply-To: References: Message-ID: Hello, The presentation recording and slides are available now. Regards, Csaba On Thu, Dec 2, 2021 at 3:52 PM Csaba Hruska wrote: > Hello, > > Today I'll do a presentation about the external stg interpreter. > If you are interested please join and ask questions. > https://skillsmatter.com/meetups/13654-haskell-stg-interp > > Regards, > Csaba Hruska > > Abstract: > Haskell: Why and How the External STG Interpreter is Useful > > The external STG interpreter is a from scratch implementation of the STG > machine in Haskell. Currently it supports almost all GHC primops and RTS > features. It can run real world Haskell programs that were compiled with > GHC Whole Program Compiler (GHC-WPC). GHC-WPC is a GHC fork that exports > the whole program STG IR. > > The external STG interpreter is an excellent tool to study the runtime > behaviour of Haskell programs, i.e. it can run/interpret GHC or Pandoc. The > implementation of the interpreter is in plain simple Haskell, so it makes > compiler backend and tooling development approachable for everyone. It > already has a programmable debugger which supports step-by-step evaluation, > breakpoints and execution region based inspection. It also can export the > whole program memory state and call-graphs to files for further > investigation. These features make it easy to find a memory leak or to > identify a performance bottleneck in a large real world Haskell application. > > https://github.com/grin-compiler/ghc-whole-program-compiler-project > -------------- next part -------------- An HTML attachment was scrubbed... URL: From csaba.hruska at gmail.com Sun Dec 5 15:37:23 2021 From: csaba.hruska at gmail.com (Csaba Hruska) Date: Sun, 5 Dec 2021 16:37:23 +0100 Subject: Why and How the External STG interpreter is Useful (Online presentation, Dec 2, Friday, 17:00 UTC) In-Reply-To: <7C4CEA7F-8C26-4EAF-8748-48B961DEFBA4@icloud.com> References: <7C4CEA7F-8C26-4EAF-8748-48B961DEFBA4@icloud.com> Message-ID: Hi, I'm aware of the debug adapter protocol but I have not used it yet. The external stg interpreter debugger has a terminal based UI currently, because I was focusing on the debug and profile features so far and not on UI. My debugger is programmable though, you can write a debug script to automate the breakpoint setup along with other debug commands. see: https://youtu.be/wt6iCgYmVGA?t=2809 Cheers, Csaba On Thu, Dec 2, 2021 at 4:08 PM YueCompl wrote: > This sounds pretty exciting! > > Can we expect a full fledged stepping debugger integrated with IDEs via > https://github.com/phoityne/haskell-debug-adapter ? > https://github.com/phoityne/ghci-dap is still quite limited feature-wise. > > On 2021-12-02, at 22:52, Csaba Hruska wrote: > > Hello, > > Today I'll do a presentation about the external stg interpreter. > If you are interested please join and ask questions. > https://skillsmatter.com/meetups/13654-haskell-stg-interp > > Regards, > Csaba Hruska > > Abstract: > Haskell: Why and How the External STG Interpreter is Useful > > The external STG interpreter is a from scratch implementation of the STG > machine in Haskell. Currently it supports almost all GHC primops and RTS > features. It can run real world Haskell programs that were compiled with > GHC Whole Program Compiler (GHC-WPC). GHC-WPC is a GHC fork that exports > the whole program STG IR. > > The external STG interpreter is an excellent tool to study the runtime > behaviour of Haskell programs, i.e. it can run/interpret GHC or Pandoc. The > implementation of the interpreter is in plain simple Haskell, so it makes > compiler backend and tooling development approachable for everyone. It > already has a programmable debugger which supports step-by-step evaluation, > breakpoints and execution region based inspection. It also can export the > whole program memory state and call-graphs to files for further > investigation. These features make it easy to find a memory leak or to > identify a performance bottleneck in a large real world Haskell application. > > https://github.com/grin-compiler/ghc-whole-program-compiler-project > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From nr at cs.tufts.edu Mon Dec 6 16:26:58 2021 From: nr at cs.tufts.edu (Norman Ramsey) Date: Mon, 06 Dec 2021 11:26:58 -0500 Subject: is there a way to alter every line of an SDoc? Message-ID: <20211206162658.904612C1E06@homedog.cs.tufts.edu> I'm emitting code in which I want to prettyprint the contents of a Cmm Block as a comment, by which I mean putting `--` to the left of *each* line of the rendered Cmm Block. Using GHC.Utils.Outputable, is this even possible? How would I tackle it? (The Haddock page does not suggest any obvious combinators.) Norman From ben at smart-cactus.org Mon Dec 6 16:45:35 2021 From: ben at smart-cactus.org (Ben Gamari) Date: Mon, 06 Dec 2021 11:45:35 -0500 Subject: is there a way to alter every line of an SDoc? In-Reply-To: <20211206162658.904612C1E06@homedog.cs.tufts.edu> References: <20211206162658.904612C1E06@homedog.cs.tufts.edu> Message-ID: <87czm9u1gl.fsf@smart-cactus.org> Norman Ramsey writes: > I'm emitting code in which I want to prettyprint the contents > of a Cmm Block as a comment, by which I mean putting `--` to the left > of *each* line of the rendered Cmm Block. Using GHC.Utils.Outputable, > is this even possible? How would I tackle it? (The Haddock page > does not suggest any obvious combinators.) > It's not easy. One (rather horrible) approach would be to render the document to a string, split the output into lines and add the comment symbols, and then splice the result into another document. Perhaps you could rather use block comment syntax? This would be much easier. Cheers, - Ben -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From nr at cs.tufts.edu Mon Dec 6 21:24:46 2021 From: nr at cs.tufts.edu (Norman Ramsey) Date: Mon, 06 Dec 2021 16:24:46 -0500 Subject: is there a way to alter every line of an SDoc? In-Reply-To: <87czm9u1gl.fsf@smart-cactus.org> (sfid-H-20211206-114543-+42.62-1@multi.osbf.lua) References: <20211206162658.904612C1E06@homedog.cs.tufts.edu> <87czm9u1gl.fsf@smart-cactus.org> (sfid-H-20211206-114543-+42.62-1@multi.osbf.lua) Message-ID: <20211206212446.8D8962C2649@homedog.cs.tufts.edu> > > I'm emitting code in which I want to prettyprint the contents > > of a Cmm Block as a comment, by which I mean putting `--` to the left > > of *each* line of the rendered Cmm Block. Using GHC.Utils.Outputable, > > is this even possible? How would I tackle it? (The Haddock page > > does not suggest any obvious combinators.) > > > It's not easy. One (rather horrible) approach would be to render the > document to a string, split the output into lines and add the comment > symbols, and then splice the result into another document. I'd be willing to go there, except I don't know how I would guess the width of the string I'm hoping to render to. Maybe it wouldn't matter. > Perhaps you could rather use block comment syntax? This would be > much easier. Alas, one goal is to make the comment very obvious to the human reader. Some prettyprinters can be jiggered to insert a specified string at every line break. But I gather SDoc is not one of those. *Sadness* N From nr at cs.tufts.edu Mon Dec 6 21:50:57 2021 From: nr at cs.tufts.edu (Norman Ramsey) Date: Mon, 06 Dec 2021 16:50:57 -0500 Subject: Alternatives for representing a reverse postorder numbering Message-ID: <20211206215057.C802A2C2649@homedog.cs.tufts.edu> Reverse postorder numbering is a superpower for control-flow analysis and other back-end things. For example, - In a reducible flow graph, a node Q is a loop header if and only if it is targeted by an edge P -> Q where Q's reverse postorder number is not greater than P's. - If a loop has multiple exits, the reverse postorder numbering of the exit nodes tells exactly the order in which the nodes must appear so they can be reached by multilevel `exit`/`break` statements, as are found in WebAssembly. - Reverse postorder numbers enable efficient computations of set intersection for dominator analysis. One could go on. In a perfect world, our representation of control-flow graphs would provide a place to store a reverse postorder number for each (reachable) basic block. Alas, we live in an imperfect world, and I am struggling to figure out how best to store reverse postorder numbers for the blocks in a `GenCmmGraph`. 1. One could apply ideas from "trees that grow" to the `Block` type from `GHC.Cmm.Dataflow.Block`. But this type is already one of the more complicated types I have ever encountered in the wild, and the thought of further complexity fills me with horror. 2. One could generalize quite a few types in `Cmm`. In particular, one could create an analog of the `GenCmmGraph` type. The analog, instead of taking a node type `n` as its parameter, would take a block type as its parameter. It would use `Graph'` as defined in `GHC.Cmm.Dataflow.Graph`. This change would ripple into `GHC.Cmm.Dataflow` without doing a whole lot of violence to the code that it there. It would then become possible to do dataflow analysis (and perhaps other operations) over graphs with annotated blocks. It's worth noting that the `Graph'` representation already exists, but it doesn't seem to be used anywhere. I'm not sure how it survived earlier rounds of culling. 3. One could simple build an auxiliary `LabelMap` that includes the reverse postorder number of every node. This idea bugs me a bit. I don't love spending O(log N) steps twice every time I look to see the direction of a control-flow edge. But what I *really* don't love is what happens to the interfaces. I can compute the reverse postorder map twice, or I can pollute my interfaces by saying "here is the dominator relation, and by the way, here also are the reverse postorder numbers, which I had to compute." I'm currently using alternative 3: when I need reverse postorder numbers, I call `revPostorderFrom` (defined in `GHC.Cmm.Dataflow.Graph`), then zip with `[1..]`. But I'm really tempted by alternative 2, which would allow me to, for example, define a graph annotated with reverse postorder numbers, then do both the dominator analysis and my translation on that graph. How deep into the weeds should I go? Make dataflow analysis even more polymorphic? or learn to love `LabelMap`? Norman From sgraf1337 at gmail.com Wed Dec 8 10:19:33 2021 From: sgraf1337 at gmail.com (Sebastian Graf) Date: Wed, 08 Dec 2021 10:19:33 +0000 Subject: Transparently hooking into the STG stack to validate an escape analysis Message-ID: Hi Devs, my master's student Sebastian and I (also Sebastian :)) are working on an escape analysis in STG, see https://gitlab.haskell.org/ghc/ghc/-/issues/16891#note_347903. We have a prototype for the escape analysis that we want to validate/exploit now. The original plan was to write the transformation that allocates non-escaping objects on the STG stack. But that is quite tricky for many reasons, one of them being treatment of the GC. This mail is rather lengthy, so I suggest you skip to "What we hope could work" and see if you can answer it without the context I provide below. If you can't, I would be very grateful if you were willing to suffer through the exposition. # Instrumentation So instead we thought about doing a (easily changed and thus versatile) instrumentation-based approach: Assign a sequence number to every instantiation (a term I will use to mean "allocation of g's closure") of things that we our analysis determines as escaping or non-escaping, such as STG's let bindings (focusing on non-let-no-escape functions for now). (One sequence number *per allocation* of the let binding's closure, not based on its syntactic identity.) Then, we set a bit in a (dynamically growing) global bit vector whenever the let "RHS is entered" and then unset it when we "leave the let-body". Example: f = \x y -> let { g = [y] \z -> y + z; } in g x Here, our analysis could see that no instantiation (which I say instead of "allocation of g's closure") of g will ever escape its scope within f. Our validation would give a fresh sequence number to the instantiation of g whenever f is called and store it in g's closure (which we arrange by piggy-backing on -prof and adding an additional field to the profiling header). Then, when g's RHS is entered, we set the bit in the global bit vector, indicating "this instantiation of g might escape". After leaving the RHS of g, we also leave the body of the defining let, which means we unset the bit in the bit vector, meaning "every use so far wasn't in an escaping scenario". So far so good. Modifying the code upon entering g takes a bit of tinkering but can be done by building on TickyTicky in StgToCmm. But what is not done so easily is inserting the continuation upon entering the let that will unset the bit! # What doesn't work: Modifying the Sequel At first, we tried to modify the sequel of the let-body to an `AssignTo`. That requires us to know the registers in which the let-body will return its results, which in turn means we have to know the representation of those results, so we have to write a function `stgExprPrimRep :: GenStgExpr p -> [PrimRep]`. Urgh! We were very surprised that there was no such function. And while we tested our function, we very soon knew why. Consider the following pattern synonym matcher: GHC.Natural.$mNatJ# :: forall {rep :: GHC.Types.RuntimeRep} {r :: TYPE rep}. GHC.Num.Natural.Natural -> (GHC.Num.BigNat.BigNat -> r) -> ((# #) -> r) -> r = {} \r [scrut_sBE cont_sBF fail_sBG] case scrut_sBE of { GHC.Num.Natural.NS _ -> fail_sBG GHC.Prim.(##); GHC.Num.Natural.NB ds_sBJ -> let { sat_sBK :: GHC.Num.BigNat.BigNat = CCCS GHC.Num.BigNat.BN#! [ds_sBJ]; } in cont_sBF sat_sBK; }; Note how its result is representation-polymorphic! It only works because our machine implementation allows tail-calls. It's obvious in hindsight that we could never write `stgExprPrimRep` in such a way that it will work on the expression `cont_sBF sat_sBK`. So the sequel approach seems infeasible. # What we hope could work: A special stack frame The other alternative would be to insert a special continuation frame on the stack when we enter the let-body (inspired by stg_restore_cccs). This continuation frame would simply push all registers (FP regs, GP regs, Vec regs, ...) to the C stack, do its work (unsetting the bit), then pop all registers again and jump to the topmost continuation on the STG stack. Example: f :: forall rep (r :: TYPE rep). Int# -> (Int# -> r) -> r f = \x g -> let { h = [x] \a -> x + a; } in case h x of b { __DEFAULT -> g b } We are only interested in unsetting the bit for h here. Consider the stack when entering the body of h. caller_of_f_cont_info <- Sp Now push our special continuation frame: caller_of_f_cont_info seq_h unset_bit_stk_info <- Sp E.g., the stack frame contains the info pointer and the sequence number. (Btw., I hope I got the stack layout about right and this is even possible) Then, after we entered the continuation of the __DEFAULT alt, we do a jump to g. Plot twist: g returns an unboxed 8-tuple of `Int#`s (as caller_of_f_cont_info knows, but f certainly doesn't!), so before it returns it will push two args on the stack (correct?): caller_of_f_cont_info seq_h unset_bit_stk_info unboxed tuple component 7 unboxed tuple component 8 <- Sp And then `g` jumps directly to the entry code for `unset_bit_stk_info` (which does the register saving I mentioned), which absolutely can't figure out from Sp alone where seq_h is. Darn! I think Luite's recent work on the StgToByteCode had to work around similar issues, I found this wiki page . But we aren't in a position where we know the representation of `r` *at all*! So our idea was to scan the stack, beginning from `Sp`, until I find `unset_bit_stk_info`, giving us the following situation: caller_of_f_cont_info seq_h unset_bit_stk_info <- Bp unboxed tuple component 7 unboxed tuple component 8 <- Sp I suggestively named the register in which we store the result Bp in analogy to the traditional base pointer. This information would be enough to unset the bit at index seq_h and then copy the unboxed tuple components 7 and 8 up by two words: caller_of_f_cont_info unboxed tuple component 7 unboxed tuple component 8 <- Sp Then jump to caller_of_f_cont_info, which knows what to make of the stack and the register state. The stack scanning is horrible and too brittle and slow for a production setting, as any of the unboxed tuple components could have the same bit pattern as `unset_bit_stk_info`. We simply hope that is never the case and it's fine for the purposes of a quick-and-dirty instrumentation. QUESTION 1: What do you think? Could this work? Can you anticipate pit falls of this approach? # What about Thunks and StgRhsCon? QUESTION 2: The instrumentation as described won't work for Thunks (which are only entered once) and constructor applications (like sat_sBK in the BigNat matcher above). Not sure how to change that without incurring huge performance hits (by deactivating memoisation). Ideas are welcome here. Thanks for reading this far. I hope you could follow and are now fizzing with excitement because you have a much better idea of how to do this and will let me know :) Sebastian -------------- next part -------------- An HTML attachment was scrubbed... URL: From lexi.lambda at gmail.com Wed Dec 8 19:40:15 2021 From: lexi.lambda at gmail.com (Alexis King) Date: Wed, 8 Dec 2021 13:40:15 -0600 Subject: Concrete#, SpecialPred, and CtEvidence invariants Message-ID: Hi all, After a recent off-list conversation, I’ve been exploring using the new SpecialPred mechanism to implement a custom constraint, but I’ve run into some questions regarding evidence for CSpecialCan constraints. Specifically, I’m uncertain about how to use them while satisfying Note [CtEvidence invariants] in GHC.Tc.Types.Constraint. Specifically, the Note in question states that ctev_pred must always be kept in sync with the type of the evidence. For example, if ctev_dest for a wanted constraint is a coercion hole, ctev_pred must be `varType (coHoleCoVar hole)`. However, this seems rather odd in the case of Concrete# constraints, since the evidence for a `Concrete# (ty :: ki)` constraint is a coercion of type `ty ~# alpha`. In canNonDecomposableConcretePrim in GHC.Tc.Solver.Canonical, setCtEvPredType is used to update the type of a CSpecialCan constraint, and setCtEvPredType automatically maintains the CtEvidence invariant mentioned above. But this results in setCoHoleType being used to modify the type of the CoVar from `ty ~# alpha` to `Concrete# (ty :: ki)`, which does not make much sense, since that is definitely *not* the type of the CoVar. As far as I can tell, the only reason that doesn’t cause Core Lint errors is that the modified CoVar is never actually used, only the original one is. This suggests to me that the invariant is not actually the right one. The Note suggests that the reason the invariant exists is that the type must be kept fully-zonked… which in turn suggests that perhaps fully-zonkedness is the invariant that’s actually desired. In that case, ctev_pred would *not* necessarily always be a cache for the type of the evidence, since in the case of Concrete#, the evidence has a different type (though for other types of constraints the old invariant would still coincidentally hold). Rather, it would be the responsibility of canNonDecomposableConcretePrim to merely ensure the evidence’s type is appropriately zonked. Does this all sound right to people? If so, I will update the Note with the modified invariant and update the code to match. Alexis -------------- next part -------------- An HTML attachment was scrubbed... URL: From lists at richarde.dev Wed Dec 8 20:20:52 2021 From: lists at richarde.dev (Richard Eisenberg) Date: Wed, 8 Dec 2021 20:20:52 +0000 Subject: Concrete#, SpecialPred, and CtEvidence invariants In-Reply-To: References: Message-ID: <010f017d9bb4bbde-66f1f310-7439-4f2a-9688-fc1668fa5be2-000000@us-east-2.amazonses.com> Good point. The Note is wrong. But I don't like an invariant on a data structure about zonkedness: the zonkedness of a type can change whenever we fill in a unification variable, possibly very far away. (On the other hand, it does make sense for functions to talk about zonkedness in preconditions and postconditions, because function calls happen at a certain instant of time.) Instead, I think the invariant is good as it is except for CSpecialPreds. Which are special. :) And, you're right that a CoVar should never have a type of Concrete# blah. Find that code and kill it. Does this help? Richard > On Dec 8, 2021, at 2:40 PM, Alexis King wrote: > > Hi all, > > After a recent off-list conversation, I’ve been exploring using the new SpecialPred mechanism to implement a custom constraint, but I’ve run into some questions regarding evidence for CSpecialCan constraints. Specifically, I’m uncertain about how to use them while satisfying Note [CtEvidence invariants] in GHC.Tc.Types.Constraint. > > Specifically, the Note in question states that ctev_pred must always be kept in sync with the type of the evidence. For example, if ctev_dest for a wanted constraint is a coercion hole, ctev_pred must be `varType (coHoleCoVar hole)`. However, this seems rather odd in the case of Concrete# constraints, since the evidence for a `Concrete# (ty :: ki)` constraint is a coercion of type `ty ~# alpha`. > > In canNonDecomposableConcretePrim in GHC.Tc.Solver.Canonical, setCtEvPredType is used to update the type of a CSpecialCan constraint, and setCtEvPredType automatically maintains the CtEvidence invariant mentioned above. But this results in setCoHoleType being used to modify the type of the CoVar from `ty ~# alpha` to `Concrete# (ty :: ki)`, which does not make much sense, since that is definitely not the type of the CoVar. As far as I can tell, the only reason that doesn’t cause Core Lint errors is that the modified CoVar is never actually used, only the original one is. > > This suggests to me that the invariant is not actually the right one. The Note suggests that the reason the invariant exists is that the type must be kept fully-zonked… which in turn suggests that perhaps fully-zonkedness is the invariant that’s actually desired. In that case, ctev_pred would not necessarily always be a cache for the type of the evidence, since in the case of Concrete#, the evidence has a different type (though for other types of constraints the old invariant would still coincidentally hold). Rather, it would be the responsibility of canNonDecomposableConcretePrim to merely ensure the evidence’s type is appropriately zonked. > > Does this all sound right to people? If so, I will update the Note with the modified invariant and update the code to match. > > Alexis -------------- next part -------------- An HTML attachment was scrubbed... URL: From klebinger.andreas at gmx.at Thu Dec 9 13:24:17 2021 From: klebinger.andreas at gmx.at (Andreas Klebinger) Date: Thu, 9 Dec 2021 14:24:17 +0100 Subject: Alternatives for representing a reverse postorder numbering In-Reply-To: <20211206215057.C802A2C2649@homedog.cs.tufts.edu> References: <20211206215057.C802A2C2649@homedog.cs.tufts.edu> Message-ID: <27f05c9d-9479-1a4c-672a-da3dd4fdad6c@gmx.at> Hello Norman, There is no invariant that Cmm control flow is reducible. So we can't always rely on this being the case. Depending on what you want to use this for this might or might not matter. In terms of implementation I think the question is if doing lookups in a LabelMap is more expensive than making the CmmGraph representation both more polymorphic, and putting more info into the Graph. Which I guess mostly depends on how much mileage we get out of the numbering. Which is impossible for me to say in advance. If you only need/use this info for a small part of the pipeline then keeping it as LabelMap seems more reasonable. If you have plans to improve all sorts of passes with this information at various stages in the pipeline integrating it into CmmGraph seems better. It's impossible to say without knowing the details. All that being said I rarely have lost sleep over the overhead of looking things up in IntMaps. The constants are pretty good there and it seems reasonable easy to change it later if needed. Am 06/12/2021 um 22:50 schrieb Norman Ramsey: > Reverse postorder numbering is a superpower for control-flow analysis > and other back-end things. For example, > > - In a reducible flow graph, a node Q is a loop header if and only if > it is targeted by an edge P -> Q where Q's reverse postorder > number is not greater than P's. > > - If a loop has multiple exits, the reverse postorder numbering of > the exit nodes tells exactly the order in which the nodes must > appear so they can be reached by multilevel `exit`/`break` > statements, as are found in WebAssembly. > > - Reverse postorder numbers enable efficient computations of set > intersection for dominator analysis. > > One could go on. > > In a perfect world, our representation of control-flow graphs would > provide a place to store a reverse postorder number for each > (reachable) basic block. Alas, we live in an imperfect world, and I > am struggling to figure out how best to store reverse postorder > numbers for the blocks in a `GenCmmGraph`. > > 1. One could apply ideas from "trees that grow" to the `Block` type > from `GHC.Cmm.Dataflow.Block`. But this type is already one of > the more complicated types I have ever encountered in the wild, > and the thought of further complexity fills me with horror. > > 2. One could generalize quite a few types in `Cmm`. In particular, > one could create an analog of the `GenCmmGraph` type. The analog, > instead of taking a node type `n` as its parameter, would take a > block type as its parameter. It would use `Graph'` as defined in > `GHC.Cmm.Dataflow.Graph`. This change would ripple into > `GHC.Cmm.Dataflow` without doing a whole lot of violence to the > code that it there. It would then become possible to do dataflow > analysis (and perhaps other operations) over graphs with annotated > blocks. > > It's worth noting that the `Graph'` representation already exists, > but it doesn't seem to be used anywhere. I'm not sure how it > survived earlier rounds of culling. > > 3. One could simple build an auxiliary `LabelMap` that includes the > reverse postorder number of every node. This idea bugs me a bit. > I don't love spending O(log N) steps twice every time I look to > see the direction of a control-flow edge. But what I *really* > don't love is what happens to the interfaces. I can compute the > reverse postorder map twice, or I can pollute my interfaces by > saying "here is the dominator relation, and by the way, here also > are the reverse postorder numbers, which I had to compute." > > I'm currently using alternative 3: when I need reverse postorder > numbers, I call `revPostorderFrom` (defined in `GHC.Cmm.Dataflow.Graph`), > then zip with `[1..]`. But I'm really tempted by alternative 2, > which would allow me to, for example, define a graph annotated with > reverse postorder numbers, then do both the dominator analysis and my > translation on that graph. > > How deep into the weeds should I go? Make dataflow analysis even more polymorphic? > or learn to love `LabelMap`? > > > Norman > > > > > > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs From lexi.lambda at gmail.com Thu Dec 9 18:33:20 2021 From: lexi.lambda at gmail.com (Alexis King) Date: Thu, 9 Dec 2021 12:33:20 -0600 Subject: Concrete#, SpecialPred, and CtEvidence invariants In-Reply-To: <010f017d9bb4bbde-66f1f310-7439-4f2a-9688-fc1668fa5be2-000000@us-east-2.amazonses.com> References: <010f017d9bb4bbde-66f1f310-7439-4f2a-9688-fc1668fa5be2-000000@us-east-2.amazonses.com> Message-ID: On Wed, Dec 8, 2021 at 2:20 PM Richard Eisenberg wrote: > Does this help? > Yes, it does, and what you said makes sense. Really, the intent of the invariant isn’t “zonkedness” *per se*, but merely that the type of the evidence should be “at least as zonked” as ctev_pred is. But that is a little vague and handwavy, so I’ll go the route you suggest and just reword things a little to explain. Thanks! -------------- next part -------------- An HTML attachment was scrubbed... URL: From nr at cs.tufts.edu Thu Dec 9 19:14:18 2021 From: nr at cs.tufts.edu (Norman Ramsey) Date: Thu, 09 Dec 2021 14:14:18 -0500 Subject: Alternatives for representing a reverse postorder numbering In-Reply-To: <27f05c9d-9479-1a4c-672a-da3dd4fdad6c@gmx.at> (sfid-H-20211209-091818-+40.61-1@multi.osbf.lua) References: <20211206215057.C802A2C2649@homedog.cs.tufts.edu> <27f05c9d-9479-1a4c-672a-da3dd4fdad6c@gmx.at> (sfid-H-20211209-091818-+40.61-1@multi.osbf.lua) Message-ID: <20211209191418.AE6002C1EDF@homedog.cs.tufts.edu> > Which I guess mostly depends on how much mileage we get out of the > numbering... I rarely have lost sleep over the overhead of looking > things up in IntMaps. Thank you!! I found your analysis very helpful. I will stick with the IntMaps until and unless things reach a stage where they look really ugly. > There is no invariant that Cmm control flow is reducible. So we > can't always rely on this being the case. Good to know. I would still like to have a simple Haskell example that generates an irreducible control-flow graph, but for now I can just write them by hand using .cmm files. BTW *every* control-flow graph has at least one reverse-postorder numbering, whether it is reducible or not. Norman From sgraf1337 at gmail.com Thu Dec 9 20:41:07 2021 From: sgraf1337 at gmail.com (Sebastian Graf) Date: Thu, 9 Dec 2021 20:41:07 +0000 Subject: Alternatives for representing a reverse postorder numbering In-Reply-To: <20211209191418.AE6002C1EDF@homedog.cs.tufts.edu> References: <20211206215057.C802A2C2649@homedog.cs.tufts.edu> <27f05c9d-9479-1a4c-672a-da3dd4fdad6c@gmx.at> (sfid-H-20211209-091818-+40.61-1@multi.osbf.lua) <20211209191418.AE6002C1EDF@homedog.cs.tufts.edu> Message-ID: FWIW, performance of IntMap could be even better if we had mutable fields and a transient (one with freeze/thaw conversion) interface. We'd need a GHC with https://github.com/ghc-proposals/ghc-proposals/pull/8/files for that, though... I think we could also speed up substitution by using a transient substitution type. ________________________________ Von: ghc-devs im Auftrag von Norman Ramsey Gesendet: Donnerstag, Dezember 9, 2021 8:16 PM An: Andreas Klebinger Cc: ghc-devs at haskell.org Betreff: Re: Alternatives for representing a reverse postorder numbering > Which I guess mostly depends on how much mileage we get out of the > numbering... I rarely have lost sleep over the overhead of looking > things up in IntMaps. Thank you!! I found your analysis very helpful. I will stick with the IntMaps until and unless things reach a stage where they look really ugly. > There is no invariant that Cmm control flow is reducible. So we > can't always rely on this being the case. Good to know. I would still like to have a simple Haskell example that generates an irreducible control-flow graph, but for now I can just write them by hand using .cmm files. BTW *every* control-flow graph has at least one reverse-postorder numbering, whether it is reducible or not. Norman _______________________________________________ ghc-devs mailing list ghc-devs at haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs -------------- next part -------------- An HTML attachment was scrubbed... URL: From ben at well-typed.com Mon Dec 13 00:58:41 2021 From: ben at well-typed.com (Ben Gamari) Date: Sun, 12 Dec 2021 19:58:41 -0500 Subject: GitLab upgrade starting soon Message-ID: <87o85liamv.fsf@smart-cactus.org> Hi all, I will be performing an upgrade of gitlab.haskell.org in roughly 30 minutes if there are no objections. I expect the down-time to be on the order of 20 minutes. Cheers, - Ben -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From nr at cs.tufts.edu Tue Dec 14 19:23:35 2021 From: nr at cs.tufts.edu (Norman Ramsey) Date: Tue, 14 Dec 2021 14:23:35 -0500 Subject: Coping with multiple meanings of `<>` Message-ID: <20211214192335.312562C1EE8@homedog.cs.tufts.edu> I find myself wanting to define instances of Semigroup (and Monoid) in a file that also imports GHC.Utils.Outputable and its `<>` operation on SDocs. At the moment I am dealing with the incompatibility by hiding the Outputable version and instead of writing `s1 <> s2` I write `hcat [s1, s2]`. This workaround seems ugly and vaguely embarrassing. How are others dealing with this issue? Would it be sensible simply to make SDoc an instance of Semigroup (and Monoid), or would we be concerned about potential additional overhead at compile time? Norman From cheng.shao at tweag.io Wed Dec 15 02:06:34 2021 From: cheng.shao at tweag.io (Cheng Shao) Date: Wed, 15 Dec 2021 03:06:34 +0100 Subject: Thoughts on async RTS API? Message-ID: Hi devs, To invoke Haskell computation in C, we need to call one of rts_eval* functions, which enters the scheduler loop, and returns only when the specified Haskell thread is finished or killed. We'd like to enhance the scheduler and add async variants of the rts_eval* functions, which take C callbacks to consume the Haskell thread result, kick off the scheduler loop, and the loop is allowed to exit when the Haskell thread is blocked. Sync variants of RTS API will continue to work with unchanged behavior. The main intended use case is async foreign calls for the WebAssembly target. When an async foreign call is made, the Haskell thread will block on an MVar to be fulfilled with the call result. But the scheduler will eventually fail to find work due to empty run queue and exit with error! We need a way to gracefully exit the scheduler, so the RTS API caller can process the async foreign call, fulfill that MVar and resume Haskell computation later. Question I: does the idea of adding async RTS API sound acceptable by GHC HQ? To be honest, it's not impossible to workaround lack of async RTS API: reuse the awaitEvent() logic in non-threaded RTS, pretend each async foreign call reads from a file descriptor and can be handled by the POSIX select() function in awaitEvent(). But it'd surely be nice to avoid such hacks and do things the principled way. Question II: how to modify the scheduler loop to implement this feature? Straightforward answer seems to be: check some RTS API non-blocking flag, if present, allow early exit due to empty run queue. Thanks a lot for reading this, I appreciate any suggestions or questions :) Best regards, Cheng From sylvain at haskus.fr Wed Dec 15 08:53:42 2021 From: sylvain at haskus.fr (Sylvain Henry) Date: Wed, 15 Dec 2021 09:53:42 +0100 Subject: Coping with multiple meanings of `<>` In-Reply-To: <20211214192335.312562C1EE8@homedog.cs.tufts.edu> References: <20211214192335.312562C1EE8@homedog.cs.tufts.edu> Message-ID: <9ec721b1-41f7-e4ff-3edf-5083aa544159@haskus.fr> Hi Norman, Usually in the compiler Semigoup's <> is imported qualified. But I agree it's ugly. The trouble with Outputable's <> is that: 1) it doesn't have the same associativity as Semigroup's <> 2) <+> interacts weirdly with <> (cf https://mail.haskell.org/pipermail/libraries/2011-November/017066.html) I have rediscovered this when trying to fix it 2 months ago: https://gitlab.haskell.org/hsyl20/ghc/-/commits/hsyl20/outputable-append I have tried to add a new constructor to fix (2) https://gitlab.haskell.org/hsyl20/ghc/-/commit/5d09acf4825a816ddb2ca2ec7294639b969ff64b but it's still failing (https://gitlab.haskell.org/hsyl20/ghc/-/jobs/791114). Any help fixing these issues would be appreciated :) Cheers, Sylvain On 14/12/2021 20:23, Norman Ramsey wrote: > I find myself wanting to define instances of Semigroup (and Monoid) > in a file that also imports GHC.Utils.Outputable and its `<>` operation > on SDocs. At the moment I am dealing with the incompatibility by > hiding the Outputable version and instead of writing `s1 <> s2` I write > `hcat [s1, s2]`. This workaround seems ugly and vaguely embarrassing. > > How are others dealing with this issue? Would it be sensible simply > to make SDoc an instance of Semigroup (and Monoid), or would we be > concerned about potential additional overhead at compile time? > > > Norman > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs From csaba.hruska at gmail.com Wed Dec 15 15:16:27 2021 From: csaba.hruska at gmail.com (Csaba Hruska) Date: Wed, 15 Dec 2021 16:16:27 +0100 Subject: Transparently hooking into the STG stack to validate an escape analysis In-Reply-To: References: Message-ID: Hi, IMO the Cmm STG machine implementation is just too complex for student projects. It's not fun to work with at all. Why did you choose this approach? IMO the escape analysis development and validation would be much smoother and fun when you'd use the external STG interpreter. When you have a solid and working design of your analysis and transformations then you could implement it in GHC's native backend if it needs any changes at all. What do you think? Do you disagree? Have you seen my presentation about the stg interpreter? https://www.youtube.com/watch?v=Ey5OFPkxF_w Cheers, Csaba On Wed, Dec 8, 2021 at 11:20 AM Sebastian Graf wrote: > Hi Devs, > > my master's student Sebastian and I (also Sebastian :)) are working on an > escape analysis in STG, see > https://gitlab.haskell.org/ghc/ghc/-/issues/16891#note_347903. > > We have a prototype for the escape analysis that we want to > validate/exploit now. > The original plan was to write the transformation that allocates > non-escaping objects on the STG stack. But that is quite tricky for many > reasons, one of them being treatment of the GC. > > This mail is rather lengthy, so I suggest you skip to "What we hope could > work" and see if you can answer it without the context I provide below. If > you can't, I would be very grateful if you were willing to suffer through > the exposition. > > # Instrumentation > > So instead we thought about doing a (easily changed and thus versatile) > instrumentation-based approach: > Assign a sequence number to every instantiation (a term I will use to > mean "allocation of g's closure") of things that we our analysis > determines as escaping or non-escaping, such as STG's let bindings > (focusing on non-let-no-escape functions for now). > (One sequence number *per allocation* of the let binding's closure, not > based on its syntactic identity.) > Then, we set a bit in a (dynamically growing) global bit vector whenever > the let "RHS is entered" and then unset it when we "leave the let-body". > Example: > > f = \x y -> > let { > g = [y] \z -> y + z; > } in g x > > Here, our analysis could see that no instantiation (which I say instead of > "allocation of g's closure") of g will ever escape its scope within f. > Our validation would give a fresh sequence number to the instantiation of > g whenever f is called and store it in g's closure (which we arrange by > piggy-backing on -prof and adding an additional field to the profiling > header). > Then, when g's RHS is entered, we set the bit in the global bit vector, > indicating "this instantiation of g might escape". > After leaving the RHS of g, we also leave the body of the defining let, > which means we unset the bit in the bit vector, meaning "every use so far > wasn't in an escaping scenario". > > So far so good. Modifying the code upon entering g takes a bit of > tinkering but can be done by building on TickyTicky in StgToCmm. > But what is not done so easily is inserting the continuation upon entering > the let that will unset the bit! > > # What doesn't work: Modifying the Sequel > > At first, we tried to modify the sequel > of > the let-body to an `AssignTo`. > That requires us to know the registers in which the let-body will return > its results, which in turn means we have to know the representation of > those results, so we have to write a function `stgExprPrimRep :: GenStgExpr > p -> [PrimRep]`. > Urgh! We were very surprised that there was no such function. And while we > tested our function, we very soon knew why. Consider the following pattern > synonym matcher: > > GHC.Natural.$mNatJ# > :: forall {rep :: GHC.Types.RuntimeRep} {r :: TYPE rep}. > GHC.Num.Natural.Natural > -> (GHC.Num.BigNat.BigNat -> r) -> ((# #) -> r) -> r > = {} \r [scrut_sBE cont_sBF fail_sBG] > case scrut_sBE of { > GHC.Num.Natural.NS _ -> fail_sBG GHC.Prim.(##); > GHC.Num.Natural.NB ds_sBJ -> > let { > sat_sBK :: GHC.Num.BigNat.BigNat > = CCCS GHC.Num.BigNat.BN#! [ds_sBJ]; > } in cont_sBF sat_sBK; > }; > > Note how its result is representation-polymorphic! It only works because > our machine implementation allows tail-calls. > It's obvious in hindsight that we could never write `stgExprPrimRep` in > such a way that it will work on the expression `cont_sBF sat_sBK`. > So the sequel approach seems infeasible. > > # What we hope could work: A special stack frame > > The other alternative would be to insert a special continuation frame on > the stack when we enter the let-body (inspired by stg_restore_cccs). > This continuation frame would simply push all registers (FP regs, GP regs, > Vec regs, ...) to the C stack, do its work (unsetting the bit), then pop > all registers again and jump to the topmost continuation on the STG stack. > Example: > > f :: forall rep (r :: TYPE rep). Int# -> (Int# -> r) -> r > f = \x g -> > let { > h = [x] \a -> x + a; > } in > case h x of b { > __DEFAULT -> g b > } > > We are only interested in unsetting the bit for h here. Consider the stack > when entering the body of h. > > caller_of_f_cont_info <- Sp > > Now push our special continuation frame: > > caller_of_f_cont_info > seq_h > unset_bit_stk_info <- Sp > > E.g., the stack frame contains the info pointer and the sequence number. > (Btw., I hope I got the stack layout about right and this is even possible) > Then, after we entered the continuation of the __DEFAULT alt, we do a jump > to g. > Plot twist: g returns an unboxed 8-tuple of `Int#`s (as caller_of_f_cont_info > knows, but f certainly doesn't!), so before it returns it will push two > args on the stack (correct?): > > caller_of_f_cont_info > seq_h > unset_bit_stk_info > unboxed tuple component 7 > unboxed tuple component 8 <- Sp > > And then `g` jumps directly to the entry code for `unset_bit_stk_info` > (which does the register saving I mentioned), which absolutely can't figure > out from Sp alone where seq_h is. > Darn! I think Luite's recent work on the StgToByteCode had to work around > similar issues, I found this wiki page > . > But we aren't in a position where we know the representation of `r` *at > all*! > > So our idea was to scan the stack, beginning from `Sp`, until I find > `unset_bit_stk_info`, giving us the following situation: > > caller_of_f_cont_info > seq_h > unset_bit_stk_info <- Bp > unboxed tuple component 7 > unboxed tuple component 8 <- Sp > > I suggestively named the register in which we store the result Bp in > analogy to the traditional base pointer. This information would be enough > to unset the bit at index seq_h > and then copy the unboxed tuple components 7 and 8 up by two words: > > caller_of_f_cont_info > unboxed tuple component 7 > unboxed tuple component 8 <- Sp > > Then jump to caller_of_f_cont_info, which knows what to make of the stack > and the register state. > > The stack scanning is horrible and too brittle and slow for a production > setting, as any of the unboxed tuple components could have the same bit > pattern as `unset_bit_stk_info`. > We simply hope that is never the case and it's fine for the purposes of a > quick-and-dirty instrumentation. > > QUESTION 1: What do you think? Could this work? Can you anticipate pit > falls of this approach? > > # What about Thunks and StgRhsCon? > > QUESTION 2: The instrumentation as described won't work for Thunks (which > are only entered once) and constructor applications (like sat_sBK in the > BigNat matcher above). Not sure how to change that without incurring huge > performance hits (by deactivating memoisation). Ideas are welcome here. > > Thanks for reading this far. I hope you could follow and are now fizzing > with excitement because you have a much better idea of how to do this and > will let me know :) > > Sebastian > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ben at well-typed.com Wed Dec 15 16:05:15 2021 From: ben at well-typed.com (Ben Gamari) Date: Wed, 15 Dec 2021 11:05:15 -0500 Subject: Thoughts on async RTS API? In-Reply-To: References: Message-ID: <87lf0lc0rd.fsf@smart-cactus.org> Cheng Shao writes: > Hi devs, > > To invoke Haskell computation in C, we need to call one of rts_eval* > functions, which enters the scheduler loop, and returns only when the > specified Haskell thread is finished or killed. We'd like to enhance > the scheduler and add async variants of the rts_eval* functions, which > take C callbacks to consume the Haskell thread result, kick off the > scheduler loop, and the loop is allowed to exit when the Haskell > thread is blocked. Sync variants of RTS API will continue to work with > unchanged behavior. > > The main intended use case is async foreign calls for the WebAssembly > target. When an async foreign call is made, the Haskell thread will > block on an MVar to be fulfilled with the call result. But the > scheduler will eventually fail to find work due to empty run queue and > exit with error! We need a way to gracefully exit the scheduler, so > the RTS API caller can process the async foreign call, fulfill that > MVar and resume Haskell computation later. > > Question I: does the idea of adding async RTS API sound acceptable by > GHC HQ? To be honest, it's not impossible to workaround lack of async > RTS API: reuse the awaitEvent() logic in non-threaded RTS, pretend > each async foreign call reads from a file descriptor and can be > handled by the POSIX select() function in awaitEvent(). But it'd > surely be nice to avoid such hacks and do things the principled way. > While the idea here sounds reasonable, I'm not sure I quite understand how this will be used in Asterius's case. Specifically, I would be worried about the lack of fairness in this scheme: no progress will be made on any foreign call until all Haskell evaluation has blocked. Is this really the semantics that you want? > Question II: how to modify the scheduler loop to implement this > feature? Straightforward answer seems to be: check some RTS API > non-blocking flag, if present, allow early exit due to empty run > queue. > `schedule` is already a very large function with loops, gotos, mutability, and quite complex control flow. I would be reluctant to add to this complexity without first carrying out some simplification. Instead of adding yet another bail-out case to the loop, I would probably rather try to extract the loop body into a new function. That is, currently `schedule` is of the form: // Perform work until we are asked to shut down. Capability *schedule (Capability *initialCapability, Task *task) { Capability *cap = initialCapability; while (1) { scheduleYield(&cap, task); if (emptyRunQueue(cap)) { continue; } if (shutting_down) { return cap; } StgTSO *t = popRunQueue(cap); if (! t.can_run_on_capability(cap)) { // Push back on the run queue and loop around again to // yield the capability to the appropriate task pushOnRunQueue(cap, t); continue; } runMutator(t); if (needs_gc) { scheduleDoGC(); } } } I might rather extract this into something like: enum ScheduleResult { NoWork, // There was no work to do PerformedWork, // Ran precisely one thread Yield, // The next thread scheduled to run cannot run on the // given capability; yield. ShuttingDown, // We were asked to shut down } // Schedule at most one thread once ScheduleResult scheduleOnce (Capability **cap, Task *task) { if (emptyRunQueue(cap)) { return NoWork; } if (shutting_down) { return ShuttingDown; } StgTSO *t = popRunQueue(cap); if (! t.can_run_on_capability(cap)) { pushOnRunQueue(cap, t); return Yield; } runMutator(t); if (needs_gc) { scheduleDoGC(); } return PerformedWork; } This is just a sketch but I hope it's clear that with something like this this you can easily implement the existing `schedule` function, as well as your asynchronous variant. Cheers, - Ben -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 905 bytes Desc: not available URL: From cheng.shao at tweag.io Wed Dec 15 17:06:19 2021 From: cheng.shao at tweag.io (Cheng Shao) Date: Wed, 15 Dec 2021 18:06:19 +0100 Subject: Thoughts on async RTS API? In-Reply-To: <87lf0lc0rd.fsf@smart-cactus.org> References: <87lf0lc0rd.fsf@smart-cactus.org> Message-ID: > While the idea here sounds reasonable, I'm not sure I quite understand > how this will be used in Asterius's case. Specifically, I would be > worried about the lack of fairness in this scheme: no progress will be > made on any foreign call until all Haskell evaluation has blocked. > Is this really the semantics that you want? Asterius runtime scheduler divides work into individual "tick"s. Each tick does some work, much like one single iteration in the while(1) scheduler loop. Ticks are not synchronously invoked by previous ticks, instead they are started asynchronously and placed inside the host event loop, fully interleaved with other host events. This way, Haskell concurrency works with host concurrency without requiring host multi-threading. It's possible to wait for run queue to be emptied, then process all blocking foreign calls in one batch, similar to awaitEvent() logic in non-threaded RTS. It's also possible to exit scheduler and resume it many more times, similar to current Asterius scheduler. Both semantics can be implemented, to guarantee fairness, the latter sounds more preferrable. The key issue is finding a way to break up the current while(1) loop in schedule() in a principled way. > `schedule` is already a very large function with loops, gotos, > mutability, and quite complex control flow. I would be reluctant > to add to this complexity without first carrying out some > simplification. Instead of adding yet another bail-out case to the loop, > I would probably rather try to extract the loop body into a new > function. That is, currently `schedule` is of the form: > > // Perform work until we are asked to shut down. > Capability *schedule (Capability *initialCapability, Task *task) { > Capability *cap = initialCapability; > while (1) { > scheduleYield(&cap, task); > > if (emptyRunQueue(cap)) { > continue; > } > > if (shutting_down) { > return cap; > } > > StgTSO *t = popRunQueue(cap); > > if (! t.can_run_on_capability(cap)) { > // Push back on the run queue and loop around again to > // yield the capability to the appropriate task > pushOnRunQueue(cap, t); > continue; > } > > runMutator(t); > > if (needs_gc) { > scheduleDoGC(); > } > } > } > > I might rather extract this into something like: > > enum ScheduleResult { > NoWork, // There was no work to do > PerformedWork, // Ran precisely one thread > Yield, // The next thread scheduled to run cannot run on the > // given capability; yield. > ShuttingDown, // We were asked to shut down > } > > // Schedule at most one thread once > ScheduleResult scheduleOnce (Capability **cap, Task *task) { > if (emptyRunQueue(cap)) { > return NoWork; > } > > if (shutting_down) { > return ShuttingDown; > } > > StgTSO *t = popRunQueue(cap); > > if (! t.can_run_on_capability(cap)) { > pushOnRunQueue(cap, t); > return Yield; > } > > runMutator(t); > > if (needs_gc) { > scheduleDoGC(); > } > > return PerformedWork; > } > > This is just a sketch but I hope it's clear that with something like > this this you can easily implement the existing `schedule` function, as > well as your asynchronous variant. > Thanks for the sketch! I definitely agree we should simplify schedule() in some way instead of adding ad-hoc bail out case. The ScheduleResult type and scheduleOnce() function looks good to me, although I need to do a lot more experiments to confirm. Cheers, Cheng From cheng.shao at tweag.io Thu Dec 16 13:57:19 2021 From: cheng.shao at tweag.io (Cheng Shao) Date: Thu, 16 Dec 2021 14:57:19 +0100 Subject: Thoughts on async RTS API? In-Reply-To: References: Message-ID: Hi Alex, Thanks for reminding. hs_try_put_mvar() wouldn't work for our use case. If the C function finishes work and calls hs_try_put_mvar() synchronously, in Haskell takeMVar wouldn't block at all, which is all fine. However, if the C function is expected to call hs_try_put_mvar() asynchronously, the non-threaded RTS will hang! Here's a minimal repro. It works with non-threaded RTS at first, but if you change scheduleCallback() in C so hs_try_put_mvar() is only invoked in a detached pthread, then the program hangs. The proposed async RTS API and related scheduler refactorings can't be avoided, if the MVar is intended to be fulfilled in an async manner, using the non-threaded RTS, on a platform with extremely limited syscall capabilities. ```haskell import Control.Concurrent import Control.Exception import Foreign import Foreign.C import GHC.Conc main :: IO () main = makeExternalCall >>= print makeExternalCall :: IO CInt makeExternalCall = mask_ $ do mvar <- newEmptyMVar sp <- newStablePtrPrimMVar mvar fp <- mallocForeignPtr withForeignPtr fp $ \presult -> do (cap,_) <- threadCapability =<< myThreadId scheduleCallback sp cap presult takeMVar mvar peek presult foreign import ccall "scheduleCallback" scheduleCallback :: StablePtr PrimMVar -> Int -> Ptr CInt -> IO () ``` ```c #include "HsFFI.h" #include "Rts.h" #include "RtsAPI.h" #include #include struct callback { HsStablePtr mvar; int cap; int *presult; }; void* callback(struct callback *p) { usleep(1000); *p->presult = 42; hs_try_putmvar(p->cap,p->mvar); free(p); return NULL; } void scheduleCallback(HsStablePtr mvar, HsInt cap, int *presult) { pthread_t t; struct callback *p = malloc(sizeof(struct callback)); p->mvar = mvar; p->cap = cap; p->presult = presult; // pthread_create(&t, NULL, callback, p); // pthread_detach(t); callback(p); } ``` On Thu, Dec 16, 2021 at 12:10 PM Alexander V Vershilov wrote: > > Hello, replying off-the thread as it would be basically an offtopic. > > But you can achieve the solution using MVars only. > The idea is that you can call mkStablePtr on the MVar that way it will > not be marked as dead, so RTS will not exit. > Then you can use hs_try_put_mvar in C thread to call the thread back. > > On Wed, 15 Dec 2021 at 05:07, Cheng Shao wrote: > > > > Hi devs, > > > > To invoke Haskell computation in C, we need to call one of rts_eval* > > functions, which enters the scheduler loop, and returns only when the > > specified Haskell thread is finished or killed. We'd like to enhance > > the scheduler and add async variants of the rts_eval* functions, which > > take C callbacks to consume the Haskell thread result, kick off the > > scheduler loop, and the loop is allowed to exit when the Haskell > > thread is blocked. Sync variants of RTS API will continue to work with > > unchanged behavior. > > > > The main intended use case is async foreign calls for the WebAssembly > > target. When an async foreign call is made, the Haskell thread will > > block on an MVar to be fulfilled with the call result. But the > > scheduler will eventually fail to find work due to empty run queue and > > exit with error! We need a way to gracefully exit the scheduler, so > > the RTS API caller can process the async foreign call, fulfill that > > MVar and resume Haskell computation later. > > > > Question I: does the idea of adding async RTS API sound acceptable by > > GHC HQ? To be honest, it's not impossible to workaround lack of async > > RTS API: reuse the awaitEvent() logic in non-threaded RTS, pretend > > each async foreign call reads from a file descriptor and can be > > handled by the POSIX select() function in awaitEvent(). But it'd > > surely be nice to avoid such hacks and do things the principled way. > > > > Question II: how to modify the scheduler loop to implement this > > feature? Straightforward answer seems to be: check some RTS API > > non-blocking flag, if present, allow early exit due to empty run > > queue. > > > > Thanks a lot for reading this, I appreciate any suggestions or > > questions :) > > > > Best regards, > > Cheng > > _______________________________________________ > > ghc-devs mailing list > > ghc-devs at haskell.org > > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > > > > -- > -- > Alexander From sgraf1337 at gmail.com Thu Dec 16 16:29:19 2021 From: sgraf1337 at gmail.com (Sebastian Graf) Date: Thu, 16 Dec 2021 16:29:19 +0000 Subject: Transparently hooking into the STG stack to validate an escape analysis In-Reply-To: References: Message-ID: Hey Csaba, After catching up on your talk and reflecting about it a bit, it seems quite obvious that your tool is the right way to collect the data and validate our analysis! Even if meanwhile we decided that a "transparent stack frame" (which I believe is something similar to what you are doing here , with an explicit `argCount` which we do not know) is not the ideal solution we've been looking for (for different reasons). Essentially, we have two things An escape analysis, implemented as an STG-to-STG pass that attaches a boolean flag to each Id whether it "escapes its scope" (for a suitable definition of that). We'd like to implement it in a way that it would be reusable within GHC with moderate effort (e.g., renaming Binder to Id or accounting for different fields), operating on a module at a time rather than whole program.The instrumentation that tries to measure how many heap objects could be allocated on the stack. E.g., set a closure-specific flag whenever the closure is entered, unset that bit (once) when we "leave the scope" that defines the closure. If my understanding is right, we could just implement this "instrumentation" as a simple extra field to Closure , right? Neat! A bit tangential: I see that your interpreter currently allocates a fresh closure for let-no-escapes when it could just re-use the closure of its defining scope. That would skew our numbers somewhat compared to instrumenting GHC-compiled programs, but I think we'd be able to work around that. I also wonder if the semantics of your let-no-escapes are actually as typically specified (operationally), in that a jump to a let-no-escape should also reset the stack pointer. It should hardly matter for the programs that GHC generates, though. I would also be interested in knowing whether the +RTS -s "bytes allocated in the heap" metric (very nearly) coincides with a similar metric you could produce. It would be fantastic if that was the case! Theoretically, that should be possible, right? I think your interpreter work is very valuable to collect data we otherwise would only be able to measure with a TickyTicky-based approach. Nice! Another, similar use case would be to identify the fraction of closures that are only entered once. I remember that there was a ticky-based patch with which Joachim used to measure this fraction (and similarly validate the analysis results), but unfortunately it couldn't end up in master. Ah, yes, we have a whole open ticket about it: #10613 . In fact, that instrumentation is also somewhat similar (adding a field to every closure) as what we want to do. Anyway, it seems like your work will be very valuable in replacing some of the annoying ticky-based instrumentation ideas! Maybe we can have a call some time this or next week to discuss details, once Sebastian and I are more familiar with the code base? Thanks for sticking with the project and doing all the hard work that can build upon! Sebastian ------ Originalnachricht ------ Von: "Csaba Hruska" An: "Sebastian Graf" Cc: "ghc-devs" ; "Sebastian Scheper" Gesendet: 15.12.2021 16:16:27 Betreff: Re: Transparently hooking into the STG stack to validate an escape analysis >Hi, > >IMO the Cmm STG machine implementation is just too complex for student >projects. It's not fun to work with at all. >Why did you choose this approach? >IMO the escape analysis development and validation would be much >smoother and fun when you'd use the external STG interpreter. >When you have a solid and working design of your analysis and >transformations then you could implement it in GHC's native backend if >it needs any changes at all. > >What do you think? >Do you disagree? > >Have you seen my presentation about the stg interpreter? >https://www.youtube.com/watch?v=Ey5OFPkxF_w > >Cheers, >Csaba > >On Wed, Dec 8, 2021 at 11:20 AM Sebastian Graf >wrote: >>Hi Devs, >> >>my master's student Sebastian and I (also Sebastian :)) are working on >>an escape analysis in STG, see >>https://gitlab.haskell.org/ghc/ghc/-/issues/16891#note_347903. >> >>We have a prototype for the escape analysis that we want to >>validate/exploit now. >>The original plan was to write the transformation that allocates >>non-escaping objects on the STG stack. But that is quite tricky for >>many reasons, one of them being treatment of the GC. >> >>This mail is rather lengthy, so I suggest you skip to "What we hope >>could work" and see if you can answer it without the context I provide >>below. If you can't, I would be very grateful if you were willing to >>suffer through the exposition. >> >># Instrumentation >> >>So instead we thought about doing a (easily changed and thus >>versatile) instrumentation-based approach: >>Assign a sequence number to every instantiation (a term I will use to >>mean "allocation of g's closure") of things that we our analysis >>determines as escaping or non-escaping, such as STG's let bindings >>(focusing on non-let-no-escape functions for now). >>(One sequence number *per allocation* of the let binding's closure, >>not based on its syntactic identity.) >>Then, we set a bit in a (dynamically growing) global bit vector >>whenever the let "RHS is entered" and then unset it when we "leave the >>let-body". Example: >> >>f = \x y -> >> let { >> g = [y] \z -> y + z; >> } in g x >> >>Here, our analysis could see that no instantiation (which I say >>instead of "allocation of g's closure") of g will ever escape its >>scope within f. >>Our validation would give a fresh sequence number to the instantiation >>of g whenever f is called and store it in g's closure (which we >>arrange by piggy-backing on -prof and adding an additional field to >>the profiling header). >>Then, when g's RHS is entered, we set the bit in the global bit >>vector, indicating "this instantiation of g might escape". >>After leaving the RHS of g, we also leave the body of the defining >>let, which means we unset the bit in the bit vector, meaning "every >>use so far wasn't in an escaping scenario". >> >>So far so good. Modifying the code upon entering g takes a bit of >>tinkering but can be done by building on TickyTicky in StgToCmm. >>But what is not done so easily is inserting the continuation upon >>entering the let that will unset the bit! >> >># What doesn't work: Modifying the Sequel >> >>At first, we tried to modify the sequel >> >>of the let-body to an `AssignTo`. >>That requires us to know the registers in which the let-body will >>return its results, which in turn means we have to know the >>representation of those results, so we have to write a function >>`stgExprPrimRep :: GenStgExpr p -> [PrimRep]`. >>Urgh! We were very surprised that there was no such function. And >>while we tested our function, we very soon knew why. Consider the >>following pattern synonym matcher: >> >>GHC.Natural.$mNatJ# >> :: forall {rep :: GHC.Types.RuntimeRep} {r :: TYPE rep}. >> GHC.Num.Natural.Natural >> -> (GHC.Num.BigNat.BigNat -> r) -> ((# #) -> r) -> r >> = {} \r [scrut_sBE cont_sBF fail_sBG] >> case scrut_sBE of { >> GHC.Num.Natural.NS _ -> fail_sBG GHC.Prim.(##); >> GHC.Num.Natural.NB ds_sBJ -> >> let { >> sat_sBK :: GHC.Num.BigNat.BigNat >> = CCCS GHC.Num.BigNat.BN#! [ds_sBJ]; >> } in cont_sBF sat_sBK; >> }; >> >>Note how its result is representation-polymorphic! It only works >>because our machine implementation allows tail-calls. >>It's obvious in hindsight that we could never write `stgExprPrimRep` >>in such a way that it will work on the expression `cont_sBF sat_sBK`. >>So the sequel approach seems infeasible. >> >># What we hope could work: A special stack frame >> >>The other alternative would be to insert a special continuation frame >>on the stack when we enter the let-body (inspired by >>stg_restore_cccs). >>This continuation frame would simply push all registers (FP regs, GP >>regs, Vec regs, ...) to the C stack, do its work (unsetting the bit), >>then pop all registers again and jump to the topmost continuation on >>the STG stack. >>Example: >> >>f :: forall rep (r :: TYPE rep). Int# -> (Int# -> r) -> r >>f = \x g -> >> let { >> h = [x] \a -> x + a; >> } in >> case h x of b { >> __DEFAULT -> g b >> } >> >>We are only interested in unsetting the bit for h here. Consider the >>stack when entering the body of h. >> >>caller_of_f_cont_info <- Sp >> >>Now push our special continuation frame: >> >>caller_of_f_cont_info >>seq_h >>unset_bit_stk_info <- Sp >> >>E.g., the stack frame contains the info pointer and the sequence >>number. (Btw., I hope I got the stack layout about right and this is >>even possible) >>Then, after we entered the continuation of the __DEFAULT alt, we do a >>jump to g. >>Plot twist: g returns an unboxed 8-tuple of `Int#`s (as >>caller_of_f_cont_info knows, but f certainly doesn't!), so before it >>returns it will push two args on the stack (correct?): >> >>caller_of_f_cont_info >>seq_h >>unset_bit_stk_info >>unboxed tuple component 7 >>unboxed tuple component 8 <- Sp >> >>And then `g` jumps directly to the entry code for `unset_bit_stk_info` >>(which does the register saving I mentioned), which absolutely can't >>figure out from Sp alone where seq_h is. >>Darn! I think Luite's recent work on the StgToByteCode had to work >>around similar issues, I found this wiki page >>. >>But we aren't in a position where we know the representation of `r` >>*at all*! >> >>So our idea was to scan the stack, beginning from `Sp`, until I find >>`unset_bit_stk_info`, giving us the following situation: >> >>caller_of_f_cont_info >>seq_h >>unset_bit_stk_info <- Bp >>unboxed tuple component 7 >>unboxed tuple component 8 <- Sp >> >>I suggestively named the register in which we store the result Bp in >>analogy to the traditional base pointer. This information would be >>enough to unset the bit at index seq_h >>and then copy the unboxed tuple components 7 and 8 up by two words: >> >>caller_of_f_cont_info >>unboxed tuple component 7 >>unboxed tuple component 8 <- Sp >> >>Then jump to caller_of_f_cont_info, which knows what to make of the >>stack and the register state. >> >>The stack scanning is horrible and too brittle and slow for a >>production setting, as any of the unboxed tuple components could have >>the same bit pattern as `unset_bit_stk_info`. >>We simply hope that is never the case and it's fine for the purposes >>of a quick-and-dirty instrumentation. >> >>QUESTION 1: What do you think? Could this work? Can you anticipate pit >>falls of this approach? >> >># What about Thunks and StgRhsCon? >> >>QUESTION 2: The instrumentation as described won't work for Thunks >>(which are only entered once) and constructor applications (like >>sat_sBK in the BigNat matcher above). Not sure how to change that >>without incurring huge performance hits (by deactivating memoisation). >>Ideas are welcome here. >> >>Thanks for reading this far. I hope you could follow and are now >>fizzing with excitement because you have a much better idea of how to >>do this and will let me know :) >> >>Sebastian >>_______________________________________________ >>ghc-devs mailing list >>ghc-devs at haskell.org >>http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs -------------- next part -------------- An HTML attachment was scrubbed... URL: From csaba.hruska at gmail.com Thu Dec 16 17:22:30 2021 From: csaba.hruska at gmail.com (Csaba Hruska) Date: Thu, 16 Dec 2021 18:22:30 +0100 Subject: Transparently hooking into the STG stack to validate an escape analysis In-Reply-To: References: Message-ID: Thanks for the feedback! Let's have a video meeting. My schedule is flexible. What time is best for you? Cheers, Csaba On Thu, Dec 16, 2021 at 5:29 PM Sebastian Graf wrote: > Hey Csaba, > > After catching up on your talk and reflecting about it a bit, it seems > quite obvious that your tool is the right way to collect the data and > validate our analysis! > Even if meanwhile we decided that a "transparent stack frame" (which I > believe is something similar to what you are doing here > , > with an explicit `argCount` which we do not know) is not the ideal solution > we've been looking for (for different reasons). > > Essentially, we have two things > > 1. An escape analysis, implemented as an STG-to-STG pass that attaches > a boolean flag to each Id whether it "escapes its scope" (for a suitable > definition of that). > We'd like to implement it in a way that it would be reusable within > GHC with moderate effort (e.g., renaming Binder to Id or accounting for > different fields), operating on a module at a time rather than whole > program. > 2. The instrumentation that tries to measure how many heap objects > could be allocated on the stack. E.g., set a closure-specific flag whenever > the closure is entered, unset that bit (once) when we "leave the scope" > that defines the closure. > If my understanding is right, we could just implement this > "instrumentation" as a simple extra field to Closure > , > right? Neat! > > A bit tangential: I see that your interpreter currently allocates a fresh > closure for let-no-escapes > > when it could just re-use the closure of its defining scope. That would > skew our numbers somewhat compared to instrumenting GHC-compiled programs, > but I think we'd be able to work around that. I also wonder if the > semantics of your let-no-escapes are actually as typically specified > (operationally), in that a jump to a let-no-escape should also reset the > stack pointer. It should hardly matter for the programs that GHC generates, > though. > > I would also be interested in knowing whether the +RTS -s "bytes allocated > in the heap" metric (very nearly) coincides with a similar metric you could > produce. It would be fantastic if that was the case! Theoretically, that > should be possible, right? > > I think your interpreter work is very valuable to collect data we > otherwise would only be able to measure with a TickyTicky-based approach. > Nice! > Another, similar use case would be to identify the fraction of closures > that are only entered once. I remember that there was a ticky-based patch > with which Joachim used to measure this fraction > > (and similarly validate the analysis results), but unfortunately it > couldn't end up in master. Ah, yes, we have a whole open ticket about it: > #10613 . In fact, that > instrumentation is also somewhat similar (adding a field to every closure) > as what we want to do. > > Anyway, it seems like your work will be very valuable in replacing some of > the annoying ticky-based instrumentation ideas! > > Maybe we can have a call some time this or next week to discuss details, > once Sebastian and I are more familiar with the code base? > > Thanks for sticking with the project and doing all the hard work that can > build upon! > Sebastian > > ------ Originalnachricht ------ > Von: "Csaba Hruska" > An: "Sebastian Graf" > Cc: "ghc-devs" ; "Sebastian Scheper" < > sebastian.scheper at student.kit.edu> > Gesendet: 15.12.2021 16:16:27 > Betreff: Re: Transparently hooking into the STG stack to validate an > escape analysis > > Hi, > > IMO the Cmm STG machine implementation is just too complex for student > projects. It's not fun to work with at all. > Why did you choose this approach? > IMO the escape analysis development and validation would be much smoother > and fun when you'd use the external STG interpreter. > When you have a solid and working design of your analysis and > transformations then you could implement it in GHC's native backend if it > needs any changes at all. > > What do you think? > Do you disagree? > > Have you seen my presentation about the stg interpreter? > https://www.youtube.com/watch?v=Ey5OFPkxF_w > > Cheers, > Csaba > > On Wed, Dec 8, 2021 at 11:20 AM Sebastian Graf > wrote: > >> Hi Devs, >> >> my master's student Sebastian and I (also Sebastian :)) are working on an >> escape analysis in STG, see >> https://gitlab.haskell.org/ghc/ghc/-/issues/16891#note_347903. >> >> We have a prototype for the escape analysis that we want to >> validate/exploit now. >> The original plan was to write the transformation that allocates >> non-escaping objects on the STG stack. But that is quite tricky for many >> reasons, one of them being treatment of the GC. >> >> This mail is rather lengthy, so I suggest you skip to "What we hope could >> work" and see if you can answer it without the context I provide below. If >> you can't, I would be very grateful if you were willing to suffer through >> the exposition. >> >> # Instrumentation >> >> So instead we thought about doing a (easily changed and thus versatile) >> instrumentation-based approach: >> Assign a sequence number to every instantiation (a term I will use to >> mean "allocation of g's closure") of things that we our analysis >> determines as escaping or non-escaping, such as STG's let bindings >> (focusing on non-let-no-escape functions for now). >> (One sequence number *per allocation* of the let binding's closure, not >> based on its syntactic identity.) >> Then, we set a bit in a (dynamically growing) global bit vector whenever >> the let "RHS is entered" and then unset it when we "leave the let-body". >> Example: >> >> f = \x y -> >> let { >> g = [y] \z -> y + z; >> } in g x >> >> Here, our analysis could see that no instantiation (which I say instead >> of "allocation of g's closure") of g will ever escape its scope within f. >> Our validation would give a fresh sequence number to the instantiation of >> g whenever f is called and store it in g's closure (which we arrange by >> piggy-backing on -prof and adding an additional field to the profiling >> header). >> Then, when g's RHS is entered, we set the bit in the global bit vector, >> indicating "this instantiation of g might escape". >> After leaving the RHS of g, we also leave the body of the defining let, >> which means we unset the bit in the bit vector, meaning "every use so far >> wasn't in an escaping scenario". >> >> So far so good. Modifying the code upon entering g takes a bit of >> tinkering but can be done by building on TickyTicky in StgToCmm. >> But what is not done so easily is inserting the continuation upon >> entering the let that will unset the bit! >> >> # What doesn't work: Modifying the Sequel >> >> At first, we tried to modify the sequel >> of >> the let-body to an `AssignTo`. >> That requires us to know the registers in which the let-body will return >> its results, which in turn means we have to know the representation of >> those results, so we have to write a function `stgExprPrimRep :: GenStgExpr >> p -> [PrimRep]`. >> Urgh! We were very surprised that there was no such function. And while >> we tested our function, we very soon knew why. Consider the following >> pattern synonym matcher: >> >> GHC.Natural.$mNatJ# >> :: forall {rep :: GHC.Types.RuntimeRep} {r :: TYPE rep}. >> GHC.Num.Natural.Natural >> -> (GHC.Num.BigNat.BigNat -> r) -> ((# #) -> r) -> r >> = {} \r [scrut_sBE cont_sBF fail_sBG] >> case scrut_sBE of { >> GHC.Num.Natural.NS _ -> fail_sBG GHC.Prim.(##); >> GHC.Num.Natural.NB ds_sBJ -> >> let { >> sat_sBK :: GHC.Num.BigNat.BigNat >> = CCCS GHC.Num.BigNat.BN#! [ds_sBJ]; >> } in cont_sBF sat_sBK; >> }; >> >> Note how its result is representation-polymorphic! It only works because >> our machine implementation allows tail-calls. >> It's obvious in hindsight that we could never write `stgExprPrimRep` in >> such a way that it will work on the expression `cont_sBF sat_sBK`. >> So the sequel approach seems infeasible. >> >> # What we hope could work: A special stack frame >> >> The other alternative would be to insert a special continuation frame on >> the stack when we enter the let-body (inspired by stg_restore_cccs). >> This continuation frame would simply push all registers (FP regs, GP >> regs, Vec regs, ...) to the C stack, do its work (unsetting the bit), then >> pop all registers again and jump to the topmost continuation on the STG >> stack. >> Example: >> >> f :: forall rep (r :: TYPE rep). Int# -> (Int# -> r) -> r >> f = \x g -> >> let { >> h = [x] \a -> x + a; >> } in >> case h x of b { >> __DEFAULT -> g b >> } >> >> We are only interested in unsetting the bit for h here. Consider the >> stack when entering the body of h. >> >> caller_of_f_cont_info <- Sp >> >> Now push our special continuation frame: >> >> caller_of_f_cont_info >> seq_h >> unset_bit_stk_info <- Sp >> >> E.g., the stack frame contains the info pointer and the sequence number. >> (Btw., I hope I got the stack layout about right and this is even possible) >> Then, after we entered the continuation of the __DEFAULT alt, we do a >> jump to g. >> Plot twist: g returns an unboxed 8-tuple of `Int#`s (as caller_of_f_cont_info >> knows, but f certainly doesn't!), so before it returns it will push two >> args on the stack (correct?): >> >> caller_of_f_cont_info >> seq_h >> unset_bit_stk_info >> unboxed tuple component 7 >> unboxed tuple component 8 <- Sp >> >> And then `g` jumps directly to the entry code for `unset_bit_stk_info` >> (which does the register saving I mentioned), which absolutely can't figure >> out from Sp alone where seq_h is. >> Darn! I think Luite's recent work on the StgToByteCode had to work around >> similar issues, I found this wiki page >> . >> But we aren't in a position where we know the representation of `r` *at >> all*! >> >> So our idea was to scan the stack, beginning from `Sp`, until I find >> `unset_bit_stk_info`, giving us the following situation: >> >> caller_of_f_cont_info >> seq_h >> unset_bit_stk_info <- Bp >> unboxed tuple component 7 >> unboxed tuple component 8 <- Sp >> >> I suggestively named the register in which we store the result Bp in >> analogy to the traditional base pointer. This information would be enough >> to unset the bit at index seq_h >> and then copy the unboxed tuple components 7 and 8 up by two words: >> >> caller_of_f_cont_info >> unboxed tuple component 7 >> unboxed tuple component 8 <- Sp >> >> Then jump to caller_of_f_cont_info, which knows what to make of the >> stack and the register state. >> >> The stack scanning is horrible and too brittle and slow for a production >> setting, as any of the unboxed tuple components could have the same bit >> pattern as `unset_bit_stk_info`. >> We simply hope that is never the case and it's fine for the purposes of a >> quick-and-dirty instrumentation. >> >> QUESTION 1: What do you think? Could this work? Can you anticipate pit >> falls of this approach? >> >> # What about Thunks and StgRhsCon? >> >> QUESTION 2: The instrumentation as described won't work for Thunks (which >> are only entered once) and constructor applications (like sat_sBK in the >> BigNat matcher above). Not sure how to change that without incurring huge >> performance hits (by deactivating memoisation). Ideas are welcome here. >> >> Thanks for reading this far. I hope you could follow and are now fizzing >> with excitement because you have a much better idea of how to do this and >> will let me know :) >> >> Sebastian >> _______________________________________________ >> ghc-devs mailing list >> ghc-devs at haskell.org >> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sgraf1337 at gmail.com Fri Dec 17 09:24:38 2021 From: sgraf1337 at gmail.com (Sebastian Graf) Date: Fri, 17 Dec 2021 09:24:38 +0000 Subject: Transparently hooking into the STG stack to validate an escape analysis In-Reply-To: References: Message-ID: (Moving the conversation off the list.) ------ Originalnachricht ------ Von: "Csaba Hruska" An: "Sebastian Graf" Cc: "ghc-devs" ; "Sebastian Scheper" Gesendet: 16.12.2021 18:22:30 Betreff: Re: Re[2]: Transparently hooking into the STG stack to validate an escape analysis >Thanks for the feedback! >Let's have a video meeting. My schedule is flexible. What time is best >for you? > >Cheers, >Csaba > >On Thu, Dec 16, 2021 at 5:29 PM Sebastian Graf >wrote: >>Hey Csaba, >> >>After catching up on your talk and reflecting about it a bit, it seems >>quite obvious that your tool is the right way to collect the data and >>validate our analysis! >>Even if meanwhile we decided that a "transparent stack frame" (which I >>believe is something similar to what you are doing here >>, >>with an explicit `argCount` which we do not know) is not the ideal >>solution we've been looking for (for different reasons). >> >>Essentially, we have two things >>An escape analysis, implemented as an STG-to-STG pass that attaches a >>boolean flag to each Id whether it "escapes its scope" (for a suitable >>definition of that). >>We'd like to implement it in a way that it would be reusable within >>GHC with moderate effort (e.g., renaming Binder to Id or accounting >>for different fields), operating on a module at a time rather than >>whole program.The instrumentation that tries to measure how many heap >>objects could be allocated on the stack. E.g., set a closure-specific >>flag whenever the closure is entered, unset that bit (once) when we >>"leave the scope" that defines the closure. >>If my understanding is right, we could just implement this >>"instrumentation" as a simple extra field to Closure >>, >>right? Neat! >>A bit tangential: I see that your interpreter currently allocates a >>fresh closure for let-no-escapes >> >>when it could just re-use the closure of its defining scope. That >>would skew our numbers somewhat compared to instrumenting GHC-compiled >>programs, but I think we'd be able to work around that. I also wonder >>if the semantics of your let-no-escapes are actually as typically >>specified (operationally), in that a jump to a let-no-escape should >>also reset the stack pointer. It should hardly matter for the programs >>that GHC generates, though. >> >>I would also be interested in knowing whether the +RTS -s "bytes >>allocated in the heap" metric (very nearly) coincides with a similar >>metric you could produce. It would be fantastic if that was the case! >>Theoretically, that should be possible, right? >> >>I think your interpreter work is very valuable to collect data we >>otherwise would only be able to measure with a TickyTicky-based >>approach. Nice! >>Another, similar use case would be to identify the fraction of >>closures that are only entered once. I remember that there was a >>ticky-based patch with which Joachim used to measure this fraction >> >>(and similarly validate the analysis results), but unfortunately it >>couldn't end up in master. Ah, yes, we have a whole open ticket about >>it: #10613 . In >>fact, that instrumentation is also somewhat similar (adding a field to >>every closure) as what we want to do. >> >>Anyway, it seems like your work will be very valuable in replacing >>some of the annoying ticky-based instrumentation ideas! >> >>Maybe we can have a call some time this or next week to discuss >>details, once Sebastian and I are more familiar with the code base? >> >>Thanks for sticking with the project and doing all the hard work that >>can build upon! >>Sebastian >> >>------ Originalnachricht ------ >>Von: "Csaba Hruska" >>An: "Sebastian Graf" >>Cc: "ghc-devs" ; "Sebastian Scheper" >> >>Gesendet: 15.12.2021 16:16:27 >>Betreff: Re: Transparently hooking into the STG stack to validate an >>escape analysis >> >>>Hi, >>> >>>IMO the Cmm STG machine implementation is just too complex for >>>student projects. It's not fun to work with at all. >>>Why did you choose this approach? >>>IMO the escape analysis development and validation would be much >>>smoother and fun when you'd use the external STG interpreter. >>>When you have a solid and working design of your analysis and >>>transformations then you could implement it in GHC's native backend >>>if it needs any changes at all. >>> >>>What do you think? >>>Do you disagree? >>> >>>Have you seen my presentation about the stg interpreter? >>>https://www.youtube.com/watch?v=Ey5OFPkxF_w >>> >>>Cheers, >>>Csaba >>> >>>On Wed, Dec 8, 2021 at 11:20 AM Sebastian Graf >>>wrote: >>>>Hi Devs, >>>> >>>>my master's student Sebastian and I (also Sebastian :)) are working >>>>on an escape analysis in STG, see >>>>https://gitlab.haskell.org/ghc/ghc/-/issues/16891#note_347903. >>>> >>>>We have a prototype for the escape analysis that we want to >>>>validate/exploit now. >>>>The original plan was to write the transformation that allocates >>>>non-escaping objects on the STG stack. But that is quite tricky for >>>>many reasons, one of them being treatment of the GC. >>>> >>>>This mail is rather lengthy, so I suggest you skip to "What we hope >>>>could work" and see if you can answer it without the context I >>>>provide below. If you can't, I would be very grateful if you were >>>>willing to suffer through the exposition. >>>> >>>># Instrumentation >>>> >>>>So instead we thought about doing a (easily changed and thus >>>>versatile) instrumentation-based approach: >>>>Assign a sequence number to every instantiation (a term I will use >>>>to mean "allocation of g's closure") of things that we our analysis >>>>determines as escaping or non-escaping, such as STG's let bindings >>>>(focusing on non-let-no-escape functions for now). >>>>(One sequence number *per allocation* of the let binding's closure, >>>>not based on its syntactic identity.) >>>>Then, we set a bit in a (dynamically growing) global bit vector >>>>whenever the let "RHS is entered" and then unset it when we "leave >>>>the let-body". Example: >>>> >>>>f = \x y -> >>>> let { >>>> g = [y] \z -> y + z; >>>> } in g x >>>> >>>>Here, our analysis could see that no instantiation (which I say >>>>instead of "allocation of g's closure") of g will ever escape its >>>>scope within f. >>>>Our validation would give a fresh sequence number to the >>>>instantiation of g whenever f is called and store it in g's closure >>>>(which we arrange by piggy-backing on -prof and adding an additional >>>>field to the profiling header). >>>>Then, when g's RHS is entered, we set the bit in the global bit >>>>vector, indicating "this instantiation of g might escape". >>>>After leaving the RHS of g, we also leave the body of the defining >>>>let, which means we unset the bit in the bit vector, meaning "every >>>>use so far wasn't in an escaping scenario". >>>> >>>>So far so good. Modifying the code upon entering g takes a bit of >>>>tinkering but can be done by building on TickyTicky in StgToCmm. >>>>But what is not done so easily is inserting the continuation upon >>>>entering the let that will unset the bit! >>>> >>>># What doesn't work: Modifying the Sequel >>>> >>>>At first, we tried to modify the sequel >>>> >>>>of the let-body to an `AssignTo`. >>>>That requires us to know the registers in which the let-body will >>>>return its results, which in turn means we have to know the >>>>representation of those results, so we have to write a function >>>>`stgExprPrimRep :: GenStgExpr p -> [PrimRep]`. >>>>Urgh! We were very surprised that there was no such function. And >>>>while we tested our function, we very soon knew why. Consider the >>>>following pattern synonym matcher: >>>> >>>>GHC.Natural.$mNatJ# >>>> :: forall {rep :: GHC.Types.RuntimeRep} {r :: TYPE rep}. >>>> GHC.Num.Natural.Natural >>>> -> (GHC.Num.BigNat.BigNat -> r) -> ((# #) -> r) -> r >>>> = {} \r [scrut_sBE cont_sBF fail_sBG] >>>> case scrut_sBE of { >>>> GHC.Num.Natural.NS _ -> fail_sBG GHC.Prim.(##); >>>> GHC.Num.Natural.NB ds_sBJ -> >>>> let { >>>> sat_sBK :: GHC.Num.BigNat.BigNat >>>> = CCCS GHC.Num.BigNat.BN#! [ds_sBJ]; >>>> } in cont_sBF sat_sBK; >>>> }; >>>> >>>>Note how its result is representation-polymorphic! It only works >>>>because our machine implementation allows tail-calls. >>>>It's obvious in hindsight that we could never write `stgExprPrimRep` >>>>in such a way that it will work on the expression `cont_sBF >>>>sat_sBK`. >>>>So the sequel approach seems infeasible. >>>> >>>># What we hope could work: A special stack frame >>>> >>>>The other alternative would be to insert a special continuation >>>>frame on the stack when we enter the let-body (inspired by >>>>stg_restore_cccs). >>>>This continuation frame would simply push all registers (FP regs, GP >>>>regs, Vec regs, ...) to the C stack, do its work (unsetting the >>>>bit), then pop all registers again and jump to the topmost >>>>continuation on the STG stack. >>>>Example: >>>> >>>>f :: forall rep (r :: TYPE rep). Int# -> (Int# -> r) -> r >>>>f = \x g -> >>>> let { >>>> h = [x] \a -> x + a; >>>> } in >>>> case h x of b { >>>> __DEFAULT -> g b >>>> } >>>> >>>>We are only interested in unsetting the bit for h here. Consider the >>>>stack when entering the body of h. >>>> >>>>caller_of_f_cont_info <- Sp >>>> >>>>Now push our special continuation frame: >>>> >>>>caller_of_f_cont_info >>>>seq_h >>>>unset_bit_stk_info <- Sp >>>> >>>>E.g., the stack frame contains the info pointer and the sequence >>>>number. (Btw., I hope I got the stack layout about right and this is >>>>even possible) >>>>Then, after we entered the continuation of the __DEFAULT alt, we do >>>>a jump to g. >>>>Plot twist: g returns an unboxed 8-tuple of `Int#`s (as >>>>caller_of_f_cont_info knows, but f certainly doesn't!), so before it >>>>returns it will push two args on the stack (correct?): >>>> >>>>caller_of_f_cont_info >>>>seq_h >>>>unset_bit_stk_info >>>>unboxed tuple component 7 >>>>unboxed tuple component 8 <- Sp >>>> >>>>And then `g` jumps directly to the entry code for >>>>`unset_bit_stk_info` (which does the register saving I mentioned), >>>>which absolutely can't figure out from Sp alone where seq_h is. >>>>Darn! I think Luite's recent work on the StgToByteCode had to work >>>>around similar issues, I found this wiki page >>>>. >>>>But we aren't in a position where we know the representation of `r` >>>>*at all*! >>>> >>>>So our idea was to scan the stack, beginning from `Sp`, until I find >>>>`unset_bit_stk_info`, giving us the following situation: >>>> >>>>caller_of_f_cont_info >>>>seq_h >>>>unset_bit_stk_info <- Bp >>>>unboxed tuple component 7 >>>>unboxed tuple component 8 <- Sp >>>> >>>>I suggestively named the register in which we store the result Bp in >>>>analogy to the traditional base pointer. This information would be >>>>enough to unset the bit at index seq_h >>>>and then copy the unboxed tuple components 7 and 8 up by two words: >>>> >>>>caller_of_f_cont_info >>>>unboxed tuple component 7 >>>>unboxed tuple component 8 <- Sp >>>> >>>>Then jump to caller_of_f_cont_info, which knows what to make of the >>>>stack and the register state. >>>> >>>>The stack scanning is horrible and too brittle and slow for a >>>>production setting, as any of the unboxed tuple components could >>>>have the same bit pattern as `unset_bit_stk_info`. >>>>We simply hope that is never the case and it's fine for the purposes >>>>of a quick-and-dirty instrumentation. >>>> >>>>QUESTION 1: What do you think? Could this work? Can you anticipate >>>>pit falls of this approach? >>>> >>>># What about Thunks and StgRhsCon? >>>> >>>>QUESTION 2: The instrumentation as described won't work for Thunks >>>>(which are only entered once) and constructor applications (like >>>>sat_sBK in the BigNat matcher above). Not sure how to change that >>>>without incurring huge performance hits (by deactivating >>>>memoisation). Ideas are welcome here. >>>> >>>>Thanks for reading this far. I hope you could follow and are now >>>>fizzing with excitement because you have a much better idea of how >>>>to do this and will let me know :) >>>> >>>>Sebastian >>>>_______________________________________________ >>>>ghc-devs mailing list >>>>ghc-devs at haskell.org >>>>http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs -------------- next part -------------- An HTML attachment was scrubbed... URL: From hecate at glitchbra.in Fri Dec 17 16:47:36 2021 From: hecate at glitchbra.in (=?UTF-8?Q?H=c3=a9cate?=) Date: Fri, 17 Dec 2021 17:47:36 +0100 Subject: [PSA] Patches to Haddock regarding GHC development move to GitLab! Message-ID: <74faca7a-aa77-0af2-cdf7-b6a185130df8@glitchbra.in> Hi everyone, just a PSA from the Haddock team and GHC release managers to warn everyone that, effective immediately, patches to Haddock needed for GHC merge requests are not to be directed to GitHub anymore. Those patches are to be sent to the GitLab mirror: https://gitlab.haskell.org/ghc/haddock. I will try and warn folks who currently have PRs opened on GitHub targeting ghc-head, but please know that they'll be closed in the next few days. Have a great end of the week, Hécate, for the Haddock Team. -- Hécate ✨ 🐦: @TechnoEmpress IRC: Hecate WWW: https://glitchbra.in RUN: BSD From lists at richarde.dev Fri Dec 17 20:40:06 2021 From: lists at richarde.dev (Richard Eisenberg) Date: Fri, 17 Dec 2021 20:40:06 +0000 Subject: [PSA] Patches to Haddock regarding GHC development move to GitLab! In-Reply-To: <74faca7a-aa77-0af2-cdf7-b6a185130df8@glitchbra.in> References: <74faca7a-aa77-0af2-cdf7-b6a185130df8@glitchbra.in> Message-ID: <010f017dca1f91f3-da2c1318-9467-4add-9005-f436e739a5bb-000000@us-east-2.amazonses.com> Hi Hécate, Thanks for this announcement. Can you also update the documentation in https://gitlab.haskell.org/ghc/ghc/-/wikis/repositories to reflect this change? That wiki page remains my go-to source for this kind of information. That page also says that https://gitlab.haskell.org/ghc/ghc/-/blob/master/packages is the authoritative place to look for submodule information. Does that need updating, too? Thanks! :) Richard > On Dec 17, 2021, at 11:47 AM, Hécate wrote: > > Hi everyone, just a PSA from the Haddock team and GHC release managers to warn everyone that, effective immediately, patches to Haddock needed for GHC merge requests are not to be directed to GitHub anymore. > > Those patches are to be sent to the GitLab mirror: https://gitlab.haskell.org/ghc/haddock. > > I will try and warn folks who currently have PRs opened on GitHub targeting ghc-head, but please know that they'll be closed in the next few days. > > Have a great end of the week, > Hécate, for the Haddock Team. > > -- > Hécate ✨ > 🐦: @TechnoEmpress > IRC: Hecate > WWW: https://glitchbra.in > RUN: BSD > > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs From ben at well-typed.com Tue Dec 21 17:23:51 2021 From: ben at well-typed.com (Ben Gamari) Date: Tue, 21 Dec 2021 12:23:51 -0500 Subject: Merge request merge timing Message-ID: <87r1a57tyn.fsf@smart-cactus.org> Hello all, Recently there have been a few instances where merge requests were merged extremely quickly (e.g. within an hour of the MR being opened). While prompt review is a good thing, we also want to make sure that all potentially interested reviewers have an opportunity to comment. In this vein, I propose that we wait at least 24 hours from the time an MR is opened before sending it to Marge. I have documented this to the merge-request review guidelines on the Wiki [1]. Cheers, - Ben [1] https://gitlab.haskell.org/ghc/ghc/-/wikis/gitlab/merge-requests -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From mail at joachim-breitner.de Wed Dec 22 12:19:21 2021 From: mail at joachim-breitner.de (Joachim Breitner) Date: Wed, 22 Dec 2021 13:19:21 +0100 Subject: CI: Choice of base commit for perf comparisons Message-ID: <0e495872fde9c2e6a6c6af2a7fdd355393515933.camel@joachim-breitner.de> Hi, the new (or “new”?) handling of perf numbers, where CI just magically records and compares them, without us having to manually edit the `all.T` files, is a big improvement, thanks! However, I found the choice of the base commit to compare against unhelpful. Assume master is at commit M, and I start a feature branch and MR with commit A. CI runs, and tells me about a performance regressions, and CI is red. I now fix the issue and push commit B to the branch. CI runs, but it picks A to compare against, and now it is red because of an seemingly unexpected performance improvement! I would have expected that all CI runs for this MR to compare the performance against the base branch on master, and to look for perf change notices in all commit messages in between. I see these advantages: * The reported perf changes correspond to the changes shown on the MR  page * Green CI = the MR is ready (after squashing) * CI will have numbers for the base commit more reliably (else, if I push commit C quickly after B, then the job for B might be cancelled and Ci will report changes of C against A instead of B, which is unexpected). I have used this logic of reporting perf changes (or any other “differential CI”) against the base branch in the Motoko project and it was quite natural. Would it be desirable and possible for us here, too? (A possible rebuttal might be: we don’t push new commits to feature branches, but always squash and rebase, as that’s what we have to do before merging anyways. If that’s the case then ok, although I generally lean to having chronological commits on feature branches and a nice squashed commit on master.) Cheers, Joachim -- Joachim Breitner mail at joachim-breitner.de http://www.joachim-breitner.de/ From lists at richarde.dev Wed Dec 22 13:13:12 2021 From: lists at richarde.dev (Richard Eisenberg) Date: Wed, 22 Dec 2021 13:13:12 +0000 Subject: CI: Choice of base commit for perf comparisons In-Reply-To: <0e495872fde9c2e6a6c6af2a7fdd355393515933.camel@joachim-breitner.de> References: <0e495872fde9c2e6a6c6af2a7fdd355393515933.camel@joachim-breitner.de> Message-ID: <010f017de246367e-50b65a0f-c774-4a94-bd13-564bb4a17605-000000@us-east-2.amazonses.com> It seems to be that this thought is in the air right now. This was done just a few days ago: https://gitlab.haskell.org/ghc/ghc/-/merge_requests/7184 https://gitlab.haskell.org/ghc/ghc/-/merge_requests/7231 also looks relevant. Richard > On Dec 22, 2021, at 7:19 AM, Joachim Breitner wrote: > > Hi, > > the new (or “new”?) handling of perf numbers, where CI just magically > records and compares them, without us having to manually edit the > `all.T` files, is a big improvement, thanks! > > However, I found the choice of the base commit to compare against > unhelpful. Assume master is at commit M, and I start a feature branch > and MR with commit A. CI runs, and tells me about a performance > regressions, and CI is red. I now fix the issue and push commit B to > the branch. CI runs, but it picks A to compare against, and now it is > red because of an seemingly unexpected performance improvement! > > I would have expected that all CI runs for this MR to compare the > performance against the base branch on master, and to look for perf > change notices in all commit messages in between. > > I see these advantages: > > * The reported perf changes correspond to the changes shown on the MR > page > * Green CI = the MR is ready (after squashing) > * CI will have numbers for the base commit more reliably > (else, if I push commit C quickly after B, then the job for B might > be cancelled and Ci will report changes of C against A instead of B, > which is unexpected). > > I have used this logic of reporting perf changes (or any other > “differential CI”) against the base branch in the Motoko project and it > was quite natural. > > Would it be desirable and possible for us here, too? > > > (A possible rebuttal might be: we don’t push new commits to feature > branches, but always squash and rebase, as that’s what we have to do > before merging anyways. If that’s the case then ok, although I > generally lean to having chronological commits on feature branches and > a nice squashed commit on master.) > > Cheers, > Joachim > > > -- > Joachim Breitner > mail at joachim-breitner.de > http://www.joachim-breitner.de/ > > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs From mail at joachim-breitner.de Wed Dec 22 14:07:17 2021 From: mail at joachim-breitner.de (Joachim Breitner) Date: Wed, 22 Dec 2021 14:07:17 +0000 (UTC) Subject: CI: Choice of base commit for perf comparisons In-Reply-To: <010f017de246367e-50b65a0f-c774-4a94-bd13-564bb4a17605-000000@us-east-2.amazonses.com> References: <0e495872fde9c2e6a6c6af2a7fdd355393515933.camel@joachim-breitner.de> <010f017de246367e-50b65a0f-c774-4a94-bd13-564bb4a17605-000000@us-east-2.amazonses.com> Message-ID: Thanks! I like it when my feature suggestions are implemented even before I voice them ;-) 22.12.2021 14:13:24 Richard Eisenberg : > It seems to be that this thought is in the air right now. This was done just a few days ago: https://gitlab.haskell.org/ghc/ghc/-/merge_requests/7184 > > https://gitlab.haskell.org/ghc/ghc/-/merge_requests/7231 also looks relevant. > > Richard > >> On Dec 22, 2021, at 7:19 AM, Joachim Breitner wrote: >> >> Hi, >> >> the new (or “new”?) handling of perf numbers, where CI just magically >> records and compares them, without us having to manually edit the >> `all.T` files, is a big improvement, thanks! >> >> However, I found the choice of the base commit to compare against >> unhelpful. Assume master is at commit M, and I start a feature branch >> and MR with commit A. CI runs, and tells me about a performance >> regressions, and CI is red. I now fix the issue and push commit B to >> the branch. CI runs, but it picks A to compare against, and now it is >> red because of an seemingly unexpected performance improvement! >> >> I would have expected that all CI runs for this MR to compare the >> performance against the base branch on master, and to look for perf >> change notices in all commit messages in between. >> >> I see these advantages: >> >> * The reported perf changes correspond to the changes shown on the MR >>   page >> * Green CI = the MR is ready (after squashing) >> * CI will have numbers for the base commit more reliably >>   (else, if I push commit C quickly after B, then the job for B might >>   be cancelled and Ci will report changes of C against A instead of B, >>   which is unexpected). >> >> I have used this logic of reporting perf changes (or any other >> “differential CI”) against the base branch in the Motoko project and it >> was quite natural. >> >> Would it be desirable and possible for us here, too? >> >> >> (A possible rebuttal might be: we don’t push new commits to feature >> branches, but always squash and rebase, as that’s what we have to do >> before merging anyways. If that’s the case then ok, although I >> generally lean to having chronological commits on feature branches and >> a nice squashed commit on master.) >> >> Cheers, >> Joachim >> >> >> -- >> Joachim Breitner >> mail at joachim-breitner.de >> http://www.joachim-breitner.de/ >> >> _______________________________________________ >> ghc-devs mailing list >> ghc-devs at haskell.org >> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs From shayne.fletcher.50 at gmail.com Fri Dec 24 17:13:46 2021 From: shayne.fletcher.50 at gmail.com (Shayne Fletcher) Date: Fri, 24 Dec 2021 12:13:46 -0500 Subject: 9.2.2 release schedule? Message-ID: Anyone know of any scheduling plans for ghc-9.2.2? Happy holidays y'all :) -- Shayne Fletcher -------------- next part -------------- An HTML attachment was scrubbed... URL: From ben at well-typed.com Fri Dec 24 18:31:25 2021 From: ben at well-typed.com (Ben Gamari) Date: Fri, 24 Dec 2021 13:31:25 -0500 Subject: 9.2.2 release schedule? In-Reply-To: References: Message-ID: <87lf097t3u.fsf@smart-cactus.org> Shayne Fletcher writes: > Anyone know of any scheduling plans for ghc-9.2.2? > It should be out by mid-January. There are quite a few backports that we need to push through and the focus recently has been on finalizing 9.0.2 but I'll be turning my attention to 9.2.2 after I the holiday. Happy Holidays, Shayne! Cheers, - Ben [1] https://gitlab.haskell.org/ghc/ghc/-/merge_requests?scope=all&state=all&label_name[]=backport%20needed%3A9.2 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From zubin at well-typed.com Sat Dec 25 20:20:27 2021 From: zubin at well-typed.com (Zubin Duggal) Date: Sun, 26 Dec 2021 01:50:27 +0530 Subject: [Haskell] [ANNOUNCE] GHC 9.0.2 released Message-ID: <20211225202027.jxlafyybul4bltfb@zubin-msi> The GHC developers are very happy to at long last announce the availability of GHC 9.0.2. Binary distributions, source distributions, and documentation are available at the [usual place](https://downloads.haskell.org/ghc/9.0.2/). Download Page: https://www.haskell.org/ghc/download_ghc_9_0_2.html Blog Post: https://www.haskell.org/ghc/blog/20211225-ghc-9.0.2-released.html GHC 9.0.2 adds first class AArch64/Darwin support using the LLVM backend, as well as fixing a number of critical correctness bugs with the 9.0.1 release, along with numerous improvements to compiler performance and memory usage: * First-class support for Apple M1 hardware using GHC's LLVM ARM backend. * Fix a number of incorrect rewrite rules for `fromInteger` (#19345, #20066). * Fix a number of runtime system and GC bugs which typically manifested in crashes (#19417, #19297, #19645, #19667, #18033, #20132, #19715, #20399, #20414, #20649, #19147, #20051, #20093) * Fixes for various compiler performance and memory usage issues (#20134, #13586, #20509, #19668, #19996, #19471) * Improvements and bug fixes for the code generator and optimiser (#18638, #19581, #19700, #19996, #19181, #19822) * Fixes for typechecker bugs (#19410 , #19191, #19522, #19400, #19849) * Bug fixes for the frontend and parser (#19397, #19838, #19518, #19777, #20242, #19786) * Emit warnings if Unicode Bidirectional Formatting characters are found in the source, possibly misleading code reviewers (#20263). * Many packaging related fixes, including versioned `ghc-pkg` executables (#20087), and actually distributing GHC versions linked against the `native` big integer backend (#18967, #19953) on both Windows and Alpine Linux. Some previous releases were still linked against the `GMP` library due to a misconfiguration of the builders. * A significant refactoring of `process` fixing numerous bugs mostly on Apple platforms (#19994, [process refactoring](https://github.com/haskell/process/pull/208)). * Bug fixes for the linker on Darwin platforms (#20004, #19968, #19950). A complete list of bug fixes and improvements can be found in the [release notes](https://downloads.haskell.org/ghc/9.0.2/docs/html/users_guide/9.0.2-notes.html). Finally, thank you to Microsoft Research, GitHub, IOHK, the Zw3rk stake pool, Tweag I/O, Serokell, Equinix, SimSpace, and other anonymous contributors whose on-going financial and in-kind support has facilitated GHC maintenance and release management over the years. Moreover, this release would not have been possible without the hundreds of open-source contributors whose work comprise this release. As always, do open a [ticket][] if you see anything amiss. Happy Holidays and Happy Hacking! - Zubin -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 488 bytes Desc: not available URL: From shayne.fletcher.50 at gmail.com Sat Dec 25 20:37:38 2021 From: shayne.fletcher.50 at gmail.com (Shayne Fletcher) Date: Sat, 25 Dec 2021 15:37:38 -0500 Subject: [Haskell] [ANNOUNCE] GHC 9.0.2 released In-Reply-To: <20211225202027.jxlafyybul4bltfb@zubin-msi> References: <20211225202027.jxlafyybul4bltfb@zubin-msi> Message-ID: Awesome. Thank-you! There appears not to be a ghc-9.0.2-release tag. Might this be addressed please? On Sat, Dec 25, 2021 at 3:22 PM Zubin Duggal wrote: > The GHC developers are very happy to at long last announce the > availability of GHC 9.0.2. Binary distributions, source distributions, > and documentation are available at the > [usual place](https://downloads.haskell.org/ghc/9.0.2/). > > Download Page: https://www.haskell.org/ghc/download_ghc_9_0_2.html > Blog Post: > https://www.haskell.org/ghc/blog/20211225-ghc-9.0.2-released.html > > GHC 9.0.2 adds first class AArch64/Darwin support using the LLVM > backend, as well as fixing a number of critical correctness bugs > with the 9.0.1 release, along with numerous improvements to compiler > performance and memory usage: > > * First-class support for Apple M1 hardware using GHC's LLVM ARM > backend. > > * Fix a number of incorrect rewrite rules for `fromInteger` (#19345, > #20066). > > * Fix a number of runtime system and GC bugs which typically manifested > in crashes (#19417, #19297, #19645, #19667, #18033, #20132, #19715, > #20399, #20414, #20649, #19147, #20051, #20093) > > * Fixes for various compiler performance and memory usage issues > (#20134, #13586, #20509, #19668, #19996, #19471) > > * Improvements and bug fixes for the code generator and optimiser > (#18638, #19581, #19700, #19996, #19181, #19822) > > * Fixes for typechecker bugs (#19410 , #19191, #19522, #19400, #19849) > > * Bug fixes for the frontend and parser (#19397, #19838, #19518, #19777, > #20242, #19786) > > * Emit warnings if Unicode Bidirectional Formatting characters are found > in the source, possibly misleading code reviewers (#20263). > > * Many packaging related fixes, including versioned `ghc-pkg` > executables (#20087), and actually distributing GHC versions linked > against the `native` big integer backend (#18967, #19953) on both > Windows and Alpine Linux. Some previous releases were still linked > against the `GMP` library due to a misconfiguration of the builders. > > * A significant refactoring of `process` fixing numerous bugs mostly on > Apple platforms (#19994, [process refactoring]( > https://github.com/haskell/process/pull/208)). > > * Bug fixes for the linker on Darwin platforms (#20004, #19968, #19950). > > A complete list of bug fixes and improvements can be found in the [release > notes]( > https://downloads.haskell.org/ghc/9.0.2/docs/html/users_guide/9.0.2-notes.html > ). > > Finally, thank you to Microsoft Research, GitHub, IOHK, the Zw3rk > stake pool, Tweag I/O, Serokell, Equinix, SimSpace, and other > anonymous contributors whose on-going financial and in-kind support > has facilitated GHC maintenance and release management over the years. > Moreover, this release would not have been possible without the hundreds > of open-source contributors whose work comprise this release. > > As always, do open a [ticket][] if you see anything amiss. > > Happy Holidays and Happy Hacking! > > - Zubin > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > -- Shayne Fletcher -------------- next part -------------- An HTML attachment was scrubbed... URL: From zubin at well-typed.com Sat Dec 25 20:42:17 2021 From: zubin at well-typed.com (Zubin Duggal) Date: Sun, 26 Dec 2021 02:12:17 +0530 Subject: [Haskell] [ANNOUNCE] GHC 9.0.2 released In-Reply-To: References: <20211225202027.jxlafyybul4bltfb@zubin-msi> Message-ID: <20211225204217.z76iampqlg7n3jzp@zubin-msi> Oops. Pushed! On 21/12/25 15:37, Shayne Fletcher wrote: >Awesome. Thank-you! There appears not to be a ghc-9.0.2-release tag. Might >this be addressed please? > From Gergo.Erdi at sc.com Tue Dec 28 09:12:04 2021 From: Gergo.Erdi at sc.com (Erdi, Gergo) Date: Tue, 28 Dec 2021 09:12:04 +0000 Subject: Source locations from Core Message-ID: PUBLIC Hi, I'm looking for ways to map Core fragments back to source locations. I see there is an annotated version of Core in `GHC/Core.hs` called `AnnExpr`, which I could see being useful for this if I set the annotation type to `SrcSpan`, but that's not what I get out of GHC's desugarer, simplifier or tidier. If there's no built-in mechanism for this, my only idea would be to create a HsExpr-to-HsExpr transformation that wraps every node in a call that is opaque enough to be persisted through Core-to-Core transformations but still transparent enough that it doesn't block optimization opportunities. Is that even possible? Alternatively, would it make it easer if I was content with only getting source locations for variable occurrences? Thanks, Gergo This email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please delete all copies and notify the sender immediately. You may wish to refer to the incorporation details of Standard Chartered PLC, Standard Chartered Bank and their subsidiaries at https: //www.sc.com/en/our-locations Where you have a Financial Markets relationship with Standard Chartered PLC, Standard Chartered Bank and their subsidiaries (the "Group"), information on the regulatory standards we adhere to and how it may affect you can be found in our Regulatory Compliance Statement at https: //www.sc.com/rcs/ and Regulatory Compliance Disclosures at http: //www.sc.com/rcs/fm Insofar as this communication is not sent by the Global Research team and contains any market commentary, the market commentary has been prepared by the sales and/or trading desk of Standard Chartered Bank or its affiliate. It is not and does not constitute research material, independent research, recommendation or financial advice. Any market commentary is for information purpose only and shall not be relied on for any other purpose and is subject to the relevant disclaimers available at https: //www.sc.com/en/regulatory-disclosures/#market-disclaimer. Insofar as this communication is sent by the Global Research team and contains any research materials prepared by members of the team, the research material is for information purpose only and shall not be relied on for any other purpose, and is subject to the relevant disclaimers available at https: //research.sc.com/research/api/application/static/terms-and-conditions. Insofar as this e-mail contains the term sheet for a proposed transaction, by responding affirmatively to this e-mail, you agree that you have understood the terms and conditions in the attached term sheet and evaluated the merits and risks of the transaction. We may at times also request you to sign the term sheet to acknowledge the same. Please visit https: //www.sc.com/en/regulatory-disclosures/dodd-frank/ for important information with respect to derivative products. -------------- next part -------------- An HTML attachment was scrubbed... URL: From matthewtpickering at gmail.com Tue Dec 28 09:36:13 2021 From: matthewtpickering at gmail.com (Matthew Pickering) Date: Tue, 28 Dec 2021 09:36:13 +0000 Subject: Source locations from Core In-Reply-To: References: Message-ID: Hi Gergo, Source Notes are what you are looking for. Currently the only way to enable them is to either use `-g` or `-finfo-table-map`. The result will be core which contains nodes which attempt to describe where the core expression came from, it's not perfect though! See Section 5.4 - https://etheses.whiterose.ac.uk/8321/1/thesis.pdf Matt On Tue, Dec 28, 2021 at 9:14 AM Erdi, Gergo via ghc-devs wrote: > > PUBLIC > > > Hi, > > > > I’m looking for ways to map Core fragments back to source locations. > > > > I see there is an annotated version of Core in `GHC/Core.hs` called `AnnExpr`, which I could see being useful for this if I set the annotation type to `SrcSpan`, but that’s not what I get out of GHC’s desugarer, simplifier or tidier. > > > > If there’s no built-in mechanism for this, my only idea would be to create a HsExpr-to-HsExpr transformation that wraps every node in a call that is opaque enough to be persisted through Core-to-Core transformations but still transparent enough that it doesn’t block optimization opportunities. Is that even possible? > > > > Alternatively, would it make it easer if I was content with only getting source locations for variable occurrences? > > > > Thanks, > > Gergo > > > This email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please delete all copies and notify the sender immediately. You may wish to refer to the incorporation details of Standard Chartered PLC, Standard Chartered Bank and their subsidiaries at https: //www.sc.com/en/our-locations > > Where you have a Financial Markets relationship with Standard Chartered PLC, Standard Chartered Bank and their subsidiaries (the "Group"), information on the regulatory standards we adhere to and how it may affect you can be found in our Regulatory Compliance Statement at https: //www.sc.com/rcs/ and Regulatory Compliance Disclosures at http: //www.sc.com/rcs/fm > > Insofar as this communication is not sent by the Global Research team and contains any market commentary, the market commentary has been prepared by the sales and/or trading desk of Standard Chartered Bank or its affiliate. It is not and does not constitute research material, independent research, recommendation or financial advice. Any market commentary is for information purpose only and shall not be relied on for any other purpose and is subject to the relevant disclaimers available at https: //www.sc.com/en/regulatory-disclosures/#market-disclaimer. > > Insofar as this communication is sent by the Global Research team and contains any research materials prepared by members of the team, the research material is for information purpose only and shall not be relied on for any other purpose, and is subject to the relevant disclaimers available at https: //research.sc.com/research/api/application/static/terms-and-conditions. > > Insofar as this e-mail contains the term sheet for a proposed transaction, by responding affirmatively to this e-mail, you agree that you have understood the terms and conditions in the attached term sheet and evaluated the merits and risks of the transaction. We may at times also request you to sign the term sheet to acknowledge the same. > > Please visit https: //www.sc.com/en/regulatory-disclosures/dodd-frank/ for important information with respect to derivative products. > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs From Gergo.Erdi at sc.com Tue Dec 28 10:00:05 2021 From: Gergo.Erdi at sc.com (Erdi, Gergo) Date: Tue, 28 Dec 2021 10:00:05 +0000 Subject: [External] Re: Source locations from Core In-Reply-To: References: Message-ID: PUBLIC Thank you, this looks exactly like what I'm looking for. Now I'll just have to try it on larger examples to see how approximate it is 😊 -----Original Message----- From: Matthew Pickering Sent: Tuesday, December 28, 2021 10:36 AM To: Erdi, Gergo Cc: GHC Subject: [External] Re: Source locations from Core ATTENTION: This email came from an external source. Do not open attachments or click on links from unknown senders or unexpected emails. Always report suspicious emails using the Report As Phishing button in Outlook to protect the Bank and our clients. Hi Gergo, Source Notes are what you are looking for. Currently the only way to enable them is to either use `-g` or `-finfo-table-map`. The result will be core which contains nodes which attempt to describe where the core expression came from, it's not perfect though! See Section 5.4 - https://clicktime.symantec.com/3QgzZCkQPV23ts2BtD4WH4Z7VN?u=https%3A%2F%2Fetheses.whiterose.ac.uk%2F8321%2F1%2Fthesis.pdf Matt This email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please delete all copies and notify the sender immediately. You may wish to refer to the incorporation details of Standard Chartered PLC, Standard Chartered Bank and their subsidiaries at https: //www.sc.com/en/our-locations Where you have a Financial Markets relationship with Standard Chartered PLC, Standard Chartered Bank and their subsidiaries (the "Group"), information on the regulatory standards we adhere to and how it may affect you can be found in our Regulatory Compliance Statement at https: //www.sc.com/rcs/ and Regulatory Compliance Disclosures at http: //www.sc.com/rcs/fm Insofar as this communication is not sent by the Global Research team and contains any market commentary, the market commentary has been prepared by the sales and/or trading desk of Standard Chartered Bank or its affiliate. It is not and does not constitute research material, independent research, recommendation or financial advice. Any market commentary is for information purpose only and shall not be relied on for any other purpose and is subject to the relevant disclaimers available at https: //www.sc.com/en/regulatory-disclosures/#market-disclaimer. Insofar as this communication is sent by the Global Research team and contains any research materials prepared by members of the team, the research material is for information purpose only and shall not be relied on for any other purpose, and is subject to the relevant disclaimers available at https: //research.sc.com/research/api/application/static/terms-and-conditions. Insofar as this e-mail contains the term sheet for a proposed transaction, by responding affirmatively to this e-mail, you agree that you have understood the terms and conditions in the attached term sheet and evaluated the merits and risks of the transaction. We may at times also request you to sign the term sheet to acknowledge the same. Please visit https: //www.sc.com/en/regulatory-disclosures/dodd-frank/ for important information with respect to derivative products. From harendra.kumar at gmail.com Tue Dec 28 14:57:31 2021 From: harendra.kumar at gmail.com (Harendra Kumar) Date: Tue, 28 Dec 2021 20:27:31 +0530 Subject: Measuring thread cpu time (using bound threads) Message-ID: Hi GHC devs, I want to measure the CPU time spent in a particular Haskell thread across thread yield points. I am assuming Haskell threads do not keep track of per thread cpu time. Therefore, to measure it I have to use the thread cpu clock (CLOCK_THREAD_CPUTIME_ID) provided by the OS (Linux). But Haskell threads can keep jumping across multiple OS threads so the OS thread's CPU time cannot be used reliably. To solve that issue I thought I could use bound threads. From the documentation it sounded like bound threads are exclusively bound to a particular OS thread so we can get thread cpu time from the OS and use it. However, if I print the thread id for a bound thread using "gettid" (OS thread id on Linux) or using "pthread_self" (pthread id), both these thread ids are changing for the same Haskell thread which is a bound thread. Is that expected, or am I doing something wrong? My questions: * Are bound threads really bound to a particular OS thread or they can use multiple OS threads for executing Haskell code, and only use a fixed OS thread for FFI code? * Is there a way to make this work? Can we obtain the thread cpu time reliably across thread yield points? Is there a way to really bind Haskell threads to OS threads? * Is there a better or alternative way to get the thread CPU time? Thanks, Harendra -------------- next part -------------- An HTML attachment was scrubbed... URL: From Gergo.Erdi at sc.com Tue Dec 28 15:57:10 2021 From: Gergo.Erdi at sc.com (Erdi, Gergo) Date: Tue, 28 Dec 2021 15:57:10 +0000 Subject: What's the benefit of taking "do" blocks apart? Is there a way to turn that off? Message-ID: PUBLIC Hi, I'm seeing 'do' blocks getting taking apart into top-level definitions, so e.g. main = do some complicated expression 1 some complicated expression 2 is compiled into sat_sKv = some complicated expression 1 sat_sKw = \_ -> some complicated expression 2 main = bindIO sat_sKv sat_sKw This seems to happen regardless of any common subexpressions, i.e. it is not the case that sat_sKv or sat_sKw are used anywhere else. What is the intended benefit of this floating-out? Is there a particular Core-to-Core pass that causes this? Is it possible to turn it off? Thanks, Gergo This email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please delete all copies and notify the sender immediately. You may wish to refer to the incorporation details of Standard Chartered PLC, Standard Chartered Bank and their subsidiaries at https: //www.sc.com/en/our-locations Where you have a Financial Markets relationship with Standard Chartered PLC, Standard Chartered Bank and their subsidiaries (the "Group"), information on the regulatory standards we adhere to and how it may affect you can be found in our Regulatory Compliance Statement at https: //www.sc.com/rcs/ and Regulatory Compliance Disclosures at http: //www.sc.com/rcs/fm Insofar as this communication is not sent by the Global Research team and contains any market commentary, the market commentary has been prepared by the sales and/or trading desk of Standard Chartered Bank or its affiliate. It is not and does not constitute research material, independent research, recommendation or financial advice. Any market commentary is for information purpose only and shall not be relied on for any other purpose and is subject to the relevant disclaimers available at https: //www.sc.com/en/regulatory-disclosures/#market-disclaimer. Insofar as this communication is sent by the Global Research team and contains any research materials prepared by members of the team, the research material is for information purpose only and shall not be relied on for any other purpose, and is subject to the relevant disclaimers available at https: //research.sc.com/research/api/application/static/terms-and-conditions. Insofar as this e-mail contains the term sheet for a proposed transaction, by responding affirmatively to this e-mail, you agree that you have understood the terms and conditions in the attached term sheet and evaluated the merits and risks of the transaction. We may at times also request you to sign the term sheet to acknowledge the same. Please visit https: //www.sc.com/en/regulatory-disclosures/dodd-frank/ for important information with respect to derivative products. -------------- next part -------------- An HTML attachment was scrubbed... URL: From benjamin.redelings at gmail.com Tue Dec 28 17:10:55 2021 From: benjamin.redelings at gmail.com (Benjamin Redelings) Date: Tue, 28 Dec 2021 09:10:55 -0800 Subject: Output language of typechecking pass? In-Reply-To: References: <010f017cc31f63bb-475c1cf3-17f9-4904-9d14-45fbe88cbe51-000000@us-east-2.amazonses.com> <27aaf5dc-8a98-228d-bf41-c32784517f95@gmail.com> Message-ID: I was thinking about the relationship between the wiki and the notes in the GHC source. Would it be possible to link directly to [compiler notes] in the GHC source from the wiki, using hyperlinks?  Right now, I'm seeing references that look like: (See |Note [Constraint flavours]|.) (I can see the motivation to include comments in the source, but I also think that the wiki is more discoverable than the compiler source code.  So, in the interests of pursuing both approaches, it would be nice to be able to link to notes FROM the wiki.  I suppose one could include a hyperlink to the file on github that contains the note...) I'm not sure how much web infrastructure would be required to make hyperlinks for notes... -BenRI On 11/8/21 5:35 AM, Simon Peyton Jones wrote: > > Is there anywhere on the GHC wiki that explains how to interpret this > output, and says that the type and dictionary applications ARE there, > just not shown by '-ddump-tc'? > > Perhaps it would be helpful to add some basic description of what > comes out of the typechecker to a page like this one? (below) > > https://gitlab.haskell.org/ghc/ghc/-/wikis/commentary/compiler/hsc-main > > > Yes it would!  Would you care to start such a wiki page (a new one; > don’t just clutter up the one you point to)?    You can write down > what you know.  Don’t worry if you aren’t 100% sure – we can correct > it.  And if you outright don’t know, leave a “What should I say here?” > note. > > "This late desugaring is somewhat unusual. It is much more common to > desugar the program before typechecking, or renaming, because that > presents the renamer and typechecker with a much smaller language to > deal with. However, GHC's organisation means that > > This note is now slightly out of date.  We are now, very carefully, > doing some desugaring *before* typechecking.  See > > * Note [Handling overloaded and rebindable constructs]  in > GHC.Rename.Expr > * Note [Rebindable syntax and HsExpansion] in GHC.Hs.Expr > > You can and should point to these and similar Notes from the wiki page > you write.  Indeed there may be some part of what you write that would > be better framed as Note in GHC’s source code. > > Thanks! > > Simon > > PS: I am leaving Microsoft at the end of November 2021, at which point > simonpj at microsoft.com will cease to > work.  Use simon.peytonjones at gmail.com > instead.  (For now, it just > forwards to simonpj at microsoft.com.) > > *From:*ghc-devs *On Behalf Of *Benjamin > Redelings > *Sent:* 08 November 2021 13:12 > *To:* Richard Eisenberg > *Cc:* ghc-devs at haskell.org > *Subject:* Re: Output language of typechecking pass? > > Hi, > > Questions: > > 1. It seems like this separation is actually necessary, in order to apply generalization only to let arguments written by the programmer, and not to let bindings introduced during desugaring. Is that right? > > I don't think so. That is, if we did it all in one pass, I still think we could get generalization right. > > I guess I asked this question wrong.  I mean to say, if we did the two > passes in the reverse order (desugaring first, followed by > typechecking), that would not work, right? > > As the wiki says: > > "This late desugaring is somewhat unusual. It is much more common to > desugar the program before typechecking, or renaming, because that > presents the renamer and typechecker with a much smaller language to > deal with. However, GHC's organisation means that > > * error messages can display precisely the syntax that the user > wrote; and > * desugaring is not required to preserve type-inference properties. > > " > > 2. Does the output of type checking contain type lambdas? > > Yes. See below. > > 3. Does the type checking pass determine where to add dictionary arguments? > > Yes. See below. > > 4. Are there any other resources I should be looking at? > > Yes. You want to enable -fprint-typechecker-elaboration (and possible -fprint-explicit-coercions). With the former, you get to see all this stuff you're looking for. It's normally suppressed so that the output resembles the user's code. > > I hope this helps! > > Richard > > Hmm... so, I think I see how this works now.  I don't think > '-fprint-explicit-coercions' does anything here though. > > $ ghc -ddump-tc Test2.hs -fprint-typechecker-elaboration > > ... > > AbsBinds [a_a2hp] [$dNum_a2hB] >   {Exports: [g <= g_a2hz >                wrap: <>] >    Exported types: g :: forall a. Num a => a -> a -> a >                    [LclId] >    Binds: g x_aYk y_aYl = (y_aYl * x_aYk) + 1 >    Evidence: [EvBinds{[W] $dNum_a2hs = $dNum_a2hq >                       [W] $dNum_a2hw = $dNum_a2hq >                       [W] $dNum_a2hq = $dNum_a2hB}]} > > ... > > The type and dictionary arguments are visible here (along with the > evidence bindings), but type and dictionary applications are only > visible if you use -ddump-tc-ast, which is a lot more verbose.  (I > don't think there is another flag that shows these applications?)  > Since I didn't initially know what "evidence" was, and there is > nothing to say that a_a2hp is a type lambda argument, this was pretty > opaque until I managed to read the tc-ast and the light went on. > > I can see now that the type and dictionary arguments are added by > annotating the AST. > > Is there anywhere on the GHC wiki that explains how to interpret this > output, and says that the type and dictionary applications ARE there, > just not shown by '-ddump-tc'? > > Perhaps it would be helpful to add some basic description of what > comes out of the typechecker to a page like this one? (below) > > https://gitlab.haskell.org/ghc/ghc/-/wikis/commentary/compiler/hsc-main > > > -BenRI > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lists at richarde.dev Tue Dec 28 19:27:34 2021 From: lists at richarde.dev (Richard Eisenberg) Date: Tue, 28 Dec 2021 19:27:34 +0000 Subject: Output language of typechecking pass? In-Reply-To: References: <010f017cc31f63bb-475c1cf3-17f9-4904-9d14-45fbe88cbe51-000000@us-east-2.amazonses.com> <27aaf5dc-8a98-228d-bf41-c32784517f95@gmail.com> Message-ID: <010f017e02831fb0-6188717a-c9c2-41f2-92a4-0a49cd4d4f34-000000@us-east-2.amazonses.com> We could always make a hyperlink to the source code as hosted on GitLab. But I actually argue not to: such links would quickly become outdated, in one of two ways: either we make a permalink, in which case the linked Note text will become outdated; or we make a link to a particular file & line, in which case the Note might move somewhere else. Instead, just by naming the Note title, we have a slightly-harder-to-use link, where you use it by grepping the source code. This is less convenient, but it will stay up-to-date. Until we have better tooling to, say, create an HTML anchor based on a Note, I think this is the best we can do. Richard > On Dec 28, 2021, at 12:10 PM, Benjamin Redelings wrote: > > I was thinking about the relationship between the wiki and the notes in the GHC source. > > Would it be possible to link directly to [compiler notes] in the GHC source from the wiki, using hyperlinks? Right now, I'm seeing references that look like: (See Note [Constraint flavours].) > > (I can see the motivation to include comments in the source, but I also think that the wiki is more discoverable than the compiler source code. So, in the interests of pursuing both approaches, it would be nice to be able to link to notes FROM the wiki. I suppose one could include a hyperlink to the file on github that contains the note...) > > I'm not sure how much web infrastructure would be required to make hyperlinks for notes... > > -BenRI > > On 11/8/21 5:35 AM, Simon Peyton Jones wrote: >> Is there anywhere on the GHC wiki that explains how to interpret this output, and says that the type and dictionary applications ARE there, just not shown by '-ddump-tc'? >> >> Perhaps it would be helpful to add some basic description of what comes out of the typechecker to a page like this one? (below) >> >> https://gitlab.haskell.org/ghc/ghc/-/wikis/commentary/compiler/hsc-main >> Yes it would! Would you care to start such a wiki page (a new one; don’t just clutter up the one you point to)? You can write down what you know. Don’t worry if you aren’t 100% sure – we can correct it. And if you outright don’t know, leave a “What should I say here?” note. >> >> "This late desugaring is somewhat unusual. It is much more common to desugar the program before typechecking, or renaming, because that presents the renamer and typechecker with a much smaller language to deal with. However, GHC's organisation means that >> >> This note is now slightly out of date. We are now, very carefully, doing some desugaring before typechecking. See >> Note [Handling overloaded and rebindable constructs] in GHC.Rename.Expr >> Note [Rebindable syntax and HsExpansion] in GHC.Hs.Expr >> >> You can and should point to these and similar Notes from the wiki page you write. Indeed there may be some part of what you write that would be better framed as Note in GHC’s source code. >> >> Thanks! >> >> Simon >> >> PS: I am leaving Microsoft at the end of November 2021, at which point simonpj at microsoft.com will cease to work. Use simon.peytonjones at gmail.com instead. (For now, it just forwards to simonpj at microsoft.com .) >> >> From: ghc-devs On Behalf Of Benjamin Redelings >> Sent: 08 November 2021 13:12 >> To: Richard Eisenberg >> Cc: ghc-devs at haskell.org >> Subject: Re: Output language of typechecking pass? >> >> Hi, >> >> >> Questions: >> >> 1. It seems like this separation is actually necessary, in order to apply generalization only to let arguments written by the programmer, and not to let bindings introduced during desugaring. Is that right? >> >> I don't think so. That is, if we did it all in one pass, I still think we could get generalization right. >> I guess I asked this question wrong. I mean to say, if we did the two passes in the reverse order (desugaring first, followed by typechecking), that would not work, right? >> >> As the wiki says: >> >> "This late desugaring is somewhat unusual. It is much more common to desugar the program before typechecking, or renaming, because that presents the renamer and typechecker with a much smaller language to deal with. However, GHC's organisation means that >> >> error messages can display precisely the syntax that the user wrote; and >> desugaring is not required to preserve type-inference properties. >> " >> >> >> 2. Does the output of type checking contain type lambdas? >> >> Yes. See below. >> >> >> 3. Does the type checking pass determine where to add dictionary arguments? >> >> Yes. See below. >> >> >> 4. Are there any other resources I should be looking at? >> >> Yes. You want to enable -fprint-typechecker-elaboration (and possible -fprint-explicit-coercions). With the former, you get to see all this stuff you're looking for. It's normally suppressed so that the output resembles the user's code. >> >> I hope this helps! >> Richard >> Hmm... so, I think I see how this works now. I don't think '-fprint-explicit-coercions' does anything here though. >> >> $ ghc -ddump-tc Test2.hs -fprint-typechecker-elaboration >> >> ... >> >> AbsBinds [a_a2hp] [$dNum_a2hB] >> {Exports: [g <= g_a2hz >> wrap: <>] >> Exported types: g :: forall a. Num a => a -> a -> a >> [LclId] >> Binds: g x_aYk y_aYl = (y_aYl * x_aYk) + 1 >> Evidence: [EvBinds{[W] $dNum_a2hs = $dNum_a2hq >> [W] $dNum_a2hw = $dNum_a2hq >> [W] $dNum_a2hq = $dNum_a2hB}]} >> >> ... >> >> The type and dictionary arguments are visible here (along with the evidence bindings), but type and dictionary applications are only visible if you use -ddump-tc-ast, which is a lot more verbose. (I don't think there is another flag that shows these applications?) Since I didn't initially know what "evidence" was, and there is nothing to say that a_a2hp is a type lambda argument, this was pretty opaque until I managed to read the tc-ast and the light went on. >> >> I can see now that the type and dictionary arguments are added by annotating the AST. >> >> Is there anywhere on the GHC wiki that explains how to interpret this output, and says that the type and dictionary applications ARE there, just not shown by '-ddump-tc'? >> >> Perhaps it would be helpful to add some basic description of what comes out of the typechecker to a page like this one? (below) >> >> https://gitlab.haskell.org/ghc/ghc/-/wikis/commentary/compiler/hsc-main >> -BenRI >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From alan.zimm at gmail.com Tue Dec 28 20:52:44 2021 From: alan.zimm at gmail.com (Alan & Kim Zimmerman) Date: Tue, 28 Dec 2021 20:52:44 +0000 Subject: Output language of typechecking pass? In-Reply-To: <010f017e02831fb0-6188717a-c9c2-41f2-92a4-0a49cd4d4f34-000000@us-east-2.amazonses.com> References: <010f017cc31f63bb-475c1cf3-17f9-4904-9d14-45fbe88cbe51-000000@us-east-2.amazonses.com> <27aaf5dc-8a98-228d-bf41-c32784517f95@gmail.com> <010f017e02831fb0-6188717a-c9c2-41f2-92a4-0a49cd4d4f34-000000@us-east-2.amazonses.com> Message-ID: FYI, it is possible to make a "permalink" on github, which points to the code at a specific commit. Perhaps gitlab has something similar? Alan On Tue, 28 Dec 2021 at 19:28, Richard Eisenberg wrote: > We could always make a hyperlink to the source code as hosted on GitLab. > But I actually argue not to: such links would quickly become outdated, in > one of two ways: either we make a permalink, in which case the linked Note > text will become outdated; or we make a link to a particular file & line, > in which case the Note might move somewhere else. Instead, just by naming > the Note title, we have a slightly-harder-to-use link, where you use it by > grepping the source code. This is less convenient, but it will stay > up-to-date. Until we have better tooling to, say, create an HTML anchor > based on a Note, I think this is the best we can do. > > Richard > > On Dec 28, 2021, at 12:10 PM, Benjamin Redelings < > benjamin.redelings at gmail.com> wrote: > > I was thinking about the relationship between the wiki and the notes in > the GHC source. > > Would it be possible to link directly to [compiler notes] in the GHC > source from the wiki, using hyperlinks? Right now, I'm seeing references > that look like: (See Note [Constraint flavours].) > > (I can see the motivation to include comments in the source, but I also > think that the wiki is more discoverable than the compiler source code. > So, in the interests of pursuing both approaches, it would be nice to be > able to link to notes FROM the wiki. I suppose one could include a > hyperlink to the file on github that contains the note...) > > I'm not sure how much web infrastructure would be required to make > hyperlinks for notes... > > -BenRI > On 11/8/21 5:35 AM, Simon Peyton Jones wrote: > > Is there anywhere on the GHC wiki that explains how to interpret this > output, and says that the type and dictionary applications ARE there, just > not shown by '-ddump-tc'? > > Perhaps it would be helpful to add some basic description of what comes > out of the typechecker to a page like this one? (below) > > > https://gitlab.haskell.org/ghc/ghc/-/wikis/commentary/compiler/hsc-main > > Yes it would! Would you care to start such a wiki page (a new one; don’t > just clutter up the one you point to)? You can write down what you > know. Don’t worry if you aren’t 100% sure – we can correct it. And if you > outright don’t know, leave a “What should I say here?” note. > > > "This late desugaring is somewhat unusual. It is much more common to > desugar the program before typechecking, or renaming, because that presents > the renamer and typechecker with a much smaller language to deal with. > However, GHC's organisation means that > This note is now slightly out of date. We are now, very carefully, doing > some desugaring *before* typechecking. See > > - Note [Handling overloaded and rebindable constructs] in > GHC.Rename.Expr > - Note [Rebindable syntax and HsExpansion] in GHC.Hs.Expr > > > You can and should point to these and similar Notes from the wiki page you > write. Indeed there may be some part of what you write that would be > better framed as Note in GHC’s source code. > > Thanks! > > Simon > > PS: I am leaving Microsoft at the end of November 2021, at which point > simonpj at microsoft.com will cease to work. Use simon.peytonjones at gmail.com > instead. (For now, it just forwards to simonpj at microsoft.com.) > > *From:* ghc-devs > *On Behalf Of *Benjamin Redelings > *Sent:* 08 November 2021 13:12 > *To:* Richard Eisenberg > *Cc:* ghc-devs at haskell.org > *Subject:* Re: Output language of typechecking pass? > > > Hi, > > > > Questions: > > > > 1. It seems like this separation is actually necessary, in order to apply generalization only to let arguments written by the programmer, and not to let bindings introduced during desugaring. Is that right? > > > > I don't think so. That is, if we did it all in one pass, I still think we could get generalization right. > > I guess I asked this question wrong. I mean to say, if we did the two > passes in the reverse order (desugaring first, followed by typechecking), > that would not work, right? > > As the wiki says: > > "This late desugaring is somewhat unusual. It is much more common to > desugar the program before typechecking, or renaming, because that presents > the renamer and typechecker with a much smaller language to deal with. > However, GHC's organisation means that > > - error messages can display precisely the syntax that the user wrote; > and > - desugaring is not required to preserve type-inference properties. > > " > > > > 2. Does the output of type checking contain type lambdas? > > > > Yes. See below. > > > > > > 3. Does the type checking pass determine where to add dictionary arguments? > > > > Yes. See below. > > > > > > 4. Are there any other resources I should be looking at? > > > > Yes. You want to enable -fprint-typechecker-elaboration (and possible -fprint-explicit-coercions). With the former, you get to see all this stuff you're looking for. It's normally suppressed so that the output resembles the user's code. > > > > I hope this helps! > > Richard > > Hmm... so, I think I see how this works now. I don't think > '-fprint-explicit-coercions' does anything here though. > > $ ghc -ddump-tc Test2.hs -fprint-typechecker-elaboration > > ... > > AbsBinds [a_a2hp] [$dNum_a2hB] > {Exports: [g <= g_a2hz > wrap: <>] > Exported types: g :: forall a. Num a => a -> a -> a > [LclId] > Binds: g x_aYk y_aYl = (y_aYl * x_aYk) + 1 > Evidence: [EvBinds{[W] $dNum_a2hs = $dNum_a2hq > [W] $dNum_a2hw = $dNum_a2hq > [W] $dNum_a2hq = $dNum_a2hB}]} > > ... > > The type and dictionary arguments are visible here (along with the > evidence bindings), but type and dictionary applications are only visible > if you use -ddump-tc-ast, which is a lot more verbose. (I don't think > there is another flag that shows these applications?) Since I didn't > initially know what "evidence" was, and there is nothing to say that a_a2hp > is a type lambda argument, this was pretty opaque until I managed to read > the tc-ast and the light went on. > > I can see now that the type and dictionary arguments are added by > annotating the AST. > > Is there anywhere on the GHC wiki that explains how to interpret this > output, and says that the type and dictionary applications ARE there, just > not shown by '-ddump-tc'? > > Perhaps it would be helpful to add some basic description of what comes > out of the typechecker to a page like this one? (below) > > > https://gitlab.haskell.org/ghc/ghc/-/wikis/commentary/compiler/hsc-main > > > -BenRI > > > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lists at richarde.dev Tue Dec 28 21:13:25 2021 From: lists at richarde.dev (Richard Eisenberg) Date: Tue, 28 Dec 2021 21:13:25 +0000 Subject: Output language of typechecking pass? In-Reply-To: References: <010f017cc31f63bb-475c1cf3-17f9-4904-9d14-45fbe88cbe51-000000@us-east-2.amazonses.com> <27aaf5dc-8a98-228d-bf41-c32784517f95@gmail.com> <010f017e02831fb0-6188717a-c9c2-41f2-92a4-0a49cd4d4f34-000000@us-east-2.amazonses.com> Message-ID: <010f017e02e405c9-e34faa90-b0eb-4e58-b293-9fbcfe97d277-000000@us-east-2.amazonses.com> Yes, GitLab supports such a permalink. The problem is that we frequently revise Notes as we understand problems better / improve the implementation. So a reader might link to a Note from the wiki only to study an old implementation that has been superseded. Richard > On Dec 28, 2021, at 3:52 PM, Alan & Kim Zimmerman wrote: > > FYI, it is possible to make a "permalink" on github, which points to the code at a specific commit. Perhaps gitlab has something similar? > > Alan > > On Tue, 28 Dec 2021 at 19:28, Richard Eisenberg > wrote: > We could always make a hyperlink to the source code as hosted on GitLab. But I actually argue not to: such links would quickly become outdated, in one of two ways: either we make a permalink, in which case the linked Note text will become outdated; or we make a link to a particular file & line, in which case the Note might move somewhere else. Instead, just by naming the Note title, we have a slightly-harder-to-use link, where you use it by grepping the source code. This is less convenient, but it will stay up-to-date. Until we have better tooling to, say, create an HTML anchor based on a Note, I think this is the best we can do. > > Richard > >> On Dec 28, 2021, at 12:10 PM, Benjamin Redelings > wrote: >> >> I was thinking about the relationship between the wiki and the notes in the GHC source. >> >> Would it be possible to link directly to [compiler notes] in the GHC source from the wiki, using hyperlinks? Right now, I'm seeing references that look like: (See Note [Constraint flavours].) >> >> (I can see the motivation to include comments in the source, but I also think that the wiki is more discoverable than the compiler source code. So, in the interests of pursuing both approaches, it would be nice to be able to link to notes FROM the wiki. I suppose one could include a hyperlink to the file on github that contains the note...) >> >> I'm not sure how much web infrastructure would be required to make hyperlinks for notes... >> >> -BenRI >> >> On 11/8/21 5:35 AM, Simon Peyton Jones wrote: >>> Is there anywhere on the GHC wiki that explains how to interpret this output, and says that the type and dictionary applications ARE there, just not shown by '-ddump-tc'? >>> >>> Perhaps it would be helpful to add some basic description of what comes out of the typechecker to a page like this one? (below) >>> >>> https://gitlab.haskell.org/ghc/ghc/-/wikis/commentary/compiler/hsc-main >>> Yes it would! Would you care to start such a wiki page (a new one; don’t just clutter up the one you point to)? You can write down what you know. Don’t worry if you aren’t 100% sure – we can correct it. And if you outright don’t know, leave a “What should I say here?” note. >>> >>> "This late desugaring is somewhat unusual. It is much more common to desugar the program before typechecking, or renaming, because that presents the renamer and typechecker with a much smaller language to deal with. However, GHC's organisation means that >>> >>> This note is now slightly out of date. We are now, very carefully, doing some desugaring before typechecking. See >>> Note [Handling overloaded and rebindable constructs] in GHC.Rename.Expr >>> Note [Rebindable syntax and HsExpansion] in GHC.Hs.Expr >>> >>> You can and should point to these and similar Notes from the wiki page you write. Indeed there may be some part of what you write that would be better framed as Note in GHC’s source code. >>> >>> Thanks! >>> >>> Simon >>> >>> PS: I am leaving Microsoft at the end of November 2021, at which point simonpj at microsoft.com will cease to work. Use simon.peytonjones at gmail.com instead. (For now, it just forwards to simonpj at microsoft.com .) >>> >>> From: ghc-devs On Behalf Of Benjamin Redelings >>> Sent: 08 November 2021 13:12 >>> To: Richard Eisenberg >>> Cc: ghc-devs at haskell.org >>> Subject: Re: Output language of typechecking pass? >>> >>> Hi, >>> >>> >>> Questions: >>> >>> 1. It seems like this separation is actually necessary, in order to apply generalization only to let arguments written by the programmer, and not to let bindings introduced during desugaring. Is that right? >>> >>> I don't think so. That is, if we did it all in one pass, I still think we could get generalization right. >>> I guess I asked this question wrong. I mean to say, if we did the two passes in the reverse order (desugaring first, followed by typechecking), that would not work, right? >>> >>> As the wiki says: >>> >>> "This late desugaring is somewhat unusual. It is much more common to desugar the program before typechecking, or renaming, because that presents the renamer and typechecker with a much smaller language to deal with. However, GHC's organisation means that >>> >>> error messages can display precisely the syntax that the user wrote; and >>> desugaring is not required to preserve type-inference properties. >>> " >>> >>> >>> 2. Does the output of type checking contain type lambdas? >>> >>> Yes. See below. >>> >>> >>> 3. Does the type checking pass determine where to add dictionary arguments? >>> >>> Yes. See below. >>> >>> >>> 4. Are there any other resources I should be looking at? >>> >>> Yes. You want to enable -fprint-typechecker-elaboration (and possible -fprint-explicit-coercions). With the former, you get to see all this stuff you're looking for. It's normally suppressed so that the output resembles the user's code. >>> >>> I hope this helps! >>> Richard >>> Hmm... so, I think I see how this works now. I don't think '-fprint-explicit-coercions' does anything here though. >>> >>> $ ghc -ddump-tc Test2.hs -fprint-typechecker-elaboration >>> >>> ... >>> >>> AbsBinds [a_a2hp] [$dNum_a2hB] >>> {Exports: [g <= g_a2hz >>> wrap: <>] >>> Exported types: g :: forall a. Num a => a -> a -> a >>> [LclId] >>> Binds: g x_aYk y_aYl = (y_aYl * x_aYk) + 1 >>> Evidence: [EvBinds{[W] $dNum_a2hs = $dNum_a2hq >>> [W] $dNum_a2hw = $dNum_a2hq >>> [W] $dNum_a2hq = $dNum_a2hB}]} >>> >>> ... >>> >>> The type and dictionary arguments are visible here (along with the evidence bindings), but type and dictionary applications are only visible if you use -ddump-tc-ast, which is a lot more verbose. (I don't think there is another flag that shows these applications?) Since I didn't initially know what "evidence" was, and there is nothing to say that a_a2hp is a type lambda argument, this was pretty opaque until I managed to read the tc-ast and the light went on. >>> >>> I can see now that the type and dictionary arguments are added by annotating the AST. >>> >>> Is there anywhere on the GHC wiki that explains how to interpret this output, and says that the type and dictionary applications ARE there, just not shown by '-ddump-tc'? >>> >>> Perhaps it would be helpful to add some basic description of what comes out of the typechecker to a page like this one? (below) >>> >>> https://gitlab.haskell.org/ghc/ghc/-/wikis/commentary/compiler/hsc-main >>> -BenRI >>> > > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs -------------- next part -------------- An HTML attachment was scrubbed... URL: From lists at richarde.dev Wed Dec 29 16:53:43 2021 From: lists at richarde.dev (Richard Eisenberg) Date: Wed, 29 Dec 2021 16:53:43 +0000 Subject: convention around pattern synonyms Message-ID: <010f017e071c9e25-632a3872-29de-4e3a-b0f1-70cdc000c22f-000000@us-east-2.amazonses.com> Hi devs, Maybe I'm just old fashioned, but I've come to find pattern synonyms really confusing. Because pattern synonyms will tend to appear next to proper data constructors in code (and they look just like data constructors), when I see one, I think it *is* a data constructor. This problem was motivated by a recent MR that introduces a new pattern synonym that caught me off-guard. So, I'd like to propose the following convention: Every pattern synonym satisfies one of the following two criteria: 1. The pattern synonym is a member of a set of synonyms/constructors that expresses a view of a type. There would naturally be a `COMPLETE` pragma including the set. `GHC.Types.Var.Inferred` is an example. 2. The pattern synonym begins with the prefix `PS_`. In the end, I'd probably prefer just (2). With Inferred, for example, I've been caught in the past trying to figure just what the constructors of ArgFlag were (there seemed to be too many), until I realized what was going on. Pattern synonyms are useful abstractions. I like them. But my mental model of a pattern match is that it matches the structure of the scrutinee and performs no computation. Pattern synonyms violate both of these assumptions, and so (as a reader) I like to know when to put these assumptions to the side. Future IDE support that could, say, color pattern synonyms differently to regular constructors would obviate the need for this convention. What do others think here? `PS_` is ugly. I don't need something quite so loud and ugly, but it's also easy to remember and recognize. Thanks! Richard -------------- next part -------------- An HTML attachment was scrubbed... URL: From ekmett at gmail.com Wed Dec 29 17:19:16 2021 From: ekmett at gmail.com (Edward Kmett) Date: Wed, 29 Dec 2021 12:19:16 -0500 Subject: convention around pattern synonyms In-Reply-To: <010f017e071c9e25-632a3872-29de-4e3a-b0f1-70cdc000c22f-000000@us-east-2.amazonses.com> References: <010f017e071c9e25-632a3872-29de-4e3a-b0f1-70cdc000c22f-000000@us-east-2.amazonses.com> Message-ID: Please no. I use them to pun constructors between multiple types that will be in scope at the same time, (e.g. when I have 8 Var constructors on different types in scope between my core language term language and type language...) and often overload them on classes. I can't write the pragma, and the PS_ destroys any utiity I get from any common name. I use them as a migration guide, when I add functionality. PS_ destroys that usecase, but then COMPLETE pragmas are a hacky mess in their current state and often simply can't be applied. All the existing pattern constructors in the lens library would fail either bar. So I have to say, either of these would probably destroy *every* use of pattern synonyms I use today. -Edward On Wed, Dec 29, 2021 at 11:55 AM Richard Eisenberg wrote: > Hi devs, > > Maybe I'm just old fashioned, but I've come to find pattern synonyms > really confusing. Because pattern synonyms will tend to appear next to > proper data constructors in code (and they look just like data > constructors), when I see one, I think it *is* a data constructor. This > problem was motivated by a recent MR that introduces a new pattern synonym > that > caught me off-guard. > > So, I'd like to propose the following convention: Every pattern synonym > satisfies one of the following two criteria: > 1. The pattern synonym is a member of a set of synonyms/constructors that > expresses a view of a type. There would naturally be a `COMPLETE` pragma > including the set. `GHC.Types.Var.Inferred` is an example. > 2. The pattern synonym begins with the prefix `PS_`. > > In the end, I'd probably prefer just (2). With Inferred, for example, I've > been caught in the past trying to figure just what the constructors of > ArgFlag were (there seemed to be too many), until I realized what was going > on. > > Pattern synonyms are useful abstractions. I like them. But my mental model > of a pattern match is that it matches the structure of the scrutinee and > performs no computation. Pattern synonyms violate both of these > assumptions, and so (as a reader) I like to know when to put these > assumptions to the side. > > Future IDE support that could, say, color pattern synonyms differently to > regular constructors would obviate the need for this convention. > > What do others think here? `PS_` is ugly. I don't need something quite so > loud and ugly, but it's also easy to remember and recognize. > > Thanks! > Richard > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ekmett at gmail.com Wed Dec 29 18:19:51 2021 From: ekmett at gmail.com (Edward Kmett) Date: Wed, 29 Dec 2021 13:19:51 -0500 Subject: convention around pattern synonyms In-Reply-To: References: <010f017e071c9e25-632a3872-29de-4e3a-b0f1-70cdc000c22f-000000@us-east-2.amazonses.com> Message-ID: If this is just about GHC internals, then by all means carry on. -Edward On Wed, Dec 29, 2021 at 12:19 PM Edward Kmett wrote: > Please no. > > I use them to pun constructors between multiple types that will be in > scope at the same time, (e.g. when I have 8 Var constructors on different > types in scope between my core language term language and type language...) > and often overload them on classes. I can't write the pragma, and the PS_ > destroys any utiity I get from any common name. > > I use them as a migration guide, when I add functionality. PS_ destroys > that usecase, but then COMPLETE pragmas are a hacky mess in their current > state and often simply can't be applied. > > All the existing pattern constructors in the lens library would fail > either bar. > > So I have to say, either of these would probably destroy *every* use of > pattern synonyms I use today. > > -Edward > > On Wed, Dec 29, 2021 at 11:55 AM Richard Eisenberg > wrote: > >> Hi devs, >> >> Maybe I'm just old fashioned, but I've come to find pattern synonyms >> really confusing. Because pattern synonyms will tend to appear next to >> proper data constructors in code (and they look just like data >> constructors), when I see one, I think it *is* a data constructor. This >> problem was motivated by a recent MR that introduces a new pattern >> synonym >> that >> caught me off-guard. >> >> So, I'd like to propose the following convention: Every pattern synonym >> satisfies one of the following two criteria: >> 1. The pattern synonym is a member of a set of synonyms/constructors that >> expresses a view of a type. There would naturally be a `COMPLETE` pragma >> including the set. `GHC.Types.Var.Inferred` is an example. >> 2. The pattern synonym begins with the prefix `PS_`. >> >> In the end, I'd probably prefer just (2). With Inferred, for example, >> I've been caught in the past trying to figure just what the constructors of >> ArgFlag were (there seemed to be too many), until I realized what was going >> on. >> >> Pattern synonyms are useful abstractions. I like them. But my mental >> model of a pattern match is that it matches the structure of the scrutinee >> and performs no computation. Pattern synonyms violate both of these >> assumptions, and so (as a reader) I like to know when to put these >> assumptions to the side. >> >> Future IDE support that could, say, color pattern synonyms differently to >> regular constructors would obviate the need for this convention. >> >> What do others think here? `PS_` is ugly. I don't need something quite so >> loud and ugly, but it's also easy to remember and recognize. >> >> Thanks! >> Richard >> _______________________________________________ >> ghc-devs mailing list >> ghc-devs at haskell.org >> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mail at joachim-breitner.de Wed Dec 29 19:39:02 2021 From: mail at joachim-breitner.de (Joachim Breitner) Date: Wed, 29 Dec 2021 20:39:02 +0100 Subject: What's the benefit of taking "do" blocks apart? Is there a way to turn that off? In-Reply-To: References: Message-ID: <3bace85605cbc915a4b912fb70e6779c6977c568.camel@joachim-breitner.de> Hi Gergo, Am Dienstag, dem 28.12.2021 um 15:57 +0000 schrieb Erdi, Gergo via ghc- devs: > PUBLIC phew > I’m seeing ‘do’ blocks getting taking apart into top-level > definitions, so e.g. >   > main = do >   some complicated expression 1 >   some complicated expression 2 >   > is compiled into >   > sat_sKv = some complicated expression 1 > sat_sKw = \_ -> some complicated expression 2 > main = bindIO sat_sKv sat_sKw >   > This seems to happen regardless of any common subexpressions, i.e. it > is not the case that sat_sKv or sat_sKw are used anywhere else. >   > What is the intended benefit of this floating-out? Is there a > particular Core-to-Core pass that causes this? Is it possible to turn > it off? didn’t investigate deeper (maybe if you provide a small example I would), but just from looking at this: * It is generally preferable to turn local lambda expressions into top-level functions. This way, instead of dynamically allocating a FUN heap object, it’s just a static function. * sat_sKv is an IO expression? Then it is actually a function in a way (taking the “State token” as an argument). So the above applies. * I think this is the FloatOut pass. You can turn it out using -fno-full-laziness. Not sure if some others passes might do similar things, though. Cheers, Joachim -- Joachim Breitner mail at joachim-breitner.de http://www.joachim-breitner.de/ From lists at richarde.dev Wed Dec 29 22:07:20 2021 From: lists at richarde.dev (Richard Eisenberg) Date: Wed, 29 Dec 2021 22:07:20 +0000 Subject: convention around pattern synonyms In-Reply-To: References: <010f017e071c9e25-632a3872-29de-4e3a-b0f1-70cdc000c22f-000000@us-east-2.amazonses.com> Message-ID: <010f017e083bbdc6-f5fdb718-cf4a-4e60-9e4c-5cce52e925a0-000000@us-east-2.amazonses.com> > On Dec 29, 2021, at 1:19 PM, Edward Kmett wrote: > > If this is just about GHC internals, then by all means carry on. Yes, I should have clarified: this is just and solely about GHC internals! I have no designs on suggesting a wider convention like this. Indeed, both of the designs you describe below make great sense to use pattern synonyms for -- and without any herald that you are doing so. Within GHC, however, we now have a few cases where pattern synonyms are effectively behaving like or-patterns, which I find unexpected and confusing without a marker telling me Something Unusual is going on. (Why? Because in both cases, the pattern synonym is really designed to act like a constructor. In the first case, if I understand correctly, the pattern synonym is just a renaming of an existing constructor, with no extra computation. In the second case, if I understand correctly, the pattern synonym captures what used to be a constructor. It's plausible that GHC will want to adopt either of these patterns at some point in the future, but we do not do either one today, and so I would say to address any change to my proposed coding convention when this happens.) Richard > > -Edward > > On Wed, Dec 29, 2021 at 12:19 PM Edward Kmett > wrote: > Please no. > > I use them to pun constructors between multiple types that will be in scope at the same time, (e.g. when I have 8 Var constructors on different types in scope between my core language term language and type language...) and often overload them on classes. I can't write the pragma, and the PS_ destroys any utiity I get from any common name. > > I use them as a migration guide, when I add functionality. PS_ destroys that usecase, but then COMPLETE pragmas are a hacky mess in their current state and often simply can't be applied. > > All the existing pattern constructors in the lens library would fail either bar. > > So I have to say, either of these would probably destroy every use of pattern synonyms I use today. > > -Edward > > On Wed, Dec 29, 2021 at 11:55 AM Richard Eisenberg > wrote: > Hi devs, > > Maybe I'm just old fashioned, but I've come to find pattern synonyms really confusing. Because pattern synonyms will tend to appear next to proper data constructors in code (and they look just like data constructors), when I see one, I think it *is* a data constructor. This problem was motivated by a recent MR that introduces a new pattern synonym that caught me off-guard. > > So, I'd like to propose the following convention: Every pattern synonym satisfies one of the following two criteria: > 1. The pattern synonym is a member of a set of synonyms/constructors that expresses a view of a type. There would naturally be a `COMPLETE` pragma including the set. `GHC.Types.Var.Inferred` is an example. > 2. The pattern synonym begins with the prefix `PS_`. > > In the end, I'd probably prefer just (2). With Inferred, for example, I've been caught in the past trying to figure just what the constructors of ArgFlag were (there seemed to be too many), until I realized what was going on. > > Pattern synonyms are useful abstractions. I like them. But my mental model of a pattern match is that it matches the structure of the scrutinee and performs no computation. Pattern synonyms violate both of these assumptions, and so (as a reader) I like to know when to put these assumptions to the side. > > Future IDE support that could, say, color pattern synonyms differently to regular constructors would obviate the need for this convention. > > What do others think here? `PS_` is ugly. I don't need something quite so loud and ugly, but it's also easy to remember and recognize. > > Thanks! > Richard > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs -------------- next part -------------- An HTML attachment was scrubbed... URL: From ietf-dane at dukhovni.org Wed Dec 29 23:12:13 2021 From: ietf-dane at dukhovni.org (Viktor Dukhovni) Date: Wed, 29 Dec 2021 18:12:13 -0500 Subject: convention around pattern synonyms In-Reply-To: <010f017e083bbdc6-f5fdb718-cf4a-4e60-9e4c-5cce52e925a0-000000@us-east-2.amazonses.com> References: <010f017e071c9e25-632a3872-29de-4e3a-b0f1-70cdc000c22f-000000@us-east-2.amazonses.com> <010f017e083bbdc6-f5fdb718-cf4a-4e60-9e4c-5cce52e925a0-000000@us-east-2.amazonses.com> Message-ID: Some "GHC-internal" types leak to users via TH, and their constructors occasionally pick up new fields, causing breakage downstream. The extra field often has a sensible default (Nothing, [], ...) and it should be best practice to rename the constructor when adding the new field, while replacing the original constructor with a pattern synonym with the "old" signature. data Foo = ... | NewImprovedMkFoo X Y Z -- was MkFoo Y Z pattern MkFoo :: Foo pattern MkFoo Y Z = NewImprovedMkFoo Nothing Y Z When pattern synonyms are used to maintain a backwards-compatible API, there should of course be no special signalling to differentiate them from "real" constructors. The boundary between "GHC-internal" and external may not always be obvious, some care is required to reduce leaking breakage via TH. -- Viktor. From Gergo.Erdi at sc.com Thu Dec 30 09:03:37 2021 From: Gergo.Erdi at sc.com (Erdi, Gergo) Date: Thu, 30 Dec 2021 09:03:37 +0000 Subject: Avoiding full laziness xform / floating-out (Re: What's the benefit of taking "do" blocks apart? Is there a way to turn that off?) Message-ID: PUBLIC Hi Joachim, Thanks for the hints! > Hi Gergo, > > Am Dienstag, dem 28.12.2021 um 15:57 +0000 schrieb Erdi, Gergo via ghc- > devs: > > PUBLIC > > phew Yeah obviously I'm sitting here not only adding these tags, but also coming up with the automated systems and also the company policies forcing the usage of said systems ;) > didn't investigate deeper (maybe if you provide a small example I would), but just from looking at this: Unfortunately, it's hard to make a small example because this all lives in bizarro world where "IO" isn't IO, "String" isn't [Char] etc, so I can't just take the input program and pass it to vanilla GHC. > > * It is generally preferable to turn local lambda expressions > into top-level functions. This way, instead of dynamically > allocating a FUN heap object, it's just a static function. > > * sat_sKv is an IO expression? Then it is actually a function in a way > (taking the "State token" as an argument). So the above applies. This is "IO" but not GHC's IO, just another type with an opaque definition and a Monad instance. There's no reason for GHC to think that sat_sKv would be a function. Of course, sat_sKw is necessarily a function since it is the second argument to bind. But only now I am noticing that even sat_sKw's definition includes *yet another* floated-out variable: sat_sKv = some complicated expression 1 sat_sKu = some complicated expression 2 sat_sKw = \_ -> sat_sKu main = bindIO sat_sKv sat_sKw Here, I don't see why GHC should think that sat_sKv and sat_sKu are functions. For example, here's the definition of sat_sKu: sat_sKu :: IO () [LclId] sat_sKu = let { sat_sRy [Occ=Once1] :: String [LclId] sat_sRy = let { sat_sRx [Occ=Once1] :: String [LclId] sat_sRx = unpackCStringUtf8# "\n"# } in let { sat_sRw [Occ=Once1] :: String [LclId] sat_sRw = case foobar True of { False -> unpackCStringUtf8# "False"#; True -> unpackCStringUtf8# "True"# } } in sApp# sat_sRw sat_sRx } in putStrOut sat_sRy Here, putStrOut is so opaque that it doesn't even have a definition, it is merely a name registered into GHC's name tables. So I really don't see why this looks "function-y" to GHC. The reason this is all a problem for me is because I would like to then interpret all these let-bindings, including the toplevel ones, as eagerly defined variables. But then there is of course a difference between main = let a1 = someIOAction1 a2 = someOtherIOAction2 a2' = \_ -> a2 in bindIO a1 a2' and main = let a1 = someIOAction1 a2 = \_ -> someIOAction2 in bindIO a1 a2 because if the *definition* of "a2" throws (not its action), then the first version will throw immediately whereas the second will only throw when "main" is *run*, after running "a1". > * I think this is the FloatOut pass. You can turn it out using > -fno-full-laziness. Not sure if some others passes might > do similar things, though. I tried turning off Opt_FullLaziness, but unfortunately I am still getting the same result. I was also hopeful when I saw there's an Opt_FloatIn flag, but alas that doesn't change this behaviour either (and it's turned on by default anyway...) I'll try to make a self-contained minimal example next week. Thanks, Gergo -----Original Message----- From: ghc-devs On Behalf Of Joachim Breitner Sent: Wednesday, December 29, 2021 8:39 PM To: ghc-devs at haskell.org Subject: [External] Re: What's the benefit of taking "do" blocks apart? Is there a way to turn that off? [You don't often get email from mail at joachim-breitner.de. Learn why this is important at http://aka.ms/LearnAboutSenderIdentification.] ATTENTION: This email came from an external source. Do not open attachments or click on links from unknown senders or unexpected emails. Always report suspicious emails using the Report As Phishing button in Outlook to protect the Bank and our clients. Hi Gergo, Am Dienstag, dem 28.12.2021 um 15:57 +0000 schrieb Erdi, Gergo via ghc- devs: > PUBLIC phew > I’m seeing ‘do’ blocks getting taking apart into top-level > definitions, so e.g. > > main = do > some complicated expression 1 > some complicated expression 2 > > is compiled into > > sat_sKv = some complicated expression 1 sat_sKw = \_ -> some > complicated expression 2 main = bindIO sat_sKv sat_sKw > > This seems to happen regardless of any common subexpressions, i.e. it > is not the case that sat_sKv or sat_sKw are used anywhere else. > > What is the intended benefit of this floating-out? Is there a > particular Core-to-Core pass that causes this? Is it possible to turn > it off? didn’t investigate deeper (maybe if you provide a small example I would), but just from looking at this: * It is generally preferable to turn local lambda expressions into top-level functions. This way, instead of dynamically allocating a FUN heap object, it’s just a static function. * sat_sKv is an IO expression? Then it is actually a function in a way (taking the “State token” as an argument). So the above applies. * I think this is the FloatOut pass. You can turn it out using -fno-full-laziness. Not sure if some others passes might do similar things, though. Cheers, Joachim -- Joachim Breitner mail at joachim-breitner.de https://clicktime.symantec.com/33oTTj3Bye1kkqsmxZWi3QZ6xU?u=http%3A%2F%2Fwww.joachim-breitner.de%2F _______________________________________________ ghc-devs mailing list ghc-devs at haskell.org https://clicktime.symantec.com/3bo4KFUFPcVSdMe4B1CUzr6xU?u=http%3A%2F%2Fmail.haskell.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fghc-devs This email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please delete all copies and notify the sender immediately. You may wish to refer to the incorporation details of Standard Chartered PLC, Standard Chartered Bank and their subsidiaries at https: //www.sc.com/en/our-locations Where you have a Financial Markets relationship with Standard Chartered PLC, Standard Chartered Bank and their subsidiaries (the "Group"), information on the regulatory standards we adhere to and how it may affect you can be found in our Regulatory Compliance Statement at https: //www.sc.com/rcs/ and Regulatory Compliance Disclosures at http: //www.sc.com/rcs/fm Insofar as this communication is not sent by the Global Research team and contains any market commentary, the market commentary has been prepared by the sales and/or trading desk of Standard Chartered Bank or its affiliate. It is not and does not constitute research material, independent research, recommendation or financial advice. Any market commentary is for information purpose only and shall not be relied on for any other purpose and is subject to the relevant disclaimers available at https: //www.sc.com/en/regulatory-disclosures/#market-disclaimer. Insofar as this communication is sent by the Global Research team and contains any research materials prepared by members of the team, the research material is for information purpose only and shall not be relied on for any other purpose, and is subject to the relevant disclaimers available at https: //research.sc.com/research/api/application/static/terms-and-conditions. Insofar as this e-mail contains the term sheet for a proposed transaction, by responding affirmatively to this e-mail, you agree that you have understood the terms and conditions in the attached term sheet and evaluated the merits and risks of the transaction. We may at times also request you to sign the term sheet to acknowledge the same. Please visit https: //www.sc.com/en/regulatory-disclosures/dodd-frank/ for important information with respect to derivative products. From matthewtpickering at gmail.com Thu Dec 30 09:41:37 2021 From: matthewtpickering at gmail.com (Matthew Pickering) Date: Thu, 30 Dec 2021 09:41:37 +0000 Subject: Avoiding full laziness xform / floating-out (Re: What's the benefit of taking "do" blocks apart? Is there a way to turn that off?) In-Reply-To: References: Message-ID: Hi Gergo, Sounds like you might be better off writing your own optimisation pass rather than relying on making GHC do what you want. Cheers Matt On Thu, Dec 30, 2021 at 9:05 AM Erdi, Gergo via ghc-devs wrote: > > PUBLIC > > Hi Joachim, > > Thanks for the hints! > > > Hi Gergo, > > > > Am Dienstag, dem 28.12.2021 um 15:57 +0000 schrieb Erdi, Gergo via ghc- > > devs: > > > PUBLIC > > > > phew > > Yeah obviously I'm sitting here not only adding these tags, but also > coming up with the automated systems and also the company policies > forcing the usage of said systems ;) > > > didn't investigate deeper (maybe if you provide a small example I would), but just from looking at this: > > Unfortunately, it's hard to make a small example because this all > lives in bizarro world where "IO" isn't IO, "String" isn't [Char] etc, > so I can't just take the input program and pass it to vanilla GHC. > > > > > * It is generally preferable to turn local lambda expressions > > into top-level functions. This way, instead of dynamically > > allocating a FUN heap object, it's just a static function. > > > > * sat_sKv is an IO expression? Then it is actually a function in a way > > (taking the "State token" as an argument). So the above applies. > > This is "IO" but not GHC's IO, just another type with an opaque > definition and a Monad instance. There's no reason for GHC to think > that sat_sKv would be a function. > > Of course, sat_sKw is necessarily a function since it is the second > argument to bind. But only now I am noticing that even sat_sKw's > definition includes *yet another* floated-out variable: > > sat_sKv = some complicated expression 1 > sat_sKu = some complicated expression 2 > sat_sKw = \_ -> sat_sKu > main = bindIO sat_sKv sat_sKw > > Here, I don't see why GHC should think that sat_sKv and sat_sKu are > functions. For example, here's the definition of sat_sKu: > > sat_sKu :: IO () > [LclId] > sat_sKu > = let { > sat_sRy [Occ=Once1] :: String > [LclId] > sat_sRy > = let { > sat_sRx [Occ=Once1] :: String > [LclId] > sat_sRx = unpackCStringUtf8# "\n"# } in > let { > sat_sRw [Occ=Once1] :: String > [LclId] > sat_sRw > = case foobar True of { > False -> unpackCStringUtf8# "False"#; > True -> unpackCStringUtf8# "True"# > } } in > sApp# sat_sRw sat_sRx } in > putStrOut sat_sRy > > Here, putStrOut is so opaque that it doesn't even have a definition, > it is merely a name registered into GHC's name tables. So I really > don't see why this looks "function-y" to GHC. > > The reason this is all a problem for me is because I would like to > then interpret all these let-bindings, including the toplevel ones, as > eagerly defined variables. But then there is of course a difference between > > main = > let a1 = someIOAction1 > a2 = someOtherIOAction2 > a2' = \_ -> a2 > in bindIO a1 a2' > > and > > main = > let a1 = someIOAction1 > a2 = \_ -> someIOAction2 > in bindIO a1 a2 > > because if the *definition* of "a2" throws (not its action), then the > first version will throw immediately whereas the second will only > throw when "main" is *run*, after running "a1". > > > * I think this is the FloatOut pass. You can turn it out using > > -fno-full-laziness. Not sure if some others passes might > > do similar things, though. > > I tried turning off Opt_FullLaziness, but unfortunately I am still > getting the same result. I was also hopeful when I saw there's an > Opt_FloatIn flag, but alas that doesn't change this behaviour either > (and it's turned on by default anyway...) > > I'll try to make a self-contained minimal example next week. > > Thanks, > Gergo > > -----Original Message----- > From: ghc-devs On Behalf Of Joachim Breitner > Sent: Wednesday, December 29, 2021 8:39 PM > To: ghc-devs at haskell.org > Subject: [External] Re: What's the benefit of taking "do" blocks apart? Is there a way to turn that off? > > [You don't often get email from mail at joachim-breitner.de. Learn why this is important at http://aka.ms/LearnAboutSenderIdentification.] > > ATTENTION: This email came from an external source. Do not open attachments or click on links from unknown senders or unexpected emails. Always report suspicious emails using the Report As Phishing button in Outlook to protect the Bank and our clients. > > > Hi Gergo, > > Am Dienstag, dem 28.12.2021 um 15:57 +0000 schrieb Erdi, Gergo via ghc- > devs: > > PUBLIC > > phew > > > I’m seeing ‘do’ blocks getting taking apart into top-level > > definitions, so e.g. > > > > main = do > > some complicated expression 1 > > some complicated expression 2 > > > > is compiled into > > > > sat_sKv = some complicated expression 1 sat_sKw = \_ -> some > > complicated expression 2 main = bindIO sat_sKv sat_sKw > > > > This seems to happen regardless of any common subexpressions, i.e. it > > is not the case that sat_sKv or sat_sKw are used anywhere else. > > > > What is the intended benefit of this floating-out? Is there a > > particular Core-to-Core pass that causes this? Is it possible to turn > > it off? > > > didn’t investigate deeper (maybe if you provide a small example I would), but just from looking at this: > > * It is generally preferable to turn local lambda expressions > into top-level functions. This way, instead of dynamically > allocating a FUN heap object, it’s just a static function. > > * sat_sKv is an IO expression? Then it is actually a function in a way > (taking the “State token” as an argument). So the above applies. > > * I think this is the FloatOut pass. You can turn it out using > -fno-full-laziness. Not sure if some others passes might > do similar things, though. > > Cheers, > Joachim > > > -- > Joachim Breitner > mail at joachim-breitner.de > https://clicktime.symantec.com/33oTTj3Bye1kkqsmxZWi3QZ6xU?u=http%3A%2F%2Fwww.joachim-breitner.de%2F > > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > https://clicktime.symantec.com/3bo4KFUFPcVSdMe4B1CUzr6xU?u=http%3A%2F%2Fmail.haskell.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fghc-devs > > This email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please delete all copies and notify the sender immediately. You may wish to refer to the incorporation details of Standard Chartered PLC, Standard Chartered Bank and their subsidiaries at https: //www.sc.com/en/our-locations > > Where you have a Financial Markets relationship with Standard Chartered PLC, Standard Chartered Bank and their subsidiaries (the "Group"), information on the regulatory standards we adhere to and how it may affect you can be found in our Regulatory Compliance Statement at https: //www.sc.com/rcs/ and Regulatory Compliance Disclosures at http: //www.sc.com/rcs/fm > > Insofar as this communication is not sent by the Global Research team and contains any market commentary, the market commentary has been prepared by the sales and/or trading desk of Standard Chartered Bank or its affiliate. It is not and does not constitute research material, independent research, recommendation or financial advice. Any market commentary is for information purpose only and shall not be relied on for any other purpose and is subject to the relevant disclaimers available at https: //www.sc.com/en/regulatory-disclosures/#market-disclaimer. > > Insofar as this communication is sent by the Global Research team and contains any research materials prepared by members of the team, the research material is for information purpose only and shall not be relied on for any other purpose, and is subject to the relevant disclaimers available at https: //research.sc.com/research/api/application/static/terms-and-conditions. > > Insofar as this e-mail contains the term sheet for a proposed transaction, by responding affirmatively to this e-mail, you agree that you have understood the terms and conditions in the attached term sheet and evaluated the merits and risks of the transaction. We may at times also request you to sign the term sheet to acknowledge the same. > > Please visit https: //www.sc.com/en/regulatory-disclosures/dodd-frank/ for important information with respect to derivative products. > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs From Gergo.Erdi at sc.com Thu Dec 30 11:56:13 2021 From: Gergo.Erdi at sc.com (Erdi, Gergo) Date: Thu, 30 Dec 2021 11:56:13 +0000 Subject: [External] Re: Avoiding full laziness xform / floating-out (Re: What's the benefit of taking "do" blocks apart? Is there a way to turn that off?) In-Reply-To: References: Message-ID: PUBLIC Turning on various Opt_D_dump flags, I can see that this tranformation happens in the first iteration of the simplifier. Looking at GHC.Core.Opt.getCoreToDo, I would guess this corresponds to this line: -- initial simplify: mk specialiser happy: minimum effort please runWhen do_presimplify simpl_gently, If I wrote my own optimisation pass, I'm not sure what it could do to inhibit GHC from doing this transformation later on in its pipeline. And I don't want to give up on the simplifier pipeline completely, since there are parts of it that I very much need. One of them is, unfortunately, the specializer itself -- and that comment in getCoreToDo makes me worried that if I somehow managed to get rid of this transformation (or if I skipped the simplifier pipeline completely and just ran the specializer out-of-band), I would break specialization anyway... -----Original Message----- From: Matthew Pickering Sent: Thursday, December 30, 2021 10:42 AM To: Erdi, Gergo Cc: Joachim Breitner ; ghc-devs at haskell.org Subject: [External] Re: Avoiding full laziness xform / floating-out (Re: What's the benefit of taking "do" blocks apart? Is there a way to turn that off?) Hi Gergo, Sounds like you might be better off writing your own optimisation pass rather than relying on making GHC do what you want. Cheers Matt On Thu, Dec 30, 2021 at 9:05 AM Erdi, Gergo via ghc-devs wrote: > > PUBLIC > > Hi Joachim, > > Thanks for the hints! > > > Hi Gergo, > > > > Am Dienstag, dem 28.12.2021 um 15:57 +0000 schrieb Erdi, Gergo via > > ghc- > > devs: > > > PUBLIC > > > > phew > > Yeah obviously I'm sitting here not only adding these tags, but also > coming up with the automated systems and also the company policies > forcing the usage of said systems ;) > > > didn't investigate deeper (maybe if you provide a small example I would), but just from looking at this: > > Unfortunately, it's hard to make a small example because this all > lives in bizarro world where "IO" isn't IO, "String" isn't [Char] etc, > so I can't just take the input program and pass it to vanilla GHC. > > > > > * It is generally preferable to turn local lambda expressions > > into top-level functions. This way, instead of dynamically > > allocating a FUN heap object, it's just a static function. > > > > * sat_sKv is an IO expression? Then it is actually a function in a way > > (taking the "State token" as an argument). So the above applies. > > This is "IO" but not GHC's IO, just another type with an opaque > definition and a Monad instance. There's no reason for GHC to think > that sat_sKv would be a function. > > Of course, sat_sKw is necessarily a function since it is the second > argument to bind. But only now I am noticing that even sat_sKw's > definition includes *yet another* floated-out variable: > > sat_sKv = some complicated expression 1 sat_sKu = some complicated > expression 2 sat_sKw = \_ -> sat_sKu main = bindIO sat_sKv sat_sKw > > Here, I don't see why GHC should think that sat_sKv and sat_sKu are > functions. For example, here's the definition of sat_sKu: > > sat_sKu :: IO () > [LclId] > sat_sKu > = let { > sat_sRy [Occ=Once1] :: String > [LclId] > sat_sRy > = let { > sat_sRx [Occ=Once1] :: String > [LclId] > sat_sRx = unpackCStringUtf8# "\n"# } in > let { > sat_sRw [Occ=Once1] :: String > [LclId] > sat_sRw > = case foobar True of { > False -> unpackCStringUtf8# "False"#; > True -> unpackCStringUtf8# "True"# > } } in > sApp# sat_sRw sat_sRx } in > putStrOut sat_sRy > > Here, putStrOut is so opaque that it doesn't even have a definition, > it is merely a name registered into GHC's name tables. So I really > don't see why this looks "function-y" to GHC. > > The reason this is all a problem for me is because I would like to > then interpret all these let-bindings, including the toplevel ones, as > eagerly defined variables. But then there is of course a difference > between > > main = > let a1 = someIOAction1 > a2 = someOtherIOAction2 > a2' = \_ -> a2 > in bindIO a1 a2' > > and > > main = > let a1 = someIOAction1 > a2 = \_ -> someIOAction2 > in bindIO a1 a2 > > because if the *definition* of "a2" throws (not its action), then the > first version will throw immediately whereas the second will only > throw when "main" is *run*, after running "a1". > > > * I think this is the FloatOut pass. You can turn it out using > > -fno-full-laziness. Not sure if some others passes might > > do similar things, though. > > I tried turning off Opt_FullLaziness, but unfortunately I am still > getting the same result. I was also hopeful when I saw there's an > Opt_FloatIn flag, but alas that doesn't change this behaviour either > (and it's turned on by default anyway...) > > I'll try to make a self-contained minimal example next week. > > Thanks, > Gergo > > -----Original Message----- > From: ghc-devs On Behalf Of Joachim > Breitner > Sent: Wednesday, December 29, 2021 8:39 PM > To: ghc-devs at haskell.org > Subject: [External] Re: What's the benefit of taking "do" blocks apart? Is there a way to turn that off? > > [You don't often get email from mail at joachim-breitner.de. Learn why > this is important at > https://clicktime.symantec.com/3M5MtkHX9TecmxRkvk6ZtdN6xU?u=http%3A%2F > %2Faka.ms%2FLearnAboutSenderIdentification.] > > ATTENTION: This email came from an external source. Do not open attachments or click on links from unknown senders or unexpected emails. Always report suspicious emails using the Report As Phishing button in Outlook to protect the Bank and our clients. > > > Hi Gergo, > > Am Dienstag, dem 28.12.2021 um 15:57 +0000 schrieb Erdi, Gergo via > ghc- > devs: > > PUBLIC > > phew > > > I’m seeing ‘do’ blocks getting taking apart into top-level > > definitions, so e.g. > > > > main = do > > some complicated expression 1 > > some complicated expression 2 > > > > is compiled into > > > > sat_sKv = some complicated expression 1 sat_sKw = \_ -> some > > complicated expression 2 main = bindIO sat_sKv sat_sKw > > > > This seems to happen regardless of any common subexpressions, i.e. > > it is not the case that sat_sKv or sat_sKw are used anywhere else. > > > > What is the intended benefit of this floating-out? Is there a > > particular Core-to-Core pass that causes this? Is it possible to > > turn it off? > > > didn’t investigate deeper (maybe if you provide a small example I would), but just from looking at this: > > * It is generally preferable to turn local lambda expressions > into top-level functions. This way, instead of dynamically > allocating a FUN heap object, it’s just a static function. > > * sat_sKv is an IO expression? Then it is actually a function in a way > (taking the “State token” as an argument). So the above applies. > > * I think this is the FloatOut pass. You can turn it out using > -fno-full-laziness. Not sure if some others passes might > do similar things, though. > > Cheers, > Joachim > > > -- > Joachim Breitner > mail at joachim-breitner.de > > https://clicktime.symantec.com/33oTTj3Bye1kkqsmxZWi3QZ6xU?u=http%3A%2F > %2Fwww.joachim-breitner.de%2F > > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > https://clicktime.symantec.com/3bo4KFUFPcVSdMe4B1CUzr6xU?u=http%3A%2F% > 2Fmail.haskell.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fghc-devs > > This email and any attachments are confidential and may also be > privileged. If you are not the intended recipient, please delete all > copies and notify the sender immediately. You may wish to refer to the > incorporation details of Standard Chartered PLC, Standard Chartered > Bank and their subsidiaries at https: //www.sc.com/en/our-locations > > Where you have a Financial Markets relationship with Standard > Chartered PLC, Standard Chartered Bank and their subsidiaries (the > "Group"), information on the regulatory standards we adhere to and how > it may affect you can be found in our Regulatory Compliance Statement > at https: //www.sc.com/rcs/ and Regulatory Compliance Disclosures at > http: //www.sc.com/rcs/fm > > Insofar as this communication is not sent by the Global Research team and contains any market commentary, the market commentary has been prepared by the sales and/or trading desk of Standard Chartered Bank or its affiliate. It is not and does not constitute research material, independent research, recommendation or financial advice. Any market commentary is for information purpose only and shall not be relied on for any other purpose and is subject to the relevant disclaimers available at https: //www.sc.com/en/regulatory-disclosures/#market-disclaimer. > > Insofar as this communication is sent by the Global Research team and contains any research materials prepared by members of the team, the research material is for information purpose only and shall not be relied on for any other purpose, and is subject to the relevant disclaimers available at https: //research.sc.com/research/api/application/static/terms-and-conditions. > > Insofar as this e-mail contains the term sheet for a proposed transaction, by responding affirmatively to this e-mail, you agree that you have understood the terms and conditions in the attached term sheet and evaluated the merits and risks of the transaction. We may at times also request you to sign the term sheet to acknowledge the same. > > Please visit https: //www.sc.com/en/regulatory-disclosures/dodd-frank/ for important information with respect to derivative products. > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > https://clicktime.symantec.com/38vRWe46h69pkzJy45pDppa6xU?u=http%3A%2F > %2Fmail.haskell.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fghc-devs This email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please delete all copies and notify the sender immediately. You may wish to refer to the incorporation details of Standard Chartered PLC, Standard Chartered Bank and their subsidiaries at https: //www.sc.com/en/our-locations Where you have a Financial Markets relationship with Standard Chartered PLC, Standard Chartered Bank and their subsidiaries (the "Group"), information on the regulatory standards we adhere to and how it may affect you can be found in our Regulatory Compliance Statement at https: //www.sc.com/rcs/ and Regulatory Compliance Disclosures at http: //www.sc.com/rcs/fm Insofar as this communication is not sent by the Global Research team and contains any market commentary, the market commentary has been prepared by the sales and/or trading desk of Standard Chartered Bank or its affiliate. It is not and does not constitute research material, independent research, recommendation or financial advice. Any market commentary is for information purpose only and shall not be relied on for any other purpose and is subject to the relevant disclaimers available at https: //www.sc.com/en/regulatory-disclosures/#market-disclaimer. Insofar as this communication is sent by the Global Research team and contains any research materials prepared by members of the team, the research material is for information purpose only and shall not be relied on for any other purpose, and is subject to the relevant disclaimers available at https: //research.sc.com/research/api/application/static/terms-and-conditions. Insofar as this e-mail contains the term sheet for a proposed transaction, by responding affirmatively to this e-mail, you agree that you have understood the terms and conditions in the attached term sheet and evaluated the merits and risks of the transaction. We may at times also request you to sign the term sheet to acknowledge the same. Please visit https: //www.sc.com/en/regulatory-disclosures/dodd-frank/ for important information with respect to derivative products. From Gergo.Erdi at sc.com Thu Dec 30 15:04:58 2021 From: Gergo.Erdi at sc.com (Erdi, Gergo) Date: Thu, 30 Dec 2021 15:04:58 +0000 Subject: Avoiding full laziness xform / floating-out (Re: What's the benefit of taking "do" blocks apart? Is there a way to turn that off?) Message-ID: PUBLIC Ah, at least I've figured out why exactly the simplifier does this (regardless of floating out due to full laziness or not): it is because the definition of the default implementation of (>>) is inlined. I have (>>) defined as such: ma >> mb = ma >>= \_ -> mb So when the simplifier encounters a call site "foo >> bar", it needs to bind "foo" and "bar" to avoid duplication (since "ma" and "mb" could occur multiple times on the right-hand side of (>>)'s definition). This means that the best I can hope for would be to avoid floating out "ma" and "mb", but the variable bindings themselves are unavoidable in the face of inlining (and of course I do want inlining). So I'll need to rethink my approach anyway. -----Original Message----- From: Erdi, Gergo Sent: Thursday, December 30, 2021 12:56 PM To: Matthew Pickering Cc: Joachim Breitner ; ghc-devs at haskell.org Subject: RE: [External] Re: Avoiding full laziness xform / floating-out (Re: What's the benefit of taking "do" blocks apart? Is there a way to turn that off?) PUBLIC Turning on various Opt_D_dump flags, I can see that this tranformation happens in the first iteration of the simplifier. Looking at GHC.Core.Opt.getCoreToDo, I would guess this corresponds to this line: -- initial simplify: mk specialiser happy: minimum effort please runWhen do_presimplify simpl_gently, If I wrote my own optimisation pass, I'm not sure what it could do to inhibit GHC from doing this transformation later on in its pipeline. And I don't want to give up on the simplifier pipeline completely, since there are parts of it that I very much need. One of them is, unfortunately, the specializer itself -- and that comment in getCoreToDo makes me worried that if I somehow managed to get rid of this transformation (or if I skipped the simplifier pipeline completely and just ran the specializer out-of-band), I would break specialization anyway... -----Original Message----- From: Matthew Pickering Sent: Thursday, December 30, 2021 10:42 AM To: Erdi, Gergo Cc: Joachim Breitner ; ghc-devs at haskell.org Subject: [External] Re: Avoiding full laziness xform / floating-out (Re: What's the benefit of taking "do" blocks apart? Is there a way to turn that off?) Hi Gergo, Sounds like you might be better off writing your own optimisation pass rather than relying on making GHC do what you want. Cheers Matt On Thu, Dec 30, 2021 at 9:05 AM Erdi, Gergo via ghc-devs wrote: > > PUBLIC > > Hi Joachim, > > Thanks for the hints! > > > Hi Gergo, > > > > Am Dienstag, dem 28.12.2021 um 15:57 +0000 schrieb Erdi, Gergo via > > ghc- > > devs: > > > PUBLIC > > > > phew > > Yeah obviously I'm sitting here not only adding these tags, but also > coming up with the automated systems and also the company policies > forcing the usage of said systems ;) > > > didn't investigate deeper (maybe if you provide a small example I would), but just from looking at this: > > Unfortunately, it's hard to make a small example because this all > lives in bizarro world where "IO" isn't IO, "String" isn't [Char] etc, > so I can't just take the input program and pass it to vanilla GHC. > > > > > * It is generally preferable to turn local lambda expressions > > into top-level functions. This way, instead of dynamically > > allocating a FUN heap object, it's just a static function. > > > > * sat_sKv is an IO expression? Then it is actually a function in a way > > (taking the "State token" as an argument). So the above applies. > > This is "IO" but not GHC's IO, just another type with an opaque > definition and a Monad instance. There's no reason for GHC to think > that sat_sKv would be a function. > > Of course, sat_sKw is necessarily a function since it is the second > argument to bind. But only now I am noticing that even sat_sKw's > definition includes *yet another* floated-out variable: > > sat_sKv = some complicated expression 1 sat_sKu = some complicated > expression 2 sat_sKw = \_ -> sat_sKu main = bindIO sat_sKv sat_sKw > > Here, I don't see why GHC should think that sat_sKv and sat_sKu are > functions. For example, here's the definition of sat_sKu: > > sat_sKu :: IO () > [LclId] > sat_sKu > = let { > sat_sRy [Occ=Once1] :: String > [LclId] > sat_sRy > = let { > sat_sRx [Occ=Once1] :: String > [LclId] > sat_sRx = unpackCStringUtf8# "\n"# } in > let { > sat_sRw [Occ=Once1] :: String > [LclId] > sat_sRw > = case foobar True of { > False -> unpackCStringUtf8# "False"#; > True -> unpackCStringUtf8# "True"# > } } in > sApp# sat_sRw sat_sRx } in > putStrOut sat_sRy > > Here, putStrOut is so opaque that it doesn't even have a definition, > it is merely a name registered into GHC's name tables. So I really > don't see why this looks "function-y" to GHC. > > The reason this is all a problem for me is because I would like to > then interpret all these let-bindings, including the toplevel ones, as > eagerly defined variables. But then there is of course a difference > between > > main = > let a1 = someIOAction1 > a2 = someOtherIOAction2 > a2' = \_ -> a2 > in bindIO a1 a2' > > and > > main = > let a1 = someIOAction1 > a2 = \_ -> someIOAction2 > in bindIO a1 a2 > > because if the *definition* of "a2" throws (not its action), then the > first version will throw immediately whereas the second will only > throw when "main" is *run*, after running "a1". > > > * I think this is the FloatOut pass. You can turn it out using > > -fno-full-laziness. Not sure if some others passes might > > do similar things, though. > > I tried turning off Opt_FullLaziness, but unfortunately I am still > getting the same result. I was also hopeful when I saw there's an > Opt_FloatIn flag, but alas that doesn't change this behaviour either > (and it's turned on by default anyway...) > > I'll try to make a self-contained minimal example next week. > > Thanks, > Gergo > > -----Original Message----- > From: ghc-devs On Behalf Of Joachim > Breitner > Sent: Wednesday, December 29, 2021 8:39 PM > To: ghc-devs at haskell.org > Subject: [External] Re: What's the benefit of taking "do" blocks apart? Is there a way to turn that off? > > > Hi Gergo, > > Am Dienstag, dem 28.12.2021 um 15:57 +0000 schrieb Erdi, Gergo via > ghc- > devs: > > PUBLIC > > phew > > > I’m seeing ‘do’ blocks getting taking apart into top-level > > definitions, so e.g. > > > > main = do > > some complicated expression 1 > > some complicated expression 2 > > > > is compiled into > > > > sat_sKv = some complicated expression 1 sat_sKw = \_ -> some > > complicated expression 2 main = bindIO sat_sKv sat_sKw > > > > This seems to happen regardless of any common subexpressions, i.e. > > it is not the case that sat_sKv or sat_sKw are used anywhere else. > > > > What is the intended benefit of this floating-out? Is there a > > particular Core-to-Core pass that causes this? Is it possible to > > turn it off? > > > didn’t investigate deeper (maybe if you provide a small example I would), but just from looking at this: > > * It is generally preferable to turn local lambda expressions > into top-level functions. This way, instead of dynamically > allocating a FUN heap object, it’s just a static function. > > * sat_sKv is an IO expression? Then it is actually a function in a way > (taking the “State token” as an argument). So the above applies. > > * I think this is the FloatOut pass. You can turn it out using > -fno-full-laziness. Not sure if some others passes might > do similar things, though. > > Cheers, > Joachim > > > -- > Joachim Breitner > mail at joachim-breitner.de > This email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please delete all copies and notify the sender immediately. You may wish to refer to the incorporation details of Standard Chartered PLC, Standard Chartered Bank and their subsidiaries at https: //www.sc.com/en/our-locations Where you have a Financial Markets relationship with Standard Chartered PLC, Standard Chartered Bank and their subsidiaries (the "Group"), information on the regulatory standards we adhere to and how it may affect you can be found in our Regulatory Compliance Statement at https: //www.sc.com/rcs/ and Regulatory Compliance Disclosures at http: //www.sc.com/rcs/fm Insofar as this communication is not sent by the Global Research team and contains any market commentary, the market commentary has been prepared by the sales and/or trading desk of Standard Chartered Bank or its affiliate. It is not and does not constitute research material, independent research, recommendation or financial advice. Any market commentary is for information purpose only and shall not be relied on for any other purpose and is subject to the relevant disclaimers available at https: //www.sc.com/en/regulatory-disclosures/#market-disclaimer. Insofar as this communication is sent by the Global Research team and contains any research materials prepared by members of the team, the research material is for information purpose only and shall not be relied on for any other purpose, and is subject to the relevant disclaimers available at https: //research.sc.com/research/api/application/static/terms-and-conditions. Insofar as this e-mail contains the term sheet for a proposed transaction, by responding affirmatively to this e-mail, you agree that you have understood the terms and conditions in the attached term sheet and evaluated the merits and risks of the transaction. We may at times also request you to sign the term sheet to acknowledge the same. Please visit https: //www.sc.com/en/regulatory-disclosures/dodd-frank/ for important information with respect to derivative products. From lists at richarde.dev Thu Dec 30 16:46:29 2021 From: lists at richarde.dev (Richard Eisenberg) Date: Thu, 30 Dec 2021 16:46:29 +0000 Subject: convention around pattern synonyms In-Reply-To: References: <010f017e071c9e25-632a3872-29de-4e3a-b0f1-70cdc000c22f-000000@us-east-2.amazonses.com> <010f017e083bbdc6-f5fdb718-cf4a-4e60-9e4c-5cce52e925a0-000000@us-east-2.amazonses.com> Message-ID: <010f017e0c3c5bfc-d0067ada-d408-4594-ba2a-446cd4f13fd1-000000@us-east-2.amazonses.com> I agree that this kind of backward-compatibility pattern synonym is good and shouldn't be prefixed with PS_. But do you have a concrete example of this leakage of an internal GHC type via TH? While I can imagine this happening, I don't know of any examples in practice. Note that even enumeration types (like Role) have separate TH counterparts. Richard > On Dec 29, 2021, at 6:12 PM, Viktor Dukhovni wrote: > > Some "GHC-internal" types leak to users via TH, and their constructors > occasionally pick up new fields, causing breakage downstream. The extra > field often has a sensible default (Nothing, [], ...) and it should be > best practice to rename the constructor when adding the new field, while > replacing the original constructor with a pattern synonym with the "old" > signature. > > data Foo = ... > | NewImprovedMkFoo X Y Z -- was MkFoo Y Z > > pattern MkFoo :: Foo > pattern MkFoo Y Z = NewImprovedMkFoo Nothing Y Z > > When pattern synonyms are used to maintain a backwards-compatible API, > there should of course be no special signalling to differentiate them > from "real" constructors. > > The boundary between "GHC-internal" and external may not always be > obvious, some care is required to reduce leaking breakage via TH. > > -- > Viktor. > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs From ietf-dane at dukhovni.org Thu Dec 30 20:25:26 2021 From: ietf-dane at dukhovni.org (Viktor Dukhovni) Date: Thu, 30 Dec 2021 15:25:26 -0500 Subject: convention around pattern synonyms In-Reply-To: <010f017e0c3c5bfc-d0067ada-d408-4594-ba2a-446cd4f13fd1-000000@us-east-2.amazonses.com> References: <010f017e071c9e25-632a3872-29de-4e3a-b0f1-70cdc000c22f-000000@us-east-2.amazonses.com> <010f017e083bbdc6-f5fdb718-cf4a-4e60-9e4c-5cce52e925a0-000000@us-east-2.amazonses.com> <010f017e0c3c5bfc-d0067ada-d408-4594-ba2a-446cd4f13fd1-000000@us-east-2.amazonses.com> Message-ID: On Thu, Dec 30, 2021 at 04:46:29PM +0000, Richard Eisenberg wrote: > I agree that this kind of backward-compatibility pattern synonym is > good and shouldn't be prefixed with PS_. > > But do you have a concrete example of this leakage of an internal GHC > type via TH? While I can imagine this happening, I don't know of any > examples in practice. Note that even enumeration types (like Role) > have separate TH counterparts. Perhaps my assumption that TH types directly mirror the internal AST is not correct... A recent user-visible change is in `ConP` https://github.com/nikita-volkov/contravariant-extras/pull/9 -- Viktor.