From juhpetersen at gmail.com Sat Nov 6 04:59:41 2021 From: juhpetersen at gmail.com (Jens Petersen) Date: Sat, 6 Nov 2021 12:59:41 +0800 Subject: [ANNOUNCE] GHC 9.2.1 now available In-Reply-To: <87lf2bu98a.fsf@smart-cactus.org> References: <87lf2bu98a.fsf@smart-cactus.org> Message-ID: On Fri, 29 Oct 2021 at 23:54, Ben Gamari wrote: > https://downloads.haskell.org/ghc/9.2.1 > If you are on Fedora you can now install ghc-9.2.1 with: sudo dnf --enablerepo=updates-testing-modular module install ghc:9.2/default Cheers, Jens -------------- next part -------------- An HTML attachment was scrubbed... URL: From christiaan.baaij at gmail.com Sun Nov 7 21:07:49 2021 From: christiaan.baaij at gmail.com (Christiaan Baaij) Date: Sun, 7 Nov 2021 22:07:49 +0100 Subject: Alpha-equivalence for recursive let-bindings Message-ID: Hi list, I was looking at the `Eq (DeBruijn CoreExpr)` instance and I noticed that the types of recursive let-bindings aren't checked for alpha-equivalence: https://gitlab.haskell.org/ghc/ghc/-/blob/master/compiler/GHC/Core/Map/Expr.hs#L166-174 go (Let (Rec ps1) e1) (Let (Rec ps2) e2) = equalLength ps1 ps2 && D env1' rs1 == D env2' rs2 && D env1' e1 == D env2' e2 where (bs1,rs1) = unzip ps1 (bs2,rs2) = unzip ps2 env1' = extendCMEs env1 bs1 env2' = extendCMEs env2 bs2 But doesn't that mean that: let (x :: Int) = x in x and let (y :: Bool) = y in y are considered alpha-equivalent? If that is the case, then I think that's wrong. Agree? I understand that you don't have to check types for non-recursive let-bindings: when the RHSs match, the types must be the same. -- Christiaan -------------- next part -------------- An HTML attachment was scrubbed... URL: From simonpj at microsoft.com Mon Nov 8 12:02:39 2021 From: simonpj at microsoft.com (Simon Peyton Jones) Date: Mon, 8 Nov 2021 12:02:39 +0000 Subject: Alpha-equivalence for recursive let-bindings In-Reply-To: References: Message-ID: Huh! Dead right! Would you like to: * Open a ticket (you can use the text from this email) * Submit a MR? On the MR, * Add a Note that again gives your killer example; and mention why we don't need the check for NonRec * Worth also pointing out that letrec { x = e1; y = e2 } in b is NOT considered equal to letrec { y = e1; x = e1 } in b. Nor are let x=e1 in let y = e2 in b considered equal to let y = e1 in let x = e1 in b. This is fine; but worth pointing out. Thanks for pointing this out! Simon PS: I am leaving Microsoft at the end of November 2021, at which point simonpj at microsoft.com will cease to work. Use simon.peytonjones at gmail.com instead. (For now, it just forwards to simonpj at microsoft.com.) From: ghc-devs On Behalf Of Christiaan Baaij Sent: 07 November 2021 21:08 To: ghc-devs Subject: Alpha-equivalence for recursive let-bindings Hi list, I was looking at the `Eq (DeBruijn CoreExpr)` instance and I noticed that the types of recursive let-bindings aren't checked for alpha-equivalence: https://gitlab.haskell.org/ghc/ghc/-/blob/master/compiler/GHC/Core/Map/Expr.hs#L166-174 go (Let (Rec ps1) e1) (Let (Rec ps2) e2) = equalLength ps1 ps2 && D env1' rs1 == D env2' rs2 && D env1' e1 == D env2' e2 where (bs1,rs1) = unzip ps1 (bs2,rs2) = unzip ps2 env1' = extendCMEs env1 bs1 env2' = extendCMEs env2 bs2 But doesn't that mean that: let (x :: Int) = x in x and let (y :: Bool) = y in y are considered alpha-equivalent? If that is the case, then I think that's wrong. Agree? I understand that you don't have to check types for non-recursive let-bindings: when the RHSs match, the types must be the same. -- Christiaan -------------- next part -------------- An HTML attachment was scrubbed... URL: From christiaan.baaij at gmail.com Mon Nov 8 13:00:48 2021 From: christiaan.baaij at gmail.com (Christiaan Baaij) Date: Mon, 8 Nov 2021 14:00:48 +0100 Subject: Alpha-equivalence for recursive let-bindings In-Reply-To: References: Message-ID: Alright, I opened an issue, https://gitlab.haskell.org/ghc/ghc/-/issues/20641, and I'll make an MR later. Any objections if I change the implementation of GHC.Core.Utils.eqExpr :: InScopeSet -> Expr -> Expr -> Bool https://gitlab.haskell.org/ghc/ghc/-/blob/master/compiler/GHC/Core/Utils.hs#L2124 to eqExpr _ e1 e2 = deBruijnize e1 == deBruijnize e2 and at the same time mark it deprecated telling people to use deBruijnize? (I don't want to remove it since GHC API users might be using it) `eqExpr` also does alpha-equivalence on CoreExpr, and has the same mistake w.r.t. alpha-equivalence as the `Eq (DeBruijn CoreExpr)` instance -- Christiaan On Mon, 8 Nov 2021 at 13:02, Simon Peyton Jones wrote: > Huh! Dead right! > > > > Would you like to: > > - Open a ticket (you can use the text from this email) > - Submit a MR? > > > > On the MR, > > - Add a Note that again gives your killer example; and mention why we > don’t need the check for NonRec > - Worth also pointing out that letrec { x = e1; y = e2 } in b is NOT > considered equal to letrec { y = e1; x = e1 } in b. Nor are let x=e1 in > let y = e2 in b considered equal to let y = e1 in let x = e1 in b. > This is fine; but worth pointing out. > > > > Thanks for pointing this out! > > > > Simon > > > > PS: I am leaving Microsoft at the end of November 2021, at which point > simonpj at microsoft.com will cease to work. Use simon.peytonjones at gmail.com > instead. (For now, it just forwards to simonpj at microsoft.com.) > > > > *From:* ghc-devs *On Behalf Of *Christiaan > Baaij > *Sent:* 07 November 2021 21:08 > *To:* ghc-devs > *Subject:* Alpha-equivalence for recursive let-bindings > > > > Hi list, > > > > I was looking at the `Eq (DeBruijn CoreExpr)` instance and I noticed that > the types of recursive let-bindings aren't checked for alpha-equivalence: > > > > > https://gitlab.haskell.org/ghc/ghc/-/blob/master/compiler/GHC/Core/Map/Expr.hs#L166-174 > > > > > go (Let (Rec ps1) e1) (Let (Rec ps2) e2) > = equalLength ps1 ps2 > && D env1' rs1 == D env2' rs2 > && D env1' e1 == D env2' e2 > where > (bs1,rs1) = unzip ps1 > (bs2,rs2) = unzip ps2 > env1' = extendCMEs env1 bs1 > env2' = extendCMEs env2 bs2 > > > > But doesn't that mean that: > > let (x :: Int) = x in x > > and > > let (y :: Bool) = y in y > > are considered alpha-equivalent? > > If that is the case, then I think that's wrong. Agree? > > I understand that you don't have to check types for non-recursive > let-bindings: when the RHSs match, the types must be the same. > > > > -- Christiaan > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From benjamin.redelings at gmail.com Mon Nov 8 13:12:01 2021 From: benjamin.redelings at gmail.com (Benjamin Redelings) Date: Mon, 8 Nov 2021 08:12:01 -0500 Subject: Output language of typechecking pass? In-Reply-To: <010f017cc31f63bb-475c1cf3-17f9-4904-9d14-45fbe88cbe51-000000@us-east-2.amazonses.com> References: <010f017cc31f63bb-475c1cf3-17f9-4904-9d14-45fbe88cbe51-000000@us-east-2.amazonses.com> Message-ID: <27aaf5dc-8a98-228d-bf41-c32784517f95@gmail.com> Hi, >> Questions: >> >> 1. It seems like this separation is actually necessary, in order to apply generalization only to let arguments written by the programmer, and not to let bindings introduced during desugaring. Is that right? > I don't think so. That is, if we did it all in one pass, I still think we could get generalization right. I guess I asked this question wrong.  I mean to say, if we did the two passes in the reverse order (desugaring first, followed by typechecking), that would not work, right? As the wiki says: "This late desugaring is somewhat unusual. It is much more common to desugar the program before typechecking, or renaming, because that presents the renamer and typechecker with a much smaller language to deal with. However, GHC's organisation means that * error messages can display precisely the syntax that the user wrote; and * desugaring is not required to preserve type-inference properties. " >> 2. Does the output of type checking contain type lambdas? > Yes. See below. > >> 3. Does the type checking pass determine where to add dictionary arguments? > Yes. See below. > >> 4. Are there any other resources I should be looking at? > Yes. You want to enable -fprint-typechecker-elaboration (and possible -fprint-explicit-coercions). With the former, you get to see all this stuff you're looking for. It's normally suppressed so that the output resembles the user's code. > > I hope this helps! > Richard Hmm... so, I think I see how this works now.  I don't think '-fprint-explicit-coercions' does anything here though. $ ghc -ddump-tc Test2.hs -fprint-typechecker-elaboration ... AbsBinds [a_a2hp] [$dNum_a2hB]   {Exports: [g <= g_a2hz                wrap: <>]    Exported types: g :: forall a. Num a => a -> a -> a                    [LclId]    Binds: g x_aYk y_aYl = (y_aYl * x_aYk) + 1    Evidence: [EvBinds{[W] $dNum_a2hs = $dNum_a2hq                       [W] $dNum_a2hw = $dNum_a2hq                       [W] $dNum_a2hq = $dNum_a2hB}]} ... The type and dictionary arguments are visible here (along with the evidence bindings), but type and dictionary applications are only visible if you use -ddump-tc-ast, which is a lot more verbose.  (I don't think there is another flag that shows these applications?)  Since I didn't initially know what "evidence" was, and there is nothing to say that a_a2hp is a type lambda argument, this was pretty opaque until I managed to read the tc-ast and the light went on. I can see now that the type and dictionary arguments are added by annotating the AST. Is there anywhere on the GHC wiki that explains how to interpret this output, and says that the type and dictionary applications ARE there, just not shown by '-ddump-tc'? Perhaps it would be helpful to add some basic description of what comes out of the typechecker to a page like this one? (below) https://gitlab.haskell.org/ghc/ghc/-/wikis/commentary/compiler/hsc-main -BenRI -------------- next part -------------- An HTML attachment was scrubbed... URL: From simonpj at microsoft.com Mon Nov 8 13:35:24 2021 From: simonpj at microsoft.com (Simon Peyton Jones) Date: Mon, 8 Nov 2021 13:35:24 +0000 Subject: Output language of typechecking pass? In-Reply-To: <27aaf5dc-8a98-228d-bf41-c32784517f95@gmail.com> References: <010f017cc31f63bb-475c1cf3-17f9-4904-9d14-45fbe88cbe51-000000@us-east-2.amazonses.com> <27aaf5dc-8a98-228d-bf41-c32784517f95@gmail.com> Message-ID: Is there anywhere on the GHC wiki that explains how to interpret this output, and says that the type and dictionary applications ARE there, just not shown by '-ddump-tc'? Perhaps it would be helpful to add some basic description of what comes out of the typechecker to a page like this one? (below) https://gitlab.haskell.org/ghc/ghc/-/wikis/commentary/compiler/hsc-main Yes it would! Would you care to start such a wiki page (a new one; don't just clutter up the one you point to)? You can write down what you know. Don't worry if you aren't 100% sure - we can correct it. And if you outright don't know, leave a "What should I say here?" note. "This late desugaring is somewhat unusual. It is much more common to desugar the program before typechecking, or renaming, because that presents the renamer and typechecker with a much smaller language to deal with. However, GHC's organisation means that This note is now slightly out of date. We are now, very carefully, doing some desugaring before typechecking. See * Note [Handling overloaded and rebindable constructs] in GHC.Rename.Expr * Note [Rebindable syntax and HsExpansion] in GHC.Hs.Expr You can and should point to these and similar Notes from the wiki page you write. Indeed there may be some part of what you write that would be better framed as Note in GHC's source code. Thanks! Simon PS: I am leaving Microsoft at the end of November 2021, at which point simonpj at microsoft.com will cease to work. Use simon.peytonjones at gmail.com instead. (For now, it just forwards to simonpj at microsoft.com.) From: ghc-devs On Behalf Of Benjamin Redelings Sent: 08 November 2021 13:12 To: Richard Eisenberg Cc: ghc-devs at haskell.org Subject: Re: Output language of typechecking pass? Hi, Questions: 1. It seems like this separation is actually necessary, in order to apply generalization only to let arguments written by the programmer, and not to let bindings introduced during desugaring. Is that right? I don't think so. That is, if we did it all in one pass, I still think we could get generalization right. I guess I asked this question wrong. I mean to say, if we did the two passes in the reverse order (desugaring first, followed by typechecking), that would not work, right? As the wiki says: "This late desugaring is somewhat unusual. It is much more common to desugar the program before typechecking, or renaming, because that presents the renamer and typechecker with a much smaller language to deal with. However, GHC's organisation means that * error messages can display precisely the syntax that the user wrote; and * desugaring is not required to preserve type-inference properties. " 2. Does the output of type checking contain type lambdas? Yes. See below. 3. Does the type checking pass determine where to add dictionary arguments? Yes. See below. 4. Are there any other resources I should be looking at? Yes. You want to enable -fprint-typechecker-elaboration (and possible -fprint-explicit-coercions). With the former, you get to see all this stuff you're looking for. It's normally suppressed so that the output resembles the user's code. I hope this helps! Richard Hmm... so, I think I see how this works now. I don't think '-fprint-explicit-coercions' does anything here though. $ ghc -ddump-tc Test2.hs -fprint-typechecker-elaboration ... AbsBinds [a_a2hp] [$dNum_a2hB] {Exports: [g <= g_a2hz wrap: <>] Exported types: g :: forall a. Num a => a -> a -> a [LclId] Binds: g x_aYk y_aYl = (y_aYl * x_aYk) + 1 Evidence: [EvBinds{[W] $dNum_a2hs = $dNum_a2hq [W] $dNum_a2hw = $dNum_a2hq [W] $dNum_a2hq = $dNum_a2hB}]} ... The type and dictionary arguments are visible here (along with the evidence bindings), but type and dictionary applications are only visible if you use -ddump-tc-ast, which is a lot more verbose. (I don't think there is another flag that shows these applications?) Since I didn't initially know what "evidence" was, and there is nothing to say that a_a2hp is a type lambda argument, this was pretty opaque until I managed to read the tc-ast and the light went on. I can see now that the type and dictionary arguments are added by annotating the AST. Is there anywhere on the GHC wiki that explains how to interpret this output, and says that the type and dictionary applications ARE there, just not shown by '-ddump-tc'? Perhaps it would be helpful to add some basic description of what comes out of the typechecker to a page like this one? (below) https://gitlab.haskell.org/ghc/ghc/-/wikis/commentary/compiler/hsc-main -BenRI -------------- next part -------------- An HTML attachment was scrubbed... URL: From nr at cs.tufts.edu Mon Nov 8 18:42:38 2021 From: nr at cs.tufts.edu (Norman Ramsey) Date: Mon, 08 Nov 2021 13:42:38 -0500 Subject: advice on minor cleanups? Message-ID: <20211108184238.586362C26F1@homedog.cs.tufts.edu> All, It appears that `runHscBackendPhase` returns a tuple that include an element of type `Maybe Linkable` but that in practice is always `Nothing`. I'm thinking of removing this element from the tuple, which would also entail simplifying the type of value constructor `T_HscBackend` for the `TPhase` type. Simplifying the code seems like a good thing, but perhaps there is a reason to leave well enough alone? Please advise me. Norman From ben at smart-cactus.org Mon Nov 8 18:53:19 2021 From: ben at smart-cactus.org (Ben Gamari) Date: Mon, 08 Nov 2021 13:53:19 -0500 Subject: advice on minor cleanups? In-Reply-To: <20211108184238.586362C26F1@homedog.cs.tufts.edu> References: <20211108184238.586362C26F1@homedog.cs.tufts.edu> Message-ID: <87sfw6xz02.fsf@smart-cactus.org> Norman Ramsey writes: > All, > > It appears that `runHscBackendPhase` returns a tuple that include an > element of type `Maybe Linkable` but that in practice is always > `Nothing`. I'm thinking of removing this element from the tuple, > which would also entail simplifying the type of value constructor > `T_HscBackend` for the `TPhase` type. Simplifying the code seems like > a good thing, but perhaps there is a reason to leave well enough > alone? Please advise me. > Hi Norman, In cases like this I generally just try to push the change through, open a merge request, and note my uncertainty in the merge request description, and let review and CI sort it out. Cheers, - Ben -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From zubin at well-typed.com Mon Nov 8 18:59:43 2021 From: zubin at well-typed.com (Zubin Duggal) Date: Tue, 9 Nov 2021 00:29:43 +0530 Subject: advice on minor cleanups? In-Reply-To: <20211108184238.586362C26F1@homedog.cs.tufts.edu> References: <20211108184238.586362C26F1@homedog.cs.tufts.edu> Message-ID: <20211108185943.fgj4exnbxqmqvxoi@zubin-msi> Looking at the code, it seems like it does return an actual `Linkable` when we are in interpreted mode: runHscBackendPhase pipe_env hsc_env mod_name src_flavour location result = do ... -> case backend dflags of Interpreter -> do ... return ([], final_iface, Just linkable, panic "interpreter") From nr at cs.tufts.edu Mon Nov 8 19:54:18 2021 From: nr at cs.tufts.edu (Norman Ramsey) Date: Mon, 08 Nov 2021 14:54:18 -0500 Subject: advice on minor cleanups? In-Reply-To: <20211108185943.fgj4exnbxqmqvxoi@zubin-msi> (sfid-H-20211108-140005-+38.75-1@multi.osbf.lua) References: <20211108184238.586362C26F1@homedog.cs.tufts.edu> <20211108185943.fgj4exnbxqmqvxoi@zubin-msi> (sfid-H-20211108-140005-+38.75-1@multi.osbf.lua) Message-ID: <20211108195418.DA6512C26F1@homedog.cs.tufts.edu> > Looking at the code, it seems like it does return an actual `Linkable` > when we are in interpreted mode: Oy. I don't know how I missed that. Thanks. Norman From hecate at glitchbra.in Mon Nov 8 23:18:08 2021 From: hecate at glitchbra.in (=?UTF-8?Q?H=c3=a9cate?=) Date: Tue, 9 Nov 2021 00:18:08 +0100 Subject: [RFC] Split the GHC.* hierarchy from `base` in its own package, `ghc-base` Message-ID: <4c6af6e2-b621-1379-3a14-65b8034c0dc4@glitchbra.in> Hello everyone, after a chat with Ben and seeing the idea thrown around, I have opened an RFC ticket to explore designs regarding a split of the GHC.* hierarchy from base, into its own package. The RFC is here: https://gitlab.haskell.org/ghc/ghc/-/issues/20647 Please do keep the comments in the ticket so that this discussion may be better archived. Cheers! -- Hécate ✨ 🐦: @TechnoEmpress IRC: Hecate WWW: https://glitchbra.in RUN: BSD From sgraf1337 at gmail.com Tue Nov 9 13:17:47 2021 From: sgraf1337 at gmail.com (Sebastian Graf) Date: Tue, 9 Nov 2021 14:17:47 +0100 Subject: Case split uncovered patterns in warnings or not? Message-ID: Hi Devs, In https://gitlab.haskell.org/ghc/ghc/-/issues/20642 we saw that GHC >= 8.10 outputs pattern match warnings a little differently than it used to. Example from there: {-# OPTIONS_GHC -Wincomplete-uni-patterns #-}foo :: [a] -> [a]foo [] = []foo xs = ys where (_, ys@(_:_)) = splitAt 0 xsmain :: IO ()main = putStrLn $ foo $ "Hello, coverage checker!" Instead of saying ListPair.hs:7:3: warning: [-Wincomplete-uni-patterns] Pattern match(es) are non-exhaustive In a pattern binding: Patterns not matched: (_, []) We now say ListPair.hs:7:3: warning: [-Wincomplete-uni-patterns] Pattern match(es) are non-exhaustive In a pattern binding: Patterns of type ‘([a], [a])’ not matched: ([], []) ((_:_), []) E.g., newer versions do (one) case split on pattern variables that haven't even been scrutinised by the pattern match. That amounts to quantitatively more pattern suggestions and for each variable a list of constructors that could be matched on. The motivation for the change is outlined in https://gitlab.haskell.org/ghc/ghc/-/issues/20642#note_390110, but I could easily be swayed not to do the case split. Which arguably is less surprising, as Andreas Abel points out. Considering the other examples from my post, which would you prefer? Cheers, Sebastian -------------- next part -------------- An HTML attachment was scrubbed... URL: From lists at richarde.dev Wed Nov 10 03:44:50 2021 From: lists at richarde.dev (Richard Eisenberg) Date: Wed, 10 Nov 2021 03:44:50 +0000 Subject: Case split uncovered patterns in warnings or not? In-Reply-To: References: Message-ID: <010f017d07f2c3df-5a0fe22c-63b6-4191-97b3-2ea484ca6a4a-000000@us-east-2.amazonses.com> Maybe the answer should depend on whether the scrutinee has already been forced. The new output ("We now say", below) offers up patterns that will change the strictness behavior of the code. The old output did not. Reading the link below, I see that, previously, there was an inconsistency with -XEmptyCase, which *did* unroll one level of constructor. But maybe that made sense because -XEmptyCase is strict (unlike normal case). I'm just saying this because I think listing the constructors in the -XEmptyCase case is a good practice, but otherwise I think they're clutterful... and strictness is a perhaps more principled way of making this distinction. Richard > On Nov 9, 2021, at 8:17 AM, Sebastian Graf wrote: > > Hi Devs, > > In https://gitlab.haskell.org/ghc/ghc/-/issues/20642 we saw that GHC >= 8.10 outputs pattern match warnings a little differently than it used to. Example from there: > > {-# OPTIONS_GHC -Wincomplete-uni-patterns #-} > > foo :: [a] -> [a] > foo [] = [] > foo xs = ys > where > (_, ys@(_:_)) = splitAt 0 xs > > main :: IO () > main = putStrLn $ foo $ "Hello, coverage checker!" > Instead of saying > > ListPair.hs:7:3: warning: [-Wincomplete-uni-patterns] > Pattern match(es) are non-exhaustive > In a pattern binding: Patterns not matched: (_, []) > > We now say > > ListPair.hs:7:3: warning: [-Wincomplete-uni-patterns] > Pattern match(es) are non-exhaustive > In a pattern binding: > Patterns of type ‘([a], [a])’ not matched: > ([], []) > ((_:_), []) > > E.g., newer versions do (one) case split on pattern variables that haven't even been scrutinised by the pattern match. That amounts to quantitatively more pattern suggestions and for each variable a list of constructors that could be matched on. > The motivation for the change is outlined in https://gitlab.haskell.org/ghc/ghc/-/issues/20642#note_390110 , but I could easily be swayed not to do the case split. Which arguably is less surprising, as Andreas Abel points out. > > Considering the other examples from my post, which would you prefer? > > Cheers, > Sebastian > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs -------------- next part -------------- An HTML attachment was scrubbed... URL: From sgraf1337 at gmail.com Wed Nov 10 07:28:42 2021 From: sgraf1337 at gmail.com (Sebastian Graf) Date: Wed, 10 Nov 2021 07:28:42 +0000 Subject: Case split uncovered patterns in warnings or not? In-Reply-To: <010f017d07f2c3df-5a0fe22c-63b6-4191-97b3-2ea484ca6a4a-000000@us-east-2.amazonses.com> References: <010f017d07f2c3df-5a0fe22c-63b6-4191-97b3-2ea484ca6a4a-000000@us-east-2.amazonses.com> Message-ID: I agree in principle, but then what about data types with strict fields? E.g. data SMaybe a = SNothing | SJust !a f :: SMaybe Bool -> () f SNothing = () Today, we'd suggest `SJust _`. But the checker can't differentiate between evaluation done by a pattern-match of the user vs. something like a strict field that was unlifted to begin with. So we'd suggest `SJust True` and `SJust False`. Similarly, we'd case split unlifted data types by default, but not lifted data types. I think I can easily make the whole function (`GHC.HsToCore.Pmc.Solver.generateInhabitingPatterns`) dependent on whether it's called from an EmptyCase or not, to recover the behavior pre-8.10. But actually I had hoped we can come up with something more general and less ad-hoc than the behavior of 8.8. Maybe there isn't and 8.8 already lived in the sweet spot. ------ Originalnachricht ------ Von: "Richard Eisenberg" An: "Sebastian Graf" Cc: "ghc-devs" Gesendet: 10.11.2021 04:44:50 Betreff: Re: Case split uncovered patterns in warnings or not? >Maybe the answer should depend on whether the scrutinee has already >been forced. The new output ("We now say", below) offers up patterns >that will change the strictness behavior of the code. The old output >did not. > >Reading the link below, I see that, previously, there was an >inconsistency with -XEmptyCase, which *did* unroll one level of >constructor. But maybe that made sense because -XEmptyCase is strict >(unlike normal case). > >I'm just saying this because I think listing the constructors in the >-XEmptyCase case is a good practice, but otherwise I think they're >clutterful... and strictness is a perhaps more principled way of making >this distinction. > >Richard > >>On Nov 9, 2021, at 8:17 AM, Sebastian Graf >>wrote: >> >>Hi Devs, >> >>In https://gitlab.haskell.org/ghc/ghc/-/issues/20642 we saw that GHC >> >= 8.10 outputs pattern match warnings a little differently than it >>used to. Example from there: >> >>{-# OPTIONS_GHC -Wincomplete-uni-patterns #-}foo::[a]->[a]foo[]=[]fooxs=yswhere(_,ys@(_:_))=splitAt0xsmain::IO()main=putStrLn$foo$"Hello, coverage checker!" >>Instead of saying >> >>ListPair.hs:7:3: warning: [-Wincomplete-uni-patterns] Pattern match(es) are non-exhaustive In a pattern binding: Patterns not matched: (_, []) >>We now say >> >>ListPair.hs:7:3: warning: [-Wincomplete-uni-patterns] Pattern match(es) are non-exhaustive In a pattern binding: Patterns of type ‘([a], [a])’ not matched: ([], []) ((_:_), []) >>E.g., newer versions do (one) case split on pattern variables that >>haven't even been scrutinised by the pattern match. That amounts to >>quantitatively more pattern suggestions and for each variable a list >>of constructors that could be matched on. >>The motivation for the change is outlined in >>https://gitlab.haskell.org/ghc/ghc/-/issues/20642#note_390110, but I >>could easily be swayed not to do the case split. Which arguably is >>less surprising, as Andreas Abel points out. >> >>Considering the other examples from my post, which would you prefer? >> >>Cheers, >>Sebastian >>_______________________________________________ >>ghc-devs mailing list >>ghc-devs at haskell.org >>http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > -------------- next part -------------- An HTML attachment was scrubbed... URL: From arnaud.spiwack at tweag.io Wed Nov 10 09:13:58 2021 From: arnaud.spiwack at tweag.io (Spiwack, Arnaud) Date: Wed, 10 Nov 2021 10:13:58 +0100 Subject: Case split uncovered patterns in warnings or not? In-Reply-To: References: <010f017d07f2c3df-5a0fe22c-63b6-4191-97b3-2ea484ca6a4a-000000@us-east-2.amazonses.com> Message-ID: > But actually I had hoped we can come up with something more general and less ad-hoc than the behavior of 8.8. Maybe there isn't and 8.8 already lived in the sweet spot. I think that it's quite fine (maybe desirable even) for error and warning messages to be ad hoc. On Wed, Nov 10, 2021 at 8:29 AM Sebastian Graf wrote: > I agree in principle, but then what about data types with strict fields? > E.g. > > data SMaybe a = SNothing | SJust !a > > f :: SMaybe Bool -> () > f SNothing = () > > Today, we'd suggest `SJust _`. > But the checker can't differentiate between evaluation done by a > pattern-match of the user vs. something like a strict field that was > unlifted to begin with. > So we'd suggest `SJust True` and `SJust False`. > > Similarly, we'd case split unlifted data types by default, but not lifted > data types. > > I think I can easily make the whole function > (`GHC.HsToCore.Pmc.Solver.generateInhabitingPatterns`) dependent on whether > it's called from an EmptyCase or not, to recover the behavior pre-8.10. > But actually I had hoped we can come up with something more general and > less ad-hoc than the behavior of 8.8. Maybe there isn't and 8.8 already > lived in the sweet spot. > > ------ Originalnachricht ------ > Von: "Richard Eisenberg" > An: "Sebastian Graf" > Cc: "ghc-devs" > Gesendet: 10.11.2021 04:44:50 > Betreff: Re: Case split uncovered patterns in warnings or not? > > Maybe the answer should depend on whether the scrutinee has already been > forced. The new output ("We now say", below) offers up patterns that will > change the strictness behavior of the code. The old output did not. > > Reading the link below, I see that, previously, there was an inconsistency > with -XEmptyCase, which *did* unroll one level of constructor. But maybe > that made sense because -XEmptyCase is strict (unlike normal case). > > I'm just saying this because I think listing the constructors in the > -XEmptyCase case is a good practice, but otherwise I think they're > clutterful... and strictness is a perhaps more principled way of making > this distinction. > > Richard > > On Nov 9, 2021, at 8:17 AM, Sebastian Graf wrote: > > Hi Devs, > > In https://gitlab.haskell.org/ghc/ghc/-/issues/20642 we saw that GHC >= > 8.10 outputs pattern match warnings a little differently than it used to. > Example from there: > > {-# OPTIONS_GHC -Wincomplete-uni-patterns #-}foo :: [a] -> [a]foo [] = []foo xs = ys where (_, ys@(_:_)) = splitAt 0 xsmain :: IO ()main = putStrLn $ foo $ "Hello, coverage checker!" > > Instead of saying > > ListPair.hs:7:3: warning: [-Wincomplete-uni-patterns] Pattern match(es) are non-exhaustive In a pattern binding: Patterns not matched: (_, []) > > We now say > > ListPair.hs:7:3: warning: [-Wincomplete-uni-patterns] Pattern match(es) are non-exhaustive In a pattern binding: Patterns of type ‘([a], [a])’ not matched: ([], []) ((_:_), []) > > E.g., newer versions do (one) case split on pattern variables that haven't > even been scrutinised by the pattern match. That amounts to quantitatively > more pattern suggestions and for each variable a list of constructors that > could be matched on. > The motivation for the change is outlined in > https://gitlab.haskell.org/ghc/ghc/-/issues/20642#note_390110, but I > could easily be swayed not to do the case split. Which arguably is less > surprising, as Andreas Abel points out. > > Considering the other examples from my post, which would you prefer? > > Cheers, > Sebastian > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > > > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > -------------- next part -------------- An HTML attachment was scrubbed... URL: From oleg.grenrus at iki.fi Wed Nov 10 09:48:14 2021 From: oleg.grenrus at iki.fi (Oleg Grenrus) Date: Wed, 10 Nov 2021 11:48:14 +0200 Subject: Case split uncovered patterns in warnings or not? In-Reply-To: References: Message-ID: <86dabd42-cde3-08ad-68dd-5db9963f194a@iki.fi> It should not. Not even when forced. I have seen an `Integer` constructors presented to me, for example:     module Ex where         foo :: Bool -> Integer -> Integer     foo True i = i With GHC-8.8 the warning is good:     % ghci-8.8.4 -Wall Ex.hs     GHCi, version 8.8.4: https://www.haskell.org/ghc/  :? for help     Loaded GHCi configuration from /home/phadej/.ghci     [1 of 1] Compiling Ex               ( Ex.hs, interpreted )         Ex.hs:4:1: warning: [-Wincomplete-patterns]         Pattern match(es) are non-exhaustive         In an equation for ‘foo’: Patterns not matched: False _       |     4 | foo True i = i       | ^^^^^^^^^^^^^^ With GHC-8.10 is straight up awful. I'm glad I don't have to explain it to any beginner, or person who don't know how Integer is implemented. (9.2 is about as bad too).     % ghci-8.10.4 -Wall Ex.hs     GHCi, version 8.10.4: https://www.haskell.org/ghc/  :? for help     Loaded GHCi configuration from /home/phadej/.ghci     [1 of 1] Compiling Ex               ( Ex.hs, interpreted )         Ex.hs:4:1: warning: [-Wincomplete-patterns]         Pattern match(es) are non-exhaustive         In an equation for ‘foo’:             Patterns not matched:                 False (integer-gmp-1.0.3.0:GHC.Integer.Type.S# _)                 False (integer-gmp-1.0.3.0:GHC.Integer.Type.Jp# _)                 False (integer-gmp-1.0.3.0:GHC.Integer.Type.Jn# _)       |     4 | foo True i = i       | ^^^ - Oleg On 9.11.2021 15.17, Sebastian Graf wrote: > Hi Devs, > > In https://gitlab.haskell.org/ghc/ghc/-/issues/20642 > we saw that GHC >= > 8.10 outputs pattern match warnings a little differently than it used > to. Example from there: > > {-# OPTIONS_GHC -Wincomplete-uni-patterns #-} > > foo :: [a] -> [a] > foo [] = [] > foo xs = ys > where > (_, ys@(_:_)) = splitAt 0 xs > > main :: IO () > main = putStrLn $ foo $ "Hello, coverage checker!" > Instead of saying > > ListPair.hs:7:3: warning: [-Wincomplete-uni-patterns] > Pattern match(es) are non-exhaustive > In a pattern binding: Patterns not matched: (_, []) > > We now say > > ListPair.hs:7:3: warning: [-Wincomplete-uni-patterns] > Pattern match(es) are non-exhaustive > In a pattern binding: > Patterns of type ‘([a], [a])’ not matched: > ([], []) > ((_:_), []) > > E.g., newer versions do (one) case split on pattern variables that > haven't even been scrutinised by the pattern match. That amounts to > quantitatively more pattern suggestions and for each variable a list > of constructors that could be matched on. > The motivation for the change is outlined in > https://gitlab.haskell.org/ghc/ghc/-/issues/20642#note_390110 > , but I > could easily be swayed not to do the case split. Which arguably is > less surprising, as Andreas Abel points out. > > Considering the other examples from my post, which would you prefer? > > Cheers, > Sebastian > > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs -------------- next part -------------- An HTML attachment was scrubbed... URL: From vladislav at serokell.io Wed Nov 10 09:51:03 2021 From: vladislav at serokell.io (Vladislav Zavialov) Date: Wed, 10 Nov 2021 12:51:03 +0300 Subject: Case split uncovered patterns in warnings or not? In-Reply-To: <86dabd42-cde3-08ad-68dd-5db9963f194a@iki.fi> References: <86dabd42-cde3-08ad-68dd-5db9963f194a@iki.fi> Message-ID: <3A5391F1-7852-4592-B33B-B8DB008848DD@serokell.io> Integer is an interesting example. I think it reveals another issue: exhaustiveness checking should account for abstract data types. If the constructors are not exported, do not case split. - Vlad > On 10 Nov 2021, at 12:48, Oleg Grenrus wrote: > > It should not. Not even when forced. > > I have seen an `Integer` constructors presented to me, for example: > > module Ex where > > foo :: Bool -> Integer -> Integer > foo True i = i > > With GHC-8.8 the warning is good: > > % ghci-8.8.4 -Wall Ex.hs > GHCi, version 8.8.4: https://www.haskell.org/ghc/ :? for help > Loaded GHCi configuration from /home/phadej/.ghci > [1 of 1] Compiling Ex ( Ex.hs, interpreted ) > > Ex.hs:4:1: warning: [-Wincomplete-patterns] > Pattern match(es) are non-exhaustive > In an equation for ‘foo’: Patterns not matched: False _ > | > 4 | foo True i = i > | ^^^^^^^^^^^^^^ > > With GHC-8.10 is straight up awful. > I'm glad I don't have to explain it to any beginner, > or person who don't know how Integer is implemented. > (9.2 is about as bad too). > > % ghci-8.10.4 -Wall Ex.hs > GHCi, version 8.10.4: https://www.haskell.org/ghc/ :? for help > Loaded GHCi configuration from /home/phadej/.ghci > [1 of 1] Compiling Ex ( Ex.hs, interpreted ) > > Ex.hs:4:1: warning: [-Wincomplete-patterns] > Pattern match(es) are non-exhaustive > In an equation for ‘foo’: > Patterns not matched: > False (integer-gmp-1.0.3.0:GHC.Integer.Type.S# _) > False (integer-gmp-1.0.3.0:GHC.Integer.Type.Jp# _) > False (integer-gmp-1.0.3.0:GHC.Integer.Type.Jn# _) > | > 4 | foo True i = i > | ^^^ > > - Oleg > > > On 9.11.2021 15.17, Sebastian Graf wrote: >> Hi Devs, >> >> In https://gitlab.haskell.org/ghc/ghc/-/issues/20642 we saw that GHC >= 8.10 outputs pattern match warnings a little differently than it used to. Example from there: >> >> {-# OPTIONS_GHC -Wincomplete-uni-patterns #-} >> >> foo :: [a] -> [a] >> foo [] = [] >> foo xs = ys >> where >> (_, ys@(_:_)) = splitAt 0 xs >> >> main :: IO () >> main = putStrLn $ foo $ "Hello, coverage checker!" >> Instead of saying >> >> >> >> ListPair.hs:7:3: warning: [-Wincomplete-uni-patterns] >> Pattern match(es) are non-exhaustive >> In a pattern binding: Patterns not matched: (_, []) >> >> >> >> We now say >> >> >> >> ListPair.hs:7:3: warning: [-Wincomplete-uni-patterns] >> Pattern match(es) are non-exhaustive >> In a pattern binding: >> Patterns of type ‘([a], [a])’ not matched: >> ([], []) >> ((_:_), []) >> >> >> >> E.g., newer versions do (one) case split on pattern variables that haven't even been scrutinised by the pattern match. That amounts to quantitatively more pattern suggestions and for each variable a list of constructors that could be matched on. >> The motivation for the change is outlined in https://gitlab.haskell.org/ghc/ghc/-/issues/20642#note_390110, but I could easily be swayed not to do the case split. Which arguably is less surprising, as Andreas Abel points out. >> >> Considering the other examples from my post, which would you prefer? >> >> Cheers, >> Sebastian >> >> >> _______________________________________________ >> ghc-devs mailing list >> >> ghc-devs at haskell.org >> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs From sgraf1337 at gmail.com Wed Nov 10 09:58:20 2021 From: sgraf1337 at gmail.com (Sebastian Graf) Date: Wed, 10 Nov 2021 09:58:20 +0000 Subject: Case split uncovered patterns in warnings or not? In-Reply-To: <3A5391F1-7852-4592-B33B-B8DB008848DD@serokell.io> References: <86dabd42-cde3-08ad-68dd-5db9963f194a@iki.fi> <3A5391F1-7852-4592-B33B-B8DB008848DD@serokell.io> Message-ID: Yes, but that is an entirely different issue: See https://gitlab.haskell.org/ghc/ghc/-/issues/13964, https://gitlab.haskell.org/ghc/ghc/-/issues/20311 and my problems in https://gitlab.haskell.org/ghc/ghc/-/merge_requests/4116#note_301577 and following. Help is appreciated there, I don't know how to get the necessary information in `DsM`. Would need to poke at `mi_exports`, which is quite unreachable at that point. I'd probably have to add a field to the `DsGblEnv`. I agree that Integer is another nail in the coffin, but only by coincidence. As I said in the issue, if you do an EmptyCase on `Integer` (which you rarely should do), then you'd be presented with the abstract constructors in GHC 8.8, too. As for the issue at hand, I'll go for "case split on EmptyCase only", which should get back the behavior from 8.8. ------ Originalnachricht ------ Von: "Vladislav Zavialov" An: "Oleg Grenrus" Cc: "ghc-devs" Gesendet: 10.11.2021 10:51:03 Betreff: Re: Case split uncovered patterns in warnings or not? >Integer is an interesting example. I think it reveals another issue: exhaustiveness checking should account for abstract data types. If the constructors are not exported, do not case split. > >- Vlad > >> On 10 Nov 2021, at 12:48, Oleg Grenrus wrote: >> >> It should not. Not even when forced. >> >> I have seen an `Integer` constructors presented to me, for example: >> >> module Ex where >> >> foo :: Bool -> Integer -> Integer >> foo True i = i >> >> With GHC-8.8 the warning is good: >> >> % ghci-8.8.4 -Wall Ex.hs >> GHCi, version 8.8.4: https://www.haskell.org/ghc/ :? for help >> Loaded GHCi configuration from /home/phadej/.ghci >> [1 of 1] Compiling Ex ( Ex.hs, interpreted ) >> >> Ex.hs:4:1: warning: [-Wincomplete-patterns] >> Pattern match(es) are non-exhaustive >> In an equation for ‘foo’: Patterns not matched: False _ >> | >> 4 | foo True i = i >> | ^^^^^^^^^^^^^^ >> >> With GHC-8.10 is straight up awful. >> I'm glad I don't have to explain it to any beginner, >> or person who don't know how Integer is implemented. >> (9.2 is about as bad too). >> >> % ghci-8.10.4 -Wall Ex.hs >> GHCi, version 8.10.4: https://www.haskell.org/ghc/ :? for help >> Loaded GHCi configuration from /home/phadej/.ghci >> [1 of 1] Compiling Ex ( Ex.hs, interpreted ) >> >> Ex.hs:4:1: warning: [-Wincomplete-patterns] >> Pattern match(es) are non-exhaustive >> In an equation for ‘foo’: >> Patterns not matched: >> False (integer-gmp-1.0.3.0:GHC.Integer.Type.S# _) >> False (integer-gmp-1.0.3.0:GHC.Integer.Type.Jp# _) >> False (integer-gmp-1.0.3.0:GHC.Integer.Type.Jn# _) >> | >> 4 | foo True i = i >> | ^^^ >> >> - Oleg >> >> >> On 9.11.2021 15.17, Sebastian Graf wrote: >>> Hi Devs, >>> >>> In https://gitlab.haskell.org/ghc/ghc/-/issues/20642 we saw that GHC >= 8.10 outputs pattern match warnings a little differently than it used to. Example from there: >>> >>> {-# OPTIONS_GHC -Wincomplete-uni-patterns #-} >>> >>> foo :: [a] -> [a] >>> foo [] = [] >>> foo xs = ys >>> where >>> (_, ys@(_:_)) = splitAt 0 xs >>> >>> main :: IO () >>> main = putStrLn $ foo $ "Hello, coverage checker!" >>> Instead of saying >>> >>> >>> >>> ListPair.hs:7:3: warning: [-Wincomplete-uni-patterns] >>> Pattern match(es) are non-exhaustive >>> In a pattern binding: Patterns not matched: (_, []) >>> >>> >>> >>> We now say >>> >>> >>> >>> ListPair.hs:7:3: warning: [-Wincomplete-uni-patterns] >>> Pattern match(es) are non-exhaustive >>> In a pattern binding: >>> Patterns of type ‘([a], [a])’ not matched: >>> ([], []) >>> ((_:_), []) >>> >>> >>> >>> E.g., newer versions do (one) case split on pattern variables that haven't even been scrutinised by the pattern match. That amounts to quantitatively more pattern suggestions and for each variable a list of constructors that could be matched on. >>> The motivation for the change is outlined in https://gitlab.haskell.org/ghc/ghc/-/issues/20642#note_390110, but I could easily be swayed not to do the case split. Which arguably is less surprising, as Andreas Abel points out. >>> >>> Considering the other examples from my post, which would you prefer? >>> >>> Cheers, >>> Sebastian >>> >>> >>> _______________________________________________ >>> ghc-devs mailing list >>> >>>ghc-devs at haskell.org >>>http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs >> _______________________________________________ >> ghc-devs mailing list >>ghc-devs at haskell.org >>http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > >_______________________________________________ >ghc-devs mailing list >ghc-devs at haskell.org >http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs From oleg.grenrus at iki.fi Wed Nov 10 10:04:38 2021 From: oleg.grenrus at iki.fi (Oleg Grenrus) Date: Wed, 10 Nov 2021 12:04:38 +0200 Subject: Case split uncovered patterns in warnings or not? In-Reply-To: <3A5391F1-7852-4592-B33B-B8DB008848DD@serokell.io> References: <86dabd42-cde3-08ad-68dd-5db9963f194a@iki.fi> <3A5391F1-7852-4592-B33B-B8DB008848DD@serokell.io> Message-ID: This is good idea. I think it would work, if you don't consider empty types. There is no way to not export & import uninhabitness of a type. For Void it doesn't matter, but for     newtype Fin j = UnsafeFin { indexFin :: Int }         absurdFin :: Fin Z -> a     absurdFin x = x `seq` error "absurd: Fin Z" as an efficient version of     data Fin n where         FZ :: Fin (S n)         FS :: Fin n -> Fin (S n)     absurdFin :: Fin z -> a     absurdFin x = case x of {} it does. For GADT version users can directly use EmptyCase instead of `absurdFin`, for unsafe&efficient one the `absurdFin` is the only way today. Alternatively (to not exporting safe `Fin Z` uninhabitness), we should be able to say     {-# COMPLETE :: Fin Z #-} i.e. `Fin Z` doesn't need to be matched for pattern match to be complete. - Oleg P.S. I'm not sure if this is relevant to the tickets sgraf linked in his reply. On 10.11.2021 11.51, Vladislav Zavialov wrote: > Integer is an interesting example. I think it reveals another issue: exhaustiveness checking should account for abstract data types. If the constructors are not exported, do not case split. > > - Vlad > >> On 10 Nov 2021, at 12:48, Oleg Grenrus wrote: >> >> It should not. Not even when forced. >> >> I have seen an `Integer` constructors presented to me, for example: >> >> module Ex where >> >> foo :: Bool -> Integer -> Integer >> foo True i = i >> >> With GHC-8.8 the warning is good: >> >> % ghci-8.8.4 -Wall Ex.hs >> GHCi, version 8.8.4: https://www.haskell.org/ghc/ :? for help >> Loaded GHCi configuration from /home/phadej/.ghci >> [1 of 1] Compiling Ex ( Ex.hs, interpreted ) >> >> Ex.hs:4:1: warning: [-Wincomplete-patterns] >> Pattern match(es) are non-exhaustive >> In an equation for ‘foo’: Patterns not matched: False _ >> | >> 4 | foo True i = i >> | ^^^^^^^^^^^^^^ >> >> With GHC-8.10 is straight up awful. >> I'm glad I don't have to explain it to any beginner, >> or person who don't know how Integer is implemented. >> (9.2 is about as bad too). >> >> % ghci-8.10.4 -Wall Ex.hs >> GHCi, version 8.10.4: https://www.haskell.org/ghc/ :? for help >> Loaded GHCi configuration from /home/phadej/.ghci >> [1 of 1] Compiling Ex ( Ex.hs, interpreted ) >> >> Ex.hs:4:1: warning: [-Wincomplete-patterns] >> Pattern match(es) are non-exhaustive >> In an equation for ‘foo’: >> Patterns not matched: >> False (integer-gmp-1.0.3.0:GHC.Integer.Type.S# _) >> False (integer-gmp-1.0.3.0:GHC.Integer.Type.Jp# _) >> False (integer-gmp-1.0.3.0:GHC.Integer.Type.Jn# _) >> | >> 4 | foo True i = i >> | ^^^ >> >> - Oleg >> >> >> On 9.11.2021 15.17, Sebastian Graf wrote: >>> Hi Devs, >>> >>> In https://gitlab.haskell.org/ghc/ghc/-/issues/20642 we saw that GHC >= 8.10 outputs pattern match warnings a little differently than it used to. Example from there: >>> >>> {-# OPTIONS_GHC -Wincomplete-uni-patterns #-} >>> >>> foo :: [a] -> [a] >>> foo [] = [] >>> foo xs = ys >>> where >>> (_, ys@(_:_)) = splitAt 0 xs >>> >>> main :: IO () >>> main = putStrLn $ foo $ "Hello, coverage checker!" >>> Instead of saying >>> >>> >>> >>> ListPair.hs:7:3: warning: [-Wincomplete-uni-patterns] >>> Pattern match(es) are non-exhaustive >>> In a pattern binding: Patterns not matched: (_, []) >>> >>> >>> >>> We now say >>> >>> >>> >>> ListPair.hs:7:3: warning: [-Wincomplete-uni-patterns] >>> Pattern match(es) are non-exhaustive >>> In a pattern binding: >>> Patterns of type ‘([a], [a])’ not matched: >>> ([], []) >>> ((_:_), []) >>> >>> >>> >>> E.g., newer versions do (one) case split on pattern variables that haven't even been scrutinised by the pattern match. That amounts to quantitatively more pattern suggestions and for each variable a list of constructors that could be matched on. >>> The motivation for the change is outlined in https://gitlab.haskell.org/ghc/ghc/-/issues/20642#note_390110, but I could easily be swayed not to do the case split. Which arguably is less surprising, as Andreas Abel points out. >>> >>> Considering the other examples from my post, which would you prefer? >>> >>> Cheers, >>> Sebastian >>> >>> >>> _______________________________________________ >>> ghc-devs mailing list >>> >>> ghc-devs at haskell.org >>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs >> _______________________________________________ >> ghc-devs mailing list >> ghc-devs at haskell.org >> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > From simonpj at microsoft.com Mon Nov 15 17:55:44 2021 From: simonpj at microsoft.com (Simon Peyton Jones) Date: Mon, 15 Nov 2021 17:55:44 +0000 Subject: Defaulting design choices Message-ID: Sam, Richard, Adam, Vlad, Ryan, and other people interested in GHC's type system. Following discussion with Richard, I have created a new ticket #20686 for Make the design of defaulting explicit. This should inform !6851 which is trying to finish up Concrete# constraints. Please have a look - we 'd love your views on The Choice. Richard especially, but everyone really: feel free to edit the Description to make it more clear/accurate. I propose to treat it as our master design document. Simon PS: I am leaving Microsoft at the end of November 2021, at which point simonpj at microsoft.com will cease to work. Use simon.peytonjones at gmail.com instead. (For now, it just forwards to simonpj at microsoft.com.) -------------- next part -------------- An HTML attachment was scrubbed... URL: From facundo.dominguez at tweag.io Tue Nov 16 14:26:30 2021 From: facundo.dominguez at tweag.io (=?UTF-8?Q?Dom=C3=ADnguez=2C_Facundo?=) Date: Tue, 16 Nov 2021 11:26:30 -0300 Subject: Interpreter thread in ghc-8.10.6 in macOS Message-ID: Dear devs, I found recently that ghc-8.10.6 and ghc-8.10.7 might be behaving differently than previous compilers when running Template Haskell code. When upgrading HaskellR to use a newer ghc I found that it would work fine with ghc-8.10.4, but not ghc-8.10.6 or ghc-8.10.7. HaskellR, and the R runtime in particular, is sensitive to local state in the threads that are used to execute the runtime. HaskellR is arranging to run the R runtime when running TemplateHaskell splices, and this stopped working in MacOS with ghc-8.10.6. The error I got is not new. It comes from R and we would get it in ghci if we try to use HaskellR without passing the command line flag -fno-ghci-sandbox. But this is the first time we get the error when using TemplateHaskell. Does anyone know if there have been changes to the threads in which the TH splices are run? And were there any changes specific to TH in MacOS? Thanks in advance, Facundo [1] https://github.com/tweag/HaskellR/pull/368#issuecomment-968864170 -------------- next part -------------- An HTML attachment was scrubbed... URL: From carter.schonwald at gmail.com Tue Nov 16 17:21:01 2021 From: carter.schonwald at gmail.com (Carter Schonwald) Date: Tue, 16 Nov 2021 12:21:01 -0500 Subject: Interpreter thread in ghc-8.10.6 in macOS In-Reply-To: References: Message-ID: Hey Facundo: can you share a link to the underlying error ? I tried to find the error output in the linked ticket, but I only see a failure message at the stack_build_tool level. Rather than a ghc or linker error How would I replicate the build failure locally? Would cabal get haskellR ; cd haskelR* ; cabal build ; cabal test suffice ? On Tue, Nov 16, 2021 at 9:27 AM Domínguez, Facundo < facundo.dominguez at tweag.io> wrote: > Dear devs, > > I found recently that ghc-8.10.6 and ghc-8.10.7 might be behaving > differently than previous compilers when running Template Haskell code. > > When upgrading HaskellR to use a newer ghc I found that it would work fine > with ghc-8.10.4, but not ghc-8.10.6 or ghc-8.10.7. HaskellR, and the R > runtime in particular, is sensitive to local state in the threads that are > used to execute the runtime. HaskellR is arranging to run the R runtime > when running TemplateHaskell splices, and this stopped working in MacOS > with ghc-8.10.6. > > The error I got is not new. It comes from R and we would get it in ghci if > we try to use HaskellR without passing the command line flag > -fno-ghci-sandbox. But this is the first time we get the error when using > TemplateHaskell. > > Does anyone know if there have been changes to the threads in which the TH > splices are run? And were there any changes specific to TH in MacOS? > > Thanks in advance, > Facundo > > [1] https://github.com/tweag/HaskellR/pull/368#issuecomment-968864170 > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > -------------- next part -------------- An HTML attachment was scrubbed... URL: From facundo.dominguez at tweag.io Tue Nov 16 17:57:05 2021 From: facundo.dominguez at tweag.io (=?UTF-8?Q?Dom=C3=ADnguez=2C_Facundo?=) Date: Tue, 16 Nov 2021 14:57:05 -0300 Subject: Interpreter thread in ghc-8.10.6 in macOS In-Reply-To: References: Message-ID: Hello Carter, thanks for taking a look. > Rather than a ghc or linker error There is no such error. The error is produced instead by the R runtime when it finds it is running in a thread not quite like it expects. Here's a failing job [2]. And the error message looks like: HaskellR-examples > Error: C stack usage 17587399033768 is too close to the limit 5246 HaskellR-examples > Error: C stack usage 17587399033816 is too close to the limit 5247 HaskellR-examples > Error: C stack usage 17587399033352 is too close to the limit 5248 HaskellR-examples > Fatal error: unable to initialize the JIT 5249 HaskellR-examples > Here's [3] where the FAQ of HaskellR suggests using -fno-ghci-sandbox. > How would I replicate the build failure locally? Would cabal get haskellR ; cd haskelR* ; cabal build ; cabal test suffice ? I think it will depend on how you have installed the R runtime. It should be possible to install with cabal-install, but I don't have any up-to-date instructions to share. The readme [4] goes with the stack tool instead. Best, Facundo [2] https://github.com/tweag/HaskellR/runs/4224936110?check_suite_focus=true [3] https://tweag.github.io/HaskellR/docs/faq.html [4] https://github.com/tweag/HaskellR#the-haskellr-project On Tue, Nov 16, 2021 at 2:21 PM Carter Schonwald wrote: > Hey Facundo: can you share a link to the underlying error ? I tried to > find the error output in the linked ticket, but I only see a failure > message at the stack_build_tool level. Rather than a ghc or linker error > > How would I replicate the build failure locally? Would cabal get haskellR > ; cd haskelR* ; cabal build ; cabal test suffice ? > > On Tue, Nov 16, 2021 at 9:27 AM Domínguez, Facundo < > facundo.dominguez at tweag.io> wrote: > >> Dear devs, >> >> I found recently that ghc-8.10.6 and ghc-8.10.7 might be behaving >> differently than previous compilers when running Template Haskell code. >> >> When upgrading HaskellR to use a newer ghc I found that it would work >> fine with ghc-8.10.4, but not ghc-8.10.6 or ghc-8.10.7. HaskellR, and the R >> runtime in particular, is sensitive to local state in the threads that are >> used to execute the runtime. HaskellR is arranging to run the R runtime >> when running TemplateHaskell splices, and this stopped working in MacOS >> with ghc-8.10.6. >> >> The error I got is not new. It comes from R and we would get it in ghci >> if we try to use HaskellR without passing the command line flag >> -fno-ghci-sandbox. But this is the first time we get the error when using >> TemplateHaskell. >> >> Does anyone know if there have been changes to the threads in which the >> TH splices are run? And were there any changes specific to TH in MacOS? >> >> Thanks in advance, >> Facundo >> >> [1] https://github.com/tweag/HaskellR/pull/368#issuecomment-968864170 >> _______________________________________________ >> ghc-devs mailing list >> ghc-devs at haskell.org >> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From zubin at well-typed.com Tue Nov 16 18:11:09 2021 From: zubin at well-typed.com (Zubin Duggal) Date: Tue, 16 Nov 2021 23:41:09 +0530 Subject: Interpreter thread in ghc-8.10.6 in macOS In-Reply-To: References: Message-ID: <20211116181109.abkypcubav474y3v@zubin-msi> A patch to run Template Haskell code in a forked thread was included in ghc-8.10.6: https://gitlab.haskell.org/ghc/ghc/-/merge_requests/5042 This was chosen as the only robust way to avoid reporting any asynchronous exceptions thrown to the compilation process during TH evaluation as a user error in TH code. If you cancel a compile by throwing the compilation thread an exception, we don't want to catch that error in GHC and report it as if the TH code had a regular exception. By running the TH code in a forked thread and not sharing the ThreadId, we ensure that any exception caught during the evaluation of TH code either originated from the code itself or was thrown by the RTS directly to the TH code (BlockedIndefinetelyOnMVar,... etc.). There may be alternative solutions to this or ways to make this behaviour configurable. I would urge you to open an ticket on the GHC tracker detailing your use case. On 21/11/16 11:26, Domínguez, Facundo wrote: >Dear devs, > >I found recently that ghc-8.10.6 and ghc-8.10.7 might be behaving >differently than previous compilers when running Template Haskell code. > >When upgrading HaskellR to use a newer ghc I found that it would work fine >with ghc-8.10.4, but not ghc-8.10.6 or ghc-8.10.7. HaskellR, and the R >runtime in particular, is sensitive to local state in the threads that are >used to execute the runtime. HaskellR is arranging to run the R runtime >when running TemplateHaskell splices, and this stopped working in MacOS >with ghc-8.10.6. > >The error I got is not new. It comes from R and we would get it in ghci if >we try to use HaskellR without passing the command line flag >-fno-ghci-sandbox. But this is the first time we get the error when using >TemplateHaskell. > >Does anyone know if there have been changes to the threads in which the TH >splices are run? And were there any changes specific to TH in MacOS? > >Thanks in advance, >Facundo > >[1] https://github.com/tweag/HaskellR/pull/368#issuecomment-968864170 >_______________________________________________ >ghc-devs mailing list >ghc-devs at haskell.org >http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs From nr at cs.tufts.edu Tue Nov 16 20:17:55 2021 From: nr at cs.tufts.edu (Norman Ramsey) Date: Tue, 16 Nov 2021 15:17:55 -0500 Subject: request for comments: foreign calls (#20693) Message-ID: <20211116201755.AA40F2C2A17@homedog.cs.tufts.edu> I'd like it to be easier to extend GHC with support for new species of foreign calls. Existing support makes it easy to *replace* GHC's implementation of foreign calls but not to *extend* it. Details in #20693 (https://gitlab.haskell.org/ghc/ghc/-/issues/20693). I'd welcome comments, especially from anyone involved with GHCJS. Norman From facundo.dominguez at tweag.io Tue Nov 16 22:07:17 2021 From: facundo.dominguez at tweag.io (=?UTF-8?Q?Dom=C3=ADnguez=2C_Facundo?=) Date: Tue, 16 Nov 2021 19:07:17 -0300 Subject: Interpreter thread in ghc-8.10.6 in macOS In-Reply-To: <20211116181109.abkypcubav474y3v@zubin-msi> References: <20211116181109.abkypcubav474y3v@zubin-msi> Message-ID: Thanks Zubin! That looks very much on the mark. We might not need to change GHC after all if we can avoid calling the R runtime from Template Haskell [4]. [4] https://github.com/tweag/HaskellR/issues/369 On Tue, Nov 16, 2021 at 3:11 PM Zubin Duggal wrote: > A patch to run Template Haskell code in a forked thread was included in > ghc-8.10.6: https://gitlab.haskell.org/ghc/ghc/-/merge_requests/5042 > > This was chosen as the only robust way to avoid reporting any > asynchronous exceptions thrown to the compilation process during TH > evaluation as a user error in TH code. If you cancel a compile by > throwing the compilation thread an exception, we don't want to catch > that error in GHC and report it as if the TH code had a regular > exception. By running the TH code in a forked thread and not sharing > the ThreadId, we ensure that any exception caught during the evaluation > of TH code either originated from the code itself or was thrown by the > RTS directly to the TH code (BlockedIndefinetelyOnMVar,... etc.). > > There may be alternative solutions to this or ways to make this > behaviour configurable. I would urge you to open an ticket on the GHC > tracker detailing your use case. > > On 21/11/16 11:26, Domínguez, Facundo wrote: > >Dear devs, > > > >I found recently that ghc-8.10.6 and ghc-8.10.7 might be behaving > >differently than previous compilers when running Template Haskell code. > > > >When upgrading HaskellR to use a newer ghc I found that it would work fine > >with ghc-8.10.4, but not ghc-8.10.6 or ghc-8.10.7. HaskellR, and the R > >runtime in particular, is sensitive to local state in the threads that are > >used to execute the runtime. HaskellR is arranging to run the R runtime > >when running TemplateHaskell splices, and this stopped working in MacOS > >with ghc-8.10.6. > > > >The error I got is not new. It comes from R and we would get it in ghci if > >we try to use HaskellR without passing the command line flag > >-fno-ghci-sandbox. But this is the first time we get the error when using > >TemplateHaskell. > > > >Does anyone know if there have been changes to the threads in which the TH > >splices are run? And were there any changes specific to TH in MacOS? > > > >Thanks in advance, > >Facundo > > > >[1] https://github.com/tweag/HaskellR/pull/368#issuecomment-968864170 > > >_______________________________________________ > >ghc-devs mailing list > >ghc-devs at haskell.org > >http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lists at richarde.dev Wed Nov 17 12:01:16 2021 From: lists at richarde.dev (Richard Eisenberg) Date: Wed, 17 Nov 2021 12:01:16 +0000 Subject: DWARF support In-Reply-To: <05A36B9D-DD0F-4279-B992-F50E6D4E8BC9@richarde.dev> References: <05A36B9D-DD0F-4279-B992-F50E6D4E8BC9@richarde.dev> Message-ID: <010f017d2dc5c7f2-ae165255-a6b8-48fc-84e7-d8db6ae68958-000000@us-east-2.amazonses.com> Hi devs, I was intrigued by Bodigrim's comment about HasCallStack in base (https://github.com/haskell/core-libraries-committee/issues/5#issuecomment-970942580) that there are other alternatives, such as DWARF. Over the years, I had tuned out every time I saw the word DWARF: it was (and is!) an unknown acronym and seems like a low-level detail. But Bodigrim's comment made me want to re-think this stance. I found Ben's series of blog posts on DWARF, starting with https://www.haskell.org/ghc/blog/20200403-dwarf-1.html. These are very helpful! In particular, they taught me that DWARF = platform-native debugging metadata. Is that translation accurate? If so, perhaps we should use both names: if I see that GHC x.y.z has DWARF support, I quickly scroll to the next bullet. If I see that GHC x.y.z has support for platform-native debugging metadata and is now compatible with e.g. gdb, I'm interested. Going further, I have a key question for my use case: is this support available on Mac? The first post in the series describes support for "Linux and several BSDs" and the last post says that "Windows PDB support" is future work. (Is "PDB" platform-native debugging metadata for Windows? I don't know.) But I don't see any mention of Mac. What's the status here? It would be very cool if this conversation ends with me making a video on how a few simple GHC flags can allow us to, say, get a stack trace on a pattern-match failure in a Haskell program. Thanks! Richard From moritz.angermann at gmail.com Wed Nov 17 12:27:52 2021 From: moritz.angermann at gmail.com (Moritz Angermann) Date: Wed, 17 Nov 2021 20:27:52 +0800 Subject: DWARF support In-Reply-To: <010f017d2dc5c7f2-ae165255-a6b8-48fc-84e7-d8db6ae68958-000000@us-east-2.amazonses.com> References: <05A36B9D-DD0F-4279-B992-F50E6D4E8BC9@richarde.dev> <010f017d2dc5c7f2-ae165255-a6b8-48fc-84e7-d8db6ae68958-000000@us-east-2.amazonses.com> Message-ID: Hi Richard, I’m not sure using platform native AND the term DWARF would help rather than add to confusion. Let me still try to help a bit with context here. For Linux and most BSDs, we have settled on the Executable and Linking Format (ELF) as the container format for your machine code. And you might see where the inspiration for DWARF might come from. For macOS, we have mach object (mach-o) as the container format. Its distinctly different to ELF, and also the reason why Linux/BSD and macOS are sometimes substantially different, wrt to executable packaging and linking. For windows we have Portable Executable (PE) as the container format. My recollection is that we implemented DWARF in the NCG only for ELF. I've always wanted to scratch an itch and try to make it work for mach-o as well, but never got around to it (yet?). The NCGs have flags that specify if we want to emit debug info or not. I believe most codegens except for x86_64/elf ignore that flag. This is a non-trivial engineering effort to get done properly, I believe. And we all spend time on many other things. Depending on how familiar you are with development on macOS, you might know the notion of dSYM folders, where macOS usually separate the application binary into the binary, and then stores the (d)ebug (SYM)bols in a separate folder. Those are iirc DWARF objects in the end. Hope this helps a bit; my recollection might be a bit rusty. Best, Moritz On Wed 17. Nov 2021 at 20:02, Richard Eisenberg wrote: > Hi devs, > > I was intrigued by Bodigrim's comment about HasCallStack in base ( > https://github.com/haskell/core-libraries-committee/issues/5#issuecomment-970942580) > that there are other alternatives, such as DWARF. Over the years, I had > tuned out every time I saw the word DWARF: it was (and is!) an unknown > acronym and seems like a low-level detail. But Bodigrim's comment made me > want to re-think this stance. > > I found Ben's series of blog posts on DWARF, starting with > https://www.haskell.org/ghc/blog/20200403-dwarf-1.html. These are very > helpful! In particular, they taught me that DWARF = platform-native > debugging metadata. Is that translation accurate? If so, perhaps we should > use both names: if I see that GHC x.y.z has DWARF support, I quickly scroll > to the next bullet. If I see that GHC x.y.z has support for platform-native > debugging metadata and is now compatible with e.g. gdb, I'm interested. > > Going further, I have a key question for my use case: is this support > available on Mac? The first post in the series describes support for "Linux > and several BSDs" and the last post says that "Windows PDB support" is > future work. (Is "PDB" platform-native debugging metadata for Windows? I > don't know.) But I don't see any mention of Mac. What's the status here? > > It would be very cool if this conversation ends with me making a video on > how a few simple GHC flags can allow us to, say, get a stack trace on a > pattern-match failure in a Haskell program. > > Thanks! > Richard > > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > -------------- next part -------------- An HTML attachment was scrubbed... URL: From carter.schonwald at gmail.com Wed Nov 17 13:08:33 2021 From: carter.schonwald at gmail.com (Carter Schonwald) Date: Wed, 17 Nov 2021 08:08:33 -0500 Subject: DWARF support In-Reply-To: References: <05A36B9D-DD0F-4279-B992-F50E6D4E8BC9@richarde.dev> <010f017d2dc5c7f2-ae165255-a6b8-48fc-84e7-d8db6ae68958-000000@us-east-2.amazonses.com> Message-ID: My understanding is that the platform specific part of ghc dwarf support atm is the stack walking to generate dwarf data in stack traces. This is because the dwarf stack walking Libs that are relatively mature are mostly centered around elf? It should still be possible with some work to use perf and gdb style tools, though the complications are a) you have to make sure all the Libs are built with dwarf b) there’s some complications around loading / placing the dwarf files adjacent to the object code files on Darwin (at least last time I checked which was years ago following the wiki entry johan tibbel wrote up I think?) C) scheduler yields make gdb stepping through a program a tad more annoying, I think the “setting the yield timer to zero” is the work around D) the “source” you step through is essentially the c— z-encoded code? So you still need to do some puzzling out of stuff On Wed, Nov 17, 2021 at 7:28 AM Moritz Angermann wrote: > Hi Richard, > > I’m not sure using platform native AND the term DWARF would help rather > than add to confusion. Let me still try to > help a bit with context here. > > For Linux and most BSDs, we have settled on the Executable and Linking > Format (ELF) as the container format for > your machine code. And you might see where the inspiration for DWARF > might come from. > > For macOS, we have mach object (mach-o) as the container format. Its > distinctly different to ELF, and also the > reason why Linux/BSD and macOS are sometimes substantially different, wrt > to executable packaging and linking. > > For windows we have Portable Executable (PE) as the container format. > > My recollection is that we implemented DWARF in the NCG only for ELF. > I've always wanted to scratch an itch > and try to make it work for mach-o as well, but never got around to it > (yet?). The NCGs have flags that specify > if we want to emit debug info or not. I believe most codegens except for > x86_64/elf ignore that flag. > > This is a non-trivial engineering effort to get done properly, I believe. > And we all spend time on many other things. > > Depending on how familiar you are with development on macOS, you might > know the notion of dSYM folders, > where macOS usually separate the application binary into the binary, and > then stores the (d)ebug (SYM)bols in > a separate folder. Those are iirc DWARF objects in the end. > > Hope this helps a bit; my recollection might be a bit rusty. > > Best, > Moritz > > > > On Wed 17. Nov 2021 at 20:02, Richard Eisenberg > wrote: > >> Hi devs, >> >> I was intrigued by Bodigrim's comment about HasCallStack in base ( >> https://github.com/haskell/core-libraries-committee/issues/5#issuecomment-970942580) >> that there are other alternatives, such as DWARF. Over the years, I had >> tuned out every time I saw the word DWARF: it was (and is!) an unknown >> acronym and seems like a low-level detail. But Bodigrim's comment made me >> want to re-think this stance. >> >> I found Ben's series of blog posts on DWARF, starting with >> https://www.haskell.org/ghc/blog/20200403-dwarf-1.html. These are very >> helpful! In particular, they taught me that DWARF = platform-native >> debugging metadata. Is that translation accurate? If so, perhaps we should >> use both names: if I see that GHC x.y.z has DWARF support, I quickly scroll >> to the next bullet. If I see that GHC x.y.z has support for platform-native >> debugging metadata and is now compatible with e.g. gdb, I'm interested. >> >> Going further, I have a key question for my use case: is this support >> available on Mac? The first post in the series describes support for "Linux >> and several BSDs" and the last post says that "Windows PDB support" is >> future work. (Is "PDB" platform-native debugging metadata for Windows? I >> don't know.) But I don't see any mention of Mac. What's the status here? >> >> It would be very cool if this conversation ends with me making a video on >> how a few simple GHC flags can allow us to, say, get a stack trace on a >> pattern-match failure in a Haskell program. >> >> Thanks! >> Richard >> >> _______________________________________________ >> ghc-devs mailing list >> ghc-devs at haskell.org >> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs >> > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > -------------- next part -------------- An HTML attachment was scrubbed... URL: From moritz.angermann at gmail.com Wed Nov 17 13:24:07 2021 From: moritz.angermann at gmail.com (Moritz Angermann) Date: Wed, 17 Nov 2021 21:24:07 +0800 Subject: DWARF support In-Reply-To: References: <05A36B9D-DD0F-4279-B992-F50E6D4E8BC9@richarde.dev> <010f017d2dc5c7f2-ae165255-a6b8-48fc-84e7-d8db6ae68958-000000@us-east-2.amazonses.com> Message-ID: Thanks Carter! Yes I completely forgot about the unwinding librarys. Sorry. My bad! Best, Moritz On Wed 17. Nov 2021 at 21:08, Carter Schonwald wrote: > My understanding is that the platform specific part of ghc dwarf support > atm is the stack walking to generate dwarf data in stack traces. This is > because the dwarf stack walking Libs that are relatively mature are mostly > centered around elf? > > It should still be possible with some work to use perf and gdb style > tools, though the complications are > > a) you have to make sure all the Libs are built with dwarf > > b) there’s some complications around loading / placing the dwarf files > adjacent to the object code files on Darwin (at least last time I checked > which was years ago following the wiki entry johan tibbel wrote up I > think?) > > C) scheduler yields make gdb stepping through a program a tad more > annoying, I think the “setting the yield timer to zero” is the work around > > D) the “source” you step through is essentially the c— z-encoded code? So > you still need to do some puzzling out of stuff > > > On Wed, Nov 17, 2021 at 7:28 AM Moritz Angermann < > moritz.angermann at gmail.com> wrote: > >> Hi Richard, >> >> I’m not sure using platform native AND the term DWARF would help rather >> than add to confusion. Let me still try to >> help a bit with context here. >> >> For Linux and most BSDs, we have settled on the Executable and Linking >> Format (ELF) as the container format for >> your machine code. And you might see where the inspiration for DWARF >> might come from. >> >> For macOS, we have mach object (mach-o) as the container format. Its >> distinctly different to ELF, and also the >> reason why Linux/BSD and macOS are sometimes substantially different, wrt >> to executable packaging and linking. >> >> For windows we have Portable Executable (PE) as the container format. >> >> My recollection is that we implemented DWARF in the NCG only for ELF. >> I've always wanted to scratch an itch >> and try to make it work for mach-o as well, but never got around to it >> (yet?). The NCGs have flags that specify >> if we want to emit debug info or not. I believe most codegens except for >> x86_64/elf ignore that flag. >> >> This is a non-trivial engineering effort to get done properly, I believe. >> And we all spend time on many other things. >> >> Depending on how familiar you are with development on macOS, you might >> know the notion of dSYM folders, >> where macOS usually separate the application binary into the binary, and >> then stores the (d)ebug (SYM)bols in >> a separate folder. Those are iirc DWARF objects in the end. >> >> Hope this helps a bit; my recollection might be a bit rusty. >> >> Best, >> Moritz >> >> >> >> On Wed 17. Nov 2021 at 20:02, Richard Eisenberg >> wrote: >> >>> Hi devs, >>> >>> I was intrigued by Bodigrim's comment about HasCallStack in base ( >>> https://github.com/haskell/core-libraries-committee/issues/5#issuecomment-970942580) >>> that there are other alternatives, such as DWARF. Over the years, I had >>> tuned out every time I saw the word DWARF: it was (and is!) an unknown >>> acronym and seems like a low-level detail. But Bodigrim's comment made me >>> want to re-think this stance. >>> >>> I found Ben's series of blog posts on DWARF, starting with >>> https://www.haskell.org/ghc/blog/20200403-dwarf-1.html. These are very >>> helpful! In particular, they taught me that DWARF = platform-native >>> debugging metadata. Is that translation accurate? If so, perhaps we should >>> use both names: if I see that GHC x.y.z has DWARF support, I quickly scroll >>> to the next bullet. If I see that GHC x.y.z has support for platform-native >>> debugging metadata and is now compatible with e.g. gdb, I'm interested. >>> >>> Going further, I have a key question for my use case: is this support >>> available on Mac? The first post in the series describes support for "Linux >>> and several BSDs" and the last post says that "Windows PDB support" is >>> future work. (Is "PDB" platform-native debugging metadata for Windows? I >>> don't know.) But I don't see any mention of Mac. What's the status here? >>> >>> It would be very cool if this conversation ends with me making a video >>> on how a few simple GHC flags can allow us to, say, get a stack trace on a >>> pattern-match failure in a Haskell program. >>> >>> Thanks! >>> Richard >>> >>> _______________________________________________ >>> ghc-devs mailing list >>> ghc-devs at haskell.org >>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs >>> >> _______________________________________________ >> ghc-devs mailing list >> ghc-devs at haskell.org >> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lists at richarde.dev Wed Nov 17 16:08:28 2021 From: lists at richarde.dev (Richard Eisenberg) Date: Wed, 17 Nov 2021 16:08:28 +0000 Subject: DWARF support In-Reply-To: References: <05A36B9D-DD0F-4279-B992-F50E6D4E8BC9@richarde.dev> <010f017d2dc5c7f2-ae165255-a6b8-48fc-84e7-d8db6ae68958-000000@us-east-2.amazonses.com> Message-ID: <010f017d2ea81bc1-f84743c7-4957-45b1-970b-1b24683b8d0c-000000@us-east-2.amazonses.com> Thanks for this! > On Nov 17, 2021, at 7:27 AM, Moritz Angermann wrote: > > For Linux and most BSDs, we have settled on the Executable and Linking Format (ELF) as the container format for > your machine code. And you might see where the inspiration for DWARF might come from. This suggests to me that DWARF is the canonical format for debugging information on Linux and most BSDs. Is that statement correct? If so, how is that different to "platform-native"? Actually, the precise wording doesn't matter: I think I'm just requesting for a more direct relationship between "DWARF" and "compatibility with all the debugging and profiling tools you use for other languages". > > For macOS, we have mach object (mach-o) as the container format. Its distinctly different to ELF, and also the > reason why Linux/BSD and macOS are sometimes substantially different, wrt to executable packaging and linking. OK. So there is no macOS support here. That's fine -- I'm just trying to understand the status quo. > > For windows we have Portable Executable (PE) as the container format. This implies that the DWARF work is (unsurprisingly) completely inapplicable for Windows. > Depending on how familiar you are with development on macOS, you might know the notion of dSYM folders, > where macOS usually separate the application binary into the binary, and then stores the (d)ebug (SYM)bols in > a separate folder. Those are iirc DWARF objects in the end. This suggests to me that the DWARF work is applicable to the macOS use case, but much more work still needs to be done. OK. Looking for more information, I checked the manual. And I found this: https://downloads.haskell.org/ghc/latest/docs/html/users_guide/debug-info.html It's really helpful! And it suggests that I actually *can* do this on macOS. Perhaps that video will really happen after all. Forgive the noise -- I should have checked the manual first. :) Richard From klebinger.andreas at gmx.at Wed Nov 17 16:26:02 2021 From: klebinger.andreas at gmx.at (Andreas Klebinger) Date: Wed, 17 Nov 2021 17:26:02 +0100 Subject: DWARF support In-Reply-To: <010f017d2ea81bc1-f84743c7-4957-45b1-970b-1b24683b8d0c-000000@us-east-2.amazonses.com> References: <05A36B9D-DD0F-4279-B992-F50E6D4E8BC9@richarde.dev> <010f017d2dc5c7f2-ae165255-a6b8-48fc-84e7-d8db6ae68958-000000@us-east-2.amazonses.com> <010f017d2ea81bc1-f84743c7-4957-45b1-970b-1b24683b8d0c-000000@us-east-2.amazonses.com> Message-ID: Am 17/11/2021 um 17:08 schrieb Richard Eisenberg: >> For windows we have Portable Executable (PE) as the container format. > This implies that the DWARF work is (unsurprisingly) completely inapplicable for Windows. It's not quite as simple. Dwarf info can be embedded into windows executables/libraries. Tools that understand dwarf can read that information and use it. However many tools on windows don't understand dwarf debugging information since windows has it's own debugging format. So it's not quite as helpful as on linux where everyone agreed to use dwarf. But it still can be used for some things. E.g. there are debuggers on windows (lldbg, gdb) that can read and use this information. Further we could in theory start emitting the same information in the windows native format. In that case we could re-use much of the work that went into GHC to allows us to collect debugging information. Putting into the right format comes fairly late in the pipeline and all steps up to that could be shared. Cheers Andreas From qdunkan at gmail.com Thu Nov 18 03:42:07 2021 From: qdunkan at gmail.com (Evan Laforge) Date: Wed, 17 Nov 2021 19:42:07 -0800 Subject: [Haskell-cafe] [ANNOUNCE] GHC 9.2.1 now available In-Reply-To: <87lf2bu98a.fsf@smart-cactus.org> References: <87lf2bu98a.fsf@smart-cactus.org> Message-ID: It seems the OS X distribution is missing profiling libs, details at https://gitlab.haskell.org/ghc/ghc/-/issues/20707 but that's basically the whole story :) From ben at smart-cactus.org Thu Nov 18 15:29:11 2021 From: ben at smart-cactus.org (Ben Gamari) Date: Thu, 18 Nov 2021 10:29:11 -0500 Subject: DWARF support In-Reply-To: <010f017d2dc5c7f2-ae165255-a6b8-48fc-84e7-d8db6ae68958-000000@us-east-2.amazonses.com> References: <05A36B9D-DD0F-4279-B992-F50E6D4E8BC9@richarde.dev> <010f017d2dc5c7f2-ae165255-a6b8-48fc-84e7-d8db6ae68958-000000@us-east-2.amazonses.com> Message-ID: <87o86hwki6.fsf@smart-cactus.org> Richard Eisenberg writes: > Hi devs, > > I was intrigued by Bodigrim's comment about HasCallStack in base > (https://github.com/haskell/core-libraries-committee/issues/5#issuecomment-970942580) > that there are other alternatives, such as DWARF. Over the years, I > had tuned out every time I saw the word DWARF: it was (and is!) an > unknown acronym and seems like a low-level detail. But Bodigrim's > comment made me want to re-think this stance. > > I found Ben's series of blog posts on DWARF, starting with > https://www.haskell.org/ghc/blog/20200403-dwarf-1.html. These are very > helpful! In particular, they taught me that DWARF = platform-native > debugging metadata. Is that translation accurate? If so, perhaps we > should use both names: if I see that GHC x.y.z has DWARF support, I > quickly scroll to the next bullet. If I see that GHC x.y.z has support > for platform-native debugging metadata and is now compatible with e.g. > gdb, I'm interested. > > Going further, I have a key question for my use case: is this support > available on Mac? The first post in the series describes support for > "Linux and several BSDs" and the last post says that "Windows PDB > support" is future work. (Is "PDB" platform-native debugging metadata > for Windows? I don't know.) But I don't see any mention of Mac. What's > the status here? > The problem is that DWARF is only one format for debugging information. Windows uses PDB, which is practically undocumented and consequently essentially unsupportable, AFAICT. I believe some other platforms still use the STABS format, which I do not believe we will ever support. Sadly, Darwin is, as usual, a bit of a tricky case. The platform seems to use a variant of the typical "split-debug" packaging configuration for debug information. Apple calls this separate debug information `dsym` files. In typical Apple fasion, the documentation surrounding this mechanism is very poor. It appears that the user or packager must generate these files manually using dsymutils but it's quite unclear how a consuming library is to find the dsym for a particular object. To make matters worse, the library that we use for unwinding, libdw, has absolutely no support for it. Ultimately, my goals for working on DWARF were: 1. To ensure that we had *some* baseline symbol and line information support on platforms for low-level debugging. I have found even basic line information invaluable when looking at code generation issues and thankfully gdb can always use inline DWARF information, even on platforms like on Windows where it isn't quite "native" 2. Providing an interface for users to unwind the stack from Haskell on the "easy" platforms (e.g. the ELF-based BSDs and Linux). I was hoping someone would express interest in picking up Darwin and Windows, at which point we could start to push this as a universal backtrace mechanism, but sadly this hasn't yet happened. 3. Provide a basis for statistic profiling support. This is sadly complicated by the stack pointer issues that Andreas and I note in our respective blog posts [1,2]. Sadly, overcoming this without breaking potential users is harder than we thought due to limitations in LLVM. At this point, for backtrace support I would rather put my money is on a native Haskell stack unwinder (such as Sven Tennie's work [3,4]). Not only is it more portable but it is also more robust (whereas with DWARF any single object lacking debug information would break unwinding), and is significantly less costly since we know much more about the structure of our stack than a DWARF unwinder would. Cheers, - Ben [1] https://well-typed.com/blog/2020/04/dwarf-4/ [2] https://well-typed.com/blog/2021/07/ghc-sp-profiling/ [3] https://gitlab.haskell.org/ghc/ghc/-/issues/18163 [4] https://gitlab.haskell.org/ghc/ghc/-/merge_requests/5456 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From ben at smart-cactus.org Thu Nov 18 15:37:37 2021 From: ben at smart-cactus.org (Ben Gamari) Date: Thu, 18 Nov 2021 10:37:37 -0500 Subject: DWARF support In-Reply-To: <010f017d2ea81bc1-f84743c7-4957-45b1-970b-1b24683b8d0c-000000@us-east-2.amazonses.com> References: <05A36B9D-DD0F-4279-B992-F50E6D4E8BC9@richarde.dev> <010f017d2dc5c7f2-ae165255-a6b8-48fc-84e7-d8db6ae68958-000000@us-east-2.amazonses.com> <010f017d2ea81bc1-f84743c7-4957-45b1-970b-1b24683b8d0c-000000@us-east-2.amazonses.com> Message-ID: <87lf1lwk46.fsf@smart-cactus.org> Richard Eisenberg writes: > Thanks for this! > >> On Nov 17, 2021, at 7:27 AM, Moritz Angermann wrote: >> >> For Linux and most BSDs, we have settled on the Executable and Linking Format (ELF) as the container format for >> your machine code. And you might see where the inspiration for DWARF might come from. > > This suggests to me that DWARF is the canonical format for debugging information on Linux and most BSDs. Is that statement correct? If so, how is that different to "platform-native"? Actually, the precise wording doesn't matter: I think I'm just requesting for a more direct relationship between "DWARF" and "compatibility with all the debugging and profiling tools you use for other languages". > >> >> For macOS, we have mach object (mach-o) as the container format. Its distinctly different to ELF, and also the >> reason why Linux/BSD and macOS are sometimes substantially different, wrt to executable packaging and linking. > > OK. So there is no macOS support here. That's fine -- I'm just trying to understand the status quo. > >> >> For windows we have Portable Executable (PE) as the container format. > > This implies that the DWARF work is (unsurprisingly) completely inapplicable for Windows. > >> Depending on how familiar you are with development on macOS, you might know the notion of dSYM folders, >> where macOS usually separate the application binary into the binary, and then stores the (d)ebug (SYM)bols in >> a separate folder. Those are iirc DWARF objects in the end. > > This suggests to me that the DWARF work is applicable to the macOS use case, but much more work still needs to be done. OK. > > Looking for more information, I checked the manual. And I found this: > https://downloads.haskell.org/ghc/latest/docs/html/users_guide/debug-info.html > It's really helpful! And it suggests that I actually *can* do this on > macOS. Perhaps that video will really happen after all. > I should clarify here that there are really two pieces to "debug information support": * the code generation logic responsible for producing the DWARF metadata * the RTS support for using that information to unwind the stack at runtime That section really describes the former. As noted in my earlier email and in #20702, the library used by the RTS to implement unwinding only targets ELF platforms. Cheers, - Ben -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From lists at richarde.dev Thu Nov 18 16:23:08 2021 From: lists at richarde.dev (Richard Eisenberg) Date: Thu, 18 Nov 2021 16:23:08 +0000 Subject: DWARF support In-Reply-To: <87o86hwki6.fsf@smart-cactus.org> References: <05A36B9D-DD0F-4279-B992-F50E6D4E8BC9@richarde.dev> <010f017d2dc5c7f2-ae165255-a6b8-48fc-84e7-d8db6ae68958-000000@us-east-2.amazonses.com> <87o86hwki6.fsf@smart-cactus.org> Message-ID: <010f017d33dbe3f9-f5d8a751-b85f-443d-ad3c-58f5dd9f963f-000000@us-east-2.amazonses.com> > On Nov 18, 2021, at 10:29 AM, Ben Gamari wrote: > > At this point, for backtrace support I would rather put my money is on a > native Haskell stack unwinder (such as Sven Tennie's work [3,4]). Not only > is it more portable but it is also more robust (whereas with DWARF any > single object lacking debug information would break unwinding), and is > significantly less costly since we know much more about the structure of > our stack than a DWARF unwinder would. Interesting -- this is helpful to know. I had heard about DWARF support for some years and thought that it would deliver stack traces. Now I will look for other sources. All good -- I understand how this is hard! -- and nice to know about. Thanks for the writeup, Ben. Richard -------------- next part -------------- An HTML attachment was scrubbed... URL: From cdsmith at gmail.com Thu Nov 18 16:31:09 2021 From: cdsmith at gmail.com (Chris Smith) Date: Thu, 18 Nov 2021 11:31:09 -0500 Subject: DWARF support In-Reply-To: <010f017d33dbe3f9-f5d8a751-b85f-443d-ad3c-58f5dd9f963f-000000@us-east-2.amazonses.com> References: <05A36B9D-DD0F-4279-B992-F50E6D4E8BC9@richarde.dev> <010f017d2dc5c7f2-ae165255-a6b8-48fc-84e7-d8db6ae68958-000000@us-east-2.amazonses.com> <87o86hwki6.fsf@smart-cactus.org> <010f017d33dbe3f9-f5d8a751-b85f-443d-ad3c-58f5dd9f963f-000000@us-east-2.amazonses.com> Message-ID: Just to satisfy my curiosity here, when talking about backtraces here, are you talking about a lexical call stack, or an execution stack? On Thu, Nov 18, 2021 at 11:24 AM Richard Eisenberg wrote: > > > On Nov 18, 2021, at 10:29 AM, Ben Gamari wrote: > > At this point, for backtrace support I would rather put my money is on a > native Haskell stack unwinder (such as Sven Tennie's work [3,4]). Not only > is it more portable but it is also more robust (whereas with DWARF any > single object lacking debug information would break unwinding), and is > significantly less costly since we know much more about the structure of > our stack than a DWARF unwinder would. > > > Interesting -- this is helpful to know. I had heard about DWARF support > for some years and thought that it would deliver stack traces. Now I will > look for other sources. All good -- I understand how this is hard! -- and > nice to know about. > > Thanks for the writeup, Ben. > > Richard > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > -------------- next part -------------- An HTML attachment was scrubbed... URL: From a.pelenitsyn at gmail.com Thu Nov 18 19:27:37 2021 From: a.pelenitsyn at gmail.com (Artem Pelenitsyn) Date: Thu, 18 Nov 2021 14:27:37 -0500 Subject: DWARF support In-Reply-To: References: <05A36B9D-DD0F-4279-B992-F50E6D4E8BC9@richarde.dev> <010f017d2dc5c7f2-ae165255-a6b8-48fc-84e7-d8db6ae68958-000000@us-east-2.amazonses.com> <87o86hwki6.fsf@smart-cactus.org> <010f017d33dbe3f9-f5d8a751-b85f-443d-ad3c-58f5dd9f963f-000000@us-east-2.amazonses.com> Message-ID: Another question would be where do I read about Haskell-native stack unwinder. The issue and MR Ben referenced have descriptions, but the MR didn't touch anything inside `docs` which is a bit scary. Are there any good recourses to dive into it besides the source code in the MR? -- Best, Artem On Thu, Nov 18, 2021, 11:31 AM Chris Smith wrote: > Just to satisfy my curiosity here, when talking about backtraces here, are > you talking about a lexical call stack, or an execution stack? > > On Thu, Nov 18, 2021 at 11:24 AM Richard Eisenberg > wrote: > >> >> >> On Nov 18, 2021, at 10:29 AM, Ben Gamari wrote: >> >> At this point, for backtrace support I would rather put my money is on a >> native Haskell stack unwinder (such as Sven Tennie's work [3,4]). Not only >> is it more portable but it is also more robust (whereas with DWARF any >> single object lacking debug information would break unwinding), and is >> significantly less costly since we know much more about the structure of >> our stack than a DWARF unwinder would. >> >> >> Interesting -- this is helpful to know. I had heard about DWARF support >> for some years and thought that it would deliver stack traces. Now I will >> look for other sources. All good -- I understand how this is hard! -- and >> nice to know about. >> >> Thanks for the writeup, Ben. >> >> Richard >> _______________________________________________ >> ghc-devs mailing list >> ghc-devs at haskell.org >> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs >> > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > -------------- next part -------------- An HTML attachment was scrubbed... URL: From nr at cs.tufts.edu Thu Nov 18 20:06:35 2021 From: nr at cs.tufts.edu (Norman Ramsey) Date: Thu, 18 Nov 2021 15:06:35 -0500 Subject: How to build Haddock documentation quickly? Message-ID: <20211118200635.B016E2C2ECB@homedog.cs.tufts.edu> In service of #20528, I'm trying to clean up some Haddock documentation in the GHC sources. The only command I know is ./hadrian/build -j _build/docs/html/index.html It turns out that if I change Haddock comments in one file (GHC/Tc/Gen/Foreign.hs), rebuilding the HTML takes a minute and a quarter. That's a little long. (And I had one build take over four minutes, which is even longer.) What tricks can I use to speed up this process? Might there be a way to run Haddock directly until things are to my liking? Norman From hecate at glitchbra.in Thu Nov 18 20:11:20 2021 From: hecate at glitchbra.in (=?UTF-8?Q?H=c3=a9cate?=) Date: Thu, 18 Nov 2021 21:11:20 +0100 Subject: How to build Haddock documentation quickly? In-Reply-To: <20211118200635.B016E2C2ECB@homedog.cs.tufts.edu> References: <20211118200635.B016E2C2ECB@homedog.cs.tufts.edu> Message-ID: <9a2ef2b2-5b25-1e3d-f9e8-10975793e714@glitchbra.in> Hi Norman, Could you see if the command-line described here is of any help? https://gitlab.haskell.org/ghc/ghc/-/issues/17929#building-the-docs Particularly the "--freeze1 --flavour=Quick" bit. Cheers! Le 18/11/2021 à 21:06, Norman Ramsey a écrit : > In service of #20528, I'm trying to clean up some Haddock > documentation in the GHC sources. The only command I know is > > ./hadrian/build -j _build/docs/html/index.html > > It turns out that if I change Haddock comments in one file > (GHC/Tc/Gen/Foreign.hs), rebuilding the HTML takes a minute > and a quarter. That's a little long. (And I had one build > take over four minutes, which is even longer.) > > What tricks can I use to speed up this process? Might there be a way > to run Haddock directly until things are to my liking? > > > Norman > > > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs -- Hécate ✨ 🐦: @TechnoEmpress IRC: Hecate WWW: https://glitchbra.in RUN: BSD From ben at smart-cactus.org Fri Nov 19 00:09:19 2021 From: ben at smart-cactus.org (Ben Gamari) Date: Thu, 18 Nov 2021 19:09:19 -0500 Subject: DWARF support In-Reply-To: References: <05A36B9D-DD0F-4279-B992-F50E6D4E8BC9@richarde.dev> <010f017d2dc5c7f2-ae165255-a6b8-48fc-84e7-d8db6ae68958-000000@us-east-2.amazonses.com> <87o86hwki6.fsf@smart-cactus.org> <010f017d33dbe3f9-f5d8a751-b85f-443d-ad3c-58f5dd9f963f-000000@us-east-2.amazonses.com> Message-ID: <87h7c9vwfb.fsf@smart-cactus.org> Artem Pelenitsyn writes: > Another question would be where do I read about Haskell-native stack > unwinder. The issue and MR Ben referenced have descriptions, but the MR > didn't touch anything inside `docs` which is a bit scary. Are there any > good recourses to dive into it besides the source code in the MR? > Indeed the user's guide documentation surrounding info table provenance should be updated to note this new capability. There is a long Note in compiler/GHC/Driver/GenerateCgIPEStub.hs which describes the implementation. As far as the interface is concerned, it's quite straightforward: * One can call GHC.Stack.CloneStack.cloneMyStack, which returns a "frozen" copy of the calling context's stack * One can then call GHC.Stack.CloneStack.decode to decode the stack into a list of frames. Did you have any particular questions? Cheers, - Ben -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From sven.tennie at gmail.com Fri Nov 19 06:57:07 2021 From: sven.tennie at gmail.com (Sven Tennie) Date: Fri, 19 Nov 2021 07:57:07 +0100 Subject: DWARF support In-Reply-To: <87h7c9vwfb.fsf@smart-cactus.org> References: <05A36B9D-DD0F-4279-B992-F50E6D4E8BC9@richarde.dev> <010f017d2dc5c7f2-ae165255-a6b8-48fc-84e7-d8db6ae68958-000000@us-east-2.amazonses.com> <87o86hwki6.fsf@smart-cactus.org> <010f017d33dbe3f9-f5d8a751-b85f-443d-ad3c-58f5dd9f963f-000000@us-east-2.amazonses.com> <87h7c9vwfb.fsf@smart-cactus.org> Message-ID: Am Fr., 19. Nov. 2021 um 01:09 Uhr schrieb Ben Gamari : > Artem Pelenitsyn writes: > > > Another question would be where do I read about Haskell-native stack > > unwinder. The issue and MR Ben referenced have descriptions, but the MR > > didn't touch anything inside `docs` which is a bit scary. Are there any > > good recourses to dive into it besides the source code in the MR? > > > Indeed the user's guide documentation surrounding info table provenance > should be updated to note this new capability. > Hey Ben, Hey Artem, Hey all, Thanks for bringing this up! I'll add some documentation to the user's guide and will likely write a blog post covering the "material" from notes in a more approachable way. Two details to note regarding the IPE based backtrace mechanism: - It's based on return frames on the STG stack. So, it doesn't work for trivial programs. A return frame is produced when the scrutinee of a case expression is evaluated. The test [1] shows a working example. - To get backtraces containing references in libraries, those have to be compiled with `-finfo-table-map`, too. Now that I've got the attention of many GHC-Devs: A big Thank-You! goes to Ben and Matthew that were always reachable and always very helpful during the whole development process of this feature! Best regards, Sven [1] - https://gitlab.haskell.org/ghc/ghc/-/blob/aed98ddaf72cc38fb570d8415cac5de9d8888818/testsuite/tests/rts/decodeMyStack.hs -------------- next part -------------- An HTML attachment was scrubbed... URL: From a.pelenitsyn at gmail.com Fri Nov 19 15:05:30 2021 From: a.pelenitsyn at gmail.com (Artem Pelenitsyn) Date: Fri, 19 Nov 2021 10:05:30 -0500 Subject: DWARF support In-Reply-To: References: <05A36B9D-DD0F-4279-B992-F50E6D4E8BC9@richarde.dev> <010f017d2dc5c7f2-ae165255-a6b8-48fc-84e7-d8db6ae68958-000000@us-east-2.amazonses.com> <87o86hwki6.fsf@smart-cactus.org> <010f017d33dbe3f9-f5d8a751-b85f-443d-ad3c-58f5dd9f963f-000000@us-east-2.amazonses.com> <87h7c9vwfb.fsf@smart-cactus.org> Message-ID: Thanks everyone! A blog post would be awesome, indeed. Ben, I didn't have anything particular in mind, just curious about possibly more ergonomic alternatives to profiling builds for getting this sort of information. (Motivation similar to Richard's, I think.) On Fri, Nov 19, 2021, 1:57 AM Sven Tennie wrote: > Am Fr., 19. Nov. 2021 um 01:09 Uhr schrieb Ben Gamari < > ben at smart-cactus.org>: > >> Artem Pelenitsyn writes: >> >> > Another question would be where do I read about Haskell-native stack >> > unwinder. The issue and MR Ben referenced have descriptions, but the MR >> > didn't touch anything inside `docs` which is a bit scary. Are there any >> > good recourses to dive into it besides the source code in the MR? >> > >> Indeed the user's guide documentation surrounding info table provenance >> should be updated to note this new capability. >> > > Hey Ben, Hey Artem, Hey all, > > Thanks for bringing this up! I'll add some documentation to the user's > guide and will likely write a blog post covering the "material" from notes > in a more approachable way. > > Two details to note regarding the IPE based backtrace mechanism: > > - It's based on return frames on the STG stack. So, it doesn't work for > trivial programs. A return frame is produced when the scrutinee of a case > expression is evaluated. > The test [1] shows a working example. > > - To get backtraces containing references in libraries, those have to be > compiled with `-finfo-table-map`, too. > > Now that I've got the attention of many GHC-Devs: A big Thank-You! goes to > Ben and Matthew that were always reachable and always very helpful during > the whole development process of this feature! > > Best regards, > > Sven > > [1] - > https://gitlab.haskell.org/ghc/ghc/-/blob/aed98ddaf72cc38fb570d8415cac5de9d8888818/testsuite/tests/rts/decodeMyStack.hs > -------------- next part -------------- An HTML attachment was scrubbed... URL: From nr at cs.tufts.edu Fri Nov 19 18:34:14 2021 From: nr at cs.tufts.edu (Norman Ramsey) Date: Fri, 19 Nov 2021 13:34:14 -0500 Subject: How to build Haddock documentation quickly? In-Reply-To: <9a2ef2b2-5b25-1e3d-f9e8-10975793e714@glitchbra.in> (sfid-H-20211118-151216-+48.58-1@multi.osbf.lua) References: <20211118200635.B016E2C2ECB@homedog.cs.tufts.edu> <9a2ef2b2-5b25-1e3d-f9e8-10975793e714@glitchbra.in> (sfid-H-20211118-151216-+48.58-1@multi.osbf.lua) Message-ID: <20211119183414.4700C2C2F18@homedog.cs.tufts.edu> > Hi Norman, > > Could you see if the command-line described here is of any help? > https://gitlab.haskell.org/ghc/ghc/-/issues/17929#building-the-docs > > Particularly the "--freeze1 --flavour=Quick" bit. It was reasonably fast, but it doesn't build the documentation I'm interested in. The file I wish to rebuild is _build/docs/html/libraries/ghc/GHC-Tc-Gen-Foreign.html and the command recommended there is hadrian/build -j --freeze1 --flavour=Quick _build/docs/html/libraries/base/base.haddock Just adding `--freeze= --flavour=Quick` to my original command may have helped a little; after editing GHC/Tc/Gen/Foreign.hs, my build time was reduced to 3 minutes 11 seconds. I also took a flyer on these two commands: hadrian/build -j --freeze1 --flavour=Quick _build/docs/html/libraries/ghc/GHC-Tc-Gen-Foreign.html hadrian/build -j _build/docs/html/libraries/ghc/GHC-Tc-Gen-Foreign.html But neither of these commands rebuilt the file. I also tried hadrian/build -j --freeze1 --flavour=Quick --docs=no-sphinx docs which succeeded but took 3 minutes 18 seconds. I'm more than willing to dive into Hadrian and figure out how it works. I could even add a new target to build just what I'm interested in. But I would need help. I've spent some time poking around the `doc` directory, and I've read the Shake papers (and some of Andrei's work) but I've never used these tools myself. I got as far as $GHC/hadrian/src/Rules/Documentation.hs, which led me to try the `docs-haddock` target, and I got a slightly better result: the document is rebuilt in 2 minutes 30 seconds. There has to be a better way. I suppose I could try to come up with a set of options to `haddock` and to run it directly. But maybe someone can suggest another idea? Is there a command-line option to Hadrian that will get it to show every external command ("builder"?) as it is executed? Norman > Le 18/11/2021 à 21:06, Norman Ramsey a écrit : > > In service of #20528, I'm trying to clean up some Haddock > > documentation in the GHC sources. The only command I know is > > > > ./hadrian/build -j _build/docs/html/index.html > > > > It turns out that if I change Haddock comments in one file > > (GHC/Tc/Gen/Foreign.hs), rebuilding the HTML takes a minute > > and a quarter. That's a little long. (And I had one build > > take over four minutes, which is even longer.) > > > > What tricks can I use to speed up this process? Might there be a way > > to run Haddock directly until things are to my liking? From ietf-dane at dukhovni.org Sat Nov 20 00:56:34 2021 From: ietf-dane at dukhovni.org (Viktor Dukhovni) Date: Fri, 19 Nov 2021 19:56:34 -0500 Subject: Unexpected duplicate join points in "Core" output? Message-ID: The below "Core" output from "ghc -O2" (9.2/8.10) for the attached program shows seemingly rendundant join points: join { exit :: State# RealWorld -> (# State# RealWorld, () #) exit (ipv :: State# RealWorld) = jump $s$j ipv } in join { exit1 :: State# RealWorld -> (# State# RealWorld, () #) exit1 (ipv :: State# RealWorld) = jump $s$j ipv } in that are identical in all but name. These correspond to fallthrough to the "otherwise" case in: ... | acc < q || (acc == q && d <= 5) -> loop (ptr `plusPtr` 1) (acc * 10 + d) | otherwise -> return Nothing but it seems that the generated X86_64 code (also below) ultimately consolidates these into a single target... Is that why it is harmless to leave these duplicated in the generated "Core"? [ Separately, in the generated machine code, it'd also be nice to avoid comparing the same "q" with the accumulator twice. A single load and compare should I think be enough, as I'd expect the status flags to persist across the jump the second test. This happens to not be performance critical in my case, because most calls should satisfy the first test, but generally I think that 3-way "a < b", "a == b", "a > b" branches ideally avoid comparing twice... ] ======== Associated Core output -- RHS size: {terms: 1, types: 0, coercions: 0, joins: 0/0} main2 :: Addr# main2 = "12345678901234567890 junk"# -- RHS size: {terms: 129, types: 114, coercions: 0, joins: 6/8} main1 :: State# RealWorld -> (# State# RealWorld, () #) main1 = \ (eta :: State# RealWorld) -> let { end :: Addr# end = plusAddr# main2 25# } in join { $s$j :: State# RealWorld -> (# State# RealWorld, () #) $s$j _ = hPutStr2 stdout $fShowMaybe4 True eta } in join { exit :: State# RealWorld -> (# State# RealWorld, () #) exit (ipv :: State# RealWorld) = jump $s$j ipv } in join { exit1 :: State# RealWorld -> (# State# RealWorld, () #) exit1 (ipv :: State# RealWorld) = jump $s$j ipv } in join { exit2 :: Addr# -> Word# -> State# RealWorld -> (# State# RealWorld, () #) exit2 (ww :: Addr#) (ww1 :: Word#) (ipv :: State# RealWorld) = case eqAddr# ww main2 of { __DEFAULT -> hPutStr2 stdout (++ $fShowMaybe1 (case $w$cshowsPrec3 11# (integerFromWord# ww1) [] of { (# ww3, ww4 #) -> : ww3 ww4 })) True eta; 1# -> jump $s$j ipv } } in joinrec { $wloop :: Addr# -> Word# -> State# RealWorld -> (# State# RealWorld, () #) $wloop (ww :: Addr#) (ww1 :: Word#) (w :: State# RealWorld) = join { getDigit :: State# RealWorld -> (# State# RealWorld, () #) getDigit (eta1 :: State# RealWorld) = case eqAddr# ww end of { __DEFAULT -> case readWord8OffAddr# ww 0# eta1 of { (# ipv, ipv1 #) -> let { ipv2 :: Word# ipv2 = minusWord# (word8ToWord# ipv1) 48## } in case gtWord# ipv2 9## of { __DEFAULT -> case ltWord# ww1 1844674407370955161## of { __DEFAULT -> case ww1 of { __DEFAULT -> jump exit ipv; 1844674407370955161## -> case leWord# ipv2 5## of { __DEFAULT -> jump exit1 ipv; 1# -> jump $wloop (plusAddr# ww 1#) (plusWord# 18446744073709551610## ipv2) ipv } }; 1# -> jump $wloop (plusAddr# ww 1#) (plusWord# (timesWord# ww1 10##) ipv2) ipv }; 1# -> jump exit2 ww ww1 ipv } }; 1# -> jump exit2 ww ww1 eta1 } } in jump getDigit w; } in jump $wloop main2 0## realWorld# ======== Executable disassembly The jumps at "-1->" and "-2->" that correspond that "otherwise" have the same target. The duplicate "load+cmp" with "q" is at "-3->" and "-4->": 0000000000408de8 : 408de8: 48 8d 45 e8 lea -0x18(%rbp),%rax 408dec: 4c 39 f8 cmp %r15,%rax 408def: 0f 82 c8 00 00 00 jb 408ebd 408df5: b8 79 dd 77 00 mov $0x77dd79,%eax 408dfa: 31 db xor %ebx,%ebx 408dfc: b9 60 dd 77 00 mov $0x77dd60,%ecx 408e01: 48 39 c1 cmp %rax,%rcx 408e04: 74 66 je 408e6c 408e06: 0f b6 11 movzbl (%rcx),%edx 408e09: 48 83 c2 d0 add $0xffffffffffffffd0,%rdx 408e0d: 48 83 fa 09 cmp $0x9,%rdx 408e11: 77 59 ja 408e6c -3-> 408e13: 48 be 99 99 99 99 99 mov $0x1999999999999999,%rsi 408e1a: 99 99 19 408e1d: 48 39 f3 cmp %rsi,%rbx 408e20: 73 0c jae 408e2e 408e22: 48 6b db 0a imul $0xa,%rbx,%rbx 408e26: 48 01 d3 add %rdx,%rbx 408e29: 48 ff c1 inc %rcx 408e2c: eb d3 jmp 408e01 -4-> 408e2e: 48 be 99 99 99 99 99 mov $0x1999999999999999,%rsi 408e35: 99 99 19 408e38: 48 39 f3 cmp %rsi,%rbx -1-> 408e3b: 75 49 jne 408e86 408e3d: 48 83 fa 05 cmp $0x5,%rdx -2-> 408e41: 77 43 ja 408e86 408e43: 48 8d 5a fa lea -0x6(%rdx),%rbx 408e47: 48 ff c1 inc %rcx 408e4a: eb b5 jmp 408e01 408e4c: 0f 1f 40 00 nopl 0x0(%rax) 408e50: c2 00 00 retq $0x0 -- Viktor. -------------- next part -------------- A non-text attachment was scrubbed... Name: main.hs Type: text/x-haskell Size: 1527 bytes Desc: not available URL: From ietf-dane at dukhovni.org Sat Nov 20 01:02:49 2021 From: ietf-dane at dukhovni.org (Viktor Dukhovni) Date: Fri, 19 Nov 2021 20:02:49 -0500 Subject: [Take 2] Unexpected duplicate join points in "Core" output? Message-ID: [ Sorry wrong version of attachment in previous message. ] The below "Core" output from "ghc -O2" (9.2/8.10) for the attached program shows seemingly rendundant join points: join { exit :: State# RealWorld -> (# State# RealWorld, () #) exit (ipv :: State# RealWorld) = jump $s$j ipv } in join { exit1 :: State# RealWorld -> (# State# RealWorld, () #) exit1 (ipv :: State# RealWorld) = jump $s$j ipv } in that are identical in all but name. These correspond to fallthrough to the "otherwise" case in: ... | acc < q || (acc == q && d <= 5) -> loop (ptr `plusPtr` 1) (acc * 10 + d) | otherwise -> return Nothing but it seems that the generated X86_64 code (also below) ultimately consolidates these into a single target... Is that why it is harmless to leave these duplicated in the generated "Core"? [ Separately, in the generated machine code, it'd also be nice to avoid comparing the same "q" with the accumulator twice. A single load and compare should I think be enough, as I'd expect the status flags to persist across the jump the second test. This happens to not be performance critical in my case, because most calls should satisfy the first test, but generally I think that 3-way "a < b", "a == b", "a > b" branches ideally avoid comparing twice... ] ======== Associated Core output -- RHS size: {terms: 1, types: 0, coercions: 0, joins: 0/0} main2 :: Addr# main2 = "12345678901234567890 junk"# -- RHS size: {terms: 129, types: 114, coercions: 0, joins: 6/8} main1 :: State# RealWorld -> (# State# RealWorld, () #) main1 = \ (eta :: State# RealWorld) -> let { end :: Addr# end = plusAddr# main2 25# } in join { $s$j :: State# RealWorld -> (# State# RealWorld, () #) $s$j _ = hPutStr2 stdout $fShowMaybe4 True eta } in join { exit :: State# RealWorld -> (# State# RealWorld, () #) exit (ipv :: State# RealWorld) = jump $s$j ipv } in join { exit1 :: State# RealWorld -> (# State# RealWorld, () #) exit1 (ipv :: State# RealWorld) = jump $s$j ipv } in join { exit2 :: Addr# -> Word# -> State# RealWorld -> (# State# RealWorld, () #) exit2 (ww :: Addr#) (ww1 :: Word#) (ipv :: State# RealWorld) = case eqAddr# ww main2 of { __DEFAULT -> hPutStr2 stdout (++ $fShowMaybe1 (case $w$cshowsPrec3 11# (integerFromWord# ww1) [] of { (# ww3, ww4 #) -> : ww3 ww4 })) True eta; 1# -> jump $s$j ipv } } in joinrec { $wloop :: Addr# -> Word# -> State# RealWorld -> (# State# RealWorld, () #) $wloop (ww :: Addr#) (ww1 :: Word#) (w :: State# RealWorld) = join { getDigit :: State# RealWorld -> (# State# RealWorld, () #) getDigit (eta1 :: State# RealWorld) = case eqAddr# ww end of { __DEFAULT -> case readWord8OffAddr# ww 0# eta1 of { (# ipv, ipv1 #) -> let { ipv2 :: Word# ipv2 = minusWord# (word8ToWord# ipv1) 48## } in case gtWord# ipv2 9## of { __DEFAULT -> case ltWord# ww1 1844674407370955161## of { __DEFAULT -> case ww1 of { __DEFAULT -> jump exit ipv; 1844674407370955161## -> case leWord# ipv2 5## of { __DEFAULT -> jump exit1 ipv; 1# -> jump $wloop (plusAddr# ww 1#) (plusWord# 18446744073709551610## ipv2) ipv } }; 1# -> jump $wloop (plusAddr# ww 1#) (plusWord# (timesWord# ww1 10##) ipv2) ipv }; 1# -> jump exit2 ww ww1 ipv } }; 1# -> jump exit2 ww ww1 eta1 } } in jump getDigit w; } in jump $wloop main2 0## realWorld# ======== Executable disassembly The jumps at "-1->" and "-2->" that correspond that "otherwise" have the same target. The duplicate "load+cmp" with "q" is at "-3->" and "-4->": 0000000000408de8 : 408de8: 48 8d 45 e8 lea -0x18(%rbp),%rax 408dec: 4c 39 f8 cmp %r15,%rax 408def: 0f 82 c8 00 00 00 jb 408ebd 408df5: b8 79 dd 77 00 mov $0x77dd79,%eax 408dfa: 31 db xor %ebx,%ebx 408dfc: b9 60 dd 77 00 mov $0x77dd60,%ecx 408e01: 48 39 c1 cmp %rax,%rcx 408e04: 74 66 je 408e6c 408e06: 0f b6 11 movzbl (%rcx),%edx 408e09: 48 83 c2 d0 add $0xffffffffffffffd0,%rdx 408e0d: 48 83 fa 09 cmp $0x9,%rdx 408e11: 77 59 ja 408e6c -3-> 408e13: 48 be 99 99 99 99 99 mov $0x1999999999999999,%rsi 408e1a: 99 99 19 408e1d: 48 39 f3 cmp %rsi,%rbx 408e20: 73 0c jae 408e2e 408e22: 48 6b db 0a imul $0xa,%rbx,%rbx 408e26: 48 01 d3 add %rdx,%rbx 408e29: 48 ff c1 inc %rcx 408e2c: eb d3 jmp 408e01 -4-> 408e2e: 48 be 99 99 99 99 99 mov $0x1999999999999999,%rsi 408e35: 99 99 19 408e38: 48 39 f3 cmp %rsi,%rbx -1-> 408e3b: 75 49 jne 408e86 408e3d: 48 83 fa 05 cmp $0x5,%rdx -2-> 408e41: 77 43 ja 408e86 408e43: 48 8d 5a fa lea -0x6(%rdx),%rbx 408e47: 48 ff c1 inc %rcx 408e4a: eb b5 jmp 408e01 408e4c: 0f 1f 40 00 nopl 0x0(%rax) 408e50: c2 00 00 retq $0x0 -- Viktor. -------------- next part -------------- A non-text attachment was scrubbed... Name: main.hs Type: text/x-haskell Size: 1460 bytes Desc: not available URL: From klebinger.andreas at gmx.at Sat Nov 20 11:49:08 2021 From: klebinger.andreas at gmx.at (Andreas Klebinger) Date: Sat, 20 Nov 2021 12:49:08 +0100 Subject: [Take 2] Unexpected duplicate join points in "Core" output? In-Reply-To: References: Message-ID: <6558d36b-283e-3fd4-f8ec-fce3f49f1900@gmx.at> Hello Victor, generally GHC does try to common up join points and duplicate expressions like that. But since that's relatively expensive most of the duplication happens during the core-cse pass which only happens once. We don't create them because they are harmless. They are simple a side product of optimizations happening after the main cse pass has run. There is no feasible way to fix this I think. As you say with some luck they get caught at the Cmm stage and deduplicated there. Sadly it doesn't always happen. In most cases the impact of this is thankfully rather small. For the assembly I opened a ticket: https://gitlab.haskell.org/ghc/ghc/-/issues/20714 Am 20/11/2021 um 02:02 schrieb Viktor Dukhovni: > [ Sorry wrong version of attachment in previous message. ] > > The below "Core" output from "ghc -O2" (9.2/8.10) for the attached > program shows seemingly rendundant join points: > > join { > exit :: State# RealWorld -> (# State# RealWorld, () #) > exit (ipv :: State# RealWorld) = jump $s$j ipv } in > > join { > exit1 :: State# RealWorld -> (# State# RealWorld, () #) > exit1 (ipv :: State# RealWorld) = jump $s$j ipv } in > > that are identical in all but name. These correspond to fallthrough > to the "otherwise" case in: > > ... > | acc < q || (acc == q && d <= 5) > -> loop (ptr `plusPtr` 1) (acc * 10 + d) > | otherwise -> return Nothing > > but it seems that the generated X86_64 code (also below) ultimately > consolidates these into a single target... Is that why it is harmless to > leave these duplicated in the generated "Core"? > > [ Separately, in the generated machine code, it'd also be nice to avoid > comparing the same "q" with the accumulator twice. A single load and > compare should I think be enough, as I'd expect the status flags to > persist across the jump the second test. > > This happens to not be performance critical in my case, because most > calls should satisfy the first test, but generally I think that 3-way > "a < b", "a == b", "a > b" branches ideally avoid comparing twice... ] > > ======== Associated Core output > > -- RHS size: {terms: 1, types: 0, coercions: 0, joins: 0/0} > main2 :: Addr# > main2 = "12345678901234567890 junk"# > > -- RHS size: {terms: 129, types: 114, coercions: 0, joins: 6/8} > main1 :: State# RealWorld -> (# State# RealWorld, () #) > main1 > = \ (eta :: State# RealWorld) -> > let { > end :: Addr# > end = plusAddr# main2 25# } in > join { > $s$j :: State# RealWorld -> (# State# RealWorld, () #) > $s$j _ = hPutStr2 stdout $fShowMaybe4 True eta } in > join { > exit :: State# RealWorld -> (# State# RealWorld, () #) > exit (ipv :: State# RealWorld) = jump $s$j ipv } in > join { > exit1 :: State# RealWorld -> (# State# RealWorld, () #) > exit1 (ipv :: State# RealWorld) = jump $s$j ipv } in > join { > exit2 > :: Addr# -> Word# -> State# RealWorld -> (# State# RealWorld, () #) > exit2 (ww :: Addr#) (ww1 :: Word#) (ipv :: State# RealWorld) > = case eqAddr# ww main2 of { > __DEFAULT -> > hPutStr2 > stdout > (++ > $fShowMaybe1 > (case $w$cshowsPrec3 11# (integerFromWord# ww1) [] of > { (# ww3, ww4 #) -> > : ww3 ww4 > })) > True > eta; > 1# -> jump $s$j ipv > } } in > joinrec { > $wloop > :: Addr# -> Word# -> State# RealWorld -> (# State# RealWorld, () #) > $wloop (ww :: Addr#) (ww1 :: Word#) (w :: State# RealWorld) > = join { > getDigit :: State# RealWorld -> (# State# RealWorld, () #) > getDigit (eta1 :: State# RealWorld) > = case eqAddr# ww end of { > __DEFAULT -> > case readWord8OffAddr# ww 0# eta1 of { (# ipv, ipv1 #) -> > let { > ipv2 :: Word# > ipv2 = minusWord# (word8ToWord# ipv1) 48## } in > case gtWord# ipv2 9## of { > __DEFAULT -> > case ltWord# ww1 1844674407370955161## of { > __DEFAULT -> > case ww1 of { > __DEFAULT -> jump exit ipv; > 1844674407370955161## -> > case leWord# ipv2 5## of { > __DEFAULT -> jump exit1 ipv; > 1# -> > jump $wloop > (plusAddr# ww 1#) > (plusWord# 18446744073709551610## ipv2) > ipv > } > }; > 1# -> > jump $wloop > (plusAddr# ww 1#) (plusWord# (timesWord# ww1 10##) ipv2) ipv > }; > 1# -> jump exit2 ww ww1 ipv > } > }; > 1# -> jump exit2 ww ww1 eta1 > } } in > jump getDigit w; } in > jump $wloop main2 0## realWorld# > > ======== Executable disassembly > > The jumps at "-1->" and "-2->" that correspond that "otherwise" have the > same target. The duplicate "load+cmp" with "q" is at "-3->" and "-4->": > > 0000000000408de8 : > 408de8: 48 8d 45 e8 lea -0x18(%rbp),%rax > 408dec: 4c 39 f8 cmp %r15,%rax > 408def: 0f 82 c8 00 00 00 jb 408ebd > 408df5: b8 79 dd 77 00 mov $0x77dd79,%eax > 408dfa: 31 db xor %ebx,%ebx > 408dfc: b9 60 dd 77 00 mov $0x77dd60,%ecx > 408e01: 48 39 c1 cmp %rax,%rcx > 408e04: 74 66 je 408e6c > 408e06: 0f b6 11 movzbl (%rcx),%edx > 408e09: 48 83 c2 d0 add $0xffffffffffffffd0,%rdx > 408e0d: 48 83 fa 09 cmp $0x9,%rdx > 408e11: 77 59 ja 408e6c > -3-> 408e13: 48 be 99 99 99 99 99 mov $0x1999999999999999,%rsi > 408e1a: 99 99 19 > 408e1d: 48 39 f3 cmp %rsi,%rbx > 408e20: 73 0c jae 408e2e > 408e22: 48 6b db 0a imul $0xa,%rbx,%rbx > 408e26: 48 01 d3 add %rdx,%rbx > 408e29: 48 ff c1 inc %rcx > 408e2c: eb d3 jmp 408e01 > -4-> 408e2e: 48 be 99 99 99 99 99 mov $0x1999999999999999,%rsi > 408e35: 99 99 19 > 408e38: 48 39 f3 cmp %rsi,%rbx > -1-> 408e3b: 75 49 jne 408e86 > 408e3d: 48 83 fa 05 cmp $0x5,%rdx > -2-> 408e41: 77 43 ja 408e86 > 408e43: 48 8d 5a fa lea -0x6(%rdx),%rbx > 408e47: 48 ff c1 inc %rcx > 408e4a: eb b5 jmp 408e01 > 408e4c: 0f 1f 40 00 nopl 0x0(%rax) > 408e50: c2 00 00 retq $0x0 > > > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs -------------- next part -------------- An HTML attachment was scrubbed... URL: From andrey.mokhov at newcastle.ac.uk Sat Nov 20 18:44:50 2021 From: andrey.mokhov at newcastle.ac.uk (Andrey Mokhov) Date: Sat, 20 Nov 2021 18:44:50 +0000 Subject: How to build Haddock documentation quickly? Message-ID: Hi Norman, > I'm more than willing to dive into Hadrian and figure out how it works. > I could even add a new target to build just what I'm interested in. > But I would need help. I've spent some time poking around the `doc` directory, and I've read > the Shake papers (and some of Andrei's work) but I've never used these tools myself. I would be delighted to help you (or anyone else!) navigate Hadrian source code. Please feel free to get in touch directly. (Alas, in the last couple of years I couldn't contribute to improving Hadrian but that's not for the lack of desire - just due to some life changes. Hopefully I can at least help by helping others!) Cheers, Andrey -------------- next part -------------- An HTML attachment was scrubbed... URL: From ietf-dane at dukhovni.org Sat Nov 20 18:54:36 2021 From: ietf-dane at dukhovni.org (Viktor Dukhovni) Date: Sat, 20 Nov 2021 13:54:36 -0500 Subject: [Take 2] Unexpected duplicate join points in "Core" output? In-Reply-To: <6558d36b-283e-3fd4-f8ec-fce3f49f1900@gmx.at> References: <6558d36b-283e-3fd4-f8ec-fce3f49f1900@gmx.at> Message-ID: On Sat, Nov 20, 2021 at 12:49:08PM +0100, Andreas Klebinger wrote: > For the assembly I opened a ticket: > https://gitlab.haskell.org/ghc/ghc/-/issues/20714 Thanks, much appreciated. Understood re redundant join points, though in the non-toy context the redundnat point code is noticeably larger. join { exit4 :: Addr# -> Word# -> State# RealWorld -> Maybe (Int64, ByteString) exit4 (ww4 :: Addr#) (ww5 :: Word#) (ipv :: State# RealWorld) = case touch# dt1 ipv of { __DEFAULT -> let { dt3 :: Int# dt3 = minusAddr# ww4 dt } in case ==# dt3 dt2 of { __DEFAULT -> jump exit1 ww2 wild dt dt1 dt2 cs dt3 ww5; 1# -> jump $wconsume cs (orI# ww2 dt3) ww5 } } } in join { exit5 :: Addr# -> Word# -> State# RealWorld -> Maybe (Int64, ByteString) exit5 (ww4 :: Addr#) (ww5 :: Word#) (w1 :: State# RealWorld) = case touch# dt1 w1 of { __DEFAULT -> let { dt3 :: Int# dt3 = minusAddr# ww4 dt } in case ==# dt3 dt2 of { __DEFAULT -> jump exit1 ww2 wild dt dt1 dt2 cs dt3 ww5; 1# -> jump $wconsume cs (orI# ww2 dt3) ww5 } } } in FWIW, these don't appear to be deduplicated, both result from the same conditional: `acc < q || acc == q && d < 5`. I need some way to make this compute a single boolean value without forking the continuation. There's a another source of code bloat that I'd like to run by you... In the WIP code for Lazy ByteString 'readInt', I started with: readInt !q !r = \ !s -> consume s False 0 where -- All done consume s at Empty !valid !acc = if valid then convert acc s else Nothing -- skip empty chunk consume (Chunk (BI.BS _ 0) cs) !valid !acc -- Recurse = consume cs valid acc -- process non-empty chunk consume s@(Chunk c@(BI.BS _ !len) cs) !valid !acc = case _digits q r c acc of Result used acc' | used <= 0 -- No more digits present -> if valid then convert acc' s else Nothing | used < len -- valid input not entirely digits -> let !c' = BU.unsafeDrop used c in convert acc' $ Chunk c' cs | otherwise -- try to read more digits -- Recurse -> consume cs True acc' Overflow -> Nothing Now _digits is the I/O loop I shared before, and the calling code gets inlined into that recursive loop with various join points. But the loop gets forked into multiple copies which are compiled separately, because there are two different recursive calls into "consume" that got compiled into separate "joinrec { ... }". So I tried instead: readInt !q !r = \ !s -> consume s False 0 where -- All done consume s at Empty !valid !acc = if valid then convert acc s else Nothing consume s@(Chunk c@(BI.BS _ !len) cs) !valid !acc = case _digits q r c acc of Result used acc' | used == len -- try to read more digits -- Recurse -> consume cs (valid || used > 0) acc' | used > 0 -- valid input not entirely digits -> let !c' = BU.unsafeDrop used c in convert acc' $ Chunk c' cs | otherwise -- No more digits present -> if valid then convert acc' s else Nothing Overflow -> Nothing But was slightly surprised to find even more duplication (3 copies instead of tw) of the I/O loop, because in the call: consume cs (valid || used > 0) acc' the boolean argument got floated out, giving: case valid of True -> consume cs True acc' _ -> case used > 0 of True -> consume cs True acc' _ -> consume cs False acc' and each of these then generates essentially the same code. To get the code to be emitted just once, I had to switch from a Bool "valid" to a bitwise "valid": readInt !q !r = \ !s -> consume s 0 0 where -- All done consume s at Empty !valid !acc = if valid /= 0 then convert acc s else Nothing consume s@(Chunk c@(BI.BS _ !len) cs) !valid !acc = case _digits q r c acc of Result used acc' | used == len -- try to read more digits -- Recurse -> consume cs (valid .|. used) acc' | used > 0 -- valid input not entirely digits -> let !c' = BU.unsafeDrop used c in convert acc' $ Chunk c' cs | otherwise -- No more digits present -> if valid /= 0 then convert acc' s else Nothing Overflow -> Nothing Is there some way for GHC to figure out to not float out such cheap computations? The 'Result' constructor is strict, so there's no cost to evaluating `used > 0`, and cloning the entire computation is I think the more unfortunate choice... Adding redundant BangPatterns on `Result !used !acc'` didn't make a difference. Switching to bitwise logical "or" finally produces just one copy of the loop. -- Viktor. From ietf-dane at dukhovni.org Sat Nov 20 19:36:40 2021 From: ietf-dane at dukhovni.org (Viktor Dukhovni) Date: Sat, 20 Nov 2021 14:36:40 -0500 Subject: [Take 2] Unexpected duplicate join points in "Core" output? In-Reply-To: References: <6558d36b-283e-3fd4-f8ec-fce3f49f1900@gmx.at> Message-ID: On Sat, Nov 20, 2021 at 01:54:36PM -0500, Viktor Dukhovni wrote: > Is there some way for GHC to figure out to not float out such cheap > computations? The 'Result' constructor is strict, so there's no cost to > evaluating `used > 0`, and cloning the entire computation is I think > the more unfortunate choice... I managed to get the loop to not emit duplicate code bloat by inserting another NOINLINE term: !keepGoing = acc < q || acc == q && d <= r {-# NOINLINE keepGoing #-} Thus the below produces Core with no significant bloat, matching roughly what one might (reasonably?/naively?) expect. But I am reluctant to actually include such work-arounds in the PR, the code that produces more "bloated" Core is easier to understand and maintain... _digits :: Accum -> Accum -> BI.ByteString -> Accum -> Result {-# INLINE _digits #-} _digits !q !r !(BI.BS !fp !len) = \ !acc -> BI.accursedUnutterablePerformIO $ BI.unsafeWithForeignPtr fp $ \ptr -> do let end = ptr `plusPtr` len go ptr end ptr acc where go start end = loop where loop !ptr !acc | ptr == end = return $ Result (ptr `minusPtr` start) acc loop !ptr !acc = getDigit >>= \ !d -> if | d <= 9 -> update d | otherwise -> return $ Result (ptr `minusPtr` start) acc where fromDigit = \w -> fromIntegral w - 0x30 -- i.e. w - '0' -- {-# NOINLINE getDigit #-} getDigit | ptr /= end = fromDigit <$> peek ptr | otherwise = pure 10 -- End of input -- update d | keepGoing = loop (ptr `plusPtr` 1) (acc * 10 + d) | otherwise = return Overflow where {-# NOINLINE keepGoing #-} !keepGoing = acc < q || acc == q && d <= r The Core code is now, with the duplicate comparison as the only visible inefficiency. -- The exit/exit3 joins could be combined but are small, -- ditto with exit1/exit2. Rec { -- RHS size: {terms: 190, types: 146, coercions: 0, joins: 8/10} $wconsume :: ByteString -> Int# -> Word# -> Maybe (Word64, ByteString) $wconsume = \ (w :: ByteString) (ww :: Int#) (ww1 :: Word#) -> case w of wild { Empty -> case ww of { __DEFAULT -> Just (W64# ww1, Empty); 0# -> Nothing }; Chunk dt dt1 dt2 cs -> let { end :: Addr# end = plusAddr# dt dt2 } in join { $s$j :: Int# -> Word# -> State# RealWorld -> Maybe (Word64, ByteString) $s$j (sc :: Int#) (sc1 :: Word#) (sc2 :: State# RealWorld) = case touch# dt1 sc2 of { __DEFAULT -> case ==# sc dt2 of { __DEFAULT -> case ># sc 0# of { __DEFAULT -> case ww of { __DEFAULT -> Just (W64# sc1, wild); 0# -> Nothing }; 1# -> Just (W64# sc1, Chunk (plusAddr# dt sc) dt1 (-# dt2 sc) cs) }; 1# -> $wconsume cs (orI# ww sc) sc1 } } } in join { exit :: Addr# -> Word# -> State# RealWorld -> Maybe (Word64, ByteString) exit (ww2 :: Addr#) (ww3 :: Word#) (ipv :: State# RealWorld) = jump $s$j (minusAddr# ww2 dt) ww3 ipv } in join { exit1 :: State# RealWorld -> Maybe (Word64, ByteString) exit1 (ipv :: State# RealWorld) = case touch# dt1 ipv of { __DEFAULT -> Nothing } } in join { exit2 :: State# RealWorld -> Maybe (Word64, ByteString) exit2 (ipv :: State# RealWorld) = case touch# dt1 ipv of { __DEFAULT -> Nothing } } in join { exit3 :: Addr# -> Word# -> State# RealWorld -> Maybe (Word64, ByteString) exit3 (ww2 :: Addr#) (ww3 :: Word#) (w1 :: State# RealWorld) = jump $s$j (minusAddr# ww2 dt) ww3 w1 } in joinrec { $wloop :: Addr# -> Word# -> State# RealWorld -> Maybe (Word64, ByteString) $wloop (ww2 :: Addr#) (ww3 :: Word#) (w1 :: State# RealWorld) = case eqAddr# ww2 end of { __DEFAULT -> join { getDigit :: State# RealWorld -> Maybe (Word64, ByteString) getDigit (eta :: State# RealWorld) = case readWord8OffAddr# ww2 0# eta of { (# ipv, ipv1 #) -> let { ipv2 :: Word# ipv2 = minusWord# (word8ToWord# ipv1) 48## } in case leWord# ipv2 9## of { __DEFAULT -> jump exit ww2 ww3 ipv; 1# -> join { keepGoing :: Maybe (Word64, ByteString) keepGoing = case ltWord# ww3 1844674407370955161## of { __DEFAULT -> case ww3 of { __DEFAULT -> jump exit1 ipv; 1844674407370955161## -> case leWord# ipv2 5## of { __DEFAULT -> jump exit2 ipv; 1# -> jump $wloop (plusAddr# ww2 1#) (plusWord# 18446744073709551610## ipv2) ipv } }; 1# -> jump $wloop (plusAddr# ww2 1#) (plusWord# (timesWord# ww3 10##) ipv2) ipv } } in jump keepGoing } } } in jump getDigit w1; 1# -> jump exit3 ww2 ww3 w1 }; } in jump $wloop dt ww1 realWorld# } end Rec } -- Viktor. From klebinger.andreas at gmx.at Sat Nov 20 20:47:36 2021 From: klebinger.andreas at gmx.at (Andreas Klebinger) Date: Sat, 20 Nov 2021 21:47:36 +0100 Subject: [Take 2] Unexpected duplicate join points in "Core" output? In-Reply-To: References: <6558d36b-283e-3fd4-f8ec-fce3f49f1900@gmx.at> Message-ID: At this point I think it would be good if you could put your problem into a ghc-ticket. I can't look in detail into this in greater detail atm because of time constraints. And without a ticket it's likely to fall by the wayside eventually. But it does seem like something where we maybe could do better. And having good examples for the problematic behaviour is always immensely helpful to solve these kinds of problems. Cheers Andreas Am 20/11/2021 um 19:54 schrieb Viktor Dukhovni: > On Sat, Nov 20, 2021 at 12:49:08PM +0100, Andreas Klebinger wrote: > >> For the assembly I opened a ticket: >> https://gitlab.haskell.org/ghc/ghc/-/issues/20714 > Thanks, much appreciated. Understood re redundant join points, though > in the non-toy context the redundnat point code is noticeably larger. > > join { > exit4 > :: Addr# -> Word# -> State# RealWorld -> Maybe (Int64, ByteString) > exit4 (ww4 :: Addr#) (ww5 :: Word#) (ipv :: State# RealWorld) > = case touch# dt1 ipv of { __DEFAULT -> > let { > dt3 :: Int# > dt3 = minusAddr# ww4 dt } in > case ==# dt3 dt2 of { > __DEFAULT -> jump exit1 ww2 wild dt dt1 dt2 cs dt3 ww5; > 1# -> jump $wconsume cs (orI# ww2 dt3) ww5 > } > } } in > join { > exit5 > :: Addr# -> Word# -> State# RealWorld -> Maybe (Int64, ByteString) > exit5 (ww4 :: Addr#) (ww5 :: Word#) (w1 :: State# RealWorld) > = case touch# dt1 w1 of { __DEFAULT -> > let { > dt3 :: Int# > dt3 = minusAddr# ww4 dt } in > case ==# dt3 dt2 of { > __DEFAULT -> jump exit1 ww2 wild dt dt1 dt2 cs dt3 ww5; > 1# -> jump $wconsume cs (orI# ww2 dt3) ww5 > } > } } in > > FWIW, these don't appear to be deduplicated, both result from the same > conditional: `acc < q || acc == q && d < 5`. I need some way to make > this compute a single boolean value without forking the continuation. > > There's a another source of code bloat that I'd like to run by you... > In the WIP code for Lazy ByteString 'readInt', I started with: > > readInt !q !r = > \ !s -> consume s False 0 > where > -- All done > consume s at Empty !valid !acc > = if valid then convert acc s else Nothing > -- skip empty chunk > consume (Chunk (BI.BS _ 0) cs) !valid !acc > -- Recurse > = consume cs valid acc > -- process non-empty chunk > consume s@(Chunk c@(BI.BS _ !len) cs) !valid !acc > = case _digits q r c acc of > Result used acc' > | used <= 0 -- No more digits present > -> if valid then convert acc' s else Nothing > | used < len -- valid input not entirely digits > -> let !c' = BU.unsafeDrop used c > in convert acc' $ Chunk c' cs > | otherwise -- try to read more digits > -- Recurse > -> consume cs True acc' > Overflow -> Nothing > > Now _digits is the I/O loop I shared before, and the calling code gets > inlined into that recursive loop with various join points. But the loop > gets forked into multiple copies which are compiled separately, because > there are two different recursive calls into "consume" that got compiled > into separate "joinrec { ... }". > > So I tried instead: > > readInt !q !r = > \ !s -> consume s False 0 > where > -- All done > consume s at Empty !valid !acc > = if valid then convert acc s else Nothing > consume s@(Chunk c@(BI.BS _ !len) cs) !valid !acc > = case _digits q r c acc of > Result used acc' > | used == len -- try to read more digits > -- Recurse > -> consume cs (valid || used > 0) acc' > | used > 0 -- valid input not entirely digits > -> let !c' = BU.unsafeDrop used c > in convert acc' $ Chunk c' cs > | otherwise -- No more digits present > -> if valid then convert acc' s else Nothing > Overflow -> Nothing > > But was slightly surprised to find even more duplication (3 copies > instead of tw) of the I/O loop, because in the call: > > consume cs (valid || used > 0) acc' > > the boolean argument got floated out, giving: > > case valid of > True -> consume cs True acc' > _ -> case used > 0 of > True -> consume cs True acc' > _ -> consume cs False acc' > > and each of these then generates essentially the same code. To get the > code to be emitted just once, I had to switch from a Bool "valid" to a > bitwise "valid": > > readInt !q !r = > \ !s -> consume s 0 0 > where > -- All done > consume s at Empty !valid !acc > = if valid /= 0 then convert acc s else Nothing > consume s@(Chunk c@(BI.BS _ !len) cs) !valid !acc > = case _digits q r c acc of > Result used acc' > | used == len -- try to read more digits > -- Recurse > -> consume cs (valid .|. used) acc' > | used > 0 -- valid input not entirely digits > -> let !c' = BU.unsafeDrop used c > in convert acc' $ Chunk c' cs > | otherwise -- No more digits present > -> if valid /= 0 then convert acc' s else Nothing > Overflow -> Nothing > > Is there some way for GHC to figure out to not float out such cheap > computations? The 'Result' constructor is strict, so there's no cost to > evaluating `used > 0`, and cloning the entire computation is I think > the more unfortunate choice... > > Adding redundant BangPatterns on `Result !used !acc'` didn't make a > difference. Switching to bitwise logical "or" finally produces just one > copy of the loop. > From simonpj at microsoft.com Sat Nov 20 21:15:15 2021 From: simonpj at microsoft.com (Simon Peyton Jones) Date: Sat, 20 Nov 2021 21:15:15 +0000 Subject: [EXTERNAL] Unexpected duplicate join points in "Core" output? In-Reply-To: References: Message-ID: There is absolutely no reason not to common-up those to join points. But we can't common up some join points when we could if they were let's. Consider join j1 x = x+1 in case v of A -> f (join j2 x = x+1 in ...j2...) B -> ....j1... C -> ....j1... Even though j2 is identical to j1's, we can't eliminate j2 in favour of j1 because then j1 wouldn't be a join point any more. GHC.Core.Opt.CSE is conservative at the moment, and never CSE's *any* join point. It would not be hard to make it clever enough to CSE join points, but no one has yet done it. Do open a ticket! Simon PS: I am leaving Microsoft at the end of November 2021, at which point simonpj at microsoft.com will cease to work. Use simon.peytonjones at gmail.com instead. (For now, it just forwards to simonpj at microsoft.com.) | -----Original Message----- | From: ghc-devs On Behalf Of Viktor | Dukhovni | Sent: 20 November 2021 00:57 | To: ghc-devs at haskell.org | Subject: [EXTERNAL] Unexpected duplicate join points in "Core" output? | | The below "Core" output from "ghc -O2" (9.2/8.10) for the attached | program shows seemingly rendundant join points: | | join { | exit :: State# RealWorld -> (# State# RealWorld, () #) | exit (ipv :: State# RealWorld) = jump $s$j ipv } in | | join { | exit1 :: State# RealWorld -> (# State# RealWorld, () #) | exit1 (ipv :: State# RealWorld) = jump $s$j ipv } in | | that are identical in all but name. These correspond to fallthrough to | the "otherwise" case in: | | ... | | acc < q || (acc == q && d <= 5) | -> loop (ptr `plusPtr` 1) (acc * 10 + d) | | otherwise -> return Nothing | | but it seems that the generated X86_64 code (also below) ultimately | consolidates these into a single target... Is that why it is harmless | to leave these duplicated in the generated "Core"? | | [ Separately, in the generated machine code, it'd also be nice to avoid | comparing the same "q" with the accumulator twice. A single load and | compare should I think be enough, as I'd expect the status flags to | persist across the jump the second test. | | This happens to not be performance critical in my case, because most | calls should satisfy the first test, but generally I think that 3-way | "a < b", "a == b", "a > b" branches ideally avoid comparing twice... | ] | | ======== Associated Core output | | -- RHS size: {terms: 1, types: 0, coercions: 0, joins: 0/0} | main2 :: Addr# | main2 = "12345678901234567890 junk"# | | -- RHS size: {terms: 129, types: 114, coercions: 0, joins: 6/8} | main1 :: State# RealWorld -> (# State# RealWorld, () #) | main1 | = \ (eta :: State# RealWorld) -> | let { | end :: Addr# | end = plusAddr# main2 25# } in | join { | $s$j :: State# RealWorld -> (# State# RealWorld, () #) | $s$j _ = hPutStr2 stdout $fShowMaybe4 True eta } in | join { | exit :: State# RealWorld -> (# State# RealWorld, () #) | exit (ipv :: State# RealWorld) = jump $s$j ipv } in | join { | exit1 :: State# RealWorld -> (# State# RealWorld, () #) | exit1 (ipv :: State# RealWorld) = jump $s$j ipv } in | join { | exit2 | :: Addr# -> Word# -> State# RealWorld -> (# State# | RealWorld, () #) | exit2 (ww :: Addr#) (ww1 :: Word#) (ipv :: State# | RealWorld) | = case eqAddr# ww main2 of { | __DEFAULT -> | hPutStr2 | stdout | (++ | $fShowMaybe1 | (case $w$cshowsPrec3 11# (integerFromWord# | ww1) [] of | { (# ww3, ww4 #) -> | : ww3 ww4 | })) | True | eta; | 1# -> jump $s$j ipv | } } in | joinrec { | $wloop | :: Addr# -> Word# -> State# RealWorld -> (# State# | RealWorld, () #) | $wloop (ww :: Addr#) (ww1 :: Word#) (w :: State# RealWorld) | = join { | getDigit :: State# RealWorld -> (# State# RealWorld, | () #) | getDigit (eta1 :: State# RealWorld) | = case eqAddr# ww end of { | __DEFAULT -> | case readWord8OffAddr# ww 0# eta1 of { (# | ipv, ipv1 #) -> | let { | ipv2 :: Word# | ipv2 = minusWord# (word8ToWord# ipv1) 48## | } in | case gtWord# ipv2 9## of { | __DEFAULT -> | case ltWord# ww1 1844674407370955161## of | { | __DEFAULT -> | case ww1 of { | __DEFAULT -> jump exit ipv; | 1844674407370955161## -> | case leWord# ipv2 5## of { | __DEFAULT -> jump exit1 ipv; | 1# -> | jump $wloop | (plusAddr# ww 1#) | (plusWord# | 18446744073709551610## ipv2) | ipv | } | }; | 1# -> | jump $wloop | (plusAddr# ww 1#) (plusWord# | (timesWord# ww1 10##) ipv2) ipv | }; | 1# -> jump exit2 ww ww1 ipv | } | }; | 1# -> jump exit2 ww ww1 eta1 | } } in | jump getDigit w; } in | jump $wloop main2 0## realWorld# | | ======== Executable disassembly | | The jumps at "-1->" and "-2->" that correspond that "otherwise" have | the same target. The duplicate "load+cmp" with "q" is at "-3->" and "- | 4->": | | 0000000000408de8 : | 408de8: 48 8d 45 e8 lea -0x18(%rbp),%rax | 408dec: 4c 39 f8 cmp %r15,%rax | 408def: 0f 82 c8 00 00 00 jb 408ebd | | 408df5: b8 79 dd 77 00 mov $0x77dd79,%eax | 408dfa: 31 db xor %ebx,%ebx | 408dfc: b9 60 dd 77 00 mov $0x77dd60,%ecx | 408e01: 48 39 c1 cmp %rax,%rcx | 408e04: 74 66 je 408e6c | | 408e06: 0f b6 11 movzbl (%rcx),%edx | 408e09: 48 83 c2 d0 add | $0xffffffffffffffd0,%rdx | 408e0d: 48 83 fa 09 cmp $0x9,%rdx | 408e11: 77 59 ja 408e6c | | -3-> 408e13: 48 be 99 99 99 99 99 mov | $0x1999999999999999,%rsi | 408e1a: 99 99 19 | 408e1d: 48 39 f3 cmp %rsi,%rbx | 408e20: 73 0c jae 408e2e | | 408e22: 48 6b db 0a imul $0xa,%rbx,%rbx | 408e26: 48 01 d3 add %rdx,%rbx | 408e29: 48 ff c1 inc %rcx | 408e2c: eb d3 jmp 408e01 | | -4-> 408e2e: 48 be 99 99 99 99 99 mov | $0x1999999999999999,%rsi | 408e35: 99 99 19 | 408e38: 48 39 f3 cmp %rsi,%rbx | -1-> 408e3b: 75 49 jne 408e86 | | 408e3d: 48 83 fa 05 cmp $0x5,%rdx | -2-> 408e41: 77 43 ja 408e86 | | 408e43: 48 8d 5a fa lea -0x6(%rdx),%rbx | 408e47: 48 ff c1 inc %rcx | 408e4a: eb b5 jmp 408e01 | | 408e4c: 0f 1f 40 00 nopl 0x0(%rax) | 408e50: c2 00 00 retq $0x0 | | -- | Viktor. From ietf-dane at dukhovni.org Sun Nov 21 00:28:23 2021 From: ietf-dane at dukhovni.org (Viktor Dukhovni) Date: Sat, 20 Nov 2021 19:28:23 -0500 Subject: [EXTERNAL] Unexpected duplicate join points in "Core" output? In-Reply-To: References: Message-ID: On Sat, Nov 20, 2021 at 09:15:15PM +0000, Simon Peyton Jones via ghc-devs wrote: > GHC.Core.Opt.CSE is conservative at the moment, and never CSE's *any* > join point. It would not be hard to make it clever enough to CSE join > points, but no one has yet done it. > > Do open a ticket! Thanks, I opened https://gitlab.haskell.org/ghc/ghc/-/issues/20717 -- Viktor. From carter.schonwald at gmail.com Sun Nov 21 11:53:53 2021 From: carter.schonwald at gmail.com (Carter Schonwald) Date: Sun, 21 Nov 2021 06:53:53 -0500 Subject: [EXTERNAL] Unexpected duplicate join points in "Core" output? In-Reply-To: References: Message-ID: In this example: why would it stop being a join point ? Admittedly, my intuition might be skewed by my own ideas about how join points are sortah a semantic special case of other constructs. On Sat, Nov 20, 2021 at 4:17 PM Simon Peyton Jones via ghc-devs < ghc-devs at haskell.org> wrote: > There is absolutely no reason not to common-up those to join points. But > we can't common up some join points when we could if they were let's. > Consider > > join j1 x = x+1 > in case v of > A -> f (join j2 x = x+1 in ...j2...) > B -> ....j1... > C -> ....j1... > > Even though j2 is identical to j1's, we can't eliminate j2 in favour of j1 > because then j1 wouldn't be a join point any more. > > GHC.Core.Opt.CSE is conservative at the moment, and never CSE's *any* join > point. It would not be hard to make it clever enough to CSE join points, > but no one has yet done it. > > Do open a ticket! > > Simon > > PS: I am leaving Microsoft at the end of November 2021, at which point > simonpj at microsoft.com will cease to work. Use simon.peytonjones at gmail.com > instead. (For now, it just forwards to simonpj at microsoft.com.) > > | -----Original Message----- > | From: ghc-devs On Behalf Of Viktor > | Dukhovni > | Sent: 20 November 2021 00:57 > | To: ghc-devs at haskell.org > | Subject: [EXTERNAL] Unexpected duplicate join points in "Core" output? > | > | The below "Core" output from "ghc -O2" (9.2/8.10) for the attached > | program shows seemingly rendundant join points: > | > | join { > | exit :: State# RealWorld -> (# State# RealWorld, () #) > | exit (ipv :: State# RealWorld) = jump $s$j ipv } in > | > | join { > | exit1 :: State# RealWorld -> (# State# RealWorld, () #) > | exit1 (ipv :: State# RealWorld) = jump $s$j ipv } in > | > | that are identical in all but name. These correspond to fallthrough to > | the "otherwise" case in: > | > | ... > | | acc < q || (acc == q && d <= 5) > | -> loop (ptr `plusPtr` 1) (acc * 10 + d) > | | otherwise -> return Nothing > | > | but it seems that the generated X86_64 code (also below) ultimately > | consolidates these into a single target... Is that why it is harmless > | to leave these duplicated in the generated "Core"? > | > | [ Separately, in the generated machine code, it'd also be nice to avoid > | comparing the same "q" with the accumulator twice. A single load and > | compare should I think be enough, as I'd expect the status flags to > | persist across the jump the second test. > | > | This happens to not be performance critical in my case, because most > | calls should satisfy the first test, but generally I think that 3-way > | "a < b", "a == b", "a > b" branches ideally avoid comparing twice... > | ] > | > | ======== Associated Core output > | > | -- RHS size: {terms: 1, types: 0, coercions: 0, joins: 0/0} > | main2 :: Addr# > | main2 = "12345678901234567890 junk"# > | > | -- RHS size: {terms: 129, types: 114, coercions: 0, joins: 6/8} > | main1 :: State# RealWorld -> (# State# RealWorld, () #) > | main1 > | = \ (eta :: State# RealWorld) -> > | let { > | end :: Addr# > | end = plusAddr# main2 25# } in > | join { > | $s$j :: State# RealWorld -> (# State# RealWorld, () #) > | $s$j _ = hPutStr2 stdout $fShowMaybe4 True eta } in > | join { > | exit :: State# RealWorld -> (# State# RealWorld, () #) > | exit (ipv :: State# RealWorld) = jump $s$j ipv } in > | join { > | exit1 :: State# RealWorld -> (# State# RealWorld, () #) > | exit1 (ipv :: State# RealWorld) = jump $s$j ipv } in > | join { > | exit2 > | :: Addr# -> Word# -> State# RealWorld -> (# State# > | RealWorld, () #) > | exit2 (ww :: Addr#) (ww1 :: Word#) (ipv :: State# > | RealWorld) > | = case eqAddr# ww main2 of { > | __DEFAULT -> > | hPutStr2 > | stdout > | (++ > | $fShowMaybe1 > | (case $w$cshowsPrec3 11# (integerFromWord# > | ww1) [] of > | { (# ww3, ww4 #) -> > | : ww3 ww4 > | })) > | True > | eta; > | 1# -> jump $s$j ipv > | } } in > | joinrec { > | $wloop > | :: Addr# -> Word# -> State# RealWorld -> (# State# > | RealWorld, () #) > | $wloop (ww :: Addr#) (ww1 :: Word#) (w :: State# RealWorld) > | = join { > | getDigit :: State# RealWorld -> (# State# RealWorld, > | () #) > | getDigit (eta1 :: State# RealWorld) > | = case eqAddr# ww end of { > | __DEFAULT -> > | case readWord8OffAddr# ww 0# eta1 of { (# > | ipv, ipv1 #) -> > | let { > | ipv2 :: Word# > | ipv2 = minusWord# (word8ToWord# ipv1) 48## > | } in > | case gtWord# ipv2 9## of { > | __DEFAULT -> > | case ltWord# ww1 1844674407370955161## of > | { > | __DEFAULT -> > | case ww1 of { > | __DEFAULT -> jump exit ipv; > | 1844674407370955161## -> > | case leWord# ipv2 5## of { > | __DEFAULT -> jump exit1 ipv; > | 1# -> > | jump $wloop > | (plusAddr# ww 1#) > | (plusWord# > | 18446744073709551610## ipv2) > | ipv > | } > | }; > | 1# -> > | jump $wloop > | (plusAddr# ww 1#) (plusWord# > | (timesWord# ww1 10##) ipv2) ipv > | }; > | 1# -> jump exit2 ww ww1 ipv > | } > | }; > | 1# -> jump exit2 ww ww1 eta1 > | } } in > | jump getDigit w; } in > | jump $wloop main2 0## realWorld# > | > | ======== Executable disassembly > | > | The jumps at "-1->" and "-2->" that correspond that "otherwise" have > | the same target. The duplicate "load+cmp" with "q" is at "-3->" and "- > | 4->": > | > | 0000000000408de8 : > | 408de8: 48 8d 45 e8 lea -0x18(%rbp),%rax > | 408dec: 4c 39 f8 cmp %r15,%rax > | 408def: 0f 82 c8 00 00 00 jb 408ebd > | > | 408df5: b8 79 dd 77 00 mov $0x77dd79,%eax > | 408dfa: 31 db xor %ebx,%ebx > | 408dfc: b9 60 dd 77 00 mov $0x77dd60,%ecx > | 408e01: 48 39 c1 cmp %rax,%rcx > | 408e04: 74 66 je 408e6c > | > | 408e06: 0f b6 11 movzbl (%rcx),%edx > | 408e09: 48 83 c2 d0 add > | $0xffffffffffffffd0,%rdx > | 408e0d: 48 83 fa 09 cmp $0x9,%rdx > | 408e11: 77 59 ja 408e6c > | > | -3-> 408e13: 48 be 99 99 99 99 99 mov > | $0x1999999999999999,%rsi > | 408e1a: 99 99 19 > | 408e1d: 48 39 f3 cmp %rsi,%rbx > | 408e20: 73 0c jae 408e2e > | > | 408e22: 48 6b db 0a imul $0xa,%rbx,%rbx > | 408e26: 48 01 d3 add %rdx,%rbx > | 408e29: 48 ff c1 inc %rcx > | 408e2c: eb d3 jmp 408e01 > | > | -4-> 408e2e: 48 be 99 99 99 99 99 mov > | $0x1999999999999999,%rsi > | 408e35: 99 99 19 > | 408e38: 48 39 f3 cmp %rsi,%rbx > | -1-> 408e3b: 75 49 jne 408e86 > | > | 408e3d: 48 83 fa 05 cmp $0x5,%rdx > | -2-> 408e41: 77 43 ja 408e86 > | > | 408e43: 48 8d 5a fa lea -0x6(%rdx),%rbx > | 408e47: 48 ff c1 inc %rcx > | 408e4a: eb b5 jmp 408e01 > | > | 408e4c: 0f 1f 40 00 nopl 0x0(%rax) > | 408e50: c2 00 00 retq $0x0 > | > | -- > | Viktor. > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > -------------- next part -------------- An HTML attachment was scrubbed... URL: From nr at cs.tufts.edu Mon Nov 22 15:36:42 2021 From: nr at cs.tufts.edu (Norman Ramsey) Date: Mon, 22 Nov 2021 10:36:42 -0500 Subject: how to map CLabel to Label for procedure entry point? Message-ID: <20211122153642.7B7F12C26B6@homedog.cs.tufts.edu> I'm struggling to figure out how to get from a CLabel found in a CmmProc to a Label used to key the blocks in the control-flow graph (CmmGraph) for the CmmProc. The code generator needs such a function, so it must be around somewhere, but I have not yet found it. I've poked around the respective modules for CLabel and for Label, and also modules like CmmToAsm. Can anyone help me out? Norman From nr at cs.tufts.edu Mon Nov 22 15:48:50 2021 From: nr at cs.tufts.edu (Norman Ramsey) Date: Mon, 22 Nov 2021 10:48:50 -0500 Subject: SOLVED: how to map CLabel to Label for procedure entry point? In-Reply-To: <27103_1637595542_619BB996_27103_2779_1_20211122153642.7B7F12C26B6@homedog.cs.tufts.edu> (sfid-H-20211122-103922-+68.67-1@multi.osbf.lua) References: <27103_1637595542_619BB996_27103_2779_1_20211122153642.7B7F12C26B6@homedog.cs.tufts.edu> (sfid-H-20211122-103922-+68.67-1@multi.osbf.lua) Message-ID: <20211122154850.BD4122C26B6@homedog.cs.tufts.edu> > I'm struggling to figure out how to get from a CLabel found in a > CmmProc to a Label used to key the blocks in the control-flow graph > (CmmGraph) for the CmmProc. I figured it out---this is the wrong problem. I need to take the `g_entry` BlockId straight out of the CmmGraph. Norman From nr at cs.tufts.edu Mon Nov 22 19:01:26 2021 From: nr at cs.tufts.edu (Norman Ramsey) Date: Mon, 22 Nov 2021 14:01:26 -0500 Subject: asking for an extra set of eyes on Cmm.Dataflow (Hoopl) change proposal Message-ID: <20211122190126.114132C26B6@homedog.cs.tufts.edu> At present, Cmm dataflow analysis works only on Cmm code. I'd like to do a dataflow analysis on native code. Details at https://gitlab.haskell.org/ghc/ghc/-/issues/20725 where I'd love an extra set of eyes from the Cmm/Dataflow/Hoopl crowd. Norman From nr at cs.tufts.edu Mon Nov 22 19:51:54 2021 From: nr at cs.tufts.edu (Norman Ramsey) Date: Mon, 22 Nov 2021 14:51:54 -0500 Subject: can GHC generate an irreducible control-flow graph? If so, how? Message-ID: <20211122195154.61DF42C26B6@homedog.cs.tufts.edu> I'm trying to figure out how to persuade GHC to generate an irreducible control-flow graph (so I can test an algorithm to convert it to structured control flow). The attached image shows (on the left) the classic simple irreducible CFG: there is a loop between nodes A and B, but neither one dominates the other, so there is no loop header. I tried to get GHC to generate this CFG using the following source code: length'' :: Bool -> List a -> Int length'' trigger xs = if trigger then countA 0 xs else countB 0 xs where countA n Nil = case n of m -> m countA n (Cons _ as) = case n + 1 of m -> countB m as countB n Nil = case n of m -> m countB n (Cons _ as) = case n + 2 of m -> countA m as Unfortunately (for my purposes), GHC generates a perfectly lovely reducible flow graph with a single header node. It is even possible for GHC to generate an irreducible control-flow graph? If so, how can it be done? Norman -------------- next part -------------- A non-text attachment was scrubbed... Name: On-the-left-is-an-unstructured-program-and-the-corresponding-irreducible-control-flow.png Type: image/png Size: 21365 bytes Desc: not available URL: From alan.zimm at gmail.com Mon Nov 22 20:16:58 2021 From: alan.zimm at gmail.com (Alan & Kim Zimmerman) Date: Mon, 22 Nov 2021 20:16:58 +0000 Subject: LINE pragma behaviour Message-ID: I am working through some ghc-exactprint test cases with GHC 9.2.1 and came across an oddity. If I parse some source with {-# LINE 93 "Foo.chs" #-} on line five, it shows up in the ParseSource as (L (Anchor { LINE:5:1-25 } (UnchangedAnchor)) (EpaComment (EpaLineComment "{-# LINE 93 \"Foo.chs\" #-}") { LINE:5:1-25 })) and the following item locations are unchanged. The effect seems to be to change the name of the file in the RealSrcSpan to "LINE", but just for that line, and no other effect. Is this expected? Alan -------------- next part -------------- An HTML attachment was scrubbed... URL: From simonpj at microsoft.com Mon Nov 22 20:35:04 2021 From: simonpj at microsoft.com (Simon Peyton Jones) Date: Mon, 22 Nov 2021 20:35:04 +0000 Subject: [EXTERNAL] can GHC generate an irreducible control-flow graph? If so, how? In-Reply-To: <20211122195154.61DF42C26B6@homedog.cs.tufts.edu> References: <20211122195154.61DF42C26B6@homedog.cs.tufts.edu> Message-ID: GHC breaks strongly connected components with a so-called loop-breaker. In this case, maybe countA is the loop-breaker; then countB can inline at all its call sites, and it'll look very reducible. See "Secrets of the GHC inliner". If you make countA and countB each call themselves, as well as the other, that will defeat this plan, and you may get closer to your goal. I'm guessing a bit, but hope this helps. Simon PS: I am leaving Microsoft at the end of November 2021, at which point simonpj at microsoft.com will cease to work. Use simon.peytonjones at gmail.com instead. (For now, it just forwards to simonpj at microsoft.com.) | -----Original Message----- | From: ghc-devs On Behalf Of Norman | Ramsey | Sent: 22 November 2021 19:52 | To: ghc-devs at haskell.org | Subject: [EXTERNAL] can GHC generate an irreducible control-flow graph? | If so, how? | | I'm trying to figure out how to persuade GHC to generate an irreducible | control-flow graph (so I can test an algorithm to convert it to | structured control flow). | | The attached image shows (on the left) the classic simple irreducible | CFG: there is a loop between nodes A and B, but neither one dominates | the other, so there is no loop header. I tried to get GHC to generate | this CFG using the following source code: | | length'' :: Bool -> List a -> Int | length'' trigger xs = if trigger then countA 0 xs else countB 0 xs | where countA n Nil = case n of m -> m | countA n (Cons _ as) = case n + 1 of m -> countB m as | countB n Nil = case n of m -> m | countB n (Cons _ as) = case n + 2 of m -> countA m as | | Unfortunately (for my purposes), GHC generates a perfectly lovely | reducible flow graph with a single header node. | | It is even possible for GHC to generate an irreducible control-flow | graph? If so, how can it be done? | | | Norman From sgraf1337 at gmail.com Mon Nov 22 21:04:40 2021 From: sgraf1337 at gmail.com (Sebastian Graf) Date: Mon, 22 Nov 2021 22:04:40 +0100 Subject: [EXTERNAL] can GHC generate an irreducible control-flow graph? If so, how? In-Reply-To: References: <20211122195154.61DF42C26B6@homedog.cs.tufts.edu> Message-ID: An alternative would be to mark both functions as NOINLINE, which the Simplifier will adhere to. You might also want to have `countA` and `countB` close over a local variable in order for them not to be floated to the top-level. If top-level bindings aren't an issue for you, you could simply use mutually recursive even/odd definitions. Otherwise, something like this might do: foo :: Bool -> Int -> Bool foo b n | n > 10 = even n | otherwise = odd n where even 0 = b even n = odd (n-1) {-# NOINLINE even #-} odd 0 = b odd n = even (n-1) {-# NOINLINE odd #-} GHC 8.10 will simply duplicate both functions into each branch, but GHC master produces irreducible control flow for me: Lib.$wfoo = \ (b_sTr :: Bool) (ww_sTu :: GHC.Prim.Int#) -> joinrec { $wodd_sTi [InlPrag=NOINLINE] :: GHC.Prim.Int# -> Bool [LclId[JoinId(1)], Arity=1, Str=<1L>, Unf=OtherCon []] $wodd_sTi (ww1_sTf :: GHC.Prim.Int#) = case ww1_sTf of wild_X1 { __DEFAULT -> jump $weven_sTp (GHC.Prim.-# wild_X1 1#); 0# -> b_sTr }; $weven_sTp [InlPrag=NOINLINE, Occ=LoopBreaker] :: GHC.Prim.Int# -> Bool [LclId[JoinId(1)], Arity=1, Str=<1L>, Unf=OtherCon []] $weven_sTp (ww1_sTm :: GHC.Prim.Int#) = case ww1_sTm of wild_X1 { __DEFAULT -> jump $wodd_sTi (GHC.Prim.-# wild_X1 1#); 0# -> b_sTr }; } in case GHC.Prim.># ww_sTu 10# of { __DEFAULT -> jump $wodd_sTi ww_sTu; 1# -> jump $weven_sTp ww_sTu } Cheers, Sebastian Am Mo., 22. Nov. 2021 um 21:37 Uhr schrieb Simon Peyton Jones via ghc-devs < ghc-devs at haskell.org>: > GHC breaks strongly connected components with a so-called loop-breaker. In > this case, maybe countA is the loop-breaker; then countB can inline at all > its call sites, and it'll look very reducible. See "Secrets of the GHC > inliner". > > If you make countA and countB each call themselves, as well as the other, > that will defeat this plan, and you may get closer to your goal. > > I'm guessing a bit, but hope this helps. > > Simon > > PS: I am leaving Microsoft at the end of November 2021, at which point > simonpj at microsoft.com will cease to work. Use simon.peytonjones at gmail.com > instead. (For now, it just forwards to simonpj at microsoft.com.) > > | -----Original Message----- > | From: ghc-devs On Behalf Of Norman > | Ramsey > | Sent: 22 November 2021 19:52 > | To: ghc-devs at haskell.org > | Subject: [EXTERNAL] can GHC generate an irreducible control-flow graph? > | If so, how? > | > | I'm trying to figure out how to persuade GHC to generate an irreducible > | control-flow graph (so I can test an algorithm to convert it to > | structured control flow). > | > | The attached image shows (on the left) the classic simple irreducible > | CFG: there is a loop between nodes A and B, but neither one dominates > | the other, so there is no loop header. I tried to get GHC to generate > | this CFG using the following source code: > | > | length'' :: Bool -> List a -> Int > | length'' trigger xs = if trigger then countA 0 xs else countB 0 xs > | where countA n Nil = case n of m -> m > | countA n (Cons _ as) = case n + 1 of m -> countB m as > | countB n Nil = case n of m -> m > | countB n (Cons _ as) = case n + 2 of m -> countA m as > | > | Unfortunately (for my purposes), GHC generates a perfectly lovely > | reducible flow graph with a single header node. > | > | It is even possible for GHC to generate an irreducible control-flow > | graph? If so, how can it be done? > | > | > | Norman > > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ben at smart-cactus.org Tue Nov 23 20:13:27 2021 From: ben at smart-cactus.org (Ben Gamari) Date: Tue, 23 Nov 2021 15:13:27 -0500 Subject: LINE pragma behaviour In-Reply-To: References: Message-ID: <874k82wrze.fsf@smart-cactus.org> "Alan & Kim Zimmerman" writes: > I am working through some ghc-exactprint test cases with GHC 9.2.1 and came > across an oddity. > > If I parse some source with > > {-# LINE 93 "Foo.chs" #-} > > on line five, it shows up in the ParseSource as > > (L > (Anchor > { LINE:5:1-25 } > (UnchangedAnchor)) > (EpaComment > (EpaLineComment > "{-# LINE 93 \"Foo.chs\" #-}") > { LINE:5:1-25 })) > > and the following item locations are unchanged. > > The effect seems to be to change the name of the file in the RealSrcSpan to > "LINE", but just for that line, and no other effect. > > Is this expected? > It sounds wrong to me, although it is surprising that there aren't any tests covering this. A quick search of the lexer reveals that GHC.Parser.Lexer.mkParserOpts takes an argument which dictates whether LINE and COLUMN pragmas update the lexer's source location or are instead just emitted in the lexeme stream. That being said, `GHC.Driver.Config.Parser.initParserOpts` seems to pass `True` here, which should update the source location. Were you testing with GHC or are you using the GHC API? Cheers, - Ben -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From ietf-dane at dukhovni.org Wed Nov 24 21:27:05 2021 From: ietf-dane at dukhovni.org (Viktor Dukhovni) Date: Wed, 24 Nov 2021 16:27:05 -0500 Subject: [EXTERNAL] Unexpected duplicate join points in "Core" output? In-Reply-To: References: Message-ID: On Sun, Nov 21, 2021 at 06:53:53AM -0500, Carter Schonwald wrote: > On Sat, Nov 20, 2021 at 4:17 PM Simon Peyton Jones via ghc-devs < > ghc-devs at haskell.org> wrote: > > > There is absolutely no reason not to common-up those to join points. But > > we can't common up some join points when we could if they were let's. > > Consider > > > > join j1 x = x+1 > > in case v of > > A -> f (join j2 x = x+1 in ...j2...) > > B -> ....j1... > > C -> ....j1... > > > > Even though j2 is identical to j1's, we can't eliminate j2 in favour of j1 > > because then j1 wouldn't be a join point any more. > > In this example: why would it stop being a join point ? > > Admittedly, my intuition might be skewed by my own ideas about how > join points are sortah a semantic special case of other constructs. I think the point is that join points are tail calls that don't return to the caller. But here even though `j1` and `j2` have the same body j1's continuation is not the same as j2's continuation. Rather the result of `j2` is the input to `f`, but the result of j1 is a possible output of the whole `case` block in the B and C branches. For two join points to be duplicates they need to not only be alpha equivalent but to also have the same continuation. Something like join j1 x = x + 1 in join j2 y = y + 1 in ... j1 ... ... j2 ... where eliminating j2 in favour of j1 should be correct. -- VIktor. From simonpj at microsoft.com Wed Nov 24 23:14:00 2021 From: simonpj at microsoft.com (Simon Peyton Jones) Date: Wed, 24 Nov 2021 23:14:00 +0000 Subject: [EXTERNAL] Unexpected duplicate join points in "Core" output? In-Reply-To: References: Message-ID: | For two join points to be duplicates they need to not only be alpha | equivalent but to also have the same continuation. Yes exactly. And it would not be hard to adapt the existing CSE pass to support this. Just needs doing. A ticket and a repo case would be really helpful. Simon PS: I am leaving Microsoft at the end of November 2021, at which point simonpj at microsoft.com will cease to work. Use simon.peytonjones at gmail.com instead. (For now, it just forwards to simonpj at microsoft.com.) | -----Original Message----- | From: ghc-devs On Behalf Of Viktor | Dukhovni | Sent: 24 November 2021 21:27 | To: ghc-devs at haskell.org | Subject: Re: [EXTERNAL] Unexpected duplicate join points in "Core" | output? | | On Sun, Nov 21, 2021 at 06:53:53AM -0500, Carter Schonwald wrote: | | > On Sat, Nov 20, 2021 at 4:17 PM Simon Peyton Jones via ghc-devs < | > ghc-devs at haskell.org> wrote: | > | > > There is absolutely no reason not to common-up those to join | points. | > > But we can't common up some join points when we could if they were | let's. | > > Consider | > > | > > join j1 x = x+1 | > > in case v of | > > A -> f (join j2 x = x+1 in ...j2...) | > > B -> ....j1... | > > C -> ....j1... | > > | > > Even though j2 is identical to j1's, we can't eliminate j2 in | favour | > > of j1 because then j1 wouldn't be a join point any more. | > | > In this example: why would it stop being a join point ? | > | > Admittedly, my intuition might be skewed by my own ideas about how | > join points are sortah a semantic special case of other constructs. | | I think the point is that join points are tail calls that don't return | to the caller. But here even though `j1` and `j2` have the same body | j1's continuation is not the same as j2's continuation. | | Rather the result of `j2` is the input to `f`, but the result of j1 is | a possible output of the whole `case` block in the B and C branches. | For two join points to be duplicates they need to not only be alpha | equivalent but to also have the same continuation. Something like | | join j1 x = x + 1 in | join j2 y = y + 1 in | ... j1 ... | ... j2 ... | | where eliminating j2 in favour of j1 should be correct. | | -- | VIktor. | _______________________________________________ | ghc-devs mailing list | ghc-devs at haskell.org | https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmail.h | askell.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fghc- | devs&data=04%7C01%7Csimonpj%40microsoft.com%7Cc5bef423b62e469b382d0 | 8d9af9156f4%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C63773386151737 | 6728%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTi | I6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=nWOBjpnIGGX2RbwIT%2BofdqfGJYq | xY%2FvKKExGB%2B2Vi3k%3D&reserved=0 From ietf-dane at dukhovni.org Wed Nov 24 23:32:04 2021 From: ietf-dane at dukhovni.org (Viktor Dukhovni) Date: Wed, 24 Nov 2021 18:32:04 -0500 Subject: [EXTERNAL] Unexpected duplicate join points in "Core" output? In-Reply-To: References: Message-ID: On Wed, Nov 24, 2021 at 11:14:00PM +0000, Simon Peyton Jones via ghc-devs wrote: > | For two join points to be duplicates they need to not only be alpha > | equivalent but to also have the same continuation. > > Yes exactly. And it would not be hard to adapt the existing CSE pass > to support this. Just needs doing. > > A ticket and a repo case would be really helpful. I'll do my best to construct a standalone reproducer that is not mired in ByteString code. The ByteString example should not be too difficult to mimmic in code that relies only on base. Though I might still have to use Foreign.Storable and Foreign.Ptr and some sort of unsafePerformIO variant in there, so that I get essentially the same basic structure of inlining and join points. I guess I'll try removing excess baggage while the basic structure persists, and ideally end up with something small enough. -- Viktor. From ietf-dane at dukhovni.org Thu Nov 25 02:43:54 2021 From: ietf-dane at dukhovni.org (Viktor Dukhovni) Date: Wed, 24 Nov 2021 21:43:54 -0500 Subject: [EXTERNAL] Unexpected duplicate join points in "Core" output? In-Reply-To: References: Message-ID: On Wed, Nov 24, 2021 at 06:32:04PM -0500, Viktor Dukhovni wrote: > > Yes exactly. And it would not be hard to adapt the existing CSE pass > > to support this. Just needs doing. > > > > A ticket and a repo case would be really helpful. > > I'll do my best to construct a standalone reproducer that is not mired > in ByteString code. The ByteString example should not be too difficult > to mimmic in code that relies only on base. Just noticed a complication, it seems that the placemnt of the IO state token in the join point argument list is non-deterministic, so I'm starting to see join points in which the argument lists are permuted, with an equivalent permutation at the jump/call site... :-( Two exit points returning equivalent data, the first returns early, the second returns after first performing some I/O: return $ Result valid acc (ptr `minusPtr` start) become respectively (ipv2 and w3 are IO state tokens): 1. jump exit2 ww4 ww5 valid ipv2 -- acc ptr valid s# 2. jump exit3 ww4 ww5 w3 valid -- acc ptr s# valid So the join points are then only alpha equivalent up to argument permutation: join { exit2 :: Word# -> Addr# -> Bool -> State# RealWorld -> Maybe (Int, ByteString) exit2 (ww4 :: Word#) (ww5 :: Addr#) (valid :: Bool) (ipv2 :: State# RealWorld) = ... join { exit3 :: Word# -> Addr# -> State# RealWorld -> Bool -> Maybe (Int, ByteString) exit3 (ww4 :: Word#) (ww5 :: Addr#) (w2 :: State# RealWorld) (valid :: Bool) = ... I don't how argument lists to join points are ordered, would it be possible to make them predictably consistent? -- Viktor. From harendra.kumar at gmail.com Fri Nov 26 19:31:27 2021 From: harendra.kumar at gmail.com (Harendra Kumar) Date: Sat, 27 Nov 2021 01:01:27 +0530 Subject: No subject Message-ID: Hi GHC devs, While compiling the primitive package using ghc head I ran into the following error: Data/Primitive/Types.hs:265:870: error: • Couldn't match type ‘Word64#’ with ‘Word#’ Expected: Word64_# Actual: Word64# • In the fourth argument of ‘setWord64Array#’, namely ‘x#’ In the first argument of ‘internal’, namely ‘(setWord64Array# arr# i n x#)’ In the first argument of ‘unsafeCoerce#’, namely ‘(internal (setWord64Array# arr# i n x#))’ | 265 | derivePrim(Word64, W64#, sIZEOF_WORD64, aLIGNMENT_WORD64, | Any idea what this is and how it can be fixed? -harendra -------------- next part -------------- An HTML attachment was scrubbed... URL: From harendra.kumar at gmail.com Fri Nov 26 19:35:08 2021 From: harendra.kumar at gmail.com (Harendra Kumar) Date: Sat, 27 Nov 2021 01:05:08 +0530 Subject: Compiling "primitive" with ghc head In-Reply-To: References: Message-ID: Forgot to add subject in the previous email. On Sat, 27 Nov 2021 at 01:01, Harendra Kumar wrote: > Hi GHC devs, > > While compiling the primitive package using ghc head I ran into the > following error: > > Data/Primitive/Types.hs:265:870: error: > • Couldn't match type ‘Word64#’ with ‘Word#’ > Expected: Word64_# > Actual: Word64# > • In the fourth argument of ‘setWord64Array#’, namely ‘x#’ > In the first argument of ‘internal’, namely > ‘(setWord64Array# arr# i n x#)’ > In the first argument of ‘unsafeCoerce#’, namely > ‘(internal (setWord64Array# arr# i n x#))’ > | > 265 | derivePrim(Word64, W64#, sIZEOF_WORD64, aLIGNMENT_WORD64, > | > > > > Any idea what this is and how it can be fixed? > > -harendra > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sylvain at haskus.fr Fri Nov 26 20:00:12 2021 From: sylvain at haskus.fr (Sylvain Henry) Date: Fri, 26 Nov 2021 21:00:12 +0100 Subject: Compiling "primitive" with ghc head In-Reply-To: References: Message-ID: Hi, We now always use Word64# to implement Word64. You can find the patch for primitive and many other packages in head.hackage: https://gitlab.haskell.org/ghc/head.hackage/-/blob/master/patches/primitive-0.7.3.0.patch Cheers, Sylvain ⁣ Le 26 nov. 2021 à 20:35, à 20:35, Harendra Kumar a écrit: >Forgot to add subject in the previous email. > >On Sat, 27 Nov 2021 at 01:01, Harendra Kumar >wrote: > >> Hi GHC devs, >> >> While compiling the primitive package using ghc head I ran into the >> following error: >> >> Data/Primitive/Types.hs:265:870: error: >> • Couldn't match type ‘Word64#’ with ‘Word#’ >> Expected: Word64_# >> Actual: Word64# >> • In the fourth argument of ‘setWord64Array#’, namely ‘x#’ >> In the first argument of ‘internal’, namely >> ‘(setWord64Array# arr# i n x#)’ >> In the first argument of ‘unsafeCoerce#’, namely >> ‘(internal (setWord64Array# arr# i n x#))’ >> | >> 265 | derivePrim(Word64, W64#, sIZEOF_WORD64, aLIGNMENT_WORD64, >> | >> >> >> >> Any idea what this is and how it can be fixed? >> >> -harendra >> > > >------------------------------------------------------------------------ > >_______________________________________________ >ghc-devs mailing list >ghc-devs at haskell.org >http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs -------------- next part -------------- An HTML attachment was scrubbed... URL: From lists at richarde.dev Fri Nov 26 21:25:59 2021 From: lists at richarde.dev (Richard Eisenberg) Date: Fri, 26 Nov 2021 21:25:59 +0000 Subject: GitLab likes Chrome more than Firefox Message-ID: <010f017d5e24074e-c7951d9e-b8ec-4267-9e93-ee41eecad459-000000@us-east-2.amazonses.com> Hi all, On a tip from Ben, I'm using Chrome to review a large patch (!7033 for the curious). It is qualitatively better than Firefox, my usual tool. I will be routinely using Chrome to review large patches from now on. You may want to, too. Richard From nr at cs.tufts.edu Tue Nov 30 19:38:08 2021 From: nr at cs.tufts.edu (Norman Ramsey) Date: Tue, 30 Nov 2021 14:38:08 -0500 Subject: GitLab likes Chrome more than Firefox In-Reply-To: <010f017d5e24074e-c7951d9e-b8ec-4267-9e93-ee41eecad459-000000@us-east-2.amazonses.com> (sfid-H-20211128-082547-+56.81-1@multi.osbf.lua) References: <010f017d5e24074e-c7951d9e-b8ec-4267-9e93-ee41eecad459-000000@us-east-2.amazonses.com> (sfid-H-20211128-082547-+56.81-1@multi.osbf.lua) Message-ID: <20211130193808.3D31C2C30A0@homedog.cs.tufts.edu> > On a tip from Ben, I'm using Chrome to review a large patch (!7033 for the > curious). It is qualitatively better than Firefox, my usual tool. And to follow up from this morning, I fear that Magit is not going to be useful as a tool for reviewing: it seems unable to access the "conversation" part of a Github PR, and I assume it will have similar issues with Gitlab. Norman