From zubin at well-typed.com Mon Jul 1 09:39:14 2024 From: zubin at well-typed.com (Zubin Duggal) Date: Mon, 1 Jul 2024 15:09:14 +0530 Subject: [ANNOUNCE] GHC 9.6.6 is now available Message-ID: <7vc5hz2hk3rgaxbic2paonhuctnl3ze32wml5rvr3fgd7yhfyk@ulc5hg2uateq> The GHC developers are happy to announce the availability of GHC 9.6.6. Binary distributions, source distributions, and documentation are available on the [release page](https://www.haskell.org/ghc/download_ghc_9_6_6.html). Blog Post: https://www.haskell.org/ghc/blog/20240701-ghc-9.6.6-released.html This release is primarily a bugfix release addressing some issues found in the 9.6 series. These include: * A fix for a bug in the NCG that could lead to incorrect runtime results due to erroneously removing a jump instruction (#24507). * A fix for a linker error that manifested on certain platform/toolchain combinations, particularly darwin with a brew provisioned toolchain, arising due to a confusion in linker options between GHC and cabal (#22210). * A fix for a compiler panic in the simplifier due to incorrect eta expansion (#24718). * A fix for possible segfaults when using the bytecode interpreter due to incorrect constructor tagging (#24870). * And a few more fixes A full accounting of changes can be found in the [release notes]. As some of the fixed issues do affect correctness users are encouraged to upgrade promptly. We would like to thank Microsoft Azure, GitHub, IOG, the Zw3rk stake pool, Well-Typed, Tweag I/O, Serokell, Equinix, SimSpace, Haskell Foundation, and other anonymous contributors whose on-going financial and in-kind support has facilitated GHC maintenance and release management over the years. Finally, this release would not have been possible without the hundreds of open-source contributors whose work comprise this release. As always, do give this release a try and open a [ticket][] if you see anything amiss. Happy Hacking! -Zubin [ticket]: https://gitlab.haskell.org/ghc/ghc/-/issues/new [release notes]: https://downloads.haskell.org/~ghc/9.6.6/docs/users_guide/9.6.6-notes.html -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 488 bytes Desc: not available URL: From simon.peytonjones at gmail.com Tue Jul 2 08:06:06 2024 From: simon.peytonjones at gmail.com (Simon Peyton Jones) Date: Tue, 2 Jul 2024 09:06:06 +0100 Subject: CountDeps Message-ID: Does anyone know what the CountDeps test does? I'm getting the failure below in my branch. Should I just accept it? I think it's because GHC.Core.FamInstEnv now depends on GHC.Builtin.Types.Literals, a very reasonable dependency Thanks Simon +++ "/builds/ghc/ghc/tmp/ghctest-su6yq239/test spaces/testsuite/tests/count-deps/CountDepsAst.run/CountDepsAst.run.stdout.normalised" 2024-07-01 18:37:33.372548372 +0000 @@ -2,6 +2,7 @@ GHC.Builtin.Names GHC.Builtin.PrimOps GHC.Builtin.Types +GHC.Builtin.Types.Literals GHC.Builtin.Types.Prim GHC.Builtin.Uniques GHC.ByteCode.Types --- "/builds/ghc/ghc/tmp/ghctest-su6yq239/test spaces/testsuite/tests/count-deps/CountDepsParser.run/CountDepsParser.stdout.normalised" 2024-07-01 18:37:33.393548751 +0000 +++ "/builds/ghc/ghc/tmp/ghctest-su6yq239/test spaces/testsuite/tests/count-deps/CountDepsParser.run/CountDepsParser.run.stdout.normalised" 2024-07-01 18:37:33.393548751 +0000 @@ -2,6 +2,7 @@ GHC.Builtin.Names GHC.Builtin.PrimOps GHC.Builtin.Types +GHC.Builtin.Types.Literals GHC.Builtin.Types.Prim GHC.Builtin.Uniques GHC.ByteCode.Types -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodrigo.m.mesquita at gmail.com Tue Jul 2 09:50:17 2024 From: rodrigo.m.mesquita at gmail.com (Rodrigo Mesquita) Date: Tue, 2 Jul 2024 10:50:17 +0100 Subject: CountDeps In-Reply-To: References: Message-ID: Hey Simon CountDepsParser and CountDepsAST list the closure of the Parser/AST modules dependencies respectively. This test helps keep in check our goal of making the AST not depend on GHC.* modules (the overarching goal of #21592). In your case, you’ve extended the closure by adding dependencies to modules which are already in the GHC.* namespace, which is perfectly fine. Rodrigo > On 2 Jul 2024, at 09:06, Simon Peyton Jones wrote: > > Does anyone know what the CountDeps test does? > > I'm getting the failure below in my branch. Should I just accept it? > > I think it's because GHC.Core.FamInstEnv now depends on GHC.Builtin.Types.Literals, a very reasonable dependency > > Thanks > > Simon > > +++ "/builds/ghc/ghc/tmp/ghctest-su6yq239/test spaces/testsuite/tests/count-deps/CountDepsAst.run/CountDepsAst.run.stdout.normalised" 2024-07-01 18:37:33.372548372 +0000 > @@ -2,6 +2,7 @@ > GHC.Builtin.Names > GHC.Builtin.PrimOps > GHC.Builtin.Types > +GHC.Builtin.Types.Literals > GHC.Builtin.Types.Prim > GHC.Builtin.Uniques > GHC.ByteCode.Types > --- "/builds/ghc/ghc/tmp/ghctest-su6yq239/test spaces/testsuite/tests/count-deps/CountDepsParser.run/CountDepsParser.stdout.normalised" 2024-07-01 18:37:33.393548751 +0000 > +++ "/builds/ghc/ghc/tmp/ghctest-su6yq239/test spaces/testsuite/tests/count-deps/CountDepsParser.run/CountDepsParser.run.stdout.normalised" 2024-07-01 18:37:33.393548751 +0000 > @@ -2,6 +2,7 @@ > GHC.Builtin.Names > GHC.Builtin.PrimOps > GHC.Builtin.Types > +GHC.Builtin.Types.Literals > GHC.Builtin.Types.Prim > GHC.Builtin.Uniques > GHC.ByteCode.Types > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs -------------- next part -------------- An HTML attachment was scrubbed... URL: From hecate at glitchbra.in Tue Jul 2 22:29:56 2024 From: hecate at glitchbra.in (=?UTF-8?Q?H=C3=A9cate?=) Date: Wed, 3 Jul 2024 00:29:56 +0200 Subject: How to implement type-level list concatenation as a GHC primitive type family Message-ID: <69589ef1-7c32-41f8-878e-1180433276fa@glitchbra.in> Hi GHC devs, I was wondering recently, considering that type family evaluation is notoriously slow, how one would implement type-level list concatenation directly within GHC in a way that is much less expensive to run. So I am looking for pointers, as this part of GHC is fairly unknown to me. Thinking about it, I'm actually unsure where the tyfam evaluation is happening. Any advice would be appreciated. Cheers, Hécate -- Hécate ✨ 🐦: @TechnoEmpress IRC: Hecate WWW: https://glitchbra.in RUN: BSD From liyao at pdx.edu Tue Jul 2 23:08:44 2024 From: liyao at pdx.edu (Yao Li) Date: Tue, 2 Jul 2024 18:08:44 -0500 Subject: Deriving mechanisms in GHC Message-ID: Dear All, My name is Yao Li. I am an assistant professor at Portland State University. I am currently working with my Ph.D. student Nicholas Coltharp on updating a tool called hs-to-coq (https://github.com/plclub/hs-to-coq) to support GHC 8.10 and the corresponding base library. However, we are running into some trouble involving the "deriving" mechanisms in GHC. It seems the derived type class instances are no longer generated by hs-to-coq after we updated GHC. I wonder what changes happened between GHC 8.4 and 8.10 that could possibly lead to this issue? Or if we hope to dig into this issue ourselves, what are some GHC resources that we should look into? Any pointers would be much appreciated! For context, hs-to-coq uses GHC's parser to parse Haskell code. Our translation is based on the source code but on the GhcRn pass. Thanks, Yao Li, Ph.D., (He/Him) Assistant Professor, Computer Science Department, Portland State University -------------- next part -------------- An HTML attachment was scrubbed... URL: From gergo at erdi.hu Wed Jul 3 04:35:00 2024 From: gergo at erdi.hu (=?ISO-8859-2?Q?=C9RDI_Gerg=F5?=) Date: Wed, 3 Jul 2024 12:35:00 +0800 (+08) Subject: How to implement type-level list concatenation as a GHC primitive type family In-Reply-To: <69589ef1-7c32-41f8-878e-1180433276fa@glitchbra.in> References: <69589ef1-7c32-41f8-878e-1180433276fa@glitchbra.in> Message-ID: <0638f8c8-8ad3-59be-6d1b-d998392d70fe@erdi.hu> I know this is not exactly what you're asking, but in similar situations I've had very good results from implementing my type family as a type checker plugin. Unfortunately it's not code that I can share, but it roughly goes like this: 1. Declare a closed type family without defining any clauses: type family Append (xs :: [k]) (ys :: [k]) :: [k] where 2. Create a typechecker plugin that in `tcPluginInit` looks up this `Append` tyfam's tycon, and in `tcPluginRewrite` maps it to a `TcPluginRewriter` 3. Inside the `TcPluginRewriter`, you have a `[TcType]` which are your arguments (so the two type-level lists that are concatenated in your example). If you can compute your tyfam result from that, you can then return a `TcPluginRewriteTo` with the "trust-me" coercion `UnivCo`. This is what really helps with performance vs. a native Haskell-defined tyfam, since it avoids Core passing around huge coercion terms corresponding to every reduction step. (https://gitlab.haskell.org/ghc/ghc/-/issues/8095 https://gitlab.haskell.org/ghc/ghc/-/issues/22323 and many more) The only tricky part is that in step 3, you have to be prepared to handle pratially meta arguments. For example, if your arguments are `String : Double : []` and `Int : Bool : α` (with some typechecker metavariable `α`), you can of course reduce that to `String : Double : Int : Bool : α`. But if they are flipped, you can either bail out until `α` is solved, or make partial progress to `Int : Bool : Append α (String : Double : [])`. I don't know which is the right choice in general, since bailing out might expose less info to type inference, but partial progressing means your coercion terms will grow, since you're restoring some of that step-by-step representation. Let me know if any of this makes sense, I've got the flu atm so the above might be too rambly :) Bye, Gergo On Wed, 3 Jul 2024, Hécate via ghc-devs wrote: > Hi GHC devs, > > I was wondering recently, considering that type family evaluation is > notoriously slow, how one would implement type-level list concatenation > directly within GHC in a way that is much less expensive to run. So I am > looking for pointers, as this part of GHC is fairly unknown to me. > > Thinking about it, I'm actually unsure where the tyfam evaluation is > happening. > > Any advice would be appreciated. > > Cheers, > Hécate > > -- .--= ULLA! =-----------------. \ http://gergo.erdi.hu \ `---= gergo at erdi.hu =-------' From hecate at glitchbra.in Wed Jul 3 07:42:30 2024 From: hecate at glitchbra.in (=?UTF-8?Q?H=C3=A9cate?=) Date: Wed, 3 Jul 2024 09:42:30 +0200 Subject: How to implement type-level list concatenation as a GHC primitive type family In-Reply-To: <0638f8c8-8ad3-59be-6d1b-d998392d70fe@erdi.hu> References: <69589ef1-7c32-41f8-878e-1180433276fa@glitchbra.in> <0638f8c8-8ad3-59be-6d1b-d998392d70fe@erdi.hu> Message-ID: <39bd21c7-eee5-46a9-82b1-99c28ad4e6b2@glitchbra.in> Thank you Gergő, this delightfully cursed! :D I was pointed by someone off-list to “Note [Adding built-in type families]” which is actually very complete! One last thing is that since everything is Xi-typed (new form of "stringly-typed" uncovered), the meaning of the parameters is a bit blurred when they all are called x1, z1, y1, in GHC.Builtin.Types.Literals. See for instance: https://hackage.haskell.org/package/ghc-9.10.1/docs/src/GHC.Builtin.Types.Literals.html#interactInertAppendSymbol Le 03/07/2024 à 06:35, ÉRDI Gergő a écrit : > I know this is not exactly what you're asking, but in similar > situations I've had very good results from implementing my type family > as a type checker plugin. Unfortunately it's not code that I can > share, but it roughly goes like this: > > 1. Declare a closed type family without defining any clauses: > >     type family Append (xs :: [k]) (ys :: [k]) :: [k] where > > 2. Create a typechecker plugin that in `tcPluginInit` looks up this > `Append` tyfam's tycon, and in `tcPluginRewrite` maps it to a > `TcPluginRewriter` > > 3. Inside the `TcPluginRewriter`, you have a `[TcType]` which are your > arguments (so the two type-level lists that are concatenated in your > example). If you can compute your tyfam result from that, you can then > return a `TcPluginRewriteTo` with the "trust-me" coercion `UnivCo`. > This is what really helps with performance vs. a native > Haskell-defined tyfam, since it avoids Core passing around huge > coercion terms corresponding to every reduction step. > (https://gitlab.haskell.org/ghc/ghc/-/issues/8095 > https://gitlab.haskell.org/ghc/ghc/-/issues/22323 and many more) > > The only tricky part is that in step 3, you have to be prepared to > handle pratially meta arguments. For example, if your arguments are > `String : Double : []` and `Int : Bool : α` (with some typechecker > metavariable `α`), you can of course reduce that to `String : Double : > Int : Bool : α`. But if they are flipped, you can either bail out > until `α` is solved, or make partial progress to `Int : Bool : Append > α (String : Double : [])`. I don't know which is the right choice in > general, since bailing out might expose less info to type inference, > but partial progressing means your coercion terms will grow, since > you're restoring some of that step-by-step representation. > > Let me know if any of this makes sense, I've got the flu atm so the > above might be too rambly :) > > Bye, >     Gergo > > On Wed, 3 Jul 2024, Hécate via ghc-devs wrote: > >> Hi GHC devs, >> >> I was wondering recently, considering that type family evaluation is >> notoriously slow, how one would implement type-level list >> concatenation directly within GHC in a way that is much less >> expensive to run. So I am looking for pointers, as this part of GHC >> is fairly unknown to me. >> >> Thinking about it, I'm actually unsure where the tyfam evaluation is >> happening. >> >> Any advice would be appreciated. >> >> Cheers, >> Hécate >> >> > -- Hécate ✨ 🐦: @TechnoEmpress IRC: Hecate WWW: https://glitchbra.in RUN: BSD From ryan.gl.scott at gmail.com Wed Jul 3 12:46:14 2024 From: ryan.gl.scott at gmail.com (Ryan Scott) Date: Wed, 3 Jul 2024 08:46:14 -0400 Subject: Deriving mechanisms in GHC Message-ID: (Disclaimer: I have never used hs-to-coq before, so everything below is merely an educated guess.) Looking at the source code for hs-to-coq, I see this [1]: addDerivedInstances :: GhcMonad m => TypecheckedModule -> m > TypecheckedModule > addDerivedInstances tcm = do > let Just (hsgroup, a, b, c) = tm_renamed_source tcm > > (_gbl_env, inst_infos, _rn_binds) <- initTcHack tcm $ do > let tcg_env = fst (tm_internals_ tcm) > tcg_env_hack = tcg_env { tcg_mod = fakeDerivingMod, > tcg_semantic_mod = fakeDerivingMod } > -- Set the module to make it look like we are in GHCi > -- so that GHC will allow us to re-typecheck existing > instances > setGblEnv tcg_env_hack $ > #if __GLASGOW_HASKELL__ >= 810 > tcInstDeclsDeriv [] (hs_derivds hsgroup) > #else > tcInstDeclsDeriv [] (hs_tyclds hsgroup >>= group_tyclds) > (hs_derivds hsgroup) > #endif If I understand this code correctly, it is using the GHC API to produce deriving-generated code and typecheck it, which seems reasonable. The part that confuses me is the #if __GLASGOW_HASKELL__ >= 810 bit. Prior to GHC 8.10, this code would obtain deriving-related information from two places: 1. The deriving clauses of data type declarations (i.e., the hs_tyclds) 2. Standalone deriving declarations (i.e., the hs_derivds) In GHC 8.10 and later, however, the code only obtains deriving-related information from standalone deriving declarations. This means that you'll completely skip over any derived instances that arise from deriving clauses, which is likely the source of the trouble you're encountering. In order to give advice on what you *should* be doing, let me briefly describe why the type of tcInstDeclsDeriv changed in GHC 8.10. The commit that changed tcInstDeclsDeriv is [2]. Prior to that commit, tcInstDeclsDeriv would obtain information related to deriving clauses via its first two arguments: 1. The first argument (of type [DerivInfo]) contained all of the deriving clauses for data family instances. (Note that hs-to-coq uses an empty list, so this means that hs-to-coq will always skip over data family instances, before or after GHC 8.10. I'm not sure if this should be considered as a separate bug.) 2. The second argument (of type [LTyClDecl GhcRn]) contained all of the data type definitions, which might have deriving clauses attached to them. The linked commit changes things so that *all* deriving clause–related information (both for ordinary data types as well as data family instances) is passed as a [DerivInfo] list. This means that hs-to-coq needs to produce those DerivInfo values somehow. As inspiration, you might want to look at how GHC calls tcInstDeclsDeriv here [3]. GHC first calls the tcTyAndClassDecls function to produce the DerivInfo values, and then it passes the DerivInfo values to tcInstDeclsDeriv. I would hope that something similar would work for hs-to-coq. Best, Ryan ----- [1] https://github.com/plclub/hs-to-coq/blob/03e823972fc7c5f85a300e554c691563f89a3e5f/src/lib/HsToCoq/Util/GHC/Deriving.hs#L50-L64 [2] https://gitlab.haskell.org/ghc/ghc/-/commit/679427f878e50ba5a9981bac4c2f9c76f4de3c3c#4c1af4850cb90ab2963edb06b69b57e87ccdf9c1 [3] https://gitlab.haskell.org/ghc/ghc/-/blob/bc1d435e399d8376b4e33d5d936424ff76cb686a/compiler/GHC/Tc/Module.hs#L1704-L1723 -------------- next part -------------- An HTML attachment was scrubbed... URL: From liyao at pdx.edu Wed Jul 3 19:02:30 2024 From: liyao at pdx.edu (Yao Li) Date: Wed, 3 Jul 2024 14:02:30 -0500 Subject: Deriving mechanisms in GHC In-Reply-To: References: Message-ID: Ah that looks like the reason. Thank you for catching that! Thanks for the pointers to the specific commit in GHC as well as the example! I am also curious if there is a recommended way to check the GHC API changes across different versions. We use GHC APIs extensively in hs-to-coq and there seem to be a number of changes between 8.4 and 8.10, so it's possible that we will run into other similar issues at some point, or when we update hs-to-coq to newer GHC versions in the future... Or are there any community recommendations on the best way to maintain our code base, which relies on GHC APIs? Thanks, Yao On Wed, Jul 3, 2024 at 7:46 AM Ryan Scott wrote: > (Disclaimer: I have never used hs-to-coq before, so everything below is > merely an educated guess.) > > Looking at the source code for hs-to-coq, I see this [1]: > > addDerivedInstances :: GhcMonad m => TypecheckedModule -> m >> TypecheckedModule >> addDerivedInstances tcm = do >> let Just (hsgroup, a, b, c) = tm_renamed_source tcm >> >> (_gbl_env, inst_infos, _rn_binds) <- initTcHack tcm $ do >> let tcg_env = fst (tm_internals_ tcm) >> tcg_env_hack = tcg_env { tcg_mod = fakeDerivingMod, >> tcg_semantic_mod = fakeDerivingMod } >> -- Set the module to make it look like we are in GHCi >> -- so that GHC will allow us to re-typecheck existing >> instances >> setGblEnv tcg_env_hack $ >> #if __GLASGOW_HASKELL__ >= 810 >> tcInstDeclsDeriv [] (hs_derivds hsgroup) >> #else >> tcInstDeclsDeriv [] (hs_tyclds hsgroup >>= group_tyclds) >> (hs_derivds hsgroup) >> #endif > > > If I understand this code correctly, it is using the GHC API to produce > deriving-generated code and typecheck it, which seems reasonable. The > part that confuses me is the #if __GLASGOW_HASKELL__ >= 810 bit. Prior to > GHC 8.10, this code would obtain deriving-related information from two > places: > > 1. The deriving clauses of data type declarations (i.e., the hs_tyclds) > 2. Standalone deriving declarations (i.e., the hs_derivds) > > In GHC 8.10 and later, however, the code only obtains deriving-related > information from standalone deriving declarations. This means that you'll > completely skip over any derived instances that arise from deriving > clauses, which is likely the source of the trouble you're encountering. > > In order to give advice on what you *should* be doing, let me briefly > describe why the type of tcInstDeclsDeriv changed in GHC 8.10. The commit > that changed tcInstDeclsDeriv is [2]. Prior to that commit, > tcInstDeclsDeriv would obtain information related to deriving clauses via > its first two arguments: > > 1. The first argument (of type [DerivInfo]) contained all of the > deriving clauses for data family instances. (Note that hs-to-coq uses an > empty list, so this means that hs-to-coq will always skip over data > family instances, before or after GHC 8.10. I'm not sure if this should be > considered as a separate bug.) > 2. The second argument (of type [LTyClDecl GhcRn]) contained all of > the data type definitions, which might have deriving clauses attached to > them. > > The linked commit changes things so that *all* deriving clause–related > information (both for ordinary data types as well as data family instances) > is passed as a [DerivInfo] list. This means that hs-to-coq needs to > produce those DerivInfo values somehow. As inspiration, you might want to > look at how GHC calls tcInstDeclsDeriv here [3]. GHC first calls the > tcTyAndClassDecls function to produce the DerivInfo values, and then it > passes the DerivInfo values to tcInstDeclsDeriv. I would hope that > something similar would work for hs-to-coq. > > Best, > > Ryan > ----- > [1] > https://github.com/plclub/hs-to-coq/blob/03e823972fc7c5f85a300e554c691563f89a3e5f/src/lib/HsToCoq/Util/GHC/Deriving.hs#L50-L64 > [2] > https://gitlab.haskell.org/ghc/ghc/-/commit/679427f878e50ba5a9981bac4c2f9c76f4de3c3c#4c1af4850cb90ab2963edb06b69b57e87ccdf9c1 > [3] > https://gitlab.haskell.org/ghc/ghc/-/blob/bc1d435e399d8376b4e33d5d936424ff76cb686a/compiler/GHC/Tc/Module.hs#L1704-L1723 > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > -------------- next part -------------- An HTML attachment was scrubbed... URL: From simon.peytonjones at gmail.com Wed Jul 3 19:20:30 2024 From: simon.peytonjones at gmail.com (Simon Peyton Jones) Date: Wed, 3 Jul 2024 20:20:30 +0100 Subject: Deriving mechanisms in GHC In-Reply-To: References: Message-ID: > I am also curious if there is a recommended way to check the GHC API changes across different versions. The difficulty is that we don't *have* a published, well designed GHC API for clients to use, and for GHC devs to strive to maintain stable. You aren't the only one to find this problem see these tickets . You might want to turn this thread into a new ticket, so you get included in that list. There is a project to design the API, but it is stalled https://discourse.haskell.org/t/charting-a-course-toward-a-stable-api-for-ghc/7646 It would be fantastic if someone, or a small group, was willing to take the lead here. Simon On Wed, 3 Jul 2024 at 20:02, Yao Li wrote: > Ah that looks like the reason. Thank you for catching that! Thanks for the > pointers to the specific commit in GHC as well as the example! > > I am also curious if there is a recommended way to check the GHC API > changes across different versions. We use GHC APIs extensively in hs-to-coq > and there seem to be a number of changes between 8.4 and 8.10, so it's > possible that we will run into other similar issues at some point, or when > we update hs-to-coq to newer GHC versions in the future... Or are there any > community recommendations on the best way to maintain our code base, which > relies on GHC APIs? > > Thanks, > Yao > > On Wed, Jul 3, 2024 at 7:46 AM Ryan Scott wrote: > >> (Disclaimer: I have never used hs-to-coq before, so everything below is >> merely an educated guess.) >> >> Looking at the source code for hs-to-coq, I see this [1]: >> >> addDerivedInstances :: GhcMonad m => TypecheckedModule -> m >>> TypecheckedModule >>> addDerivedInstances tcm = do >>> let Just (hsgroup, a, b, c) = tm_renamed_source tcm >>> >>> (_gbl_env, inst_infos, _rn_binds) <- initTcHack tcm $ do >>> let tcg_env = fst (tm_internals_ tcm) >>> tcg_env_hack = tcg_env { tcg_mod = fakeDerivingMod, >>> tcg_semantic_mod = fakeDerivingMod } >>> -- Set the module to make it look like we are in GHCi >>> -- so that GHC will allow us to re-typecheck existing >>> instances >>> setGblEnv tcg_env_hack $ >>> #if __GLASGOW_HASKELL__ >= 810 >>> tcInstDeclsDeriv [] (hs_derivds hsgroup) >>> #else >>> tcInstDeclsDeriv [] (hs_tyclds hsgroup >>= group_tyclds) >>> (hs_derivds hsgroup) >>> #endif >> >> >> If I understand this code correctly, it is using the GHC API to produce >> deriving-generated code and typecheck it, which seems reasonable. The >> part that confuses me is the #if __GLASGOW_HASKELL__ >= 810 bit. Prior >> to GHC 8.10, this code would obtain deriving-related information from >> two places: >> >> 1. The deriving clauses of data type declarations (i.e., the hs_tyclds >> ) >> 2. Standalone deriving declarations (i.e., the hs_derivds) >> >> In GHC 8.10 and later, however, the code only obtains deriving-related >> information from standalone deriving declarations. This means that you'll >> completely skip over any derived instances that arise from deriving >> clauses, which is likely the source of the trouble you're encountering. >> >> In order to give advice on what you *should* be doing, let me briefly >> describe why the type of tcInstDeclsDeriv changed in GHC 8.10. The >> commit that changed tcInstDeclsDeriv is [2]. Prior to that commit, >> tcInstDeclsDeriv would obtain information related to deriving clauses >> via its first two arguments: >> >> 1. The first argument (of type [DerivInfo]) contained all of the >> deriving clauses for data family instances. (Note that hs-to-coq uses an >> empty list, so this means that hs-to-coq will always skip over data >> family instances, before or after GHC 8.10. I'm not sure if this should be >> considered as a separate bug.) >> 2. The second argument (of type [LTyClDecl GhcRn]) contained all of >> the data type definitions, which might have deriving clauses attached to >> them. >> >> The linked commit changes things so that *all* deriving clause–related >> information (both for ordinary data types as well as data family instances) >> is passed as a [DerivInfo] list. This means that hs-to-coq needs to >> produce those DerivInfo values somehow. As inspiration, you might want >> to look at how GHC calls tcInstDeclsDeriv here [3]. GHC first calls the >> tcTyAndClassDecls function to produce the DerivInfo values, and then it >> passes the DerivInfo values to tcInstDeclsDeriv. I would hope that >> something similar would work for hs-to-coq. >> >> Best, >> >> Ryan >> ----- >> [1] >> https://github.com/plclub/hs-to-coq/blob/03e823972fc7c5f85a300e554c691563f89a3e5f/src/lib/HsToCoq/Util/GHC/Deriving.hs#L50-L64 >> [2] >> https://gitlab.haskell.org/ghc/ghc/-/commit/679427f878e50ba5a9981bac4c2f9c76f4de3c3c#4c1af4850cb90ab2963edb06b69b57e87ccdf9c1 >> [3] >> https://gitlab.haskell.org/ghc/ghc/-/blob/bc1d435e399d8376b4e33d5d936424ff76cb686a/compiler/GHC/Tc/Module.hs#L1704-L1723 >> _______________________________________________ >> ghc-devs mailing list >> ghc-devs at haskell.org >> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs >> > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > -------------- next part -------------- An HTML attachment was scrubbed... URL: From liyao at pdx.edu Wed Jul 3 19:44:51 2024 From: liyao at pdx.edu (Yao Li) Date: Wed, 3 Jul 2024 14:44:51 -0500 Subject: Deriving mechanisms in GHC In-Reply-To: References: Message-ID: Hi Simon, Thanks for all these pointers! > You aren't the only one to find this problem see these tickets. You might > want to turn this thread into a new ticket, so you get included in that > list. > Good to know! From the list, I see that the LiquidHaskell team also ran into the same issue and they proposed this ticket, which I should thumb up: https://gitlab.haskell.org/ghc/ghc/-/issues/24096 It would be fantastic if someone, or a small group, was willing to take the > lead here. Indeed! Best, Yao -------------- next part -------------- An HTML attachment was scrubbed... URL: From gergo at erdi.hu Mon Jul 8 04:21:07 2024 From: gergo at erdi.hu (=?ISO-8859-2?Q?=C9RDI_Gerg=F5?=) Date: Mon, 8 Jul 2024 12:21:07 +0800 (+08) Subject: Cannot build `master`, tries to use GHC 9.2?! Message-ID: <9ac3ac55-343f-e7cb-4d8e-dd34127484ad@erdi.hu> Hi, I'm trying to build GHC 5b1aefb70edbd54ac899896df39d8f3d6c579518 but I'm running into a weird problem. Using `hadrian/build-stack`, it fails quite early on with: | Run Ghc CompileCWithGhc (Stage0 GlobalLibs): utils/hsc2hs/cbits/utils.c => _build/stageBoot/utils/hsc2hs/build/c/cbits/utils.o Command line: ghc -Wall -hisuf hi -osuf o -hcsuf hc -static -hide-all-packages -no-user-package-db '-package-env -' '-package-db _build/stageBoot/inplace/package.conf.d' '-this-unit-id hsc2hs-0.68.10-inplace' '-this-package-name hsc2hs' '-package-id base-4.16.4.0' '-package-id containers-0.6.5.1' '-package-id directory-1.3.6.2' '-package-id filepath-1.4.2.2' '-package-id process-1.6.16.0' -i -i/home/cactus/prog/haskell/ghc/ghc.master/_build/stageBoot/utils/hsc2hs/build -i/home/cactus/prog/haskell/ghc/ghc.master/_build/stageBoot/utils/hsc2hs/build/hsc2hs/autogen -i/home/cactus/prog/haskell/ghc/ghc.master/utils/hsc2hs/src/ -I_build/stageBoot/utils/hsc2hs/build -I/home/cactus/.ghcup/ghc/9.2.8/lib/ghc-9.2.8/process-1.6.16.0/include -I/home/cactus/.ghcup/ghc/9.2.8/lib/ghc-9.2.8/unix-2.7.2.2/include -I/home/cactus/.ghcup/ghc/9.2.8/lib/ghc-9.2.8/time-1.11.1.1/include -I/home/cactus/.ghcup/ghc/9.2.8/lib/ghc-9.2.8/bytestring-0.11.4.0/include -I/home/cactus/.ghcup/ghc/9.2.8/lib/ghc-9.2.8/base-4.16.4.0/include -I/home/cactus/.ghcup/ghc/9.2.8/lib/ghc-9.2.8/ghc-bignum-1.2/include -I/home/cactus/.ghcup/ghc/9.2.8/lib/ghc-9.2.8/include -optP-include -optP_build/stageBoot/utils/hsc2hs/build/hsc2hs/autogen/cabal_macros.h -optP-DIN_GHC_TREE -outputdir _build/stageBoot/utils/hsc2hs/build -fdiagnostics-color=always -Wall -XHaskell2010 -optc-I_build/stageBoot/utils/hsc2hs/build -optc-I/home/cactus/.ghcup/ghc/9.2.8/lib/ghc-9.2.8/process-1.6.16.0/include -optc-I/home/cactus/.ghcup/ghc/9.2.8/lib/ghc-9.2.8/unix-2.7.2.2/include -optc-I/home/cactus/.ghcup/ghc/9.2.8/lib/ghc-9.2.8/time-1.11.1.1/include -optc-I/home/cactus/.ghcup/ghc/9.2.8/lib/ghc-9.2.8/bytestring-0.11.4.0/include -optc-I/home/cactus/.ghcup/ghc/9.2.8/lib/ghc-9.2.8/base-4.16.4.0/include -optc-I/home/cactus/.ghcup/ghc/9.2.8/lib/ghc-9.2.8/ghc-bignum-1.2/include -optc-I/home/cactus/.ghcup/ghc/9.2.8/lib/ghc-9.2.8/include -optc-Wno-error=inline -c utils/hsc2hs/cbits/utils.c -o _build/stageBoot/utils/hsc2hs/build/c/cbits/utils.o -O -H32m ===> Command failed with error code: 1 ghc: unrecognised flag: -this-package-name hsc2hs I don't know where all those references to GHC 9.2.8 come from. I've run `configure` with GHC 9.10.1 on the $PATH and it correctly reports it as the bootstrapping version: Bootstrapping using : ghc which is version : 9.10.1 What am I doing wrong? Gergo From matthewtpickering at gmail.com Mon Jul 8 13:30:40 2024 From: matthewtpickering at gmail.com (Matthew Pickering) Date: Mon, 8 Jul 2024 14:30:40 +0100 Subject: Final Call for Talks: Haskell Implementors' Workshop 2024 In-Reply-To: References: Message-ID: ``` ACM SIGPLAN Haskell Implementors' Workshop https://icfp24.sigplan.org/home/hiw-2024 Milan, Italy, September 2, 2024 Co-located with ICFP 2024 https://icfp24.sigplan.org/ ``` ## Important dates * Deadline: July 19, 2024 (AoE) * Notification: August 2, 2024 * Workshop: September 2, 2024 The 16th Haskell Implementors’ Workshop is to be held alongside ICFP 2024 this year in Milan. It is a forum for people involved in the design and development of Haskell implementations, tools, libraries, and supporting infrastructure to share their work and to discuss future directions and collaborations with others. Talks and/or demos are proposed by submitting an abstract, and selected by a small program committee. There will be no published proceedings. The workshop will be informal and interactive, with open spaces in the timetable and room for ad-hoc discussion, demos, and lightning talks. ## Scope and Target Audience It is important to distinguish the Haskell Implementors’ Workshop from the Haskell Symposium which is also co-located with ICFP 2024. The Haskell Symposium is for the publication of Haskell-related research. In contrast, the Haskell Implementors’ Workshop will have no proceedings – although we will aim to make talk videos, slides, and presented data available with the consent of the speakers. The Implementors’ Workshop is an ideal place to describe a Haskell extension, describe works-in-progress, demo a new Haskell-related tool, or even propose future lines of Haskell development. Members of the wider Haskell community are encouraged to attend the workshop – we need your feedback to keep the Haskell ecosystem thriving. Students working with Haskell are especially encouraged to share their work. The scope covers any of the following topics. There may be some topics that people feel we’ve missed, so by all means submit a proposal even if it doesn’t fit exactly into one of these buckets: * Compilation techniques * Language features and extensions * Type system implementation * Concurrency and parallelism: language design and implementation * Performance, optimization and benchmarking * Virtual machines and run-time systems * Libraries and tools for development or deployment ## Talks We invite proposals from potential speakers for talks and demonstrations. We are aiming for 20-minute talks with 5 minutes for questions and changeovers. We want to hear from people writing compilers, tools, or libraries, people with cool ideas for directions in which we should take the platform, proposals for new features to be implemented, and half-baked crazy ideas. Please submit a talk title and abstract of no more than 300 words. Submissions can be made via HotCRP at https://hiw2024.hotcrp.com until July 19 (anywhere on earth). We will also have a lightning talks session. These have been very well received in recent years, and we aim to increase the time available to them. Lightning talks should be ~7mins and are scheduled on the day of the workshop. Suggested topics for lightning talks are to present a single idea, a work-in-progress project, a problem to intrigue and perplex Haskell implementors, or simply to ask for feedback and collaborators. ## Program Committee * Matthew Pickering (Well-Typed LLP) * David Binder (University of Tübingen) * Gabriella Gonzalez (Mercury) * Sylvain Henry (IOG) * Teofil Camarasu (CircuitHub) ## Contact * Matthew Pickering From simon.peytonjones at gmail.com Tue Jul 9 08:57:43 2024 From: simon.peytonjones at gmail.com (Simon Peyton Jones) Date: Tue, 9 Jul 2024 09:57:43 +0100 Subject: Cabal woe Message-ID: Friends I'm trying to repro #25064 with my development build of GHC. The test case depends on packages `primitive` and `hashtables`. So I try this (see below). Alas, apparently after the `cabal install`, it can't find Prelude. What should I do? I tried removing the "environments" file, whatever that is, which then meant it could find Prelude -- but the libraries were no longer installed. I lack a decent model of what is going on with installing packages for my development builds. Is there a write up anywhere? Thanks Simon bash$ cabal install --lib hashtables primitive --with-compiler $HOME/code/HEAD/$s2 Warning: Unknown/unsupported 'ghc' version detected (Cabal 3.10.1.0 supports 'ghc' version < 9.8): /home/simonpj/code/HEAD/_build/stage1/bin/ghc is version 9.11.20240517 Warning: Unknown/unsupported 'ghc' version detected (Cabal 3.10.1.0 supports 'ghc' version < 9.8): /home/simonpj/code/HEAD/_build/stage1/bin/ghc is version 9.11.20240517 Resolving dependencies... bash$ ~/code/HEAD/$s2 -c T25064.hs Loaded package environment from T25064.hs:1:8: error: [ ]8;;https://errors.haskell.org/messages/GHC-87110 \GHC-87110 ]8;; \] Could not load module ‘Prelude’. It is a member of the hidden package ‘base-4.20.0.0’. Use -v to see a list of the files searched for. | 1 | module Bug where | ^^^ bash$ rm /home/simonpj/.ghc/x86_64-linux-9.11.20240517/environments/default bash$ ~/code/HEAD/$s2 -c T25064.hs T25064.hs:3:1: error: [ ]8;;https://errors.haskell.org/messages/GHC-61948 \GHC-61948 ]8;; \] Could not find module ‘Control.Monad.Primitive’. Perhaps you meant Control.Monad.Writer (from mtl-2.3.1) Use -v to see a list of the files searched for. | 3 | import Control.Monad.Primitive | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ T25064.hs:4:1: error: [ ]8;;https://errors.haskell.org/messages/GHC-87110 \GHC-87110 ]8;; \] Could not find module ‘Data.HashTable.ST.Basic’. Use -v to see a list of the files searched for. | 4 | import qualified Data.HashTable.ST.Basic as H | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -------------- next part -------------- An HTML attachment was scrubbed... URL: From matthewtpickering at gmail.com Tue Jul 9 09:20:56 2024 From: matthewtpickering at gmail.com (Matthew Pickering) Date: Tue, 9 Jul 2024 10:20:56 +0100 Subject: Cabal woe In-Reply-To: References: Message-ID: `cabal install --lib` is very hard to use properly. I would advise against it. It is much easier to create a simple cabal project and use normal cabal build commands. I packaged things in a repo for you. https://github.com/mpickering/t25064 ``` cabal build -w ghc-9.6.2 (fails) cabal build -w /path/to/devel/compiler (fails with error message) ``` Matt On Tue, Jul 9, 2024 at 9:58 AM Simon Peyton Jones wrote: > > Friends > > I'm trying to repro #25064 with my development build of GHC. The test case depends on packages `primitive` and `hashtables`. So I try this (see below). > > Alas, apparently after the `cabal install`, it can't find Prelude. > > What should I do? I tried removing the "environments" file, whatever that is, which then meant it could find Prelude -- but the libraries were no longer installed. > > I lack a decent model of what is going on with installing packages for my development builds. Is there a write up anywhere? > > Thanks > > Simon > > > bash$ cabal install --lib hashtables primitive --with-compiler $HOME/code/HEAD/$s2 > Warning: Unknown/unsupported 'ghc' version detected (Cabal 3.10.1.0 supports > 'ghc' version < 9.8): /home/simonpj/code/HEAD/_build/stage1/bin/ghc is version > 9.11.20240517 > Warning: Unknown/unsupported 'ghc' version detected (Cabal 3.10.1.0 supports > 'ghc' version < 9.8): /home/simonpj/code/HEAD/_build/stage1/bin/ghc is version > 9.11.20240517 > Resolving dependencies... > bash$ ~/code/HEAD/$s2 -c T25064.hs > Loaded package environment from > T25064.hs:1:8: error: [ ]8;;https://errors.haskell.org/messages/GHC-87110 \GHC-87110 ]8;; \] > Could not load module ‘Prelude’. > It is a member of the hidden package ‘base-4.20.0.0’. > Use -v to see a list of the files searched for. > | > 1 | module Bug where > | ^^^ > > bash$ rm /home/simonpj/.ghc/x86_64-linux-9.11.20240517/environments/default > bash$ ~/code/HEAD/$s2 -c T25064.hs > T25064.hs:3:1: error: [ ]8;;https://errors.haskell.org/messages/GHC-61948 \GHC-61948 ]8;; \] > Could not find module ‘Control.Monad.Primitive’. > Perhaps you meant Control.Monad.Writer (from mtl-2.3.1) > Use -v to see a list of the files searched for. > | > 3 | import Control.Monad.Primitive > | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > > T25064.hs:4:1: error: [ ]8;;https://errors.haskell.org/messages/GHC-87110 \GHC-87110 ]8;; \] > Could not find module ‘Data.HashTable.ST.Basic’. > Use -v to see a list of the files searched for. > | > 4 | import qualified Data.HashTable.ST.Basic as H > | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > > > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs From sgraf1337 at gmail.com Tue Jul 9 12:23:13 2024 From: sgraf1337 at gmail.com (Sebastian Graf) Date: Tue, 9 Jul 2024 14:23:13 +0200 Subject: Cabal woe In-Reply-To: References: Message-ID: Hi Simon, Hi Matt, I think Simon has a point. I really do not enjoy creating a new throwaway project for every small reproducer I want to test. A cabal project means that I can't simply pass `-fforce-recomp -ddump-simpl -O` to a GHC invocation, for example. Instead I have to create a cabal file and remember all the fields that need to be set. (Or myself through a perceived 30 step wizard with `cabal new`, only to find that I have to open the file anyway to tweak build-depends...) Stupid as it may be, that subconsciously really affects my willingness to tackle an issue. Presumably this can be solved through some amount of automation/scripting. Perhaps it would be enough to share these scripts in a prominent location, for example the Wiki. Alternatively, I think I would enjoy something like `nix-shell -p`, where I can just fire up a temporary shell with all the deps in a locally visible package db. That way I could keep calling `ghc` directly. (Not that I suffer *much* from this issue at the moment anyway.) Sebastian Am Di., 9. Juli 2024 um 11:21 Uhr schrieb Matthew Pickering < matthewtpickering at gmail.com>: > `cabal install --lib` is very hard to use properly. I would advise against > it. > > It is much easier to create a simple cabal project and use normal > cabal build commands. I packaged things in a repo for you. > > https://github.com/mpickering/t25064 > > ``` > cabal build -w ghc-9.6.2 (fails) > cabal build -w /path/to/devel/compiler (fails with error message) > ``` > > Matt > > On Tue, Jul 9, 2024 at 9:58 AM Simon Peyton Jones > wrote: > > > > Friends > > > > I'm trying to repro #25064 with my development build of GHC. The test > case depends on packages `primitive` and `hashtables`. So I try this (see > below). > > > > Alas, apparently after the `cabal install`, it can't find Prelude. > > > > What should I do? I tried removing the "environments" file, whatever > that is, which then meant it could find Prelude -- but the libraries were > no longer installed. > > > > I lack a decent model of what is going on with installing packages for > my development builds. Is there a write up anywhere? > > > > Thanks > > > > Simon > > > > > > bash$ cabal install --lib hashtables primitive --with-compiler > $HOME/code/HEAD/$s2 > > Warning: Unknown/unsupported 'ghc' version detected (Cabal 3.10.1.0 > supports > > 'ghc' version < 9.8): /home/simonpj/code/HEAD/_build/stage1/bin/ghc is > version > > 9.11.20240517 > > Warning: Unknown/unsupported 'ghc' version detected (Cabal 3.10.1.0 > supports > > 'ghc' version < 9.8): /home/simonpj/code/HEAD/_build/stage1/bin/ghc is > version > > 9.11.20240517 > > Resolving dependencies... > > bash$ ~/code/HEAD/$s2 -c T25064.hs > > Loaded package environment from > > T25064.hs:1:8: error: [ ]8;; > https://errors.haskell.org/messages/GHC-87110 \GHC-87110 ]8;; \] > > Could not load module ‘Prelude’. > > It is a member of the hidden package ‘base-4.20.0.0’. > > Use -v to see a list of the files searched for. > > | > > 1 | module Bug where > > | ^^^ > > > > bash$ rm > /home/simonpj/.ghc/x86_64-linux-9.11.20240517/environments/default > > bash$ ~/code/HEAD/$s2 -c T25064.hs > > T25064.hs:3:1: error: [ ]8;; > https://errors.haskell.org/messages/GHC-61948 \GHC-61948 ]8;; \] > > Could not find module ‘Control.Monad.Primitive’. > > Perhaps you meant Control.Monad.Writer (from mtl-2.3.1) > > Use -v to see a list of the files searched for. > > | > > 3 | import Control.Monad.Primitive > > | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > > > > T25064.hs:4:1: error: [ ]8;; > https://errors.haskell.org/messages/GHC-87110 \GHC-87110 ]8;; \] > > Could not find module ‘Data.HashTable.ST.Basic’. > > Use -v to see a list of the files searched for. > > | > > 4 | import qualified Data.HashTable.ST.Basic as H > > | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > > > > > > _______________________________________________ > > ghc-devs mailing list > > ghc-devs at haskell.org > > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > -------------- next part -------------- An HTML attachment was scrubbed... URL: From simon.peytonjones at gmail.com Tue Jul 9 13:35:52 2024 From: simon.peytonjones at gmail.com (Simon Peyton Jones) Date: Tue, 9 Jul 2024 14:35:52 +0100 Subject: Cabal woe In-Reply-To: References: Message-ID: > > I think Simon has a point. I really do not enjoy creating a new throwaway > project for every small reproducer I want to test. My (clearly faulty) mental model is this. - I have a particular stage2 compiler, say $(GHC). Maybe built in my build tree, maybe installed. - If I say `$(GHC) -package primitive`, it right says "I don't know about primitiver" - I expect to be able to say `cabal install primitive --with-compiler=$(GHC)`, to extend my $(GHC) with the package `primitive`. - Thereafter I expect `$(GHC) -package primitive` to work -- after all, I have extended GHC to know about it. If I'm in a cabal project then these installed-by-default packages shouldn't make any difference; cabal will control everything. I'm sure it used to be like this, but something has changed, and I'm not sure why. Clearly my model of cabal is faulty, but I don't know how to get an accurate one. Is there a writeup I can read? e.g. What is the "environments" file? Why does it fail to find Prelude? Would it be possible to support the simple story above, as well? Simon On Tue, 9 Jul 2024 at 13:23, Sebastian Graf wrote: > Hi Simon, Hi Matt, > > I think Simon has a point. I really do not enjoy creating a new throwaway > project for every small reproducer I want to test. > A cabal project means that I can't simply pass `-fforce-recomp > -ddump-simpl -O` to a GHC invocation, for example. > Instead I have to create a cabal file and remember all the fields that > need to be set. (Or myself through a perceived 30 step wizard with > `cabal new`, only to find that I have to open the file anyway to tweak > build-depends...) > > Stupid as it may be, that subconsciously really affects my willingness to > tackle an issue. > > Presumably this can be solved through some amount of automation/scripting. > Perhaps it would be enough to share these scripts in a prominent location, > for example the Wiki. > Alternatively, I think I would enjoy something like `nix-shell -p`, where > I can just fire up a temporary shell with all the deps in a locally visible > package db. That way I could keep calling `ghc` directly. > > (Not that I suffer *much* from this issue at the moment anyway.) > > Sebastian > > Am Di., 9. Juli 2024 um 11:21 Uhr schrieb Matthew Pickering < > matthewtpickering at gmail.com>: > >> `cabal install --lib` is very hard to use properly. I would advise >> against it. >> >> It is much easier to create a simple cabal project and use normal >> cabal build commands. I packaged things in a repo for you. >> >> https://github.com/mpickering/t25064 >> >> ``` >> cabal build -w ghc-9.6.2 (fails) >> cabal build -w /path/to/devel/compiler (fails with error message) >> ``` >> >> Matt >> >> On Tue, Jul 9, 2024 at 9:58 AM Simon Peyton Jones >> wrote: >> > >> > Friends >> > >> > I'm trying to repro #25064 with my development build of GHC. The test >> case depends on packages `primitive` and `hashtables`. So I try this (see >> below). >> > >> > Alas, apparently after the `cabal install`, it can't find Prelude. >> > >> > What should I do? I tried removing the "environments" file, whatever >> that is, which then meant it could find Prelude -- but the libraries were >> no longer installed. >> > >> > I lack a decent model of what is going on with installing packages for >> my development builds. Is there a write up anywhere? >> > >> > Thanks >> > >> > Simon >> > >> > >> > bash$ cabal install --lib hashtables primitive --with-compiler >> $HOME/code/HEAD/$s2 >> > Warning: Unknown/unsupported 'ghc' version detected (Cabal 3.10.1.0 >> supports >> > 'ghc' version < 9.8): /home/simonpj/code/HEAD/_build/stage1/bin/ghc is >> version >> > 9.11.20240517 >> > Warning: Unknown/unsupported 'ghc' version detected (Cabal 3.10.1.0 >> supports >> > 'ghc' version < 9.8): /home/simonpj/code/HEAD/_build/stage1/bin/ghc is >> version >> > 9.11.20240517 >> > Resolving dependencies... >> > bash$ ~/code/HEAD/$s2 -c T25064.hs >> > Loaded package environment from >> > T25064.hs:1:8: error: [ ]8;; >> https://errors.haskell.org/messages/GHC-87110 \GHC-87110 ]8;; \] >> > Could not load module ‘Prelude’. >> > It is a member of the hidden package ‘base-4.20.0.0’. >> > Use -v to see a list of the files searched for. >> > | >> > 1 | module Bug where >> > | ^^^ >> > >> > bash$ rm >> /home/simonpj/.ghc/x86_64-linux-9.11.20240517/environments/default >> > bash$ ~/code/HEAD/$s2 -c T25064.hs >> > T25064.hs:3:1: error: [ ]8;; >> https://errors.haskell.org/messages/GHC-61948 \GHC-61948 ]8;; \] >> > Could not find module ‘Control.Monad.Primitive’. >> > Perhaps you meant Control.Monad.Writer (from mtl-2.3.1) >> > Use -v to see a list of the files searched for. >> > | >> > 3 | import Control.Monad.Primitive >> > | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ >> > >> > T25064.hs:4:1: error: [ ]8;; >> https://errors.haskell.org/messages/GHC-87110 \GHC-87110 ]8;; \] >> > Could not find module ‘Data.HashTable.ST.Basic’. >> > Use -v to see a list of the files searched for. >> > | >> > 4 | import qualified Data.HashTable.ST.Basic as H >> > | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ >> > >> > >> > _______________________________________________ >> > ghc-devs mailing list >> > ghc-devs at haskell.org >> > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs >> _______________________________________________ >> ghc-devs mailing list >> ghc-devs at haskell.org >> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From oleg.grenrus at iki.fi Tue Jul 9 13:59:19 2024 From: oleg.grenrus at iki.fi (Oleg Grenrus) Date: Tue, 9 Jul 2024 16:59:19 +0300 Subject: Cabal woe In-Reply-To: References: Message-ID: On 9.7.2024 16.35, Simon Peyton Jones wrote: > Would it be possible to support the simple story above, as well? From oleg.grenrus at iki.fi Tue Jul 9 14:07:07 2024 From: oleg.grenrus at iki.fi (Oleg Grenrus) Date: Tue, 9 Jul 2024 17:07:07 +0300 Subject: Cabal woe In-Reply-To: References: Message-ID: On 9.7.2024 16.35, Simon Peyton Jones wrote: > Would it be possible to support the simple story above, as well? TL;DR the GHC developers (e.g. you, richard, sebastian) are virtually the only group of people who want to invoke GHC manually, from your POV the story might seem simple, but in grand scheme of things it's niche and obscure. `cabal-install` (and `stack` FWIW) don't want you to have any (implicit) state. That's not great for "real world projects". You may want to read through https://github.com/haskell/cabal/issues/6481 issue, which is kind of an opener towards support what you want. But again, it's niche feature. Creating a package as Matt proposes doesn't take much time (once you have done it few times), and allows to easily share the snippet (as everything can be contained inside package or/and project files). - Oleg From oleg.grenrus at iki.fi Tue Jul 9 14:08:09 2024 From: oleg.grenrus at iki.fi (Oleg Grenrus) Date: Tue, 9 Jul 2024 17:08:09 +0300 Subject: Cabal woe In-Reply-To: References: Message-ID: <41f2123b-6300-422f-8644-9e5d72609499@iki.fi> On 9.7.2024 15.23, Sebastian Graf wrote: > I can't simply pass `-fforce-recomp -ddump-simpl -O` From oleg.grenrus at iki.fi Tue Jul 9 14:11:56 2024 From: oleg.grenrus at iki.fi (Oleg Grenrus) Date: Tue, 9 Jul 2024 17:11:56 +0300 Subject: Cabal woe In-Reply-To: References: Message-ID: On 9.7.2024 15.23, Sebastian Graf wrote: > I can't simply pass `-fforce-recomp -ddump-simpl -O` You can, `cabal build --ghc-options= "-fforce-recomp -ddump-simpl -O"` should work (though it applies to all packages). > and remember all the fields that need to be set. ( `cabal init` is your friend. That said, there are very little fields to be set for minimal package, `cabal-version, name, version` and the library pragma with `build-depends` and `exposed-modules`. After you have done such package few times, you will remember those easily. I'd argue a lot more easily that remembering how to use `cabal install --lib` properly. - Oleg From allbery.b at gmail.com Tue Jul 9 14:14:53 2024 From: allbery.b at gmail.com (Brandon Allbery) Date: Tue, 9 Jul 2024 10:14:53 -0400 Subject: Cabal woe In-Reply-To: References: Message-ID: You can also `cabal init -n` to get a minimal cabal file, then `cabal repl -b primitive` or whatever. On Tue, Jul 9, 2024 at 10:12 AM Oleg Grenrus wrote: > > On 9.7.2024 15.23, Sebastian Graf wrote: > > I can't simply pass `-fforce-recomp -ddump-simpl -O` > > You can, `cabal build --ghc-options= "-fforce-recomp -ddump-simpl -O"` > should work (though it applies to all packages). > > > and remember all the fields that need to be set. ( > > `cabal init` is your friend. That said, there are very little fields to > be set for minimal package, `cabal-version, name, version` and the > library pragma with `build-depends` and `exposed-modules`. > > After you have done such package few times, you will remember those > easily. I'd argue a lot more easily that remembering how to use `cabal > install --lib` properly. > > - Oleg > > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > -- brandon s allbery kf8nh allbery.b at gmail.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From jaro.reinders at gmail.com Tue Jul 9 15:04:27 2024 From: jaro.reinders at gmail.com (J. Reinders) Date: Tue, 9 Jul 2024 17:04:27 +0200 Subject: Cabal woe In-Reply-To: References: Message-ID: <2145C591-190E-4241-8C05-2FDE3917FA04@gmail.com> One other alternative is to use a cabal script. You can simply add a multiline comment to your haskell files: {- cabal: build-depends: base, primitive, hashtables -} Then you can run the file with ‘cabal run T25064.hs’, or you can add a shebang at the top of the file and make it executable: #!/usr/bin/env cabal For more info see the cabal docs: https://cabal.readthedocs.io/en/latest/cabal-commands.html#cabal-run Cheers, Jaro From amindfv at mailbox.org Tue Jul 9 17:06:02 2024 From: amindfv at mailbox.org (amindfv at mailbox.org) Date: Tue, 9 Jul 2024 13:06:02 -0400 Subject: Cabal woe In-Reply-To: References: Message-ID: On Tue, Jul 09, 2024 at 05:07:07PM +0300, Oleg Grenrus wrote: > > On 9.7.2024 16.35, Simon Peyton Jones wrote: > > Would it be possible to support the simple story above, as well? > > TL;DR the GHC developers (e.g. you, richard, sebastian) are virtually the > only group of people who want to invoke GHC manually [...] Genuinely curious, what makes you say this? My only data is anecdata, but I've talked to quite a lot of people who miss the old days of `cabal install`-ing a package (particularly if it requires tweaking flags) and then having it at your fingertips whenever you run GHCi, `runghc`, or GHC. Some packages are so central to our workflows that it's almost like needing to `mkdir foo ; cd foo ; cabal init ; $EDITOR foo.cabal ; cabal repl` to get the Prelude. Nobody misses the package conflicts that could come from long-term thoughtless default-global installation, but to say that almost nobody wants to invoke GHC manually seems like a major stretch. If we had a cleaner solution, I'm willing to bet a lot of people would drop the `cabal init` workflow for everyday tinkering like a hot potato. Tom P.S. I've built my own hacks on top of GHC environment files to recapture this way of working, but that's another story. From klebinger.andreas at gmx.at Tue Jul 9 17:13:46 2024 From: klebinger.andreas at gmx.at (Andreas Klebinger) Date: Tue, 9 Jul 2024 19:13:46 +0200 Subject: Sunsetting CentOS support Message-ID: <50cccb30-36e7-4b70-839f-3c722d17739f@gmx.at> Hello devs, with CentOS 7 reaching EOL we are demoting it to a Tier 2 platform and will eventually drop support completely. There are no immediate plans to stop supporting it, but we will not allocate any more resources towards support for it and will likely drop support completely should it break in the future. We introduced support for Rocky Linux as replacement for CentOS some time ago, so users who need a RHEL compatible GHC distribution should be able to use GHC packaged for Rocky Linux instead without issues. Best Regards Andreas Klebinger From amindfv at mailbox.org Tue Jul 9 17:31:41 2024 From: amindfv at mailbox.org (amindfv at mailbox.org) Date: Tue, 9 Jul 2024 13:31:41 -0400 Subject: Cabal woe In-Reply-To: References: Message-ID: On Tue, Jul 09, 2024 at 01:06:02PM -0400, amindfv--- via ghc-devs wrote: > [...]I've talked to quite a lot of people who miss the old days of `cabal install`-ing a package (particularly if it requires tweaking flags) [...] Forgot to mention an additional common workflow issue: having pre-installed packages that come from non-{H,St}ackage sources. Just another source of friction, where typing g-h-c-i-ENTER was by far the easiest way to use the package. Tom From george.colpitts at gmail.com Tue Jul 9 17:32:55 2024 From: george.colpitts at gmail.com (George Colpitts) Date: Tue, 9 Jul 2024 14:32:55 -0300 Subject: Cabal woe In-Reply-To: References: Message-ID: I agree and believe cabal v2-install has regressed from v1-install. My use case is I'm a hobbyist who writes small programs. I assume there are learners working outside a university class who are in a similar situation. Even experienced developers like Richard Eisenberg seem to agree, from https://github.com/haskell/cabal/issues/7832: Further, we do think the use case Richard has outlined of a "nice universe for a playground" is an important one. Great -- glad to hear it. Thank you. However, experience has repeatedly shown that "do what I mean" algos in package management lead to frustrated and confused users, and while things should be as easy as possible, they should also be no easier than that, and finding that balance is not going to be simple. I agree here, and have not thought about the specific of the algorithms above. But, there is an algorithm that is known to (mostly) work: what v1 did. That is, the universe expands only by user request, when that user asks to add a new package to their universe. At that point, cabal selects the most recent release of that package (or whatever version the user asked for). Then, cabal uses the same algorithm it usually uses to decide its dependencies. This worked for the lifetime of v1. But it only mostly worked, because of cabal-hell: that's why v2 exists! So there would be a potential pitfall in that users try to build an inconsistent universe, or they build a consistent universe that doesn't serve their actual needs. I'm not sure those pitfalls are avoidable... but maybe they could be mitigated with cabal destroy-universe which blows away the current universe, allowing the user to re-expand it from scratch. (In my own practice, a universe went stale at about the same rate as GHC released, and so I just moved to a new GHC when things got rough in the universe.)... IMHO the current situation is a deterrent to beginners trying to learn Haskell. There is a huge amount to learn and adding cabal and stack etc can appear as a huge pain to them, particularly if they have used pip3, the Python package manager. Also I'm surprised that Cabal isn't under the control of the Core Library committee, I think that would have made for a smoother transition from cabal v1 to v2 Thanks George On Tue, Jul 9, 2024 at 2:06 PM amindfv--- via ghc-devs wrote: > On Tue, Jul 09, 2024 at 05:07:07PM +0300, Oleg Grenrus wrote: > > > > On 9.7.2024 16.35, Simon Peyton Jones wrote: > > > Would it be possible to support the simple story above, as well? > > > > TL;DR the GHC developers (e.g. you, richard, sebastian) are virtually the > > only group of people who want to invoke GHC manually [...] > > Genuinely curious, what makes you say this? > > My only data is anecdata, but I've talked to quite a lot of people who > miss the old days of `cabal install`-ing a package (particularly if it > requires tweaking flags) and then having it at your fingertips whenever you > run GHCi, `runghc`, or GHC. Some packages are so central to our workflows > that it's almost like needing to `mkdir foo ; cd foo ; cabal init ; $EDITOR > foo.cabal ; cabal repl` to get the Prelude. > > Nobody misses the package conflicts that could come from long-term > thoughtless default-global installation, but to say that almost nobody > wants to invoke GHC manually seems like a major stretch. If we had a > cleaner solution, I'm willing to bet a lot of people would drop the `cabal > init` workflow for everyday tinkering like a hot potato. > > Tom > > P.S. I've built my own hacks on top of GHC environment files to recapture > this way of working, but that's another story. > > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > -------------- next part -------------- An HTML attachment was scrubbed... URL: From oleg.grenrus at iki.fi Tue Jul 9 17:50:18 2024 From: oleg.grenrus at iki.fi (Oleg Grenrus) Date: Tue, 9 Jul 2024 20:50:18 +0300 Subject: Cabal woe In-Reply-To: References: Message-ID: cabal-install has various means to address that too. You can have `packages: /anywhere/onyourdisk` or `packages: http://somewhere.else/tar.gz in your cabal.project. Or you can have `source-repository-packages` for accessing remote git repositories. Or you can setup https://cabal.readthedocs.io/en/stable/config.html#local-no-index-repositories for a bit more permanent and reusable solution. cabal-install has ways to approach those problems *without having a stateful/implicit setup*. And the workflow is the same as usual workflow with packages and projects. In fact, real work projects do use source-repository-packages (and sometimes in project vendored `.tar.gz` source distributions). An anecdote from my side: I wrote`cabal-env` [1] for myself somewhat five years ago. I don't think I have used in the last two years. The package workflow is often better. Matt's example highlights a strength: cabal build -w ghc-9.6.2 (fails) cabal build -w /path/to/devel/compiler (fails with error message) As all the dependencies/build-info is written down in the .cabal file, trying out with different versions of GHC is trivial. It's not with environment files. [1] https://github.com/phadej/cabal-extras/tree/master/cabal-env On 9.7.2024 20.31, amindfv at mailbox.org wrote: > On Tue, Jul 09, 2024 at 01:06:02PM -0400, amindfv--- via ghc-devs wrote: >> [...]I've talked to quite a lot of people who miss the old days of `cabal install`-ing a package (particularly if it requires tweaking flags) [...] > Forgot to mention an additional common workflow issue: having pre-installed packages that come from non-{H,St}ackage sources. Just another source of friction, where typing g-h-c-i-ENTER was by far the easiest way to use the package. > > Tom > From amindfv at mailbox.org Tue Jul 9 19:05:46 2024 From: amindfv at mailbox.org (amindfv at mailbox.org) Date: Tue, 9 Jul 2024 15:05:46 -0400 Subject: Cabal woe In-Reply-To: References: Message-ID: On Tue, Jul 09, 2024 at 08:50:18PM +0300, Oleg Grenrus wrote: > cabal-install has various means to address that too. You can have `packages: > /anywhere/onyourdisk` or `packages: http://somewhere.else/tar.gz in your > cabal.project. Or you can have `source-repository-packages` for accessing > remote git repositories. Or you can setup https://cabal.readthedocs.io/en/stable/config.html#local-no-index-repositories > for a bit more permanent and reusable solution. > > cabal-install has ways to approach those problems *without having a > stateful/implicit setup*. And the workflow is the same as usual workflow > with packages and projects. In fact, real work projects do use > source-repository-packages (and sometimes in project vendored `.tar.gz` > source distributions). Yep, I'm aware (and in fact have used `source-repository-package` on business-critical work projects. I appreciate its existence!). However, I'm specifically trying to get to the bottom of the sentiment that "the GHC developers (e.g. you, richard, sebastian) are virtually the only group of people who want to invoke GHC manually." My impression these past few years is that it's desirable to have one or more simple global states, accessible from e.g. ghci, and it's also desirable to avoid package conflicts, and there's tension between these needs. The above sentiment, though, seems to imply the former simply isn't desirable to cabal devs in the first place. > An anecdote from my side: I wrote`cabal-env` [1] for myself somewhat five > years ago. I don't think I have used in the last two years. The package > workflow is often better. I've used it! My current work is another attempt to tackle a similar use-case. I disagree (strongly) that for my needs the package workflow is better. Cheers, Tom From gergo at erdi.hu Wed Jul 10 06:53:05 2024 From: gergo at erdi.hu (=?ISO-8859-2?Q?=C9RDI_Gerg=F5?=) Date: Wed, 10 Jul 2024 14:53:05 +0800 (+08) Subject: Cannot build `master`, tries to use GHC 9.2?! In-Reply-To: <9ac3ac55-343f-e7cb-4d8e-dd34127484ad@erdi.hu> References: <9ac3ac55-343f-e7cb-4d8e-dd34127484ad@erdi.hu> Message-ID: By playing around with GHC versions, Stack resolvers, and trying `hadrian/build` instead of `hadrian/build-stack`, I now got somewhat further: Error: hadrian: Missing dependency on a foreign library: * Missing (or bad) header file: HsBase.h But `HsBase.h` should come from the GHC source tree itself, right? So it's not an unresolved external dependency. It's very much possible that this is all because of some local configuration problem on my machine, but I'm still going to need some hints on where to even start looking. Thanks, Gerg On Mon, 8 Jul 2024, ÉRDI Gergő wrote: > Hi, > > I'm trying to build GHC 5b1aefb70edbd54ac899896df39d8f3d6c579518 but I'm > running into a weird problem. Using `hadrian/build-stack`, it fails quite > early on with: > > | Run Ghc CompileCWithGhc (Stage0 GlobalLibs): utils/hsc2hs/cbits/utils.c => > _build/stageBoot/utils/hsc2hs/build/c/cbits/utils.o > Command line: ghc -Wall -hisuf hi -osuf o -hcsuf hc -static > -hide-all-packages -no-user-package-db '-package-env -' '-package-db > _build/stageBoot/inplace/package.conf.d' '-this-unit-id > hsc2hs-0.68.10-inplace' '-this-package-name hsc2hs' '-package-id > base-4.16.4.0' '-package-id containers-0.6.5.1' '-package-id > directory-1.3.6.2' '-package-id filepath-1.4.2.2' '-package-id > process-1.6.16.0' -i > -i/home/cactus/prog/haskell/ghc/ghc.master/_build/stageBoot/utils/hsc2hs/build > -i/home/cactus/prog/haskell/ghc/ghc.master/_build/stageBoot/utils/hsc2hs/build/hsc2hs/autogen > -i/home/cactus/prog/haskell/ghc/ghc.master/utils/hsc2hs/src/ > -I_build/stageBoot/utils/hsc2hs/build > -I/home/cactus/.ghcup/ghc/9.2.8/lib/ghc-9.2.8/process-1.6.16.0/include > -I/home/cactus/.ghcup/ghc/9.2.8/lib/ghc-9.2.8/unix-2.7.2.2/include > -I/home/cactus/.ghcup/ghc/9.2.8/lib/ghc-9.2.8/time-1.11.1.1/include > -I/home/cactus/.ghcup/ghc/9.2.8/lib/ghc-9.2.8/bytestring-0.11.4.0/include > -I/home/cactus/.ghcup/ghc/9.2.8/lib/ghc-9.2.8/base-4.16.4.0/include > -I/home/cactus/.ghcup/ghc/9.2.8/lib/ghc-9.2.8/ghc-bignum-1.2/include > -I/home/cactus/.ghcup/ghc/9.2.8/lib/ghc-9.2.8/include -optP-include > -optP_build/stageBoot/utils/hsc2hs/build/hsc2hs/autogen/cabal_macros.h > -optP-DIN_GHC_TREE -outputdir _build/stageBoot/utils/hsc2hs/build > -fdiagnostics-color=always -Wall -XHaskell2010 > -optc-I_build/stageBoot/utils/hsc2hs/build > -optc-I/home/cactus/.ghcup/ghc/9.2.8/lib/ghc-9.2.8/process-1.6.16.0/include > -optc-I/home/cactus/.ghcup/ghc/9.2.8/lib/ghc-9.2.8/unix-2.7.2.2/include > -optc-I/home/cactus/.ghcup/ghc/9.2.8/lib/ghc-9.2.8/time-1.11.1.1/include > -optc-I/home/cactus/.ghcup/ghc/9.2.8/lib/ghc-9.2.8/bytestring-0.11.4.0/include > -optc-I/home/cactus/.ghcup/ghc/9.2.8/lib/ghc-9.2.8/base-4.16.4.0/include > -optc-I/home/cactus/.ghcup/ghc/9.2.8/lib/ghc-9.2.8/ghc-bignum-1.2/include > -optc-I/home/cactus/.ghcup/ghc/9.2.8/lib/ghc-9.2.8/include > -optc-Wno-error=inline -c utils/hsc2hs/cbits/utils.c -o > _build/stageBoot/utils/hsc2hs/build/c/cbits/utils.o -O -H32m > ===> Command failed with error code: 1 > ghc: unrecognised flag: -this-package-name hsc2hs > > > I don't know where all those references to GHC 9.2.8 come from. I've run > `configure` with GHC 9.10.1 on the $PATH and it correctly reports it as the > bootstrapping version: > > Bootstrapping using : ghc > which is version : 9.10.1 > > > What am I doing wrong? > > Gergo > -- .--= ULLA! =-----------------. \ http://gergo.erdi.hu \ `---= gergo at erdi.hu =-------' From simon.peytonjones at gmail.com Wed Jul 10 08:27:21 2024 From: simon.peytonjones at gmail.com (Simon Peyton Jones) Date: Wed, 10 Jul 2024 09:27:21 +0100 Subject: Cabal woe In-Reply-To: References: Message-ID: Oleg (a cabal hero, btw, thank you Oleg) writes `cabal-install` (and `stack` FWIW) don't want you to have any (implicit) state. That's not great for "real world projects". But you necessarily have global state anyway! When I say `ghc -c Foo.hs` I get a particular Prelude, which in turn depends on a bunch of libraries. My mental model was simply that I can extend that global state, so that rather than only getting Prelude I get whatever libraries I install. Of course, if I go via cabal, all that global state would be ignored, because cabal would specify everything explicitly. To put it another way, I can't figure out why ghc -c Foo.hs -package this -package that is any different from cabal build with a cabal file that has build-depends: this, that. Except that the former is a lot more convenient for quick compilations. My impression these past few years is that it's desirable to have one or more simple global states, accessible from e.g. ghci, and it's also desirable to avoid package conflicts, and there's tension between these needs. The above sentiment, though, seems to imply the former simply isn't desirable to cabal devs in the first place. Oleg mentioned https://github.com/haskell/cabal/issues/6481, and Julian drew my attention to https://github.com/haskell/cabal/issues/10098. I am not qualified to have a well-informed opinion about any of this. I'm just reporting one user's surprising (and disappointing) experience. I'm an outlier in some ways, yes, but here all I wanted to do (as a random GHC user) was compile Foo.hs with a package available, and was surprised that apparently this is no longer possible without creating an auxiliary cabal file and invoking ghc via cabal. I wonder if there is an articulate writeup of Cabal's mental model. e.g. What is an environment file? Why can't my GHC find Prelude? If you invoke plain ghc, what packages does it "see"? Etc. The user manual is good once you have the basic framework in your head, but it's that "big picture" that I'm missing. Thanks Simon On Tue, 9 Jul 2024 at 20:06, amindfv--- via ghc-devs wrote: > On Tue, Jul 09, 2024 at 08:50:18PM +0300, Oleg Grenrus wrote: > > cabal-install has various means to address that too. You can have > `packages: > > /anywhere/onyourdisk` or `packages: http://somewhere.else/tar.gz in your > > cabal.project. Or you can have `source-repository-packages` for accessing > > remote git repositories. Or you can setup > https://cabal.readthedocs.io/en/stable/config.html#local-no-index-repositories > > for a bit more permanent and reusable solution. > > > > cabal-install has ways to approach those problems *without having a > > stateful/implicit setup*. And the workflow is the same as usual workflow > > with packages and projects. In fact, real work projects do use > > source-repository-packages (and sometimes in project vendored `.tar.gz` > > source distributions). > > Yep, I'm aware (and in fact have used `source-repository-package` on > business-critical work projects. I appreciate its existence!). > > However, I'm specifically trying to get to the bottom of the sentiment > that "the GHC developers (e.g. you, richard, sebastian) are virtually the > only group of people who want to invoke GHC manually." > > My impression these past few years is that it's desirable to have one or > more simple global states, accessible from e.g. ghci, and it's also > desirable to avoid package conflicts, and there's tension between these > needs. The above sentiment, though, seems to imply the former simply isn't > desirable to cabal devs in the first place. > > > An anecdote from my side: I wrote`cabal-env` [1] for myself somewhat five > > years ago. I don't think I have used in the last two years. The package > > workflow is often better. > > I've used it! My current work is another attempt to tackle a similar > use-case. I disagree (strongly) that for my needs the package workflow is > better. > > Cheers, > Tom > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > -------------- next part -------------- An HTML attachment was scrubbed... URL: From hecate at glitchbra.in Wed Jul 10 09:01:27 2024 From: hecate at glitchbra.in (=?UTF-8?Q?H=C3=A9cate?=) Date: Wed, 10 Jul 2024 11:01:27 +0200 Subject: The reason for non-GHC boot dependencies Message-ID: <7e7045b2-3714-460d-800f-1a6a4ea56af3@glitchbra.in> Hi devs, I had a chat earlier today with someone and found myself unable to explain the reason why GHC came with boot dependencies like xhtml, that are dependencies of Haddock and HPC. Obviously, the binaries are (haskell-)dynamically linked when distributed, but what is the reason why haddock, hpc, etc can't be (haskell-)statically linked when distributed? Surely the size of binaries can't be the only concern, otherwise we'd use upx¹ on them when distributing them. Cheers, Hécate 1: https://upx.github.io -- Hécate ✨ 🐦: @TechnoEmpress IRC: Hecate WWW: https://glitchbra.in RUN: BSD From oleg.grenrus at iki.fi Wed Jul 10 09:53:34 2024 From: oleg.grenrus at iki.fi (Oleg Grenrus) Date: Wed, 10 Jul 2024 12:53:34 +0300 Subject: Cabal woe In-Reply-To: References: Message-ID: On 10.7.2024 11.27, Simon Peyton Jones wrote: > I wonder if there is an articulate writeup of Cabal's mental model. Sadly, i'm not aware of any. I'm also afraid, that Duncan's (who made initial v2-setup), Herbert's (who pushed v2 over the line) and mine models differ at least slightly. The https://well-typed.com/blog/2015/01/how-we-might-abolish-cabal-hell-part-2/ is Duncan's view from 2015. A side comment, we really should rename either the Cabal-the-library or cabal-install. Because I'm sure you are asking about mental model behind cabal-install v2-build, as "nothing" has changed in Cabal-the-library for a decade. The idea is that cabal-install v2-build installs everything into single global store, and gives consistent views to that store. In practice the global store is unusable directly, as a whole it may not be consistent nor unambiguous (see e.g. https://gitlab.haskell.org/ghc/ghc/-/issues/24823 for funny side-effects of that, you may have dozens of installed units in the package db with the same package name and version). > e.g.  What is an environment file? Essentially it's a set of `-package-db` and `-package-id` flags. In other words, environment files are (specialized) response-files which ghc (tool, or lib) may pick up automatically. They are a way to encode a view into cabal's store (but technically there is no reason they cannot be used to give a view into stack's snapshot package databases too) > Why can't my GHC find Prelude? Probably because there is -hide-all-packages, -clear-package-db (or both); without -package-db and -package-id for base (a package with module Prelude). > If you invoke plain ghc, what packages does it "see"? It depends on the state you have. This is complicated (use cabal-install). If you have a default or local environment file, ghc will use those. If there aren't it will use a global package db. To be honest, I'm not so sure what GHC does by default. I'm mostly familiar with programmatic execution, which often starts with `-clear-package-db`, `-hide-all-packages` etc to clear all the implicit stuff GHC may do. - Oleg From hecate at glitchbra.in Wed Jul 10 09:57:07 2024 From: hecate at glitchbra.in (=?UTF-8?Q?H=C3=A9cate?=) Date: Wed, 10 Jul 2024 11:57:07 +0200 Subject: Cabal woe In-Reply-To: References: Message-ID: <49aacafd-4a29-410a-826b-931e4efa4235@glitchbra.in> Reading this certainly motivates me to push for a better design of the respective boundaries of GHC and cabal. Removing magic from GHC that it uses to compensate for the absence of cabal or other build system would certainly help using it for simpler use-cases. Le 10/07/2024 à 11:53, Oleg Grenrus a écrit : > > On 10.7.2024 11.27, Simon Peyton Jones wrote: >> I wonder if there is an articulate writeup of Cabal's mental model. > > Sadly, i'm not aware of any. I'm also afraid, that Duncan's (who made > initial v2-setup), Herbert's (who pushed v2 over the line) and mine > models differ at least slightly. > > The > https://well-typed.com/blog/2015/01/how-we-might-abolish-cabal-hell-part-2/ > is Duncan's view from 2015. > > A side comment, we really should rename either the Cabal-the-library > or cabal-install. Because I'm sure you are asking about mental model > behind cabal-install v2-build, as "nothing" has changed in > Cabal-the-library for a decade. > > The idea is that cabal-install v2-build installs everything into > single global store, and gives consistent views to that store. In > practice the global store is unusable directly, as a whole it may not > be consistent nor unambiguous (see e.g. > https://gitlab.haskell.org/ghc/ghc/-/issues/24823 for funny > side-effects of that, you may have dozens of installed units in the > package db with the same package name and version). > >> e.g.  What is an environment file? > > Essentially it's a set of `-package-db` and `-package-id` flags. > In other words, environment files are (specialized) response-files > which ghc (tool, or lib) may pick up automatically. > They are a way to encode a view into cabal's store (but technically > there is no reason they cannot be used to give a view into stack's > snapshot package databases too) > >> Why can't my GHC find Prelude? > Probably because there is -hide-all-packages, -clear-package-db (or > both); without -package-db and -package-id for base (a package with > module Prelude). > >> If you invoke plain ghc, what packages does it "see"? > > It depends on the state you have. This is complicated (use > cabal-install). If you have a default or local environment file, ghc > will use those. If there aren't it will use a global package db. > > To be honest, I'm not so sure what GHC does by default. I'm mostly > familiar with programmatic execution, which often starts with > `-clear-package-db`, `-hide-all-packages` etc to clear all the > implicit stuff GHC may do. > > - Oleg > > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs -- Hécate ✨ 🐦: @TechnoEmpress IRC: Hecate WWW: https://glitchbra.in RUN: BSD From x at tomsmeding.com Wed Jul 10 10:52:10 2024 From: x at tomsmeding.com (Tom Smeding) Date: Wed, 10 Jul 2024 12:52:10 +0200 Subject: Cabal woe In-Reply-To: <49aacafd-4a29-410a-826b-931e4efa4235@glitchbra.in> References: <49aacafd-4a29-410a-826b-931e4efa4235@glitchbra.in> Message-ID: So far, the discussion around environment files in this thread has always been entangled with the idea of a "global state". GHC environment files (as written by `cabal install --lib`) are only global if they are placed in a directory where GHC will always look regardless of where GHC is currently running. By default, `cabal install --lib PKG` will place the file in such a location (~/.ghc/x86_64-linux-9.4.7/environments/default in my particular setup), but one can use the `--package-env DIR` flag to change the directory to use. GHC will look in the current directory and its parent directories for environment files, hence the following works: (choosing `some` as an arbitrary light-weight non-boot package; the `cabal install` command prints little because I already have a compiled copy in my global store) ~$ mkdir experiments ~$ cd experiments ~/experiments$ cabal install --package-env . --lib some Resolving dependencies... ~/experiments$ ghci Loaded package environment from /home/tom/experiments/.ghc.environment.x86_64-linux-9.4.7 GHCi, version 9.4.7: https://www.haskell.org/ghc/ :? for help Loaded GHCi configuration from /home/tom/git/dotfiles/.ghci Prelude> import Data.Some Prelude Data.Some> :q Leaving GHCi. ~/experiments$ cd .. ~$ rm -r experiments Of course, normal GHC (i.e. not GHCi) will also work and pick up the environment file. Using these "local" environment files allows one to create a local view of the global store that includes only a number of (presumably consistent) packages. Add to it by running `cabal install --package-env . --lib PKG` multiple times for different PKGs. No global state is modified, except potentially for compilation products of the libraries being installed -- but that is _append-only_ global state (the global cabal store), which is much less dangerous than general mutable global state. Furthermore, as already noted indirectly by Oleg, GHC environment files are human-readable, if not very easily human-writable. - Tom On 10/07/2024 11:57, Hécate via ghc-devs wrote: > Reading this certainly motivates me to push for a better design of the > respective boundaries of GHC and cabal. Removing magic from GHC that > it uses to compensate for the absence of cabal or other build system > would certainly help using it for simpler use-cases. > > Le 10/07/2024 à 11:53, Oleg Grenrus a écrit : >> >> On 10.7.2024 11.27, Simon Peyton Jones wrote: >>> I wonder if there is an articulate writeup of Cabal's mental model. >> >> Sadly, i'm not aware of any. I'm also afraid, that Duncan's (who made >> initial v2-setup), Herbert's (who pushed v2 over the line) and mine >> models differ at least slightly. >> >> The >> https://well-typed.com/blog/2015/01/how-we-might-abolish-cabal-hell-part-2/ >> is Duncan's view from 2015. >> >> A side comment, we really should rename either the Cabal-the-library >> or cabal-install. Because I'm sure you are asking about mental model >> behind cabal-install v2-build, as "nothing" has changed in >> Cabal-the-library for a decade. >> >> The idea is that cabal-install v2-build installs everything into >> single global store, and gives consistent views to that store. In >> practice the global store is unusable directly, as a whole it may not >> be consistent nor unambiguous (see e.g. >> https://gitlab.haskell.org/ghc/ghc/-/issues/24823 for funny >> side-effects of that, you may have dozens of installed units in the >> package db with the same package name and version). >> >>> e.g.  What is an environment file? >> >> Essentially it's a set of `-package-db` and `-package-id` flags. >> In other words, environment files are (specialized) response-files >> which ghc (tool, or lib) may pick up automatically. >> They are a way to encode a view into cabal's store (but technically >> there is no reason they cannot be used to give a view into stack's >> snapshot package databases too) >> >>> Why can't my GHC find Prelude? >> Probably because there is -hide-all-packages, -clear-package-db (or >> both); without -package-db and -package-id for base (a package with >> module Prelude). >> >>> If you invoke plain ghc, what packages does it "see"? >> >> It depends on the state you have. This is complicated (use >> cabal-install). If you have a default or local environment file, ghc >> will use those. If there aren't it will use a global package db. >> >> To be honest, I'm not so sure what GHC does by default. I'm mostly >> familiar with programmatic execution, which often starts with >> `-clear-package-db`, `-hide-all-packages` etc to clear all the >> implicit stuff GHC may do. >> >> - Oleg >> >> _______________________________________________ >> ghc-devs mailing list >> ghc-devs at haskell.org >> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > From a.pelenitsyn at gmail.com Wed Jul 10 14:05:24 2024 From: a.pelenitsyn at gmail.com (Artem Pelenitsyn) Date: Wed, 10 Jul 2024 10:05:24 -0400 Subject: Cabal woe In-Reply-To: References: Message-ID: Hi Simon, The direct answer to your particular problem is that cabal-install-3.10.1 (which you're using) does not add `base` in the package environment created by `install --lib` by default. So, to fix the problem you'd need to say `cabal install --lib base hashtables primitive` instead. Better yet, you could upgrade your cabal-install to version 3.10.3 (recommended by GHCup), where this behavior was reversed (base is added by default). To add to how brittle `install --lib` is: the 3.10.1 behavior (no base by default) bothered people [1] and was changed in 3.10.3 [2]. But the 3.10.3 behavior (add base by default) also bothers people [3]! And note that although you have an easy way to solve your issue (add base in the list of libraries in the call to install --lib), the users of 3.10.3 who don't want base added by default don't have such an easy way. (By easy I mean using only cabal CLI and don't edit env files manually, for example.) More general, I think, it's possible to develop a mental model for today's `install --lib` (either 3.10.1 or 3.10.3, doesn't matter in the grand scheme), and use it effectively for "little experiments" as advocated by Richard in the above-mentioned issue [4] (note that the issue was solved since!). The prerequisite for that is to understand GHC environment files and that `install --lib` is mostly a (n incomplete) CLI interface to them. Yet, people argue for a more holistic "environment" experience, see, e.g., the above mentioned [5] (*the* cabal-env issue) or a very recent push from the old cabal sandboxes perspective: https://github.com/haskell/cabal/issues/10098 or this thread :-) Also, (much in the spirit of *Carthago delenda est*) I try to conclude my every reply about install --lib with what's already mentioned by Tom: *always* use `cabal install --lib` with `--package-env=.` It will save you a bunch of pain related to the "global state" (not all of it, perhaps). -- Best, Artem [1]: https://github.com/haskell/cabal/issues/8894 [2]: https://github.com/haskell/cabal/pull/8903 [3]: https://github.com/haskell/cabal/issues/9672 [4]: https://github.com/haskell/cabal/issues/7832 [5]: https://github.com/haskell/cabal/issues/6481 On Tue, Jul 9, 2024 at 4:58 AM Simon Peyton Jones < simon.peytonjones at gmail.com> wrote: > Friends > > I'm trying to repro #25064 with my development build of GHC. The test > case depends on packages `primitive` and `hashtables`. So I try this (see > below). > > Alas, apparently after the `cabal install`, it can't find Prelude. > > What should I do? I tried removing the "environments" file, whatever that > is, which then meant it could find Prelude -- but the libraries were no > longer installed. > > I lack a decent model of what is going on with installing packages for my > development builds. Is there a write up anywhere? > > Thanks > > Simon > > > bash$ cabal install --lib hashtables primitive --with-compiler > $HOME/code/HEAD/$s2 > Warning: Unknown/unsupported 'ghc' version detected (Cabal 3.10.1.0 > supports > 'ghc' version < 9.8): /home/simonpj/code/HEAD/_build/stage1/bin/ghc is > version > 9.11.20240517 > Warning: Unknown/unsupported 'ghc' version detected (Cabal 3.10.1.0 > supports > 'ghc' version < 9.8): /home/simonpj/code/HEAD/_build/stage1/bin/ghc is > version > 9.11.20240517 > Resolving dependencies... > bash$ ~/code/HEAD/$s2 -c T25064.hs > Loaded package environment from > T25064.hs:1:8: error: [ ]8;;https://errors.haskell.org/messages/GHC-87110 > \GHC-87110 ]8;; \] > Could not load module ‘Prelude’. > It is a member of the hidden package ‘base-4.20.0.0’. > Use -v to see a list of the files searched for. > | > 1 | module Bug where > | ^^^ > > bash$ rm /home/simonpj/.ghc/x86_64-linux-9.11.20240517/environments/default > bash$ ~/code/HEAD/$s2 -c T25064.hs > T25064.hs:3:1: error: [ ]8;;https://errors.haskell.org/messages/GHC-61948 > \GHC-61948 ]8;; \] > Could not find module ‘Control.Monad.Primitive’. > Perhaps you meant Control.Monad.Writer (from mtl-2.3.1) > Use -v to see a list of the files searched for. > | > 3 | import Control.Monad.Primitive > | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > > T25064.hs:4:1: error: [ ]8;;https://errors.haskell.org/messages/GHC-87110 > \GHC-87110 ]8;; \] > Could not find module ‘Data.HashTable.ST.Basic’. > Use -v to see a list of the files searched for. > | > 4 | import qualified Data.HashTable.ST.Basic as H > | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > > > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > -------------- next part -------------- An HTML attachment was scrubbed... URL: From klebinger.andreas at gmx.at Wed Jul 10 16:10:21 2024 From: klebinger.andreas at gmx.at (Andreas Klebinger) Date: Wed, 10 Jul 2024 18:10:21 +0200 Subject: The reason for non-GHC boot dependencies In-Reply-To: <7e7045b2-3714-460d-800f-1a6a4ea56af3@glitchbra.in> References: <7e7045b2-3714-460d-800f-1a6a4ea56af3@glitchbra.in> Message-ID: I think they could be statically linked. But those boot libraries don't change much and generally don't really cause us nor users pain so it seems like there is little reason to do so to me. > Surely the size of binaries can't be the only concern, otherwise we'd use upx¹ on them when distributing them. I believe Ben experimented with executable compression tools in the past with little success. But there were segfaults, executables being flagged by antivirus and perhaps more issues I forgot which just made using it unrealistic at the time. But perhaps the tooling has matured in the meantime. Since our distributions are already compressed purely for *distribution* purposes I would expect the gains there to be rather slim anyway. So it's not really that we don't care about size, just that these tools seemed not reliable enough for the benefits they offer in the past. Am 10/07/2024 um 11:01 schrieb Hécate via ghc-devs: > Hi devs, > > I had a chat earlier today with someone and found myself unable to > explain the reason why GHC came with boot dependencies like xhtml, > that are dependencies of Haddock and HPC. > > Obviously, the binaries are (haskell-)dynamically linked when > distributed, but what is the reason why haddock, hpc, etc can't be > (haskell-)statically linked when distributed? > > Surely the size of binaries can't be the only concern, otherwise we'd > use upx¹ on them when distributing them. > > Cheers, > > Hécate > > > 1: https://upx.github.io > From a.pelenitsyn at gmail.com Wed Jul 10 16:22:24 2024 From: a.pelenitsyn at gmail.com (Artem Pelenitsyn) Date: Wed, 10 Jul 2024 12:22:24 -0400 Subject: The reason for non-GHC boot dependencies In-Reply-To: References: <7e7045b2-3714-460d-800f-1a6a4ea56af3@glitchbra.in> Message-ID: > I think they could be statically linked. But those boot libraries don't > change much and generally > don't really cause us nor users pain so it seems like there is little > reason to do so to me. There once was a sizeable patch to Haddock to switch from xhtml to Lucid, and it was rejected, seemingly, solely on the grounds that Lucid can't be added to boot libraries [1]. Since then Lucid dropped the heavier part of its dependency tree, so it's maybe not an issue anymore, but my point is that there's more to this story than what you mentioned: the need for keeping Haddock and HPC's dependencies in the set of boot libraries may slow down development of those tools. [1]: https://github.com/haskell/haddock/pull/1598#issuecomment-1621765685 On Wed, Jul 10, 2024 at 12:10 PM Andreas Klebinger via ghc-devs < ghc-devs at haskell.org> wrote: > I think they could be statically linked. But those boot libraries don't > change much and generally > don't really cause us nor users pain so it seems like there is little > reason to do so to me. > > > Surely the size of binaries can't be the only concern, otherwise we'd > use upx¹ on them when distributing them. > > I believe Ben experimented with executable compression tools in the past > with little success. > But there were segfaults, executables being flagged by antivirus and > perhaps more issues I forgot > which just made using it unrealistic at the time. > > But perhaps the tooling has matured in the meantime. > > Since our distributions are already compressed purely for *distribution* > purposes I would expect the gains there to be rather slim anyway. > > So it's not really that we don't care about size, just that these tools > seemed not reliable enough for the benefits they offer in the past. > > Am 10/07/2024 um 11:01 schrieb Hécate via ghc-devs: > > Hi devs, > > > > I had a chat earlier today with someone and found myself unable to > > explain the reason why GHC came with boot dependencies like xhtml, > > that are dependencies of Haddock and HPC. > > > > Obviously, the binaries are (haskell-)dynamically linked when > > distributed, but what is the reason why haddock, hpc, etc can't be > > (haskell-)statically linked when distributed? > > > > Surely the size of binaries can't be the only concern, otherwise we'd > > use upx¹ on them when distributing them. > > > > Cheers, > > > > Hécate > > > > > > 1: https://upx.github.io > > > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > -------------- next part -------------- An HTML attachment was scrubbed... URL: From hecate at glitchbra.in Wed Jul 10 16:48:33 2024 From: hecate at glitchbra.in (=?UTF-8?Q?H=C3=A9cate?=) Date: Wed, 10 Jul 2024 18:48:33 +0200 Subject: The reason for non-GHC boot dependencies In-Reply-To: References: <7e7045b2-3714-460d-800f-1a6a4ea56af3@glitchbra.in> Message-ID: <369e9953-577d-4520-a30a-22eb5027b3af@glitchbra.in> For context, the move from xhtml to lucid2 is very much in progress, for both haddock and hpc. The necessity to avoid too many third-party libraries is that in its current (and very custom) setup, dependencies are git submodules in the GHC tree. Which somewhat make sense because these dependencies have sometimes to be adjusted when they use unstable internal APIs. Le 10/07/2024 à 18:22, Artem Pelenitsyn a écrit : > > I think they could be statically linked. But those boot libraries don't > > change much and generally > > don't really cause us nor users pain so it seems like there is little > > reason to do so to me. > > There once was a sizeable patch to Haddock to switch from xhtml to Lucid, > and it was rejected, seemingly, solely on the grounds that Lucid can't > be added to boot libraries [1]. > Since then Lucid dropped the heavier part of its dependency tree, so > it's maybe not an issue anymore, > but my point is that there's more to this story than what you > mentioned: the need for keeping Haddock and HPC's > dependencies in the set of boot libraries may slow down development of > those tools. > > [1]: https://github.com/haskell/haddock/pull/1598#issuecomment-1621765685 > > On Wed, Jul 10, 2024 at 12:10 PM Andreas Klebinger via ghc-devs > wrote: > > I think they could be statically linked. But those boot libraries > don't > change much and generally > don't really cause us nor users pain so it seems like there is little > reason to do so to me. > >  > Surely the size of binaries can't be the only concern, > otherwise we'd > use upx¹ on them when distributing them. > > I believe Ben experimented with executable compression tools in > the past > with little success. > But there were segfaults, executables being flagged by antivirus and > perhaps more issues I forgot > which just made using it unrealistic at the time. > > But perhaps the tooling has matured in the meantime. > > Since our distributions are already compressed purely for > *distribution* > purposes I would expect the gains there to be rather slim anyway. > > So it's not really that we don't care about size, just that these > tools > seemed not reliable enough for the benefits they offer in the past. > > Am 10/07/2024 um 11:01 schrieb Hécate via ghc-devs: > > Hi devs, > > > > I had a chat earlier today with someone and found myself unable to > > explain the reason why GHC came with boot dependencies like xhtml, > > that are dependencies of Haddock and HPC. > > > > Obviously, the binaries are (haskell-)dynamically linked when > > distributed, but what is the reason why haddock, hpc, etc can't be > > (haskell-)statically linked when distributed? > > > > Surely the size of binaries can't be the only concern, otherwise > we'd > > use upx¹ on them when distributing them. > > > > Cheers, > > > > Hécate > > > > > > 1: https://upx.github.io > > > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > > > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs -- Hécate ✨ 🐦: @TechnoEmpress IRC: Hecate WWW:https://glitchbra.in RUN: BSD -------------- next part -------------- An HTML attachment was scrubbed... URL: From hecate at glitchbra.in Wed Jul 10 16:55:09 2024 From: hecate at glitchbra.in (=?UTF-8?Q?H=C3=A9cate?=) Date: Wed, 10 Jul 2024 18:55:09 +0200 Subject: The reason for non-GHC boot dependencies In-Reply-To: References: <7e7045b2-3714-460d-800f-1a6a4ea56af3@glitchbra.in> Message-ID: <85f13b4a-e7a4-4865-bc69-6da7d1526034@glitchbra.in> Thanks a lot for the clarification Andreas. So, the present situation is that we seem to have an implicit set of packages that are readily available to users of ghc and ghci, but which serve no direct goal for the end-user, and whose presence is fairly arbitrary. In light of the recent discussions about the mental model of dependencies, cabal and GHC, I was wondering if anything was keeping us expose these packages the user does not explicitly download. I will take some time to see how to switch to (haskell-)static linking for distribution releases. Have a nice day, Hécate Le 10/07/2024 à 18:10, Andreas Klebinger via ghc-devs a écrit : > I think they could be statically linked. But those boot libraries don't > change much and generally > don't really cause us nor users pain so it seems like there is little > reason to do so to me. > > > Surely the size of binaries can't be the only concern, otherwise we'd > use upx¹ on them when distributing them. > > I believe Ben experimented with executable compression tools in the past > with little success. > But there were segfaults, executables being flagged by antivirus and > perhaps more issues I forgot > which just made using it unrealistic at the time. > > But perhaps the tooling has matured in the meantime. > > Since our distributions are already compressed purely for *distribution* > purposes I would expect the gains there to be rather slim anyway. > > So it's not really that we don't care about size, just that these tools > seemed not reliable enough for the benefits they offer in the past. > > Am 10/07/2024 um 11:01 schrieb Hécate via ghc-devs: >> Hi devs, >> >> I had a chat earlier today with someone and found myself unable to >> explain the reason why GHC came with boot dependencies like xhtml, >> that are dependencies of Haddock and HPC. >> >> Obviously, the binaries are (haskell-)dynamically linked when >> distributed, but what is the reason why haddock, hpc, etc can't be >> (haskell-)statically linked when distributed? >> >> Surely the size of binaries can't be the only concern, otherwise we'd >> use upx¹ on them when distributing them. >> >> Cheers, >> >> Hécate >> >> >> 1: https://upx.github.io >> > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs -- Hécate ✨ 🐦: @TechnoEmpress IRC: Hecate WWW: https://glitchbra.in RUN: BSD From amindfv at mailbox.org Thu Jul 11 02:59:18 2024 From: amindfv at mailbox.org (amindfv at mailbox.org) Date: Wed, 10 Jul 2024 22:59:18 -0400 Subject: Cabal woe In-Reply-To: References: <49aacafd-4a29-410a-826b-931e4efa4235@glitchbra.in> Message-ID: On Wed, Jul 10, 2024 at 12:52:10PM +0200, Tom Smeding wrote: > [...] > Furthermore, as already noted indirectly by Oleg, GHC environment files are > human-readable, if not very easily human-writable. They're actually pretty human-writable too. The script I've written to regain simple ghci workflows more or less just copies `package-id` lines from one environment file to another. I have to make sure I don't end up with different versions of transitive dependencies, but it actually works beautifully for keeping a default set of packages that's much larger than just `base`. Tom From gergo at erdi.hu Thu Jul 11 04:10:28 2024 From: gergo at erdi.hu (=?ISO-8859-2?Q?=C9RDI_Gerg=F5?=) Date: Thu, 11 Jul 2024 12:10:28 +0800 (+08) Subject: Cannot build `master`, tries to use GHC 9.2?! In-Reply-To: References: <9ac3ac55-343f-e7cb-4d8e-dd34127484ad@erdi.hu> Message-ID: <99dd6d2e-75e7-b759-6720-51f668944650@erdi.hu> OK so it turns out there were two issues at play here: * I had `_build` as a symlink, and that's no bueno since Hadrian passes some relative paths like `_build/../libraries` to some command lines. This isn't ideal but is not a dealbreaker. * It turns out Hadrian recently was broken with regard to Hadrian-building vs. GHC-bootstrapping Haskell compilers. I've filed this as https://gitlab.haskell.org/ghc/ghc/-/issues/25076. On Wed, 10 Jul 2024, ÉRDI Gergő wrote: > By playing around with GHC versions, Stack resolvers, and trying > `hadrian/build` instead of `hadrian/build-stack`, I now got somewhat further: > > Error: hadrian: Missing dependency on a foreign library: > * Missing (or bad) header file: HsBase.h > > But `HsBase.h` should come from the GHC source tree itself, right? So it's > not an unresolved external dependency. > > It's very much possible that this is all because of some local configuration > problem on my machine, but I'm still going to need some hints on where to > even start looking. > > Thanks, > Gerg > > On Mon, 8 Jul 2024, ÉRDI Gergő wrote: > >> Hi, >> >> I'm trying to build GHC 5b1aefb70edbd54ac899896df39d8f3d6c579518 but I'm >> running into a weird problem. Using `hadrian/build-stack`, it fails quite >> early on with: >> >> | Run Ghc CompileCWithGhc (Stage0 GlobalLibs): utils/hsc2hs/cbits/utils.c >> => _build/stageBoot/utils/hsc2hs/build/c/cbits/utils.o >> Command line: ghc -Wall -hisuf hi -osuf o -hcsuf hc -static >> -hide-all-packages -no-user-package-db '-package-env -' '-package-db >> _build/stageBoot/inplace/package.conf.d' '-this-unit-id >> hsc2hs-0.68.10-inplace' '-this-package-name hsc2hs' '-package-id >> base-4.16.4.0' '-package-id containers-0.6.5.1' '-package-id >> directory-1.3.6.2' '-package-id filepath-1.4.2.2' '-package-id >> process-1.6.16.0' -i >> -i/home/cactus/prog/haskell/ghc/ghc.master/_build/stageBoot/utils/hsc2hs/build >> -i/home/cactus/prog/haskell/ghc/ghc.master/_build/stageBoot/utils/hsc2hs/build/hsc2hs/autogen >> -i/home/cactus/prog/haskell/ghc/ghc.master/utils/hsc2hs/src/ >> -I_build/stageBoot/utils/hsc2hs/build >> -I/home/cactus/.ghcup/ghc/9.2.8/lib/ghc-9.2.8/process-1.6.16.0/include >> -I/home/cactus/.ghcup/ghc/9.2.8/lib/ghc-9.2.8/unix-2.7.2.2/include >> -I/home/cactus/.ghcup/ghc/9.2.8/lib/ghc-9.2.8/time-1.11.1.1/include >> -I/home/cactus/.ghcup/ghc/9.2.8/lib/ghc-9.2.8/bytestring-0.11.4.0/include >> -I/home/cactus/.ghcup/ghc/9.2.8/lib/ghc-9.2.8/base-4.16.4.0/include >> -I/home/cactus/.ghcup/ghc/9.2.8/lib/ghc-9.2.8/ghc-bignum-1.2/include >> -I/home/cactus/.ghcup/ghc/9.2.8/lib/ghc-9.2.8/include -optP-include >> -optP_build/stageBoot/utils/hsc2hs/build/hsc2hs/autogen/cabal_macros.h >> -optP-DIN_GHC_TREE -outputdir _build/stageBoot/utils/hsc2hs/build >> -fdiagnostics-color=always -Wall -XHaskell2010 >> -optc-I_build/stageBoot/utils/hsc2hs/build >> -optc-I/home/cactus/.ghcup/ghc/9.2.8/lib/ghc-9.2.8/process-1.6.16.0/include >> -optc-I/home/cactus/.ghcup/ghc/9.2.8/lib/ghc-9.2.8/unix-2.7.2.2/include >> -optc-I/home/cactus/.ghcup/ghc/9.2.8/lib/ghc-9.2.8/time-1.11.1.1/include >> -optc-I/home/cactus/.ghcup/ghc/9.2.8/lib/ghc-9.2.8/bytestring-0.11.4.0/include >> -optc-I/home/cactus/.ghcup/ghc/9.2.8/lib/ghc-9.2.8/base-4.16.4.0/include >> -optc-I/home/cactus/.ghcup/ghc/9.2.8/lib/ghc-9.2.8/ghc-bignum-1.2/include >> -optc-I/home/cactus/.ghcup/ghc/9.2.8/lib/ghc-9.2.8/include >> -optc-Wno-error=inline -c utils/hsc2hs/cbits/utils.c -o >> _build/stageBoot/utils/hsc2hs/build/c/cbits/utils.o -O -H32m >> ===> Command failed with error code: 1 >> ghc: unrecognised flag: -this-package-name hsc2hs >> >> >> I don't know where all those references to GHC 9.2.8 come from. I've run >> `configure` with GHC 9.10.1 on the $PATH and it correctly reports it as the >> bootstrapping version: >> >> Bootstrapping using : ghc >> which is version : 9.10.1 >> >> >> What am I doing wrong? >> >> Gergo >> > > -- .--= ULLA! =-----------------. \ http://gergo.erdi.hu \ `---= gergo at erdi.hu =-------' From matthewtpickering at gmail.com Thu Jul 11 08:41:12 2024 From: matthewtpickering at gmail.com (Matthew Pickering) Date: Thu, 11 Jul 2024 09:41:12 +0100 Subject: The reason for non-GHC boot dependencies In-Reply-To: <85f13b4a-e7a4-4865-bc69-6da7d1526034@glitchbra.in> References: <7e7045b2-3714-460d-800f-1a6a4ea56af3@glitchbra.in> <85f13b4a-e7a4-4865-bc69-6da7d1526034@glitchbra.in> Message-ID: The `haddock` executable must be dynamically linked as it must compile source files. It is much more robust to use dynamic linking when loading dependencies for TH evaluation than static linking. HLS is dynamically linked for the same reason. As such we have to distribute at least shared libraries for dependencies for haddock etc. It seems therefore that we might as well advertise these libraries are available in the package database. The things that we definitely have to advertise in the package database are `Cabal` and `ghc` and dependencies. The point about which libraries are boot libraries is a separate one. If you are developing a library which is a boot library then you are restricted to a small set of dependencies. Additional dependencies introduce a large amount of friction due to the coordindation needed during a release process. Anyone is free to develop their own libraries which use whichever dependencies they wish in user-space. Cheers, Matt On Wed, Jul 10, 2024 at 5:55 PM Hécate via ghc-devs wrote: > > Thanks a lot for the clarification Andreas. > > So, the present situation is that we seem to have an implicit set of > packages that are readily available to users of ghc and ghci, but which > serve no direct goal for the end-user, and whose presence is fairly > arbitrary. > > In light of the recent discussions about the mental model of > dependencies, cabal and GHC, I was wondering if anything was keeping us > expose these packages the user does not explicitly download. > > I will take some time to see how to switch to (haskell-)static linking > for distribution releases. > > Have a nice day, > > Hécate > > Le 10/07/2024 à 18:10, Andreas Klebinger via ghc-devs a écrit : > > I think they could be statically linked. But those boot libraries don't > > change much and generally > > don't really cause us nor users pain so it seems like there is little > > reason to do so to me. > > > > > Surely the size of binaries can't be the only concern, otherwise we'd > > use upx¹ on them when distributing them. > > > > I believe Ben experimented with executable compression tools in the past > > with little success. > > But there were segfaults, executables being flagged by antivirus and > > perhaps more issues I forgot > > which just made using it unrealistic at the time. > > > > But perhaps the tooling has matured in the meantime. > > > > Since our distributions are already compressed purely for *distribution* > > purposes I would expect the gains there to be rather slim anyway. > > > > So it's not really that we don't care about size, just that these tools > > seemed not reliable enough for the benefits they offer in the past. > > > > Am 10/07/2024 um 11:01 schrieb Hécate via ghc-devs: > >> Hi devs, > >> > >> I had a chat earlier today with someone and found myself unable to > >> explain the reason why GHC came with boot dependencies like xhtml, > >> that are dependencies of Haddock and HPC. > >> > >> Obviously, the binaries are (haskell-)dynamically linked when > >> distributed, but what is the reason why haddock, hpc, etc can't be > >> (haskell-)statically linked when distributed? > >> > >> Surely the size of binaries can't be the only concern, otherwise we'd > >> use upx¹ on them when distributing them. > >> > >> Cheers, > >> > >> Hécate > >> > >> > >> 1: https://upx.github.io > >> > > _______________________________________________ > > ghc-devs mailing list > > ghc-devs at haskell.org > > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > > -- > Hécate ✨ > 🐦: @TechnoEmpress > IRC: Hecate > WWW: https://glitchbra.in > RUN: BSD > > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs From me at michaelpj.com Thu Jul 11 09:28:46 2024 From: me at michaelpj.com (Michael Peyton Jones) Date: Thu, 11 Jul 2024 10:28:46 +0100 Subject: The reason for non-GHC boot dependencies In-Reply-To: References: <7e7045b2-3714-460d-800f-1a6a4ea56af3@glitchbra.in> <85f13b4a-e7a4-4865-bc69-6da7d1526034@glitchbra.in> Message-ID: > The `haddock` executable must be dynamically linked as it must compile source files. Is this still true in the post-hi-haddock world? I thought we removed the code path in cabal that needed that, so recent versions of haddock shouldn't need it either? M On Thu, Jul 11, 2024 at 9:41 AM Matthew Pickering < matthewtpickering at gmail.com> wrote: > The `haddock` executable must be dynamically linked as it must compile > source files. It is much more robust to use dynamic linking when > loading dependencies for TH evaluation than static linking. HLS is > dynamically linked for the same reason. > > As such we have to distribute at least shared libraries for > dependencies for haddock etc. It seems therefore that we might as well > advertise these libraries are available in the package database. The > things that we definitely have to advertise in the package database > are `Cabal` and `ghc` and dependencies. > > The point about which libraries are boot libraries is a separate one. > If you are developing a library which is a boot library then you are > restricted to a small set of dependencies. Additional dependencies > introduce a large amount of friction due to the coordindation needed > during a release process. Anyone is free to develop their own > libraries which use whichever dependencies they wish in user-space. > > Cheers, > > Matt > > On Wed, Jul 10, 2024 at 5:55 PM Hécate via ghc-devs > wrote: > > > > Thanks a lot for the clarification Andreas. > > > > So, the present situation is that we seem to have an implicit set of > > packages that are readily available to users of ghc and ghci, but which > > serve no direct goal for the end-user, and whose presence is fairly > > arbitrary. > > > > In light of the recent discussions about the mental model of > > dependencies, cabal and GHC, I was wondering if anything was keeping us > > expose these packages the user does not explicitly download. > > > > I will take some time to see how to switch to (haskell-)static linking > > for distribution releases. > > > > Have a nice day, > > > > Hécate > > > > Le 10/07/2024 à 18:10, Andreas Klebinger via ghc-devs a écrit : > > > I think they could be statically linked. But those boot libraries don't > > > change much and generally > > > don't really cause us nor users pain so it seems like there is little > > > reason to do so to me. > > > > > > > Surely the size of binaries can't be the only concern, otherwise we'd > > > use upx¹ on them when distributing them. > > > > > > I believe Ben experimented with executable compression tools in the > past > > > with little success. > > > But there were segfaults, executables being flagged by antivirus and > > > perhaps more issues I forgot > > > which just made using it unrealistic at the time. > > > > > > But perhaps the tooling has matured in the meantime. > > > > > > Since our distributions are already compressed purely for > *distribution* > > > purposes I would expect the gains there to be rather slim anyway. > > > > > > So it's not really that we don't care about size, just that these tools > > > seemed not reliable enough for the benefits they offer in the past. > > > > > > Am 10/07/2024 um 11:01 schrieb Hécate via ghc-devs: > > >> Hi devs, > > >> > > >> I had a chat earlier today with someone and found myself unable to > > >> explain the reason why GHC came with boot dependencies like xhtml, > > >> that are dependencies of Haddock and HPC. > > >> > > >> Obviously, the binaries are (haskell-)dynamically linked when > > >> distributed, but what is the reason why haddock, hpc, etc can't be > > >> (haskell-)statically linked when distributed? > > >> > > >> Surely the size of binaries can't be the only concern, otherwise we'd > > >> use upx¹ on them when distributing them. > > >> > > >> Cheers, > > >> > > >> Hécate > > >> > > >> > > >> 1: https://upx.github.io > > >> > > > _______________________________________________ > > > ghc-devs mailing list > > > ghc-devs at haskell.org > > > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > > > > -- > > Hécate ✨ > > 🐦: @TechnoEmpress > > IRC: Hecate > > WWW: https://glitchbra.in > > RUN: BSD > > > > _______________________________________________ > > ghc-devs mailing list > > ghc-devs at haskell.org > > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > -------------- next part -------------- An HTML attachment was scrubbed... URL: From matthewtpickering at gmail.com Thu Jul 11 09:33:38 2024 From: matthewtpickering at gmail.com (Matthew Pickering) Date: Thu, 11 Jul 2024 10:33:38 +0100 Subject: The reason for non-GHC boot dependencies In-Reply-To: References: <7e7045b2-3714-460d-800f-1a6a4ea56af3@glitchbra.in> <85f13b4a-e7a4-4865-bc69-6da7d1526034@glitchbra.in> Message-ID: Haddock is not migrated yet to only require interface files. Sjoerd Visscher has been working on this recently. On Thu, Jul 11, 2024 at 10:28 AM Michael Peyton Jones wrote: > > > The `haddock` executable must be dynamically linked as it must compile > source files. > > Is this still true in the post-hi-haddock world? I thought we removed the code path in cabal that needed that, so recent versions of haddock shouldn't need it either? > > M > > On Thu, Jul 11, 2024 at 9:41 AM Matthew Pickering wrote: >> >> The `haddock` executable must be dynamically linked as it must compile >> source files. It is much more robust to use dynamic linking when >> loading dependencies for TH evaluation than static linking. HLS is >> dynamically linked for the same reason. >> >> As such we have to distribute at least shared libraries for >> dependencies for haddock etc. It seems therefore that we might as well >> advertise these libraries are available in the package database. The >> things that we definitely have to advertise in the package database >> are `Cabal` and `ghc` and dependencies. >> >> The point about which libraries are boot libraries is a separate one. >> If you are developing a library which is a boot library then you are >> restricted to a small set of dependencies. Additional dependencies >> introduce a large amount of friction due to the coordindation needed >> during a release process. Anyone is free to develop their own >> libraries which use whichever dependencies they wish in user-space. >> >> Cheers, >> >> Matt >> >> On Wed, Jul 10, 2024 at 5:55 PM Hécate via ghc-devs >> wrote: >> > >> > Thanks a lot for the clarification Andreas. >> > >> > So, the present situation is that we seem to have an implicit set of >> > packages that are readily available to users of ghc and ghci, but which >> > serve no direct goal for the end-user, and whose presence is fairly >> > arbitrary. >> > >> > In light of the recent discussions about the mental model of >> > dependencies, cabal and GHC, I was wondering if anything was keeping us >> > expose these packages the user does not explicitly download. >> > >> > I will take some time to see how to switch to (haskell-)static linking >> > for distribution releases. >> > >> > Have a nice day, >> > >> > Hécate >> > >> > Le 10/07/2024 à 18:10, Andreas Klebinger via ghc-devs a écrit : >> > > I think they could be statically linked. But those boot libraries don't >> > > change much and generally >> > > don't really cause us nor users pain so it seems like there is little >> > > reason to do so to me. >> > > >> > > > Surely the size of binaries can't be the only concern, otherwise we'd >> > > use upx¹ on them when distributing them. >> > > >> > > I believe Ben experimented with executable compression tools in the past >> > > with little success. >> > > But there were segfaults, executables being flagged by antivirus and >> > > perhaps more issues I forgot >> > > which just made using it unrealistic at the time. >> > > >> > > But perhaps the tooling has matured in the meantime. >> > > >> > > Since our distributions are already compressed purely for *distribution* >> > > purposes I would expect the gains there to be rather slim anyway. >> > > >> > > So it's not really that we don't care about size, just that these tools >> > > seemed not reliable enough for the benefits they offer in the past. >> > > >> > > Am 10/07/2024 um 11:01 schrieb Hécate via ghc-devs: >> > >> Hi devs, >> > >> >> > >> I had a chat earlier today with someone and found myself unable to >> > >> explain the reason why GHC came with boot dependencies like xhtml, >> > >> that are dependencies of Haddock and HPC. >> > >> >> > >> Obviously, the binaries are (haskell-)dynamically linked when >> > >> distributed, but what is the reason why haddock, hpc, etc can't be >> > >> (haskell-)statically linked when distributed? >> > >> >> > >> Surely the size of binaries can't be the only concern, otherwise we'd >> > >> use upx¹ on them when distributing them. >> > >> >> > >> Cheers, >> > >> >> > >> Hécate >> > >> >> > >> >> > >> 1: https://upx.github.io >> > >> >> > > _______________________________________________ >> > > ghc-devs mailing list >> > > ghc-devs at haskell.org >> > > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs >> > >> > -- >> > Hécate ✨ >> > 🐦: @TechnoEmpress >> > IRC: Hecate >> > WWW: https://glitchbra.in >> > RUN: BSD >> > >> > _______________________________________________ >> > ghc-devs mailing list >> > ghc-devs at haskell.org >> > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs >> _______________________________________________ >> ghc-devs mailing list >> ghc-devs at haskell.org >> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs From hecate at glitchbra.in Thu Jul 11 13:06:34 2024 From: hecate at glitchbra.in (=?UTF-8?Q?H=C3=A9cate?=) Date: Thu, 11 Jul 2024 15:06:34 +0200 Subject: The reason for non-GHC boot dependencies In-Reply-To: References: <7e7045b2-3714-460d-800f-1a6a4ea56af3@glitchbra.in> <85f13b4a-e7a4-4865-bc69-6da7d1526034@glitchbra.in> Message-ID: <6d1666e6-3dca-421c-96ce-66e5edb571ae@glitchbra.in> Ah, that is a much better explanation for the distribution of shared libraries, thank you very much Matthew! I am not sure however that I follow your reasoning that we might as well expose them, but this is not a hill I am ready to die on right now. :) Have a nice day and thank you for the clarification! Hécate Le 11/07/2024 à 10:41, Matthew Pickering a écrit : > The `haddock` executable must be dynamically linked as it must compile > source files. It is much more robust to use dynamic linking when > loading dependencies for TH evaluation than static linking. HLS is > dynamically linked for the same reason. > > As such we have to distribute at least shared libraries for > dependencies for haddock etc. It seems therefore that we might as well > advertise these libraries are available in the package database. The > things that we definitely have to advertise in the package database > are `Cabal` and `ghc` and dependencies. > > The point about which libraries are boot libraries is a separate one. > If you are developing a library which is a boot library then you are > restricted to a small set of dependencies. Additional dependencies > introduce a large amount of friction due to the coordindation needed > during a release process. Anyone is free to develop their own > libraries which use whichever dependencies they wish in user-space. > > Cheers, > > Matt > > On Wed, Jul 10, 2024 at 5:55 PM Hécate via ghc-devs > wrote: >> Thanks a lot for the clarification Andreas. >> >> So, the present situation is that we seem to have an implicit set of >> packages that are readily available to users of ghc and ghci, but which >> serve no direct goal for the end-user, and whose presence is fairly >> arbitrary. >> >> In light of the recent discussions about the mental model of >> dependencies, cabal and GHC, I was wondering if anything was keeping us >> expose these packages the user does not explicitly download. >> >> I will take some time to see how to switch to (haskell-)static linking >> for distribution releases. >> >> Have a nice day, >> >> Hécate >> >> Le 10/07/2024 à 18:10, Andreas Klebinger via ghc-devs a écrit : >>> I think they could be statically linked. But those boot libraries don't >>> change much and generally >>> don't really cause us nor users pain so it seems like there is little >>> reason to do so to me. >>> >>>> Surely the size of binaries can't be the only concern, otherwise we'd >>> use upx¹ on them when distributing them. >>> >>> I believe Ben experimented with executable compression tools in the past >>> with little success. >>> But there were segfaults, executables being flagged by antivirus and >>> perhaps more issues I forgot >>> which just made using it unrealistic at the time. >>> >>> But perhaps the tooling has matured in the meantime. >>> >>> Since our distributions are already compressed purely for *distribution* >>> purposes I would expect the gains there to be rather slim anyway. >>> >>> So it's not really that we don't care about size, just that these tools >>> seemed not reliable enough for the benefits they offer in the past. >>> >>> Am 10/07/2024 um 11:01 schrieb Hécate via ghc-devs: >>>> Hi devs, >>>> >>>> I had a chat earlier today with someone and found myself unable to >>>> explain the reason why GHC came with boot dependencies like xhtml, >>>> that are dependencies of Haddock and HPC. >>>> >>>> Obviously, the binaries are (haskell-)dynamically linked when >>>> distributed, but what is the reason why haddock, hpc, etc can't be >>>> (haskell-)statically linked when distributed? >>>> >>>> Surely the size of binaries can't be the only concern, otherwise we'd >>>> use upx¹ on them when distributing them. >>>> >>>> Cheers, >>>> >>>> Hécate >>>> >>>> >>>> 1: https://upx.github.io >>>> >>> _______________________________________________ >>> ghc-devs mailing list >>> ghc-devs at haskell.org >>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs >> -- >> Hécate ✨ >> 🐦: @TechnoEmpress >> IRC: Hecate >> WWW: https://glitchbra.in >> RUN: BSD >> >> _______________________________________________ >> ghc-devs mailing list >> ghc-devs at haskell.org >> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs -- Hécate ✨ 🐦: @TechnoEmpress IRC: Hecate WWW: https://glitchbra.in RUN: BSD From simon.peytonjones at gmail.com Fri Jul 12 07:58:25 2024 From: simon.peytonjones at gmail.com (Simon Peyton Jones) Date: Fri, 12 Jul 2024 08:58:25 +0100 Subject: CI stuck? Message-ID: Dear GHC devs Is GHC's CI stuck in some way? My !12928 has been scheduled by Marge over 10 times now, and each time the commit has failed. Ten seems... a lot. Thanks Simon -------------- next part -------------- An HTML attachment was scrubbed... URL: From sylvain at haskus.fr Fri Jul 12 08:54:28 2024 From: sylvain at haskus.fr (Sylvain Henry) Date: Fri, 12 Jul 2024 10:54:28 +0200 Subject: CI stuck? In-Reply-To: References: Message-ID: Hi Simon, There was a MR failing with the JS job [1,2]. I've fixed it 1 hour ago so it should pass now. Sylvain [1] https://gitlab.haskell.org/ghc/ghc/-/merge_requests/13025#note_575683 [2] https://gitlab.haskell.org/ghc/ghc/-/merge_requests/12991#note_575777 On 12/07/2024 09:58, Simon Peyton Jones wrote: > Dear GHC devs > > Is GHC's CI stuck in some way?  My !12928 has been scheduled by Marge > over 10 times now, and each time the commit has failed. Ten seems...  > a lot. > > Thanks > > Simon > > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs -------------- next part -------------- An HTML attachment was scrubbed... URL: From allbery.b at gmail.com Fri Jul 12 17:24:43 2024 From: allbery.b at gmail.com (Brandon Allbery) Date: Fri, 12 Jul 2024 13:24:43 -0400 Subject: submodules update warning? Message-ID: I got several of these while updating my local ghc checkout: ``` Fetching submodule libraries/binary remote: error: MIDX does not contain the BTMP chunk remote: warning: unable to load pack: 'pack-1d2d40f15dad726ad220801f9d5bd5b527bc7493.idx', disabling pack-reuse >From https://gitlab.haskell.org/ghc/packages/binary * [new branch] wip/edsko -> origin/wip/edsko ``` Is this something I need to worry about? -- brandon s allbery kf8nh allbery.b at gmail.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From klebinger.andreas at gmx.at Mon Jul 15 13:45:27 2024 From: klebinger.andreas at gmx.at (Andreas Klebinger) Date: Mon, 15 Jul 2024 15:45:27 +0200 Subject: Gitlab upgrade tomorrow Message-ID: Hi all, Gitlab will be temporarily unavailable tomorrow around 9:00 AM CEST while it's updated. This should not take long, but a short downtime is expected. Cheers, - Andreas From matthewtpickering at gmail.com Tue Jul 16 09:08:29 2024 From: matthewtpickering at gmail.com (Matthew Pickering) Date: Tue, 16 Jul 2024 10:08:29 +0100 Subject: Gitlab upgrade tomorrow In-Reply-To: References: Message-ID: The upgrade to 1.17.1 is completed now, migrations are running in the background. Please post if there are any issues, it's the first upgrade I have done. I will perform the upgrade to 1.17.2 once the corresponding patch lands in upstream. (https://github.com/NixOS/nixpkgs/pull/326259/files) Cheers, Matt On Mon, Jul 15, 2024 at 2:45 PM Andreas Klebinger via ghc-devs wrote: > > Hi all, > > Gitlab will be temporarily unavailable tomorrow around 9:00 AM CEST > while it's updated. > This should not take long, but a short downtime is expected. > > Cheers, > > - Andreas > > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs From simon.peytonjones at gmail.com Tue Jul 16 10:20:26 2024 From: simon.peytonjones at gmail.com (Simon Peyton Jones) Date: Tue, 16 Jul 2024 11:20:26 +0100 Subject: Cabal and GHC: the big picture Message-ID: Friends You may remember a recent thread on ghc-devs about GHC and Cabal . In it I say how I feel I lack the "big picture" of how GHC and Cabal interact, and that my mental model is probably faulty. Tom Ellis took pity on me, and together we wrote this big-picture overview about how GHC and Cabal interact . Would you like to: - Read it as a consumer. - Does it tell you stuff that is useful? - What else would you like to know? - What is un-clear or missing? - Read it as an expert. - Is it accurate? - Are any bits misleading? - Do the links go to appropriate places? - What other links or resource would be helpful. It is not intended as a replacement for the GHC user guide, nor the Cabal user guide; rather it is littered with links to those guides which give much fuller details. Rather, it is intended to put you (well, me for one!) in a position where you can more easily make sense of those documents. We'd love to have your help in improving it. Simon -------------- next part -------------- An HTML attachment was scrubbed... URL: From oleg.grenrus at iki.fi Tue Jul 16 11:35:50 2024 From: oleg.grenrus at iki.fi (Oleg Grenrus) Date: Tue, 16 Jul 2024 14:35:50 +0300 Subject: Cabal and GHC: the big picture In-Reply-To: References: Message-ID: <4476693a-140f-460f-b9f5-60612e457ded@iki.fi> My first comment, which applies across the whole document is Don't write "package (unit)". Write unit. Leave the package to be used solely as "A package is the unit of distribution and versioning.", and use "unit" consistently for compilation units, and/or "component" (or more specifically "library" etc). The naming of flags is a history artifact. The key observation is that "package is the unit of distribution" is nowadays only a Cabal concept. Only PackageImports and "imprecise" flags like "-package" (c.f. "-package-id" which ought to be called "-unit-id") in GHC really know or care about that. Second comment, is that be mindful about `cabal-install` and Cabal difference. The "3 Cabal" section is really "3 cabal-install", and e.g. stack does things differently. > Suppose version 2.3.7 of package P, called P-2.3.7, depends on package Q. Is therefore wrong. You should write "Suppose version 2.3.7 of library P, called "P-2.3.7", depends on library Q". Also libraries can depend on executables: e.g. happy, GHC doesn't care about those dependencies, but Cabal (the library, which does the building) does. > Each unit has a unit-id, looking like *may* look. The unit identifier is a random string invented by a build tool. It's informative, but it really doesn't matter much. > Q: "installed package" means the same as "unit" Not exactly. > Q: "package id" means the same as "unit-id" I think so. And I'd argue to not use "package id" going forward. >  recompiling with no change could change the binary (non-determinism). Does that change the unit-id? It doesn't. Unit-id is invented prior to compilation. Therefore at least *interface determinism* is important. Though, cabal-install v2 *never* re-install units to store database, so determinism is not a hard requirement. > A package database can contain many installed versions of the same package P, or even of a particular version of P, say P-2.4.3, compiled against different dependencies. Even against the same dependencies, even with the same flags, if for some reason the build tool changes the way it computes the unit-id. Also s/package/library/. Re-call, there exist non-main sublibraries. > documentation for -package does not clearly specify how the name of the package is mapped to a unit-id. Important bit to remember about "-package" is that it's a legacy flag, not used by tools anymore. -package-id looks for the unit exactly. -package scans to find a matching one, there may be many (and e.g. in case of the same version, probably non-deterministic choice is made). > This .cabal/store is not a package database. .cabal/store/ **is** an ordinary package database. > Rather, cabal will invoke ghc with a long list of -package-id flags Yes. This is not mutually exclusive. Package database flags tell where, `-package-id` flags tell what units to use. > Can a package contain multiple public libraries? Yes. public/private doesn't matter for GHC though. Cabal enforce the dependency visibility. I.e. private/public is a Cabal concept. (The visibility is written to interface files, but it's there solely for Cabal to figure out what the visibility was. GHC doesn't or at least shouldn't use that info). > Difference between unit-id and ABI hash? As far as I remember, unit-id tries to approximate ABI hash. In fact, there was a request to have GHC output something like ABI-hash given the set of flags. Currently Cabal has an ad-hoc implementation to filter out flags which should not affect the ABI of a package (like `-fprint-explicit-foralls`. Side note: it would been clearer if flag name convention would suggest already whether they affect ABI or not. E.g. `-ddump` flags or generally `-d` flags don't, but `-f` flags do, except e.g. `-fprint...` which is kind of `-ddump` like flag). On 16.7.2024 13.20, Simon Peyton Jones wrote: > Friends > > You may remember a recent thread on ghc-devs about GHC and Cabal > .  > In it I say how I feel I lack the "big picture" of how GHC and Cabal > interact, and that my mental model is probably faulty. > > Tom Ellis took pity on me, and together we wrote this big-picture > overview about how GHC and Cabal interact > .  > Would you like to: > > * Read it as a consumer. > o Does it tell you stuff that is useful? > o What else would you like to know? > o What is un-clear or missing? > * Read it as an expert. > o Is it accurate? > o Are any bits misleading? > o Do the links go to appropriate places? > o What other links or resource would be helpful. > > It is not intended as a replacement for the GHC user guide, nor the > Cabal user guide; rather it is littered with links to those guides > which give much fuller details. Rather, it is intended to put you > (well, me for one!) in a position where you can more easily make sense > of those documents. > > We'd love to have your help in improving it. > > Simon > > > > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs -------------- next part -------------- An HTML attachment was scrubbed... URL: From tom-lists-haskell-cafe-2023 at jaguarpaw.co.uk Tue Jul 16 11:43:34 2024 From: tom-lists-haskell-cafe-2023 at jaguarpaw.co.uk (Tom Ellis) Date: Tue, 16 Jul 2024 12:43:34 +0100 Subject: Cabal and GHC: the big picture In-Reply-To: <4476693a-140f-460f-b9f5-60612e457ded@iki.fi> References: <4476693a-140f-460f-b9f5-60612e457ded@iki.fi> Message-ID: Thanks Oleg. On Tue, Jul 16, 2024 at 02:35:50PM +0300, Oleg Grenrus wrote: > > This .cabal/store is not a package database. > > .cabal/store/ **is** an ordinary package database. The package database is the thing that contains the .conf files, isn't it? So perhaps you mean .cabal/store//package.db is an ordinary package database? If not it would be good if we can establish precisely what is meant by "package database" and then clarify the document. From oleg.grenrus at iki.fi Tue Jul 16 11:51:07 2024 From: oleg.grenrus at iki.fi (Oleg Grenrus) Date: Tue, 16 Jul 2024 14:51:07 +0300 Subject: Cabal and GHC: the big picture In-Reply-To: References: <4476693a-140f-460f-b9f5-60612e457ded@iki.fi> Message-ID: <14afe39c-142d-410f-b199-ddce09b43a0f@iki.fi> package.db directory has only .conf files, but those conf files point to directories with interface and object files. I think every tool puts the interface and object files next to .conf files. ./package.db is the "index", ./aeson-2.2.3.0-3b4881079a148ac154e9112c43bd71f61140bdb9e78660803e826af798de2108 ./aeson-2.2.3.0-9bfa3c3833e85d9e62afa4f60b2820c47e8c3382991299ab624dedb682bb786a ... is the data. Also global package db is somewhat similar with index at e.g. /opt/ghcup/.ghcup/ghc/9.10.1/lib/ghc-9.10.1/lib/package.conf.d and data at /opt/ghcup/.ghcup/ghc/9.10.1/lib/ghc-9.10.1/lib/x86_64-linux-ghc-9.10.1 - Oleg On 16.7.2024 14.43, Tom Ellis wrote: > Thanks Oleg. > > On Tue, Jul 16, 2024 at 02:35:50PM +0300, Oleg Grenrus wrote: >>> This .cabal/store is not a package database. >> .cabal/store/ **is** an ordinary package database. > The package database is the thing that contains the .conf files, isn't > it? So perhaps you mean .cabal/store//package.db is an ordinary > package database? If not it would be good if we can establish > precisely what is meant by "package database" and then clarify the > document. > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs From tom-lists-haskell-cafe-2023 at jaguarpaw.co.uk Tue Jul 16 12:19:20 2024 From: tom-lists-haskell-cafe-2023 at jaguarpaw.co.uk (Tom Ellis) Date: Tue, 16 Jul 2024 13:19:20 +0100 Subject: Cabal and GHC: the big picture In-Reply-To: <14afe39c-142d-410f-b199-ddce09b43a0f@iki.fi> References: <4476693a-140f-460f-b9f5-60612e457ded@iki.fi> <14afe39c-142d-410f-b199-ddce09b43a0f@iki.fi> Message-ID: On Tue, Jul 16, 2024 at 02:51:07PM +0300, Oleg Grenrus wrote: > package.db directory has only .conf files, but those conf files point to > directories with interface and object files. > I think every tool puts the interface and object files next to .conf files. Right, but in principle I could do something different, couldn't I? I could take some subset of the .conf files in package.db, put them in a directory called /tmp/mynewpackagedatabase, and *that* would be a perfectly valid package database (as long as the files it points to still exist), wouldn't it? Tom From simon.peytonjones at gmail.com Tue Jul 16 14:08:33 2024 From: simon.peytonjones at gmail.com (Simon Peyton Jones) Date: Tue, 16 Jul 2024 15:08:33 +0100 Subject: Cabal and GHC: the big picture In-Reply-To: <4476693a-140f-460f-b9f5-60612e457ded@iki.fi> References: <4476693a-140f-460f-b9f5-60612e457ded@iki.fi> Message-ID: Thanks Oleg Don't write "package (unit)". Write unit. OK. But: > Q: "installed package" means the same as "unit" > Not exactly. If a unit is not an installed library, what (precisely) is a unit? Thanks Simon On Tue, 16 Jul 2024 at 12:36, Oleg Grenrus wrote: > My first comment, which applies across the whole document is > > Don't write "package (unit)". Write unit. > > Leave the package to be used solely as "A package is the unit of > distribution and versioning.", and use "unit" consistently for compilation > units, and/or "component" (or more specifically "library" etc). > > The naming of flags is a history artifact. > > The key observation is that "package is the unit of distribution" is > nowadays only a Cabal concept. Only PackageImports and "imprecise" flags > like "-package" (c.f. "-package-id" which ought to be called "-unit-id") in > GHC really know or care about that. > > Second comment, is that be mindful about `cabal-install` and Cabal > difference. The "3 Cabal" section is really "3 cabal-install", and e.g. > stack does things differently. > > > Suppose version 2.3.7 of package P, called P-2.3.7, depends on package Q. > > Is therefore wrong. You should write "Suppose version 2.3.7 of library P, > called "P-2.3.7", depends on library Q". > > Also libraries can depend on executables: e.g. happy, GHC doesn't care > about those dependencies, but Cabal (the library, which does the building) > does. > > > Each unit has a unit-id, looking like > > *may* look. The unit identifier is a random string invented by a build > tool. It's informative, but it really doesn't matter much. > > > Q: "installed package" means the same as "unit" > > Not exactly. > > > Q: "package id" means the same as "unit-id" > > I think so. And I'd argue to not use "package id" going forward. > > > recompiling with no change could change the binary (non-determinism). > Does that change the unit-id? > > It doesn't. Unit-id is invented prior to compilation. Therefore at least > *interface determinism* is important. Though, cabal-install v2 *never* > re-install units to store database, so determinism is not a hard > requirement. > > > A package database can contain many installed versions of the same > package P, or even of a particular version of P, say P-2.4.3, compiled > against different dependencies. > > Even against the same dependencies, even with the same flags, if for some > reason the build tool changes the way it computes the unit-id. > > Also s/package/library/. Re-call, there exist non-main sublibraries. > > > documentation for -package does not clearly specify how the name of the > package is mapped to a unit-id. > > Important bit to remember about "-package" is that it's a legacy flag, not > used by tools anymore. > -package-id looks for the unit exactly. -package scans to find a matching > one, there may be many (and e.g. in case of the same version, probably > non-deterministic choice is made). > > > This .cabal/store is not a package database. > > .cabal/store/ **is** an ordinary package database. > > > Rather, cabal will invoke ghc with a long list of -package-id > flags > > Yes. This is not mutually exclusive. Package database flags tell where, > `-package-id` flags tell what units to use. > > > Can a package contain multiple public libraries? > > Yes. public/private doesn't matter for GHC though. Cabal enforce the > dependency visibility. I.e. private/public is a Cabal concept. (The > visibility is written to interface files, but it's there solely for Cabal > to figure out what the visibility was. GHC doesn't or at least shouldn't > use that info). > > > Difference between unit-id and ABI hash? > > As far as I remember, unit-id tries to approximate ABI hash. In fact, > there was a request to have GHC output something like ABI-hash given the > set of flags. Currently Cabal has an ad-hoc implementation to filter out > flags which should not affect the ABI of a package (like > `-fprint-explicit-foralls`. Side note: it would been clearer if flag name > convention would suggest already whether they affect ABI or not. E.g. > `-ddump` flags or generally `-d` flags don't, but `-f` flags do, except > e.g. `-fprint...` which is kind of `-ddump` like flag). > > On 16.7.2024 13.20, Simon Peyton Jones wrote: > > Friends > > You may remember a recent thread on ghc-devs about GHC and Cabal > . In > it I say how I feel I lack the "big picture" of how GHC and Cabal interact, > and that my mental model is probably faulty. > > Tom Ellis took pity on me, and together we wrote this big-picture > overview about how GHC and Cabal interact > . > Would you like to: > > - Read it as a consumer. > - Does it tell you stuff that is useful? > - What else would you like to know? > - What is un-clear or missing? > - Read it as an expert. > - Is it accurate? > - Are any bits misleading? > - Do the links go to appropriate places? > - What other links or resource would be helpful. > > It is not intended as a replacement for the GHC user guide, nor the Cabal > user guide; rather it is littered with links to those guides which give > much fuller details. Rather, it is intended to put you (well, me for one!) > in a position where you can more easily make sense of those documents. > > We'd love to have your help in improving it. > > Simon > > > > _______________________________________________ > ghc-devs mailing listghc-devs at haskell.orghttp://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > -------------- next part -------------- An HTML attachment was scrubbed... URL: From oleg.grenrus at iki.fi Tue Jul 16 14:41:10 2024 From: oleg.grenrus at iki.fi (Oleg Grenrus) Date: Tue, 16 Jul 2024 17:41:10 +0300 Subject: Cabal and GHC: the big picture In-Reply-To: References: <4476693a-140f-460f-b9f5-60612e457ded@iki.fi> Message-ID: <6ff8c684-97ac-44fc-a02f-e927eacb9445@iki.fi> On 16.7.2024 17.08, Simon Peyton Jones wrote: > Thanks Oleg > > Don't write "package (unit)". Write unit. > > > OK. But: > > > Q: "installed package" means the same as "unit" > > Not exactly. > > If a unit is not an installed library, what (precisely) is a unit? There are "buts". Open units (backpack) are not really libraries. Executables, test-suites and benchmarks are not libraries, but still compilation units and cabal-install gives them unit ids; and probably even tells it to GHC as "-this-unit-id". > > Thanks > > Simon > > > On Tue, 16 Jul 2024 at 12:36, Oleg Grenrus wrote: > > My first comment, which applies across the whole document is > > Don't write "package (unit)". Write unit. > > Leave the package to be used solely as "A package is the unit of > distribution and versioning.", and use "unit" consistently for > compilation units, and/or "component" (or more specifically > "library" etc). > > The naming of flags is a history artifact. > > The key observation is that "package is the unit of distribution" > is nowadays only a Cabal concept. Only PackageImports and > "imprecise" flags like "-package" (c.f. "-package-id" which ought > to be called "-unit-id") in GHC really know or care about that. > > Second comment, is that be mindful about `cabal-install` and Cabal > difference. The "3 Cabal" section is really "3 cabal-install", and > e.g. stack does things differently. > > > Suppose version 2.3.7 of package P, called P-2.3.7, depends on > package Q. > > Is therefore wrong. You should write "Suppose version 2.3.7 of > library P, called "P-2.3.7", depends on library Q". > > Also libraries can depend on executables: e.g. happy, GHC doesn't > care about those dependencies, but Cabal (the library, which does > the building) does. > > > Each unit has a unit-id, looking like > > *may* look. The unit identifier is a random string invented by a > build tool. It's informative, but it really doesn't matter much. > > > Q: "installed package" means the same as "unit" > > Not exactly. > > > Q: "package id" means the same as "unit-id" > > I think so. And I'd argue to not use "package id" going forward. > > >  recompiling with no change could change the binary > (non-determinism). Does that change the unit-id? > > It doesn't. Unit-id is invented prior to compilation. Therefore at > least *interface determinism* is important. Though, cabal-install > v2 *never* re-install units to store database, so determinism is > not a hard requirement. > > > A package database can contain many installed versions of the > same package P, or even of a particular version of P, say P-2.4.3, > compiled against different dependencies. > > Even against the same dependencies, even with the same flags, if > for some reason the build tool changes the way it computes the > unit-id. > > Also s/package/library/. Re-call, there exist non-main sublibraries. > > > documentation for -package does not clearly specify how the name > of the package is mapped to a unit-id. > > Important bit to remember about "-package" is that it's a legacy > flag, not used by tools anymore. > -package-id looks for the unit exactly. -package scans to find a > matching one, there may be many (and e.g. in case of the same > version, probably non-deterministic choice is made). > > > This .cabal/store is not a package database. > > .cabal/store/ **is** an ordinary package database. > > > Rather, cabal will invoke ghc with a long list of -package-id > flags > > Yes. This is not mutually exclusive. Package database flags tell > where, `-package-id` flags tell what units to use. > > > Can a package contain multiple public libraries? > > Yes. public/private doesn't matter for GHC though. Cabal enforce > the dependency visibility. I.e. private/public is a Cabal concept. > (The visibility is written to interface files, but it's there > solely for Cabal to figure out what the visibility was. GHC > doesn't or at least shouldn't use that info). > > > Difference between unit-id and ABI hash? > > As far as I remember, unit-id tries to approximate ABI hash. In > fact, there was a request to have GHC output something like > ABI-hash given the set of flags. Currently Cabal has an ad-hoc > implementation to filter out flags which should not affect the ABI > of a package (like `-fprint-explicit-foralls`. Side note: it would > been clearer if flag name convention would suggest already whether > they affect ABI or not. E.g. `-ddump` flags or generally `-d` > flags don't, but `-f` flags do, except e.g. `-fprint...` which is > kind of `-ddump` like flag). > > On 16.7.2024 13.20, Simon Peyton Jones wrote: >> Friends >> >> You may remember a recent thread on ghc-devs about GHC and Cabal >> .  >> In it I say how I feel I lack the "big picture" of how GHC and >> Cabal interact, and that my mental model is probably faulty. >> >> Tom Ellis took pity on me, and together we wrote this big-picture >> overview about how GHC and Cabal interact >> .  >> Would you like to: >> >> * Read it as a consumer. >> o Does it tell you stuff that is useful? >> o What else would you like to know? >> o What is un-clear or missing? >> * Read it as an expert. >> o Is it accurate? >> o Are any bits misleading? >> o Do the links go to appropriate places? >> o What other links or resource would be helpful. >> >> It is not intended as a replacement for the GHC user guide, nor >> the Cabal user guide; rather it is littered with links to those >> guides which give much fuller details.  Rather, it is intended to >> put you (well, me for one!) in a position where you can more >> easily make sense of those documents. >> >> We'd love to have your help in improving it. >> >> Simon >> >> >> >> _______________________________________________ >> ghc-devs mailing list >> ghc-devs at haskell.org >> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > -------------- next part -------------- An HTML attachment was scrubbed... URL: From oleg.grenrus at iki.fi Tue Jul 16 14:51:51 2024 From: oleg.grenrus at iki.fi (Oleg Grenrus) Date: Tue, 16 Jul 2024 17:51:51 +0300 Subject: Cabal and GHC: the big picture In-Reply-To: References: <4476693a-140f-460f-b9f5-60612e457ded@iki.fi> <14afe39c-142d-410f-b199-ddce09b43a0f@iki.fi> Message-ID: <914207ef-e4d4-4b5f-81c2-026318808dfc@iki.fi> Sure. But let's not be too nitpicky. GHC indeed wants the "package.db" directory when you give it as -package-db flag argument. The directory of .conf files is though not what GHC reads, it reads the cache file in that directory, i.e. package.cache file, so you can hack together a tool (like ghc-pkg) which manages the package.cache file, and then .conf files don't need to be in that directory at all. I don't think it ever made sense to differentiate "cache file" from "just conf files" and/or "conf files with interface and object files". IIRC "ghc-pkg check" checks that interface files exists (or at least the directories exist they should be in). My point was that .cabal/store//package.db (or just .cabal/store/ as it's not ambiguous here) are ordinary package databases. - Oleg On 16.7.2024 15.19, Tom Ellis wrote: > On Tue, Jul 16, 2024 at 02:51:07PM +0300, Oleg Grenrus wrote: >> package.db directory has only .conf files, but those conf files point to >> directories with interface and object files. >> I think every tool puts the interface and object files next to .conf files. > Right, but in principle I could do something different, couldn't I? I > could take some subset of the .conf files in package.db, put them in a > directory called /tmp/mynewpackagedatabase, and *that* would be a > perfectly valid package database (as long as the files it points to > still exist), wouldn't it? > > Tom > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs From sylvain at haskus.fr Thu Jul 18 07:48:59 2024 From: sylvain at haskus.fr (Sylvain Henry) Date: Thu, 18 Jul 2024 09:48:59 +0200 Subject: Cabal and GHC: the big picture In-Reply-To: References: Message-ID: <1ca2b4d2-e262-4db0-b52f-f72d70d2523e@haskus.fr> > What other links or resource would be helpful. On almost the same topic (only the GHC side) I wrote a big picture note "About units" a few years ago: https://gitlab.haskell.org/ghc/ghc/-/blob/12d3b66cedd3c80e7c1e030238c92d26631cab8d/compiler/GHC/Unit.hs#L23 Sylvain On 16/07/2024 12:20, Simon Peyton Jones wrote: > Friends > > You may remember a recent thread on ghc-devs about GHC and Cabal > .  > In it I say how I feel I lack the "big picture" of how GHC and Cabal > interact, and that my mental model is probably faulty. > > Tom Ellis took pity on me, and together we wrote this big-picture > overview about how GHC and Cabal interact > .  > Would you like to: > > * Read it as a consumer. > o Does it tell you stuff that is useful? > o What else would you like to know? > o What is un-clear or missing? > * Read it as an expert. > o Is it accurate? > o Are any bits misleading? > o Do the links go to appropriate places? > o What other links or resource would be helpful. > > It is not intended as a replacement for the GHC user guide, nor the > Cabal user guide; rather it is littered with links to those guides > which give much fuller details. Rather, it is intended to put you > (well, me for one!) in a position where you can more easily make sense > of those documents. > > We'd love to have your help in improving it. > > Simon > > > > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs -------------- next part -------------- An HTML attachment was scrubbed... URL: From matthewtpickering at gmail.com Thu Jul 18 16:35:29 2024 From: matthewtpickering at gmail.com (Matthew Pickering) Date: Thu, 18 Jul 2024 17:35:29 +0100 Subject: Gitlab upgrade tomorrow In-Reply-To: References: Message-ID: The upgrade to 17.1.2 is now completed. Cheers, Matt On Tue, Jul 16, 2024 at 10:08 AM Matthew Pickering wrote: > > The upgrade to 1.17.1 is completed now, migrations are running in the > background. Please post if there are any issues, it's the first > upgrade I have done. > > I will perform the upgrade to 1.17.2 once the corresponding patch > lands in upstream. > (https://github.com/NixOS/nixpkgs/pull/326259/files) > > Cheers, > > Matt > > On Mon, Jul 15, 2024 at 2:45 PM Andreas Klebinger via ghc-devs > wrote: > > > > Hi all, > > > > Gitlab will be temporarily unavailable tomorrow around 9:00 AM CEST > > while it's updated. > > This should not take long, but a short downtime is expected. > > > > Cheers, > > > > - Andreas > > > > _______________________________________________ > > ghc-devs mailing list > > ghc-devs at haskell.org > > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs From jappieklooster at hotmail.com Mon Jul 22 23:38:49 2024 From: jappieklooster at hotmail.com (jappie klooster) Date: Mon, 22 Jul 2024 19:38:49 -0400 Subject: questions about head.hackage Message-ID: Hello everyone, For the stability working group I'm trying to understand how many packages head.hackage allows you to build. Does anyone know the answer to this or have an idea how I can find out? For context, the stability working group is a group of interested people who try and allow ghc and haskell system to innovate, while reducing the amount of friction caused to the wider Haskell community. Current efforts include the marking language proposals as stable, and my work of investigating historic breakage. Investigating this breakage will allow us to make better informed decisions on which pain points to prioritize. -- With regards, Jappie Klooster e: hi at jappie.me w: https://jappie.me m: +31644237437 From moritz.angermann at gmail.com Tue Jul 23 03:03:19 2024 From: moritz.angermann at gmail.com (Moritz Angermann) Date: Tue, 23 Jul 2024 12:03:19 +0900 Subject: questions about head.hackage In-Reply-To: References: Message-ID: Hi Jappie, This answer will only tangentially answer your question. Head hackage by design is GhC development internal and should never ever leak out of ghc development. As such the packages building with it today may not build tomorrow (ghc HEAD is a moving target). What you _can_ do is look at the package patches in head.hackage; and the look at the dependencies for them in hackage to get a rough estimate of direct and indirect packages being enabled. Getting an accurate count would require trying to compile all hackage against some ghc from master. And that number likely changes every day due to head of ghc and hackage being moving targets. L A more sensible option might be to try and use a subset like stackage or the CLC stackage set to buidl against. Best, Moritz. On Tue, 23 Jul 2024 at 8:39 AM, jappie klooster wrote: > Hello everyone, > > For the stability working group I'm trying to understand how many > packages head.hackage allows you to build. > Does anyone know the answer to this or have an idea how I can find out? > > For context, the stability working group is a group of interested people > who try and allow ghc and haskell > system to innovate, while reducing the amount of friction caused to the > wider Haskell community. > Current efforts include the marking language proposals as stable, > and my work of investigating historic breakage. > Investigating this breakage will allow us to make better informed > decisions on which pain points to prioritize. > > > -- > With regards, > > Jappie Klooster > > e: hi at jappie.me > w: https://jappie.me > m: +31644237437 > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > -------------- next part -------------- An HTML attachment was scrubbed... URL: From andrea at andreabedini.com Tue Jul 23 03:29:03 2024 From: andrea at andreabedini.com (Andrea Bedini) Date: Tue, 23 Jul 2024 11:29:03 +0800 Subject: questions about head.hackage In-Reply-To: References: Message-ID: On Tue, 23 Jul 2024, at 7:38 AM, jappie klooster wrote: > For the stability working group I'm trying to understand how many > packages head.hackage allows you to build. > Does anyone know the answer to this or have an idea how I can find out? I don't have the numbers you are looking for but one data-point is that cabal-install itself (which has a reasonable number of dependencies) typically does not compile with a freshly released GHC. This forces to temporarily add head.hackage in CI when a new GHC releases. IMHO, using head.hackage to test a source-distribution defeats the purposes of testing. We cannot (and should not) assume those building cabal's source distribution(s) are using it, and therefore `cabal install cabal-install` will end up succeeding in our CI and failing for everybody else. My 2c, Andrea -- Andrea Bedini https://www.andreabedini.com From teofilcamarasu at gmail.com Tue Jul 23 07:06:40 2024 From: teofilcamarasu at gmail.com (Teofil Camarasu) Date: Tue, 23 Jul 2024 08:06:40 +0100 Subject: questions about head.hackage In-Reply-To: References: Message-ID: Hi Jappie, There is a GHC grafana with some statistics. Unfortunately the relevant dashboards seem to be broken at present. Hopefully someone on this list could fix it. See: https://grafana.gitlab.haskell.org/d/7T7oEMlMz/head-hackage-performance?orgId=2&viewPanel=3 There's also https://grafana.gitlab.haskell.org/d/RHebDhq7z/head-hackage-residency-profiling?orgId=2&refresh=30m, which is for residency information, but looking at the filters suggests that the peak amount of packages is around 900. Cheers, Teo On Tue, Jul 23, 2024 at 4:29 AM Andrea Bedini wrote: > On Tue, 23 Jul 2024, at 7:38 AM, jappie klooster wrote: > > For the stability working group I'm trying to understand how many > > packages head.hackage allows you to build. > > Does anyone know the answer to this or have an idea how I can find out? > > I don't have the numbers you are looking for but one data-point is that > cabal-install itself (which has a reasonable number of dependencies) > typically does not compile with a freshly released GHC. This forces to > temporarily add head.hackage in CI when a new GHC releases. > > IMHO, using head.hackage to test a source-distribution defeats the > purposes of testing. We cannot (and should not) assume those building > cabal's source distribution(s) are using it, and therefore `cabal install > cabal-install` will end up succeeding in our CI and failing for everybody > else. > > My 2c, > Andrea > > -- > Andrea Bedini > https://www.andreabedini.com > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sam.derbyshire at gmail.com Tue Jul 23 14:36:13 2024 From: sam.derbyshire at gmail.com (Sam Derbyshire) Date: Tue, 23 Jul 2024 16:36:13 +0200 Subject: Acknowledgement of CI instability Message-ID: Hi all, The GHC team would like to acknowledge some ongoing CI instabilities in the GHC project: - regular failures of the i386 job, possibly related to the bump to debian 12 (although the job still failed occasionally before this), - flakiness of the MultiLayerModulesDefsGhciReload test causing the fedora33-release job to fail, - lack of availability of darwin runners, causing aarch64-darwin and x86_64-darwin jobs to time out. These issues are currently causing a string of marge batch failures, holding up several MRs. We are currently short on resources for addressing problems with CI, so please bear with us while we sort the situation out. In the meantime, feel free to let us know of any other CI issues that are impacting your work on GHC. Best, Sam -------------- next part -------------- An HTML attachment was scrubbed... URL: From moritz.angermann at gmail.com Tue Jul 23 23:57:23 2024 From: moritz.angermann at gmail.com (Moritz Angermann) Date: Wed, 24 Jul 2024 08:57:23 +0900 Subject: Acknowledgement of CI instability In-Reply-To: References: Message-ID: Hi, As I’ve now arrived in Seoul and have my Mac’s back, I can bring two more M1 Mac’s back online today. However they are hooked up on a serviced apartment internet which might not be the fastest. Best, Moritz On Tue, 23 Jul 2024 at 11:36 PM, Sam Derbyshire wrote: > Hi all, > > The GHC team would like to acknowledge some ongoing CI instabilities in > the GHC project: > > - regular failures of the i386 job, possibly related to the bump to > debian 12 > > (although the job still failed occasionally before this), > - flakiness of the MultiLayerModulesDefsGhciReload test causing the > fedora33-release job to fail, > - lack of availability of darwin runners, causing aarch64-darwin and > x86_64-darwin jobs to time out. > > These issues are currently causing a string of marge batch failures, > holding up several MRs. > > We are currently short on resources for addressing problems with CI, so > please bear with us while we sort the situation out. In the meantime, feel > free to let us know of any other CI issues that are impacting your work on > GHC. > > Best, > > Sam > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jappieklooster at hotmail.com Wed Jul 24 00:54:31 2024 From: jappieklooster at hotmail.com (jappie klooster) Date: Tue, 23 Jul 2024 20:54:31 -0400 Subject: questions about head.hackage In-Reply-To: References: Message-ID: Thanks Teo, that's exactly what I was looking for I'll keep in mind this number changes over time. @Andrea I think the intention is that the patches eventually get upstreamed to the libraries It's just a good measure point for me to do quantitative breakage analysis upon. On 23-07-2024 03:06, Teofil Camarasu wrote: > Hi Jappie, > > There is a GHC grafana with some statistics. Unfortunately the > relevant dashboards seem to be broken at present. Hopefully someone on > this list could fix it. > See: > https://grafana.gitlab.haskell.org/d/7T7oEMlMz/head-hackage-performance?orgId=2&viewPanel=3 > > > There's also > https://grafana.gitlab.haskell.org/d/RHebDhq7z/head-hackage-residency-profiling?orgId=2&refresh=30m > , > which is for residency information, but looking at the filters > suggests that the peak amount of packages is around 900. > > Cheers, > Teo > > On Tue, Jul 23, 2024 at 4:29 AM Andrea Bedini > wrote: > > On Tue, 23 Jul 2024, at 7:38 AM, jappie klooster wrote: > > For the stability working group I'm trying to understand how many > > packages head.hackage allows you to build. > > Does anyone know the answer to this or have an idea how I can > find out? > > I don't have the numbers you are looking for but one data-point is > that cabal-install itself (which has a reasonable number of > dependencies) typically does not compile with a freshly released > GHC. This forces to temporarily add head.hackage in CI when a new > GHC releases. > > IMHO, using head.hackage to test a source-distribution defeats the > purposes of testing. We cannot (and should not) assume those > building cabal's source distribution(s) are using it, and > therefore `cabal install cabal-install` will end up succeeding in > our CI and failing for everybody else. > > My 2c, > Andrea > > -- > Andrea Bedini > https://www.andreabedini.com > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > -- With regards, Jappie Klooster e:hi at jappie.me w:https://jappie.me m: +31644237437 -------------- next part -------------- An HTML attachment was scrubbed... URL: From matthewtpickering at gmail.com Mon Jul 29 10:38:25 2024 From: matthewtpickering at gmail.com (Matthew Pickering) Date: Mon, 29 Jul 2024 11:38:25 +0100 Subject: Haskell Implementors' Workshop - Change of date Message-ID: This year the decision has been taken to combine together Haskell Implementors' Workshop and Haskell Symposium. Therefore the workshop will now take place on the dates September 6/7 (the dates of the Haskell Symposium). Please email me if you have any questions. Cheers, Matt -------------- next part -------------- An HTML attachment was scrubbed... URL: From celeste.hollenbeck at gmail.com Tue Jul 30 14:57:52 2024 From: celeste.hollenbeck at gmail.com (Celeste Hollenbeck) Date: Tue, 30 Jul 2024 15:57:52 +0100 Subject: Question about profiling reports Message-ID: Hi GHC Team, I have a question for a research project. I'm looking at GHC's profiler, and the documentation says a profiling report displays "a break-down by cost centre of the most costly functions in the program". Here's an example of the report that I'm talking about: MAIN MAIN 102 0 0.0 0.0 100.0 100.0 CAF GHC.IO.Handle.FD 128 0 0.0 0.0 0.0 0.0 CAF GHC.IO.Encoding.Iconv 120 0 0.0 0.0 0.0 0.0 CAF GHC.Conc.Signal 110 0 0.0 0.0 0.0 0.0 CAF Main 108 0 0.0 0.0 100.0 100.0 main Main 204 1 0.0 0.0 100.0 100.0 fib Main 205 2692537 100.0 100.0 100.0 100.0 This example is under Section 8.1 of the GHC User's Guide https://downloads.haskell.org/ghc/latest/docs/users_guide/profiling.html#:~:text=GHC's%20profiling%20system%20assigns%20costs,to%20the%20enclosing%20cost%20centre. It looks like the numbers often add up to less than 100% for the %time, but I don't see any documentation on a threshold for what makes a cost centre "costly"—so I assume that "costly" means that it takes up any time whatsoever, and any cost centres that take up any time at all are included in the report? So perhaps the numbers under %time don't add up to 100% all the time because of rounding error or perhaps garbage collection? Or something else that is not profiled? Is that correct? Thanks, Celeste -------------- next part -------------- An HTML attachment was scrubbed... URL: