From benjamin.redelings at gmail.com Thu Dec 1 02:42:23 2022 From: benjamin.redelings at gmail.com (Benjamin Redelings) Date: Wed, 30 Nov 2022 21:42:23 -0500 Subject: GHC and type-family rewriting? Message-ID: <3b922ff9-af7a-ff62-06a9-822751262564@gmail.com> Hi, I've managed to code up implications and GADTs, and am now working on adding type families.  I've been following the OutsideIn paper, but it seems that GHC is not really following the same plan for the solver.  For example, instead of replacing every type family with a metavariable, it only does this for occur-check failures.  It talks about "cycle breakers" instead of "flattening substitutions".  It creates flatting substitutions for both givens and wanteds (I think). And I see this note: > -- | A 'Xi'-type is one that has been fully rewritten with respect > -- to the inert set; that is, it has been rewritten by the algorithm > -- in GHC.Tc.Solver.Rewrite. (Historical note: 'Xi', for years and years, > -- meant that a type was type-family-free. It does *not* mean this > -- any more.) > type Xi = TcType If I'm understanding correctly, the inert set is now thought of as a "generalized substitution" that replaces either an LHS (untouchable type variables or type family applications) with an RHS. I guess what I'm wondering is: (Q1) Did GHC evolve to this point starting from something fairly close to the OutsideIn paper? (Q2) Is the new approach (i.e. eager type family rewriting) mostly to making rewriting faster? (Q3) Does it sound reasonable to implement the approach from the OutsideIn paper, and than gradually transform it to look more like GHC? -BenRI From bryan at haskell.foundation Thu Dec 1 08:21:18 2022 From: bryan at haskell.foundation (Bryan Richter) Date: Thu, 1 Dec 2022 10:21:18 +0200 Subject: DKIM failures for gitlab mail In-Reply-To: References: <15a1738d1daedaad7a7f7b8a3b2db9944dd3ed7b.camel@joachim-breitner.de> Message-ID: Thanks for noticing, Joachim! Ben Gamari is still the primary contact for GitLab configuration... Ben, maybe you know something about this? On Wed, Nov 30, 2022 at 7:12 PM Viktor Dukhovni wrote: > > On Wed, Nov 30, 2022 at 05:33:44PM +0100, Joachim Breitner wrote: > > > I noticed that a small number of Gitlab notification emails end up in > > my spamfilter. While there is not much you can do about triggering some > > bayesian style spam filter at my email provider (mailbox.org), I did > > notice this in the headers: > > > > X-Spam-Status: No, score=2.704 tagged_above=2 required=6 > > tests=[DKIM_INVALID=0.1, DKIM_SIGNED=0.1, HS_RSPAMD_10_11=2.5, > > HTML_MESSAGE=0.001, SPF_HELO_NONE=0.001, SPF_NONE=0.001, > > URIBL_BLOCKED=0.001] autolearn=no autolearn_force=no > > Authentication-Results: spamfilter01.heinlein-hosting.de (amavisd-new); > > dkim=fail (1024-bit key) reason="fail (bad RSA signature)" > > header.d=gitlab.haskell.org > > DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; > > d=gitlab.haskell.org; > > s=mail; t=1669733134; > > bh=D0NUcHiskEnwSP99umP3zo8Fz8fl74OgAJ8NRDKCsp4=; > > h=Date:From:Reply-To:To:In-Reply-To:References:Subject:List-Id: > > List-Unsubscribe; > > b=R+WMLfhRZZdYxMd6K6w+iodDe8EHzwONNArNyboqsU5NnafPRhKZ1UeGxO/BCMvEK > > M7XHRRrBsPfRYpTph7xSGY427KGXieASVg1GDhAiwKSLBCiqDdkBaoJLLUIfUD02NS > > ouI3tvQ9mddNdaEK7retq8N+29hzs/ezf9cpgy+Q= > > Indeed the signature in "b=" was not made by the key at > mail._domainkey.gitlab.haskell.org. Running the below: > > sig=$( > printf "%s\n%s\n%s\n" \ > R+WMLfhRZZdYxMd6K6w+iodDe8EHzwONNArNyboqsU5NnafPRhKZ1UeGxO/BCMvE \ > KM7XHRRrBsPfRYpTph7xSGY427KGXieASVg1GDhAiwKSLBCiqDdkBaoJLLUIfUD0 \ > 2NSouI3tvQ9mddNdaEK7retq8N+29hzs/ezf9cpgy+Q= > ) > > pkey=$( > dig +short -t txt mail._domainkey.gitlab.haskell.org | > perl -MMIME::Base64 -ne ' > /^"v=DKIM1;/ or next; > print decode_base64($1) if m{;\s*p=(\S+?)(?:;|$)} > ' | > openssl pkey -pubin -inform DER > ) > > openssl rsautl -raw -encrypt -pubin \ > -inkey <( printf "%s\n" "$pkey" ) \ > -in <(printf "%s\n" "$sig" | openssl base64 -d) | > xxd -p > > the output is: > > 509bfc93a492f1b5328308e51624d9a7ed1378861f577b11413c5034bc0c > 673d61660434d4bc30844e7648da0f9605923805973a313a8c3bc82215cc > ac447e47551087c544a0592ac3ae48474584bad7d9ca5b850a67493a7977 > d28aaa3a9a7580d165dc4f31ff484bdbc40e94a2be1750e71c51c555b5c1 > 6bc051947bb07ae4 > > Which is not a PKCS#1.5 padded signature block. So either the > "b=" value was corrupted in transit, or it was signed by a key > that is different from what is published in DNS. > > > but maybe Postfix is not using the right key? > > Strictly speaking that's not Postfix itself, but some DKIM milter, but > nits aside, more likely a stale public key is published in DNS. > > -- > Viktor. > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs From lists at richarde.dev Fri Dec 9 04:48:44 2022 From: lists at richarde.dev (Richard Eisenberg) Date: Fri, 9 Dec 2022 04:48:44 +0000 Subject: GHC and type-family rewriting? In-Reply-To: <3b922ff9-af7a-ff62-06a9-822751262564@gmail.com> References: <3b922ff9-af7a-ff62-06a9-822751262564@gmail.com> Message-ID: <010f0184f536dd95-2169874b-47da-49d2-82ba-79426baf1513-000000@us-east-2.amazonses.com> > On Nov 30, 2022, at 9:42 PM, Benjamin Redelings wrote: > > (Q1) Did GHC evolve to this point starting from something fairly close to the OutsideIn paper? Yes. > > (Q2) Is the new approach (i.e. eager type family rewriting) mostly to making rewriting faster? No. Simpler, not faster (and not slower). Or that was the intent. > > (Q3) Does it sound reasonable to implement the approach from the OutsideIn paper, and than gradually transform it to look more like GHC? > Sure, but I'm not sure what the advantage of doing so would be. This is all my doing: for years and years, GHC's treatment of type families was as described in OutsideIn. But I never could quite figure out why we needed to have flattening variables. And so I got rid of them -- this seemed like a simplification. I'm not sure it really panned out, though: without flattening variables, we need these cycle-breaker variables (which are pretty gross). On the flip side, I think the new approach might enable the possibility of reducing type families only in "strict" positions (e.g. the argument to another type family or perhaps a class during instance lookup). In the end, I don't think either the old way or the new way is the Right Answer. Maybe you can come up with something better than both! Richard > -BenRI > > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs From matthewtpickering at gmail.com Wed Dec 14 12:51:00 2022 From: matthewtpickering at gmail.com (Matthew Pickering) Date: Wed, 14 Dec 2022 12:51:00 +0000 Subject: 9.4.4 release Message-ID: Hi all, We are going to release the latest release in the 9.4.* series before Christmas. The release fixes a number of small bugs which have surfaced to do with 9.4. For the current full list of backports see: https://gitlab.haskell.org/ghc/ghc/-/merge_requests/9523 I am preparing the release branch and then Ben will perform the final steps necessary for the release. Cheers, Matt From Gergo.Erdi at sc.com Thu Dec 15 01:23:06 2022 From: Gergo.Erdi at sc.com (Erdi, Gergo) Date: Thu, 15 Dec 2022 01:23:06 +0000 Subject: 9.4.4 release In-Reply-To: References: Message-ID: PUBLIC I'd like to nominate !9153 since it fixes a regression from 9.2 to 9.4. Also, I don't know why but I was unable to make this comment on the MR, because I keep getting: "Your comment could not be submitted! Please check your network connection and try again." Even though I can otherwise browse around Gitlab. -----Original Message----- From: ghc-devs On Behalf Of Matthew Pickering Sent: Wednesday, December 14, 2022 8:51 PM To: GHC developers ; hasufell at posteo.de; juhpetersen at gmail.com Subject: [External] 9.4.4 release Hi all, We are going to release the latest release in the 9.4.* series before Christmas. The release fixes a number of small bugs which have surfaced to do with 9.4. For the current full list of backports see: https://clicktime.symantec.com/15tSyRHEtFR816m1QoQTB?h=nKTqAjBvyF3MovWDef-WEenA4Y75aM3oDv2bE0u5ZmI=&u=https://gitlab.haskell.org/ghc/ghc/-/merge_requests/9523 I am preparing the release branch and then Ben will perform the final steps necessary for the release. Cheers, Matt _______________________________________________ ghc-devs mailing list ghc-devs at haskell.org https://clicktime.symantec.com/15tStb5xRdjXb9w5sF1JZ?h=sH-g7rK8uaH4H-qR-px1N-_RFZw104O5bEveNx9eEqo=&u=http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs This email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please delete all copies and notify the sender immediately. You may wish to refer to the incorporation details of Standard Chartered PLC, Standard Chartered Bank and their subsidiaries at https: //www.sc.com/en/our-locations Where you have a Financial Markets relationship with Standard Chartered PLC, Standard Chartered Bank and their subsidiaries (the "Group"), information on the regulatory standards we adhere to and how it may affect you can be found in our Regulatory Compliance Statement at https: //www.sc.com/rcs/ and Regulatory Compliance Disclosures at http: //www.sc.com/rcs/fm Insofar as this communication is not sent by the Global Research team and contains any market commentary, the market commentary has been prepared by the sales and/or trading desk of Standard Chartered Bank or its affiliate. It is not and does not constitute research material, independent research, recommendation or financial advice. Any market commentary is for information purpose only and shall not be relied on for any other purpose and is subject to the relevant disclaimers available at https: //www.sc.com/en/regulatory-disclosures/#market-disclaimer. Insofar as this communication is sent by the Global Research team and contains any research materials prepared by members of the team, the research material is for information purpose only and shall not be relied on for any other purpose, and is subject to the relevant disclaimers available at https: //research.sc.com/research/api/application/static/terms-and-conditions. Insofar as this e-mail contains the term sheet for a proposed transaction, by responding affirmatively to this e-mail, you agree that you have understood the terms and conditions in the attached term sheet and evaluated the merits and risks of the transaction. We may at times also request you to sign the term sheet to acknowledge the same. Please visit https: //www.sc.com/en/regulatory-disclosures/dodd-frank/ for important information with respect to derivative products. From matthewtpickering at gmail.com Thu Dec 15 09:23:52 2022 From: matthewtpickering at gmail.com (Matthew Pickering) Date: Thu, 15 Dec 2022 09:23:52 +0000 Subject: 9.4.4 release In-Reply-To: References: Message-ID: That issue has not been resolved for 9.4.4 but we will still provide CentOS7 builds (which link against the older glibc). On Thu, Dec 15, 2022 at 1:19 AM Julian Ospald wrote: > > Has there been any changes to the bindists? Does https://gitlab.haskell.org/ghc/ghc/-/issues/22268 still exist? > > On December 14, 2022 12:51:00 PM UTC, Matthew Pickering wrote: >> >> Hi all, >> >> We are going to release the latest release in the 9.4.* series before Christmas. >> >> The release fixes a number of small bugs which have surfaced to do with 9.4. >> >> For the current full list of backports see: >> >> https://gitlab.haskell.org/ghc/ghc/-/merge_requests/9523 >> >> I am preparing the release branch and then Ben will perform the final >> steps necessary for the release. >> >> Cheers, >> >> Matt From matthewtpickering at gmail.com Thu Dec 15 09:27:30 2022 From: matthewtpickering at gmail.com (Matthew Pickering) Date: Thu, 15 Dec 2022 09:27:30 +0000 Subject: 9.4.4 release In-Reply-To: References: Message-ID: !9153 is already present in 9.4.3 at commit hash edfa9f4653b10cb0a897ace15b25b3b52cde5c39 (which was from this specific fix for the 9.4 branch https://gitlab.haskell.org/ghc/ghc/-/merge_requests/9137) Cheers, Matt On Thu, Dec 15, 2022 at 1:23 AM Erdi, Gergo wrote: > > PUBLIC > > I'd like to nominate !9153 since it fixes a regression from 9.2 to 9.4. > Also, I don't know why but I was unable to make this comment on the MR, because I keep getting: > > "Your comment could not be submitted! Please check your network connection and try again." > > Even though I can otherwise browse around Gitlab. > > -----Original Message----- > From: ghc-devs On Behalf Of Matthew Pickering > Sent: Wednesday, December 14, 2022 8:51 PM > To: GHC developers ; hasufell at posteo.de; juhpetersen at gmail.com > Subject: [External] 9.4.4 release > > > Hi all, > > We are going to release the latest release in the 9.4.* series before Christmas. > > The release fixes a number of small bugs which have surfaced to do with 9.4. > > For the current full list of backports see: > > https://clicktime.symantec.com/15tSyRHEtFR816m1QoQTB?h=nKTqAjBvyF3MovWDef-WEenA4Y75aM3oDv2bE0u5ZmI=&u=https://gitlab.haskell.org/ghc/ghc/-/merge_requests/9523 > > I am preparing the release branch and then Ben will perform the final steps necessary for the release. > > Cheers, > > Matt > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > https://clicktime.symantec.com/15tStb5xRdjXb9w5sF1JZ?h=sH-g7rK8uaH4H-qR-px1N-_RFZw104O5bEveNx9eEqo=&u=http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > > This email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please delete all copies and notify the sender immediately. You may wish to refer to the incorporation details of Standard Chartered PLC, Standard Chartered Bank and their subsidiaries at https: //www.sc.com/en/our-locations > > Where you have a Financial Markets relationship with Standard Chartered PLC, Standard Chartered Bank and their subsidiaries (the "Group"), information on the regulatory standards we adhere to and how it may affect you can be found in our Regulatory Compliance Statement at https: //www.sc.com/rcs/ and Regulatory Compliance Disclosures at http: //www.sc.com/rcs/fm > > Insofar as this communication is not sent by the Global Research team and contains any market commentary, the market commentary has been prepared by the sales and/or trading desk of Standard Chartered Bank or its affiliate. It is not and does not constitute research material, independent research, recommendation or financial advice. Any market commentary is for information purpose only and shall not be relied on for any other purpose and is subject to the relevant disclaimers available at https: //www.sc.com/en/regulatory-disclosures/#market-disclaimer. > > Insofar as this communication is sent by the Global Research team and contains any research materials prepared by members of the team, the research material is for information purpose only and shall not be relied on for any other purpose, and is subject to the relevant disclaimers available at https: //research.sc.com/research/api/application/static/terms-and-conditions. > > Insofar as this e-mail contains the term sheet for a proposed transaction, by responding affirmatively to this e-mail, you agree that you have understood the terms and conditions in the attached term sheet and evaluated the merits and risks of the transaction. We may at times also request you to sign the term sheet to acknowledge the same. > > Please visit https: //www.sc.com/en/regulatory-disclosures/dodd-frank/ for important information with respect to derivative products. From benjamin.redelings at gmail.com Sat Dec 17 15:17:01 2022 From: benjamin.redelings at gmail.com (Benjamin Redelings) Date: Sat, 17 Dec 2022 07:17:01 -0800 Subject: GHC and type-family rewriting? In-Reply-To: <010f0184f536dd95-2169874b-47da-49d2-82ba-79426baf1513-000000@us-east-2.amazonses.com> References: <3b922ff9-af7a-ff62-06a9-822751262564@gmail.com> <010f0184f536dd95-2169874b-47da-49d2-82ba-79426baf1513-000000@us-east-2.amazonses.com> Message-ID: <3277cd25-0afb-87cd-5be7-a3c435cdc0b4@gmail.com> Hi Richard, Thanks!  This context of where the current GHC approach came from is quite helpful.  I think I see what you mean by reducing type families only in "strict" positions... and maybe I see what you mean by trying to make the whole approach less ugly.  I still need to do quite a bit of plumbing work before I get to anything that interesting. But supposing I do get there, I'm curious if there are some papers on term-rewriting that would be helpful to set the context?  The OutsideIn paper mentions Kapur (1997) "Shostak's congruence closure as completion" in support of the flattening idea. -BenRI On 12/8/22 11:48 PM, Richard Eisenberg wrote: > >> On Nov 30, 2022, at 9:42 PM, Benjamin Redelings wrote: >> >> (Q1) Did GHC evolve to this point starting from something fairly close to the OutsideIn paper? > Yes. > >> (Q2) Is the new approach (i.e. eager type family rewriting) mostly to making rewriting faster? > No. Simpler, not faster (and not slower). Or that was the intent. > >> (Q3) Does it sound reasonable to implement the approach from the OutsideIn paper, and than gradually transform it to look more like GHC? >> > Sure, but I'm not sure what the advantage of doing so would be. > > This is all my doing: for years and years, GHC's treatment of type families was as described in OutsideIn. But I never could quite figure out why we needed to have flattening variables. And so I got rid of them -- this seemed like a simplification. I'm not sure it really panned out, though: without flattening variables, we need these cycle-breaker variables (which are pretty gross). On the flip side, I think the new approach might enable the possibility of reducing type families only in "strict" positions (e.g. the argument to another type family or perhaps a class during instance lookup). In the end, I don't think either the old way or the new way is the Right Answer. Maybe you can come up with something better than both! > > Richard Thanks!  This is quite helpful. -BenRI From lists at richarde.dev Sun Dec 18 03:41:31 2022 From: lists at richarde.dev (Richard Eisenberg) Date: Sun, 18 Dec 2022 03:41:31 +0000 Subject: GHC and type-family rewriting? In-Reply-To: <3277cd25-0afb-87cd-5be7-a3c435cdc0b4@gmail.com> References: <3b922ff9-af7a-ff62-06a9-822751262564@gmail.com> <010f0184f536dd95-2169874b-47da-49d2-82ba-79426baf1513-000000@us-east-2.amazonses.com> <3277cd25-0afb-87cd-5be7-a3c435cdc0b4@gmail.com> Message-ID: <010f018523528e99-1f851e68-926a-43eb-b947-c23d810366e7-000000@us-east-2.amazonses.com> > On Dec 17, 2022, at 10:17 AM, Benjamin Redelings wrote: > > But supposing I do get there, I'm curious if there are some papers on term-rewriting that would be helpful to set the context? The OutsideIn paper mentions Kapur (1997) "Shostak's congruence closure as completion" in support of the flattening idea. I'm not aware of any, but I think looking for some is a good idea. For better or worse, the current approach was freshly invented; looking for prior art might have yielded something better. Richard -------------- next part -------------- An HTML attachment was scrubbed... URL: From ben at well-typed.com Sat Dec 24 22:36:15 2022 From: ben at well-typed.com (Ben Gamari) Date: Sat, 24 Dec 2022 17:36:15 -0500 Subject: [ANNOUNCE] GHC 9.4.4 is now available Message-ID: <87k02gbfas.fsf@smart-cactus.org> The GHC developers are happy to announce the availability of GHC 9.4.4. Binary distributions, source distributions, and documentation are available at downloads.haskell.org [1]. This release is primarily a bugfix release. Bugs fixed include: * An issue where the wrong labels were used in the eventlog for events emitted when using info table profiling which was broken in 9.4.3 (#22452) * Fix a long standing bug where invalid eventlogs would be produced with long command-lines (#20221) * Fix a regression in eta-expansion which resulted in serious performance regressions in 9.4.3 (#22424) * Fix the `-fdefer-diagnostics` flag (#22391) * Fixes to several subtle compiler panics (#22491, #22416, #22549, #22475, #22039) * Add necessary write barriers to `IORef` operations, avoiding potential soundness issues on architectures with weakly-ordered memory models (#22468) Note that, as GHC 9.4 is the first release series where the release artifacts are all generated by our new Hadrian build system, it is possible that there will be packaging issues. If you enounter trouble while using a binary distribution, please open a [ticket]. Likewise, if you are a downstream packager, do consider migrating to [Hadrian] to run your build; the Hadrian build system can be built using `cabal-install`, `stack`, or the in-tree [bootstrap script]. See the accompanying [blog post] for details on migrating packaging to Hadrian. We would also like to emphasize that GHC 9.4 must be used in conjunction with Cabal-3.8 or later. This is particularly important for Windows users due to changes in GHC's Windows toolchain. We would like to thank Microsoft Azure, GitHub, IOG, the Zw3rk stake pool, Well-Typed, Tweag I/O, Serokell, Equinix, SimSpace, Haskell Foundation, and other anonymous contributors whose on-going financial and in-kind support has facilitated GHC maintenance and release management over the years. Finally, this release would not have been possible without the hundreds of open-source contributors whose work comprise this release. As always, do give this release a try and open a [ticket] if you see anything amiss. Happy Haskelling, - Ben [1]: https://downloads.haskell.org/ghc/9.4.4 [ticket]: https://gitlab.haskell.org/ghc/ghc/-/issues/new [bootstrap script]: https://gitlab.haskell.org/ghc/ghc/-/blob/e2520df3fffa0cf22fb19c5fb872832d11c07d35/hadrian/bootstrap/README.md [Hadrian]: https://gitlab.haskell.org/ghc/ghc/-/blob/e2520df3fffa0cf22fb19c5fb872832d11c07d35/hadrian [blog post]: https://www.haskell.org/ghc/blog/20220805-make-to-hadrian.html -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From david.feuer at gmail.com Sun Dec 25 00:00:02 2022 From: david.feuer at gmail.com (David Feuer) Date: Sat, 24 Dec 2022 19:00:02 -0500 Subject: [ANNOUNCE] GHC 9.4.4 is now available In-Reply-To: <87k02gbfas.fsf@smart-cactus.org> References: <87k02gbfas.fsf@smart-cactus.org> Message-ID: Does this release include the fix for #22549 (infinite loops for some undecidable instances)? On Sat, Dec 24, 2022, 5:36 PM Ben Gamari wrote: > The GHC developers are happy to announce the availability of GHC 9.4.4. > Binary > distributions, source distributions, and documentation are available at > downloads.haskell.org [1]. > > This release is primarily a bugfix release. Bugs fixed include: > > * An issue where the wrong labels were used in the eventlog for events > emitted > when using info table profiling which was broken in 9.4.3 (#22452) > > * Fix a long standing bug where invalid eventlogs would be produced with > long > command-lines (#20221) > > * Fix a regression in eta-expansion which resulted in serious performance > regressions in 9.4.3 (#22424) > > * Fix the `-fdefer-diagnostics` flag (#22391) > > * Fixes to several subtle compiler panics > (#22491, #22416, #22549, #22475, #22039) > > * Add necessary write barriers to `IORef` operations, avoiding potential > soundness issues on architectures with weakly-ordered memory models > (#22468) > > Note that, as GHC 9.4 is the first release series where the release > artifacts > are all generated by our new Hadrian build system, it is possible that > there > will be packaging issues. If you enounter trouble while using a binary > distribution, please open a [ticket]. Likewise, if you are a downstream > packager, do consider migrating to [Hadrian] to run your build; the Hadrian > build system can be built using `cabal-install`, `stack`, or the in-tree > [bootstrap script]. See the accompanying [blog post] for details on > migrating packaging to Hadrian. > > We would also like to emphasize that GHC 9.4 must be used in conjunction > with > Cabal-3.8 or later. This is particularly important for Windows users due to > changes in GHC's Windows toolchain. > > We would like to thank Microsoft Azure, GitHub, IOG, the Zw3rk stake pool, > Well-Typed, Tweag I/O, Serokell, Equinix, SimSpace, Haskell Foundation, and > other anonymous contributors whose on-going financial and in-kind support > has > facilitated GHC maintenance and release management over the years. Finally, > this release would not have been possible without the hundreds of > open-source > contributors whose work comprise this release. > > As always, do give this release a try and open a [ticket] if you see > anything amiss. > > Happy Haskelling, > > - Ben > > > [1]: https://downloads.haskell.org/ghc/9.4.4 > [ticket]: https://gitlab.haskell.org/ghc/ghc/-/issues/new > [bootstrap script]: > https://gitlab.haskell.org/ghc/ghc/-/blob/e2520df3fffa0cf22fb19c5fb872832d11c07d35/hadrian/bootstrap/README.md > [Hadrian]: > https://gitlab.haskell.org/ghc/ghc/-/blob/e2520df3fffa0cf22fb19c5fb872832d11c07d35/hadrian > [blog post]: > https://www.haskell.org/ghc/blog/20220805-make-to-hadrian.html > _______________________________________________ > Glasgow-haskell-users mailing list > Glasgow-haskell-users at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/glasgow-haskell-users > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ben at well-typed.com Sun Dec 25 03:29:00 2022 From: ben at well-typed.com (Ben Gamari) Date: Sat, 24 Dec 2022 22:29:00 -0500 Subject: [Haskell-cafe] [ANNOUNCE] GHC 9.4.4 is now available In-Reply-To: References: <87k02gbfas.fsf@smart-cactus.org> Message-ID: <87h6xkb1qv.fsf@smart-cactus.org> David Feuer writes: > Does this release include the fix for #22549 (infinite loops for some > undecidable instances)? > Yes, it includes a backport of !9485. Cheers, - Ben -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From david.feuer at gmail.com Sun Dec 25 03:35:36 2022 From: david.feuer at gmail.com (David Feuer) Date: Sat, 24 Dec 2022 22:35:36 -0500 Subject: [Haskell-cafe] [ANNOUNCE] GHC 9.4.4 is now available In-Reply-To: <87h6xkb1qv.fsf@smart-cactus.org> References: <87k02gbfas.fsf@smart-cactus.org> <87h6xkb1qv.fsf@smart-cactus.org> Message-ID: Excellent! On Sat, Dec 24, 2022, 10:29 PM Ben Gamari wrote: > David Feuer writes: > > > Does this release include the fix for #22549 (infinite loops for some > > undecidable instances)? > > > Yes, it includes a backport of !9485. > > Cheers, > > - Ben > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From hecate at glitchbra.in Tue Dec 27 17:09:59 2022 From: hecate at glitchbra.in (=?UTF-8?Q?H=c3=a9cate?=) Date: Tue, 27 Dec 2022 18:09:59 +0100 Subject: Deprecating Safe Haskell, or heavily investing in it? Message-ID: <4b5304a6-f035-b72c-0816-d450d9347ea7@glitchbra.in> Hi everyone, and happy holidays. I am looking into whether or not Safe Haskell is still worth maintaining. Currently there are two sides on which Safe Haskell hurts us: 1. GHC Development 2. Library development For point n°1: You can easily take the measure of what Safe Haskell raises in GHC as of today by visiting the bug tracker¹ and see that for example Unboxed Types cannot be used because their home modules (GHC.Prim and GHC.Types) are marked as Unsafe. Moreover, interactions between GHC2021 and Safe make the latter quite unpleasant to work with². Regarding point n°2: Safe Haskell is badly integrated within our existing frameworks for API compatibility. Neither the PVP nor the extension's documentation mention compatibility, or define what stance we should take. Thus we are bound to fight on the letter versus the spirit of the PVP (which is not an unreasonable debate since there is no formalism anywhere). This provokes debates without clear resolution beyond "Friend don't let friends use {-# Safe #-}"³ Now I am not saying that Safe Haskell does not bring any kind of good idea, far from it. However there are two things that live inside Safe Haskell that would benefit from being separated: 1. Strict type-safety 2. Restricted IO A lot of the public use-cases of Safe Haskell seem to be on the "Restricted IO", such as Lambdabot and other code evaluators out there. It's a fairly reasonable feature, I'd even say it's something that we should be publicising more when speaking about GHC's more advanced  features. However, "Strict type-safety" seems to be the root of many problems that we encounter downstream. A lot of it stems from GeneralizedNewtypeDeriving being marked as unsafe (under point 1), which is fair enough, but we've had DerivingVia not marked as Safe until May 2021, which reveals a big problem: Options have to be marked as forbidden under Safe to be caught, which leaves a lot of work to the human factor of GHC development. Now, there are two options (convenient!) that are left to us: 1. Deprecate Safe Haskell: We remove the Safe mechanism as it exists today, and keep the IO restriction under another name. This will certainly cause much joy amongst maintainers and GHC developers alike. The downside is that we don't have a mechanism to enforce "Strict type-safety" anymore. 2. We heavily invest in Safe Haskell: This is the option where we amend the PVP to take changes of Safety annotations into account, invest in workforce to fix the bugs on the GHC side. Which means we also invest in the tools that check for PVP compatibility to check for Safety. This is not the matter of a GSoC, or a 2-days hackathon, and I would certainly have remorse sending students to the salt mines like that. I do not list the Status Quo as an option because it is terrible and has led us to regularly have complaints from both GHC & Ecosystem libraries maintainers. There can be no half-measures that they usually tend to make us slide back into the status quo. So, what do you think? ¹ https://gitlab.haskell.org/ghc/ghc/-/issues/?label_name%5B%5D=Safe%20Haskell ² https://gitlab.haskell.org/ghc/ghc/-/issues/19605 ³ https://github.com/haskell/directory/issues/147 -- Hécate ✨ 🐦: @TechnoEmpress IRC: Hecate WWW: https://glitchbra.in RUN: BSD From ietf-dane at dukhovni.org Tue Dec 27 20:12:44 2022 From: ietf-dane at dukhovni.org (Viktor Dukhovni) Date: Tue, 27 Dec 2022 15:12:44 -0500 Subject: Deprecating Safe Haskell, or heavily investing in it? In-Reply-To: <4b5304a6-f035-b72c-0816-d450d9347ea7@glitchbra.in> References: <4b5304a6-f035-b72c-0816-d450d9347ea7@glitchbra.in> Message-ID: On Tue, Dec 27, 2022 at 06:09:59PM +0100, Hécate wrote: > Now, there are two options (convenient!) that are left to us: > > 1. Deprecate Safe Haskell: We remove the Safe mechanism as it exists > today, and keep the IO restriction under another name. This will > certainly cause much joy amongst maintainers and GHC developers alike. > The downside is that we don't have a mechanism to enforce "Strict > type-safety" anymore. > > 2. We heavily invest in Safe Haskell: This is the option where we amend > the PVP to take changes of Safety annotations into account, invest in > workforce to fix the bugs on the GHC side. Which means we also invest in > the tools that check for PVP compatibility to check for Safety. This is > not the matter of a GSoC, or a 2-days hackathon, and I would certainly > have remorse sending students to the salt mines like that. > > I do not list the Status Quo as an option because it is terrible and has > led us to regularly have complaints from both GHC & Ecosystem libraries > maintainers. There can be no half-measures that they usually tend to > make us slide back into the status quo. > > So, what do you think? I think that "Restricted IO" would in principle be the more sensible approach. HOWEVER, for robust "sandboxing" of untrusted code what's required is more than just hiding the raw IO Monad from the sandboxed code. Doing that securely is much too difficult to do correctly, as evidenced by the ultimate failure (long history of bypass issues) of similar efforts for enabling restricted execution of untrusted code in Java (anyone still using Java "applets", or running Flash in their browser???). The only way to do this correctly is to provide strong memory separation between the untrusted code and the TCB. The only mainstream working examples of this that I know of are: * Kernel vs. user space memory separation. * Tcl's multiple interpreters, where untrusted code runs in slave interpreters stripped of most verbs, with aliases added to wrappers that call back into the parent interpreter for argument validation and restricted execution. Both systems provide strong memory isolation of untrusted code, only data passes between the untrusted code and the TCB through a limited set of callbacks (system calls if you like). For "Safe Haskell" to really be *safe*, memory access from untrusted code would need to be "virtualised", with a separate heap and foreign memory allocator for evaluation of untrusted code, and the RTS rewriting and restricting all direct memory access. This means that "peek" and "poke" et. al. would not directly read memory, but rather be restricted to specific address ranges allocated to the untrusted task. Essentially the RTS would have to become a user-space microkernel. This is in principle possible, but it is not clear whether this is worth doing, given limited resources. To achieve "safe" execution, restricted code needs to give up some runtime performance, just compile-time safety checks are not sufficiently robust in practice. For example, the underlying byte arrays (pinned or not) behind ByteString and Text when used from untrusted code would not allow access to data beyond the array bounds (range checked on every access), ... which again speaks to some "virtualisation" of memory access by the RTS, at least to the extent of always performing range checks when running untrusted code. Bottom line, I don't trust systems like Safe Haskell, or Java's type-system-based sandboxing of untrusted code, ... that try to perform sandboxing in a shared address space by essentially static analysis alone. We've long left shared address space security systems DOS and MacOS 9 behind... good riddance. -- Viktor. From hecate at glitchbra.in Tue Dec 27 20:39:22 2022 From: hecate at glitchbra.in (=?UTF-8?Q?H=c3=a9cate?=) Date: Tue, 27 Dec 2022 21:39:22 +0100 Subject: Deprecating Safe Haskell, or heavily investing in it? In-Reply-To: References: <4b5304a6-f035-b72c-0816-d450d9347ea7@glitchbra.in> Message-ID: <5e716326-6a23-3776-b1ee-4eba796fc65c@glitchbra.in> Thanks for your input Viktor! I came across the nsjail system from Google a little while after posting this thread: https://github.com/google/nsjail/#overview Perhaps we could get the most value for our buck if we externalise the solution to work with OS-level mechanisms? What do you think of that? Something based upon eBPF would certainly incur less modifications to the RTS? Le 27/12/2022 à 21:12, Viktor Dukhovni a écrit : > On Tue, Dec 27, 2022 at 06:09:59PM +0100, Hécate wrote: > >> Now, there are two options (convenient!) that are left to us: >> >> 1. Deprecate Safe Haskell: We remove the Safe mechanism as it exists >> today, and keep the IO restriction under another name. This will >> certainly cause much joy amongst maintainers and GHC developers alike. >> The downside is that we don't have a mechanism to enforce "Strict >> type-safety" anymore. >> >> 2. We heavily invest in Safe Haskell: This is the option where we amend >> the PVP to take changes of Safety annotations into account, invest in >> workforce to fix the bugs on the GHC side. Which means we also invest in >> the tools that check for PVP compatibility to check for Safety. This is >> not the matter of a GSoC, or a 2-days hackathon, and I would certainly >> have remorse sending students to the salt mines like that. >> >> I do not list the Status Quo as an option because it is terrible and has >> led us to regularly have complaints from both GHC & Ecosystem libraries >> maintainers. There can be no half-measures that they usually tend to >> make us slide back into the status quo. >> >> So, what do you think? > I think that "Restricted IO" would in principle be the more sensible > approach. HOWEVER, for robust "sandboxing" of untrusted code what's > required is more than just hiding the raw IO Monad from the sandboxed > code. Doing that securely is much too difficult to do correctly, as > evidenced by the ultimate failure (long history of bypass issues) of > similar efforts for enabling restricted execution of untrusted code in > Java (anyone still using Java "applets", or running Flash in their > browser???). > > The only way to do this correctly is to provide strong memory separation > between the untrusted code and the TCB. The only mainstream working > examples of this that I know of are: > > * Kernel vs. user space memory separation. > > * Tcl's multiple interpreters, where untrusted code runs in > slave interpreters stripped of most verbs, with aliases > added to wrappers that call back into the parent interpreter > for argument validation and restricted execution. > > Both systems provide strong memory isolation of untrusted code, only > data passes between the untrusted code and the TCB through a limited > set of callbacks (system calls if you like). > > For "Safe Haskell" to really be *safe*, memory access from untrusted > code would need to be "virtualised", with a separate heap and foreign > memory allocator for evaluation of untrusted code, and the RTS rewriting > and restricting all direct memory access. This means that "peek" and > "poke" et. al. would not directly read memory, but rather be restricted > to specific address ranges allocated to the untrusted task. > > Essentially the RTS would have to become a user-space microkernel. > > This is in principle possible, but it is not clear whether this is worth > doing, given limited resources. > > To achieve "safe" execution, restricted code needs to give up some > runtime performance, just compile-time safety checks are not > sufficiently robust in practice. For example, the underlying byte > arrays (pinned or not) behind ByteString and Text when used from > untrusted code would not allow access to data beyond the array bounds > (range checked on every access), ... which again speaks to some > "virtualisation" of memory access by the RTS, at least to the extent of > always performing range checks when running untrusted code. > > Bottom line, I don't trust systems like Safe Haskell, or Java's > type-system-based sandboxing of untrusted code, ... that try to perform > sandboxing in a shared address space by essentially static analysis > alone. We've long left shared address space security systems DOS and > MacOS 9 behind... good riddance. > -- Hécate ✨ 🐦: @TechnoEmpress IRC: Hecate WWW: https://glitchbra.in RUN: BSD From b.gohla at gmx.de Tue Dec 27 20:48:47 2022 From: b.gohla at gmx.de (=?utf-8?Q?Bj=C3=B6rn?= Gohla) Date: Tue, 27 Dec 2022 20:48:47 +0000 Subject: Deprecating Safe Haskell, or heavily investing in it? In-Reply-To: References: <4b5304a6-f035-b72c-0816-d450d9347ea7@glitchbra.in> Message-ID: <87cz848s01.fsf@gmx.de> Viktor Dukhovni writes: > On Tue, Dec 27, 2022 at 06:09:59PM +0100, Hécate wrote: > >> Now, there are two options (convenient!) that are left to us: >> >> 1. Deprecate Safe Haskell: We remove the Safe mechanism as it exists >> today, and keep the IO restriction under another name. This will >> certainly cause much joy amongst maintainers and GHC developers alike. >> The downside is that we don't have a mechanism to enforce "Strict >> type-safety" anymore. >> >> 2. We heavily invest in Safe Haskell: This is the option where we amend >> the PVP to take changes of Safety annotations into account, invest in >> workforce to fix the bugs on the GHC side. Which means we also invest in >> the tools that check for PVP compatibility to check for Safety. This is >> not the matter of a GSoC, or a 2-days hackathon, and I would certainly >> have remorse sending students to the salt mines like that. >> >> I do not list the Status Quo as an option because it is terrible and has >> led us to regularly have complaints from both GHC & Ecosystem libraries >> maintainers. There can be no half-measures that they usually tend to >> make us slide back into the status quo. >> >> So, what do you think? > > I think that "Restricted IO" would in principle be the more sensible > approach. HOWEVER, for robust "sandboxing" of untrusted code what's > required is more than just hiding the raw IO Monad from the sandboxed > code. Doing that securely is much too difficult to do correctly, as > evidenced by the ultimate failure (long history of bypass issues) of > similar efforts for enabling restricted execution of untrusted code in > Java (anyone still using Java "applets", or running Flash in their > browser???). > > The only way to do this correctly is to provide strong memory separation > between the untrusted code and the TCB. The only mainstream working > examples of this that I know of are: > > * Kernel vs. user space memory separation. > > * Tcl's multiple interpreters, where untrusted code runs in > slave interpreters stripped of most verbs, with aliases > added to wrappers that call back into the parent interpreter > for argument validation and restricted execution. > > Both systems provide strong memory isolation of untrusted code, only > data passes between the untrusted code and the TCB through a limited > set of callbacks (system calls if you like). > > For "Safe Haskell" to really be *safe*, memory access from untrusted > code would need to be "virtualised", with a separate heap and foreign > memory allocator for evaluation of untrusted code, and the RTS rewriting > and restricting all direct memory access. This means that "peek" and > "poke" et. al. would not directly read memory, but rather be restricted > to specific address ranges allocated to the untrusted task. > > Essentially the RTS would have to become a user-space microkernel. > > This is in principle possible, but it is not clear whether this is worth > doing, given limited resources. [...] The other possibility is to break up your application into functional components, that run in separate processes and communicate through channels. That doesn't require any special RTS support, but it requires some infratructure, e.g., to automatically derive message types from function signatures, and to manage the subprocesses. (Ideally, one would also restrict the available kernel interface of each component using, e.g., seccomp on Linux or pledge on OpenBSD) From jaro.reinders at gmail.com Tue Dec 27 21:31:07 2022 From: jaro.reinders at gmail.com (Jaro Reinders) Date: Tue, 27 Dec 2022 22:31:07 +0100 Subject: Deprecating Safe Haskell, or heavily investing in it? In-Reply-To: References: <4b5304a6-f035-b72c-0816-d450d9347ea7@glitchbra.in> Message-ID: <18B4EBC3-75BF-4EE9-9D9D-01C84992D3FC@gmail.com> The bytestring package does have run time bounds checks. So maybe Safe Haskell is safer than you think? On December 27, 2022 9:12:44 PM GMT+01:00, Viktor Dukhovni wrote: >On Tue, Dec 27, 2022 at 06:09:59PM +0100, Hécate wrote: > >> Now, there are two options (convenient!) that are left to us: >> >> 1. Deprecate Safe Haskell: We remove the Safe mechanism as it exists >> today, and keep the IO restriction under another name. This will >> certainly cause much joy amongst maintainers and GHC developers alike. >> The downside is that we don't have a mechanism to enforce "Strict >> type-safety" anymore. >> >> 2. We heavily invest in Safe Haskell: This is the option where we amend >> the PVP to take changes of Safety annotations into account, invest in >> workforce to fix the bugs on the GHC side. Which means we also invest in >> the tools that check for PVP compatibility to check for Safety. This is >> not the matter of a GSoC, or a 2-days hackathon, and I would certainly >> have remorse sending students to the salt mines like that. >> >> I do not list the Status Quo as an option because it is terrible and has >> led us to regularly have complaints from both GHC & Ecosystem libraries >> maintainers. There can be no half-measures that they usually tend to >> make us slide back into the status quo. >> >> So, what do you think? > >I think that "Restricted IO" would in principle be the more sensible >approach. HOWEVER, for robust "sandboxing" of untrusted code what's >required is more than just hiding the raw IO Monad from the sandboxed >code. Doing that securely is much too difficult to do correctly, as >evidenced by the ultimate failure (long history of bypass issues) of >similar efforts for enabling restricted execution of untrusted code in >Java (anyone still using Java "applets", or running Flash in their >browser???). > >The only way to do this correctly is to provide strong memory separation >between the untrusted code and the TCB. The only mainstream working >examples of this that I know of are: > > * Kernel vs. user space memory separation. > > * Tcl's multiple interpreters, where untrusted code runs in > slave interpreters stripped of most verbs, with aliases > added to wrappers that call back into the parent interpreter > for argument validation and restricted execution. > >Both systems provide strong memory isolation of untrusted code, only >data passes between the untrusted code and the TCB through a limited >set of callbacks (system calls if you like). > >For "Safe Haskell" to really be *safe*, memory access from untrusted >code would need to be "virtualised", with a separate heap and foreign >memory allocator for evaluation of untrusted code, and the RTS rewriting >and restricting all direct memory access. This means that "peek" and >"poke" et. al. would not directly read memory, but rather be restricted >to specific address ranges allocated to the untrusted task. > >Essentially the RTS would have to become a user-space microkernel. > >This is in principle possible, but it is not clear whether this is worth >doing, given limited resources. > >To achieve "safe" execution, restricted code needs to give up some >runtime performance, just compile-time safety checks are not >sufficiently robust in practice. For example, the underlying byte >arrays (pinned or not) behind ByteString and Text when used from >untrusted code would not allow access to data beyond the array bounds >(range checked on every access), ... which again speaks to some >"virtualisation" of memory access by the RTS, at least to the extent of >always performing range checks when running untrusted code. > >Bottom line, I don't trust systems like Safe Haskell, or Java's >type-system-based sandboxing of untrusted code, ... that try to perform >sandboxing in a shared address space by essentially static analysis >alone. We've long left shared address space security systems DOS and >MacOS 9 behind... good riddance. > >-- > Viktor. >_______________________________________________ >ghc-devs mailing list >ghc-devs at haskell.org >http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs From ietf-dane at dukhovni.org Tue Dec 27 23:03:30 2022 From: ietf-dane at dukhovni.org (Viktor Dukhovni) Date: Tue, 27 Dec 2022 18:03:30 -0500 Subject: Deprecating Safe Haskell, or heavily investing in it? In-Reply-To: <18B4EBC3-75BF-4EE9-9D9D-01C84992D3FC@gmail.com> References: <4b5304a6-f035-b72c-0816-d450d9347ea7@glitchbra.in> <18B4EBC3-75BF-4EE9-9D9D-01C84992D3FC@gmail.com> Message-ID: On Tue, Dec 27, 2022 at 10:31:07PM +0100, Jaro Reinders wrote: > The bytestring package does have run time bounds checks. So maybe Safe > Haskell is safer than you think? No. The safety depends on careful Safe/Unsafe marking of an unmanageable and growing set of modules. How does GHC know that "Data.ByteString.Unsafe" is actually "unsafe" in the sense of "Safe" Haskell? λ> BS.index x 10 *** Exception: Data.ByteString.index: index too large: 10, length = 6 CallStack (from HasCallStack): error, called at libraries/bytestring/Data/ByteString.hs:2026:23 in bytestring-0.11.3.1:Data.ByteString moduleError, called at libraries/bytestring/Data/ByteString.hs:1232:24 in bytestring-0.11.3.1:Data.ByteString index, called at :7:1 in interactive:Ghci3 λ> import Data.ByteString.Unsafe as UBS λ> UBS.unsafeIndex x 30000 27 λ> UBS.unsafeIndex x 1000000 162 λ> UBS.unsafeIndex x 10000000 185 λ> UBS.unsafeIndex x 100000000 Segmentation fault (core dumped) This is too brittle to be safe on an ongoing basis in practice. -- Viktor. From ietf-dane at dukhovni.org Tue Dec 27 23:08:04 2022 From: ietf-dane at dukhovni.org (Viktor Dukhovni) Date: Tue, 27 Dec 2022 18:08:04 -0500 Subject: Deprecating Safe Haskell, or heavily investing in it? In-Reply-To: <5e716326-6a23-3776-b1ee-4eba796fc65c@glitchbra.in> References: <4b5304a6-f035-b72c-0816-d450d9347ea7@glitchbra.in> <5e716326-6a23-3776-b1ee-4eba796fc65c@glitchbra.in> Message-ID: On Tue, Dec 27, 2022 at 09:39:22PM +0100, Hécate wrote: > I came across the nsjail system from Google a little while after posting > this thread: https://github.com/google/nsjail/#overview Yes, this is the sort of thing that one can begin to trust, provided that the exposed capabalities are managed only by inclusion, all system calls, filesystem namespaces, network namespaces, ... that are not explicitly allowed are denied. > Perhaps we could get the most value for our buck if we externalise the > solution to work with OS-level mechanisms? What do you think of that? > Something based upon eBPF would certainly incur less modifications to > the RTS? Indeed, it would be simpler to leverage existing virtualisation and/or containerisation technologies, than build a new microkernel within the RTS. Consequently, I guess I am saying that "Safe Haskell" was an interesting research project, but may be a practical dead-end. -- Viktor. From iavor.diatchki at gmail.com Wed Dec 28 00:13:28 2022 From: iavor.diatchki at gmail.com (Iavor Diatchki) Date: Tue, 27 Dec 2022 16:13:28 -0800 Subject: Deprecating Safe Haskell, or heavily investing in it? In-Reply-To: References: <4b5304a6-f035-b72c-0816-d450d9347ea7@glitchbra.in> <5e716326-6a23-3776-b1ee-4eba796fc65c@glitchbra.in> Message-ID: Hello, I disagree that Safe Haskell is a failed experiment, and I think with a little work could be a very valuable tool for auditing Haskell source code. The main change I think we should make is to completely remove the source code annotations, and instead expose an external mechanism (e.g. some sort of file) for specifying which potentially unsafe modules one trusts and which modules should be safe under those assumptions. Then GHC can do the checking and inference just like now, and people can make their own safety configurations depending on the threat model. Iavor On Tue, Dec 27, 2022, 17:08 Viktor Dukhovni wrote: > On Tue, Dec 27, 2022 at 09:39:22PM +0100, Hécate wrote: > > > I came across the nsjail system from Google a little while after posting > > this thread: https://github.com/google/nsjail/#overview > > Yes, this is the sort of thing that one can begin to trust, provided > that the exposed capabalities are managed only by inclusion, all system > calls, filesystem namespaces, network namespaces, ... that are not > explicitly allowed are denied. > > > Perhaps we could get the most value for our buck if we externalise the > > solution to work with OS-level mechanisms? What do you think of that? > > Something based upon eBPF would certainly incur less modifications to > > the RTS? > > Indeed, it would be simpler to leverage existing virtualisation and/or > containerisation technologies, than build a new microkernel within the > RTS. Consequently, I guess I am saying that "Safe Haskell" was an > interesting research project, but may be a practical dead-end. > > -- > Viktor. > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > -------------- next part -------------- An HTML attachment was scrubbed... URL: From b.gohla at gmx.de Wed Dec 28 00:23:56 2022 From: b.gohla at gmx.de (b.gohla) Date: Wed, 28 Dec 2022 00:23:56 +0000 Subject: Deprecating Safe Haskell, or heavily investing in it? In-Reply-To: <5e716326-6a23-3776-b1ee-4eba796fc65c@glitchbra.in> Message-ID: <1N63VY-1om6JV17rE-016L0O@mail.gmx.net> I have implemented something like that actually: https://github.com/cgohla/pledgeWorking out a portable API could be difficult.Sent from my Galaxy -------- Original message --------From: Hécate Date: 27/12/22 20:39 (GMT+00:00) To: ghc-devs at haskell.org Subject: Re: Deprecating Safe Haskell, or heavily investing in it? Thanks for your input Viktor!I came across the nsjail system from Google a little while after posting this thread: https://github.com/google/nsjail/#overviewPerhaps we could get the most value for our buck if we externalise the solution to work with OS-level mechanisms?What do you think of that? Something based upon eBPF would certainly incur less modifications to the RTS?Le 27/12/2022 à 21:12, Viktor Dukhovni a écrit :> On Tue, Dec 27, 2022 at 06:09:59PM +0100, Hécate wrote:>>> Now, there are two options (convenient!) that are left to us:>>>> 1. Deprecate Safe Haskell: We remove the Safe mechanism as it exists>> today, and keep the IO restriction under another name. This will>> certainly cause much joy amongst maintainers and GHC developers alike.>> The downside is that we don't have a mechanism to enforce "Strict>> type-safety" anymore.>>>> 2. We heavily invest in Safe Haskell: This is the option where we amend>> the PVP to take changes of Safety annotations into account, invest in>> workforce to fix the bugs on the GHC side. Which means we also invest in>> the tools that check for PVP compatibility to check for Safety. This is>> not the matter of a GSoC, or a 2-days hackathon, and I would certainly>> have remorse sending students to the salt mines like that.>>>> I do not list the Status Quo as an option because it is terrible and has>> led us to regularly have complaints from both GHC & Ecosystem libraries>> maintainers. There can be no half-measures that they usually tend to>> make us slide back into the status quo.>>>> So, what do you think?> I think that "Restricted IO" would in principle be the more sensible> approach.  HOWEVER, for robust "sandboxing" of untrusted code what's> required is more than just hiding the raw IO Monad from the sandboxed> code.  Doing that securely is much too difficult to do correctly, as> evidenced by the ultimate failure (long history of bypass issues) of> similar efforts for enabling restricted execution of untrusted code in> Java (anyone still using Java "applets", or running Flash in their> browser???).>> The only way to do this correctly is to provide strong memory separation> between the untrusted code and the TCB.  The only mainstream working> examples of this that I know of are:>>      * Kernel vs. user space memory separation.>>      * Tcl's multiple interpreters, where untrusted code runs in>        slave interpreters stripped of most verbs, with aliases>        added to wrappers that call back into the parent interpreter>        for argument validation and restricted execution.>> Both systems provide strong memory isolation of untrusted code, only> data passes between the untrusted code and the TCB through a limited> set of callbacks (system calls if you like).>> For "Safe Haskell" to really be *safe*, memory access from untrusted> code would need to be "virtualised", with a separate heap and foreign> memory allocator for evaluation of untrusted code, and the RTS rewriting> and restricting all direct memory access.  This means that "peek" and> "poke" et. al. would not directly read memory, but rather be restricted> to specific address ranges allocated to the untrusted task.>> Essentially the RTS would have to become a user-space microkernel.>> This is in principle possible, but it is not clear whether this is worth> doing, given limited resources.>> To achieve "safe" execution, restricted code needs to give up some> runtime performance, just compile-time safety checks are not> sufficiently robust in practice.  For example, the underlying byte> arrays (pinned or not) behind ByteString and Text when used from> untrusted code would not allow access to data beyond the array bounds> (range checked on every access), ...  which again speaks to some> "virtualisation" of memory access by the RTS, at least to the extent of> always performing range checks when running untrusted code.>> Bottom line, I don't trust systems like Safe Haskell, or Java's> type-system-based sandboxing of untrusted code, ... that try to perform> sandboxing in a shared address space by essentially static analysis> alone.  We've long left shared address space security systems DOS and> MacOS 9 behind... good riddance.>-- Hécate ✨🐦: @TechnoEmpressIRC: HecateWWW: https://glitchbra.inRUN: BSD_______________________________________________ghc-devs mailing listghc-devs at haskell.orghttp://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs -------------- next part -------------- An HTML attachment was scrubbed... URL: From cdsmith at gmail.com Wed Dec 28 03:33:04 2022 From: cdsmith at gmail.com (Chris Smith) Date: Tue, 27 Dec 2022 20:33:04 -0700 Subject: Deprecating Safe Haskell, or heavily investing in it? In-Reply-To: <4b5304a6-f035-b72c-0816-d450d9347ea7@glitchbra.in> References: <4b5304a6-f035-b72c-0816-d450d9347ea7@glitchbra.in> Message-ID: Thanks for writing this up! I was a user of SafeHaskell briefly in 2011-2012, and it certainly has theoretical benefits. That said, though, I agree that the balance of the cost-benefit analysis is solidly against it, and I would much rather ease the burden of GHC and library development than continue to maintain a burdensome niche feature with limited users. That said, I think some of the comments in this thread are unnecessarily dismissive of SafeHaskell's usefulness and correctness. So here are a few things worth keeping in mind: 1. The strict type safety aspect of SafeHaskell is, in fact, the mechanism that makes SafeHaskell effective at controlling effects through limited IO access. You simply cannot have the latter without the former. Generalized newtype deriving, for example, is incompatible with SafeHaskell precisely because it can be used to break restricted IO access. It looks like a lot of the responses here ignore that SafeHaskell did a lot of work to make runtime-cost-free safe execution reliable, then make hand-wavy references to other systems that don't do any of that, and use that as a basis for asserting distrust in SafeHaskell. That's entirely unfair. 2. The assertion that safety in SafeHaskell depends on the correctness of a bunch of libraries assertions about their safety reveals a fundamental lack of understanding of how SafeHaskell works. Safety is inferred compositionally from the source code and the safety of the modules it uses, and a source code annotation about safety can never compromise the safety guarantees of SafeHaskell. The only thing that can compromise safety is a misplaced trust decision. Trust decisions are not in the source code at all. They are made outside the source by running `ghc-pkg trust`. This conversation reminds me of a parable I encountered somewhere, in which someone declares "I don't understand why this decision was ever made, and I we should change it", and someone responds, "No, if you don't understand the decision was made, then you don't know enough to change it. If you learn why it was decided that way in the first place, then you will have the understanding to decide whether to change it." Just like in the parable, I am not disagreeing with the idea of deprecating SafeHaskell, but I am skeptical that the decision should be made on the basis of misrepresentations and crude analogies that completely miss what's unique and interesting about SafeHaskell in the first place. On Tue, Dec 27, 2022, 10:10 AM Hécate wrote: > Hi everyone, and happy holidays. > > I am looking into whether or not Safe Haskell is still worth maintaining. > > Currently there are two sides on which Safe Haskell hurts us: > > 1. GHC Development > 2. Library development > > For point n°1: You can easily take the measure of what Safe Haskell > raises in GHC as of today by visiting the bug tracker¹ and see that for > example Unboxed Types cannot be used because their home modules > (GHC.Prim and GHC.Types) are marked as Unsafe. Moreover, interactions > between GHC2021 and Safe make the latter quite unpleasant to work with². > > Regarding point n°2: Safe Haskell is badly integrated within our > existing frameworks for API compatibility. Neither the PVP nor the > extension's documentation mention compatibility, or define what stance > we should take. Thus we are bound to fight on the letter versus the > spirit of the PVP (which is not an unreasonable debate since there is no > formalism anywhere). This provokes debates without clear resolution > beyond "Friend don't let friends use {-# Safe #-}"³ > > Now I am not saying that Safe Haskell does not bring any kind of good > idea, far from it. > > However there are two things that live inside Safe Haskell that would > benefit from being separated: > > 1. Strict type-safety > > 2. Restricted IO > > A lot of the public use-cases of Safe Haskell seem to be on the > "Restricted IO", such as Lambdabot and other code evaluators out there. > > It's a fairly reasonable feature, I'd even say it's something that we > should be publicising more when speaking about GHC's more advanced > features. However, "Strict type-safety" seems to be the root of many > problems that we encounter downstream. A lot of it stems from > GeneralizedNewtypeDeriving being marked as unsafe (under point 1), which > is fair enough, but we've had DerivingVia not marked as Safe until May > 2021, which reveals a big problem: Options have to be marked as > forbidden under Safe to be caught, which leaves a lot of work to the > human factor of GHC development. > > Now, there are two options (convenient!) that are left to us: > > 1. Deprecate Safe Haskell: We remove the Safe mechanism as it exists > today, and keep the IO restriction under another name. This will > certainly cause much joy amongst maintainers and GHC developers alike. > The downside is that we don't have a mechanism to enforce "Strict > type-safety" anymore. > > 2. We heavily invest in Safe Haskell: This is the option where we amend > the PVP to take changes of Safety annotations into account, invest in > workforce to fix the bugs on the GHC side. Which means we also invest in > the tools that check for PVP compatibility to check for Safety. This is > not the matter of a GSoC, or a 2-days hackathon, and I would certainly > have remorse sending students to the salt mines like that. > > I do not list the Status Quo as an option because it is terrible and has > led us to regularly have complaints from both GHC & Ecosystem libraries > maintainers. There can be no half-measures that they usually tend to > make us slide back into the status quo. > > So, what do you think? > > > ¹ > > https://gitlab.haskell.org/ghc/ghc/-/issues/?label_name%5B%5D=Safe%20Haskell > > ² https://gitlab.haskell.org/ghc/ghc/-/issues/19605 > > ³ https://github.com/haskell/directory/issues/147 > > > -- > Hécate ✨ > 🐦: @TechnoEmpress > IRC: Hecate > WWW: https://glitchbra.in > RUN: BSD > > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > -------------- next part -------------- An HTML attachment was scrubbed... URL: From howard.b.golden at gmail.com Wed Dec 28 05:14:39 2022 From: howard.b.golden at gmail.com (howard.b.golden at gmail.com) Date: Tue, 27 Dec 2022 21:14:39 -0800 Subject: Deprecating Safe Haskell, or heavily investing in it? In-Reply-To: References: <4b5304a6-f035-b72c-0816-d450d9347ea7@glitchbra.in> Message-ID: <96e7be19a2064fe9dc73f223a8aa18361ee30a15.camel@gmail.com> I hope that some effort can be expended to specifically identify what changes would be necessary to make Safe Haskell work consistent with its name. Only then can a reasoned decision about how to proceed be made. Also, rejecting the status quo implies that making a decision is a high priority. Is it really that urgent? If Safe Haskell isn't being used much, then why hurry to fix it or kill it? On Tue, 2022-12-27 at 20:33 -0700, Chris Smith wrote: > Thanks for writing this up! > > I was a user of SafeHaskell briefly in 2011-2012, and it certainly > has theoretical benefits.  That said, though, I agree that the > balance of the cost-benefit analysis is solidly against it, and I > would much rather ease the burden of GHC and library development than > continue to maintain a burdensome niche feature with limited users. > > That said, I think some of the comments in this thread are > unnecessarily dismissive of SafeHaskell's usefulness and > correctness.  So here are a few things worth keeping in mind: > > 1. The strict type safety aspect of SafeHaskell is, in fact, the > mechanism that makes SafeHaskell effective at controlling effects > through limited IO access.  You simply cannot have the latter without > the former.  Generalized newtype deriving, for example, is > incompatible with SafeHaskell precisely because it can be used to > break restricted IO access. It looks like a lot of the responses here > ignore that SafeHaskell did a lot of work to make runtime-cost-free > safe execution reliable, then make hand-wavy references to other > systems that don't do any of that, and use that as a basis for > asserting distrust in SafeHaskell.  That's entirely unfair. > 2. The assertion that safety in SafeHaskell depends on the > correctness of a bunch of libraries assertions about their safety > reveals a fundamental lack of understanding of how SafeHaskell > works.  Safety is inferred compositionally from the source code and > the safety of the modules it uses, and a source code annotation about > safety can never compromise the safety guarantees of SafeHaskell.  > The only thing that can compromise safety is a misplaced trust > decision.  Trust decisions are not in the source code at all.  They > are made outside the source by running `ghc-pkg trust`. > > This conversation reminds me of a parable I encountered somewhere, in > which someone declares "I don't understand why this decision was ever > made, and I we should change it", and someone responds, "No, if you > don't understand the decision was made, then you don't know enough to > change it.  If you learn why it was decided that way in the first > place, then you will have the understanding to decide whether to > change it."  Just like in the parable, I am not disagreeing with the > idea of deprecating SafeHaskell, but I am skeptical that the decision > should be made on the basis of misrepresentations and crude analogies > that completely miss what's unique and interesting about SafeHaskell > in the first place.  > > On Tue, Dec 27, 2022, 10:10 AM Hécate wrote: > > Hi everyone, and happy holidays. > > > > I am looking into whether or not Safe Haskell is still worth > > maintaining. > > > > Currently there are two sides on which Safe Haskell hurts us: > > > > 1. GHC Development > > 2. Library development > > > > For point n°1: You can easily take the measure of what Safe Haskell > > raises in GHC as of today by visiting the bug tracker¹ and see that > > for > > example Unboxed Types cannot be used because their home modules > > (GHC.Prim and GHC.Types) are marked as Unsafe. Moreover, > > interactions > > between GHC2021 and Safe make the latter quite unpleasant to work > > with². > > > > Regarding point n°2: Safe Haskell is badly integrated within our > > existing frameworks for API compatibility. Neither the PVP nor the > > extension's documentation mention compatibility, or define what > > stance > > we should take. Thus we are bound to fight on the letter versus the > > spirit of the PVP (which is not an unreasonable debate since there > > is no > > formalism anywhere). This provokes debates without clear resolution > > beyond "Friend don't let friends use {-# Safe #-}"³ > > > > Now I am not saying that Safe Haskell does not bring any kind of > > good > > idea, far from it. > > > > However there are two things that live inside Safe Haskell that > > would > > benefit from being separated: > > > > 1. Strict type-safety > > > > 2. Restricted IO > > > > A lot of the public use-cases of Safe Haskell seem to be on the > > "Restricted IO", such as Lambdabot and other code evaluators out > > there. > > > > It's a fairly reasonable feature, I'd even say it's something that > > we > > should be publicising more when speaking about GHC's more advanced  > > features. However, "Strict type-safety" seems to be the root of > > many > > problems that we encounter downstream. A lot of it stems from > > GeneralizedNewtypeDeriving being marked as unsafe (under point 1), > > which > > is fair enough, but we've had DerivingVia not marked as Safe until > > May > > 2021, which reveals a big problem: Options have to be marked as > > forbidden under Safe to be caught, which leaves a lot of work to > > the > > human factor of GHC development. > > > > Now, there are two options (convenient!) that are left to us: > > > > 1. Deprecate Safe Haskell: We remove the Safe mechanism as it > > exists > > today, and keep the IO restriction under another name. This will > > certainly cause much joy amongst maintainers and GHC developers > > alike. > > The downside is that we don't have a mechanism to enforce "Strict > > type-safety" anymore. > > > > 2. We heavily invest in Safe Haskell: This is the option where we > > amend > > the PVP to take changes of Safety annotations into account, invest > > in > > workforce to fix the bugs on the GHC side. Which means we also > > invest in > > the tools that check for PVP compatibility to check for Safety. > > This is > > not the matter of a GSoC, or a 2-days hackathon, and I would > > certainly > > have remorse sending students to the salt mines like that. > > > > I do not list the Status Quo as an option because it is terrible > > and has > > led us to regularly have complaints from both GHC & Ecosystem > > libraries > > maintainers. There can be no half-measures that they usually tend > > to > > make us slide back into the status quo. > > > > So, what do you think? > > > > > > ¹ > > https://gitlab.haskell.org/ghc/ghc/-/issues/?label_name%5B%5D=Safe%20Haskell > > > > ² https://gitlab.haskell.org/ghc/ghc/-/issues/19605 > > > > ³ https://github.com/haskell/directory/issues/147 > > > > > > _______________________________________________ > > ghc-devs mailing list > > ghc-devs at haskell.org > > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs From tom-lists-haskell-cafe-2017 at jaguarpaw.co.uk Wed Dec 28 12:13:49 2022 From: tom-lists-haskell-cafe-2017 at jaguarpaw.co.uk (Tom Ellis) Date: Wed, 28 Dec 2022 12:13:49 +0000 Subject: Deprecating Safe Haskell, or heavily investing in it? In-Reply-To: References: <4b5304a6-f035-b72c-0816-d450d9347ea7@glitchbra.in> Message-ID: On Tue, Dec 27, 2022 at 08:33:04PM -0700, Chris Smith wrote: > This conversation reminds me of a parable I encountered somewhere, in which > someone declares "I don't understand why this decision was ever made, and I > we should change it", and someone responds, "No, if you don't understand > the decision was made, then you don't know enough to change it. If you > learn why it was decided that way in the first place, then you will have > the understanding to decide whether to change it." That parable is Chesterton's fence: https://en.wikipedia.org/wiki/G._K._Chesterton#Chesterton's_fence From davean at xkcd.com Wed Dec 28 23:04:31 2022 From: davean at xkcd.com (davean) Date: Wed, 28 Dec 2022 18:04:31 -0500 Subject: Deprecating Safe Haskell, or heavily investing in it? In-Reply-To: References: <4b5304a6-f035-b72c-0816-d450d9347ea7@glitchbra.in> Message-ID: The only part of Safe Haskell I ever really cared about was type safety. That's what matters, I think. I've wanted to use it a number of times and played with it, but it's never actually managed to become an important part of anything for me. So take that as you will. I'd love it if it worked well, its issues have limited what I attempt, but at the end of the day it's never hurt me too bad to not have it. -davean On Wed, Dec 28, 2022 at 7:14 AM Tom Ellis < tom-lists-haskell-cafe-2017 at jaguarpaw.co.uk> wrote: > On Tue, Dec 27, 2022 at 08:33:04PM -0700, Chris Smith wrote: > > This conversation reminds me of a parable I encountered somewhere, in > which > > someone declares "I don't understand why this decision was ever made, > and I > > we should change it", and someone responds, "No, if you don't understand > > the decision was made, then you don't know enough to change it. If you > > learn why it was decided that way in the first place, then you will have > > the understanding to decide whether to change it." > > That parable is Chesterton's fence: > > https://en.wikipedia.org/wiki/G._K._Chesterton#Chesterton's_fence > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > -------------- next part -------------- An HTML attachment was scrubbed... URL: From clintonmead at gmail.com Fri Dec 30 12:07:18 2022 From: clintonmead at gmail.com (Clinton Mead) Date: Fri, 30 Dec 2022 23:07:18 +1100 Subject: Fwd: Will there be a GHC 9.2.6? In-Reply-To: References: Message-ID: Hi All I just noticed Haskell Language Server (HLS) 1.9 has been released, which supports GHC 9.2.5. My organisation is currently using GHC 9.2.2 and I don't see any immediate need to jump to GHC 9.4, it would be good to settle on the latest most stable version of the GHC 9.2 series as it includes all the features we need at the moment. Also I understand that HLS tends to have long term(ish) support for the latest GHC in a release series. I'd rather not do this upgrade twice so I was just wondering whether it has been decided or not whether there will be a 9.2.6 release (if this is still unknown that's okay). I did notice this page: https://gitlab.haskell.org/ghc/ghc/-/milestones/385 but was unsure whether this page was autogenerated or a deliberate indication that there will be a GHC 9.2.6 at some point. Thanks, Clinton -------------- next part -------------- An HTML attachment was scrubbed... URL: