From djsamperi at gmail.com Thu May 4 04:12:59 2023 From: djsamperi at gmail.com (Dominick Samperi) Date: Thu, 4 May 2023 00:12:59 -0400 Subject: Contributing packages? Message-ID: I just tried to upload a new Haskell package and ran into some difficulties. If I just try to upload, it fails and says I need to contact somebody in the trustees group to be added to the Uploaders group. If I try to upload a package candidate instead, it says Forbidden, No access for this resource. The Uploading packages help page is not very helpful. Also, I have a Hackage account and I can get to the account management page, and generate an auth key, but I don't know what I can do with that. It seems the package candidate feature is the way to go, since direct uploads are irreversible and more difficult to manage. Any tips would be appreciated. Thanks, Dominick -------------- next part -------------- An HTML attachment was scrubbed... URL: From bryan at haskell.foundation Sun May 7 10:07:45 2023 From: bryan at haskell.foundation (Bryan Richter) Date: Sun, 7 May 2023 13:07:45 +0300 Subject: GHC GitLab change to login: disabling OAuth via GitHub Message-ID: Hi folks, Spammers have been abusing GitHub OAuth to create accounts on gitlab.haskell.org and post spam. Due to team resources, the only good way to stop spammers right now is to manually approve each new user. GitHub OAuth defeats that by automatically approving new users. So, sadly, I have disabled GitHub OAuth for the time being. If you used it to sign up and have trouble logging in now, one of us admins can help you out. -Bryan -------------- next part -------------- An HTML attachment was scrubbed... URL: From jakob.bruenker at gmail.com Sun May 7 12:50:45 2023 From: jakob.bruenker at gmail.com (=?UTF-8?Q?Jakob_Br=C3=BCnker?=) Date: Sun, 7 May 2023 14:50:45 +0200 Subject: GHC GitLab change to login: disabling OAuth via GitHub In-Reply-To: References: Message-ID: Hi, for what it's worth, as someone who has been using GitHub OAuth, it looks like using the password reset functionality with one's github email address is enough to be able to log in. Best, Jakob On Sun, May 7, 2023 at 12:08 PM Bryan Richter via ghc-devs < ghc-devs at haskell.org> wrote: > Hi folks, > > Spammers have been abusing GitHub OAuth to create accounts on > gitlab.haskell.org and post spam. > > Due to team resources, the only good way to stop spammers right now is to > manually approve each new user. GitHub OAuth defeats that by automatically > approving new users. > > So, sadly, I have disabled GitHub OAuth for the time being. If you used it > to sign up and have trouble logging in now, one of us admins can help you > out. > > -Bryan > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bryan at haskell.foundation Sun May 7 13:00:01 2023 From: bryan at haskell.foundation (Bryan Richter) Date: Sun, 7 May 2023 16:00:01 +0300 Subject: GHC GitLab change to login: disabling OAuth via GitHub In-Reply-To: References: Message-ID: Hi Jakob, thanks for the confirmation. I've had one report that the password reset email ended up in the spam folder, so be on the lookout for that. -Bryan On Sun, 7 May 2023 at 15:51, Jakob Brünker wrote: > Hi, > > for what it's worth, as someone who has been using GitHub OAuth, it looks > like using the password reset functionality with one's github email address > is enough to be able to log in. > > Best, > Jakob > > On Sun, May 7, 2023 at 12:08 PM Bryan Richter via ghc-devs < > ghc-devs at haskell.org> wrote: > >> Hi folks, >> >> Spammers have been abusing GitHub OAuth to create accounts on >> gitlab.haskell.org and post spam. >> >> Due to team resources, the only good way to stop spammers right now is to >> manually approve each new user. GitHub OAuth defeats that by automatically >> approving new users. >> >> So, sadly, I have disabled GitHub OAuth for the time being. If you used >> it to sign up and have trouble logging in now, one of us admins can help >> you out. >> >> -Bryan >> _______________________________________________ >> ghc-devs mailing list >> ghc-devs at haskell.org >> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From matthewtpickering at gmail.com Fri May 12 08:33:05 2023 From: matthewtpickering at gmail.com (Matthew Pickering) Date: Fri, 12 May 2023 09:33:05 +0100 Subject: Release schedules: 9.6.2 and 9.8.1 Message-ID: Hi all, * We are preparing the 9.6.2 release to happen within 1-2 weeks. * The proposed 9.8.1 schedule is as follows: Fork Date: June 16th Release Date: Early September Feature patches not in the tree by June 16th will not be included in the release. This is the six-month release cadence which we have proposed to follow. Cheers, Matt From sam.halliday at gmail.com Thu May 18 21:05:34 2023 From: sam.halliday at gmail.com (Sam Halliday) Date: Thu, 18 May 2023 22:05:34 +0100 Subject: help migrating a tool that uses the ghc api Message-ID: <87v8gpl59d.fsf@gmail.com> Hi all, A few years ago I wrote a tool (under a pseudonym) that uses the ghc api. I have not been working in Haskell and I've found that it's bitrotted since the 8.10.x series that I was most recently using. The impact of refactors to the ghc codebase have hit me really badly, and with the help of Sylvain, I've been able to get it to the point where it compiles with 9.0.2 and 9.2.7. I'm currently working my way through 9.4.5 support but this is becoming extremely time consuming and draining, so I'm asking for help from anybody who has been involved in these refactors if they could help further. There's only so much git pickaxe can help with, when functions names, types and parameters are moved around in a way that is hard to keep track of. Especially with the changes impacting the way packages, units, errors, and compiler invocation all work. The code is at https://gitlab.com/tseenshe/hsinspect (and also on hackage) and I'm doing all my work-in-progress under the ghc9 branch. As a follow up question, I have a few ideas for how this level of disruption to my codebase could be avoided and I was hoping to get some thoughts on it: 1. some programming language communities have a "community build" that is periodically built by snapshots of the compiler. This allows unexpected regressions to be caught early in the dev cycle and would allow the author of refactor changes to send a courtesy patch to keep the broken code running if the change is intended to be kept in the compiler. I'd like to propose hsinspect for such a community build. 2. propose that my code is merged into the ghc api, so that my code becomes trivial to maintain from that moment on because I've handed the responsibility on, very explicitly, to whoever wishes to make breaking changes in the compiler. I'd like to propose adding all my modules (minus the compiler plugin and machine readable format stuff) to the compiler code tree. are either of these options realistic? I don't think I would be interested in using Haskell anymore if my tooling stopped working: I've invested so much time and energy into it at this point that to start again or have to set my tools aside would be too much of a disincentive. I really love the Haskell language, so I hope that this is not the case. For those interested further in my tool and/or helping... As a quick architectural overview (motivations and goals are described well enough in the README to not need repeating): my tool is very simple: - the plugin is a ghc compiler plugin, all it does is dump out the flags that the batch compiler was invoked with. I understand that HLS has its own solution these days that involves extracting this information from the build tool, but I still prefer getting it from the compiler because it's simple, build tool agnostic, and guaranteed to be correct. I spent many years doing it the build tool route for Scala tooling and I think that was a mistake. - the runtime binary, which is user-invoked (rarely) during their dev cycle, and has several features. This binary is able parse the user's Haskell file's pragmas, on top of the flags that the compiler was invoked with, to extract the exact dflags to use for any interactive compiler usage. The specific features are then: - find all the packages that are depended upon by the home unit. Go and find all the symbols contained therein, including their type signatures. Dump it all out to a machine readable format. - parse just the imports section of the current file to extract the full list of imports in use. Then go and lookup all the symbol names that it implies and their package name, and dump it all out to a machine readable format. - parse the file, find all the data types that the user has defined, and output it into a simpler AST that is designed for code navigation and boilerplate generation tools, comments are not preserved. An example tool that can use this is at https://gitlab.com/tseenshe/boilerplate (and on hackage) On top of these machine readable files I have Emacs tooling that can provide me with all the semantic code editor support that I need, and leaves the door open for some things I haven't implemented yet (such as Hoogle like search within the project and dependencies). Code for the Emacs stuff is at https://gitlab.com/tseenshe/haskell-tng.el under the haskell-tng-hsinspect.el file The approach has some limitations, for example I cannot support RecordDotSyntax because completing on a dot would mean knowing the type under the dot. I also wrote an LSP so that my VSCode colleagues could use it too. Everybody that used it with me was free to use HIE (so called at the time) if they wished. Those of us who used hsinspect preferred it because it's basically zero overhead and is therefore really kind on CPU, RAM and battery life. -- Best regards, Sam -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 227 bytes Desc: not available URL: From vaibhavsagar at gmail.com Fri May 19 02:14:42 2023 From: vaibhavsagar at gmail.com (Vaibhav Sagar) Date: Fri, 19 May 2023 12:14:42 +1000 Subject: help migrating a tool that uses the ghc api In-Reply-To: <87v8gpl59d.fsf@gmail.com> References: <87v8gpl59d.fsf@gmail.com> Message-ID: Hi Sam, I have some experience with GHCI API version migrations from maintaining IHaskell (https://github.com/IHaskell/IHaskell/). During my maintainership I've upgraded to every release of GHC from 8.2 to 9.6 (9 unless I'm counting incorrectly). In my experience, doing an incremental version upgrade takes me a relaxed weekend or less each time (for a project of IHaskell's size), but I can see it being incredibly frustrating if you put your code down for enough time that you have to jump multiple versions. I have a workflow that I'm pretty happy with, which I outline in https://vaibhavsagar.com/blog/2021/05/02/updating-ihaskell-newer-ghc/ (mostly so I don't forget how I did things 6 months later). I'm personally happy with the pace of GHC development and refactors and I see the breaking changes as the necessary cost of improving the codebase and fixing (sometimes long-standing) issues. I think you are asking a lot of the compiler developers here. I don't think that merely consuming an API brings with it an expectation that the API developers should proactively fix your code when they make breaking changes (unless that is clearly outlined at the beginning, which I don't think it ever has been for GHC), and I think this would introduce a lot of friction into the development process which I personally would not want. I think a "community build" would be a great idea, but only for a small handful of projects that have outsized importance to the Haskell ecosystem (e.g. Pandoc, ShellCheck, HLS etc.) which excludes my project (and IMHO yours). I'm sure there are people who are qualified to help you keep your project up-to-date without impacting GHC development or including your project in the codebase. Thanks, Vaibhav On Fri, May 19, 2023 at 7:04 AM Sam Halliday wrote: > Hi all, > > A few years ago I wrote a tool (under a pseudonym) that uses the ghc > api. I have not been working in Haskell and I've found that it's > bitrotted since the 8.10.x series that I was most recently using. > > The impact of refactors to the ghc codebase have hit me really badly, > and with the help of Sylvain, I've been able to get it to the point > where it compiles with 9.0.2 and 9.2.7. I'm currently working my way > through 9.4.5 support but this is becoming extremely time consuming and > draining, so I'm asking for help from anybody who has been involved in > these refactors if they could help further. There's only so much git > pickaxe can help with, when functions names, types and parameters are > moved around in a way that is hard to keep track of. Especially with the > changes impacting the way packages, units, errors, and compiler > invocation all work. > > The code is at https://gitlab.com/tseenshe/hsinspect (and also on > hackage) and I'm doing all my work-in-progress under the ghc9 branch. > > > As a follow up question, I have a few ideas for how this level of > disruption to my codebase could be avoided and I was hoping to get some > thoughts on it: > > 1. some programming language communities have a "community build" that > is periodically built by snapshots of the compiler. This allows > unexpected regressions to be caught early in the dev cycle and would > allow the author of refactor changes to send a courtesy patch to keep > the broken code running if the change is intended to be kept in the > compiler. I'd like to propose hsinspect for such a community build. > > 2. propose that my code is merged into the ghc api, so that my code > becomes trivial to maintain from that moment on because I've handed > the responsibility on, very explicitly, to whoever wishes to make > breaking changes in the compiler. I'd like to propose adding all my > modules (minus the compiler plugin and machine readable format stuff) > to the compiler code tree. > > are either of these options realistic? > > I don't think I would be interested in using Haskell anymore if my > tooling stopped working: I've invested so much time and energy into it > at this point that to start again or have to set my tools aside would be > too much of a disincentive. I really love the Haskell language, so I > hope that this is not the case. > > > For those interested further in my tool and/or helping... > > As a quick architectural overview (motivations and goals are described > well enough in the README to not need repeating): my tool is very > simple: > > - the plugin is a ghc compiler plugin, all it does is dump out the flags > that the batch compiler was invoked with. I understand that HLS has > its own solution these days that involves extracting this information > from the build tool, but I still prefer getting it from the compiler > because it's simple, build tool agnostic, and guaranteed to be > correct. I spent many years doing it the build tool route for Scala > tooling and I think that was a mistake. > > - the runtime binary, which is user-invoked (rarely) during their dev > cycle, and has several features. This binary is able parse the user's > Haskell file's pragmas, on top of the flags that the compiler was > invoked with, to extract the exact dflags to use for any interactive > compiler usage. The specific features are then: > > - find all the packages that are depended upon by the home unit. Go > and find all the symbols contained therein, including their type > signatures. Dump it all out to a machine readable format. > > - parse just the imports section of the current file to extract the > full list of imports in use. Then go and lookup all the symbol names > that it implies and their package name, and dump it all out to a > machine readable format. > > - parse the file, find all the data types that the user has defined, > and output it into a simpler AST that is designed for code > navigation and boilerplate generation tools, comments are not > preserved. An example tool that can use this is at > https://gitlab.com/tseenshe/boilerplate (and on hackage) > > On top of these machine readable files I have Emacs tooling that can > provide me with all the semantic code editor support that I need, and > leaves the door open for some things I haven't implemented yet (such as > Hoogle like search within the project and dependencies). Code for the > Emacs stuff is at https://gitlab.com/tseenshe/haskell-tng.el under the > haskell-tng-hsinspect.el file > > The approach has some limitations, for example I cannot support > RecordDotSyntax because completing on a dot would mean knowing the type > under the dot. > > I also wrote an LSP so that my VSCode colleagues could use it too. > Everybody that used it with me was free to use HIE (so called at the > time) if they wished. Those of us who used hsinspect preferred it > because it's basically zero overhead and is therefore really kind on > CPU, RAM and battery life. > > -- > Best regards, > Sam > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > -------------- next part -------------- An HTML attachment was scrubbed... URL: From simon.peytonjones at gmail.com Fri May 19 08:12:14 2023 From: simon.peytonjones at gmail.com (Simon Peyton Jones) Date: Fri, 19 May 2023 09:12:14 +0100 Subject: help migrating a tool that uses the ghc api In-Reply-To: References: <87v8gpl59d.fsf@gmail.com> Message-ID: I'm sorry you've been having such a torrid time, Sam. We (the GHC developers) are acutely aware of the difficulties surrounding the GHC API. It was discussed quite a bit at the Haskell Implementers Meeting at last year's ICFP. The difficulty is that there simply *is *no defined GHC API. GHC defines many thousands of functions spread across over 700 modules. Of course, we change them all the time! And GHC API users simply reach out and use the most convenient function for them -- but of course we don't know which functions they are using, and so we may change them without warning. And they may be entirely inappropriate "seeing too much of the internals" functions anyway. What we need is - An API for GHC that is *designed *and *actively curated*. - Which clients use with confidence - And GHC devs respect and try to change as little as possible - If a client needs a function that is not in the API, they can bid for it to be added (we'd need a process and, yes, a committee); or they can just reach around and grab one from GHC, but thereby expose themselves to un-managed changes in that function. Even if we had all that, I'd expect the API to shift quite rapidly. For example, the new error-message infrastructure was developed specifically in response to client needs (esp HLS) but those changes necessarily affect all clients, yourself included. At least, with a defined API and process, you might get more warning and migration advice. But to achieve that we need a group of people with the expertise and commitment to create and curate that API. It is possible that the Haskell Foundation will help with this, but meanwhile I'm just sharing the need again in case anyone says "oh I'd love to help with that". This is how open source works: we work together towards common goals. But it does need volunteers, and everyone is busy. Everyone: if you'd like to contribute to a GHC API design and curation process, please do say. Even if you don't feel able to lead it, saying "I'd be happy to help" is a big thing. I know this isn't helping Sam much in the short term -- apologies for that. Simon On Fri, 19 May 2023 at 03:15, Vaibhav Sagar wrote: > Hi Sam, > > I have some experience with GHCI API version migrations from maintaining > IHaskell (https://github.com/IHaskell/IHaskell/). During my > maintainership I've upgraded to every release of GHC from 8.2 to 9.6 (9 > unless I'm counting incorrectly). In my experience, doing an incremental > version upgrade takes me a relaxed weekend or less each time (for a project > of IHaskell's size), but I can see it being incredibly frustrating if you > put your code down for enough time that you have to jump multiple versions. > I have a workflow that I'm pretty happy with, which I outline in > https://vaibhavsagar.com/blog/2021/05/02/updating-ihaskell-newer-ghc/ > (mostly so I don't forget how I did things 6 months later). > > I'm personally happy with the pace of GHC development and refactors and I > see the breaking changes as the necessary cost of improving the codebase > and fixing (sometimes long-standing) issues. > > I think you are asking a lot of the compiler developers here. I don't > think that merely consuming an API brings with it an expectation that the > API developers should proactively fix your code when they make breaking > changes (unless that is clearly outlined at the beginning, which I don't > think it ever has been for GHC), and I think this would introduce a lot of > friction into the development process which I personally would not want. I > think a "community build" would be a great idea, but only for a small > handful of projects that have outsized importance to the Haskell ecosystem > (e.g. Pandoc, ShellCheck, HLS etc.) which excludes my project (and IMHO > yours). > > I'm sure there are people who are qualified to help you keep your project > up-to-date without impacting GHC development or including your project in > the codebase. > > Thanks, > Vaibhav > > On Fri, May 19, 2023 at 7:04 AM Sam Halliday > wrote: > >> Hi all, >> >> A few years ago I wrote a tool (under a pseudonym) that uses the ghc >> api. I have not been working in Haskell and I've found that it's >> bitrotted since the 8.10.x series that I was most recently using. >> >> The impact of refactors to the ghc codebase have hit me really badly, >> and with the help of Sylvain, I've been able to get it to the point >> where it compiles with 9.0.2 and 9.2.7. I'm currently working my way >> through 9.4.5 support but this is becoming extremely time consuming and >> draining, so I'm asking for help from anybody who has been involved in >> these refactors if they could help further. There's only so much git >> pickaxe can help with, when functions names, types and parameters are >> moved around in a way that is hard to keep track of. Especially with the >> changes impacting the way packages, units, errors, and compiler >> invocation all work. >> >> The code is at https://gitlab.com/tseenshe/hsinspect (and also on >> hackage) and I'm doing all my work-in-progress under the ghc9 branch. >> >> >> As a follow up question, I have a few ideas for how this level of >> disruption to my codebase could be avoided and I was hoping to get some >> thoughts on it: >> >> 1. some programming language communities have a "community build" that >> is periodically built by snapshots of the compiler. This allows >> unexpected regressions to be caught early in the dev cycle and would >> allow the author of refactor changes to send a courtesy patch to keep >> the broken code running if the change is intended to be kept in the >> compiler. I'd like to propose hsinspect for such a community build. >> >> 2. propose that my code is merged into the ghc api, so that my code >> becomes trivial to maintain from that moment on because I've handed >> the responsibility on, very explicitly, to whoever wishes to make >> breaking changes in the compiler. I'd like to propose adding all my >> modules (minus the compiler plugin and machine readable format stuff) >> to the compiler code tree. >> >> are either of these options realistic? >> >> I don't think I would be interested in using Haskell anymore if my >> tooling stopped working: I've invested so much time and energy into it >> at this point that to start again or have to set my tools aside would be >> too much of a disincentive. I really love the Haskell language, so I >> hope that this is not the case. >> >> >> For those interested further in my tool and/or helping... >> >> As a quick architectural overview (motivations and goals are described >> well enough in the README to not need repeating): my tool is very >> simple: >> >> - the plugin is a ghc compiler plugin, all it does is dump out the flags >> that the batch compiler was invoked with. I understand that HLS has >> its own solution these days that involves extracting this information >> from the build tool, but I still prefer getting it from the compiler >> because it's simple, build tool agnostic, and guaranteed to be >> correct. I spent many years doing it the build tool route for Scala >> tooling and I think that was a mistake. >> >> - the runtime binary, which is user-invoked (rarely) during their dev >> cycle, and has several features. This binary is able parse the user's >> Haskell file's pragmas, on top of the flags that the compiler was >> invoked with, to extract the exact dflags to use for any interactive >> compiler usage. The specific features are then: >> >> - find all the packages that are depended upon by the home unit. Go >> and find all the symbols contained therein, including their type >> signatures. Dump it all out to a machine readable format. >> >> - parse just the imports section of the current file to extract the >> full list of imports in use. Then go and lookup all the symbol names >> that it implies and their package name, and dump it all out to a >> machine readable format. >> >> - parse the file, find all the data types that the user has defined, >> and output it into a simpler AST that is designed for code >> navigation and boilerplate generation tools, comments are not >> preserved. An example tool that can use this is at >> https://gitlab.com/tseenshe/boilerplate (and on hackage) >> >> On top of these machine readable files I have Emacs tooling that can >> provide me with all the semantic code editor support that I need, and >> leaves the door open for some things I haven't implemented yet (such as >> Hoogle like search within the project and dependencies). Code for the >> Emacs stuff is at https://gitlab.com/tseenshe/haskell-tng.el under the >> haskell-tng-hsinspect.el file >> >> The approach has some limitations, for example I cannot support >> RecordDotSyntax because completing on a dot would mean knowing the type >> under the dot. >> >> I also wrote an LSP so that my VSCode colleagues could use it too. >> Everybody that used it with me was free to use HIE (so called at the >> time) if they wished. Those of us who used hsinspect preferred it >> because it's basically zero overhead and is therefore really kind on >> CPU, RAM and battery life. >> >> -- >> Best regards, >> Sam >> _______________________________________________ >> ghc-devs mailing list >> ghc-devs at haskell.org >> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs >> > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sam.halliday at gmail.com Fri May 19 09:34:37 2023 From: sam.halliday at gmail.com (Sam Halliday) Date: Fri, 19 May 2023 10:34:37 +0100 Subject: help migrating a tool that uses the ghc api In-Reply-To: References: <87v8gpl59d.fsf@gmail.com> Message-ID: <87ilcoll5e.fsf@gmail.com> Thanks Simon, Simon Peyton Jones wrote: > What we need is > - An API for GHC that is *designed *and *actively curated*. > ... Some may believe that the API would need to be some huge reflection of the internal API, with maximal reuse in mind. That is a mammoth task, but an API could also end up being a lot of the code from tools pushed further into the ghc codebase (although perhaps a very inefficient way of doing it for everybody if it's going through committees). I propose an alternative: that tooling authors can submit test code (in the sense that it is compiled as part of the ghc build) to ghc as documentation of their most sensitive uses of the ghc APIs. It wouldn't be distributed and therefore there is no risk to pollution of the ghc api. Breaking changes would require fixing inside the ghc test codebase at the point of the breaking change by the author of the change (who presumably understands it the most!) and then when ghc is released, the tooling author knows exactly what to do to fix their code without having to involve the ghc developers any further, except to extend their thanks. One of my biggest fears is that somebody just *removes* something from the api entirely and I can't do what I need at all anymore. I don't think that kind of problem can be addressed retrospectively. > I know this isn't helping Sam much in the short term -- apologies for that. Bringing attention to it is helping, so thank you for that. My immediate problem can probably be solved by some kind soul dedicating 30 mins of their time to help push my code over the line. I'm more than happy to barter my own time for something they'd like in return, or send a small gift! :-) -- Best regards, Sam -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 227 bytes Desc: not available URL: From tom-lists-haskell-cafe-2017 at jaguarpaw.co.uk Fri May 19 09:43:37 2023 From: tom-lists-haskell-cafe-2017 at jaguarpaw.co.uk (Tom Ellis) Date: Fri, 19 May 2023 10:43:37 +0100 Subject: help migrating a tool that uses the ghc api In-Reply-To: <87v8gpl59d.fsf@gmail.com> References: <87v8gpl59d.fsf@gmail.com> Message-ID: On Thu, May 18, 2023 at 10:05:34PM +0100, Sam Halliday wrote: > 1. some programming language communities have a "community build" that > is periodically built by snapshots of the compiler. This allows > unexpected regressions to be caught early in the dev cycle and would > allow the author of refactor changes to send a courtesy patch to keep > the broken code running if the change is intended to be kept in the > compiler. I'd like to propose hsinspect for such a community build. Hi Sam, there are a couple of ways something like this could work: 1. Team GHC tracks a set of packages and can send fixes ahead of time. 2. Packages track GHC and can fix themselves ahead of time. I believe you're proposing 1, and I think it already exists: http://ghc.gitlab.haskell.org/head.hackage/ I understood that head.hackage ran on the entirety of Hackage so hsinspect[1] should have been picked up by it. I lack a lot of knowledge about head.hackage though, so perhaps someone with more knowledge can chime in. If you're interested in 2 then the Haskell Foundation is working on making nightly builds of GHC accessible. The latest I've heard is from David Christiansen (HF Executive Director) on Discourse: https://discourse.haskell.org/t/haskell-foundation-february-2023-update/5896#requirements-gathering-for-nightly-releases-6 If you're interested I suggest you get in touch with David (perhaps on that thread). Tom [1] https://hackage.haskell.org/package/hsinspect-0.0.18 From sam.halliday at gmail.com Mon May 22 21:00:44 2023 From: sam.halliday at gmail.com (Sam Halliday) Date: Mon, 22 May 2023 22:00:44 +0100 Subject: help migrating a tool that uses the ghc api In-Reply-To: References: <87v8gpl59d.fsf@gmail.com> Message-ID: <877ct0krnn.fsf@gmail.com> Thanks Tom, It looks like head.hackage is for maintainers to opt-in to push patches to accomodate ghc devs. It doesn't look like it's scanning everything and it doesn't look like it's run as part of the ghc build, although I am just trying to gleam this from the README so I could be wrong. The discourse link, and the plan to publish snapshots of the compiler, looks awesome. I'll be following along. I could then compile my code weekly or monthly to try and catch breakages much closer to the time of the change. I'm sure the gitlab CI could be setup to do this automatically, although that may require some investigation. That would help mitigate my fear that the API changes in a way that I can't recover from. Tom Ellis wrote: > On Thu, May 18, 2023 at 10:05:34PM +0100, Sam Halliday wrote: >> 1. some programming language communities have a "community build" that >> is periodically built by snapshots of the compiler. This allows >> unexpected regressions to be caught early in the dev cycle and would >> allow the author of refactor changes to send a courtesy patch to keep >> the broken code running if the change is intended to be kept in the >> compiler. I'd like to propose hsinspect for such a community build. > > Hi Sam, there are a couple of ways something like this could work: > > 1. Team GHC tracks a set of packages and can send fixes ahead of time. > > 2. Packages track GHC and can fix themselves ahead of time. > > > I believe you're proposing 1, and I think it already exists: > > http://ghc.gitlab.haskell.org/head.hackage/ > > I understood that head.hackage ran on the entirety of Hackage so > hsinspect[1] should have been picked up by it. I lack a lot of > knowledge about head.hackage though, so perhaps someone with more > knowledge can chime in. > > > If you're interested in 2 then the Haskell Foundation is working on > making nightly builds of GHC accessible. The latest I've heard is > from David Christiansen (HF Executive Director) on Discourse: > > https://discourse.haskell.org/t/haskell-foundation-february-2023-update/5896#requirements-gathering-for-nightly-releases-6 > > If you're interested I suggest you get in touch with David (perhaps on > that thread). > > Tom > > > [1] https://hackage.haskell.org/package/hsinspect-0.0.18 > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs -- Best regards, Sam -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 227 bytes Desc: not available URL: From sam.halliday at gmail.com Mon May 22 21:14:11 2023 From: sam.halliday at gmail.com (Sam Halliday) Date: Mon, 22 May 2023 22:14:11 +0100 Subject: help migrating a tool that uses the ghc api In-Reply-To: <87ilcoll5e.fsf@gmail.com> References: <87v8gpl59d.fsf@gmail.com> <87ilcoll5e.fsf@gmail.com> Message-ID: <874jo4kr18.fsf@gmail.com> Thanks to everybody who chimed in on this thread. I have now updated ghcflags / hsinspect to work with all versions of ghc from 8.8.3 through to 9.6.1. You can see the churn in various parts of the ghc api in the commit history https://gitlab.com/tseenshe/hsinspect/-/commits/develop ... if the authors of some refactors inside ghc were aware that some of these functions and data types were being used externally then perhaps they could have left functions in place that would have continued to work as before. I'd like to extract this one part from the thread, and ask if there would be any objections to me submitting a standalone test file to the ghc codebase: > tooling authors can submit test code (in the sense that it is compiled > as part of the ghc build) to ghc as documentation of their most > sensitive uses of the ghc APIs. It wouldn't be distributed and > therefore there is no risk to pollution of the ghc api. It will take me some time to do this so I don't want to do it without understanding if it would be accepted. I think it will pay for itself after about 2 or 3 major releases of ghc, but it mitigates against anybody removing core functionality that I'm making use of which is the most important thing. If that is ok, where should I start? I don't know how to add (or run) a test file to ghc that would simply test that the file compiles. Sam Halliday wrote: > Thanks Simon, > > Simon Peyton Jones wrote: >> What we need is >> - An API for GHC that is *designed *and *actively curated*. >> ... > > Some may believe that the API would need to be some huge reflection of > the internal API, with maximal reuse in mind. That is a mammoth task, > but an API could also end up being a lot of the code from tools pushed > further into the ghc codebase (although perhaps a very inefficient way > of doing it for everybody if it's going through committees). > > I propose an alternative: that tooling authors can submit test code (in > the sense that it is compiled as part of the ghc build) to ghc as > documentation of their most sensitive uses of the ghc APIs. It wouldn't > be distributed and therefore there is no risk to pollution of the ghc > api. > > Breaking changes would require fixing inside the ghc test codebase at > the point of the breaking change by the author of the change (who > presumably understands it the most!) and then when ghc is released, the > tooling author knows exactly what to do to fix their code without having > to involve the ghc developers any further, except to extend their > thanks. > > One of my biggest fears is that somebody just *removes* something from > the api entirely and I can't do what I need at all anymore. I don't > think that kind of problem can be addressed retrospectively. > > >> I know this isn't helping Sam much in the short term -- apologies for that. > > Bringing attention to it is helping, so thank you for that. My immediate > problem can probably be solved by some kind soul dedicating 30 mins of > their time to help push my code over the line. I'm more than happy to > barter my own time for something they'd like in return, or send a small > gift! :-) > > > -- > Best regards, > Sam -- Best regards, Sam -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 227 bytes Desc: not available URL: From ben at well-typed.com Tue May 23 21:31:19 2023 From: ben at well-typed.com (Ben Gamari) Date: Tue, 23 May 2023 17:31:19 -0400 Subject: [ANNOUNCE] GHC 9.6.2 is now available Message-ID: <87a5xuohv2.fsf@smart-cactus.org> The GHC developers are happy to announce the availability of GHC 9.6.2. Binary distributions, source distributions, and documentation are available at https://downloads.haskell.org/ghc/9.6.2 This release is primarily a bug-fix release addressing a few issues found in 9.6.2. These include: * a number of simplifier and specialisation issues (#22761, #22549) * A bug resulting in crashes of programs using the new ``listThreads#`` primop (#23071). * A compiler crash triggered by certain uses of quantified constraints (#23171) * Various bugs in the Javascript backend have been fixed (#23399, #23360, #23346) * A missing write barrier in the non-moving collector's handling of selector thunks, resulting in undefined behavior (#22930). * The non-moving garbage collector's treatment of weak pointers has been revamped which should allow more reliable finalization of ``Weak#`` closures (#22327) * The non-moving garbage collector now bounds the amount of marking it will do during the post-marking stop-the-world phase, greatly reducing tail latencies in some programs (#22929) A full accounting of changes can be found in the [release notes]. As some of the fixed issues do affect correctness users are encouraged to upgrade promptly. We would like to thank Microsoft Azure, GitHub, IOG, the Zw3rk stake pool, Well-Typed, Tweag I/O, Serokell, Equinix, SimSpace, Haskell Foundation, and other anonymous contributors whose on-going financial and in-kind support has facilitated GHC maintenance and release management over the years. Finally, this release would not have been possible without the hundreds of open-source contributors whose work comprise this release. As always, do give this release a try and open a [ticket] if you see anything amiss. Happy compiling, ~ Ben [ticket]: https://gitlab.haskell.org/ghc/ghc/-/issues/new [release notes]: https://downloads.haskell.org/~ghc/9.6.2/docs/users_guide/9.6.2-notes.html -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From zubin at well-typed.com Fri May 26 14:06:00 2023 From: zubin at well-typed.com (Zubin Duggal) Date: Fri, 26 May 2023 19:36:00 +0530 Subject: [Haskell] [ANNOUNCE] GHC 9.2.8 released Message-ID: <7g2acy7hyoxggm6qqx6br4bmfwnw6cc3s3yv7f24eebgxzanes@r57zpmb7f3lp> The GHC developers are happy to announce the availability of GHC 9.2.8. Binary distributions, source distributions, and documentation are available at [downloads.haskell.org](https://downloads.haskell.org/ghc/9.2.8). This release is primarily a bugfix release addressing one issue found in 9.2.7: - Fix a bug with RTS linker failing with 'internal error: m32_allocator_init: Failed to map' on newer Linux kernels (#19421). We would like to thank Microsoft Azure, GitHub, IOG, the Zw3rk stake pool, Well-Typed, Tweag I/O, Serokell, Equinix, SimSpace, Haskell Foundation, and other anonymous contributors whose on-going financial and in-kind support has facilitated GHC maintenance and release management over the years. Finally, this release would not have been possible without the hundreds of open-source contributors whose work comprise this release. As always, do give this release a try and open a [ticket][] if you see anything amiss. Happy compiling, - Zubin [ticket]: https://gitlab.haskell.org/ghc/ghc/-/issues/new [release notes]: https://downloads.haskell.org/~ghc/9.2.8/docs/html/users_guide/9.2.8-notes.html -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 488 bytes Desc: not available URL: From lists at qseep.net Wed May 31 03:14:31 2023 From: lists at qseep.net (Lyle Kopnicky) Date: Tue, 30 May 2023 20:14:31 -0700 Subject: Trouble building GHC Message-ID: Hi folks, I’m new here. I’ll be attending the GHC Contributors’ Workshop next week, and in preparation, I’m trying to build GHC, both the native code backend and the JS backend. So far, I’ve only tried to build it with the native code backend, but I haven’t been able to get it to work. I’ve gotten help from some friendly folks on the #ghc channel on Matrix, and made some progress, but I’m still stuck. Is there anyone here who could be a point person for helping me get it to build? BTW I’m located on the west coast of the US (until next week when I’ll be in Switzerland), so time lag may be a factor. I’m using a Mac with aarch64 and macOS 13.3. Here are some of the things I’ve tried, and issues I’ve run into: - Started with the advice from this wiki: https://gitlab.haskell.org/ghc/ghc/-/wikis/building - Checked out the ghc source, on HEAD. - The rest of the steps were at https://gitlab.haskell.org/ghc/ghc/-/wikis/building/preparation/mac-osx - Already had Apple’s command line tools, but when I tried to do operations using them, I got an error saying I needed the full Xcode, so I installed that. - brew install autoconf automake python sphinx-doc - Worked fine, also added sphinx-build to the path - Initially tried using ghc 9.2.7 to build - later tried switching to 9.4.4 and eventually 9.4.5 - cabal update; cabal install alex happy haddock - This is where I ran into trouble - couldn’t build haddock. Cabal said it couldn’t resolve the dependencies. - I tried switching to ghc 9.4.4 and cabal 3.10.1.0 - same problem - User romes (Rodrigo) on #ghc helped with this - was able to reproduce it and filed a ticket: https://github.com/haskell/haddock/issues/1596 - However he pointed out that haddock was already supplied through ghcup so I don’t need to build it. - Already had MacTex installed and I installed the DejaVu font family. - ./boot && ./configure - Worked fine, although ./boot gave me lots of autoconf warnings like: configure.ac:9: warning: The macro `AC_HELP_STRING' is obsolete. configure.ac:9: You should run autoupdate. - Apparently autoconf has renamed these macros to pluralize them, and also encloses the arguments in square brackets. - hadrian/build - Ran into another problem: The build failed with: Error, file does not exist and no rule available: /Users/lyle/devel/haskell/ghc/_build/stage1/libraries/ghc-prim/build/GHC/Classes.hi - Rodrigo helped me out with this. Introduced me to other build flags like -j —flavour-quick. - The issue proved quite persistent, but it might fail on different .hi files. - It turns out if you ask Hadrian to build that specific file, it works. So somehow it can find a rule! Then you can proceed to rebuild and it will fail on a different .hi file. Obviously it would be tedious to do this for all the possible files it can fail on. - I also tried not building profiled libraries. - I tried, instead of using the Apple toolchain, using llvm 12, then llvm 16. But I got different errors from that. - Rodrigo was unable to reproduce the issue. - So, I thought I’d try the Nix approach. - First I had to repair my nix installation, because apparently updating macOS overwrites /etc/zshrc, overwriting the bit that sources '/nix/var/nix/profiles/default/etc/profile.d/nix-daemon.sh’ - Nix reminded me when I ran commands that I needed to add the flag '--extra-experimental-features nix-command’ and sometimes '--extra-experimental-features flakes’ - The instructions from the wiki didn’t work, got an error: - - nix build -f .gitlab/darwin/toolchain.nix -o toolchain.sh --extra-experimental-features nix-command - error: cannot evaluate a function that has an argument without a value ('system’) - But folks on #ghc recommended I use ghc.nix instead - Pointed me to https://ghc.dev/ - That linked to these instructions: https://github.com/alpmestan/ghc.nix#building-ghc - But those instructions failed with: - error: nix-shell requires a single derivation - User Artem said my command was wrong, instead recommended: - nix develop https://github.com/alpmestan/ghc.nix/archive/master.tar.gz - Once I added the magic experimental flags… - It failed, with: - error: NAR hash mismatch in input 'github:commercialhaskell/all-cabal-hashes/02bb1361217e690d83af9cc132b1d2bf2096763c' (/nix/store/zbav3qqbyz7mn7rh4iwybs0ni01ppdbj-source), expected 'sha256-HdAnlSc4U8ftnZrBZr2CewsPQs03V9K2gkTVHKG8IfA=', got 'sha256-86BgvJ+ebMxTp+nPxo659hsNJbhE6CYJWiIbQXX+sBM= - User MangoIV helped me out with this. - First I tried regenerating the lock file, with nix flake update. - Didn’t fix the problem. - Then, I checked out the branch update_flake_lock_action - This is apparently a branch designed to fix the problem - Didn’t fix the problem - Then I regenerated the lock file again, with nix flake update - That worked! Well, at least that got me to a ghc.nix prompt - So, I did: - ./boot - ./configure - cabal v2-update - hadrian/build -j --flavour=quick - It chugged along for quite a while. Sadly, it ended with this error: - - Linking /Users/lyle/Sync/devel/haskell/ghc/hadrian/dist-newstyle/build/aarch64-osx/ghc-9.2.4/hadrian-0.1.0.0/x/hadrian/build/hadrian/hadrian ... Data.Binary.Get.runGet at position 1181: Unknown encoding for constructor - What also worries me is that I thought I read somewhere that ghc.nix doesn’t work for cross-compilers, so even if it works for the native backend, it may not work for the JS backend. Any help would be much appreciated! — Lyle -------------- next part -------------- An HTML attachment was scrubbed... URL: From sgraf1337 at gmail.com Wed May 31 07:13:03 2023 From: sgraf1337 at gmail.com (Sebastian Graf) Date: Wed, 31 May 2023 09:13:03 +0200 Subject: Trouble building GHC In-Reply-To: References: Message-ID: Hi Lyle, I'm sorry that you have so much trouble in getting your first build done. The Classes.hi issue sounds like something I had experienced in the past, but I'm not having it at the moment. Are you also using symlinks by any chance? Then it is very likely that you have been bitten by https://gitlab.haskell.org/ghc/ghc/-/issues/22451, the workaround to which would be to do something like `cd "$(readlink -f .)"` before you start your build. Regarding your second issue using ghc.nix, a quick google turned up https://gitlab.haskell.org/ghc/ghc/-/issues/20429#note_379762. Is it possible that you didn't start from a clean build? E.g., at the least you should `rm -rf _build` (note that `hadrian/cabal clean` sadly is insufficient IIRC for reasons I don't recall). I often simply do `git clean -fxd` to be extra sure. After that, you'll have to boot, configure (including passing $CONFIGURE_ARGS) and build again. By the way, are you using ZSH? I'm using it and I have to pass the CONFIGURE_ARGS in a slightly different way : `./configure ${=CONFIGURE_ARGS}`. I also updated https://gitlab.haskell.org/ghc/ghc/-/wikis/building/preparation/linux#nixnixos to account for new-style flakified builds+direnv, if that's a workflow that you are familiar with. Hope that helps, Sebastian Am Mi., 31. Mai 2023 um 05:14 Uhr schrieb Lyle Kopnicky : > Hi folks, I’m new here. I’ll be attending the GHC Contributors’ Workshop > next week, and in preparation, I’m trying to build GHC, both the native > code backend and the JS backend. So far, I’ve only tried to build it with > the native code backend, but I haven’t been able to get it to work. I’ve > gotten help from some friendly folks on the #ghc channel on Matrix, and > made some progress, but I’m still stuck. > > Is there anyone here who could be a point person for helping me get it to > build? BTW I’m located on the west coast of the US (until next week when > I’ll be in Switzerland), so time lag may be a factor. > > I’m using a Mac with aarch64 and macOS 13.3. Here are some of the things > I’ve tried, and issues I’ve run into: > > > - Started with the advice from this wiki: > https://gitlab.haskell.org/ghc/ghc/-/wikis/building > - Checked out the ghc source, on HEAD. > - The rest of the steps were at > https://gitlab.haskell.org/ghc/ghc/-/wikis/building/preparation/mac-osx > - Already had Apple’s command line tools, but when I tried to do > operations using them, I got an error saying I needed the full Xcode, so I > installed that. > - brew install autoconf automake python sphinx-doc > - Worked fine, also added sphinx-build to the path > - Initially tried using ghc 9.2.7 to build - later tried switching to > 9.4.4 and eventually 9.4.5 > - cabal update; cabal install alex happy haddock > - This is where I ran into trouble - couldn’t build haddock. Cabal > said it couldn’t resolve the dependencies. > - I tried switching to ghc 9.4.4 and cabal 3.10.1.0 - same problem > - User romes (Rodrigo) on #ghc helped with this - was able to > reproduce it and filed a ticket: > https://github.com/haskell/haddock/issues/1596 > - However he pointed out that haddock was already supplied through > ghcup so I don’t need to build it. > - Already had MacTex installed and I installed the DejaVu font family. > - ./boot && ./configure > - Worked fine, although ./boot gave me lots of autoconf warnings > like: > configure.ac:9: warning: The macro `AC_HELP_STRING' is obsolete. > configure.ac:9: You should run autoupdate. > - Apparently autoconf has renamed these macros to pluralize them, > and also encloses the arguments in square brackets. > - hadrian/build > - Ran into another problem: The build failed with: > Error, file does not exist and no rule available: > > /Users/lyle/devel/haskell/ghc/_build/stage1/libraries/ghc-prim/build/GHC/Classes.hi > - Rodrigo helped me out with this. Introduced me to other build > flags like -j —flavour-quick. > - The issue proved quite persistent, but it might fail on different > .hi files. > - It turns out if you ask Hadrian to build that specific file, it > works. So somehow it can find a rule! Then you can proceed to rebuild and > it will fail on a different .hi file. Obviously it would be tedious > to do this for all the possible files it can fail on. > - I also tried not building profiled libraries. > - I tried, instead of using the Apple toolchain, using llvm 12, > then llvm 16. But I got different errors from that. > - Rodrigo was unable to reproduce the issue. > - So, I thought I’d try the Nix approach. > - First I had to repair my nix installation, because apparently > updating macOS overwrites /etc/zshrc, overwriting the bit that sources > '/nix/var/nix/profiles/default/etc/profile.d/nix-daemon.sh’ > - Nix reminded me when I ran commands that I needed to add the > flag '--extra-experimental-features nix-command’ and > sometimes '--extra-experimental-features flakes’ > - The instructions from the wiki > > didn’t work, got an error: > - > - nix build -f .gitlab/darwin/toolchain.nix -o toolchain.sh > --extra-experimental-features nix-command > - error: cannot evaluate a function that has an argument without a > value ('system’) > - But folks on #ghc recommended I use ghc.nix instead > - Pointed me to https://ghc.dev/ > - That linked to these instructions: > https://github.com/alpmestan/ghc.nix#building-ghc > - But those instructions failed with: > - error: nix-shell requires a single derivation > - User Artem said my command was wrong, instead recommended: > - nix develop > https://github.com/alpmestan/ghc.nix/archive/master.tar.gz > - Once I added the magic experimental flags… > - It failed, with: > - error: NAR hash mismatch in input > 'github:commercialhaskell/all-cabal-hashes/02bb1361217e690d83af9cc132b1d2bf2096763c' > (/nix/store/zbav3qqbyz7mn7rh4iwybs0ni01ppdbj-source), expected > 'sha256-HdAnlSc4U8ftnZrBZr2CewsPQs03V9K2gkTVHKG8IfA=', got > 'sha256-86BgvJ+ebMxTp+nPxo659hsNJbhE6CYJWiIbQXX+sBM= > - User MangoIV helped me out with this. > - First I tried regenerating the lock file, with nix flake update. > - Didn’t fix the problem. > - Then, I checked out the branch update_flake_lock_action > - This is apparently a branch designed to fix the problem > - Didn’t fix the problem > - Then I regenerated the lock file again, with nix flake update > - That worked! Well, at least that got me to a ghc.nix prompt > - So, I did: > - ./boot > - ./configure > - cabal v2-update > - hadrian/build -j --flavour=quick > - It chugged along for quite a while. Sadly, it ended with this > error: > - > - Linking > /Users/lyle/Sync/devel/haskell/ghc/hadrian/dist-newstyle/build/aarch64-osx/ghc-9.2.4/hadrian-0.1.0.0/x/hadrian/build/hadrian/hadrian > ... > Data.Binary.Get.runGet at position 1181: Unknown encoding for > constructor > - What also worries me is that I thought I read somewhere that > ghc.nix doesn’t work for cross-compilers, so even if it works for > the native backend, it may not work for the JS backend. > > > Any help would be much appreciated! > — Lyle > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > -------------- next part -------------- An HTML attachment was scrubbed... URL: From godzbanebane at gmail.com Wed May 31 07:49:37 2023 From: godzbanebane at gmail.com (Georgi Lyubenov) Date: Wed, 31 May 2023 10:49:37 +0300 Subject: Trouble building GHC In-Reply-To: References: Message-ID: <5b99bed9-7c03-f547-b941-abd308990ea8@gmail.com> Just chiming in to mention that I'm on zsh, and I've had no issues with the configure step by running the configure_ghc shell function that the flake provides. On 5/31/23 10:13, Sebastian Graf wrote: > Hi Lyle, > > I'm sorry that you have so much trouble in getting your first build done. > The Classes.hi issue sounds like something I had experienced in the > past, but I'm not having it at the moment. > Are you also using symlinks by any chance? Then it is very likely that > you have been bitten by > https://gitlab.haskell.org/ghc/ghc/-/issues/22451, the workaround to > which would be to do something like `cd "$(readlink -f .)"` before you > start your build. > > Regarding your second issue using ghc.nix, a quick google turned up > https://gitlab.haskell.org/ghc/ghc/-/issues/20429#note_379762. > Is it possible that you didn't start from a clean build? > E.g., at the least you should `rm -rf _build` (note that > `hadrian/cabal clean` sadly is insufficient IIRC for reasons I don't > recall). > I often simply do `git clean -fxd` to be extra sure. > After that, you'll have to boot, configure (including passing > $CONFIGURE_ARGS) and build again. > By the way, are you using ZSH? I'm using it and I have topass the > CONFIGURE_ARGS in a slightly different way > : `./configure > ${=CONFIGURE_ARGS}`. > > I also updated > https://gitlab.haskell.org/ghc/ghc/-/wikis/building/preparation/linux#nixnixos > to account for new-style flakified builds+direnv, if that's a workflow > that you are familiar with. > > Hope that helps, > Sebastian > > Am Mi., 31. Mai 2023 um 05:14 Uhr schrieb Lyle Kopnicky : > > Hi folks, I’m new here. I’ll be attending the GHC Contributors’ > Workshop next week, and in preparation, I’m trying to build GHC, > both the native code backend and the JS backend. So far, I’ve only > tried to build it with the native code backend, but I haven’t been > able to get it to work. I’ve gotten help from some friendly folks > on the #ghc channel on Matrix, and made some progress, but I’m > still stuck. > > Is there anyone here who could be a point person for helping me > get it to build? BTW I’m located on the west coast of the US > (until next week when I’ll be in Switzerland), so time lag may be > a factor. > > I’m using a Mac with aarch64 and macOS 13.3. Here are some of the > things I’ve tried, and issues I’ve run into: > > * Started with the advice from this wiki: > https://gitlab.haskell.org/ghc/ghc/-/wikis/building > * Checked out the ghc source, on HEAD. > * The rest of the steps were at > https://gitlab.haskell.org/ghc/ghc/-/wikis/building/preparation/mac-osx > * Already had Apple’s command line tools, but when I tried to do > operations using them, I got an error saying I needed the full > Xcode, so I installed that. > * brew install autoconf automake python sphinx-doc > o Worked fine, also added sphinx-build to the path > * Initially tried using ghc 9.2.7 to build - later tried > switching to 9.4.4 and eventually 9.4.5 > * cabal update; cabal install alex happy haddock > o This is where I ran into trouble - couldn’t build haddock. > Cabal said it couldn’t resolve the dependencies. > o I tried switching to ghc 9.4.4 and cabal 3.10.1.0 - same > problem > o User romes (Rodrigo) on #ghc helped with this - was able > to reproduce it and filed a ticket: > https://github.com/haskell/haddock/issues/1596 > o However he pointed out that haddock was already supplied > through ghcup so I don’t need to build it. > * Already had MacTex installed and I installed the DejaVu font > family. > * ./boot && ./configure > o Worked fine, although |./boot| gave me lots of autoconf > warnings like: > configure.ac:9 : warning: The macro > `AC_HELP_STRING' is obsolete. > configure.ac:9 : You should run > autoupdate. > o Apparently autoconf has renamed these macros to pluralize > them, and also encloses the arguments in square brackets. > * hadrian/build > o Ran into another problem: The build failed with: > Error, file does not exist and no rule available: > /Users/lyle/devel/haskell/ghc/_build/stage1/libraries/ghc-prim/build/GHC/Classes.hi > o Rodrigo helped me out with this. Introduced me to other > build flags like |-j —flavour-quick|. > o The issue proved quite persistent, but it might fail on > different |.hi| files. > o It turns out if you ask Hadrian to build that specific > file, it works. So somehow it can find a rule! Then you > can proceed to rebuild and it will fail on a different > |.hi| file. Obviously it would be tedious to do this for > all the possible files it can fail on. > o I also tried not building profiled libraries. > o I tried, instead of using the Apple toolchain, using llvm > 12, then llvm 16. But I got different errors from that. > o Rodrigo was unable to reproduce the issue. > * So, I thought I’d try the Nix approach. > * First I had to repair my nix installation, because apparently > updating macOS overwrites /etc/zshrc, overwriting the bit that > sources > '/nix/var/nix/profiles/default/etc/profile.d/nix-daemon.sh’ > * Nix reminded me when I ran commands that I needed to add the > flag '--extra-experimental-features nix-command’ and > sometimes '--extra-experimental-features flakes’ > * The instructions from the wiki > > didn’t work, got an error: > o > > o nix build -f .gitlab/darwin/toolchain.nix -o toolchain.sh > --extra-experimental-features nix-command > o error: cannot evaluate a function that has an argument > without a value ('system’) > * But folks on #ghc recommended I use |ghc.nix| instead > o Pointed me to https://ghc.dev/ > o That linked to these instructions: > https://github.com/alpmestan/ghc.nix#building-ghc > o But those instructions failed with: > + error: nix-shell requires a single derivation > o User Artem said my command was wrong, instead recommended: > + nix develop > https://github.com/alpmestan/ghc.nix/archive/master.tar.gz > o Once I added the magic experimental flags… > o It failed, with: > + error: NAR hash mismatch in input > 'github:commercialhaskell/all-cabal-hashes/02bb1361217e690d83af9cc132b1d2bf2096763c' > (/nix/store/zbav3qqbyz7mn7rh4iwybs0ni01ppdbj-source), > expected > 'sha256-HdAnlSc4U8ftnZrBZr2CewsPQs03V9K2gkTVHKG8IfA=', > got 'sha256-86BgvJ+ebMxTp+nPxo659hsNJbhE6CYJWiIbQXX+sBM= > o User MangoIV helped me out with this. > o First I tried regenerating the lock file, with |nix flake > update|. > + Didn’t fix the problem. > o Then, I checked out the branch update_flake_lock_action > + This is apparently a branch designed to fix the problem > + Didn’t fix the problem > o Then I regenerated the lock file again, with |nix flake > update| > + That worked! Well, at least that got me to a > |ghc.nix| prompt > o So, I did: > + ./boot > + ./configure > + cabal v2-update > + hadrian/build -j --flavour=quick > o It chugged along for quite a while. Sadly, it ended with > this error: > + > > + Linking > /Users/lyle/Sync/devel/haskell/ghc/hadrian/dist-newstyle/build/aarch64-osx/ghc-9.2.4/hadrian-0.1.0.0/x/hadrian/build/hadrian/hadrian > ... > Data.Binary.Get.runGet at position 1181: Unknown > encoding for constructor > o What also worries me is that I thought I read somewhere > that |ghc.nix| doesn’t work for cross-compilers, so even > if it works for the native backend, it may not work for > the JS backend. > > > Any help would be much appreciated! > — Lyle > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > > > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs -------------- next part -------------- An HTML attachment was scrubbed... URL: From matthewtpickering at gmail.com Wed May 31 09:16:18 2023 From: matthewtpickering at gmail.com (Matthew Pickering) Date: Wed, 31 May 2023 10:16:18 +0100 Subject: Trouble building GHC In-Reply-To: References: Message-ID: Hi Lyle, It seems that you set-up a proper native toolchain (without nix) and that you might be running into this issue: https://gitlab.haskell.org/ghc/ghc/-/issues/22451 Do you have any symlinks anywhere around the build folder? Cheers, Matt On Wed, May 31, 2023 at 4:14 AM Lyle Kopnicky wrote: > > Hi folks, I’m new here. I’ll be attending the GHC Contributors’ Workshop next week, and in preparation, I’m trying to build GHC, both the native code backend and the JS backend. So far, I’ve only tried to build it with the native code backend, but I haven’t been able to get it to work. I’ve gotten help from some friendly folks on the #ghc channel on Matrix, and made some progress, but I’m still stuck. > > Is there anyone here who could be a point person for helping me get it to build? BTW I’m located on the west coast of the US (until next week when I’ll be in Switzerland), so time lag may be a factor. > > I’m using a Mac with aarch64 and macOS 13.3. Here are some of the things I’ve tried, and issues I’ve run into: > > Started with the advice from this wiki: https://gitlab.haskell.org/ghc/ghc/-/wikis/building > Checked out the ghc source, on HEAD. > The rest of the steps were at https://gitlab.haskell.org/ghc/ghc/-/wikis/building/preparation/mac-osx > Already had Apple’s command line tools, but when I tried to do operations using them, I got an error saying I needed the full Xcode, so I installed that. > brew install autoconf automake python sphinx-doc > > Worked fine, also added sphinx-build to the path > > Initially tried using ghc 9.2.7 to build - later tried switching to 9.4.4 and eventually 9.4.5 > cabal update; cabal install alex happy haddock > > This is where I ran into trouble - couldn’t build haddock. Cabal said it couldn’t resolve the dependencies. > I tried switching to ghc 9.4.4 and cabal 3.10.1.0 - same problem > User romes (Rodrigo) on #ghc helped with this - was able to reproduce it and filed a ticket: https://github.com/haskell/haddock/issues/1596 > However he pointed out that haddock was already supplied through ghcup so I don’t need to build it. > > Already had MacTex installed and I installed the DejaVu font family. > ./boot && ./configure > > Worked fine, although ./boot gave me lots of autoconf warnings like: > configure.ac:9: warning: The macro `AC_HELP_STRING' is obsolete. > configure.ac:9: You should run autoupdate. > Apparently autoconf has renamed these macros to pluralize them, and also encloses the arguments in square brackets. > > hadrian/build > > Ran into another problem: The build failed with: > Error, file does not exist and no rule available: > /Users/lyle/devel/haskell/ghc/_build/stage1/libraries/ghc-prim/build/GHC/Classes.hi > Rodrigo helped me out with this. Introduced me to other build flags like -j —flavour-quick. > The issue proved quite persistent, but it might fail on different .hi files. > It turns out if you ask Hadrian to build that specific file, it works. So somehow it can find a rule! Then you can proceed to rebuild and it will fail on a different .hi file. Obviously it would be tedious to do this for all the possible files it can fail on. > I also tried not building profiled libraries. > I tried, instead of using the Apple toolchain, using llvm 12, then llvm 16. But I got different errors from that. > Rodrigo was unable to reproduce the issue. > > So, I thought I’d try the Nix approach. > First I had to repair my nix installation, because apparently updating macOS overwrites /etc/zshrc, overwriting the bit that sources '/nix/var/nix/profiles/default/etc/profile.d/nix-daemon.sh’ > Nix reminded me when I ran commands that I needed to add the flag '--extra-experimental-features nix-command’ and sometimes '--extra-experimental-features flakes’ > The instructions from the wiki didn’t work, got an error: > > nix build -f .gitlab/darwin/toolchain.nix -o toolchain.sh --extra-experimental-features nix-command > error: cannot evaluate a function that has an argument without a value ('system’) > > But folks on #ghc recommended I use ghc.nix instead > > Pointed me to https://ghc.dev/ > That linked to these instructions: https://github.com/alpmestan/ghc.nix#building-ghc > But those instructions failed with: > > error: nix-shell requires a single derivation > > User Artem said my command was wrong, instead recommended: > > nix develop https://github.com/alpmestan/ghc.nix/archive/master.tar.gz > > Once I added the magic experimental flags… > It failed, with: > > error: NAR hash mismatch in input 'github:commercialhaskell/all-cabal-hashes/02bb1361217e690d83af9cc132b1d2bf2096763c' (/nix/store/zbav3qqbyz7mn7rh4iwybs0ni01ppdbj-source), expected 'sha256-HdAnlSc4U8ftnZrBZr2CewsPQs03V9K2gkTVHKG8IfA=', got 'sha256-86BgvJ+ebMxTp+nPxo659hsNJbhE6CYJWiIbQXX+sBM= > > User MangoIV helped me out with this. > First I tried regenerating the lock file, with nix flake update. > > Didn’t fix the problem. > > Then, I checked out the branch update_flake_lock_action > > This is apparently a branch designed to fix the problem > Didn’t fix the problem > > Then I regenerated the lock file again, with nix flake update > > That worked! Well, at least that got me to a ghc.nix prompt > > So, I did: > > ./boot > ./configure > cabal v2-update > hadrian/build -j --flavour=quick > > It chugged along for quite a while. Sadly, it ended with this error: > > Linking /Users/lyle/Sync/devel/haskell/ghc/hadrian/dist-newstyle/build/aarch64-osx/ghc-9.2.4/hadrian-0.1.0.0/x/hadrian/build/hadrian/hadrian ... > Data.Binary.Get.runGet at position 1181: Unknown encoding for constructor > > What also worries me is that I thought I read somewhere that ghc.nix doesn’t work for cross-compilers, so even if it works for the native backend, it may not work for the JS backend. > > > Any help would be much appreciated! > — Lyle > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs From sgraf1337 at gmail.com Wed May 31 09:41:28 2023 From: sgraf1337 at gmail.com (Sebastian Graf) Date: Wed, 31 May 2023 11:41:28 +0200 Subject: Trouble building GHC In-Reply-To: <5b99bed9-7c03-f547-b941-abd308990ea8@gmail.com> References: <5b99bed9-7c03-f547-b941-abd308990ea8@gmail.com> Message-ID: > I've had no issues with the configure step by running the configure_ghc shell function that the flake provides. For reference, this is the relevant ghc.nix issue: https://github.com/alpmestan/ghc.nix/issues/111 It seems that the `configure_ghc` used to be a shell function in https://github.com/alpmestan/ghc.nix/blob/b200a76a4f28d6434e4678827a0373002e641b12/default.nix#L156 . Nowadays, it became a standalone bash script (great work, Magnus!), which explains why it works for you: https://github.com/alpmestan/ghc.nix/blob/f34c21877257fc37bbcf8962dc006024bfd0f946/ghc.nix#L138 You can't do `./configure $CONFIGURE_ARGS` in zsh directly, though. Am Mi., 31. Mai 2023 um 09:49 Uhr schrieb Georgi Lyubenov < godzbanebane at gmail.com>: > Just chiming in to mention that I'm on zsh, and I've had no issues with > the configure step by running the configure_ghc shell function that the > flake provides. > On 5/31/23 10:13, Sebastian Graf wrote: > > Hi Lyle, > > I'm sorry that you have so much trouble in getting your first build done. > The Classes.hi issue sounds like something I had experienced in the past, > but I'm not having it at the moment. > Are you also using symlinks by any chance? Then it is very likely that you > have been bitten by https://gitlab.haskell.org/ghc/ghc/-/issues/22451, > the workaround to which would be to do something like `cd "$(readlink -f > .)"` before you start your build. > > Regarding your second issue using ghc.nix, a quick google turned up > https://gitlab.haskell.org/ghc/ghc/-/issues/20429#note_379762. > Is it possible that you didn't start from a clean build? > E.g., at the least you should `rm -rf _build` (note that `hadrian/cabal > clean` sadly is insufficient IIRC for reasons I don't recall). > I often simply do `git clean -fxd` to be extra sure. > After that, you'll have to boot, configure (including passing > $CONFIGURE_ARGS) and build again. > By the way, are you using ZSH? I'm using it and I have to pass the > CONFIGURE_ARGS in a slightly different way > : `./configure > ${=CONFIGURE_ARGS}`. > > I also updated > https://gitlab.haskell.org/ghc/ghc/-/wikis/building/preparation/linux#nixnixos > to account for new-style flakified builds+direnv, if that's a workflow that > you are familiar with. > > Hope that helps, > Sebastian > > Am Mi., 31. Mai 2023 um 05:14 Uhr schrieb Lyle Kopnicky : > >> Hi folks, I’m new here. I’ll be attending the GHC Contributors’ Workshop >> next week, and in preparation, I’m trying to build GHC, both the native >> code backend and the JS backend. So far, I’ve only tried to build it with >> the native code backend, but I haven’t been able to get it to work. I’ve >> gotten help from some friendly folks on the #ghc channel on Matrix, and >> made some progress, but I’m still stuck. >> >> Is there anyone here who could be a point person for helping me get it to >> build? BTW I’m located on the west coast of the US (until next week when >> I’ll be in Switzerland), so time lag may be a factor. >> >> I’m using a Mac with aarch64 and macOS 13.3. Here are some of the things >> I’ve tried, and issues I’ve run into: >> >> >> - Started with the advice from this wiki: >> https://gitlab.haskell.org/ghc/ghc/-/wikis/building >> - Checked out the ghc source, on HEAD. >> - The rest of the steps were at >> https://gitlab.haskell.org/ghc/ghc/-/wikis/building/preparation/mac-osx >> - Already had Apple’s command line tools, but when I tried to do >> operations using them, I got an error saying I needed the full Xcode, so I >> installed that. >> - brew install autoconf automake python sphinx-doc >> - Worked fine, also added sphinx-build to the path >> - Initially tried using ghc 9.2.7 to build - later tried switching to >> 9.4.4 and eventually 9.4.5 >> - cabal update; cabal install alex happy haddock >> - This is where I ran into trouble - couldn’t build haddock. Cabal >> said it couldn’t resolve the dependencies. >> - I tried switching to ghc 9.4.4 and cabal 3.10.1.0 - same problem >> - User romes (Rodrigo) on #ghc helped with this - was able to >> reproduce it and filed a ticket: >> https://github.com/haskell/haddock/issues/1596 >> - However he pointed out that haddock was already supplied through >> ghcup so I don’t need to build it. >> - Already had MacTex installed and I installed the DejaVu font family. >> - ./boot && ./configure >> - Worked fine, although ./boot gave me lots of autoconf warnings >> like: >> configure.ac:9: warning: The macro `AC_HELP_STRING' is obsolete. >> configure.ac:9: You should run autoupdate. >> - Apparently autoconf has renamed these macros to pluralize them, >> and also encloses the arguments in square brackets. >> - hadrian/build >> - Ran into another problem: The build failed with: >> Error, file does not exist and no rule available: >> >> /Users/lyle/devel/haskell/ghc/_build/stage1/libraries/ghc-prim/build/GHC/Classes.hi >> - Rodrigo helped me out with this. Introduced me to other build >> flags like -j —flavour-quick. >> - The issue proved quite persistent, but it might fail on >> different .hi files. >> - It turns out if you ask Hadrian to build that specific file, it >> works. So somehow it can find a rule! Then you can proceed to rebuild and >> it will fail on a different .hi file. Obviously it would be >> tedious to do this for all the possible files it can fail on. >> - I also tried not building profiled libraries. >> - I tried, instead of using the Apple toolchain, using llvm 12, >> then llvm 16. But I got different errors from that. >> - Rodrigo was unable to reproduce the issue. >> - So, I thought I’d try the Nix approach. >> - First I had to repair my nix installation, because apparently >> updating macOS overwrites /etc/zshrc, overwriting the bit that sources >> '/nix/var/nix/profiles/default/etc/profile.d/nix-daemon.sh’ >> - Nix reminded me when I ran commands that I needed to add the >> flag '--extra-experimental-features nix-command’ and >> sometimes '--extra-experimental-features flakes’ >> - The instructions from the wiki >> >> didn’t work, got an error: >> - >> - nix build -f .gitlab/darwin/toolchain.nix -o toolchain.sh >> --extra-experimental-features nix-command >> - error: cannot evaluate a function that has an argument without a >> value ('system’) >> - But folks on #ghc recommended I use ghc.nix instead >> - Pointed me to https://ghc.dev/ >> - That linked to these instructions: >> https://github.com/alpmestan/ghc.nix#building-ghc >> - But those instructions failed with: >> - error: nix-shell requires a single derivation >> - User Artem said my command was wrong, instead recommended: >> - nix develop >> https://github.com/alpmestan/ghc.nix/archive/master.tar.gz >> - Once I added the magic experimental flags… >> - It failed, with: >> - error: NAR hash mismatch in input >> 'github:commercialhaskell/all-cabal-hashes/02bb1361217e690d83af9cc132b1d2bf2096763c' >> (/nix/store/zbav3qqbyz7mn7rh4iwybs0ni01ppdbj-source), expected >> 'sha256-HdAnlSc4U8ftnZrBZr2CewsPQs03V9K2gkTVHKG8IfA=', got >> 'sha256-86BgvJ+ebMxTp+nPxo659hsNJbhE6CYJWiIbQXX+sBM= >> - User MangoIV helped me out with this. >> - First I tried regenerating the lock file, with nix flake update. >> - Didn’t fix the problem. >> - Then, I checked out the branch update_flake_lock_action >> - This is apparently a branch designed to fix the problem >> - Didn’t fix the problem >> - Then I regenerated the lock file again, with nix flake update >> - That worked! Well, at least that got me to a ghc.nix prompt >> - So, I did: >> - ./boot >> - ./configure >> - cabal v2-update >> - hadrian/build -j --flavour=quick >> - It chugged along for quite a while. Sadly, it ended with this >> error: >> - >> - Linking >> /Users/lyle/Sync/devel/haskell/ghc/hadrian/dist-newstyle/build/aarch64-osx/ghc-9.2.4/hadrian-0.1.0.0/x/hadrian/build/hadrian/hadrian >> ... >> Data.Binary.Get.runGet at position 1181: Unknown encoding for >> constructor >> - What also worries me is that I thought I read somewhere that >> ghc.nix doesn’t work for cross-compilers, so even if it works for >> the native backend, it may not work for the JS backend. >> >> >> Any help would be much appreciated! >> — Lyle >> _______________________________________________ >> ghc-devs mailing list >> ghc-devs at haskell.org >> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs >> > > _______________________________________________ > ghc-devs mailing listghc-devs at haskell.orghttp://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lists at qseep.net Wed May 31 19:47:42 2023 From: lists at qseep.net (Lyle Kopnicky) Date: Wed, 31 May 2023 12:47:42 -0700 Subject: Trouble building GHC In-Reply-To: References: Message-ID: Hi folks, I managed to get GHC to build with the help of user chreekat on #ghc, who pointed out that the issue I was having with building .hi files had been filed by Richard Eisenberg, and a workaround had been written by Ben Gamari. I cherry-picked that workaround, and it fixed the problem! After that, it only took 13 1/2 minutes to build GHC. I still haven’t gotten it working via nix, but that’s OK - I only needed to get it working one way or the other. Thanks, Lyle > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lists at qseep.net Wed May 31 22:03:05 2023 From: lists at qseep.net (Lyle Kopnicky) Date: Wed, 31 May 2023 15:03:05 -0700 Subject: Trouble building GHC In-Reply-To: References: <5b99bed9-7c03-f547-b941-abd308990ea8@gmail.com> Message-ID: > > Thanks, Georgi and Sebastian. > The instructions at https://github.com/alpmestan/ghc.nix and https://ghc.dev/ seem a bit outdated, then. I’m not sure who maintains those. In the README.md for ghc.nix, it says to run: nix-shell https://github.com/alpmestan/ghc.nix/archive/master.tar.gz --attr devShells..default That failed didn’t work for me and I was told I needed to use nix-develop, which worked. Then later, it says to run: $ ./boot && ./configure $CONFIGURE_ARGS # In zsh, use ${=CONFIGURE_ARGS} …which as you pointed out should be updated to configure_ghc. Anyway, that got me further, but I still get several fatal errors of this sort: compiler/GHC/Tc/Zonk/Type.hs:1277:13: error: • Illegal equational constraint Anno (StmtLR GhcTc GhcTc (LocatedA (body GhcTc))) ~ SrcSpanAnnA (Use GADTs or TypeFamilies to permit this) • In the type signature: zonkStmt :: Anno (StmtLR GhcTc GhcTc (LocatedA (body GhcTc))) ~ SrcSpanAnnA => (LocatedA (body GhcTc) -> ZonkTcM (LocatedA (body GhcTc))) -> Stmt GhcTc (LocatedA (body GhcTc)) -> ZonkBndrTcM (Stmt GhcTc (LocatedA (body GhcTc))) | 1277 | zonkStmt :: Anno (StmtLR GhcTc GhcTc (LocatedA (body GhcTc))) ~ SrcSpanAnnA | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^... Command failed Build failed. Georgi, I might just ping you on Slack about it. (We’re coworkers.) Thanks, Lyle -------------- next part -------------- An HTML attachment was scrubbed... URL: