<div dir="ltr"><div class="gmail_default" style="font-family:tahoma,sans-serif">I'm sorry you've been having such a torrid time, Sam.</div><div class="gmail_default" style="font-family:tahoma,sans-serif"><br></div><div class="gmail_default" style="font-family:tahoma,sans-serif">We (the GHC developers) are acutely aware of the difficulties surrounding the GHC API. It was discussed quite a bit at the Haskell Implementers Meeting at last year's ICFP.</div><div class="gmail_default" style="font-family:tahoma,sans-serif"><br></div><div class="gmail_default" style="font-family:tahoma,sans-serif">The difficulty is that there simply <b>is </b>no defined GHC API. GHC defines many thousands of functions spread across over 700 modules. Of course, we change them all the time! And GHC API users simply reach out and use the most convenient function for them -- but of course we don't know which functions they are using, and so we may change them without warning. And they may be entirely inappropriate "seeing too much of the internals" functions anyway.</div><div class="gmail_default" style="font-family:tahoma,sans-serif"><br></div><div class="gmail_default" style="font-family:tahoma,sans-serif">What we need is</div><div class="gmail_default" style="font-family:tahoma,sans-serif"><ul><li>An API for GHC that is <b>designed </b>and <b>actively curated</b>.</li><li>Which clients use with confidence</li><li>And GHC devs respect and try to change as little as possible</li><li>If a client needs a function that is not in the API, they can bid for it to be added (we'd need a process and, yes, a committee); or they can just reach around and grab one from GHC, but thereby expose themselves to un-managed changes in that function.<br></li></ul><div></div><div></div><div>Even if we had all that, I'd expect the API to shift quite rapidly. For example, the new error-message infrastructure was developed specifically in response to client needs (esp HLS) but those changes necessarily affect all clients, yourself included. At least, with a defined API and process, you might get more warning and migration advice.</div><div><br></div><div>
But to achieve that we need a group of people with the expertise and
commitment to create and curate that API. It is possible that the
Haskell Foundation will help with this, but meanwhile I'm just sharing
the need again in case anyone says "oh I'd love to help with that".
This is how open source works: we work together towards common goals.
But it does need volunteers, and everyone is busy. <br></div><div><br></div><div>Everyone: if you'd like to contribute to a GHC API design and curation process, please do say. Even if you don't feel able to lead it, saying "I'd be happy to help" is a big thing.<br></div><div><br></div><div></div><div>I know this isn't helping Sam much in the short term -- apologies for that.<br></div><div><br></div><div>Simon<br></div></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Fri, 19 May 2023 at 03:15, Vaibhav Sagar <<a href="mailto:vaibhavsagar@gmail.com">vaibhavsagar@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div>Hi Sam,</div><div><br></div><div>I have some experience with GHCI API version migrations from maintaining IHaskell (<a href="https://github.com/IHaskell/IHaskell/" target="_blank">https://github.com/IHaskell/IHaskell/</a>). During my maintainership I've upgraded to every release of GHC from 8.2 to 9.6 (9 unless I'm counting incorrectly). In my experience, doing an incremental version upgrade takes me a relaxed weekend or less each time (for a project of IHaskell's size), but I can see it being incredibly frustrating if you put your code down for enough time that you have to jump multiple versions. I have a workflow that I'm pretty happy with, which I outline in <a href="https://vaibhavsagar.com/blog/2021/05/02/updating-ihaskell-newer-ghc/" target="_blank">https://vaibhavsagar.com/blog/2021/05/02/updating-ihaskell-newer-ghc/</a> (mostly so I don't forget how I did things 6 months later).<br></div><div><br></div><div>I'm personally happy with the pace of GHC development and refactors and I see the breaking changes as the necessary cost of improving the codebase and fixing (sometimes long-standing) issues.</div><div><br></div><div>I think you are asking a lot of the compiler developers here. I don't think that merely consuming an API brings with it an expectation that the API developers should proactively fix your code when they make breaking changes (unless that is clearly outlined at the beginning, which I don't think it ever has been for GHC), and I think this would introduce a lot of friction into the development process which I personally would not want. I think a "community build" would be a great idea, but only for a small handful of projects that have outsized importance to the Haskell ecosystem (e.g. Pandoc, ShellCheck, HLS etc.) which excludes my project (and IMHO yours).</div><div><br></div><div>I'm sure there are people who are qualified to help you keep your project up-to-date without impacting GHC development or including your project in the codebase.</div><div><br></div><div>Thanks,</div><div>Vaibhav<br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Fri, May 19, 2023 at 7:04 AM Sam Halliday <<a href="mailto:sam.halliday@gmail.com" target="_blank">sam.halliday@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Hi all,<br>
<br>
A few years ago I wrote a tool (under a pseudonym) that uses the ghc<br>
api. I have not been working in Haskell and I've found that it's<br>
bitrotted since the 8.10.x series that I was most recently using.<br>
<br>
The impact of refactors to the ghc codebase have hit me really badly,<br>
and with the help of Sylvain, I've been able to get it to the point<br>
where it compiles with 9.0.2 and 9.2.7. I'm currently working my way<br>
through 9.4.5 support but this is becoming extremely time consuming and<br>
draining, so I'm asking for help from anybody who has been involved in<br>
these refactors if they could help further. There's only so much git<br>
pickaxe can help with, when functions names, types and parameters are<br>
moved around in a way that is hard to keep track of. Especially with the<br>
changes impacting the way packages, units, errors, and compiler<br>
invocation all work.<br>
<br>
The code is at <a href="https://gitlab.com/tseenshe/hsinspect" rel="noreferrer" target="_blank">https://gitlab.com/tseenshe/hsinspect</a> (and also on<br>
hackage) and I'm doing all my work-in-progress under the ghc9 branch.<br>
<br>
<br>
As a follow up question, I have a few ideas for how this level of<br>
disruption to my codebase could be avoided and I was hoping to get some<br>
thoughts on it:<br>
<br>
1. some programming language communities have a "community build" that<br>
is periodically built by snapshots of the compiler. This allows<br>
unexpected regressions to be caught early in the dev cycle and would<br>
allow the author of refactor changes to send a courtesy patch to keep<br>
the broken code running if the change is intended to be kept in the<br>
compiler. I'd like to propose hsinspect for such a community build.<br>
<br>
2. propose that my code is merged into the ghc api, so that my code<br>
becomes trivial to maintain from that moment on because I've handed<br>
the responsibility on, very explicitly, to whoever wishes to make<br>
breaking changes in the compiler. I'd like to propose adding all my<br>
modules (minus the compiler plugin and machine readable format stuff)<br>
to the compiler code tree.<br>
<br>
are either of these options realistic?<br>
<br>
I don't think I would be interested in using Haskell anymore if my<br>
tooling stopped working: I've invested so much time and energy into it<br>
at this point that to start again or have to set my tools aside would be<br>
too much of a disincentive. I really love the Haskell language, so I<br>
hope that this is not the case.<br>
<br>
<br>
For those interested further in my tool and/or helping...<br>
<br>
As a quick architectural overview (motivations and goals are described<br>
well enough in the README to not need repeating): my tool is very<br>
simple:<br>
<br>
- the plugin is a ghc compiler plugin, all it does is dump out the flags<br>
that the batch compiler was invoked with. I understand that HLS has<br>
its own solution these days that involves extracting this information<br>
from the build tool, but I still prefer getting it from the compiler<br>
because it's simple, build tool agnostic, and guaranteed to be<br>
correct. I spent many years doing it the build tool route for Scala<br>
tooling and I think that was a mistake.<br>
<br>
- the runtime binary, which is user-invoked (rarely) during their dev<br>
cycle, and has several features. This binary is able parse the user's<br>
Haskell file's pragmas, on top of the flags that the compiler was<br>
invoked with, to extract the exact dflags to use for any interactive<br>
compiler usage. The specific features are then:<br>
<br>
- find all the packages that are depended upon by the home unit. Go<br>
and find all the symbols contained therein, including their type<br>
signatures. Dump it all out to a machine readable format.<br>
<br>
- parse just the imports section of the current file to extract the<br>
full list of imports in use. Then go and lookup all the symbol names<br>
that it implies and their package name, and dump it all out to a<br>
machine readable format.<br>
<br>
- parse the file, find all the data types that the user has defined,<br>
and output it into a simpler AST that is designed for code<br>
navigation and boilerplate generation tools, comments are not<br>
preserved. An example tool that can use this is at<br>
<a href="https://gitlab.com/tseenshe/boilerplate" rel="noreferrer" target="_blank">https://gitlab.com/tseenshe/boilerplate</a> (and on hackage)<br>
<br>
On top of these machine readable files I have Emacs tooling that can<br>
provide me with all the semantic code editor support that I need, and<br>
leaves the door open for some things I haven't implemented yet (such as<br>
Hoogle like search within the project and dependencies). Code for the<br>
Emacs stuff is at <a href="https://gitlab.com/tseenshe/haskell-tng.el" rel="noreferrer" target="_blank">https://gitlab.com/tseenshe/haskell-tng.el</a> under the<br>
haskell-tng-hsinspect.el file<br>
<br>
The approach has some limitations, for example I cannot support<br>
RecordDotSyntax because completing on a dot would mean knowing the type<br>
under the dot.<br>
<br>
I also wrote an LSP so that my VSCode colleagues could use it too.<br>
Everybody that used it with me was free to use HIE (so called at the<br>
time) if they wished. Those of us who used hsinspect preferred it<br>
because it's basically zero overhead and is therefore really kind on<br>
CPU, RAM and battery life.<br>
<br>
-- <br>
Best regards,<br>
Sam<br>
_______________________________________________<br>
ghc-devs mailing list<br>
<a href="mailto:ghc-devs@haskell.org" target="_blank">ghc-devs@haskell.org</a><br>
<a href="http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs" rel="noreferrer" target="_blank">http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs</a><br>
</blockquote></div>
_______________________________________________<br>
ghc-devs mailing list<br>
<a href="mailto:ghc-devs@haskell.org" target="_blank">ghc-devs@haskell.org</a><br>
<a href="http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs" rel="noreferrer" target="_blank">http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs</a><br>
</blockquote></div>