Implementing a compilation server

[Matthew Pickering]

Hi Facundo

Some pointers...

1. Only put things in the EPS if they are not going to change
throughout the whole compilation
2. Treat everything which can change as a home package
2a. I suppose you have performed your own dependency analysis, so
build your own `ModGraph` and start looking from `load'`, you might
just want to call `upsweep_mod/compileOne'` directly yourself.
2b. I suppose you are NOT targeting 9.4.1 yet, but that will make
things easier as you can use support for multiple home packages,
otherwise you will get into severe difficulties if you load a package
you later want to compile into the EPS. The only thing you can do here
      is restart the compilation session I think.
3. To my knowledge, there is no issue using different -this-unit-id in
the same session. Not sure what errors you have seen.
4. You need to use --make mode rather than -c (oneshot) because
oneshot mode loads all interfaces into the EPS (see point 1)

ghcide is the closest program to this kind of compilation server you
imagine so you can look at how that uses the GHC API.

Cheers,

Matt


On Thu, May 5, 2022 at 1:06 AM Domínguez, Facundo
<facundo.dominguez at tweag.io> wrote:
>
> Dear ghc devs,
>
> I'm using the ghc API to write a compilation server (a.k.a. persistent worker). The idea is to serve requests to compile individual modules. In this fashion, we can compile modules with different compilation flags and yet pay only once for the startup overheads of the compiler.
>
> One challenge of this approach is to reuse as much as possible from the ghc API session/environment from one compilation request to the next, so we save the trouble of reconstructing it each time. This message is to ask for advise on how to better accomplish this reuse.
>
> I tried reusing the whole environment for multiple requests, but I'm conjecturing that this might cause troubles when the requests require building modules with different values of -this-unit-id. Another problem that streams from this is that recompiling a module which defines a type class instance fails because it encounters in the environment the type class instance from the
> previous compilation.
>
> My work-in-progress implementation is here [1]. There appears to be multiple ways to compile a module in the API, so far I have been trying DriverPipeline.compileFile.
>
> My best lead right now is to look for inspiration in how GHCi implements the load command, but this does a sort of --make compilation while I'm going here for the one-shot style.
>
> Thanks in advance,
> Facundo
>
> [1] https://github.com/tweag/rules_haskell/blob/16ba422457ea4daa5dbf40d46327ebcb20588e97/tools/haskell_module_worker/src/Compile.hs#L188
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs