slow load and typecheck

[Evan Laforge]

I'm not sure if I'm doing the same thing as you, but I use a GHC repl
for my program.  It loads a 200-300 modules in under a second, and is
able to reload changed ones dynamically, just like ghci.

The source is https://github.com/elaforge/karya/blob/work/Cmd/ReplGhc.hs,
see 'parse_flags' and its call in 'interpreter'.

The main thing is getting ghc to load the .o files, but if ghci will
do it, then the ghc API will do it.  You just have to get the flags to
be the same, and ghc is pretty opaque about why it doesn't want to
load.  There is a -ddump-something flag but it doesn't say what flags
actually changed.  I actually wound up patching ghc to add that
feature.

On Tue, Oct 8, 2019 at 7:15 AM Sam Halliday <sam.halliday at gmail.com> wrote:
>
> Hello all,
>
> I am writing an interactive tool using the ghc api. It is able to load
> and typecheck a source file in a user's package.
>
> I obtain the flags that cabal uses to compile the user's package via the
> hie-bios trick, and I `parseDynamicFlagsCmdLine' them inside my tool,
> then I `setTargets' all the home modules (with targetAllowObjCode=True).
>
> I use HscNothing and NoLink because I only want access to the trees, I
> don't want to produce any output files.
>
> For the file that I wish to inspect, I `removeTarget' the module and
> `addTarget` it again but this time providing the full path to the file
> and don't allow object code.
>
> Then I LoadUpTo and typecheck. For the sake of simplicity, let's assume
> that the file under inspection only has a module definiton and no
> imports or top levels.
>
> Functionally, my code is working great and I am able to do what I want
> with the typechecked tree.
>
> However, load is very slow (~10 seconds user time) on large projects.
> Here is a cpu time trace of my program (milliseconds):
>
>   main              1
>   parse flags      93
>   load          20436
>   typecheck     20437
>
> I can enable a bit more ghc timing info via -Rghc-timings and I see
>
>   !!! Chasing dependencies: finished in 157.20 milliseconds, allocated
>       528.112 megabytes
>
> This seems fine, anything sub-second is ok.
>
> But then I see a bunch of home modules in CodeGen that I was not expecting:
>
>    !!! CodeGen [My.Module.Dependency]:
>        finished in 3335.62 milliseconds, allocated 270.615 megabytes
>
> So it looks like the targetAllowObjCode is being ignored... is there any
> way to force it? Actually I'd prefer to fail fast than to ever compile
> or codegen a dependency module.
>
>
> I know that it should be possible to load the module a lot faster
> because if I make a small change in the file under inspection and ask
> cabal to recompile the module it is super fast (less than a second).
>
> Could somebody who understands how incremental/partial compiles work
> please help me out?
>
>
> PS: If this textual description is confusing, I could put together a
> minimal reproduction and example project but it will take me some time
> to do that.
>
> --
> Best regards,
> Sam
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs