slow load and typecheck

[Brandon Allbery]

I think the only path for loading a dependency that doesn't involve loading
object code of some kind is the {-# SOURCE #-} hack as part of .hs-boot
files, which isn't general enough to be reused here as I understand it. A
decent chunk of the compiler would need to be duplicated to avoid this, and
it might use a fair amount of memory and end up generating at least part of
the object into memory.

Also recall that if any TH or quasiquotation is involved, it'll need to
load object code in support of that; and it might well need to prepare for
this in the general case rather than again having to duplicate a bunch of
code to support different no-TH and TH paths.

Cabal will build all that stuff the first time and then reuse it the next,
so it's not quite the same thing. Since you told ghc no object code, it
discards what it generates here and may not use existing compiled modules;
or you may have specified settings incompatible with any it did find.

In short, you may want to rethink this; ghc is a compiler, not an IDE, and
doesn't quite work the way you had hoped.

On Tue, Oct 8, 2019 at 10:15 AM Sam Halliday <sam.halliday at gmail.com> wrote:

> Hello all,
>
> I am writing an interactive tool using the ghc api. It is able to load
> and typecheck a source file in a user's package.
>
> I obtain the flags that cabal uses to compile the user's package via the
> hie-bios trick, and I `parseDynamicFlagsCmdLine' them inside my tool,
> then I `setTargets' all the home modules (with targetAllowObjCode=True).
>
> I use HscNothing and NoLink because I only want access to the trees, I
> don't want to produce any output files.
>
> For the file that I wish to inspect, I `removeTarget' the module and
> `addTarget` it again but this time providing the full path to the file
> and don't allow object code.
>
> Then I LoadUpTo and typecheck. For the sake of simplicity, let's assume
> that the file under inspection only has a module definiton and no
> imports or top levels.
>
> Functionally, my code is working great and I am able to do what I want
> with the typechecked tree.
>
> However, load is very slow (~10 seconds user time) on large projects.
> Here is a cpu time trace of my program (milliseconds):
>
>   main              1
>   parse flags      93
>   load          20436
>   typecheck     20437
>
> I can enable a bit more ghc timing info via -Rghc-timings and I see
>
>   !!! Chasing dependencies: finished in 157.20 milliseconds, allocated
>       528.112 megabytes
>
> This seems fine, anything sub-second is ok.
>
> But then I see a bunch of home modules in CodeGen that I was not expecting:
>
>    !!! CodeGen [My.Module.Dependency]:
>        finished in 3335.62 milliseconds, allocated 270.615 megabytes
>
> So it looks like the targetAllowObjCode is being ignored... is there any
> way to force it? Actually I'd prefer to fail fast than to ever compile
> or codegen a dependency module.
>
>
> I know that it should be possible to load the module a lot faster
> because if I make a small change in the file under inspection and ask
> cabal to recompile the module it is super fast (less than a second).
>
> Could somebody who understands how incremental/partial compiles work
> please help me out?
>
>
> PS: If this textual description is confusing, I could put together a
> minimal reproduction and example project but it will take me some time
> to do that.
>
> --
> Best regards,
> Sam
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>


-- 
brandon s allbery kf8nh
allbery.b at gmail.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191008/5daa5ebb/attachment.html>