slow load and typecheck

[Sam Halliday]

A quick follow-up to this, Rahul Muttinieni gave me some advice to try
out

  HscInterpreted / LinkInMemory

instead of

  HscNothing / NoLink

and now I am no longer seeing home modules being compiled, and
everything is a lot faster. Woohoo!


But I have no idea why this speeds things up... my code isn't using
TemplateHaskell so HscNothing should really mean "don't do any codegen".
Something is causing the HscNothing to be ignored. I'd still really like
to get to the bottom of this so if anybody knows how the batch compiler
is able to avoid recompiling home modules then please let me know... I
would like to continue using HscNothing instead of HscInterpreted.


Sam Halliday <sam.halliday at gmail.com> writes:

> Hello all,
>
> I am writing an interactive tool using the ghc api. It is able to load
> and typecheck a source file in a user's package.
>
> I obtain the flags that cabal uses to compile the user's package via the
> hie-bios trick, and I `parseDynamicFlagsCmdLine' them inside my tool,
> then I `setTargets' all the home modules (with targetAllowObjCode=True).
>
> I use HscNothing and NoLink because I only want access to the trees, I
> don't want to produce any output files.
>
> For the file that I wish to inspect, I `removeTarget' the module and
> `addTarget` it again but this time providing the full path to the file
> and don't allow object code.
>
> Then I LoadUpTo and typecheck. For the sake of simplicity, let's assume
> that the file under inspection only has a module definiton and no
> imports or top levels.
>
> Functionally, my code is working great and I am able to do what I want
> with the typechecked tree.
>
> However, load is very slow (~10 seconds user time) on large projects.
> Here is a cpu time trace of my program (milliseconds):
>
>   main              1
>   parse flags      93
>   load          20436
>   typecheck     20437
>
> I can enable a bit more ghc timing info via -Rghc-timings and I see
>
>   !!! Chasing dependencies: finished in 157.20 milliseconds, allocated
>       528.112 megabytes
>
> This seems fine, anything sub-second is ok.
>
> But then I see a bunch of home modules in CodeGen that I was not expecting:
>
>    !!! CodeGen [My.Module.Dependency]:
>        finished in 3335.62 milliseconds, allocated 270.615 megabytes
>
> So it looks like the targetAllowObjCode is being ignored... is there any
> way to force it? Actually I'd prefer to fail fast than to ever compile
> or codegen a dependency module.
>
>
> I know that it should be possible to load the module a lot faster
> because if I make a small change in the file under inspection and ask
> cabal to recompile the module it is super fast (less than a second).
>
> Could somebody who understands how incremental/partial compiles work
> please help me out?
>
>
> PS: If this textual description is confusing, I could put together a
> minimal reproduction and example project but it will take me some time
> to do that.
>
> -- 
> Best regards,
> Sam

-- 
Best regards,
Sam
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 194 bytes
Desc: not available
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191008/2064b059/attachment.sig>