slow load and typecheck

[Matthew Pickering]

Are you writing interface files (-fwrite-interface)? It makes no sense
for HscInterpreted to be faster than HscNothing.

Cheers,

Matt

On Tue, Oct 8, 2019 at 3:30 PM Sam Halliday <sam.halliday at gmail.com> wrote:
>
> A quick follow-up to this, Rahul Muttinieni gave me some advice to try
> out
>
>   HscInterpreted / LinkInMemory
>
> instead of
>
>   HscNothing / NoLink
>
> and now I am no longer seeing home modules being compiled, and
> everything is a lot faster. Woohoo!
>
>
> But I have no idea why this speeds things up... my code isn't using
> TemplateHaskell so HscNothing should really mean "don't do any codegen".
> Something is causing the HscNothing to be ignored. I'd still really like
> to get to the bottom of this so if anybody knows how the batch compiler
> is able to avoid recompiling home modules then please let me know... I
> would like to continue using HscNothing instead of HscInterpreted.
>
>
> Sam Halliday <sam.halliday at gmail.com> writes:
>
> > Hello all,
> >
> > I am writing an interactive tool using the ghc api. It is able to load
> > and typecheck a source file in a user's package.
> >
> > I obtain the flags that cabal uses to compile the user's package via the
> > hie-bios trick, and I `parseDynamicFlagsCmdLine' them inside my tool,
> > then I `setTargets' all the home modules (with targetAllowObjCode=True).
> >
> > I use HscNothing and NoLink because I only want access to the trees, I
> > don't want to produce any output files.
> >
> > For the file that I wish to inspect, I `removeTarget' the module and
> > `addTarget` it again but this time providing the full path to the file
> > and don't allow object code.
> >
> > Then I LoadUpTo and typecheck. For the sake of simplicity, let's assume
> > that the file under inspection only has a module definiton and no
> > imports or top levels.
> >
> > Functionally, my code is working great and I am able to do what I want
> > with the typechecked tree.
> >
> > However, load is very slow (~10 seconds user time) on large projects.
> > Here is a cpu time trace of my program (milliseconds):
> >
> >   main              1
> >   parse flags      93
> >   load          20436
> >   typecheck     20437
> >
> > I can enable a bit more ghc timing info via -Rghc-timings and I see
> >
> >   !!! Chasing dependencies: finished in 157.20 milliseconds, allocated
> >       528.112 megabytes
> >
> > This seems fine, anything sub-second is ok.
> >
> > But then I see a bunch of home modules in CodeGen that I was not expecting:
> >
> >    !!! CodeGen [My.Module.Dependency]:
> >        finished in 3335.62 milliseconds, allocated 270.615 megabytes
> >
> > So it looks like the targetAllowObjCode is being ignored... is there any
> > way to force it? Actually I'd prefer to fail fast than to ever compile
> > or codegen a dependency module.
> >
> >
> > I know that it should be possible to load the module a lot faster
> > because if I make a small change in the file under inspection and ask
> > cabal to recompile the module it is super fast (less than a second).
> >
> > Could somebody who understands how incremental/partial compiles work
> > please help me out?
> >
> >
> > PS: If this textual description is confusing, I could put together a
> > minimal reproduction and example project but it will take me some time
> > to do that.
> >
> > --
> > Best regards,
> > Sam
>
> --
> Best regards,
> Sam
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs