whole program optimization

[Simon Peyton-Jones]

The GHC API ought to make this relatively easy, but sadly doesn't at the moment.  What you'd want:

- read in each module in turn, typecheck, desugar, perform modest simplification, and keep the resulting Core bindings
- now glom all those bindings together and optimize
- code generate, write out a .o file

There is absolutely no difficulty in principle, but it'd require care.   For example, do we write out one .hi file or many?  If one, what should its name be?  If many, which bindings should end up in which file?  Similarly for the .o file.

How would you control it, so that your multi-module project got compiled in the groups you wanted?  Of course one group would be best, but you might run out of memory...

It'd be an interesting project.  Some transformations (e.g. defunctionalisation) only work if you can guarantee to see the whole program.  But many others should just work better when you can see more of the program at once.

Simon



| -----Original Message-----
| From: glasgow-haskell-users-bounces at haskell.org [mailto:glasgow-haskell-
| users-bounces at haskell.org] On Behalf Of Neil Mitchell
| Sent: 23 December 2007 12:48
| To: Isaac Dupree
| Cc: GHC Users
| Subject: Re: whole program optimization
|
| Hi Isaac,
|
| I've had similar thoughts before. Once GHC can read and write external
| Core this should become a half hour hack. I did a similar thing in
| Yhc, and it really is trivial
| (http://darcs.haskell.org/yhc/src/compiler98/Core/Linker.hs). I'm sure
| there would be a few corner cases, but those kind of things can easily
| be patched over.
|
| I think there would be speed improvements, but not as dramatic as for
| something that was designed to do whole-program from the outset.
|
| Thanks
|
| Neil - who really wants external Core
|
|
| On 12/23/07, Isaac Dupree <isaacdupree at charter.net> wrote:
| > GHC optimizes less effectively when parts of a program are in different
| > modules, or are exported from a module.  Therefore, merging all the code
| > into one "module Main (main)" would help.  Unfortunately, Haskell source
| > syntax is ill-suited to this; for example:
| >
| > module-boundary monomorphism restriction
| > different defaults in different modules
| > deriving (Show, Read) --for renaming everything that's
| >    -- in different modules, so it doesn't conflict
| > new GHC "{..}" etc. record stuff
| > imports as, unknown import lists....
| >
| > Also, program correctness can be wrongly increased; for example, using
| > instances that were defined in some module imported by Main, but not by
| > the module that uses the instance.  I suppose this might affect
| > overlapping instances somehow?
| >
| > But it seems plausible to merge modules on the GHC Core level, after
| > desugaring and type-checking.  Then the one infelicity would be that
| > orphan RULES could apply in more places (RULES are converted and applied
| > to Core!).  Is this a problem?  Does it already happen when modules
| > import small inlinings from .hi files of modules that don't have access
| > to the RULES?
| >
| > It would not actually be quite "whole-program" as long as all the
| > libraries are compiled separately and only available through their .o
| > and incomplete .hi files.  Which is okay; but it would be nice if they
| > could all also be compiled to the "complete-.hi" that we thought might
| > serve for External Core -- and then one would just need to tell GHC to
| > take all those complete-.hi's and compile them together into one
| > executable.  (except for complications like C _stub files, etc, etc.
| > Also some parts of `base` might be wired-in enough that they need some
| > thought before messing with them.)  Then _any_ function that was only
| > used once would be inlined, and GHC would generally be free to make
| > decisions like that (at the GHC-speed price of having to look at all
| > program and library code in detail) (however the GHC interface to this
| > works, Cabal could probably help automate it somehow?)
| >
| > (Unfortunately, ghc-haskell source syntax doesn't provide an easy way to
| > encode Core, even if we wanted to convert it back into GHC-compilable
| > Haskell, without unsafeCoerce everywhere, I think?)
| >
| > So it's more complicated than it seems on the surface, but it could be
| > done; I only wonder how effective it would be on various programs.  I
| > assume GHC doesn't currently implement any optimizations that can _only_
| > be whole-program, so it would still not be quite like Jhc, though this
| > basic support might allow some such optimizations to be implemented.
| >
| > Isaac
| > _______________________________________________
| > Glasgow-haskell-users mailing list
| > Glasgow-haskell-users at haskell.org
| > http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
| >
| _______________________________________________
| Glasgow-haskell-users mailing list
| Glasgow-haskell-users at haskell.org
| http://www.haskell.org/mailman/listinfo/glasgow-haskell-users