[GHC] #1009: GHC Performance Index

[GHC]

#1009: GHC Performance Index
---------------------------------+------------------------------------------
    Reporter:  simonmar          |       Owner:         
        Type:  task              |      Status:  new    
    Priority:  normal            |   Milestone:  _|_    
   Component:  Build System      |     Version:  6.6    
    Severity:  normal            |    Keywords:         
  Difficulty:  Moderate (1 day)  |    Testcase:  N/A    
Architecture:  Unknown           |          Os:  Unknown
---------------------------------+------------------------------------------
We want a lightweight way to detect when either (a) the compiler's
 performance changes, or (b) the performance of compiled code changes. This
 was triggerred by my noticing that the HEAD nightly builds seem somewhat
 slower than usual right now.

 So the idea is pretty simple. We decide on a fixed set of nofib programs
 --intiially all of them, but we want the set to remain stable in the
 future -- and as part of the nightly build we time how long it takes to
 compile this set with a fixed set of flags (probably -O -fasm, to rule out
 changes in gcc). Then we time how long it takes to run all those programs,
 maybe 5 times each. Then we keep track of these two figures for every
 successful nightly build, and plot the results, giving us a nice way to
 track GHC's performance over time. We can go back and get some historical
 figures too. Of course the results are specific to a particular machine,
 so we'd have to keep track of results per nightly build machine or
 something.

 So basically: some small changes to nofib to allow "just build everything"
 (I think even NoFibRuns=0 might do this), and to build a fixed set of
 programs (the set could be built into the nofib build system, no need to
 make it dynamic). And some way to collect the results and plot graphs.

-- 
Ticket URL: <http://hackage.haskell.org/trac/ghc/ticket/1009>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler