another wash compile problem

[Frederik Eaton]

> When I compile WASH it takes forever for ghc to finish compiling
> HTMLMonad98, and on my powerpc system with 151MB free RAM, gcc crashes
> during the compile. Should some of the larger WASH modules maybe be
> split into multiple files? I get the sense that ghc compilation
> time/memory is super-linear in the size of the module being compiled.

I've now verified my earlier "ghc compilation complexity is
super-linear" belief. I split up HTMLMonad98 into 10 files, and then
split the 6th and largest, which consists of the following lines -

instance AdmitChildHEAD HTML
instance AdmitChildFRAMESET HTML
...(2692 lines)
instance AdmitChildLABEL ACRONYM
instance AdmitChildBUTTON ACRONYM

into 14 files:

$ wc -l HTMLMonad98No6.hs
2696 HTMLMonad98No6.hs
$ wc -l HTMLMonad98No6?.hs
  200 HTMLMonad98No6a.hs
  204 HTMLMonad98No6b.hs
  204 HTMLMonad98No6c.hs
...
 2748 total

Compiling a module which imports the 14 smaller modules takes much
less time than compiling the monolithic module - it's almost 5 times
faster (see below). This is especially strange for this example since
it is just a bunch of empty instance declarations. The direct
conclusion is that splitting up HTMLMonad98 (5849 lines total) should
speed the WASH compile considerably.

But this also makes me wonder what the reason for the super-linearity
in ghc's compilation time is. Could we fix ghc so that compilation
time is mostly linear in the size of modules? Where is the
inefficiency? It would seem that at the very least ghc could do a pass
extracting strongly connected components of the symbol dependency
graph into independent temporary modules, and then compile these
pieces separately.

Frederik

=====

$ time ghc --make HTMLMonad98No6all.hs                      wash*islay:~/WashNGo-2.4.6-modified/WASH
Chasing modules from: HTMLMonad98No6all.hs
Skipping  HTMLMonad98No5   ( ./HTMLMonad98No5.hs, ./HTMLMonad98No5.o )
Skipping  HTMLBase         ( ./HTMLBase.hs, ./HTMLBase.o )
Skipping  HTMLMonadBase    ( ./HTMLMonadBase.hs, ./HTMLMonadBase.o )
Skipping  HTMLMonad        ( ./HTMLMonad.hs, ./HTMLMonad.o )
Skipping  HTMLMonad98No4   ( ./HTMLMonad98No4.hs, ./HTMLMonad98No4.o )
Skipping  HTMLMonad98No3   ( ./HTMLMonad98No3.hs, ./HTMLMonad98No3.o )
Skipping  HTMLMonad98No2   ( ./HTMLMonad98No2.hs, ./HTMLMonad98No2.o )
Skipping  HTMLMonad98No6m  ( ./HTMLMonad98No6m.hs, ./HTMLMonad98No6m.o )
Skipping  HTMLMonad98No6l  ( ./HTMLMonad98No6l.hs, ./HTMLMonad98No6l.o )
Skipping  HTMLMonad98No6k  ( ./HTMLMonad98No6k.hs, ./HTMLMonad98No6k.o )
Skipping  HTMLMonad98No6j  ( ./HTMLMonad98No6j.hs, ./HTMLMonad98No6j.o )
Skipping  HTMLMonad98No6i  ( ./HTMLMonad98No6i.hs, ./HTMLMonad98No6i.o )
Skipping  HTMLMonad98No6h  ( ./HTMLMonad98No6h.hs, ./HTMLMonad98No6h.o )
Skipping  HTMLMonad98No6g  ( ./HTMLMonad98No6g.hs, ./HTMLMonad98No6g.o )
Skipping  HTMLMonad98No6f  ( ./HTMLMonad98No6f.hs, ./HTMLMonad98No6f.o )
Skipping  HTMLMonad98No6e  ( ./HTMLMonad98No6e.hs, ./HTMLMonad98No6e.o )
Skipping  HTMLMonad98No6d  ( ./HTMLMonad98No6d.hs, ./HTMLMonad98No6d.o )
Skipping  HTMLMonad98No6c  ( ./HTMLMonad98No6c.hs, ./HTMLMonad98No6c.o )
Skipping  HTMLMonad98No6b  ( ./HTMLMonad98No6b.hs, ./HTMLMonad98No6b.o )
Skipping  HTMLMonad98No6a  ( ./HTMLMonad98No6a.hs, ./HTMLMonad98No6a.o )
Skipping  HTMLMonad98No6n  ( ./HTMLMonad98No6n.hs, ./HTMLMonad98No6n.o )
Skipping  HTMLMonad98No6all ( HTMLMonad98No6all.hs, HTMLMonad98No6all.o )
ghc --make HTMLMonad98No6all.hs  0.26s user 0.04s system 75% cpu 0.396 total
$ rm -f *.o
$ time ghc --make HTMLMonad98No6all.hs
Chasing modules from: HTMLMonad98No6all.hs
Compiling HTMLMonad98No5   ( ./HTMLMonad98No5.hs, ./HTMLMonad98No5.o )
Compiling HTMLBase         ( ./HTMLBase.hs, ./HTMLBase.o )
Compiling HTMLMonadBase    ( ./HTMLMonadBase.hs, ./HTMLMonadBase.o )
Compiling HTMLMonad        ( ./HTMLMonad.hs, ./HTMLMonad.o )
Compiling HTMLMonad98No4   ( ./HTMLMonad98No4.hs, ./HTMLMonad98No4.o )
Compiling HTMLMonad98No3   ( ./HTMLMonad98No3.hs, ./HTMLMonad98No3.o )
Compiling HTMLMonad98No2   ( ./HTMLMonad98No2.hs, ./HTMLMonad98No2.o )
Compiling HTMLMonad98No6m  ( ./HTMLMonad98No6m.hs, ./HTMLMonad98No6m.o )
Compiling HTMLMonad98No6l  ( ./HTMLMonad98No6l.hs, ./HTMLMonad98No6l.o )
Compiling HTMLMonad98No6k  ( ./HTMLMonad98No6k.hs, ./HTMLMonad98No6k.o )
Compiling HTMLMonad98No6j  ( ./HTMLMonad98No6j.hs, ./HTMLMonad98No6j.o )
Compiling HTMLMonad98No6i  ( ./HTMLMonad98No6i.hs, ./HTMLMonad98No6i.o )
Compiling HTMLMonad98No6h  ( ./HTMLMonad98No6h.hs, ./HTMLMonad98No6h.o )
Compiling HTMLMonad98No6g  ( ./HTMLMonad98No6g.hs, ./HTMLMonad98No6g.o )
Compiling HTMLMonad98No6f  ( ./HTMLMonad98No6f.hs, ./HTMLMonad98No6f.o )
Compiling HTMLMonad98No6e  ( ./HTMLMonad98No6e.hs, ./HTMLMonad98No6e.o )
Compiling HTMLMonad98No6d  ( ./HTMLMonad98No6d.hs, ./HTMLMonad98No6d.o )
Compiling HTMLMonad98No6c  ( ./HTMLMonad98No6c.hs, ./HTMLMonad98No6c.o )
Compiling HTMLMonad98No6b  ( ./HTMLMonad98No6b.hs, ./HTMLMonad98No6b.o )
Compiling HTMLMonad98No6a  ( ./HTMLMonad98No6a.hs, ./HTMLMonad98No6a.o )
Compiling HTMLMonad98No6n  ( ./HTMLMonad98No6n.hs, ./HTMLMonad98No6n.o )
Compiling HTMLMonad98No6all ( HTMLMonad98No6all.hs, HTMLMonad98No6all.o )
ghc --make HTMLMonad98No6all.hs  18.12s user 0.31s system 51% cpu 35.452 total
$ rm -f *.o
$ time ghc --make HTMLMonad98No6.hs
Chasing modules from: HTMLMonad98No6.hs
Compiling HTMLMonad98No5   ( ./HTMLMonad98No5.hs, ./HTMLMonad98No5.o )
Compiling HTMLBase         ( ./HTMLBase.hs, ./HTMLBase.o )
Compiling HTMLMonadBase    ( ./HTMLMonadBase.hs, ./HTMLMonadBase.o )
Compiling HTMLMonad        ( ./HTMLMonad.hs, ./HTMLMonad.o )
Compiling HTMLMonad98No4   ( ./HTMLMonad98No4.hs, ./HTMLMonad98No4.o )
Compiling HTMLMonad98No3   ( ./HTMLMonad98No3.hs, ./HTMLMonad98No3.o )
Compiling HTMLMonad98No2   ( ./HTMLMonad98No2.hs, ./HTMLMonad98No2.o )
Compiling HTMLMonad98No6   ( HTMLMonad98No6.hs, HTMLMonad98No6.o )
ghc --make HTMLMonad98No6.hs  113.98s user 0.92s system 67% cpu 2:49.65 total
$ cat /proc/cpuinfo
processor       : 0
vendor_id       : GenuineIntel
cpu family      : 15
model           : 2
model name      : Intel(R) Pentium(R) 4 CPU 2.80GHz
stepping        : 9
cpu MHz         : 2794.145
cache size      : 512 KB
fdiv_bug        : no
hlt_bug         : no
f00f_bug        : no
coma_bug        : no
fpu             : yes
fpu_exception   : yes
cpuid level     : 2
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe cid xtpr
bogomips        : 5537.79