T3294 variies in whether it fails or not

[Simon Peyton-Jones]

Ian

Perf/compiler/T3294 has very different max_bytes_used when BINDIST=YES than without. See below.  

I don't know why.  I don't know whether it matters.  Maybe it's just the moment that GC strikes.  But it's uncomfortable to have such an erratic test failure.

Simon

simonpj at cam-05-unx:~/5builds/HEAD-1/testsuite/tests/perf/compiler$ make TEST=T3294
python2 ../../../driver/runtests.py  -e ghc_compiler_always_flags="'-fforce-recomp -dcore-lint -dcmm-lint -dno-debug-output -no-user-package-db -rtsopts '" -e ghc_debugged=False -e ghc_with_native_codegen=1 -e ghc_with_vanilla=1 -e ghc_with_dynamic=1 -e ghc_with_profiling=0 -e ghc_with_threaded_rts=1 -e ghc_with_dynamic_rts=1 -e ghc_with_interpreter=1 -e ghc_unregisterised=0 -e ghc_dynamic_by_default=False -e ghc_dynamic=True -e ghc_with_smp=1 -e ghc_with_llvm=1 -e windows=False -e darwin=False -e in_tree_compiler=True -e clean_only=False --rootdir=. --config=../../../config/ghc -e 'config.confdir="../../../config"' -e 'config.compiler="/5playpen/simonpj/HEAD-1/inplace/bin/ghc-stage2"' -e 'config.ghc_pkg="/5playpen/simonpj/HEAD-1/inplace/bin/ghc-pkg"' -e 'config.hp2ps="/5playpen/simonpj/HEAD-1/inplace/bin/hp2ps"' -e 'config.hpc="/5playpen/simonpj/HEAD-1/inplace/bin/hpc"' -e 'config.gs="gs"' -e 'config.platform="x86_64-unknown-linux"' -e 'config.os="linux"' -e 'config.arch="x86_64"' -e 'config.wordsize="64"' -e 'default_testopts.cleanup=""' -e 'config.timeout=int() or config.timeout' -e 'config.timeout_prog="../../../timeout/install-inplace/bin/timeout"' -e 'config.exeext=""' -e 'config.top="/5playpen/simonpj/HEAD-1/testsuite"'  \
		 --only=T3294 \
		 \
		 \
		 \
		 \
		 \
		
Timeout is 300
Found 1 .T files...
Beginning test run at Tue May 28 09:21:29 BST 2013
====> Scanning ./all.T
=====> T3294(normal) 2 of 14 [0, 0, 0]
cd . && '/5playpen/simonpj/HEAD-1/inplace/bin/ghc-stage2' -fforce-recomp -dno-debug-output -no-user-package-db -rtsopts -fno-ghci-history -c T3294.hs   +RTS -V0 -tT3294.comp.stats --machine-readable -RTS  >T3294.comp.stderr 2>&1

OVERALL SUMMARY for test run started at Tue May 28 09:21:29 BST 2013
      14 total tests, which gave rise to
      47 test cases, of which
      46 were skipped

       0 had missing libraries
       1 expected passes
       0 expected failures

       0 caused framework failures
       0 unexpected passes
       0 unexpected failures

simonpj at cam-05-unx:~/5builds/HEAD-1/testsuite/tests/perf/compiler$ cat T3294.comp.stats 
/5playpen/simonpj/HEAD-1/inplace/lib/bin/ghc-stage2 -B/5playpen/simonpj/HEAD-1/inplace/lib -fforce-recomp -dno-debug-output -no-user-package-db -rtsopts -fno-ghci-history -c T3294.hs +RTS -V0 -tT3294.comp.stats --machine-readable 
 [("bytes allocated", "2721056936")
 ,("num_GCs", "540")
 ,("average_bytes_used", "15190408")
 ,("max_bytes_used", "36726792")
 ,("num_byte_usage_samples", "15")
 ,("peak_megabytes_allocated", "105")
 ,("init_cpu_seconds", "0.00")
 ,("init_wall_seconds", "0.00")
 ,("mutator_cpu_seconds", "2.64")
 ,("mutator_wall_seconds", "3.25")
 ,("GC_cpu_seconds", "2.03")
 ,("GC_wall_seconds", "2.03")
 ]
simonpj at cam-05-unx:~/5builds/HEAD-1/testsuite/tests/perf/compiler$ make TEST=T3294 BINDIST=YES
python2 ../../../driver/runtests.py  -e ghc_compiler_always_flags="'-fforce-recomp -dcore-lint -dcmm-lint -dno-debug-output -no-user-package-db -rtsopts '" -e ghc_debugged=False -e ghc_with_native_codegen=1 -e ghc_with_vanilla=1 -e ghc_with_dynamic=1 -e ghc_with_profiling=0 -e ghc_with_threaded_rts=1 -e ghc_with_dynamic_rts=1 -e ghc_with_interpreter=1 -e ghc_unregisterised=0 -e ghc_dynamic_by_default=False -e ghc_dynamic=True -e ghc_with_smp=1 -e ghc_with_llvm=1 -e windows=False -e darwin=False -e in_tree_compiler=True -e clean_only=False --rootdir=. --config=../../../config/ghc -e 'config.confdir="../../../config"' -e 'config.compiler="/5playpen/simonpj/HEAD-1/bindisttest/install   dir/bin/ghc"' -e 'config.ghc_pkg="/5playpen/simonpj/HEAD-1/bindisttest/install   dir/bin/ghc-pkg"' -e 'config.hp2ps="/5playpen/simonpj/HEAD-1/bindisttest/install   dir/bin/hp2ps"' -e 'config.hpc="/5playpen/simonpj/HEAD-1/bindisttest/install   dir/bin/hpc"' -e 'config.gs="gs"' -e 'config.platform="x86_64-unknown-linux"' -e 'config.os="linux"' -e 'config.arch="x86_64"' -e 'config.wordsize="64"' -e 'default_testopts.cleanup=""' -e 'config.timeout=int() or config.timeout' -e 'config.timeout_prog="../../../timeout/install-inplace/bin/timeout"' -e 'config.exeext=""' -e 'config.top="/5playpen/simonpj/HEAD-1/testsuite"'  \
		 --only=T3294 \
		 \
		 \
		 \
		 \
		 \
		
Timeout is 300
Found 1 .T files...
Beginning test run at Tue May 28 09:21:49 BST 2013
====> Scanning ./all.T
=====> T3294(normal) 2 of 14 [0, 0, 0]
cd . && '/5playpen/simonpj/HEAD-1/bindisttest/install   dir/bin/ghc' -fforce-recomp -dno-debug-output -no-user-package-db -rtsopts -fno-ghci-history -c T3294.hs   +RTS -V0 -tT3294.comp.stats --machine-readable -RTS  >T3294.comp.stderr 2>&1
max_bytes_used value is too high:
    Expected    max_bytes_used: 36904752 +/-15%
    Lower bound max_bytes_used: 31369039 
    Upper bound max_bytes_used: 42440465 
    Actual      max_bytes_used: 43818472 
*** unexpected failure for T3294(normal)

Unexpected results from:
TEST="T3294"

OVERALL SUMMARY for test run started at Tue May 28 09:21:49 BST 2013
      14 total tests, which gave rise to
      47 test cases, of which
      46 were skipped

       0 had missing libraries
       0 expected passes
       0 expected failures

       0 caused framework failures
       0 unexpected passes
       1 unexpected failures

Unexpected failures:
   .  T3294 [stat not good enough] (normal)

simonpj at cam-05-unx:~/5builds/HEAD-1/testsuite/tests/perf/compiler$ cat T3294.comp.stats 
/5playpen/simonpj/HEAD-1/bindisttest/install   dir/lib/ghc-7.7/bin/ghc -B/5playpen/simonpj/HEAD-1/bindisttest/install   dir/lib/ghc-7.7 -fforce-recomp -dno-debug-output -no-user-package-db -rtsopts -fno-ghci-history -c T3294.hs +RTS -V0 -tT3294.comp.stats --machine-readable 
 [("bytes allocated", "2724158504")
 ,("num_GCs", "525")
 ,("average_bytes_used", "16042228")
 ,("max_bytes_used", "43818472")
 ,("num_byte_usage_samples", "15")
 ,("peak_megabytes_allocated", "116")
 ,("init_cpu_seconds", "0.00")
 ,("init_wall_seconds", "0.00")
 ,("mutator_cpu_seconds", "2.77")
 ,("mutator_wall_seconds", "3.38")
 ,("GC_cpu_seconds", "2.38")
 ,("GC_wall_seconds", "2.38")
 ]
simonpj at cam-05-unx:~/5builds/HEAD-1/testsuite/tests/perf/compiler$