nightly build nofib failures

[Donald Bruce Stewart]

Hey,

There is a bug in the nightly build scripts for running the nofib
suite. Nofib programs can fail, yet the build output will still
be "ok.". In fact, every single nofib program can fail to
compile, but it will still be "ok." in the nightly mail.

In a normal nightly build, we get this kind of output for
a nofib run:
    "**** running nofib                ... ok."
Taking
    http://www.haskell.org/pipermail/cvs-ghc/2003-July/017999.html
as an example, we see that all nofib's were "ok". Now, if we go
and look at the logs:
    http://haskell.cs.yale.edu/ghc/dist/current/logs/

Looking in nofib-2, we see that there are lots (all?) fails: primes, 
queens, etc.  Looking in nofib-5 all tests fail, because it
is build the -unreg way witout library support. 

So here is a quick sh patch against nightly/bin/nightly-wrk to
print the number of failures a nofib run produced. It produces
output like this, where failures are printed if they occur.

    **** running nofib                ... ok.
    **** running nofib                ... ok.
    **** running nofib                ... ok. (4 failures)
    **** running nofib                ... ok. (4 failures)

Cheers,
    Don Stewart

-------------------------------------------------------------------


Index: nightly/bin/nightly-wrk
===================================================================
RCS file: /cvs/nightly/bin/nightly-wrk,v
retrieving revision 1.36
diff -u -r1.36 nightly-wrk
--- nightly/bin/nightly-wrk	30 May 2003 01:44:40 -0000	1.36
+++ nightly/bin/nightly-wrk	14 Jul 2003 01:09:46 -0000
@@ -713,7 +713,18 @@
 	    "$nofib_1_opts"		\
 	    "$nofib_1_categories"	\
 	    '' > $LOG_DIR/$arch-nofib-1 2>&1; then
-	    $ECHO "ok."
+
+            F="$LOG_DIR/$arch-nofib-1"
+            E='not remade because of errors'
+            FAIL=`grep "$E" $F | wc -l | sed 's/^ *//'`
+
+            $ECHO -n "ok."
+            if [ "$FAIL" != 0 ] ; then
+                $ECHO " ($FAIL failures)"
+            else
+                $ECHO
+            fi
+
 	else
 	    $ECHO "failed."
 	fi
@@ -727,7 +738,18 @@
 	    "$nofib_2_opts"		\
 	    "$nofib_2_categories"	\
 	    '' > $LOG_DIR/$arch-nofib-2 2>&1; then
-	    $ECHO "ok."
+
+            F="$LOG_DIR/$arch-nofib-2"
+            E='not remade because of errors'
+            FAIL=`grep "$E" $F | wc -l | sed 's/^ *//'`
+
+            $ECHO -n "ok."
+            if [ "$FAIL" != 0 ] ; then
+                $ECHO " ($FAIL failures)"
+            else
+                $ECHO
+            fi
+
 	else
 	    $ECHO "failed."
 	fi
@@ -741,7 +763,18 @@
 	    "$nofib_3_opts"		\
 	    "$nofib_3_categories"	\
 	    '' > $LOG_DIR/$arch-nofib-3 2>&1; then
-	    $ECHO "ok."
+
+            F="$LOG_DIR/$arch-nofib-3"
+            E='not remade because of errors'
+            FAIL=`grep "$E" $F | wc -l | sed 's/^ *//'`
+
+            $ECHO -n "ok."
+            if [ "$FAIL" != 0 ] ; then
+                $ECHO " ($FAIL failures)"
+            else
+                $ECHO
+            fi
+
 	else
 	    $ECHO "failed."
 	fi
@@ -755,7 +788,18 @@
 	    "$nofib_4_opts"		\
 	    "$nofib_4_categories"	\
 	    '' > $LOG_DIR/$arch-nofib-4 2>&1; then
-	    $ECHO "ok."
+
+            F="$LOG_DIR/$arch-nofib-4"
+            E='not remade because of errors'
+            FAIL=`grep "$E" $F | wc -l | sed 's/^ *//'`
+
+            $ECHO -n "ok."
+            if [ "$FAIL" != 0 ] ; then
+                $ECHO " ($FAIL failures)"
+            else
+                $ECHO
+            fi
+
 	else
 	    $ECHO "failed."
 	fi
@@ -769,7 +813,18 @@
 	    "$nofib_5_opts"		\
 	    "$nofib_5_categories"	\
 	    '' > $LOG_DIR/$arch-nofib-5 2>&1; then
-	    $ECHO "ok."
+
+            F="$LOG_DIR/$arch-nofib-5"
+            E='not remade because of errors'
+            FAIL=`grep "$E" $F | wc -l | sed 's/^ *//'`
+
+            $ECHO -n "ok."
+            if [ "$FAIL" != 0 ] ; then
+                $ECHO " ($FAIL failures)"
+            else
+                $ECHO
+            fi
+
 	else
 	    $ECHO "failed."
 	fi
@@ -783,7 +838,18 @@
 	    "$nofib_6_opts"		\
 	    "$nofib_6_categories"	\
 	    '' > $LOG_DIR/$arch-nofib-6 2>&1; then
-	    $ECHO "ok."
+
+            F="$LOG_DIR/$arch-nofib-6"
+            E='not remade because of errors'
+            FAIL=`grep "$E" $F | wc -l | sed 's/^ *//'`
+
+            $ECHO -n "ok."
+            if [ "$FAIL" != 0 ] ; then
+                $ECHO " ($FAIL failures)"
+            else
+                $ECHO
+            fi
+
 	else
 	    $ECHO "failed."
 	fi