More on integer division

[Alastair Reid]

[Mail I send to Dylan tends to bounce so I'm using both email
addresses I have for him and hoping that he's subscribed to ghc-bugs
in case both of those addresses fail...]

Dylan Thurston <dpt at lotus.bostoncoop.net> writes:
> Ken Shan told me about QuickCheck, which seems like the right way to
> test such things.  Incorporating QuickCheck into ghc or hugs test
> suites is more work than I want to do right now, but it would be a
> nice project sometime.

I seem to remember being told that the GHC folk had started using QC
but now preferred not to.  I never heard the reason but I presume it's
because there's many ways for a buggy Haskell compiler to 'correctly'
print

  True

without executing every single line of code correctly but many fewer
ways for a buggy Haskell compiler to correctly print

   [0.0,0.0998334,0.198669,0.29552,0.389418,0.479426,0.564642,0.644218,0.717356,0.783327,0.841471]

without executing every line of code correctly.


This isn't to say that QuickCheck is bad (in fact, I think it's a great
tool) just that it is better at detecting errors in libraries than
in the compiler used to compile the libraries.


A useful thing to do would be to adapt the QuickCheck approach so that
instead of generating a bunch of Bools you generate a bunch of pairs
of values which must be equal.  For example, instead of checks _which
boil down to_ code like this:

  print $ and [ a + b == a - (-b) | (a,b) <- testinputs ]

you should use checks which boil down to code like this:

  print       [ (a + b, a - (-b)) | (a,b) <- testinputs ]

(Of course, you want to achieve this whilst maintaining most of
QuickCheck's general look and feel - an exercise left for the reader.)


Even better would be to be able to run in three different modes of operation:

1) Given two methods m1 and m2 of generating each answer (in the example,
   these are 'a+b' and 'a-(-b)', check that m1 and m2 generate the same
   answers as each other.  This is (a special case of) the standard
   QuickCheck behaviour.

2) Given a previously generated file of answers, check that m1
   generates those answers and check that m2 generates those answers.

3) Generate a file of sample answers for use in mode 2.
   [Note that you want to run the checker in mode 1 when doing this
   to check that your sample answers make some sort of sense.  Ideally,
   you'd generate the file on a separate Haskell compiler...]



I wonder if someone out there who teaches Haskell and/or software
engineering could assign this project to their students.  It'd be
incredibly useful to have such a tool and to have extensive
compiler-independent test-suites for Haskell98 (and common extensions)
and a test harness (like the current GHC test harness) which runs
these tests at regular intervals.  And I think it'd be incredibly good
for students to know how to go about setting up such a test suite: how
to test, how not to test, how to construct a good test framework, etc.


-- 
Alastair Reid        reid at cs.utah.edu        http://www.cs.utah.edu/~reid/