How does GHC's testsuite work?

[Sébastien Hinderer]

Dear Joachim,

Many thanks for your prompt, positive and encouraging response.

Joachim Breitner (2017/10/30 13:44 -0400):
> Hi Sebastien,
> 
> I’m looking forward to your report, surely there will be some
> interesting inspirations for us.

Thanks. Unfortunately, the paper has been written in French. i'll see
whether I can find the time to translate it into english soon. If I can
do so I'll definitely post a link.

> Am Montag, den 30.10.2017, 11:25 -0400 schrieb Edward Z. Yang:
> > Actually, it's the reverse of what you said: like OCaml, GHC essentially
> > has ~no unit tests; it's entirely Haskell programs which we compile
> > (and sometimes run; a lot of tests are for the typechecker only so
> > we don't bother running those.)  The .T file is just a way of letting
> > the Python driver know what tests exist.
> 
> let me add that these tests rarely check the actual output of the
> compiler (i.e. the program, or even the simplified code). Often it is
> enough to check
>  * whether the compile succeeds or fails as expected, or maybe
>  * what messages the compiler prints.

I see. I think it is quite similar for OCaml.

> In a few cases we do dump the complete intermediate code (-ddump-
> simpl), but then the test case specifies a “normalization function”
> that checks the output for a certain property, e.g. by grepping for
> certain patterns.

Got it, thanks. We also have options to pretty-print the few internal
representations we have, but as far as I know, we don't use any
normalization function in such cases and just make a diff with an
expected reference (yuk!).

> The only real unit tests that I know of are these:
> http://git.haskell.org/ghc.git/tree/HEAD:/testsuite/tests/callarity/unittest
> These are effectively programs using “GHC-the-library”

Okay, thanks!

Will come back with a link ASAP!

Sébastien.