[Haskell-cafe] Testing of GHC extensions & optimizations

[Rodrigo Stevaux]

Thanks for the clarification.

What I am hinting at is, the Csmith project caught many bugs in C compilers
by using random testing -- feeding random programs and testing if the
optimizations preserved program behavior.

Haskell, having tens of optimizations, could be a potential application of
the same technique.

I have no familiarity with the GHC or with any compilers in general; I am
just looking for something to study.

My questions in its most direct form is, as in your view, could GHC
optimizations hide bugs that could be potentially be revealed by exploring
program spaces?

Em dom, 2 de set de 2018 às 16:58, Sven Panne <svenpanne at gmail.com>
escreveu:

> Am So., 2. Sep. 2018 um 20:05 Uhr schrieb Rodrigo Stevaux <
> roehst at gmail.com>:
>
>> Hi Omer, thanks for the reply. The tests you run are for regression
>> testing, that is, non-functional aspects, is my understanding right? [...]
>>
>
> Quite the opposite, the usual steps are:
>
>    * A bug is reported.
>    * A regression test is added to GHC's test suite, reproducing the bug (
> https://ghc.haskell.org/trac/ghc/wiki/Building/RunningTests/Adding).
>    * The bug is fixed.
>
> This way it is made sure that the bug doesn't come back later. Do this for
> a few decades, and you have a very comprehensive test suite for functional
> aspects. :-) The reasoning behind this: Blindly adding tests is wasted
> effort most of time, because this way you often test things which only very
> rarely break: Bugs OTOH hint you very concretely at
> problematic/tricky/complicated parts of your SW.
>
> Catching increases in runtime/memory consumption is a slightly different
> story, because you have to come up with "typical" scenarios to make useful
> comparisons. You can have synthetic scenarios for very specific parts of
> the compiler, too, like pattern matching with tons of constructors, or
> using gigantic literals, or type checking deeply nested tricky things,
> etc., but I am not sure if such things are usually called "regression
> tests".
>
> Cheers,
>    S.
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/haskell-cafe/attachments/20180902/48132772/attachment.html>