[Haskell-cafe] quickcheck for compiler testing

[Andrey Chudnov]

If you just need an example for generating and shrinking arbitrary ASTs, 
you can have a look here: 
https://github.com/jswebtools/language-ecmascript/blob/master/src/Language/ECMAScript3/Syntax/Arbitrary.hs 
. Note that I use the testing-feat package to generate Gen instances --- 
it tends to do better (explore interesting cases earlier) than 
handwritten code --- and then fix-up incorrect ASTs (another option is 
to simply discard them).

On 02/16/2015 12:33 PM, Maurizio Vitale wrote:
> By 'test size reduction' I mean the 'shrink' function. It seems to me 
> (as I said, first Haskell program and no experience with quickcheck) 
> that it works nicely with a topdown generation of the test, but I 
> don't see how to easily generate correct programs that way.
>
> Even if you cannot release your tests, maybe you can help me with a 
> very simple case. Consider a trivial AST. A program is a possibly 
> nested block. Each block is a bunch of declarations of variables and 
> some use of them
>
>     data Block = Block [Declaration] [Statement]
>
>     data Declaration = Var String String
>
>     data Statement = Statement Block | Use String
>
>
> How one would generate things like:
>
>     Block [Var "a" "int"] [Use "a"] -- here a is declared in the same
>     block
>
>     Block [] [Statement Block [Var "s" "int"] [Statement Block [] [Use
>     "s"]]] -- here s is declared in some other visible scope
>
>
> etc.
>
> Or am I trying to approach the problem from the wrong angle?
>
>
> On Mon, Feb 16, 2015 at 12:02 PM, Andrey Chudnov <achudnov at gmail.com 
> <mailto:achudnov at gmail.com>> wrote:
>
>     I use QuichCheck for compiler testing where I generate random, but
>     well-formed programs and check some high-level syntactic
>     properties on results. The QuickCheck instance is open-source (see
>     language-ecmascript), but the compiler-test code is closed-source
>     at this time. Still, I found that it's not the ultimate answer:
>     many properties are hard to formalize, so I have to resort to unit
>     tests. I'm not sure what you mean by "how this would play with
>     test size reduction". I think it's worth giving a try, but keep in
>     mind that you might still need to use unit tests.
>
>     Let me know if you have any questions.
>
>     /Andrey
>
>
>     On 02/16/2015 11:53 AM, Maurizio Vitale wrote:
>>     I'm starting to work on my first real haskell program (I've only
>>     RWH exercises under my belt) and wondering whether people use
>>     quickcheck at all for compiler testing.
>>
>>     I've seen uses of quickcheck for testing parsers, but I'm
>>     interested in generating well-formed programs (e.g. random
>>     programs with all declarations in reasonable random places). This
>>     could be used to test passes other than parsing (or even parsing,
>>     for languages that needs to distinguish identifiers, like the
>>     'typedef' problem in C/C++).
>>
>>     The only thing I can think of, is to use quickcheck for randomly
>>     generating statements, go over them and figure out free variables
>>     (incl. functions) and generate declarations in random places for
>>     them. But I'm not sure how this would play with test size
>>     reduction and doesn't look like a nice solution anyhow.
>>
>>     Any idea or pointers to examples? or should I give up on
>>     quickcheck for this and just do direct testing?
>>
>>     Thanks,
>>
>>       Maurizio
>>
>>
>>     _______________________________________________
>>     Haskell-Cafe mailing list
>>     Haskell-Cafe at haskell.org  <mailto:Haskell-Cafe at haskell.org>
>>     http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
>
>
>     _______________________________________________
>     Haskell-Cafe mailing list
>     Haskell-Cafe at haskell.org <mailto:Haskell-Cafe at haskell.org>
>     http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/haskell-cafe/attachments/20150216/0fdb373a/attachment.html>