[Haskell-cafe] quickcheck for compiler testing

[Andrey Chudnov]

Do you know if there is an implementation?

On 02/17/2015 03:15 AM, Paul Brauner wrote:
>
> There's a recent paper by Jonas Duregård et al where they introduce a 
> variant of feat that allows for efficient discarding of unwanted 
> values, à la lazy Smallcheck. The motivating example of the paper is 
> exactly what you're looking for: discarding ill-formed programs.
>
>
> On Mon, Feb 16, 2015, 19:44 Andrey Chudnov <achudnov at gmail.com 
> <mailto:achudnov at gmail.com>> wrote:
>
>     If you just need an example for generating and shrinking arbitrary
>     ASTs, you can have a look here:
>     https://github.com/jswebtools/language-ecmascript/blob/master/src/Language/ECMAScript3/Syntax/Arbitrary.hs
>     . Note that I use the testing-feat package to generate Gen
>     instances --- it tends to do better (explore interesting cases
>     earlier) than handwritten code --- and then fix-up incorrect ASTs
>     (another option is to simply discard them).
>
>
>     On 02/16/2015 12:33 PM, Maurizio Vitale wrote:
>>     By 'test size reduction' I mean the 'shrink' function. It seems
>>     to me (as I said, first Haskell program and no experience with
>>     quickcheck) that it works nicely with a topdown generation of the
>>     test, but I don't see how to easily generate correct programs
>>     that way.
>>
>>     Even if you cannot release your tests, maybe you can help me with
>>     a very simple case. Consider a trivial AST. A program is a
>>     possibly nested block. Each block is a bunch of declarations of
>>     variables and some use of them
>>
>>         data Block = Block [Declaration] [Statement]
>>
>>         data Declaration = Var String String
>>
>>         data Statement = Statement Block | Use String
>>
>>
>>     How one would generate things like:
>>
>>         Block [Var "a" "int"] [Use "a"] -- here a is declared in the
>>         same block
>>
>>         Block [] [Statement Block [Var "s" "int"] [Statement Block []
>>         [Use "s"]]] -- here s is declared in some other visible scope
>>
>>
>>     etc.
>>
>>     Or am I trying to approach the problem from the wrong angle?
>>
>>
>>     On Mon, Feb 16, 2015 at 12:02 PM, Andrey Chudnov
>>     <achudnov at gmail.com <mailto:achudnov at gmail.com>> wrote:
>>
>>         I use QuichCheck for compiler testing where I generate
>>         random, but well-formed programs and check some high-level
>>         syntactic properties on results. The QuickCheck instance is
>>         open-source (see language-ecmascript), but the compiler-test
>>         code is closed-source at this time. Still, I found that it's
>>         not the ultimate answer: many properties are hard to
>>         formalize, so I have to resort to unit tests. I'm not sure
>>         what you mean by "how this would play with test size
>>         reduction". I think it's worth giving a try, but keep in mind
>>         that you might still need to use unit tests.
>>
>>         Let me know if you have any questions.
>>
>>         /Andrey
>>
>>
>>         On 02/16/2015 11:53 AM, Maurizio Vitale wrote:
>>>         I'm starting to work on my first real haskell program (I've
>>>         only RWH exercises under my belt) and wondering whether
>>>         people use quickcheck at all for compiler testing.
>>>
>>>         I've seen uses of quickcheck for testing parsers, but I'm
>>>         interested in generating well-formed programs (e.g. random
>>>         programs with all declarations in reasonable random places).
>>>         This could be used to test passes other than parsing (or
>>>         even parsing, for languages that needs to distinguish
>>>         identifiers, like the 'typedef' problem in C/C++).
>>>
>>>         The only thing I can think of, is to use quickcheck for
>>>         randomly generating statements, go over them and figure out
>>>         free variables (incl. functions) and generate declarations
>>>         in random places for them. But I'm not sure how this would
>>>         play with test size reduction and doesn't look like a nice
>>>         solution anyhow.
>>>
>>>         Any idea or pointers to examples? or should I give up on
>>>         quickcheck for this and just do direct testing?
>>>
>>>         Thanks,
>>>
>>>           Maurizio
>>>
>>>
>>>         _______________________________________________
>>>         Haskell-Cafe mailing list
>>>         Haskell-Cafe at haskell.org  <mailto:Haskell-Cafe at haskell.org>
>>>         http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
>>
>>
>>         _______________________________________________
>>         Haskell-Cafe mailing list
>>         Haskell-Cafe at haskell.org <mailto:Haskell-Cafe at haskell.org>
>>         http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
>>
>>
>
>     _______________________________________________
>     Haskell-Cafe mailing list
>     Haskell-Cafe at haskell.org <mailto:Haskell-Cafe at haskell.org>
>     http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/haskell-cafe/attachments/20150217/ef32b765/attachment.html>