[Haskell-cafe] quickcheck for compiler testing

[Maurizio Vitale]

also a longer thesis (which I believe is the backtracking optimization they
refer to in the paper)  is at
http://publications.lib.chalmers.se/records/fulltext/157525.pdf
I'll have to play with a very small language and see where I get.
Thanks to everybody, and if there're other examples around I'll be glad to
know about them.

On Tue, Feb 17, 2015 at 3:15 AM, Paul Brauner <polux2001 at gmail.com> wrote:

> There's a recent paper by Jonas Duregård et al where they introduce a
> variant of feat that allows for efficient discarding of unwanted values, à
> la lazy Smallcheck. The motivating example of the paper is exactly what
> you're looking for: discarding ill-formed programs.
>
> On Mon, Feb 16, 2015, 19:44 Andrey Chudnov <achudnov at gmail.com> wrote:
>
>>  If you just need an example for generating and shrinking arbitrary ASTs,
>> you can have a look here:
>> https://github.com/jswebtools/language-ecmascript/blob/master/src/Language/ECMAScript3/Syntax/Arbitrary.hs
>> . Note that I use the testing-feat package to generate Gen instances --- it
>> tends to do better (explore interesting cases earlier) than handwritten
>> code --- and then fix-up incorrect ASTs (another option is to simply
>> discard them).
>>
>>
>> On 02/16/2015 12:33 PM, Maurizio Vitale wrote:
>>
>> By 'test size reduction' I mean the 'shrink' function. It seems to me (as
>> I said, first Haskell program and no experience with quickcheck) that it
>> works nicely with a topdown generation of the test, but I don't see how to
>> easily generate correct programs that way.
>>
>>  Even if you cannot release your tests, maybe you can help me with a
>> very simple case. Consider a trivial AST. A program is a possibly nested
>> block. Each block is a bunch of declarations of variables and some use of
>> them
>>
>>   data Block = Block [Declaration] [Statement]
>>
>>   data Declaration = Var String String
>>
>>   data Statement = Statement Block | Use String
>>
>>
>>  How one would generate things like:
>>
>>  Block [Var "a" "int"] [Use "a"] -- here a is declared in the same block
>>
>>  Block [] [Statement Block [Var "s" "int"] [Statement Block [] [Use
>> "s"]]] -- here s is declared in some other visible scope
>>
>>
>>  etc.
>>
>>  Or am I trying to approach the problem from the wrong angle?
>>
>>
>>
>> On Mon, Feb 16, 2015 at 12:02 PM, Andrey Chudnov <achudnov at gmail.com>
>> wrote:
>>
>>>  I use QuichCheck for compiler testing where I generate random, but
>>> well-formed programs and check some high-level syntactic properties on
>>> results. The QuickCheck instance is open-source (see language-ecmascript),
>>> but the compiler-test code is closed-source at this time. Still, I found
>>> that it's not the ultimate answer: many properties are hard to formalize,
>>> so I have to resort to unit tests. I'm not sure what you mean by "how this
>>> would play with test size reduction". I think it's worth giving a try, but
>>> keep in mind that you might still need to use unit tests.
>>>
>>> Let me know if you have any questions.
>>>
>>> /Andrey
>>>
>>>
>>> On 02/16/2015 11:53 AM, Maurizio Vitale wrote:
>>>
>>>  I'm starting to work on my first real haskell program (I've only RWH
>>> exercises under my belt) and wondering whether people use quickcheck at all
>>> for compiler testing.
>>>
>>>  I've seen uses of quickcheck for testing parsers, but I'm interested
>>> in generating well-formed programs (e.g. random programs with all
>>> declarations in reasonable random places). This could be used to test
>>> passes other than parsing (or even parsing, for languages that needs to
>>> distinguish identifiers, like the 'typedef' problem in C/C++).
>>>
>>>  The only thing I can think of, is to use quickcheck for randomly
>>> generating statements, go over them and figure out free variables (incl.
>>> functions) and generate declarations in random places for them. But I'm not
>>> sure how this would play with test size reduction and doesn't look like a
>>> nice solution anyhow.
>>>
>>>  Any idea or pointers to examples? or should I give up on quickcheck
>>> for this and just do direct testing?
>>>
>>>  Thanks,
>>>
>>>    Maurizio
>>>
>>>
>>>  _______________________________________________
>>> Haskell-Cafe mailing listHaskell-Cafe at haskell.orghttp://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
>>>
>>>
>>>
>>> _______________________________________________
>>> Haskell-Cafe mailing list
>>> Haskell-Cafe at haskell.org
>>> http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
>>>
>>>
>>
>>  _______________________________________________
>> Haskell-Cafe mailing list
>> Haskell-Cafe at haskell.org
>> http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
>>
>
> _______________________________________________
> Haskell-Cafe mailing list
> Haskell-Cafe at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/haskell-cafe/attachments/20150217/92f1dec9/attachment.html>