[Haskell-cafe] quickcheck for compiler testing
achudnov at gmail.com
Tue Feb 17 12:54:39 UTC 2015
Do you know if there is an implementation?
On 02/17/2015 03:15 AM, Paul Brauner wrote:
> There's a recent paper by Jonas Duregård et al where they introduce a
> variant of feat that allows for efficient discarding of unwanted
> values, à la lazy Smallcheck. The motivating example of the paper is
> exactly what you're looking for: discarding ill-formed programs.
> On Mon, Feb 16, 2015, 19:44 Andrey Chudnov <achudnov at gmail.com
> <mailto:achudnov at gmail.com>> wrote:
> If you just need an example for generating and shrinking arbitrary
> ASTs, you can have a look here:
> . Note that I use the testing-feat package to generate Gen
> instances --- it tends to do better (explore interesting cases
> earlier) than handwritten code --- and then fix-up incorrect ASTs
> (another option is to simply discard them).
> On 02/16/2015 12:33 PM, Maurizio Vitale wrote:
>> By 'test size reduction' I mean the 'shrink' function. It seems
>> to me (as I said, first Haskell program and no experience with
>> quickcheck) that it works nicely with a topdown generation of the
>> test, but I don't see how to easily generate correct programs
>> that way.
>> Even if you cannot release your tests, maybe you can help me with
>> a very simple case. Consider a trivial AST. A program is a
>> possibly nested block. Each block is a bunch of declarations of
>> variables and some use of them
>> data Block = Block [Declaration] [Statement]
>> data Declaration = Var String String
>> data Statement = Statement Block | Use String
>> How one would generate things like:
>> Block [Var "a" "int"] [Use "a"] -- here a is declared in the
>> same block
>> Block  [Statement Block [Var "s" "int"] [Statement Block 
>> [Use "s"]]] -- here s is declared in some other visible scope
>> Or am I trying to approach the problem from the wrong angle?
>> On Mon, Feb 16, 2015 at 12:02 PM, Andrey Chudnov
>> <achudnov at gmail.com <mailto:achudnov at gmail.com>> wrote:
>> I use QuichCheck for compiler testing where I generate
>> random, but well-formed programs and check some high-level
>> syntactic properties on results. The QuickCheck instance is
>> open-source (see language-ecmascript), but the compiler-test
>> code is closed-source at this time. Still, I found that it's
>> not the ultimate answer: many properties are hard to
>> formalize, so I have to resort to unit tests. I'm not sure
>> what you mean by "how this would play with test size
>> reduction". I think it's worth giving a try, but keep in mind
>> that you might still need to use unit tests.
>> Let me know if you have any questions.
>> On 02/16/2015 11:53 AM, Maurizio Vitale wrote:
>>> I'm starting to work on my first real haskell program (I've
>>> only RWH exercises under my belt) and wondering whether
>>> people use quickcheck at all for compiler testing.
>>> I've seen uses of quickcheck for testing parsers, but I'm
>>> interested in generating well-formed programs (e.g. random
>>> programs with all declarations in reasonable random places).
>>> This could be used to test passes other than parsing (or
>>> even parsing, for languages that needs to distinguish
>>> identifiers, like the 'typedef' problem in C/C++).
>>> The only thing I can think of, is to use quickcheck for
>>> randomly generating statements, go over them and figure out
>>> free variables (incl. functions) and generate declarations
>>> in random places for them. But I'm not sure how this would
>>> play with test size reduction and doesn't look like a nice
>>> solution anyhow.
>>> Any idea or pointers to examples? or should I give up on
>>> quickcheck for this and just do direct testing?
>>> Haskell-Cafe mailing list
>>> Haskell-Cafe at haskell.org <mailto:Haskell-Cafe at haskell.org>
>> Haskell-Cafe mailing list
>> Haskell-Cafe at haskell.org <mailto:Haskell-Cafe at haskell.org>
> Haskell-Cafe mailing list
> Haskell-Cafe at haskell.org <mailto:Haskell-Cafe at haskell.org>
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Haskell-Cafe