[Haskell] Interesting experiences of test automation in Haskell? Automation of Software Test 2010

[Tom Hawkins]

On Fri, Nov 27, 2009 at 9:30 PM, John Hughes <rjmh at chalmers.se> wrote:
> This is a "heads up" about a workshop on test automation that I just joined
> the programme committee of. Automation of Software Test will be co-located
> with ICSE in Cape Town in May--the workshop home page is here:
>
> http://www.cs.allegheny.edu/ast2010/
>
> Tools like QuickCheck, SmallCheck and HUnit fit the call for papers
> perfectly. So if you're doing some interesting automated testing in Haskell,
> why not submit a paper about it, and show a new audience what the Haskell
> community is up to? Both research papers and case studies are welcome, and
> the latter can even be in the form of a presentation of up to 15 slides--so
> there's no excuse for not putting something together!

Recently we added a unit testing framework to Atom: a haskell DSL for
hard realtime embedded applications.  The framework hooks into Atom's
assertion and functional coverage statements and provides some basic
randomization utilities for automatic test generation.  We've used
this framework to test the embedded software of Eaton's parallel
hydraulic hybrid application (HLA).

One nice feature we've incorporated for our application is in the
event of an assertion violation, simulation data is formated such that
we can use the same analysis and debug tools that we use for real
vehicle telemetry.

I don't think we can make it to Cape Town, but I'll see if one of us
can write up a case study.  (Shruti, Tejas, Jake, any interest?)

Here's an example test log from Language.Atom.Unit:

pass:  processTest1
 cycles =    1000
pass:  dualProcessTest
 cycles =    1000
pass:  processNoDeactivationGuard
 cycles =    1000
pass:  processNoActivationGuard
 cycles =    1000
pass:  processDuelingProcesses
 cycles =    1000
pass:  arbiterTest1
 cycles =    1000
pass:  arbiterTest2
 cycles =    1000
pass:  arbiterTest3
 cycles =    1000
pass:  arbiterTest8
 cycles =    1000
pass:  arbiterTest10
 cycles =    1000
pass:  absDisengagementTest
 cycles =   35000
pass:  cruiseControlDisengagementTest
 cycles =   35000
pass:  engineSpeedDisengagementTest
 cycles =   35000
pass:  pumpSpeedDisengagementTest
 cycles =   35000
pass:  ignitionDisengagementTest
 cycles =   35000
pass:  selectedGearDisengagementTest
 cycles =   35000
pass:  currentGearDisengagementTest
 cycles =   35000
pass:  modeDisengagementTest
 cycles =   35000
pass:  commissioningProcessDisengagementTest
 cycles =   35000
pass:  canEEC1DisengagementTest
 cycles =   35000
pass:  canEEC2DisengagementTest
 cycles =   35000
pass:  canETC1DisengagementTest
 cycles =   35000
pass:  canETC2DisengagementTest
 cycles =   35000
pass:  canEBCDisengagementTest
 cycles =   35000
pass:  canECCDisengagementTest
 cycles =   35000
pass:  driverisolationFaultDisengagementTest
 cycles =   35000
pass:  drivermodeDrainFaultDisengagementTest
 cycles =   35000
pass:  drivermodeFillFaultDisengagementTest
 cycles =   35000
pass:  drivermainBypassFaultDisengagementTest
 cycles =   35000
pass:  driverchargeBypassFaultDisengagementTest
 cycles =   35000
pass:  driverclutch1FaultDisengagementTest
 cycles =   35000
pass:  driverclutch2FaultDisengagementTest
 cycles =   35000
pass:  driverclutch3FaultDisengagementTest
 cycles =   35000
pass:  driverpumpFaultDisengagementTest
 cycles =   35000
pass:  drivermotorFaultDisengagementTest
 cycles =   35000
pass:  coolerFanOnOffTest
 cycles =   40000
pass:  stateTransitionTest
 cycles =   75000
pass:  swashCmdInBootstrapTest
 cycles =   40000
pass:  brakePressureAboveRangeAndTxSpeedDisengagementTest
 cycles =   35000
pass:  brakePressureAboveRangeDisengagementTest
 cycles =   35000
pass:  accumulatorPressureBelowRangeDisengagementTest
 cycles =   35000
pass:  accumulatorPressureAboveRangeDisengagementTest
 cycles =   35000
pass:  sawsh1BelowRangeDisengagementTest
 cycles =   35000
pass:  swash1AboveRangeDisengagementTest
 cycles =   35000
pass:  swash2BelowRangeDisengagementTest
 cycles =   35000
pass:  swash2AboveRangeDisengagementTest
 cycles =   35000
pass:  swashDiffOnBoundaryDisengagementTest
 cycles =   35000
pass:  swashDiffAboveRangeDisengagementTest
 cycles =   35000
pass:  pumpCaseOilTempCAboveRangeDisengagementTest
 cycles =   35000
pass:  pumpCaseOilTempCBelowRangeDisengagementTest
 cycles =   35000
pass:  pumpCaseOilTempCBoundaryDisengagementTest
 cycles =   35000
pass:  reservoirOilTempCAboveRangeDisengagementTest
 cycles =   35000
pass:  reservoirOilTempCBelowRangeDisengagementTest
 cycles =   35000
pass:  reservoirOilTempCBoundaryDisengagementTest
 cycles =   35000
pass:  tempSensorMiscompareAboveRangeDisengagementTest
 cycles =   35000
pass:  tempSensorMiscompareBoundaryDisengagementTest
 cycles =   35000
pass:  reservoirOilTempCTooExtremeDisengagementTest
 cycles =   35000
pass:  reservoirOilTempCTooExtremeBoundaryDisengagementTest
 cycles =   35000
pass:  reservoirOilLevelGalLDisengagementTest
 cycles =   35000
pass:  reservoirOilLevelGalHDisengagementTest
 cycles =   35000
pass:  pumpSpeedAboveRangeLDisengagementTest
 cycles =   35000
pass:  pumpSpeedAboveRangeHDisengagementTest
 cycles =   35000
pass:  accumOverPressureBoundaryDisengagementTest
 cycles =   35000
pass:  accumOverPressureAboveRangeDisengagementTest
 cycles =   35000
pass:  filterPressureBelowRangeDisengagementTest
 cycles =   35000
pass:  filterPressureAboveRangeDisengagementTest
 cycles =   35000
pass:  5VSupplyBelowRangeDisengagementTest
 cycles =   35000
pass:  5VSupplyAboveRangeDisengagementTest
 cycles =   35000
pass:  12VSupplyBelowRangeDisengagementTest
 cycles =   35000
pass:  12VSupplyAboveRangeDisengagementTest
 cycles =   35000
pass:  stuckEngagedDisengagementTest
 cycles =   35000
pass:  stuckOilLevelDisengagementTest
 cycles =   35000
pass:  clutchdisengagementFaultTest
 cycles =   40000
pass:  clutchSensorMiscompareFault
 cycles =   40000
pass:  clutchSensorMiscompareFaultAbovePumpSpeedandabsRangeTest
 cycles =   40000
pass:  clutchSensorMiscompareFaultAbovePumpSpeedandBelowabsRangeTest
 cycles =   40000
pass:  clutchSensorMiscompareFaultBoundaryTest
 cycles =   40000
pass:  clutchSensorMiscompareFaultAboveRangeTest
 cycles =   40000
pass:  unexpectedDisengagementTest
 cycles =   40000
pass:  failedEngagementAttemptsTest
 cycles =   40000
pass:  fanOffTest
 cycles =   30000
pass:  evuTest
 cycles =   30000
pass:  swashCalTest
 cycles =   40000
pass:  seedKey32Test1
 cycles =     100
pass:  seedKey32Test2
 cycles =     100
pass:  seedKey32Test3
 cycles =     100
pass:  seedKey32Test4
 cycles =     100
pass:  seedKey32Test5
 cycles =     100
pass:  seedKey32Test6
 cycles =     100
pass:  seedKey32Test7
 cycles =     100

Total Passing Tests     : 90 / 90
Total Simulation Cycles : 2650700
Total Function Coverage : 102 / 102

GREEN LIGHT


>
> So how about it? It would be great to see some Haskell papers at the
> workshop! Deadline 20 January.
>
> John Hughes
>
> PS Check out the ICSE web site for information on the location:
> http://www.sbs.co.za/ICSE2010/
>
> _______________________________________________
> Haskell mailing list
> Haskell at haskell.org
> http://www.haskell.org/mailman/listinfo/haskell
>
>