[Hat] GUI for Hat

[Colin Runciman]

Peter,

Pleased to hear that you are planning to work towards some advances in 
the Hat tools.

> Cover
>
>> It will use hs-colour to try and show which functions are being used
>> for a set of inputs. If the new tracing library is used, and it  
>> supports
>> taking the union of traces, this will be used to show coverage from  
>> lots
>> of traces. This can be tied into quickcheck.
>
> Excellent, that's a really useful tool for me.  You might find some  
> useful code in hat-delta and in hat-cover.  Hat-cover contains code  
> that will give you full coverage for a call, while hat-delta contains  
> code that will generate the coverage in the top level function call.

Hat-cover is pretty limited.  Just in the last few weeks I have 
implemented a new and more powerful coverage tool called hpc in 
collaboration with Andy Gill at Galois.  Hpc is not actually a Hat tool; 
its coverage records and index files are much simpler than Hat traces.  
But the approach is similar as we have a source transformation, run-time 
recording via a library, and separate viewing tools.  Hpc seems to work 
quite nicely with QuickCheck, and a simple script can be used to keep 
testing until coverage reaches a fixpoint.  Operations to combine 
coverage records (eg. union, intersection, difference) are still on our 
to-do list, but should be straightforward.  I attach a README and a 
tar-ball of our current development version.  Comments welcome.  We have 
a paper in preparation and I'd be happy to send you a copy when the 
first draft is finished.

I suppose there could be scope for an hpc GUI too! :-)

Colin R

-------------- next part --------------
A non-text attachment was scrubbed...
Name: hpcproto.tar.gz
Type: application/x-gzip
Size: 84744 bytes
Desc: not available
Url : http://www.haskell.org//pipermail/hat/attachments/20060728/268488c8/hpcproto.tar-0001.bin
-------------- next part --------------
HPC PRE-RELEASE PROTOTYPE
=========================

We are distributing the prototype of a Haskell tool called hpc to a few
potential users, such as yourself.  We hope you will try it, and give us
your reactions.

*** Please do NOT distribute this prototype any further. ***

Quite soon we expect to release a version of hpc for general use. It will
be open-source.

WHAT IS HPC?
------------

Hpc is a tool-kit to record and display Haskell Program Coverage. Hpc
includes tools that instrument Haskell programs to record program
coverage, run instrumented programs, and display the coverage information
obtained.

Hpc works by applying a source-to-source transformation; this
transformation also generates as a by-product a program-index file (.pix)
and module-index files (.mix). The transformed program is compiled with a
library; in addition to its usual run-time behaviour the program generates
a coverage record in a program-ticks file (.tix).  If the program is run
more than once, coverage data is accumulated to reflect all runs.

Hpc provides coverage information of two kinds: source coverage and
boolean-control coverage. Source coverage is the extent to which every
part of the  program was used, measured at three different levels:
declarations (both top-level and local), alternatives (among several
equations or case branches) and expressions (at every level).  Boolean
coverage is the extent to which each of the values True and False is
obtained in every syntactic boolean context (ie. guard, condition,
qualifier).

Hpc displays both kinds of information in two different ways: textual
reports with summary statistics (hpc-report) and sources with colour
mark-up (hpc-source).  For boolean coverage, there are four possible
outcomes for each guard, condition or qualifier: both True and False
values occur; only True; only False; never evaluated. In hpc-source
output, highlighting with a yellow background indicates a part of
the program that was never evaluated; a green background indicates an
always-True expression and a red background indicates an always-False one.

INSTALLING HPC
--------------

Requires ghc (6.4*); hmake (3.10); linux or MacOSX; bash.

There is not yet a slick installation kit.  Pick a suitable directory in
which to build hpc and try this:

1. tar xf hpcproto.tar
2. add HPCROOT=$(pwd)/hpc to your environment
3. cd hpc
4. make
5. add $(pwd)/bin to your PATH

USING HPC
---------

There is no fancy packaging of the hpc tools, just a few shell scripts.
Try this:

1. cd to the source directory for a Haskell program
   -- let's assume the main module is in Main.hs
2. hpc-build Main [build options such as -i..., -I... or -package ...]
   -- transforms for coverage and builds new executable
   -- in a subdirectory .hpc, along with index files
   -- and copies of original sources
3. hpc-run Main [args for Main]
   -- runs the transformed Main, recording coverage info
4. hpc-report Main [module-name...]
   -- writes a textual summary of coverage to stdout
   -- if no modules specified, reports on all
5. hpc-source Main [module-name...]
   -- generates highlighted <module-name>.hs.html for each module
   -- if no modules specified, generates all

See the ${HPCROOT}/test directory for examples, including a nice
way to use hpc with QuickCheck (hpc-fix).

LIMITATIONS
-----------

* The source transformation is not informed by operator priorities.
  Operands are treated as the immediate subexpressions of any chain of
  infix applications.

* Most ghc extensions are not yet supported.

* Various minor glitches -- you will encounter them soon enough!

* Developed using ghc under linux.  However, as hpc uses very few
  extensions beyond Haskell 98 it should port readily to other compilers
  and environments.

EXTENSIONS
----------

There are various ideas for hpc that we have discussed, put aside
for now, but might come back to. They include:

* extending boolean coverage to boolean arguments of Prelude functions;

* extending boolean coverage to functions and argument positions specified
  by the user;

* generalising boolean coverage to value-coverage for other datatypes;

* success and failure coverage of argument patterns;

* path coverage in the functional dependency graph;

* indicating frequency of use, not just coverage;

* combining information from more than one coverage record.

Andy Gill (andy at galois.com) & Colin Runciman (colin at cs.york.ac.uk)

19 July 2006 (revised 28 July 2006)