[GHC] #10951: HPC program has poor error reporting / strange CLI in general

[GHC]

#10951: HPC program has poor error reporting / strange CLI in general
-------------------------------------+-------------------------------------
           Reporter:  mgsloan        |             Owner:
               Type:  bug            |            Status:  new
           Priority:  low            |         Milestone:
          Component:  Code Coverage  |           Version:  7.10.2
           Keywords:                 |  Operating System:  Unknown/Multiple
       Architecture:                 |   Type of failure:  None/Unknown
  Unknown/Multiple                   |
          Test Case:                 |        Blocked By:
           Blocking:                 |   Related Tickets:
Differential Rev(s):                 |
-------------------------------------+-------------------------------------
 1. Many erroneous usages of the hpc program result in reporting 100% (0/0)
 coverage. Proposed resolution:

 * Report an error and exit with failure.  I can't think of a legitimate
 case where you would want a report that doesn't consider any expressions /
 declarations.

 * If a --include flag is specified, I think it would be valuable to output
 all the module names, so that it's clearer why the --include flag didn't
 work.

 * At the very least, (0/0) coverage shouldn't be reported as "100%".

 2. Positional arguments after the initial tix file get interpreted as
 --include filters, filtering which modules / packages are considered.  By
 default, HPC includes all modules in the report, unless include filters
 are supplied.  If you accidentally provide a positional argument which
 isn't a valid thing for "--include", then your reports will always be
 (0/0) coverage.  Proposed resolution:

 * Deprecate using positional arguments for include filters.  This way it
 will complain about this instead of yielding trivial (0/0) coverage
 reports

 3. Ideally, you'd be able to pass in multiple tix files and combine them
 in memory instead of using "hpc combine" and generating intermediate tix
 files.  This would be a better usage of positional arguments than include
 filters.  For now, the additional tix files would need to be flag
 arguments.  This also isn't all that useful until
 https://ghc.haskell.org/trac/ghc/ticket/1853 is fixed.

 4. `--srcdir`, `--hpcdir`, and `--reset-hpcdirs` interface has strange
 semantics for searching for mix files.  It literally does something
 roughly like `[sd </> hd | sd <- srcDirs, hd <- hpcDirs]`, and uses this
 when looking for mix files.  This would be OK, except that the `srcdir`
 must be the work dir used for compilation, in order for source markup to
 be generated.  This means that you are tied to having your hpcdir be a
 sub-directory of the compilation work dir.  Proposed resolution:

 * Add `--mixdir`, specifying a path to a mix file directory.  Can be
 supplied multiple times.  If `--mixdir` is provided, then it's an error to
 also use `--hpcdir` / `--reset-hpcdirs`.

 Instead of fixing the hpc program, I have considered forking it and giving
 it a new executable name, so that the CLI and behavior can be entirely
 changed.  Thoughts on this alternative?  It is appealing because it seems
 tricky to fix the hpc program's CLI while ensuring that backwards
 compatibility is maintained.

 I am keen on working on fixing this situation, but I want consensus from
 y'all about the approach that should be taken.

 Maybe it makes sense for this hpc-ng program to be developed outside of
 the GHC repo?  This way it can use more recent libraries for generating
 HTML and parsing command line arguments.

--
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/10951>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler