Validating with Haddock

[Austin Seipp]

Yes, the skipped tests are normal. The testsuite has a concept of
tests being built a certain 'way' - for example, you might test a
piece of code by making sure it works compiled with -threaded,
non-threaded, profiling, the LLVM backend, or any combination of
those, etc. So a single *test* gives rise to multiple *test cases*.

When you run validate, it runs it in a 'fast' mode by default as
opposed to the slow mode. The fast mode only runs a subset of the
overall test cases - it runs the most basic tests per file, which
generally gives a pretty good indication as to what is going on.

Also, the performance failures you're seeing are (I speculate) due to
out of date performance numbers. Sometimes these numbers go up or down
just due to code churn, but they're sometimes finnicky, because they
may depend on the exact time a major GC happens or something. So a
small wibble can cause them to sometimes occasionally fail.

In any case, these results seem to indicate your branch looks quite
OK, so I can try to merge this soon, if you think it is actually
complete and ready.

On Tue, Jan 7, 2014 at 12:13 PM, Mateusz Kowalczyk
<fuuzetsu at fuuzetsu.co.uk> wrote:
> On 07/01/14 14:42, Austin Seipp wrote:
>> Hi Mateusz,
>>
>> I remember your email and I believe I responded with the OK at the
>> time - my impression was that it was ready to be merged and would
>> shortly be done after that, but I didn't hear anything back about it.
>> I apologize for my dropping the ball.
>
> We contacted you because we thought it wouldn't be this much trouble
> to get an OK from the validate process. The code was technically more
> or less ready months ago, although Simon has been making some changes
> here and there.
>
>> As for your actual error - ghc-paths is only used in Haddock when it's
>> not built in the GHC tree (as per the cabal file,) so I find it very
>> suspicious that your package check is mentioning it at all (it's not
>> mentioned anywhere else in any GHC sources.) Can you verify that it's
>> there with `./inplace/bin/ghc-pkg list`? I'm not even sure how it
>> could possibly get involved.
>>
>> Finally, can you be more specific about exactly how you tested these
>> changes with your modified Haddock? I presume it was something like:
>
> I had ran it on a as-is tree so that I could compare the results from
> before and after I put my changes in. I had just ran validate
> yesterday again (after sync-all) and I no longer get this package failure!
>
>> $ ... clone ghc source ...
>> $ cd ghc
>> $ ... get extra stuff with ./sync-all ...
>> $ cd utils/haddock
>> $ ... use git to grab your code from github ...
>> $ cd ../..
>> $ sh ./validate
>
> As I mention, it was on unchanged tree but this is how I'll do it when
> testing the changes.
>
>> But I'd like to make sure I know exactly what's going on. I can try
>> testing your branch later today.
>
> I think the original issue is now gone. I do get 8 unexpected failures
> and about 11000+ skipped tests! Is this normal? Should I be filing
> bugs? Should I create a separate thread? Can someone look at my log?
> You can download it from [1], it's about 8MB. If GHC itself compiles
> the test information into some files you'd prefer, please let me know,
> I simply redirected all output from the validate script into a log.
>
> [1]: http://fuuzetsu.co.uk/misc/validatelog
>
> --
> Mateusz K.
>



-- 
Regards,

Austin Seipp, Haskell Consultant
Well-Typed LLP, http://www.well-typed.com/