Validating with Haddock

[Mateusz Kowalczyk]

On 07/01/14 20:15, Ian Lynagh wrote:
> On Tue, Jan 07, 2014 at 06:39:36PM +0000, Mateusz Kowalczyk wrote:
>> On 07/01/14 18:21, Austin Seipp wrote:
>>>
>>> Also, the performance failures you're seeing are (I speculate) due to
>>> out of date performance numbers. Sometimes these numbers go up or down
>>> just due to code churn, but they're sometimes finnicky, because they
>>> may depend on the exact time a major GC happens or something. So a
>>> small wibble can cause them to sometimes occasionally fail.
>>
>> These are the numbers from the clean tree.
> 
> The haddock perf numbers look pretty bad, especially the
> peak_megabytes_allocated:
> 
> =====> haddock.base(normal) 429 of 3855 [0, 0, 0]
> peak_megabytes_allocated value is too high:
>     Expected    peak_megabytes_allocated: 139 +/-1%
>     Actual      peak_megabytes_allocated: 180
> 
> =====> haddock.Cabal(normal) 430 of 3855 [0, 1, 0]
> peak_megabytes_allocated value is too high:
>     Expected    peak_megabytes_allocated:  89 +/-1%
>     Actual      peak_megabytes_allocated: 150
> 
> =====> haddock.compiler(normal) 431 of 3855 [0, 2, 0]
> max_bytes_used value is too high:
>     Expected    peak_megabytes_allocated: 663 +/-1%
>     Actual      peak_megabytes_allocated: 794
> 
> I think it would be worth working out what's going on before merging
> more haddock changes.
> 
> 
> Thanks
> Ian
> 

Hi Ian,

Is there any guidance on how these tests are performed? More
importantly, is there any log of how the performance changed over time?
Is it Haddock's fault that it has become slower or is it the cause of
GHC changes?

PS: If there's no performance over time log, it might be worth
introducing something!
-- 
Mateusz K.