Validating with Haddock

[Austin Seipp]

For the record and other people reading - after a quick discussion on
IRC, it simply looks like the 32-bit peak_megabytes_allocated numbers
for those tests probably weren't updated at the same time as the 64bit
ones, leaving them out of date.

On Tue, Jan 7, 2014 at 3:07 PM, Mateusz Kowalczyk
<fuuzetsu at fuuzetsu.co.uk> wrote:
> On 07/01/14 20:15, Ian Lynagh wrote:
>> On Tue, Jan 07, 2014 at 06:39:36PM +0000, Mateusz Kowalczyk wrote:
>>> On 07/01/14 18:21, Austin Seipp wrote:
>>>>
>>>> Also, the performance failures you're seeing are (I speculate) due to
>>>> out of date performance numbers. Sometimes these numbers go up or down
>>>> just due to code churn, but they're sometimes finnicky, because they
>>>> may depend on the exact time a major GC happens or something. So a
>>>> small wibble can cause them to sometimes occasionally fail.
>>>
>>> These are the numbers from the clean tree.
>>
>> The haddock perf numbers look pretty bad, especially the
>> peak_megabytes_allocated:
>>
>> =====> haddock.base(normal) 429 of 3855 [0, 0, 0]
>> peak_megabytes_allocated value is too high:
>>     Expected    peak_megabytes_allocated: 139 +/-1%
>>     Actual      peak_megabytes_allocated: 180
>>
>> =====> haddock.Cabal(normal) 430 of 3855 [0, 1, 0]
>> peak_megabytes_allocated value is too high:
>>     Expected    peak_megabytes_allocated:  89 +/-1%
>>     Actual      peak_megabytes_allocated: 150
>>
>> =====> haddock.compiler(normal) 431 of 3855 [0, 2, 0]
>> max_bytes_used value is too high:
>>     Expected    peak_megabytes_allocated: 663 +/-1%
>>     Actual      peak_megabytes_allocated: 794
>>
>> I think it would be worth working out what's going on before merging
>> more haddock changes.
>>
>>
>> Thanks
>> Ian
>>
>
> Hi Ian,
>
> Is there any guidance on how these tests are performed? More
> importantly, is there any log of how the performance changed over time?
> Is it Haddock's fault that it has become slower or is it the cause of
> GHC changes?
>
> PS: If there's no performance over time log, it might be worth
> introducing something!
> --
> Mateusz K.
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://www.haskell.org/mailman/listinfo/ghc-devs
>



-- 
Regards,

Austin Seipp, Haskell Consultant
Well-Typed LLP, http://www.well-typed.com/