[commit: ghc] master: Bump haddock.base max_bytes_used (8df7fea)

[Simon Marlow]

On 04/08/2014 13:13, Joachim Breitner wrote:
> Hi,
>
>
> Am Montag, den 04.08.2014, 13:08 +0200 schrieb Joachim Breitner:
>> Am Montag, den 04.08.2014, 12:02 +0100 schrieb Edward Z.Yang:
>>> Yes, on my box, this test is now failing (because the stat is too good):
>>>
>>>      Expected    haddock.base(normal) max_bytes_used: 127954488 +/-10%
>>>      Lower bound haddock.base(normal) max_bytes_used: 115159039
>>>      Upper bound haddock.base(normal) max_bytes_used: 140749937
>>>      Actual      haddock.base(normal) max_bytes_used: 113167424
>>>      Deviation   haddock.base(normal) max_bytes_used:     -11.6 %
>>
>> ugh.
>>
>> What are your compilation settings? Plain "validate"?
>>
>> Looks like the ghcspeed instance settings still don’t quite match what
>> validate does...
>>
>> But I don’t see anything in
>>          mk/validate-settings.mk
>> which would yield different results than
>>          echo 'GhcLibHcOpts += -O -dcore-lint'  >> mk/build.mk
>>          echo 'GhcStage2HcOpts += -O -dcore-lint'  >> mk/build.mk
>>
>> I’m starting a plain validate run on that machine, to see if it is
>> compilation settings or some other variable.
>
> validate goes through without a problem. So it seems to be dependent on
> other things.
>
> Are these very flaky measures (max_bytes_used) at all useful? So far, I
> have only seen friction due to them, and any real problem would likely
> be caught by either bytes_allocated or nofib measurements (I hope).
> Maybe we should simply remove them from the test suite, and stop
> worrying?

 From testsuite/tests/perf/compiler/all.T:

# Note [residency]
#
# Residency (peak_megabytes_allocated and max_bytes_used) is sensitive
# to when the major GC runs, which makes it inherently inaccurate.
# Sometime an innocuous change somewhere can shift things around such
# that the samples occur at a different time, and the residency
# appears to change (up or down) when the underlying profile hasn't
# really changed.
#
# However, please don't just ignore changes in residency.  If you see
# a change in one of these figures, please check whether it is real or
# not as follows:
#
#  * Run the test with old and new compilers, adding +RTS -h -i0.01
#    (you don't need to compile anything for profiling or enable profiling
#    libraries to get a heap profile).
#  * view the heap profiles, read off the maximum residency.  If it has
#    really changed, then you know there's an issue.

This advice is hard to follow for the Haddock tests, because the test 
doesn't actually run anything, it just uses the information previously 
collected.  I think we should probably remove the max_bytes_used limits 
for the Haddock tests.

Cheers,
Simon