Unreliability of the build system

[Gabriel Dos Reis]

On Fri, May 17, 2013 at 8:52 AM, Ian Lynagh <ian at well-typed.com> wrote:
> On Wed, May 15, 2013 at 12:25:14PM -0700, David Terei wrote:
>>
>> I'd point out though that GHC's 'no validate failures' is very
>> frequently violated which did confuse me a lot when getting started.
>
> This is at least partly caused by inconsistencies in test results. For
> example, a Linux developer might push a change that causes a failure on
> OS X, or one person might push a change that means that a perf test
> fails on someone else's machine but not his own (or perhaps even fails
> randomly, and they got lucky on their validate run).
>
> The first person is not aware that anything is wrong, and the second
> person just ignores the failure because their patches didn't cause it.
>
> In an ideal world, the second person would either fix the bug, revert
> thc bad change, or file a ticket and mark the test as broken.
> Unfortunately, the second person is generally trying to do something
> else when they discover the problem, and doesn't want to spend time
> fixing something they didn't break.

Can't the build bots instructed to systematically fire up on every
commit for testing on the various platforms you currently do?  Maybe
you probably want to optimize for the cases of "whitespaces only"
patches.

Volunteers have set up something similar to this and it has
helped up tremendously in the GCC land.

-- Gaby