Nightlies

[Austin Seipp]

FWIW, this is pretty much what it was going to do. Except it also
needs to host things like publicly accessible binary snapshots, as
people want to use them. And that's a no-go for firewalls or otherwise
non-controllable infrastructure that people may have bots on, so it
must have a central place for the results to be located on
haskell.org. It can then send summarized reports to the list based
e.g. on a cron job (asking bot runners to manage their own emails for
bots is annoying and painful, and doesn't scale nicely for them as we
add more.) And also, of course, there needs to be some aspect of
non-repudiation to the results, so that people know builds and emails
are legitimate (i.e. signed by an GPG pubkey and verified on the
server.)

On Sat, Jan 25, 2014 at 9:50 PM, Mateusz Kowalczyk
<fuuzetsu at fuuzetsu.co.uk> wrote:
> On 26/01/14 03:29, Austin Seipp wrote:
>> As of right now, Pali's FreeBSD builds seem to be the only nightly
>> that is still consistently running (and thanks to him for that!)
>>
>> The build infrastructure in its current status is mainly just
>> 'unmaintained'. Furthermore there's not really a good roster of
>> machines that were/were not part of the system AFAIK aside from the
>> old list, and it's unclear what the status of many of those machines
>> are (as you said, many haven't checked in in a while.)
>>
>> There is much interest in a better nightly infrastructure and people
>> have asked me several times about setting one up on IRC. We have
>> historically had some problems with the nightly infrastructure, mainly
>> things like network disconnectivity or firewalling policies, since
>> most people aren't running dedicated internet facing machines (or even
>> a dedicated machine at all. Firewalls have been a problem for places
>> like MSR from what I understand.)
>
> Why not simply have the clients post the results once a night? If the
> builds are nightly, is there really any need to have an open daemon
> listening? From what I can tell from
> http://darcs.haskell.org/ghcBuilder/builders/ it is simply the matter of
> building once a day/night and then posting the results in an e-mail to
> the list and uploading the binaries and test results elsewhere. Could we
> not simply have a wrapper script around GHC build process that in the
> end posts all these results to relevant places? The clients could simply
> have a nightly cron job and it'd be up to the slave owner to keep these
> builds going as often or as rarely as they want. The only downside is
> that you guys can't tell the clients precisely when to run but looking
> at build times, it's only once a day anyway.
>
>> Several individual people run Jenkins individually, and I like it, but
>> I'm not sure how well it does when spread across the globe in terms of
>> networking (and realistically builders will look like that, as we
>> can't possibly have a dedicated farm somewhere.) I was also at one
>> point worried about the size of such a tool on systems like ARM
>> machines where resources are at a premium, but in hindsight this looks
>> OK. I'd like any opinions on this if people have deployed things in
>> these highly distributed scenarios.
>>
>> I have had some ideas for an extremely-minimal nightly build
>> infrastructure that would ideally require minimal setup and let
>> clients have power over choosing how and when to build, but I have yet
>> to find the time to finish the basic implementation to try it.
>>
>>
>> On Sat, Jan 25, 2014 at 8:22 PM, Mateusz Kowalczyk
>> <fuuzetsu at fuuzetsu.co.uk> wrote:
>>> Hi all,
>>>
>>> I'd just like to query the status of the nightly builds. Is anything
>>> happening in that area? [1] is right on the front page of the GHC Trac
>>> even though no builds were ran for ~5 months. Perhaps it should be
>>> moved out of the way if there's no plan to resume these in the near
>>> future.
>>>
>>> Does anything specific need doing to get these to run again?
>>>
>>> [1]: https://ghc.haskell.org/trac/ghc/wiki/Builder
>>> --
>>> Mateusz K.
>>> _______________________________________________
>>> ghc-devs mailing list
>>> ghc-devs at haskell.org
>>> http://www.haskell.org/mailman/listinfo/ghc-devs
>>>
>>
>>
>>
>
>
> --
> Mateusz K.
>



-- 
Regards,

Austin Seipp, Haskell Consultant
Well-Typed LLP, http://www.well-typed.com/