[Haskell-cafe] Where did these threads come from?

[Gregory Collins]

What idle GC does for you is tries to make you take the latency hit for
garbage collection when your program is not CPU-bound, rather than doing it
when you hit an arbitrary allocation threshold (probably during request
processing when you're actually doing work). So in that case it's a good
thing, yes --- but for yours I'd probably turn it off too. Other tweaks you
can try include tweaking the RTS gc settings if you haven't already,
especially increasing -A.

On Fri, Nov 13, 2015 at 3:50 AM, David Turner <dct25-561bs at mythic-beasts.com
> wrote:

> Ah yes, and the more obvious question: should I just run it with -I0? I'm
> kind of assuming that the idle GC is a Good Thing since it's on by default.
>
> Cheers,
>
> On 13 November 2015 at 11:11, David Turner <dct25-561bs at mythic-beasts.com>
> wrote:
>
>> Hi all,
>>
>> I'm working on a server process that has (by design) fairly high memory
>> usage but serves relatively few requests and spends most of its time idle.
>> By 'fairly high' I mean 100s of MB, nothing ridiculous. I'm trying to
>> improve its quiescent CPU usage as this is currently the limiting factor
>> for how cheaply we can run this service - we're aiming for the cheapest
>> t-class EC2 machine we can get.
>>
>> The quiescent CPU usage is currently about 50%. This seems to be largely
>> due to the idle GC, because running with -I0 cuts it down to about 0.3%.
>> Pretty much every time any thread wakes up it triggers the idle GC a short
>> while later and because of the high memory usage these GCs are somewhat
>> expensive.
>>
>> Using threadscope I found the biggest problem was the reaper thread in
>> resource-pool which wakes up every second whether there's anything to reap
>> or not. Fortunately this was easy to spot as the reaper thread labels
>> itself as such. I've replaced this with an event-driven reaper that only
>> wakes up on demand and cut the baseline from 50% down to just under 10%.
>>
>> There are two more threads I can now see waking up periodically in
>> threadscope, with respective periods approximately 10 sec and 30 sec. I
>> have a hunch what the 30-sec one is but no idea about the 10-sec one. If I
>> can kill off the 10 sec one then that should take CPU usage down to <5%
>> which would be just fine.
>>
>> My question: is there any reliable way to determine where these threads
>> came from, in terms of the location of the forkIO call that made them? I've
>> just been looking for likely candidates and labelling threads and this is
>> rather tedious.
>>
>> Cheers,
>>
>> David
>>
>
>
> _______________________________________________
> Haskell-Cafe mailing list
> Haskell-Cafe at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
>
>


-- 
Gregory Collins <greg at gregorycollins.net>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/haskell-cafe/attachments/20151113/432cf64f/attachment.html>