[Haskell-cafe] Mapping of async tasks to threads?

[Viktor Dukhovni]

I have an application that is largely latency-bound, it
operates on hundreds of concurrent DNS lookup tasks,
to process workloads of millions to tens of millions
such tasks.

Each task is spawned via Control.Concurrent.Async.async,
and the code is compiled with "-threaded", and runs with
the "capability" count set to 4:

  $ ps -o command= -p $(pgrep -x danescan)
  danescan ... +RTS -N4 ...

The main task collects batches of outputs from the worker
tasks and stores the results in a database.  With the worker
count set to 350, looking at per-thread "ps" output on FreeBSD,
I see 24 threads of which 3 look idle, 20 seem to be doing the
DNS work (with a few noticeably less than the rest), and one
is the main thread doing the database updates:

  $ ps -o pid,lwp,time,mwchan,msgrcv,msgsnd,state -H -p $(pgrep -x danescan) | sort -k5n
    PID    LWP    TIME MWCHAN   MSGRCV   MSGSND STAT
  38083 100763 0:00.00 uwait         0        0 I+
  38083 101152 0:00.00 uwait         0        0 I+
  38083 101922 0:00.00 uwait         0        0 I+
  38083 101343 0:02.29 uwait     27194    27627 I+
  38083 101335 0:04.30 uwait     51689    52517 I+
  38083 101391 0:13.33 uwait    173618   177160 S+
  38083 101917 0:22.63 kqread   295420   300894 S+
  38083 101362 0:28.73 uwait    371419   378350 I+
  38083 101349 0:46.11 uwait    594307   604816 I+
  38083 101352 0:49.65 uwait    641703   653557 S+
  38083 101370 0:53.93 select   699057   712217 S+
  38083 101358 0:58.15 uwait    757636   771437 S+
  38083 101376 1:00.84 kqread   794335   809328 S+
  38083 101375 1:02.31 uwait    806276   821284 S+
  38083 101342 1:02.34 uwait    809367   823963 S+
  38083 101244 1:03.26 select   812395   827329 S+
  38083 101360 1:03.47 uwait    815955   831710 S+
  38083 101356 1:03.03 uwait    818215   833649 I+
  38083 101364 1:03.54 uwait    825807   841300 S+
  38083 101347 1:03.89 uwait    832981   848893 S+
  38083 101334 1:04.42 kqread   836132   851386 S+
  38083 101357 1:06.04 uwait    859308   875754 S+
  38083 101134 1:06.67 kqread   859410   875176 S+
  38083 100655 5:36.98 uwait  11780649 11780685 S+

The number of open UDP DNS query sockets matches what I
expect (each active query uses a fresh socket, closing
it when the results arrive): 

  $ lsof -n -P -i -a -p $(pgrep -x danescan) | egrep -c '(TCP|UDP).*:53$'
  349

How is the actual thread count related to the RTS "-N" option?
I did not expect to see 20 threads...

-- 
	Viktor.