[commit: ghc] master: rts: Add api to pin a thread to a numa node but without fixing a capability (122d826)

[git at git.haskell.org]

Repository : ssh://git@git.haskell.org/ghc

On branch  : master
Link       : http://ghc.haskell.org/trac/ghc/changeset/122d826d1d1b7ba6e73866331863fa1e0b3e99ea/ghc

>---------------------------------------------------------------

commit 122d826d1d1b7ba6e73866331863fa1e0b3e99ea
Author: Darshan Kapashi <darshankapashi at gmail.com>
Date:   Thu Nov 10 15:21:08 2016 -0500

    rts: Add api to pin a thread to a numa node but without fixing a capability
    
    `rts_setInCallCapability` sets the thread affinity as well as pins the
    numa node. We should also have the ability to set the numa node without
    setting the capability affinity. `rts_pinNumaNodeForCapability` function
    is added and exported via `RtsAPI.h`.
    
    Previous callers of `rts_setInCallCapability` should now also call
    `rts_pinNumaNodeForCapability` to get the same effect as before.
    
    Test Plan:
      ./validate
    
    Reviewers: austin, simonmar, bgamari
    
    Reviewed By: simonmar, bgamari
    
    Subscribers: thomie, niteria
    
    Differential Revision: https://phabricator.haskell.org/D2637
    
    GHC Trac Issues: #12764


>---------------------------------------------------------------

122d826d1d1b7ba6e73866331863fa1e0b3e99ea
 includes/RtsAPI.h | 24 ++++++++++++++++++++++++
 rts/Task.c        | 18 +++++++++++++-----
 2 files changed, 37 insertions(+), 5 deletions(-)

diff --git a/includes/RtsAPI.h b/includes/RtsAPI.h
index 230c982..3b6e1dc 100644
--- a/includes/RtsAPI.h
+++ b/includes/RtsAPI.h
@@ -172,6 +172,24 @@ void rts_unlock (Capability *token);
 // when there is no current capability.
 Capability *rts_unsafeGetMyCapability (void);
 
+/* ----------------------------------------------------------------------------
+   Which cpu should the OS thread and Haskell thread run on?
+
+   1. Run the current thread on the given capability:
+     rts_setInCallCapability(cap, 0);
+
+   2. Run the current thread on the given capability and set the cpu affinity
+      for this thread:
+     rts_setInCallCapability(cap, 1);
+
+   3. Run the current thread on the given numa node:
+     rts_pinThreadToNumaNode(node);
+
+   4. Run the current thread on the given capability and on the given numa node:
+     rts_setInCallCapability(cap, 0);
+     rts_pinThreadToNumaNode(cap);
+   ------------------------------------------------------------------------- */
+
 // Specify the Capability that the current OS thread should run on when it calls
 // into Haskell.  The actual capability will be calculated as the supplied
 // value modulo the number of enabled Capabilities.
@@ -185,6 +203,12 @@ Capability *rts_unsafeGetMyCapability (void);
 // specified capability, set by either +RTS -qa or +RTS --numa.
 void rts_setInCallCapability (int preferred_capability, int affinity);
 
+// Specify the CPU Node that the current OS thread should run on when it calls
+// into Haskell. The argument can be either a node number or capability number.
+// The actual node will be calculated as the supplied value modulo the number
+// of numa nodes.
+void rts_pinThreadToNumaNode (int node);
+
 /* ----------------------------------------------------------------------------
    Building Haskell objects from C datatypes.
    ------------------------------------------------------------------------- */
diff --git a/rts/Task.c b/rts/Task.c
index 253520f..8ce4ecc 100644
--- a/rts/Task.c
+++ b/rts/Task.c
@@ -506,11 +506,19 @@ void rts_setInCallCapability (
         if (RtsFlags.ParFlags.setAffinity) {
             setThreadAffinity(preferred_capability, n_capabilities);
         }
-        if (RtsFlags.GcFlags.numa) {
-            task->node = capNoToNumaNode(preferred_capability);
-            if (!DEBUG_IS_ON || !RtsFlags.DebugFlags.numa) { // faking NUMA
-                setThreadNode(numa_map[task->node]);
-            }
+    }
+#endif
+}
+
+void rts_pinThreadToNumaNode (
+    int node USED_IF_THREADS)
+{
+#ifdef THREADED_RTS
+    if (RtsFlags.GcFlags.numa) {
+        Task *task = getTask();
+        task->node = capNoToNumaNode(node);
+        if (!DEBUG_IS_ON || !RtsFlags.DebugFlags.numa) { // faking NUMA
+            setThreadNode(numa_map[task->node]);
         }
     }
 #endif