[sajith at gmail.com: Google Summer of Code: a NUMA wishlist!]

[Tyson Whitehead]

On March 28, 2012 04:41:16 Simon Marlow wrote:
> Sure.  Do you have a NUMA machine to test on?

My understanding is non-NUMA machines went away when the AMD and Intel moved 
away from frontside buses (FSB) and integrated the memory controllers on die.

Intel is more recent to this game.  I believe AMD's last non-NUMA machines 
where the Athalon XP series and Intel's the Core 2 series.

An easy way to see what you've got is to see what 'numactl --hardware' says.  
If the node distance matrix is not uniform, you have NUMA hardware.

As an example, on a 8 socket Opteron machine (32 cores) you get

$ numactl --hardware
available: 8 nodes (0-7)
node 0 size: 16140 MB
node 0 free: 3670 MB
node 1 size: 16160 MB
node 1 free: 3472 MB
node 2 size: 16160 MB
node 2 free: 4749 MB
node 3 size: 16160 MB
node 3 free: 4542 MB
node 4 size: 16160 MB
node 4 free: 3110 MB
node 5 size: 16160 MB
node 5 free: 1963 MB
node 6 size: 16160 MB
node 6 free: 1715 MB
node 7 size: 16160 MB
node 7 free: 2862 MB
node distances:
node   0   1   2   3   4   5   6   7 
  0:  10  20  20  20  20  20  20  20 
  1:  20  10  20  20  20  20  20  20 
  2:  20  20  10  20  20  20  20  20 
  3:  20  20  20  10  20  20  20  20 
  4:  20  20  20  20  10  20  20  20 
  5:  20  20  20  20  20  10  20  20 
  6:  20  20  20  20  20  20  10  20 
  7:  20  20  20  20  20  20  20  10 

On our more traditional NUMA there are 64 nodes and the numbers range from 
10-37.  But it's an older SGI Itanium solution, so that comes with its own set 
of problems, and most modern machines are already out performing it.

Cheers!  -Tyson