[sajith at gmail.com: Google Summer of Code: a NUMA wishlist!]

[Simon Marlow]

On 28/03/2012 16:57, Tyson Whitehead wrote:
> On March 28, 2012 04:41:16 Simon Marlow wrote:
>> Sure.  Do you have a NUMA machine to test on?
>
> My understanding is non-NUMA machines went away when the AMD and Intel moved
> away from frontside buses (FSB) and integrated the memory controllers on die.
>
> Intel is more recent to this game.  I believe AMD's last non-NUMA machines
> where the Athalon XP series and Intel's the Core 2 series.
>
> An easy way to see what you've got is to see what 'numactl --hardware' says.
> If the node distance matrix is not uniform, you have NUMA hardware.
>
> As an example, on a 8 socket Opteron machine (32 cores) you get
>
> $ numactl --hardware
> available: 8 nodes (0-7)
> node 0 size: 16140 MB
> node 0 free: 3670 MB
> node 1 size: 16160 MB
> node 1 free: 3472 MB
> node 2 size: 16160 MB
> node 2 free: 4749 MB
> node 3 size: 16160 MB
> node 3 free: 4542 MB
> node 4 size: 16160 MB
> node 4 free: 3110 MB
> node 5 size: 16160 MB
> node 5 free: 1963 MB
> node 6 size: 16160 MB
> node 6 free: 1715 MB
> node 7 size: 16160 MB
> node 7 free: 2862 MB
> node distances:
> node   0   1   2   3   4   5   6   7
>    0:  10  20  20  20  20  20  20  20
>    1:  20  10  20  20  20  20  20  20
>    2:  20  20  10  20  20  20  20  20
>    3:  20  20  20  10  20  20  20  20
>    4:  20  20  20  20  10  20  20  20
>    5:  20  20  20  20  20  10  20  20
>    6:  20  20  20  20  20  20  10  20
>    7:  20  20  20  20  20  20  20  10

Well, you learn something new every day!  On the new 32-core Opteron box 
we have here:

available: 8 nodes (0-7)
node 0 cpus: 0 4 8 12
node 0 size: 8182 MB
node 0 free: 1994 MB
node 1 cpus: 16 20 24 28
node 1 size: 8192 MB
node 1 free: 2783 MB
node 2 cpus: 3 7 11 15
node 2 size: 8192 MB
node 2 free: 2961 MB
node 3 cpus: 19 23 27 31
node 3 size: 8192 MB
node 3 free: 5359 MB
node 4 cpus: 2 6 10 14
node 4 size: 8192 MB
node 4 free: 3030 MB
node 5 cpus: 18 22 26 30
node 5 size: 8192 MB
node 5 free: 4667 MB
node 6 cpus: 1 5 9 13
node 6 size: 8192 MB
node 6 free: 3240 MB
node 7 cpus: 17 21 25 29
node 7 size: 8192 MB
node 7 free: 4031 MB
node distances:
node   0   1   2   3   4   5   6   7
   0:  10  16  16  22  16  22  16  22
   1:  16  10  16  22  22  16  22  16
   2:  16  16  10  16  16  16  16  22
   3:  22  22  16  10  16  16  22  16
   4:  16  22  16  16  10  16  16  16
   5:  22  16  16  16  16  10  22  22
   6:  16  22  16  22  16  22  10  16
   7:  22  16  22  16  16  22  16  10

The node distances on this box are less uniform than yours.

Cheers,
	Simon