Performance and Capacity Planning 740

NUMA is non-uniform memory architecture.

basically take a small CEC (say one to four processor board) with its own private memory. then create a second order memory bus that interconnects multiple of the memories of multiple of such CECs in a uniform memory addressing architecture.

the latency for a CEC to get at its local memory is much shorter than the latency to get at memory located at a different CEC. the addressing is uniform ... but the latency is different.

this is somewhat analogous to extended store on the 3090 ... where it was the same memory technology but futher away and higher latency ... however in the 3090, it wasn't uniform addressing ... local memory and extended store used different memory addressing and explicity software management.

from an operational standpoint ... it somewhat harks back to LCS from the 360 days. You could get 8mbytes of 8mic letency memory from Ampex (and other vendors) that attached to various 360 models. (they were found on some number of 360-65s and 360-75s where the normal memory latency was .75mics ... 750ns).

in the late 80s time-frame and early 90s ... about the same time LLNL was pushing serial fiber channel into becoming a standard, people at SLAC was pushing SCI into being a standard. FCS & SCI used similar fiber hardware technology .... however, SCI developed architectures for asyncronous bus operation over fiber links (rather than just simply asyncronous i-o operation over fiber links). SCI asyncronous memory bus architecture supported 64 ports.

Convex used SCI and HP two-processor snake boards to build 128-way system. Sequent and DG used SCI and pentium 4-processor boards to build 256-way systems.

Convex evolved a version of MACH (from CMU) for support the 128-way Exampler ... and supported some of the same thruput trade-off decisions that should be familiar to people doing LCS support in 360 days (did you execute directly in LCS ... or copy to higher-speed storage for end). Some of NUMA thruput issues are analogous to cache considerations ... thruput is affected by latency.

Sequent enhanced their Dynix UNIX system that they had developed for supporting up to 32-way "uniform" SMP operation.

Since that time, HP absorbed Convex, DG has gone away (there is some legacy of their disk raid products around) and IBM has bought Sequent.

We were somewhat involved in both of these fiber efforts (fiber channel standard and SCI).

random past fcs, sci, &-or numa posts:

--