IBM 610 workstation computer 3452

+--------------- Using any approach requiring a global system lock seems to prevent any system from scaling to more than 4-5 CPUs available for useful work. +---------------

Yup, exactly. That's when you have to start spending serious bucks on re-architecting the kernel to replace the BKL (Big Kernel Lock) a.k.a. "spl7()" to old-timers! with multiple fine-grained data structure locks -- not just per-process, per-CPU, per-network-socket, etc., but often per-queue *within* those structures. E.g., ISTR that the "ifnet" struct (the per-interface structure in BSD-style networking) in the Irix kernel had 4 or 5 distinct *pieces* of it that could be locked separately. While this may seem complex, it really paid off -- a single high-speed streaming TCP connection could easily bring 3 CPUs to bear on the task: one for the user process that was doing the output, one for transmit & receive interrupts, and a third for the "netisr()" upcall processing of received data (mainly the ACKs, in this case).

Similar parallelism gains were realized in the filesystem: In order to reach the 7.5-10 GB-s rates that were demonstrated for a single Irix user process reading a single file, there had to be 8-11 CPUs brought to bear -- the user CPU and roughly one CPU full of filesystem activity in the kernel for each GB-s of read performance. Remember, this was back in the days of "merely" fast-and-wide SCSI, not even "ultra", IIRC. Getting those bandwidths required that the filesystem be striped across more than a hundred(!) SCSI controllers, each with 4-5 disks on it.

-Rob

----- San Mateo, CA 94403 (650)572-2607