virtual memory 4480

another characteristic of the real memory resource vis-a-vis disk resource trade-off was the original os-360 design using CKD DASD.

the disk technology was referred to count-key-data ... it was possible to format disk space with specified record sizes and tag each record with various flags and attributes (not the fixed record size and sequential numbered records that you find in much of today's much more familiar disk technology). os-360 made a decision about conserving real storage requirements by leaving the disk layout information resident on disk. the "VTOC" (volume table of contents, somewhat akin to a MFD-master file directory) had the individual records labled. When there was a request to open some file, an I-O request was built by the system to "search" the VTOC for the required information.

a similar design trade-off was created for file libraries that had a special file format called PDS (parbreastioned data set) that included a on-disk directory of all the members in the file. when there was a request to retrieve a specific library member, there was an I-O request built by the system to do ("multi-track") search of the PDS directory for the required information.

these two design trade-offs required exorbitant amounts of disk I-O resources ... however, the savings in scarce real memory resource was deemed to be a reasonable trade-off.

roll-forward to mid-70s ... when the amounts of available real memory was starting to dramatically increase ... and the improvement in disk throughput was dramatically falling behind the thruput improvement of other system components. as a result, major system bottleneck was shifting from real storage to disk activity.

a widely deployed disk drive was the 3330 (a lots of other vendors produced clones of the 3330 that made their way into a variety of systems). it had 20 surfaces and 20 heads per (cylinder) arm position ... only 19 of the heads were addressable (the 20th head was used for something called rotational position sensing). the device spun at 3600 rpm ... or 60 revolutions per second. a multi-track search of a multi-cylinder PDS directory took approx. 1-3 of a second elapsed time ... per cylinder (arm position). The avg. elapsed time to find a member in a large program library could take 1-2 second or more elapsed time.

in the late 70s, a very large national retailer was starting to experience severe performance degradation of its in-store, online applications. the retailer had its stores organizied into regions ... and the datacenter had dedicated processor complex per region. however, all the processor complexes had shared access to a large common pool of disks. there was a single large application library for the online, in-store operations that was shared across all the dedicated regional processor complexes.

over period of several months, numerous people had been brought in attempting to diagnose the problem. finally i was brought into a clbutt room that was had a dozen or so long clbutt tables. all the tables were nearly covered with high stacks of paper performance reports from all the operating systems.

people were expected to scan the performance reports looking for resource activity that corresponded with periods of extremely poor performance. the problem was that most people were accustomed to looking at extremely high activity counts as an indication of bottlenecked resources. all the periodic disk i-o counts failed to indicated any specific disk in the large pool (disk drives in the large score of drives) with high i-o count activity.

one problem was that the performance reports gave interval i-o activity counts by disk for specific processor complex. to get the total i-o activity for a disk ... you had to manually sum the i-o count activity per processor complex across all the processor complexes. the other problem was that there was no actual service time numbers or queuing delay numbers ... just raw activity counts.

I started quickly scanning the piles of paper and after maybe 20-30 minutes started to realize that a specific disk drive had an aggregate activity count consistently of approx. 6-7 operations per second during the periods of bad performance and thruput. it wasn't normally considered a high i-o rate ... but it started to appear to be the only value that consistently correlated with periods of poor performance. all the other values reported in the reams of paper performance data appeared to pretty much randomly vary all over the place.

zero'ing on that particular disk ... closer examination showed that it was typically doing a full-cylinder multi-track search of the PDS directory (to find a library member to load) taking nearly 1-3rd second elapsed time ... followed by reading the library memory (taking a couple tens of millisecond). basically the whole national datacenter complex was reduced to loading an aggregate of three application library programs per second.

all the system configuration easily had sufficient real memory for caching the PDS directory of library members ... and the in-store application load would be reduced to possibly 10-15milliseconds rather than 340milliseconds. however, the system implementation structure had been cast in concret from the mid-60s ... so they had to find other compensating procedures to speed up in-store application load.

furthermore, within a few years, typical system configurations would not only be able to cache the directory of members but also all the members themselves.

misc. past posts about the 60s ckd-disk-storage performance trade-off and its lingering effects long after it was no longer valid

--