50th Anniversary of invention of disk drives 4591

there was another "comprise" between disks and high speed core for 360s. disks (drums, datacells, etc) were referred to as "DASD" (direct access storage device) ... more specifically "CKD" DASD (count-key-data).

the trade-off was extremely scarce real storage vis-a-vis realatively abundant i-o resources. typically, filesystems have an index of where things are on the disk. most systems these days, use the relatively abundant real storage to cache these indexes (in addition to caching the data itself). however of 360, the indexes were kept on disk (saving real storage).

CKD allowed for essentially allowed filesystem metadata to be written along with the data itself. the indexes were kept on disk with filesystem metadata indexes. rather than reading the indexes into real storage (and possibly caching them), CKD DASD i-o programming provided for doing a sequential search of the indexes on disk ... trading off scarce real storage for abundant i-o capacity.

however, by at least the mid-70s, the trade-off was reversing ... with real storage starting to become abundant and disk i-o was becoming more and more of a system bottleneck.

in the late 70s, i was brought in to investigate a severe throughput-performance problem for a large national retail chain. they had central dataprocessing facility providing support for all stores nationally ... with several clustered mainframes sharing common application library. it turns out that the CKD program library dasd-disk search was taking approx. 1-2 second elapsed time (actual program load took maybe 10-20 milliseconds ..., but the on-disk index serial search was taking 500 milliseconds) and all retail store software application program loads were serialized through this process.

this trade-off left-over from the mid-60s included having the argument for the on-disk serial search kept in processor real storage (further optimizing real storage constraint) ... however it required that there was a dedicated exclusive i-o path between the device and the processor real storage for the duration of the search. this further exasherbated the throughput. typically multiple disks (between 8 to 32) might share a common disk controller and i-o channel-bus. not only was the disk performing the search, busy for the duration ... but because of the requirement for the dedicated open channel between the disk and processor storage (for accessing the search argument) busy for the duration of the search ... it wasn't possible to perform any operations for any of the other disks (sharing the same controller and-or i-o channel-bus).

misc. past posts discussing this subject

... not the above is a different collection of posts than

which primarily references working with the people in bldg. 14 (disk engineering) and bldg. 15 (disk product test) on the san jose plant site.

in any case, this and other factors prompted my observation that over a period of ten to fifteen years, disk relative system performance had declined by an order of magnitude i.e. other system resources increased by a factor of fifty while disk resources (in terms of operations per second) increased by possibly only a factor of five.

the initial take was that the disk division buttigned their disk performance and modeling group to refute my statements ... however, after several weeks they came back and said that I may have actually slightly understated the issue.

the change in the relative thruput of different system components ... especially with respect to each other ... results in having to change in various strategies and trade-offs ... which is also somewhat the recent thread from comp.arch

another series of posts about similar change in disk-memory trade-offs involves system-r ... original relational-sql

and RDBMS. in the 70s, there were something of pro-con argument between the people in santa teresa lab (bldg 90) dealing with 60s "physical" databases and system-r work going on in bldg. 28. the stl people were claiming that system-r indexes doubled the typical physical disk space requirements and significantly increased the search time to find a specific record (requiring potentially reading multiple different indexes). this was compared to the 60s physical databases were physical record pointers were exposed as part of the data paradigm.

the counter argument was that there was significant manual and administrative effort required to managed the exposed physical record pointers ... that were eliminated in the RDBMS paradigm.

what you saw going into the 80s, was the significant increase in disk space (the number of bits per disk arm increased by an order of magnitude, the the disk arm accesses-sec only showed slight improvement) and the significant decrease in the price per megabyte of disk space ... somewhat made the issue mute about the size of the RDBMS indexes. furthermore, the ever increasing abundance of real storage made it possible to cache a significant portion of RDBMS index in real storage (eliminating the significant number of additional I-Os to process the index ... vis-a-vis the physical databases from the 60s).

the issue during the 80s for RDBMS was that relative importance of the "cons" against RDBMS were significantly reduced ... while the "cons" against the 60 physical databases (manual people time and expertise) significantly increased. a few past posts on the changing relative amount of different resources for RDBMS:

misc. other past posts about change in relative system thruput and performance of various system components over a period of years

other posts in this disk thread: