CA1153126A - Virtual storage system and method - Google Patents

Abstract

Abstract of the Disclosure A data storage management system and method for use in conjunction with a "host" digital computer. The system responds to requests by the host computer to store and to retrieve information in the format of a conventional tape drive assembly. The data, however, is actually stored or retrieved in piecemeal form from a plurality of disk storage elements so that the data management system functionally mimics a tape drive and accepts and produces data in serial (tape) format, but in fact stores the data in available spaces in disk units.

Description

~31~:6 lhe present invention relates to means for storing digital data, and more particularly to an improved system and method for managing and allocating data in a disk storage arrangement.
One of the main considerations in designing and operating a digital data system is the provision of suitable means for storing information receiv-ed from the central processing unit (CPU) of the computer and providing appropriate information to the CPU when it is requested.
It is generally conceded that both magnetic tape and disk storage systems have different and unique functional characteristics. Accordingly, each type of storage medium lends itself particularly well to certain types of applications, and not to others. In particular, data files which are serially organized, and which are to be serially accessed, are most compatible with tape drive technology~ inasmuch as information is necessarily written upon and read from tape in serial fashion. Disk memory systems, on the other hand, generally have the capability of having information written upon and read from practically any portion of the disks almost instantaneously. As a consequence, multiple data sets all stored on the same disk can be active at once, which is not practical with tape. Therefore, it would always be pre-ferred to use disk memory systems, except for the fact that disk memory is much more expensive than tape.
As a result, while serially-organized data can conveniently be continuously written upon and read from a tape; oftentimes writing or obtain-ing data which is not serially organized presents a problem. If the data is located upon a distant portion of the tape, the tape must be wound or un-; wound to the point at which the data is located. A second far more serious drawback to tape memory is the fact that, for most large-scale applications, the number of reels of tape required to store information is far in excess ,~

~;3126 of, perhaps hundreds of times greater than, the number of tape drives or mounts available to read or write from the tapes. In fact, many large scale data processing operations make use of thousands of reels of tape. In general, scme 90% of all tapes contain but one data set (e.g. a customer list or a payroll file) since as a rule the physical characteristics of tape make it impractical to read or write from or to two or more data sets at once, which is very often desirable. Clearly then, tape storage is very inefficient, since only coincidentally will the length of a given data set approximate the available memory area. In fact, studies show tha~ normally only about one percent of tape is actually used.
Obviously, no user can provide a tape mount or drive for each reel of tape; rather, individual reels are constantly brought from storage, mounted for use, then demounted and replaced in storage. This is time-consuming and inefficient, since it involves the intervention of a human operator.
; Efforts have been made to overcome this shortcoming of tape media, for instance by providing a large number of small cartridges of tape and an automatic mechanism to rapidly remove the cartridges from a storage rack and place them on a tape mount. Automatic mounting devices have also been made for use with conventional tape. To date, however, no completely successful method exists for making large quantities of tape immediately available to tape drives.
At first glance, disk storage systems would appear to overcome the shortcomings of tape. Practically any portion of any disk can be accessed very rapidly, so that a disk system does not have to scan through all of the data recorded thereon in serial fashion. Accordingly, by responding to address codes, a disk drive can access a given block of data almost instantly.
Thus, multiple data sets can be stored on a single disk and be concurrently

- 2 -~S3126 processed. However, the cost of disk drives~ is comparatively great, and more-over state-of-the-art disk drives do not make highly efficient use of the potential storage area.
Studies have demonstrated that at any given time in a disk system, only approximately 75% of potential storage space is actually available for data storage purposes. The "overhead", or total storage space used to support the read/write activity needed to maintain and log data, is thus approximately 20% to 25%. In addition, for most applications, only about two-thirds of the available data space is actually used to store data, due to the space requirements of the addressing systems commonly used. Hence, the total utili-zation factor of most state-of-the-art disk systems is only about 50% or less.
erefore disk systems, although offering attractively high data access and retrieval speeds, are relatively cost inefficient, and may therefore be expensive in terms of the storage space which is actually used. By comparison, the loss of space in tape storage due to provision of address marks is small, since only a single address need be provided for a single sequentially organ-i~ed record. The storage efficiency is thus increased to this extent.
It will therefore be seen that it would be highly advantageous to provide a system which exhibits the addressing efficiensy of the tape medium and the rapid-access characteristic of disk storage. Desirably, such a memory system would not require any modification to the remainder of the com-puting system involved; that is, it would "look" to the computer either as tape or disk memories do in the prior art.
It is therefore an object of the present invention to provide an improved storage system for a digital computer or the like.
It is another object of the invention to provide a system and method for storing and accessing serially-arranged data more quickly than , ~;31Z6 state~of-the-art storage systems.
Still another object is to provide a storage system which responds to com~ands from the host system in the manner of a tape drive, but makes use of the rapid access capability of disk storage units.
Yet another object of the present invention is to provide a data storage management system which accepts and delivers data in serial form, while recording and retrieving the information from disk storage units in optimum piecemeal form.
Briefly stated, in accordance with one aspect of the invention the foregoing objects are achieved by providing a "virtual storage system"
comprising a host interface stage responsive to commands from a host computer for receiving and transmitting information in a serial, tape format, and a first data buffer within said host interface for temporarily storing data received from the host or directed thereto. A plurality of disk storage units are also provided, along with a disk storage interface and an associated second data buffer which temporarily stores data directed to and received from the disk storage units. The interface stages are coupled together, and also coupled in common to a large, main memory which receives and queues infor-mation flowing from one interface to the other. Finally, a master control processor is coupled to the interface units and to the main memory and - directs the operation of the apparatus so that the host interface stages respond to the host computer in the same manner as would a tape system, while the disk interface stages read and write data upon disk memories in random fashion in accordance with currently available memory space.
Broadly described, the present invention provides apparatus for storage of a user defined data set, said apparatus adapted to be connected to an input/output channel of a host computer, said channel of said host outputting peripheral commands including specified addresses for storage of . '~

`. ` ': ~
-.

, ~3~26 data sets on peripheral storage means, said apparatus for storage of data comprising: long-term magnetic storage media for storing data; channel inter-face means for mating with said channel of said host, device interface means for connection to said long-term magnetic storage media; data path means con-necting said channel interface means and said device interface means; and controller means for maintaining records of available storage locations on said long-term magnetic storage media, for assigning said locations to sub-portions of a data to be stored, for dividing said data set by intelligence external to the host computer into subportions sized to fit available storage locations on said long-term magnetic storage media, for controlling flow of data from said channel interface means over said data path means to said device interface means for storage on said long-term magnetic storage media, and for maintaining a record showing the correspondence between said addresses for storage of data output by said host and the actual locations on said long-term magnetic storage media at which the subportions of said data set are stored.
According to another aspect of the invention, there is provided method of improving the efficiency of utility of a data storage subsystem, said sub-system being adapted to be connected to an input/output channel of a host com-puter, said host being of the class which passes commands for data storage over such channels, said commands comprising addresses for storage of data sets on peripheral devices connected to said channel, the method comprising the steps of receiving a data set from the host and storing the data set in short-term solid state storage means connected to said host channel consulting a list of available storage locations to determine what storage locations are available for storage of subportions of said data set on long-term magnetic storage media, dividing said data set by intelligence external to the host computer . ~
..

~53126 into subportions sized to fit such determined available storage locations on said magnetic storage media, writing said subportions to said storage locations on said magnetic storage media, deleting said storage locations from the list of available storage locations, and maintaining a second list showing the correspondence between the address for storage of said data output by said host and the actual locations on storage media which said data set has been stored.
The invention will now be described in greater detail with reference to the accompanying drawings in which:
Figure 1 illustrates the overall organization of a virtual storage system constructed in accordance with the invention;
Figure 2 is a schematic diagram illustrating the flow of control and information signals within the system of Figure l; and Figures 3 and 4 are illustrations of the flow of command signals and information in "read" and "write" situations, respectively.
Figure 1 shows in simplified form the basic architecture of the virtual storage system of the invention when coupled to a host computer 10.
The invention constitutes means for controlling one or more extremely fast and accessible disk memory unit(s) to accept data and respond in precisely the same fashion as a tape drive, while providing much better use of available storage, thus in effect having vastly expanded storage capability and a far more rapid response time than either tape or disks. The host computcr 10 may be of any appropriate type, although it is anticipated that economical opera-tion of the present invention will dictate its use with a large mainframe host computer such as *IBM System/360, System/370 computers, and *IBM Models 3031, * trade mark -5a-:
;, ~3126 3032, and 3033 processor complexes. The virtual storage system of the inven-tion can also be used with a plurality of host computers. The storage system comprises a bank of host interface stages 12 through 12 , the number of inter-face stages being dependent upon the amount of data to be transferred. Since the host interface stages are identical, the actual number which is to be used is for present purposes immaterial. The host interface stages are coupled to the host computer lO by means of conventional channels 14 and receive and transmit information to the computer 10 in the ordinary, serialized format which is associated with tape drives.
One or more disk interface stages 16 through 16 are provided, and are coupled to all of the host interface stages by means of a common informa-tion bus 18. In this manner, information may be transferred between any given host interface stage and any disk interface stage.
The disk interface stages are each coupled to respective ones of disk memories 20a through 20n. Each disk memory typically comprises a plural-ity of disks arranged on a common axis but other configurations are possible.
The combination of each disk memory with its associated disk interface stage forms a disk storage unit. As will be discussed in further detail, each disk interface stage comprises a memory interface, a data buffer, and a disk inter-face, along with a local microprocessor controller for operating the various elements of the interface stage.
Also connected to the main bus 18 is a main memory 22. Memory 22 forms a repository for information flowing from the host interface stages to the disk interface stages, and may further serve as a memory for control pro-cessor 24 also coupled to the main bus 18. The control processor 24, which is the central processing unit or CPU of the virtual storage system, is coupled to each of the host and disk interface stages 12 and 16 respectively, ~, .' '' .
-;3126 and directs the operation thereof so that information may be received, queued,organized, and stored in the proper manner. The control processor 24 further instructs the various host interface stages-12 to cause them to respond to the host computer 10 in a manner which simulates a tape drive.
Turning now to Figure 2, there is- shown in further detail the structure of exemplary elements of the system. Processor 24 is coupled direct-ly to host interface stage 12, and in particular to an internal interface 26 which simply facilitates the transfer of signals between the control processor 24 and a microprocessor 28 which forms the controller for the host interface stage. Also forming part of the host interface stage are a data buffer 30, a host interface 32 which makes the data buffer compatible with the host computer 10, and a memory interface 34 through which the data buffer communi-cates to data bus 18.
The function of the host interface stage 12 is to simulate a tape drive system to the host computer; that is, to convert the host computer lO's commands to tape drives into commands to the virtual storage system of the invention, and to send the host computer 10 data in the form it would be sent by a tape drive. ~ore particularly, host interface stage 12 accepts the sig-nals from a host computer 10 which are of the type used to operate the tape drives. Such signals are obtained by coupling the host interface stage to the byte multiplex, or block multiplex, or selector channels of the host 10. It will be appreciated that these signals include operator commands such as "mount tape" and "demount tape" as well as machine commands such as "read", "write", and "forward space file". The host interface stage responds to both kinds of signals as if it were an operator and a tape drive, acknowledging the signals, responding that "mounting" of an imaginary ~virtual) tape reel has been accomplished, and the like.

.~ .

A

i31~:6 The data buffer 30 of each host interface stage accepts data from the host computer 10 in serial form, usually nine bits in parallel, in the precise manner that it would be reapplied to a tape system for writing upon a tape. The buffer 30 in conjunction with the host interface 32 during a write operation deserializes the information; that is, stores it in parallel fields holding the individual bits until eight entire bytes are available. Typically 72 bits are transmitted at a time thus reducing transmission time. In this manner, up to 90% compression of the time required to transmit the data on the line 18 can be achieved. When data is to be exchanged between buffer 30 and the bus 18, the data flows through memory interface 34, again in eight parallel bytes so that an extremely rapid exchange of data may take place.
The size of said buffer can vary greatly, depending on system requirements but in a preferred embodiment is large enough, 64,000 bytes, to hold an entire record, which expedites time sharing of bus 18.
Ultimately, data received from the host computer 10 is written onto magnetic disks mounted on one or more disk drive units 20a to 20n. Disk interface stage 16 couples data bus 18 to a disk drive 20 (not shown), and is comprised of a local microprocessor controller 36 which operates the various elements of the interface stage in accordance with instructions from control processor 24. As was the case with the host interface stages 12, with disk interface stages 16 instructions from the control processor are transmitted through an interface 38 to the local controller 36. The latter then responds by causing a data buffer 40J disk interface 42, and memory interface 44 to transfer data to and from the disk drive.
In particularJ memory interface 44 serves to receive eight bytes of data from bus 18 and serialize the data so that it is installed in buffer 40 in a single stream of bits. Arrangement and queueing of buffer storage is .

, :
: .

: ' ;31Z6 accomplished by the local controller 36, as is the operation of the disk in-terface 42 which passes data from buffer 40 to the disk drive at the appropri-ate times. Still further, the disk interface stages serve to seek and log the locations of data upon the various disks of the disk drive associated therewith, so that the information can be retrieved when needed.
Between the time data is in a host interface 12 and is received in a disk interface 16, it is stored in a main memory 22. The high speed main memory 22 thus acts as a "bank" or "cache" which holds the data until one of the disk drive units 20 is ready to accept it. When this occurs, the disk interface stage 42 associated with the disk drive 20 signals control processor 24 of its availability. The control processor 24 then instructs the main memory or cache 22 to discharge data to the disk interface stage 42 by way of bus 18.
In like manner, when the host computer 10 is seeking data, the identity of the data sought is transferred through host interface stage 12 to the control processor 24, which then determines if the requested data has been previously moved into the main memory. If it has not, the control pro-cessor 24 in turn will apply a "transmitter" signal to the disk interface stage 16, and the local controller 36 therein will cause data to be read out of the associated buffer 40, through the associated memory interface 44, and onto the data bus 18, and thence to the main memory 22.
At the same time, the local controller 28 of the available host interface stage 12 will enable the associated memory interface 34, so that the newly-read data will be received from main memory 22 and gated into the host interface data buffer 30. On the other hand, if it has, the data will be immediately gated out of the buffer 30, serialized by the host inter-face module 32, and transmitted to the host computer 10.

, ;31Z6 In an alternative embodimentJ both host buffers 30 and disk buffers 40 may be dispensed ~ith, their functions then being performed by main memory 22.
In this fashion, segments of data which had been dispersed throughout a number of disk drives are compiled, queued, and then re-assembled automatically into serial form. Accordingly, the information flowing to the host computer 10 from the various disk drives 20 appears in serial form, precisely as if it were being read from a tape.
In the foregoing mannerJ the high speed main memory provides a cache from which data can be selected and queued for subsequent reassembly into serial form before being directed to the host computer.
As will be recognized by those skilled in the art upon learning the teachings of the present invention, the various elements of the present virtual storage system may be assembled from commercially-available elements, and coupled together in any convenient fashion. For instance, while all of the various elements of disk interface stage 16 are illustrated as being dis-posed in one location, remote from control processor 24, in fact the elements may be disposed at different locations, and coupled by means of appropriate cables, buses, or the like. The local microprocessor controllers 24 used for operating the components of each interface stage need only be of rather limit-ed capability and may be comprised of any of the various high speed micro-processors available on the market. An example of such microprocessors is the LSI-ll marketed by DEC Incorporated of Boston, Mass.; alternatively, a suitable unit can be assembled from the AMD Co.'s 2900 series of parts. In like manner, the host and disk interfaces 12 and 16 which accomplish the serializing and deserializing of information may be standard units, such as the model 370 block multiplexor available from the IBM Corporation of Armonk, :

~1~3126 New York. In like manner, an IBM block multiplexor may be used for direct memory access, the actual connection of the various units being well under-stood by those skilled in the art.
Similarly, the buffers 30 and 40 used for temporary storage of data in the host and disk interface stages 12 and 16 may be of any appro-priate type, although in a presently preferred embodiment a memory of at least 64K bytes is p~eferred. One commercially available buffer of this type is manufactured by Fairchild Semiconductor, and comprises a N-MOS random ac-cess memory having a speed of 200 nanoseconds.
The high speed memory 22, which serves as a data cache should be of the type generally designated a fast access memory, i.e. one having a cycle time of 400 nanoseconds or less. In a preferred embodiment, the high speed cache has a capacity of 16 megabytes. One commercially-available memory appropriate for use with the present invention is manufactured by Intersil Corp., Sunnyvale, California, and is marketed by Storage Technology Corp. of Louisville, Colorado, assignee of the present application, and designated models 3758 and 3768.
Figure 3 is a functional illustration of the operation of the inventive system in a "read" mode wherein the host computer 10 has requested information from memory. Control signals are shown by solid lines, and data flow by dashed lines. Accordingly~ the host 10 produces a command signal to the human operator requesting certain information ~e.g. "Mount Tape N"), and since it is assumed by the host 10 that the data is stored on tape in serial form, only the initial identification or "label" need be specified. Other elements of the virtual storage system, particularly the control processor 24, respond to the information by accessing all data which corresponds to the requested information and causing the data to be assembled in serial form.

, ;3126 Thus, the host computer 10 simply calls for Tape N which, it assumes, according to information supplied by the individual program being run on the host computer 10, contains a particular data set. In fact, accor-ding to the virtual storage system of the invention, this particular data set may be spread over a number of different disks and/or may be in a number of locations on a single disk. Therefore, the virtual storage system of the invention responds to an operator command such as "Mount Tape N" by searching through its own memory for data indicating where the various parts of the record which the host computer has denominated "Tape N" have been stored.
Thus, the response of the system is not to direct an operator to mount Tape N, but merely to recall the information as to the location(s) of the data set identified as "Tape N" from its memory.
When the host interface stage 12 indicates that a "Mount Tape N" instruction has been completed, which is practically instantaneous in com-parison to the mounting of an actual tape, the host 10 outputs a command to the interface 12 designating the specific identity of the block of information to be read. This command is transmitted directly by the host interface stage 12 to the control processor 24 which identifies the various areas in which the information is stored. This may be accomplished through the use of a data log 46 which is coupled to the control processor. In a presently prefer-red embodiment data log 46 comprises a random access memory ~RAM) (which may be "backed-up" by a tape drive unit and associated tape in case of loss of memory in the RAM) on which the location of the various sub-blocks of data which together constitute the data requested by the host computer 10 are recorded.
When the locations of the various sub-blocks of data have been identified to the control processor 24, signals are supplied to the controller ... .

-:

. :
.-' ' ~1~3126 portions of the various disk interface stages 36, whereupon data is accessed from the various disk storage units 20. The data is then read out into the buffer 40 of the interface stage 16 and prepared for transmission to the main memory cache 22 for temporary residence there until needed by the various host interface stages 12.
When the buffer 40 is filled with data, a signal is outputted from the disk interface stage 16 to the control processor 24 indicating this fact, at which time the control processor 24 transmits the buffered informa-tion directly to a previously-allocated space in the main, high speed memory cache 22.
With prior art tape systems, the initial "Mount" operation ordinarily occupies approximately 30 sec. - 5 min. The total time required to execute a "read" command varies between 1-10 milliseconds depending on the characteristics of the individual device and of course on the length of the record being read. In a preferred embodiment ofthe illustrated virtual stor-age system, a "Mount" operation typically occupies less than one second. The actual "read" operation typically occupies 1-10 milliseconds as before, de-pending on record length and the characteristics of the cache 22. At this point, therefore, the information to have been read will have been very rapidly placed in the cache 22 where it is stored until a signal from memory interface 12 indicates to the cache that the host computer is ready to receive the data. At this time, then, the data has been gathered from the various points in the various disks 20 where it has been stored, has been put into serial form in the proper order, and is ready to be read into the host com-puter 10 with no delay for address time or for searching for the information on the various disks. Clearly, therefore, the memory system of the invention offers significant time advantages over disks; and as compared to tape systems, . '. . ' : ~ "

: . , .. ~ - ' , , .~ . .

. ~ ' ' ' ' ', ~5i31Z6 avoids the mounting time delays completely. In this way, the "anticipatory buffering" provided by the use of the cache combines the advantages of disk and tape in that it combines the lack of mounting required for disks with the serial format of tapes, thus saving time to the host computer by having the data completely organized in the cache. At the same time, owing to the superior organization and cooperation of the various elements, very little of the actual disk storage is dedicated to directing read/write activity or maintaining or logging the data thereon, sometimes referred to as the "over-head" factor in disk memory capacity. Accordingly, a much higher percentage of actual disk memory is available for data. In this way, the storage efficiency of disk memory can be upgraded to a level similar to that of tape.
In the host "write" mode illustrated in Figure 4, the procedure is substantially the reverse of the "read" mode just described, As in Figure

3, control signals are shown by solid lines and the flow of data by dashed lines. When the host computer 10 signals, through conventional tape drive codes, that it desires to "write" or record data~ this command is transmitted through interface stage 12 to the control processor 24. The latter instructs interface 12 to acknowledge the command, and at the same time interrogates the disk interface stages 16 in order to find a buffer 40 which will accept some or all of the data. When such locations are determined, they are logged in data log 46 by the control processor 24.
The control processor 24 then directs the host interface stage 12 to accept the data from the host computer 10 and direct it to appropriate locations in the data buffer 30 of the host interface stage 12. Should the buffer 30 become filled before corresponding space is available in the buffers 40 of the disk interface stages, an "allocate space" instruction is trans-mitted by the control processor to the memory cache 22, after which data from ~53126 the host interface stage buffer is applied to the memory cache 22. At some subsequent time, when adequate buffer capacity is available in one or more disk interface stages 16, the control processor instructs the cache memory 22 to supply the stored data to the appropriate disk interface stage buffers 16 for disposition upon the disks of the associated disk drives 20.
As each block of data is applied to a host interface 32 by the host 10, the memory interface module 34 determines where in the associat-ed buffer 30 the newly-received information will be stored; compresses the data in serial fashion so as to produce, for instance, eight channels of data; and adds an identification bit or bits to the information block indicating the size of the block. In this manner, data is continually fed to the buffers 30 until a given block of data is filled or until the host computer generates a signal indicating the end of a given record. In accor-dance with an important aspect of the present invention, such a signal is interpreted by the local controller 28 as an end of message signal subsequent to which no more buffer memory space need be allocated.
At this point, the control processor 24 directs the data now stored in data buffer 30 to the high-speed cache 22. During the time the host computer is storing data in data buffer 30 as described above, the control processor 24 can be "looking for" sufficient space upon disk drives 20, and can allocate that space to the data then being stored in data buffer 30. Thus, when the end of record signal is received in the control processor, data it can be serialized and fed into the high speed cache, as a temporary storage space, and is then passed to disk drives 20 for storage.
Put differently, when the host computer produces an initial "Mount Tape" command (i.e. directs the operator to provide a blank tape for data storage purposes) this command is passed through the host interface ,, .

;` . . : , , : ;

~53126 stage 12 to control processor 24. The control processor 24 responds to the "Mount Tape" instruction by seeking space upon one or more disks 20, and by reserving adequate space in one or more of the disk interface stage buffers 30. This done, the control processor 24 instructs the host interface stage 12 to accept data from the host 10 and to deserialize it and store it in the associated buffer 30. Once this buffer is full, if no more buffer space is available, this data is passed to the memory cache 22. When sufficient disk space is available, the next step is for the control processor 24 to instruct the disk control interface 42 of the disk interface stage 16 to write the data upon the associated disk device 20 after which the interface module 44 reads and serializes the information, placing it in its associated buffer 40, whence it is written onto the associated disk 20. It will be understood that another possibility is to have the data copied or stored onto a conventional tape drive, if for example duplication be deemed desirable; that is, the data storage system of the inventîon can be paralleled with a conventional prior art tape system, if desired.
It will be apparent to those skilled in the art that the association of a high speed cache with a control processor, along with host interface and disk interface stages and associated control modules permits the combination of the advantages of prior art tape and disk drive systems;that is, the ready accessibility of disk drives is combined with the storage efficiency of tape drives to yield a fast high efficiency "virtual storage system" in which a disk system is made, through the intermediaries of the invention, to look to the central processor like a tape drive, but without the drawbacks thereof. Since the memory disks according to the invention are permanently mounted, there is no need for an operator's intervention when ; a tape-type record (i.e. a sequential record) is sought. Moreover, prior art : . ..

, . , ~1~3il26 disk drives can be used with the virtual storage system of the invention without modification; that is, the system of the invention can be viewed as a unit or interface inserted between a prior art host computer and prior art disk drives. Therefore, there is no need to modify the disk drives or the host computer in order that they may be operated in conjunction with the virtual storage system of the invention.
Furthermore, it will be appreciated that the virtual storage system of the invention can be used to interface between any number of host computers and any number of disk drives. Therefore, it will be appreciated that should an operator have more than one host compute and a limited number of disk drives, the virtual storage system of the invention can be used to achieve maximum storage efficiency and eliminate undue multiplication of disk or tape storage systems. The host interface units of the invention may be coupled to more than one host computer. Moreover, it will be appreci-ated that the provision of a high speed memory or cache is essential to the "anticipatory buffering" which is a very important feature of the invention.
By using the high speed cache as an intermediate buffer between the host computer interface and the various disk drive interface stages, buffering of multiplex data can be accomp~shed; that is, according to the invention data stored in a number of places upon a number of disk drives ~referred to as "in random form") can be assembled, sequenced, and stored temporarily in the high speed cache until it is called for by the host computer. In this way, there ;~ need be no delays in reading the information from memory into the host com-puter. In a similar fashion, data output from the host computer can be arranged, divided, and stored in the high speed cache until such time as various disk storage areas are available to store the information.
A particularly significant possibility which is provided by the , ~

,~ ' .
.
.

. .
.

;31Z6 storage system of the invention is that of data compression. Heretofore a common scheme of data CompTessiOn, which involves the replacement of a long string of digital "l's" or "O's" with symbols indicating the length of the string, had not been useful with disk drive units, since this scheme com-pressed addressing information as well as data. However, since according to the present application disk drives appear as tape, this difficulty of the prior art is obviated, and data compression is made compatible with disk storage. Preferably, data compression is implemented in the host interface stage;that is during a write operation long continuous strings of "l's" or "O's"
may be detected and replaced with shorter symbols; during a read operation, these are detected and replaced by the thus-defined data.
Finally, it will be appreciated that there are a number of other modifications and refinements which can be made to the virtual storage system of the invention and that the examples discussed above are exemplary only, and that therefore thescope of the invention should be interpreted as limited only by the following claims.

., `~:

~, .

.~ .
' :, ~ .

Claims (9)

THE EMBODIMENTS OF THE INVENTION IN WHICH AN EXCLUSIVE
PROPERTY OR PRIVILEGE IS CLAIMED ARE DEFINED AS FOLLOWS:

1. Apparatus for storage of a user defined data set, said apparatus adapted to be connected to an input/output channel of a host computer, said channel of said host outputting peripheral commands including specified addresses for storage of data sets on peripheral storage means, said apparatus for storage of data comprising: long-term magnetic storage media for storing data; channel interface means for mating with said channel of said host; device interface means for connection to said long-term magnetic storage media; data path means connecting said channel interface means and said device interface means; and controller means for maintaining records of available storage locations on said long-term magnetic storage media, for assigning said locations to sub-portions of a data to be stored, for dividing said data set by intelligence external to the host computer into subportions sized to fit available storage locations on said long-term magnetic storage media, for controlling flow of data from said channel interface means over said data path means to said device interface means for storage on said long-term magnetic storage media, and for maintaining a record showing the correspondence between said addresses for storage of data output by said host and the actual locations on said long-term magnetic storage media at which the subportions of said data set are stored.

2. The apparatus of claim 1 wherein said controller is additionally capable of performing data compression on said data prior to its storage on magnetic storage media.

3. The apparatus of claim 1 further comprising solid state cache memory means for storage of a data set received from said host prior to its division into subportions for storage on long-term magnetic storage media.

4. The apparatus of claim 3 wherein said cache is further used to assemble subportions of said data upon the data set being accessed by a host computer in performance of a READ command.

5. Method of improving the efficiency of utility of a data storage sub-system, said subsystem being adapted to be connected to an input/output chan-nel of a host computer, said host being of the class which passes commands for data storage over such channels, said commands comprising addresses for storage of data sets on peripheral devices connected to said channel, the method com-prising the steps of receiving a data set from the host and storing the data set in short-term solid state storage means connected to said host channel consulting a list of available storage locations to determine what storage locations are available for storage of subportions of said data set on long-term magnetic storage media, dividing said data set by intelligence external to the host computer into subportions sized to fit such determined available storage locations on said magnetic storage media, writing said subportions to said storage locations on said magnetic storage media, deleting said storage locations from the list of available storage locations, and maintaining a second list showing the correspondence between the address for storage of said data output by said host and the actual locations on storage media which said data set has been stored.

6. The method of claim 5 further comprising the step of performing data compression on said data prior to storage on said magnetic storage media.

7. Apparatus for storage of user defined data sets comprising channel interface means for mating with an input/output channel of a host computer of the type which outputs an address for storage of data on storage media together with such data set, solid state cache storage means for storing a data set prior to storage of said data set on long-term magnetic storage media, data path means connecting said channel interface means, said cache memory and said long-term magnetic storage media, and controller means for: maintaining a table of available storage locations on said magnetic storage media for storage of subportions of said data set; division of said data set by intelligence external to the host computer into subportions sized to fit said storage loca-tions for controlling writing of said subportions to said available storage locations; and for maintaining a table of the correspondence between the addresses for storage output by said host and the actual storage locations at which the subportions of said data set are stored, and of the relative sequen-tial ordering of said subportions of said data set.

8, The apparatus of claim 7 wherein said controller means additionally performs data compression on said data prior to dividing it into subportions for storage onto said magnetic storage media.

9. The apparatus of claim 7 wherein said controller responds to a READ
command from a host computer by reassembling the subportions of the data set in their original order prior to retransmission of the data set to the host computer.