GB2285525A - A tape database control process - Google Patents
A tape database control process Download PDFInfo
- Publication number
- GB2285525A GB2285525A GB9400387A GB9400387A GB2285525A GB 2285525 A GB2285525 A GB 2285525A GB 9400387 A GB9400387 A GB 9400387A GB 9400387 A GB9400387 A GB 9400387A GB 2285525 A GB2285525 A GB 2285525A
- Authority
- GB
- United Kingdom
- Prior art keywords
- data
- tape
- record
- database
- controller
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
-
- G—PHYSICS
- G11—INFORMATION STORAGE
- G11B—INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
- G11B20/00—Signal processing not specific to the method of recording or reproducing; Circuits therefor
- G11B20/10—Digital recording or reproducing
- G11B20/12—Formatting, e.g. arrangement of data block or words on the record carriers
-
- G—PHYSICS
- G11—INFORMATION STORAGE
- G11B—INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
- G11B27/00—Editing; Indexing; Addressing; Timing or synchronising; Monitoring; Measuring tape travel
- G11B27/002—Programmed access in sequence to a plurality of record carriers or indexed parts, e.g. tracks, thereof, e.g. for editing
-
- G—PHYSICS
- G11—INFORMATION STORAGE
- G11B—INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
- G11B27/00—Editing; Indexing; Addressing; Timing or synchronising; Monitoring; Measuring tape travel
- G11B27/10—Indexing; Addressing; Timing or synchronising; Measuring tape travel
- G11B27/11—Indexing; Addressing; Timing or synchronising; Measuring tape travel by using information not detectable on the record carrier
-
- G—PHYSICS
- G11—INFORMATION STORAGE
- G11B—INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
- G11B27/00—Editing; Indexing; Addressing; Timing or synchronising; Monitoring; Measuring tape travel
- G11B27/10—Indexing; Addressing; Timing or synchronising; Measuring tape travel
- G11B27/19—Indexing; Addressing; Timing or synchronising; Measuring tape travel by using information detectable on the record carrier
- G11B27/28—Indexing; Addressing; Timing or synchronising; Measuring tape travel by using information detectable on the record carrier by using information signals recorded by the same method as the main recording
- G11B27/30—Indexing; Addressing; Timing or synchronising; Measuring tape travel by using information detectable on the record carrier by using information signals recorded by the same method as the main recording on the same track as the main recording
- G11B27/3027—Indexing; Addressing; Timing or synchronising; Measuring tape travel by using information detectable on the record carrier by using information signals recorded by the same method as the main recording on the same track as the main recording used signal is digitally coded
-
- G—PHYSICS
- G11—INFORMATION STORAGE
- G11B—INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
- G11B27/00—Editing; Indexing; Addressing; Timing or synchronising; Monitoring; Measuring tape travel
- G11B27/10—Indexing; Addressing; Timing or synchronising; Measuring tape travel
- G11B27/19—Indexing; Addressing; Timing or synchronising; Measuring tape travel by using information detectable on the record carrier
- G11B27/28—Indexing; Addressing; Timing or synchronising; Measuring tape travel by using information detectable on the record carrier by using information signals recorded by the same method as the main recording
- G11B27/30—Indexing; Addressing; Timing or synchronising; Measuring tape travel by using information detectable on the record carrier by using information signals recorded by the same method as the main recording on the same track as the main recording
- G11B27/3027—Indexing; Addressing; Timing or synchronising; Measuring tape travel by using information detectable on the record carrier by using information signals recorded by the same method as the main recording on the same track as the main recording used signal is digitally coded
- G11B27/3036—Time code signal
- G11B27/3054—Vertical Interval Time code [VITC]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F2003/0697—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers device management, e.g. handlers, drivers, I/O schedulers
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Signal Processing (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
A tape database controller operates in conjunction with a tape controller of a tape system to allow not only batch archival and batch retrieval of data, but also on-line access to data as if the database were stored on disk. Each database is broken down into a number of datasets and a single tape volume (31) is assigned to each dataset. Data blocks (36) in each tape volume include a number of records (37) each comprising a variable number of sub-records (41) and a single header (40). An index file (Fig. 4) includes an index entry record for each primary key, this record having a number of cycles, there being one cycle for each data record (37). Therefore, by using a primary key and a storage date, the database controller 10 may determine the identifier for the tape volume (31) and of the data block (36), the particular data record (37) being retrieved by use of the primary key. The primary key is stored in a field (51) of the header (40). <IMAGE>
Description
"A taPe database control process"
The invention relates to a process carried out by a tape database controller connected between a data processor and a tape system.
It is well known that storage of data on tapes is considerably less expensive than on disk. For this reason, tapes are often used for archival of large amounts of data. An example is archival of statement records for a utility company such as a telephone company. Tape systems such as those described in U.S. Patent
Specification No. 4145724 (Consolidated Electronic
Industries) or in PCT Patent Specification No. 9301595 (Storage Technology Corp.) are used for handling of tapes in an automated manner.
At present, retrieval of data from tapes is generally quite a slow process. Indeed, because it is so slow it is generally not used in an ^ line processing situation.
This leaves the considerable commercial disadvantages for companies such as utility companies as it causes delays in replying to customer enquiries, for example. An example is where a customer telephones a telephone company to enquire about a statement which issued several months previously. More than likely, the data for this statement will have been archived to tape and will not be accessible on disk. It is then necessary to generate the necessary instructions for retrieval of the correct tape and to then read the data and to reply to the customer at least two hours later.
The invention is directed towards providing a process for storage and retrieval of data from tapes which is carried out quickly enough to be of use in an on-line data processing situation.
According to the invention, there is provided a process for storage and retrieval of data on a tape system, the process being carried out by a tape database controller connected between a data processor and a tape system controller, the processor comprising the steps of:
setting up a database structure comprising a plurality
of datasets within the database;
setting up a general tape structure of a plurality of
variable-length data blocks for tapes handled by the
tape system;
receiving a data storage request from the data
processor and carrying out the steps of::
assigning a single tape volume to the relevant
dataset, directing creation of a new record header
within a data block of the tape volume, the header
including a primary key and a storage date, and
writing the data to a plurality of sub-records in
sequence after the record header;
setting up an index file for the database comprising a
single index control record with tape system
initialisation data;
after writing of data to the tape system, the tape
database controller creating a cycle for an index entry
record for the data record, the index entry record
including the primary key and storage date and the
associated volume sequence number and block identifier;
receiving a data retrieval request and carrying out the
steps of::
addressing the index file with the primary key and
the storage date and accessing the relevant index
entry record to determine the data record volume
sequence identifier and block identifier; and
transmitting read instructions to the tape system
controller.
In one embodiment, the tape database controller assigns a dataset name comprising a common high level qualifier and a low level qualifier including the volume sequence identifier.
Preferably, there is a variable number of sub-records for each data record, and a single data record may not span two or more blocks unless it is of greater size.
In another embodiment, the index control record of the index file includes a current block count for the tape volume and this is automatically updated by the tape database controller after each new block is written to.
The invention will be more clearly understood from the following description of some preferred embodiments thereof, given by way of example only with reference to the accompanying drawings in which:
Fig. 1 is a block diagram showing a data processing
system incorporating a tape system and a tape
database controller of the invention;
Fig. 2 is a flow chart showing a flow of instructions
in the tape database controller;
Fig. 3 is a hierarchial diagrammatic representation
of a tape database generated by a process of the
invention; and
Fig 4 is a representation of an index file generated
by the tape database controller.
Referring to Fig. 1, a data processing system 1 is shown.
The system 1 is for the carrying out of processing operations, primarily with reference to databases. The system 1 comprises a data processor 2 which controls the various processing operations. One of the primary tasks of the data processor 2 is to access data on fixed disk databases and to process this data. However, quite regularly it will be necessary to access data on a large tape database system 3 having a capacity of up to 26 terrabytes. A tape controller 4 is constructed to receive data or data requests and the tape system addresses for -.çriting or retrieval of the data and to control the robotic systems of the tape system 3. To enable the data processor 2 to access databases of the tape system 3, the system 1 comprises a tape database controller 10 connected between the data processor 2 and the tape controller 4.
In addition, there is a tape controller interface 11 which is connected to intercept instructions from the tape database controller 10 and to provide auxiliary robotic control instructions for the tape controller 4. The tape database controller 10 operates in conjunction with an index file 12 and a catalog file 13, described below.
Fig. 2 illustrates the manner in which the tape database controller 10 and the tape controller interface 11 interact. An instruction 20 may be transmitted from the tape database controller 10 and this is intercepted in step 21 by the tape controller interface 11. If the instruction is for reading or writing of data from a tape currently loaded in a tape head, the instruction is transmitted straight through to the tape controller 4 as indicated by the arrow 22. However, if robotic control is required to retrieve the relevant tape not already inserted in the tape reading head, the tape controller interface 11 generates the necessary robotic control instructions for the tape controller 4 before the instruction may be transmitted, in this case indirectly, as indicated by the arrow 24.
The important features of the invention are the manner in which the tape database controller 10 operates in order to provide for quick writing and reading of data from a tape database on the tape system 3. Retrieval of data is more important as writing is generally carried out in an archive batch process.
The controller 10 sets up a database structure comprising a plurality of datasets. In Fig. 3, a database 30 is shown. In addition, the controller 10 sets up a general tape structure for each of the tapes handled by the tape system 3. This general tape structure comprises a plurality of fixed-length data blocks, the length of each data block being a maximum of 32 kB. The controller 10 also sets up the index file 12. The index file 12 is set up with a single index control record which includes all of the necessary initialisation data for the database.
There may be a number of databases stored on the tape system 3, and there is one index file for each database.
The index control record includes various upper level tolerance values for storage of data on the tapes, including a maximum number of blocks per tape volume and a maximum block size for a tape dataset, a maximum number of archive periods, i.e. the length of time that data is held in the database before automatic deletion. The index control record also includes the current block being utilised. The index file 12 is shown in more detail in
Fig. 4 in which the index control record is indicated by the numeral 60.
When the controller 10 receives a data storage request from the data processor 2, it initially determines the relevant dataset and sets up a single tape volume for that dataset if this has not been done already. As shown in
Fig. 3, there is a single tape volume 31 for each dataset.
Each tape volume is identified by a volume sequence identifier, namely, a volume sequence number in the range of 1 to n. Once a tape volume has been assigned to a dataset, the dataset name is generated. This comprises a common or standard high level qualifier followed by a low level qualifier which includes the volume sequence number.
In this embodiment, there is a maximum of 65,535 datasets and therefore the low level qualifier is in the form nnnnn.
The structure of each tape volume 31 is shown in Fig. 3.
There is a volume serial number record 32, followed by a pair of headers 33 and 34, and by a tape mark 35. There is then a sequence of the fixed-length data blocks 36. A typical data block 36 is also illustrated in Fig. 3 and this comprises a number of records indicated by the numeral 37. Writing of data to the particular tape volume involves writing to a data block 36 which involves creation of a new record header for a record 37. Each record 37 comprises a variable number of variable-length sub-records. In the example shown in Fig. 3, there are three sub-records 41 for each data record 37. The header 40 of each archived data record is shown in detail in Fig.
3 and this comprises a fixed pair of characters in this embodiment @@, indicated by the numeral 50. This is followed by a primary key field 51 a storage date key 52.
The information which is applied by the data processor 2 to the controller 10 includes the primary key which may, for example, by the customer account number for a telephone company or an electricity supply company. The storage date is determined by reference to a real-time clock. Thus, the controller 10 has all of the information necessary to generate a header 40 when writing of data to a data block 36. The manner in which the data is broken down into the sub-records 41 is dependent on the nature of the data. All of the information necessary for writing of the data to the relevant tape and the location on the tape is generated by the controller 10 and this is embedded in the instructions transmitted to the tape controller 4 either directly or indirectly. The database controller 10 does not carry out the actual writing of the data or the robotic tape handling involved.
After writing of data to the tape, the controller 10 creates an index entry record in the index file 12 for the data record 37. The index entry record is indicated by the numeral 70 in Fig. 4 and it comprises a fixed length prefix made up of a primary key field 71 and a half-word field 72 containing the number of cycles in the index entry record 70. The fixed-length prefix is followed by a number of cycles, each comprising fields 73 to 76. Each cycle comprises a six-byte archive or storage date 73 and a half-word binary field 74 giving the number of tape blocks occupied by this unit of archived data. An important aspect of the writing process is that an archived data record does not span more than one data block unless it alone is a larger size than the block size (32kB).The cycle also includes a full-word binary field 75 giving the block identifier of the first tape block which contains data for the record and finally, there is half-word binary field 76 giving the volume sequence number.
It will thus be noted that for any particular primary key such as customer account number, there is only a single index entry record in the index file 12. The relevant cycle gives the data retrieval information for retrieval of data from the relevant tape. To retrieve data from a database on the tape system 3, the data processor 2 transmits the primary key and archive date for the data.
By accessing the index file 12, the database controller 10 determines the volume sequence number and the block start identifier from the fields 76 and 75 respectively in the relevant cycle of the index entry record 70. Once the relevant block in the relevant tape volume is identified, the particular data record 37 is determined by using the primary key to search the fields 51 of the data record headers 40.
When a write operation has been completed, the controller 10 updates the index control record for the relevant index file 12 with an identifier for the current block being written to. It is important to note that blocks are used sequentially, irrespective of the nature of the data which has been written. This minimises the amount of tape handling required on the tape system 3. The fact that data associated with a single primary key such as a customer account number may be located on different tape volumes in different datasets is of little importance because of the manner in which these may be retrieved very quickly using the process as described above.It has been found that the particular data structure used and the manner in which retrievals are made by access to the index file 12 provide for a very fast data retrieval and indeed it can almost be guaranteed that there will be an average retrieval time of 30 seconds using a state of the art tape system 3. The controller 10 may operate in conjunction with a larger number of data processors 2 controlled by a central controller. To do this, it transmits program code as a sub-task to perform all tape functions for a particular processor of the data processing system. The sub-task program code is transmitted to the address space for the particular processor. This ensures that the other processors are not delayed by operation of a single processor in accessing data on tape.
The invention is not limited to the embodiments hereinbefore described, but may be varied in construction and detail.
Claims (5)
1. A process for storage and retrieval of data on a tape
system, the process being carried out by a tape
database controller connected between a data
processor and a tape system controller, the processor
comprising the steps of:
setting up a database structure comprising a
plurality of datasets within the database;
setting up a general tape structure of a plurality of
variable-length data blocks for tapes handled by the
tape system;
receiving a data storage request from the data
processor and carrying out the steps of:: assIgning a single tape volume to the relevant dataset, , directing =creation of a new record header within a data block of the tape volume,
the header including a primary key and a storage
date, and writing the data to a plurality of
sub-records in sequence after the record header;
setting up an index file for the database comprising
a single index control record with tape system
initialisation data;
after writing of data to the tape system, the tape
database controller creating a cycle for an index
entry record for the data record, the index entry
record including the primary key and storage date and
the associated volume sequence number and block
identifier;
receiving a data retrieval request and carrying out
the steps of::
addressing the index file with the primary key
and the storage date and accessing the relevant
index entry record to determine the data record
volume sequence identifier and block identifier;
and
transmitting read instructions to the tape
system controller.
2. A process as claimed in claim 1, wherein the tape
database controller assigns a dataset name comprising
a common high level qualifier and a low level
qualifier including the volume sequence identifier.
3. A process as claimed in any preceding claim, wherein
there is a variable number of sub-records sor each
data record, no a single data record may not span
two or more blocks unless it is of greater size.
4. A process as claimed in any preceding claim, wherein
the index control record of the index file includes
a current block count for the tape volume and this is
automatically updated by the tape database controller
after each new block is written to.
5. A process as claimed in any preceding claim, wherein
each data record header comprises a fixed, common
character string followed by the primary key and the
storage date.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| GB9400387A GB2285525B (en) | 1994-01-11 | 1994-01-11 | A tape database control process |
| BE9400057A BE1006042A6 (en) | 1994-01-11 | 1994-01-18 | Process management magnetic database on tape. |
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| GB9400387A GB2285525B (en) | 1994-01-11 | 1994-01-11 | A tape database control process |
| BE9400057A BE1006042A6 (en) | 1994-01-11 | 1994-01-18 | Process management magnetic database on tape. |
Publications (3)
| Publication Number | Publication Date |
|---|---|
| GB9400387D0 GB9400387D0 (en) | 1994-03-09 |
| GB2285525A true GB2285525A (en) | 1995-07-12 |
| GB2285525B GB2285525B (en) | 1998-04-01 |
Family
ID=25662860
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| GB9400387A Expired - Fee Related GB2285525B (en) | 1994-01-11 | 1994-01-11 | A tape database control process |
Country Status (2)
| Country | Link |
|---|---|
| BE (1) | BE1006042A6 (en) |
| GB (1) | GB2285525B (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP0936561A2 (en) * | 1998-02-17 | 1999-08-18 | Sony Corporation | Data transfer method and apparatus on an SCSI bus |
| GB2342214A (en) * | 1998-09-28 | 2000-04-05 | Dascom Software Dev Services L | A tape system storage and retrieval process |
| EP1548598A1 (en) * | 2002-09-10 | 2005-06-29 | Masaharu Tamatsu | Database re-organizing system and database |
| DE10123377B4 (en) * | 2001-05-14 | 2011-07-14 | Océ Printing Systems GmbH, 85586 | Method for reading data and reading device stored on a data tape |
-
1994
- 1994-01-11 GB GB9400387A patent/GB2285525B/en not_active Expired - Fee Related
- 1994-01-18 BE BE9400057A patent/BE1006042A6/en not_active IP Right Cessation
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP0936561A2 (en) * | 1998-02-17 | 1999-08-18 | Sony Corporation | Data transfer method and apparatus on an SCSI bus |
| EP0936561A3 (en) * | 1998-02-17 | 2003-10-15 | Sony Corporation | Data transfer method and apparatus on an SCSI bus |
| GB2342214A (en) * | 1998-09-28 | 2000-04-05 | Dascom Software Dev Services L | A tape system storage and retrieval process |
| US6189001B1 (en) | 1998-09-28 | 2001-02-13 | Dascom Software Development Services Limited | Tape system storage and retrieval process |
| GB2342214B (en) * | 1998-09-28 | 2003-02-26 | Dascom Software Dev Services L | A tape system storage and retrieval process |
| DE10123377B4 (en) * | 2001-05-14 | 2011-07-14 | Océ Printing Systems GmbH, 85586 | Method for reading data and reading device stored on a data tape |
| EP1548598A1 (en) * | 2002-09-10 | 2005-06-29 | Masaharu Tamatsu | Database re-organizing system and database |
| EP1548598A4 (en) * | 2002-09-10 | 2009-04-08 | Masaharu Tamatsu | Database re-organizing system and database |
Also Published As
| Publication number | Publication date |
|---|---|
| GB9400387D0 (en) | 1994-03-09 |
| BE1006042A6 (en) | 1994-04-26 |
| GB2285525B (en) | 1998-04-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US5768532A (en) | Method and distributed database file system for implementing self-describing distributed file objects | |
| US6449607B1 (en) | Disk storage with modifiable data management function | |
| JPH0728631A (en) | Method and system that install software application reversibly | |
| JPH06318168A (en) | Hierarchical data storage management device, method and its network | |
| JPH04505977A (en) | Object-oriented distributed processing system | |
| US6029229A (en) | Digital data storage subsystem including directory for efficiently providing formatting information for stored records | |
| US7159139B2 (en) | Digital data storage subsystem including directory for efficiently providing formatting information for stopped records and utilization of a check value for verifying that a record is from a particular storage location | |
| US6330655B1 (en) | Digital data storage subsystem including directory for efficiently providing formatting information for stored records | |
| US5519860A (en) | Central processor index sort followed by direct record sort and write by an intelligent control unit | |
| GB2285525A (en) | A tape database control process | |
| US5666527A (en) | System for dynamically changing logical data structure of database | |
| IE69554B1 (en) | A tape database control process | |
| JPH096653A (en) | Data base checking information processor | |
| JP3166675B2 (en) | Transaction search processor by time specification | |
| JPH11265389A (en) | Shared data base device ahd shared data base system | |
| JP2720775B2 (en) | File parallel processing unit | |
| JPH02151944A (en) | Data processor | |
| KR20020037399A (en) | method for processing a transaction of the database management system | |
| JPH0477938A (en) | Data storage method | |
| KR20000014913A (en) | Online personal directory providing service system and method thereof | |
| JP2817911B2 (en) | Access control method for keyed files | |
| EP0537893B1 (en) | File storage apparatus | |
| Brandt | The Library in Academic Computing: Downloading Revisited, or Putting up with Clumsy Data Transfer All Over Again… | |
| JP3313207B2 (en) | Information storage device and information management method | |
| JP3008500B2 (en) | Update record reading mechanism |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PCNP | Patent ceased through non-payment of renewal fee |
Effective date: 20050111 |