JPS63263524A - Data structure converting system - Google Patents

Abstract

PURPOSE:To reduce the data arrangement change processing in the data reception side, to improve the processing efficiency and to shorten the processing time by converting the data structure in data transfer units in accordance with a data structure conversion request to transfer data at the time of transferring data existing in a data storage medium to another data storage medium. CONSTITUTION:A data structure converter 3 is newly provided between an external storage device 4 and a main storage device 2; and when a CPU 1 issues a request for data transfer from the external storage device 4 to the main storage device 2 or from the main storage device 2 to the external storage device 4, the data structure converting request which prompts transposition of significant data set units in each block as a data transfer unit is transferred to the data structure converter and data arrangement in the block is converted on the basis of this request to transfer data to the main storage device 2 or the external storage device 4. Thus, the data arrangement processing time in the vector instruction generation processing on the side of the CPU 1 is shortened.

Description

[Detailed Description of the Invention] [Industrial Field of Application] The present invention relates to a computer system in which data is stored in a computer system from an external storage device (data storage device, generally referred to) to a main storage device (temporary storage device for data manipulation, generally referred to). Alternatively, it relates to a method of converting the arrangement of data according to a specified data structure conversion request when transferring data from a main storage device to an external storage device.

[Conventional technology]

In the conventional method, as described in Japanese Patent Application Laid-Open No. 134945/1982, the access unit in database processing is made into multiple records, which are converted into storage addresses in the main storage device at once, and the multiple records are converted into vector format. After expanding into main memory, vector processing was performed to speed up processing. However, when transferring data on an external storage device to main memory, no consideration was given to converting the arrangement of logical data within a record so that it is expanded into a vector format before being transferred.

[Problem that the invention seeks to solve]

The above conventional technology does not consider changing the structure of data that is easy to process or want to retain on the data receiving side when transferring data between the external storage device and the main storage device. There is a problem in that it takes time to convert the data structure because the data structure is converted (rearranged) so that the received data can be easily processed or desired to be retained.

An object of the present invention is to transfer data from an external storage device to a main storage device, or from a main storage device to an external storage device, by changing the arrangement of the data and sending it in accordance with a data structure conversion request. The purpose of this invention is to improve processing efficiency and reduce processing time by reducing data arrangement change processing.

[Means for solving problems]

The above purpose is between external storage and main storage.

When a new data structure conversion device is installed and a request is issued from the CPU to transfer data from an external storage device to a main storage device or from a main storage device to an external storage device, each block, which is a unit of data transfer, is A data structure conversion request that instructs the sorting of meaningful data blocks is passed to the data structure conversion device, the data arrangement within the block is converted based on the request, and the data is transferred to the main storage or external storage. This is achieved by:

[Operation] In response to a data transfer request and a data structure conversion request from the CPU, the data structure conversion device inputs the data of the file on the external storage device or main storage device in block units, and issues the data structure conversion request. Based on the information on the data structure conversion request, which transfers the arrangement of the corresponding data in the block to the external storage device or main storage device on the receiving side of the changed data, the address information (before change) of the data in the block is It consists of the relative address and length from the beginning of the block and the address information after the change. As a result, the data structure conversion device converts the structure of the data in the block from the file 1 on the storage device on the data sending side to the block east position, and transfers it to the storage device on the data receiving side.
There is no need for U to perform structural conversion of data on the main memory.

〔Example〕

Hereinafter, one embodiment of the present invention will be described in detail with reference to the drawings. The configuration of the embodiment is shown in FIG. 1 is the central processing unit (
CPU), 2 is a main storage device, 4 is an external storage device, and 3 is a data structure conversion device, which are newly installed devices by Kenshin. 31 is a transfer and data structure conversion command receiving unit;
Receive command information from 1 to 3. The information received was 35.
Store it in each table of the processing request reception unit. 33 is a block acquisition unit that takes in a specified block from the external storage device 4. 32 is a data structure conversion processing unit that converts the data structure in the block that was taken in by the block acquisition unit 33 in response to a data structure conversion request from 1. Convert. A data conversion result sending unit 34 transfers the block in the block buffer after data structure conversion in the buffer unit 36 to the main storage device 2°. Reference numeral 36 denotes a buffer section which includes one or more pre-conversion block buffers and one or more post-conversion block buffers, and the blocks received from the external storage device 4 are stored in the pre-conversion block buffers. The block whose data structure has been converted is stored in the converted block buffer. 37 is a buffer management unit, which has a table for managing the status of the pre-conversion block buffer and the post-conversion block buffer;
D” is.

36 pre-conversion and post-conversion block buffers are identified. The ``buffer status'' in the pre-conversion block buffer status table indicates that the block buffer is ``empty'', ``block is contained but conversion by 32 is not completed'', ``block is contained and conversion by 32 is completed''. On the other hand, the status is expressed by, for example, "Pa0", "IH%a2", etc., respectively.

``Buffer status'' in the post-conversion block buffer status table indicates that the block buffer is ``empty''.

``Data is included but conversion by 32 is not completed'''
For example, if the state is ``data is contained and conversion by 32 is complete'', ``A #l, 11'' respectively.
B''.

The "processing request ID" in the one-car state table, which is represented by "C't," indicates which block corresponds to which processing request the block buffer is filled with.

Next, the operation of the embodiment configured as described above will be explained.

First, the following processing request is issued on the CPUI.

In other words, a specified operation (transferring the data to a specified position) is performed on specific data that constitutes a record, which is a logical unit of data, in a specific file. . The CPUI identifies the ID (identification number) of the block (there may be more than one block) containing the file and its external storage bag fi! The storage location ID in i4 is checked by accessing the external storage device 4 before issuing a processing request.

Next, the transfer and data structure conversion command receiving unit 31 receives from 1 the ID of the processing request, the above block storage location ID,
Receives conversion processing information for layout conversion of target data (within a block) and passes it to the data structure conversion processing unit 32. 3
2 searches for an empty table from the beginning in the order of the arrangement number of the tables attached to the processing request table, and stores the processing request ID and block storage location ID therein.

Secure a free conversion processing table (conversion processing table with the same number as the array number of the processing request table).
Enter the conversion processing information (data ID, relative address in the record before conversion, length (of data), relative address in the record after conversion), and also the block storage location ID.
is passed to the block acquisition unit 33.

32 imports the specified block (group) from the external storage device 4, and in the buffer status table for blocks before conversion of the buffer management unit 37, the buffer status is 'o'' (
Check the buffer ID(group) that is empty) and select the buffer part 3.
The block (group) is stored in the corresponding pre-conversion block buffer (group) of No. 6. Thereafter, the buffer management unit 37 changes the "buffer status" of the buffer status table for blocks before conversion corresponding to the buffer ID (group) to "gll19clK block is contained, but the conversion by 32 is not completed"). .

On the other hand, the data structure conversion processing unit 32
Check the buffer status table for block 7 before conversion,
Search for one buffer ID whose buffer status is tt1'rcat (contains a block but the conversion by 32 is not completed...), read the corresponding processing request ID, and read the information of the corresponding conversion processing table. 32 firstly performs the following processing on the pre-conversion block buffer of the buffer section 36 corresponding to the buffer ID, that is, starting from the first record in the block, Data ID indicated by address and length
The data is stored at the location (address) indicated by the relative address in the post-conversion record in the post-conversion block buffer. The converted block buffer for storing the data of the block is determined by the data structure conversion processing unit 32 as follows. First, search for the buffer status of 37 post-conversion block buffer status table z A p'' (empty), set the buffer status to "B", and set there the same processing request ID as the processing request ID. .

Then, once the data (multiple data) to be rearranged in the l record are moved to the buffer for the block after the corresponding variable,
The next record is also rearranged in the same way, and when the processing of the block is completed, the buffer status of the 37 pre-conversion block buffer status tables corresponding to the pre-conversion block buffer is set to “0” (empty), and processing is performed. Also clears the request ID.

In addition, the buffer status of the 37 post-conversion block buffer status table corresponding to the post-conversion block buffer is 'C' (data is contained and conversion in 32 is completed).
Set to .

Then, the data structure conversion processing unit 32 repeats the same process as above for the other 36 pre-conversion block buffers.

On the other hand, the data conversion result sending unit 34 sends a buffer I whose buffer status in the post-conversion block buffer status table 37 is C'' (data is contained and conversion in step 32 is completed).
O and the processing request ID are found, and then the block in the modified block buffer corresponding to the buffer ID and the corresponding processing request ID are transferred from the buffer unit 36 to the main storage device 2. After the transfer is completed, the buffer for the changed block and the buffer status table for the changed block are also
The data conversion result sending unit 32 clears it. Here, the transfer timing is determined at regular intervals or when the buffer status of the post-conversion block buffer status table is 1 in 37.
1 When the number of buffers in C#l exceeds a certain value. Further, the data conversion result sending unit 34 transmits the processing request I
At the time when all blocks of the file corresponding to D (data structure converted) are transferred to the main storage device, the processing request table storing the processing request IO of the processing request reception unit 35 and the corresponding conversion processing table Clear.

Above, we have described an example in which the data structure of the file on the external storage device is converted in block units and the data is transferred to the main storage device. When converting the structure and transferring data to an external storage device, the procedure can be almost the same as above.

Next, details before and after data structure conversion will be explained in terms of actual database processing.

A database consists of pages (logical access units) that store tables and indexes, and the horizontal columns of the table are called rows and the vertical columns are called columns.

1 Pages are numbered sequentially from the beginning, and these are identified as page numbers. FIG. 2 shows this row stored within a page. An example of the structure of this page will be explained using FIG. 2. The page 5 includes a row 51 and an offset section 52. The offset section 52 stores a relative address from the start address of page 5 to the start of row 51. An offset section exists for each row within a page, and is numbered sequentially from the end of the page. This number is called a slot number. In order to batch process rows consisting of data structures like this on the CPU side using vector instructions, as shown in FIG. There is a need to. According to the procedure shown in one embodiment of the present invention, the data structure shown in FIG. 2 is converted into the data structure of the group B and data vector shown in FIG. 3, and the data is transferred to the main storage device 2. Is possible. The data vector 6 shown in FIG. 3 is data necessary for processing a vector instruction, and the row address 61 indicates the address when each row 51 is stored in the main memory 2, and where the row is stored in the table. It consists of a row 1d62 indicating whether the row exists (the row ID can be expressed using the page number and offset part of the page where the row exists), where the row address 61 is an address on the main storage device 2. Therefore, when a data structure conversion request is issued from the CPUI, the transfer and data structure conversion instruction receiving unit 31 receives the start address on the main storage device 2 that stores the row group, and updates the processing request table in the processing request receiving unit 35. It must be stored in . The data structure conversion processing unit 32 includes the 1 shown in FIG.
Each time a row in the pre-conversion block buffer of the buffer unit 36 that stores one page is stored in the post-conversion block buffer of the same buffer unit 36, the row size and the first row to be stored in the main storage device 2 are determined. The row address is calculated based on the first address on the main storage device 2 where the row group is stored, and the row address is calculated in the post-conversion block buffer (the row address of the data vector 61 in Figure 3). 61). By adding this process to the data structure conversion embodiment described above, the data structure conversion shown in FIGS. 2 and 3 can be realized using the same data structure conversion process procedure.

In addition, by adding a request to transfer the data before data structure conversion to the main storage device 2 at the time of a data structure conversion request from the CPU, data in the block buffer before conversion of the buffer 36 can be transferred to the main storage device 2. Can be transferred. This can be done by storing the pre-conversion data transfer request in the processing request table of the processing request receiving unit 35.
The data conversion result sending unit 34 transfers the data in the pre-conversion block buffer (for example, the page in FIG. 2) of the buffer unit 36 to the data in the post-conversion block buffer (for example, the third page).
This can be realized by sending the data vectors together with the group B in the figure and the data vector). This procedure can be added to the data structure conversion example described above. Through this process, in beta base update processing, etc., the efficiency of update processing can be improved by reducing the data input/output processing before data structure conversion. .

〔Effect of the invention〕

As explained above, according to the present invention described above, the structure of data specified by a processing request (data structure conversion request) from an external storage device to a main storage device or from a main storage device to an external storage device is transmitted. All you have to do is convert and transfer. Therefore, the CPU's need for detecting relevant data and structural conversion processing is reduced, so that the CPU usage time can be reduced. In particular, it is possible to reduce the time required for data arrangement processing in vector instruction creation processing on the CPU side.

Further, by sending the data before data structure conversion together with the data after data structure conversion, it is possible to improve the efficiency of update processing of databases and the like.

[Brief explanation of drawings]

FIG. 1 is a diagram showing a schematic configuration of an embodiment of the present invention, and FIG.
The figure is an explanatory diagram showing data before conversion on the external storage device 4 in one embodiment of the present invention, and FIG. FIG. 4 is a structural diagram of data after converting the data on FIG. 4 to be sent to the main storage device. 3... Data structure conversion device, 32... Data structure conversion processing unit, 35... Processing request receiving unit, processing request table, conversion processing table. Figure 2

Claims (1)

[Claims] 1. In a computer system that has a main storage device and an external storage device, when transferring data existing in a data storage medium to another data storage medium, the data structure is changed for each data transfer unit in response to a data structure conversion request. A data structure conversion method characterized by converting and transmitting.