JPH06348554A - Data processor - Google Patents

Abstract

PURPOSE:To structure a data processor which has a dedicated data base arithmetic processor so as to perform a sorting process and retrieve a relation data base at a high speed and can perform efficient data processing on the whole system while maintaining the cooperation between a host device and the data base arithmetic processor. CONSTITUTION:When file capacity is deficient at the time of the output of arithmetic result data, the data base arithmetic processor 16 temporarily stops an output processing and sends a request to extend an output file by the capacity of the deficiency to the host device 10; and the host device 10 answers the start address of the part of the output file extended by the capacity of deficiency to the data arithmetic processor 16 and the data base arithmetic processor 16 restarts the output processing which is stopped so far to carry on an arithmetic processing without interruption.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a data processing device having an arithmetic processing device for performing sorting or relational algebraic arithmetic processing on file data to be operated in response to a request from a host device.

[0002]

2. Description of the Related Art Generally, a computer system is provided with a dedicated database arithmetic processing unit in order to execute sorting and search processing of a relational database (RDB) at high speed.

This database arithmetic processing device is provided with an arithmetic circuit called a hardware sorter, and by using this arithmetic circuit, sorting and relational algebraic arithmetic etc. according to arithmetic processing requests from the host device can be performed at high speed. Run.

Such a database arithmetic processing device is
As such, it directly accesses the catalog part of the file on the secondary storage device, allocates or deletes the file,
It does not have the function of the so-called file management subsystem on the host system, such as expansion.

Therefore, when an attempt is made to output a calculation result to a file while the database calculation processing device completes the calculation processing inside the database and leaves only the output thereafter, the capacity of the output file is insufficient. And, it had the drawback that all the arithmetic processing up to that point was wasted.

To avoid this, the host device reserves an output file in the secondary storage device in advance with a capacity corresponding to the size of the operation result data, and then requests the database operation processing device for the operation processing. There was a way to issue.
However, in this case, the host device needs to calculate the capacity of the calculation result in advance, but accurate prediction calculation is difficult or impossible, and the capacity of the output file must be calculated by a trial and error method. I didn't get it.

[0007]

As described above, when there is a shortage of output files of the arithmetic processing result, the database arithmetic processing unit suspends the output and invalidates the entire arithmetic processing as described above. Was causing a waste of money. Further, the calculation of the output file capacity and the retry of the arithmetic processing by the host device at the time of the arithmetic processing are non-computational, and the existence of the database arithmetic processing apparatus aiming at the improvement of the data processing efficiency is essentially denied. It is a thing.

The present invention has been made in view of the above circumstances, and based on the fact that the database arithmetic processing device itself accurately grasps the data amount of the arithmetic result, the file is output at the time of outputting the arithmetic result data. When the capacity is insufficient, the output processing is temporarily stopped, and a request is issued to the host device to extend the output file by the amount of the insufficient capacity, and the host device extends the output file by the amount of the insufficient capacity. Efficient data by responding the start sector address of the part to the database arithmetic processing unit, the database arithmetic processing unit restarting the output processing that has been stopped until then, and allowing the arithmetic processing to continue without interruption An object is to provide a data processing device that can realize a processing mechanism.

[0009]

The present invention has a host device, a secondary storage device in which file data to be operated is stored, and a path used when directly accessing the secondary storage device. In a data processing device, which comprises an arithmetic processing device that executes a predetermined arithmetic processing on the file data in response to an arithmetic request issued from a host device, the arithmetic processing device calculates according to an arithmetic request from the host device. Data input means for obtaining target file data from the secondary storage device via the path, arithmetic processing means for performing arithmetic operation on the input file data, operation target file data and the arithmetic processing means An internal memory that stores the calculation result data,
Output means for outputting the operation result data to the secondary storage device, and during the output processing of the operation result data by the output means,
Output capacity expansion means for detecting a capacity shortage of the output file, suspending the output processing, designating the capacity shortage, and requesting the host apparatus to expand the capacity shortage, the host apparatus calculates The output capacity expansion request issued from the output capacity expansion request means of the processing device is received, the output file is expanded by the specified capacity, and the address on the secondary storage device corresponding to the file expansion part is given to the arithmetic processing device. A file extension means for responding to the file extension means, and the arithmetic processing unit receives an extension completion response from the file extension means in the host device, and restarts the output during the suspension from the address corresponding to the file extension part. It is characterized by having done.

According to the present invention, the arithmetic processing device further includes a first processor for executing communication with the host device and data input / output with the secondary storage device, and arithmetic such as sorting or relational algebraic arithmetic. An arithmetic circuit for performing logical operations,
A second processor for controlling execution of operations by the operation circuit; an internal memory for storing operation target file data input via the first processor and an operation result by the operation circuit; and the first processor A third processor that performs synchronous control of the first and second processors based on an instruction from the host device supplied via the host processor. Completing the calculation operation according to the calculation request from, and outputting the calculation result data to the output file, when the output is insufficient, the lack is detected, and the host device is requested to expand the insufficient capacity, and the request is made. The host device that has received the extension expands the output file by the extension capacity designated by the third processor, and the memory address corresponding to the extension portion By responding to the third processor, and characterized in that a resume configure output processing paused which was the result data.

[0011]

According to the present invention, when the arithmetic processing unit completes the arithmetic operation in accordance with the arithmetic operation request from the host device and the output file runs short while outputting the operation result data to the output file, the shortage is detected, and the shortage capacity is detected. To the host device. The host device expands the output file by the expansion capacity specified by the arithmetic processing unit, and responds the start sector address of the expanded portion to the arithmetic processing unit to restart the output processing of the result data that was paused and output it. The first feature is that even if the file capacity is insufficient, the operation can be continued without failing.

Based on the fact that the calculation process itself accurately grasps the data amount of the calculation result, if the file capacity becomes insufficient at the time of outputting the calculation result data, the output process is temporarily stopped and the shortage occurs. A request is issued to the host device to extend the output file by the amount corresponding to the capacity, and the host device extends the output file by the insufficient capacity and responds with the start sector address of the extended portion to the arithmetic processing unit, and performs the arithmetic processing. The apparatus restarts the output processing that has been stopped until then, and enables the above-mentioned arithmetic processing to be continuously executed without being interrupted, whereby an efficient data processing apparatus can be constructed.

Further, according to the present invention, a multiprocessor configuration including first to third processors is adopted in the database operation processing device, and the data input / output between the database operation processing device and the secondary storage device is the first. To the processor
The operation processing using the operation circuit is distributed to the second processor, and the overall control is distributed to the third processor, and the operation processing from the host device and the output processing are temporarily stopped when the output file capacity is insufficient. Request the host device to expand the output file by the insufficient capacity, receive the expansion completion response of the above output from the host, and based on the start sector address of the expansion part of the output file containing the response A second feature is that the third processor is configured to restart the output processing that has been temporarily stopped. In this configuration, the load can be efficiently distributed by the first to third processors, and the data input / output by the first processor and the second processor can be performed.
The parallel processing with the arithmetic processing of the other processor becomes possible, the parallel processing and the output processing is temporarily stopped when the output file capacity is insufficient, the output file expansion request is issued to the host device, and the output file expansion processing by the host device The third processor executes the output processing of the temporary stop upon receiving the completion response and the operation performance of the database arithmetic processing apparatus can be further improved.

[0014]

Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a block diagram showing the configuration of an apparatus according to an embodiment of the present invention. The data processing apparatus according to the embodiment of the present invention includes a host computer 10, a database operation processing apparatus (database engine; DBE) 16, and a magnetic disk device 17.

The host computer 10 includes a CPU 11,
The main memory 12 and the first and second channel devices (channel 1, channel 2) 14 and 15 are used.
The CPU 11, the main storage device 12, and the channel devices 14 and 15 are commonly connected via a system bus 13.

The host computer 10 and the database arithmetic processing unit 16 are connected via the channel unit 14 of the host computer 10, and the host computer 10 and the magnetic disk unit 17 are connected via the channel unit 15. Further, the database processing device 1
6 and the magnetic disk 17 are connected via a dedicated access path 18.

The CPU 11 controls the host computer 10 as a whole, and the database arithmetic processing unit 16
It delegates the execution of various arithmetic processes such as sorting and relational algebraic arithmetic.

The main storage device 12 stores a file to be operated by the database operation processing device 16 and a command for instructing an operation content. The database arithmetic processing unit 16 executes arithmetic processing on the data of the operation target file based on the command issued from the CPU 11, and uses the path 18 for input of the operation target file and output of the operation result. The magnetic disk device 17 is directly accessed via the.

The database processing unit 16 includes an engine interface processor (EIP) 161, an engine control processor (ECP) 162, and a large capacity memory (E).
BDM) 163, hardware sorter control processor (ECAM) 164, parallel sorting module (P
SOM) 165 and parallel relational algebra calculation module (P
RAM) 166 and the like.

Engine interface processor 16
1, the engine control processor 162, and the hardware sorter control processor 164 are connected to each other by an internal bus 167 to form a tightly coupled multiprocessor that shares and uses the large capacity memory 163. .

Therefore, the large capacity memory 163 is arranged in the common address space of each of the three processors.
Further, these three processors, that is, the engine interface processor 161, the engine control processor 1
62, and hardware sorter control processor 164
The functions are distributed in.

In this case, in order to efficiently carry out the respective unique roles, the respective processors are configured to be tightly coupled and controlled by independent monitors suitable for the respective processors.

Engine interface processor 16
The functions of the engine control processor 162, the engine control processor 162, and the hardware sorter control processor 164 are distributed as follows.

That is, the engine interface processor 161 communicates between the host computer 10 and the database arithmetic processing unit 16, and is also connected to the disk controller of the magnetic disk unit 17 via the path 18 to make a magnetic disk. Controls data input / output with the device 17. The engine interface processor 161 also performs an output file reconstruction process when outputting data to the magnetic disk device 17.

The engine interface processor 161 communicates with the host computer 10 by
It receives a command sent from the CPU 11 via the first channel device 14 and sends it to the engine control processor 162.

Further, the engine interface processor 161 receives the status as a result of command transmission sent from the engine control processor 162, and sends it back to the CPU 11 via the first channel device 14.

Further, the engine interface processor 161 receives an input / output request from the engine control processor 162 in inputting / outputting data to / from the magnetic disk device 17, and connects between the large capacity memory 163 and the magnetic disk device 17. To transfer data. At that time, output file data reconstruction processing is also performed according to a request from the engine control processor 162.

The engine control processor 162 includes an engine interface processor 161, a large capacity memory 16
3 and the hardware sorter control processor 164 via the internal bus 167. When the command issued from the CPU 11 is an operation processing command such as sort or relational algebraic operation, various corresponding commands Create and execute a command processing process. In the portion related to the present invention, when the calculation result is being output to the output file on the magnetic disk device 17 and the output file has insufficient capacity, the output processing is temporarily stopped and the following processing is executed.

That is, the shortage capacity is calculated, and the CPU 11
Requesting that the output file be expanded by that amount, and the magnetic disk device 17 corresponding to the expanded portion is requested.
Of the start sector address of the engine interface processor 1
The data is sent to 61 to restart the output processing that has been paused.

The large-capacity memory 163 stores the operation target file data, CP, read from the magnetic disk device 17.
Various calculation processing commands sent from U11, calculation processing results of the hardware sorter control processor 164, parallel sorting module 165, parallel relational algebra calculation module 166, etc., engine control processor 16
2 stores the integration results of the calculation results by 2 respectively, and further, here, when the engine control processor 162 detects the capacity shortage while outputting the result data, the shortage capacity is CP.
The expansion request command (reference numeral 2 in FIG. 2) for requesting expansion to U11 and the CPU 11 in the expansion request command
As a result of the expansion by the CPU, the CP including the start sector address on the magnetic disk device 17 of the output file expansion portion as a parameter and responding to the engine control processor 162.
Extended response command issued from U11 (3 in Fig. 2)
It is a shared memory in the database arithmetic processing unit 16 for storing the above.

The file data to be calculated is stored in the input buffer section in the large capacity memory 163, and the calculation processing result is stored in the output buffer section in the large capacity memory 163.
The hardware sorter control processor 164 controls the calculation by the parallel sorting module 165 and the parallel relational algebra calculation module 166 based on the command from the engine control processor 162. In this case, the hardware sorter control processor 164 uses the large capacity memory 1
The data on 63 is input to the parallel sorting module 165, and the calculation result output from the parallel relational algebra calculation module 166 is stored in the large capacity memory 163. When inputting data to the parallel sorting module 165, first, Execute the clipping process.

In this key cut-out processing, the hardware sorter control processor 164 cuts out only the calculation-target key field necessary for the calculation from each record of the calculation target and outputs the record identification number (the large-capacity memory 163).
The record top address) is added and transmitted to the parallel sorting module 165.

The parallel sorting module 165 is a dedicated hardware circuit that is driven under the control of the hardware sorter control processor 164 to execute sorting processing in parallel, and is connected to the parallel relation algebra operation module 166. This parallel sorting module 165
Is called the pipeline merge sorter,
A plurality of sort cells for performing 2-way merge are connected in cascade.

The parallel relational algebra operation module 166 is a dedicated hardware circuit for executing relational algebraic operations such as join (join) and list (constraint) in the relational database in parallel, and the data sorted from the parallel sorting module 165. To output the calculation result to the hardware sorter control processor 164.
When only the sorting process is executed, the parallel relational algebra operation module 166 functions as the last-stage sort cell.

As described above, the engine interface processor 161, the engine control processor 162, and the hardware sorter control processor 164 have functions for executing various arithmetic processes, such as sort process and select process, dispersed therein. There is.

Each of these processors has a large-capacity memory 1.
It operates asynchronously except for the buffer exchange on 63,
Execute each function in parallel. That is, processing such as sorting and selecting is usually composed of input processing, arithmetic processing, and output processing, but the hardware sorter control processor 1
The calculation process under the control of 64 is executed in parallel with the input process of the calculation target data by the engine interface processor 161.

When it is necessary for the engine control processor 162 to perform integrated processing on the calculation result obtained from the hardware sorter control processor 164, the integrated processing engine interface processor 161 also executes output processing in parallel.

2 to 5 are diagrams for explaining the operation of the embodiment of the present invention. FIG. 2 is a diagram showing a format of each of the extension request command 2 and the extension response command 3 described above.

The extension request command 2 is command code 2
1, the extension target file descriptor 22 and the extension capacity 23
Has as a parameter. The extended response command 3 includes the start sector address 3 of the extended portion in addition to the command code 31.
It has 3 as a parameter.

The expansion request command 2 sets the shortage capacity to C
When the PU 11 is requested to expand, it is issued from the engine control processor 162 to the CPU 11.

The extension response command 3 is issued from the CPU 11 to the engine control processor 162 in response to the extension request command 2, and as a result of the extension by the CPU 11, the output file extension portion on the magnetic disk device 17 is expanded. The start sector address of is included as a parameter and the response is made.

4 and 5 are flowcharts showing the operation of the embodiment of the present invention, FIG. 4 is a flowchart showing an output processing procedure, and FIG. 3 is a flowchart showing an extension processing procedure.

The operation of the apparatus having the configuration of the embodiment of the present invention shown in FIG. 1 will be described below with reference to FIGS. When the engine control processor 162 detects that the capacity of the output file is insufficient while outputting the calculation result data to the output file (step 41 in FIG. 4), the output process is temporarily stopped (step 42 in FIG. 4).

Here, an output file descriptor indicating what the output file is is set in the output file descriptor parameter 22 of the extended command 2 (step 43 in FIG. 4), and the insufficient capacity is [insufficient capacity = result data Amount-output file capacity] (step 44 in FIG. 4), the insufficient capacity calculated here is set as the expansion capacity parameter 23 of the expansion request command 2 (step 45), and the expansion request command 2 is completed. , CPU11 is issued.

When the CPU 11 receives the extension request command 2 from the engine control processor (ECP) 162 (step 51 in FIG. 5), it extracts the extension target file descriptor from the target file descriptor parameter 22 in the extension request command 2. , Specify the file to be expanded (Fig. 5
Step 52), the capacity for expanding the file to be expanded is extracted from the expansion capacity parameter 23 of the expansion request command 2 (step 53 in FIG. 5), and the file 2 in the file descriptor 22 is expanded by the expansion capacity 23 (FIG. 5). Step 5
4) The start sector address on the magnetic disk unit 17 of the extension part is taken out and set in the start sector address parameter 32 of the extension part of the extension response command 3 (step 55 in FIG. 5), and the extension response command 3 is set to the engine control processor. 162 (step 56 in FIG. 5).

Next, the engine control processor 162
The extended response command 3 is received (step 47 in FIG. 4), and the start sector address 32 of the extended portion of the extended response command 3 is received.
To the engine interface processor 161 as an output restart sector address (step 48 in FIG. 4), and the engine control processor 162 restarts the output processing (step 49 in FIG. 4).

The above description is of the embodiment in the case where the output files are insufficient.
When the sum total of the amount of data to be calculated and the amount of calculation result data exceeds the capacity of the large capacity memory 163, the calculation processing is performed using the work file. Even if the work file is insufficient, the work file can be treated in the same way by treating it as an output file. Therefore, the processing can be continued by the completely same means without interruption.

[0048]

As described above, according to the present invention, when the database arithmetic processing unit detects an output file shortage during output processing, the output is temporarily stopped and the shortage capacity is stored in the host unit. In response to an expansion request, the host device expands the output file by the capacity of the request, and by returning the start sector address of the expanded portion to the database arithmetic processing device again, restarts the output of the operation result data, By continuing the entire arithmetic processing, as a result, without interrupting the arithmetic processing and retrying, and while maintaining the cooperation between the host device and the database arithmetic processing device,
Efficient data processing can be realized in the entire system.

[Brief description of drawings]

FIG. 1 is a block diagram showing a configuration of an exemplary embodiment of the present invention.

FIG. 2 is a diagram showing a structure of an extension request command used in the above embodiment.

FIG. 3 is a diagram showing the structure of an extended response command used in the above embodiment.

FIG. 4 is a flowchart showing an output processing procedure of the above embodiment.

FIG. 5 is a flowchart showing an extension processing procedure of the above embodiment.

[Explanation of symbols]

10 ... Host computer, 13 ... System bus, 1
4, 15 ... Channel, 16 ... Database arithmetic processing device, 17 ... Magnetic disk device, 161, 162, 164
... processor, 163 ... mass memory, 165 ... parallel sorting module, 166 ... parallel relational algebra operation module.

Claims (2)

[Claims] 1. An operation issued from the host device, which has a host device, a secondary storage device in which file data to be operated is stored, and a path used when directly accessing the secondary storage device. In a data processing device comprising an arithmetic processing device for executing a predetermined arithmetic processing on the file data in response to a request, the arithmetic processing device processes the file data to be operated according to an arithmetic request from the host device. Data input means obtained from the secondary storage device via the path, arithmetic processing means for performing arithmetic operation on the input file data, the operation target file data and arithmetic result data by the arithmetic processing means are stored. Internal memory, output means for outputting the operation result data to the secondary storage device, and operation result data by the output means. Output capacity expansion means for detecting a capacity shortage of the output file during the output processing of the above, suspending the output processing, specifying the capacity shortage, and requesting the host apparatus to expand the capacity shortage, The host device receives the output capacity expansion request issued from the output capacity expansion request means of the arithmetic processing device,
The output file is expanded by a specified capacity, and a file expansion means for responding to the arithmetic processing device an address on the secondary storage device corresponding to the file expansion portion is provided, wherein the arithmetic processing device is the host device. A data processing apparatus, which receives a response indicating completion of extension from a file extension means and resumes output during suspension from an address corresponding to a file extension portion. 2. A host device, a secondary storage device for storing file data to be operated, and a path used when directly accessing the secondary storage device, and issued from the host device. In a data processing device comprising an arithmetic processing device for executing a predetermined arithmetic processing on file data in response to an arithmetic request, the arithmetic processing device communicates with the host device and communicates with the secondary storage device. A first processor for executing data input / output between the two; an arithmetic circuit for performing arithmetic logic operations such as sorting or relational algebraic operations; a second processor for executing and controlling operations by the arithmetic circuits; and the first processor described above. An operation target file data input via the above and an operation result of the operation circuit is stored in an internal memory;
A third processor that performs synchronous control of the first and second processors based on an instruction from the host device supplied via the processor of the third processor. When the calculation operation is completed according to the calculation request from the host device and the output is insufficient while outputting the calculation result data to the output file, the shortage is detected and a request is made to extend the insufficient capacity to the host device. Then, in response to the request, the host device expands the output file by the expansion capacity designated by the third processor, and responds to the third processor with a memory address corresponding to the expanded portion, A data processing device characterized by restarting output processing of result data that has been temporarily stopped.