JPH0293720A - Data base processing method - Google Patents

Abstract

PURPOSE:To parallelly access data and to improve through-put by obtaining a record housing area for user on a cache storage device and scheduling an input device so that cache collision can be minimum between plural processors. CONSTITUTION:A main memory 20 is obtained by processors 10-13 and a data base is housed in disks 30-35. Buffers 200-203 are allocated on the main memory 20 of the respective processors 10-13 and respective processor times are set so that the data in the buffer can not be mutually interfered by referring or updating, etc. Clusters 330-333 to be housed in the disk 33 are defined as a reading subject and applied results of selection and projection processing, etc., are respectively transferred to the buffers 200-203. When the data can be parallelly accessed, with considering the number of the processors or the performance of the respective processors, an input / output processing request can be scheduled so that the cache collision, etc., can be minimum between the plural processors.

Description

DETAILED DESCRIPTION OF THE INVENTION [Field of Industrial Application] The present invention relates to a database processing method suitable for parallel processing suitable for a relational database management system in a computer system.

[Conventional technology]

The database is assumed to consist of relations that can be seen by the user in the form of a two-dimensional table, and tuples correspond to the rows of the table. Further, a tuple consists of one or more attributes (referred to as an r column).

In database processing, for example, in relational database processing, the data to be processed exists on a secondary storage device, and each database operation requires reading and transferring a large amount of data. In such a database system, when a large amount of data is transferred, the data transfer time becomes a performance bottleneck of the database system.

Therefore, a method can be considered to make effective use of the time to transfer data from the secondary storage device. This overlaps the data transfer time and the time required for database processing of the data, and is well known as a conventional technique. in general.

In computer systems, the capacity of programs and data greatly exceeds the growth in main memory capacity, so the current concept of storage hierarchy is essential.

Furthermore, a well-known approach is to place an input/output device including a semiconductor memory device between a main memory device and a secondary memory device, aiming at shortening the time required for input/output processing. This semiconductor storage device reads data close to the record that is the target of attendance in units of multiple records all at once. Then, if the type of processing is such that it can be assumed with a statistically high probability that the record semiconductor storage device that received the access request immediately after, the input/output processing time can be dramatically reduced. That is, it would be more effective if the database processing system DBMS on the central processing unit could pre-read and store data that can be predicted to be accessed in the semiconductor storage device in advance. However, as mentioned above, database processing involves a mixture of sequential access type processing and random access type processing.
It has been found that in conventional control methods for semiconductor memory devices, the input/output processing time increases due to phenomena such as frequent read errors.

Furthermore, to improve the processing performance of the CPU.

Architectures with multiple processing units have been implemented. In particular, an approach in which the main memory is shared by each processing unit is promising. Usually, each processing unit has a cache to increase the throughput of data reading. However, since a single main storage device is shared, data collisions may occur between the caches of each processing device, and the performance of each processing device may deteriorate due to invalidation of the cache or the like.

[Problem to be solved by the invention]

In the above conventional technology, a central processing unit consisting of multiple processing units that share main memory utilizes access characteristics that can be predicted in advance by a DBMS, and actively schedules the cache conflicts among the processing units to minimize them. , the parallel processing method to shorten processing time was not clear.

The purpose of the present invention is to analyze queries from users, understand the access characteristics to the database from the created internal processing procedures, consider the number of processing devices and the performance of each processing device, and Reduces input/output processing time by scheduling to minimize cache collisions, etc., determining whether parallel operations are involved when starting input/output processing, and comprehensively determining parallel processing units based on system characteristics, etc. The object of the present invention is to provide a database processing method that improves the performance of the entire system.

[Means to solve the problem]

In order to achieve the above object, the present invention includes a central processing unit consisting of one or more processing units that analyze queries from a plurality of users to a database, determine a corresponding internal processing procedure, and execute the procedure; a secondary storage device that stores data that can be shared by users; and an input/output processing device that controls transfer of a plurality of records requested for input/output processing between the central storage device and the secondary storage device. A process of determining whether or not a parallel processing operation is to be performed when starting input/output processing based on an internal processing procedure created based on an input timing in a computer system including the above.

If the parallel processing operation is involved, a step of transferring input/output processing results to an area executable by each processing device capable of image processing;
It is characterized by the process of dividing the input/output processing results using a hash function or the like and scheduling each input/output processing device so as to minimize cache collisions between each processing device.

[Effect]

According to the database processing method of the present invention, the CBMS
Acquire one or more record storage areas for each user on the cache storage device according to the access characteristics predicted by Schedule input processing devices to minimize collisions, determine whether parallel operations are involved when starting input/output processing, and comprehensively determine parallel processing units based on system characteristics to efficiently perform parallel processing. By doing this, the input/output processing time can be significantly reduced and the throughput of the entire system can be improved.

〔Example〕

Hereinafter, one embodiment of the present invention will be described in detail based on the drawings.

FIG. 1 shows a DBMS database processing method to which the present invention is applied. However, the explanation of FIG. 1 will be given later.

FIG. 2 is an example of a functional configuration of a database system to which the present invention is applied.

The database system is an application program (hereinafter abbreviated as AP) 40.41 created by a user.
, a DBMS 50 that manages the entire database system such as reconciliation processing and resource management, and an operating system that manages the entire system, such as an input/output management program 61 that reads and writes data that is subject to input/output processing in database processing. It is composed of a system (hereinafter, the operating system will be abbreviated as O8) 60 and a database 70.

The DBMS 50 includes a system control unit 51 that manages and controls the entire system as well as input/output management, and a timing analysis unit 520 that performs syntax analysis and semantic analysis of queries. A logic processing unit 52 consisting of an optimization unit 521 that generates an appropriate processing procedure, and a code generation logic processing execution unit 522 that generates internal code and interprets and executes this code, and the DBM.
A database buffer control unit 57 that manages the database buffer 58 that is processed in S50, a data access processing unit 54 that implements condition judgment 1 editing of accessed data, etc., and an exclusive control 55 that implements exclusive control of resources shared by the system. ．． and a physical processing unit 53 that performs database physical processing, etc., and a mapping processing unit 56 that performs mapping from a logical database space (page access space) to a physical database space (daily access space). has been done.

FIG. 3 shows an example of a hardware configuration to which the present invention is applied. Specifically, there is a central processing unit (CPU) 80 that holds the main memory (MM) 20, a channel unit (CHU) 81 that performs input/output control, and a processing unit that performs selection, constraint, projection, combination, etc. in database processing. An intelligent file control device (IFC) 82 (
This intelligent file control device 82 consists of a disk 30 (corresponding to the above-mentioned input/output control device) and a disk 30 (this disk 30 corresponds to the above-mentioned secondary storage device).

A path 87 is used to transfer relative block addresses, input/output data, etc. between the intelligent file control bag @82 and the disk 30. Further, filtering processing results, search conditions, etc. are transferred between the channel device 81 and the intelligent file control device (IFC:) 82, and routes 85 and 86 are used. Furthermore, a path 84 is provided between the central processing unit 80 and the channel device 81 for data transfer and control data exchange.
is used. A plurality of channel devices 81 and intelligent file control devices 82 may be connected to both the paths 84 and 86, respectively. The route 85 is
Indicates that it may be used to share the intelligent file controller 82 with other systems.

This configuration differs from the system in which the 08MS 50 implemented on the central processing unit 80 executes all database processing. That is, among the database processing, relational algebraic operations corresponding to the internal processing code generated by the code logic processing execution unit 552 of 08MS50, that is, various operations such as selection, constraint, projection, and combination, are shared by the intelligent file control device. It is. This corresponds to what is called an input/output device with various arithmetic functions (for example, a filtering processing device) added thereto.

This basic operation is shown below.

(1) The 08MS50 analyzes the inquiry input from the application program and determines the processing procedure.

(2) In accordance with the above processing procedure, a relational algebraic operation is extracted and a processing request is issued to the intelligent file control device 81.

(3) The intelligent file control device 81 interprets the processing request and issues a physical input/output request to the disk 30. The database 7° is stored in a relation format, and pages, which are management units of this database, are read in synchronization with the data transfer of the disk 30 and set in the local storage 820.

(4) Perform operations such as selection, constraint, projection, and combination on the data in the local storage 820, and perform filtering processing.

(5) Set this processing result in the database database buffer 58 of the 08MS50.

(6) 08 The MS 50 edits the self-processing results and outputs the adjustment results to the application program.

Next, FIG. 1 will be explained. This is a database processing method of a DBMS in a database system to which the present invention is applied.
0, 31, 32, 33° 34. 35 and buffers 200, 201, 202 that store temporary copies of the database.
, 203, and processing devices 10, 11,
It consists of 12°13. Further, each disk 30, 31, 32, 33, 34, 35 is composed of a cluster group consisting of a plurality of pages. Specifically, the disk 33 is
It is shown that clusters 330, 331, 332 and 333 are formed. In this configuration example, four processing devices 10,
11, 12, and 13 share the main memory 20, and the database is stored on six disks 30, 31, 32, 33, and 3.
4,35. Each processing device 10, 11, 1
At 2.13, there is a buffer 200.201 on the main memory 20.
．． 202 and 203 are allocated, and settings are made so that data in the buffer does not interfere with each other when referenced or updated during each processing device time. The clusters 330, 331, 332, and 333 stored on the disk 33 are read, and the results of selection, projection, and the like are transferred to the corresponding buffers 200, 201, 202, and 203, respectively.

As a modification of the 0-wire configuration example in which processing is performed on each disk in the same way and the input/output processing results are set in each buffer,
If there are 8 processing units.

It is possible to schedule access to each cluster on each disk and transfer the processing results to each buffer using a hash function or the like. Also.

If there are two or more central processing units, it is also possible to access each cluster on each disk and schedule the I/O processing results to be transferred to each buffer of each central processing unit using a hash function. be.

The feature of this embodiment is that if data can be accessed in parallel, input/output processing requests are made in such a way that cache conflicts among multiple processing units are minimized, taking into account the number of processing units and the performance of each processing unit. It is possible to schedule.

A comprehensive explanation of the DBMS that realizes this feature will be provided.

FIG. 4 is a flowchart of DBMS processing to which the present invention is applied. Below, detailed flowcharts of each processing unit will be explained.

If the processing request received from the application program is an open request, the system control unit 51 allocates control blocks etc. necessary for executing the command (processing 5100-5101), and if the processing request is a close request, Releases control blocks etc. necessary for executing this command (processing 5102-5103), and if it is a commit request, all pages locked in this transaction are unlocked (processing 5104-5105), or if it is a fetch request. For example, the process 51 of calling the code generation logic processing unit 522
06-5107), if it is a setup request, the process of calling the logic processing unit 522 5108-5109),
Each ends.

The logic processing unit 52 executes syntactic analysis and semantic analysis of the makeshift command (process 520), determines the optimal data access processing procedure (process 521), develops this processing procedure into an executable format, interprets it, and executes it (process 520). 522), end.

The physical processing unit 53 includes a data access processing unit 54, a database buffer control unit 57, and a mapping processing unit 56.
and an earth discharge control section 55.

If the search range information for the data to be read has been created, the data access processing unit 54 proceeds to process 5404.
Otherwise, if the search range spans multiple clusters, that is, check whether data can be accessed in parallel (5401), if possible, the search range should be set in units of disk volumes that can be accessed in parallel. Create (processing 5401゜5402.5404), otherwise,
Create search range (processing 5401, 5403゜540
4). Allocate a hash function evaluation value corresponding to a processing device capable of parallel processing, and schedule the transfer of input/output processing results to each buffer in main memory 6. Next, lock the page set that is the target of input processing. (processing 540
5). Furthermore, the page set to be input processing is read, and the data format, calculation result information, etc. specified in the input processing request are created (processing 5406). If the page set that is the target of input processing is composed only of records that do not satisfy the conditional expression, this page set will undergo unlock processing (processing 5407, 5408, and 5409), and will become the target of input processing. If the conditional expression specified in the input processing request cannot be evaluated due to record overflow, etc. (processing 5409), the current record is read using the record address string pointing to the record set for which the conditional expression cannot be evaluated. Lock the page set that stores the set 5410), create a disk storage address string that stores this page set 5411), access the thermal data, evaluate the search conditions, and retrieve the data specified in the input processing request. Process to create format, calculation result information, etc. 541
2) If a page is composed of only records that do not satisfy the conditional expression (processing 5413), the page set is unlocked (processing 5413, 5414, 5416),
It also accesses the record set that satisfies one conditional expression, and transfers the processing results to each buffer in the main memory in the specified data format according to the hash function evaluation value (processing 5415).
, 5416) If the page set that is the target of input processing does not hit the buffer, the process proceeds to step 54218. Otherwise, the page set that was hit in the buffer is accessed, the search condition expression is evaluated, and the customization is performed according to the input processing request. Create and process the specified data format, calculation result information, etc. 5417),
If the page is composed only of records that do not satisfy the conditional expression (processing 5418), the page set is unlocked (processing 5418.5420.5421), and the record set that satisfies one conditional expression is accessed. , transfers the processing results to each buffer in the main memory according to the hash function evaluation value in the specified data format (processing 5419.5
421), if the entire search range has not been read (processing 542
1), proceed to process 5400.

Otherwise, follow the process below.

1: When sorting is not specified If there is no sorting specified, the processing results are accessed and the data is passed to the application program (processing 54).
22,5428).

Case 2: When sort processing is specified, sort processing is performed for each processing device, and a series is created for each process 5422,
5423), and processes for merging multiple series and creating sorting results (5424 and 5429), and further accessing the processing results and passing the data to the application program (processing 5428).

Case 3: When join processing is specified, perform sort processing for each processing device and create a series for each processing 5422
, 5423), then merge the runs created by each processing device to create an intermediate processing result. (species 5424,
5425).

Further, the intermediate processing result is subjected to margin processing, a join processing result is created (processing 5426), the processing result is accessed, and the data is passed to the application program (processing 5428).

The database buffer control unit 57 determines which hash function evaluation value to allocate to the buffers 200, 201, 202, and 203 on the main memory 20, and assigns the buffers 200, 201, 202, and 2 for each input/output processing request unit to be read in parallel.
03 (processing 530), searches the database buffer 58 to find a page set to be input processed (processing 571), and if all page sets to be input processed are found, the buffer storing this page set is locked. Processing 572, 577) is completed.

Otherwise, as a result of searching the database buffer 58, lock only the buffer storing the page set found (573), and as a result of the above search,
The page set that was not found is converted into a disk storage address string (574).

Process 575 to acquire a buffer for storing the data format, calculation result information, etc. specified in the input processing request, and issue an input/output request to the input/output management program 61 of 0860 for each disk unit that can be accessed in parallel. ), the data format specified in the input processing request, calculation result information, etc. are received (process 576), and the process ends.

The mapping processing unit 56 converts the search range given by the page set into a disk storage address string (process 560), and if this disk storage address string spans multiple disks, the disk storage address string is 561, 562), and ends.

If it is an unlock process, the exclusive control unit 56 performs a DEQ process on the resource to be unlocked (process 560).
, 562). For lock processing.

Performs ENQ processing for the resource subject to lock processing (processing 5
60,561). However, if a deadlock is detected, the resources subject to the putlock processing are released (processes 561, 563, 564).

The processing flow has been explained above.

The present invention can be realized through a software system of a large computer, or through a multiprocessor system in which a dedicated process is prepared for each processing unit.

〔Effect of the invention〕

According to the present invention, one or more record storage areas are acquired for each user on the cache storage device according to the access characteristics predicted in advance by the DBMS, and the number of processing devices and the performance of each processing device are taken into consideration. In addition, by scheduling input devices to minimize cache collisions among multiple processing devices, it becomes possible to efficiently perform parallel processing, and the throughput of the entire system improves.

Furthermore, since input/output processing is performed in parallel, there is an effect of reducing input/output processing time.

[Brief explanation of drawings]

FIG. 1 is a diagram showing an example of database processing of a DBMS to which the present invention is applied, and FIG. 2 is a diagram showing an example of the configuration of a database system to which the present invention is applied. FIG. 3 is a diagram showing an example of a hardware configuration to which the present invention is applied, and FIG. 4 is a processing flowchart of a DBMS to which the present invention is applied.

Claims (1)

[Claims] 1. A central processing unit consisting of one or more processing units that analyzes queries from multiple users to the database, determines a corresponding internal processing procedure, and executes the same, and is shared by each user. a secondary storage device that stores data that can be processed, and an input/output processing device that controls the transfer of a plurality of records requested for input/output processing between the central storage device and the secondary storage device. In the system, there is a process of determining whether or not parallel processing is to be performed at the start of input/output processing based on internal processing procedures created from an input query, and if parallel processing is involved, the process of determining whether or not the input/output processing results are to be performed. The process of transferring to an area that can be executed by each processing device that can perform parallel processing, and dividing the I/O processing results using a hash function, and scheduling each I/O processing device to minimize cache collisions between each processing device. A database processing method characterized by comprising the steps of: 2. A database processing method according to claim 1, comprising a step of acquiring one or more record storage areas for each user's arrangement, and performing parallel processing operations based on a specified search range. 3. A database processing method according to claim 1, comprising the step of performing exclusive control on a plurality of records to be read in advance through lock processing. 4. Claim 1 is characterized in that it includes a step of securing an area for reading a plurality of records into a database buffer on main memory, and a step of sequentially replacing and controlling and managing this area. Database processing method. 5. In claim 1, there is provided a process for storing an address string for storing records that satisfy a given search condition expression, and an address for storing pages containing only records that do not satisfy the given search condition expression. A process of storing columns, a process of storing an address column that stores records for which the given search condition expression could not be evaluated due to record overflow, and at least one record that satisfies the given search condition expression. The process of storing the address string that stores the page, the process of storing the results of editing the records that satisfy the given search condition expression into page format with only the specified fields, and the process of storing the results that satisfy the given search condition expression. 1. A database processing method characterized by comprising the step of storing only. 6. In claim 1, if there are multiple central processing units, each secondary storage device is accessed via each input/output processing unit, and the input/output processing result is converted into a hash function evaluation value. A database processing method comprising the step of scheduling transfer to each database buffer on each central processing unit. 7. Claim 1 includes the step of dividing the record storage range to be searched into a record address string that can be accessed in parallel, and the step of issuing an input/output processing request using the record address string. A database processing method characterized by: