JPH1153404A - Multidimensional totalizing process method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To shorten the time needed for a totalizing process by dividing a multidimensional data base into totalization units and not using a totalization unit which is used for totalization in each step in the next step. SOLUTION: A request for multidimensional totalization is inputted through an input device 1 and analyzed in a data controller 2. Then division into partial totalization units is performed on the basis of the analysis result. Further, parallelism is calculated by steps as to the divided partial totalization units and the execution procedure of partial totalizing process performed in the respective steps is determined in consideration of the number of DB dedicated processors 5. Then the partial totalization units are allocated to the DB dedicated processors 5 until all the steps of parallel execution end and the parallel execution is carried out. To perform execution in partial totalization units, a data file is read out of an external storage device 4 and after a totalizing process is performed, the result is written to the external storage device 4. The totalization result which is obtained finally is outputted to an output device 3.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a database processing method and, more particularly, to an improvement in a tallying method of a multidimensional database management system.

[0002]

2. Description of the Related Art In general, in a multidimensional database management system, a method using an index is employed for realizing a high-speed aggregation process. When an index is used, a method of generating and processing an index in advance and a method of generating and processing an index at the time of execution are roughly divided into two methods. Detailed data, which is data at the detailed level, such as a multidimensional database, is frequently updated, so that the index is frequently regenerated during a search.

On the other hand, as another approach, there is a method of performing a totalizing process by realizing a high-speed sorting without using an index. When a multi-dimensional total result is to be obtained from the detailed data, this is a method of obtaining the final total result by performing totaling for each dimension by data processing centered on sorting. When the result of totaling all the dimensions of the detailed data as one key is used as primary aggregated data, N-dimensional aggregation can be performed by performing the aggregation processing for each dimension N times for the primary aggregated data. .

[0004] Next, the operation of the prior art in the multidimensional aggregation processing without using the index will be described. FIG.
The tabulation result calculated by the four-dimensional tabulation process is shown.
Is the primary aggregation result for the detail data, 52 is the aggregation result for dimension 1 when the primary aggregation data 51 is input, and 53 is the
Counting result for dimension 2 when 51 and 52 are input, 54
Is a totaling result for dimension 3 when 51, 52, 53 is input, and 55 is a totaling result for dimension 4 when 51, 52, 53, 54 is input. Conventionally, when calculating a four-dimensional tally result, as shown in FIG. 14, the first tally result 0 (data shown in black) is input to the tally result 0 (data shown in black) as a step 1 as shown in FIG. 10 (data shown in black) is calculated and the tally results 0 and 10 are output. In step 2, the tally results 0 and 10 are input to calculate tally results 20 and the tally results 0, 10, and 20 are output. The totaling result 0, 10, 20, 30 is calculated as the input, the totaling result 30 is calculated, the totaling result 0, 10, 20, 30 is output, and the totaling result 0, 10, 20, 30 is input as the step 4, the totaling result 40 is input as the step 4. And output the tally results 0, 10, 20, 30, and 40.

[0005]

In the conventional multidimensional totaling method, it is necessary to wait for all the output results of the step (N-1) in order to perform the totaling process for the dimension N in the step N. In a system having a CPU or a plurality of data processing-dedicated devices, it is not possible to perform the parallel processing by dividing the tallying process, and it is not suitable for high-speed processing.

The processing in step N is performed in step (N-
This is a tallying process in which the tallying result of 1) is input. As the output result, in addition to the tallying result of the dimension N, the input data is also output as a result. There has been a problem that the input data increases in a snowball manner and the processing time increases.

SUMMARY OF THE INVENTION The present invention has been made to solve the above-mentioned problems, and is intended to divide the totaling process of each step into processing units which can be executed in parallel and to process the partial totaling unit at high speed. The purpose is to provide a method.

[0008]

According to the present invention, there is provided a multi-dimensional totaling method, comprising: a multi-dimensional database system having a detailed level data file and a hierarchical structure information table holding a hierarchical structure of each dimension; A step of totalizing the first aggregation data, which is a totaling unit of each dimension obtained by directly totaling, a second totaling, which is a totaling unit of each dimension, obtained by directly totaling each of the first totaling data; It consists of a step of counting data.

Further, the first total data or the second total data composed of a plurality of total units are totaled simultaneously.

[0010] Furthermore, the totaling unit to be processed at the same time is 2
The tallying process is performed in more than one time.

In addition, a totaling result record having a totaling value of 0 is generated for each dimension, and break key information consisting of numbers indicating the totaling result record corresponding to each dimension is stored in the totaling result record in ascending or descending order. Is added to.

Further, when it is possible to total the same total unit from any of the plural total units, one of the plural total units is exclusively monopolized and totaled. is there.

[0013] The present invention also compares and verifies the tally results obtained from each of the plurality of tally units.

Further, in a system having a plurality of multi-dimensional databases having a data file at a detailed level and a hierarchical structure information table holding a hierarchical structure of each dimension, a first totalization of a first multi-dimensional database and a second multi-dimensional database is performed. Are simultaneously processed.

[0015] The primary tabulation of the first multidimensional database is tabulated, and the primary tabulation of the second multidimensional database is tabulated simultaneously with the first step of the first multidimensional database. Of the multi-dimensional database are sequentially and simultaneously counted.

Furthermore, the primary tabulation of the first multidimensional database is tabulated, and the primary tabulation of the second multidimensional database is converted to the second tabulation of the first multidimensional database.
Is performed at the same time as the above step, and thereafter, successive steps of each multidimensional database are sequentially and simultaneously processed.

[0017]

BEST MODE FOR CARRYING OUT THE INVENTION

Embodiment 1 FIG. FIG. 1 is a configuration diagram of a multidimensional database processing apparatus according to Embodiment 1 of the present invention, in which 1 is an input device for inputting a request for multidimensional aggregation, 2 is a data control device for controlling aggregation processing, and 3 is an aggregation result. An output device that outputs
Reference numeral 4 denotes a multidimensional database and an external storage device for storing the results of counting, and reference numeral 5 denotes a database dedicated processing device (hereinafter, referred to as a DB dedicated processing device) that performs a counting process for each counting unit of each dimension.

Next, based on the processing procedure of FIG. 2, the process of processing will be described while corresponding to the apparatus of FIG. First, a request for multidimensional aggregation is input from the input device 1 (Step S).
1), the request is analyzed in the data control device 2 (step S2). Next, based on the analysis result, the data is divided into partial tabulation units as shown in FIG. 3 (step S3). Further, the degree of parallelism is calculated for each step as shown in FIG. 4 with respect to the divided partial aggregation units, and is executed for each step in consideration of the number of DB dedicated processing devices 5 connected to the present apparatus. The execution procedure of the partial tabulation process is determined (step S4). Next, until all the steps of the parallel execution are completed, the respective partial aggregation units are allocated to the DB dedicated processing device 5 and the parallel execution is performed (step S5). In the execution of the partial counting unit, the data file is read from the external storage device 4, the counting process is performed, and the result is written in the external storage device 4. Finally, the obtained totaling result is output to the output device 3
(Step S6).

FIG. 3 shows a state in which a partial tabulation process is executed by dividing the data into partial tabulation units. The difference from the prior art is that the tabulation unit is subdivided corresponding to each dimension. The point is that there is an aggregation unit that can be executed in parallel. In FIG. 3, the one added in parentheses for each tabulation unit, for example, (← 0) indicates that the tabulation data can be directly tabulated from the tabulation data of the tabulation unit 0 which is the primary tabulation data. The tallying units are the tallying unit 10, the tallying unit 30, and the tallying unit 40, which means that they can be executed in parallel.

FIG. 4 is a diagram showing the degree of parallelism at each counting stage in the four-dimensional counting process. At the stage of the primary aggregation, the degree of parallelism is 1, and in the first step, the aggregation processing for calculating the aggregation results 10, 20, 30, and 40, which are the aggregation results, can be executed in parallel by inputting the aggregation unit 0. Yes, the degree of parallelism is 4. The degrees of parallelism in the partial aggregation processes in the second, third, and fourth steps are 6, 4, and 1, respectively. In general, the degree of parallelism in step i in the N-dimensional totaling process is NCi (a combination of extracting i pieces from N pieces).

FIG. 5 is a diagram showing the tally result calculated in each step with respect to the four-dimensional tally result shown in FIG. In each step, the data transfer amount is minimized by outputting only the aggregation result for the dimension to the input data. In FIG. 5, step 1 shows a state in which the first tabulation data including the tabulation units 10, 20, 30, and 40 is tabulated from the tabulation data of tabulation unit 0, and step 2 is a tabulation unit 10 From the first aggregation data consisting of 20, 30, and 40 aggregation units, the aggregation units 21, 31, 3
The figure shows how the second tabulation data composed of 2, 41, 42, and 44 tabulation units are tabulated. Similarly, step 3,
By repeating step 4, the totaling result finally totaled for all four dimensions is totaled in the totaling unit 47. FIG. 6 shows a specific example of the counting result in step 1 at this time. The numbers in FIG. 6 indicate the numbers of the aggregation units in FIG.

As described above, the processing of multidimensional aggregation is divided into partial aggregation units, assigned to a plurality of CPUs or an external data processing device, and executed in parallel.
It is possible to perform high-speed multidimensional aggregation processing. Here, when there is a limit to the number of the DB dedicated processing device 5 or the number of CPUs or external data processing dedicated devices, it is possible to divide the processing into two or more times in the processing stage of each totaling processing. Further, since only the tally result record excluding the input data is used as the input data in the next step, the processing time can be reduced.

As shown in FIG. 4, in a set of partial totalization units that can be executed in parallel, the input data file may be common. A method of supplying data to each DB dedicated processing device 5 that executes a partial totaling unit in such a case will be described with reference to FIG. The number of the partial aggregation processing units is set to N (step S11). First, a common input data file is read into a storage area (not shown) in the data control device 2 (step S12), and this data is stored in each D.
By writing to the B-dedicated processing device 5 in order, each DB-dedicated processing device 5 performs the process of totaling and writing for the partial totaling processing unit (step S13), and the number of times of data writing becomes N The process is executed until the values become equal (step S14). If the written data is not the last data of the input data file, the same processing is repeated (step S15). By adopting such a data supply system, data transfer between the external storage device 4 and the data control device 2 can be reduced, and overhead due to data transfer can be suppressed.

FIG. 8 shows the configuration of the DB dedicated processing unit 5 that executes a partial tabulation unit, and comprises a data processing unit 6 and a sort processing unit 7. The data processor 6 exchanges data with the data controller 5 and the sort processor 7 by DMA.
At the same time as performing using the function, data processing such as selection processing and total calculation is performed. FIG. 9 shows a processing procedure of the data processing unit 6. Here, a table in the format shown in FIG. 10 is assumed as the detailed data file, and the category structure corresponding to the fields of this table is shown in FIG. First, the data control device 2 reads the category information into a storage area (not shown) in the DB dedicated processing device 5 (step S21). Next, the data of the detailed data file is read into the storage area by DMA transfer (step S22).
At the same time, selection / projection processing according to the request is performed (step S23). In the projection processing, the data is converted into a data format for inputting data to the sort processing unit 7. Further, after setting the key break information for this data (step S24), the data is input to the sort processing unit by DMA transfer (step S26).

This process is repeated until reading of the detailed data file is completed (step S25). If the end of the data of the detailed data file is detected, data of the total value 0 is generated according to the category information (step S).
27) After setting the key break information (step S24), the generated data is input to the sort processing unit 7. This process is repeated until the generation of the tally data ends (step S28). The total data to be generated is performed on the partial total result for each dimension as shown in FIG. The data sorted by the sort processing unit 7 is read to the storage area of the data processing unit 6 by DMA transfer (step S29). After performing a set operation process (step S30) and a selection process for the set operation result (step S31) on the data, the totalized result is output to the data control device 2 (step S32).
As the data to be output here, only a partial aggregation result excluding the input data is output.

FIG. 12 shows a data format to be input to the sort processing unit 7 and its data. The data format input to the sort processing unit 7 includes a sort key string and an output item string. The sort key column specifies a value obtained by converting the category information for each dimension as shown in FIG. 11 into a unique ID. At the end of the sort key sequence, break key information is set. In addition to the input data, the sort processing unit 7 also inputs and sorts the generated data of the tally value 0 corresponding to the partial tally result. As a result, the key break processing for grouping performed on the sorting result conventionally performed the key comparison, but the setting of the break key information eliminates the necessity of the key comparison, and the processing speed can be expected to be increased.

Embodiment 2 FIG. In the second embodiment, a system having a plurality of multidimensional databases having a data file at a specification level and a hierarchical structure information table holding a hierarchical structure of each dimension will be described. In the system of FIG. 1, the multidimensional database is stored in a plurality of external storage devices 4, and the database is controlled by the data control device 2.
By using the dedicated processing device 5 in common among the multidimensional databases, it is possible to execute a plurality of multidimensional databases simultaneously.

Scheduling is important in order to use the DB dedicated processing device 5 in common among the multidimensional databases. In the case where there are a plurality of sets of partial aggregation units that can be executed in parallel from the detailed data in FIG. In order to effectively use the limited DB dedicated processing device 5, the following scheduling is conceivable.

First, in the stage of the primary aggregation of the degree of parallelism 1, the aggregation unit 0 of the first and second multidimensional databases is simultaneously executed, and the first step of the stage of the degree of parallelism 4 of the second multidimensional database is performed. To the degree of parallelism of the first multidimensional database
This is a scheduling in which the tabulation process is performed simultaneously with the second step, which is the stage, and the successive steps of the multidimensional database are sequentially and simultaneously performed. Naturally, if the third multidimensional database exists, the subsequent processing is performed in the same manner as the relationship between the first and second multidimensional databases. This scheduling enables load distribution at the stage of primary aggregation. In addition, the schedule after the totaling unit 0 of the first and second multidimensional databases is simultaneously executed at the stage of the primary aggregation is not limited to the above-described case, and various cases can be considered. Further, depending on the number of DB-dedicated processing devices 5, in each step of each multidimensional database, it is also possible to carry out the tallying process in two or more steps.

Next, after the primary aggregation of the first multidimensional database is totalized, the primary aggregation of the second multidimensional database is aggregated simultaneously with the first step of the first multidimensional database. This is a scheduling in which successive steps of the multidimensional database are sequentially and simultaneously counted. This scheduling is suitable for distributing the load when the type of the detailed data is small and performing the totaling processing in a relatively short time.

Next, after the primary tabulation of the first multidimensional database is processed, the primary tabulation of the second multidimensional database is tabulated simultaneously with the second step of the first multidimensional database. This is a scheduling in which successive steps of the multidimensional database are sequentially and simultaneously counted. With this scheduling, when the type of detailed data is large, it is suitable for distributing the load to a small number of DB-dedicated processing devices 5 and performing the aggregation process.

[0032]

Since the present invention is configured as described above, it has the following effects.

Since the multidimensional database is divided into aggregation units and the aggregation units used for aggregation in each step are not used in the next step, the time required for aggregation processing can be shortened.

Further, since the totalizing unit in each step is configured to perform the totalizing process at the same time, the processing speed of the totalizing process can be increased.

Further, the totaling unit to be processed at the same time is 2
Since it is configured to perform the tallying process more than once,
Even when the number of U or dedicated devices for data processing is limited, the processing speed can be improved by parallel execution.

Further, since the ascending or descending break key information consisting of numbers is added to the totaling result record, the totaling result record is added to the totaling data and sorted, so that the key can be compared without performing key comparison. Break processing can be achieved.

Furthermore, since any one of the common tabulation units is exclusively and exclusively tabulated, the tabulation process can be executed without any input conflict. .

In addition, since the counting results obtained from the respective counting units of the common counting unit are configured to be compared and verified, it is possible to check whether the counting result is correct or incorrect in the process of performing the counting process.

Furthermore, in a system having a plurality of multi-dimensional databases, the first multi-dimensional database and the second multi-dimensional database are configured to process the primary total at the same time. Can be processed.

Further, in a system having a plurality of multidimensional databases, the aggregation processing steps of the first multidimensional database and the second multidimensional database are shifted by one step, so that there are few types of detail level data files. In such a case, the load can be distributed and the aggregation process can be performed in a relatively short time.

Furthermore, in a system having a plurality of multi-dimensional databases, the total multi-steps of the first multi-dimensional database and the second multi-dimensional database are shifted by two steps, so that there are many types of detail level data files. In this case, the load can be distributed to a small number of DB-dedicated processing devices 5 to perform the aggregation process.

[Brief description of the drawings]

FIG. 1 is a configuration diagram illustrating a multidimensional database processing device according to a first embodiment of the present invention.

FIG. 2 is a flowchart illustrating a processing procedure of the multidimensional database processing device according to the first embodiment of the present invention.

FIG. 3 is a schematic diagram illustrating a multidimensional aggregation processing method according to the first embodiment of the present invention;

FIG. 4 is a timing chart illustrating a multidimensional aggregation processing method according to the first embodiment of the present invention;

FIG. 5 is an explanatory diagram showing a multidimensional aggregation processing method according to the first embodiment of the present invention;

FIG. 6 is an application example showing the multidimensional aggregation processing method according to the first embodiment of the present invention.

FIG. 7 is a flowchart showing a processing procedure when the same file is input according to the first embodiment of the present invention.

FIG. 8 is a configuration diagram illustrating a DB-dedicated processing device according to the first embodiment of the present invention;

FIG. 9 is a flowchart illustrating a processing procedure of the DB dedicated processing device according to the first embodiment of the present invention;

FIG. 10 is a table showing a detailed data table according to the first embodiment of the present invention.

FIG. 11 is a configuration diagram showing a category structure according to the first embodiment of the present invention.

FIG. 12 is an explanatory diagram showing a data input format to a sort processing unit.

FIG. 13 is a schematic diagram showing a conventional multidimensional aggregation processing method.

FIG. 14 is an explanatory diagram illustrating a conventional multidimensional aggregation processing method.

[Explanation of symbols]

1 input device, 2 data control device, 3 output device, 4
External storage device, 5 database (DB) dedicated processing device.

────────────────────────────────────────────────── ───

[Procedure amendment]

[Submission date] December 10, 1997

[Procedure amendment 1]

[Document name to be amended] Drawing

[Correction target item name] FIG.

[Correction method] Change

[Correction contents]

FIG.

Claims (9)

[Claims] 1. A multi-dimensional database system having a detailed level data file and a hierarchical structure information table holding a hierarchical structure of each dimension, a totaling unit of each dimension obtained by directly totaling the totaling data. 1. A multidimensional tabulation processing method, comprising: a step of tabulating one tabulated data; and a step of tabulating second tabulated data which is a tabulation unit of each dimension obtained by directly tabulating the first tabulated data. 2. The multidimensional aggregation processing method according to claim 1, wherein the first aggregation data or the second aggregation data comprising a plurality of aggregation units is aggregated simultaneously. 3. The multidimensional aggregation processing method according to claim 1, wherein the aggregation processing is performed by dividing the aggregation unit to be aggregated into two or more times at the same time. 4. A totaling result record having a totaling value of 0 is generated for each dimension, and break key information in ascending or descending order consisting of a number indicating a totaling result record corresponding to each dimension is stored in the totaling result record. 4. The multidimensional aggregation processing method according to claim 1, wherein the number is added to the total number. 5. The method according to claim 1, wherein when the same total unit can be totaled from any of the plural total units, one of the plural total units is exclusively monopolized and totaled. The multidimensional aggregation processing method according to any one of claims 1 to 4. 6. The multidimensional tabulation processing method according to claim 5, wherein a comparison result obtained from each of the plurality of tabulation units is compared and verified. 7. A system having a plurality of multidimensional databases each having a detailed level data file and a hierarchical structure information table holding a hierarchical structure of each dimension,
7. The multidimensional totaling method according to claim 1, wherein the first totalizing of the multidimensional database and the second totaling of the second multidimensional database are simultaneously performed. 8. A primary tabulation of the first multidimensional database is tabulated, and a primary tabulation of the second multidimensional database is tabulated simultaneously with the first step of the first multidimensional database. 7. The multidimensional totaling method according to claim 1, wherein the successive steps of the multidimensional database are sequentially and simultaneously processed. 9. A first tabulation of the first multidimensional database is tabulated, and a first tabulation of the second multidimensional database is tabulated simultaneously with the second step of the first multidimensional database. 7. The multidimensional totaling method according to claim 1, wherein the successive steps of the multidimensional database are sequentially and simultaneously processed.