JP2007310592A - Data processing system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To increase search speed by reducing disk I/O in a DB server. <P>SOLUTION: A data processing system 10 has a DB server 14 and an AP server 12. The DB server 14 has a database management system 28 and a database 30 storing a table. The AP server 12 has a memory 26, a data processing part 22 for commanding the DB server 14 to read data, and a data compression section 24 for converting data transmitted from the DB server 14 into data of a tree structure and storing it in the memory 26. The data processing part 22 executes the retrieval processing to data of the tree structure. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a data processing system, and more particularly to a technique for realizing an increase in search speed in a data processing system including an AP server and a DB server.

Along with the development of client-server systems, in order to meet the demand for larger-scale information processing, in addition to the client that displays data, it has an AP server that processes data and a DB server that stores data. A so-called three-layer client-server system has become widespread.
In order to improve the processing speed, a load is distributed by arranging a plurality of AP servers in parallel.
JP 2005-165610 A

By the way, the AP server can be configured with an inexpensive PC server, so it is relatively easy to increase the processing speed by increasing the number of installed servers, but it is necessary to maintain data synchronization for the DB server Therefore, it is not possible to shift to distributed processing as easily as AP server.
Of course, database system vendors have been using various technologies to speed up the DB server itself from both the software and hardware sides, and as a result, certain results have been achieved. It cannot be denied that prices will rise.
In addition, as long as the scale of the database handled by the client-server system continues to increase, the disk I / O (data read / write) speed will eventually become a barrier, and there will be a time when the performance of the DB server cannot be improved. is expected.

  The present invention has been devised to solve the above-described problems that the conventional data management system has, and does not depend on the performance improvement of the DB server itself, but the DB server is stored in the memory of the AP server. The purpose is to improve the search speed by reproducing the table.

  In order to achieve the above object, a data processing system according to claim 1 is a data processing system including a DB server and an AP server, and the DB server includes a database management system and a database storing tables. Provided, the AP server issues a SQL statement to the memory and the DB server, and instructs to read the data stored in the table; and the data transmitted from the DB server is converted into tree-structured data. Data compression means for converting and storing in the memory, and means for executing search processing on the tree-structured data are provided.

  The data processing system according to claim 2 is the system according to claim 1, wherein the data compression unit further includes at least the highest data item in each record transmitted from the DB server. A first process that leaves data of one record when values overlap with each other, deletes data of other records, and associates subordinate data that was subordinate to the deleted data with the remaining data; The remaining top-level data items are combined into one root element, and a second process for generating tree-structured data is executed.

  The data processing system according to claim 3 is the system according to claim 2, wherein the data reading unit specifies one or a plurality of data items in the SQL statement, so that the table is sent from the DB server. Is divided into groups, and the data compression unit executes the first process in units of groups.

3. The data processing system according to claim 1, wherein the table stored in the database of the DB server is compressed as tree-structured data on the memory of the AP server that can be accessed at a higher speed than the disk. Since the AP server does not access the DB server, and therefore does not generate disk I / O, high-speed retrieval of necessary data is possible, and the processing speed is dramatically improved. realizable.
In addition, duplicate data items are removed in the process of converting tabular data into tree-structured data, and the total amount of data is compressed. As a result, the AP server memory with a relatively small capacity is required efficiently. Data can be accommodated.

  In the case of the data processing system according to claim 3, when the AP server retrieves the record from the DB server, the entire table is not received at once, but is divided into groups by specifying values of specific data items. Since the record for the next group is received when compression processing is completed for the group, even a relatively large table can be stored in the AP server memory. .

FIG. 1 is an overall configuration diagram of a data processing system 10 according to the present invention, and this system 10 includes a plurality of AP servers 12, a DB server 14, and a load balancer (load balancer) 16.
The load balancer 16 and each AP server 12, and each AP server 12 and DB server 14 are connected by a network.
A large number of client terminals 20 are connected to each AP server 12 via a network such as an intranet 18 or the Internet and a load balancer 16.

Each AP server 12 includes a data processing unit 22, a data compression unit 24, and a memory 26.
The hard disk (not shown) of each AP server 12 is set up with an OS and an application program dedicated to this system, and the CPU of the AP server 12 operates according to these programs. The compression unit 24 is realized.

The DB server 14 includes a database management system (DBMS) 28 and a database 30 in which various tables for business processing are stored.
The database management system 28 manages the database 30, and performs input / output and update of data stored in the database 30, and predetermined calculations.

FIG. 2 shows an example of a table stored in the database 30, and this table includes data items of date, store Cd (store code), and product Cd (product code).
There is a conceptual hierarchical structure between these data items, the date corresponds to the top item, the store Cd corresponds to the middle item, and the product Cd corresponds to the lowest item.

  The load balancer 16 serves to distribute the request transmitted from the client terminal 20 according to the load applied to each AP server 12.

  The client terminal 20 includes a computer such as a PC, and a Web browser program and a dedicated application program are set up in addition to the OS.

Hereinafter, the processing procedure in the system 10 will be described with reference to the flowcharts of FIGS. 3 and 4.
First, when the data processing unit 22 of the AP server 12 is activated (S10), an SQL statement is issued to the DB server 14 to request data extraction (S12).
At this time, for example, the data processing unit 22 arranges the records stored in the table of FIG. 2 in ascending order based on the date, the store code, and the product code in units of groups specified by date × store code. The SQL statement instructing to be transmitted is generated and transmitted to the DB server 14.

When the corresponding record is transmitted from the database management system 28 of the DB server 14 in groups of date × store code, the data processing unit 22 passes this to the data compression unit 24 (S14).
The data compression unit 24 checks whether there is a duplicate value in the date and store code of each record, and if there is a duplicate, leaves one date and one store code, and removes the other data Thereafter, each data is stored in the memory 26 (S16).

  When data compression (deletion of duplicate data) and storage in the memory 26 are completed for one group, the data processing unit 22 sends an SQL statement that instructs the extraction of records belonging to the next group (date × store code) to the DB server. 14 (S18, S12), the data compression unit 24 compresses the data and stores it in the memory 26 (S14, S16).

The data processing unit 22 and the data compression unit 24 of the AP server 12 repeat steps S12 to S16 until processing is completed for all groups on the target table (S18).
FIG. 5 shows an image of data stored in the memory 26, and (a) corresponds to the data of the group relating to the store code: 600 stores on March 15, 2006. As shown in the figure, the data of the date (20060315) and the store code (600) are left only for the first record and are deleted from the other records. At this time, the data of the product code related to the record from which the date and the store code are deleted is associated with the date and the store code left as an array together with the product code of the top record.
Further, (g) corresponds to the group data relating to the store code: 601 stores on March 17, 2006.

Next, the data compressing unit 24 aggregates the same dates in each group under one date, and combines them under one root (the duplicate dates at this time are Deleted).
As a result, as shown in FIG. 6, the records stored in the database 30 of the DB server 14 are reproduced as tree-structured data on the memory 26 of the AP server 12 (S20).
In the table of FIG. 2, date and store code data are provided for each record. However, in the case of the tree-structured data shown in FIG. 5, the date that is the upper data item has no duplication. The store codes, which are the lower data items, are also aggregated in such a way that there is no duplication within the group of the same date, so that the data capacity is greatly reduced.

  Here, when the AP server 12 receives a search request from the client terminal 20 via the load balancer 16 (S22 in FIG. 4), the data processing unit 22 searches the tree structure data formed on the memory 26 with a search condition. (S24), and the result is sent to the client terminal 22 (S26).

  In this data processing system 10, the table stored in the database 30 of the DB server 14 is compressed as tree-structured data on the memory 26 of the AP server 12 that can be accessed at a higher speed than the disk. Since the data processing unit 22 of the AP server does not access the DB server 14 and thus does not generate disk I / O, the necessary data can be retrieved at high speed.

  Further, in the process of converting tabular data into tree-structured data, duplicate data item data is removed and the entire data amount is compressed. As a result, the memory 26 of the AP server 12 having a relatively small capacity (generally 2 GB) However, the necessary data can be efficiently accommodated.

  Furthermore, when retrieving records from the DB server 14, the entire table is not received at once, but by specifying the values of the date and store code data items, the records are received divided into groups, and compression processing is performed for the group. Since a method of receiving a record relating to the next group at the time of completion is adopted, even a relatively large table can be stored in the memory 26 of the AP server 12.

1 is an overall configuration diagram of a data processing system according to the present invention. It is a figure which shows an example of the table stored in the database of DB server. It is a flowchart which shows the process sequence in this system. It is a flowchart which shows the process sequence in this system. It is an image figure which shows the group data stored in memory. It is an image figure which shows the data structure of the tree structure formed on memory.

Explanation of symbols

10 Data processing system
12 AP server
14 DB server
16 Load balancer
18 Intranet
20 Client terminal
22 Data processing section
24 Data compression section
26 memory
28 Database management system
30 database

Claims (3)

A data processing system comprising a DB server and an AP server,
The DB server includes a database management system and a database storing tables.
The AP server issues an SQL statement to the memory and the DB server, and reads data stored in the table, and converts the data transmitted from the DB server into tree structure data. A data processing system comprising: data compression means for storing in the memory; and means for executing search processing on the tree-structured data. The above data compression means leaves data of one record when the values overlap each other at least for the highest data item in each record transmitted from the DB server. A first process that associates the subordinate data that was dependent on the deleted data with the remaining data;
2. The data processing system according to claim 1, wherein the second highest level data item is generated by combining the remaining top-level data items with one root element and generating tree-structured data. The data reading means executes a process of reading the table divided into groups from the DB server by specifying one or more data item values in the SQL statement,
The data processing system according to claim 2, wherein the data compression unit executes the first process in units of groups.

Images