WO2016092604A1 - Data processing system and data access method - Google Patents

Abstract

It takes time to combine data being managed separately in a plurality of database management systems and to perform processing such as analysis on the data. In the present invention, when a query is to be executed in an external database, a management server and an external DB server work in cooperation with each other and modify a query to be transmitted to the external DB server, on the basis of the computer resources of the external DB server, so that the modified query can be executed at high speed. Further, the amount of resources to be allocated for execution of a query is determined on the basis of the importance of the user who executes the query and the importance of the database to be used, and the query is executed in accordance with the levels of importance.

Description

Data processing system and data access method

Information communication systems, especially the use of database management systems in information systems.

In order to conduct business and management, companies have built multiple databases for different purposes, such as transaction processing systems for managing production and purchasing, and data warehouses that store data generated by corporate activities. In recent years, there is a movement to analyze a large amount of data accumulated every day, find new value, and utilize it for management. In order to realize this, it is necessary to execute an analysis query by combining data stored in a plurality of databases.

As a means for processing data by combining data of a plurality of database management systems (hereinafter referred to as DBMS), in Patent Document 1, a user registers an external function so that a query to another DBMS is sent to another DBMS. Send and get the execution result.

Special table 2007-513426

When performing analysis processing by combining external data managed separately in addition to the data stored in the analysis server, the DBMS that manages the external data is optimally configured for the queries required by the analysis server. There is a problem that it takes time to execute the query. In particular, when a plurality of DBMSs to be combined operate on the same physical computer, there is no method for processing a query by changing to an optimal configuration based on the processing contents of the query and the analysis process being executed.

The query processing provided by the present invention has a function of changing a configuration in which a part of the processing is executed by an external DBMS to a configuration in which the management server and the external DB server can cooperate to execute the query at high speed. The configuration change function for each query also realizes control suitable for each query by changing the configuration based on the priority according to the analysis server or the user.
When a query referring to data of an external DBMS is detected in the DBMS of the analysis server, the external query processing unit of the DBMS transmits the query, the execution plan of the query, and additional information to the management server that manages the external DB server, Request the DBMS to be configured optimally for query execution.
Based on the request, the resource adjustment unit of the management server inquires the external DBMS about an execution plan when the external DBMS executes the target query.
The plan information linkage unit of the external DBMS provides the management server with an execution plan of a query executed by the DBMS.
The resource adjustment unit of the management server manages the resources of the external DB server, specifies the resources necessary to execute the query quickly by acquiring and combining the execution plans of the queries executed by the external DB server, Request resource allocation to an external DB server.

In the analysis process that combines the data managed by the external DB server and the analysis server data, the external DB server adjusts the resource allocation based on the contents of the query and executes the query. Can be executed with

1 shows a configuration of a data processing system targeted by the present invention. 7 shows a query processing flow in the embodiment. 7 shows a query processing flow in the embodiment. 7 shows a query processing flow in the embodiment. An example of a query is shown. The example of the execution plan produced | generated as a result of having analyzed the query by the plan production | generation part of the analysis server is shown. The processing flow of the external query process part in an Example is shown. The example of the execution management table in an Example is shown. The example of the analysis DB information which the analysis DB management part of a management server manages in the Example is shown. The example of the user information which the analyst information management part of a management server manages in the Example is shown. The example of the priority correspondence table | surface which the resource adjustment part of a management server manages in the Example is shown. The processing flow in which the resource adjustment unit determines resource allocation from the plan cost information received from the external DB server and transmits the resource allocation information to the external server is shown. The processing flow in which the resource adjustment unit determines resource allocation from the plan cost information received from the external DB server and transmits the resource allocation information to the external server is shown. The resource adjustment unit receives a query execution completion from the analysis server and recalculates the resource allocation. An example of an external DB server resource managed by the resource adjustment unit is shown. The example of the resource allocation table which a resource adjustment part manages is shown. The example of the information of the external DB server which the external DB management part of a management server manages is shown. Another example query is shown. An embodiment in which the function of the management server exists in the external DB server is shown. An embodiment in which an external DB server is composed of a plurality of servers and a shared storage is shown. In the embodiment, the external DB server is composed of a plurality of servers, and the storage of each server stores different external DB data.

This will be described using examples.

FIG. 1 shows the configuration of a data processing system targeted by the present invention. The analysis server 1 (100) is a computer on which the database management system DBMS1 (101) operates. The analysis server 1 (100) has a hardware configuration in which a CPU 151, a memory 153, a disk 155, and a network interface card (NIC 157) used for communication with other computers are connected by a system bus (not shown). In the analysis server 1 (101), the operating system (OS 159) controls hardware and application software.

DBMS1 (101) is data processing software that operates as one of the applications of the OS 159 and has the following functions.

The communication unit 103 includes an external input / output unit 141 used for information communication with the outside of the DBMS. The communication destination includes a query distribution unit 143 and a query collection unit 145 including an analysis application, a DB management console, a management server, and an external DBMS.

The access management unit 105 determines whether a request arriving at the external input / output unit 141 of the DBMS 1 (101) can be processed. The information that is the basis of the determination includes a user name, password, authority information set for the user name, connection source computer name, connection source IP address, and authority information set for the data to be accessed. .

The SQL analysis unit 107 analyzes a query written in SQL received from the client, and confirms the grammar. In addition, the character string query is broken down into meaningful units such as tokens for later processing.

The plan generation unit 109 determines a query execution plan (internal processing order) by using data information such as the execution record and size stored in the DBMS 1 (101). In the present embodiment, description will be made using a cost-based plan.

The external query processing unit 111 is a unique part in the present invention, and operates when requesting a processing result of an external DBMS in processing a query.

The SQL execution unit 113 executes a query by manipulating data based on the query execution plan generated by the plan generation unit 109.

Analysis data 120 indicates data managed by DBMS 1 (101). Here, among various types of data, a sales table 123 that stores the sales of each product for each period, and an index 121 that is data for efficiently operating the sales table are illustrated.

The remote DB data 130 retains the type of data that needs to refer to the external DBMS and the latest data when analyzing the analysis data 120 and the data in the external DBMS in combination. In this embodiment, economic index data (131) possessed by DBMS3 (340), which will be described later, and SNS data (133) possessed by DBMS4 (370) can be referred to.

Although the data processing software is described here as a database management system, software in which a query processing function is added to a data processing base such as Apache (TM) Hadoop (R) may be used.

There may be a plurality of analysis DBMSs that use external DBMSs, such as DBMS 2 (401) of analysis server 2 (400) as well as analysis server 1 (100).

When the analysis DBMS (DBMS1, DBMS2, etc.) uses an external DBMS, the management server 200 instructs the configuration change of the external DBMS in accordance with the query to be processed. Similar to the analysis server 1 (100), the management server 200 is a computer having a CPU 201, a memory 203, a disk 205, and a NIC 207 as hardware and connected by a system bus (not shown). As an application of an operating system (OS) 209 that operates on the management server 200, software that manages information necessary for configuration change operates.

The resource adjustment unit 211 receives a query and a query execution plan from the DBMS1 (101) and DBMS2 (401), and instructs a configuration change from the management unit information described later and the execution plan of the external DBMS.

The external DB management unit 221 manages basic information of an external DBMS that can be used by the analysis server.

The analysis DB management unit 223 manages basic information of an analysis server that uses an external database.

The analyst information management unit 225 manages the correspondence between the user priority in each analysis server and the priority when using an external DBMS.

In this embodiment, the management server 200 is described as an independent server. However, the same processing may be performed by an analysis server or an external DB server (an example in which an operation is performed by an external DB server is described in the second embodiment). ).

The external DB server 300 is a server that operates a DBMS that is used as an auxiliary to the analysis separately from the analysis DBMS. Similar to the analysis server 1 (100) and the management server 200, the basic hardware and system software include a CPU 301, a memory 303, a disk 305, a NIC 307, and an OS 309.

The external DB server 300 stores two types of data as an example. DBMS 3 (340) manages economic indicators 357, and DBMS 4 (370) manages information (387) exchanged by social networking service (SNS). DBMSs are managed by different DBMSs because the optimum management method differs for each type of data. However, in this embodiment, the DBMS 3 (340) and the DBMS 4 (370) have the same configuration for explanation.

The communication units 341 and 371 include reception units 432 and 372 and transmission units 344 and 374, respectively, and communicate with the upper DBMS and the management server 200.

The access management units 343 and 373 have the same role as the access management unit 105 of the DBMS 1 (101).

The SQL analysis units 345 and 375 have the same role as the SQL analysis unit 107.

The plan generation units 347 and 377 generate an execution plan for processing the query.

The plan information cooperation units 349 and 379 are one of the features of the present invention, and perform preprocessing to process the query generated by the external query processing unit 111 of the DBMS 1 (101). Further, regarding preprocessing, the communication units 341 and 371 are used to cooperate with the management server 200 and the resource allocation unit 320.

The query processing units 352 and 382 have the same role as the SQL execution unit 113.

The DB basic information management units 353 and 383 hold management information of the DBMS.

The resource allocation unit 320 changes the resources allocated to the DBMS 3 (340) and the DBMS 4 (370) in cooperation with the function of the OS (309) and the DB basic information management unit (353, 383) based on the instruction of the management server 200. To do.

In this embodiment, the DBMS 3 (340) and the DBMS 4 (370) are operating on the external DB server 300, but each DBMS may be operating on a separate server. Each DBMS may operate as a process of the OS (309) or a process on a virtual machine realized by a virtualization mechanism supported by a recent computer. In addition, each DBMS does not use the emulation necessary for virtual machine execution, but the name space of various resources is different and usage is tuned by the container mechanism provided by the OS (309) and other system software. It may operate as a container.

The analysis application uses an analysis database and an external database via the analysis database. Although not shown, the client 1 (500) has the same hardware and OS as the analysis server 1 (100). The DB management console 1 (503) has a function of managing the DBMS 1 (101) and can set a connection for using an external DBMS. The analysis application 1 (501) is an application program that issues a query for data analysis. There can be a plurality of clients, and FIG. 1 shows a client 2 (510) and a client 3 (520) as an example in addition to the client 1 (500). The client and server, and the server are connected by a network (561, 563).

FIG. 2 shows the flow of processing when the analysis application 1 requests the analysis server 1 for an analysis query that combines the data of the analysis server and the external DB server in the system of FIG.

The external input / output unit 141 of the DBMS 1 (101) receives the query transmitted by the analysis application 1 (501) (step S1001). The access management unit 105 and the SQL analysis unit 107 of the DBMS 1 (101) authenticate and analyze the received query. The access management unit 105 determines whether or not the query is issued from a user whose processing is permitted by the DBMS 1 from the user name and password used for the connection. In order to facilitate the subsequent processing, the SQL analysis unit 107 divides the authenticated query in units of meaningful character strings (tokens) and determines whether the query conforms to the SQL grammar. When the access authority is set for the table referred to by the query, the access management unit 105 checks whether the user who has requested the query has the access authority to the table to be operated.

The plan generation unit 109 generates a function used when executing the query, the order of execution, and the processing cost as an execution plan (step S1003). Although not shown in FIG. 1, the statistical information accumulated in the processing so far is used for generating the execution plan. Note that at the time of step S1003, the process using the external DB server does not need to know the detailed cost value.

The external query processing unit 111 of the DBMS 1 (101) is a part that is one of the features of the present invention, and specifies a part to be processed by the external DB server from the received query (step S1006). The external query processing unit 111 refers to the query and the execution plan of the query, and sets the query that refers to the remote DB data 130 as the processing target of the external DB server. Furthermore, the external query processing unit 111 defines, as a section, a query that can be requested in parallel for a query processed by the external DB server. For example, for the product information stored in the sales table of the analysis server, the query shown in FIG. 5 for examining the relationship between the sales in the top 10 districts, the economic status of the relevant district, and the number of appearances in the SNS is shown in FIG. It becomes an execution plan. In FIG. 6, functions used such as scan, filter, and combination to be used for executing a query, and an estimate for executing the function are illustrated. Further, a section is defined in which a query is requested in parallel to the DBMS 3 (340) that manages the economic index and the DBMS 4 (370) that manages the SNS.

The external query processing unit 111 of the DBMS 1 (101) executes query conversion on the part processed by the external DB server specified in step S1006 among the queries received in step S1001, and the converted query and corresponding execution are executed. The plan and cost information (plan cost information) are transmitted to the management server 200 (step S1008). Details of the query conversion are shown in FIG. The external query processing unit 111 uses the function of the query distribution unit 143 to transmit the converted query and plan cost information to be executed by the external DB server to the management server 200.

The resource adjustment unit 211 of the management server 200 receives the new query and plan cost information of the analysis server transmitted in step S1008. The resource adjustment unit 211 analyzes the received query, and requests detailed plan cost information for executing the query in the external DB server for each external DB server (step S1022).

The DBMS receiving units (342, 372) operating on the external DB server 300 receive the plan cost information presentation request from the management server 200. The access management unit (343, 373) confirms that the reception is a request from the management server 200 (step S1032).

The DBMS plan information linkage unit (349, 379) operating on the external DB server 300 is one of the features of the present invention. Based on the request from the management server, the execution plan can be generated without executing the query. And has a function of notifying the resource adjustment unit of plan cost information.

The plan information cooperation unit (349, 379) takes out the query included in the plan cost information presentation request, and requests the SQL analysis unit (345, 375) for analysis. The SQL analysis unit (345, 375) analyzes the character string constituting the query, and returns the result to the plan information cooperation unit (349, 379). The plan information cooperation unit (349, 379) passes the analysis result to the plan generation unit (347, 377), and requests the execution plan generation of the query. The plan generation unit (347, 377) calculates an execution plan and a cost estimate for executing the query (in the case of the query in FIG. 5, the inside of the dotted line in FIG. 6 corresponds, except for the underlined portion).

The plan information cooperation unit (349, 379) calculates, from the calculated cost, characteristic information such as whether a CPU or I / O is important, necessary resources, parallel execution degree, etc. when executing a query. . The characteristic information includes the degree of using the CPU calculated from the cost, the degree of emphasizing I / O, the possible parallelism within the DBMS, and the output format of the result. In the case of the query in FIG. 5, the underlined information in FIG. 6 corresponds.

For example, when a search for a large index is included in the execution of a query, the degree of emphasis on I / O increases because random disk I / O is involved. Also, the degree of parallel execution is calculated from the data storage method and the query processing method. When a sort process and a filter process are included in a part of a query, the degree of parallelism decreases because the process is unified even if there are abundant calculation resources. Conversely, if the data is partitioned, the degree of parallelism can be increased. As a result output format, there are a case where the result can be returned every time data matching the condition is found, and a case where the result is collectively output for aggregation.

The plan information cooperation unit (349, 379) requests the transmission unit (344, 374) to transmit the plan cost information including the characteristic information. The transmission units (344, 374) transmit the requested plan cost information to the management server 200 (step S1045).

The resource adjustment unit 211 of the management server 200 receives the processing result from the external DB server (step S1051).

FIG. 3 shows a continuation of FIG. 2 and subsequent diagrams of the query processing flow in the embodiment.

The resource adjustment unit 211 includes plan cost information received from the external DB server, external DBMS information managed by the external DB management unit 221, analysis server information managed by the analysis DB management unit 223, and analyst information management unit 225. Based on the user information to be managed, the configuration of the external DB server for processing the query requested by the analysis server is calculated (step S1053). Details will be described later.

The resource adjustment unit 211 requests the resource allocation unit 320 of the external DB server 300 to change the configuration as resource allocation information (step S1055).

The resource allocation unit 320 of the external DB server 300 receives the resource allocation information transmitted in step S1055, and changes the configuration of resources allocated to the DBMS based on the resource allocation information. After changing the assignment, the resource assignment unit 320 transmits assignment completion to the management server 200 (step S1062).

The resource adjustment unit 211 of the management server 200 receives the assignment completion transmitted in step S1065 (step S1071).

The resource adjustment unit 211 transmits the query processing completion of the external DBMS as a reply to the converted query and the plan cost information received in step S1008 to the DBMS1 (101) of the analysis server 1 (100) (step S1073). At this time, hint information for correcting the execution plan of the query may be transmitted at the same time.

The external query processing unit 111 of the DBMS 1 (101) receives the query processing preparation completion transmitted in step S1073 using the function of the query collection unit 145 (step S1081). If there is hint information for correcting the execution plan, the process may return to step S1003 and the execution plan may be corrected again.

The SQL execution unit 113 of the DBMS 1 (101) starts query processing according to the execution plan (step S1083).

FIG. 4 shows a continuation of FIG. 3 and subsequent drawings of the query processing flow in the embodiment.

When the SQL execution unit 113 refers to external DBMS information, the SQL execution unit 113 transmits a query to the external DB server via the external query processing unit 111. The external query processing unit 111 transmits the converted query to the external DB server using the function of the query distribution unit 143 (step S1085).

The DBMS (340, 370) operating on the external DB server 300 receives and executes the query transmitted from the DBMS1 (101), and transmits the processing result (step S1091). The receiving units (342, 372) receive the query transmitted from the DBMS1 (101). The access management unit (343, 373) of the DBMS operating on the external DB server 300 confirms that the received query is communication from a correct server and an authorized user. The SQL analysis unit (345, 375) of the DBMS analyzes a character string constituting the query. The plan generation unit (347, 377) generates, as an execution plan, the order of processing when executing a query and the calculation method to be used. The query processing units (352, 382) execute the operation of data held in the DBMS based on the plan generated by the plan generation unit (347, 377). The execution result of the query is transmitted to the DBMS 1 (101) of the analysis server 1 (100) that issued the query by the transmission unit (344, 374).

The query collection unit 145 of the DBMS 1 (101) receives the execution result of the query requested in step S1085 from the external DB server, and advances the query processing (step S1111).

The external query processing unit (111) of the DBMS1 (101) performs the execution to the management server 200 by using the function of the query distribution unit 143 when the specific section specified in step S1073 ends (including the completion of the query). The completion of the section is notified (step S1119).

The resource adjustment unit 211 of the management server 200 receives the section end notification from the DBMS 1 (101) (step S1131).

The resource adjustment unit 211 of the management server 200 calculates the resource allocation again from the section completion notification, the query and plan cost information received in step S1022, and the resource allocation calculated in step S1053 (step S1133).

The resource adjustment unit 211 of the management server 200 transmits the calculated resource allocation information to the external DB server when the allocation is necessary again (step S1135).

The processing in the external DB server has already been described as step S1062.

The resource adjustment unit 211 of the management server 200 receives the resource allocation change result from the external DB server, and confirms the resource allocation result (step S1071).

When the query execution is completed, the DBMS 1 (101) transmits the query execution result received in step S1001 to the analysis application 1 (501) from the external input / output unit 141 (step S1117).

FIG. 7 shows a processing flow of the external query processing unit 111 of the analysis server 1 (100). The external query processing unit 111 converts (1) a part of the query received from the client to be executed by the external DB server (steps S1006 and S1008), and (2) a reception process of query processing preparation completion transmitted from the management server ( (Step S1081), (3) Query transmission to external DB server (Step S1085), (4) Query execution result reception from external DB server (Step S1111), (5) End of section to management server at end of section (Step S1119) are five main functions.

When the external query processing unit 111 is called, it first determines whether it is a call from the plan generation unit 109 and conversion of a new query is requested (step S2001). When it corresponds, the process after step S2003 is performed (function (1)). As a determination method, there are a method of holding a caller module name in a transaction management data structure not shown in FIG. 1 and the like, and a method of holding a flag indicating whether or not query conversion is completed.

The external query processing unit 111 searches for an execution plan of the query from the execution plans managed by the plan generation unit 109 (step S2003). The search includes a method of making an inquiry to the plan generation unit 109 and having pointer information to the execution plan in the management data structure.

The external query processing unit 111 specifies a portion of the execution plan acquired by the search that requires a query to the external DBMS, determines whether the query can be executed in parallel on a plurality of external DBMSs, A section is set in the executable part (step S2005). The external query processing unit 111 first identifies a part that needs to be inquired of an external DBMS when a table to be operated in the execution plan is managed by the remote DB data 130. For example, as shown in FIG. 6, the economic index table is referred to in the query for analyzing the district where the sales of the product X shown in FIG. 5 are high, the economic index of the district, and the SNS information about the product (A). The part (a) referring to the SNS table is an inquiry to the external DBMS.

Furthermore, the external query processing unit 111 determines a portion that can be executed in parallel in the execution plan for inquiring an external DBMS, sets a unit that can be executed in parallel as a section, and sets an ID for each section. The external query processing unit 111 uses data dependent on the execution plan and information on functions used in order to identify parallel execution locations. The portion where the subsequent processing cannot be performed unless the data is prepared becomes the boundary of the section. In addition, in the combining process, the part for calculating the data to be combined can be executed independently if there is a sufficient computing resource, so that it can be determined that the part can be executed in parallel. In the example of the execution plan in FIG. 6, the parts (a) and (b) are processes that can be calculated after the district ID is calculated, and are based on nested loop joins that are generally processed in parallel in a parallel DBMS. For the data acquisition process, it is determined as the same section.

The external query processing unit 111 gives information such as analysis server information, user information requesting the query, and transaction ID in addition to the section ID to the query (step S2007). The analysis server information includes a server name and an IP address. As a setting method, there are a method of using the comment sentence shown in step S2005 and a method of adding to a part of data to be sent to the management server.

The external query processing unit 111 transmits the query and the plan cost information to the management server 200 using the function of the query distribution unit 143 (step S2009).

If the call is not from the plan generation unit 109 in step S2001, the external query processing unit 111 is a call to the query collection unit 145, and determines whether the message passed is ready for query processing (step S2021). When it corresponds, it progresses to step 2027 (function (2)). If not, the process proceeds to step S2031.

The external query processing unit 111 calls the SQL execution unit 113 and starts executing a query using data of the external DB server (step S2027). Although not shown in this flow, when the query processing preparation completion message includes hint information for plan correction of the external DB, this flow may be terminated and the process may return to step S1008.

The external query processing unit 111 determines from the communication contents whether the called trigger is a call from the query collection unit 145 and reception of a query processing result (step S2031). If applicable, the process proceeds to step S2033 (function (4)). Otherwise, the process proceeds to step S2041.

If the call source is neither the plan generation unit 109 nor the communication unit 103, the external query processing unit 111 determines whether it is an execution request to the external DB server by a call from the SQL execution unit 113 (step S2041). When it corresponds, it progresses to step S2043 (function (3)). If not, step S2045 is executed (function (5)).

The external query processing unit 111 uses the query distribution unit 143 to send the corresponding part of the query converted in step S2007 to the target external DB server in order to acquire the data of the external DBMS required by the SQL execution unit 113. (Step S2043).

If the external query processing unit 111 is a call from the SQL execution unit 113 and the cause of the call is not a query execution to the external DB server, the external query processing unit 111 determines whether the query currently being executed has been completed or whether all the queries have been completed. Confirmation is made (step S2045). The external query processing unit 111 uses the function of the external input / output unit 141 to transmit information about the external DBMS and the completed section to the management server 200.

The resource adjustment unit 211 of the management server 200 executes the following four processes broadly: (1) Request plan cost information from the external DB server, (2) Assign resources from the plan cost information received from the external DB server And send resource allocation information to the external DB server, (3) receive resource allocation completion from the external DB server, send query preprocessing completion to the analysis server, and (4) receive query execution completion from the analysis server Then, the resource allocation is recalculated and the resource allocation of the external DB server is changed as in (2).

The process (1) of the resource adjustment unit 211 corresponds to step S1022 shown in FIG. When the resource adjustment unit 211 receives the query and the plan cost information from the DBMS 1 (101) of the analysis server 1 (100), the resource adjustment unit 211 registers the query information in the execution management table 1100 illustrated in FIG. The execution management table 1100 is a data structure that holds information about a query being executed while the analysis server requests a query from an external DBMS. The execution management table 1100 holds the identifier (1101) of the analysis DB server, the transaction ID (1103) assigned by the analysis DB server, the total number of sections of the requested query (1105), and the section ID (1107) currently being executed. To do. Further, as external DB server information, the external DBMS (1109) being used, the status of the query (1111), the number of messages waiting for a response (1113), and the ratio of target resources to be set in the external DBMS (1115) It has information on the setting resource (1117) actually set, and a pointer (1118) to a resource allocation table allocated for each external DBMS. Further, a list (1119) of a section for changing the resource allocation, information (1121) of the address where the requested query is recorded, and data (1123) for referring to the plan cost information acquired from the analysis server and the external DB server ). In FIG. 8, information (transaction ID 1) of a query requested by DBMS 1 is registered as 1151.

The query received by the resource adjustment unit 211 from the analysis server includes the analysis DBMS information, user information, external DBMS information, and the like given by the external query processing unit 111 of the DBMS 1 (101) in step S2007 in addition to the SQL sentence. Embedded.

In the plan cost information received by the resource adjustment unit 211 from the analysis server, the section information calculated by the query processing unit 111 in step S2005 is added to the cost-based execution plan used in a general DBMS. Note that the query plan executed by the external DBMS at this time is not necessarily accurate. For example, in the query of FIG. 5, the cost of (a) (b) in the plan of FIG.

The resource adjustment unit 211 inquires the external DB server for detailed plan cost information of a query executed by the external DB server. Specifically, a query for which a section is set is extracted for each external DB server from the query character string. The resource adjustment unit 211 manages information for connecting to an external DBMS and information on a server on which an external DB server operates, using external database information 2100 shown in FIG. The resource adjustment unit 211 specifies the host name, IP address, and port number for inquiring plan cost information by combining the extracted query and the external database information 2100. The resource adjustment unit 211 transmits a plan cost information inquiry message to the specified destination for the external DBMS. The resource adjustment unit 221 changes the status 1111 of the execution management table 1100 during the plan inquiry.

The process (2) of the resource adjustment unit 211 receives the plan cost information inquired from the external DBMS and determines the resource allocation. First, the resource adjustment unit 211 receives the above-described inquiry result of the plan cost information from the external DBMS for each section. The resource procurement unit 211 receives from the external DB server the plan cost information of the query that the analysis server requests to the external DBMS. The resource adjustment unit 211 decreases the response waiting 1113 that holds the number of messages waiting for a response to the query in the execution management table 1100 by one. The resource adjustment unit 211 determines whether the response waiting 1113 of the query is 0. If 0, message reception is complete. If it is not 0, the resource adjustment unit 211 waits for the arrival of the plan cost information because all the plan cost information from the external DBMS has not arrived.

FIG. 12 shows a processing flow when determining the resource allocation of the external DB server. This flow is a detailed version of step S1053.

The resource adjustment unit 211 first searches the analysis DB information 1200 shown in FIG. 9 and acquires the priority of the DBMS that requested the query (step S4201). The analysis DB information 1200 stores the DBMS name, IP address, priority, and the like of the analysis server that requests the query. The analysis DB information 1200 is data managed by the external DB management unit 221 and is updated by a system administrator or a system management program (not shown in FIG. 1) when an analysis server is added to or deleted from the information system.

The resource adjustment unit 211 extracts user information included in the query 1121 of the execution management table 1100. The resource adjustment unit 211 searches the user information 1300 shown in FIG. 10, and acquires the priority when using the external DB server from the analysis DBMS name and user name that requested the query (step S4203). The user information 1300 in FIG. 10 is data managed by the analyst information management unit 225, and is updated by the system administrator or the system management program in accordance with the analysis server change and the user change.

The resource adjustment unit 211 refers to the priority correspondence table 1400 in FIG. 11 and acquires the target priority corresponding to the DBMS priority obtained in step S4201 and the user priority obtained in step S4203 (step S4203). S4207). The priority correspondence table 1400 is a correspondence table between query priorities and target allocation resource amounts managed in advance by the resource adjustment unit 211. The resource adjustment unit 211 assigns the acquired target allocation resource amount to the target allocation resource 1115 of the execution management table 1100 as the target allocation resource amount of the query.

The resource adjustment unit 211 recalculates the section and calculates the number of the section whose allocation is changed from the plan cost information acquired from each external DBMS (step S4209). When an external DBMS generates a plan for a query requested by the analysis server, the detailed cost of the plan and the parts that can be executed in parallel are known. The resource adjustment unit 211 reconfigures information related to a specific section of the DBMS based on the plan cost information. In the execution plan, the resource adjustment unit 211 sets a temporary internal section at a place where the processing time exceeds a certain time or a place where the degree of parallelism changes. For example, in the part (a) of FIG. 6, if the table scan takes a lot of execution time, the table scan part is changed to “1-DBMS4-internal section 1”, “1-DBMS4-internal section 2”, etc. To divide. Further, a portion where processing takes a certain amount of time and a result is returned to the DBMS that requested the query is defined as an allocation change section.

The resource adjustment unit 211 calculates a basic allocation amount to be allocated until the first allocation change section is reached from the received plan cost information and the target resource calculated in step S4207 (step S4211). The basic allocation amount corresponds to the amount of resources to be allocated when no change is necessary in later determination processing, such as when the resources of the external DB server can be occupied in the execution of a query. The plan cost information includes the characteristic information in the query execution calculated in step S1034. The resource adjustment unit 211 refers to the characteristic information of the section to be allocated, and determines that the CPU resource for the query is increased when the CPU utilization characteristic of the execution plan is high. Also, the CPU resource allocation is increased even when the degree of parallel execution is high. On the other hand, if the I / O load is high due to an index operation or the like, I / O resources are increased. If the size of the operation target table exceeds a predetermined criterion, it is determined to increase the memory resource. As the reference value, a value determined for the entire system or a value set for each external DBMS may be used.

The resource adjustment unit 211 checks whether or not the resource allocation target range is on a single external DB server within the execution interval of the query to be processed (step S4213). The resource adjustment unit 211 confirms whether or not the query is executed by a plurality of external DBMSs within the interval for calculating the resource allocation in the execution management table 1100 for the interval calculated in step S4209. For this purpose, referring to the external database information 2100 in FIG. 17, the physical server of the external DBMS to be executed is confirmed. When a query is executed on a plurality of external DBMSs in the target section, and these external DBMSs are actually operating on one external DB server, the process proceeds to step S4215. Otherwise, the process proceeds to step S4233. When the section is reconstructed in step S4209 when proceeding to step S4233, the section number of the entire query is updated (in the above example, “1-DBMS4-internal section 1” is changed to “section 1”, “ 1-DBMS4-internal section 2 "is set to" section 2 ").

The resource adjustment unit 211 recalculates sections and allocation change sections with respect to DBMS queries operating on the same server (step S4215). First, the resource adjustment unit 211 recalculates the internal section calculated in step 4209 in consideration of a query that operates in parallel. Among plans executed in the section set by the analysis server, a plan that takes a certain time (eg, 60 seconds) for execution time estimation and returns a result to the analysis server is defined as a new section. The resource adjustment unit 211 adds a section number to the review section 1119 of the execution management table as a section for changing the allocation for a section exceeding a certain time. Further, when the resource adjustment unit 211 executes certain sections N and N + 1, (a) one consumes a large amount of resource x in the section N, the other consumes a small amount of resource y in the section N + 1, or (b) one of them Allocation is also changed for a section where the resources used are greatly different at the change of section, such as a small consumption of resource x in section N and a large consumption of resource y in section N + 1 on the other side. It should be noted that the external query processing unit 111 of the analysis server 100 can use means for designating a section in which resource allocation is changed.

The resource adjustment unit 211 refers to the plan cost information of each external DBMS in the section, and confirms the execution result output method and execution time (step S4217). For example, in a query composed of four sections that refer to DBMS3 and DBMS4, when section 1 and section 2 are executed with the same resource allocation in step S4215, the sum of the estimated execution times of section 3 and section 2 of DBMS3 is When the difference is larger (for example, three times) than the sum of the estimated execution times of the section 1 and the section 2 of the DBMS 4, it is determined to be true. Moreover, in the same example, if there is a section in which the results are returned little by little in the sections 1 and 2 of the DBMS 3 and the DBMS 4, the result is true. If the query has a relevant property, the process advances to step S4219. Otherwise, the process proceeds to step S4221. FIG. 13 shows a processing flow in which the resource adjustment unit 211 determines resource allocation following FIG.

The resource adjustment unit 211 assigns resources preferentially to queries that return results little by little in step S4217 and queries that have a long execution time (step S4219). For example, in a query with a short execution time and a query that returns execution results collectively at the end, half of the calculation result of the basic allocation amount calculated in S4211 is allocated to a resource amount of a query with a long execution time and a query that returns results little by little. . When the results are returned little by little, the analysis server that has requested the query can start the subsequent processing based on the results by allocating resources preferentially.

The resource adjustment unit 211 compares the CPU execution cost and the I / O load cost ratio among the plan cost information of queries executed by a plurality of external DBMSs in the same section (step S4221). If there is a bias exceeding the ratio determined by the system, the process proceeds to step S4223. Otherwise, the process proceeds to step S4225.

The resource adjustment unit 211 distributes the CPU resource and the I / O resource according to the difference in the ratio when the query executed at the same time has different load characteristics between the CPU and the I / O (step S4223). For example, when the CPU usage cost of the DBMS 3 is 90, the I / O cost is 10, the CPU usage cost of the DBMS 4 is 10, and the I / O cost is 90 in a certain section, the DBMS 3 and the DBMS 4 The ratio of each usage cost is used as the resource allocation amount.

The resource adjustment unit 211 determines whether the parallel execution degree of the query executed in the section is high (step S4225). The reference value used for the determination here can be (a) a reference value set in advance or (b) the number of cores assigned to the DBMS executing the query. If it is determined that the degree of parallelism is high, the process proceeds to step S4227. Otherwise, the process proceeds to step S4229.

If the parallel execution degree of the query is high, the resource adjustment unit 211 assigns the CPU (core) of the server on which the DBMS operates to the DBMS that executes the query in consideration of the priority (step S4227). The resource adjustment unit 211 holds a resource management table 2000 shown in FIG. 15 for each external DB server to be managed. In the resource management table 2000, the amount of resources, the amount of unallocated remaining resources, and the allocated resource amount for each DBMS operating on the server are managed for each type of resource mounted on the server. The resource adjustment unit 211 allocates a core to a DBMS of a query with a high degree of parallel execution within the allocation amount determined in step S4209.

The resource adjustment unit 211 determines whether or not the work memory required from the cost is larger than a certain amount within the section to be executed with the same resource allocation (step S4229). If applicable, the process proceeds to step S4231. Otherwise, the process proceeds to step S4233.

When the size of the work memory necessary for executing the query is large, the resource adjustment unit 211 uses the allocation amount calculated in step S4207, the resource management table 2000 of the external DB server on which the query is executed, and the ratio calculated in step S4229. The memory capacity to be added to each DBMS is determined (step S4231).

The resource adjustment unit 211 sets the resource allocation determined in the previous steps in the resource allocation table 1600 shown in FIG. 16 (step 4233). The resource allocation table 1600 is created for each transaction transmitted from the analysis server (101) to the external DB server for the number of external DBMSs to be used. In the resource allocation table 1600, the resource adjustment unit 211 determines the allocation, and records the target resource amount and the actually set resource amount for the elements of the resource to be allocated and executed by the resource allocation unit 320. The resource adjustment unit 211 sets a target allocated resource amount based on the resource allocation obtained by calculation and the resource amount in the resource management table 2000. The amount set as the target allocation resource in the resource allocation table 1600 is set to an ideal resource amount when the analysis server that requested the query can occupy the external DB server. As the setting resource of the resource allocation table 1600, the same amount as the target allocation resource is registered as an initial value. For example, when another query is already operating and the query is executed in parallel, the adjusted resource amount is registered by calculation in a subsequent step such as step S4235.

The resource adjustment unit 211 compares the resource amount set in step S4233 with the remaining resource amount in the resource management table 2000 of FIG. 15 to determine whether allocation is possible (step S4234). If so, complete the process. Otherwise, step S4235 is executed.

When resource allocation cannot be performed, the resource adjustment unit 211 refers to the setting resource 1117 and the priority correspondence table 1400 of the execution management table 1100 and searches for an allocation amount having a lower priority than the currently set resource 1117. For the type of resource to be allocated, the allocated resource amount at the searched allocation amount is calculated (step S4235).

When the resource allocation calculation is completed, the resource adjustment unit 211 sends a resource allocation request to the external DB server used for executing the query. The flow of this process is as follows. The resource adjustment unit 211 subtracts the determined resource allocation and the value of the set resource of the resource allocation table 1600 from the remaining resource amount of the resource management table 2000, and adds it to the resource amount of each DBMS. The resource adjustment unit 211 determines whether to change the resources of a plurality of external DBMSs. When changing the resources of a plurality of external DBMSs, the resource adjustment unit 211 refers to the external database information 2100 in FIG. 17 and confirms whether the databases to be changed are the same physical server. The external database information 2100 holds information on an address, a port number, a user name and password, a data size, and the number of analysis databases that may be connected for each external DBMS. When a plurality of external DBMSs are operating on one physical server, a resource allocation request for the plurality of DBMSs is transmitted as one allocation request. When there is one DBMS to which resources are to be allocated, or when a plurality of DBMSs are operating on different physical servers, a resource allocation request is transmitted to the corresponding external DB server as each resource allocation request. .

The process (3) of the resource adjustment unit 211 receives the resource allocation completion from the external DB server, and transmits the completion of the query pre-processing to the analysis server.

The resource adjustment unit 211 receives a resource allocation completion message from the resource allocation unit 320 of the external DB server 300. The resource adjustment unit 211 decreases the number of response waits 1113 related to the query execution in the execution management table 1100 by one.

The resource adjustment unit 211 determines whether the value of the response wait number 1113 in the query execution is 0. When it is not 0, it is a case where a query is executed in parallel by a plurality of external DB servers in the section, and a completion notification of a resource adjustment request to another external DB server is waited for.

When the response wait number 1113 is 0, the resource adjustment unit 211 notifies the DBMS1 (101) of the analysis server 100 that the query processing preparation is complete. The resource adjustment unit 211 updates the status 1111 of the execution management table 1100 during execution. Next, the resource adjustment unit 211 transmits query processing preparation completion to the DBMS 1 (101) of the analysis server 100 that uses the external DB server. When transmitting the query processing preparation completion, the resource adjustment unit 211 simultaneously includes the section information recorded in the review section 1119 of the execution management table 1100 when calculating the resource allocation. When the section is reconfigured in steps S4213 and S4215, the resource adjustment unit 211 also transmits the reconfigured section information. Further, the resource adjustment unit 211 sets a response waiting 1113 in the execution management table 1100 in order to receive the section completion message transmitted by the analysis server.

The process (4) of the resource adjustment unit 211 receives the query execution section completion from the analysis server and recalculates the resource allocation. FIG. 14 shows this processing flow. As a premise of this processing flow, the resource adjustment unit 211 receives the section completion notification transmitted by the DBMS 1 (101) of the analysis server 100 in step S1119. When the resource adjustment unit 211 receives the notification, the resource adjustment unit 211 decreases the response waiting number stored in the response waiting 1113 of the execution management table 1100 by 1 and waits until the completion notification of the same section is completed. The resource adjustment unit 211 updates the section ID (1107) of the execution management table 1100 and starts the flow of FIG.

The resource adjustment unit 211 checks whether or not the completed section is the last section (step S4407). For this purpose, the resource adjustment unit 211 compares the number of all sections 1105 in the execution management table 1100 with the section ID (1107) updated in step S4405. If the total number of sections 1105 matches the section ID (1107), it means that the last section has been executed, and the process proceeds to step S4409. If they do not coincide with each other, it means that the resource adjustment unit 211 has reached the section for performing the reallocation, calculated at the time of resource allocation, and proceeds to step S4411.

The resource adjustment unit 211 refers to the execution management table 1100, the priority correspondence table 1400, and the resource management table 2000, and calculates the amount of resources added to execute the query (step S4409). Since the query processing is completed, the set resource of the resource allocation table 1600 linked to 1118 of the execution management table 1100 is released. Further, the resource adjustment unit 211 searches for queries in which the setting resource 1117 does not reach the target resource 1115 from other queries registered in the execution management table 1100 and executed in parallel, and finds the insufficient resources. It may be assigned.

The resource adjustment unit 211 recalculates the resource allocation for executing the next section from the information stored in the execution management table 1100, the query traced from the execution management table, and the plan cost information for the query (step S4411). ).

The resource adjustment unit 211 requests the external DB server to allocate the resources determined in steps S4409 and S4411 (step S4413). As the detailed execution contents, the same method as the method for transmitting the resource allocation request to the external DB server when the resource allocation is completed can be used. When the transaction is the last section, the transaction information of the query is deleted from the execution management table 1100.

When the resource allocation unit 320 of the external DB server 300 receives a resource allocation request from the resource adjustment unit 211 of the management server 200, it cooperates with the DBMS 3 (340) and DBMS 4 (370) and functions of the operating system (OS) (309). Add and delete resources using. For example, when the DBMS is operating as an application process of the OS and the CPU resource is changed, the number of cores used is increased or decreased using an affinity function that designates a core to be executed. Further, the resource allocation unit 320 may change not only the number of cores but also the process execution priority. When changing resources related to external storage I / O, the resource allocation unit 320 adjusts the bandwidth of the I / O device used by the target DBMS using the management function of the external storage. When allocating memory resources, the resource allocation unit 320 confirms that the memory managed by the OS (309) has a free space, and notifies the DB basic information management units 353 and 383) of the memory allocation. The DB basic information management unit (353, 383) secures a working memory based on the notified information. When the memory resource is reduced, the resource allocation unit 320 notifies the DB basic information management unit (353, 383) of the release request. Upon receiving the notification, the DB basic information management unit (353, 383) releases the working memory.

When the DBMS is operating on the container mechanism provided by the OS, the resource allocation unit 320 uses the management function of the container to increase or decrease the CPU utilization rate allocated to the container as a change in CPU resources, or Change the allocation by increasing or decreasing the number of cores to be executed. When changing the disk I / O resource, the resource allocation unit 320 changes the reserved bandwidth for the I / O device used by the container. The same bandwidth change is performed for the network. When increasing the memory resource, the resource allocation unit 320 adds the memory allocated to the container and notifies the DB basic information management unit (353, 383) of the DBMS 3, 4 (340, 370). Based on the notified information, the DB basic information management unit (353, 383) secures a working memory for placing an intermediate table, which is used in executing a query. In order to reduce memory resources, a memory area such as an intermediate table is first released, and the DB basic information management unit (353, 383) notifies the resource allocation unit 320, so that a part allocated to the container This memory is removed from the container usage target.

When the DBMSs 3 and 4 (340 and 370) are applications in the virtual machine using the virtualization mechanism, the resource allocation unit 320 performs resource allocation in cooperation with the virtualization platform and the DBMS operating on the OS (309). Execute. When increasing or decreasing the CPU resource, the resource allocation unit 320 changes the CPU utilization rate or the number of execution cores of the virtual machine on which the target DBMS is operating. Furthermore, the resource allocation unit 320 may change the number of cores and the utilization rate of the virtual CPU allocated to the DBMS by using the function of the OS on the virtual machine. When increasing or decreasing the I / O resource, the resource allocation unit 320 changes the I / O bandwidth of the virtual device used by the virtual machine on which the target DBMS operates. When adding a memory resource, the resource allocation unit 320 uses a virtualization platform function to increase the memory allocated to the virtual machine on which the target DBMS operates. The resource allocation unit 320 communicates with the DBMS on the virtual machine and notifies the DB basic information management unit (353, 383) of the memory addition. If there is sufficient memory in the virtual machine, the memory expansion for the virtual machine may be omitted. When reducing the memory resources, the resource allocation unit 320 communicates with the DBMS on the virtual machine, notifies the DB basic information management unit (353, 383) of the memory release, and releases a part of the memory used by the DBMS. . The resource allocation unit 320 reduces the memory allocation to the virtual machine by using the virtualization infrastructure function.

When the resource allocation is completed, the resource allocation unit 320 notifies the management server 200 of the completion of the resource allocation change.

As described above, the query in FIG. 5 sorts the sales by the analysis server and searches the top 10 cases for the economic index and SNS. As shown in the execution plan of FIG. 6, the search to the external DB server cannot be performed until the top 10 cases are determined.

FIG. 18 shows an example of a query different from FIG. The query in FIG. 18 calculates the number of individual products taken up in 2013 by sales, economic indicators, and social media for sales in 2013 among the sales data stored in DBMS 1 of the analysis server. To do. Since the economic index can be obtained regardless of the product, the sales table search and the economic index search can be executed simultaneously. Moreover, since the economic index search cost is lower than the SNS table search cost, it is more efficient to obtain the result of the economic index first and allocate resources to the DBMS 4 in which the SNS information is accumulated after the execution is completed. As described above, by performing resource allocation for each query execution, the allocation is optimal for query execution, and the query can be executed at high speed.

FIG. 19 is a different embodiment, and shows an example in which the functions of the management server 200 and the external DB server 300 in FIG. 1 operate on the external DB server 1800 on the same computer (hardware such as a CPU). And details of software modules are omitted). When the external DB server 1800 is used, communication between the resource adjustment unit 211, the external DBMS (340, 370), and the resource allocation unit 320 is not a network interface card (NIC) normally used for communication between computers, but lighter. Use interprocess communication that can be handled. Specifically, the plan cost information request of the query from the resource adjustment unit 211 to the external DBMS, the transmission of the plan cost information from the external DBMS to the resource adjustment unit 211, the resource allocation from the resource adjustment unit 211 to the resource allocation unit 320 A request (including reassignment) and an assignment completion notification from the resource assignment unit 320 to the resource adjustment unit 211 are applicable. Note that data read / write to the shared memory may be used as a method for realizing inter-process communication.

Since the communication overhead is reduced in the resource adjustment unit 211 in the configuration of FIG. 19 compared to the configuration of FIG. 1, the time for determining whether to execute in the same section used in steps S4209 and S4215 is shorter. Can be.

The resource adjustment unit 211 and the external DB management unit 221 can use the resource information managed by the resource management table 2000 and the external database information 2100 by directly querying the OS of the external DB server 1800. By directly using the resource adjustment unit 211 and the external DB management unit 221, the resource allocation calculation can be performed with a more accurate resource amount managed by the OS.

FIG. 20 shows a further different embodiment, in which DBMSs for managing external data operate on different servers, and the data is stored in the shared storage. The external DB management unit 221 uses the storage information area of the external database information 2100 to manage information related to data storage in the DBMS. Thereby, even when the IP address and the physical server name are different, it is possible to detect the case where the storage apparatus is shared. When determining whether the allocation targets are the same in step S4213, the resource adjustment unit 211 executes the flow of step S4215 to allocate I / O resources even if the servers differ depending on the storage performance and the shared state. adjust.

When the resource adjustment unit 211 increases or decreases the resource allocation of the shared storage, the resource allocation unit 1 (2510) and the resource allocation unit 2 (2520) consider the increase or decrease of the cache amount and the increase or decrease of the bandwidth in the shared storage. I do. Further, when the storage apparatus has a plurality of I / O ports and the bandwidth can be changed by changing the number of ports to be used, the setting of the ports may be changed using the management function of the storage apparatus.

In addition, when a plurality of instances can be prepared in each external DB server (for example, the execution binary of DBMS4 (370) is also arranged in the external DB server 1 (2401) in addition to the execution binary of DBMS3 (340)), the resource adjustment unit 211 can also manage the external DB server 1 (2501) to activate the DBMS 4 (370) and execute the query. For example, if the query to be executed with high priority in steps S4201 and S4203 cannot be assigned in step S4233, instead of executing step S4235, the query is executed on another external DB server sharing the storage. Start the DBMS to be executed. As another example, when recalculating the query interval and the allocation change interval in step S4215, the resource adjustment unit 211 refers to the execution management table 1100 and the resource management table 2000, and the server on which one DBMS operates. If the remaining resource amount is less than a preset reference value, a DBMS that executes a query to the other server is started. When starting the DBMS, the resource adjustment unit 211 issues a request to the resource allocation unit of the target server. The resource allocation unit of the server activates the DBMS based on the request. When there are a plurality of external DB servers for the same DBMS, the resource adjustment unit 211 notifies the analysis server as to which server the query should be requested as hint information when the query processing preparation transmitted in step 1073 is completed.

Note that the resource adjustment unit 211 does not start the DBMS when resources are insufficient for query execution of the DBMS 4 (370) in the external DB server 1. In addition, management may be performed so that the query is executed by the DBMS of the external DB server 1.

FIG. 21 shows a part of a configuration of another embodiment in which the DBMS of external data operates on different external DB servers, and the storage is connected to each server. In this example, the storage is not shared, but the storage 2510 and the storage 2540 hold the same data SNS387. The DBMS 4 (370) that executes the query related to the SNS can be executed by both the external DB server 1 (2501) and the external DB server 2 (2531). The external DB management unit 221 records the server in which the data is written as storage information of the external database information 2100 and also records information of another server that provides the same role as the DBMS. For example, when the DBMS 6 is another DBMS that executes an SNS query, the external DB management unit 221 registers the DBMS 6 in the column of another server of the DBMS 4 in the external database information 2100 (in this case, it is illustrated in FIG. 17). (The DBMS6 line is also registered separately).

When there are a plurality of external DB servers that execute the same external data query, the resource adjustment unit 211 requests all candidate external DB servers when requesting the plan cost information to the external DB server in step S1022 To do. In step S4209, the resource adjustment unit 211 selects which DBMS to inquire from the obtained plan cost information. For example, the DBMS having the shortest execution time is selected from the cost information and the available resource information, and the section in which recalculation and allocation of the section is changed is selected by the execution plan of the DBMS. Alternatively, one or a plurality of DBMSs to be preferentially used may be determined for each external data to be processed, and plan cost information may be requested only for the target DBMS.

Similarly to the third embodiment, the resource adjustment unit 211 selects another DBMS that executes the query of the same external data, instead of step S4235, if the allocation cannot be performed in step S4233, and allocates resources. Calculate whether is possible.

101 analysis server 1, 109 plan generation unit, 111 external query processing unit, 113 SQL execution unit, 200 management server, 211 resource adjustment unit, 300 external DB server, 320 resource allocation unit, 347 plan generation unit, 349 plan information linkage unit 377 Plan generation unit, 379 Plan information linkage unit, 1100 execution management table, 1200 analysis DB information, 1300 user information, 1400 priority correspondence table, 2000 resource management table, 2100 external database information

Claims (4)

1. A data processing system comprising a first DBMS, a plurality of second DBMSs, and a management server,
   The first DBMS is
   An external receiver for receiving the first query;
   The data managed by another DBMS is extracted from the first query received by the external receiver, and the extracted data is divided into a plurality of sections that can be processed at the same time. An external query processing unit for generating a second query;
   A query distribution unit that distributes the plurality of generated second queries;
   And a query collection unit that receives the processing result of the distributed second query.
   A receiving unit for receiving the second query transmitted by the query distributing unit;
   A query processing unit for executing the received query;
   A plan information linkage unit that answers the execution plan and cost of the second query;
   The management server includes a transmission unit that transmits the result of the query processing to the first DBMS.
   Queries the execution plans and costs of the distributed multiple second queries, changes the interval based on the received execution plans and costs, and calculates the amount of computer resources to execute the second query based on the changed intervals A resource adjustment unit for notifying the first DBMS of creation and distribution of the third query based on the changed section;
   The second DBMS sends the result of processing the third query based on the distributed computer resource amount to the first DBMS,
   A data processing system, wherein the processing results of the third query sent to the first DBMS are aggregated.
2. The external reception unit accepts queries from a plurality of users, the resource adjustment unit refers to a priority correspondence table in which resource allocation amount is determined based on DBMS priority and user priority,
   A resource allocation amount is determined based on a user who receives the query and a DBMS referred to by the query;
   The data processing system according to claim 1, wherein a third query is created.
3. Data including a first DBMS including an external reception unit, an external query processing unit, a query distribution unit, and a query collection unit, a plurality of second DBMSs including a reception unit, a query processing unit, and a plan information cooperation unit, and a management server A data processing method for a processing system,
   The external receiver included in the first DBMS receives the first query,
   The external query processing unit extracts data managed by another DBMS from the first query received by the external reception unit, and divides the extracted data into a plurality of sections that can be processed simultaneously. , Generate a second query for each segment,
   Distribute multiple second queries generated by the query distributor,
   Receive the processing result of the second query distributed by the query collection department,
   The receiving unit of the second DBMS receives the second query transmitted by the query distributing unit,
   Execute the query received by the query processing unit,
   The plan information cooperation department answers the execution plan and cost of the second query,
   The transmission unit transmits the result processed by the query process to the first DBMS,
   The resource coordinator of the management server inquires about the execution plan and cost of the multiple second queries distributed, changes the interval based on the received execution plan and cost, and executes the second query based on the changed interval The amount of computer resources to be calculated, and notification of the creation and distribution of the third query to the first DBMS based on the changed section,
   The second DBMS sends the result of processing the third query based on the distributed computer resource amount to the first DBMS,
   A data processing method for a data processing system, wherein the processing results of a third query sent to the first DBMS are aggregated.
4. The external reception unit accepts queries from a plurality of users, the resource adjustment unit refers to a priority correspondence table in which resource allocation amount is determined based on DBMS priority and user priority,
   A resource allocation amount is determined based on a user who receives the query and a DBMS referred to by the query;
   The data processing method of the data processing system according to claim 3, wherein a third query is created.

Images