WO2013153620A1 - Système de traitement de données et procédé de traitement de données - Google Patents

Système de traitement de données et procédé de traitement de données Download PDF

Info

Publication number
WO2013153620A1
WO2013153620A1 PCT/JP2012/059789 JP2012059789W WO2013153620A1 WO 2013153620 A1 WO2013153620 A1 WO 2013153620A1 JP 2012059789 W JP2012059789 W JP 2012059789W WO 2013153620 A1 WO2013153620 A1 WO 2013153620A1
Authority
WO
WIPO (PCT)
Prior art keywords
key
column
processing
data
server device
Prior art date
Application number
PCT/JP2012/059789
Other languages
English (en)
Japanese (ja)
Inventor
雅輝 四ツ谷
康郎 國信
敬行 河野
吉田　順
Original Assignee
株式会社日立製作所
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 株式会社日立製作所 filed Critical 株式会社日立製作所
Priority to PCT/JP2012/059789 priority Critical patent/WO2013153620A1/fr
Publication of WO2013153620A1 publication Critical patent/WO2013153620A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements

Definitions

  • the present invention relates to a data processing system and a data processing method, and is suitable for application to a data processing system and a data processing method for parallel and distributed processing of large-scale data.
  • a simple method for improving the processing performance of the arithmetic processing device is to use a high-performance arithmetic processing device equipped with a computing speed and a component with a high processing speed. There is a problem that the cost is higher than that of the apparatus.
  • parallel distributed processing is a method of speeding up processing by a plurality of arithmetic processing devices executing processes in parallel and in parallel.
  • Patent Document 1 discloses a distributed processing framework called MapReduce as one of methods for realizing parallel distributed processing.
  • MapReduce is a data analysis process that extracts data that is subject to parallel distributed processing from input data and generates data consisting of a combination of Key and Value, and Reduce that aggregates the data extracted by Map processing This is a simplified programming model for processing.
  • Map processing After dividing data consisting of a set of key (key) and value is extracted from input data, intermediate data in which the divided data is bundled in units of keys is generated.
  • intermediate data is aggregated by combining values of intermediate data having the same key.
  • the execution engine of MapReduce executes Map processing on a plurality of computers in parallel, and controls the generated intermediate data to allocate the intermediate data to each computer according to the key and execute the Reduce processing.
  • MapReduce is suitable for distributed processing with a large-scale parallel configuration because processing can be executed by dynamically allocating processing to a plurality of computers.
  • MapReduce also has the advantage that the developer only needs to define the group extraction method in Map processing and the data aggregation method in Reduce processing.
  • Patent Document 2 discloses an inquiry processing method for speeding up the inquiry processing.
  • statistical information is calculated by examining the frequency of appearance of keys for all items (columns) of data to be processed, and the difference in data allocated according to the calculated statistical information.
  • Data processing is assigned to a plurality of nodes so as to be small.
  • the query processing method disclosed in Patent Document 2 is expected to allocate data processing to a plurality of nodes so that the difference in allocated data becomes small when applied to MapReduce. Since it is difficult to pre-define the columns to be used, statistical information is calculated for all the columns of the analysis target data, and there is a problem that the processing load increases due to a large amount of calculation processing.
  • the present invention has been made in consideration of the above points, and intends to propose a data processing system and a data processing method capable of executing parallel distributed processing at high speed.
  • an extraction process for extracting a column and a column value for each record from the analysis target data, and an aggregation process for aggregating the aggregation target data using the key column as a key A plurality of first server devices that perform analysis processing in parallel on each server device, processing for determining the key column based on processing results of past analysis processing by the plurality of first server devices, A second server that performs processing for creating key distribution information indicating the appearance ratio of each column value in a key column, and processing for determining the plurality of first server devices that perform the aggregation processing based on the key distribution information And the second server device divides the data to be analyzed for each column value of the key column.
  • a plurality of first server devices that perform analysis processing including data extraction processing and aggregation processing in parallel in each server device, and the plurality of first devices
  • the plurality of first server devices block from the data to be analyzed
  • a first step of extracting a column and a column value for each record in units, and the second server device determines the key column based on an extraction result by the plurality of first server devices, and each of the key columns
  • the key distribution information indicating the appearance ratio of the column value is created, and the aggregation processing is performed based on the key distribution information.
  • the second server device divides the analysis target data into column values of the key columns. Is provided to the plurality of first server devices in accordance with the appearance ratio of the column value.
  • FIG. 1st Embodiment It is a block diagram which shows the structure of the data processing system by 1st Embodiment. It is explanatory drawing which shows an example of the process target data on a memory. It is a flowchart which shows the whole process procedure of the data processing system shown in FIG. It is a flowchart which shows the process procedure by a parallel distributed processing management server apparatus. It is a table which shows the structure of a data arrangement information management table. It is a table which shows the structure of a process execution status management table. It is a flowchart which shows the process procedure by a parallel distributed processing execution server apparatus. It is a flowchart (the 1) which shows the process procedure which produces the key distribution information of a key column. It is a table which shows the structure of an analysis application execution history management table.
  • reference numeral 100 denotes a data processing system according to the first embodiment as a whole.
  • the data processing system 100 includes a parallel distributed processing management server device 120, a parallel distributed processing execution server device 130 (130A, 130B,..., 130N), and a data distribution control server device 140 that are connected to each other via a network 110.
  • the data processing system 100 is connected to the client device 310 via the network 110. Below, each structure of the data processing system 100 is demonstrated.
  • the parallel distributed processing management server device 120 includes a network interface 121, a CPU (Central Processing Unit) 122, a main storage device 123, which are connected to each other via a bus 125. And a secondary storage device 124.
  • the parallel distributed processing management server device 120 manages execution of parallel distributed processing by the parallel distributed processing execution server device 130.
  • the network interface 121 is an interface for the client device 120 to connect to the network 110.
  • the CPU 122 is an arithmetic processing unit that executes a program stored in the main storage device 123.
  • the main storage device 123 is a storage device such as a RAM (Random Access Memory) that stores a program executed by the CPU 122 and data necessary for executing the program.
  • the main storage device 123 stores a data registration processing unit 1231, a distributed processing management unit 1232, and an OS (Operating System) 1233.
  • the data registration processing unit 1231 and the distributed processing management unit 1232 are, for example, application programs.
  • the OS 1233 is a software program that provides basic functions that can be used by application programs and manages the entire parallel distributed processing management server device 120 by the operation of the CPU 122.
  • processing by the data registration processing unit 1231 is actually realized by “the CPU 122 operates using basic functions provided to the OS 1231 according to the program of the data registration processing unit 1231”. For convenience, it is described as “the data registration processing unit 1231 performs processing”. Similarly, in the processing of other programs, operations by the CPU and OS may be omitted.
  • the secondary storage device 124 is a storage device such as a hard disk drive (HDD: Hard Disk Drive) that stores data.
  • the secondary storage device 124 is not limited to a magnetic storage device such as an HDD, and may be another storage device, for example, a semiconductor storage device such as a flash memory.
  • the secondary storage device 124 stores a data arrangement information management table 1241 and a process execution status management table 1242.
  • each unit stored in the main storage device 123 and the specific structure of each table stored in the secondary storage device 124 will be described later in the description of the parallel distributed processing according to this embodiment.
  • the functional description of each unit stored in the main storage devices 133 and 143 of the parallel distributed processing execution server device 130 and the data distribution control server device 140 and the details of each table stored in the secondary storage devices 134 and 144 are provided.
  • the specific structure will also be described later in the description of the parallel distributed processing according to the present embodiment.
  • the parallel distributed processing management server device 120, the parallel distributed processing execution server device 130, and the data distribution control server device 140 each have different application programs and tables, but the hardware configuration is “network interface, The CPU, the main storage device, and the secondary storage device are connected to each other by a bus ”. Further, the hardware configuration of the client device 310 is the same as that of the parallel distributed processing management server device 120. Therefore, in the description of the configurations of the parallel distributed processing execution server device 130, the data distribution control server device 140, and the client device 310, which will be described later, the description of the same parts as the parallel distributed processing management server device 120 is omitted.
  • the parallel distributed processing execution server device 130 (130A, 130B,..., 130N) includes a network interface 131, a CPU 132, A plurality of computers having a main storage device 133 and a secondary storage device 134.
  • parallel distributed processing is executed by a plurality of computers performing data extraction processing and aggregation processing in parallel.
  • Aggregation processing by the parallel distributed processing execution server device 130 is performed using a key column determined by the data distribution control server device 140 as a key.
  • the main storage device 133 stores a parallel distributed processing execution unit 1331 and an OS 1332.
  • the parallel distributed processing execution unit 1331 extracts the column and the column value for each record from the data to be analyzed, and the aggregation processing using the key column as a key for the data divided for each column value of the key column
  • the analysis application stored in each of the parallel distributed processing execution server devices 130A to 130N may be a heterogeneous analysis application that performs different extraction processing, or may be the same type of analysis application that performs the same extraction processing.
  • the secondary storage device 134 stores processing target data 1341 registered as data for performing parallel distributed processing.
  • the processing target data 1341 is sequentially read on the main storage device 133 in units of at least one block, and the read processing (processing target data 1333 on the memory) is subjected to extraction processing by the analysis application.
  • FIG. 2 shows, as an example of the processing target data 1333 on the memory, website browsing logs by a plurality of users, one record per line.
  • the processing target data 1333 in FIG. 2 includes a block name column 1333A in which a block name to which the record is allocated is described, a date / time information column 1333B in which the creation time of the record is described, and a user who can specify an executor of the record
  • a user name column 1333C in which an ID is described
  • a content column 1333D in which a browsing destination URL (Uniform Resource Locator) is described
  • a category name column 1333E in which a category name previously associated with the browsing destination URL is described. It has a table structure for items.
  • a plurality of records are divided into a plurality of blocks in units of a predetermined number of records or time.
  • the block columns 1333F to 1333J records divided into blocks are described.
  • the top record in FIG. 2 belongs to block 1 and indicates that a URL classified as category A was browsed by a user with user ID 00001 on December 22, 2011 at 10:11. Yes.
  • the parallel distributed processing execution unit 1331 performs extraction processing on the processing target data 1333 on the memory in FIG. 2, date information, user ID, content, and category name are extracted as columns, for example, column [category name ], [Category A] to [Category D] are extracted as column values.
  • the data distribution control server device 140 includes a network interface 141, a CPU 142, a main storage device 143, and a secondary storage device 144 that are connected to each other via a bus 145. It is a computer.
  • the data distribution control server device 140 analyzes the extraction processing result of the processing target data 1341 by the parallel distributed processing execution unit 1331 of the parallel distributed processing execution server device 130 and uses it as a key in the aggregation processing in the parallel distributed processing execution server device 130.
  • a key column that is a column to be processed is determined, and key distribution information based on each column value (key column value) of the key column in the processing target data 1341 is created.
  • the data distribution control server device 140 distributes data (aggregation target data) to be subjected to aggregation processing to a plurality of parallel distributed processing execution server devices 130 and performs aggregation processing based on key distribution information of key columns.
  • the process of determining the parallel distributed processing execution server device 130 that is the allocation destination of the aggregation target data is performed.
  • the main storage device 143 stores an analysis application execution history management unit 1431, a key column candidate extraction unit 1432, a key distribution calculation unit 1433, a data distribution destination determination unit 1434, and an OS 1435.
  • the analysis application execution history management unit 1431, the key column candidate extraction unit 1432, the key distribution calculation unit 1433, and the data distribution destination determination unit 1434 are, for example, application programs.
  • the secondary storage device 144 includes an analysis application execution history management table 1441, a key distribution calculation target column management table 1442, a key distribution management table 1443, a distribution destination server device management table 1444, a key column candidate management table 1445, and a key column threshold management table 1446.
  • a key distribution calculation target column management table 1447, an aggregation processing start condition management table 1448, a distribution destination candidate server device management table 1449, and a distribution data allocation management table 1450 are stored.
  • the client device 310 is a computer having a network interface 311, a CPU 312, a main storage device 313, and a secondary storage device 314 that are connected to each other via a bus 315.
  • the client device 310 transmits a data registration execution request for requesting registration of processing target data or a data analysis processing execution request for requesting analysis of processing target data to the data processing system 100 in accordance with a user operation.
  • the main storage device 313 stores a client processing unit 3131 and an OS 3132.
  • the client processing unit 3131 is an application program that performs processing such as reading and writing of client processing data 3141 and transmission of an execution request to the data processing system 100.
  • the secondary storage device 314 stores client processing data 3141 that is the source of the processing target data 1341.
  • the client processing data 3141 is transmitted following the execution request after the data registration execution request is transmitted from the client device 310 to the data processing system 100, and written into the secondary storage device 134 of the parallel distributed processing execution server device 130.
  • the processing target data 1341 is updated.
  • the parallel distributed processing management server device 120 manages execution of parallel distributed processing by the parallel distributed processing execution server device 130. Then, the plurality of parallel distributed processing execution server devices 130 perform analysis processing including the extraction processing of the processing target data 1341 and the aggregation processing of the aggregation target data in parallel in each parallel distributed processing execution server device 130. Realize parallel distributed processing.
  • the aggregation target data is data obtained by dividing the processing target data 1341 for each key column value of the key column determined by the data distribution control server device 140.
  • the data distribution control server device 140 analyzes the extraction processing result of the processing target data 1341 by the parallel distributed processing execution server device 130 and creates key distribution information based on the key column value of the key column in the processing target data 1341. Further, the data distribution control server device 140 determines the parallel distributed processing execution server device 130 that is the allocation destination of the aggregation target data based on the key distribution information of the key column.
  • step S101 the client processing unit 1131 causes the parallel distributed processing management server device 120 and the data distribution control to start when a predetermined operation for requesting analysis of processing target data is performed by the user in the client device 310.
  • a data analysis processing execution request is transmitted to the server device 140.
  • the parallel distributed processing management server device 120 and the data distribution control server device 140 start processing in parallel.
  • an aggregation key designated by the user is also transmitted to the data distribution control server apparatus 140.
  • the aggregation key is a key column candidate in the aggregation process, and at the time of the first analysis by the parallel distributed processing execution unit 1331, the designated aggregation key is acquired as the key column candidate.
  • the distributed processing management unit 1232 is activated by the CPU 122 and manages the parallel distributed processing to be executed by the parallel distributed processing execution server device 130 (step S102).
  • the outline during the time when the parallel distributed processing is managed by the parallel distributed processing management server apparatus 120 (step S102) is described in steps S103 to S108.
  • step S103 the distributed processing management unit 1232 of the parallel distributed processing management server device 120 activates the parallel distributed processing execution unit 1331 of the parallel distributed processing execution server device 130. Then, the started parallel distributed processing execution unit 1331 executes parallel distributed processing (step S104). During the execution of the parallel distributed processing in step S104, the key distribution information is determined (step S105), the aggregation target data is distributed to the plurality of parallel distributed processing execution server devices 130 (step S106), and each parallel distributed processing is executed. Aggregation processing of the aggregation target data is performed in the server 130 (step S107).
  • step S108 when the distributed processing management unit 1232 receives a message indicating the completion of execution of the parallel distributed processing from the parallel distributed processing execution unit 1331 (step S108), the management of the parallel distributed processing is terminated and the processing is completed.
  • step S109 key distribution information of the key column is created (step S109).
  • a key column that is a calculation target of the key distribution is determined based on the extraction result by the parallel distributed processing execution unit 1331 of the parallel distributed processing execution server device 130, and the key distribution indicating the appearance ratio for each key column value of the key column Information is created.
  • the key column key distribution information created in step S109 is used for the determination in step S105 in the parallel distributed processing execution server device 130.
  • the data distribution control server device 140 creates a table for determining the parallel distributed processing execution server device 130 that is the distribution destination of the aggregation target data based on the key distribution information of the key column (step S110).
  • This table corresponds to the distribution destination server apparatus management table 1444, and its specific structure will be described later.
  • the data distribution in step S108 on the parallel distributed processing execution server device 130 side is performed with reference to the table created in step S110.
  • the processing by the parallel distributed processing management server device 120 and the parallel distributed processing execution server device 130 are performed.
  • Processing is performed in parallel with reference to each other.
  • the parallel distributed processing management server device 120 registers the processing target data in accordance with the execution request received from the client device 310 (steps S203 to S204) or manages the execution of the parallel distributed processing of the processing target data. (Steps S205 to S208) are performed. Note that the processing shown in steps S205 to S208 in FIG. 4 corresponds to the processing shown in step S102 in FIG.
  • the parallel distributed processing management server device 120 checks whether the received message is a processing target data registration execution request (step S202). ).
  • the data registration processing unit 1231 is transmitted from the client device 310 after the processing target data registration execution request.
  • the incoming client processing data 3141 or a part of the client processing data 3141 is stored in block units in any of the secondary storage devices 134 of the parallel distributed processing execution server devices 130A to 130N, and the processing target data 1341 is updated (step S203).
  • the data registration processing unit 1231 updates the processing target data 1341 updated in step S203 by writing predetermined information in the data arrangement information management table 1241 ( Step S204).
  • the data arrangement information management table 1241 is a table for managing in which parallel distributed processing execution server device 130 the processing target data 1341 is stored in block units in step S203.
  • the data arrangement information management table 1241 includes a data block ID column 1241A in which a block name of data is described, and a data arrangement server apparatus name in which the name of the server apparatus in which the data of the block is stored is described.
  • the structure has a column 1241B. “Server apparatus 1”, “Server apparatus 2”, and “Server apparatus 3” described in the data arrangement server apparatus name column 1241B in FIG. 5 are assigned to any of the parallel distributed processing execution server apparatuses 130A to 130N. Correspond.
  • step S204 the parallel distributed processing management server device 120 ends the processing target data registration process.
  • step S202 if the message received from the client device 310 in step S202 is not a processing target data registration execution request, that is, if it is an analysis processing execution request (NO in step S202), the distributed processing management unit 1232 performs parallel processing.
  • the parallel distributed processing execution unit 1331 of the distributed processing execution server device 130 is activated (step S205).
  • step S207 for the parallel distributed processing (step S207) executed by the parallel distributed processing execution unit 1331 started in step S205, the distributed processing management unit 1232 stores information indicating the execution status of the processing in the processing execution status management table 1242.
  • the execution status of the parallel distributed processing is managed by updating (step S206).
  • the processing execution status management table 1242 includes a server device name column 1242A, an arrangement data block number column 1242B, a processing completion block number column 1242C, a progress rate column 1242D, a start time column 1242E, and a completion time column 1242F. It has the structure which has.
  • the server device name column 1242A the name of a server device on which parallel distributed processing is performed (any of the parallel distributed processing execution server devices 130A to 130N) is described.
  • the arrangement data block number column 1242B describes the number of blocks of the processing target data 1341 assigned to the server device (the server device) described in the server device name column 1242A.
  • the progress rate column 1242D describes the ratio of the value in the processing completed block number column 1242C to the value in the arrangement data block number column 1242B of the server device as the processing progress rate in the server device.
  • the start time column 1242E and the completion time column 1242F the start time and completion time of the analysis processing in the server device are described.
  • step S208 when the distributed processing management unit 1232 receives a message indicating that the execution of the parallel distributed processing is completed from the parallel distributed processing execution unit 1331 (step S208), the series of processing ends.
  • step S301 the parallel distributed processing execution unit 1331 of the parallel distributed processing execution server device 130 is activated in accordance with an instruction from the distributed processing management unit 1232 of the parallel distributed processing management server device 120. Then, the parallel distributed processing execution unit 1331 starts analysis processing including extraction processing and aggregation processing. In the extraction process, a column and a column value for each record are extracted in block units from the processing target data 1333 on the memory. The result of the extraction process is referred to by the analysis application execution history management unit 1431 of the data distribution control server device 140 described later.
  • the parallel distributed processing execution unit 1331 refers to the key distribution management table 1443 of the data distribution control server device 140 and confirms whether there is key distribution information of the key column (step S302).
  • the key distribution management table 1443 will be described later with reference to FIGS.
  • step S302 If there is key distribution information of the key column in step S302 (YES in step S302), the parallel distributed processing execution unit 1331 collects data in the parallel distributed processing execution server device 130 based on the distribution destination server device management table 1444. Data is distributed (step S303).
  • the aggregation target data is data obtained by dividing the processing target data 1341 for each column value of the key column, if there is a record that does not include the key column in the processing target data 1341, the record is the aggregation target data. Excluded from. Thereafter, the process of step S305 is executed.
  • step S302 If there is no key distribution information of the key column in step S302 (NO in step S302), the parallel distributed processing execution unit 1331 sends the processing target data 1341 to the parallel distributed processing execution server device 130 regardless of the bias of the key distribution. Distribute (step S304). Thereafter, the process of step S305 is executed.
  • step S305 aggregation processing using the key column as a key is performed in each parallel distributed processing execution server device 130 to which data has been distributed in step S303 or step S304.
  • step S305 commonly used key aggregation processing can be used.
  • the parallel distributed processing execution unit 1331 sends a message notifying the completion of the parallel distributed processing execution to the distributed processing management unit 1232 of the parallel distributed processing management server device 120. (Step S306), the process ends.
  • the parallel distributed processing execution unit 1331 transmits the result of the aggregation processing in step S305 to the parallel distributed processing management server device 120. Then, after the distributed processing management unit 1232 of the parallel sentence finger processing management server device 120 receives the result of the aggregation processing from all the parallel distributed processing execution units 1331 of the parallel distributed processing execution server device 130, the received result of the aggregation processing Is transmitted to the client device 310 and output to an output unit (not shown) of the client device 310.
  • processing by the data distribution control management server device 140 will be described.
  • the extraction processing result of the processing target data 1341 (or processing target data 1333 on the memory) in the parallel distributed processing execution server device 130 is analyzed, and the key column in the processing target data 1341 is analyzed.
  • the processing for creating the key distribution information will be described with reference to FIGS. 8, 10, and 14.
  • FIG. the allocation destination of the processing target data 1341 is determined based on the key distribution information for a plurality of parallel distributed processing execution server devices 130 that perform aggregate information using the key column as a key. Processing to be performed will be described with reference to FIG. 8, 10, and 14 correspond to the process shown in step S ⁇ b> 109 of FIG. 3, and the process of FIG. 17 corresponds to the process shown in step S ⁇ b> 110 of FIG. 3.
  • FIG. 8 shows that the analysis application execution history management unit 1431 performs extraction by the parallel distributed processing execution unit 1331 in the process of creating the key column key distribution information. Processing for updating the analysis application execution history table 1441 with reference to the result is described.
  • step S401 when the data distribution control server device 140 receives an analysis processing execution request from the client device 310, the data distribution control server device 140 activates the analysis application execution history management unit 1431.
  • the analysis application execution history management unit 1431 checks whether or not the value of the parallel distributed processing execution counter is 0 (step S402).
  • the parallel distributed processing execution counter is one of the parameters held in the main storage device 143, and the number of processes is counted each time the parallel distributed processing execution 1331 performs extraction processing in units of blocks.
  • step S402 If the value of the parallel distributed processing execution counter is 0 in step S402 (YES in step S402), it indicates that the extraction processing by the parallel distributed processing execution unit 1331 is the first time.
  • the analysis application execution history management unit 1431 selects the analysis application name and the analysis process of the parallel distributed processing execution unit 1331 used for the extraction process in block units from the processing target data 1333 on the memory for which the extraction process has been completed. And the column designated as the aggregation key at the time of transmission of the data analysis processing execution request in the client device 310 is acquired (step S403).
  • the analysis application execution history management unit 1431 uses the column acquired in step S403 as a key column candidate, and updates the analysis application execution history management table 1441 together with the other data acquired in step S403 (step S404).
  • the analysis application execution history management table 1441 includes an analysis application name column 1441A in which the analysis application name acquired in step S403 is described, an execution date / time column 1441B in which the execution date and time of analysis processing is described, and a key column.
  • the structure has a key column candidate name field 1441C in which candidates are described.
  • the analysis application name described in the analysis application name column 1441A of the analysis application execution history management table 1441 is described every time extraction processing is performed in block units. Therefore, when analysis processing by the same kind of analysis application is performed on a plurality of blocks, the analysis application name is described a plurality of times in the analysis application name column 1441A, and a plurality of blocks are analyzed by different types of analysis applications. When analysis processing is performed, a plurality of analysis application names are described in the analysis application name column 1441A.
  • step S402 If the value of the parallel / distributed process execution counter is not 0 in step S402 (NO in step S402), the key column candidate has already been acquired during the first analysis process, so it is necessary to acquire the key column candidate again.
  • the analysis application execution history management unit 1431 does not update the analysis application execution history management unit 1431. Then, when the value of the parallel distributed processing execution counter is not 0 in step S402, or after the processing of step S404, the processing of FIG. 10 is executed. Note that the analysis application execution history management unit 1431 may add 1 to the value of the parallel distributed processing execution counter before performing the processing of FIG.
  • FIG. 10 shows a process for creating key column key distribution information.
  • the key column candidate extraction unit 1432 determines a key column candidate having an appearance degree exceeding a predetermined threshold as a key column, and performs analysis. A process of updating the application execution history management table 1441 is described.
  • step S501 the key column candidate extraction unit 1432 is activated. Then, the key column candidate extraction unit 1432 refers to the analysis application execution history management table 1441 and counts the value of the total number of occurrences of key column candidates (key column candidate appearance total value) described in the key column candidate name field 1441C for each analysis application. (Step S502).
  • the key column candidate extraction unit 1432 refers to the analysis application execution history management table 1441 and the key column candidate appearance total value counted in step S502, and for each key column candidate in the past analysis processing execution history by the analysis application. Is calculated as the key column candidate degree, and the key column candidate management table 1445 is updated (step S503).
  • the key column candidate management table 1445 includes a key column candidate name field 1445A in which a key column candidate name is described, a key column candidate appearance number field 1445B in which the number of appearances of the key column candidate is described, and the appearance of the key column candidate.
  • the structure has a key column candidate degree column 1445C in which the ratio is described.
  • each item of the key column candidate management table 1445 is calculated for the columns from [analysis application 1] to [analysis application 4] in the analysis application execution history table 1441 shown in FIG. 9, it is described in the key column candidate name 1441C.
  • the total number of key column candidates that is the total number of times is 6.
  • the key column candidate extraction unit 1432 refers to the key column candidate management table 1445, and determines whether the key column candidate degree of each key column candidate exceeds the key column threshold described in the key column threshold management table 1446 (Step S1). S504).
  • the key column threshold value management table 1446 has a structure having a key column threshold value field 1446A in which preset threshold values are described. In FIG. 12, the key column threshold value table is set to [0.4]. Yes.
  • step S504 If it is determined in step S504 that there is a key column candidate degree exceeding the key column threshold (YES in step S504), the key column candidate extraction unit 1432 determines a key column candidate having a key column candidate degree exceeding the key column threshold as a key column.
  • the management table 1447 is updated (step S505). As shown in FIG. 13, the key column management table 1447 has a structure having a key column name column 1447 in which the name of the key column determined in step S505 is described. In FIG. 13, since the key column candidate degree of [category name] in the key column candidate management table 1445 of FIG. 11 is [0.50] and exceeds the key column threshold value [0.4] of FIG. ] Is described.
  • step S505 When the key column candidate degree analysis including the execution history of the past analysis processing included in the analysis target is performed and the key column is determined by the processing in step S505, the key column candidate extraction unit 1432 completes the update of the key column management table 1447. Then, an update completion message is transmitted (step S506), and then the process shown in step S601 of FIG. 14 is executed.
  • step S504 If it is determined in step S504 that there is no key column candidate degree exceeding the key column threshold (NO in step S504), the key column candidate extraction unit 1432 assumes that there is no key column candidate degree to be determined as a key column at present.
  • the process shown in step S606 of FIG. 14 is executed without determining the key column.
  • FIG. 14 shows a key column value for each key column value in the key column determined by the key distribution calculation unit 1433 in the process of creating key distribution information of the key column. The process of calculating the appearance ratio (key distribution value) of and updating the key distribution management table 1443 is described.
  • step S601 when the data distribution control server device 140 receives an update completion message for the key column management table 1447 from the key column candidate extraction unit 1432, the key distribution calculation unit 1433 is activated. Then, the key distribution calculation unit 1433 refers to the processing target data 1333 on the memory, and for the key column described in the key column management table 1447, the key distribution candidate 1300 records the number of records of the key column that appears in the processing target data 1333 on the memory. The total number of occurrences is counted (step S602).
  • the key distribution calculation unit 1433 counts the number of appearances for each key column value of the key column from the processing target data 1333 in the memory, and uses the appearance ratio with respect to the total number of key column candidate record appearances counted in step S602 as the key distribution value.
  • the key distribution management table 1443 is calculated and updated (step S603).
  • the key distribution management table 1443 includes a key column value field 1443A in which a key column value is described, a key appearance number field 1443B in which the number of occurrences of the key column value is described, and a key distribution value of the key column value.
  • the key distribution value column 1443C is described.
  • [category name] is set as the key column
  • [category A] to [category D] are described as the key column values in the key column value field 1443A, and correspond to each key column value.
  • the key appearance number [10], [5], [3], and [2] are described in order from the top of the key appearance number column 1443B.
  • the key distribution calculation unit 1433 determines whether or not a predefined aggregation processing start condition is satisfied.
  • the pre-defined aggregation processing start condition includes, for example, the processing progress rate of analysis processing in the parallel distributed processing execution server device 130, the number of execution times of analysis processing, the number of records subjected to analysis processing, or the progress from the start of analysis processing. Time etc. can be set.
  • the processing progress rate of analysis processing in the parallel distributed processing execution server device 130 is defined in advance as a start condition of aggregation processing will be described with reference to FIGS. 6 and 16.
  • a predefined threshold value [0.5] is described in the job progress rate threshold value field 1448A.
  • the key distribution calculation unit 1433 refers to the processing execution status management table 1242 (FIG. 6) of the parallel distributed processing management server device 120 (FIG. 6), and executes the parallel distributed processing execution described in the server device column 1242A. It is determined whether the progress rate in the server device 130 exceeds the threshold value [0.5] of the aggregation processing start condition management table 1448.
  • the key distribution calculation unit 1433 It is determined that the start condition for the aggregation process is not satisfied.
  • step S604 When it is determined in step S604 that the aggregation processing start condition is satisfied (YES in step S604), the key distribution management table 1242 is created as key distribution information of the key column, and the aggregation processing may be started. Therefore, the key distribution calculation unit 1433 transmits an update completion message informing that the update of the key distribution management table 1242 is completed (step S605), and ends the process.
  • step S604 determines whether the aggregation process start condition is not satisfied (NO in step S604), or if it is determined in step S504 in FIG. 10 that there is no key column candidate degree exceeding the key column threshold ( In step S504, it is determined that sufficient analysis results have not yet been collected to create key column key distribution information.
  • the parallel distributed processing execution server device 130 reads the next block of the processing target data 1333 on the memory in order to continue the analysis processing (step S606), and performs the analysis processing on the read next block. (Step S607). In step S607, in the analysis processing for the next block, the processing described so far with reference to FIGS. 7, 8, and 10 is performed again.
  • the key distribution calculation unit 1433 waits for the progress of analysis processing for the next block. Thereafter, after a predetermined time has elapsed, the key distribution calculation unit 1433 confirms whether the key column is described in the key column management table 1447 (step S608).
  • step S608 If the key column is confirmed in step S608 (YES in step S608), the key distribution calculation unit 1433 returns to the process in step S602 and starts calculating the key distribution value for the newly confirmed key column. If the key column is not confirmed in step S608 (NO in step S608), the key distribution calculation unit 1433 returns to step S604, and determines again whether the aggregation processing start condition is satisfied. Do.
  • the key distribution management table 1242 is created as the key column distribution information of the key column if the key column is determined by the processes described in FIGS. 8, 10, and 14 and the start condition of the aggregation process is satisfied. The In addition, in a situation where the key column has not been determined or a condition where the aggregation process start condition is not satisfied, the process of creating key distribution information of the key column is not completed. In steps S302 to S304 of FIG. 7 described above, the parallel distributed processing execution unit 1331 confirms the presence / absence of the key distribution management table 1242, and sends the aggregation target data to the parallel distributed processing execution server device 130 according to the confirmation result. Distribute.
  • the data distribution control server device 140 receives the update completion notification of the key distribution management table 1443 in FIG. 14 and executes a process of determining a data distribution destination server device (described later with reference to FIG. 17).
  • the data distribution destination determination unit 1434 is activated (step S701).
  • the data distribution destination determination unit 1434 refers to the processing execution status management table 1242 to extract the parallel distributed processing execution server device 130 (distribution destination candidate server device) that can distribute the aggregation target data, and extracts the extraction result as the distribution destination.
  • the candidate server device management table 1449 is updated (step S702). As shown in FIG. 18, the distribution destination candidate server device management table 1449 has a structure having a distribution destination candidate server device name column 1449A in which distribution destination candidate server devices are described. In step S702, the data distribution destination determination unit 1434 counts the number of distribution destination candidate server devices described in the distribution destination candidate server device name column 1449A.
  • the data distribution destination determination unit 1434 allocates the aggregation target data divided for each key column value based on the number of distribution destination candidate server devices counted in step S702 and the description content of the key distribution management table 1443.
  • the number of server devices is calculated, and the distribution data allocation management table 1450 is updated based on the calculation result (step S703).
  • the distributed data assignment management table 1450 has a structure having a key column value field 1450A, a record number field 1450B, a key distribution value field 1450C, and an assigned server device number field 1450D.
  • the key distribution value column 1450C describes the key distribution values in the key distribution management table 1443 in descending order.
  • the key column value column 1450A and the record number column 1450B include the key distribution calculation target column value and key corresponding to the key distribution value. The number of occurrences is listed.
  • the number of assigned server devices column 1450D describes the number of assigned server devices based on the key distribution value in the column and the number of distribution destination candidate server devices counted in step S702.
  • the data distribution destination determination unit 1434 assigns the distribution destination candidate server device described in the distribution destination candidate server device name field 1449A to the head of the data distribution destination determination unit 1434 according to the number of allocation server devices in the distribution data allocation management table 1450. Are assigned as distribution server devices in order. Then, the data distribution destination determination unit 1434 updates the distribution destination server device management table 1444 with the assigned distribution server device and the corresponding key column value (step S704). As shown in FIG. 20, the distribution destination server device management table 1444 has a structure having a key column value column 1444A in which a key column value is described and a distribution server device column 1444B in which a distribution server device name is described.
  • the data distribution control server device 140 adds the key columns from the key column candidates with high accuracy as the analysis processing by the parallel distributed processing execution server device 130 is repeated. Can be determined. Then, the parallel distributed processing execution server device 130 performs the aggregation process by assigning the data divided for each column value of the key column based on the key distribution information of the appropriate key column so that the processing amount is allocated as evenly as possible. The aggregation process can be performed while avoiding the bottleneck due to the bias of the key distribution.
  • each parallel distributed processing execution server device 130 As a result, the time difference required for the analysis processing in each parallel distributed processing execution server device 130 is reduced, and the parallel distributed processing execution server device 130 as a whole can be expected to realize parallel distributed processing at high speed. This effect is particularly effective when each of the parallel distributed processing execution server apparatuses 130A to 130N is composed of computers having substantially the same processing performance.
  • the data distribution control server device 140 refers to the past analysis processing results, extracts key column candidates that are key column candidates, and narrows down the key column candidates to statistical information ( (Corresponding to the analysis application execution history management table 1441) is created, it is not necessary to create statistical information for all the columns of the processing target data 1341, and unnecessary statistical processing is not performed, and the effect of reducing the processing load can be expected. .
  • the key distribution information is generated by calculating the appearance ratio for each column value of the key column until the progress of the data analysis process satisfies a predetermined condition, and therefore sufficient statistics are provided.
  • the key distribution information of the key column can be created based on the information, and by using the key distribution information of the key column thus created, the parallel distributed processing execution server device 130 that is the data distribution destination in the aggregation processing can be accurately Can be well determined.
  • the data processing system according to the second embodiment has the first feature when assigning aggregation target data to a server device in parallel distributed processing.
  • the data processing system 100 according to the embodiment is characterized in that the allocation is performed in consideration of the bias of the key distribution, whereas the allocation is performed in consideration of the processing capability of the server device in addition to the bias of the key distribution. To do.
  • the configuration of the data processing system 200 is the same as that of the data processing system 100 of the first embodiment except that the data distribution control server device 140 is a data distribution control server device 240. Since this is a configuration, description of the common configuration is omitted. In addition, regarding the internal configuration of the data distribution control server device 240, those having the same functions as those of the data distribution control server device 140 are denoted by the same reference numerals, and description thereof is omitted.
  • the data distribution destination determination unit 2434 is an application program for performing processing different from that of the data distribution destination determination unit 1434 to determine the allocation destination of the aggregation target data. For the operation, refer to FIG. It will be described later.
  • the distribution processing efficiency management table 2450 is a table used for determining an allocation destination of aggregation target data in the aggregation processing.
  • the distribution server device management table 2444 is a table in which the allocation destination of the aggregation target data determined by the data distribution destination determination unit 2434 is described.
  • the structure of the distribution processing efficiency management table 2450 will be described later with reference to FIG. 23, and the structure of the distribution server apparatus management table 2444 will be described later with reference to FIG.
  • the processing by the data processing system 200 is the processing by the data processing system 100 of the first embodiment, except for the processing in which the data distribution destination determination unit 2434 determines the distribution destination server device.
  • the description of common processing is omitted.
  • processing performed by the data distribution destination determination unit 2434 will be described with reference to FIG.
  • the data distribution destination determination unit 2434 is activated, updates the distribution destination candidate server device management table 1449, counts the number of distribution destination candidate server devices, and updates the distribution data allocation management table 1450 ( Steps S801 to S803) are the same as the processing described in steps S701 to S703 in FIG.
  • step S804 the data distribution destination determination unit 2434 updates the analysis processing efficiency management table 2450 with reference to the processing execution status management table 1242.
  • the analysis processing efficiency management table 2450 has a structure including a server device name column 2450A, a processing completion block total number column 2450B, a total processing time column 2450C, a processing efficiency column 2450D, and a processing capacity ratio column 2450E. ing.
  • the processing completion block total number column 2450B, the total processing time column 2450C, the processing efficiency column 2450D, and the processing capacity ratio column 2450E correspond to predetermined parallel processing execution server devices 130 described in the server device name column 2450A. Data is described.
  • the total number of completed blocks of analysis processing executed in the past is written in the total number of processed blocks column 2450B.
  • the total number of blocks completed in the analysis process can be calculated by accumulating the number of process completed blocks in the process execution status management table 1242 every time the analysis process is completed.
  • the total processing time column 2450C describes the total processing time [h] required for the analysis processing executed in the past.
  • the total processing time is calculated from the start time and the completion time described in the start time column 1242E and the completion time column 1242F of the process execution status management table 1242, and this calculated value is calculated every time block-based analysis processing is completed. It can be calculated by accumulating.
  • the processing efficiency column 2450D describes the processing efficiency indicating the number of processing blocks per unit time that can be calculated from the total number of processing completed blocks and the total processing time.
  • the processing capacity ratio column 2450E describes the processing capacity ratio between server apparatuses described in the server apparatus name column 2450A. For example, in FIG. 23, the processing capability ratio is described with the processing capability value of the server device in the top row as the reference value (1.0).
  • the data distribution destination determination unit 2434 selects the distribution destination candidate server devices described in the distribution destination candidate server device name column 1449A in order from the server devices with the highest processing capability ratio in the analysis processing efficiency management table 2450.
  • the number of assigned server devices is assigned based on the key distribution value and the distribution destination server device management table 2444 is updated (step S805).
  • the distribution destination server apparatus management table 2444 has a structure having a key column value field 2444A in which a key column value is described and a distribution server apparatus name field 2444B in which a distribution server apparatus name is described.
  • the number of server devices to which the processing target data 1341 is allocated from the distribution destination candidate server device management table 1449 is four servers [server device 1] to [server device 4].
  • the processing capacity ratio of [Server apparatus 1] to [Server apparatus 4] to be assigned is set to 1 as a whole. Each processing capacity ratio is calculated. At this time, [Server 1] is 0.25, [Server 2] is 0.15, [Server 3] is 0.10, and [Server 4] is 0.50.
  • [category A] having the largest number of records in the distribution data allocation management table 1450 is described.
  • [server device 4] having the highest processing capability ratio in FIG. 23 is described. Since the key distribution value of [Category A] is 0.50 and the processing capacity ratio of [Server 4] is also 0.50, [Server 4] is the only server device to which [Category A] processing is assigned. I know it ’s good.
  • [Category B] is described in the key column value field 2444A in the next row of the distribution destination server device management table 2444, and the same assignment as in [Category A] is performed.
  • server devices to be assigned are determined and described in the distribution destination server device management table 2444.
  • the parallel distributed processing execution server devices 130A to 130N are not the same model or have different specifications and processing capabilities because of different manufacturing ages. In particular, it is possible to determine a distribution destination server in which a bottleneck is unlikely to occur in consideration of a difference in specifications and processing capability, and an effect of realizing high-speed parallel distributed processing can be expected.
  • the client processing data 3141 that is the source of the processing target data 1341 is the secondary storage of the client device 310.
  • the present invention is not limited to this.
  • data that is the source of the processing target data 1341 is accumulated in an external backbone server system (not shown), and the accumulated data May be configured to be transmitted from the core server system to the data processing system 100 or 200 intermittently or periodically.
  • the aggregate key specified by the user is the data distribution control server device.
  • the present invention may be configured to be defined in the program of the parallel distributed processing execution unit 1331 of the parallel distributed processing execution server device 130. Good.
  • the analysis application execution history management unit 1431 may acquire the aggregate key as a key column candidate from the parallel distributed processing execution unit 1331 during the first analysis process by the parallel distributed processing execution unit 1331.
  • the present invention is not limited to this, for example, a plurality of streams.
  • the data processing method according to the present invention may be applied to a data processing system configured using a processing infrastructure and inputting a data group aggregated for each key to the stream processing infrastructure and performing time-series processing.
  • a data processing system configured as described above it is possible to avoid a bottleneck caused by the concentration of data input to a specific stream processing platform, and it can be expected to perform parallel distributed processing of streams at high speed.
  • the parallel distributed processing execution server device 130 obtains a column and a column value for each record in block units from the data to be analyzed. It is an example of the some 1st server apparatus which performs the analysis process which consists of the extraction process to extract, and the aggregation process which aggregates the data of aggregation object by using a key column as a key in each server apparatus.
  • the data distribution control server devices 140 and 240 create key distribution information indicating a key column based on processing results of extraction processing by a plurality of first server devices, and key distribution information indicating the appearance ratio of each column value in the key column 4 is an example of a second server device that performs a process of determining a plurality of first server devices that perform aggregation processing based on key distribution information.
  • the analysis application execution history management unit 1431 is an example of an aggregate key acquisition unit.
  • the present invention can be applied to a data processing system and a data processing method for parallel and distributed processing of large-scale data.

Abstract

Le problème décrit par la présente invention est de proposer un système de traitement de données pouvant exécuter un traitement réparti parallèle à une vitesse élevée. La solution selon la présente invention consiste à configurer un système de traitement de données (100) comprenant : une pluralité de premiers dispositifs serveur (130) qui, en parallèle, réalisent un traitement d'analyse faisant appel à un traitement d'extraction permettant d'extraire des colonnes et des valeurs de colonne à partir de données devant être analysées et un traitement d'agrégation permettant d'agréger, avec une colonne de clés en tant que clé, des données devant être agrégées ; et un second dispositif serveur (140) permettant de réaliser un traitement permettant de déterminer la colonne de clés sur la base des résultats de traitement d'un traitement d'analyse antérieur, un traitement permettant de produire des informations de répartition de clés indiquant le pourcentage d'apparition de chaque valeur de colonne dans la colonne de clés, et un traitement permettant de déterminer la pluralité de premiers serveurs (130) afin de réaliser le traitement d'agrégation sur la base des informations de répartition de clés. En outre, le second dispositif serveur (140) détermine la pluralité de premiers dispositifs serveur (130) pour réaliser le traitement d'agrégation au moyen de l'attribution de données devant être agrégées, qui résultent de la division des données devant être analysées dans chaque valeur de colonne de la colonne de clés, à la pluralité de premiers dispositifs serveur (130) en fonction du pourcentage d'apparition des valeurs de colonne.
PCT/JP2012/059789 2012-04-10 2012-04-10 Système de traitement de données et procédé de traitement de données WO2013153620A1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/JP2012/059789 WO2013153620A1 (fr) 2012-04-10 2012-04-10 Système de traitement de données et procédé de traitement de données

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/JP2012/059789 WO2013153620A1 (fr) 2012-04-10 2012-04-10 Système de traitement de données et procédé de traitement de données

Publications (1)

Publication Number Publication Date
WO2013153620A1 true WO2013153620A1 (fr) 2013-10-17

Family

ID=49327228

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2012/059789 WO2013153620A1 (fr) 2012-04-10 2012-04-10 Système de traitement de données et procédé de traitement de données

Country Status (1)

Country Link
WO (1) WO2013153620A1 (fr)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2016024612A (ja) * 2014-07-18 2016-02-08 富士通株式会社 データ処理制御方法、データ処理制御プログラムおよびデータ処理制御装置
JP2018515844A (ja) * 2015-05-04 2018-06-14 アリババ グループ ホウルディング リミテッド データ処理方法及びシステム

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH06139119A (ja) * 1992-10-28 1994-05-20 Hitachi Ltd データベース管理システムおよびデータベース管理方法
JPH06214843A (ja) * 1993-01-20 1994-08-05 Hitachi Ltd データベース管理システムおよび問合せの処理方法
JPH06259478A (ja) * 1993-03-02 1994-09-16 Toshiba Corp 分散データベースのデータ再配置方式
JPH07160557A (ja) * 1993-12-13 1995-06-23 Hitachi Ltd データベースアクセス処理方法

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH06139119A (ja) * 1992-10-28 1994-05-20 Hitachi Ltd データベース管理システムおよびデータベース管理方法
JPH06214843A (ja) * 1993-01-20 1994-08-05 Hitachi Ltd データベース管理システムおよび問合せの処理方法
JPH06259478A (ja) * 1993-03-02 1994-09-16 Toshiba Corp 分散データベースのデータ再配置方式
JPH07160557A (ja) * 1993-12-13 1995-06-23 Hitachi Ltd データベースアクセス処理方法

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2016024612A (ja) * 2014-07-18 2016-02-08 富士通株式会社 データ処理制御方法、データ処理制御プログラムおよびデータ処理制御装置
JP2018515844A (ja) * 2015-05-04 2018-06-14 アリババ グループ ホウルディング リミテッド データ処理方法及びシステム
US10872070B2 (en) 2015-05-04 2020-12-22 Advanced New Technologies Co., Ltd. Distributed data processing

Similar Documents

Publication Publication Date Title
WO2019114128A1 (fr) Procédé de traitement de bloc de transaction de chaîne de blocs, dispositif électronique et support de stockage lisible
KR101630749B1 (ko) 데이터센터 리소스 할당
US10108645B1 (en) Database monitoring for online migration optimization
US9405589B2 (en) System and method of optimization of in-memory data grid placement
JP5664098B2 (ja) 複合イベント分散装置、複合イベント分散方法および複合イベント分散プログラム
US8898422B2 (en) Workload-aware distributed data processing apparatus and method for processing large data based on hardware acceleration
US9965327B2 (en) Dynamically scalable data collection and analysis for target device
WO2019019644A1 (fr) Procédé et appareil d'attribution de serveur de poussée, et dispositif informatique et support de stockage
CN110532067A (zh) 事件处理方法、装置、设备及存储介质
WO2015043528A1 (fr) Procédé et dispositif de traitement parallèle de message de fils d'exécution multiples
US9596298B1 (en) Load balancing in a distributed processing system
JP2013536492A (ja) 複数のシステムを用いたデータ解析
WO2017028696A1 (fr) Procédé et dispositif pour surveiller la charge d'un système de stockage distribué
JP5245711B2 (ja) 分散データ処理システム、分散データ処理方法および分散データ処理用プログラム
CN105094981B (zh) 一种数据处理的方法及装置
JP6798564B2 (ja) リソース設定制御装置、リソース設定制御システム、リソース設定制御方法、および、リソース設定制御プログラム
CN108573029B (zh) 一种获取网络访问关系数据的方法、装置及存储介质
JP2014049129A (ja) 仮想マシンの負荷管理システム及びその方法
JPWO2016178316A1 (ja) 計算機調達予測装置、計算機調達予測方法、及び、プログラム
JP2017037492A (ja) 分散処理プログラム、分散処理方法および分散処理装置
TWI454949B (zh) 分散式資源管理系統及其分散式資源管理方法
JP2017535012A (ja) ソフトウェアの計測に基づいたリアルタイム報告
CN108153584B (zh) 一种用于确定目标网络设备需分配的线程数的方法与设备
WO2013153620A1 (fr) Système de traitement de données et procédé de traitement de données
CN103729417A (zh) 一种数据扫描的方法及装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 12873963

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 12873963

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: JP