CN103186427A - System and method for analyzing data record set - Google Patents
System and method for analyzing data record set Download PDFInfo
- Publication number
- CN103186427A CN103186427A CN2011104571304A CN201110457130A CN103186427A CN 103186427 A CN103186427 A CN 103186427A CN 2011104571304 A CN2011104571304 A CN 2011104571304A CN 201110457130 A CN201110457130 A CN 201110457130A CN 103186427 A CN103186427 A CN 103186427A
- Authority
- CN
- China
- Prior art keywords
- data
- data block
- unit
- analytic
- data recording
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
- G06F9/505—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals considering the load
Abstract
The invention discloses a system and a method for analyzing a data record set. The system comprises a data receiving module configured to receive the data record set, a static configuration module configured to divide the data record set into a plurality of data blocks each comprising one or more data records, a dynamic decision module configured to distribute the data records in the same data block to the same analysis unit according to the load situation of a processing module, and the processing module comprising a plurality of analysis units used for receiving and analyzing the data records distributed by the dynamic decision module.
Description
Technical field
The present invention relates to computer system distributed load equalizing technology, and the system and method for the analysis data record set that relates in particular to.
Background technology
In the analytic process to the data set of records ends, for example, about the data recording of the transaction of bank card, because the difference between the data recording quantity of the transaction of different bank card, tend to cause computer system distributed load equalizing difficulty, cause the computer system overall performance to descend.
Usually, the data recording of enormous amount can be carried out distributed analysis by a plurality of analytic units, but under some specific analysis situations, requires the ownership main body of every data recording to be analyzed by same analytic unit.For example, in the bank card business dealing analysis, require the transaction of same bank card in same analytic unit, to analyze continuously according to the order of sequence, just can analyze the trading activity track of this bank card.In this case, because the time that the transaction of every bank card takes place is at random, under the prerequisite that the ownership main body that requires every data recording is analyzed by same analytic unit, realize that the load balancing of a plurality of analytic units becomes difficult point.
And for example, when the fixed amount of analytic unit and the distribution situation of data recording to be analyzed when changing or when the quantity of analytic unit changes, computer system often is difficult to adapt to the load variations that each analytic unit produces, the feasible resource that is difficult to the associated computer system, thus cause the computer system overall performance to descend.Simultaneously, for the quantity that adapts to data recording distribution situation or analytic unit changes, need the artificial analyst coverage of adjusting each analytic unit could reach load balancing again usually.
The objective of the invention is to improve the internal performance of computer system, carry out a kind of internal system improvement in performance program-con-trolled computer internal system object by computing machine, make it possible to distribute neatly pending data record set, and then reach desirable load balancing state, thereby improve the handling property of computer system.
Summary of the invention
For addressing the above problem, the invention provides a kind of system that analyzes data record set, it is characterized in that this system comprises: data reception module is configured to receive data record set; The static configuration module is configured to described data record set is divided into a plurality of data blocks, and wherein each data block comprises one or more data recording; The dynamic decision module is configured to the loading condition according to processing module, and the data recording in the same data block is distributed to same analytic unit; Processing module comprises a plurality of analytic units, and these a plurality of analytic units are for the data recording that receives and analyze from the dynamic decision module assignment.
In described disclosed scheme, described static configuration module comprises: the attribute setup unit, and be configured as data recording and set one or more attributes, wherein, described one or more attributes are set according to the one or more fields in the data recording.The granularity dispensing unit, the one or more attributes that are configured to the setting of dependency setup unit are selected one or more particular communitys, wherein, the granularity that described one or more particular community is divided as data, the data division unit, the data recording that is configured to have the property value of identical particular community is divided into same data block, makes that the record in the described data record set is divided into a plurality of data blocks.
In described disclosed scheme, described dynamic decision module comprises: recognition unit, be configured to identify the affiliated data block of data recording to be analyzed, matching unit, be configured to a coupling in the data block under the data recording to be analyzed of described identification and the described a plurality of analytic unit, allocation units, be configured to described data recording to be analyzed distribute to its under the analytic unit of data block coupling.
In described disclosed scheme, described matching unit comprises that the matching relationship storage unit is used for the matching relationship of storage data block and analytic unit, described matching unit further be configured to according to matching relationship judge under the data recording to be analyzed data block whether with described a plurality of analytic units in a coupling, described allocation units further are configured to distribute described data recording to be analyzed according to matching relationship.
In described disclosed scheme, described matching unit further is configured to, data block under judging current data recording to be analyzed not with described a plurality of analytic units in a coupling time, with a coupling in this data block and the described a plurality of analytic unit, and store this matching relationship.
In described disclosed scheme, described matching unit is configured to described data block and described analytic unit are mated equably.
In described disclosed scheme, described matching unit is configured to, when the data block under this current data recording to be analyzed not with described a plurality of analytic units in a coupling time, in the described a plurality of analytic units of circulation selection one, and the analytic unit that will select according to the order of sequence and this data block coupling.
In described disclosed scheme, described data division unit is each data block setting data block identifier, wherein said data block identifier is recorded in the corresponding data recording as adding field, and described judging unit is judged data block under the data recording to be analyzed according to data block identifier.
In described disclosed scheme, described processing module also comprises feedback unit, is used for the status information to described dynamic decision module feedback analysis unit, and described status information comprises the quantity of analytic unit.Described dynamic decision module also comprises state storage unit, and state storage unit is configured to the inventory analysis element number, and data block and described a plurality of analytic unit that described matching unit will mate according to the quantity of analytic unit mate
In described disclosed scheme, described state storage unit further is configured to described analytic unit and stores corresponding natural number identifier according to the order of sequence, described data block identifier is the natural number that distributes in order, described matching unit by based on the natural number identifier of the corresponding analytic unit of remainder that the natural number of data block identifier correspondence is obtained divided by the quantity of described analytic unit with not with the data recording current to be analyzed of analytic unit coupling under data block mate with this corresponding analytic unit.
In described disclosed scheme, when the quantity of the analytic unit of described feedback unit feedback changed, described matching unit was configured to described data block and described analytic unit are mated again.
According to another object of the present invention, a kind of method of analyzing data record set is provided, it is characterized in that this method comprises: DRP data reception process receives data record set in this process; The static configuration process is divided into a plurality of data blocks with described data record set in this process, wherein each data block comprises one or more data recording; The dynamic decision process according to the loading condition of processing procedure, is distributed to same analytic process with the data recording in the same data block in this process; Processing procedure comprises the process of utilizing a plurality of analytic units to analyze, receives and analyze the data recording of distributing in the dynamic decision process in the analytic unit.
In described disclosed scheme, described static configuration process comprises: the attribute assignment procedure, in this process, set one or more attributes for data recording, and wherein, described one or more attributes are set according to the one or more fields in the data recording.The granularity layoutprocedure, one or more attributes that the dependency assignment procedure is set in this process are selected one or more particular communitys, wherein, the granularity that described one or more particular community is divided as data, the data partition process, the data recording that will have the property value of identical particular community in this process is divided into same data block, makes that the record in the described data record set is divided into a plurality of data blocks.
In described disclosed scheme, described dynamic decision process comprises: identifying, data block in this process under the identification data recording to be analyzed, matching process, in this process with a coupling in the data block under the data recording to be analyzed of described identification and the described a plurality of analytic unit, assigning process, the analytic unit of the data block coupling under in this process, distributing to described data recording to be analyzed with it.
In described disclosed scheme, described matching process comprises that the matching relationship storing process is used for the matching relationship of storage data block and analytic unit, described matching process further comprise according to matching relationship judge under the data recording to be analyzed data block whether with described a plurality of analytic units in a coupling, described assigning process further comprises according to matching relationship and distributes described data recording to be analyzed.
In described disclosed scheme, described matching process further comprises, data block under judging current data recording to be analyzed not with described a plurality of analytic units in a coupling time, with a coupling in this data block and the described a plurality of analytic unit, and store this matching relationship.
In described disclosed scheme, describedly in matching process, described data block and described analytic unit are mated equably.
In described disclosed scheme, in described matching process, when the data block under this current data recording to be analyzed not with described a plurality of analytic units in a coupling time, in the described a plurality of analytic units of circulation selection one, and the analytic unit that will select according to the order of sequence and this data block coupling.
In described disclosed scheme, described data partition process is each data block setting data block identifier, wherein said data block identifier is recorded in the corresponding data recording as adding field, and described deterministic process is judged data block under the data recording to be analyzed according to data block identifier.
In described disclosed scheme, described processing procedure also comprises feedback procedure, is used for the status information to described dynamic decision process feedback analysis unit, and described status information comprises the quantity of analytic unit.Described dynamic decision process also comprises the state storing process, and the state storing process is the inventory analysis element number in this process, and data block and described a plurality of analytic unit that described matching process will mate according to the quantity of analytic unit mate.
In described disclosed scheme, described state storing process is further stored corresponding natural number identifier according to the order of sequence at analytic unit described in this process, described data block identifier is the natural number that distributes in order, described matching process by based on the natural number identifier of the corresponding analytic unit of remainder that the natural number of data block identifier correspondence is obtained divided by the quantity of described analytic unit with not with the data recording current to be analyzed of analytic unit coupling under data block mate with this corresponding analytic unit.
In described disclosed scheme, when the quantity of the analytic unit of described feedback procedure feedback changed, described matching process mated again to described data block and described analytic unit in this process.
The objective of the invention is to improve the inside computer system performance, carry out a kind of internal system improvement in performance program-con-trolled computer internal system object by computing machine, finish a series of settings or adjustment that each ingredient of this computer system is implemented, thereby obtain the effect of inside computer system improvement in performance.
Description of drawings
After the reference accompanying drawing had been read the specific embodiment of the present invention, those skilled in the art will become apparent various aspects of the present invention.One skilled in the art will appreciate that these accompanying drawings only are used for cooperating embodiment that technical scheme of the present invention is described, and be not to be intended to protection scope of the present invention is construed as limiting.
Fig. 1 is the composition synoptic diagram of analyzing the system of data record set according to an embodiment of the invention.
Fig. 2 is the flow chart of steps according to the method for the analysis data record set of the embodiment of the invention.
Embodiment
With reference to the accompanying drawings, the specific embodiment of the present invention is described in further detail.
Fig. 1 is the composition synoptic diagram according to the system of the analysis data record set of the embodiment of the invention.As shown in Figure 1, the system of analysis data record set comprises data reception module, static configuration module, dynamic decision module, processing module.Wherein, data reception module is configured to receive data record set, and this data record set comprises one or more data recording.Data reception module is sent to the static configuration module with the data record set that receives.The static configuration module is configured to described data record set is carried out pre-service, is about to described data record set and is divided into a plurality of data blocks, and wherein each data block comprises one or more data recording.The dynamic decision module receives by the data recording of piecemeal from the static configuration module, and is configured to the loading condition according to processing module, and the data recording in the same data block is distributed to same analytic unit in the processing module.Processing module comprises a plurality of analytic units, and these a plurality of analytic units are for the data recording that receives and analyze from the dynamic decision module assignment.Because can utilize the static configuration module that the data set of records ends is carried out piecemeal handles, therefore, even when the distribution situation of data recording changes, also can change the branch block mode of data record set by the static configuration module neatly, make new branch block mode can adapt to the variation of the distribution situation of data recording, thus the feasible load balancing of handling the analytic unit of each database.Dynamic decision module of the present invention can dynamically be adjusted the analysis task of distributing to analytic unit according to the loading condition of processing module, even therefore the quantity when analytic unit changes, computer system also can adapt to the load variations that each analytic unit produces.
In the above-described embodiments, the static configuration module comprises attribute setup unit, granularity dispensing unit, data division unit alternatively, as shown in Figure 1.Wherein, the attribute setup unit is configured to data recording and sets one or more attributes, and wherein, described one or more attributes are set according to the one or more fields in the data recording.Exemplarily, for the situation that bank card business dealing is analyzed, can be with an end attribute as data recording of the bank's card number in the transaction data record, credit card issuer, bank card character, bank card number.The granularity dispensing unit is configured to one or more attributes of dependency setup unit setting and selects one or more particular communitys, wherein, and the granularity that described one or more particular communitys are divided as data.For above-mentioned example, for example, can be with two selections in end of credit card issuer, bank card character, bank card number as particular community.The data division unit, the data recording that is configured to have the property value of identical particular community is divided into same data block, makes that the record in the described data record set is divided into a plurality of data blocks.For above-mentioned example, for example, hair fastener behavior 200 tame banks, bank card character are that 3 kinds, the end position of bank card number are 10 kinds, and the record in the data record set can be divided into 600(200 * 3 * 10 so) individual data block.What it will be appreciated by those skilled in the art that is, distribution situation that can the single data record changes to define attribute and the particular community of data recording.
In the above-described embodiments, the dynamic decision module comprises recognition unit, matching unit, allocation units alternatively, as shown in Figure 1.Wherein, recognition unit is configured to identify the affiliated data block of data recording to be analyzed.Matching unit is configured to a coupling in the data block under the data recording to be analyzed of described identification and the described a plurality of analytic unit.Here, the data recording in the data block will be assigned to corresponding analytic unit after the coupling.After coupling is finished, allocation units be configured to described data recording to be analyzed distribute to its under the analytic unit of data block coupling.
In the above-described embodiments, matching unit can also comprise the matching relationship storage unit, and the matching relationship storage unit is used for the matching relationship of storage data block and analytic unit.Be understandable that data block and analytic unit are man-to-man corresponding relations.Hereinafter will describe, this matching relationship can change according to the status information of the analytic unit in the processing module, namely mates again.Matching unit further be configured to according to matching relationship judge under the data recording to be analyzed data block whether with described a plurality of analytic units in a coupling.Allocation units further are configured to distribute described data recording to be analyzed according to matching relationship.
In the above-described embodiments, matching unit further is configured to, data block under judging current data recording to be analyzed not with described a plurality of analytic units in a coupling time, with a coupling in this data block and the described a plurality of analytic unit, and store this matching relationship.
In the above-described embodiments, matching unit is configured to described data block and described analytic unit are mated equably.
In the above-described embodiments, matching unit is configured to, when the data block under this current data recording to be analyzed not with described a plurality of analytic units in a coupling time, in described a plurality of analytic units one is selected in circulation according to the order of sequence, and the analytic unit that will select and this data block are mated.For instance, when P analytic unit, data record set is divided into N data block in static configuration, matching unit can be with the data block under article one data recording that reads and first analytic unit coupling in this case, thereby giving this first analytic unit by allocation units with this article one data recording analyzes, when second data recording that matching unit reads, if this second data recording and this article one data recording belong to identical data block, allocation units will this second data recording be given this first analytic unit and are analyzed, otherwise matching unit is with the data block under the second data recording and next analytic unit coupling, so circulation, data recording can be distributed to all P analysis process equably.More specifically, for instance, matching unit can be considered as round-robin queue with described a plurality of analytic units, when the data block under judging current data recording to be analyzed and the coupling in described a plurality of analytic unit, the pointer of the described analytic unit of the mobile sensing of described matching unit round-robin queue, data block under judging current data recording to be analyzed not with described a plurality of analytic units in coupling the time, described matching unit is with the next analytic unit of the described analytic unit of described pointed round-robin queue.
In the above-described embodiments, the data division unit is each data block setting data block identifier, wherein said data block identifier is recorded in the corresponding data recording as adding field, and judging unit is judged data block under the data recording to be analyzed according to data block identifier.
In the above-described embodiments, described processing module also comprises feedback unit alternatively, and described dynamic decision module also comprises state storage unit alternatively, as shown in Figure 1.Wherein, feedback unit is used for the status information to described dynamic decision module feedback analysis unit, and described status information comprises the quantity of analytic unit.State storage unit is configured to the inventory analysis element number.Data block and described a plurality of analytic unit that described matching unit will mate according to the quantity of analytic unit mate.
In the above-described embodiments, described state storage unit further is configured to described analytic unit is stored corresponding natural number identifier according to the order of sequence, and described data block identifier is the natural number that distributes in order.In this case, matching unit can by based on the natural number identifier of the corresponding analytic unit of remainder that the natural number of data block identifier correspondence is obtained divided by the quantity of described analytic unit with not with the data recording current to be analyzed of analytic unit coupling under data block mate with this corresponding analytic unit.For instance, for example when P analytic unit, data record set is divided into N data block in static configuration, and n data block will be mated with (N%P) individual analytic unit so.
In the above-described embodiments, when the quantity of the analytic unit of described feedback unit feedback changed, described matching unit was configured to described data block and described analytic unit are mated again.Can utilize the process that disclosed method is mated again in above-described embodiment.
Fig. 2 is the flow chart of steps according to the method for the analysis data record set of the embodiment of the invention.According to another object of the present invention, a kind of method of analyzing data record set has also been proposed, this method comprises as shown in Figure 2: DRP data reception process receives data record set in this process; The static configuration process is divided into a plurality of data blocks with described data record set in this process, wherein each data block comprises one or more data recording; The dynamic decision process according to the loading condition of processing procedure, is distributed to same analytic process with the data recording in the same data block in this process; Processing procedure comprises the process of utilizing a plurality of analytic units to analyze, receives and analyze the data recording of distributing in the dynamic decision process in the analytic unit.
In the above-described embodiments, described static configuration process comprises: the attribute assignment procedure, in this process, set one or more attributes for data recording, and wherein, described one or more attributes are set according to the one or more fields in the data recording.The granularity layoutprocedure, one or more attributes that the dependency assignment procedure is set in this process are selected one or more particular communitys, wherein, the granularity that described one or more particular community is divided as data, the data partition process, the data recording that will have the property value of identical particular community in this process is divided into same data block, makes that the record in the described data record set is divided into a plurality of data blocks.
In the above-described embodiments, described dynamic decision process comprises: identifying, data block in this process under the identification data recording to be analyzed, matching process, in this process with a coupling in the data block under the data recording to be analyzed of described identification and the described a plurality of analytic unit, assigning process, the analytic unit of the data block coupling under in this process, distributing to described data recording to be analyzed with it.
In the above-described embodiments, described matching process comprises that the matching relationship storing process is used for the matching relationship of storage data block and analytic unit, described matching process further comprise according to matching relationship judge under the data recording to be analyzed data block whether with described a plurality of analytic units in a coupling, described assigning process further comprises according to matching relationship and distributes described data recording to be analyzed.
In the above-described embodiments, described matching process further comprises, data block under judging current data recording to be analyzed not with described a plurality of analytic units in a coupling time, with a coupling in this data block and the described a plurality of analytic unit, and store this matching relationship.
In the above-described embodiments, describedly in matching process, described data block and described analytic unit are mated equably.
In the above-described embodiments, in described matching process, when the data block under this current data recording to be analyzed not with described a plurality of analytic units in a coupling time, in described a plurality of analytic units one is selected in circulation according to the order of sequence, and the analytic unit that will select and this data block are mated.
In the above-described embodiments, described data partition process is each data block setting data block identifier, wherein said data block identifier is recorded in the corresponding data recording as adding field, and described deterministic process is judged data block under the data recording to be analyzed according to data block identifier.
In the above-described embodiments, described processing procedure also comprises feedback procedure, is used for the status information to described dynamic decision process feedback analysis unit, and described status information comprises the quantity of analytic unit.Described dynamic decision process also comprises the state storing process, and the state storing process is the inventory analysis element number in this process, and data block and described a plurality of analytic unit that described matching process will mate according to the quantity of analytic unit mate.
In the above-described embodiments, described state storing process is further stored corresponding natural number identifier according to the order of sequence at analytic unit described in this process, described data block identifier is the natural number that distributes in order, described matching process by based on the natural number identifier of the corresponding analytic unit of remainder that the natural number of data block identifier correspondence is obtained divided by the quantity of described analytic unit with not with the data recording current to be analyzed of analytic unit coupling under data block mate with this corresponding analytic unit.
In the above-described embodiments, when the quantity of the analytic unit of described feedback procedure feedback changed, described matching process mated again to described data block and described analytic unit in this process.
By the description of above embodiment, those skilled in the art can be well understood to the present invention and can realize by software and/or hardware mode.Those skilled in the art can understand, and under situation without departing from the spirit and scope of the present invention, can also do various changes and replacement to the specific embodiment of the present invention.These changes and replace all drop in claims of the present invention institute restricted portion.
Claims (22)
1. a system that analyzes data record set is characterized in that, this system comprises:
Data reception module is configured to receive data record set;
The static configuration module is configured to described data record set is divided into a plurality of data blocks, and wherein each data block comprises one or more data recording;
The dynamic decision module is configured to the loading condition according to processing module, and the data recording in the same data block is distributed to same analytic unit;
Processing module comprises a plurality of analytic units, and these a plurality of analytic units are for the data recording that receives and analyze from the dynamic decision module assignment.
2. the system as claimed in claim 1 is characterized in that, described static configuration module comprises:
The attribute setup unit is configured as data recording and sets one or more attributes, and wherein, described one or more attributes are set according to the one or more fields in the data recording,
The granularity dispensing unit, the one or more attributes that are configured to the setting of dependency setup unit are selected one or more particular communitys, wherein, the granularity that described one or more particular communitys are divided as data,
The data division unit, the data recording that is configured to have the property value of identical particular community is divided into same data block, makes that the record in the described data record set is divided into a plurality of data blocks.
3. system as claimed in claim 2 is characterized in that, described dynamic decision module comprises:
Recognition unit is configured to identify the affiliated data block of data recording to be analyzed,
Matching unit is configured to a coupling in the data block under the data recording to be analyzed of described identification and the described a plurality of analytic unit,
Allocation units, be configured to described data recording to be analyzed distribute to its under the analytic unit of data block coupling.
4. system as claimed in claim 3 is characterized in that,
Described matching unit comprises that the matching relationship storage unit is used for the matching relationship of storage data block and analytic unit,
Described matching unit further be configured to according to matching relationship judge under the data recording to be analyzed data block whether with described a plurality of analytic units in a coupling,
Described allocation units further are configured to distribute described data recording to be analyzed according to matching relationship.
5. system as claimed in claim 4, it is characterized in that, described matching unit further is configured to, data block under judging current data recording to be analyzed not with described a plurality of analytic units in a coupling time, with a coupling in this data block and the described a plurality of analytic unit, and store this matching relationship.
6. system as claimed in claim 5 is characterized in that, described matching unit is configured to described data block and described analytic unit are mated equably.
7. system as claimed in claim 6, it is characterized in that, described matching unit is configured to, when the data block under this current data recording to be analyzed not with described a plurality of analytic units in a coupling time, in the described a plurality of analytic units of circulation selection one, and the analytic unit that will select according to the order of sequence and this data block coupling.
8. system as claimed in claim 7 is characterized in that,
Described data division unit is each data block setting data block identifier, and wherein said data block identifier is recorded in the corresponding data recording as adding field,
Described judging unit is judged data block under the data recording to be analyzed according to data block identifier.
9. system as claimed in claim 8 is characterized in that,
Described processing module also comprises feedback unit, is used for the status information to described dynamic decision module feedback analysis unit, and described status information comprises the quantity of analytic unit,
Described dynamic decision module also comprises state storage unit,
State storage unit is configured to the inventory analysis element number,
Data block and described a plurality of analytic unit that described matching unit will mate according to the quantity of analytic unit mate.
10. system as claimed in claim 9 is characterized in that,
Described state storage unit further is configured to described analytic unit and stores corresponding natural number identifier according to the order of sequence,
Described data block identifier is the natural number that distributes in order,
Described matching unit by based on the natural number identifier of the corresponding analytic unit of remainder that the natural number of data block identifier correspondence is obtained divided by the quantity of described analytic unit with not with the data recording current to be analyzed of analytic unit coupling under data block mate with this corresponding analytic unit.
11. system as claimed in claim 10 is characterized in that, when the quantity of the analytic unit of described feedback unit feedback changed, described matching unit was configured to described data block and described analytic unit are mated again.
12. a method of analyzing data record set is characterized in that, this method comprises:
DRP data reception process receives data record set in this process;
The static configuration process is divided into a plurality of data blocks with described data record set in this process, wherein each data block comprises one or more data recording;
The dynamic decision process according to the loading condition of processing procedure, is distributed to same analytic process with the data recording in the same data block in this process;
Processing procedure comprises the process of utilizing a plurality of analytic units to analyze, receives and analyze the data recording of distributing in the dynamic decision process in the analytic unit.
13. method as claimed in claim 12 is characterized in that, described static configuration process comprises:
The attribute assignment procedure is set one or more attributes for data recording in this process, wherein, described one or more attributes are set according to the one or more fields in the data recording,
The granularity layoutprocedure, one or more attributes that the dependency assignment procedure is set in this process are selected one or more particular communitys, wherein, the granularity that described one or more particular communitys are divided as data,
The data partition process, the data recording that will have the property value of identical particular community in this process is divided into same data block, makes that the record in the described data record set is divided into a plurality of data blocks.
14. method as claimed in claim 13 is characterized in that, described dynamic decision process comprises:
Identifying, the data block in this process under the identification data recording to be analyzed,
Matching process, in this process with a coupling in the data block under the data recording to be analyzed of described identification and the described a plurality of analytic unit,
Assigning process, the analytic unit of the data block coupling under in this process, distributing to described data recording to be analyzed with it.
15. method as claimed in claim 14 is characterized in that,
Described matching process comprises that the matching relationship storing process is used for the matching relationship of storage data block and analytic unit,
Described matching process further comprise according to matching relationship judge under the data recording to be analyzed data block whether with described a plurality of analytic units in a coupling,
Described assigning process further comprises according to matching relationship and distributes described data recording to be analyzed.
16. method as claimed in claim 15, it is characterized in that, described matching process further comprises, data block under judging current data recording to be analyzed not with described a plurality of analytic units in a coupling time, with a coupling in this data block and the described a plurality of analytic unit, and store this matching relationship.
17. method as claimed in claim 16 is characterized in that, describedly in matching process described data block and described analytic unit is mated equably.
18. method as claimed in claim 17, it is characterized in that, in described matching process, when the data block under this current data recording to be analyzed not with described a plurality of analytic units in a coupling time, in the described a plurality of analytic units of circulation selection one, and the analytic unit that will select according to the order of sequence and this data block coupling.
19. method as claimed in claim 18 is characterized in that,
Described data partition process is each data block setting data block identifier, and wherein said data block identifier is recorded in the corresponding data recording as adding field,
Described deterministic process is judged data block under the data recording to be analyzed according to data block identifier.
20. method as claimed in claim 19 is characterized in that,
Described processing procedure also comprises feedback procedure, is used for the status information to described dynamic decision process feedback analysis unit, and described status information comprises the quantity of analytic unit,
Described dynamic decision process also comprises the state storing process,
The state storing process is the inventory analysis element number in this process,
Data block and described a plurality of analytic unit that described matching process will mate according to the quantity of analytic unit mate.
21. method as claimed in claim 20 is characterized in that,
Described state storing process is further stored corresponding natural number identifier according to the order of sequence at analytic unit described in this process,
Described data block identifier is the natural number that distributes in order,
Described matching process by based on the natural number identifier of the corresponding analytic unit of remainder that the natural number of data block identifier correspondence is obtained divided by the quantity of described analytic unit with not with the data recording current to be analyzed of analytic unit coupling under data block mate with this corresponding analytic unit.
22. method as claimed in claim 21 is characterized in that, when the quantity of the analytic unit of described feedback procedure feedback changed, described matching process mated again to described data block and described analytic unit in this process.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201110457130.4A CN103186427B (en) | 2011-12-31 | 2011-12-31 | A kind of system and method for analytical data set of records ends |
| PCT/CN2012/087722 WO2013097751A1 (en) | 2011-12-31 | 2012-12-27 | System and method for analyzing data record set |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201110457130.4A CN103186427B (en) | 2011-12-31 | 2011-12-31 | A kind of system and method for analytical data set of records ends |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN103186427A true CN103186427A (en) | 2013-07-03 |
| CN103186427B CN103186427B (en) | 2016-09-07 |
Family
ID=48677605
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201110457130.4A Active CN103186427B (en) | 2011-12-31 | 2011-12-31 | A kind of system and method for analytical data set of records ends |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN103186427B (en) |
| WO (1) | WO2013097751A1 (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103473371A (en) * | 2013-09-29 | 2013-12-25 | 方正国际软件有限公司 | Pushing system and method of fetch data |
| CN103530334A (en) * | 2013-09-29 | 2014-01-22 | 方正国际软件有限公司 | System and method for data matching based on comparison module |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1573754A (en) * | 2003-06-24 | 2005-02-02 | 微软公司 | System and method for online analytical processing using dimension attributes and multiple hierarchies per dimension |
| CN101197876A (en) * | 2006-12-06 | 2008-06-11 | 中兴通讯股份有限公司 | Method and system for multi-dimensional analysis of message service data |
| US20100010968A1 (en) * | 2008-07-10 | 2010-01-14 | Redlich Ron M | System and method to identify, classify and monetize information as an intangible asset and a production model based thereon |
-
2011
- 2011-12-31 CN CN201110457130.4A patent/CN103186427B/en active Active
-
2012
- 2012-12-27 WO PCT/CN2012/087722 patent/WO2013097751A1/en active Application Filing
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1573754A (en) * | 2003-06-24 | 2005-02-02 | 微软公司 | System and method for online analytical processing using dimension attributes and multiple hierarchies per dimension |
| CN101197876A (en) * | 2006-12-06 | 2008-06-11 | 中兴通讯股份有限公司 | Method and system for multi-dimensional analysis of message service data |
| US20100010968A1 (en) * | 2008-07-10 | 2010-01-14 | Redlich Ron M | System and method to identify, classify and monetize information as an intangible asset and a production model based thereon |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103473371A (en) * | 2013-09-29 | 2013-12-25 | 方正国际软件有限公司 | Pushing system and method of fetch data |
| CN103530334A (en) * | 2013-09-29 | 2014-01-22 | 方正国际软件有限公司 | System and method for data matching based on comparison module |
| CN103530334B (en) * | 2013-09-29 | 2018-01-23 | 北大医疗信息技术有限公司 | Based on the data matching system and method for comparing template |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2013097751A1 (en) | 2013-07-04 |
| CN103186427B (en) | 2016-09-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Katok et al. | Lot sizing in general assembly systems with setup costs, setup times, and multiple constrained resources | |
| US11321722B2 (en) | Assortment optimization using incremental swapping with demand transference | |
| Dörmer et al. | Master production scheduling and sequencing at mixed-model assembly lines in the automotive industry | |
| Rusmevichientong et al. | Assortment optimization with mixtures of logits | |
| US20170011292A1 (en) | Systems and Methods for Multi-Objective Evolutionary Algorithms with Category Discovery | |
| CN108255838B (en) | Method and system for establishing intermediate data warehouse for big data analysis | |
| CN101308560A (en) | Store management system and program | |
| CN107038521A (en) | Intelligent scheduling and work item distribution | |
| CN109558421A (en) | Data processing method, system, device and storage medium based on caching | |
| US8286175B2 (en) | Method and apparatus for capacity optimization and planning in an on-demand computing environment | |
| CN109767070A (en) | Case distribution method, device, computer equipment and storage medium | |
| US20180365605A1 (en) | Supply chain simulation system and supply chain simulation method | |
| Arbabian et al. | Capacity expansions with bundled supplies of attributes: An application to server procurement in cloud computing | |
| CN103186427A (en) | System and method for analyzing data record set | |
| CN106933907A (en) | The processing method and processing device of tables of data extended counter | |
| CN105701602A (en) | Resource distribution method and device | |
| US20210216573A1 (en) | Algorithm to apply a predicate to data sets | |
| CN109033248A (en) | The method and apparatus of storing data record, the method and apparatus for inquiring data record | |
| CN111796934B (en) | Task issuing method and device, storage medium and electronic equipment | |
| CN112990709A (en) | Approval task allocation method, device, equipment and storage medium | |
| Chotayakul et al. | The chance-constrained programming for the lot-sizing problem with stochastic demand on parallel machines | |
| Cheng et al. | A two stage push-pull production planning model | |
| CN110648090A (en) | Logistics process node management method and system | |
| Balkhi et al. | Improving inventory control for SABIC products | |
| de Souza Amaro et al. | Evaluating process flexibility in lot sizing problems: an approach based on multicriteria decision making |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant |