CN106936899A - The collocation method of distributed statistical analysis system and distributed statistical analysis system - Google Patents
The collocation method of distributed statistical analysis system and distributed statistical analysis system Download PDFInfo
- Publication number
- CN106936899A CN106936899A CN201710105031.7A CN201710105031A CN106936899A CN 106936899 A CN106936899 A CN 106936899A CN 201710105031 A CN201710105031 A CN 201710105031A CN 106936899 A CN106936899 A CN 106936899A
- Authority
- CN
- China
- Prior art keywords
- node
- data
- leader
- statistical analysis
- shard
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000007619 statistical method Methods 0.000 title claims abstract description 78
- 238000000034 method Methods 0.000 title claims abstract description 49
- 238000013467 fragmentation Methods 0.000 claims abstract description 48
- 238000006062 fragmentation reaction Methods 0.000 claims abstract description 48
- 239000012634 fragment Substances 0.000 claims abstract description 6
- 230000005012 migration Effects 0.000 claims description 18
- 238000013508 migration Methods 0.000 claims description 18
- 238000009826 distribution Methods 0.000 claims description 16
- 238000007726 management method Methods 0.000 claims description 14
- 238000004364 calculation method Methods 0.000 claims description 10
- 230000004913 activation Effects 0.000 claims description 6
- 238000013500 data storage Methods 0.000 description 11
- 230000008569 process Effects 0.000 description 10
- 238000004458 analytical method Methods 0.000 description 8
- 230000006870 function Effects 0.000 description 8
- 230000008859 change Effects 0.000 description 6
- 238000012546 transfer Methods 0.000 description 6
- 238000013507 mapping Methods 0.000 description 5
- 239000012792 core layer Substances 0.000 description 4
- 230000006872 improvement Effects 0.000 description 4
- 239000010410 layer Substances 0.000 description 4
- 238000012423 maintenance Methods 0.000 description 4
- 230000004048 modification Effects 0.000 description 4
- 238000012986 modification Methods 0.000 description 4
- 238000013461 design Methods 0.000 description 3
- 230000003362 replicative effect Effects 0.000 description 3
- 238000000354 decomposition reaction Methods 0.000 description 2
- 238000005194 fractionation Methods 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 241000208340 Araliaceae Species 0.000 description 1
- 241001269238 Data Species 0.000 description 1
- 235000005035 Panax pseudoginseng ssp. pseudoginseng Nutrition 0.000 description 1
- 235000003140 Panax quinquefolius Nutrition 0.000 description 1
- 230000009471 action Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000012217 deletion Methods 0.000 description 1
- 230000037430 deletion Effects 0.000 description 1
- 235000013399 edible fruits Nutrition 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 235000008434 ginseng Nutrition 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 230000008520 organization Effects 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
- 239000002699 waste material Substances 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1001—Protocols in which an application is distributed across nodes in the network for accessing one among a plurality of replicated servers
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1001—Protocols in which an application is distributed across nodes in the network for accessing one among a plurality of replicated servers
- H04L67/1034—Reaction to server failures by a load balancer
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a kind of collocation method of distributed statistical analysis system, the distributed statistical analysis system includes ZooKeeper clusters, service node and calculate node cluster, and methods described includes：Leader node in election calculate node cluster, according to data fragmentation principle in each calculate node fragment data, and election leadership's burst in the copy of data fragmentation；After service node receives statistical analysis request, to leader node application calculate node, the minimum calculate node of task amount is fed back to service node by leader node；After service node obtains the calculate node of feedback, statistics request is sent to the calculate node；Calculate node is searched leader's burst and the data fragmentation copy of free time is obtained to leader's burst application, and statistics task is distributed into data fragmentation copy execution statistics task.Present invention also offers the distributed statistical analysis system based on above-mentioned collocation method.
Description
Technical field
The present invention relates to a kind of collocation method of distributed statistical analysis system, particularly a kind of configuration cluster management, number
According to storage and the collocation method of migration, Statistic Query and statistics task load balancing function, and obtained by the collocation method
The distributed statistical analysis system for arriving.
Background technology
Distributed system is a kind of computer system interconnected by multiple process resources.These process resources
Node device is properly termed as again, gets off to perform same personage in unified control.As Chinese patent CN102497280 is disclosed
A kind of distributed system, it can realize the mutual perception between multiple equipment node.Improve the efficiency of management.But it is not
There is the open management and configuration that task is specifically performed to each device node.
Distributed system generally needs to have the function of statistical analysis.But in current distributed statistical system not
Practical operation situation according to process resource is come the distribution statistical analysis task between device node.Therefore process resource can occur
Situation about being not efficiently used.
The content of the invention
The distribution that the present invention provides a kind of collocation method of distributed statistical analysis system and obtained by the collocation method
Statistical analysis system.The high availability and enhanced scalability of guarantee system, and realize the load balancing of statistical analysis task.
To solve the above problems, in a first aspect, the present invention provides a kind of collocation method of distributed statistical analysis system, institute
Stating distributed statistical analysis system includes ZooKeeper clusters, service node and calculate node cluster, and methods described includes：
Election calculate node cluster in leader node, according to data fragmentation principle in each calculate node fragment data, and
Election leadership's burst in the copy of data fragmentation；
After service node receives statistical analysis request, to leader node application calculate node, leader node is by task amount minimum
Calculate node feeds back to service node；
After service node obtains the calculate node of feedback, statistics request is sent to the calculate node；
Calculate node is searched leader's burst and the data fragmentation copy of free time is obtained to leader's burst application, and statistics task is divided
Dispensing data slice copy execution statistics task.
Alternatively and preferably, methods described also includes：
To leader node, leader node finds all copies of corresponding data fragmentation to the data is activation that service node will update
And by each copy of data syn-chronization to the data fragmentation.
Alternatively and preferably, the data syn-chronization uses two-phase commitment protocol, if all successes, return successfully,
If any one copy fails, failure is retracted and returned.
Alternatively and preferably, methods described also includes：In the case of adjustment data fragmentation number or burst rule,
The relation of the data that remap and data fragmentation.
Alternatively and preferably, when methods described is additionally included in equipment state and changes, distributed statistical system is entered
The step of row adjustment, the adjustment includes following at least one：
When equipment fault is detected, the number and distribution situation of each shard copies are checked, if the number of each shard copies is poor
It is not larger, then to shard carry out be distributed adjustment and migration, with ensure each shard copies number it is essentially identical；And
When new equipment addition is detected, the number and distribution situation of each shard copies are checked, if the number of the copy of shard
It is variant, then the few shard of copy number is copied into new equipment, if number is identical, re-start burst.
Alternatively and preferably, the calculate node is searched leader's burst and the data of free time is obtained to leader's burst application
Burst copy, and by statistics task distribute to the data slice copy perform statistics task include：
Assignment statisticses are split into multiple tasks by the calculate node for performing calculating task, and will need system according to burst rule
The ID lists of meter are grouped, and task and ID groups are sent to corresponding leader's burst by calculate node, and choosing is responsible for by leader's burst
Choose friends and task is performed by load less data burst copy, after all of tasks carrying is finished, perform node and collect all
Result of calculation is simultaneously merged to result, then returns to service node.
Alternatively and preferably, if assignment statisticses need repeatedly to access data, knot is being collected and combined by execution node
After fruit, the task that carried out again according to assignment statisticses demand splits and result merges, untill assignment statisticses operation is finished.
Second aspect, the present invention provides a kind of distributed statistical analysis system, including ZooKeeper clusters, service node
With calculate node cluster.Also include：
Cluster management module, for electing the leader node in calculate node cluster, section is calculated according to data fragmentation principle each
Fragment data in point, and election leadership's burst in the copy of data fragmentation；
Statistical analysis module, after receiving statistical analysis request for service node, to leader node application calculate node, Ling Daojie
The minimum calculate node of task amount is fed back to service node by point；After service node obtains the calculate node of feedback, please by statistics
Ask and be sent to the calculate node；Calculate node is searched leader's burst and the data fragmentation pair of free time is obtained to leader's burst application
This, and statistics task is distributed into data slice copy execution statistics task.
Alternatively and preferably, also include：
Data memory module, for service node will update data is activation to leader node, leader node finds corresponding
All copies of data fragmentation and by each copy of data syn-chronization to the data fragmentation.
Alternatively and preferably, the data syn-chronization uses two-phase commitment protocol, if all successes, return successfully,
If any one copy fails, failure is retracted and returned.
The collocation method of the distributed statistical system that the present invention is provided and distributed statistical analysis system, can be by task point
Load, and preferentially distribute to the less calculate node of load and data districting operation so that the process resource of relative free can be certainly
It is dynamic to be fully used, statistical efficiency is improve, and ensure that system has high scalability and high stability.
Brief description of the drawings
The Organization Chart of the distributed statistical analysis system that Fig. 1 is provided by the embodiment of the present invention one；
The collocation method flow chart of the distributed statistical analysis system that Fig. 2 is provided by the embodiment of the present invention one；
Another flow chart of the collocation method of the distributed statistical analysis system that Fig. 3 is provided by the embodiment of the present invention one；
Another flow chart of the collocation method of the distributed statistical analysis system that Fig. 4 is provided by the embodiment of the present invention one；
Another flow chart of the collocation method of the distributed statistical analysis system that Fig. 5 is provided by the embodiment of the present invention one；
The structural representation of the distributed statistical analysis system that Fig. 6 is provided by the embodiment of the present invention two.
Specific embodiment
In order that those skilled in the art more fully understand the present invention program, with reference to the accompanying drawings and detailed description
The present invention is described in further detail.
Embodiment one
A kind of collocation method of distributed statistical analysis system is present embodiments provided, wherein, as shown in figure 1, the distribution is united
Score analysis system is mainly made up of three parts：ZooKeeper clusters, service node and calculate node cluster.ZooKeeper clusters
For the condition managing of calculate node cluster.Service node is responsible for the decomposition and integration of statistical fractals and the renewal control of data
Deng work.The statistical analysis request of service node receiving front-end, and content search and statistical analysis two parts are resolved to, content is searched
The task of rope part transfers to content searching engine to run, and the task of statistical analysis part transfers to calculate node cluster to run.Calculate
Node cluster is used for the work such as burst and backup, the fractionation of calculating task and result merging, the load of calculating task of data.
Term is explained：Unless otherwise instructed, the relevant term being related in the present invention has following implication：
Term is explained：
ZooKeeper：ZooKeeper is one distributed, and the distributed application program coordination service of open source code is
Mono- realization increased income of Chubby of Google, is the significant components of Hadoop and Hbase.It is one for Distributed Application is carried
For the software of Consistency service, there is provided function include：Configuring maintenance, domain name service, distributed synchronization, group service etc..
ZooKeeper includes a simple primitive collection, there is provided the interface of Java and C.In the present invention, ZooKeeper clusters are used for
The condition managing of calculate node cluster.
Solr：Solr is with written in Java, operates in Servlet containers（Such as Apache Tomcat or Jetty）One
Individual independent full-text search server.Its major function includes full-text search, hit sign, facet search, dynamic clustering, data
Storehouse is integrated, and rich text（Such as Word, PDF）Treatment.Solr is highly scalable, and there is provided distributed search and rope
Draw duplication.
Shard：Data fragmentation；
Shard copies：Data fragmentation copy（Shard-1 Replica、Shard-2 Replica、Shard-3
Replica……）；
Core：Manage shard copies and the function set of statistic analysis is carried out for copy data（Core1 in Fig. 1,
Core2……）；Core includes two parts content：Statistical analysis and data storage.For data storage, core is used as two benches
The participant of committing protocol participates in the renewal operation of data.Each copy of shard is deposited using way of example in Core
Storage, local directly invokes example, the binding remote service of distal end.
overseer：Leader node in calculate node.
Based on above-mentioned distributed statistical analysis system, as shown in Figure 2-5, its collocation method includes the steps：
S1：Election calculate node cluster in leader node, according to data fragmentation principle in each calculate node fragment data, and
Election leadership's burst in the copy of data fragmentation.
In step sl, main purpose is to be managed according to predetermined strategy to each calculate node cluster.
This partial function can be completed by ZooKeeper clusters.
The original state and final state of cluster are as follows：
1）Original state
ZooKeeper clusters；
Each independent calculate node, core is initialized, specifies the shard belonging to core；
2）Final state
Calculate node elects overseer, and each node knows overseer and other calculate nodes；
Each core distributes a shard, and the copy of same shard can not be appeared in same calculate node；
Each copy that shard a leader, same shard are elected between multiple copies of each shard knows this
The leader of shard and other member
It is specifically described below between original state and final state, each sub-steps of cluster management step：
S101：Overseer is elected
Based on ZooKeeper clusters, numerous calculate nodes can easily elect overseer.Step is as follows：
1）Machine is opened, each calculate node registers the IP of each calculate node to ZooKeeper；
Path：StatsAnalysis/live_nodes/
2）Each calculate node monitors StatsAnalysis/live_nodes/, obtains and preserve the IP letters of all of calculate node
Breath；
3）After each calculate node IP is obtained, each calculate node is to zookeeper registration contentions overseer；
Path：StatsAnalysis/overseer/
4）Each calculate node monitors StatsAnalysis/overseer/, obtains and preserve the IP information of overseer；
S102：Data fragmentation
During data fragmentation, the shard as belonging to configuration file specifies core need to ensure the copy of same shard not
Can be in same calculate node.
Data fragmentation is comprised the following steps that：
1）Machine is opened, each calculate node initializes core according to the number of the core of configuration, and registers each core to zookeeper；
Path is：StatsAnalysis/cores/；
2）Shards of the overseer according to belonging to each core determines data fragmentation rule, and is synchronized to other calculate nodes, specifically
See data storage section（Description）；Path：StatsAnalysis/shard/Barrel2Shard；
S103：Shard leader are elected
After shard is assigned, copy carries out the election of leader in each shard.Step is as follows：
1）Core monitors StatsAnalysis/cores/, obtains affiliated shard；
2）Each copies of Shard are to ZooKeeper clusters registration contention shard leader；
Path is：StatsAnalysis/shardleader/1, StatsAnalysis/shardleader/2 ...；
3）Each copy monitors each affiliated paths of the shard leader in zookeeper, obtains and preserve respective shard
The information of leader；
To sum up, the configured in one piece flow of cluster management module can be generalized into following steps：
1）Configuration ZooKeeper clusters；
2）Calculate node opens machine, is set up with the control page and contacted, and configuration shard numbers, single shard copy numbers, triggering add
Enter cluster action；
3）Each calculate node creates the Cores of oneself（Number is determined by configuration parameter）, registered to ZooKeeper, registered paths
It is StatsAnalysis/cores/；
4）Each calculate node is registered to ZooKeeper, and registered paths are StatsAnalysis/live_nodes/；
5）Each calculate node fights for overseer to ZooKeeper, and path is StatsAnalysis/overseer/；
6）Overseer monitors StatsAnalysis/live_nodes/ and obtains all of calculate node；
7）Overseer Makes Path StatsAnalysis/shardleader/ { shard numberings } to zookeeper, is used for
Shard leader are fought between shard copies；
8）Overseer monitors StatsAnalysis/cores/ and obtains each core, is carried out according to cores, number of copies, shard numbers
Shard copies are distributed, and shard numberings are write into each core；
9）Each core monitors each self-corresponding node, reads the shard numberings that nodal value obtains distribution；
10）To ZooKeeper registration contention shard leader, path is StatsAnalysis/shardleader/ to each core
{ shard numberings }；
11）Shard leader monitoring StatsAnalysis/cores/ obtains each copy of this shard.
S2:After service node receives statistical analysis request, to leader node application calculate node, leader node is by task amount
Minimum calculate node feeds back to service node；After service node obtains the calculate node of feedback, statistics request is sent to this
Calculate node；Calculate node is searched leader's burst and the data fragmentation copy of free time is obtained to leader's burst application, and will statistics
Task distributes to the data fragmentation copy and performs statistics task.
In step s 2, service node receive client statistical analysis request after, by request be decomposed into query statement and
Statistics sentence, holds enterprise's ID lists that query statement obtains matching from solr first, then obtains idle meter from overseer
Enterprise ID lists and statistics sentence are simultaneously sent to the calculate node and carry out statistical analysis by operator node.
Statistical analysis can substantially be divided into three steps：Statistics Sentence analysis, task splits and result merges.Statistics language
Parsing statistics sentence is responsible in sentence analysis.When overseer already is made in statistical analysis hands to execution node, performing node will make
Industry splits into multiple tasks, and according to burst rule enterprise's ID lists of statistics will be needed to be grouped, perform node be responsible for by
Task and enterprise's ID groups are sent to the leader of correspondence shard, and the pair for selecting to transfer to load less is responsible for by shard leader
This execution task.After all of tasks carrying is finished, perform node and collect all result of calculations and result is merged, after
And return to service node.Task splits the single subtask for being and only takes up less process resource, and can be rapidly completed, and has
Beneficial to redistributing for subtask, it is to avoid occur the situation of idling-resource waste caused by task execution time process.At some
In embodiment, if operation needs repeatedly to access data, as a further improvement, then collecting and combining result by execution node
Afterwards, carry out task again according to job requirements to split and result merging, untill job run is finished.Accordingly even when repeatedly visiting
Ask the uniformity that data can also keep statistics.
In three steps of statistical analysis：
The form of statistical analysis sentence can set according to the icon analysis rule of existing DP data platforms.Parsing code is taken from
Standalone version statistical analysis code.
The fractionation of task is specially and the related enterprise ID lists of statistical analysis operation is grouped according to burst rule, group
Number for shard number.Each task is responsible for a statistical work for the data of group.The code of specific statistical analysis is by list
Machine version statistical analysis code revision is realized.
As a result combining step includes collecting the statistics of each task, and is post-processed according to statistical analysis demand.
The specific code for merging is realized by standalone version statistical analysis code revision.
To sum up, the idiographic flow that Statistic Query module is performed can be summarized as：
1）Service node receives statistical analysis request, and request is split as into two parts：Searching request and statistics are asked；
2）Service node calls solr service-seeking relevant enterprise ID lists；
3）Service node monitors catalogue StatsAnalysis/overseer/ and obtains current overseer, and to overseer
The minimum calculate node of task amount is fed back to service node by application calculate node, overseer（Between completion calculate node
Load balancing）；
4）Service node obtains the calculate node of task run, and statistics request and relevant enterprise ID lists are sent into the calculating
Node；
5）Be grouped for enterprise's ID lists according to burst rule by calculate node；
6）Calculate node finds the position of each shard leader and to the idle shard copies of its application（Complete shard pairs
Load balancing between this）；
7）Calculate node will count sentence and the enterprise ID of correlation is sent to a shard copy, and statistics is completed by them, if one
Secondary interaction cannot obtain result, and repeatedly interaction is responsible for by calculate node；
In the statistical work of the subtask of statistical analysis, can be obtained from local Shard copies according to the enterprise ID lists of input
Corresponding company information is taken, and is analyzed according to statistical rules.
8）Calculate node collects the statistics of each shard copies, and returns to service node and carry out result cache.
S3：To leader node, leader node finds corresponding data fragmentation to the data is activation that service node will update
All copies and by each copy of data syn-chronization to the data fragmentation.
Step S3 is mainly for the storage work of completion data.Data syn-chronization uses two-phase commitment protocol, if all into
Work(, then return successfully, if any one copy fails, retracts and return failure.
Determine that data fragmentation rule includes：
1）The low 10 of enterprise ID is obtained, according to value packet obtains 1024 buckets（That is Barrel）；
2）The mapping relations of Barrel to shard are stored using table.
Storage location of the mapping relations of Barrel ID to Shard ID in ZooKeeper clusters be
The entitled Barrel2Shard of StatsAnalysis/shard/, znode, is worth the json character strings for mapping relations.
To sum up, the overall flow of data storage can be summarized as follows：
1）Service node monitors StatsAnalysis/overseer/, obtains current overseer, and will update the data transmission
To overseer；
2）Overseer is calculated according to burst rule and is obtained shard id, and finds the leader of the shard, the number that will be updated
According to being sent to shard leader；
3）Shard leader are by all copies of data consistent updates to the shard.
S4：In the case of adjustment data fragmentation number or burst rule, the relation of the data that remap and data fragmentation.
Step S4 is the migration work of data.Data Migration is divided into two kinds of situations：Replicate single copy and the modification of shard
Shard numbers burst again.During Data Migration, the renewal of data is not allowed to operate.
In reproduction replica, if running into the situation of equipment fault or addition new equipment, it may be necessary to from shard
Leader synchrodatas, now can be using the strategy for replicating whole copy.
Again in burst, when shard numbers are changed, data are re-started with burst, modification Barrel to shard's reflects
Penetrate relation and update to ZooKeeper, the data for changing the Barrel of mapping relations are copied into new shard from former shard, and
Deleted from former shard.
S5：When equipment state changes, distributed statistical system is adjusted, it is described adjustment include it is following at least
It is a kind of：
When equipment fault is detected, the number and distribution situation of each shard copies are checked, if the number of each shard copies is poor
It is not larger, then to shard carry out be distributed adjustment and migration, with ensure each shard copies number it is essentially identical；And
When new equipment addition is detected, the number and distribution situation of each shard copies are checked, if the number of the copy of shard
It is variant, then the few shard of copy number is copied into new equipment, if number is identical, re-start burst.
So, when device fails or when thering is new equipment to be added, system can automatically timely adjustresources point
Match somebody with somebody, it is ensured that the autgmentability of system.
Embodiment two
A kind of distributed statistical analysis system is present embodiments provided, the system is configured by the collocation method implemented in, and
To the corresponding module for performing the collocation method.Specifically include：Cluster management module, data storage and transferring module, statistical analysis
Enquiry module and statistics task load balancing module（As shown in Figure 6）.
Cluster management module is used for establishment of connection of each calculate node with ZooKeeper clusters and maintenance, specifically for
Overseer nodes in election calculate node cluster（That is leader node）, divided in each calculate node according to data fragmentation principle
Sheet data, and election leadership's burst in the copy of data fragmentation（That is shard leader）.
When equipment state changes, it is necessary to adjust the distribution of shard copies.Equipment state change mainly includes equipment fault
Two kinds of situations are added with new equipment.
When equipment fault is detected, the number and distribution situation of each shard copies of cluster management module check, if respectively
The number difference of shard copies is larger, then shard is carried out being distributed adjustment and migration, to ensure the number of each shard copies
It is essentially identical, strengthen the robustness of cluster.
When new equipment addition is detected, the number and distribution situation of each shard copies are checked, if the copy of shard
Number is variant, then the few shard of copy number is copied into new equipment, if number is identical, can re-start burst.
Herein, shard numberings can be specified in configuration file, not use cluster to specify burst situation automatically.
The distribution for adjusting shard copies can be by cluster management module and data transferring module（Description）Coordinate
Into.
The original state and final state of cluster are as follows：
1）Original state
ZooKeeper clusters；
Each independent calculate node, core is initialized, specifies the shard belonging to core；
2）Final state
Calculate node elects overseer, and each node knows overseer and other calculate nodes；
Each core distributes a shard, and the copy of same shard can not be appeared in same calculate node；
Each copy that shard a leader, same shard are elected between multiple copies of each shard knows this
The leader of shard and other members.
Between original state and final state, the configuration process of cluster management module corresponds to step in embodiment one
The content of S101-S103.Will not be repeated here.
Data storage and transferring module, for service node will update data is activation to overseer, overseer is looked into
Find all copies of corresponding shard and by each copy of data syn-chronization to the shard.Data syn-chronization is carried using two benches
Agreement is handed over, if all successes, return successfully, if any one copy fails, failure is retracted and return.When equipment fault or add
When entering new equipment, the state of cluster can change, after cluster state is stable, the shard situations in cluster, by
Overseer supports the work of Data Migration.
According to above-mentioned collocation method, data storage and transferring module specifically include data memory module and Data Migration mould
Block（Correspond respectively to implement the content of S3 and S4 steps in）.
Data memory module is used to update data at each copy of each shard, and ensures that data are consistent in shard
Property.Further, data memory module is additionally operable to determination burst rule and consistency protocol, determines that burst rule includes：
1）The low 10 of enterprise ID is obtained, according to value packet obtains 1024 buckets；
2）The mapping relations of Barrel to shard are stored using table.
Consistency protocol includes：The atom that data update is ensured using two-phase commitment protocol between each copy in Shard
Property.
Two-phase commitment protocol is specifically made up of two parts two-phase commitment protocol module：Coordinator
（Coordinator）And participant（Participant）.Coordinator plays management synergy, and participant is data access
Actual operator.
Data Migration module is used to adjust shard distributions, in the case where shard numbers and burst rule are constant, only uses
In replicating and deleting shard copy contents, in the case of adjustment shard numbers or burst rule, for the data that remap
Relation.
Data Migration is divided into two kinds of situations：Replicate the single copy of shard and change shard numbers burst again.In number
During according to migration, the renewal of data is not allowed to operate.Detailed process may be referred to implement the implementation of step S4 in.
Statistic Query module, receives the statistical analysis request of client and is decomposed into request and look into for service node
After asking sentence and statistics sentence, enterprise's ID lists that query statement obtains matching from solr are held first, then obtained from overseer
Take the calculate node of free time and enterprise ID lists and statistics sentence are sent to the calculate node and carry out statistical analysis.Calculate node
Statistics sentence is rewritten, each shard leader is then accessed and is obtained the less copy of load in each shard, and operation task is sent out
Node where the copy is sent to perform.Collected after calculate node receives the result of calculation that each shard is returned and returned to system
Meter result.
When task is not carried out finishing for a long time, calculate node is killed overtime task and reselects a shard copy
Operation task.When using rmi synchronization calls in code, then the function of overtime task is not realized killing.
Statistic Query module is specifically configured to perform below scheme：
Statistical analysis can substantially be divided into three steps：Statistics Sentence analysis, task splits and result merges.Statistics sentence point
Parsing statistics sentence is responsible in analysis.When overseer already is made in statistical analysis hands to execution node, perform node and tear operation open
It is divided into multiple tasks, and according to burst rule enterprise's ID lists of statistics will be needed to be grouped, performs node and be responsible for task
With the leader that enterprise ID groups are sent to correspondence shard, it is responsible for selecting the copy for transferring to load less to hold by shard leader
Row task.After all of tasks carrying is finished, perform node and collect all result of calculations and result is merged, then return
Back to service node.If operation needs repeatedly to access data, by execution node after result is collected and combined, according to operation need
Ask and carry out again task to split and result merging, untill job run is finished.Search the less calculate node of load or
The task of shard copies transfers to load balancing module（Description）Perform.
The specific implementation of three steps of statistical analysis may be referred to the corresponding content of S2 in embodiment one to realize.
Statistics task load balancing module, for being used cooperatively with cluster management module and Statistic Query module.
In some embodiments, it is also possible to as a part for Statistic Query module, be performed during statistical analysis relevant
The task of load balancing.Load balancing is divided into two kinds of situations：The load balancing of load balancing and task based access control based on operation, base
In the calculate node that the load balancing of operation is responsible for selecting operating statistic to analyze operation, the load balancing of task based access control is responsible for selection
Run the copy of the shard of calculating task.
Load balancing plays an important role in terms of utilization rate of equipment and installations is improved, and working contents are the effective computings of monitoring
Resource and task distribution condition, for the applying for calculation resources of the task chooses more idle calculation resources.
It is as stated above in the implementation procedure of Statistic Query module, mainly including following two load balancings
Step：
The first performs node and task run copy to choose.This part can individually be integrated into a selection submodule, solely
It is vertical to use, it is to improve code reuse.
Be for second the task amount for extracting calculate node to each calculate node from overseer, from shard leader to
Each copy extracts the task amount of each copies of shard.It is final to be selected according to the Selection Strategy for choosing the minimum calculation resources of task amount
Select calculation resources.
As a further improvement, the system can also include following module：
Equipment fault adjustment module.The influence of equipment fault can be divided into three aspects：Influence to cluster state, to statistical analysis
Influence and the influence to data storage.Therefore equipment fault adjustment module can be used for performing following three kinds of steps：
1st, to the influence of cluster state
When calculate node and cores lose, re-elect overseer and shard leader's.Way to elect is automatic choosing
Lift, it is not necessary to write code again；
2nd, to the influence of statistical analysis
If calculate node failure where statistical analysis operation, service node retriggered statistical analysis process；
If calculate node failure where subtask, by calculate node retriggered task where operation；
3rd, to the influence of data storage
Any response is not done to data storage.
Equipment adds adjustment module.There are two kinds of situations in addition equipment：Flash and new equipment are added.For the feelings of flash
Condition, equipment addition adjustment module is used to add overtime limitation, only after cluster state changes time-out, cluster state change（Do not wrap
Include overseer and shard leader）The change of ability trigger data storage（The equipment for reconnecting is added as new equipment, original number
According to deletion）If cluster state recovers in time-out time（Do not include overseer and shard leader）, then data are not done
The change of storage.In the overtime computing interval, the renewal of data is not allowed to operate.
The addition of new equipment may cause both sides consequence：The duplication of shard copies and migration, again burst.Two kinds
In units of bucket, using json string formats, the mode pulled from source by target is migrated situation.In data conversion period
Between, do not allow the renewal of data to operate, but normal function of statistic analysis can not be influenceed.The migration work of data（Including replicating
Again burst）There is overseer to preside over.
The duplication of Shard copies：Replicated in units of bucket, only when whole copy, all migration finishes just calculation completion,
In order to prevent source failure during duplication or transfer.
Again burst：Shifted in units of bucket, transfer finishes a bucket, then changed bucket to the map information of shard,
The burst success again of the information of mark one bucket.Again burst is only when Shard copy numbers are satisfied by specifying copy number and remaining
Can just be carried out when remaining enough core numbers, and can not simultaneously be carried out with migration with the duplication of Shard copies.
During the Interface design of modules, the embodiment of the present invention take into account the level of distributed system.Calculate
Node cluster includes three different levels：Calculate node layer, Core layers, Statistics and DataStore layers.Calculate node
The maintenance of the responsible cluster topology of layer, the maintenance of burst rule, the burst of data, the duplication and migration of data, again burst, statistics
Decomposition and the result for analyzing operation such as collect at the work.Core layers is responsible for the uniformity that the data of each Shard update.Statistics
It is responsible for the statistical analysis of single shard copies, and DataStore is responsible for the storage of Shard copy datas.
Load balancing is divided into two levels：The load balancing and Core layers of load balancing of calculate node layer, two loads
Share the content for only processing different.Uniformity between Core layers of multiple copies of same shard uses two-phase commitment protocol,
Ensure the atomicity of data manipulation, the core of Shard leader serves as telegon, and each core of shard copies serves as ginseng
With unit.The condition managing of whole cluster and leader's election etc. are realized based on zookeeper clusters.
Therefore, in design interface, according to above-mentioned level, and with reference to the data exchange relational design between modules
.
Collocation method above to distributed statistical analysis system provided by the present invention and distributed statistical analysis system
It is described in detail.Specific case used herein is set forth to principle of the invention and implementation method, above reality
The explanation for applying example is only intended to help and understands core concept of the invention.It should be pointed out that for the ordinary skill of the art
For personnel, under the premise without departing from the principles of the invention, some improvement and modification can also be carried out to the present invention, these improvement
Also fallen into the protection domain of the claims in the present invention with modification.
Claims (10)
1. the collocation method of distributed statistical analysis system, the distributed statistical analysis system includes ZooKeeper clusters, clothes
Business node and calculate node cluster, it is characterised in that methods described includes：
Election calculate node cluster in leader node, according to data fragmentation principle in each calculate node fragment data, and
Election leadership's burst in the copy of data fragmentation；
After service node receives statistical analysis request, to leader node application calculate node, leader node is by task amount minimum
Calculate node feeds back to service node；After service node obtains the calculate node of feedback, statistics request is sent to the calculating section
Point；Calculate node is searched leader's burst and the data fragmentation copy of free time is obtained to leader's burst application, and statistics task is divided
Dispensing data fragmentation copy execution statistics task.
2. the collocation method of distributed statistical analysis system according to claim 1, it is characterised in that methods described is also
Including：
To leader node, leader node finds all copies of corresponding data fragmentation to the data is activation that service node will update
And by each copy of data syn-chronization to the data fragmentation.
3. the collocation method of distributed statistical analysis system according to claim 2, it is characterised in that the data syn-chronization
Two-phase commitment protocol is used, if all successes, return successfully, if any one copy fails, failure is retracted and return.
4. the collocation method of distributed statistical analysis system according to claim 2, it is characterised in that methods described is also
Including：In the case of adjustment data fragmentation number or burst rule, the relation of the data that remap and data fragmentation.
5. the collocation method of distributed statistical analysis system according to claim 1, it is characterised in that methods described is also wrapped
Include when equipment state changes, the step of being adjusted to distributed statistical system, the adjustment includes following at least one
Kind：
When equipment fault is detected, the number and distribution situation of each shard copies are checked, if the number of each shard copies is poor
It is not larger, then to shard carry out be distributed adjustment and migration, with ensure each shard copies number it is essentially identical；And
When new equipment addition is detected, the number and distribution situation of each shard copies are checked, if the number of the copy of shard
It is variant, then the few shard of copy number is copied into new equipment, if number is identical, re-start burst.
6. the collocation method of distributed statistical analysis system according to claim 1, it is characterised in that the calculate node
Search leader's burst and the data fragmentation copy of free time is obtained to leader's burst application, and statistics task is distributed into the data point
Piece copy performs statistics task to be included：
Assignment statisticses are split into multiple tasks by the calculate node for performing calculating task, and will need system according to burst rule
The ID lists of meter are grouped, and task and ID groups are sent to corresponding leader's burst by calculate node, and choosing is responsible for by leader's burst
Choose friends and task is performed by load less data burst copy, after all of tasks carrying is finished, perform node and collect all
Result of calculation is simultaneously merged to result, then returns to service node.
7. the collocation method of distributed statistical analysis system according to claim 6, it is characterised in that if assignment statisticses
Need repeatedly to access data, then by execution node after result is collected and combined, the task that carried out again according to assignment statisticses demand is torn open
Divide and result merges, untill assignment statisticses operation is finished.
8. a kind of distributed statistical analysis system, including ZooKeeper clusters, service node and calculate node cluster, its feature
It is also to include：
Cluster management module, for electing the leader node in calculate node cluster, section is calculated according to data fragmentation principle each
Fragment data in point, and election leadership's burst in the copy of data fragmentation；
Statistical analysis module, after receiving statistical analysis request for service node, to leader node application calculate node, Ling Daojie
The minimum calculate node of task amount is fed back to service node by point；After service node obtains the calculate node of feedback, please by statistics
Ask and be sent to the calculate node；Calculate node is searched leader's burst and the data fragmentation pair of free time is obtained to leader's burst application
This, and statistics task is distributed into data fragmentation copy execution statistics task.
9. distributed statistical analysis system according to claim 8, it is characterised in that also include：
Data memory module, for service node will update data is activation to leader node, leader node finds corresponding
All copies of data fragmentation and by each copy of data syn-chronization to the data fragmentation.
10. distributed statistical analysis system according to claim 9, it is characterised in that the data syn-chronization uses two ranks
Section committing protocol, if all successes, return successfully, if any one copy fails, retracts and returns failure.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710105031.7A CN106936899B (en) | 2017-02-25 | 2017-02-25 | Configuration method of distributed statistical analysis system and distributed statistical analysis system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710105031.7A CN106936899B (en) | 2017-02-25 | 2017-02-25 | Configuration method of distributed statistical analysis system and distributed statistical analysis system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106936899A true CN106936899A (en) | 2017-07-07 |
| CN106936899B CN106936899B (en) | 2021-02-05 |
Family
ID=59424728
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710105031.7A Expired - Fee Related CN106936899B (en) | 2017-02-25 | 2017-02-25 | Configuration method of distributed statistical analysis system and distributed statistical analysis system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106936899B (en) |
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107231437A (en) * | 2017-07-18 | 2017-10-03 | 郑州云海信息技术有限公司 | A kind of task backup management method and device |
| CN108572845A (en) * | 2018-03-15 | 2018-09-25 | 华为技术有限公司 | The upgrade method and related system of distributed micro services cluster |
| CN108737543A (en) * | 2018-05-21 | 2018-11-02 | 天津中兴智联科技有限公司 | A kind of distribution Internet of things middleware and working method |
| CN108829709A (en) * | 2018-05-03 | 2018-11-16 | 广州金山安全管理系统技术有限公司 | Distributed database management method, apparatus, storage medium and processor |
| CN109214132A (en) * | 2018-10-30 | 2019-01-15 | 中国运载火箭技术研究院 | A kind of big Throughput Asynchronous task processing system of non-coupled streaming towards LVC emulation |
| CN109951508A (en) * | 2017-12-21 | 2019-06-28 | 中国移动通信集团重庆有限公司 | Request coordination approach and device, computer equipment and storage medium based on Zookeeper |
| CN110209910A (en) * | 2019-05-20 | 2019-09-06 | 无线生活（杭州）信息科技有限公司 | Index switching dispatching method and dispatching device |
| CN110597809A (en) * | 2019-08-21 | 2019-12-20 | 中山大学 | Consistency algorithm system supporting tree-shaped data structure and implementation method thereof |
| CN110933107A (en) * | 2019-12-16 | 2020-03-27 | 许昌学院 | Configuration method of distributed statistical analysis system and distributed statistical analysis system |
| CN111737018A (en) * | 2020-08-26 | 2020-10-02 | 腾讯科技（深圳）有限公司 | ZooKeeper configuration file storage processing method, device, equipment and medium thereof |
| CN112765262A (en) * | 2019-11-05 | 2021-05-07 | 中兴通讯股份有限公司 | Data redistribution method, electronic equipment and storage medium |
| CN113268472A (en) * | 2021-07-15 | 2021-08-17 | 北京华品博睿网络技术有限公司 | Distributed data storage system and method |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101504663A (en) * | 2009-03-17 | 2009-08-12 | 北京大学 | Swarm intelligence based spatial data copy self-adapting distribution method |
| CN101697526A (en) * | 2009-10-10 | 2010-04-21 | 中国科学技术大学 | Method and system for load balancing of metadata management in distributed file system |
| CN103023970A (en) * | 2012-11-15 | 2013-04-03 | 中国科学院计算机网络信息中心 | Method and system for storing mass data of Internet of Things (IoT) |
| CN103258036A (en) * | 2013-05-15 | 2013-08-21 | 广州一呼百应网络技术有限公司 | Distributed real-time search engine based on p2p |
| US20130290249A1 (en) * | 2010-12-23 | 2013-10-31 | Dwight Merriman | Large distributed database clustering systems and methods |
| CN103414761A (en) * | 2013-07-23 | 2013-11-27 | 北京工业大学 | Mobile terminal cloud resource scheduling method based on Hadoop framework |
| CN103929500A (en) * | 2014-05-06 | 2014-07-16 | 刘跃 | Method for data fragmentation of distributed storage system |
| CN104008152A (en) * | 2014-05-21 | 2014-08-27 | 华南理工大学 | Distributed file system architectural method supporting mass data access |
| CN105550229A (en) * | 2015-12-07 | 2016-05-04 | 北京奇虎科技有限公司 | Method and device for repairing data of distributed storage system |
| US20160191508A1 (en) * | 2014-12-31 | 2016-06-30 | Nexenta Systems, Inc. | Methods and Systems for Block Sharding of Objects Stored in Distributed Storage System |
| US20160203174A1 (en) * | 2015-01-09 | 2016-07-14 | Dinesh Shahane | Elastic sharding of data in a multi-tenant cloud |
-
2017
- 2017-02-25 CN CN201710105031.7A patent/CN106936899B/en not_active Expired - Fee Related
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101504663A (en) * | 2009-03-17 | 2009-08-12 | 北京大学 | Swarm intelligence based spatial data copy self-adapting distribution method |
| CN101697526A (en) * | 2009-10-10 | 2010-04-21 | 中国科学技术大学 | Method and system for load balancing of metadata management in distributed file system |
| US20130290249A1 (en) * | 2010-12-23 | 2013-10-31 | Dwight Merriman | Large distributed database clustering systems and methods |
| CN103023970A (en) * | 2012-11-15 | 2013-04-03 | 中国科学院计算机网络信息中心 | Method and system for storing mass data of Internet of Things (IoT) |
| CN103258036A (en) * | 2013-05-15 | 2013-08-21 | 广州一呼百应网络技术有限公司 | Distributed real-time search engine based on p2p |
| CN103414761A (en) * | 2013-07-23 | 2013-11-27 | 北京工业大学 | Mobile terminal cloud resource scheduling method based on Hadoop framework |
| CN103929500A (en) * | 2014-05-06 | 2014-07-16 | 刘跃 | Method for data fragmentation of distributed storage system |
| CN104008152A (en) * | 2014-05-21 | 2014-08-27 | 华南理工大学 | Distributed file system architectural method supporting mass data access |
| US20160191508A1 (en) * | 2014-12-31 | 2016-06-30 | Nexenta Systems, Inc. | Methods and Systems for Block Sharding of Objects Stored in Distributed Storage System |
| US20160203174A1 (en) * | 2015-01-09 | 2016-07-14 | Dinesh Shahane | Elastic sharding of data in a multi-tenant cloud |
| CN105550229A (en) * | 2015-12-07 | 2016-05-04 | 北京奇虎科技有限公司 | Method and device for repairing data of distributed storage system |
Non-Patent Citations (1)
| Title |
|---|
| YAOSHENGTING: "SolrCloud之分布式索引及与Zookeeper的集成", 《HTTPS://BLOG.CSDN.NET/YSTYAOSHENGTING/ARTICLE/DETAILS/48028485》 * |
Cited By (19)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107231437A (en) * | 2017-07-18 | 2017-10-03 | 郑州云海信息技术有限公司 | A kind of task backup management method and device |
| CN109951508A (en) * | 2017-12-21 | 2019-06-28 | 中国移动通信集团重庆有限公司 | Request coordination approach and device, computer equipment and storage medium based on Zookeeper |
| CN108572845A (en) * | 2018-03-15 | 2018-09-25 | 华为技术有限公司 | The upgrade method and related system of distributed micro services cluster |
| CN108572845B (en) * | 2018-03-15 | 2022-05-31 | 华为技术有限公司 | Upgrading method of distributed micro-service cluster and related system |
| CN108829709A (en) * | 2018-05-03 | 2018-11-16 | 广州金山安全管理系统技术有限公司 | Distributed database management method, apparatus, storage medium and processor |
| CN108737543A (en) * | 2018-05-21 | 2018-11-02 | 天津中兴智联科技有限公司 | A kind of distribution Internet of things middleware and working method |
| CN108737543B (en) * | 2018-05-21 | 2021-09-24 | 高新兴智联科技有限公司 | Distributed Internet of things middleware and working method |
| CN109214132B (en) * | 2018-10-30 | 2023-06-30 | 中国运载火箭技术研究院 | LVC simulation-oriented uncoupled streaming type large-flux asynchronous task processing system |
| CN109214132A (en) * | 2018-10-30 | 2019-01-15 | 中国运载火箭技术研究院 | A kind of big Throughput Asynchronous task processing system of non-coupled streaming towards LVC emulation |
| CN110209910A (en) * | 2019-05-20 | 2019-09-06 | 无线生活（杭州）信息科技有限公司 | Index switching dispatching method and dispatching device |
| CN110209910B (en) * | 2019-05-20 | 2021-06-04 | 无线生活（杭州）信息科技有限公司 | Index switching scheduling method and scheduling device |
| CN110597809A (en) * | 2019-08-21 | 2019-12-20 | 中山大学 | Consistency algorithm system supporting tree-shaped data structure and implementation method thereof |
| CN110597809B (en) * | 2019-08-21 | 2023-05-23 | 中山大学 | Consistency algorithm system supporting tree-like data structure and implementation method thereof |
| CN112765262A (en) * | 2019-11-05 | 2021-05-07 | 中兴通讯股份有限公司 | Data redistribution method, electronic equipment and storage medium |
| CN112765262B (en) * | 2019-11-05 | 2023-02-28 | 金篆信科有限责任公司 | Data redistribution method, electronic equipment and storage medium |
| CN110933107A (en) * | 2019-12-16 | 2020-03-27 | 许昌学院 | Configuration method of distributed statistical analysis system and distributed statistical analysis system |
| CN111737018A (en) * | 2020-08-26 | 2020-10-02 | 腾讯科技（深圳）有限公司 | ZooKeeper configuration file storage processing method, device, equipment and medium thereof |
| CN113268472B (en) * | 2021-07-15 | 2021-10-12 | 北京华品博睿网络技术有限公司 | Distributed data storage system and method |
| CN113268472A (en) * | 2021-07-15 | 2021-08-17 | 北京华品博睿网络技术有限公司 | Distributed data storage system and method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106936899B (en) | 2021-02-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106936899A (en) | The collocation method of distributed statistical analysis system and distributed statistical analysis system | |
| Campbell et al. | Extreme scale with full sql language support in microsoft sql azure | |
| CN104486445B (en) | Distributed extendable resource monitoring system based on cloud platform | |
| EP3069228B1 (en) | Partition-based data stream processing framework | |
| EP2932370B1 (en) | System and method for performing a transaction in a massively parallel processing database | |
| CN102110121B (en) | A kind of data processing method and system thereof | |
| US20140195489A1 (en) | Logical replication in clustered database system with adaptive cloning | |
| CN108234306A (en) | Network equipment, network method and computer readable storage medium | |
| CN106993019A (en) | Distributed task dispatching method and system | |
| CN106980678A (en) | Data analysing method and system based on zookeeper technologies | |
| CN107430603A (en) | The system and method for MPP database | |
| US20160065498A1 (en) | Distributed transaction subsystem | |
| CN105518641B (en) | Point-to-Point Data clone method, equipment and system and host node switching method, equipment and system | |
| CN108280080A (en) | A kind of method of data synchronization, device and electronic equipment | |
| CN106484713A (en) | A kind of based on service-oriented Distributed Request Processing system | |
| CN104852965B (en) | A kind of user account project management method and system | |
| CN105468720A (en) | Method for integrating distributed data processing systems, corresponding systems and data processing method | |
| CN103473696A (en) | Method and system for collecting, analyzing and distributing internet business information | |
| CN109819004A (en) | For disposing the method and system at more live data centers | |
| CN106021370A (en) | Memory database instance management method and device | |
| CN105610947A (en) | Method, device and system for realizing high-available distributed queue service | |
| CN101340368B (en) | Web service distributed management system and method based on WSDM | |
| Smid et al. | Case study on data communication in microservice architecture | |
| CN110019231A (en) | A kind of method that parallel database dynamically associates and node | |
| Leibert et al. | Automatic management of partitioned, replicated search services |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20210205 |
|
| CF01 | Termination of patent right due to non-payment of annual fee |