CN105243125B - Operation method, device, cluster and the inquiry data method and device of PrestoDB clusters - Google Patents
Operation method, device, cluster and the inquiry data method and device of PrestoDB clusters Download PDFInfo
- Publication number
- CN105243125B CN105243125B CN201510633927.3A CN201510633927A CN105243125B CN 105243125 B CN105243125 B CN 105243125B CN 201510633927 A CN201510633927 A CN 201510633927A CN 105243125 B CN105243125 B CN 105243125B
- Authority
- CN
- China
- Prior art keywords
- survival
- coordinator node
- node
- clusters
- coordinator
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/33—Querying
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
Abstract
The invention discloses a kind of operation method of PrestoDB clusters, device, cluster and inquiry data method and devices.The operation method of PrestoDB clusters includes:ZooKeeper clusters receive respective IP address and the port that at least two coordinator nodes are sent;The coordinator node that the IP address that first is received and port represent is determined as coordinator node of surviving, and the coordinator node other than coordinator node of surviving notifies survival coordinator node to calculate node as spare coordinator node;During inquiry is performed, the current viability of detection survival coordinator node;If detecting that survival coordinator node breaks down, a coordinator node is elected in spare coordinator node, as new survival coordinator node, which is notified to calculate node.The present invention improves the usable efficiency of PrestoDB clusters, realizes the High Availabitity of PrestoDB clusters.
Description
Technical field
The present embodiments relate to computer technology more particularly to a kind of operation method of PrestoDB clusters, device,
The method and apparatus of PrestoDB clusters and PrestoDB clusters inquiry data.
Background technology
Along with the rise of big data, the business datum amount of Internet company rises year by year, therefore major Internet company
All carry out big data technology in inside, and data warehouse is built for core business system, data warehouse is divided into two kinds at present
Type:Off-line data warehouse and real-time data warehouse.Wherein, the representative products in off-line data warehouse are exactly hive, the product by
In bottom Computational frame be MapReduce, therefore be suitable for off-line analysis and the calculating of super large data set, for requirement of real-time
Compare high data analysis and calculating is not appropriate for;The representative products of real-time data warehouse are PrestoDB, the product by
FaceBook is developed, and employs distributed data calculating and the transmission mode of PipeLine, analysis and calculating for big data
It disclosure satisfy that and real-time data analysis and the requirement calculated within -20 minutes, are met at 100 milliseconds.
Since PrestoDB is a distributed computing framework based on memory, carry out data analysis and calculate when
It waits, all calculating (Worker) node in PrestoDB clusters carries out practical data processing and calculating, and coordinates
(Coordinator) node is substantially carried out carrying out heartbeat detection and each between the scheduling of query task and each node
The state and information of the calculating task run on Worker nodes summarize and count.Therefore Coordinator nodes are the equal of
The management node of entire PrestoDB clusters, for all Worker nodes under the overall leadership and inquiry and calculating task.
In the prior art, PrestoDB clusters can only specify a Coordinator node, may result in this way
There are Single Point of Faliures for Coordinator nodes:Once hardware fault occurs for the server where Coordinator nodes, then need
Using PrestoDB clusters stop service, modification PrestoDB cluster configuration files reassign a new server as
Coordinator nodes, restart cluster.PrestoDB clusters just can be subnormal again after above-mentioned a series of operation
Service is provided, but operation can cause PrestoDB clusters to be off providing service within a period of time in this way, reduce
The usable efficiency of PrestoDB clusters.
Invention content
In view of this, the embodiment of the present invention provides a kind of operation method of PrestoDB clusters, device, PrestoDB clusters
And the method and apparatus of PrestoDB clusters inquiry data, to improve the usable efficiency of PrestoDB clusters.
In a first aspect, an embodiment of the present invention provides a kind of operation method of PrestoDB clusters, the method includes:
ZooKeeper clusters receive respective IP address and the port that at least two coordinator nodes are sent;
The coordinator node that the IP address and port that ZooKeeper clusters receive first represent is determined as survival and coordinates
Node, the coordinator node other than the survival coordinator node as spare coordinator node, by the survival coordinator node notify to
Calculate node accepts querying command by the survival coordinator node and calculating task is handed down to calculate node;
During inquiry is performed, the current viability of the ZooKeeper clusters detection survival coordinator node;
If ZooKeeper clusters detect that the survival coordinator node breaks down, in the spare coordinator node
In elect a coordinator node, as new survival coordinator node, which is notified to calculate node.
Second aspect, the embodiment of the present invention additionally provide a kind of running gear of PrestoDB clusters, and described device includes:
Address receiving module, for receiving respective IP address and the port that at least two coordinator nodes are sent;
Node determining module, the coordinator node represented for the IP address for receiving first and port are determined as surviving
Coordinator node, the coordinator node other than the survival coordinator node lead to the survival coordinator node as spare coordinator node
Know to calculate node, querying command is accepted by the survival coordinator node and calculating task is handed down to calculate node;
Survival detection module, for during inquiry is performed, detecting the current viability of the survival coordinator node;
Module is elected, if for detecting that the survival coordinator node breaks down, in the spare coordinator node
In elect a coordinator node, as new survival coordinator node, which is notified to calculate node.
The third aspect, the embodiment of the present invention additionally provide a kind of PrestoDB clusters, including at least two coordinator nodes, meter
Operator node and ZooKeeper clusters;
The ZooKeeper clusters include the running gear of the PrestoDB clusters described in any embodiment of the present invention.
Fourth aspect, the embodiment of the present invention additionally provides a kind of method of PrestoDB clusters inquiry data, using this hair
PrestoDB clusters described in bright any embodiment perform, the method includes:
Client specifies IP address and the port of ZooKeeper clusters;
Client receives querying command, and current survival coordinator node is obtained from the ZooKeeper clusters;
The querying command is submitted to the survival coordinator node by client, is looked by the survival coordinator node described
Order is ask to be handled to obtain calculating task and calculating task is handed down to calculate node to carry out inquiry calculating;
Client obtains query result from the survival coordinator node.
5th aspect, the embodiment of the present invention additionally provide a kind of device of PrestoDB clusters inquiry data, described device
Including:
Module is specified in address, for specifying the IP address of ZooKeeper clusters and port;
Inquire-receive module for receiving querying command, and obtains current survival association from the ZooKeeper clusters
Point of adjustment;
Module is submitted in inquiry, for the querying command to be submitted to the survival coordinator node, is coordinated by the survival
Node is handled to obtain calculating task and calculating task is handed down to calculate node to carry out inquiry calculating to the querying command;
As a result acquisition module, for obtaining query result from the survival coordinator node.
Operation method, device, PrestoDB clusters and the PrestoDB of PrestoDB clusters provided in an embodiment of the present invention
Cluster inquire data method and apparatus, by be configured in PrestoDB clusters at least two coordinator nodes, calculate node and
ZooKeeper clusters elect a coordinator node as survival by ZooKeeper clusters at least two coordinator node
Coordinator node, other coordinator nodes detect the survival coordinator node as spare coordinator node, and during inquiry is performed
Current viability once detecting that the survival coordinator node breaks down, is then selected again in the spare coordinator node
A coordinator node is lifted as new survival coordinator node, PrestoDB clusters is avoided and stops after coordinator node failure
Service improves the usable efficiency of PrestoDB clusters, realizes the High Availabitity of PrestoDB clusters.
Description of the drawings
Fig. 1 is a kind of flow chart of the operation method for PrestoDB clusters that the embodiment of the present invention one provides;
Fig. 2 is a kind of structure diagram of the running gear of PrestoDB clusters provided by Embodiment 2 of the present invention;
Fig. 3 is a kind of deployment schematic diagram for PrestoDB clusters that the embodiment of the present invention three provides;
Fig. 4 is the flow chart of the method for a kind of PrestoDB clusters inquiry data that the embodiment of the present invention four provides;
Fig. 5 is the structure diagram of the device of a kind of PrestoDB clusters inquiry data that the embodiment of the present invention five provides.
Specific embodiment
The present invention is described in further detail with reference to the accompanying drawings and examples.It is understood that this place is retouched
The specific embodiment stated is used only for explaining the present invention rather than limitation of the invention.It also should be noted that in order to just
Part related to the present invention rather than full content are illustrated only in description, attached drawing.
In embodiments of the present invention, the address of at least two coordinator nodes is specified in the configuration file of PrestoDB clusters
With the address of ZooKeeper clusters.When including three servers including two coordinator nodes, ZooKeeper clusters, configuration rule
Fan Ru:Coordinator node address=IP address 1:Port 1;IP address 2:Port 2, that is, it is address 1 and address 2 to specify IP address
For two nodes as candidate coordinator node, port is respectively port 1 and port 2.ZooKeeper addresses=IP address 1:Port
1;IP address 2:Port 2;IP address 3:Port 3 specifies 3 three IP address 1, IP address 2 and IP address server compositions
ZooKeeper clusters, and port 1, port 2 and port 3 are all equal under normal circumstances.Coordinator node specified in this way and
Behind the address of ZooKeeper clusters, convenient for the communication between follow-up each node.At least two are specified to coordinate in configuration file
The address of node and ZooKeeper clusters can be realized by adding configuration item, that is, add two configuration items, a configuration item
For specifying the IP address of at least two coordinator nodes and port, another configuration item is used to specify the IP of ZooKeeper clusters
Address and port.
Embodiment one
Fig. 1 be the embodiment of the present invention one provide a kind of PrestoDB clusters operation method flow chart, the present embodiment
The situation of election new survival coordinator node when the survival coordinator node in PrestoDB clusters breaks down is applicable to, it should
Method can be performed by ZooKeeper clusters, be specifically comprised the following steps:
Step 110, ZooKeeper clusters receive respective IP address and the port that at least two coordinator nodes are sent.
In the present embodiment, PrestoDB clusters include at least two coordinator nodes, multiple calculate nodes and ZooKeeper
Cluster.Wherein, at least two coordinator nodes are configured in PrestoDB clusters, the failures such as the machine of delaying occur convenient for survival coordinator node
Afterwards, new survival coordinator node is elected from other coordinator node, it is not necessary to stop the service of PrestoDB clusters, so as to protect
Demonstrate,prove the high availability of PrestoDB clusters.Above-mentioned work(can be realized in two coordinator nodes of preferred disposition in PrestoDB clusters
Energy.
After coordinator node starts, IP address and port to the pocket transmission itself of ZooKeeper collection, ZooKeeper clusters connect
After receiving the IP address and port that coordinator node sends, the IP address and port are saved in coordinator node list, are convenient for
Subsequently to the election for coordinator node of surviving.
Wherein, ZooKeeper is one distributed, the distributed application program coordination service of open source code, target
It is the error-prone key service of packaged complexity, the system of interface and performance efficiency easy to use, function-stable is supplied to
User.When disposing ZooKeeper clusters, preferred deployment odd number node, because ZooKeeper clusters are with machine number mistake of delaying
Entire cluster partly can just be allowed to be delayed machine.
Step 120, the coordinator node that the IP address and port that ZooKeeper clusters receive first represent is determined as
It survives coordinator node, the survival is coordinated section as spare coordinator node by the coordinator node other than the survival coordinator node
Point notice accepts querying command by the survival coordinator node and calculating task is handed down to calculate node to calculate node.
When ZooKeeper clusters start election survival coordinator node, the IP address that first is received and port represent
Coordinator node be determined as coordinator node of surviving, the coordinator node that subsequently received IP address and port represent is determined as standby
With coordinator node, the survival coordinator node is notified to calculate node, querying command is accepted simultaneously by the survival coordinator node
Querying command is handled to obtain calculating task, calculating task is handed down to calculate node, is performed by calculate node specific
Calculating task.
Step 130, during inquiry is performed, the ZooKeeper clusters detection survival coordinator node is current to be deposited
Activity.
During PrestoDB clusters perform inquiry, ZooKeeper clusters pass through the communication with coordinator node of surviving
Interaction detects the current viability of the survival coordinator node in real time.
Wherein, the current viability of the ZooKeeper clusters detection survival coordinator node preferably includes:
ZooKeeper clusters receive the status information that the survival coordinator node is sent every setting time;
If reach the state letter that the setting time ZooKeeper clusters are not received by the survival coordinator node
Breath, then extend to the second setting time, if reaching second setting time, is not received by the status information, then really
The fixed survival coordinator node breaks down.
Survival coordinator node, to the primary status information of oneself of ZooKeeper collection pocket transmission, is reached every setting time
During setting time, ZooKeeper clusters receive the status information that survival coordinator node is sent, it is believed that survival coordinator node is
Normal work, without failure;When reaching setting time, ZooKeeper clusters are not received by the survival coordinator node hair
The status information sent then extends the time of receiving status information to the second setting time, if reaching second setting time,
Still without the status information for receiving survival coordinator node transmission, it is determined that the survival coordinator node breaks down.
Step 140, if ZooKeeper clusters detect that the survival coordinator node breaks down, described spare
A coordinator node is elected in coordinator node, as new survival coordinator node, by the new survival coordinator node notify to
Calculate node.
When ZooKeeper clusters detect that the survival coordinator node machine of delaying occurs when failures, in the spare coordination
A coordinator node is elected in node, using the coordinator node as new survival coordinator node, which is coordinated into section
Point notice subsequently accepts querying command by the new survival coordinator node and is responsible for meter to the calculate node in PrestoDB clusters
Calculation task issues.
The present embodiment receives respective IP address and the end that at least two coordinator nodes are sent by ZooKeeper clusters
Mouthful, the coordinator node of the IP address that first is received and port expression is determined as coordinator node of surviving, and the survival is coordinated
The survival coordinator node is notified to calculate node, is deposited by described as spare coordinator node by the coordinator node other than node
Coordinator node living accepts querying command and calculating task is handed down to calculate node, during inquiry is performed, ZooKeeper
The current viability of the cluster detection survival coordinator node, if the survival coordinator node breaks down, in spare association
A coordinator node is elected in point of adjustment, as new survival coordinator node, which is notified to meter
Operator node once detecting that survival coordinator node breaks down, elects a new survival from spare coordinator node immediately
Coordinator node avoids PrestoDB clusters and stops service after coordinator node failure, improves PrestoDB clusters
Efficiency can be used, realize the High Availabitity of PrestoDB clusters.
Based on the above technical solution, it after the survival coordinator node that this is new notifies to calculate node, also wraps
It includes:
ZooKeeper clusters instruction calculate node forces the calculating unsuccessfully submitted by the survival coordinator node of failure
Task.
ZooKeeper clusters notify the new survival coordinator node to meter after new survival coordinator node is elected
Operator node, and indicate that calculate node forces what is unsuccessfully submitted by the i.e. preceding survival coordinator node of the survival coordinator node of failure
Calculating task continues to execute the calculating task to avoid calculate node, so as to save the memory headroom of calculate node, further carries
The usable efficiency of high PrestoDB clusters.
Embodiment two
Fig. 2 is a kind of structure diagram of the running gear of PrestoDB clusters provided by Embodiment 2 of the present invention, such as Fig. 2
Shown, the running gear of the PrestoDB clusters described in the present embodiment is configured in ZooKeeper clusters, including:Address receives
Module 210, node determining module 220, survival detection module 230 and election module 240.
Wherein, address receiving module 210 is used to receive respective IP address and the end that at least two coordinator nodes are sent
Mouthful;
Node determining module 220 is used for the IP address for receiving first and the coordinator node of port expression is determined as depositing
Coordinator node living, the coordinator node other than the survival coordinator node is as spare coordinator node, by the survival coordinator node
It notifies to calculate node, querying command is accepted by the survival coordinator node and calculating task is handed down to calculate node;
Detection module 230 of surviving is used for during inquiry is performed, and detects the current survival of the survival coordinator node
Property;
If election module 240 is for detecting that the survival coordinator node breaks down, in the spare coordination section
A coordinator node is elected in point, as new survival coordinator node, which is notified to save to calculating
Point.
Preferably, it further includes:
Indicating module, after notifying to calculate node in the survival coordinator node that this is new, instruction calculate node is strong
The calculating task that system is unsuccessfully submitted by the survival coordinator node of failure.
Preferably, the detection module includes:
Receiving unit, for receiving the status information that the survival coordinator node is sent every setting time;
Determination unit, if for reaching the state letter that the setting time is not received by the survival coordinator node
Breath, then extend to the second setting time, if reaching second setting time, is not received by the status information, then really
The fixed survival coordinator node breaks down.
The said goods can perform the operation method of PrestoDB clusters that any embodiment of the present invention is provided, and have execution
The corresponding function module of method and advantageous effect.
Embodiment three
Fig. 3 is a kind of deployment schematic diagram for PrestoDB clusters that the embodiment of the present invention three provides, as shown in figure 3, this reality
The PrestoDB clusters described in example are applied to include:At least two coordinator nodes 310, calculate node 320 and ZooKeeper clusters 330.
Wherein, ZooKeeper clusters 330 include the PrestoDB clusters described in any embodiment of the present invention, ZooKeeper
Cluster is used for the election survival coordinator node at least two coordinator node, and detects the survival of the survival coordinator node
Property, detecting the survival coordinator node failure, then elect a coordinator node as new survival coordinator node, and
The new survival coordinator node is notified to calculate node.The ZooKeeper clusters preferably include at least three servers.
When performing inquiry, survival coordinator node carries out heartbeat for dispatching between query task and each calculate node
Detect and summarize and count the state and information of the calculating task run in each calculate node;Calculate node is used to carry out data
Processing and calculating.
The present embodiment in PrestoDB clusters by being configured at least two coordinator nodes, calculate node and ZooKeeper
Cluster can realize the High Availabitity of PrestoDB clusters, improve the usable efficiency of PrestoDB clusters.
Example IV
Fig. 4 be the embodiment of the present invention four provide a kind of PrestoDB clusters inquiry data method flow chart, this reality
It applies example and is applicable to situation according to the PrestoDB clusters inquiry data described in any embodiment of the present invention, this method can be by
Client performs, and specifically comprises the following steps:
Step 410, client specifies IP address and the port of ZooKeeper clusters.
Client submit inquire when, need IP address and the port of specified ZooKeeper clusters, convenient for subsequently from
ZooKeeper clusters get survival coordinator node.
Step 420, client receives querying command, and obtains current survival from the ZooKeeper clusters and coordinate
Node.
Client receives querying command input by user, according to the IP address of ZooKeeper clusters and port, from described
Current survival coordinator node is obtained in ZooKeeper clusters.
Step 430, the querying command is submitted to the survival coordinator node by client, by the survival coordinator node
The querying command is handled to obtain calculating task and calculating task is handed down to calculate node to carry out inquiry calculating.
The querying command is submitted to the survival coordinator node by client, is looked into described in the survival coordinator node parsing
It askes order and obtains query execution plan, the query execution stage is produced according to the query execution plan, by the query execution rank
Section is separated into multiple calculating tasks, and multiple calculating tasks are handed down to calculate node, are carried out by calculate node according to calculating task
Inquiry calculates, and the survival coordinator node is summarized and counted to the inquiry result of calculation of calculate node, obtains query result.
Step 440, client obtains query result from the survival coordinator node.
Client obtains final query result from the survival coordinator node.
The present embodiment specifies IP address and the port of ZooKeeper clusters by client, after querying command is received,
Current survival coordinator node is obtained from ZooKeeper clusters, querying command is submitted into the survival coordinator node, by depositing
Coordinator node living is handled to obtain calculating task and calculating task is handed down to calculate node to look into the querying command
It askes and calculates, obtain query result from the survival coordinator node, realize and data are inquired according to PrestoDB clusters, ensure that
The high availability of PrestoDB clusters improves the usable efficiency of PrestoDB clusters.
Embodiment five
Fig. 5 is the structure diagram of the device of a kind of PrestoDB clusters inquiry data that the embodiment of the present invention five provides,
As shown in figure 5, the device of the PrestoDB clusters inquiry data described in the present embodiment is configured in client, including:Address refers to
Module 530 and result acquisition module 540 are submitted in cover half block 510, inquire-receive module 520, inquiry.
Wherein, module 510 is specified for specifying IP address and the port of ZooKeeper clusters in address;
Inquire-receive module 520 is used to receive querying command, and obtain current survival from the ZooKeeper clusters
Coordinator node;
Inquiry submits module 530 to be used to the querying command submitting to the survival coordinator node, is assisted by the survival
Point of adjustment is handled to obtain calculating task and calculating task is handed down to calculate node to carry out inquiry meter to the querying command
It calculates;
As a result acquisition module 540 is used to obtain query result from the survival coordinator node.
The said goods can perform the method that the embodiment of the present invention four is provided, have the corresponding function module of execution method and
Advantageous effect.
Note that it above are only presently preferred embodiments of the present invention and institute's application technology principle.It will be appreciated by those skilled in the art that
The present invention is not limited to specific embodiment described here, can carry out for a person skilled in the art various apparent variations,
It readjusts and substitutes without departing from protection scope of the present invention.Therefore, although being carried out by above example to the present invention
It is described in further detail, but the present invention is not limited only to above example, without departing from the inventive concept, also
It can include other more equivalent embodiments, and the scope of the present invention is determined by scope of the appended claims.
Claims (10)
1. a kind of operation method of PrestoDB clusters, which is characterized in that the PrestoDB clusters, which include at least two, to be coordinated
Node, multiple calculate nodes and ZooKeeper clusters, the method includes:
ZooKeeper clusters receive respective IP address and the port that at least two coordinator nodes are sent;
The coordinator node that the IP address and port that ZooKeeper clusters receive first represent is determined as survival and coordinates section
Point, the coordinator node other than the survival coordinator node notify the survival coordinator node to meter as spare coordinator node
Operator node accepts querying command by the survival coordinator node and calculating task is handed down to calculate node;
During inquiry is performed, the current viability of the ZooKeeper clusters detection survival coordinator node;
If ZooKeeper clusters detect that the survival coordinator node breaks down, selected in the spare coordinator node
A coordinator node is enumerated, as new survival coordinator node, which is notified to calculate node.
2. it according to the method described in claim 1, it is characterized in that, is notified by the new survival coordinator node to calculate node
Later, it further includes:
ZooKeeper clusters instruction calculate node forces the calculating unsuccessfully submitted by the survival coordinator node of failure to be appointed
Business.
3. method according to claim 1 or 2, which is characterized in that ZooKeeper clusters detect the survival coordinator node
Current viability includes:
ZooKeeper clusters receive the status information that the survival coordinator node is sent every setting time;
If reaching the status information that the setting time ZooKeeper clusters are not received by the survival coordinator node,
The second setting time is extended to, if reaching second setting time, is not received by the status information, it is determined that described
Coordinator node of surviving breaks down.
4. a kind of running gear of PrestoDB clusters, which is characterized in that the PrestoDB clusters, which include at least two, to be coordinated
Node, multiple calculate nodes and ZooKeeper clusters, described device include:
Address receiving module, for receiving respective IP address and the port that at least two coordinator nodes are sent;
Node determining module, the coordinator node represented for the IP address for receiving first and port are determined as survival coordination
Node, the coordinator node other than the survival coordinator node as spare coordinator node, by the survival coordinator node notify to
Calculate node accepts querying command by the survival coordinator node and calculating task is handed down to calculate node;
Survival detection module, for during inquiry is performed, detecting the current viability of the survival coordinator node;
Module is elected, if for detecting that the survival coordinator node breaks down, is selected in the spare coordinator node
A coordinator node is enumerated, as new survival coordinator node, which is notified to calculate node.
5. device according to claim 4, which is characterized in that further include:
Indicating module, after notifying to calculate node in the survival coordinator node that this is new, instruction calculate node forces to lose
Lose the calculating task submitted by the survival coordinator node of failure.
6. device according to claim 4 or 5, which is characterized in that the detection module includes:
Receiving unit, for receiving the status information that the survival coordinator node is sent every setting time;
Determination unit, if for reaching the status information that the setting time is not received by the survival coordinator node,
The second setting time is extended to, if reaching second setting time, is not received by the status information, it is determined that described
Coordinator node of surviving breaks down.
7. a kind of PrestoDB clusters, which is characterized in that including at least two coordinator nodes, calculate node and ZooKeeper collection
Group;
The ZooKeeper clusters include the running gear of any PrestoDB clusters of claim 4-6.
8. PrestoDB clusters according to claim 7, which is characterized in that the ZooKeeper clusters include at least three
Platform server.
9. a kind of method of PrestoDB clusters inquiry data, is held using the PrestoDB clusters described in claim 7 or 8
Row, which is characterized in that the method includes:
Client specifies IP address and the port of ZooKeeper clusters;
Client receives querying command, and current survival coordinator node is obtained from the ZooKeeper clusters;
The querying command is submitted to the survival coordinator node by client, and the inquiry is ordered by the survival coordinator node
Order is handled to obtain calculating task and calculating task is handed down to calculate node to carry out inquiry calculating;
Client obtains query result from the survival coordinator node.
10. a kind of device of PrestoDB clusters inquiry data, which is characterized in that including:
Module is specified in address, for specifying the IP address of ZooKeeper clusters and port;
Inquire-receive module for receiving querying command, and obtains current survival from the ZooKeeper clusters and coordinates section
Point;
Module is submitted in inquiry, for the querying command to be submitted to the survival coordinator node, by the survival coordinator node
The querying command is handled to obtain calculating task and calculating task is handed down to calculate node to carry out inquiry calculating;
As a result acquisition module, for obtaining query result from the survival coordinator node.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201510633927.3A CN105243125B (en) | 2015-09-29 | 2015-09-29 | Operation method, device, cluster and the inquiry data method and device of PrestoDB clusters |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201510633927.3A CN105243125B (en) | 2015-09-29 | 2015-09-29 | Operation method, device, cluster and the inquiry data method and device of PrestoDB clusters |
Publications (2)
Publication Number | Publication Date |
---|---|
CN105243125A CN105243125A (en) | 2016-01-13 |
CN105243125B true CN105243125B (en) | 2018-07-06 |
Family
ID=55040773
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201510633927.3A Active CN105243125B (en) | 2015-09-29 | 2015-09-29 | Operation method, device, cluster and the inquiry data method and device of PrestoDB clusters |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN105243125B (en) |
Families Citing this family (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN107193960B (en) * | 2017-05-24 | 2020-11-10 | 南京大学 | Distributed crawler system and periodic incremental grabbing method |
CN107807608A (en) * | 2017-11-02 | 2018-03-16 | 腾讯科技(深圳)有限公司 | Data processing method, data handling system and storage medium |
CN111131361B (en) * | 2018-10-31 | 2023-03-24 | 北京国双科技有限公司 | Processing method and device for connection nodes in cluster query system |
CN111522649B (en) * | 2019-02-01 | 2022-04-12 | 北京京东尚科信息技术有限公司 | Distributed task allocation method, device and system |
CN109977161A (en) * | 2019-03-28 | 2019-07-05 | 上海中通吉网络技术有限公司 | The monitoring system of presto cluster |
Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN101977124A (en) * | 2010-11-05 | 2011-02-16 | 山东中创软件工程股份有限公司 | Service clustering method and system based on ZooKeeper technology |
CN103188110A (en) * | 2013-03-29 | 2013-07-03 | 汉柏科技有限公司 | Zookeeper-based cloud platform service hot backup method |
CN103312809A (en) * | 2013-06-24 | 2013-09-18 | 北京汉柏科技有限公司 | Distributed management method for service in cloud platform |
CN103401704A (en) * | 2013-07-24 | 2013-11-20 | 佳都新太科技股份有限公司 | Implementation scheme of distributed log collecting server |
CN103729436A (en) * | 2013-12-27 | 2014-04-16 | 中国科学院信息工程研究所 | Distributed metadata management method and system |
CN104486108A (en) * | 2014-12-08 | 2015-04-01 | 畅捷通信息技术股份有限公司 | Node configuration method base on Zookeeper and node configuration system based on Zookeeper |
Family Cites Families (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US9690671B2 (en) * | 2013-11-01 | 2017-06-27 | Cloudera, Inc. | Manifest-based snapshots in distributed computing environments |
-
2015
- 2015-09-29 CN CN201510633927.3A patent/CN105243125B/en active Active
Patent Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN101977124A (en) * | 2010-11-05 | 2011-02-16 | 山东中创软件工程股份有限公司 | Service clustering method and system based on ZooKeeper technology |
CN103188110A (en) * | 2013-03-29 | 2013-07-03 | 汉柏科技有限公司 | Zookeeper-based cloud platform service hot backup method |
CN103312809A (en) * | 2013-06-24 | 2013-09-18 | 北京汉柏科技有限公司 | Distributed management method for service in cloud platform |
CN103401704A (en) * | 2013-07-24 | 2013-11-20 | 佳都新太科技股份有限公司 | Implementation scheme of distributed log collecting server |
CN103729436A (en) * | 2013-12-27 | 2014-04-16 | 中国科学院信息工程研究所 | Distributed metadata management method and system |
CN104486108A (en) * | 2014-12-08 | 2015-04-01 | 畅捷通信息技术股份有限公司 | Node configuration method base on Zookeeper and node configuration system based on Zookeeper |
Also Published As
Publication number | Publication date |
---|---|
CN105243125A (en) | 2016-01-13 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN105243125B (en) | Operation method, device, cluster and the inquiry data method and device of PrestoDB clusters | |
CN104281496B (en) | The distribution method and distributor of virtual resource | |
CN107547595B (en) | Cloud resource scheduling system, method and device | |
CN105677462A (en) | Distributed task system based on internet of things and business processing method | |
CN108243012B (en) | Charging application processing system, method and device in OCS (online charging System) | |
CN106919445A (en) | A kind of method and apparatus of the container of Parallel Scheduling in the cluster | |
CN105812423B (en) | A kind of cloud system configuration method, server and device | |
CN105162632A (en) | Automatic processing system for server cluster failures | |
CN109614227A (en) | Task resource concocting method, device, electronic equipment and computer-readable medium | |
US20150365311A1 (en) | Scalable software monitoring infrastructure, using parallel task queuing, to operate in elastic cloud environments | |
US20180239761A1 (en) | Collision detection using state management of configuration items | |
CN104484228B (en) | Distributed parallel task processing system based on Intelli DSC | |
CN109359034A (en) | A kind of operation system test method, computer readable storage medium and terminal device | |
CN103326880B (en) | Genesys calling system high availability cloud computing monitoring system and method | |
CN108038126A (en) | Method, apparatus, terminal device and storage medium derived from a kind of data | |
CN114598586B (en) | Multi-cloud scene computing power gridding method and system | |
Mahato et al. | Reliability modeling and analysis for deadline-constrained grid service | |
US8751640B2 (en) | System and method for enhancing efficiency and/or efficacy of switchover and/or failover in providing network based services with high availability | |
CN110247818A (en) | A kind of data monitoring method, device, storage medium and server | |
CN113722141B (en) | Method and device for determining delay reason of data task, electronic equipment and medium | |
Zhang et al. | A relationship-based VM placement framework of cloud environment | |
Wang et al. | Research on key technology of edge-node resource scheduling based on linear programming | |
CN115130911A (en) | Method and device for processing delivery tasks failed to execute by robot | |
CN105245375B (en) | A kind of network-based high-efficiency storage method | |
CN114706675A (en) | Task deployment method and device based on cloud edge cooperative system |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
C06 | Publication | ||
PB01 | Publication | ||
C10 | Entry into substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |