CN111459909A - Method for constructing PostgreSQL L database cluster - Google Patents
Method for constructing PostgreSQL L database cluster Download PDFInfo
- Publication number
- CN111459909A CN111459909A CN202010175693.3A CN202010175693A CN111459909A CN 111459909 A CN111459909 A CN 111459909A CN 202010175693 A CN202010175693 A CN 202010175693A CN 111459909 A CN111459909 A CN 111459909A
- Authority
- CN
- China
- Prior art keywords
- node
- database
- cluster
- postgresql
- master
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 21
- 238000010367 cloning Methods 0.000 claims abstract description 5
- 230000036541 health Effects 0.000 claims description 12
- 238000011084 recovery Methods 0.000 claims description 4
- 230000001737 promoting effect Effects 0.000 claims description 2
- 230000010076 replication Effects 0.000 description 6
- 230000007246 mechanism Effects 0.000 description 4
- 210000004556 brain Anatomy 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 230000001360 synchronised effect Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000002265 prevention Effects 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
Abstract
The invention provides a method for constructing a PostgreSQL L database cluster, which comprises the steps of initializing uninitialized nodes and adding the nodes into a candidate node list of a main node, setting the nodes as the main node and setting L eader Key parameters of a cluster as the node ID of the nodes when only one node exists in the candidate node list of the main node, starting a PostgreSQL L example by the main node and initializing a database, starting a PostgreSQL L example after a slave node detects L eader Key of the cluster and cloning the database from the main node into a slave node, and starting the PostgreSQL L example by other nodes after waiting for setting L eader Key in the cluster and cloning the database from the main node into the slave node.
Description
Technical Field
The embodiment of the invention relates to the technical field of information, in particular to a method for constructing a PostgreSQL L database cluster together with a distributed key value storage system.
Background
PostgreSQL L is a powerful open source database system and is widely used in various industries, and it is very important to deploy and manage a PostgreSQL L database cluster with high availability and high performance.
Disclosure of Invention
Aiming at the problem that the prior art lacks effective deployment and management of a PostgreSQL L database cluster with high availability and high performance, the embodiment of the invention provides a method for constructing a PostgreSQL L database cluster together with a distributed key value storage system, which comprises the following steps:
initializing the nodes: initializing uninitialized nodes and adding the nodes into a candidate node list of a main node;
initializing a main node, namely setting the node as the main node and setting L eader Key parameter of a cluster as the node ID of the node when only one node exists in a candidate node list of the main node, starting a PostgreSQL L instance and initializing a database by the main node;
a slave node synchronization step, namely starting a PostgreSQL L instance by the slave node after detecting L eader Key of the cluster, and cloning the database from the master node to become the slave node;
and in the step of updating the main node, other nodes need to wait for setting L eader keys in the cluster to start a PostgreSQL L instance, and clone the database from the main node into a slave node.
In some embodiments, the L reader Key value of each node has a life cycle TT L (Time To L ive) parameter, and when the life cycle is reached, the following main node updating steps are carried out:
at the expiration of TT L, the cluster checks the L reader Key parameter to determine if the master node is alive, updates the TT L value if the master node is normal, otherwise performs the master node election step.
In some embodiments, the master node election step comprises:
when detecting that L reader Key is empty, checking the position of a node in the location information of WA L log in a distributed configuration storage system (DCS) system and a health check result corresponding to the node, and determining a master node.
In some embodiments, the master node election step further comprises:
if the master node survives, the current master node is maintained and the master node update step is resumed at the next TT L.
In some embodiments, the method further comprises:
when detecting that the node ID of the node itself exists in a database data directory of PostgreSQL L, the node is connected to DCS and then starts a PostgreSQL L database instance in a master-node-free mode;
after the database is started, the node obtains L eader Key parameter values from the DCS system, and determines the current main node according to L eader Key parameters;
if the L reader Key parameter is that the node is a main node, promoting the node to be the main node;
if another node is the master node, the node follows the master node to become a slave node;
if the L reader Key parameter is null, the node will first go through health check and promote to the master node after passing through the health check.
And the master node and the slave node in the cluster synchronize the position information value of the WA L log of the node database into the DCS at regular intervals, and when the cluster fails, the master node and the slave node need to acquire the WA L log position information from the DCS for database recovery.
In some embodiments, the method further comprises the DCS system copying the L reader Key parameter to each node in the cluster other than the master node.
The technical scheme of the invention has the following beneficial effects:
(1) the existing DCS system cluster is adopted for uniformly configuring and managing the PostgreSQL L cluster, the invention of wheels is not required to be repeated, the cluster is more robust and stable, and the consistency of parameters and states of all nodes of the cluster is kept;
(2) the cluster brain split problem can be avoided, and high fault tolerance and stability are achieved;
(3) the high availability of the PostgreSQL L database cluster is improved, and the fault recovery time of the database cluster is shortened;
(4) the cluster mechanism is simple, flexible in configuration and easy to maintain.
Drawings
Fig. 1 is a flowchart illustrating a method for constructing a PostgreSQ L database cluster according to an embodiment of the present invention.
Detailed Description
In order to make the technical problems, technical solutions and advantages of the present invention more apparent, the following detailed description is given with reference to the accompanying drawings and specific embodiments.
The embodiment of the invention provides a method for cooperatively constructing a PostgreSQL L database cluster by adopting a distributed configuration storage system.
The distributed configuration storage system (DCS) stores configuration and state information of a PostgreSQL L database by adopting key values and interacts with a PostgreSQL L database by using a REST API (representational State application program interface), wherein the DCS can be distributed on each node of a cluster and can also be independently and centrally arranged, in order to prevent the DCS node from failing, a large-scale DCS cluster can be applied to perform disaster prevention backup in the embodiment of the invention, meanwhile, the DCS adopts a distributed consistent algorithm in the embodiment of the invention, and all nodes changed by the key values in the cluster can be automatically distributed and copied, so that the parameter consistency of the cluster is ensured.
Specifically, the system provided by the embodiment of the invention is established by deploying a distributed configuration storage system (DCS) cluster for storing parameter information and state information of a PostgreSQL L database, constructing a multi-point cluster by adopting a PostgreSQL L stream replication technology, and having the functions of creating a PostgreSQL L cluster, starting stream replication, automatically recovering faults and the like.
The method for constructing the PostgreSQ L database cluster according to the embodiment of the present invention may be exemplarily divided into the following three stages, it should be understood that the segmentation in the embodiment of the present invention is only for easy understanding and is not a limitation to the embodiment of the present invention, and it should be understood that these stages may be executed in any time slot:
1. managing uninitialized database phases
The method comprises the steps of adding an uninitialized node into a master node election program when the uninitialized node appears, initializing a first node in a cluster to become a master node, setting L reader Key of the cluster as the node ID of the node, initializing a database after the master node starts a PostgreSQL L instance, starting PostgreSQL L instances after other nodes need to wait for setting L reader Key in the cluster, cloning the database from the master node to become slave nodes, wherein L reader Key parameters are automatically copied to other nodes in the cluster through a DCS system.
The L reader Key of each node has TT L parameter, generally TT L is set to 30 seconds, this means that the cluster automatically checks L reader Key every 30 seconds to determine whether the master node is alive, if L reader Key is found to be empty after 30, it indicates that the master node is down due to failure, the cluster will enter the master node election program.
2. Managing an initialization database phase
When the database is started and operated, the node obtains L eader Key parameter values from the DCS system, if the parameter values are the ID of the node, the node is promoted to be a master node, if the other node is the master node, the node starts to follow the master node to be a slave node, and if L eader Key is empty, the node firstly carries out health check and is promoted to be the master node after the health check.
And the master node and the slave node in the cluster synchronize the position information value of the WA L log of the node database into the DCS at regular intervals, and when the cluster fails, the master node and the slave node need to acquire the WA L log position information from the DCS for database recovery.
3. Stage of electing main node
All nodes in the cluster check the L eader Key every TT L (which can be set, for example, 30 seconds), once the L eader Key changes, all nodes will perform the election master node procedure, if the L eader Key does not change in TT L, then the recheck is performed only at the next TT L.
The method specifically comprises the steps of checking WA L log position information in a DCS in each node and needing node health check, comparing whether any node is more ahead than WA L log position information of the node, selecting the node as a main node if WA L log position information of the node is more ahead, and selecting the node as the main node if no WA L log position of the node is more ahead.
If the master node elects, the ID of the node is set to L reader Key, the TT L value is set to 30 seconds, then the selected master node promotes the node to be the master node, and if the process fails, the node is waited to assume the role of the master node again.
The mechanism has the functions of creating a cluster, starting stream replication, processing synchronous requirements, monitoring the states of master nodes and slave nodes of the cluster, and modifying the configuration of a cluster member database. A reboot loading configuration file or reboot database operation may be performed on the selected cluster member. Planned node switchover and unplanned failover can be achieved. The failed master node can be rolled back to restore it to normal, and all the duplicate connections are restarted to point to the newly promoted master node.
The parameter and the state information of the PostgreSQL L database are stored by using a DCS multi-node cluster, and the consistency of the parameter and the state information of the PostgreSQL L cluster is ensured by using a distributed consistency algorithm of the DCS cluster, so that cluster brain cracks are avoided.
The cluster uses a stream replication technique built into the PostgreSQL L database to achieve data synchronization of the primary and backup nodes, this mechanism applies to all versions beyond PostgreSQL L.0. according to the data synchronization mechanism supported by the PostgreSQL L version, asynchronous replication and synchronous replication can be achieved.
When the cluster master node is unavailable, the slave nodes execute a node health check program, and elect the master node by comparing the health degree, the health degree check mainly compares the position information of WA L logs received and played back by all the current slave nodes, and the slave node with the shortest distance from the position of the playback WA L log to the position of the original master node WA L log is elected as the master node (the state information of the master and slave nodes WA L logs needs to be stored in the DCS).
L reader key is a key parameter in the cluster, stored in the DCS for each node in the cluster, holding the ID value of the cluster master node L reader key has TT L attribute, set to 30 seconds in general, check for the presence or change of L reader key value every 30 seconds in a loop, which is equivalent to a heartbeat for each node in the cluster.
While the foregoing is directed to the preferred embodiment of the present invention, it will be understood by those skilled in the art that various changes and modifications may be made without departing from the spirit and scope of the invention as defined in the appended claims.
Claims (6)
1. A method for constructing a PostgreSQL L database cluster is characterized by comprising the following steps:
initializing the nodes: initializing uninitialized nodes and adding the nodes into a candidate node list of a main node;
initializing a main node, namely setting the node as the main node and setting L eader Key parameter of a cluster as the node ID of the node when only one node exists in a candidate node list of the main node, starting a PostgreSQL L instance and initializing a database by the main node;
a slave node synchronization step, namely starting a PostgreSQL L instance by the slave node after detecting L eader Key of the cluster, and cloning the database from the master node to become the slave node;
and in the step of updating the main node, other nodes need to wait for setting L eader keys in the cluster to start a PostgreSQL L instance, and clone the database from the main node into a slave node.
2. The method for constructing the PostgreSQL L database cluster according to claim 1, wherein the L eader Key Key value of each node has the parameter of life cycle TT L, and when the life cycle is reached, the following main node updating steps are carried out:
at the expiration of TT L, the cluster checks the L reader Key parameter to determine if the master node is alive, updates the TT L value if the master node is normal, otherwise performs the master node election step.
3. The method of building a PostgreSQ L database cluster according to claim 2, wherein the master node election step comprises:
when detecting that L reader Key is null, checking the position of the node in WA L log position information in the DCS and the corresponding health check result, and determining the master node.
4. The method of building a PostgreSQ L database cluster according to claim 2, wherein the master node election step further comprises:
if the master node survives, the current master node is maintained and the master node update step is resumed at the next TT L.
5. The method of building a PostgreSQ L database cluster according to claim 1, further comprising:
when detecting that the node ID of the node itself exists in a database data directory of PostgreSQL L, the node is connected to DCS and then starts a PostgreSQL L database instance in a master-node-free mode;
after the database is started, the node obtains L reader Key parameter values from the DCS system, and determines the current master node according to L reader Key parameters;
if the L reader Key parameter is that the node is a main node, promoting the node to be the main node;
if another node is the master node, the node follows the master node to become a slave node;
if the L reader Key parameter is null, the node will first go through health check and promote to the master node after passing through the health check.
And the master node and the slave node in the cluster synchronize the position information value of the WA L log of the node database into the DCS at regular intervals, and when the cluster fails, the master node and the slave node need to acquire the WA L log position information from the DCS for database recovery.
6. The method of building a PostgreSQL L database cluster according to claim 1, further comprising the DCS system copying the L eader Key parameter to each node in the cluster except the master node.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010175693.3A CN111459909A (en) | 2020-03-13 | 2020-03-13 | Method for constructing PostgreSQL L database cluster |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010175693.3A CN111459909A (en) | 2020-03-13 | 2020-03-13 | Method for constructing PostgreSQL L database cluster |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111459909A true CN111459909A (en) | 2020-07-28 |
Family
ID=71684481
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010175693.3A Pending CN111459909A (en) | 2020-03-13 | 2020-03-13 | Method for constructing PostgreSQL L database cluster |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111459909A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113328899A (en) * | 2021-08-04 | 2021-08-31 | 苏州浪潮智能科技有限公司 | Fault processing method and system for cluster nodes |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102202425A (en) * | 2011-06-24 | 2011-09-28 | 中国人民解放军国防科学技术大学 | Satellite cluster self-organization networking method based on master-slave heterogeneous data transmission module |
| CN105095486A (en) * | 2015-08-17 | 2015-11-25 | 浪潮(北京)电子信息产业有限公司 | Cluster database disaster recovery method and device |
| CN107948017A (en) * | 2017-09-30 | 2018-04-20 | 用友金融信息技术股份有限公司 | Highly available cluster system |
| CN110635941A (en) * | 2019-08-30 | 2019-12-31 | 苏州浪潮智能科技有限公司 | Database node cluster fault migration method and device |
| CN110647580A (en) * | 2019-09-05 | 2020-01-03 | 南京邮电大学 | Distributed container cluster mirror image management main node, slave node, system and method |
-
2020
- 2020-03-13 CN CN202010175693.3A patent/CN111459909A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102202425A (en) * | 2011-06-24 | 2011-09-28 | 中国人民解放军国防科学技术大学 | Satellite cluster self-organization networking method based on master-slave heterogeneous data transmission module |
| CN105095486A (en) * | 2015-08-17 | 2015-11-25 | 浪潮(北京)电子信息产业有限公司 | Cluster database disaster recovery method and device |
| CN107948017A (en) * | 2017-09-30 | 2018-04-20 | 用友金融信息技术股份有限公司 | Highly available cluster system |
| CN110635941A (en) * | 2019-08-30 | 2019-12-31 | 苏州浪潮智能科技有限公司 | Database node cluster fault migration method and device |
| CN110647580A (en) * | 2019-09-05 | 2020-01-03 | 南京邮电大学 | Distributed container cluster mirror image management main node, slave node, system and method |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113328899A (en) * | 2021-08-04 | 2021-08-31 | 苏州浪潮智能科技有限公司 | Fault processing method and system for cluster nodes |
| CN113328899B (en) * | 2021-08-04 | 2021-10-22 | 苏州浪潮智能科技有限公司 | Fault processing method and system for cluster nodes |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11360854B2 (en) | Storage cluster configuration change method, storage cluster, and computer system | |
| US7620845B2 (en) | Distributed system and redundancy control method | |
| US7779295B1 (en) | Method and apparatus for creating and using persistent images of distributed shared memory segments and in-memory checkpoints | |
| WO2021136422A1 (en) | State management method, master and backup application server switching method, and electronic device | |
| CN106062717A (en) | Distributed storage replication system and method | |
| GB2484086A (en) | Reliability and performance modes in a distributed storage system | |
| CN110515557B (en) | Cluster management method, device and equipment and readable storage medium | |
| CN112579354B (en) | Method for backup and recovery of edge cloud collaborative software | |
| CN105915391B (en) | The distributed key assignments storage method of self-recovering function is submitted and had based on single phase | |
| CN108614876B (en) | Redis database-based system and data processing method | |
| CN116680256B (en) | Database node upgrading method and device and computer equipment | |
| JP6511739B2 (en) | Redundant system and redundant method | |
| JPWO2008129620A1 (en) | Fully duplexed system, system control method and system control program | |
| US10078558B2 (en) | Database system control method and database system | |
| CN115801799A (en) | System for managing middleware life cycle based on kubernets multiple clusters | |
| CN111752488A (en) | Management method and device of storage cluster, management node and storage medium | |
| CN111459909A (en) | Method for constructing PostgreSQL L database cluster | |
| CN113986450A (en) | Virtual machine backup method and device | |
| CN107786650B (en) | Storage medium and management strategy synchronization method, device and system | |
| CN113438111A (en) | Method for restoring RabbitMQ network partition based on Raft distribution and application | |
| CN112231150B (en) | Method and device for recovering fault database in database cluster | |
| CN115878361A (en) | Node management method and device for database cluster and electronic equipment | |
| JP2010146044A (en) | Redundant system | |
| JP2009265973A (en) | Data synchronization system, failure recovery method, and program | |
| CN115587099A (en) | Distributed meter lock application method and device, storage medium and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200728 |