CN111030858A - Data management method, system and related device for distributed multi-cluster system - Google Patents
Data management method, system and related device for distributed multi-cluster system Download PDFInfo
- Publication number
- CN111030858A CN111030858A CN201911243194.7A CN201911243194A CN111030858A CN 111030858 A CN111030858 A CN 111030858A CN 201911243194 A CN201911243194 A CN 201911243194A CN 111030858 A CN111030858 A CN 111030858A
- Authority
- CN
- China
- Prior art keywords
- cluster
- data
- distributed multi
- interface
- sending
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/08—Configuration management of networks or network elements
- H04L41/0893—Assignment of logical groups to network elements
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computing Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application provides a data management method of a distributed multi-cluster system, which comprises the following steps: acquiring a virtual IP of each cluster in the distributed multi-cluster system; taking the cluster with the minimum virtual IP as a central cluster, and deploying a data center on the central cluster; the data center comprises a query interface and a data updating interface; after the central cluster is determined, deploying sending units on all other clusters in the distributed multi-cluster system, where the sending units are used to send query requests for data information to the query interface and/or send update requests for data information to the data update interface. The method and the device can improve the efficiency of multi-cluster data management, improve the performance of the distributed multi-cluster system, and manage the distributed multi-cluster system more conveniently. The application also provides a data management system of the distributed multi-cluster system, a computer readable storage medium and the distributed multi-cluster system, which have the beneficial effects.
Description
Technical Field
The present application relates to the field of servers, and in particular, to a data management method and system for a distributed multi-cluster system, and a related device.
Background
At present, in management of multiple clusters, a central cluster actively sends a mode to other clusters at regular time to manage data information common to the multiple clusters, the mode needs to configure a push destination and a timing push service in the central cluster in advance, and actively sends the data information to other clusters when the service is started.
Disclosure of Invention
The application aims to provide a data management method and system of a distributed multi-cluster system, a computer readable storage medium and the distributed multi-cluster system, which can improve the multi-cluster management efficiency.
In order to solve the above technical problem, the present application provides a data management method for a distributed multi-cluster system, and the specific technical solution is as follows:
acquiring a virtual IP of each cluster in the distributed multi-cluster system;
taking the cluster with the minimum virtual IP as a central cluster, and deploying a data center on the central cluster; the data center comprises a query interface and a data updating interface;
after the central cluster is determined, deploying sending units on all other clusters in the distributed multi-cluster system, where the sending units are used to send query requests for data information to the query interface and/or send update requests for data information to the data update interface.
Wherein, when the distributed multi-cluster system adds a new cluster, the method further comprises:
deploying the sending unit on the new cluster.
Wherein when the data center is removed, further comprising:
and re-acquiring the virtual IP of the rest clusters in the distributed multi-cluster system, and taking the cluster with the minimum current virtual IP as a central cluster.
The sending unit is configured to send a query request of data information to the query interface, and/or send an update request of data information to the data update interface includes:
the sending unit is used for sending a query request of data information to the query interface periodically according to the preset frequency of the cluster where the sending unit is located, and/or sending an update request of the data information to the data update interface. .
The application provides a data management system of distributed multi-cluster system, includes:
the acquisition module is used for acquiring the virtual IP of each cluster in the distributed multi-cluster system;
the first deployment module is used for taking the cluster with the minimum virtual IP as a central cluster and deploying a data center on the central cluster; the data center comprises a query interface and a data updating interface;
a second deployment module, configured to deploy sending units on all other clusters in the distributed multi-cluster system after determining the central cluster; the sending unit is a unit for sending a query request of data information to the query interface and/or sending an update request of data information to the data update interface.
Wherein, still include:
a third deployment module for deploying the sending unit on the new cluster.
Wherein, still include:
and the central cluster deployment module is used for re-acquiring the virtual IP of the rest clusters in the distributed multi-cluster system and taking the cluster with the minimum current virtual IP as the central cluster.
The present application also provides a computer-readable storage medium having stored thereon a computer program which, when being executed by a processor, carries out the steps of the data management method as described above.
The present application further provides a distributed multi-cluster system, which includes a memory and a processor, where the memory stores a computer program, and the processor implements the steps of the data management method when calling the computer program in the memory.
The application provides a data management method of a distributed multi-cluster system, which comprises the following steps: acquiring a virtual IP of each cluster in the distributed multi-cluster system; taking the cluster with the minimum virtual IP as a central cluster, and deploying a data center on the central cluster; the data center comprises a query interface and a data updating interface; after the central cluster is determined, deploying sending units on all other clusters in the distributed multi-cluster system, where the sending units are used to send query requests for data information to the query interface and/or send update requests for data information to the data update interface.
According to the method, the data center is deployed in the center cluster, the sending units are deployed in other clusters, the center cluster does not need to actively push services to other clusters in the system regularly, each cluster can send query requests and/or update requests to the center cluster through the sending unit of the cluster according to the updating requirements of the cluster, and meanwhile any service of the center cluster does not need to be modified during updating, so that the inter-cluster communication frequency of the distributed multi-cluster system is reduced, frequent data updating and copying are avoided, communication resources are saved, and the performance of the distributed multi-cluster system is improved. The efficiency of multi-cluster data management can be improved, the performance of the distributed multi-cluster system is improved, and the distributed multi-cluster system is more convenient to manage. The application also provides a data management system of the distributed multi-cluster system, a computer readable storage medium and the distributed multi-cluster system, which have the above beneficial effects and are not described herein again.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below, it is obvious that the drawings in the following description are only embodiments of the present application, and for those skilled in the art, other drawings can be obtained according to the provided drawings without creative efforts.
Fig. 1 is a flowchart of a data management method of a distributed multi-cluster system according to an embodiment of the present application;
fig. 2 is a schematic structural diagram of a data management system of a distributed multi-cluster system according to an embodiment of the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some embodiments of the present application, but not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
Referring to fig. 1, fig. 1 is a flowchart of a data management method of a distributed multi-cluster system according to an embodiment of the present application, where the data management method includes:
s101: acquiring a virtual IP of each cluster in the distributed multi-cluster system;
s102: taking the cluster with the minimum virtual IP as a central cluster, and deploying a data center on the central cluster; the data center comprises a query interface and a data updating interface;
s103: after the central cluster is determined, deploying sending units on all other clusters in the distributed multi-cluster system; the sending unit is used for sending a query request of the data information to the query interface and/or sending an update request of the data information to the data update interface.
In a distributed multi-cluster system, a virtual IP of each cluster is first acquired. The cluster with the minimum virtual IP serves as a central cluster, a data center is deployed on the central cluster, and the data center provides a query interface and a data updating interface of data information of public external services. In particular, the data center can provide the following functions that firstly, query requests of other clusters are received through the query interface, then whether the data information of the data center is updated or not is queried, and the query requests are correspondingly replied. Secondly, after receiving the update requests of other clusters through the data information update interface, the data information of the data center is pushed to the corresponding cluster, and the data information update of the requested cluster is completed.
After the central cluster is determined, a sending unit is deployed on other clusters of the distributed multi-cluster system, and the sending unit sends a query request and an update request of data information to complete the following functions: firstly, sending a data information query request to a data center of a center cluster, and after receiving a query return of the data center, determining whether to continue to initiate an update request according to a return result. Secondly, a data information updating request is sent to a data center of the center cluster to request for updating the data information.
Specifically, the sending unit may send the query request of the data information to the query interface at regular intervals according to a preset frequency of the cluster in which the sending unit is located, and/or send the update request of the data information to the data update interface. In the prior art, query and update of each cluster are mainly completed by a central cluster, and generally, update service is regularly pushed to each cluster, and then each cluster receives the push and sends a query request and an update request to the central cluster. It is readily understood that the different cluster update requirements are not the same. Some clusters have high updating frequency, and some clusters have low updating frequency. This results in a waste of information transmission process, which is essentially a waste of communication resources between clusters. However, in the embodiment of the present application, the sending unit is added to each cluster, so that each sending unit can send the query request and the update request to the central cluster according to the situation of the cluster where the sending unit is located, thereby greatly reducing the waste of communication resources between clusters, simultaneously improving the resource utilization rate of the central cluster, and facilitating the central cluster to realize the management of other clusters in the system.
In addition, in the prior art, since the update process depends on the push service of the central cluster, once the update requirement of a certain cluster or some clusters is increased, the central server needs to be reconfigured to meet the update requirement of individual clusters. The embodiment of the application can directly modify the corresponding cluster configuration, and once the updating requirement of the cluster is improved, the request sending frequency of the sending unit of the cluster can be correspondingly improved so as to meet the updating requirement of the cluster without modifying any service of the central cluster.
On the basis of the above embodiment, as a preferred embodiment, when a new cluster is added to the distributed multi-cluster system, the sending unit may also be deployed on the new cluster.
When a new cluster is added to the distributed multi-cluster system, the sending unit is only needed to be deployed on the new cluster, and then the sending unit can be used for sending query information or data updating information to a query interface or a data updating interface of a data center, so that the new cluster is added to the distributed multi-cluster system.
According to the embodiment of the application, the data center is deployed in the center cluster, the sending unit is deployed in other clusters, the center cluster does not need to actively push service to other clusters in the system regularly, each cluster can send the query request and/or the update request to the center cluster through the sending unit according to the update requirement of the cluster, and meanwhile any service of the center cluster does not need to be modified during updating, so that the inter-cluster communication frequency of the distributed multi-cluster system is reduced, frequent data update copying is avoided, communication resources are saved, and the performance of the distributed multi-cluster system is improved. The efficiency of multi-cluster data management can be improved, the performance of the distributed multi-cluster system is improved, and the distributed multi-cluster system is more convenient to manage.
On the basis of the above embodiment, as a preferred embodiment, when the data center is removed, the method may further include:
and re-acquiring the virtual IP of the rest clusters in the distributed multi-cluster system, and taking the cluster with the minimum current virtual IP as a central cluster.
No matter what reason the central cluster in the current distributed multi-cluster system is removed, the virtual IP of the rest clusters in the system summary needs to be reacquired, and the cluster with the minimum virtual IP is used as the central cluster again to relocate the data center, so as to ensure that one central cluster exists in the distributed multi-cluster system most of the time, and the query request and the data update request sent by the rest clusters in the distributed multi-cluster system can be received, so as to meet the query update requirements of each cluster in the distributed multi-cluster system.
In the following, a data management system of a distributed multi-cluster system provided by an embodiment of the present application is introduced, and the data management system described below and the data management method described above may be referred to correspondingly.
Referring to fig. 2, fig. 2 is a schematic structural diagram of a data management system of a distributed multi-cluster system according to an embodiment of the present application, where the data management system of the distributed multi-cluster system includes:
an obtaining module 100, configured to obtain a virtual IP of each cluster in the distributed multi-cluster system;
a first deployment module 200, configured to use a cluster with the smallest virtual IP as a central cluster, and deploy a data center on the central cluster; the data center comprises a query interface and a data updating interface;
a second deployment module 300, configured to deploy sending units on all other clusters in the distributed multi-cluster system after determining the central cluster; the sending unit is a unit for sending a query request of data information to the query interface and/or sending an update request of data information to the data update interface.
Based on the above embodiment, as a preferred embodiment, the method may further include:
a third deployment module for deploying the sending unit on the new cluster.
Based on the above embodiment, as a preferred embodiment, the method may further include:
and the central cluster deployment module is used for re-acquiring the virtual IP of the rest clusters in the distributed multi-cluster system and taking the cluster with the minimum current virtual IP as the central cluster.
The present application also provides a computer readable storage medium having stored thereon a computer program which, when executed, may implement the steps provided by the above-described embodiments. The storage medium may include: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.
The present application further provides a distributed multi-cluster system, which may include a memory and a processor, where the memory stores a computer program, and when the processor calls the computer program in the memory, the steps provided in the foregoing embodiments may be implemented. Of course, the distributed multi-cluster system may also include various network interfaces, power supplies, and other components.
The embodiments are described in a progressive manner in the specification, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other. For the system provided by the embodiment, the description is relatively simple because the system corresponds to the method provided by the embodiment, and the relevant points can be referred to the method part for description.
The principles and embodiments of the present application are explained herein using specific examples, which are provided only to help understand the method and the core idea of the present application. It should be noted that, for those skilled in the art, it is possible to make several improvements and modifications to the present application without departing from the principle of the present application, and such improvements and modifications also fall within the scope of the claims of the present application.
It is further noted that, in the present specification, relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
Claims (9)
1. A data management method of a distributed multi-cluster system is characterized by comprising the following steps:
acquiring a virtual IP of each cluster in the distributed multi-cluster system;
taking the cluster with the minimum virtual IP as a central cluster, and deploying a data center on the central cluster; the data center comprises a query interface and a data updating interface;
after the central cluster is determined, deploying sending units on all other clusters in the distributed multi-cluster system; the sending unit is used for sending a query request of data information to a query interface of the data center and/or sending an update request of the data information to a data update interface of the data center.
2. The data management method of claim 1, when the distributed multi-cluster system adds a new cluster, further comprising:
deploying the sending unit on the new cluster.
3. The data management method of claim 1, when the data center is removed, further comprising:
and re-acquiring the virtual IP of the rest clusters in the distributed multi-cluster system, and taking the cluster with the minimum current virtual IP as a central cluster.
4. The data management method according to claim 1, wherein the sending unit is configured to send a query request of data information to the query interface, and/or send an update request of data information to the data update interface includes:
the sending unit is used for sending a query request of data information to the query interface periodically according to the preset frequency of the cluster where the sending unit is located, and/or sending an update request of the data information to the data update interface.
5. A data management system for a distributed multi-cluster system, comprising:
the acquisition module is used for acquiring the virtual IP of each cluster in the distributed multi-cluster system;
the first deployment module is used for taking the cluster with the minimum virtual IP as a central cluster and deploying a data center on the central cluster; the data center comprises a query interface and a data updating interface;
a second deployment module, configured to deploy sending units on all other clusters in the distributed multi-cluster system after determining the central cluster; the sending unit is a unit for sending a query request of data information to the query interface and/or sending an update request of data information to the data update interface.
6. The data management system of claim 5, further comprising:
a third deployment module for deploying the sending unit on the new cluster.
7. The data management system of claim 5, further comprising:
and the central cluster deployment module is used for re-acquiring the virtual IP of the rest clusters in the distributed multi-cluster system and taking the cluster with the minimum current virtual IP as the central cluster.
8. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the data management method according to any one of claims 1 to 4.
9. A distributed multi-cluster system comprising a memory having a computer program stored therein and a processor implementing the steps of the data management method according to any one of claims 1 to 4 when the processor invokes the computer program in the memory.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201911243194.7A CN111030858B (en) | 2019-12-06 | 2019-12-06 | Data management method, system and related device for distributed multi-cluster system |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201911243194.7A CN111030858B (en) | 2019-12-06 | 2019-12-06 | Data management method, system and related device for distributed multi-cluster system |
Publications (2)
Publication Number | Publication Date |
---|---|
CN111030858A true CN111030858A (en) | 2020-04-17 |
CN111030858B CN111030858B (en) | 2023-04-07 |
Family
ID=70204590
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201911243194.7A Active CN111030858B (en) | 2019-12-06 | 2019-12-06 | Data management method, system and related device for distributed multi-cluster system |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN111030858B (en) |
Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN103634144A (en) * | 2013-11-15 | 2014-03-12 | 新浪网技术(中国)有限公司 | Method, system and equipment for managing multi-IDC (internet data center) cluster configuration file |
CN104113597A (en) * | 2014-07-18 | 2014-10-22 | 西安交通大学 | Multi- data-centre hadoop distributed file system (HDFS) data read-write system and method |
CN105208054A (en) * | 2014-06-16 | 2015-12-30 | 中兴通讯股份有限公司 | Cloud scheduling management method, management center and management system |
CN108769199A (en) * | 2018-05-29 | 2018-11-06 | 郑州云海信息技术有限公司 | A kind of distributed file storage system host node management method and device |
-
2019
- 2019-12-06 CN CN201911243194.7A patent/CN111030858B/en active Active
Patent Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN103634144A (en) * | 2013-11-15 | 2014-03-12 | 新浪网技术(中国)有限公司 | Method, system and equipment for managing multi-IDC (internet data center) cluster configuration file |
CN105208054A (en) * | 2014-06-16 | 2015-12-30 | 中兴通讯股份有限公司 | Cloud scheduling management method, management center and management system |
CN104113597A (en) * | 2014-07-18 | 2014-10-22 | 西安交通大学 | Multi- data-centre hadoop distributed file system (HDFS) data read-write system and method |
CN108769199A (en) * | 2018-05-29 | 2018-11-06 | 郑州云海信息技术有限公司 | A kind of distributed file storage system host node management method and device |
Also Published As
Publication number | Publication date |
---|---|
CN111030858B (en) | 2023-04-07 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN106059825A (en) | Distributed system and configuration method | |
CN108551399B (en) | Service deployment method, system and related device in cloud environment | |
WO2021227999A1 (en) | Cloud computing service system and method | |
CN109766174B (en) | Resource scheduling method, resource scheduling apparatus, and computer-readable storage medium | |
CN102821000A (en) | Method for improving usability of PaaS platform | |
CN111045745A (en) | Method and system for managing configuration information | |
CN110727738B (en) | Global routing system based on data fragmentation, electronic equipment and storage medium | |
CN111427670A (en) | Task scheduling method and system | |
CN108282526B (en) | Dynamic allocation method and system for servers between double clusters | |
CN109428926B (en) | Method and device for scheduling task nodes | |
CN111030858B (en) | Data management method, system and related device for distributed multi-cluster system | |
CN111382132A (en) | Medical image data cloud storage system | |
CN103023990A (en) | Image file upgrade system and method in stack system | |
CN113032134B (en) | Method and device for realizing cloud computing resource allocation and cloud management server | |
CN111078322A (en) | Server and K8S cluster-based public configuration parameter configuration method and system | |
CN111064643A (en) | Node server and data interaction method and related device thereof | |
CN111737052A (en) | Distributed object storage system and method | |
CN116055499A (en) | Method, equipment and medium for intelligently scheduling cluster tasks based on redis | |
CN109344202B (en) | Data synchronization method and management node | |
CN110798358A (en) | Distributed service identification method and device, computer readable medium and electronic equipment | |
CN107534678B (en) | Method, device and system for establishing connection between VNFM and VIM | |
CN110909048A (en) | Data query method, device, server, client and storage medium | |
CN107273047B (en) | Cache management method, cache manager and storage management software | |
CN115981670A (en) | Container cluster service deployment method, device, server and storage medium | |
CN113687962A (en) | Request processing method, device, equipment and storage medium |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |