CN111030858B - Data management method, system and related device for distributed multi-cluster system - Google Patents
Data management method, system and related device for distributed multi-cluster system Download PDFInfo
- Publication number
- CN111030858B CN111030858B CN201911243194.7A CN201911243194A CN111030858B CN 111030858 B CN111030858 B CN 111030858B CN 201911243194 A CN201911243194 A CN 201911243194A CN 111030858 B CN111030858 B CN 111030858B
- Authority
- CN
- China
- Prior art keywords
- cluster
- data
- distributed multi
- interface
- sending
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/08—Configuration management of networks or network elements
- H04L41/0893—Assignment of logical groups to network elements
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computing Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application provides a data management method of a distributed multi-cluster system, which comprises the following steps: acquiring a virtual IP of each cluster in the distributed multi-cluster system; taking the cluster with the minimum virtual IP as a central cluster, and deploying a data center on the central cluster; the data center comprises a query interface and a data updating interface; after the central cluster is determined, deploying sending units on all other clusters in the distributed multi-cluster system, where the sending units are used to send query requests for data information to the query interface and/or send update requests for data information to the data update interface. The method and the device can improve the efficiency of multi-cluster data management, improve the performance of the distributed multi-cluster system, and manage the distributed multi-cluster system more conveniently. The application also provides a data management system of the distributed multi-cluster system, a computer readable storage medium and the distributed multi-cluster system, which have the beneficial effects.
Description
Technical Field
The present application relates to the field of servers, and in particular, to a data management method and system for a distributed multi-cluster system, and a related device.
Background
At present, in management of multiple clusters, a central cluster actively sends a mode to other clusters at regular time to manage data information common to the multiple clusters, the mode needs to configure a push destination and a timing push service in the central cluster in advance, and actively sends the data information to other clusters when the service is started.
Disclosure of Invention
The application aims to provide a data management method and system of a distributed multi-cluster system, a computer readable storage medium and the distributed multi-cluster system, which can improve the multi-cluster management efficiency.
In order to solve the above technical problem, the present application provides a data management method for a distributed multi-cluster system, which has the following specific technical scheme:
acquiring a virtual IP of each cluster in the distributed multi-cluster system;
taking the cluster with the minimum virtual IP as a central cluster, and deploying a data center on the central cluster; the data center comprises a query interface and a data updating interface;
after the central cluster is determined, deploying sending units on all other clusters in the distributed multi-cluster system, where the sending units are used to send query requests for data information to the query interface and/or send update requests for data information to the data update interface.
Wherein, when the distributed multi-cluster system adds a new cluster, the method further comprises:
deploying the sending unit on the new cluster.
Wherein when the data center is removed, further comprising:
and re-acquiring the virtual IP of the rest clusters in the distributed multi-cluster system, and taking the cluster with the minimum current virtual IP as a central cluster.
The sending unit is configured to send a query request for data information to the query interface, and/or send an update request for data information to the data update interface includes:
the sending unit is used for sending a query request of data information to the query interface periodically according to the preset frequency of the cluster where the sending unit is located, and/or sending an update request of the data information to the data update interface. .
The application provides a data management system of distributed multi-cluster system, including:
the acquisition module is used for acquiring the virtual IP of each cluster in the distributed multi-cluster system;
the first deployment module is used for taking the cluster with the minimum virtual IP as a central cluster and deploying a data center on the central cluster; the data center comprises a query interface and a data updating interface;
a second deployment module, configured to deploy sending units on all other clusters in the distributed multi-cluster system after determining the central cluster; the sending unit is a unit for sending a query request of data information to the query interface and/or sending an update request of data information to the data update interface.
Wherein, still include:
a third deployment module for deploying the sending unit on the new cluster.
Wherein, still include:
and the central cluster deployment module is used for re-acquiring the virtual IP of the rest clusters in the distributed multi-cluster system and taking the cluster with the minimum current virtual IP as the central cluster.
The present application also provides a computer-readable storage medium having stored thereon a computer program which, when being executed by a processor, carries out the steps of the data management method as described above.
The present application further provides a distributed multi-cluster system, which includes a memory and a processor, where the memory stores a computer program, and the processor implements the steps of the data management method when calling the computer program in the memory.
The application provides a data management method of a distributed multi-cluster system, which comprises the following steps: acquiring a virtual IP of each cluster in the distributed multi-cluster system; taking the cluster with the minimum virtual IP as a central cluster, and deploying a data center on the central cluster; the data center comprises a query interface and a data updating interface; after the central cluster is determined, deploying sending units on all other clusters in the distributed multi-cluster system, where the sending units are used to send query requests for data information to the query interface and/or send update requests for data information to the data update interface.
According to the method, the data center is deployed in the center cluster, the sending unit is deployed in other clusters, the center cluster does not need to actively push services to other clusters in the system periodically, each cluster can send query requests and/or update requests to the center cluster through the sending unit of the cluster according to updating requirements of the cluster, and meanwhile any service of the center cluster does not need to be modified during updating, so that the inter-cluster communication frequency of the distributed multi-cluster system is reduced, frequent data updating and copying are avoided, communication resources are saved, and the performance of the distributed multi-cluster system is improved. The efficiency of multi-cluster data management can be improved, the performance of the distributed multi-cluster system is improved, and the distributed multi-cluster system is more convenient to manage. The present application further provides a data management system of a distributed multi-cluster system, a computer readable storage medium, and a distributed multi-cluster system, which have the above beneficial effects and are not described herein again.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below, it is obvious that the drawings in the following description are only embodiments of the present application, and for those skilled in the art, other drawings can be obtained according to the provided drawings without creative efforts.
Fig. 1 is a flowchart of a data management method of a distributed multi-cluster system according to an embodiment of the present application;
fig. 2 is a schematic structural diagram of a data management system of a distributed multi-cluster system according to an embodiment of the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some embodiments of the present application, but not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
Referring to fig. 1, fig. 1 is a flowchart of a data management method of a distributed multi-cluster system according to an embodiment of the present disclosure, where the data management method includes:
s101: acquiring a virtual IP of each cluster in the distributed multi-cluster system;
s102: taking the cluster with the minimum virtual IP as a central cluster, and deploying a data center on the central cluster; the data center comprises a query interface and a data updating interface;
s103: after the central cluster is determined, deploying sending units on all other clusters in the distributed multi-cluster system; the sending unit is used for sending a query request of the data information to the query interface and/or sending an update request of the data information to the data update interface.
In a distributed multi-cluster system, the virtual IP of each cluster is first obtained. The cluster with the minimum virtual IP serves as a central cluster, a data center is deployed on the central cluster, and the data center provides a query interface and a data updating interface of data information of public external services. Specifically, the data center can provide the following functions that firstly, query requests of other clusters are received through the query interface, then whether data information of the data center is updated or not is queried, and the query requests are correspondingly replied. Secondly, after receiving the update requests of other clusters through the data information update interface, the data information of the data center is pushed to the corresponding cluster, and the data information update of the requested cluster is completed.
After the central cluster is determined, a sending unit is deployed on other clusters of the distributed multi-cluster system, and the sending unit sends a query request and an update request of data information to complete the following functions: firstly, sending a data information query request to a data center of a center cluster, and after receiving a query return of the data center, determining whether to continue to initiate an update request according to a return result. Secondly, a data information updating request is sent to a data center of the center cluster to request for updating the data information.
Specifically, the sending unit may send the query request of the data information to the query interface at regular intervals according to a preset frequency of the cluster in which the sending unit is located, and/or send the update request of the data information to the data update interface. In the prior art, query and update of each cluster are mainly completed by a central cluster, and generally, update service is regularly pushed to each cluster, and then each cluster receives the push and sends a query request and an update request to the central cluster. It is readily understood that the different cluster update requirements are not the same. Some clusters have high updating frequency, and some clusters have low updating frequency. This results in a waste of information transmission process, which is essentially a waste of communication resources between clusters. However, in the embodiment of the present application, the sending unit is added to each cluster, so that each sending unit can send the query request and the update request to the central cluster according to the situation of the cluster where the sending unit is located, thereby greatly reducing the waste of communication resources between clusters, simultaneously improving the resource utilization rate of the central cluster, and facilitating the central cluster to realize the management of other clusters in the system.
In addition, in the prior art, since the update process depends on the push service of the central cluster, once the update requirement of a certain cluster or some clusters increases, the central server needs to be reconfigured to meet the update requirement of individual clusters. The embodiment of the application can directly modify the corresponding cluster configuration, and once the update requirement of a cluster is improved, the request sending frequency of the sending unit of the cluster can be correspondingly improved so as to meet the update requirement of the cluster without modifying any service of a central cluster.
On the basis of the above embodiment, as a preferred embodiment, when a new cluster is added to the distributed multi-cluster system, the sending unit may also be deployed on the new cluster.
When a new cluster is added to the distributed multi-cluster system, the sending unit is only needed to be deployed on the new cluster, and then the sending unit can be used for sending query information or data updating information to a query interface or a data updating interface of a data center, so that the new cluster is added to the distributed multi-cluster system.
According to the embodiment of the application, the data center is deployed in the center cluster, the sending unit is deployed in other clusters, the center cluster does not need to actively push service to other clusters in the system regularly, each cluster can send the query request and/or the update request to the center cluster through the sending unit according to the update requirement of the cluster, and meanwhile any service of the center cluster does not need to be modified during updating, so that the inter-cluster communication frequency of the distributed multi-cluster system is reduced, frequent data update copying is avoided, communication resources are saved, and the performance of the distributed multi-cluster system is improved. The efficiency of multi-cluster data management can be improved, the performance of the distributed multi-cluster system is improved, and the distributed multi-cluster system is more convenient to manage.
On the basis of the above embodiment, as a preferred embodiment, when the data center is removed, the method may further include:
and re-acquiring the virtual IP of the rest clusters in the distributed multi-cluster system, and taking the cluster with the minimum current virtual IP as a central cluster.
No matter what reason the central cluster in the current distributed multi-cluster system is removed, the virtual IP of the rest clusters in the system summary needs to be obtained again, the cluster with the minimum virtual IP is used as the central cluster again, and the data center is redeployed, so that the central cluster in the distributed multi-cluster system is ensured to exist in most time, the query request and the data updating request sent by the rest clusters in the distributed multi-cluster system can be received, and the query updating requirement of each cluster in the distributed multi-cluster system is met.
In the following, a data management system of a distributed multi-cluster system provided by an embodiment of the present application is introduced, and the data management system described below and the data management method described above may be referred to correspondingly.
Referring to fig. 2, fig. 2 is a schematic structural diagram of a data management system of a distributed multi-cluster system according to an embodiment of the present application, where the data management system of the distributed multi-cluster system includes:
an obtaining module 100, configured to obtain a virtual IP of each cluster in the distributed multi-cluster system;
a first deployment module 200, configured to use a cluster with the smallest virtual IP as a central cluster, and deploy a data center on the central cluster; the data center comprises a query interface and a data updating interface;
a second deployment module 300, configured to deploy the sending unit on all other clusters in the distributed multi-cluster system after determining the central cluster; the sending unit is a unit for sending a query request of data information to the query interface and/or sending an update request of data information to the data update interface.
Based on the above embodiment, as a preferred embodiment, the method may further include:
a third deployment module for deploying the sending unit on the new cluster.
Based on the above embodiment, as a preferred embodiment, the method may further include:
and the central cluster deployment module is used for re-acquiring the virtual IP of the rest clusters in the distributed multi-cluster system and taking the cluster with the minimum current virtual IP as the central cluster.
The present application also provides a computer readable storage medium having stored thereon a computer program which, when executed, may implement the steps provided by the above-described embodiments. The storage medium may include: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.
The present application further provides a distributed multi-cluster system, which may include a memory and a processor, where the memory stores a computer program, and when the processor calls the computer program in the memory, the steps provided in the foregoing embodiments may be implemented. Of course, the distributed multi-cluster system may also include various network interfaces, power supplies, and other components.
The embodiments are described in a progressive manner in the specification, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other. For the system provided by the embodiment, the description is relatively simple because the system corresponds to the method provided by the embodiment, and the relevant points can be referred to the description of the method part.
The principles and embodiments of the present application are explained herein using specific examples, which are provided only to help understand the method and the core idea of the present application. It should be noted that, for those skilled in the art, it is possible to make several improvements and modifications to the present application without departing from the principle of the present application, and such improvements and modifications also fall within the scope of the claims of the present application.
It is further noted that, in the present specification, relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrases "comprising a," "8230," "8230," or "comprising" does not exclude the presence of additional like elements in a process, method, article, or apparatus that comprises the element.
Claims (5)
1. A data management method of a distributed multi-cluster system is characterized by comprising the following steps:
acquiring a virtual IP of each cluster in the distributed multi-cluster system;
taking the cluster with the minimum virtual IP as a central cluster, and deploying a data center on the central cluster; the data center comprises a query interface and a data updating interface;
after the central cluster is determined, deploying sending units on all other clusters in the distributed multi-cluster system; the sending unit is used for sending a query request of data information to a query interface of the data center and/or sending an update request of the data information to a data update interface of the data center;
when the data center is removed, the virtual IP of the rest clusters in the distributed multi-cluster system is obtained again, and the cluster with the minimum current virtual IP is used as a central cluster;
wherein, when the distributed multi-cluster system adds a new cluster, the method further comprises:
deploying the sending unit on the new cluster; the sending unit is used for sending the query information or the data updating information to a query interface or a data updating interface of the data center.
2. The data management method according to claim 1, wherein the sending unit is configured to send a query request of data information to the query interface, and/or send an update request of data information to the data update interface includes:
the sending unit is used for sending a query request of data information to the query interface periodically according to the preset frequency of the cluster where the sending unit is located, and/or sending an update request of the data information to the data update interface.
3. A data management system for a distributed multi-cluster system, comprising:
the acquisition module is used for acquiring the virtual IP of each cluster in the distributed multi-cluster system;
the first deployment module is used for taking the cluster with the minimum virtual IP as a central cluster and deploying a data center on the central cluster; the data center comprises a query interface and a data updating interface;
a second deployment module, configured to deploy sending units on all other clusters in the distributed multi-cluster system after determining the central cluster; the sending unit is a unit used for sending a query request of data information to the query interface and/or sending an update request of the data information to the data update interface;
the central cluster deployment module is used for re-acquiring the virtual IP of the rest clusters in the distributed multi-cluster system and taking the cluster with the minimum current virtual IP as a central cluster;
a third deployment module for deploying the sending unit on the new cluster.
4. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the data management method according to any one of claims 1-2.
5. A distributed multi-cluster system comprising a memory having a computer program stored therein and a processor which when invoked by the computer program in the memory implements the steps of the data management method of any of claims 1-2.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911243194.7A CN111030858B (en) | 2019-12-06 | 2019-12-06 | Data management method, system and related device for distributed multi-cluster system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911243194.7A CN111030858B (en) | 2019-12-06 | 2019-12-06 | Data management method, system and related device for distributed multi-cluster system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111030858A CN111030858A (en) | 2020-04-17 |
| CN111030858B true CN111030858B (en) | 2023-04-07 |
Family
ID=70204590
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911243194.7A Active CN111030858B (en) | 2019-12-06 | 2019-12-06 | Data management method, system and related device for distributed multi-cluster system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111030858B (en) |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103634144B (en) * | 2013-11-15 | 2017-06-13 | 新浪网技术(中国)有限公司 | The configuration file management method of many IDC clusters, system and equipment |
| CN105208054A (en) * | 2014-06-16 | 2015-12-30 | 中兴通讯股份有限公司 | Cloud scheduling management method, management center and management system |
| CN104113597B (en) * | 2014-07-18 | 2016-06-08 | 西安交通大学 | The HDFS data read-write method of a kind of many Data centres |
| CN108769199A (en) * | 2018-05-29 | 2018-11-06 | 郑州云海信息技术有限公司 | A kind of distributed file storage system host node management method and device |
-
2019
- 2019-12-06 CN CN201911243194.7A patent/CN111030858B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN111030858A (en) | 2020-04-17 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110737442A (en) | edge application management method and system | |
| CN113037794B (en) | Method, device and system for computing resource allocation scheduling | |
| WO2021227999A1 (en) | Cloud computing service system and method | |
| CN102821000A (en) | Method for improving usability of PaaS platform | |
| CN109766174B (en) | Resource scheduling method, resource scheduling apparatus, and computer-readable storage medium | |
| CN111522636A (en) | Application container adjusting method, application container adjusting system, computer readable medium and terminal device | |
| CN110727738B (en) | Global routing system based on data fragmentation, electronic equipment and storage medium | |
| CN109428926B (en) | Method and device for scheduling task nodes | |
| CN110417876B (en) | Session method, node server in distributed system and master control equipment | |
| CN108574718B (en) | Cloud host creation method and device | |
| CN113032134A (en) | Method and device for realizing cloud computing resource allocation and cloud management server | |
| CN114598484B (en) | Certificate updating method, device, cluster and storage medium | |
| CN111030858B (en) | Data management method, system and related device for distributed multi-cluster system | |
| CN110798358B (en) | Distributed service identification method and device, computer readable medium and electronic equipment | |
| CN116055499A (en) | Method, equipment and medium for intelligently scheduling cluster tasks based on redis | |
| CN107534678B (en) | Method, device and system for establishing connection between VNFM and VIM | |
| CN111737052A (en) | Distributed object storage system and method | |
| CN110929126A (en) | Distributed crawler scheduling method based on remote procedure call | |
| CN111064643B (en) | Node server and data interaction method and related device thereof | |
| CN114647385A (en) | Volume mapping management method, system and related device for distributed storage | |
| CN113687962A (en) | Request processing method, device, equipment and storage medium | |
| CN109995577B (en) | Method, device and system for updating strategy | |
| CN110650033B (en) | Distributed application configuration management method and distributed computing system | |
| CN108279982B (en) | Method, system and equipment for managing pbs resources and hadoop resources | |
| CN112463305A (en) | Management method, system and related device of cloud virtualization GPU |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |