CN117651051A - Dynamic and static data synchronization method for distributed high-availability architecture of embedded cloud platform - Google Patents
Dynamic and static data synchronization method for distributed high-availability architecture of embedded cloud platform Download PDFInfo
- Publication number
- CN117651051A CN117651051A CN202311595179.5A CN202311595179A CN117651051A CN 117651051 A CN117651051 A CN 117651051A CN 202311595179 A CN202311595179 A CN 202311595179A CN 117651051 A CN117651051 A CN 117651051A
- Authority
- CN
- China
- Prior art keywords
- cloud platform
- embedded
- static data
- dynamic
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000003068 static effect Effects 0.000 title claims abstract description 56
- 238000000034 method Methods 0.000 title claims abstract description 36
- 230000001360 synchronised effect Effects 0.000 claims abstract description 15
- 230000002159 abnormal effect Effects 0.000 claims description 4
- 238000011084 recovery Methods 0.000 claims description 3
- 230000001419 dependent effect Effects 0.000 abstract description 3
- 230000006870 function Effects 0.000 description 9
- 238000010586 diagram Methods 0.000 description 3
- 238000012544 monitoring process Methods 0.000 description 3
- 238000004590 computer program Methods 0.000 description 2
- 238000010276 construction Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008878 coupling Effects 0.000 description 1
- 238000010168 coupling process Methods 0.000 description 1
- 238000005859 coupling reaction Methods 0.000 description 1
- 230000003247 decreasing effect Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
Abstract
The invention discloses a dynamic and static data synchronization method of an embedded cloud platform distributed high-availability architecture, which belongs to the field of distributed cloud platforms for managing embedded nodes, and comprises the following steps: s1, building an embedded cloud platform distributed high-availability architecture, wherein the architecture at least comprises two or more cloud platform main and standby servers; s2, in the operation of the embedded cloud platform distributed high-availability architecture system, the method comprises the following steps: when real-time dynamic data are synchronized, establishing long connection between the embedded board card and the cloud platform server, generating a virtual agent corresponding to the board card chip by the cloud platform, and broadcasting chip information to all distributed cloud platform servers in real time through the virtual agent; when static data of a cloud platform landing disc is needed to be synchronized, the cloud platform main node synchronously broadcasts data to all standby node servers through the rsync component. The invention can realize real-time synchronization of dynamic and static data in a pure software mode under the condition of not increasing the complexity of the system and using a simpler and less-dependent component library.
Description
Technical Field
The invention relates to the field of distributed cloud platforms for managing embedded nodes, in particular to a dynamic and static data synchronization method for an embedded cloud platform distributed high-availability architecture.
Background
The distributed high-availability architecture consists of 2 or more servers, and needs to synchronize data in real time in the running process of the system, and at present, the following modes are generally adopted:
(1) Data synchronization, copy, data consistency and other functions are realized by means of a distributed storage system (ETCD, redis), a message middleware (Kafka) or the like;
(2) Constructing a distributed database on a server, and guaranteeing time synchronization by means of the existing data function of the distributed database;
(3) Constructing a shared disk array shared by distributed services, and storing drop data by using a shared disk;
(4) And introducing an application library related to third-party data synchronization into the platform service, and developing and realizing a self-defined data synchronization function.
Because the embedded cloud platform has limited OPS (operation per second) service and limited synchronous data volume, static data is only algorithm files, application files and configuration files required by the embedded node, dynamic data is only the characteristics of the embedded node such as real-time board card state and task state, the operation and maintenance cost of software required by the mode (1) and the mode (2) is higher, the complexity of the cloud platform system is greatly increased, and certain CPU and memory resources are occupied; the method (3) needs to build a shared disk, changes the hardware architecture of the system, increases the cost, cannot fully utilize the storage resources of the server disk, and cannot synchronize the data in the memory to only share the dropped data.
Disclosure of Invention
The invention aims to overcome the defects of the prior art and provides a dynamic and static data synchronization method of an embedded cloud platform distributed high-availability architecture, which can realize dynamic and static data synchronization applied to the embedded cloud platform distributed high-availability architecture under the condition of using a simpler and less-dependent component library in a pure software mode without increasing the complexity of the system.
The invention aims at realizing the following scheme:
a dynamic and static data synchronization method of an embedded cloud platform distributed high-availability architecture comprises the following steps:
step S1, building an embedded cloud platform distributed high-availability architecture, wherein the architecture at least comprises two or more cloud platform main and standby servers;
step S2, in the operation of the embedded cloud platform distributed high-availability architecture system, the method comprises the following steps: when real-time dynamic data are synchronized, establishing long connection between the embedded board card and the cloud platform server, generating a virtual agent corresponding to the board card chip by the cloud platform, and broadcasting chip information to all distributed cloud platform servers in real time through the virtual agent; and when static data of the cloud platform landing is needed to be synchronized, the cloud platform main node synchronously broadcasts the data to all the standby node servers.
Further, in step S1, the embedded cloud platform distributed high availability architecture can ensure that the system can handle abnormal faults and meet high availability requirements.
Further, in step S1, the embedded cloud platform distributed high availability architecture has a distributed cloud platform service and manages a plurality of embedded board card chips.
Further, in step S2, after the embedded board card establishes a long connection with the cloud platform server, the method includes the following sub steps: reporting dynamic data of the embedded board card chip in real time, wherein the dynamic data comprises state data and task data; and synchronously synchronizing static data uploaded on the cloud platform, wherein the static data comprises algorithms, applications and configuration files.
Further, in step S2, the long connection is a TCP long connection.
Further, in step S2, the establishing a long connection between the embedded board card and the cloud platform server, the cloud platform generating a virtual agent corresponding to the board card chip, specifically includes the following sub steps: the embedded board reports dynamic data, a long connection is established by the active cloud platform after the board is on line, and a virtual agent is established for each chip in the board by the cloud platform after the long connection is established.
Further, in step S2, the broadcasting the chip information to all distributed cloud platform servers in real time through the virtual proxy specifically includes the following sub-steps: the virtual agent of the chip receives dynamic data reported by the board card chip in real time and broadcasts the dynamic data to the cloud platform server in a grpc mode.
Further, in step S2, when the static data of the cloud platform landing is needed to be synchronized, the cloud platform master node synchronously broadcasts the data to all the standby node servers, which specifically includes the following sub-steps: and the cloud platform disc-falling stores static data in units of files, and the files are synchronously broadcast to all standby cloud platform servers by adopting a general component rsync of a linux system.
Further, the cloud platform disk-falling static data real-time synchronization comprises four conditions of adding, deleting, changing and checking.
Further, when the cloud platform is dropped to store static data, and when a certain cloud platform server is unavailable due to fault, the method comprises the following substeps: and comparing and pushing static data after the recovery through the server.
The beneficial effects of the invention include:
the invention is very suitable for the distributed service architecture for managing the embedded node, and realizes the real-time synchronization of dynamic and static data by a pure software mode under the condition of not increasing the complexity of the system and using a simpler and less-dependent component library; meanwhile, the data synchronization mode is at the cloud platform server without adding data synchronization service logic to the embedded node; and the high available requirement that the quantity of the distributed cloud platform servers is elastically telescopic can be met.
Drawings
In order to more clearly illustrate the embodiments of the invention or the technical solutions of the prior art, the drawings which are used in the description of the embodiments or the prior art will be briefly described, it being obvious that the drawings in the description below are only some embodiments of the invention, and that other drawings can be obtained according to these drawings without inventive faculty for a person skilled in the art.
FIG. 1 is a schematic diagram illustrating a data synchronization system architecture according to an embodiment of the present invention;
FIG. 2 is a diagram illustrating a synchronous dynamic data flow according to an embodiment of the present invention;
fig. 3 is a schematic diagram of a synchronous static data flow according to an embodiment of the invention.
Detailed Description
All of the features disclosed in all of the embodiments of this specification, or all of the steps in any method or process disclosed implicitly, except for the mutually exclusive features and/or steps, may be combined and/or expanded and substituted in any way.
In view of the problems in the background, the inventor of the present invention, aiming at the characteristics of the distributed high availability architecture of the embedded cloud platform, aiming at mode 4, conceived a dynamic and static data synchronization method applied to the distributed high availability architecture of the embedded cloud platform, applied to the distributed cloud platform for managing the embedded nodes, and solved the following technical problems:
1) Under the condition of not increasing the complexity of the system and not changing the hardware architecture of the server (namely by a pure software mode), the data synchronization of dynamic and static data is realized.
2) By adopting the simple and convenient method, the coupling between the cloud platform service and the data synchronization three-party library is reduced, and the dependence is reduced.
3) After the data synchronization function is configured, the distributed service can be dynamically increased or decreased, and the high-availability requirements of flexible quantity and failure are met.
In a specific implementation manner, according to the construction of a data synchronization system architecture as shown in fig. 1, the system architecture needs to meet the requirements of a distributed high-availability architecture, namely, the construction of a system environment meets the following requirements:
(1) The distributed high-availability architecture consists of 2 or more servers;
(2) The server operating system uses a linux system;
(3) Completing the configuration of the rsync component, configuring the account password of the server, and setting the daemon encryption-free data synchronization function of the rsync component;
(4) The cloud platform service completes the function development of receiving the embedded board card TCP connection and producing the board card chip virtual agent;
(5) The cloud platform service completes the grpc remote function calling function and realizes the development of the broadcast dynamic data function;
(6) The cloud platform service completes the interface development of receiving the added, deleted, modified and checked static data files.
As shown in fig. 2, the synchronous dynamic data flow of the present invention specifically includes the following steps:
(1) The embedded board actively establishes TCP long connection with the cloud platform server;
(2) After receiving the connection of the embedded board card, the cloud platform generates a virtual agent corresponding to the on-line board card chip;
(3) The chip virtual agent is responsible for sending heartbeats to the chip of the corresponding board card in real time, monitoring the state of the chip according to information returned by the heartbeats, and monitoring the running state of tasks in the chip;
(4) If the monitoring chip is in a normal state, the chip agent runs normally and periodically sends a heartbeat instruction; if the chip virtual agent monitors that the chip state is abnormal, the chip agent broadcasts the chip state information and the task information to all distributed cloud platform services through the grpc.
As shown in fig. 3, the static landing data synchronization process of the present invention specifically includes the following steps:
(1) Interfaces such as static data adding, deleting, modifying and checking of the cloud platform service are called, and static data such as algorithms, applications and configuration files are changed;
(2) The cloud platform service locally stores the change of the corresponding static data file into the corresponding file directory;
(3) And the cloud platform service simply and rapidly synchronously broadcasts the changed static data through the rsync component, and finally, the static data synchronization is completed by the corresponding file catalogues of all the standby node servers.
It should be noted that, within the scope of protection defined in the claims of the present invention, the following embodiments may be combined and/or expanded, and replaced in any manner that is logical from the above specific embodiments, such as the disclosed technical principles, the disclosed technical features or the implicitly disclosed technical features, etc.
Example 1
A dynamic and static data synchronization method of an embedded cloud platform distributed high-availability architecture comprises the following steps:
step S1, building an embedded cloud platform distributed high-availability architecture, wherein the architecture at least comprises two or more cloud platform main and standby servers;
step S2, in the operation of the embedded cloud platform distributed high-availability architecture system, the method comprises the following steps: when real-time dynamic data are synchronized, establishing long connection between the embedded board card and the cloud platform server, generating a virtual agent corresponding to the board card chip by the cloud platform, and broadcasting chip information to all distributed cloud platform servers in real time through the virtual agent; and when static data of the cloud platform landing is needed to be synchronized, the cloud platform main node synchronously broadcasts the data to all the standby node servers.
Example 2
Based on embodiment 1, in step S1, the embedded cloud platform distributed high availability architecture can ensure that the system can handle abnormal faults and meet high availability requirements.
Example 3
Based on embodiment 1, in step S1, the embedded cloud platform distributed high availability architecture has a distributed cloud platform service and manages a plurality of embedded board card chips.
Example 4
On the basis of embodiment 1, in step S2, after the embedded board card establishes a long connection with the cloud platform server, the method includes the following sub steps: reporting dynamic data of the embedded board card chip in real time, wherein the dynamic data comprises state data and task data; and synchronously synchronizing static data uploaded on the cloud platform, wherein the static data comprises algorithms, applications and configuration files.
Example 5
On the basis of embodiment 1, in step S2, the long connection is a TCP long connection.
Example 6
Based on embodiment 1, in step S2, the long connection is established between the embedded board card and the cloud platform server, and the cloud platform generates a virtual agent corresponding to the board card chip, which specifically includes the following sub-steps: the embedded board reports dynamic data, a long connection is established by the active cloud platform after the board is on line, and a virtual agent is established for each chip in the board by the cloud platform after the long connection is established.
Example 7
Based on embodiment 1, in step S2, the broadcasting, by the virtual agent, the chip information to all distributed cloud platform servers in real time specifically includes the following sub-steps: the virtual agent of the chip receives dynamic data reported by the board card chip in real time and broadcasts the dynamic data to the cloud platform server in a grpc mode.
Example 8
Based on embodiment 1, in step S2, when static data of a cloud platform landing is needed to be synchronized, a cloud platform master node synchronously broadcasts data to all standby node servers, which specifically includes the following sub-steps: and the cloud platform disc-falling stores static data in units of files, and the files are synchronously broadcast to all standby cloud platform servers by adopting a general component rsync of a linux system.
Example 9
Based on embodiment 8, the cloud platform disk-falling static data real-time synchronization includes adding, deleting, changing and checking.
Example 10
On the basis of embodiment 8, when the cloud platform is dropped to store static data and a certain cloud platform server is unavailable due to a fault, the method comprises the following substeps: and comparing and pushing static data after the recovery through the server.
The units involved in the embodiments of the present invention may be implemented by software, or may be implemented by hardware, and the described units may also be provided in a processor. Wherein the names of the units do not constitute a limitation of the units themselves in some cases.
According to an aspect of embodiments of the present invention, there is provided a computer program product or computer program comprising computer instructions stored in a computer readable storage medium. The computer instructions are read from the computer-readable storage medium by a processor of a computer device, and executed by the processor, cause the computer device to perform the methods provided in the various alternative implementations described above.
As another aspect, the embodiment of the present invention also provides a computer-readable medium that may be contained in the electronic device described in the above embodiment; or may exist alone without being incorporated into the electronic device. The computer-readable medium carries one or more programs which, when executed by the electronic device, cause the electronic device to implement the methods described in the above embodiments.
Claims (10)
1. The dynamic and static data synchronization method of the distributed high-availability architecture of the embedded cloud platform is characterized by comprising the following steps of:
step S1, building an embedded cloud platform distributed high-availability architecture, wherein the architecture at least comprises two or more cloud platform main and standby servers;
step S2, in the operation of the embedded cloud platform distributed high-availability architecture system, the method comprises the following steps: when real-time dynamic data are synchronized, establishing long connection between the embedded board card and the cloud platform server, generating a virtual agent corresponding to the board card chip by the cloud platform, and broadcasting chip information to all distributed cloud platform servers in real time through the virtual agent; and when static data of the cloud platform landing is needed to be synchronized, the cloud platform main node synchronously broadcasts the data to all the standby node servers.
2. The method for synchronizing dynamic and static data of an embedded cloud platform distributed high availability architecture according to claim 1, wherein in step S1, the embedded cloud platform distributed high availability architecture is capable of guaranteeing that a system can handle abnormal faults and meeting high availability requirements.
3. The method for synchronizing dynamic and static data of an embedded cloud platform distributed high availability architecture according to claim 1, wherein in step S1, the embedded cloud platform distributed high availability architecture has a distributed cloud platform service and manages a plurality of embedded board card chips.
4. The method for synchronizing dynamic and static data of an embedded cloud platform distributed high availability architecture according to claim 1, wherein in step S2, after the embedded board card establishes a long connection with the cloud platform server, the method comprises the following sub steps: reporting dynamic data of the embedded board card chip in real time, wherein the dynamic data comprises state data and task data; and synchronously synchronizing static data uploaded on the cloud platform, wherein the static data comprises algorithms, applications and configuration files.
5. The method for synchronizing dynamic and static data of an embedded cloud platform distributed high availability architecture according to claim 1, wherein in step S2, the long connection is a TCP long connection.
6. The method for synchronizing dynamic and static data of the distributed high availability architecture of the embedded cloud platform according to claim 1, wherein in step S2, the long connection is established between the embedded board card and the cloud platform server, and the cloud platform generates a virtual agent corresponding to the board card chip, specifically comprising the following sub-steps: the embedded board reports dynamic data, a long connection is established by the active cloud platform after the board is on line, and a virtual agent is established for each chip in the board by the cloud platform after the long connection is established.
7. The method for synchronizing dynamic and static data of an embedded cloud platform distributed high availability architecture according to claim 1, wherein in step S2, the broadcasting of chip information to all distributed cloud platform servers in real time through a virtual agent specifically comprises the following sub-steps: the virtual agent of the chip receives dynamic data reported by the board card chip in real time and broadcasts the dynamic data to the cloud platform server in a grpc mode.
8. The method for synchronizing dynamic and static data of an embedded cloud platform distributed high availability architecture according to claim 1, wherein in step S2, when static data of a cloud platform landing is needed to be synchronized, a cloud platform master node synchronously broadcasts data to all standby node servers, specifically comprising the following sub-steps: and the cloud platform disc-falling stores static data in units of files, and the files are synchronously broadcast to all standby cloud platform servers by adopting a general component rsync of a linux system.
9. The method for synchronizing dynamic and static data of the distributed high-availability architecture of the embedded cloud platform according to claim 8, wherein the real-time synchronization of static data stored in the cloud platform comprises adding, deleting, modifying and searching.
10. The method for synchronizing dynamic and static data of an embedded cloud platform distributed high availability architecture according to claim 8, wherein when static data is saved by a cloud platform landing, when a certain cloud platform server is unavailable due to a fault, the method comprises the following sub-steps: and comparing and pushing static data after the recovery through the server.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311595179.5A CN117651051A (en) | 2023-11-24 | 2023-11-24 | Dynamic and static data synchronization method for distributed high-availability architecture of embedded cloud platform |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311595179.5A CN117651051A (en) | 2023-11-24 | 2023-11-24 | Dynamic and static data synchronization method for distributed high-availability architecture of embedded cloud platform |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117651051A true CN117651051A (en) | 2024-03-05 |
Family
ID=90045893
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311595179.5A Pending CN117651051A (en) | 2023-11-24 | 2023-11-24 | Dynamic and static data synchronization method for distributed high-availability architecture of embedded cloud platform |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117651051A (en) |
-
2023
- 2023-11-24 CN CN202311595179.5A patent/CN117651051A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11360854B2 (en) | Storage cluster configuration change method, storage cluster, and computer system | |
| US5822531A (en) | Method and system for dynamically reconfiguring a cluster of computer systems | |
| EP1654645B1 (en) | Fast application notification in a clustered computing system | |
| US9037899B2 (en) | Automated node fencing integrated within a quorum service of a cluster infrastructure | |
| US11397647B2 (en) | Hot backup system, hot backup method, and computer device | |
| US7146532B2 (en) | Persistent session and data in transparently distributed objects | |
| US20040098717A1 (en) | System and method for creating complex distributed applications | |
| US20170315886A1 (en) | Locality based quorum eligibility | |
| CN111327467A (en) | Server system, disaster recovery backup method thereof and related equipment | |
| CN106452836B (en) | main node setting method and device | |
| CN102088490A (en) | Data storage method, device and system | |
| CN115576655B (en) | Container data protection system, method, device, equipment and readable storage medium | |
| CN105069152A (en) | Data processing method and apparatus | |
| CN104158833A (en) | Method for constructing intelligent desktop system | |
| CN109733444B (en) | Database system and train monitoring management equipment | |
| CN112631756A (en) | Distributed regulation and control method and device applied to space flight measurement and control software | |
| CN111510480A (en) | Request sending method and device and first server | |
| CN114615268B (en) | Service network, monitoring node, container node and equipment based on Kubernetes cluster | |
| CN116389233A (en) | Container cloud management platform active-standby switching system, method and device and computer equipment | |
| CN117651051A (en) | Dynamic and static data synchronization method for distributed high-availability architecture of embedded cloud platform | |
| CN103780433B (en) | Self-healing type virtual resource configuration management data architecture | |
| CN115658390A (en) | Container disaster tolerance method, system, device, equipment and computer readable storage medium | |
| CN111752892A (en) | Distributed file system, method for implementing the same, management system, device, and medium | |
| CN114676291B (en) | Database system and control method thereof | |
| CN104158835A (en) | Method for intelligent desktop system server to control clients |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |