CN108881379A - The synchronous method and apparatus of data between a kind of server cluster - Google Patents
The synchronous method and apparatus of data between a kind of server cluster Download PDFInfo
- Publication number
- CN108881379A CN108881379A CN201810414297.4A CN201810414297A CN108881379A CN 108881379 A CN108881379 A CN 108881379A CN 201810414297 A CN201810414297 A CN 201810414297A CN 108881379 A CN108881379 A CN 108881379A
- Authority
- CN
- China
- Prior art keywords
- server cluster
- data
- cluster
- backup
- server
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1095—Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1097—Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
Abstract
The embodiment of the present invention provides data between a kind of server cluster synchronous method and apparatus, is related to cluster-based storage technical field, this method includes：Monitor the health degree of the data transmission state and each server cluster itself when each server cluster obtains primary sources from the first data-source device, then the first backup server cluster is determined according to the data transmission state of each server cluster and health degree, the primary sources that finally controlling the first backup server cluster will acquire are synchronized to other server clusters except the first backup server cluster.Since the data transmission state and health degree according to each server cluster determine backup server cluster, therefore the data that backup server cluster is obtained from data-source device are complete data, to ensure that the integrality for synchronizing rear each server cluster data.Secondly, other server clusters are synchronized to by the data for obtaining backup server cluster, to realize the consistency of data between server cluster.
Description
Technical field
The present embodiments relate to a kind of synchronous sides of data between cluster-based storage technical field more particularly to server cluster
Method and device.
Background technique
As China enters big data era, the demand to data management and storage is also continuously increased.In order to meet to big
The data of scale carry out storage and management, occur building the side that server cluster is managed data and stores in recent years
Case.In the prior art, lead in order to prevent because problem in computer rooms, backbone network problem etc. lead to the not available problem of server cluster
The standing data progress storage and management set multiple server clusters and obtain same data source.However, in user to server set
During group's request data, if switching to another server cluster from a server cluster, it may appear that the data of request
Inconsistent or incoherent situation, to influence user experience.
Summary of the invention
The embodiment of the present invention provides data between a kind of server cluster synchronous method and apparatus, for solving the prior art
The inconsistent problem of data between middle server cluster.
On the one hand, the method synchronous the embodiment of the invention provides data between a kind of server cluster, including：
Monitor data transmission state when each server cluster obtains primary sources from the first data-source device and
Then the health degree of each server cluster itself determines that data transmission state is normal from each server cluster
And health degree meets the server cluster of preset condition, and health degree normal from the data transmission state meets again later
The first backup server cluster is determined in the server cluster of preset condition, finally controls the first backup server cluster
The primary sources that will acquire are synchronized to other in addition to the first backup server cluster in each server cluster
Server cluster.Data transmission state and each server cluster when obtaining primary sources by monitoring each server cluster
Health degree determine the first backup server cluster, therefore data that the server cluster for backup is obtained from data-source device
For complete data, to ensure that the integrality for synchronizing rear each server cluster data.Secondly, by by the first backup services
The data that device cluster obtains are synchronized to other server clusters, to realize the consistency of data between server cluster.
Optionally, described from the server cluster that the data transmission state is normal and health degree meets preset condition
Determine the first backup server cluster, including：By data transmission state is normal and health degree meets the service of preset condition
The server cluster of highest priority is determined as the first backup server cluster in device cluster.Due to the higher server of priority
The corresponding data transmission link of cluster is more stable, and health degree is also preferable, therefore and health degree normal in data transmission state is full
The server cluster of highest priority is chosen in the server cluster of sufficient preset condition as the first backup server cluster, is improved
The integrality of data in the first backup server cluster.
Optionally, the secondary sources saved in each server cluster are obtained, the secondary sources are from second
It is obtained in data-source device.When the secondary sources saved in determining each server cluster are inconsistent, according to described
The healthy journey of data transmission state and each server cluster itself when each server cluster obtains the secondary sources
It spends and determines the second backup server cluster, then control the second backup server cluster the secondary sources of preservation are synchronous
To other server clusters in each server cluster in addition to the second backup server cluster.By by server
Cluster save data be compared, therefore in server cluster data occur it is inconsistent when, can be in time by the second backup services
The data of device cluster are synchronized to other server clusters, to ensure that the consistency of data between server cluster.Secondly, root
The health degree of data transmission state and each server cluster itself when obtaining secondary sources according to server cluster determines the
Two backup server clusters, to ensure that the complete of the data that the second backup server cluster is obtained from data-source device
Property.
Optionally, for any one server cluster in each server cluster, the server cluster is received
The first request of data that user terminal is sent, first request of data are used for request primary sources, and then judgement is
The no primary sources saved from the first backup server cluster synchronization；If so, will be from the first backup server collection
The primary sources of group synchronization are sent to the user terminal；Otherwise the primary sources that will be obtained from the first data-source device
It is sent to the user terminal.Since server cluster is after receiving the request of data of user terminal, first judge whether to save
Synchronous data if so, then sending synchrodata to user terminal ensure that user terminal is obtained from each server cluster
The consistency and continuity of the data taken, otherwise send the data obtained from data-source device, ensure that before data synchronize
Server cluster also can provide data to user terminal.
Optionally, further include：The server cluster receives the second request of data that the user terminal is sent, and described the
Two request of data are used for request secondary sources, and the secondary sources of preservation are sent to the user terminal, the clothes
The secondary sources that business device cluster saves be obtained from second data-source device or from second backup server
It is obtained in cluster.For secondary sources, by the way that the data synchronized directly to be covered to server cluster from data-source device
The data of acquisition, therefore in user terminal requests data, the data that server cluster directly saves this cluster are sent to user
Terminal does not need to judge, responds the rate that user terminal data is requested on the one hand improve, on the other hand saved server
The memory space of cluster.
On the one hand, the device synchronous the embodiment of the invention provides data between a kind of server cluster, including monitoring module,
First processing module and synchronization module, wherein monitoring module, for monitoring each server cluster from the first data-source device
The health degree of data transmission state and each server cluster itself when acquisition primary sources；First processing mould
Block, for determining that data transmission state is normal from each server cluster and health degree meets the service of preset condition
Device cluster；From the data transmission state, normal and health degree, which meets, determines that first is standby in the server cluster of preset condition
Part server cluster；Synchronization module, the primary sources that will acquire for controlling the first backup server cluster are synchronized to
Other server clusters in each server cluster in addition to the first backup server cluster.
Optionally, the first processing module is specifically used for：By data transmission state is normal and health degree meet it is default
The server cluster of highest priority is determined as the first backup server cluster in the server cluster of condition.
Optionally, server cluster data synchronization unit further includes enquiry module, and the enquiry module is described for obtaining
The secondary sources saved in each server cluster, the secondary sources are obtained from the second data-source device；It is described
When the secondary sources that first processing module is also used to save in determining each server cluster are inconsistent, according to described each
The health degree of data transmission state and each server cluster itself when server cluster obtains the secondary sources
Determine the second backup server cluster；The synchronization module is also used to control the second backup server cluster for the of preservation
Two class data are synchronized to other server clusters in each server cluster in addition to the second backup server cluster.
Optionally, further include：Receiving module, Second processing module, wherein receiving module, for receiving user terminal hair
The first request of data sent, first request of data are used for request primary sources；Second processing module, for judging
Whether primary sources from first backup server cluster synchronization are saved；If so, will be from first backup server
The primary sources of cluster synchronization are sent to the user terminal；Otherwise the he first-class numbert that will be obtained from the first data-source device
According to being sent to the user terminal.
Optionally, the receiving module is also used to receive the second request of data that the user terminal is sent, and described second
Request of data is used for request secondary sources；The Second processing module is also used to for the secondary sources of preservation being sent to
The user terminal, the secondary sources that the server cluster saves be obtained from second data-source device or from
It is obtained in the second backup server cluster.
On the other hand, the embodiment of the invention provides a kind of terminal devices, including at least one processing unit, display unit
And at least one storage unit, wherein the storage unit is stored with computer program, when described program is single by the processing
When member executes, so that the step of processing unit executes any of the above-described the method.
Another aspect, the embodiment of the invention provides a kind of computer readable storage medium, being stored with can be set by terminal
The standby computer program executed, when described program is run on the terminal device, so that terminal device execution is any of the above-described
The step of item the method.
Technical solution provided in an embodiment of the present invention, data when obtaining data by monitoring each server cluster transmit shape
The health degree of state and each server cluster itself, by data transmission state is normal and health degree meets the service of preset condition
The server cluster of highest priority is determined as backup server cluster in device cluster, therefore compared to other server clusters, it is standby
The data that part server cluster is obtained from data-source device are more complete data, and the data of backup server cluster are synchronous
To other server clusters, the integrality for synchronizing rear each server cluster data ensure that.Secondly, by by backup server collection
The data that group obtains are synchronized to other server clusters, realize the consistency of data between server cluster, thus user to
Server cluster request data also can guarantee the consistency of requested data and link up even if the case where encountering cluster switching
Property, to promote user experience.
Detailed description of the invention
To describe the technical solutions in the embodiments of the present invention more clearly, make required in being described below to embodiment
Attached drawing is briefly introduced, it should be apparent that, drawings in the following description are only some embodiments of the invention, for this
For the those of ordinary skill in field, without any creative labor, it can also be obtained according to these attached drawings
His attached drawing.
Fig. 1 is a kind of system architecture diagram provided in an embodiment of the present invention；
Fig. 2 is the flow diagram for the method that data are synchronous between a kind of server cluster provided in an embodiment of the present invention；
Fig. 3 is the flow diagram for the method that data are synchronous between a kind of server cluster provided in an embodiment of the present invention；
Fig. 4 is the flow diagram for the method that data are synchronous between a kind of server cluster provided in an embodiment of the present invention；
Fig. 5 is the flow diagram for the method that data are synchronous between a kind of server cluster provided in an embodiment of the present invention；
Fig. 6 is a kind of structural schematic diagram of server cluster provided in an embodiment of the present invention；
Fig. 7 is a kind of structural schematic diagram of server cluster provided in an embodiment of the present invention；
Fig. 8 is the structural schematic diagram for the device that data are synchronous between a kind of server cluster provided in an embodiment of the present invention；
Fig. 9 is a kind of structural schematic diagram of terminal device provided in an embodiment of the present invention.
Specific embodiment
In order to which the purpose of the present invention, technical solution and beneficial effect is more clearly understood, below in conjunction with attached drawing and implementation
Example, the present invention will be described in further detail.It should be appreciated that specific embodiment described herein is only used to explain this hair
It is bright, it is not intended to limit the present invention.
Herein, it is to be understood that any number of elements in attached drawing be used to example rather than limit and it is any
Name is only used for distinguishing, without any restrictions meaning.In order to facilitate understanding, below to involved in the embodiment of the present invention
Noun explains.
Server cluster：Multiple server centereds are got up carries out same service together, is like only in client
There is a server.Cluster can use multiple computers and carry out parallel computations to obtain very high calculating speed, can also be with
It is backuped with multiple computers, so that any one server is broken whole system and still can operate normally.
Fig. 1 illustrates a kind of applicable system architecture schematic diagram of the embodiment of the present invention, as shown in Figure 1, of the invention
The applicable system architecture of embodiment includes user terminal 110, data-source device 120, at least two server clusters 130, control
Center 140.Control centre 140 can be separate server, can be integrated in server cluster 130 or be integrated in existing CDN
In other equipment in network.
The electronic equipment of the specific network communications capability of user terminal 110, the electronic equipment be smart phone, tablet computer or
Portable personal computer etc..Pass through wireless network between user terminal 110 and server cluster 130 and control centre 140
It is connected.
Data-source device 120 is that server cluster 130 provides initial data, and data format can be text, picture, view
Frequently, webpage, http information etc..Pass through wireless network between data-source device 120 and server cluster 130 and control centre 140
Network is connected.
Server cluster 130 obtains data and is saved from data-source device 120, while receiving user terminal 110
Request of data after, response data request be user terminal 110 data are provided.
Control centre 140 is monitored when each server cluster 130 obtains data from data-source device 120 by monitoring device
After the health degree of data transmission state and each server cluster 130 itself, and health degree normal from data transmission state
Meet in the server cluster 130 of preset condition and determine backup server cluster, controlling backup server cluster again later will
The primary sources of acquisition are synchronized to other server clusters in each server cluster 130 in addition to backup server cluster
130, therefore ensure that the consistency of data between each server cluster.When user terminal 110 is to 130 request data of server cluster
When, request of data first is sent to control centre 140, then by control centre 140 according to the loading condition of each server cluster 130
A server cluster 130, the number of request from the server cluster 130 of distribution of user terminal 110 are distributed for user terminal 110
According to.When the server cluster 130 for providing data for user terminal 110 breaks down, control centre 140 is controlled user terminal
110 switch to another server cluster 130, continue as user terminal 110 by another server cluster 130 and provide data.
Control centre 140 is the server cluster or cloud computing center of a server or several servers composition.
Optionally, above-mentioned wireless network or cable network use standard communication techniques and/or agreement.Network be usually because
Special net, it may also be any network, including but not limited to local area network (Local Area Network, LAN), Metropolitan Area Network (MAN)
(Metropolitan Area Network, MAN), wide area network (Wide Area Network, WAN), mobile, wired or nothing
Any combination of gauze network, dedicated network or Virtual Private Network).
Based on system architecture diagram described in Fig. 1, synchronous the embodiment of the invention provides data between a kind of server cluster
The process of method, this method can be executed by the synchronous device of data between server cluster, as shown in Fig. 2, including the following steps：
Step S201 monitors data transmission when each server cluster obtains primary sources from the first data-source device
The health degree of state and each server cluster itself.
In a kind of possible embodiment, primary sources are significant data, and significant data is according to specific business need
Determined, the first data-source device is to provide the equipment of significant data, is needed, and primary sources source device can be only
Significant data is generated, significant data and insignificant data can also be generated simultaneously.
In alternatively possible embodiment, primary sources real time data, i.e. user can be in real time from server clusters
The data of middle request, for example during live streaming, onlooker needs to obtain the video of main broadcaster in real time, then main broadcaster sends out as data source
Sending the data to server cluster is primary sources.First data-source device is to provide the equipment of real time data, the first number
It can be sent to each server cluster after generating primary sources according to source device, needed, primary sources source device can
Only to generate real time data, real time data and non-real-time data can also be generated simultaneously, in this regard, the embodiment of the present invention is not done specifically
It limits.Each server cluster is saved after receiving the primary sources that the first data-source device is sent.Monitoring device is real-time
The health degree of the data transmission state and each server cluster itself when each server cluster obtains primary sources is monitored,
Data transmission state includes but is not limited to the catastrophic failures such as network transmission state, transmission assembly state, machine delay machine, influences to service
The factor of the health degree of device cluster itself includes but is not limited to：Hard disk, memory, central processing unit (Central
Processing Unit, abbreviation CPU), mainboard, I/O plate and power supply.
Step S202 determines that data transmission state is normal and health degree meets preset condition from each server cluster
Server cluster.
Step S203, and health degree normal from data transmission state, which meets in the server cluster of preset condition, to be determined
First backup server cluster.
In a kind of possible embodiment, when the data transmission state of only one server cluster in each service cluster
Normal and health degree meets preset condition, then the server cluster is determined as the first backup server cluster.
In another in possible embodiment, when the data of at least two server clusters in each service cluster transmit shape
When state is normal and health degree meets preset condition, then the first backup services cluster is determined according to the priority of server cluster,
For example, by data transmission state is normal and health degree meets the server of highest priority in the server cluster of preset condition
Cluster is determined as the first backup server cluster.Since the corresponding data transmission link of the higher server cluster of priority is more steady
Fixed, health degree is also preferable, therefore selects in the server cluster that data transmission state is normal and health degree meets preset condition
It takes the server cluster of highest priority as the first backup server cluster, improves the number in the first backup server cluster
According to integrality.
Step S204, the primary sources that the first backup server cluster of control will acquire are synchronized in each server cluster
Other server clusters in addition to the first backup server cluster.
The primary sources that first backup server cluster will acquire are distributed to other server clusters, other server sets
Group is stored in local after the primary sources for receiving the distribution of the first backup server cluster, therefore between each server cluster
Data are consistent, and when user switches in another server cluster from a server cluster, also can guarantee that user requests
Data consistency and continuity, to promote user experience.In addition, other server clusters save the first backup services
The primary sources of device cluster distribution, while the primary sources obtained from the first data-source device are also saved, even if therefore
Primary sources distribution is not terminated in the first backup server cluster, other server clusters receive the number of user terminal
When according to request, first the primary sources obtained from the first data-source device can also be sent to the user terminal, be convenient for user
Primary sources are obtained in real time.Secondly, if failure when the first backup server cluster has not distributed primary sources,
Backup server cluster, and the will obtained from the first data-source device can also be determined from other server clusters again
A kind of data are sent to other server clusters, improve the stability of data backup.
Based on system architecture diagram described in Fig. 1, it is synchronous that the embodiment of the invention also provides data between a kind of server cluster
Method, the process of this method can be executed by the synchronous device of data between server cluster, as shown in figure 3, including following step
Suddenly：
Step S301, obtains the secondary sources saved in each server cluster, and secondary sources are from the second data source
It is obtained in equipment.
In a kind of possible embodiment, secondary sources are insignificant data, and insignificant data are according to specific business
Demand is determined that the second data-source device is to provide the equipment of insignificant data, is needed, secondary sources source device
Insignificant data can be only generated, significant data and insignificant data can also be generated simultaneously.In addition, the first data-source device and
Second data-source device can be same data-source device, be also possible to different data-source devices.
In alternatively possible embodiment, secondary sources are non-real-time data, i.e. server cluster regularly updates
Data, and user knows that such data regularly updates in advance, for example secondary sources are the data updated daily, the second class
After the secondary sources on the same day are sent to server cluster by data-source device, if user needs to inquire the he second-class number on the same day
According to could be inquired after needing one day.Second data-source device is to provide the equipment of non-real-time data, and the second data-source device generates
Each server cluster can be sent to after non-real-time data.It needs, secondary sources source device can only generate non-reality
When data, real time data and non-real-time data can also be generated simultaneously, in this regard, the embodiment of the present invention is not specifically limited.Monitoring
Data transmission state and each server cluster itself when each server cluster of equipment real-time monitoring obtains secondary sources
Health degree simultaneously saves data transmission state and health degree, and each server cluster is receiving the second data-source device hair
It is directly saved after the secondary sources sent, do not need real-time judge data transmission state and carries out data and is synchronous.Later again
The secondary sources that each server cluster was obtained and saved within the same period are inquired, it is manual that the mode of inquiry can be user
Inquiry, is also possible to periodically automatic inquiry, and the second data of each server cluster for then obtaining inquiry carry out user circle
Face (User Interface, abbreviation UI) shows and compares, and judges what each server cluster was obtained and saved within the same period
Whether data are consistent.
Step S302, when the secondary sources saved in determining each server cluster are inconsistent, according to each server set
The health degree of data transmission state and each server cluster itself when group obtains secondary sources determines the second backup services
Device cluster.
Since each server cluster is when obtaining secondary sources, each server cluster of monitoring device real-time monitoring is obtained
The health degree of data transmission state and each server cluster itself when secondary sources, therefore each service is determined after the comparison
When the secondary sources that device cluster was obtained and saved within the same period are inconsistent, available monitoring device is supervised within the period
The health of data transmission state and each server cluster itself when each server cluster acquisition secondary sources measured
Degree, then and health degree normal from data transmission state, which meets, determines the second backup clothes in the server cluster of preset condition
Business device cluster.From data transmission state, normal and health degree, which meets, determines the second backup in each server cluster of preset condition
The method of server cluster meets in each server cluster of preset condition really with the normal and health degree from data transmission state
The method of fixed first backup server cluster is identical, and details are not described herein again.
The secondary sources of preservation are synchronized in each server cluster by step S303, the second backup server cluster of control
Other server clusters in addition to the second backup server cluster.
The secondary sources of preservation are distributed to other server clusters by the second backup server cluster, other server sets
Group will receive secondary sources preservation, while the secondary sources obtained from secondary sources source device being deleted.Pass through
By server cluster save data be compared, therefore in server cluster data occur it is inconsistent when, can be in time by second
The data of backup server cluster are synchronized to other server clusters, to ensure that the consistent of data between server cluster
Property.It should be noted that being equally applicable to side provided in an embodiment of the present invention for the primary sources in each server cluster
Case, i.e., periodically inquire and whether compare data consistent, if inconsistent, it is synchronous to carry out data, but will not will be from when synchronous
The primary sources obtained in first data-source device are covered.
Based on system architecture diagram described in Fig. 1, it is synchronous that the embodiment of the invention also provides data between a kind of server cluster
Method, the process of this method can execute by any one server cluster in each server cluster, as shown in figure 4, packet
Include following steps：
Step S401, server cluster receive the first request of data that user terminal is sent, and the first request of data is for asking
Seek acquisition primary sources.
Server cluster saves the primary sources obtained from primary sources source device and the first backup clothes simultaneously
The primary sources of business device cluster distribution, the two are stored in different distributed data bases.It should be noted that if server
When cluster is the first backup server cluster, it is only necessary to save the primary sources obtained from primary sources source device.
Step S402, server cluster judge whether to save the he first-class numbert from the first backup server cluster synchronization
According to.
Step S403, if so, by being sent to the user terminal from the primary sources of the first backup server cluster synchronization.
Otherwise the primary sources obtained from the first data-source device are sent to the user terminal by step S404.
The first request of data that server cluster real-time reception user terminal is sent, at this time, in fact it could happen that the first backup clothes
Primary sources are not backed up to other service server clusters completely by business device cluster, therefore server cluster is receiving first
When request of data, first judge whether to save the primary sources from the first backup server cluster synchronization, i.e., the first backup clothes
Whether business device terminates primary sources distribution, if so, by sending out from the primary sources of the first backup server cluster synchronization
It send to user terminal, is otherwise sent to the user terminal the primary sources obtained from the first data-source device, to both protect
The consistency and continuity for having demonstrate,proved the data that user terminal is obtained from each server cluster in turn ensure that real-time response user is whole
The request of data at end.
Based on system architecture diagram described in Fig. 1, it is synchronous that the embodiment of the invention also provides data between a kind of server cluster
Method, the process of this method can execute by any one server cluster in each server cluster, as shown in figure 5, packet
Include following steps：
Step S501, server cluster receive the second request of data that user terminal is sent, and the second request of data is for asking
Seek acquisition secondary sources.
Server cluster saves the secondary sources obtained from secondary sources source device or the second backup server collection
The secondary sources of group's distribution.
The secondary sources of preservation are sent to the user terminal by step S502, server cluster.
The secondary sources that server cluster saves be obtained from the second data-source device or from the second backup services
It is obtained in device cluster.For secondary sources, due to user terminal non real-time requests data, therefore synchronous data are directly covered
The data that lid server cluster is obtained from data-source device, in user terminal requests data, server cluster will directly be protected
The data deposited are sent to the user terminal, and do not need to judge, on the one hand improve the rate of response user terminal data request, separately
On the one hand the memory space of server cluster has been saved.
Embodiment in order to preferably explain the present invention describes the embodiment of the present invention below by specific implement scene and provides
A kind of server cluster between the synchronous method of data for primary sources, set server cluster 1, clothes as described in Figure 6
Business device cluster 2 and server cluster 3 obtain primary sources from the first data-source device, and server cluster 1 includes receiving
Module, processing module, cache module, distribution module, memory module and enquiry module, server cluster 2 and server cluster 3
Structure and the course of work are identical as server cluster 1.Receive that the first data-source device is sent by receiving module first the
A kind of data, data format can have diversity, can be file, http information etc..Then primary sources are accessed
Primary sources, can be sent to different destinations, for example send kafka for primary sources and handle, be by processing
Data reparation and verification, receiving module can simultaneously deposit primary sources a to distributed file system.Mould is received later
Primary sources are sent to processing module by block, and processing module extracts primary sources before data loading
(extract), convert (transform), load (load) or polymerization etc. operation, the processing engine used have JSTORM,
SPARKSTREAMING etc., processing logic is according to business demand or customized.Cache module carries out the result of processing module
Caching prevents from being put in storage phenomena such as accumulating for storage, can be using storage systems such as CODIS or Aerospike.Server cluster 1
If distribution module receive control centre transmission distribution instruction, primary sources are fished for from cache module, are distributed to
Server cluster 2 and server cluster 3.If the distribution module of server cluster 1 is not received by the distribution of control centre's transmission
Instruction then receives the primary sources that server cluster 2 or server cluster 3 are distributed and preservation.Fig. 6 illustrates service
The process of the distribution module distribution primary sources of device cluster 1.For consistency and reliability, control centre is in a time
Section can only distribute to the first backup server collection pocket transmission to be instructed, and then controlling the first backup server cluster will be from he first-class numbert
Other server clusters are synchronized to according to the primary sources that source device obtains.Optionally, the distribution module of server cluster 1 can also
To obtain data transmission state when each server cluster obtains primary sources and each from the monitoring device of control centre
The health degree of server cluster itself, then according to the data transmission state of each server cluster, health degree and preferential
Grade judges whether itself is the first backup server, if so, fishing for primary sources from cache module, is distributed to server
Otherwise cluster 2 and server cluster 3 receive the primary sources that server cluster 2 or server cluster 3 are distributed and preservation.It deposits
Module is stored up, for store the of the primary sources obtained from primary sources source device and the distribution of the first backup server cluster
A kind of data, memory module are generally distributed data base, can be phoenix, hbase, mongodb etc., specific choosing
It selects and is determined according to business scenario.Enquiry module, for being used when external inquiry data.
As described in Figure 7, for secondary sources, set server cluster 1, server cluster 2 and server cluster 3 from
Secondary sources are obtained in second data-source device and are saved the data in memory module.Control centre passes through server cluster
Enquiry module periodically obtain secondary sources from the memory module of each server cluster and UI displaying and compare, judge to take
Whether the secondary sources saved in business device cluster 1, server cluster 2 and server cluster 3 are consistent, if so, directly tying
Beam, otherwise according to each server cluster obtain secondary sources when data transmission state and each server cluster itself health
Degree determines the second backup server cluster, sends synchronic command to the second backup server cluster so that the second backup server
The secondary sources of preservation are synchronized to other server clusters by cluster, and the second backup server cluster is server cluster 1, clothes
A server cluster being engaged in device cluster 2 and server cluster 3.
Technical solution provided in an embodiment of the present invention, data when obtaining data by monitoring each server cluster transmit shape
The health degree of state and each server cluster itself, by data transmission state is normal and health degree meets the clothes of preset condition
The server cluster of highest priority is determined as backup server cluster in business device cluster, therefore compared to other server clusters,
The data that backup server cluster is obtained from data-source device are more complete data, and the data of backup server cluster are same
Step ensure that the integrality for synchronizing rear each server cluster data to other server clusters.Secondly, by by backup server
The data that cluster obtains are synchronized to other server clusters, realize the consistency of data between server cluster, therefore user exists
To server cluster request data, even if the case where encountering cluster switching, also can guarantee consistency and the company of requested data
Coherence, to promote user experience.
Based on the same technical idea, the dress synchronous the embodiment of the invention also provides data between a kind of server cluster
It sets, as shown in figure 8, the device 800 includes monitoring module 801, first processing module 802 and synchronization module 803, wherein monitoring
Module 801 obtains data transmission state when primary sources for monitoring each server cluster from the first data-source device
And the health degree of each server cluster itself；First processing module 802, for true from each server cluster
It makes that data transmission state is normal and health degree meets the server cluster of preset condition；It is normal from the data transmission state
And health degree meets and determines the first backup server cluster in the server cluster of preset condition；Synchronization module 803, is used for
The primary sources that controlling the first backup server cluster will acquire are synchronized in each server cluster except described the
Other server clusters except one backup server cluster.
Optionally, the first processing module 802 is specifically used for：By data transmission state is normal and health degree meet it is pre-
If the server cluster of highest priority is determined as the first backup server cluster in the server cluster of condition.
Optionally, server cluster data synchronization unit further includes enquiry module 804, and the enquiry module 804 is for obtaining
The secondary sources saved in each server cluster are taken, the secondary sources are obtained from the second data-source device
's；When the secondary sources that the first processing module 802 is also used to save in determining each server cluster are inconsistent,
The data transmission state and each server cluster itself when the secondary sources are obtained according to each server cluster
Health degree determine the second backup server cluster；The synchronization module 803 is also used to control second backup server
The secondary sources of preservation are synchronized in each server cluster in addition to the second backup server cluster by cluster
Other server clusters.
It optionally, further include receiving module 805, the first request of data sent for receiving user terminal, described first
Request of data is used for request primary sources；Second processing module 806 is saved for judging whether from the first backup clothes
The primary sources for device cluster synchronization of being engaged in；If so, by being sent out from the primary sources of the first backup server cluster synchronization
It send to the user terminal；Otherwise the primary sources obtained from the first data-source device are sent to the user terminal.
Optionally, the receiving module 805 is also used to receive the second request of data that the user terminal is sent, and described the
Two request of data are used for request secondary sources；The secondary sources that the Second processing module 806 is also used to save
It is sent to the user terminal, the secondary sources that the server cluster saves are obtained from second data-source device
Or obtain from the second backup server cluster.
The embodiment of the invention provides a kind of terminal devices, including at least one processing unit and at least one storage list
Member, wherein the storage unit is stored with computer program, when described program is executed by the processing unit, so that described
The step of method that data synchronize between processing unit execute server cluster.As shown in figure 9, for described in the embodiment of the present invention
The hardware structural diagram of terminal device, the terminal device are specifically as follows desktop computer, portable computer, intelligent hand
Machine, tablet computer etc..Specifically, which may include memory 901, processor 902 and stores on a memory
Computer program, the processor 902 realize data between any server cluster in above-described embodiment when executing described program
The step of synchronous method.Wherein, memory 901 may include read-only memory (ROM) and random access memory (RAM),
And the program instruction stored in memory 901 and data are provided to processor 902.
Further, terminal device described in the embodiment of the present application can also include input unit 903 and output dress
Set 904 etc..Input unit 903 may include keyboard, mouse, touch screen etc.；Output device 904 may include display equipment, such as
Liquid crystal display (Liquid Crystal Display, LCD), cathode-ray tube (Cathode Ray Tube, CRT) touch
Screen etc..Memory 901, processor 902, input unit 903 and output device 904 can be connected by bus or other modes
It connects, in Fig. 9 for being connected by bus.The program instruction that processor 902 calls memory 901 to store and the journey according to acquisition
The synchronous method of data between sequence instruction execution server cluster provided by the above embodiment.
The embodiment of the invention also provides a kind of computer readable storage medium, being stored with can be by terminal device execution
Computer program, when described program is run on the terminal device, so that data between the terminal device execute server cluster
The step of synchronous method.
It should be understood by those skilled in the art that, the embodiment of the present invention can provide as method or computer program product.
Therefore, complete hardware embodiment, complete software embodiment or embodiment combining software and hardware aspects can be used in the present invention
Form.It is deposited moreover, the present invention can be used to can be used in the computer that one or more wherein includes computer usable program code
The shape for the computer program product implemented on storage media (including but not limited to magnetic disk storage, CD-ROM, optical memory etc.)
Formula.
The present invention be referring to according to the method for the embodiment of the present invention, the process of equipment (system) and computer program product
Figure and/or block diagram describe.It should be understood that every one stream in flowchart and/or the block diagram can be realized by computer program instructions
The combination of process and/or box in journey and/or box and flowchart and/or the block diagram.It can provide these computer programs
Instruct the processor of general purpose computer, special purpose computer, Embedded Processor or other programmable data processing devices to produce
A raw machine, so that being generated by the instruction that computer or the processor of other programmable data processing devices execute for real
The device for the function of being specified in present one or more flows of the flowchart and/or one or more blocks of the block diagram.
These computer program instructions, which may also be stored in, is able to guide computer or other programmable data processing devices with spy
Determine in the computer-readable memory that mode works, so that it includes referring to that instruction stored in the computer readable memory, which generates,
Enable the manufacture of device, the command device realize in one box of one or more flows of the flowchart and/or block diagram or
The function of being specified in multiple boxes.
These computer program instructions also can be loaded onto a computer or other programmable data processing device, so that counting
Series of operation steps are executed on calculation machine or other programmable devices to generate computer implemented processing, thus in computer or
The instruction executed on other programmable devices is provided for realizing in one or more flows of the flowchart and/or block diagram one
The step of function of being specified in a box or multiple boxes.
Although preferred embodiments of the present invention have been described, it is created once a person skilled in the art knows basic
Property concept, then additional changes and modifications may be made to these embodiments.So it includes excellent that the following claims are intended to be interpreted as
It selects embodiment and falls into all change and modification of the scope of the invention.
Obviously, various changes and modifications can be made to the invention without departing from essence of the invention by those skilled in the art
Mind and range.In this way, if these modifications and changes of the present invention belongs to the range of the claims in the present invention and its equivalent technologies
Within, then the present invention is also intended to include these modifications and variations.
Claims (10)
1. a kind of synchronous method of data between server cluster, which is characterized in that including：
Monitor data transmission state when each server cluster obtains primary sources from the first data-source device and described
The health degree of each server cluster itself；
It determines that data transmission state is normal from each server cluster and health degree meets the server of preset condition
Cluster；
From the data transmission state, normal and health degree, which meets, determines the first backup in the server cluster of preset condition
Server cluster；
The primary sources that controlling the first backup server cluster will acquire are synchronized in each server cluster except institute
State other server clusters except the first backup server cluster.
2. the method as described in claim 1, which is characterized in that and health degree normal from the data transmission state meets pre-
If the first backup server cluster is determined in the server cluster of condition, including：
By data transmission state is normal and health degree meets the server of highest priority in the server cluster of preset condition
Cluster is determined as the first backup server cluster.
3. the method as described in claim 1, which is characterized in that further include：
The secondary sources saved in each server cluster are obtained, the secondary sources are from the second data-source device
It obtains；
When the secondary sources saved in determining each server cluster are inconsistent, obtained according to each server cluster
The health degree of data transmission state and each server cluster itself when the secondary sources determines that the second backup takes
Business device cluster；
It controls the second backup server cluster the secondary sources of preservation are synchronized in each server cluster except institute
State other server clusters except the second backup server cluster.
4. the method as described in claim 1, which is characterized in that further include：
For any one server cluster in each server cluster, the server cluster receives user terminal and sends
The first request of data, first request of data be used for request primary sources；
The server cluster judges whether to save the primary sources from the first backup server cluster synchronization；
If so, by the user terminal is sent to from the primary sources of the first backup server cluster synchronization；
Otherwise the primary sources obtained from first data-source device are sent to the user terminal.
5. method as claimed in claim 4, which is characterized in that further include：
The server cluster receives the second request of data that the user terminal is sent, and second request of data is for requesting
Obtain secondary sources；
The secondary sources of preservation are sent to the user terminal by the server cluster, the server cluster save the
Two class data are from obtaining in second data-source device or obtain from the second backup server cluster.
6. the synchronous device of data between a kind of server cluster, which is characterized in that including：
Monitoring module obtains data transmission when primary sources for monitoring each server cluster from the first data-source device
The health degree of state and each server cluster itself；
First processing module, for determining that data transmission state is normal and health degree meets from each server cluster
The server cluster of preset condition；From the data transmission state, normal and health degree meets the server cluster of preset condition
In determine the first backup server cluster；
Synchronization module, the primary sources that will acquire for controlling the first backup server cluster are synchronized to each service
Other server clusters in device cluster in addition to the first backup server cluster.
7. device as claimed in claim 6, which is characterized in that the first processing module is specifically used for：
By data transmission state is normal and health degree meets the server of highest priority in the server cluster of preset condition
Cluster is determined as the first backup server cluster.
8. device as claimed in claim 6, which is characterized in that further include enquiry module；
The enquiry module for obtaining the secondary sources saved in each server cluster, the secondary sources be from
It is obtained in second data-source device；
When the secondary sources that the first processing module is also used to save in determining each server cluster are inconsistent, root
Data transmission state and each server cluster itself when obtaining the secondary sources according to each server cluster
Health degree determines the second backup server cluster；
The synchronization module be also used to control the second backup server cluster the secondary sources of preservation are synchronized to it is described
Other server clusters in each server cluster in addition to the second backup server cluster.
9. device as claimed in claim 6, which is characterized in that further include：
Receiving module, for receiving the first request of data of user terminal transmission, first request of data is used for request
Primary sources；
Second processing module saves the primary sources from the first backup server cluster synchronization for judging whether；
If so, by the user terminal is sent to from the primary sources of the first backup server cluster synchronization；
Otherwise the primary sources obtained from the first data-source device are sent to the user terminal.
10. device as claimed in claim 9, which is characterized in that the receiving module is also used to receive the user terminal hair
The second request of data sent, second request of data are used for request secondary sources；
The Second processing module is also used to for the secondary sources of preservation to be sent to the user terminal, the server cluster
The secondary sources of preservation are from obtaining in second data-source device or obtain from the second backup server cluster
It takes.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810414297.4A CN108881379B (en) | 2018-05-03 | 2018-05-03 | Method and device for data synchronization between server clusters |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810414297.4A CN108881379B (en) | 2018-05-03 | 2018-05-03 | Method and device for data synchronization between server clusters |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108881379A true CN108881379A (en) | 2018-11-23 |
| CN108881379B CN108881379B (en) | 2021-04-16 |
Family
ID=64327362
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810414297.4A Expired - Fee Related CN108881379B (en) | 2018-05-03 | 2018-05-03 | Method and device for data synchronization between server clusters |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108881379B (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109614271A (en) * | 2018-12-14 | 2019-04-12 | 深圳前海微众银行股份有限公司 | Control method, device, equipment and the storage medium of multiple company-data consistency |
| CN109660560A (en) * | 2019-01-23 | 2019-04-19 | 视联动力信息技术股份有限公司 | A kind of method of data synchronization and system |
| CN111371900A (en) * | 2020-03-13 | 2020-07-03 | 北京奇艺世纪科技有限公司 | Method and system for monitoring health state of synchronous link |
| CN111901389A (en) * | 2020-07-03 | 2020-11-06 | 北京达佳互联信息技术有限公司 | Data updating method, device, server and storage medium |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102025630A (en) * | 2010-12-14 | 2011-04-20 | 成都市华为赛门铁克科技有限公司 | Load balancing method and load balancing system |
| US20120180070A1 (en) * | 2011-01-11 | 2012-07-12 | Ibm Corporation | Single point, scalable data synchronization for management of a virtual input/output server cluster |
| CN102682117A (en) * | 2012-05-15 | 2012-09-19 | 浪潮电子信息产业股份有限公司 | Method for quickly copying cluster data in database |
| CN103391299A (en) * | 2012-05-08 | 2013-11-13 | 深圳市腾讯计算机系统有限公司 | Load balance method and load balance system |
| CN104462483A (en) * | 2014-12-18 | 2015-03-25 | 北京奇虎科技有限公司 | Data synchronizing method, device and system |
| CN104536971A (en) * | 2014-12-02 | 2015-04-22 | 北京锐安科技有限公司 | High-availability database |
| CN105915646A (en) * | 2016-06-20 | 2016-08-31 | 飞天诚信科技股份有限公司 | Method and device for dispersing load of servers |
| CN106888245A (en) * | 2016-06-07 | 2017-06-23 | 阿里巴巴集团控股有限公司 | A kind of data processing method, apparatus and system |
-
2018
- 2018-05-03 CN CN201810414297.4A patent/CN108881379B/en not_active Expired - Fee Related
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102025630A (en) * | 2010-12-14 | 2011-04-20 | 成都市华为赛门铁克科技有限公司 | Load balancing method and load balancing system |
| US20120180070A1 (en) * | 2011-01-11 | 2012-07-12 | Ibm Corporation | Single point, scalable data synchronization for management of a virtual input/output server cluster |
| CN103391299A (en) * | 2012-05-08 | 2013-11-13 | 深圳市腾讯计算机系统有限公司 | Load balance method and load balance system |
| CN102682117A (en) * | 2012-05-15 | 2012-09-19 | 浪潮电子信息产业股份有限公司 | Method for quickly copying cluster data in database |
| CN104536971A (en) * | 2014-12-02 | 2015-04-22 | 北京锐安科技有限公司 | High-availability database |
| CN104462483A (en) * | 2014-12-18 | 2015-03-25 | 北京奇虎科技有限公司 | Data synchronizing method, device and system |
| CN106888245A (en) * | 2016-06-07 | 2017-06-23 | 阿里巴巴集团控股有限公司 | A kind of data processing method, apparatus and system |
| CN105915646A (en) * | 2016-06-20 | 2016-08-31 | 飞天诚信科技股份有限公司 | Method and device for dispersing load of servers |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109614271A (en) * | 2018-12-14 | 2019-04-12 | 深圳前海微众银行股份有限公司 | Control method, device, equipment and the storage medium of multiple company-data consistency |
| CN109660560A (en) * | 2019-01-23 | 2019-04-19 | 视联动力信息技术股份有限公司 | A kind of method of data synchronization and system |
| CN111371900A (en) * | 2020-03-13 | 2020-07-03 | 北京奇艺世纪科技有限公司 | Method and system for monitoring health state of synchronous link |
| CN111901389A (en) * | 2020-07-03 | 2020-11-06 | 北京达佳互联信息技术有限公司 | Data updating method, device, server and storage medium |
| CN111901389B (en) * | 2020-07-03 | 2023-07-04 | 北京达佳互联信息技术有限公司 | Data updating method, device, server and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108881379B (en) | 2021-04-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110417842B (en) | Fault processing method and device for gateway server | |
| CN108023953B (en) | High-availability implementation method and device for FTP service | |
| CN108881379A (en) | The synchronous method and apparatus of data between a kind of server cluster | |
| KR102324987B1 (en) | Method and apparatus for resource allocation | |
| US10212228B2 (en) | Implementing synchronization of state information betweeen instances of an application as well as between different applications in an efficient, scalable manner | |
| Deboosere et al. | Cloud-based desktop services for thin clients | |
| US10241876B1 (en) | Cooperative fault tolerance and load balancing | |
| CN109348264B (en) | Video resource sharing method and device, storage medium and electronic equipment | |
| CN113163002B (en) | Server switching method and device and storage medium | |
| CN113018871A (en) | Service processing method, device and storage medium | |
| CN110334072A (en) | A kind of distributed file system, file updating method and device | |
| US11588890B2 (en) | System, method and apparatus having a redundant cluster for processing data | |
| US8606908B2 (en) | Wake-up server | |
| US11750711B1 (en) | Systems and methods for adaptively rate limiting client service requests at a blockchain service provider platform | |
| KR20130089779A (en) | System for proving contents based on cloud computing and method thereof | |
| JP2023520057A (en) | METHOD, APPARATUS, ELECTRONIC DEVICE, SYSTEM AND STORAGE MEDIUM FOR MICROSERVICE COMPOSITION | |
| CN112165523A (en) | Data downloading method and device | |
| CN103179182A (en) | Appratus and method for providing cloud infrastructure sharing | |
| CN107302849B (en) | Light path distribution method and device | |
| CN106570006B (en) | Multimedia file recommendation system and delivery information storage method | |
| CN112653632A (en) | Flow control realization method and device | |
| JP2012093992A (en) | Data center controlling system, data center controlling apparatus and program | |
| EP3355551A1 (en) | Data access method and device | |
| CN109831467B (en) | Data transmission method, equipment and system | |
| EP4142258A1 (en) | Edge computing network, data transmission method and apparatus, device and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20210416 |