CN109788027B

CN109788027B - Data synchronization method, device, server and computer storage medium

Info

Publication number: CN109788027B
Application number: CN201811525666.3A
Authority: CN
Inventors: 刘根生
Original assignee: Ping An Technology Shenzhen Co Ltd
Current assignee: Ping An Technology Shenzhen Co Ltd
Priority date: 2018-12-13
Filing date: 2018-12-13
Publication date: 2022-04-15
Anticipated expiration: 2038-12-13
Also published as: CN109788027A

Abstract

The embodiment of the invention discloses a data synchronization method, a data synchronization device, a server and a storage medium, wherein the method is applied to the field of incremental synchronization, and comprises the following steps: and receiving an update message sent by the data server through an incremental update data interface, wherein the update message comprises the identification, the update data and the update version number of the data server, determining the identification of each background server which has a corresponding relation with the identification of the data server, and pushing the update data and the update version number to each background server based on the identification of each background server so that each background server performs incremental update according to the update data and the update version number. By implementing the embodiment of the invention, the data synchronization between the data server and the background server can be completed.

Description

Data synchronization method, device, server and computer storage medium

Technical Field

The invention relates to the technical field of communication, in particular to a data synchronization method, a data synchronization device, a server and a computer storage medium.

Background

In background system development, data used at high frequency in a data server often needs to be loaded into an application memory to accelerate processing speed, and the existing background application is often distributed background. Therefore, data inconsistency between the data server and the memory of the background application is easily caused.

Therefore, how to complete the data synchronization between the data server and the background application becomes a technical problem to be solved urgently.

Disclosure of Invention

The embodiment of the invention discloses a data synchronization method, a data synchronization device, a server and a computer storage medium, which can complete data synchronization between a data server and a background server.

In a first aspect, an embodiment of the present invention discloses a data synchronization method, which may be applied to a synchronization server, and the method may include: receiving an update message sent by a data server through an incremental update data interface, wherein the update message comprises an identifier of the data server, update data and an update version number, and the update version number is the version number of the update data; determining the identifiers of all background servers which have corresponding relations with the identifiers of the data servers; and pushing the updating data and the updating version number to each background server based on the identification of each background server so that each background server performs incremental updating according to the updating data and the updating version number.

In one implementation, the method may further include: receiving a first data request sent by a background server, wherein the first data request is used for requesting target complete data with the same version number as an updated version number, and the first data request is sent when the background server does not have the complete data in an internal memory of the background server; acquiring target complete data, wherein the target complete data is obtained by the synchronous server through incremental updating of the complete data stored in an external memory of the synchronous server according to the updated data; and sending the target complete data to the background server.

In one implementation, the method may further include: receiving a second data request sent by a background server, wherein the second data request is used for requesting update data with the same version number as that in a version number set, the version number set comprises 1 or more version numbers, and the second data request is sent when the background server has complete data in an internal memory of the background server, and the difference between the update version number and the version number of the complete data in the internal memory is greater than a preset value; acquiring an update data set, wherein the version numbers of the update data in the update data set are the same as the version numbers in the version number set in a one-to-one correspondence manner; and sending the update data set to the background server so that the background server performs incremental update on the complete data after the last incremental update according to the received update data in the sequence of increasing the version number.

In a second aspect, an embodiment of the present invention discloses another data synchronization method, where the method may be applied to a background server, and the method may include: receiving push data sent by a synchronous server, wherein the push data comprises update data and an update version number, the update version number is the version number of the update data, and the connection state between a background server and the synchronous server is long connection; detecting whether complete data exists in an internal memory; if the complete data exists in the internal memory, acquiring the version number of the complete data; if the update version number is different from the version number of the complete data, performing incremental update on the complete data according to the update data; and associating and storing the updated complete data and the updated version number after the increment updating in the internal memory.

In one implementation, the method may further include: if the internal memory does not have complete data, a first data request is sent to the synchronous server, and the first data request is used for requesting target complete data with the same version number as the updated version number; receiving target complete data sent by a synchronous server; and associating and storing the target complete data and the update version number in an internal memory.

In an implementation manner, if the update version number is different from the version number of the complete data, a specific implementation manner of performing incremental update on the complete data according to the update data may be: if the difference value between the update version number and the version number of the complete data is a preset value, performing incremental update on the complete data according to the update data; if the difference between the update version number and the version number of the complete data is larger than a preset value, sending a second data request to the synchronous server, wherein the second data request is used for requesting the update data with the version number same as that in a version number set, and the version number set comprises 1 or more version numbers; receiving an update data set sent by a synchronous server, wherein the version numbers of the update data in the update data set are the same as the version numbers in the version number set in a one-to-one correspondence manner; and according to the ascending sequence of the version numbers, sequentially according to the received updating data, carrying out incremental updating on the complete data after the last incremental updating.

In one implementation, the method may further include: detecting the connection state between the background server and the synchronous server at intervals of preset duration, wherein the connection state comprises long connection and disconnection; if the connection state is disconnected, sending a long connection request to the synchronous server, wherein the long connection request is used for requesting to establish long connection with the synchronous server; and receiving a connection success message sent by the synchronization server, wherein the connection success message is used for indicating that the long connection is successfully established between the background server and the synchronization server.

In a third aspect, an embodiment of the present invention discloses a data synchronization apparatus, which includes a unit configured to perform the method described in the first aspect.

In a fourth aspect, an embodiment of the present invention discloses another data synchronization apparatus, which includes means for performing the method of the second aspect.

In a fifth aspect, an embodiment of the present invention discloses a server, where the server includes a memory and a processor, where the memory is used to store a computer program, and the computer program includes program instructions, and the processor is configured to call the program instructions to execute the method according to the first aspect.

In a sixth aspect, an embodiment of the present invention discloses another server, where the server includes a memory and a processor, where the memory is used to store a computer program, and the computer program includes program instructions, and the processor is configured to call the program instructions to execute the method according to the second aspect.

In a seventh aspect, an embodiment of the present invention discloses a computer storage medium storing a computer program, the computer program comprising program instructions that, when executed by a processor, cause the processor to perform the method of the first aspect.

In an eighth aspect, the present invention discloses another computer storage medium, which stores a computer program, the computer program comprising program instructions, which, when executed by a processor, cause the processor to perform the method of the second aspect.

By implementing the embodiment of the invention, after receiving the updated data and the updated version number sent by the data server, the synchronization server can determine the identifier of each background server which has a corresponding relationship with the identifier of the data server, and pushes the updated data and the updated version number to each background server based on the identifier of each background server, so that each background server performs incremental updating according to the updated data and the updated version number, thereby completing the data synchronization between the data server and each background server.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.

Fig. 1 is a schematic architecture diagram of a communication system according to an embodiment of the present invention;

fig. 2 is a schematic flow chart of a data synchronization method according to an embodiment of the present invention;

FIG. 3 is a flow chart of another data synchronization method provided by the embodiment of the invention;

FIG. 4 is a flow chart illustrating a further data synchronization method according to an embodiment of the present invention;

fig. 5 is a schematic structural diagram of a data synchronization apparatus according to an embodiment of the present invention;

FIG. 6 is a schematic structural diagram of another data synchronization apparatus according to an embodiment of the present invention;

fig. 7 is a schematic structural diagram of a synchronization server according to an embodiment of the present invention;

fig. 8 is a schematic structural diagram of a backend server according to an embodiment of the present invention.

Detailed Description

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

Referring to fig. 1, fig. 1 is a schematic diagram of a communication system according to an embodiment of the present invention. As shown in fig. 1, the communication system includes a data server 101, a synchronization server 102, a first backend server 103, a second backend server 104, and a third backend server 105. The data server 101 may be configured to store data required by the first backend server 103, the second backend server 104, and the third backend server 105. The synchronization server 102 has a publish-subscribe function, and the data server 101 may send the stored data to the synchronization server 102, so that the synchronization server 102 pushes the data to the first backend server 103, the second backend server 104, and the third backend server 105 through the publish function, so as to complete data synchronization between the data server 101 and the first backend server 103, the second backend server 104, and the third backend server 105. The synchronization server 102 may be a redis server or other servers with a publish-subscribe function, which is not limited in this embodiment of the present invention, and the synchronization server 102 is taken as a redis server for explanation in this embodiment of the present invention.

It should be noted that, in fig. 1, the data pushed by the synchronization server 102 to the 3 backend servers is only used for example and does not constitute a limitation to the embodiment of the present invention, and in a specific implementation, the synchronization server 102 may also push the data to 1, 2, 5, or other number of backend servers, which is not limited in the embodiment of the present invention.

It should be understood that the communication system described in the embodiment of the present invention is for more clearly illustrating the technical solution of the embodiment of the present invention, and does not form a limitation on the technical solution provided in the embodiment of the present invention, and as a person having ordinary skill in the art knows that along with the evolution of the system architecture and the appearance of a new service scenario, the technical solution provided in the embodiment of the present invention is also applicable to similar technical problems.

Referring to fig. 2, based on the schematic architecture diagram of the communication system shown in fig. 1, fig. 2 is a schematic flow chart of a data synchronization method according to an embodiment of the present invention. Specifically, as shown in fig. 2, the data synchronization method according to the embodiment of the present invention may include, but is not limited to, the following steps:

s201, a redis server receives an update message sent by a data server through an incremental update data interface, wherein the update message includes an identifier of the data server, update data and an update version number, and the update version number is the version number of the update data.

The identification of the data server is used for uniquely identifying the data server, and the update message may be sent to the redis server when the data server detects that the self-stored complete data is updated, it should be noted that the complete data may be all data or part of data stored in the data server, but the complete data is data required by the background server, and as long as the data required by the background server is updated, the data server may send the update message to the redis server. The update data is used for indicating data updated on the basis of the complete data, and after receiving the update data sent by the data server, the redis server can perform incremental update on the currently stored complete data according to the update data and obtain new complete data. Incremental updating refers to updating only the places needing to be changed, not the places which are not changed, and the updated places are not repeatedly updated when updating is performed, so that the incremental updating consumes less processing resources compared with full updating. Full updates refer to updates to all data. It should be noted that the update data and the complete data are different types of data.

In an implementation manner, the data server may send the complete data stored by itself to the redis server, so that when the update data is subsequently sent to the redis server, the redis server may perform incremental update on the complete data currently stored by the redis server according to the update data. It can be understood that when the data server subsequently sends other update data to the redis server, the data server does not need to send the stored complete data to the redis server again, and the redis server only needs to perform incremental update on the complete data obtained by the last incremental update according to the currently received update data.

In one implementation, the data server may store the update data in association with a version number of the update data (i.e., an update version number). In one implementation, the data server may further associate and store the self-stored complete data and a version number of the self-stored complete data. The version number of the complete data and the version number of the update data may each be an incremental version number, which may be used to indicate the order in which the update of the complete data occurs. For example, when the version number is a version number incremented by 1, the version number of the complete data 1 is v1, and the version number of the complete data 2 is v2, it indicates that the complete data 2 is obtained by performing incremental update on the basis of the complete data 1 according to the update data with the version number v 2. Wherein the update data sent by the data server to the redis server may be used for indicating data in which the target complete data changes on the basis of the first complete data. The first complete data may be the complete data stored by the data server itself before the last data update. For example, at 8:00:00 (hour: minute: second), the complete data stored in the data server is the first complete data, if the first complete data stored in the data server is updated after 3 minutes (i.e., at 8:03: 00), after the update is completed, the complete data stored in the data server is the target complete data, and the data server can obtain the update data to be sent to the redis server by comparing the target complete data with the first complete data. It should be noted that, in any case, only one complete piece of data is stored in the data server. In one implementation, multiple sets of update data may be stored in the data server, and version numbers of the multiple sets of update data are different, and among the version numbers of all update data stored in the data server, the maximum version number is the same as the version number of complete data stored in the data server.

It should be noted that the above-mentioned version number that is incremented by 1 is only used for example, and in other possible implementation manners, the version number may also be a version number that is incremented by 2, 3, or other numbers, which is not limited in this embodiment of the present invention.

In the embodiment of the invention, when the redis server receives the update message sent by the data server, the update message indicates that the data stored in the data server is updated, and the redis server pushes the update data and the update version number in the update message to each background server needing the data stored in the data server, so that the data synchronization between the data server and each background server needing the data stored in the data server can be maintained.

In an implementation manner, the incremental update data interface of the redis server may be used only to receive the update message, and may also be used to receive other data, which is not limited in this embodiment of the present invention.

In one implementation, the data server may include 1 or more databases, where each database may be used to store different application-related data. That is to say, data required by different backend servers may be stored in different databases of the data server, for example, if the data server includes 2 databases (a and B), where the backend server 1, the backend server 2, and the backend server 3 all require data in the database a, and the backend server 4, the backend server 5, and the backend server 6 all require data in the database B, when the data stored in the database a is updated, the data server a may send an update message a to the sync server, where the update message a is used to indicate that the data in the database a is updated, and accordingly, the redis server may send the update data and the update version number in the update message a to the backend server 1, the backend server 2, and the backend server 3. Similarly, when the data stored in the database B is updated, the data server B may send an update message B to the redis server, where the update message B is used to indicate that the data in the database B is updated, and correspondingly, the redis server may send the update data and the update version number in the update message B to the background server 4, the background server 5, and the background server 6.

In an implementation manner, the database in the data server may be Oracle, Mysql, redis, HBase, or another common database, which is not limited in this embodiment of the present invention.

S202, the redis server determines the identification of each background server which has a corresponding relation with the identification of the data server. Specifically, after receiving the update message, the redis server may determine, by querying the correspondence table, the identifiers of all the background servers having a correspondence with the identifier of the data server, and send the update data and the update version number included in the update message to the background server corresponding to the identifier of each background server. The redis server may store a correspondence between the identifier of the data server and the identifiers of the background services in a correspondence table in advance, where the correspondence may be sent to the redis server by the data server. In an implementation manner, the redis server may receive the update messages sent by different data servers, and the identifiers of the backend servers corresponding to the identifiers of the different data servers may be different, for example, the redis server may receive the update message 1 sent by the data server 1 and receive the update message 2 sent by the data server 2, and the backend server identifiers having a corresponding relationship with the identifier of the data server 1 may be the backend server identifier 1 and the backend server identifier 2, and the backend server identifiers having a corresponding relationship with the identifier of the data server 2 may be the backend server identifier 3 and the backend server identifier 4.

In an implementation manner, a specific implementation manner of the redis server determining the identifier of each background server having a corresponding relationship with the identifier of the data server may further be: the redis server determines a target channel in all used channels, and acquires pre-stored identifications of all background servers subscribing the target channel, wherein the target channel is used for data transmission between the data server and each background server. The redis server may use 1 or more channels, each of which may be used to publish data of a different data server, or each of which may also be used to publish different data in the same data server. Therefore, when receiving an update message sent by a data server, a redis server needs to determine a channel for issuing the update message. Specifically, when the redis server creates only one channel for the data server, the redis server may store a corresponding relationship between the server identifier and the channel identifier in advance. When the redis server receives an update message sent by the data server, the update message includes an identifier of the data server, the redis server can obtain the identifier of the data server, and according to a corresponding relationship between the server identifier and the channel identifier, in all channels used by the redis server, a channel indicated by the channel identifier corresponding to the identifier of the data server is determined as a target channel, and the target channel is used for issuing update data and an update version number.

In one implementation, the update message may further include a channel identifier, and when the redis server creates a plurality of channels for the data server, the data server may store a correspondence between the database identifier and the channel identifier in advance, where different database identifiers correspond to different channel identifiers. When data stored in a certain database is updated, the data server can determine the channel identifier corresponding to the database, and further, when the data server sends the updated data and the updated version number to the redis server, the channel identifier corresponding to the database is sent to the redis server together. Therefore, the redis server can determine a target channel from all channels used by the redis server according to the channel identifier corresponding to the database, and send the updated data and the updated version number to each background server subscribing to the target channel.

In one implementation, the redis server may use 1 or more channels, and different backend servers may subscribe to the same or different channels, that is, each channel may be subscribed to by 1 or more backend servers. The identities of all background servers subscribed to each channel may be stored in the redis server in advance, for example, when the redis server uses 3 channels (channel 1, channel 2, and channel 3), the identities of the background servers subscribed to each channel of the redis server may be as shown in table 1.

TABLE 1 identification of background servers subscribing to respective channels of a redis server

Channel identification	Identification of all background servers subscribing to respective channels
		Channel 1	Background server 1, background server 2, background server 3
Channel 2	Background server 4 and background server 5
		Channel 3	Background server 6

S203, the redis server pushes the updating data and the updating version number to each background server based on the identification of each background server so that each background server can carry out incremental updating according to the updating data and the updating version number.

Specifically, after the redis server determines the identifier of each background server having a corresponding relationship with the identifier of the data server, the update data and the update version number may be pushed to the background server corresponding to the identifier of each background server. Correspondingly, after receiving the update data and the update version number, each background server can compare the update version number with the version number of the self-stored complete data, and judge whether to perform incremental update on the self-stored complete data according to the comparison result. Because the version number is an incremental version number, if the comparison result shows that the update version number is different from the version number of the self-stored complete data, it indicates that the data stored in the data server is updated, and the background server needs to perform incremental update on the self-stored complete data according to the update data.

Specifically, when the version number is increased by a preset value, if the comparison result is that the difference between the update version number and the version number of the complete data stored in the background server is the preset value, it indicates that the data stored in the data server is updated, the complete data stored in the background server is different from the complete data currently stored in the data server, and the complete data stored in the background server is the same as the complete data stored before the last update of the data server, the complete data stored in the background server is incrementally updated according to the update data of which the version number is the update version number, and after the incremental update is completed, the complete data stored in the background server can be consistent with the complete data currently stored in the data server. It should be noted that the preset value may be 1, 2, 3, or other values, which is not limited in the embodiment of the present invention.

In the embodiment of the invention, because the incremental version number is adopted, the background server can judge whether the update version number is consistent with the version number of the self-stored complete data before updating the self-stored complete data according to the update data, and only when the update version number is inconsistent with the version number of the self-stored complete data, the background server performs incremental updating on the self-stored complete data. By the method, the situation that when the update version number is consistent with the version number of the complete data stored in the background server, the background server unnecessarily updates the complete data stored in the background server according to the update data, and resource waste is caused can be avoided.

By implementing the embodiment of the invention, after receiving the updated data and the updated version number sent by the data server, the redis server determines the identifier of each background server which has a corresponding relationship with the identifier of the data server, and pushes the updated data and the updated version number to each background server based on the identifier of each background server, so that each background server performs incremental updating according to the updated data and the updated version number, thereby completing the data synchronization between the data server and each background server.

Referring to fig. 3, fig. 3 is a schematic flowchart illustrating another data synchronization method according to an embodiment of the present invention. Specifically, as shown in fig. 3, another data synchronization method according to an embodiment of the present invention may include, but is not limited to, the following steps:

s301, the redis server receives an update message sent by the data server through an incremental update data interface, wherein the update message includes an identifier of the data server, update data and an update version number, and the update version number is the version number of the update data.

S302, the redis server determines the identification of each background server which has a corresponding relation with the identification of the data server.

S303, the redis server pushes the updating data and the updating version number to each background server based on the identification of each background server.

It should be noted that, the execution processes of steps S301 to S303 may refer to the specific descriptions in steps S201 to S203 in fig. 2, which are not described herein again.

Fig. 3 takes the number of identifiers of backend servers having a corresponding relationship with the identifier of the data server as 1 as an example, and therefore fig. 3 only shows that the redis server pushes the update data and the update version number to 1 backend server. It should be noted that, the number of the identifiers of the background servers having the correspondence with the identifiers of the data servers is 1, which is only used for example, in other feasible implementation manners, the number of the identifiers of the background servers having the correspondence with the identifiers of the data servers may also be 2, 3 or other numbers, and one identifier of one background server uniquely identifies one background server, which is not limited in the embodiment of the present invention.

S304, the background server detects whether complete data exists in the internal memory.

When the background server receives the push data sent by the redis server, it needs to be determined whether incremental updating can be performed, that is, the background server needs to detect whether complete data exists in the internal memory, and then can perform incremental updating according to the update data included in the push data on the basis of the complete data. The pushed data comprises updated data and an updated version number, the updated version number is the version number of the updated data, and the background server receives the pushed data sent by the redis server when the connection state between the background server and the redis server is long connection.

In an implementation mode, the background server can also preset a duration as an interval, and detect a connection state between the background server and the redis server, wherein the connection state comprises long connection and disconnection; if the connection state is disconnected, sending a long connection request to the redis server, wherein the long connection request is used for requesting to establish long connection with the redis server; and receiving a connection success message sent by the redis server, wherein the connection success message is used for indicating that the long connection is successfully established between the background server and the redis server. Specifically, the background server detects whether the long connection with the redis server is disconnected or not at intervals of preset duration, and if the long connection is disconnected, the long connection is reestablished by sending a long connection request to the redis server; if not, the long connection is continuously kept. By the method, the communication connection between the background server and the redis server can be ensured to be in a long connection state, and the background server can be ensured to successfully receive the push data sent by the redis server.

The long connection means that a plurality of data packets can be continuously sent on one connection, the short connection means that one connection is established when both communication parties have data interaction, and the connection is disconnected after the data transmission is completed, namely, only one service is sent after each connection. Compared with the short connection established between the background server and the redis server, the long connection established between the background server and the redis server can avoid the situation that the short connection is required to be established again every time the redis server sends push data to the background server, and is beneficial to reducing the power consumption of the background server and the redis server. The preset time period may be 5 minutes, 6 minutes or other values, which is not limited in the embodiment of the present invention.

In an implementation manner, after the background server sends the long connection request to the redis server, if a connection failure message sent by the redis server is received, the background server may continue to send the long connection request to the background server until a connection success message sent by the redis server is received. In one implementation, after the background server sends the long connection request to the redis server, if any response made by the redis server for the long connection request is not received, it indicates that the redis server may not successfully receive the long connection request, and at this time, the background server may continue to send the long connection request to the background server until a connection success message sent by the redis server is received.

S305, if the complete data exists in the internal memory, the background server acquires the version number of the complete data.

Specifically, when the complete data exists in the internal memory of the backend server, the backend server further needs to further determine whether incremental updating needs to be performed, so as to avoid resource waste caused by performing incremental updating without performing incremental updating. Specifically, the background server may determine whether incremental updating is required by comparing the updated version number with the version number of the complete data stored in the internal memory of the background server, and according to the comparison result.

And S306, if the update version number is different from the version number of the complete data, the background server performs incremental update on the complete data according to the update data.

Because the version number is an incremental version number, if the update version number is different from the version number of the complete data stored in the background server, it indicates that the data stored in the data server is updated, and the background server needs to perform incremental update on the complete data stored in the background server according to the update data.

In one implementation, when the version number is incremented by a preset value, there may be two reasons why the update version number and the version number of the complete data stored by the background server are different: firstly, updating data stored in a data server, wherein the difference between the updated version number and the version number of complete data stored in a background server is a preset value; secondly, the data stored in the data server is updated, and the background server does not successfully receive the pushed data pushed before the redis server, at this time, the difference between the updated version number and the version number of the complete data stored in the background server is greater than the preset value.

In one implementation, the backend server may determine a difference between the update version number and the version number of the complete data stored by the backend server, and determine the number of times of execution of the incremental update according to the difference.

In one implementation manner, when the version number is incremented by a preset value, if a difference value between the updated version number and the version number of the complete data stored in the background server is a preset value, it indicates that the data stored in the data server is updated, the complete data stored in the background server is different from the complete data currently stored in the data server, and the complete data stored in the background server is the same as the complete data stored before the last update of the data server, at this time, the background server only needs to perform 1 incremental update on the complete data according to the updated data, so that the complete data obtained after the update is consistent with the complete data currently stored in the data server.

In one implementation, when the version number is incremented by a preset value, if a difference between the updated version number and the version number of the complete data stored in the background server is greater than the preset value, it indicates that the background server does not successfully receive all the updated data sent by the redis server, or indicates that the background server successfully receives all the updated data sent by the redis server, but the update is not successful when the incremental update is performed. At this time, the background server may perform the following steps:

s1, the background server sends a second data request to the redis server, the second data request is used for requesting the updating data with the version number same as that in the version number set, and the version number set comprises 1 or more version numbers. And the background server sends a second data request to the redis server so as to acquire the update data with the version number same as that in the version number set. And then, performing incremental updating for multiple times according to the obtained updating data, so that the complete data obtained after the incremental updating is finished is consistent with the complete data stored in the data server. In one implementation, the set of version numbers may include version numbers between the update version number and the version number of the complete data stored by the backend server. For example, when the update version number is v5 and the version number of the complete data stored by the backend server is v1, the set of version numbers includes all version numbers: v2, v3 and v 4.

s2, the background server receives the update data set sent by the redis server, and the version number of the update data in the update data set is the same as the version number in the version number set in a one-to-one correspondence manner.

Specifically, after receiving the second data request sent by the background server, the redis server may obtain the update data with the version number the same as each version number in the version number set, obtain an update data set, and send the update data set to the background server. The version numbers of the update data in the update data set correspond to the version numbers in the version number set one to one, and for example, if all the version numbers included in the version number set are: v2, v3, and v4, and the update data set includes all update data: second update data having the same version number as v2, third update data having the same version number as v3, and fourth update data having the same version number as v 4.

s3, the background server carries out increment updating to the complete data after last increment updating according to the ascending sequence of the version number and the received updating data in turn. Specifically, the update data set may include update data with a version number that is an update version number, or may not include update data with a version number that is an update version number. When the update data set includes update data with a version number of the update version number, the specific implementation manner of the background server performing incremental update on the complete data after the last incremental update according to the received update data may be that: and the background server performs incremental updating on the complete data after the last incremental updating according to the updating data in the updating data set. When the update data set does not include update data with a version number of the update version number, the specific implementation manner of the background server performing incremental update on the complete data after the last incremental update according to the received update data may be that: and the background server performs incremental updating on the complete data after the last incremental updating according to the updating data in the updating data set and the updating data with the version number as the updating version number. Compared with the update data set comprising the update data with the version number being the update version number, when the update data set does not comprise the update data with the version number being the update version number, the redis server can send the update data set to the background server, the sent data volume is small, and the transmission efficiency is improved.

For example, if the set of version numbers includes all version numbers: v2, v3 and v4, the update data set comprises all update data as follows: the second update data, the third update data and the fourth update data, wherein v2 is the same as the second update data, v3 is the same as the third update data, v4 is the same as the fourth update data, the update version number is v5, and when the version number of the complete data currently stored by the background server is v1, the background server can firstly perform incremental update on the complete data (i.e., the complete data currently stored by the background server) obtained by the last incremental update of the background server according to the second update data to obtain complete data 2; then, according to the third updating data, performing incremental updating on the complete data (namely, the complete data 2) obtained by the last incremental updating of the background server to obtain complete data 3; thirdly, according to the fourth updating data, performing incremental updating on the complete data (namely, the complete data 3) obtained by the last incremental updating of the background server to obtain complete data 4; and finally, according to the update data pushed by the redis server, performing incremental update on the complete data (namely, the complete data 4) obtained by the last incremental update of the background server to obtain complete data 5. And further, the complete data 5 and the v5 are stored in the internal memory in an associated manner, so that when new update data are received later, the background server can compare the v5 with the version number of the new update data, and judge whether to perform incremental update on the complete data 5 according to the new update data according to the comparison result. Where the version number of the complete data 5 is v 5.

In an implementation manner, since the version number of the update data may indicate a time sequence of update of the complete data, the background server performs incremental update on the complete data after the last incremental update according to the sequence of increasing the version number and the received update data in sequence, so that the incremental update may be successfully performed. If incremental updates are performed out of the order of increasing version numbers, update failures may result. For example, if the update data 2 with the version number v2 describes data in which the complete data 2 with the version number v2 changes on the basis of the complete data 1 with the version number v1, and if the update data 3 with the version number v3 describes data in which the complete data 3 with the version number v3 changes on the basis of the complete data 2 with the version number v2, if the complete data currently stored by the backend server is the complete data 1, and the update data received by the backend server are the update data 2 and the update data 3, at this time, if the backend server performs incremental update according to the descending order of the version numbers, that is, the complete data 1 is incrementally updated according to the update data 3, the update may fail.

And S307, the background server stores the updated complete data and the updated version number in the internal memory in an associated manner. The background server stores the updated complete data and the updated version number in the internal memory of the background server in an associated manner, so that when new updated data is received later, the updated version number and the version number of the new updated data can be compared, and whether incremental updating is performed or not is judged according to the comparison result.

By implementing the embodiment of the invention, after receiving the update data and the update version number sent by the data server, the redis server can determine the identifier of the background server which has a corresponding relation with the identifier of the data server, and pushes the update data and the update version number to the background server based on the identifier of the background server, and when the update version number of the background server is inconsistent with the version number of the self-stored complete data, the background server performs incremental update on the self-stored complete data according to the update data. By the method, the situation that when the update version number is consistent with the version number of the complete data stored in the background server, the background server unnecessarily updates the complete data stored in the background server according to the update data, so that resource waste is caused can be avoided.

In one implementation, if a difference between the update version number and the version number of the complete data stored by the background server is greater than a preset value and less than or equal to a preset threshold, the first background server may perform S1, S2, and S3 in step S306. Wherein the preset threshold is greater than or equal to 2 times the preset value. When the difference value between the updated version number and the version number of the complete data stored in the background server is greater than the preset value and less than or equal to the preset threshold value, the number of times that the background server does not successfully receive the pushed data pushed before the redis server is less, compared with the method for requesting the redis server to acquire the target complete data, the target complete data comprises data far greater than the data included in the updated data, and 1 or more pieces of updated data are acquired by sending the second data request to the redis server, so that the data amount transmitted in the network can be reduced, and the transmission efficiency is improved. Also, less power is consumed to perform a fewer number of incremental update operations than to write the target integrity data.

In one implementation, if a difference between the update version number and the version number of the complete data stored in the background server is greater than a preset threshold, the background server may perform the following steps: and sending a first data request to a redis server so as to acquire target complete data, and storing the target complete data and the update version number in an internal memory of the redis server in an associated manner, wherein the first data request is used for requesting the target complete data with the same version number as the update version number.

The difference value between the update version number and the version number of the complete data stored in the background server is greater than the preset threshold value, which indicates that the number of times that the background server fails to receive the pushed data pushed before the redis server is greater, and compared with the case that the second data request is sent to the redis server, in order to obtain the update data set, because the number of times that the background server fails to receive the pushed data pushed before the redis server is greater, the number of update data included in the update data set is greater, so that the number of times that the background server needs to perform the incremental update operation is greater, and it can be understood that the probability of errors in the multiple incremental update mode is higher. However, if the first data request is sent to the redis server, the target complete data is obtained, and only 1 data writing operation is needed, so that the target complete data can be obtained more accurately by adopting the method compared with a method of performing multiple incremental updates.

Referring to fig. 4, fig. 4 is a flowchart illustrating another data synchronization method according to an embodiment of the present invention. Specifically, as shown in fig. 4, another data synchronization method according to an embodiment of the present invention may include, but is not limited to, the following steps:

s401, receiving, by a redis server, an update message sent by a data server through an incremental update data interface, where the update message includes an identifier of the data server, update data, and an update version number, and the update version number is a version number of the update data.

S402, the redis server determines the identification of each background server which has a corresponding relation with the identification of the data server.

S403, the redis server pushes the update data and the update version number to each background server based on the identifier of each background server.

Fig. 4 takes the number of identifiers of backend servers having a corresponding relationship with the identifier of the data server as 1 as an example, and therefore fig. 4 only shows that the redis server pushes the update data and the update version number to 1 backend server.

S404, the background server detects whether complete data exists in the internal memory.

It should be noted that the execution processes of steps S401 to S404 may refer to specific descriptions in steps S201 to S203 in fig. 2 and step S304 in fig. 3, respectively, and are not described herein again.

Specifically, the background server detects whether there is complete data in the internal memory, and if there is complete data, step S405 is executed; if there is no complete data, step S406 is executed.

S405, if the complete data exists in the internal memory, the background server acquires the version number of the complete data; if the update version number is different from the version number of the complete data, the background server performs incremental update on the complete data according to the update data; and the background server stores the complete data after the incremental updating and the updating version number in an internal memory in an associated manner.

It should be noted that, the execution process of step S405 may refer to specific descriptions in steps S305 to S307 in fig. 3, which is not described herein again.

S406, if the complete data does not exist in the internal memory, the background server sends a first data request to the redis server, wherein the first data request is used for requesting target complete data with the version number being the same as the update version number.

Specifically, the incremental update needs to perform incremental update on the complete data according to the update data on the basis of the existence of the complete data. Therefore, when the complete data does not exist in the background server, it indicates that the data in the internal memory of the background server is lost, and the background server cannot perform incremental updating. At this time, the background server may send a first data request to the redis server in order to obtain the target complete data. It should be noted that, because the version number of the target complete data is the same as the update version number, the target complete data acquired by the background server is the same as the complete data currently stored by the data server, that is, after the target complete data is acquired by the background server, the target complete data and the update version number only need to be stored in the internal memory in an associated manner without performing incremental update.

In an implementation manner, the reason for the data loss in the internal memory of the backend server may be that the data in the internal memory is completely lost due to power failure of the backend server, and the backend server may store the complete data in the internal memory, so that the speed of reading the complete data is faster. The integral data and the updating data stored in the redis server are not easy to lose because the redis server stores the integral data and the updating data in the external memory and the data stored in the external memory cannot be lost due to power failure. The background server can obtain the target complete data by sending the first data request to the redis server. In one implementation, the external memory may be a hard disk, a floppy disk, an optical disk, or a usb disk, which is not limited in this embodiment of the present invention.

In an implementation manner, the redis server stores the corresponding relationship between the version number and the complete data, so that when the complete data in the background server is lost, the redis server can send the complete data (i.e., the target complete data) required by the background server to the background server, which is beneficial to ensuring data synchronization between the background server and the data server.

S407, the redis server acquires target complete data, and the target complete data is obtained by the redis server through incremental updating of the complete data stored in the external memory of the redis server according to the updating data.

In one implementation manner, after receiving the complete data sent by the data server, the redis server may store the complete data in its own external memory, and after receiving the update data sent by the data server, the redis server may perform incremental update on the complete data in the external memory according to the update data, and obtain the target complete data after the update is completed. And when the version number is increased by 1, the difference between the updated version number and the version number of the complete data in the external memory is 1. Therefore, the redis server performs 1 incremental update on the complete data in the external memory according to the update data, so that the target complete data obtained after the update is completed is consistent with the complete data stored in the data server.

In an implementation manner, the redis server may pre-store a corresponding relationship between the version number and the complete data, and after the redis server receives a first data request sent by the background server, the redis server may obtain, based on the corresponding relationship between the version number and the complete data, target complete data having the same version number as the updated version number, and further send the target complete data to the background server.

S408, the redis server sends the target complete data to the background server.

And S409, the background server stores the target complete data and the updated version number in an internal memory in an associated mode.

Specifically, after receiving the target complete data sent by the redis server, the background server may associate and store the target complete data and the update version number in its internal memory, so that when new update data is subsequently received, the background server may compare the update version number with the version number of the new update data, and determine whether to perform incremental update according to a comparison result.

By implementing the embodiment of the invention, after receiving the update data and the update version number sent by the data server, the redis server can determine the identifier of the background server which has a corresponding relation with the identifier of the data server, and pushes the update data and the update version number to the background server based on the identifier of the background server, and the background server can carry out incremental update on the self-stored complete data according to the update data when the complete data exists in the internal memory and the update version number is inconsistent with the version number of the self-stored complete data. By the method, the situation that when the update version number is consistent with the version number of the complete data stored in the background server, the background server unnecessarily updates the complete data stored in the background server according to the update data, so that resource waste is caused can be avoided. In addition, the background server may obtain the target complete data by sending a first data request to the redis server when the complete data does not exist in the internal memory. By the method, the problem that the background server cannot perform incremental updating due to data loss in the internal memory of the background server can be avoided, and the reliability of the synchronization system is improved.

Referring to fig. 5, fig. 5 is a schematic structural diagram of a data synchronization apparatus according to an embodiment of the present invention, specifically, as shown in fig. 5, the data synchronization apparatus 50 may include:

a receiving unit 501, configured to receive, through an incremental update data interface, an update message sent by a data server, where the update message includes an identifier of the data server, update data, and an update version number, and the update version number is a version number of the update data.

The processing unit 502 is configured to determine the identifiers of the background servers having a corresponding relationship with the identifier of the data server.

A sending unit 503, configured to push the update data and the update version number to each background server based on the identifier of each background server, so that each background server performs incremental update according to the update data and the update version number.

In an implementation manner, the receiving unit 501 may be further configured to receive a first data request sent by the backend server, where the first data request is used to request target complete data with a version number that is the same as the update version number, and the first data request is sent when the backend server does not have the complete data in its internal memory; the data synchronization apparatus 50 may further include an obtaining unit 504, configured to obtain target complete data, where the target complete data is obtained by the synchronization server performing incremental update on complete data stored in an external memory of the synchronization server according to the update data; the sending unit 503 may also be configured to send the target complete data to the background server.

In an implementation manner, the receiving unit 501 may be further configured to receive a second data request sent by the background server, where the second data request is used to request update data with a version number that is the same as the version number in the version number set, the version number set includes 1 or more version numbers, and the second data request is sent when the background server has complete data in its internal memory, and a difference between the update version number and the version number of the complete data in the internal memory is greater than a preset value; the obtaining unit 504 may be further configured to obtain an update data set, where version numbers of update data in the update data set are the same as version numbers in the version number set in a one-to-one correspondence manner; the sending unit 503 may also be configured to send the update data set to the backend server, so that the backend server performs incremental update on the complete data after the last incremental update sequentially according to the sequence of increasing the version number and the received update data.

The embodiments of the present invention and the embodiments of the methods shown in fig. 2 to 4 are based on the same concept, and the technical effects thereof are also the same, and for the specific principle, reference is made to the description of the embodiments shown in fig. 2 to 4, which is not repeated herein.

Referring to fig. 6, fig. 6 is a schematic structural diagram of another data synchronization apparatus according to an embodiment of the present invention, specifically, as shown in fig. 6, the data synchronization apparatus 60 may include:

a receiving unit 601, configured to receive push data sent by a synchronization server, where the push data includes update data and an update version number, the update version number is a version number of the update data, and a connection state between a background server and the synchronization server is a long connection;

a detecting unit 602, configured to detect whether complete data exists in the internal memory;

an obtaining unit 603, configured to obtain a version number of complete data if the complete data exists in the internal memory;

a processing unit 604, configured to perform incremental update on the complete data according to the update data if the update version number is different from the version number of the complete data;

the processing unit 604 may further be configured to associate and store the updated complete data and the updated version number in the internal memory.

In an implementation manner, the synchronization apparatus 60 may further include a sending unit 605, configured to send a first data request to the synchronization server if there is no complete data in the internal memory, where the first data request includes an update version number, and the first data request is used to request target complete data with the same version number as the update version number; the receiving unit 601 may be further configured to receive target complete data sent by the synchronization server; the processing unit 604 may further be configured to associate and store the target complete data and the update version number in the internal memory.

In an implementation manner, the processing unit 604 is configured to, when performing incremental update on the complete data according to the update data if the update version number is different from the version number of the complete data, specifically, if a difference between the update version number and the version number of the complete data is a preset value, perform incremental update on the complete data according to the update data; if the difference between the update version number and the version number of the complete data is larger than a preset value, sending a second data request to the synchronous server, wherein the second data request is used for requesting the update data with the version number same as that in a version number set, and the version number set comprises 1 or more version numbers; receiving an update data set sent by a synchronous server, wherein the version numbers of the update data in the update data set are the same as the version numbers in the version number set in a one-to-one correspondence manner; and according to the ascending sequence of the version numbers, sequentially according to the received updating data, carrying out incremental updating on the complete data after the last incremental updating.

In an implementation manner, the detecting unit 602 may be further configured to detect a connection state between the background server and the synchronization server at intervals of a preset duration, where the connection state includes long connection and disconnection; the sending unit 605 is further configured to send a long connection request to the synchronization server if the connection status is disconnected, where the long connection request is used to request to establish a long connection with the synchronization server; the receiving unit 601 may further be configured to receive a connection success message sent by the synchronization server, where the connection success message is used to indicate that the long connection is successfully established between the background server and the synchronization server.

The embodiments of the present invention and the embodiments of the methods shown in fig. 3 to 4 are based on the same concept, and the technical effects thereof are also the same, and for the specific principle, reference is made to the description of the embodiments shown in fig. 3 to 4, which is not repeated herein.

Referring to fig. 7, fig. 7 is a schematic structural diagram of a synchronization server according to an embodiment of the present invention. The server 70 may comprise a receiver 701, a transmitter 702, a memory 703 and a processor 704, the receiver 701, the transmitter 702, the memory 703 and the processor 704 being connected by one or more communication buses.

Memory 703 may include both read-only memory and random-access memory, and provides instructions and data to processor 704. A portion of the memory 703 may also include non-volatile random access memory.

The Processor 704 may be a Central Processing Unit (CPU), and the Processor 704 may be other general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), an off-the-shelf Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic, discrete hardware components, etc. A general-purpose processor may be a microprocessor, but in the alternative, the processor 704 may be any conventional processor or the like. Wherein:

a memory 703 for storing program instructions.

A processor 704 for invoking program instructions stored in memory 703 for:

and receiving an update message sent by the data server through the incremental update data interface, wherein the update message comprises the identification of the data server, the update data and an update version number, and the update version number is the version number of the update data.

And determining the identification of each background server which has a corresponding relation with the identification of the data server.

And pushing the updating data and the updating version number to each background server based on the identification of each background server so that each background server performs incremental updating according to the updating data and the updating version number.

It should be noted that, for details that are not mentioned in the embodiment corresponding to fig. 7 and the specific implementation manner of each step, reference may be made to the embodiments shown in fig. 1 to fig. 4 and the foregoing description, and details are not repeated here.

Referring to fig. 8, fig. 8 is a schematic structural diagram of a background server according to an embodiment of the present invention. The server 80 may include a receiver 801, a memory 802, and a processor 803, the receiver 801, the memory 802, and the processor 803 being connected by one or more communication buses.

The memory 802 may include a read-only memory and a random access memory, and provides instructions and data to the processor 803. A portion of the memory 802 may also include non-volatile random access memory.

The Processor 803 may be a Central Processing Unit (CPU), and the Processor 803 may also be other general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), an off-the-shelf Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic, discrete hardware components, etc. A general purpose processor may be a microprocessor, but in the alternative, the processor 803 may be any conventional processor or the like. Wherein:

a memory 802 for storing program instructions.

A processor 803 for invoking program instructions stored in the memory 802 for:

receiving push data sent by a synchronous server, wherein the push data comprises update data and an update version number, the update version number is the version number of the update data, and the connection state between a background server and the synchronous server is long connection;

detecting whether complete data exists in an internal memory;

if the complete data exists in the internal memory, acquiring the version number of the complete data;

if the update version number is different from the version number of the complete data, performing incremental update on the complete data according to the update data;

and associating and storing the updated complete data and the updated version number after the increment updating in the internal memory.

It should be noted that, for details that are not mentioned in the embodiment corresponding to fig. 8 and the specific implementation manner of each step, reference may be made to the embodiments shown in fig. 3 to fig. 4 and the foregoing description, and details are not repeated here.

Embodiments of the present invention further provide a computer-readable storage medium, in which a computer program is stored, where the computer program includes program instructions, and when the program instructions are executed by a processor, the processor is caused to execute the steps performed by the redis server in the method embodiments shown in fig. 1 to fig. 4.

Embodiments of the present invention further provide a computer-readable storage medium, in which a computer program is stored, where the computer program includes program instructions, and when the program instructions are executed by a processor, the processor is caused to execute the steps executed by the backend server in the method embodiments shown in fig. 3 to fig. 4.

While the invention has been described with reference to a number of embodiments, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims.

Claims

1. A data synchronization method is applied to a synchronization server with a publish-subscribe function, and comprises the following steps:

receiving an update message sent by a data server through an incremental update data interface, wherein the update message comprises an identifier of the data server, update data and an update version number, and the update version number is the version number of the update data;

determining the identification of each background server which has a corresponding relation with the identification of the data server, wherein the data server is used for storing the data required by each background server, the synchronous server is allowed to receive the updating messages sent by different data servers, and the identifications of the background servers corresponding to the identifications of different data servers are different;

and pushing the updating data and the updating version number to each background server through a release function based on the identification of each background server so that each background server performs incremental updating according to the updating data and the updating version number.

2. The method of claim 1, further comprising:

receiving a first data request sent by a background server, wherein the first data request is used for requesting target complete data with the same version number as the update version number, and the first data request is sent when the background server does not have complete data in an internal memory of the background server;

acquiring the target complete data, wherein the target complete data is obtained by the synchronous server through carrying out incremental updating on the complete data stored in an external memory of the synchronous server according to the updating data;

and sending the target complete data to the background server.

3. The method of claim 1, further comprising:

receiving a second data request sent by a background server, where the second data request is used to request update data with a version number that is the same as the version number in a version number set, the version number set includes 1 or more version numbers, and the second data request is sent when the background server has complete data in its internal memory and the difference between the update version number and the version number of the complete data in the internal memory is greater than a preset value;

acquiring an update data set, wherein the version numbers of the update data in the update data set are the same as the version numbers in the version number set in a one-to-one correspondence manner;

and sending the updated data set to the background server so that the background server performs incremental updating on the complete data after the last incremental updating according to the received updated data in the sequence of increasing the version number.

4. A data synchronization method is applied to a background server and comprises the following steps:

receiving push data sent by a synchronization server with a publish-subscribe function through a publish function based on an identifier of the background server, wherein the push data comprises update data and an update version number, the update version number is the version number of the update data, the connection state between the background server and the synchronization server is long connection, the identifier of the background server is determined by the synchronization server and has a corresponding relation with the identifier of a data server, the identifier of the data server, the update data and the update version number are contained in an update message received by the synchronization server through an incremental update data interface from the data server, the data server is used for storing required data of the background server, and the synchronization server is allowed to receive the update message sent by different data servers, the marks of the background servers corresponding to the marks of different data servers are different;

detecting whether complete data exists in an internal memory;

and associating and storing the updated complete data after incremental updating and the updated version number in the internal memory.

5. The method of claim 4, further comprising:

if the complete data does not exist in the internal memory, sending a first data request to the synchronous server, wherein the first data request is used for requesting target complete data with the same version number as the update version number;

receiving the target complete data sent by the synchronous server;

and associating and storing the target complete data and the update version number in the internal memory.

6. The method of claim 4, wherein if the update version number is different from the version number of the integrity data, incrementally updating the integrity data according to the update data comprises:

if the difference value between the updated version number and the version number of the complete data is a preset value, performing incremental updating on the complete data according to the updated data;

if the difference value between the update version number and the version number of the complete data is larger than the preset value, sending a second data request to the synchronous server, wherein the second data request is used for requesting update data with the version number being the same as that in a version number set, and the version number set comprises 1 or more version numbers;

receiving an update data set sent by the synchronous server, wherein the version numbers of the update data in the update data set are the same as the version numbers in the version number set in a one-to-one correspondence manner;

and according to the ascending sequence of the version numbers, sequentially according to the received updating data, carrying out incremental updating on the complete data after the last incremental updating.

7. The method according to any one of claims 4 to 6, further comprising:

detecting a connection state between the background server and the synchronous server at intervals of preset duration, wherein the connection state comprises long connection and disconnection;

if the connection state is disconnected, sending a long connection request to the synchronous server, wherein the long connection request is used for requesting to establish long connection with the synchronous server;

and receiving a connection success message sent by the synchronous server, wherein the connection success message is used for indicating that the long connection is successfully established between the background server and the synchronous server.

8. A data synchronization apparatus, characterized in that the apparatus comprises means for performing the method according to any of claims 1 to 7.

9. A server comprising a memory for storing a computer program comprising program instructions and a processor configured to invoke the program instructions to perform the method of any of claims 1 to 7.

10. A computer storage medium, characterized in that the computer storage medium stores a computer program comprising program instructions that, when executed by a processor, cause the processor to perform the method according to any of claims 1-7.