CN113783916B - Information synchronization method and device - Google Patents
Information synchronization method and device Download PDFInfo
- Publication number
- CN113783916B CN113783916B CN202011110808.7A CN202011110808A CN113783916B CN 113783916 B CN113783916 B CN 113783916B CN 202011110808 A CN202011110808 A CN 202011110808A CN 113783916 B CN113783916 B CN 113783916B
- Authority
- CN
- China
- Prior art keywords
- target information
- updated
- target
- identification
- information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1095—Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/56—Provisioning of proxy services
- H04L67/568—Storing data temporarily at an intermediate stage, e.g. caching
- H04L67/5682—Policies or rules for updating, deleting or replacing the stored data
Abstract
The application discloses an information synchronization method and device. One embodiment of the method comprises: determining the identification of target information in a centralized cache, wherein all the target information in a server cluster is stored in the centralized cache, and each target information is provided with an identification representing the sequence; determining the identification of the target information which is not updated by the target server in the server cluster from the identification of the target information stored in the centralized cache according to the updated identification and the identification of the target information, wherein the updated identification is the identification of the updated target information in the target server; and synchronizing the target information which is not updated to the target server according to the identification of the target information which is not updated. The embodiment provides an information synchronization method for a server cluster, all target information of the server cluster is stored in a centralized cache, and the information synchronization efficiency is improved.
Description
Technical Field
The embodiment of the application relates to the technical field of computers, in particular to an information synchronization method and device.
Background
The current service end system generally adopts a distributed cluster design, and the overall throughput of the system is improved by transversely expanding the scale of the service cluster. In a server cluster, the problem of memory data loading and synchronization among multiple servers is generally solved by the following method:
1. when the server is started, the information in the database corresponding to the server cluster is pulled in batches, and the pulled information is loaded into the memory. After the starting is completed, when one server instance receives a write request, the memory data of the server is updated firstly, then the record of the database is updated, other service instances in the cluster poll the database regularly, and the updated data are loaded into the local memory.
2. When the server is started, the information in the database corresponding to the server cluster is pulled in batches, and the pulled information is loaded into the memory. After the starting is finished, when a server instance processes a write request, firstly updating the memory data of the server; and meanwhile, the write request is forwarded to other server instances in the cluster, so that the memory data of the server instances in the cluster are kept synchronous.
Disclosure of Invention
The embodiment of the application provides an information synchronization method and device.
In a first aspect, an embodiment of the present application provides an information synchronization method, including: determining the identification of target information in a centralized cache, wherein all the target information in a server cluster is stored in the centralized cache, and each target information is provided with an identification representing the sequence; determining the identification of the target information which is not updated by the target server in the server cluster from the identification of the target information stored in the centralized cache according to the updated identification and the identification of the target information, wherein the updated identification is the identification of the updated target information in the target server; and synchronizing the target information which is not updated to the target server according to the identification of the target information which is not updated.
In some embodiments, the identification is a sequence number that characterizes the precedence by an increasing relationship; the synchronizing the non-updated target information to the target server according to the identification of the non-updated target information includes: the following acquisition operations are executed until the target servers all acquire the target information which is not updated: determining a difference value between the maximum sequence number in the updated mark and the maximum sequence number in the mark of the target information which is not updated; determining the smaller value of the difference value and the preset acquisition quantity as a target quantity, and acquiring target information of the target quantity from a serial number which is next to the maximum serial number in the updated identification from the non-updated target information; and synchronizing the maximum sequence number in the updated identification to the maximum sequence number corresponding to the target information of the target number.
In some embodiments, prior to determining the identity of the target information in the centralized cache, the method further comprises: determining configuration information in the process of synchronizing the target information which is not updated to a target server according to a preset static configuration file or a dynamic configuration instruction of a user, wherein the configuration information comprises: the preset acquisition number of the acquisition operations, the theme of the acquired target information and the number of concurrent acquisition threads of the acquisition operations are acquired.
In some embodiments, the above method further comprises: determining a delay time for each of a plurality of fetch threads within a preset length of time in response to determining that the fetch operation is concurrently executed by the plurality of fetch threads; and executing the obtaining thread according to the delay time corresponding to the obtaining thread aiming at each obtaining thread in the plurality of obtaining threads.
In some embodiments, the above method further comprises: suspending the obtaining operation in response to determining to execute the storing operation, wherein the storing operation represents storing the non-updated target information obtained up to now into a memory of the target server; in response to determining that the store operation was successfully completed, the fetch operation is restarted.
In some embodiments, the above method further comprises: in response to determining that the target server receives the write operation, determining, from the centralized cache, an identification of target information characterized by the write operation, and loading the target information into the centralized cache.
In a second aspect, an embodiment of the present application provides an information synchronization apparatus, including: the first determining unit is configured to determine an identifier of target information in a centralized cache, wherein all the target information in the server cluster is stored in the centralized cache, and each target information is provided with an identifier representing a sequence; a second determining unit, configured to determine, from the identifiers of the target information stored in the centralized cache, an identifier of target information that is not updated by the target servers in the server cluster according to the updated identifier and the identifier of the target information, where the updated identifier is the identifier of the target information that is updated in the target servers; a synchronization unit configured to synchronize the non-updated target information to the target server according to the identification of the non-updated target information.
In some embodiments, the identifiers are sequence numbers that characterize the precedence order by an increasing relationship; a synchronization unit further configured to: executing the following acquisition operation until the target server acquires all the target information which is not updated: determining a difference value between the maximum sequence number in the updated mark and the maximum sequence number in the mark of the target information which is not updated; determining the smaller value of the difference value and the preset acquisition number as a target number, and acquiring target information of the target number from a serial number which is next to the maximum serial number in the updated identification from the non-updated target information; and synchronizing the maximum sequence number in the updated identification to the maximum sequence number corresponding to the target information of the target number.
In some embodiments, the above apparatus further comprises: a configuration unit configured to: determining configuration information in the process of synchronizing the target information which is not updated to a target server according to a preset static configuration file or a dynamic configuration instruction of a user, wherein the configuration information comprises: the preset acquisition number of the acquisition operations, the theme of the acquired target information and the number of concurrent acquisition threads of the acquisition operations are acquired.
In some embodiments, the above apparatus further comprises: a delay unit configured to determine a delay time for each of a plurality of fetch threads within a preset length of time in response to determining that a fetch operation is concurrently performed by the plurality of fetch threads; and executing the obtaining thread according to the delay time corresponding to the obtaining thread aiming at each obtaining thread in the plurality of obtaining threads.
In some embodiments, the above apparatus further comprises: a storage unit configured to suspend the acquisition operation in response to determining to perform a storage operation, wherein the storage operation characterizes storing of non-updated target information acquired up to now into a memory of the target server; in response to determining that the store operation was successfully completed, the fetch operation is restarted.
In some embodiments, the above apparatus further comprises: a loading unit configured to determine, in response to determining that the target server receives the write operation, an identification of target information characterized by the write operation from the centralized cache, and load the target information into the centralized cache.
In a third aspect, the present application provides a computer-readable medium, on which a computer program is stored, where the program, when executed by a processor, implements the method as described in any implementation manner of the first aspect.
In a fourth aspect, an embodiment of the present application provides an electronic device, including: one or more processors; a storage device having one or more programs stored thereon, which when executed by one or more processors, cause the one or more processors to implement a method as described in any implementation of the first aspect.
The information synchronization method and device provided by the embodiment of the application determine the identifier of the target information in the centralized cache, wherein all the target information in the server cluster is stored in the centralized cache, and each target information is provided with the identifier representing the sequence; determining the identification of the target information which is not updated by the target server in the server cluster from the identification of the target information stored in the centralized cache according to the updated identification and the identification of the target information, wherein the updated identification is the identification of the updated target information in the target server; according to the identification of the target information which is not updated, the target information which is not updated is synchronized to the target server, so that the information synchronization method for the server cluster is provided.
Drawings
Other features, objects and advantages of the present application will become more apparent upon reading of the following detailed description of non-limiting embodiments thereof, made with reference to the accompanying drawings in which:
FIG. 1 is an exemplary system architecture diagram in which one embodiment of the present application may be applied;
FIG. 2 is a flow diagram of one embodiment of an information synchronization method according to the present application;
fig. 3 is a schematic diagram of an application scenario of the information synchronization method according to the present embodiment;
FIG. 4 is a flow diagram of yet another embodiment of an information synchronization method according to the present application;
FIG. 5 is a block diagram of one embodiment of an information synchronization apparatus according to the present application;
FIG. 6 is a block diagram of a computer system suitable for use in implementing embodiments of the present application.
Detailed Description
The present application will be described in further detail with reference to the drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the relevant invention and not restrictive of the invention. It should be noted that, for convenience of description, only the portions related to the related invention are shown in the drawings.
It should be noted that, in the present application, the embodiments and features of the embodiments may be combined with each other without conflict. The present application will be described in detail below with reference to the embodiments with reference to the attached drawings.
Fig. 1 illustrates an exemplary architecture 100 to which the information synchronization methods and apparatus of the present application may be applied.
As shown in fig. 1, the system architecture 100 may include end devices 101, 102, 103, a network 104, and a server cluster 105. The network 104 serves to provide a medium of communication links between the terminal devices 101, 102, 103 and the server cluster 105. Network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, to name a few.
The terminal devices 101, 102, 103 may be hardware devices or software that support network connections for data interaction and data processing. When the terminal devices 101, 102, and 103 are hardware, they may be various electronic devices supporting network connection, information interaction, display, processing, and the like, including but not limited to smart phones, tablet computers, e-book readers, laptop portable computers, desktop computers, and the like. When the terminal apparatuses 101, 102, 103 are software, they can be installed in the electronic apparatuses listed above. It may be implemented, for example, as multiple software or software modules to provide distributed services, or as a single software or software module. And is not particularly limited herein.
The server cluster 105 may include servers providing various services, such as a background processing server receiving a write operation performed by a user on the terminal devices 101, 102, and 103 to perform information synchronization. The method comprises the steps that a background processing server determines identification of target information in a centralized cache, wherein all the target information in a server cluster is stored in the centralized cache, and each target information is provided with identification representing the sequence; determining the identification of the target information which is not updated by the target server in the server cluster from the identification of the target information stored in the centralized cache according to the updated identification and the identification of the target information, wherein the updated identification is the identification of the updated target information in the target server; and synchronizing the target information which is not updated to the target server according to the identification of the target information which is not updated. As an example, the server cluster 105 may be a cloud server.
It should be noted that the server cluster may be hardware or software. When the server is hardware, it can be implemented as a distributed server cluster composed of multiple servers. When the server cluster is software, it may be implemented as a plurality of software or software modules (e.g., software or software modules used to provide distributed services). And is not particularly limited herein.
It should be further noted that the information synchronization method provided by the embodiments of the present disclosure may be executed by a server in a server cluster. Accordingly, various parts (e.g., various units, sub-units, modules, sub-modules) included in the information synchronization apparatus may be all provided in the servers in the server cluster.
It should be understood that the number of end devices, networks, and server clusters in fig. 1 is merely illustrative. There may be any number of end devices, networks, and server clusters, as desired for implementation. When the electronic device on which the information synchronization method operates does not need to perform data transmission with other electronic devices, the system architecture may only include the electronic device (e.g., a server cluster or a terminal device) on which the information synchronization method operates.
With continued reference to FIG. 2, a flow 200 of one embodiment of an information synchronization method is shown, comprising the steps of:
In this embodiment, an executing agent of the information synchronization method (for example, a server in the server cluster in fig. 1) may determine the identification of the target information in the centralized cache. All target information in the server cluster is stored in the centralized cache, and each target information is provided with an identifier representing the sequence.
The server cluster may be a distributed server cluster that increases the overall throughput of the system by horizontally expanding the server cluster size (number of servers). Each server cluster is correspondingly provided with a centralized cache, and the target information of all the servers in the server cluster is cached in the centralized cache. The target information may be any information involved in the server cluster. As an example, the target information may be information input by the user based on a write operation.
The target information in the server cluster is centrally managed through the centralized cache, so that the consistency of the information required to be synchronized by all the servers in the server cluster can be ensured; when the cache data is excessive and the capacity of the centralized cache needs to be expanded, the cache server can be conveniently expanded horizontally.
In this embodiment, each piece of target information in the centralized cache is provided with an identifier that is distinguished from other pieces of target information and represents the order of the target information and the other pieces of target information.
The mark may be any type of mark, including but not limited to geometric patterns, characters, and the like. Taking the character identifier as an example, the execution main body or the electronic device in communication connection with the execution main body may be configured with a sequence corresponding to the character, and set the corresponding identifier for the target information in the centralized cache according to the preset sequence.
In some optional implementation manners of this embodiment, the identifier may be a sequence number that represents a sequence order by an increasing relationship. The larger the number represented by the sequence number is, the later the time for representing the centralized cache to acquire the target information is.
In this embodiment, the execution main body may determine, from the target information stored in the centralized cache, an identifier of target information that is not updated by the target server in the server cluster, according to the updated identifier and the identifier of the target information. Wherein the updated identification is an identification of updated target information in the target server. The target server may be any server in a cluster of servers. It is to be understood that the target server may be the executing agent described above.
As an example, the execution subject may compare the updated identifier with the identifier of the target information in the centralized cache, and determine that the target information updated synchronously by the target server is stored in the centralized cache when an identifier not present in the updated identifier exists in the identifier of the target information in the centralized cache. At this time, the identifier of the target information corresponding to the identifier that does not exist in the updated identifiers and exists in the identifiers of the target information in the centralized cache may be determined as the identifier of the target information that is not updated by the target server.
In this embodiment, the execution subject may synchronize the target information that is not updated to the target server according to the identifier of the target information that is not updated.
As an example, first, the execution subject may determine target information that is not updated by the target server according to the identifier of the target information that is not updated, based on the uniqueness of the identifier. The executing agent may then synchronize the target information to the target server without updating.
In some optional implementations of this embodiment, for the identifier characterized in the form of a sequence number, the following obtaining operation is performed until the target server obtains all the target information that is not updated:
first, a difference between the largest sequence number in the updated identity and the largest sequence number in the identity of the target information that is not updated is determined.
In this implementation manner, since the sequence numbers represent the sequence of obtaining the target information by the centralized cache through the increasing relationship, the target information corresponding to the largest sequence number in the updated identifier is the target information with the latest time updated by the target server, and the target information corresponding to the largest sequence number in the identifier of the target information that is not updated is the target information with the latest time in the target information that needs to be updated by the target server. When the difference between the two is greater than 0, it may be determined that target information that is not updated by the target server exists in the centralized cache.
As an example, the maximum sequence number in the determined identifier of the target information that is not updated by the target server and exists in the centralized cache is 205, and the acquisition time of the corresponding target information is "2020.10.6"; and the maximum sequence number in the updated identifier is 185, and the corresponding time is "2020.10.6. Then the acquisition time is "2020.10.6 13" - "2020.10.6" 56", and the target information with sequence number 186-205 is the target information that is not updated by the target server.
And secondly, determining the smaller value of the difference value and the preset acquisition quantity as a target quantity, and acquiring target information of the target quantity from the sequence number which is the next to the maximum sequence number in the updated identification from the non-updated target information.
In this implementation, the preset acquisition number may be a number specifically set according to an actual situation. By determining the smaller value of the difference value and the preset acquisition number as the target number and performing acquisition operation, the execution main body can be prevented from wasting operating resources.
With continued reference to the example in the first step above, the preset number of acquisitions is 15. Then in the first acquisition operation, the smaller of the difference value (205-185 = 20) and the preset acquisition number 15 is the preset acquisition number 15, and the acquisition of the target information of the serial numbers 186-200 can be performed according to the preset acquisition number 15. After the target information with the sequence numbers 186-200 is acquired, the target server also has the target information with the sequence numbers 201-205 which is not updated synchronously. A second acquisition operation needs to be performed.
When the second obtaining operation is executed, if the smaller value of the difference (205-200 = 5) and the preset obtaining number 15 is the difference 5, the executing main body only needs to obtain the target information of the sequence numbers 201-205 according to the preset obtaining number 5, and waste of operating resources caused by obtaining the target information according to the preset obtaining number 15 is avoided.
And thirdly, synchronizing the maximum sequence number in the updated identification to the maximum sequence number corresponding to the target information of the target number.
After each acquisition operation, the execution subject needs to synchronize the maximum sequence number in the updated identifier to the maximum sequence number corresponding to the target information of the target number.
As an example, after the first acquiring operation is completed, the executing entity updates the identifier of the updated information to 200; after the second acquiring operation is completed, the executing entity updates the identifier of the updated information to 205.
It is to be understood that, in this implementation manner, when the difference between the maximum sequence number in the updated identifier and the maximum sequence number in the identifier of the target information that is not updated is zero, the execution main body may determine that all target servers acquire the target information that is not updated.
With continued reference to fig. 3, fig. 3 is a schematic diagram 300 of an application scenario of the information synchronization method according to the present embodiment. In the application scenario of fig. 3, a plurality of users respectively perform information interaction with the server cluster 301 through terminal devices. A centralized cache 3011 and a plurality of servers 3012, 3013, 3014, 3015 are provided in the server cluster 301. The centralized cache 3011 may receive target information input by a plurality of users in the server cluster through the terminal device, where each target information is provided with an identifier representing a sequence. For the target information input by the user, each server in the server cluster needs to synchronously update the target information in the centralized cache 3011. In the information synchronization process of the target server 3012, the server 3012 first determines the identity of the target information in the centralized cache. Then, according to the updated identifier and the identifier of the target information, the identifier of the target information that is not updated by the target server 3012 in the server cluster is determined from the identifiers of the target information stored in the centralized cache, where the updated identifier is the identifier of the target information that is updated in the target server. Finally, the non-updated target information is synchronized to the target server 3012 based on the identity of the non-updated target information.
In the method provided by the above embodiment of the present disclosure, the identifier of the target information in the centralized cache is determined, where all the target information in the server cluster is stored in the centralized cache, and each target information is provided with an identifier representing the order; determining an identifier of target information which is not updated by a target server in a server cluster from identifiers of target information stored in a centralized cache according to an updated identifier and an identifier of the target information, wherein the updated identifier is an identifier of the updated target information in the target server; according to the identification of the target information which is not updated, the target information which is not updated is synchronized to the target server, so that the information synchronization method for the server cluster is provided.
In some optional implementation manners of this embodiment, before determining the identifier of the target information in the centralized cache, the execution main body may further determine, according to a preset static configuration file or a dynamic configuration instruction of a user, configuration information in a process of synchronizing the target information that is not updated to the target server, where the configuration information includes: the preset acquisition number of the acquisition operations, the theme of the acquired target information and the number of concurrent acquisition threads of the acquisition operations are acquired.
The preset acquisition quantity is used for representing the quantity of target information which can be acquired by executing one acquisition operation; the subject of the target information is used for indicating the main body of the target information acquired when the execution main body executes the acquisition operation; the number of concurrent acquisition threads is used to characterize the number of acquisition threads concurrently executed when performing the acquisition operation. Through presetting a static configuration file or a dynamic configuration instruction, a user can flexibly set each configuration parameter in the information synchronization process so as to improve the flexibility of the information synchronization process.
In some optional implementations of this embodiment, the execution main body is configured to, in response to determining that the fetch operation is concurrently executed by a plurality of fetch threads, determine a delay time for each of the plurality of fetch threads within a preset time length; and executing the obtaining thread according to the delay time corresponding to the obtaining thread aiming at each obtaining thread in the plurality of obtaining threads.
It is understood that the preset time length can be specifically set according to actual conditions. As an example, the number of concurrently executing acquisition threads is proportional to the preset length of time. By randomly delaying the concurrently executing acquisition threads, the concurrent access pressure of the centralized cache is reduced.
In some optional implementations of this embodiment, the executing entity suspends the obtaining operation in response to determining to execute the storing operation, where the storing operation represents that the non-updated target information obtained up to now is stored in the memory of the target server; in response to determining that the store operation was successfully completed, the fetch operation is restarted.
In the implementation mode, the storage operation and the acquisition operation cannot be executed simultaneously, but the storage operation and the acquisition operation are executed alternately, so that the statistical errors of acquisition and storage of the target information are prevented when the storage operation and the acquisition operation are executed simultaneously, and the accuracy of information synchronization is improved.
In some optional implementation manners of this embodiment, in response to determining that the target server receives the write operation, the execution main body determines, from the centralized cache, an identifier of target information that is characterized by the write operation, and loads the target information into the centralized cache.
With continuing reference to FIG. 4, an exemplary flow 400 of another embodiment of an information synchronization method according to the present application is shown, comprising the steps of:
Wherein the configuration information includes: the preset acquisition number of the acquisition operations, the theme of the acquired target information and the number of concurrent acquisition threads of the acquisition operations are acquired.
At step 402, an identification of target information in a centralized cache is determined.
All target information in the server cluster is stored in the centralized cache, and each target information is provided with a sequence number representing the sequence through an increasing relationship.
And step 403, determining the identifier of the target information which is not updated by the target server in the server cluster from the identifiers of the target information stored in the centralized cache according to the updated identifier and the identifier of the target information.
Wherein the updated identification is an identification of updated target information in the target server.
As can be seen from this embodiment, compared with the embodiment corresponding to fig. 2, the flow 400 of the information synchronization method in this embodiment specifically illustrates that information synchronization in the server cluster is performed through the sequence represented by the sequence number of each target information in the centralized cache. Therefore, the convenience and the accuracy of the information synchronization of the embodiment are further improved.
With continuing reference to fig. 5, as an implementation of the methods shown in the above-mentioned figures, the present disclosure provides an embodiment of an information synchronization apparatus, which corresponds to the embodiment of the method shown in fig. 2, and which can be applied to various electronic devices.
As shown in fig. 5, the information synchronizing apparatus includes: the method comprises the following steps: a first determining unit 501, configured to determine an identifier of target information in a centralized cache, where all the target information in a server cluster is stored in the centralized cache, and each target information is provided with an identifier representing a sequence; a second determining unit 502 configured to determine, from the identifiers of the target information stored in the centralized cache, an identifier of target information that is not updated by the target server in the server cluster according to the updated identifier and the identifier of the target information, where the updated identifier is the identifier of the target information that is updated in the target server; a synchronization unit 503 configured to synchronize the non-updated target information to the target server according to the identification of the non-updated target information.
In some optional implementation manners of this embodiment, the identifier is a sequence number representing a sequence by an increasing relationship; a synchronization unit 503, further configured to: executing the following acquisition operation until the target server acquires all the target information which is not updated: determining a difference value between the maximum sequence number in the updated identification and the maximum sequence number in the identification of the target information which is not updated; determining the smaller value of the difference value and the preset acquisition number as a target number, and acquiring target information of the target number from a serial number which is next to the maximum serial number in the updated identification from the non-updated target information; and synchronizing the maximum sequence number in the updated identification to the maximum sequence number corresponding to the target information of the target number.
In some optional implementations of this embodiment, the apparatus further includes: a configuration unit (not shown in the figure) configured to: determining configuration information in the process of synchronizing the target information which is not updated to a target server according to a preset static configuration file or a dynamic configuration instruction of a user, wherein the configuration information comprises: the preset acquisition number of the acquisition operations, the theme of the acquired target information and the number of concurrent acquisition threads of the acquisition operations are acquired.
In some optional implementations of this embodiment, the apparatus further includes: a delay unit (not shown in the figures) configured to determine a delay time for each of the plurality of fetch threads within a preset length of time in response to determining that the fetch operation is concurrently performed by the plurality of fetch threads; and executing the obtaining thread according to the delay time corresponding to the obtaining thread aiming at each obtaining thread in the plurality of obtaining threads.
In some optional implementations of this embodiment, the apparatus further includes: a storage unit (not shown in the figure) configured to suspend the acquisition operation in response to determining to perform a storage operation, wherein the storage operation characterizes storing of non-updated target information acquired up to now into a memory of the target server; in response to determining that the store operation was successfully completed, the fetch operation is restarted.
In some optional implementations of this embodiment, the apparatus further includes: a loading unit (not shown in the figures) configured to determine, in response to determining that the target server receives the write operation, an identification of target information that is characterized by the write operation from the centralized cache, and load the target information into the centralized cache.
In this embodiment, a first generation unit in the information synchronization apparatus determines an identifier of target information in a centralized cache, where all target information in a server cluster is stored in the centralized cache, and each target information is provided with an identifier representing a sequence; the second generation unit determines an identifier of target information which is not updated by a target server in the server cluster from the identifiers of the target information stored in the centralized cache according to the updated identifier and the identifier of the target information, wherein the updated identifier is the identifier of the updated target information in the target server; the synchronization unit synchronizes the non-updated target information to the target server according to the identification of the non-updated target information, so that the information synchronization method for the server cluster is provided.
Referring now to FIG. 6, shown is a block diagram of a computer system 600 suitable for use in implementing devices of embodiments of the present application (e.g., devices 101, 102, 103, 105 shown in FIG. 1). The apparatus shown in fig. 6 is only an example, and should not bring any limitation to the function and use range of the embodiments of the present application.
As shown in fig. 6, the computer system 600 includes a processor (e.g., CPU, central processing unit) 601 that can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM) 602 or a program loaded from a storage section 608 into a Random Access Memory (RAM) 603. In the RAM603, various programs and data necessary for the operation of the system 600 are also stored. The processor 601, the ROM602, and the RAM603 are connected to each other via a bus 604. An input/output (I/O) interface 605 is also connected to bus 604.
The following components are connected to the I/O interface 605: an input portion 606 including a keyboard, a mouse, and the like; an output portion 607 including a display such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, and a speaker; a storage section 608 including a hard disk and the like; and a communication section 609 including a network interface card such as a LAN card, a modem, or the like. The communication section 609 performs communication processing via a network such as the internet. The driver 610 is also connected to the I/O interface 605 as needed. A removable medium 611 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 610 as necessary, so that a computer program read out therefrom is mounted in the storage section 608 as necessary.
In particular, the processes described above with reference to the flow diagrams may be implemented as computer software programs, according to embodiments of the present disclosure. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method illustrated in the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network through the communication section 609, and/or installed from the removable medium 611. Which when executed by the processor 601 performs the above-mentioned functions as defined in the method of the present application.
Note that the computer readable medium of the present application can be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present application, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In this application, however, a computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.
Computer program code for carrying out operations for aspects of the present application may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, smalltalk, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the client computer, partly on the client computer, as a stand-alone software package, partly on the client computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the client computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The units described in the embodiments of the present application may be implemented by software or hardware. The described units may also be provided in a processor, and may be described as: a processor includes a first generation unit, a second generation unit, and a synchronization unit. Where the names of these units do not in some cases constitute a limitation on the units themselves, for example, a synchronization unit may also be described as a "unit that synchronizes the target information that is not updated to the target server based on the identification of the target information that is not updated".
As another aspect, the present application also provides a computer-readable medium, which may be contained in the apparatus described in the above embodiments; or may be separate and not incorporated into the device. The computer readable medium carries one or more programs which, when executed by the apparatus, cause the computer device to: determining the identification of target information in a centralized cache, wherein all the target information in a server cluster is stored in the centralized cache, and each target information is provided with an identification representing the sequence; determining an identifier of target information which is not updated by a target server in a server cluster from identifiers of target information stored in a centralized cache according to an updated identifier and an identifier of the target information, wherein the updated identifier is an identifier of the updated target information in the target server; and synchronizing the target information which is not updated to the target server according to the identification of the target information which is not updated.
The above description is only a preferred embodiment of the application and is illustrative of the principles of the technology employed. It will be appreciated by those skilled in the art that the scope of the invention herein disclosed is not limited to the particular combination of features described above, but also encompasses other arrangements formed by any combination of the above features or their equivalents without departing from the spirit of the invention. For example, the above features may be replaced with (but not limited to) features having similar functions disclosed in the present application.
Claims (12)
1. An information synchronization method, comprising:
determining identifiers of target information in a centralized cache, wherein all the target information in a server cluster is stored in the centralized cache, and performing centralized management on all the target information in the server cluster, wherein each target information is provided with an identifier for representing a sequence, and the identifier is a serial number for representing the sequence through an increasing relationship;
according to the updated identification and the identification of the target information, determining the identification of the target information which is not updated by a target server in the server cluster from the identification of the target information stored in the centralized cache, wherein the updated identification is the identification of the updated target information in the target server;
synchronizing the non-updated target information to the target server according to the identification of the non-updated target information, comprising:
executing the following obtaining operation until the target server obtains the target information which is not updated completely:
determining a difference value between the maximum sequence number in the updated identification and the maximum sequence number in the identification of the target information which is not updated;
determining the smaller value of the difference value and the preset acquisition quantity as a target quantity, and acquiring target information of the target quantity from the sequence number which is the next to the maximum sequence number in the updated identification from the un-updated target information;
and synchronizing the maximum sequence number in the updated identification to the maximum sequence number corresponding to the target information of the target number.
2. The method of claim 1, wherein prior to the determining the identity of the target information in the centralized cache, further comprising:
determining configuration information in the process of synchronizing the non-updated target information to the target server according to a preset static configuration file or a dynamic configuration instruction of a user, wherein the configuration information comprises: the preset acquisition number of the acquisition operations, the theme of the acquired target information and the number of concurrent acquisition threads of the acquisition operations.
3. The method of claim 2, further comprising:
in response to determining that the fetch operation is concurrently performed by multiple fetch threads, determining a delay time for each fetch thread of the multiple fetch threads within a preset length of time;
and executing the obtaining thread according to the delay time corresponding to the obtaining thread aiming at each obtaining thread in the plurality of obtaining threads.
4. The method of claim 1, further comprising:
suspending the obtaining operation in response to determining to perform a storing operation, wherein the storing operation characterizes storing the non-updated target information obtained by now into a memory of the target server;
restarting the fetch operation in response to determining that the store operation was successfully completed.
5. The method of claim 1, further comprising:
in response to determining that the target server receives a write operation, determining, from the centralized cache, an identification of target information that is characterized by the write operation, and loading the target information into the centralized cache.
6. An information synchronization apparatus, comprising:
the first determining unit is configured to determine an identifier of target information in a centralized cache, wherein all the target information in a server cluster is stored in the centralized cache, the target information in the server cluster is managed in a centralized manner, each target information is provided with an identifier representing a sequence, and the identifier is a serial number representing the sequence through an increasing relationship;
a second determining unit, configured to determine, from the identifiers of the target information stored in the centralized cache, an identifier of target information that is not updated by a target server in the server cluster according to an updated identifier and the identifier of the target information, where the updated identifier is an identifier of target information that is updated in the target server;
a synchronization unit configured to synchronize the non-updated target information to the target server according to the identification of the non-updated target information;
the synchronization unit, further configured to: executing the following obtaining operation until the target server obtains the target information which is not updated completely: determining a difference value between the maximum sequence number in the updated identification and the maximum sequence number in the identification of the target information which is not updated; determining the smaller value of the difference value and the preset acquisition quantity as a target quantity, and acquiring target information of the target quantity from the sequence number which is the next to the maximum sequence number in the updated identification from the un-updated target information; and synchronizing the maximum sequence number in the updated identification to the maximum sequence number corresponding to the target information of the target number.
7. The apparatus of claim 6, further comprising:
a configuration unit configured to: determining configuration information in the process of synchronizing the non-updated target information to the target server according to a preset static configuration file or a dynamic configuration instruction of a user, wherein the configuration information comprises: the preset acquisition number of the acquisition operation, the theme of the acquired target information and the number of concurrent acquisition threads of the acquisition operation.
8. The apparatus of claim 7, further comprising:
a delay unit configured to determine a delay time for each of a plurality of fetch threads within a preset length of time in response to determining that the fetch operation is concurrently performed by the plurality of fetch threads; and executing the obtaining thread according to the delay time corresponding to the obtaining thread aiming at each obtaining thread in the plurality of obtaining threads.
9. The apparatus of claim 6, further comprising:
a storage unit configured to suspend the obtaining operation in response to determining to perform a storing operation, wherein the storing operation characterizes storing non-updated target information obtained up to now into a memory of the target server; restarting the fetch operation in response to determining that the store operation was successfully completed.
10. The apparatus of claim 6, further comprising:
a loading unit configured to determine, in response to determining that the target server receives a write operation, an identification of target information characterized by the write operation from the centralized cache, and load the target information into the centralized cache.
11. A computer-readable medium, on which a computer program is stored which, when being executed by a processor, carries out the method according to any one of claims 1-5.
12. An electronic device, comprising:
one or more processors;
a storage device having one or more programs stored thereon,
when executed by the one or more processors, cause the one or more processors to implement the method of any one of claims 1-5.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011110808.7A CN113783916B (en) | 2020-10-16 | 2020-10-16 | Information synchronization method and device |
| PCT/CN2021/122668 WO2022078243A1 (en) | 2020-10-16 | 2021-10-08 | Information synchronization method and apparatus |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011110808.7A CN113783916B (en) | 2020-10-16 | 2020-10-16 | Information synchronization method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113783916A CN113783916A (en) | 2021-12-10 |
| CN113783916B true CN113783916B (en) | 2022-12-27 |
Family
ID=78835072
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011110808.7A Active CN113783916B (en) | 2020-10-16 | 2020-10-16 | Information synchronization method and device |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN113783916B (en) |
| WO (1) | WO2022078243A1 (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114553895B (en) * | 2022-04-24 | 2022-07-22 | 浙江中控技术股份有限公司 | Data synchronization method, system, storage medium and electronic equipment |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102035865A (en) * | 2009-09-30 | 2011-04-27 | 阿里巴巴集团控股有限公司 | Data storing and addressing methods, systems and equipment |
| CN104468163A (en) * | 2013-09-18 | 2015-03-25 | 腾讯科技（北京）有限公司 | Method and device for disaster tolerant network construction, and disaster tolerant network |
| US9613046B1 (en) * | 2015-12-14 | 2017-04-04 | Netapp, Inc. | Parallel optimized remote synchronization of active block storage |
| CN108322533A (en) * | 2018-01-31 | 2018-07-24 | 广州鼎甲计算机科技有限公司 | Configuration and synchronization method between distributed type assemblies node based on operation log |
| CN110597673A (en) * | 2019-09-25 | 2019-12-20 | 腾讯科技（深圳）有限公司 | Disaster recovery method, device and equipment of storage system and computer readable storage medium |
Family Cites Families (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20080270629A1 (en) * | 2007-04-27 | 2008-10-30 | Yahoo! Inc. | Data snychronization and device handling using sequence numbers |
| JP2013065104A (en) * | 2011-09-15 | 2013-04-11 | Toshiba Corp | Load distribution system, data access device, and load distribution method |
| CN103369051B (en) * | 2013-07-22 | 2016-04-27 | 中安消技术有限公司 | A kind of data server cluster system and method for data synchronization |
| CN103716384A (en) * | 2013-12-17 | 2014-04-09 | 创新科存储技术（深圳）有限公司 | Method and device for realizing cloud storage data synchronization in cross-data-center manner |
| CN104283956B (en) * | 2014-09-30 | 2016-01-20 | 腾讯科技（深圳）有限公司 | Strong consistency distributed data storage method, Apparatus and system |
| CN104954101B (en) * | 2015-07-24 | 2018-06-01 | 湖北文理学院 | Multiple terminals method of data synchronization based on synchronous ACK |
| CN110019256A (en) * | 2017-08-03 | 2019-07-16 | 北京京东尚科信息技术有限公司 | Data processing method and device |
-
2020
- 2020-10-16 CN CN202011110808.7A patent/CN113783916B/en active Active
-
2021
- 2021-10-08 WO PCT/CN2021/122668 patent/WO2022078243A1/en active Application Filing
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102035865A (en) * | 2009-09-30 | 2011-04-27 | 阿里巴巴集团控股有限公司 | Data storing and addressing methods, systems and equipment |
| CN104468163A (en) * | 2013-09-18 | 2015-03-25 | 腾讯科技（北京）有限公司 | Method and device for disaster tolerant network construction, and disaster tolerant network |
| US9613046B1 (en) * | 2015-12-14 | 2017-04-04 | Netapp, Inc. | Parallel optimized remote synchronization of active block storage |
| CN108322533A (en) * | 2018-01-31 | 2018-07-24 | 广州鼎甲计算机科技有限公司 | Configuration and synchronization method between distributed type assemblies node based on operation log |
| CN110597673A (en) * | 2019-09-25 | 2019-12-20 | 腾讯科技（深圳）有限公司 | Disaster recovery method, device and equipment of storage system and computer readable storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113783916A (en) | 2021-12-10 |
| WO2022078243A1 (en) | 2022-04-21 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109388626B (en) | Method and apparatus for assigning numbers to services | |
| CN108846753B (en) | Method and apparatus for processing data | |
| CN110391938B (en) | Method and apparatus for deploying services | |
| CN111198751A (en) | Service processing method and device | |
| CN113094430B (en) | Data processing method, device, equipment and storage medium | |
| CN111309747A (en) | Data synchronization method, system and device | |
| CN112965945A (en) | Data storage method and device, electronic equipment and computer readable medium | |
| CN111782235A (en) | Data upgrading and querying method and device | |
| CN111338834B (en) | Data storage method and device | |
| CN113783916B (en) | Information synchronization method and device | |
| CN110795495A (en) | Data processing method and device, electronic equipment and computer readable medium | |
| CN110888773B (en) | Method, device, medium and electronic equipment for acquiring thread identification | |
| CN112084254A (en) | Data synchronization method and system | |
| CN112115149A (en) | Method and device for providing data | |
| CN111309366B (en) | Method, device, medium and electronic equipment for managing registration core | |
| CN113742376A (en) | Data synchronization method, first server and data synchronization system | |
| CN114785770A (en) | Mirror layer file sending method and device, electronic equipment and computer readable medium | |
| CN108287792B (en) | Method and apparatus for outputting information | |
| CN112948494A (en) | Data synchronization method and device, electronic equipment and computer readable medium | |
| CN110750424A (en) | Resource inspection method and device | |
| CN112148705A (en) | Data migration method and device | |
| CN110661857A (en) | Data synchronization method and device | |
| CN110633324B (en) | Method, apparatus, electronic device and computer readable medium for synchronizing data | |
| CN116975012A (en) | File synchronization method, file updating method, device, electronic equipment and medium | |
| CN113761049A (en) | Data synchronization method and device under read-write separation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |