CN109561126B - Data synchronization method and device, storage medium and processor - Google Patents
Data synchronization method and device, storage medium and processor Download PDFInfo
- Publication number
- CN109561126B CN109561126B CN201710889039.7A CN201710889039A CN109561126B CN 109561126 B CN109561126 B CN 109561126B CN 201710889039 A CN201710889039 A CN 201710889039A CN 109561126 B CN109561126 B CN 109561126B
- Authority
- CN
- China
- Prior art keywords
- data
- synchronization
- type
- synchronization result
- failure
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1095—Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a data synchronization method and device, a storage medium and a processor, wherein the method comprises the following steps: acquiring synchronization result information of data synchronization of a first system from a second system; storing the synchronization result information into a database; the synchronization result information comprises an item identifier of the data and a synchronization result type of the data corresponding to the item identifier; searching a synchronization result type with a failure type in the database, determining an item identifier corresponding to the searched synchronization result type as a failure item identifier, and controlling the first system to resynchronize data corresponding to the failure item identifier from the second system; the invention can automatically monitor the data synchronization condition, and control the first system to resynchronize the data after determining that the data synchronization fails, thereby improving the efficiency of data synchronization.
Description
Technical Field
The present invention relates to the field of data processing technologies, and in particular, to a data synchronization method and apparatus, a storage medium, and a processor.
Background
With the development of network technology, the interaction between different data systems is increasing, and one interaction mode is that one data system synchronizes data in another data system. In general, in the process of data synchronization in a data system, data synchronization often fails due to network delay and the like.
In the prior art, after the data system completes data synchronization, a technician is required to manually check data which fails to be synchronized, and control the data system again to synchronize the data. Then, when the amount of data that fails to be synchronized is large, a technician needs to spend a lot of time and cost for processing, and manual processing often causes errors, resulting in inefficient data synchronization.
Disclosure of Invention
In view of the above, the present invention has been made to provide a data synchronization method and apparatus, a storage medium, and a processor that overcome or at least partially solve the above problems.
A method of data synchronization, comprising:
acquiring synchronization result information of data synchronization of a first system from a second system;
storing the synchronization result information into a database; the synchronization result information comprises an item identifier of the data and a synchronization result type of the data corresponding to the item identifier;
and searching a synchronization result type with a failure type in the database, determining an item identifier corresponding to the searched synchronization result type as a failure item identifier, and controlling the first system to resynchronize data corresponding to the failure item identifier from the second system.
Preferably, the synchronization result information further includes a data type corresponding to the synchronization result type, where each data type included in the data corresponding to each item identifier corresponds to a synchronization result type;
the controlling the first system to resynchronize data corresponding to the failed item identification from the second system includes:
and for the data corresponding to the failed item identification, determining the data type corresponding to the synchronization result type with the type being a failure as a failed data type, and controlling the first system to resynchronize the data corresponding to the failed data type from the second system.
Preferably, the synchronization result information further includes a synchronization time of the data corresponding to the item identification; the method further comprises the following steps:
determining target synchronization result information of the synchronization time in a preset time period in the database;
correspondingly, the searching for the synchronization result type with the failure in the database, and determining the item identifier corresponding to the searched synchronization result type as the failed item identifier include:
and searching the synchronization result type with the type of failure in the target synchronization result information in the database, and identifying the item corresponding to the searched synchronization result type as a failure item identification.
Preferably, the first system runs on at least one server;
before the searching the synchronization result type with the failure type in the database and determining the item mark identification corresponding to the searched synchronization result type as the failure item identification, the method further comprises:
determining that all servers operated by the first system complete synchronization of data.
Preferably, the acquiring synchronization result information for data synchronization performed by the first system from the second system includes:
monitoring a data synchronization process of the first system from the second system for the item identification;
and acquiring synchronization result information aiming at the project identification after the data synchronization aiming at the project identification of the first system is determined to be finished.
A data synchronization apparatus, comprising:
an information acquisition unit, configured to acquire synchronization result information of data synchronization performed by a first system from a second system;
the data storage unit is used for storing the synchronization result information into a database; the synchronization result information comprises an item identifier of the data and a synchronization result type of the data corresponding to the item identifier;
and the resynchronization unit is used for searching the synchronization result type which is found to be a failure in the database, determining the item identification corresponding to the searched synchronization result type as a failure item identification, and controlling the first system to resynchronize the data corresponding to the failure item identification from the second system.
Preferably, the synchronization result information further includes a data type of the data corresponding to the synchronization result type, where each data type included in the data corresponding to each item identifier corresponds to the synchronization result type;
the resynchronization unit includes:
the first searching module is used for searching a synchronization result type with a failure type in the database and determining an item identifier corresponding to the searched synchronization result type as a failure item identifier;
and the first synchronization module is used for determining the data type corresponding to the synchronization result type with the type being failure as the failure data type for the data corresponding to the failure item identification, and controlling the first system to resynchronize the data corresponding to the failure data type from the second system.
Preferably, the synchronization result information further includes a synchronization time of the data corresponding to the item identification; the device also includes:
a first determining unit, configured to determine, in the database, target synchronization result information of the synchronization time within a preset time period;
the resynchronization unit includes:
the second searching module is used for searching the synchronization result type with the type of failure in the target synchronization result information and identifying the item corresponding to the searched synchronization result type as a failure item identification;
and the second synchronization module is used for controlling the first system to resynchronize the data corresponding to the failed item identifier from the second system.
A storage medium comprising a stored program, wherein a device on which the storage medium is located is controlled to perform a data synchronization method as described in any one of the above when the program is run.
A processor configured to run a program, the program when running performing a method of data synchronization as claimed in any one of the preceding claims.
By means of the technical scheme, the invention provides a data synchronization method, which specifically comprises the following steps: acquiring synchronization result information of data synchronization performed by a first system from a second system, and storing the synchronization result information into a database, wherein the synchronization result information comprises an item identifier of the data and a synchronization result type of the data corresponding to the item identifier; searching a synchronization result type with a failure type in a database, determining an item identifier corresponding to the searched synchronization result type as a failure item identifier, and controlling the first system to resynchronize data corresponding to the failure item identifier from the second system; therefore, the method and the device can automatically monitor the data synchronization condition, control the first system to resynchronize the data after determining that the data synchronization fails, and improve the data synchronization efficiency.
The foregoing description is only an overview of the technical solutions of the present invention, and the embodiments of the present invention are described below in order to make the technical means of the present invention more clearly understood and to make the above and other objects, features, and advantages of the present invention more clearly understandable.
Drawings
Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are only for purposes of illustrating the preferred embodiments and are not to be construed as limiting the invention. Also, like reference numerals are used to refer to like parts throughout the drawings. In the drawings:
fig. 1 is a flow chart illustrating a data synchronization method according to an embodiment of the present invention;
fig. 2 is a flow chart illustrating a data synchronization method according to another embodiment of the present invention;
fig. 3 is a flowchart illustrating a data synchronization method according to another embodiment of the present invention;
fig. 4 is a flowchart illustrating a data synchronization method according to another embodiment of the present invention;
fig. 5 is a schematic structural diagram illustrating a data synchronization apparatus according to an embodiment of the present invention;
fig. 6 is a schematic structural diagram illustrating a data synchronization apparatus according to another embodiment of the present invention;
fig. 7 is a schematic structural diagram illustrating a data synchronization apparatus according to another embodiment of the present invention;
fig. 8 is a schematic structural diagram illustrating a data synchronization apparatus according to another embodiment of the present invention.
Detailed Description
Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.
An embodiment of the present invention discloses a data synchronization method, as shown in fig. 1, the method includes the following steps:
step 101: acquiring synchronization result information of data synchronization of a first system from a second system;
the first system is an SEMD (Search Engine Marketing operation analysis tool) system, the second system is a WD (webdistactor, website data monitoring analysis tool) system, the WD system is used for collecting backend WD data of the Search Engine, and the SEMD system is used for synchronizing WD data from the WD system.
It should be noted that the data WD at the back end of the search engine is the operation data after entering the item using the search engine as an entry.
In practical applications, in order to ensure the popularity of a project, the project is promoted in a search engine, and the project refers to a project which provides various services for users, such as project which provides shopping services for users, such as kyoto, naobao, suting and the like. Taking the example that the jingdong is popularized in hundreds, after keyword search is performed in hundreds, hundreds can push search results of the jingdong about the keyword, and after a user clicks the search results of the jingdong about the keyword, the user jumps to a website of the jingdong, and operation data performed by the user on the website of the jingdong is the back-end WD data of hundreds, for example, the back-end WD data can include order amount data, shopping cart adding data, order amount data and the like.
Optionally, the obtaining synchronization result information of the first system performing data synchronization from the second system includes the following processes:
(1) monitoring a data synchronization process of the first system from the second system for the item identification;
the item identifications are used for uniquely characterizing the items, and a data synchronization process of the first system from the second system for each item identification is monitored.
(2) And acquiring synchronization result information aiming at the project identification after the data synchronization aiming at the project identification of the first system is determined to be finished.
And when the data synchronization of the first system aiming at one item identification is finished, acquiring the synchronization result information aiming at the item identification. That is, the first system generally synchronizes data from the second system according to the item identification, and the synchronization result information is for the synchronization of the data of each item identification.
Step 102: storing the synchronization result information into a database;
the synchronization result information comprises item identification of the data and synchronization result type of the data, and contains the corresponding relation between the item identification and the synchronization result type of the data.
The synchronization result type includes a success type and a failure type.
For example, the data synchronization of the kyoton is successful, the database stores the corresponding relationship between the kyoton and the success type, the data synchronization of the suting easy-to-purchase is failed, and the database stores the corresponding relationship between the suting easy-to-purchase and the failure type, wherein both the kyoton easy-to-purchase and the suting easy-to-purchase are the item identifiers. It should be noted that the specific expression of the item identifier and the synchronization result type may be set according to the actual situation, and the present invention is not limited thereto, for example, the success type is represented by a number "1", the failure type is represented by a data "0", or the success and failure types are directly represented by "synchronization success" and "synchronization failure".
Step 103: and searching a synchronization result type with a failure type in the database, determining an item identifier corresponding to the searched synchronization result type as a failure item identifier, and controlling the first system to resynchronize data corresponding to the failure item identifier from the second system.
It is understood that the failed item id does not refer to a specific item, but is only used to characterize the item id corresponding to the type of failure found in the corresponding relationship.
It should be noted that, the data synchronization method provided by the present invention is applied to a first system, optionally, the first system may include a data monitoring module and a data synchronization module, the data monitoring module obtains synchronization result information of the first system performing data synchronization from a second system, stores the synchronization result information in a database, searches for a synchronization result type with a type of failure in the database, determines an item identifier corresponding to the searched synchronization result type as a failed item identifier, sends the failed item identifier to the data synchronization module, and the data synchronization module re-synchronizes data corresponding to the failed item identifier from the second system.
Therefore, the present embodiment provides a data synchronization method, including: acquiring synchronization result information of data synchronization performed by a first system from a second system, and storing the synchronization result information into a database, wherein the synchronization result information comprises an item identifier of the data and a synchronization result type of the data corresponding to the item identifier; searching a synchronization result type with a failure in the database, determining an item identifier corresponding to the searched synchronization result type as a failure item identifier, and controlling the first system to resynchronize data corresponding to the failure item identifier from the second system; the invention can automatically monitor the data synchronization condition, and controls the first system to resynchronize the data after determining that the data synchronization fails, thereby improving the efficiency of data synchronization.
Another embodiment of the present invention provides a data synchronization method, as shown in fig. 2, including the following steps:
step 201: acquiring synchronization result information of data synchronization of a first system from a second system;
step 202: storing the synchronization result information into a database;
the synchronization result information includes an item identifier of data and a synchronization result type of the data corresponding to the item identifier, and the synchronization result information also includes a data type of the data corresponding to the synchronization result type. And each data type contained in the data corresponding to each item identifier corresponds to a synchronization result type.
It should be noted that, an item identifier may include data of multiple data types, and when data synchronization of data corresponding to an item identifier fails, it may be that synchronization of partial data in the data corresponding to the item identifier succeeds, and synchronization of another partial data fails.
Taking WD data as an example, the data type may be an order amount, shopping cart item information, and the like.
Specifically, in the synchronization result information, the item identifier of the data may have a corresponding relationship with the data type of the data, and each data type has a synchronization result type, so as to establish a corresponding relationship between the item identifier of the data, the data type of the data, and the synchronization result type.
For example, the data types of the data include data type 1 and data type 2, the item is identified as AD1, the data synchronization of data type 1 fails, and the data synchronization of data type 2 succeeds, then the synchronization result information may include: the item identifier AD1 has data type 1 and data type 2, where data type 1 corresponds to a failure type and data type 2 corresponds to a success type.
Of course, in the synchronization result information, the item identification of the data may directly correspond to the synchronization result type of the data, and the data type of the data corresponds to the success or failure of the synchronization of the data under the data type.
For example, the data types of the data include data type 1 and data type 2, the item identifier is AD1, the data synchronization of data type 1 fails, and the data synchronization of data type 2 succeeds, then the data has failed synchronization because of the existence of the data, and the synchronization result type of the data is failure, in the synchronization result information, the item identifier AD1 corresponds to the failure type, the item identifier AD1 has data type 1 and data type 2, the data type 1 corresponds to the failure type, and the data type 2 corresponds to the success type.
It should be noted that the synchronization result information may be transmitted in a form of a table, and may also be transmitted in other forms, which is not limited in the present invention.
Step 203: and searching a synchronization result type with a failure type in the database, determining an item identifier corresponding to the searched synchronization result type as a failure item identifier, determining a data type corresponding to the synchronization result type with the failure type as a failure data type for data corresponding to the failure item identifier, and controlling the first system to resynchronize the data corresponding to the failure data type from the second system.
The item identification has different data types, each data type corresponds to a synchronization result type, the item identification corresponding to the found synchronization result type is determined as a failure item identification by finding the synchronization result type with the failure, and in the failure item identification, different data types of the data exist.
For example, the kyoto has order quantity data and order amount data, and when the order quantity data and the order amount data are synchronized, the order quantity data are successfully synchronized, and the order amount data are not synchronized. Then, the synchronization result information includes that the item identifier is kyoto, the data type of the kyoto is an order quantity data type and an order amount data type, wherein the order quantity data type corresponds to the synchronization result type with the successful type, and the order amount data type corresponds to the synchronization result type with the failed type. And determining the Jingdong corresponding to the searched synchronous result type as a failed item identifier by searching the synchronous result type with the determined type as a failure, and then determining the order amount data type corresponding to the synchronous result type with the determined type as a failure data type, so as to control the first system to resynchronize the order amount data in the Jingdong from the second system.
Therefore, the present embodiment provides a data synchronization method, including: acquiring synchronization result information of data synchronization performed by a first system from a second system, and storing the synchronization result information into a database, wherein the synchronization result information comprises an item identifier of the data and a synchronization result type of the data corresponding to the item identifier; searching a synchronization result type with a failure in the database, determining an item identifier corresponding to the searched synchronization result type as a failure item identifier, and controlling the first system to resynchronize data corresponding to the failure item identifier from the second system; the invention can automatically monitor the data synchronization condition, and controls the first system to resynchronize the data after determining that the data synchronization fails, thereby improving the efficiency of data synchronization.
Furthermore, the synchronization result information also includes the data type of the data corresponding to the synchronization result type, and the first system can be controlled to resynchronize the data corresponding to the failed data type from the second system, that is, the first system is controlled to resynchronize partial data in the failed item identifier, thereby further improving the efficiency of data synchronization.
Another embodiment of the present invention discloses a data synchronization method, as shown in fig. 3, the method includes the following steps:
step 301: acquiring synchronization result information of data synchronization of a first system from a second system;
step 302: storing the synchronization result information into a database;
the synchronization result information comprises an item identifier of the data and a synchronization result type of the data corresponding to the item identifier, and the synchronization result information further comprises synchronization time of the data corresponding to the item identifier.
The synchronization time is a synchronization time when the first system synchronizes the data identified by each item from the second system.
Step 303: determining target synchronization result information of synchronization time in a preset time period in the database;
the preset time period is data set in advance for specifying which time period of resynchronization failed synchronization. For example, the day before the current time is preset, then all target synchronization result information of the synchronization time on the day before the current time can be screened out from the database.
The data synchronization based on the actual requirement is realized by determining the target synchronization result information of the synchronization time in the preset time period, and only the data which fails in synchronization in the preset time period is synchronized, so that the data synchronization efficiency is improved.
Step 304: and searching the synchronization result type with the type of failure in the target synchronization result information in a database, identifying the item corresponding to the searched synchronization result type as a failure item identification, and controlling the first system to resynchronize the data corresponding to the failure item identification from the second system.
The step of searching for a synchronization result type with a failure type in the target synchronization result information in the database, and identifying an item corresponding to the searched synchronization result type as a failed item identification is a specific implementation of the step of searching for a synchronization result type with a failure type in the database, and determining an item identification corresponding to the searched synchronization result type as a failed item identification.
Therefore, in the embodiment, synchronization result information of data synchronization from the second system by the first system is acquired, and the synchronization result information is stored in the database, wherein the synchronization result information includes an item identifier of the data and a synchronization result type of the data corresponding to the item identifier; searching a synchronization result type with a failure in the database, determining an item identifier corresponding to the searched synchronization result type as a failure item identifier, and controlling the first system to resynchronize data corresponding to the failure item identifier from the second system; the invention can automatically monitor the data synchronization condition, and controls the first system to resynchronize the data after determining that the data synchronization fails, thereby improving the efficiency of data synchronization.
Further, after the target synchronization result information of the synchronization time within the preset time period is determined in the database, the synchronization result type with the type being failed is searched in the target synchronization result information in the database, the item corresponding to the searched synchronization result type is identified as the failed item identification, the first system is controlled to resynchronize the data corresponding to the failed item identification from the second system, the resynchronization of the data with the synchronization failure within the specified time period is realized, and the efficiency of data synchronization is further improved.
Another embodiment of the present invention provides a data synchronization method, as shown in fig. 4, including the following steps:
step 401: acquiring synchronization result information of data synchronization of a first system from a second system;
step 402: storing the synchronization result information into a database;
the synchronization result information comprises an item identification of the data and a synchronization result type of the data corresponding to the item identification.
Step 403: determining that all servers operated by the first system complete data synchronization;
the first system runs on at least one server, each server can synchronize data, specifically, each server can synchronize data at a preset time point every day.
Step 404: and searching a synchronization result type with a failure type in the database, determining an item mark corresponding to the searched synchronization result type as a failure item mark, and controlling the first system to resynchronize data corresponding to the failure item mark from the second system.
Therefore, in this embodiment, synchronization result information of data synchronization performed by the first system from the second system is obtained, and the synchronization result information is stored in the database, where the synchronization result information includes an item identifier of the data and a synchronization result type of the data corresponding to the item identifier; searching a synchronization result type with a failure in the database, determining an item identifier corresponding to the searched synchronization result type as a failure item identifier, and controlling the first system to resynchronize data corresponding to the failure item identifier from the second system; the invention can automatically monitor the data synchronization condition, and controls the first system to resynchronize the data after determining that the data synchronization fails, thereby improving the efficiency of data synchronization.
Corresponding to the above data synchronization method, the present invention also discloses a data synchronization apparatus, which is described below by various embodiments, specifically:
an embodiment of the present invention discloses a data synchronization apparatus, as shown in fig. 5, the apparatus includes: an information acquisition unit 501, a data storage unit 502, and a resynchronization unit 503, wherein:
an information acquiring unit 501, configured to acquire synchronization result information of data synchronization performed by the first system from the second system;
the synchronization result information includes item identification of the data and a synchronization result type of the data, and includes a correspondence relationship between the two.
The first system is a Search Engine Marketing operation analysis tool (SEMD) system, the second system is a Web monitor analysis tool (WD) system, the WD system is used for collecting backend WD data of the Search Engine, and the SEMD system is used for synchronizing WD data from the WD system.
It should be noted that the data WD at the back end of the search engine is the operation data after entering the item using the search engine as an entry.
Optionally, the information obtaining unit 501 includes: the system comprises a first monitoring module and a first acquisition module; wherein: the first monitoring module is used for monitoring a data synchronization process of the first system from the second system aiming at the project identification; and the first acquisition module is used for acquiring the synchronization result information aiming at the project identification after the data synchronization aiming at the project identification of the first system is determined to be finished.
The item identifications are used for uniquely characterizing the items, and a data synchronization process of the first system from the second system for each item identification is monitored.
And when the data synchronization of the first system aiming at one project identification is completed, acquiring the synchronization result information aiming at the project identification. That is, the first system generally synchronizes data from the second system according to the item identification, and the synchronization result information is for the synchronization of the data of each item identification.
A data storage unit 502, configured to store the synchronization result information in a database;
the synchronization result information comprises the corresponding relation between the item identification of the data and the synchronization result type of the data.
The synchronization result type includes a success type and a failure type.
A resynchronization unit 503, configured to search for a synchronization result type with a failure type in the database, determine an item identifier corresponding to the searched synchronization result type as a failed item identifier, and control the first system to resynchronize data corresponding to the failed item identifier from the second system.
It can be understood that the failed item identifier does not refer to a certain item, but is only used for characterizing the item identifier of which the type found in the corresponding relationship is the type of the failed synchronization result.
Therefore, in this embodiment, synchronization result information of data synchronization performed by the first system from the second system is obtained, and the synchronization result information is stored in the database, where the synchronization result information includes an item identifier of the data and a synchronization result type of the data corresponding to the item identifier; searching a synchronization result type with a failure in the database, determining an item identifier corresponding to the searched synchronization result type as a failure item identifier, and controlling the first system to resynchronize data corresponding to the failure item identifier from the second system; the invention can automatically monitor the data synchronization condition, and controls the first system to resynchronize the data after determining that the data synchronization fails, thereby improving the efficiency of data synchronization.
Another embodiment of the present invention discloses a data synchronization apparatus, as shown in fig. 6, the apparatus includes: an acquisition information unit 601, a storage data unit 602, and a resynchronization unit 603; the resynchronization unit 603 includes: a first lookup module 6031 and a first synchronization module 6032; specifically, the method comprises the following steps:
an obtaining information unit 601, configured to obtain synchronization result information for performing data synchronization from a second system by a first system;
a data storing unit 602, configured to store the synchronization result information in a database;
the synchronization result information comprises an item identifier of the data and a synchronization result type of the data corresponding to the item identifier; the synchronization result information further includes a data type of the data corresponding to the synchronization result type. And each data type contained in the data corresponding to each item identifier corresponds to a synchronization result type.
The synchronization result type may be a success type or a failure type of data for each data type.
It should be noted that, an item identifier may include data of multiple data types, and when synchronization of data corresponding to an item identifier fails, it may be that synchronization of part of the data corresponding to the item identifier succeeds, and synchronization of another part of the data fails.
A first searching module 6031, configured to search, in the database, a synchronization result type that is a failure, and determine an item identifier corresponding to the searched synchronization result type as a failure item identifier;
a first synchronization module 6032, configured to, for the data corresponding to the failed item identifier, determine the data type corresponding to the synchronization result type that is the type of failure as a failed data type, and control the first system to resynchronize the data corresponding to the failed data type from the second system.
Therefore, in this embodiment, synchronization result information of the first system performing data synchronization from the second system is obtained, and the synchronization result information is stored in the database, where the synchronization result information includes an item identifier of the data and a synchronization result type of the data corresponding to the item identifier; searching a synchronization result type with a failure in the database, determining an item identifier corresponding to the searched synchronization result type as a failure item identifier, and controlling the first system to resynchronize data corresponding to the failure item identifier from the second system; the method and the system can automatically monitor the data synchronization condition, and control the first system to resynchronize the data after determining that the data synchronization fails, so that the data synchronization efficiency is improved;
furthermore, the synchronization result information also includes the data type of the data corresponding to the synchronization result type, and the first system can be controlled to resynchronize the data corresponding to the failed data type from the second system, that is, the first system is controlled to resynchronize partial data in the failed item identifier, thereby further improving the efficiency of data synchronization.
Still another embodiment of the present invention further provides a data synchronization apparatus, as shown in fig. 7, the apparatus including: an acquisition information unit 701, a storage data unit 702, a first determination unit 703, and a resynchronization unit 704; wherein:
an obtaining information unit 701, configured to obtain synchronization result information for performing data synchronization from a second system by a first system;
a data storage unit 702, configured to store the synchronization result information in a database;
the synchronization result information comprises an item identifier of the data and a synchronization result type of the data corresponding to the item identifier, and the synchronization result information further comprises synchronization time of the data corresponding to the item identifier.
The synchronization time is a synchronization time when the first system synchronizes the data identified by each item from the second system.
A first determining unit 703, configured to determine, in the database, target synchronization result information of the synchronization time within a preset time period;
the preset time period is data set in advance for specifying which time synchronization fails to be resynchronized.
A resynchronization unit 704 comprising: a second lookup module 7041 and a second synchronization module 7042;
a second searching module 7041, configured to search, in the target synchronization result information, a synchronization result type that is a failure, and identify an item corresponding to the searched synchronization result type as a failed item identifier;
a second synchronization module 7042, configured to control the first system to resynchronize data corresponding to the failed item identifier from the second system.
Therefore, in this embodiment, synchronization result information of data synchronization performed by the first system from the second system is obtained, and the synchronization result information is stored in the database, where the synchronization result information includes an item identifier of the data and a synchronization result type of the data corresponding to the item identifier; searching a synchronization result type with a failure in the database, determining an item identifier corresponding to the searched synchronization result type as a failure item identifier, and controlling the first system to resynchronize data corresponding to the failure item identifier from the second system; the method and the system can automatically monitor the data synchronization condition, and control the first system to resynchronize the data after determining that the data synchronization fails, so that the data synchronization efficiency is improved;
further, after the target synchronization result information of the synchronization time within the preset time period is determined in the database, the synchronization result type with the type being failed is searched in the target synchronization result information in the database, the item corresponding to the searched synchronization result type is identified as the failed item identification, the first system is controlled to resynchronize the data corresponding to the failed item identification from the second system, the resynchronization of the data with the synchronization failure within the specified time period is realized, and the efficiency of data synchronization is further improved.
Another embodiment of the present invention provides a data synchronization apparatus, as shown in fig. 8, the apparatus including: an acquisition information unit 801, a storage data unit 802, a determination synchronization unit 803, and a resynchronization unit 804; wherein:
an obtaining information unit 801 configured to obtain synchronization result information of data synchronization performed by the first system from the second system;
a data storage unit 802, configured to store the synchronization result information in a database;
the synchronization result information comprises an item identification of the data and a synchronization result type of the data corresponding to the item identification.
A determining synchronization unit 803, configured to determine that all servers operated by the first system complete data synchronization;
the first system runs on at least one server, each server can synchronize data, specifically, each server can synchronize data at a preset time point every day.
A resynchronization unit 804, configured to search the database for a synchronization result type that is a failure, determine an item identifier corresponding to the searched synchronization result type as a failure item identifier, and control the first system to resynchronize data corresponding to the failure item identifier from the second system.
Therefore, in this embodiment, synchronization result information of the first system performing data synchronization from the second system is obtained, and the synchronization result information is stored in the database, where the synchronization result information includes an item identifier of the data and a synchronization result type of the data corresponding to the item identifier; searching a synchronization result type with a failure in the database, determining an item identifier corresponding to the searched synchronization result type as a failure item identifier, and controlling the first system to resynchronize data corresponding to the failure item identifier from the second system; the invention can automatically monitor the data synchronization condition, and controls the first system to resynchronize the data after determining that the data synchronization fails, thereby improving the efficiency of data synchronization.
The data processing device comprises a processor and a memory, wherein the acquisition information unit, the data storage unit, the resynchronization unit, the first determination unit, the determination synchronization unit and the like are stored in the memory as program units, and the processor executes the program units stored in the memory to realize corresponding functions.
The processor comprises a kernel, and the kernel calls the corresponding program unit from the memory. The kernel can be set to be one or more, and the data synchronization condition is automatically monitored by adjusting the kernel parameters, so that the data synchronization efficiency is improved.
The memory may include volatile memory in a computer readable medium, Random Access Memory (RAM) and/or nonvolatile memory such as Read Only Memory (ROM) or flash memory (flash RAM), and the memory includes at least one memory chip.
An embodiment of the present invention provides a storage medium on which a program is stored, the program implementing the data synchronization method when executed by a processor. Optionally, the device on which the storage medium is located is controlled to execute the data synchronization method when the program runs.
The embodiment of the invention provides a processor, which is used for running a program, wherein the data synchronization method is executed when the program runs.
The embodiment of the invention provides equipment, which comprises a processor, a memory and a program which is stored on the memory and can run on the processor, wherein the processor executes the program and realizes the following steps:
acquiring synchronization result information of data synchronization of a first system from a second system;
storing the synchronization result information into a database; the synchronization result information comprises an item identifier of the data and a synchronization result type of the data corresponding to the item identifier;
and searching a synchronization result type with a failure type in the database, determining an item identifier corresponding to the searched synchronization result type as a failure item identifier, and controlling the first system to resynchronize data corresponding to the failure item identifier from the second system.
Optionally, the synchronization result information further includes a data type of the data corresponding to the synchronization result type, where each data type included in the data corresponding to each item identifier corresponds to the synchronization result type;
the controlling the first system to resynchronize data corresponding to the failed item identification from the second system includes:
and for the data corresponding to the failed item identification, determining the data type corresponding to the synchronization result type with the type being a failure as a failed data type, and controlling the first system to resynchronize the data corresponding to the failed data type from the second system.
Optionally, the synchronization result information further includes a synchronization time of the data corresponding to the item identifier; the processor executes the program and further realizes the following steps:
determining target synchronization result information of the synchronization time in a preset time period in the database;
correspondingly, the searching for the synchronization result type with the failure in the database, and determining the item identifier corresponding to the searched synchronization result type as the failed item identifier include:
and searching the synchronization result type with the type of failure in the target synchronization result information in the database, and identifying the item corresponding to the searched synchronization result type as a failure item identification.
Optionally, the first system runs on at least one server;
before the synchronization result type with a failure is searched in the database and the item mark identification corresponding to the searched synchronization result type is determined as the failure item identification, the processor further realizes the following steps when executing a program:
determining that all servers operated by the first system complete synchronization of data.
Optionally, the obtaining synchronization result information of the first system performing data synchronization from the second system includes:
monitoring a data synchronization process of the first system from the second system for the item identification;
and acquiring synchronization result information aiming at the project identification after the data synchronization aiming at the project identification of the first system is determined to be finished.
The device herein may be a server, a PC, a PAD, a mobile phone, etc.
The present application further provides a computer program product adapted to perform a program for initializing the following method steps when executed on a data processing device:
acquiring synchronization result information of data synchronization of a first system from a second system;
storing the synchronization result information into a database; the synchronization result information comprises an item identifier of the data and a synchronization result type of the data corresponding to the item identifier;
and searching a synchronization result type with a failure type in the database, determining an item identifier corresponding to the searched synchronization result type as a failure item identifier, and controlling the first system to resynchronize data corresponding to the failure item identifier from the second system.
Optionally, the synchronization result information further includes a data type of the data corresponding to the synchronization result type, where each data type included in the data corresponding to each item identifier corresponds to the synchronization result type;
the controlling the first system to resynchronize data corresponding to the failed item identification from the second system includes:
and for the data corresponding to the failed item identification, determining the data type corresponding to the synchronization result type with the type being a failure as a failed data type, and controlling the first system to resynchronize the data corresponding to the failed data type from the second system.
Optionally, the synchronization result information further includes a synchronization time of the data corresponding to the item identifier; there may also be a program of method steps:
determining target synchronization result information of the synchronization time in a preset time period in the database;
correspondingly, the searching for the synchronization result type with the failure in the database, and determining the item identifier corresponding to the searched synchronization result type as the failed item identifier include:
and searching the synchronization result type with the type of failure in the target synchronization result information in the database, and identifying the item corresponding to the searched synchronization result type as a failure item identification.
Optionally, the first system runs on at least one server;
before the synchronization result type with the type of failure is searched in the database and the item mark identification corresponding to the searched synchronization result type is determined as the failure item identification, the following method steps can be further provided:
determining that all servers operated by the first system complete synchronization of data.
Optionally, the obtaining synchronization result information of the first system performing data synchronization from the second system includes:
monitoring a data synchronization process of the first system from the second system for the item identification;
and acquiring synchronization result information aiming at the project identification after the data synchronization aiming at the project identification of the first system is determined to be finished.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
In a typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
The memory may include forms of volatile memory in a computer readable medium, Random Access Memory (RAM) and/or non-volatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). The memory is an example of a computer-readable medium.
Computer-readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, a computer readable medium does not include a transitory computer readable medium such as a modulated data signal and a carrier wave.
It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in the process, method, article, or apparatus that comprises the element.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The above are merely examples of the present application and are not intended to limit the present application. Various modifications and changes may occur to those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.
Claims (8)
1. A method of data synchronization, comprising:
acquiring synchronization result information of data synchronization of a first system from a second system;
storing the synchronization result information into a database; the synchronization result information comprises an item identifier of the data and a synchronization result type of the data corresponding to the item identifier; the synchronization result information also comprises a data type of the data corresponding to the synchronization result type, wherein a plurality of data types contained in the data corresponding to each item identifier correspond to the synchronization result type;
and searching a synchronization result type with a type of failure in the database, determining an item identifier corresponding to the searched synchronization result type as a failure item identifier, determining a data type corresponding to the synchronization result type with the type of failure as a failure data type for data corresponding to the failure item identifier, and controlling the first system to resynchronize at least part of data corresponding to the failure data type from the second system.
2. The method of claim 1, wherein the synchronization result information further includes a synchronization time of the data corresponding to the item identification; the method further comprises the following steps:
determining target synchronization result information of the synchronization time in a preset time period in the database;
correspondingly, the searching for the synchronization result type with the failure in the database, and determining the item identifier corresponding to the searched synchronization result type as the failed item identifier include:
and searching the synchronization result type with the type of failure in the target synchronization result information in the database, and identifying the item corresponding to the searched synchronization result type as a failure item identification.
3. The method of claim 1, wherein the first system runs on at least one server;
before the searching the synchronization result type with the failure type in the database and determining the item mark identification corresponding to the searched synchronization result type as the failure item identification, the method further comprises:
determining that all servers operated by the first system complete synchronization of data.
4. The method of claim 1, wherein the obtaining synchronization result information for data synchronization of the first system from the second system comprises:
monitoring a data synchronization process of the first system from the second system for the item identification;
and acquiring synchronization result information aiming at the project identification after the data synchronization aiming at the project identification of the first system is determined to be finished.
5. A data synchronization apparatus, comprising:
an information acquisition unit, configured to acquire synchronization result information of data synchronization performed by a first system from a second system;
the data storage unit is used for storing the synchronization result information into a database; the synchronization result information comprises an item identifier of the data and a synchronization result type of the data corresponding to the item identifier; the synchronization result information also comprises a data type of the data corresponding to the synchronization result type, wherein a plurality of data types contained in the data corresponding to each item identifier correspond to the synchronization result type;
and the resynchronization unit is used for searching the synchronization result type with the failure in the database, determining the item identification corresponding to the searched synchronization result type as the failed item identification, determining the data type corresponding to the synchronization result type with the failure as the failed data type for the data corresponding to the failed item identification, and controlling the first system to resynchronize at least part of data corresponding to the failed data type from the second system.
6. The apparatus of claim 5, wherein the synchronization result information further comprises a synchronization time of the data corresponding to the item identifier; the device also includes:
a first determining unit, configured to determine, in the database, target synchronization result information of the synchronization time within a preset time period;
the resynchronization unit includes:
the second searching module is used for searching the synchronization result type with the type of failure in the target synchronization result information and identifying the item corresponding to the searched synchronization result type as a failure item identification;
and the second synchronization module is used for controlling the first system to resynchronize the data corresponding to the failed item identifier from the second system.
7. A storage medium, characterized in that the storage medium comprises a stored program, wherein a device on which the storage medium is located is controlled to perform a data synchronization method according to any one of claims 1-4 when the program is run.
8. A processor, characterized in that the processor is configured to run a program, which when running performs a data synchronization method according to any of claims 1-4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710889039.7A CN109561126B (en) | 2017-09-27 | 2017-09-27 | Data synchronization method and device, storage medium and processor |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710889039.7A CN109561126B (en) | 2017-09-27 | 2017-09-27 | Data synchronization method and device, storage medium and processor |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109561126A CN109561126A (en) | 2019-04-02 |
| CN109561126B true CN109561126B (en) | 2022-01-28 |
Family
ID=65863615
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710889039.7A Active CN109561126B (en) | 2017-09-27 | 2017-09-27 | Data synchronization method and device, storage medium and processor |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109561126B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110225119B (en) * | 2019-06-13 | 2022-09-16 | 平安科技（深圳）有限公司 | Data synchronization method and related equipment |
| CN110990405B (en) * | 2019-11-28 | 2024-04-12 | 中国银行股份有限公司 | Data loading method, device, server and storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101673295A (en) * | 2009-08-28 | 2010-03-17 | 金蝶软件（中国）有限公司 | Data synchronization control method, device and single-point logging-in system |
| CN103686300A (en) * | 2013-11-18 | 2014-03-26 | 中兴通讯股份有限公司 | Service guide synchronization method and system |
| CN103973727A (en) * | 2013-01-28 | 2014-08-06 | 深圳市腾讯计算机系统有限公司 | Data synchronizing method and device |
| CN105282253A (en) * | 2015-11-04 | 2016-01-27 | 湖南御家科技有限公司 | Data document synchronization method and system |
| CN105450682A (en) * | 2014-08-08 | 2016-03-30 | 阿里巴巴集团控股有限公司 | Method, device, and system for data synchronous storage and synchronizing data to client |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9338055B2 (en) * | 2013-03-15 | 2016-05-10 | Cisco Technology, Inc. | Virtual router upgrade via graceful restart |
-
2017
- 2017-09-27 CN CN201710889039.7A patent/CN109561126B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101673295A (en) * | 2009-08-28 | 2010-03-17 | 金蝶软件（中国）有限公司 | Data synchronization control method, device and single-point logging-in system |
| CN103973727A (en) * | 2013-01-28 | 2014-08-06 | 深圳市腾讯计算机系统有限公司 | Data synchronizing method and device |
| CN103686300A (en) * | 2013-11-18 | 2014-03-26 | 中兴通讯股份有限公司 | Service guide synchronization method and system |
| CN105450682A (en) * | 2014-08-08 | 2016-03-30 | 阿里巴巴集团控股有限公司 | Method, device, and system for data synchronous storage and synchronizing data to client |
| CN105282253A (en) * | 2015-11-04 | 2016-01-27 | 湖南御家科技有限公司 | Data document synchronization method and system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109561126A (en) | 2019-04-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107040578B (en) | Data synchronization method, device and system | |
| CN106899654B (en) | Sequence value generation method, device and system | |
| CN106897342B (en) | Data verification method and equipment | |
| CN103365758A (en) | Process monitoring method and system in virtualization environment | |
| CN107133165B (en) | Browser compatibility detection method and device | |
| CN108182258B (en) | Distributed data analysis system and method | |
| CN109358898B (en) | Information processing method and device, electronic equipment and storage medium | |
| CN109561126B (en) | Data synchronization method and device, storage medium and processor | |
| CN111767227A (en) | Recording playback test method and device | |
| CN107179965B (en) | Database recovery method and device | |
| CN113918658A (en) | Method and device for recovering data | |
| CN105847320A (en) | Resource obtaining method and device | |
| CN109086094B (en) | Method, system, device and storage medium for self-starting of application service | |
| CN111125087B (en) | Data storage method and device | |
| CN110943876A (en) | URL state detection method, device, equipment and system | |
| CN109189864B (en) | Method, device and equipment for determining data synchronization delay | |
| CN109992501A (en) | The availability inspection of small routine can use small routine list generation method and device | |
| CN109582396B (en) | Task state processing method, device and system and storage medium | |
| CN116414914A (en) | Data synchronization method and device, processor and electronic equipment | |
| CN109561121B (en) | Method and device for monitoring deployment | |
| CN108121733B (en) | Data query method and device | |
| CN114489999A (en) | Method and device for processing pipeline task, processor and electronic equipment | |
| CN104239448A (en) | Time sequence data timestamp obtaining method and device | |
| CN110019296B (en) | Database query script generation method and device, storage medium and processor | |
| CN111198818B (en) | Information acquisition method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information |
Address after: 100080 No. 401, 4th Floor, Haitai Building, 229 North Fourth Ring Road, Haidian District, Beijing Applicant after: Beijing Guoshuang Technology Co.,Ltd. Address before: 100086 Beijing city Haidian District Shuangyushu Area No. 76 Zhichun Road cuigongfandian 8 layer A Applicant before: Beijing Guoshuang Technology Co.,Ltd. |
|
| CB02 | Change of applicant information | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |