CN113254225A

CN113254225A - Data synchronization method and device

Info

Publication number: CN113254225A
Application number: CN202110579363.5A
Authority: CN
Inventors: 王文彬
Original assignee: Beijing Jingdong Century Trading Co Ltd; Beijing Wodong Tianjun Information Technology Co Ltd
Current assignee: Beijing Jingdong Century Trading Co Ltd; Beijing Wodong Tianjun Information Technology Co Ltd
Priority date: 2021-05-26
Filing date: 2021-05-26
Publication date: 2021-08-13

Abstract

The invention discloses a data synchronization method and device, and relates to the technical field of computers. The specific implementation mode of the method comprises the following steps: receiving one or more data synchronization requests, wherein the data synchronization requests indicate a target server and data to be synchronized; distributing one or more corresponding synchronous threads for the data synchronization request according to the target server indicated by the data synchronization request and the performance parameters of the target server; and sending the data to be synchronized to a target server by using the synchronization thread so as to perform data synchronization according to the data to be synchronized. The implementation method can improve the efficiency of data synchronization, reasonably distribute the synchronous thread resources and ensure the online and offline data consistency.

Description

Data synchronization method and device

Technical Field

The present invention relates to the field of computer technologies, and in particular, to a data synchronization method and apparatus.

Background

Network consumption refers to enjoying various services by taking the internet as a tool means, and is usually realized by assistance of a terminal APP and the like, and with the development of computer technology, the network consumption becomes a part of daily life.

The terminal APP can generally provide service lists of a plurality of brands, a user can enjoy brand exclusive service after registering as a brand member, and the online APP service provider and the offline brand service provider manage member data through respective CRM systems and perform interactive data synchronization; the CRM (customer Relationship management) system is a customer Relationship management system.

The existing online CRM and offline CRM data synchronization is generally transmitted through a gateway, and because the gateway layer resources are limited, when one brand service has poor performance or is unavailable, the gateway resources are occupied for a long time, so that other synchronization services cannot be performed, even the gateway layer is broken down, and the online and offline data consistency cannot be ensured.

Disclosure of Invention

In view of this, embodiments of the present invention provide a data synchronization method and apparatus, which can determine a synchronization thread of a corresponding data processing request according to performance of a brand service, and further perform resource isolation on an online synchronization request, improve efficiency of data synchronization, reasonably allocate synchronization thread resources, and ensure online and offline data consistency.

The technical scheme of the invention can realize the isolation and dynamic adjustment of request resources by judging and scheduling the data synchronization request by the client under the premise of not depending on a gateway layer and an offline server, and further realize flow control and flow shaping by matching with a flow limiting measure. The consistency of the online system data and the offline service data can be ensured to the maximum extent, and meanwhile, the synchronization performance is improved, and the timely response of the offline service is ensured.

To achieve the above object, according to an aspect of an embodiment of the present invention, there is provided a data synchronization method including:

receiving one or more data synchronization requests, wherein the data synchronization requests indicate a target server and data to be synchronized;

distributing one or more corresponding synchronous threads to the data synchronization request according to the target server indicated by the data synchronization request and the performance parameters of the target server;

and sending the data to be synchronized to the target server by using the synchronization thread so as to perform data synchronization according to the data to be synchronized.

Optionally, the method further comprises:

grouping the one or more data synchronization requests according to the performance parameters of the target server;

and determining the execution priority of the data synchronization request according to the grouping result of the data synchronization request, and distributing the synchronization thread to the data synchronization request according to the execution priority.

Optionally, the determining, according to the grouping result of the data synchronization request, the execution priority of the data synchronization request, and allocating the synchronization thread to the data synchronization request according to the execution priority includes:

determining a first number of the data synchronization requests corresponding to each execution priority according to the grouping result;

and determining the synchronous threads corresponding to the data synchronous requests corresponding to different execution priorities according to the first number, the second number of the one or more synchronous threads and the states of the synchronous threads.

Optionally, for a first data synchronization request with the execution priority higher than a preset first threshold, determining the execution priority of the data synchronization request according to a grouping result of the data synchronization request, and allocating the synchronization thread to the data synchronization request according to the execution priority, includes:

assigning a synchronization thread of the one or more synchronization threads that is in an idle state to the first data synchronization request.

Optionally, for a second data synchronization request with the execution priority lower than a preset second threshold, determining the execution priority of the data synchronization request according to the grouping result of the data synchronization request, and allocating the synchronization thread to the data synchronization request according to the execution priority, includes:

determining whether the first data synchronization request which is not executed exists, and if not, allocating a synchronization thread which is in an idle state in the one or more synchronization threads to the second data synchronization request.

Optionally, the performance parameters include: the duration of the response and/or the result of the response,

determining a server with an abnormal response result and/or a server with a response time length larger than a preset response time length threshold, dividing a data synchronization request corresponding to the server into first groups, and determining that the execution priority of the first groups is lower than a preset second threshold;

and/or the presence of a gas in the gas,

and determining the response result as a normal server side with the response time length not greater than a preset response time length threshold, dividing the data synchronization request corresponding to the server side into a second group, and determining that the execution priority of the second group is higher than a preset first threshold.

Optionally, the sending, by using the synchronization thread, the data to be synchronized to the target server includes:

and according to the type of the data to be synchronized, the synchronization thread is utilized to send the data to be synchronized to a target server corresponding to the data synchronization request in real time or asynchronously.

Optionally, after sending the data to be synchronized to the target server, the method further includes:

and determining whether the data synchronization is successful according to the response of the target server to the data to be synchronized, and if not, re-executing the data synchronization request corresponding to the data to be synchronized.

Optionally, when the number of the data synchronization requests received in a preset time period is greater than a preset number threshold, the method further includes:

intercepting part of data synchronization requests in the plurality of data synchronization requests according to the number of the received data synchronization requests and the number of the one or more synchronization threads in the preset time period, and distributing the synchronization threads to the data synchronization requests which are not intercepted.

According to still another aspect of an embodiment of the present invention, there is provided a data synchronization apparatus including:

the system comprises a receiving module, a synchronization module and a synchronization module, wherein the receiving module is used for receiving one or more data synchronization requests, and the data synchronization requests indicate a target server and data to be synchronized;

the data processing module is used for distributing one or more corresponding synchronous threads to the data synchronization request according to the target server indicated by the data synchronization request and the performance parameters of the target server;

and the synchronization module is used for sending the data to be synchronized to the target server by using the synchronization thread so as to perform data synchronization according to the data to be synchronized.

According to another aspect of the embodiments of the present invention, there is provided a data synchronization electronic device, including:

one or more processors;

a storage device for storing one or more programs,

when the one or more programs are executed by the one or more processors, the one or more processors implement the data synchronization method provided by the present invention.

According to still another aspect of embodiments of the present invention, there is provided a computer-readable medium on which a computer program is stored, the program, when executed by a processor, implementing the data synchronization method provided by the present invention.

One embodiment of the above invention has the following advantages or benefits: because the technical means of judging the data synchronization request at the client and allocating the request resource for the client according to the performance parameters of the server is adopted, the technical problems that the existing online and offline data synchronization service is poor and the online and offline data consistency cannot be ensured are solved, and the technical effects of determining the synchronization thread of the corresponding data processing request according to the performance of the brand service, further performing resource isolation on the online synchronization request, improving the data synchronization efficiency, reasonably allocating the synchronization thread resource and ensuring the online and offline data consistency are achieved.

Further effects of the above-mentioned non-conventional alternatives will be described below in connection with the embodiments.

Drawings

The drawings are included to provide a better understanding of the invention and are not to be construed as unduly limiting the invention. Wherein:

FIG. 1 illustrates an exemplary system architecture diagram of a data synchronization method or data synchronization apparatus suitable for application to embodiments of the present invention;

FIG. 2 is a schematic diagram of a main flow of a data synchronization method according to an embodiment of the present invention;

FIG. 3 is a schematic diagram of the main flow of a method of determining a synchronous thread of a data synchronization request according to one embodiment of the invention;

FIG. 4 is a schematic diagram of the main flow of a method of determining a synchronous thread of a data synchronization request according to yet another embodiment of the invention;

FIG. 5 is a schematic diagram of the main modules of a data synchronization apparatus according to an embodiment of the present invention;

fig. 6 is a schematic block diagram of a computer system suitable for use in implementing a terminal device or server of an embodiment of the invention.

Detailed Description

Exemplary embodiments of the present invention are described below with reference to the accompanying drawings, in which various details of embodiments of the invention are included to assist understanding, and which are to be considered as merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the invention. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.

Fig. 1 is a diagram illustrating an exemplary system architecture of a data synchronization method or a data synchronization apparatus suitable for being applied to an embodiment of the present invention, and as shown in fig. 1, the exemplary system architecture of the data synchronization method or the data synchronization apparatus according to the embodiment of the present invention includes:

as shown in fig. 1, the system architecture 100 may include

terminal devices

101, 102, 103, a network 104, and a server 105. The network 104 serves as a medium for providing communication links between the

terminal devices

101, 102, 103 and the server 105. Network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, to name a few.

The user may use the

terminal devices

101, 102, 103 to interact with the server 105 over the network 104 to synchronize data and the like. The

terminal devices

101, 102, 103 may have various communication client applications installed thereon, such as a shopping application, a web browser application, a search application, an instant messaging tool, a mailbox client, social platform software, and the like.

The

terminal devices

101, 102, 103 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop portable computers, desktop computers, and the like.

The server 105 may be a server that provides various services, such as a background management server that supports shopping websites browsed by users using the

terminal devices

101, 102, 103. The background management server may analyze the received member point data synchronization request and the like, and feed back a processing result (e.g., successful synchronization) to the

terminal apparatuses

101, 102, 103.

It should be noted that the data synchronization method provided by the embodiment of the present invention is generally executed by the

terminal devices

101, 102, and 103, and accordingly, the data synchronization apparatus is generally disposed in the

terminal devices

101, 102, and 103.

It should be understood that the number of terminal devices, networks, and servers in fig. 1 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.

Fig. 2 is a schematic diagram of a main flow of a data synchronization method according to an embodiment of the present invention, and as shown in fig. 2, the data synchronization method of the present invention includes:

step S201, receiving one or more data synchronization requests, where the data synchronization requests indicate a target server and data to be synchronized.

When a user enjoys brand service through a terminal APP, in order to effectively manage user data by an offline brand service end and provide better service for the user to improve user experience, the user data received by an online client needs to be synchronized to the offline brand service end in time.

In an embodiment of the present invention, an online client receives one or more data synchronization requests indicating a target server and data to be synchronized. The data synchronization request may be triggered by the user within the terminal APP.

In the embodiment of the present invention, the data to be synchronized may be membership data, including membership card information, point information, and the like. For example, the membership card information includes data such as user ID, brand ID, contact phone, birthday, and sex; the point information comprises data of point value, point increment, point decrement, point exchange, point modification and the like. Interfaces such as card opening, inputting and modifying are arranged in the terminal APP, a user operates by triggering the corresponding interfaces, and the client sends a data synchronization request to the server according to triggering responding to the user.

In an embodiment of the present invention, the data synchronization request may further indicate a synchronization time deadline of the data to be synchronized, indicating that the synchronization of the data to be synchronized must be completed before the deadline.

Step S202, according to the target server indicated by the data synchronization request and the performance parameters of the target server, allocating one or more corresponding synchronization threads to the data synchronization request.

In the embodiment of the invention, the client interacts with the server through the gateway, the data synchronization request is sent to the server through the gateway, and the gateway is provided with a plurality of synchronization threads for data transmission. And allocating one or more corresponding synchronous threads for the data synchronization request according to the performance parameters of the target server indicated by the data synchronization request, the number of the data synchronization requests, the number of synchronous threads, the state (such as whether idle) of the synchronous threads and the like.

In the embodiment of the present invention, the performance parameter of the server includes a response duration of the server and/or a response result. The performance parameter of the server can be determined according to the historical response data and/or the real-time response data of the server. For example, according to the time taken by the server to respond to the client single point increment data synchronization request, the response time of the server to the client single point increment data synchronization request is determined to be 100 ms. For another example, whether the server side responds to the client-side single point increment data synchronization request is judged, and if yes, the response result of the server side to the client-side single point increment data synchronization request is determined to be normal; if not, determining that the response result is abnormal. For another example, whether the server responds to the client single point increment data synchronization request within a certain preset time (for example, 10s) is judged, and if the server responds to the client within a preset response time threshold, it is determined that the response result of the server to the client single point increment data synchronization request is normal; and if the server side does not respond to the client side within the preset response time threshold value, determining that the response result is abnormal.

In the embodiment of the invention, when the performance parameters of the server are determined according to the historical response data of the server, the performance of the interface of the server can be monitored in real time, so that the performance parameters of the server can be adjusted in real time.

As shown in fig. 3, an embodiment of the present invention discloses a method for determining a synchronous thread of a data synchronization request, including:

and allocating one or more corresponding synchronous threads to the data synchronization request in proportion according to the performance parameter of the target server indicated by the data synchronization request. For example, the performance of the server, characterized by the performance parameters: good, medium and poor according to a ratio of 0.5: 0.3: a ratio of 0.2 allocates a synchronization thread for the data synchronization request.

Step S301, grouping one or more data synchronization requests according to the performance parameters of the target server.

In the embodiment of the invention, the data synchronization request corresponding to the target server with the response result of abnormal and/or the response time length longer than the preset response time length threshold is divided into a first group;

and/or the presence of a gas in the gas,

and dividing the data synchronization request corresponding to the target server side with the response result of normal and the response time length not greater than the preset response time length threshold into a second group.

In the embodiment of the present invention, the packet of the data synchronization request may also be determined according to the type of the data to be processed. For example, if the data to be processed is real-time synchronous data, dividing the corresponding data synchronous request into a second group; and if the data to be processed is non-real-time synchronous data, grouping the data synchronous requests according to the performance parameters of the target server within the synchronous time limit indicated by the corresponding data synchronous requests.

In the embodiment of the present invention, the packet is not limited to the first packet and the second packet, and the granularity of the packet may be set according to the need, for example, divided into a fast packet, a slow packet, and a retry packet. As another example, a fast packet, a general packet, a slow packet, a retry packet, and the like are divided. As another example, into a first packet, a second packet, … …, an nth packet, and a retry packet; the specific value of N may be determined according to different intervals of the response time length.

Step S302, determining the execution priority of the data synchronization request according to the grouping result of the data synchronization request.

In the embodiment of the invention, according to the grouping result of the data synchronization request, the execution priority of the data synchronization request corresponding to the first grouping is determined to be lower than a preset second threshold;

and/or the presence of a gas in the gas,

the execution priority of the data synchronization request corresponding to the second packet is determined to be higher than a preset first threshold.

In the embodiment of the present invention, the first threshold may be greater than or equal to the second threshold, and accordingly, the execution priority of the data synchronization request corresponding to the second packet is the first priority, and the execution priority of the data synchronization request corresponding to the first packet is the second priority.

Step S303, determining a first number of data synchronization requests corresponding to each execution priority.

In the embodiment of the invention, a first number of data synchronization requests corresponding to a first priority is determined, and/or a first number of data synchronization requests corresponding to a second priority is determined; wherein, the first number of the data synchronization requests corresponding to each execution priority may be the same or different.

Step S304, determining, according to the first number of data synchronization requests corresponding to each execution priority, the second number of one or more synchronization threads, and the state of the synchronization threads, synchronization threads corresponding to the data synchronization requests corresponding to different execution priorities.

In the embodiment of the invention, the data synchronization request can be allocated with the synchronization thread according to the execution priority, and the higher the execution priority is, the greater the proportion of the number of the synchronization threads allocated to the data synchronization request is. For example, the ratio of the number of the synchronization threads allocated to the data synchronization requests corresponding to the first priority and the second priority is 0.6: 0.2.

The existing online and offline data synchronization is directly transmitted through a gateway, resource isolation among different data synchronization requests is not carried out, if one or more brand service terminals have poor performance or are unavailable, the data transmission of the data synchronization requests can always occupy gateway resources, the gateway resources are limited, namely the data synchronization requests which can be processed are limited, so that the request amount processed by the gateway is limited, synchronization threads cannot be distributed to other data synchronization requests, and the data synchronization is blocked. And the off-line server can not receive the data processing request for a long time and respond in time, so that the performance and the data synchronization efficiency of the brand server are greatly influenced.

By judging the data synchronization request at the client, a larger proportion of synchronization threads are allocated to the data synchronization request with good performance, so that the data synchronization request is subjected to resource isolation. The method and the device realize the purposes of providing sufficient threads for the requests with good performance, controlling the threads distributed to the requests with poor performance, and synchronizing the requests with poor performance to the maximum extent while ensuring that good offline service is not influenced. The implementation method can improve the efficiency of data synchronization, reasonably distribute the synchronous thread resources and ensure the online and offline data consistency.

By identifying the data synchronization request at the client, a larger proportion of synchronization threads are allocated to the data synchronization request with good performance, the request with poor performance can be controlled, the request failure is directly returned at the client, and further more gateway thread resources are reserved for the server with normal performance, so that the server can respond to the data synchronization request in time, and the data synchronization efficiency is improved.

In the embodiment of the invention, after rejecting part of the requests of the data synchronization requests corresponding to the server with poor performance, the data synchronization requests can be alarmed in real time to inform the offline server of upgrading, the performance parameters of the data synchronization requests are determined again according to the upgraded server, and the data synchronization requests corresponding to the server are regrouped.

In the embodiment of the present invention, the number of the synchronous threads allocated to different execution priorities may be dynamically adjusted according to the first number of the data synchronization requests, the second number of the one or more synchronous threads, and the state of the synchronous threads corresponding to each execution priority. For example, the first number of data synchronization requests corresponding to the first priority and the second priority is 1:100, the second number of synchronization threads is 20, 16 of the synchronization threads are in an idle state, 1 synchronization thread is allocated to the data synchronization request corresponding to the first priority, and the remaining 15 synchronization threads are allocated to the data synchronization request corresponding to the second priority. For another example, the first number of the data synchronization requests corresponding to the first priority and the second priority is 100:80, and the second number of the synchronization threads is 20, where 10 of the synchronization threads are in an idle state, the synchronization thread is preferentially allocated to the first priority, 6 synchronization threads may be allocated to the data synchronization request corresponding to the first priority, and the remaining 4 synchronization threads are allocated to the data synchronization request corresponding to the second priority. For another example, if the first number of the data synchronization requests corresponding to the first priority and the second priority is 100:0, and the second number of the synchronization threads is 20, and 15 of the synchronization threads are in an idle state, 15 synchronization threads are allocated to the data synchronization requests corresponding to the first priority. For example, if the first number of data synchronization requests corresponding to the first priority and the second priority is 0:80, and the second number of synchronization threads is 20, where 20 of the synchronization threads are in an idle state, 20 synchronization threads are allocated to the data synchronization requests corresponding to the second priority.

The data synchronization requests are grouped at the client according to the performance of different service ends, and the resources allocated to the data synchronization requests are dynamically adjusted, so that the resource isolation of the service ends of different brands can be realized. The client limits the current of the brand with poor performance by controlling the request resource, can realize the accurate control of the request resource of off-line brand service, and reduces the pressure brought to the gateway and off-line brand service by the instant overlarge request amount.

In the embodiment of the invention, a gateway interface is requested by using a normal request mode aiming at the corresponding data synchronization request with high execution priority, so that the request can pass by according to the number of the requests, the request speed is fastest, and the gateway interface is generally used for allocating request resources to normal online brand services. Correspondingly, the consumption rate of the data synchronization request is high, data backlog generally does not exist, and all requests can be requested at the first time.

The consumption rate is controlled by the asynchronous MQ for corresponding data synchronization requests of low execution priority. After the sending, the consumption end uses a pulling mode to consume, the pulling rate can be controlled, the consumption speed is adjusted through dynamic configuration and is generally used for services with poor performance of off-line brands, so the consumption speed is controlled as much as possible, and excessive resources are not occupied when the off-line services are requested.

Further, the consumption rate may also be controlled by the asynchronous MQ for corresponding data synchronization requests with high execution priority. For example, the information of the membership card opened by the user, the data to be synchronized, which has low real-time requirement, can be processed asynchronously.

In the embodiment of the invention, after the grouping result of the data synchronization request is determined, the consumption rate is controlled by the asynchronous MQ, and further the distribution of the synchronous threads is realized.

As shown in fig. 4, an embodiment of the present invention discloses a method for determining a synchronous thread of a data synchronization request, including:

and according to the performance parameters of the target server indicated by the data synchronization request, preferentially distributing the synchronization thread for the corresponding data synchronization request with good performance, and after determining that the corresponding data synchronization request with good performance is processed, distributing the synchronization thread for the corresponding data synchronization request with poor performance. For example, the performance of the server, characterized by the performance parameters: and the data synchronization request with the excellent performance is distributed with the synchronous thread preferentially, the corresponding data synchronization request with the excellent performance is distributed with the synchronous thread after the corresponding data synchronization request with the excellent performance is determined to be processed, and the corresponding data synchronization request with the poor performance is distributed with the synchronous thread after the corresponding data synchronization request with the excellent performance is determined to be processed.

Step S401, grouping one or more data synchronization requests according to the performance parameter of the target server.

In the embodiment of the present invention, the grouping method of the data synchronization request is the same as step S301.

Step S402, determining the execution priority of the data synchronization request according to the grouping result of the data synchronization request.

In the embodiment of the present invention, the method for determining the execution priority of the data synchronization request is the same as that of step S302.

In step S403, for a first data synchronization request with an execution priority higher than a preset first threshold, a synchronization thread in an idle state among the one or more synchronization threads is allocated to the first data synchronization request.

Step S404, determining whether there is a first data synchronization request that is not executed for a second data synchronization request whose execution priority is lower than a preset second threshold, and if not, allocating a synchronization thread in an idle state among the one or more synchronization threads to the second data synchronization request.

In the embodiment of the invention, if the unexecuted first data synchronization request exists, the idle-state synchronization thread is continuously allocated to the first data synchronization request until the first data synchronization request is executed, and after the unexecuted first data synchronization request does not exist, the idle-state synchronization thread in one or more synchronization threads is allocated to the second data synchronization request.

Step S203, sending the data to be synchronized to the target server by using the synchronization thread, so as to perform data synchronization according to the data to be synchronized.

In the embodiment of the invention, after the synchronization thread corresponding to the data synchronization request is determined, the synchronization thread is utilized to send the data to be synchronized to the target server, so as to perform data synchronization according to the data to be synchronized and receive the synchronization result returned by the target server.

In the embodiment of the invention, whether the data synchronization is successful is determined according to the response (including response time length and/or response result) of the target server to the data to be synchronized, if so, the performance parameter corresponding to the synchronization result is stored so as to facilitate the real-time monitoring of the server; if not, the synchronous thread is determined again for the data synchronous request corresponding to the server side with the synchronization failure to retry, and the data synchronous request corresponding to the data to be synchronized is re-executed.

Further, when one or more corresponding synchronization threads are allocated to the data synchronization request according to the target server indicated by the data synchronization request and the performance parameters of the target server, the one or more synchronization threads are reserved in advance as synchronization retry threads for retrying the data synchronization request corresponding to the server with the synchronization failure. The retry means setting a mechanism of re-request for the data synchronization request with synchronization failure, so as to ensure the maximum synchronization success rate.

When the existing data synchronization request is retried under the condition of response overtime in an online manner, the data synchronization request which fails in synchronization usually occupies resources together with other new data synchronization requests, and if the retry mechanism is not complete and cannot be synchronized in time, data backlog can be caused, so that the resources are occupied for a long time, the synchronization performance is increasingly poor, and the backlog is increasingly serious.

One or more synchronous threads are reserved in advance for the retried data synchronization request and are used for backlogging the retried pushing of the failed data, and the online and offline data consistency can be further guaranteed under the condition that the normal synchronous threads are not influenced.

In the embodiment of the invention, the data to be synchronized can be sent to the target server corresponding to the data synchronization request in real time or asynchronously by using the synchronization thread according to the type of the data to be synchronized. For example, if the data to be processed is real-time synchronous data, the synchronous thread is used for sending the data to be synchronized to a target server corresponding to the data synchronization request in real time; and if the data to be processed is non-real-time synchronous data, the synchronous thread is utilized to asynchronously send the data to be synchronized to a target server corresponding to the data synchronization request.

In the embodiment of the present invention, when the number of the data synchronization requests received in the preset time period is greater than the preset number threshold, a part of the data synchronization requests in the plurality of data synchronization requests may be intercepted according to the number of the data synchronization requests received in the preset time period and the number of one or more synchronization threads, and a synchronization thread may be allocated to a data synchronization request that is not intercepted. By partially intercepting a large amount of data synchronization requests in a short time, the pressure of a gateway and an offline server can be reduced, the synchronization efficiency is further improved, and the consistency of online and offline data is ensured.

In the embodiment of the invention, one or more data synchronization requests are received, wherein the data synchronization requests indicate a target server and data to be synchronized; distributing one or more corresponding synchronous threads to the data synchronization request according to the target server indicated by the data synchronization request and the performance parameters of the target server; and sending the data to be synchronized to the target server by using the synchronization thread so as to perform the steps of data synchronization and the like according to the data to be synchronized, thereby improving the efficiency of data synchronization, reasonably distributing synchronization thread resources and ensuring the consistency of online and offline data.

The online and offline synchronization system performs resource isolation by grouping and then realizes flow control by using current limiting; the control of the request amount is realized through different consumption rates among the packets; and then through the retry packet for retrying the requested resource allocation. Therefore, when the performance of the off-line brand service is in a problem, the resource occupation is reduced, and the resource control and scheduling of the data synchronization request can be realized through the client without depending on the off-line brand service and the gateway.

Fig. 5 is a schematic diagram of main modules of a data synchronization apparatus according to an embodiment of the present invention, and as shown in fig. 5, a data synchronization apparatus 500 of the present invention includes:

a receiving module 501, configured to receive one or more data synchronization requests, where the data synchronization requests indicate a target server and data to be synchronized.

In the embodiment of the present invention, the receiving module 501 of the online client receives one or more data synchronization requests, where the data synchronization requests indicate a target server and data to be synchronized. The data synchronization request may be triggered by the user within the terminal APP.

In the embodiment of the present invention, the data to be synchronized may be membership data, including membership card information, point information, and the like. For example, the membership card information includes data such as user ID, brand ID, contact phone, birthday, and sex; the point information comprises data of point value, point increment, point decrement, point exchange, point modification and the like.

A data processing module 502, configured to allocate one or more corresponding synchronization threads to the data synchronization request according to the target server indicated by the data synchronization request and the performance parameter of the target server.

In this embodiment of the present invention, after the receiving module 501 receives the data synchronization request, the data processing module 502 allocates one or more corresponding synchronization threads to the data synchronization request according to the performance parameter of the target server indicated by the data synchronization request, the number of the data synchronization requests, the number of the synchronization threads, the state (for example, whether the synchronization threads are idle) of the synchronization threads, and the like.

In the embodiment of the present invention, the performance parameter of the server includes a response duration of the server and/or a response result. The performance parameter of the server can be determined according to the historical response data and/or the real-time response data of the server.

A synchronization module 503, configured to send the data to be synchronized to the target server by using the synchronization thread, so as to perform data synchronization according to the data to be synchronized.

In this embodiment of the present invention, after the data processing module 502 determines a synchronization thread corresponding to the data synchronization request, the synchronization module 503 sends the data to be synchronized to the target server by using the synchronization thread, so as to perform data synchronization according to the data to be synchronized, and receive a synchronization result returned by the target server.

In this embodiment of the present invention, the system further includes a retry module 504, configured to determine whether data synchronization is successful according to a response (including a response duration and/or a response result) of the target server to the data to be synchronized, and if not, determine a retry of the synchronization thread again for the data synchronization request corresponding to the server that fails to synchronize, and re-execute the data synchronization request corresponding to the data to be synchronized.

In this embodiment of the present invention, the current limiting module 505 is further included, configured to intercept, when the number of the data synchronization requests received in the preset time period is greater than a preset number threshold, a part of the data synchronization requests in the multiple data synchronization requests according to the number of the data synchronization requests received in the preset time period and the number of one or more synchronization threads, and allocate, by using the data processing module 502, a synchronization thread to the data synchronization request that is not intercepted.

In the embodiment of the invention, through the modules such as the receiving module, the data processing module, the synchronization module, the retry module, the current limiting module and the like, the efficiency of data synchronization can be improved, the synchronous thread resources can be reasonably distributed, and the online and offline data consistency can be ensured.

Fig. 6 is a schematic structural diagram of a computer system suitable for implementing a terminal device according to an embodiment of the present invention, and as shown in fig. 6, the computer system 600 of the terminal device according to the embodiment of the present invention includes:

a Central Processing Unit (CPU)601, which can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM)602 or a program loaded from a storage section 608 into a Random Access Memory (RAM) 603. In the RAM603, various programs and data necessary for the operation of the system 600 are also stored. The CPU601, ROM602, and RAM603 are connected to each other via a bus 604. An input/output (I/O) interface 605 is also connected to bus 604.

The following components are connected to the I/O interface 605: an input portion 606 including a keyboard, a mouse, and the like; an output portion 607 including a display such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, and a speaker; a storage section 608 including a hard disk and the like; and a communication section 609 including a network interface card such as a LAN card, a modem, or the like. The communication section 609 performs communication processing via a network such as the internet. The driver 610 is also connected to the I/O interface 605 as needed. A removable medium 611 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 610 as necessary, so that a computer program read out therefrom is mounted in the storage section 608 as necessary.

In particular, according to the embodiments of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method illustrated in the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network through the communication section 609, and/or installed from the removable medium 611. The computer program performs the above-described functions defined in the system of the present invention when executed by the Central Processing Unit (CPU) 601.

It should be noted that the computer readable medium shown in the present invention can be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present invention, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In the present invention, however, a computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.

The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

The modules described in the embodiments of the present invention may be implemented by software or hardware. The described modules may also be provided in a processor, which may be described as: a processor includes a receiving module, a data processing module, and a synchronization module. The names of these modules do not form a limitation on the modules themselves in some cases, for example, a data processing module may also be described as a "module that determines a synchronization thread of data to be synchronized according to a target server corresponding to a data synchronization request".

As another aspect, the present invention also provides a computer-readable medium that may be contained in the apparatus described in the above embodiments; or may be separate and not incorporated into the device. The computer readable medium carries one or more programs which, when executed by a device, cause the device to comprise: receiving one or more data synchronization requests, wherein the data synchronization requests indicate a target server and data to be synchronized; distributing one or more corresponding synchronous threads to the data synchronization request according to the target server indicated by the data synchronization request and the performance parameters of the target server; and sending the data to be synchronized to the target server by using the synchronization thread so as to perform data synchronization according to the data to be synchronized.

According to the technical scheme of the embodiment of the invention, the synchronous thread of the corresponding data processing request can be determined according to the performance of the brand service, so that the resource isolation is carried out on the on-line synchronous request, the data synchronization efficiency is improved, the synchronous thread resource is reasonably distributed, and the on-line and off-line data consistency is ensured.

Furthermore, the data synchronization request is judged at the client, so that the synchronization thread is reasonably distributed, and additional development/coupling of service logic at a gateway layer is not needed for resource isolation, so that the gateway layer has pure functions.

Furthermore, by means of the current limiting measure of the data synchronization request and the judgment of the data synchronization request at the client, the synchronization thread is reasonably distributed, the dynamic adjustment of gateway resources can be realized, the data synchronization efficiency is greatly optimized, and the on-line and off-line data consistency guarantee is promoted.

The above-described embodiments should not be construed as limiting the scope of the invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions can occur, depending on design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method of data synchronization, comprising:

2. The method of claim 1, further comprising:

3. The method according to claim 2, wherein the determining an execution priority of the data synchronization request according to the grouping result of the data synchronization request and allocating the synchronization thread to the data synchronization request according to the execution priority comprises:

4. The method according to claim 2, wherein for a first data synchronization request with the execution priority higher than a preset first threshold, the determining the execution priority of the data synchronization request according to the grouping result of the data synchronization request and allocating the synchronization thread to the data synchronization request according to the execution priority comprises:

5. The method according to claim 4, wherein for a second data synchronization request with the execution priority lower than a preset second threshold, the determining the execution priority of the data synchronization request according to the grouping result of the data synchronization request and allocating the synchronization thread to the data synchronization request according to the execution priority comprises:

6. The method of claim 5, wherein the performance parameters comprise: the duration of the response and/or the result of the response,

and/or the presence of a gas in the gas,

7. The method according to claim 1, wherein the sending the data to be synchronized to the target server by using the synchronization thread comprises:

8. The method according to claim 1, further comprising, after sending the data to be synchronized to the target server:

9. The method of claim 1, further comprising, when the number of data synchronization requests received within a preset time period is greater than a preset number threshold:

10. A data synchronization apparatus, comprising:

11. A data synchronization electronic device, comprising:

one or more processors;

a storage device for storing one or more programs,

when executed by the one or more processors, cause the one or more processors to implement the method of any one of claims 1-9.

12. A computer-readable medium, on which a computer program is stored, which, when being executed by a processor, carries out the method according to any one of claims 1-9.