CN110795495A

CN110795495A - Data processing method and device, electronic equipment and computer readable medium

Info

Publication number: CN110795495A
Application number: CN201810783276.XA
Authority: CN
Inventors: 段开元
Original assignee: Beijing Jingdong Century Trading Co Ltd; Beijing Jingdong Shangke Information Technology Co Ltd
Current assignee: Beijing Jingdong Century Trading Co Ltd; Beijing Jingdong Shangke Information Technology Co Ltd
Priority date: 2018-07-17
Filing date: 2018-07-17
Publication date: 2020-02-14

Abstract

The disclosure provides a data processing method, a data processing device, electronic equipment and a computer readable medium, and belongs to the technical field of internet. The method comprises the following steps: the data center acquires full data or incremental data from the database; synchronizing the full data and the corresponding version number to the cloud service so that the cloud service can store the full data and the version number; and synchronizing the incremental data and the version number corresponding to the incremental data to the client through a remote procedure call protocol (RPC) service. The client side only needs to read the incremental data from the data center by means of the version number of the existing data, the phenomenon that the database link is occupied due to the fact that the number of the client sides is too large or the quantity of the requested data is too large can be prevented, only the data center accesses the database and obtains the incremental data in the whole process, the client side does not directly access the database any more, when the access quantity of the client side is too large, effective and accurate data service can be provided through the capacity expansion data center, and a complex and expensive data capacity expansion means is avoided.

Description

Data processing method and device, electronic equipment and computer readable medium

Technical Field

The present disclosure generally relates to the field of internet technologies, and in particular, to a data processing method, an apparatus, an electronic device, and a computer-readable medium.

Background

At present, with the increase of users, the concurrency of service requests for accessing the database is larger and larger, and the requirement on the access speed is also higher and higher, and no matter the database is accessed through the service or the database is accessed through the uniform cache middleware, the network overhead cannot be avoided, so that the network overhead is increased.

In order to solve the problem of network overhead, the required data can be directly stored in the memory of the service, so that the requirement of access speed can be well met, and the concurrent processing speed can be improved. However, storing the required data directly in the memory of the service inevitably causes various problems such as consistency problems and the like.

Therefore, there is still a need for improvement in the prior art solutions.

The above information disclosed in this background section is only for enhancement of understanding of the background of the disclosure and therefore it may contain information that does not constitute prior art that is already known to a person of ordinary skill in the art.

Disclosure of Invention

The present disclosure provides a data processing method, an apparatus, an electronic device and a computer readable medium, which solve the above technical problems.

Additional features and advantages of the disclosure will be set forth in the detailed description which follows, or in part will be obvious from the description, or may be learned by practice of the disclosure.

According to an aspect of the present disclosure, there is provided a data processing method including:

the data center acquires full data or incremental data from a database;

synchronizing the full data and the corresponding version number to a cloud service so that the cloud service stores the full data and the version number;

and synchronizing the incremental data and the version number corresponding to the incremental data to a client through a remote procedure call protocol (RPC) service.

In one embodiment of the present disclosure, the data center obtaining the full amount data or the incremental data from the database includes:

judging whether the version number exists in the data center or not, and obtaining a judgment result as existence or nonexistence;

if the judgment result is that the data does not exist, acquiring the full data from the database for the first time, and storing the full data and the corresponding version number in a memory of a data center;

if the judgment result is that the incremental data exists, acquiring corresponding incremental data from the database according to the version number, and storing the incremental data in a memory of a data center;

wherein the version number is a timestamp of the retrieval of the full or incremental data from the database.

In an embodiment of the present disclosure, after the data center obtains the full amount of data from the database, the data center further includes:

storing the full data stored in the memory of the data center in combination with the corresponding version number into a local file of the data center;

and checking the full data, and synchronizing the full data and the corresponding version number to the cloud service if the full data passes the checking.

In an embodiment of the present disclosure, after the data center obtains the incremental data from the database, the method further includes:

storing the incremental data stored in the memory of the data center into a local file of the data center;

and combining the original full data stored in the local file of the data center with the incremental data and the corresponding version number to form new full data, and storing the new full data into the local file of the data center.

In one embodiment of the disclosure, synchronizing the full amount of data and the version number to a cloud service comprises:

and the data center synchronizes the first acquired full data and the corresponding version number or the new full data and the corresponding version number to the cloud service.

In an embodiment of the present disclosure, after synchronizing the incremental data and the version number corresponding to the incremental data to a client through a remote procedure call protocol (RPC) service, the method further includes:

the client acquires the full data and the corresponding version number from the cloud service;

verifying the full data and the full data stored in a local file of the client, and acquiring new full data according to a verification result;

and storing the new full data and the corresponding version number in a local file of the client.

In one embodiment of the present disclosure, further comprising:

if the client is restarted, after the client is restarted, trying to load the full data and the corresponding version number stored in the local file of the client; and if the loading fails, the client acquires the full data and the corresponding version number from the cloud service, acquires the incremental data from the data center through the RPS service, and stores the incremental data in the memory of the client.

According to still another aspect of the present disclosure, there is provided a data processing apparatus including:

the data acquisition module is configured to acquire full data or incremental data from a database by the data center;

a full synchronization module configured to synchronize the full data and the corresponding version number to a cloud service so that the cloud service stores the full data and the version number;

the increment synchronization module is configured to synchronize the increment data and the version number corresponding to the increment data to a client through a remote procedure call protocol (RPC) service.

According to yet another aspect of the present disclosure, there is provided an electronic device comprising a processor; a memory storing instructions for the processor to control the method steps as described above.

According to another aspect of the present disclosure, there is provided a computer-readable medium having stored thereon computer-executable instructions that, when executed by a processor, implement the method steps as described above.

According to the data processing method, the data processing device, the electronic equipment and the computer readable medium provided by the embodiments of the disclosure, on one hand, the data is read from the database by adding the data center, then the full data and the incremental data are respectively treated, namely, the full data is synchronized to the cloud service, and the incremental data is synchronized to the client, so that the client only needs to read the incremental data from the data center by means of the version number of the current data, the phenomenon that the database link is full due to the fact that the number of the clients is too large or the amount of the requested data is too large can be prevented, only the data center accesses the database and obtains the incremental data in the whole process, the clients do not directly access the database any more, when the access amount of the client is too large, effective and accurate data service can be provided through the capacity expansion data center, and a complex and expensive data capacity expansion means is avoided. On the other hand, as the full data and the corresponding version number are stored in the cloud service, the problem that the full data is inconsistent with the database data easily caused by the fact that the client directly writes the full data into a local file of the client can be avoided; and the cloud service can be used for conveniently managing the full data, and can quickly check questions and quickly repair error data. The scheme does not pass through hardware or middleware, and network overhead can be reduced.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.

Drawings

The above and other objects, features and advantages of the present disclosure will become more apparent by describing in detail exemplary embodiments thereof with reference to the attached drawings.

Fig. 1 shows a system architecture diagram of a data processing method and apparatus provided in an embodiment of the present disclosure.

Fig. 2 shows a flowchart of a data processing method provided in an embodiment of the present disclosure.

Fig. 3 shows a flowchart of step S210 in fig. 2 according to an embodiment of the present disclosure.

Fig. 4 shows a flowchart of step S230 in fig. 2 according to an embodiment of the present disclosure.

FIG. 5 is a data flow diagram illustrating a system architecture according to an embodiment of the present disclosure.

FIG. 6 shows an overall flowchart of data processing steps in an embodiment of the present disclosure.

Fig. 7 shows a schematic diagram of a data processing apparatus provided in another embodiment of the present disclosure.

Fig. 8 shows a schematic structural diagram of an electronic device suitable for implementing an embodiment of the present application, provided by an embodiment of the present disclosure.

Detailed Description

Example embodiments will now be described more fully with reference to the accompanying drawings. Example embodiments may, however, be embodied in many different forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of example embodiments to those skilled in the art. The drawings are merely schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and thus their repetitive description will be omitted.

Furthermore, the described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a thorough understanding of embodiments of the disclosure. One skilled in the relevant art will recognize, however, that the subject matter of the present disclosure can be practiced without one or more of the specific details, or with other methods, components, devices, steps, and the like. In other instances, well-known structures, methods, devices, implementations, materials, or operations are not shown or described in detail to avoid obscuring aspects of the disclosure.

Some of the block diagrams shown in the figures are functional entities and do not necessarily correspond to physically or logically separate entities. These functional entities may be implemented in the form of software, or in one or more hardware modules or integrated circuits, or in different networks and/or processor devices and/or microcontroller devices.

In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to specific embodiments and the accompanying drawings.

In the related embodiment of the disclosure, for the distributed memory unified scheme, each client is directly connected with the database to perform data synchronization, then persistent full data is performed locally, and then incremental data synchronization is performed through the database. This solution is described. If all the clients are directly connected with the database to update the local memory, firstly, if the clients are gradually increased, the database becomes a bottleneck of service expansion, and if the number of the clients is too large or the amount of the requested data is too large, the database link is fully occupied, so that the number of the loaded clients is limited. In addition, each service is directly connected with the database and then local file persistence is performed, so that each service needs to store a local full file and then the service is loaded from the local, and therefore problems are caused that if data are inconsistent, the data are difficult to check, the full file of each service needs to be downloaded and compared, and a lot of problems are caused to data restoration.

In addition, the implementation may be through hardware or middleware (e.g., Redis, etc.). All clients are consistent memories through hardware, for example, the data buses realize the consistency of the service memories, so that the cost is high, and the data buses are required to be accessed or cut out simultaneously if capacity expansion and capacity reduction are carried out, and the circuit is complex; in addition, a strong dependence on hardware can be generated, which can lead to various inconsistency problems if the data bus is faulty. The middleware is used for carrying out unified storage, network overhead is generated inevitably, and the performance is influenced.

Based on the above, a unified scheme for the memory of the distributed system needs to be provided, so that the memory can be updated effectively and accurately without affecting the service performance, and the consistency of the memory data in the distributed system is ensured.

As shown in fig. 1, system architecture 100 may include a database 101, a data center 102, cloud services 103, and clients 104. In addition, the system is also constructed by a network, and the network is used as a medium for providing communication links between the database and the data center, between the data center and the cloud service, between the data center and the client and between the cloud service and the client. The network may include various connection types, such as wired, wireless communication links, or fiber optic cables, to name a few. In the system architecture 100, the data center 102 and the clients 104 are both formed in a cluster form, that is, a data center service cluster and a client cluster.

Based on the above system architecture, fig. 2 shows a flowchart of a data processing method provided in an embodiment of the present disclosure, which is used in a data center in the system architecture shown in fig. 1, and includes the following steps:

as shown in fig. 2, in step S210, the data center obtains full data or incremental data from a database.

As shown in fig. 2, in step S220, the full data and the corresponding version number are synchronized to a cloud service, so that the cloud service stores the full data and the version number.

As shown in fig. 2, in step S230, the incremental data and the version number corresponding to the incremental data are synchronized to the client through a Remote Procedure call protocol (RPC) service.

According to the data processing method provided by the embodiment of the disclosure, data is read from a database through a data center, then, full data and incremental data are treated respectively, that is, the full data is synchronized to a cloud service, and the incremental data is synchronized to a client, so that the client only needs to read the incremental data from the data center by virtue of the version number of the existing data, the phenomenon that a database link is occupied due to the fact that the number of the client is too large or the requested data amount is too large can be prevented, only the data center accesses the database and obtains the incremental data in the whole process, the client does not directly access the database, when the access amount of the client is too large, effective and accurate data service can be provided through an expansion data center, and a complex and expensive data expansion means is avoided.

The following describes a detailed description of the data processing method provided by the present disclosure with reference to the flowchart shown in fig. 2, specifically as follows:

in step S210, the data center obtains full or incremental data from a database.

In one embodiment of the present disclosure, unlike the prior art, data is not obtained by a client directly accessing a database, but is accessed by a data center.

Fig. 3 shows a flow chart of data center acquiring data, comprising the following steps:

as shown in fig. 3, in step S310, it is determined whether a version number exists in the data center, and the determination result is presence or absence. If the determination result is not present, it indicates that the system is first started, and go to step S320, and if the determination result is present, go to step S330.

The version number is a timestamp for acquiring full data or incremental data from the database, the data are transmitted and stored among the database, the data center, the cloud service and the clients in sequence, time inconsistency among different components in the architecture system is avoided, meanwhile, time inconsistency among the data centers and the clients is avoided, the timestamp for acquiring the data from the database is used as the version number, the timestamp is unique and is generated along with generation of the full data, therefore, the version number and the data file can be paired, when memory data loss occurs, the incremental data can be acquired again from a version number node through the full data file and the version number, missing of the data is avoided, and data recovery can be performed quickly.

As shown in fig. 3, in step S320, the full amount of data is obtained from the database, and the full amount of data and the corresponding version number are stored in the memory of the data center, and then the process goes to step S340.

As shown in fig. 3, in step S330, corresponding incremental data is obtained from the database according to the version number, and the incremental data is stored in a memory of a data center.

As shown in fig. 3, in step S340, the RPC service data is updated according to the data in the memory of the data center, that is, the incremental data to be synchronized to the client.

Based on the steps shown in fig. 3, the data center acquires full data or incremental data from the database by using a timing trigger mechanism, and after a task of timing trigger, the data center loads a data file with a current version number in the database DB according to the version number, writes the data file into a memory of the data center, and updates RPC service data in the memory. If no version number description is the first start, the whole data is loaded from the DB and written into the memory (i.e. the memory of the data center).

In an embodiment of the present disclosure, after the step S320, the data center obtains the full amount of data from the database, the method further includes:

and storing the total data stored in the memory of the data center in a local file of the data center by adopting a timing trigger mechanism in combination with the corresponding version number. And then, verifying the full data, and synchronizing the full data and the corresponding version number to the cloud service if the full data and the corresponding version number pass the verification.

The verification can be performed by adopting a hash algorithm (such as MD5), specifically, whether the current MD5 value of the local file is correct and complete is checked by comparing the current MD5 value with a standard MD5 value, so as to ensure that the latest full amount of data and the corresponding version number are written into the cloud service.

In an embodiment of the present disclosure, after the step S330 of obtaining the incremental data from the database, the data center further includes:

firstly, storing the incremental data stored in the memory of the data center into a local file of the data center; and secondly, combining the original full data stored in the local file of the data center with the incremental data and the corresponding version number to form new full data, and storing the new full data in the local file of the data center.

In step S220, the full data and the corresponding version number are synchronized to a cloud service, so that the cloud service stores the full data and the version number.

In step S230, the incremental data and the version number corresponding to the incremental data are synchronized to the client by a remote procedure call protocol (RPC) service.

In an embodiment of the present disclosure, fig. 4 shows a step flow after step S230 synchronizes the incremental data and the version number corresponding to the incremental data to the client through a remote procedure call protocol RPC service in this embodiment, where the step includes the following steps:

as shown in fig. 4, in step S410, the client acquires the full data and the corresponding version number from the cloud service.

As shown in fig. 4, in step S420, the full data and the full data already stored in the local file of the client are verified, and a new full data is obtained according to the verification result.

As shown in fig. 4, in step S430, the new full data and the corresponding version number are stored in the local file of the client.

Based on the steps shown in fig. 4, the latest full amount of data and the corresponding version number are always stored in the local file of the client. And the client accesses the data center through the RPC range according to the starting time in the memory or the version number after the last incremental update to update the incremental data in the local memory. In addition, the client acquires the full data and the corresponding version number from the cloud service at regular time, updates the local full data file and the local version file, and ensures that the full file can be normally loaded from the local if the cloud service is abnormal when the client is started next time.

In an embodiment of the present disclosure, after step S230, the method further includes:

if the client is restarted, after the client is restarted, trying to load the full data and the corresponding version number stored in the local file of the client; if the loading fails, the client acquires full data and a corresponding version number from the cloud service, acquires incremental data from the data center through an RPC service, and stores the incremental data in a memory of the client; and if the loading is successful, directly loading the full amount of data and the corresponding version number stored in the local file of the client.

Therefore, under the condition that the client normally works and is not restarted, the client acquires full data by accessing the cloud service and stores the full data in a local file of the client, and acquires the incremental data from the data center by the RPC service and stores the incremental data in the memory of the client. If the client is restarted due to reasons, the full data and the corresponding version number in the local file of the client are loaded firstly after the client is restarted, if the recording fails, the full data and the corresponding version number are directly loaded from the cloud service, and incremental data are obtained from the data center through the RPC service at regular time to update the memory of the client, so that the data consistency of each client in the client cluster is ensured.

Based on the above, fig. 5 describes in detail the data processing method in the embodiment of the present disclosure with reference to a data flow diagram of a system architecture:

as shown in fig. 5, in step S501, the data is periodically acquired to update the cache of the data center.

As shown in fig. 5, in step S502, the full data and the version number are timing-synchronized.

As shown in fig. 5, in step S503, the RPC service synchronizes the delta data.

As shown in fig. 5, in step S504, the full amount of data and the version number are periodically acquired.

Through the system architecture shown in fig. 5 and the corresponding data stream transmission direction, in the system architecture, the data center can avoid the bottleneck that the DB becomes a cluster for capacity expansion, and prevent the database link from being full due to excessive number of clients and excessive request data amount. Through the data center, DB access is realized only by the service of the data center, the client does not directly access the DB any more, and free capacity expansion of the client is realized. The cloud service has the function in the system architecture as a persistent data warehouse for the full data files and the version numbers, and the problem that the full data files are easily inconsistent because the full data files are directly written into the local part of the client by the client is avoided. The management of the full data files is uniformly carried out after the cloud service is used, the data problems in the memory can be conveniently and quickly checked, and the purpose of quickly repairing the memory data in each service can be achieved by repairing the data of the cloud service. The client is a server and needs to acquire full data and incremental data, wherein the full data is acquired by accessing cloud services and is written into a local file under the condition of non-restart, and the incremental data in the memory of the server is updated incrementally by accessing RPC services, so that the data consistency of each server is ensured.

Fig. 6 shows an overall flow chart of the above-described data processing steps, including the following steps:

as shown in fig. 6, in step S601, the data center obtains data from the DB, including the first-initiated generation of the full-size data file and the subsequent acquisition of incremental data.

As shown in fig. 6, in step S602, the data center periodically writes the data in the memory into the local file, and verifies according to MD5 to ensure that the full amount of data is successfully written into the cloud service.

As shown in fig. 6, in step S603, the data center provides RPC service of incremental data to provide service for the client.

As shown in fig. 6, in step S604, the client cluster starts full data loading from the cloud service for the first time, writes to the local file, and periodically accesses the RPC service of the data center to update the data in the client memory.

As shown in fig. 6, in step S605, the client service acquires the full amount of data from the cloud service at regular time, performs comparison check with the MD5 of the local file, and writes the latest full amount of data and the corresponding version number into the local file.

As shown in fig. 6, in step S606, if the client is restarted, the local full amount and the corresponding version number are loaded first, and if the acquisition fails, the local full amount and the corresponding version number are directly loaded from the cloud service, and then the memory of the client is updated regularly through the RPC service.

In addition, on the basis of the step flow shown in fig. 6, when the data center is abnormal, if a certain data center service in the data center service cluster is abnormal, other data centers can normally provide RPC services and timing update of the full file; under the condition that all data center servers are abnormal, the client only does not normally update the incremental data within a certain time, and after the data center is recovered, the incremental RPC service can be accurately and continuously provided according to the full data files and the version numbers in the cloud service, and data omission does not occur. Under the condition that the cloud service is abnormal, the client can continue to access the data center at regular time through the total data and the corresponding version number in the local file to provide the memory service; the data center can continue to provide incremental data service, the full data cannot be updated only temporarily, and after the cloud service is recovered, the data center can continue to write the latest full data and the corresponding version number into the cloud service, so that the client can continue to update the local full data and the corresponding version number, and the data cannot be missed.

It should be noted that in the system architecture, the cloud service and the RPC service call can be customized, and the memory service is performed by using the pairing of the version number and the full data, so that the problem that the memory data consistency is difficult to locate and repair is solved, the dependence on unified memory hardware and the network overhead of caching middleware are avoided, and a unified memory scheme in a distributed system with a robust failover mechanism is provided.

To sum up, in the data processing method provided by the embodiment of the present disclosure, on one hand, data is read from a database by adding a data center, and then full data and incremental data are treated separately, that is, the full data is synchronized to a cloud service, and the incremental data is synchronized to a client, so that the client only needs to read the incremental data from the data center by using the version number of the currently existing data, which can prevent the database link from being occupied due to an excessive amount of client or an excessive amount of requested data. On the other hand, as the full data and the corresponding version number are stored in the cloud service, the problem that the full data is inconsistent with the database data easily caused by the fact that the client directly writes the full data into a local file of the client can be avoided; and the cloud service can be used for conveniently managing the full data, and can quickly troubleshoot problems and quickly repair error data. The scheme does not pass through hardware or middleware, and network overhead can be reduced.

Fig. 7 is a schematic diagram of a data processing apparatus provided in another embodiment of the present disclosure, and as shown in fig. 7, the apparatus 700 includes: a data acquisition module 710, a full-scale synchronization module 720, and an incremental synchronization module 730.

The data acquisition module 710 is configured to acquire full or incremental data from a database by the data center; the full synchronization module 720 is configured to synchronize the full data and the corresponding version number to a cloud service so that the cloud service stores the full data and the version number; the delta synchronization module 730 is configured to synchronize the delta data and a version number corresponding to the delta data to the client through a remote procedure call protocol, RPC, service.

The functions of each module in the apparatus are described in the above method embodiments, and are not described again here.

To sum up, in the data processing apparatus provided in the embodiment of the present disclosure, on one hand, data is read from a database by adding a data center, and then full data and incremental data are treated separately, that is, the full data is synchronized to a cloud service, and the incremental data is synchronized to a client, so that the client only needs to read the incremental data from the data center by using the version number of the currently existing data, which can prevent the database link from being occupied due to an excessive amount of client or an excessive amount of requested data. On the other hand, as the full data and the corresponding version number are stored in the cloud service, the problem that the full data is inconsistent with the database data easily caused by the fact that the client directly writes the full data into a local file of the client can be avoided; and the cloud service can be used for conveniently managing the full data, and can quickly troubleshoot problems and quickly repair error data. The scheme does not pass through hardware or middleware, and network overhead can be reduced.

In another aspect, the present disclosure also provides an electronic device, including a processor and a memory, where the memory stores operating instructions for the processor to control the following method:

the data center acquires full data or incremental data from a database; synchronizing the full data and the corresponding version number to a cloud service so that the cloud service stores the full data and the version number; and synchronizing the incremental data and the version number corresponding to the incremental data to a client through a remote procedure call protocol (RPC) service.

Referring now to FIG. 8, shown is a block diagram of a computer system 800 suitable for use in implementing the electronic devices of embodiments of the present application. The electronic device shown in fig. 8 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present application.

As shown in fig. 8, the computer system 800 includes a Central Processing Unit (CPU)801 that can perform various appropriate actions and processes in accordance with a program stored in a Read Only Memory (ROM)802 or a program loaded from a storage section 807 into a Random Access Memory (RAM) 803. In the RAM 803, various programs and data necessary for the operation of the system 800 are also stored. The CPU 801, ROM 802, and RAM 803 are connected to each other via a bus 804. An input/output (I/O) interface 805 is also connected to bus 804.

The following components are connected to the I/O interface 805: an input portion 806 including a keyboard, a mouse, and the like; an output section 807 including a signal such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, and a speaker; a storage portion 808 including a hard disk and the like; and a communication section 809 including a network interface card such as a LAN card, a modem, or the like. The communication section 809 performs communication processing via a network such as the internet. A drive 810 is also connected to the I/O interface 805 as necessary. A removable medium 811 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 810 as necessary, so that a computer program read out therefrom is mounted on the storage portion 808 as necessary.

In particular, according to an embodiment of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method illustrated by the flow chart. In such an embodiment, the computer program can be downloaded and installed from a network through the communication section 809 and/or installed from the removable medium 811. The computer program executes the above-described functions defined in the system of the present application when executed by the Central Processing Unit (CPU) 801.

It should be noted that the computer readable medium shown in the present application may be a computer readable signal medium or a computer readable medium or any combination of the two. The computer readable medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present application, a computer readable medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In this application, however, a computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may be any computer readable medium that is not a computer readable medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.

The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

The units described in the embodiments of the present application may be implemented by software or hardware. The described units may also be provided in a processor, which may be described as: a processor includes a transmitting unit, an obtaining unit, a determining unit, and a first processing unit. The names of these units do not in some cases constitute a limitation to the unit itself, and for example, the sending unit may also be described as a "unit sending a picture acquisition request to a connected server".

In another aspect, the present disclosure also provides a computer-readable medium, which may be contained in the apparatus described in the above embodiments; or may be separate and not assembled into the device. The computer readable medium carries one or more programs which, when executed by a device, cause the device to include the method steps of:

It should be clearly understood that this disclosure describes how to make and use particular examples, but the principles of this disclosure are not limited to any details of these examples. Rather, these principles can be applied to many other embodiments based on the teachings of the present disclosure.

Exemplary embodiments of the present disclosure are specifically illustrated and described above. It is to be understood that the present disclosure is not limited to the precise arrangements, instrumentalities, or instrumentalities described herein; on the contrary, the disclosure is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims.

Claims

1. A data processing method, comprising:

the data center acquires full data or incremental data from a database;

synchronizing the full data and the corresponding version number to a cloud service so that the cloud service can store the full data and the version number;

2. The data processing method of claim 1, wherein the data center obtaining the full or incremental data from the database comprises:

3. The data processing method of claim 2, wherein after the data center obtains the full amount of data from the database, the method further comprises:

4. The data processing method of claim 2, wherein after the data center obtains the incremental data from the database, the method further comprises:

5. The data processing method of claim 3 or 4, wherein synchronizing the full amount of data and the version number to a cloud service comprises:

6. The data processing method of claim 1, wherein after synchronizing the incremental data and the version number corresponding to the incremental data to the client through a remote procedure call protocol (RPC) service, the method further comprises:

7. The data processing method of claim 6, further comprising:

8. A data processing apparatus, comprising:

a data acquisition module configured to acquire full or incremental data from a database by the data center;

9. An electronic device, comprising:

a processor;

memory storing instructions for the processor to control the method steps according to any one of claims 1-7.

10. A computer-readable medium having stored thereon computer-executable instructions, which when executed by a processor, perform the method steps of any one of claims 1-7.