CN114722060A - Data synchronization method, device, equipment and computer readable storage medium - Google Patents

Abstract

The application relates to a data synchronization method, a data synchronization device and a computer readable storage medium. The method comprises the following steps: judging whether a first database of a first entity has data updating; if the first database is updated, exporting updated data from the first database; cloud storing the update data derived from the first database; judging whether data updating needs to be carried out on a second database of a second entity; if data updating needs to be carried out on a second database of the second entity, downloading updated data which are subjected to cloud storage; update data derived from the first database is imported to a second database of the second entity. The technical scheme of the application can realize data synchronization between cross-entities.

Description

Data synchronization method, device, equipment and computer readable storage medium

Technical Field

The present application relates to the field of computer networks, and in particular, to a data synchronization method, apparatus, device, and computer-readable storage medium.

Background

Data synchronization often occurs between different applications, systems, or devices, enabling backup or interaction of data between different entities. For example, the application A, B is a master application and a backup application for the same object, respectively, and when the data of the application a is updated, theoretically, the application B should also have an update. At this time, the updated data is imported from the application a to the application B, so that the data synchronization between the two applications can be completed. In the related art, a data synchronization method is to connect two different applications to the same database, and the two applications achieve data synchronization by accessing the database shared by the two applications. However, since the two applications themselves have a huge difference in the load and concurrency of data, the related art may cause the common database to bear a huge pressure and even collapse.

Disclosure of Invention

To solve or partially solve the problems in the related art, the present application provides a data synchronization method, apparatus, device and computer-readable storage medium, which can implement data synchronization across entities.

A first aspect of the present application provides a data synchronization method, including:

judging whether a first database of a first entity has data updating;

if the first database is updated, exporting updated data from the first database;

performing cloud storage on the updating data;

judging whether a second database of a second entity needs to update data or not;

if the second database of the second entity needs to update the data, downloading the updated data which is subjected to cloud storage;

and importing the updating data into a second database of the second entity.

A second aspect of the present application provides a data synchronization apparatus, including:

the first judging module is used for judging whether data updating exists in a first database of the first entity;

the export module is used for exporting the updated data from the first database if the first database is updated;

the storage module is used for carrying out cloud storage on the updated data;

the second judgment module is used for judging whether data updating needs to be carried out on a second database of the second entity;

the downloading module is used for downloading the updated data which is subjected to cloud storage if the data of the second database of the second entity needs to be updated;

and the import module is used for importing the update data into a second database of a second entity.

A third aspect of the present application provides an electronic device comprising:

a processor; and

a memory having executable code stored thereon, which when executed by the processor, causes the processor to perform the method as described above.

A fourth aspect of the present application provides a computer-readable storage medium having stored thereon executable code, which, when executed by a processor of an electronic device, causes the processor to perform the method as described above.

The technical scheme provided by the application can comprise the following beneficial effects: different from the prior art that one database is shared when data synchronization is performed, in the technical scheme of the application, the update data is exported from the first database of the first entity and is stored in the cloud, and the update data stored in the cloud is downloaded and then is imported into the second database of the second entity, so that the cloud storage is only used as a transmission or storage middleware of the update data, the export and import of the update data are still realized by accessing the database of the entity by each entity, and the characteristic attribute (for example, high concurrency) of database access by one entity can not cause the pressure of database access by another entity, so that the data synchronization across the entities is smoothly realized, and the method is particularly suitable for the data synchronization between the entities with the significant difference in the characteristic attribute.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the application.

Drawings

The foregoing and other objects, features and advantages of the application will be apparent from the following more particular descriptions of exemplary embodiments of the application as illustrated in the accompanying drawings wherein like reference numbers generally represent like parts throughout the exemplary embodiments of the application.

Fig. 1 is a schematic flow chart of a data synchronization method according to an embodiment of the present application;

fig. 2 is a schematic structural diagram of a data synchronization apparatus according to an embodiment of the present application;

fig. 3 is a schematic structural diagram of an electronic device shown in an embodiment of the present application.

Detailed Description

Embodiments of the present application will be described in more detail below with reference to the accompanying drawings. While embodiments of the present application are illustrated in the accompanying drawings, it should be understood that the present application may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used in this application and the appended claims, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.

It should be understood that although the terms "first," "second," "third," etc. may be used herein to describe various information, these information should not be limited to these terms. These terms are only used to distinguish one type of information from another. For example, first information may also be referred to as second information, and similarly, second information may also be referred to as first information, without departing from the scope of the present application. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of the present application, "a plurality" means two or more unless specifically limited otherwise.

Data synchronization often occurs between entities such as different applications, systems or devices, and backup or interaction of data between different entities is realized. For example, the application A, B is a master application and a backup application for the same object, respectively, and when the data of the application a is updated, theoretically, the application B should also have an update. At this time, the updated data is imported from the application a to the application B, so that the data synchronization between the two applications can be completed. In the related art, a data synchronization method is to connect two different applications to the same database, and the two applications achieve data synchronization by accessing the database shared by the two applications. However, since the two applications themselves have a huge difference in the load and concurrency of data, the related art may cause the common database to bear a huge pressure and even collapse.

In view of the foregoing problems, embodiments of the present application provide a data synchronization method, which can implement data synchronization across entities.

The technical solutions of the embodiments of the present application are described in detail below with reference to the accompanying drawings.

Referring to fig. 1, it is a schematic flow chart of a data synchronization method shown in the embodiment of the present application, where the method mainly includes steps S101 to S106, and is described as follows:

step S101: whether data updating exists in the first database of the first entity is judged.

In the embodiment of the present application, the entity may be an application, a system, or a device, and the like, which has a corresponding database, for example, a Remote Dictionary service (Redis) database; redis is a typical Key-Value (Key-Value) database. After data writing, deleting, etc. operations are performed on the database, the data in the database may not be the same as the data before the operations, i.e. the database is updated. Because data files in a database are generally encrypted, for example, by using the version 5 Message Digest Algorithm (MD 5), different data files are encrypted by MD5 to generate a so-called MD5 value, and therefore, one method for determining whether an update exists in the database is to compare whether the MD5 value of the current data file of the database is the same as the MD5 value of the previous data file, if the MD5 value of the current data file of the database is the same as the MD5 value of the previous data file, no data update exists in the database, otherwise, a data update exists. In the embodiment of the present application, by comparing whether the MD5 value of the current data file in the first database of the first entity is the same as the MD5 value of the previous data file, if the MD5 value is the same as the MD5 value of the previous data file, there is no data update in the first database of the first entity, otherwise, the first database of the first entity is updated with data.

Step S102: if the first database is updated, the updated data is derived from the first database.

In an embodiment of the present application, the exporting of the update data from the first database may be implemented by a data exporting process, in other words, a data exporting process may be used to export the update data from the first database. In view of the problem that when modifying a shared resource or object of multiple processes in a multi-core and multi-process environment, results and expectations are often inconsistent due to random scheduling of a central processing unit, a lock mechanism needs to be adopted to lock the processes, that is, the method of the embodiment of the present application may further include acquiring a process lock to lock the data export process. Specifically, acquiring a process lock to lock a data export process may be: judging whether the resource abundance of the process lock is greater than a preset threshold value or not according to the state information of the process lock; if the resource abundance of the process lock is greater than the preset threshold, the process lock is acquired to lock the data export process, where the preset threshold may be 0, and the state information of the process lock includes information of the resource abundance of the process lock, the process lock is occupied, the process lock is idle, and the like, where the resource abundance of the process lock is how many available process locks exist currently. When the resource abundance of the process lock is larger than the preset threshold, besides the process lock acquisition, the resource abundance of the process lock can be subtracted, and meanwhile, information that the process lock acquisition is successful is generated and returned to a process lock requester, namely the first entity. The resource abundance of the process lock is reduced by one because the process lock requester, i.e. the first entity, has acquired the right to use a process lock, and the currently available process locks are reduced by one. It should be noted that, since the subsequent processes are not executed any more when there is no data update in the first database of the first entity, the obtaining of the process lock to lock the data export process in the above embodiment may be performed before determining whether there is a data update in the first database of the first entity, or may be performed after determining that there is a data update in the first database of the first entity.

The above embodiment is a scenario when the abundance of resources of the process lock is greater than a preset threshold. If the resource abundance of the process lock is not greater than the preset threshold, for example, if the resource abundance of the process lock is not greater than 0, that is, no process lock exists, the process lock request of the process lock requester is placed in the process lock acquisition queue, and the process lock request is responded when the resource abundance of the process lock is greater than the preset threshold. It can be understood that, in order to save resources of the process lock or not waste resources of the process lock, if the determination result of step S101 is that there is no data update in the first database, the acquired process lock is released. It should be noted that, the first entity may be a Web application developed based on a larvel framework, and when the first database is a Redis database, acquiring a process lock to lock a data export process may be specifically implemented using a command such as setnx or setex of the Redis database.

As an embodiment of the present application, if the first database is updated, deriving the updated data from the first database may be: if the first database is updated, acquiring attribute information of the updated data; the update data is derived from the first database based on the attribute information of the update data. In the above-described embodiment, the attribute information of the update data describes information such as the number size, name, and position in the first database of the update data. It can be understood that when the amount of the updated data is too large, the export from the first database to the target storage unit directly has a problem of inefficiency, and therefore, in an embodiment of the present application, the export of the updated data from the first database according to the attribute information of the updated data may specifically be: determining the data volume of the updating data according to the attribute information of the updating data; if the data volume of the updated data is larger than a preset derived number threshold, dividing the updated data into a plurality of slices, wherein each slice corresponds to slice description information; according to the slice description information, obtaining a corresponding slice from a first database and caching the slice into a memory; and storing the cached slice to a local target storage unit. It should be noted that the local target storage unit here is a target storage unit of the local storage device where the first database is located, and based on that the first database and the first entity may be located in the same storage device or may not be located in the same storage device, the local target storage unit may be a local storage unit of the first entity or may not be a local storage unit of the first entity.

In the above-described embodiment, the preset derived number threshold is an upper limit number of derived data set in advance. When the data volume of the updated data is larger than the preset export number threshold, the updated data is not directly stored in the local target storage unit, but the slice of the updated data is cached in the memory, so the execution efficiency of the data export process is obviously improved; meanwhile, data export is carried out according to the slices, and each slice is only one part of the updated data, so that all the updated data are not required to be cached in a memory, and particularly under the conditions of large updated data volume and limited memory space, the method has more obvious advantages and higher practical value.

As described above, in order to save resources of the process lock or not waste resources of the process lock, the acquired process lock is released when the first database does not have data update as a result of the determination in step S101; likewise, if the update data is not successfully derived from the first database, the acquired process lock may be released to conserve or not waste resources of the process lock.

Step S103: and carrying out cloud storage on the update data derived from the first database.

In the embodiment of the present application, cloud storing the update data derived from the first database may be that the update data is directly derived from the first database and uploaded to a cloud storage component, such as a seaweedfs storage; the update data derived from the first database and stored to the local target storage unit where the first database is located may also be uploaded to the cloud storage component. Further, a cloud storage address of the update data, e.g., a Uniform/Uniform Resource Locator (URL) address of a snapshot file of the update data, may be returned to the first entity so that the first entity passes the cloud storage address of the update data to the second entity. It should be noted that, in the embodiment of the present application, the exported update data is cloud-stored, and the cloud storage address of the update data is sent to the second entity, instead of directly sending the update data to the second entity, because when the snapshot file of the update data is large, only the cloud storage address of the update data is sent instead of the snapshot file of the update data, which may reduce the data transmission pressure of the snapshot server corresponding to the first entity.

Similarly to the foregoing embodiment, if there is no data update in the first database of the first entity and/or the update data is not successfully derived from the first database, the acquired process lock is released, and similarly, if the cloud storage of the update data is completed, the acquired process lock is released, so as to save resources of the process lock or not waste resources of the process lock. As for the method for releasing the acquired process lock, as an embodiment of the present application, it may be: judging whether the process lock is acquired or not according to the state information of the process lock; if the process lock is not acquired, generating release error information and returning the release error information to the process lock requester; if the process lock is acquired, releasing the process lock when the first database of the first entity has no data update and/or the update data is not successfully exported from the first database and/or the update data is subjected to cloud storage execution, and adding one operation to the resource abundance of the process lock to indicate that an available process lock is added currently; the process lock request stored in the queue is obtained in response to the process lock. When adding an operation to the resource abundance of the process lock, a release success message can be generated and returned to the first entity, and the process lock request stored in the process lock acquisition queue is responded, so that the corresponding operation, namely the process lock acquisition or the process lock release, is performed on the process lock request.

Step S104: and judging whether data updating needs to be carried out on the second database of the second entity.

In the embodiment of the present application, the second entity may be an application, a system, or a device different from the first entity, for example, as described above, the first entity is a Web application developed by a larvel framework, the second entity may be a Web application developed based on an openness framework, and the second database of the second entity may also be a Redis database or another type of database, so that the application scenario of the present application may be data synchronization across Web applications. As for determining whether the second database of the second entity needs to be updated, an embodiment of the method may be that a data file version number of the second database is first obtained, then the data file version number is requested from the second entity, and finally the obtained data file version number is compared with the data file version number requested from the second entity, if the two are consistent, it is determined that the second database of the second entity needs to be updated, otherwise, the second database of the second entity does not need to be updated. It should be noted that, in the above embodiment, when the version number of the data file is requested to the second entity, it may be determined whether a network error occurs, and if the network error occurs, the request is unsuccessful, and the process is exited.

Step S105: and if the data of the second database of the second entity needs to be updated, downloading the updated data which is subjected to cloud storage.

Specifically, the implementation of step S105 may be: acquiring a cloud storage address of updated data; and downloading the updating data from the target storage unit identified by the cloud storage address according to the cloud storage address of the updating data. Further, whether the update data is successfully downloaded from the storage unit corresponding to the cloud storage address can be judged, and if not, the process is exited. After the update data is successfully downloaded from the storage unit corresponding to the cloud storage address, the subsequent process can be continued. In the above embodiment, when the snapshot file of the update data is downloaded through the storage address of the update data, if the snapshot file of the update data is suspended, the snapshot file of the update data may be continuously transmitted from the suspended place, so that efficiency of acquiring the snapshot file of the update data by the server corresponding to the second entity is improved.

Step S106: the update data is imported to a second database of the second entity.

As an embodiment of the present application, importing the update data into the second database of the second entity may be: acquiring a data snapshot file of the updated data corresponding to the updated data identification number; processing the data snapshot files of the updated data in batches according to a preset batch strategy; and importing the data snapshot file after batch processing into a second database of the second entity. In the above embodiment, the preset batching strategy may be obtained according to the size, the number, and the import speed of the target import data, and is a preset scheme for dividing the target import data into multiple batches, which is set to facilitate the data import process. For example, when the target import data is 10000 pieces, the target import data may be divided into 10 batches according to a preset batch policy according to 1000 pieces per batch, or the target import data may be divided into a plurality of batches with the same data size according to the data size, and so on.

In another embodiment of the present application, when the second entity is a Web application developed based on an openness framework and the second database of the second entity is a Redis database, the update data may be imported to the second database of the second entity through a Redis-load, Redis database command. Further, if the update data is successfully imported into the second database of the second entity, a kafka message may be sent to the application developed based on the larvel framework, indicating that the updated Redis received, and the data synchronization operation is successfully performed. It should be noted that, in the foregoing embodiment of the present application, when the first entity is a Web application developed by a larvel framework, the second entity may be a Web application developed based on an openreserve framework; alternatively, when the first entity is a Web application developed by an openreserve framework, the second entity may be a Web application developed based on a larvel framework. Therefore, the technical scheme of the application is suitable for cross-entity data synchronization, or further can be considered to be applicable to cross-Web application data synchronization. The concurrency number supported by the Laravel-based framework is low, and the method can be used for developing Web applications of a management system and the like, while the concurrency number supported by the OpenResty framework is high, and the method can be used for developing Web applications of a service system and the like, and further the technical scheme can be used for data synchronization between Web applications which are different in the highest supported concurrency number and different in types (such as the management system, the service system and the like).

It can be known from the data synchronization method illustrated in fig. 1 that, unlike the related art that one database is shared when data synchronization is performed, in the technical solution of the present application, update data is exported from a first database of a first entity and stored in a cloud, and the update data stored in the cloud is downloaded and then imported into a second database of a second entity, so that cloud storage is only used as a transmission or storage middleware of the update data, and export and import of the update data are still implemented by accessing the database of each entity, and it is ensured that a characteristic attribute (e.g., high concurrency) of access to the database by one entity does not cause pressure of access to the database by another entity, thereby smoothly implementing data synchronization across entities, and being particularly suitable for data synchronization between entities having significant differences in characteristic attributes.

Corresponding to the embodiment of the application function implementation method, the application also provides a data synchronization device, electronic equipment and a corresponding embodiment.

Fig. 2 is a schematic structural diagram of a data synchronization apparatus according to an embodiment of the present application. For convenience of explanation, only portions related to the embodiments of the present application are shown. The data synchronization apparatus illustrated in fig. 2 mainly includes a first determining module 201, an exporting module 202, a storing module 203, a second determining module 204, a downloading module 205, and an importing module 206, where:

a first determining module 201, configured to determine whether a first database of a first entity has data update;

an export module 202, configured to export updated data from the first database if the first database is updated with data;

a storage module 203, configured to perform cloud storage on update data derived from the first database;

a second determining module 204, configured to determine whether data update needs to be performed on a second database of the second entity;

a downloading module 205, configured to download, if data update needs to be performed on a second database of a second entity, update data that has been subjected to cloud storage;

an import module 206, configured to import the update data exported from the first database to a second database of the second entity.

Optionally, the export module 202 of the example of fig. 2 may include an attribute obtaining unit and an update data export unit, wherein:

the attribute acquisition unit is used for acquiring attribute information of the updated data if the first database is updated;

and the updating data exporting unit is used for exporting the updating data from the first database according to the attribute information of the updating data.

Optionally, the update data derivation unit of the above example may include a determination unit, a division unit, a cache unit, and a storage unit, wherein:

a determining unit configured to determine a data amount of the update data according to the attribute information of the update data;

the segmentation unit is used for segmenting the update data into a plurality of slices if the data volume of the update data is larger than a preset derived number threshold, wherein each slice corresponds to the slice description information;

the cache unit is used for acquiring the corresponding slice from the first database according to the slice description information and caching the slice into the memory;

and the storage unit is used for storing the cached slice to the local target storage unit.

Optionally, the apparatus illustrated in fig. 2 may further include an obtaining module configured to obtain a process lock to lock the data export process, where the data export process is configured to export the update data from the first database.

Optionally, the acquiring module of the above example may include a resource abundance determining unit and a process lock acquiring unit, wherein:

the resource abundance judging unit is used for judging whether the resource abundance of the process lock is greater than a preset threshold value or not according to the state information of the process lock;

and the process lock acquisition unit is used for acquiring a process lock to lock the data export process if the resource abundance of the process lock is greater than a preset threshold value.

Optionally, the apparatus illustrated in fig. 2 may further include a releasing module, configured to release the acquired process lock if the first database of the first entity has no data update and/or the update data is not successfully derived from the first database and/or cloud storage of the update data is completed.

Optionally, the release module of the foregoing example may include an acquisition determining unit, a returning unit, a process lock releasing unit, and a responding unit, where:

the acquiring and judging unit is used for judging whether the process lock is acquired or not according to the state information of the process lock;

the return unit is used for generating release error information and returning the release error information to the process lock requester if the process lock is not acquired;

the process lock releasing unit is used for releasing the process lock when the process lock is acquired and adding one to the resource abundance of the process lock;

and the response unit is used for responding to the process lock request stored in the process lock acquisition queue.

Optionally, the download module 205 illustrated in fig. 2 may include an address obtaining unit and an update data downloading unit, where:

the address acquisition unit is used for acquiring a cloud storage address of the update data;

and the update data downloading unit is used for downloading the update data from the storage unit corresponding to the cloud storage address according to the cloud storage address of the update data.

Optionally, the import module 206 illustrated in fig. 2 may include a data snapshot file obtaining unit, a batching unit, and a data import unit, where:

the data snapshot file acquiring unit is used for acquiring a data snapshot file of the updating data corresponding to the updating data identification number;

the batching unit is used for batching the data snapshot file of the updated data according to a preset batching strategy;

and the data import unit is used for importing the data snapshot files processed in batches into a second database of the second entity.

With regard to the apparatus in the above-described embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment related to the method, and will not be elaborated here.

Fig. 3 is a schematic structural diagram of an electronic device shown in an embodiment of the present application.

Referring to fig. 3, the electronic device 300 includes a memory 310 and a processor 320.

The Processor 320 may be a Central Processing Unit (CPU), other general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic, discrete hardware components, etc. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.

The memory 310 may include various types of storage units such as a system memory, a Read Only Memory (ROM), and a permanent storage device. Wherein the ROM may store static data or instructions for the processor 320 or other modules of the computer. The persistent storage device may be a read-write storage device. The persistent storage may be a non-volatile storage device that does not lose stored instructions and data even after the computer is powered down. In some embodiments, the persistent storage device employs a mass storage device (e.g., magnetic or optical disk, flash memory) as the persistent storage device. In other embodiments, the permanent storage may be a removable storage device (e.g., floppy disk, optical drive). The system memory may be a read-write memory device or a volatile read-write memory device, such as a dynamic random access memory. The system memory may store instructions and data that some or all of the processors require at run-time. Further, the memory 310 may comprise any combination of computer-readable storage media, including various types of semiconductor memory chips (e.g., DRAM, SRAM, SDRAM, flash memory, programmable read-only memory), magnetic and/or optical disks, may also be employed. In some embodiments, memory 310 may include a removable storage device that is readable and/or writable, such as a Compact Disc (CD), a digital versatile disc read only (e.g., DVD-ROM, dual layer DVD-ROM), a Blu-ray disc read only, an ultra-dense disc, a flash memory card (e.g., SD card, min SD card, Micro-SD card, etc.), a magnetic floppy disk, or the like. Computer-readable storage media do not contain carrier waves or transitory electronic signals transmitted by wireless or wired means.

The memory 310 has stored thereon executable code that, when processed by the processor 320, may cause the processor 320 to perform some or all of the methods described above.

Furthermore, the method according to the present application may also be implemented as a computer program or computer program product comprising computer program code instructions for performing some or all of the steps of the above-described method of the present application.

Alternatively, the present application may also be embodied as a computer-readable storage medium (or non-transitory machine-readable storage medium or machine-readable storage medium) having executable code (or a computer program or computer instruction code) stored thereon, which, when executed by a processor of an electronic device (or server, etc.), causes the processor to perform part or all of the various steps of the above-described method according to the present application.

Having described embodiments of the present application, the foregoing description is intended to be exemplary, not exhaustive, and not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen in order to best explain the principles of the embodiments, the practical application, or improvements made to the technology in the marketplace, or to enable others of ordinary skill in the art to understand the embodiments disclosed herein.

Claims (12)

1. A method for synchronizing data, the method comprising:

judging whether a first database of a first entity has data updating;

if the first database is updated, exporting updated data from the first database;

performing cloud storage on the updating data;

judging whether data updating needs to be carried out on a second database of a second entity;

if data updating needs to be carried out on a second database of the second entity, downloading the updated data which is subjected to cloud storage;

and importing the updating data into a second database of the second entity.

2. The data synchronization method of claim 1, wherein if the first database is updated, deriving updated data from the first database comprises:

if the first database is updated, acquiring attribute information of the updated data;

deriving the update data from a first database according to attribute information of the update data.

3. The data synchronization method according to claim 2, wherein the deriving the update data from the first database according to the attribute information of the update data comprises:

determining the data volume of the updating data according to the attribute information of the updating data;

if the data volume of the updated data is larger than a preset derived quantity threshold value, dividing the updated data into a plurality of slices, wherein each slice corresponds to slice description information;

according to the slice description information, obtaining a corresponding slice from the first database and caching the slice into a memory;

and storing the cached slice to a local target storage unit.

4. A method for synchronizing data according to any one of claims 1 to 3, characterized in that the method further comprises:

a process lock is acquired to lock a data export process for exporting the update data from the first database.

5. The data synchronization method of claim 4, wherein the obtaining a process lock to lock a data export process comprises:

judging whether the resource abundance of the process lock is greater than a preset threshold value or not according to the state information of the process lock;

and if the resource abundance of the process lock is greater than the preset threshold, acquiring the process lock to lock the data export process.

6. The data synchronization method of claim 4, further comprising:

and releasing the process lock if the first database of the first entity has no data update and/or the update data is not successfully derived from the first database and/or the cloud storage execution of the update data is completed.

7. The data synchronization method of claim 6, wherein the releasing the process lock comprises:

judging whether the process lock is acquired or not according to the state information of the process lock;

if the process lock is not acquired, generating release error information and returning the release error information to the process lock requester;

if the process lock is acquired, releasing the process lock, and adding one to the resource abundance of the process lock;

the process lock request stored in the queue is obtained in response to the process lock.

8. The data synchronization method according to claim 1, wherein the downloading the cloud-stored update data comprises:

acquiring a cloud storage address of the updated data;

and downloading the updating data from a storage unit corresponding to the cloud storage address according to the cloud storage address of the updating data.

9. The data synchronization method of claim 1, wherein importing the updated data into a second database of a second entity comprises:

acquiring a data snapshot file of the updated data corresponding to the updated data identification number;

processing the data snapshot files in batches according to a preset batch strategy;

and importing the data snapshot file after batch processing into a second database of the second entity.

10. A data synchronization apparatus, the apparatus comprising:

the first judging module is used for judging whether data updating exists in a first database of the first entity;

the export module is used for exporting the updated data from the first database if the first database is updated;

the storage module is used for carrying out cloud storage on the updated data;

the second judgment module is used for judging whether data updating needs to be carried out on a second database of the second entity;

the downloading module is used for downloading the updated data which is subjected to the cloud storage if the data of the second database of the second entity needs to be updated;

and the import module is used for importing the update data into a second database of a second entity.

11. An electronic device, comprising:

a processor; and

a memory having executable code stored thereon, which when executed by the processor, causes the processor to perform the method of any one of claims 1 to 9.

12. A computer readable storage medium having stored thereon executable code which, when executed by a processor of an electronic device, causes the processor to perform the method of any of claims 1 to 9.

Images