CN110633321B - Data synchronization method, device and equipment - Google Patents
Data synchronization method, device and equipment Download PDFInfo
- Publication number
- CN110633321B CN110633321B CN201910817901.2A CN201910817901A CN110633321B CN 110633321 B CN110633321 B CN 110633321B CN 201910817901 A CN201910817901 A CN 201910817901A CN 110633321 B CN110633321 B CN 110633321B
- Authority
- CN
- China
- Prior art keywords
- data
- synchronized
- data synchronization
- determining
- timestamp
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
- G06F16/275—Synchronous replication
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Computing Systems (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a data synchronization method, a device, equipment and a computer readable storage medium, wherein the method comprises the following steps: determining attribute information of data to be synchronized; wherein the attribute information includes at least one of a data volume of the data to be synchronized and an effective timestamp; determining a data synchronization mode based on the attribute information; and synchronizing the data to be synchronized to the target database based on the data synchronization mode.
Description
Technical Field
The present invention relates to the field of computer technologies, and in particular, to a method, an apparatus, and a device for synchronizing data.
Background
Currently, a large enterprise usually sets corresponding branches in different regions according to business requirements. Because modern enterprises require information-based unified management, the currently adopted scheme is a multi-database management mode based on the requirement, namely each branch office uses an independent database, and then data in the branch office database is synchronized with data in the main company database.
When the data in the branch database and the data in the head office database are synchronized, a fixed synchronization mode is usually adopted to synchronize the newly added data, and the synchronization cannot be performed according to the characteristics of the newly added data, so that the flexibility is poor.
Disclosure of Invention
Embodiments of the present invention provide a data synchronization method, apparatus, device, and computer readable storage medium, so as to solve the problem in the prior art that data synchronization cannot be performed according to the characteristics of newly added data, which results in poor flexibility.
In order to solve the technical problem, the invention is realized as follows:
in a first aspect, a method for synchronizing data is provided, where the method includes:
determining attribute information of data to be synchronized; wherein the attribute information includes at least one of a data amount and an effective timestamp of the data to be synchronized;
determining a data synchronization mode based on the attribute information;
and synchronizing the data to be synchronized to a target database based on the data synchronization mode.
In a second aspect, an apparatus for synchronizing data is provided, the apparatus including:
the first determining module is used for determining the attribute information of the data to be synchronized; wherein the attribute information comprises at least one of a data amount and an effective timestamp of the data to be synchronized;
the second determining module is used for determining a data synchronization mode based on the attribute information;
and the synchronization module is used for synchronizing the data to be synchronized into the target database based on the data synchronization mode.
In a third aspect, a terminal device is provided, the terminal device comprising a processor, a memory and a computer program stored on the memory and executable on the processor, the computer program, when executed by the processor, implementing the steps of the method according to the first aspect.
In a fourth aspect, a computer-readable storage medium is provided, on which a computer program is stored, which computer program, when being executed by a processor, carries out the steps of the method according to the first aspect.
In the embodiment of the present invention, attribute information of data to be synchronized may be determined, where the attribute information of the data to be synchronized may include at least one of a data amount and an effective timestamp of the data to be synchronized, and then, a data synchronization manner may be determined according to the attribute information of the data to be synchronized, and the data to be synchronized may be synchronized into the target database based on the determined data synchronization manner. As can be seen from the above, the embodiment of the present invention can flexibly select a data synchronization method suitable for the data to be synchronized according to the attribute information of the data to be synchronized, thereby effectively solving the problem of poor flexibility in the prior art.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the description serve to explain the invention and not to limit the invention. In the drawings:
fig. 1 is a schematic flow chart of a data synchronization method according to an embodiment of the present invention;
FIG. 2 is a block diagram of a data synchronization apparatus according to an embodiment of the present invention;
fig. 3 is a schematic hardware structure diagram of a data synchronization apparatus according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the technical solutions of the present invention will be clearly and completely described below with reference to the specific embodiments of the present invention and the accompanying drawings. It is to be understood that the described embodiments are merely exemplary of the invention, and not restrictive of the full scope of the invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The technical solutions provided by the embodiments of the present invention are described in detail below with reference to the accompanying drawings.
Referring to fig. 1, fig. 1 is a schematic flow chart of a data synchronization method according to an embodiment of the present invention, as shown in fig. 1, the method includes the following steps:
step 102: determining attribute information of data to be synchronized; wherein the attribute information includes at least one of a data amount of the data to be synchronized and an effective timestamp.
Step 104: and determining a data synchronization mode based on the attribute information.
Step 106: and synchronizing the data to be synchronized to the target database based on the data synchronization mode.
In the embodiment of the invention, the data to be synchronized can be obtained, and the attribute information of the data to be synchronized is determined. Wherein the attribute information of the data to be synchronized may include at least one of a data amount of the data to be synchronized and an effective timestamp.
Since determining the attribute information of the data to be synchronized according to the data to be synchronized is prior art, the embodiments of the present invention are not described herein again.
After determining the attribute information of the data to be synchronized, the data synchronization manner may be determined based on the attribute information of the data to be synchronized.
In an example, the attribute information of the data to be synchronized may include an effective timestamp of the data to be synchronized, and when the data synchronization manner is determined based on the attribute information of the data to be synchronized, the data synchronization manner may be specifically determined according to the effective timestamp of the data to be synchronized.
When the data synchronization mode is determined according to the effective timestamp of the data to be synchronized, the data synchronization mode can be determined according to the effective timestamp of the data to be synchronized and the current timestamp. Specifically, when the effective timestamp of the data to be synchronized is not later than the current timestamp, it may be determined that the data synchronization mode is data synchronization by a message queue; and when the effective timestamp of the data to be synchronized is later than the current timestamp, the data synchronization mode can be determined to be the mode of establishing batch data synchronization tasks for data synchronization.
Because the synchronization time required when data synchronization is carried out in a message queue mode is less than the synchronization time required when data synchronization is carried out in a mode of establishing batch data synchronization tasks, when the effective timestamp of the data to be synchronized is not later than the current timestamp, namely the data to be synchronized needs to take effect at the current real time, a data synchronization mode requiring less synchronization time can be selected, namely the data synchronization is carried out through the message queue, the data to be synchronized is synchronized to the target database quickly, and therefore the data to be synchronized can take effect in the target database quickly.
As can be seen from the above, when the data synchronization manner is determined according to the effective timestamp of the data to be synchronized, it can be ensured to a great extent that the data to be synchronized, which needs to be quickly effective or effective in real time, can be quickly synchronized into the target database, so that the data to be synchronized can be quickly effective in the target database. Therefore, the data synchronization method can be determined according to the time requirement of the data to be synchronized, which needs to be validated quickly or in real time.
It should be noted that, in an example, the data to be synchronized may also be policy data for personal safety, and then when the attribute information of the data to be synchronized includes an effective timestamp, it may be further detected whether the data to be synchronized includes the age and occupation of the insured person, and if the data to be synchronized includes the age and occupation of the insured person, a target effective timestamp may be determined according to the age and occupation of the insured person and the effective timestamp of the data to be synchronized, and when the target effective timestamp is later than the current timestamp, a data synchronization mode is determined to be data synchronization by establishing a batch data synchronization task.
When the target effective timestamp is determined according to the age and occupation of the insured person and the effective timestamp of the data to be synchronized, the age of the insured person and the safety factor corresponding to the occupation can be determined respectively, and then the target coefficient is determined according to the product of the safety factor corresponding to the age of the insured person and the safety factor corresponding to the occupation of the insured person. After determining the target coefficient, an effective time interval of the data to be synchronized may be determined according to an effective timestamp of the data to be synchronized and the generation timestamp. The target coefficient may then be multiplied by the determined validation interval to obtain a target validation interval. The target validation timestamp may be determined from the target validation interval and the generation timestamp of the data to be synchronized. For example, the target effective time interval may be 28 days, the generation timestamp of the data to be synchronized may be 1 month, 1 day, 8 hours in 2019, and the effective timestamp of the data to be synchronized may be 1 month, 29 days, 8 hours in 2019.
When determining the safety factor corresponding to the age and occupation of the insured person, the determination may be performed based on an age safety coefficient table and an occupation safety coefficient table which are generated in advance. Wherein, the age safety coefficient table may include safety coefficients corresponding to each age group; the occupational safety factor table may include a safety factor corresponding to each occupational. In this example, the pre-generated age safety coefficient table and occupational safety coefficient table may be set by the relevant person based on experience, or may be determined by each occupational and the number of insurance claims for personal safety corresponding to each age. The corresponding safety factors in the age safety factor table and the occupational safety factor table may be higher when the safety degree is higher, such as the more moderate age (e.g., 20 to 40 years), the safer the occupational; the corresponding safety factors in the age safety factor table and the occupational safety factor table may be lower when the safety level is lower, such as the greater or smaller the age and the more dangerous the occupational.
When the data to be synchronized is policy data for personal safety, the corresponding insurance observation periods may be different. For example, when the insured life is younger or is engaged in safer professions, the corresponding insurance observation period may be shorter, and at this time, the effective timestamp corresponding to the data to be synchronized, that is, the effective timestamp of the policy data for personal safety may be earlier; when the insured person is older or engaged in dangerous occupations, the corresponding insurance observation period can be longer, and at the moment, the effective timestamp corresponding to the data to be synchronized can be later. Therefore, if the data synchronization method of the data to be synchronized is determined only according to the effective timestamp and the current timestamp corresponding to the data to be synchronized, the data to be synchronized that should take effect earlier than the effective timestamp and that should take longer synchronization time before the current timestamp may be synchronized into the target database by the data synchronization method, so that the data to be synchronized cannot take effect in time.
In this example, the target effective timestamp of the data to be synchronized may be determined according to the age and occupation of the insured person, and the data synchronization method of the data to be synchronized may be determined according to the target timestamp and the current time, so that the above problem may be effectively avoided, and the data synchronization method more suitable for the data to be synchronized may be determined.
It should be noted that the above-mentioned policy data for personal safety is only one application scenario of the data to be synchronized in the embodiment of the present invention, and does not limit the data to be synchronized.
In another example, the attribute information of the data to be synchronized may include the data amount of the data to be synchronized and not include the effective timestamp of the data to be synchronized, and when the data synchronization manner is determined based on the attribute information of the data to be synchronized, the data synchronization manner may be specifically determined according to the data amount of the data to be synchronized.
When the data synchronization mode is determined according to the data volume of the data to be synchronized, the data synchronization mode can be determined according to the data volume of the data to be synchronized and a preset value. Specifically, when the data volume of the data to be synchronized exceeds a preset value, the data synchronization mode can be determined to be data synchronization by establishing a batch data synchronization task; and when the data volume of the data to be synchronized does not exceed the preset value, determining that the data synchronization mode is data synchronization through a message queue.
Because the efficiency of synchronizing the batch data when the data synchronization is performed by establishing the batch data synchronization task is higher than the efficiency of synchronizing the batch data when the data synchronization is performed by using the message queue, when the data amount of the data to be synchronized exceeds a preset value, namely the data amount of the data to be synchronized is large, a data synchronization mode with high transmission efficiency for a large amount of data can be selected, namely the data synchronization mode for synchronizing the data to be synchronized by establishing the batch data synchronization task is used for efficiently synchronizing the data to be synchronized to the target database.
As can be seen from the above, when the data synchronization method is determined according to the data amount of the data to be synchronized, the data to be synchronized with a large data amount can be efficiently synchronized into the target database. Therefore, the corresponding data synchronization mode can be determined according to the requirement of the data to be synchronized with large data quantity on transmission efficiency.
In the embodiment of the invention, when data synchronization is performed on the data to be synchronized in a message queue manner, the data to be synchronized can be sequentially added to the message queue, so that the data to be synchronized is synchronized to the target database through the message queue. Specifically, the Data to be synchronized may be acquired from the source database by using a database CDC (Change Data Capture) technique, and then the Data to be synchronized is sequentially added to the message queue, so that the Data to be synchronized may be synchronized to the target database through the message queue.
When data synchronization is performed on data to be synchronized by means of establishing a batch data synchronization task, the batch data synchronization task can be established based on a button tool, and the data to be synchronized is synchronized into a target database through the established batch data synchronization task.
In this example, the service type corresponding to the data to be synchronized may also be determined, and whether the data to be synchronized is data that needs to be periodically synchronized is determined according to the service type corresponding to the data to be synchronized. If the data is data which needs to be periodically synchronized, a script for periodically triggering task execution can be generated according to a synchronization period corresponding to the data to be synchronized and an address of a target database, and then the batch data synchronization task can be periodically triggered according to the script of the periodically triggered task so as to perform periodic data synchronization subsequently.
In this example, an exception reminding mechanism, such as an exception mail notification, may also be added to the established batch data synchronization task. When the abnormal reminding is performed, specific abnormal links, abnormal reasons and the like can be reminded, for example, the abnormal reminding is performed by setting the names of the abnormal links, the abnormal reasons and the like as mail titles, so that the synchronous process can be better monitored.
In this example, the data to be synchronized may further include a version identification number, and therefore, when the data to be synchronized is synchronized, the data to be synchronized may be divided into different versions of data according to the version identification number of the data to be synchronized, and then the different versions of data are synchronized into the target database according to attribute information other than the version identification number, such as an effective timestamp.
In the embodiment of the present invention, attribute information of data to be synchronized may be determined, where the attribute information of the data to be synchronized may include at least one of a data amount and an effective timestamp of the data to be synchronized, and then, a data synchronization manner may be determined according to the attribute information of the data to be synchronized, and the data to be synchronized may be synchronized into the target database based on the determined data synchronization manner. As can be seen from the above, the embodiment of the present invention can flexibly select a data synchronization method suitable for the data to be synchronized according to the attribute information of the data to be synchronized, thereby effectively solving the problem of poor flexibility in the prior art.
Corresponding to the above data synchronization method, an embodiment of the present invention further provides a data synchronization apparatus, fig. 2 is a schematic diagram of module composition of the data synchronization apparatus provided in the embodiment of the present invention, and as shown in fig. 2, the data synchronization apparatus includes:
a first determining module 21, configured to determine attribute information of data to be synchronized; wherein the attribute information includes at least one of a data amount and an effective timestamp of the data to be synchronized;
a second determination module 22, which determines a data synchronization mode based on the attribute information;
and the synchronization module 23 is configured to synchronize the data to be synchronized to a target database based on the data synchronization manner.
Optionally, if the attribute information includes an effective timestamp of the data to be synchronized, the second determining module 22 is configured to:
determining a data synchronization mode based on the effective timestamp of the data to be synchronized;
if the attribute information includes the data size of the data to be synchronized and does not include the effective timestamp of the data to be synchronized, the second determining module 22 is configured to:
and determining a data synchronization mode based on the data volume of the data to be synchronized.
Optionally, the second determining module 22 is configured to:
when the effective timestamp of the data to be synchronized is not later than the current timestamp, determining that the data synchronization mode is data synchronization through a message queue;
and when the effective timestamp of the data to be synchronized is later than the current timestamp, determining that the data synchronization mode is the mode of establishing batch data synchronization tasks for data synchronization.
Optionally, the second determining module 22 is configured to:
when the data volume of the data to be synchronized exceeds a preset value, determining that a data synchronization mode is a mode of establishing batch data synchronization tasks for data synchronization;
and when the data volume of the data to be synchronized does not exceed a preset value, determining that the data synchronization mode is data synchronization through a message queue mode.
Optionally, the second determining module 22 is configured to:
and establishing a batch data synchronization task based on a button tool, so that the data to be synchronized is synchronized to the target database through the established batch data synchronization task.
Optionally, the second determining module 22 is configured to:
and sequentially adding the data to be synchronized to a message queue so as to synchronize the data to be synchronized to the target database through the message queue.
Optionally, the data to be synchronized is policy data for personal safety; the second determining module 22 is configured to:
detecting whether the data to be synchronized contains the age and occupation of the insured person;
if the age and occupation of the insured person are included, determining a target effective time stamp according to the age and occupation of the insured person and the effective time stamp of the data to be synchronized;
and when the target effective timestamp is later than the current timestamp, determining that the data synchronization mode is the mode of establishing batch data synchronization tasks for data synchronization.
In the embodiment of the present invention, attribute information of data to be synchronized may be determined, where the attribute information of the data to be synchronized may include at least one of a data amount and an effective timestamp of the data to be synchronized, and then, a data synchronization manner may be determined according to the attribute information of the data to be synchronized, and the data to be synchronized may be synchronized into the target database based on the determined data synchronization manner. As can be seen from the above, the embodiment of the present invention can flexibly select a data synchronization method suitable for the data to be synchronized according to the attribute information of the data to be synchronized, thereby effectively solving the problem of poor flexibility in the prior art.
Corresponding to the above data synchronization method, an embodiment of the present invention further provides a data synchronization device, and fig. 3 is a schematic diagram of a hardware structure of the data synchronization device according to an embodiment of the present invention.
The data synchronization device may be a terminal device or a server for synchronizing data provided in the above embodiments.
The data synchronization device may have a large difference due to different configurations or performances, and may include one or more processors 301 and a memory 302, and the memory 302 may store one or more stored applications or data. Memory 302 may be, among other things, transient storage or persistent storage. The application program stored in memory 302 may include one or more modules (not shown), each of which may include a series of computer-executable instructions in a synchronization device for data. Still further, the processor 301 may be configured to communicate with the memory 302 to execute a series of computer-executable instructions in the memory 302 on a device for synchronizing data. The apparatus for synchronizing data may also include one or more power supplies 303, one or more wired or wireless network interfaces 304, one or more input-output interfaces 305, one or more keyboards 306.
In particular, in this embodiment, the apparatus for synchronizing data includes a memory, and one or more programs, wherein the one or more programs are stored in the memory, and the one or more programs may include one or more modules, and each module may include a series of computer-executable instructions for the apparatus for synchronizing data, and the one or more programs configured to be executed by the one or more processors include computer-executable instructions for:
determining attribute information of data to be synchronized; wherein the attribute information includes at least one of a data amount and an effective timestamp of the data to be synchronized;
determining a data synchronization mode based on the attribute information;
and synchronizing the data to be synchronized to a target database based on the data synchronization mode.
Optionally, if the attribute information includes an effective timestamp of the data to be synchronized, the computer-executable instructions, when executed, may cause the processor to:
determining a data synchronization mode based on the effective timestamp of the data to be synchronized;
if the attribute information includes the data amount of the data to be synchronized and does not include the effective timestamp of the data to be synchronized, the computer-executable instructions, when executed, may cause the processor to;
and determining a data synchronization mode based on the data volume of the data to be synchronized.
Optionally, the computer executable instructions, when executed, may cause the processor to:
when the effective timestamp of the data to be synchronized is not later than the current timestamp, determining that the data synchronization mode is data synchronization in a message queue mode;
and when the effective timestamp of the data to be synchronized is later than the current timestamp, determining that the data synchronization mode is the mode of establishing batch data synchronization tasks for data synchronization.
Optionally, the computer executable instructions, when executed, may cause the processor to:
when the data volume of the data to be synchronized exceeds a preset value, determining that a data synchronization mode is a mode of establishing batch data synchronization tasks for data synchronization;
and when the data volume of the data to be synchronized does not exceed a preset value, determining that the data synchronization mode is data synchronization through a message queue mode.
Optionally, the computer executable instructions, when executed, may cause the processor to:
and establishing a batch data synchronization task based on a button tool, so that the data to be synchronized is synchronized to the target database through the established batch data synchronization task.
Optionally, the computer executable instructions, when executed, may cause the processor to:
and sequentially adding the data to be synchronized to a message queue so as to synchronize the data to be synchronized to the target database through the message queue.
Optionally, the data to be synchronized is warranty data for personal safety, and the computer executable instructions, when executed, may cause the processor to:
detecting whether the data to be synchronized contains the age and occupation of the insured person or not;
if the age and occupation of the insured person are included, determining a target effective time stamp according to the age and occupation of the insured person and the effective time stamp of the data to be synchronized;
and when the target effective timestamp is later than the current timestamp, determining that the data synchronization mode is the mode of establishing batch data synchronization tasks for data synchronization.
In the embodiment of the present invention, attribute information of data to be synchronized may be determined, where the attribute information of the data to be synchronized may include at least one of a data amount and an effective timestamp of the data to be synchronized, and then, a data synchronization manner may be determined according to the attribute information of the data to be synchronized, and the data to be synchronized may be synchronized into the target database based on the determined data synchronization manner. As can be seen from the above, the embodiment of the present invention can flexibly select a data synchronization method suitable for the data to be synchronized according to the attribute information of the data to be synchronized, thereby effectively solving the problem of poor flexibility in the prior art.
In the 90 s of the 20 th century, improvements in a technology could clearly distinguish between improvements in hardware (e.g., improvements in circuit structures such as diodes, transistors, switches, etc.) and improvements in software (improvements in process flow). However, as technology advances, many of today's process flow improvements have been seen as direct improvements in hardware circuit architecture. Designers almost always obtain the corresponding hardware circuit structure by programming an improved method flow into the hardware circuit. Thus, it cannot be said that an improvement in the process flow cannot be realized by hardware physical modules. For example, a Programmable Logic Device (PLD), such as a Field Programmable Gate Array (FPGA), is an integrated circuit whose Logic functions are determined by programming the Device by a user. A digital system is "integrated" on a PLD by the designer's own programming without requiring the chip manufacturer to design and fabricate application-specific integrated circuit chips. Furthermore, nowadays, instead of manually making an Integrated Circuit chip, such Programming is often implemented by "logic compiler" software, which is similar to a software compiler used in program development and writing, but the original code before compiling is also written by a specific Programming Language, which is called Hardware Description Language (HDL), and HDL is not only one but many, such as abel (advanced Boolean Expression Language), ahdl (alternate Hardware Description Language), traffic, pl (core universal Programming Language), HDCal (jhdware Description Language), lang, Lola, HDL, laspam, hardward Description Language (vhr Description Language), vhal (Hardware Description Language), and vhigh-Language, which are currently used in most common. It will also be apparent to those skilled in the art that hardware circuitry that implements the logical method flows can be readily obtained by merely slightly programming the method flows into an integrated circuit using the hardware description languages described above.
The controller may be implemented in any suitable manner, for example, the controller may take the form of, for example, a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro) processor, logic gates, switches, an Application Specific Integrated Circuit (ASIC), a programmable logic controller, and an embedded microcontroller, examples of which include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320, the memory controller may also be implemented as part of the control logic of the memory. Those skilled in the art will also appreciate that, in addition to implementing the controller in purely computer readable program code means, the same functionality can be implemented by logically programming method steps such that the controller is in the form of logic gates, switches, application specific integrated circuits, programmable logic controllers, embedded microcontrollers and the like. Such a controller may thus be considered a hardware component, and the means included therein for performing the various functions may also be considered as a structure within the hardware component. Or even means for performing the functions may be regarded as being both a software module for performing the method and a structure within a hardware component.
The systems, devices, modules or units illustrated in the above embodiments may be implemented by a computer chip or an entity, or by a product with certain functions. One typical implementation device is a computer. In particular, the computer may be, for example, a personal computer, a laptop computer, a cellular telephone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.
For convenience of description, the above devices are described as being divided into various units by function, and are described separately. Of course, the functions of the units may be implemented in the same software and/or hardware or in a plurality of software and/or hardware when implementing the invention.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
In a typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
The memory may include forms of volatile memory in a computer readable medium, Random Access Memory (RAM) and/or non-volatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of a computer-readable medium.
Computer-readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, a computer readable medium does not include a transitory computer readable medium such as a modulated data signal and a carrier wave.
It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The invention may be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The invention may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media including memory storage devices.
The embodiments in the present specification are described in a progressive manner, and the same and similar parts among the embodiments are referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, for the system embodiment, since it is substantially similar to the method embodiment, the description is simple, and for the relevant points, reference may be made to the partial description of the method embodiment.
The above description is only an example of the present invention, and is not intended to limit the present invention. Various modifications and alterations to this invention will become apparent to those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the scope of the claims of the present invention.
Claims (7)
1. A method for synchronizing data, the method comprising:
determining attribute information of data to be synchronized; wherein the attribute information includes at least one of a data amount and an effective timestamp of the data to be synchronized;
if the attribute information comprises the effective timestamp of the data to be synchronized, determining a data synchronization mode based on the effective timestamp of the data to be synchronized;
if the attribute information comprises the data volume of the data to be synchronized and does not comprise the effective timestamp of the data to be synchronized, determining a data synchronization mode based on the data volume of the data to be synchronized;
synchronizing the data to be synchronized to a target database based on the data synchronization mode;
the determining a data synchronization mode based on the effective timestamp of the data to be synchronized includes:
when the effective timestamp of the data to be synchronized is not later than the current timestamp, determining that the data synchronization mode is data synchronization through a message queue;
when the effective timestamp of the data to be synchronized is later than the current timestamp, determining that the data synchronization mode is data synchronization by establishing a batch data synchronization task;
the data to be synchronized is policy data aiming at personal safety; when the effective timestamp of the data to be synchronized is later than the current timestamp, determining that the data synchronization mode is data synchronization by establishing a batch data synchronization task, wherein the data synchronization method comprises the following steps:
detecting whether the data to be synchronized contains the age and occupation of the insured person;
if the age and occupation of the insured person are included, determining a target effective time stamp according to the age and occupation of the insured person and the effective time stamp of the data to be synchronized;
and when the target effective timestamp is later than the current timestamp, determining that the data synchronization mode is the mode of establishing batch data synchronization tasks for data synchronization.
2. The method according to claim 1, wherein the determining a data synchronization mode based on the data amount of the data to be synchronized comprises:
when the data volume of the data to be synchronized exceeds a preset value, determining that a data synchronization mode is a mode of establishing batch data synchronization tasks for data synchronization;
and when the data volume of the data to be synchronized does not exceed a preset value, determining that the data synchronization mode is data synchronization through a message queue mode.
3. The method of claim 1 or 2, wherein the data synchronization by establishing a batch data synchronization task comprises:
and establishing a batch data synchronization task based on a button tool, so that the data to be synchronized is synchronized to the target database through the established batch data synchronization task.
4. The method according to claim 1 or 2, wherein the data synchronization by means of a message queue comprises:
adding the data to be synchronized to a message queue so as to synchronize the data to be synchronized to the target database through the message queue.
5. An apparatus for synchronizing data, the apparatus comprising:
the first determining module is used for determining the attribute information of the data to be synchronized; wherein the attribute information includes at least one of a data amount and an effective timestamp of the data to be synchronized;
a second determining module, configured to determine a data synchronization manner based on the effective timestamp of the data to be synchronized if the attribute information includes the effective timestamp of the data to be synchronized; if the attribute information comprises the data volume of the data to be synchronized and does not comprise the effective timestamp of the data to be synchronized, determining a data synchronization mode based on the data volume of the data to be synchronized;
the synchronization module is used for synchronizing the data to be synchronized into a target database based on the data synchronization mode;
the second determining module is used for determining that the data synchronization mode is data synchronization through a message queue mode when the effective timestamp of the data to be synchronized is not later than the current timestamp; when the effective timestamp of the data to be synchronized is later than the current timestamp, determining that the data synchronization mode is data synchronization by establishing a batch data synchronization task;
the data to be synchronized is policy data aiming at personal safety, and the second determining module is used for detecting whether the data to be synchronized contains the age and occupation of the insured person; if the age and occupation of the insured person are included, determining a target effective time stamp according to the age and occupation of the insured person and the effective time stamp of the data to be synchronized; and when the target effective timestamp is later than the current timestamp, determining that the data synchronization mode is the mode of establishing batch data synchronization tasks for data synchronization.
6. A terminal device, comprising: memory, processor and computer program stored on the memory and executable on the processor, which computer program, when executed by the processor, carries out the steps of the method according to any one of claims 1 to 4.
7. A computer-readable storage medium, characterized in that a computer program is stored on the computer-readable storage medium, which computer program, when being executed by a processor, carries out the steps of the method according to any one of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910817901.2A CN110633321B (en) | 2019-08-30 | 2019-08-30 | Data synchronization method, device and equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910817901.2A CN110633321B (en) | 2019-08-30 | 2019-08-30 | Data synchronization method, device and equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110633321A CN110633321A (en) | 2019-12-31 |
| CN110633321B true CN110633321B (en) | 2022-06-03 |
Family
ID=68969804
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910817901.2A Active CN110633321B (en) | 2019-08-30 | 2019-08-30 | Data synchronization method, device and equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110633321B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113821409A (en) * | 2021-09-23 | 2021-12-21 | 中国建设银行股份有限公司 | Method, device, storage medium and equipment for monitoring data transmission |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106528893A (en) * | 2016-12-26 | 2017-03-22 | 北京奇虎科技有限公司 | Data synchronization method and device |
| CN109492018A (en) * | 2018-09-12 | 2019-03-19 | 武汉达梦数据库有限公司 | A kind of adaptive dynamic adjusting method of data synchronous system and device |
| CN110032558A (en) * | 2019-04-12 | 2019-07-19 | 重庆天蓬网络有限公司 | A kind of method of data synchronization, device, system and storage medium |
| CN110083579A (en) * | 2019-03-21 | 2019-08-02 | 深圳壹账通智能科技有限公司 | Incremental data synchronous method, apparatus, computer equipment and computer storage medium |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7613739B2 (en) * | 2005-11-17 | 2009-11-03 | Research In Motion Limited | Method and apparatus for synchronizing databases connected by wireless interface |
-
2019
- 2019-08-30 CN CN201910817901.2A patent/CN110633321B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106528893A (en) * | 2016-12-26 | 2017-03-22 | 北京奇虎科技有限公司 | Data synchronization method and device |
| CN109492018A (en) * | 2018-09-12 | 2019-03-19 | 武汉达梦数据库有限公司 | A kind of adaptive dynamic adjusting method of data synchronous system and device |
| CN110083579A (en) * | 2019-03-21 | 2019-08-02 | 深圳壹账通智能科技有限公司 | Incremental data synchronous method, apparatus, computer equipment and computer storage medium |
| CN110032558A (en) * | 2019-04-12 | 2019-07-19 | 重庆天蓬网络有限公司 | A kind of method of data synchronization, device, system and storage medium |
Non-Patent Citations (2)
| Title |
|---|
| 一种由智能终端控制的数据同步算法;李立亚等;《西安电子科技大学学报(自然科学版)》;20130430;第40卷(第2期);172-180,212 * |
| 基于JSON的离线数据同步策略及应用;程鑫鑫等;《计算机与现代化》;20171215;第26卷(第12期);257-261 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110633321A (en) | 2019-12-31 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107450979B (en) | Block chain consensus method and device | |
| CN107391526B (en) | Data processing method and device based on block chain | |
| CN107391527B (en) | Data processing method and device based on block chain | |
| CN107577697B (en) | Data processing method, device and equipment | |
| CN109033127B (en) | Synchronous data verification method, device and equipment | |
| CN107578338B (en) | Service publishing method, device and equipment | |
| CN110263050B (en) | Data processing method, device, equipment and storage medium | |
| CN111752633A (en) | Code loading method, device and equipment | |
| CN110807013B (en) | Data migration method and device for distributed data storage cluster | |
| CN111680960A (en) | Attendance statistical method and equipment | |
| CN112631679A (en) | Preloading method and device for micro-application | |
| CN111459724B (en) | Node switching method, device, equipment and computer readable storage medium | |
| CN110602163B (en) | File uploading method and device | |
| CN111930343A (en) | Method, device and medium for acquiring random number and generating random number | |
| CN108595315B (en) | Log collection method, device and equipment | |
| CN110633321B (en) | Data synchronization method, device and equipment | |
| CN109118361B (en) | Method, device and system for managing limit | |
| CN114115016A (en) | Data processing method, device, equipment and medium | |
| CN109446271B (en) | Data synchronization method, device, equipment and medium | |
| CN109582388B (en) | Parameter configuration method, device and equipment | |
| CN113961523A (en) | Business file splitting and summarizing method, device and equipment | |
| CN113254163B (en) | Processing method and device of block chain data | |
| CN110019975B (en) | Random walk, random walk method based on cluster, random walk device and equipment | |
| CN110908429A (en) | Timer operation method and device | |
| CN109614388B (en) | Budget deduction method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |