CN107689998A - A kind of incremental data synchronization method and terminal device - Google Patents

A kind of incremental data synchronization method and terminal device Download PDF

Info

Publication number
CN107689998A
CN107689998A CN201710825934.2A CN201710825934A CN107689998A CN 107689998 A CN107689998 A CN 107689998A CN 201710825934 A CN201710825934 A CN 201710825934A CN 107689998 A CN107689998 A CN 107689998A
Authority
CN
China
Prior art keywords
incremental
incremental data
system
record
data
Prior art date
Application number
CN201710825934.2A
Other languages
Chinese (zh)
Inventor
覃娆
覃无愧
Original Assignee
平安科技(深圳)有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 平安科技(深圳)有限公司 filed Critical 平安科技(深圳)有限公司
Priority to CN201710825934.2A priority Critical patent/CN107689998A/en
Publication of CN107689998A publication Critical patent/CN107689998A/en

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network-specific arrangements or communication protocols supporting networked applications
    • H04L67/10Network-specific arrangements or communication protocols supporting networked applications in which an application is distributed across nodes in the network
    • H04L67/1095Network-specific arrangements or communication protocols supporting networked applications in which an application is distributed across nodes in the network for supporting replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes or user terminals or syncML

Abstract

The invention belongs to field of computer technology, more particularly to a kind of incremental data synchronization method and terminal device.Methods described includes:The first incremental data is obtained from each service sub-system respectively, first incremental data is each service sub-system user data actually newly-increased in default first time period;Each first incremental data of acquisition is collected, obtains the second incremental data, the user data that second incremental data should increase newly for each service sub-system in the first time period;Each first incremental data is compared with second incremental data respectively, obtains the incremental record of each service sub-system missing, the incremental record is the elementary cell of incremental data;The incremental record of missing is added separately in corresponding service sub-system, to complete the synchronization to the incremental data of each service sub-system.By the present invention, the uniformity of the incremental data between each service sub-system ensure that.

Description

A kind of incremental data synchronization method and terminal device

Technical field

The invention belongs to field of computer technology, more particularly to a kind of incremental data synchronization method and terminal device.

Background technology

Current financial business subsystem is in large scale, complicated, often classifies according to business and draws whole system It is divided into multiple service sub-systems, therefore, the incremental data of user generally requires to synchronize in multiple service sub-systems, at present Mode mainly by interface interchange mode to the real-time recording increment data of multiple different service sub-systems, data syn-chronization Uniformity is difficult to ensure that error rate is higher.

The content of the invention

In view of this, the embodiments of the invention provide a kind of incremental data synchronization method and terminal device, to solve to pass through The uniformity of existing data syn-chronization is difficult during real-time to the multiple different service sub-systems recording increment data of interface interchange mode To ensure, the problem of error rate is higher.

The first aspect of the embodiment of the present invention provides a kind of incremental data synchronization method, can include:

The first incremental data is obtained from each service sub-system respectively, first incremental data is each business Subsystem user data actually newly-increased in default first time period;

Each first incremental data of acquisition is collected, obtains the second incremental data, second incremental number According to the user data that should be increased newly in the first time period for each service sub-system;

Each first incremental data is compared with second incremental data respectively, obtains each business The incremental record of subsystem missing, the incremental record are the elementary cell of incremental data, and every incremental record includes an industry Business information;

The incremental record of missing is added separately in corresponding service sub-system, to complete to each business The synchronization of the incremental data of subsystem.

The second aspect of the embodiment of the present invention provides a kind of computer-readable recording medium, the computer-readable storage Media storage has computer program, and the computer program realizes following steps when being executed by processor:

The first incremental data is obtained from each service sub-system respectively, first incremental data is each business Subsystem user data actually newly-increased in default first time period;

Each first incremental data of acquisition is collected, obtains the second incremental data, second incremental number According to the user data that should be increased newly in the first time period for each service sub-system;

Each first incremental data is compared with second incremental data respectively, obtains each business The incremental record of subsystem missing, the incremental record are the elementary cell of incremental data, and every incremental record includes an industry Business information;

The incremental record of missing is added separately in corresponding service sub-system, to complete to each business The synchronization of the incremental data of subsystem.

The third aspect of the embodiment of the present invention provides a kind of incremental data synchronous terminal equipment, including memory, processing Device and the computer program that can be run in the memory and on the processor is stored in, described in the computing device Following steps are realized during computer program:

The first incremental data is obtained from each service sub-system respectively, first incremental data is each business Subsystem user data actually newly-increased in default first time period;

Each first incremental data of acquisition is collected, obtains the second incremental data, second incremental number According to the user data that should be increased newly in the first time period for each service sub-system;

Each first incremental data is compared with second incremental data respectively, obtains each business The incremental record of subsystem missing, the incremental record are the elementary cell of incremental data, and every incremental record includes an industry Business information;

The incremental record of missing is added separately in corresponding service sub-system, to complete to each business The synchronization of the incremental data of subsystem.

Existing beneficial effect is the embodiment of the present invention compared with prior art:By the embodiment of the present invention, distinguish first Incremental data is obtained from each service sub-system, then it is collected, the incremental data collected provides synchronous mark Standard, the incremental data of each service sub-system are compared therewith respectively, it can be deduced that it is existing between the synchronization criterion Difference records, because these differences are obtained based on same synchronization criterion, as long as by each service sub-system accordingly Processing is synchronized to respective incremental data, so that it may it is consistent to ensure the incremental data between them, and because this is same Step standard summarizes the data of each service sub-system, greatly ensure that its accuracy, can be dropped significantly using it as reference Error rate when low synchronous.

Brief description of the drawings

Technical scheme in order to illustrate the embodiments of the present invention more clearly, below will be to embodiment or description of the prior art In the required accompanying drawing used be briefly described, it should be apparent that, drawings in the following description be only the present invention some Embodiment, for those of ordinary skill in the art, without having to pay creative labor, can also be according to these Accompanying drawing obtains other accompanying drawings.

Fig. 1 is a kind of schematic diagram of implementation environment in the embodiment of the present invention;

Fig. 2 is a kind of one embodiment flow chart of incremental data synchronization method in the embodiment of the present invention;

Fig. 3 is that a kind of incremental data synchronization method is considering one under the scene of error in data be present in the embodiment of the present invention Individual embodiment flow chart;

Fig. 4 is to determine to treat the schematic flow diagram of Check System in the embodiment of the present invention;

Fig. 5 is a kind of schematic block diagram of incremental data synchronous terminal equipment provided in an embodiment of the present invention;

Fig. 6 is a kind of Program modual graph of incremental data synchronization program provided in an embodiment of the present invention.

Embodiment

To enable goal of the invention, feature, the advantage of the present invention more obvious and understandable, below in conjunction with the present invention Accompanying drawing in embodiment, the technical scheme in the embodiment of the present invention is clearly and completely described, it is clear that disclosed below Embodiment be only part of the embodiment of the present invention, and not all embodiment.Based on the embodiment in the present invention, this area All other embodiment that those of ordinary skill is obtained under the premise of creative work is not made, belongs to protection of the present invention Scope.

Fig. 1 show a kind of schematic diagram of implementation environment of the embodiment of the present invention, the executive agent in the embodiment of the present invention System of Synchronous Processing as shown in figure, System of Synchronous Processing are connected with each service sub-system, read each business subsystem The incremental data of system, and it is collected, then by the incremental data of each service sub-system and the incremental data pair collected Than obtaining the incremental data of each service sub-system missing, being finally added separately to the incremental record of missing corresponding In service sub-system, to complete the synchronization to the incremental data of each service sub-system.

Referring to Fig. 2, a kind of one embodiment of incremental data synchronization method can include in the embodiment of the present invention:

Step S201, the first incremental data is obtained from each service sub-system respectively.

First incremental data is each service sub-system use actually newly-increased in default first time period User data, in the present embodiment, System of Synchronous Processing can be at default first moment respectively from default first storage location The first incremental data of each service sub-system is read, first incremental data is by each service sub-system default Second moment was extracted and was stored into first storage location respectively.

For example, certain company's routine carries out the synchronization of an incremental data in daily morning, each service sub-system is in T+1 days Morning 00:05:00 (i.e. described second moment) started timed task, by T days 00 in respective system:00:00 to 24:00:00 (i.e. The first time period) incremental data (i.e. described first incremental data) extract, be stored into first storage location. System of Synchronous Processing is in T+1 mornings day 02:00:00 (i.e. described first moment) started timed task, from the described first storage position Put and read first incremental data.To ensure that System of Synchronous Processing can successfully read out data, when avoiding the occurrence of reading In first storage location, there has been no the situation that data are stored in, first moment should be later than second moment, and described Interval between first moment and second moment should be greater than default time interval, and the specific value of the time interval can To be set according to actual conditions, it is defined by ensureing that service sub-system each enough completes the process such as data pick-up and storage.

Step S202, each first incremental data of acquisition is collected, obtains the second incremental data.

The user that second incremental data should increase newly for each service sub-system in the first time period Data.

In the present embodiment, incremental data is made up of several incremental records, and the incremental record is incremental data Elementary cell, every incremental record includes a business information, for example, every incremental record can represent user and product account A binding relationship between family.

When each first incremental data to reading collects, each incremental data can be regarded as one Set, incremental record is element therein, then tries to achieve these union of sets collection, namely second incremental data.Specifically, All incremental records can be first extracted, and are formed an incremental data, then, to wherein including identical services information Multiple incremental records only retain one of incremental record, delete the incremental record of other repetitions.

For example, if the first incremental data of service sub-system 1 includes:Incremental record 1, incremental record 2, the and of incremental record 3 Incremental record 4, the first incremental data of service sub-system 2 include:Incremental record 1, incremental record 4, incremental record 5 and increment note Record 6, then the second incremental data obtained after collecting includes:Incremental record 1, incremental record 2, incremental record 3, incremental record 4th, incremental record 5 and incremental record 6.

Step S203, each first incremental data is compared with second incremental data respectively, obtained each The incremental record of the individual service sub-system missing.

In above-mentioned example, the first incremental data of service sub-system 1 is compared with second incremental data, The incremental record of obtained missing is incremental record 5 and incremental record 6, by the first incremental data of service sub-system 2 with it is described Second incremental data is compared, and the incremental record of obtained missing is incremental record 2 and incremental record 3.

Step S204, the incremental record of missing is added separately in corresponding service sub-system, to complete to each The synchronization of the incremental data of the individual service sub-system.

Specifically, the incremental record of missing can be stored into by System of Synchronous Processing respectively at default 3rd moment Default second storage location, so as to each service sub-system at default 4th moment respectively from the described second storage position The incremental record for reading and each lacking is put, and the incremental record of missing is added to corresponding first incremental data In, obtain the incremental data after synchronization.

Ground is readily appreciated that, the 3rd moment should be later than first moment, and when first moment and the described 3rd Interval between quarter should be greater than certain time interval, to ensure enough System of Synchronous Processing completion readings, collect and than equity Process is defined, and the 4th moment should be later than the 3rd moment, and between the 3rd moment and the 4th moment between Every should be greater than certain time interval, it is defined by ensureing that enough System of Synchronous Processing complete storing process.

For example, System of Synchronous Processing is in T+1 mornings day 02:00:00 (i.e. described first moment) started timed task and carried out Read, collect and than equivalent process, and in T+1 mornings day 05:00:00 (i.e. described 3rd moment) started timed task and deposited Storage process, each service sub-system is in T+1 mornings day 06:00:00 (i.e. described 4th moment) started timed task and is read out And the process such as addition.

It is this precondition that the data based on service sub-system are trust data in embodiment more than.But In actual applications, the data of service sub-system be probably can exist mistake, it is necessary to be taken in it.Referring to Fig. 3, Considering to exist one embodiment under the scene of error in data can include:

Step S301, the first incremental data is obtained from each service sub-system respectively.

Step S301 is identical with the step S201 in above-described embodiment, specifically can refer to the description in above-described embodiment, this Embodiment repeats no more to this.

Step S302, each first incremental data of acquisition is collected, obtains the second incremental data.

Step S302 is identical with the step S202 in above-described embodiment, specifically can refer to the description in above-described embodiment, this Embodiment repeats no more to this.

Step S303, second incremental data is calculated according to the data reliability of default each service sub-system In each bar incremental record confidence level.

The data reliability of each service sub-system is determined according to historical statistical data, if in historical statistics number In, the operation that service sub-system performs modification to incremental data (includes operation and the deletion incremental record of addition incremental record Operation) it is more, then illustrate its data occur mistake probability it is higher, its data reliability is lower.The number of service sub-system It is [0,1] according to the span of confidence level, takes its data of 0 interval scale completely insincere, takes its data of 1 interval scale completely credible.

In the present embodiment, the confidence level of incremental record can calculate in the following way:It will appear from this incremental record Each service sub-system data reliability summation, then divided by service sub-system total number, the business of last gained is should The confidence level of incremental record, similarly, the span of the confidence level of incremental record is [0,1], takes its data of 0 interval scale complete It is insincere, take its data of 1 interval scale completely credible.

If for example, share 3 service sub-systems, respectively service sub-system 1, service sub-system 2, service sub-system 3, its Data reliability is 0.9,0.8,0.7.Wherein, incremental record 1 is appeared in service sub-system 2 and service sub-system 3, but is not had Appear in service sub-system 1, then the confidence level of incremental record 1 is just (0.8+0.7)/3=0.5.

Step S304, from incremental record described in each bar in second incremental data extracting the confidence level is more than The preferred record of default Second Threshold.

The value of the Second Threshold can be configured according to actual conditions, and the present embodiment is not especially limited to this, But should be noted that if its value is bigger, the preferred record extracted is fewer, if its value is smaller, what is extracted is excellent Choosing record is more.

It is alternatively possible to simplified using the one kind of following processing mode as step S303 and step S304:Unite respectively Count the total degree that each bar incremental record in second incremental data occurs in first incremental data;From described second The preferred record that the total degree is more than default first threshold, institute are extracted in incremental record described in each bar in incremental data First threshold is stated to be determined by the total number of the service sub-system.If for example, the total number of service sub-system be 10, first threshold The half of the total number is arranged to, namely value is 5, the total degree that incremental record 1 occurs in the first incremental data is 4, is increased The total degree that amount record 2 occurs in the first incremental data is 6, then incremental record 2 is preferably to record, and incremental record 1 is not It is preferred that record.

Step S305, by second incremental data of the preferred record composition described in each bar extracted after preferred.

Compared to the second initial incremental data, preferably after the second incremental data in only remain the higher increasing of confidence level Amount record, the synchronized result obtained accordingly are more accurate.

Step S306, by each first incremental data respectively with preferably after second incremental data compared It is right, obtain the incremental record of each service sub-system missing and the incremental record of redundancy.

For example, if the first incremental data of service sub-system 1 includes:Incremental record 1, incremental record 2, the and of incremental record 3 Incremental record 4, the first incremental data of service sub-system 2 include:Incremental record 1, incremental record 2, incremental record 3 and increment note Record 5, preferably after the second incremental data include:Incremental record 1, incremental record 2, incremental record 3 and incremental record 5.

By the first incremental data of service sub-system 1 with preferably after the second incremental data be compared, obtained missing Incremental record be incremental record 5, the incremental record of redundancy is incremental record 4, by the first incremental data of service sub-system 2 with Second incremental data is compared, and both are completely the same, both the incremental record without missing, also irredundant incremental record.

Step S307, the incremental record of missing is added separately in corresponding service sub-system, by the institute of redundancy Incremental record is stated to delete from corresponding service sub-system respectively.

Specifically, System of Synchronous Processing can distinguish the incremental record of the incremental record and redundancy of missing Default second storage location is stored into, so that each service sub-system is read each from second storage location respectively The incremental record of the incremental record and redundancy of missing, and by the incremental record of missing added to corresponding the In one incremental data, and from corresponding first incremental data delete redundancy the incremental record, obtain the increasing after synchronization Measure data.

Preferably for the relatively low service sub-system of data reliability, it is necessary to be investigated to it, mistake occurs to reduce it Probability by mistake.Specifically, step as shown in Figure 4 can be included:

Step S401, the sum of the historical operation record of each service sub-system is counted respectively.

The historical operation is recorded as in default second time period the service sub-system when synchronizing processing Performed operation note (including the operation of addition incremental record and operation of deletion incremental record), in order to avoid short-term burst shape The influence of condition, the second time period should long enough, for example, can take one week, one month etc..

Step S402, the service sub-system that the sum that the historical operation is recorded is more than default 3rd threshold value are true It is set to and treats Check System;

The sum of historical operation record is bigger, then it is higher to illustrate that wrong probability occurs in its data.3rd threshold value Value can be configured according to actual conditions, and the present embodiment is not especially limited to this, if but it should be noted that its value It is bigger, it is determined that what is gone out treats that Check System is fewer, if its value is smaller, it is determined that what is gone out treats that Check System is more.

Step S403, the staff for notifying to specify treat that Check System is investigated to described.

In summary, by the embodiment of the present invention, incremental data is obtained from each service sub-system respectively first, then It is collected, the incremental data collected provides synchronous standard, and the incremental data difference of each service sub-system is therewith It is compared, it can be deduced that its existing difference record between the synchronization criterion, because these differences are based on same same Step standard obtains, as long as each service sub-system is synchronized into processing to respective incremental data accordingly, so that it may protect It is consistent to demonstrate,prove the incremental data between them, and because the synchronization criterion summarizes the data of each service sub-system, pole The earth ensure that its accuracy, error rate when can substantially reduce synchronization using it as reference.

Corresponding to the incremental data synchronization method described in foregoing embodiments, Fig. 5 shows increasing provided in an embodiment of the present invention The schematic block diagram of data syn-chronization terminal device is measured, for convenience of description, illustrate only the part related to the embodiment of the present invention.

In the present embodiment, the incremental data synchronous terminal equipment can be desktop PC, notebook, palm electricity The computing device such as brain and cloud server.The incremental data synchronous terminal equipment may include:Processor 50, memory 51 and deposit Store up the computer program 52 that can be run in the memory 51 and on the processor 50.

The processor 50 can be CPU (Central Processing Unit, CPU), can also be Other general processors, digital signal processor (Digital Signal Processor, DSP), application specific integrated circuit (Application Specific Integrated Circuit, ASIC), field programmable gate array (Field- Programmable Gate Array, FPGA) either other PLDs, discrete gate or transistor logic, Discrete hardware components etc..General processor can be microprocessor or the processor can also be any conventional processor Deng.

The memory 51 can be the internal storage unit of the incremental data synchronous terminal equipment 5, such as incremental number According to the hard disk or internal memory of synchronous terminal equipment 5.The memory 51 can also be the outer of the incremental data synchronous terminal equipment 5 The plug-in type hard disk being equipped with portion's storage device, such as the incremental data synchronous terminal equipment 5, intelligent memory card (Smart Media Card, SMC), secure digital (Secure Digital, SD) card, flash card (Flash Card) etc..Further, The memory 51 can also both include the internal storage unit of the incremental data synchronous terminal equipment 5 or including external storage Equipment.The memory 51 is used to storing the computer program and its needed for the incremental data synchronous terminal equipment 5 Its program and data.The memory 51 can be also used for temporarily storing the data that has exported or will export.

Referring to Fig. 6, it is the Program modual graph of computer program 52 provided in an embodiment of the present invention.In the present embodiment, Described computer program 52 can be divided into one or more modules, and one or more of modules are stored in described In memory 51, and as performed by the processor 50, to complete the present invention.For example, in figure 6, described computer program 52, namely incremental data synchronization program can be divided into incremental data acquisition module 601, incremental data summarizing module 602, One comparing module 603, missing record add module 604.Describe specifically to introduce the work(of the program module 601-604 below Energy.

Incremental data acquisition module 601, for obtaining the first incremental data from each service sub-system respectively, described One incremental data is each service sub-system user data actually newly-increased in default first time period;

Incremental data summarizing module 602, for collecting to each first incremental data of acquisition, obtain second Incremental data, the user that second incremental data should increase newly for each service sub-system in the first time period Data;

First comparing module 603, for each first incremental data to be carried out with second incremental data respectively Compare, obtain the incremental record of each service sub-system missing, the incremental record is the elementary cell of incremental data, often Bar incremental record includes a business information;

Missing record add module 604, for the incremental record of missing to be added separately into corresponding business subsystem In system, to complete the synchronization to the incremental data of each service sub-system.

Alternatively, the computer program 52 can also include:

Number statistical module, increase for counting each bar incremental record in second incremental data respectively described first The total degree occurred in amount data;

First preferred record abstraction module, for being extracted from incremental record described in each bar in second incremental data Go out the total degree be more than default first threshold preferred record, the first threshold by the service sub-system total number It is determined that;

First comprising modules, for the second incremental data by preferred record composition described in each bar extracted after preferred.

Alternatively, the computer program 52 can also include:

Confidence level computing module, for calculating described the according to the data reliability of default each service sub-system The confidence level of each bar incremental record in two incremental datas;

Second preferred record abstraction module, for being extracted from incremental record described in each bar in second incremental data Go out the preferred record that the confidence level is more than default Second Threshold;

Second comprising modules, for the second incremental data by preferred record composition described in each bar extracted after preferred.

Further, the computer program 52 can also include:

Second comparing module, for by each first incremental data respectively with preferably after second incremental data It is compared, obtains the incremental record of each service sub-system redundancy;

Redundant recording removing module, for the incremental record of redundancy to be deleted from corresponding service sub-system respectively Remove.

Further, the computer program 52 can also include:

Operation statistics module, the sum that the historical operation for counting each service sub-system respectively records are described Historical operation is recorded as the service sub-system performed operation note when synchronizing processing in default second time period Record;

Check System determining module is treated, the sum for the historical operation to be recorded is more than the institute of default 3rd threshold value Service sub-system is stated to be defined as treating Check System;

Notification module is investigated, the staff for notifying to specify treats that Check System is investigated to described.

It is apparent to those skilled in the art that for convenience and simplicity of description, the system of foregoing description, The specific work process of device and unit, the corresponding process in preceding method embodiment is may be referred to, will not be repeated here.

In the above-described embodiments, the description to each embodiment all emphasizes particularly on different fields, and is not described in detail or remembers in some embodiment The part of load, it may refer to the associated description of other embodiments.

Those of ordinary skill in the art are it is to be appreciated that each embodiment described with reference to the embodiments described herein Module, unit and/or method and step, it can be realized with the combination of electronic hardware or computer software and electronic hardware.This A little functions are performed with hardware or software mode actually, application-specific and design constraint depending on technical scheme.Specially Industry technical staff can realize described function using distinct methods to each specific application, but this realization is not It is considered as beyond the scope of this invention.

In several embodiments provided herein, it should be understood that disclosed system, apparatus and method can be with Realize by another way.For example, device embodiment described above is only schematical, for example, the unit Division, only a kind of division of logic function, can there is other dividing mode, such as multiple units or component when actually realizing Another system can be combined or be desirably integrated into, or some features can be ignored, or do not perform.It is another, it is shown or The mutual coupling discussed or direct-coupling or communication connection can be the indirect couplings by some interfaces, device or unit Close or communicate to connect, can be electrical, mechanical or other forms.

The unit illustrated as separating component can be or may not be physically separate, show as unit The part shown can be or may not be physical location, you can with positioned at a place, or can also be distributed to multiple On NE.Some or all of unit therein can be selected to realize the mesh of this embodiment scheme according to the actual needs 's.

In addition, each functional unit in each embodiment of the present invention can be integrated in a processing unit, can also That unit is individually physically present, can also two or more units it is integrated in a unit.Above-mentioned integrated list Member can both be realized in the form of hardware, can also be realized in the form of SFU software functional unit.

If the integrated unit is realized in the form of SFU software functional unit and is used as independent production marketing or use When, it can be stored in a computer read/write memory medium.Based on such understanding, technical scheme is substantially The part to be contributed in other words to prior art or all or part of the technical scheme can be in the form of software products Embody, the computer software product is stored in a storage medium, including some instructions are causing a computer Equipment (can be personal computer, server, or network equipment etc.) performs the complete of each embodiment methods described of the present invention Portion or part steps.And foregoing storage medium includes:USB flash disk, mobile hard disk, read-only storage (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disc or CD etc. are various can store journey The medium of sequence code.

Described above, the above embodiments are merely illustrative of the technical solutions of the present invention, rather than its limitations;Although with reference to before Embodiment is stated the present invention is described in detail, it will be understood by those within the art that:It still can be to preceding State the technical scheme described in each embodiment to modify, or equivalent substitution is carried out to which part technical characteristic;And these Modification is replaced, and the essence of appropriate technical solution is departed from the spirit and scope of various embodiments of the present invention technical scheme.

Claims (10)

  1. A kind of 1. incremental data synchronization method, it is characterised in that including:
    The first incremental data is obtained from each service sub-system respectively, first incremental data is each business subsystem System user data actually newly-increased in default first time period;
    Each first incremental data of acquisition is collected, obtains the second incremental data, second incremental data is The user data that each service sub-system should increase newly in the first time period;
    Each first incremental data is compared with second incremental data respectively, obtains each business subsystem The incremental record for missing of uniting, the incremental record are the elementary cell of incremental data, and every incremental record includes a business and believed Breath;
    The incremental record of missing is added separately in corresponding service sub-system, to complete to each business subsystem The synchronization of the incremental data of system.
  2. 2. incremental data synchronization method according to claim 1, it is characterised in that after the second incremental data is obtained, Also include:
    The total degree that each bar incremental record in second incremental data occurs in first incremental data is counted respectively;
    The total degree is extracted from incremental record described in each bar in second incremental data and is more than default first threshold The preferred record of value, the first threshold are determined by the total number of the service sub-system;
    By second incremental data of the preferred record composition described in each bar extracted after preferred.
  3. 3. incremental data synchronization method according to claim 1, it is characterised in that after the second incremental data is obtained, Also include:
    Each bar increment in second incremental data is calculated according to the data reliability of default each service sub-system The confidence level of record;
    The confidence level is extracted from incremental record described in each bar in second incremental data and is more than default second threshold The preferred record of value;
    By second incremental data of the preferred record composition described in each bar extracted after preferred.
  4. 4. the incremental data synchronization method according to Claims 2 or 3, it is characterised in that also include:
    By each first incremental data respectively with preferably after second incremental data be compared, obtain each described The incremental record of service sub-system redundancy;
    The incremental record of redundancy is deleted from corresponding service sub-system respectively.
  5. 5. incremental data synchronization method according to any one of claim 1 to 3, it is characterised in that also include:
    The sum of the historical operation record of each service sub-system is counted respectively, and the historical operation is recorded as default The service sub-system performed operation note when synchronizing processing in second time period;
    The service sub-system that the sum that the historical operation is recorded is more than default 3rd threshold value is defined as treating investigation system System;
    The staff specified is notified to treat that Check System is investigated to described.
  6. 6. a kind of computer-readable recording medium, the computer-readable recording medium storage has computer program, and its feature exists In incremental data synchronization side of the realization as any one of claim 1 to 5 when the computer program is executed by processor The step of method.
  7. 7. a kind of incremental data synchronous terminal equipment, including memory, processor and it is stored in the memory and can be The computer program run on the processor, it is characterised in that realized such as during computer program described in the computing device Lower step:
    The first incremental data is obtained from each service sub-system respectively, first incremental data is each business subsystem System user data actually newly-increased in default first time period;
    Each first incremental data of acquisition is collected, obtains the second incremental data, second incremental data is The user data that each service sub-system should increase newly in the first time period;
    Each first incremental data is compared with second incremental data respectively, obtains each business subsystem The incremental record for missing of uniting, the incremental record are the elementary cell of incremental data, and every incremental record includes a business and believed Breath;
    The incremental record of missing is added separately in corresponding service sub-system, to complete to each business subsystem The synchronization of the incremental data of system.
  8. 8. incremental data synchronous terminal equipment according to claim 7, it is characterised in that obtain the second incremental data it Afterwards, in addition to:
    The total degree that each bar incremental record in second incremental data occurs in first incremental data is counted respectively;
    The total degree is extracted from incremental record described in each bar in second incremental data and is more than default first threshold The preferred record of value, the first threshold are determined by the total number of the service sub-system;
    By second incremental data of the preferred record composition described in each bar extracted after preferred.
  9. 9. incremental data synchronous terminal equipment according to claim 7, it is characterised in that obtain the second incremental data it Afterwards, in addition to:
    Each bar increment in second incremental data is calculated according to the data reliability of default each service sub-system The confidence level of record;
    The confidence level is extracted from incremental record described in each bar in second incremental data and is more than default second threshold The preferred record of value;
    By second incremental data of the preferred record composition described in each bar extracted after preferred.
  10. 10. the incremental data synchronous terminal equipment according to any one of claim 7 to 9, it is characterised in that also include:
    The sum of the historical operation record of each service sub-system is counted respectively, and the historical operation is recorded as default The service sub-system performed operation note when synchronizing processing in second time period;
    The service sub-system that the sum that the historical operation is recorded is more than default 3rd threshold value is defined as treating investigation system System;
    The staff specified is notified to treat that Check System is investigated to described.
CN201710825934.2A 2017-09-14 2017-09-14 A kind of incremental data synchronization method and terminal device CN107689998A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201710825934.2A CN107689998A (en) 2017-09-14 2017-09-14 A kind of incremental data synchronization method and terminal device

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201710825934.2A CN107689998A (en) 2017-09-14 2017-09-14 A kind of incremental data synchronization method and terminal device
PCT/CN2018/074869 WO2019052112A1 (en) 2017-09-14 2018-01-31 Incremental data synchronization method, readable storage medium, terminal device, and apparatus

Publications (1)

Publication Number Publication Date
CN107689998A true CN107689998A (en) 2018-02-13

Family

ID=61156265

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201710825934.2A CN107689998A (en) 2017-09-14 2017-09-14 A kind of incremental data synchronization method and terminal device

Country Status (2)

Country Link
CN (1) CN107689998A (en)
WO (1) WO2019052112A1 (en)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101577817A (en) * 2009-06-10 2009-11-11 中兴通讯股份有限公司 Distributed video monitoring system and method thereof for keeping data consistency
CN102096685A (en) * 2009-12-11 2011-06-15 阿里巴巴集团控股有限公司 Method and device for synchronizing distributive data into data warehouse
CN103823884A (en) * 2014-03-06 2014-05-28 国家电网公司 Maintenance right-based loose consistency synchronization method for grid calculating data

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101577817A (en) * 2009-06-10 2009-11-11 中兴通讯股份有限公司 Distributed video monitoring system and method thereof for keeping data consistency
CN102096685A (en) * 2009-12-11 2011-06-15 阿里巴巴集团控股有限公司 Method and device for synchronizing distributive data into data warehouse
CN103823884A (en) * 2014-03-06 2014-05-28 国家电网公司 Maintenance right-based loose consistency synchronization method for grid calculating data

Also Published As

Publication number Publication date
WO2019052112A1 (en) 2019-03-21

Similar Documents

Publication Publication Date Title
CN102929901B (en) The method and apparatus improving data warehouse performance
WO2007022363A2 (en) Conformance authority reconciliation
Ansell et al. UK Renal Registry 11th Annual Report (December 2008): Chapter 7 Survival and causes of death of UK adult patients on Renal Replacement Therapy in 2007: national and centre-specific analyses
CN106055997A (en) Method and system for applying parental-control policy to media file
CN101312559A (en) Consumer short message management method based on mobile communication terminal and mobile communication terminal thereof
CN102355482B (en) Data transmission method and equipment thereof
US7908239B2 (en) System for storing event data using a sum calculator that sums the cubes and squares of events
Brännäs et al. Asymmetric time series and temporal aggregation
CN104182506A (en) Log management method
CN103593440B (en) The reading/writing method and device of journal file
Wang et al. Research and design of RFID data processing model based on complex event processing
US20110235909A1 (en) Analyzing documents using stored templates
CN104270275B (en) A kind of aided analysis method of abnormal cause, server and smart machine
CN105335509B (en) A kind of method for recommending action message, device and server
CN104348810B (en) The detection method of stolen account number, apparatus and system
CN103391311A (en) Method and system for consistency verification of data among multiple platforms
CN104156201B (en) Apparatus and method for generating user interface
CN107810500A (en) Data quality analysis
CN103678339B (en) Data backflow method and system and data access method and system in relational database
CN102831168B (en) Index Calculation Method and apparatus kpi
CN105117402B (en) Daily record data sharding method and device
CN102385736A (en) Consumption statistic system for credit card and mobile phone with consumption statistic function of credit card
CN104090835B (en) eID (electronic IDentity) and spectrum theory based cross-platform virtual asset transaction audit method
CN104636240B (en) A kind of acquisition methods and terminal of information report
US20120284268A1 (en) Space-time-nodal type signal processing

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination