CN109885614A - A kind of method and apparatus that data are synchronous - Google Patents
A kind of method and apparatus that data are synchronous Download PDFInfo
- Publication number
- CN109885614A CN109885614A CN201910008352.4A CN201910008352A CN109885614A CN 109885614 A CN109885614 A CN 109885614A CN 201910008352 A CN201910008352 A CN 201910008352A CN 109885614 A CN109885614 A CN 109885614A
- Authority
- CN
- China
- Prior art keywords
- data
- field
- target matrix
- corresponding relationship
- synchronized
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 230000001360 synchronised effect Effects 0.000 title claims abstract description 122
- 238000000034 method Methods 0.000 title claims abstract description 42
- 239000011159 matrix material Substances 0.000 claims abstract description 132
- 238000013479 data entry Methods 0.000 claims abstract description 37
- 238000003780 insertion Methods 0.000 claims abstract description 32
- 230000037431 insertion Effects 0.000 claims abstract description 32
- 239000000284 extract Substances 0.000 claims abstract description 7
- 238000012795 verification Methods 0.000 claims description 26
- 238000004590 computer program Methods 0.000 claims description 20
- 238000012545 processing Methods 0.000 abstract description 6
- 230000006870 function Effects 0.000 description 12
- 238000010586 diagram Methods 0.000 description 9
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 238000004891 communication Methods 0.000 description 2
- 238000012937 correction Methods 0.000 description 2
- 230000008878 coupling Effects 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 241001269238 Data Species 0.000 description 1
- 238000004422 calculation algorithm Methods 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000000052 comparative effect Effects 0.000 description 1
- 230000010485 coping Effects 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
- 238000005192 partition Methods 0.000 description 1
- 238000000926 separation method Methods 0.000 description 1
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The embodiment of the present invention is suitable for technical field of data processing, provide a kind of method and apparatus that data are synchronous, the described method includes: the corresponding relationship between configuration source data table and target matrix, corresponding relationship includes the field corresponding relationship in tables of data corresponding relationship and source data table between source data table itself and target matrix itself in each field and target matrix between each field；Determine the maximum value that data entry is numbered in target matrix；The maximum value and tables of data corresponding relationship and field corresponding relationship numbered according to data entry in target matrix generate the synchronous execution sentence of data, and the synchronous sentence that executes of data includes query statement and insertion sentence；It executes query statement and extracts data to be synchronized from source data table；It executes insertion sentence data to be synchronized are inserted into target matrix；Completeness check is carried out to target matrix.The present embodiment can realize increment synchronization between source data table and target matrix.
Description
Technical field
The invention belongs to technical field of data processing, synchronous more particularly to a kind of synchronous method of data, a kind of data
Device, a kind of equipment and a kind of computer readable storage medium.
Background technique
Data, which synchronize, can be synchronized to the data in certain tables of data in other tables of data.Currently, for database it
Between data synchronous mainly synchronize to realize by full dose.Full dose, which synchronizes, to be referred to the total data in source data table together
It is synchronized to target matrix.In general, the quantity stored in every tables of data is all very huge, if synchronous using full dose
Mode can all consume resources much more very per subsynchronous, and it is time-consuming also can be long, influence to tables of data or database just
It is often used.
On the other hand, in order to guarantee the consistency of synchronous front and back data, it is also necessary to periodically make to the data after synchronizing complete
Property verification.If being to be synchronized to the total data in source data table together since full dose synchronizes in such a way that full dose is synchronous
Target matrix, then data are finished whole property verification when also can only by target matrix data and source data table in
Data are directly compared, and control is lacked, so that the accuracy rate of verification is lower.
Summary of the invention
In view of this, the method and apparatus synchronous the embodiment of the invention provides a kind of data, to solve in the prior art
The low problem of data check accuracy rate after carrying out that the consumption of data isochronous resources is more, time-consuming using full dose is synchronous and synchronize.
The first aspect of the embodiment of the present invention provides a kind of method that data are synchronous, comprising:
The corresponding relationship between source data table and target matrix is configured, the corresponding relationship includes the source data table sheet
Each field and the mesh in tables of data corresponding relationship and the source data table between body and the target matrix itself
Mark the field corresponding relationship in tables of data between each field；
Determine the maximum value that data entry is numbered in the target matrix；
The maximum value and the tables of data corresponding relationship and field numbered according to data entry in the target matrix
Corresponding relationship generates the synchronous execution sentence of data, and the synchronous sentence that executes of the data includes query statement and insertion sentence；
It executes the query statement and extracts data to be synchronized from the source data table；
It executes the insertion sentence data to be synchronized are inserted into the target matrix；
Completeness check is carried out to the target matrix.
The second aspect of the embodiment of the present invention provides a kind of device that data are synchronous, comprising:
Corresponding relationship configuration module, for configuring the corresponding relationship between source data table and target matrix, the correspondence
Relationship includes the tables of data corresponding relationship and the source number between the source data table itself and the target matrix itself
According to the field corresponding relationship between each field in field each in table and the target matrix；
Entry number determining module, for determining the maximum value of data entry number in the target matrix；
Sentence generation module is executed, the maximum value for being numbered according to data entry in the target matrix, Yi Jisuo
Tables of data corresponding relationship and field corresponding relationship are stated, the synchronous execution sentence of data is generated, the synchronous sentence that executes of the data includes
Query statement and insertion sentence；
Data inquiry module to be synchronized extracts number to be synchronized from the source data table for executing the query statement
According to；
The data to be synchronized are inserted into the target for executing the insertion sentence by Data insertion module to be synchronized
In tables of data；
Data check module, for carrying out completeness check to the target matrix.
The third aspect of the embodiment of the present invention provides a kind of equipment, including memory, processor and is stored in described
In memory and the computer program that can run on the processor, the processor are realized when executing the computer program
The step of method that above-mentioned data synchronize.
The fourth aspect of the embodiment of the present invention provides a kind of computer readable storage medium, the computer-readable storage
Media storage has computer program, and the computer program realizes the step of the synchronous method of above-mentioned data when being executed by processor
Suddenly.
Compared with prior art, the embodiment of the present invention includes following advantages:
The embodiment of the present invention, by configuring the corresponding relationship between source data table and target matrix and determining target data
The maximum value that data entry is numbered in table, so as to the maximum value numbered according to data entry in target matrix, Yi Jiyuan
Tables of data corresponding relationship and field corresponding relationship between tables of data and target matrix, generating includes query statement and parenthesis
The synchronous execution sentence of data including sentence.Data to be synchronized can be extracted from source data table by executing query statement, executed
Above-mentioned data to be synchronized can be inserted into target matrix by insertion sentence, and carry out completeness check to target matrix.
The present embodiment can realize increment synchronization between source data table and target matrix, and it is only right when data are synchronous to allow
The content increased newly in source data table synchronizes, and reduces data and synchronizes occupancy to resource, improves the synchronous efficiency of data.
Detailed description of the invention
It to describe the technical solutions in the embodiments of the present invention more clearly, below will be to embodiment or description of the prior art
Needed in attached drawing be briefly described.It should be evident that the accompanying drawings in the following description is only of the invention some
Embodiment for those of ordinary skill in the art without any creative labor, can also be according to these
Attached drawing obtains other attached drawings.
Fig. 1 is a kind of step flow diagram of the synchronous method of data of one embodiment of the invention；
Fig. 2 is the step flow diagram of the synchronous method of another data of one embodiment of the invention；
Fig. 3 is a kind of schematic diagram of the synchronous device of data of one embodiment of the invention；
Fig. 4 is a kind of schematic diagram of equipment of one embodiment of the invention.
Specific embodiment
In being described below, for illustration and not for limitation, the tool of such as particular system structure, technology etc is proposed
Body details, to understand thoroughly the embodiment of the present invention.It should be apparent, however, to those skilled in the art that there is no these specific thin
The present invention also may be implemented in the other embodiments of section.In other cases, it omits to well-known system, device, circuit
And the detailed description of method, in case unnecessary details interferes description of the invention.
Illustrate technical solution of the present invention below by specific embodiment.
Referring to Fig.1, a kind of step flow diagram of the method for data synchronization of one embodiment of the invention, tool are shown
Body may include steps of:
Corresponding relationship between S101, configuration source data table and target matrix；
In embodiments of the present invention, source data table and target matrix can be stored in the data in disparate databases
Table, is also possible to the tables of data being stored in same database, and the present embodiment is not construed as limiting this.
In order to make it easy to understand, the present embodiment is carried out so that source data table is stored in disparate databases with target matrix as an example
Subsequent explanation and introduction.That is, it is considered that source data table is stored in first database, target matrix is stored in the second number
According in library, first database is different databases from the second database.
In addition, the target matrix in the present embodiment can be an empty table, i.e., it is not stored to have any other data；Or
Person, target matrix are also possible to be stored with the tables of data of other data, and the present embodiment is also not construed as limiting this.
In embodiments of the present invention, carrying out before data synchronize, can configure first source data table and target matrix it
Between corresponding relationship.Above-mentioned corresponding relationship not only includes that the tables of data between source data table itself and target matrix itself is corresponding
Relationship further includes the field corresponding relationship in source data table in each field and target matrix between each field.
For example, there are two tables of data, i.e. tables of data A and tables of data B, it is now desired to which the data in tables of data A are synchronous
Into tables of data B, then using tables of data A as source data table, tables of data B be target matrix, can initially set up tables of data A and
Tables of data corresponding relationship between tables of data B.
Then, the field corresponding relationship in further configuration data Table A and tables of data B between each field.Assuming that data
There are field Aa, Ab and Ac in Table A, there is field Ba, Bb and Bc in tables of data B, it is now desired to will be stored under field Aa in tables of data A
Data be synchronized under the field Ba in tables of data B, the data stored under field Ab in tables of data A are synchronized in tables of data B
Field Bb under, the data stored under field Ac in tables of data A are synchronized under the field Bc in tables of data B, then can establish
Corresponding relationship between above three field, as shown in the table:
Table one, the corresponding relationship example between source data table and target matrix
It should be noted that when corresponding relationship between configuration field, the whole words that can will be stored in source data table
A certain field in Duan Junyu target matrix establishes corresponding relationship, and it is synchronous only will can also to need to make data in source data table
Field and target matrix in a certain field establish corresponding relationship, the present embodiment is not construed as limiting this.
For example, it is assumed that having field Aa, Ab, Ac and Ad in tables of data A, but synchronous Aa, Ab, Ac are only needed when data are synchronous
The data stored under three fields, then can only by tri- fields of above-mentioned Aa, Ab, Ac respectively with certain in target matrix
One field establishes corresponding relationship, can also by tetra- fields of above-mentioned Aa, Ab, Ac and Ad with a certain field in target matrix
Establish corresponding relationship.
In completing corresponding relationship and two tables of data between source data table and target matrix between each field
Corresponding relationship with postponing, above-mentioned corresponding relationship can be written in configuration file and be saved.
S102, the maximum value that data entry is numbered in the target matrix is determined；
In general, the field in tables of data is stored according to certain sequence, said sequence can be by increasing certainly
ID embodies.It can be the entry number for referring to data storage in tables of data from increasing ID in tables of data.For each data,
Data entry number is all unique.For example, it is 1,2,3 ... 1000 that the data entry number in source data table, which can recorde,
Correspondingly, the data entry number in target matrix can also be shown as 1,2,3 ... 1000 accordingly.
In embodiments of the present invention, by determining that the maximum value of data entry number in target matrix can find out source number
Which it has been synchronized to target matrix according to data in table to suffer, so as to only to the number generated after the entry number
According to synchronizing, the increment synchronization of tables of data is realized, without synchronizing again to the total data in source data table, reduce
Synchronous data volume.
S103, according in the target matrix data entry number maximum value and the tables of data corresponding relationship
With field corresponding relationship, the synchronous execution sentence of data is generated；
In embodiments of the present invention, data, which synchronize, can be by executing specific SQL (Structured Query
Language, structured query language) sentence realizes.
Above-mentioned SQL statement may include the query statement that data are searched in source data table, and the data that will be found out
The insertion sentence being inserted into target matrix.
In the concrete realization, operator can generate above-mentioned query statement and parenthesis by way of scripted code
Sentence.
For example, query statement can be expressed as " select field Aa from table A ", i.e., from source data Table A
Select the data under field Aa；And " insert into table B " can be expressed as by being inserted into sentence, i.e., it will be in source data Table A
Data under field Aa are inserted into target matrix B.
Certainly, the above sentence is only a kind of example, and those skilled in the art can write suitable look into according to actual needs
Ask sentence and insertion sentence.For example, more specifically restrictive condition can be increased in SQL statement, such as search specific fields, certain
The data etc. generated in a period, to realize that personalized data synchronize.
In embodiments of the present invention, can determine field to be synchronized in source data table first, according in source data table to
The maximum value that data entry is numbered in synchronous field and target matrix generates the query statement for being directed to source data table, then
According to tables of data corresponding relationship and field corresponding relationship, the insertion sentence for being directed to target matrix is generated.
In the concrete realization, target data can be determined according to the corresponding relationship between source data table and target matrix
SQL language is added in the maximum value of the data entry number of synchronized data, the maximum value for then numbering above-mentioned data entry in table
In sentence, to only search the data for being greater than the maximum value of above-mentioned data entry number.
For example, in the examples described above, the maximum value of the data entry number in target matrix is 1000, then generating
When the SQL statement that incremental data synchronizes, the maximum value that 1000 this data entry can be numbered is added in query statement, even if
With " 1000 from table A of select field Aa over " this query statement, data are found out from source data table
Entry number is greater than 1000 data.
Then, according to tables of data corresponding relationship and field corresponding relationship, the insertion sentence for being directed to target matrix is generated
“insert into field Ba in table B”。
This variable is numbered by increasing data entry in the SQL statement for executing simultaneously operating, may be implemented to source number
Make increment synchronization according to the incremental data in table, it is synchronous to make a full dose to whole data without every time.
S104, the execution query statement extract data to be synchronized from the source data table；
It in embodiments of the present invention, include from source data table by executing SQL statement to complete the synchronous complete procedure of data
In find out data to be synchronized, and data to be synchronized are inserted into target matrix.
Therefore, the query statement that data to be synchronized are searched from source data table, such as " select can be first carried out
Field Aa from table A ", from the data stored under field Aa are extracted in source data Table A, then executing will be to be synchronized
Data are inserted into the insertion sentence in target matrix, such as " insert into table B ".
It should be noted that the data in the present embodiment, which synchronize, can be the synchronization carried out to incremental data, therefore synchronous
Frequency can be set relatively high.For example, every 10 minutes one subsynchronous SQL statements of execution can be set, to incremental data into
Row synchronizes.
It can include the SQL statement that the data entry of tables of data is numbered by execution when being synchronized to incremental data
To complete.
For example, by executing " 1000 from table A of select field Aa over " this query statement, it can
To find out the data that data entry number is greater than 1000 from source data table, the data found out are then synchronized to number of targets
According to table.
The data to be synchronized are inserted into the target matrix by S105, the execution insertion sentence；
For example, " insert into field Ba in table B " this insertion sentence can be executed, it will be from source data
Data of the data entry number greater than 1000 found out in table are inserted into the field Ba of target matrix.
S106, completeness check is carried out to the target matrix.
Data integrity verifying is the integrality and accuracy to guarantee data and a kind of verification operation for carrying out.In this hair
In bright embodiment, data integrity verifying is able to verify that whether the data after synchronizing are identical as source data.
In embodiments of the present invention, the synchronization of a full dose data can be completed first on the basis of source data table, i.e., will
Total data in source data Table A is synchronized in verification tables of data C.
Since the synchronous resource occupied of full dose is relatively more, take a long time, therefore can choose during idle time, such as morning
It is synchronous that the equal periods carry out full dose.
Complete after full dose synchronizes, can by the data verified in tables of data C with it is formerly synchronous after obtained target data
Table B is compared, with the integrality and accuracy of verify data.
In embodiments of the present invention, by configuring the corresponding relationship between source data table and target matrix and determining target
In tables of data data entry number maximum value, so as to according in target matrix data entry number maximum value, with
And tables of data corresponding relationship and field corresponding relationship between source data table and target matrix, it generates including query statement and inserts
Enter the synchronous execution sentence of data including sentence.Data to be synchronized can be extracted from source data table by executing query statement,
Executing insertion sentence can be inserted into above-mentioned data to be synchronized in target matrix, and carry out integrality school to target matrix
It tests.The present embodiment can realize increment synchronization between source data table and target matrix, allow when data are synchronous only
Only the content increased newly in source data table is synchronized, reduce data and synchronize occupancy to resource, it is synchronous to improve data
Efficiency.
Referring to Fig. 2, the step flow diagram of the method for another data synchronization of one embodiment of the invention is shown,
It can specifically include following steps:
S201, the dynamic coding for determining each field in the source data table；
In embodiments of the present invention, source data table can be the tables of data with more attribute fields.With more attribute fields
Tables of data refer to that field attribute corresponding to the pieces of data being recorded in same tables of data includes there are many.
For example, record has 4 datas as shown in the table in certain Zhang Yuan's tables of data:
Table two, more attribute field datatable examples
| Field 1 | Field 2 | Field 3 | |
| 1st data | Clothes 1 | Color | It is red |
| 2nd data | Clothes 2 | Color | It is green |
| 3rd data | Clothes 1 | Size | 38 |
| 4th data | Clothes 2 | Size | 40 |
Record has a two kinds of different attributes of clothes, i.e. color and size in above-mentioned tables of data, and with character (Chinese character or
English character) form, have recorded the corresponding attribute value of the attribute simultaneously in source data table.
As it can be seen that repeating to have recorded color and size two in the form of multiple fields in the tables of data from the example of upper table two
Attribute causes data redundancy.Method of data synchronization in through this embodiment, it is possible to reduce these data redundancies.
When doing data synchronization to the tables of data, the dynamic coding of each field in source data table can be determined first
(code)。
S202, according to the dynamic coding, the data under each field are associated, so that with identical ID
Data are corresponding with the same dynamic coding；
It in embodiments of the present invention, can be on the basis of the data under certain field, by the number of same type under the field
According to corresponding to the same dynamic coding.
For example, for the data in table two, it can be on the basis of the lower data recorded of field 1, according to what is recorded in field 1
Clothes ID determines dynamic coding, so that the pieces of data dynamic coding having the same of clothes ID having the same.
Therefore, in the data record of above-mentioned source data table, the dynamic coding of the 1st data and the 3rd data is same
It is a, can be expressed as code1, the dynamic coding of the 2nd data and the 4th data be it is same, be expressed as code2, obtain as
Related information shown in following table.
Table three, the data instance after association
| Field 1 | Field 2 | Field 3 | ||
| code1 | 1st data | Clothes 1 | Color | It is red |
| code2 | 2nd data | Clothes 2 | Color | It is green |
| code1 | 3rd data | Clothes 1 | Size | 38 |
| code2 | 4th data | Clothes 2 | Size | 40 |
So as to which the data after association are synchronized to target matrix.
S203, according to more attribute fields of the source data table, determine the gauge outfit of the target matrix, the gauge outfit packet
Include multiple attribute-names；
It in embodiments of the present invention, can be by number of targets when the data after association are synchronized to target matrix
It is realized according to the above-mentioned code of dynamic insertion in table.
In the concrete realization, multiple preset attribute-names can be written in target matrix, keep each attribute-name corresponding
In a field.Above-mentioned multiple preset attribute-names can be by being analyzed to obtain to the data recorded in source data table.Example
Such as, for the source data table in above-mentioned example, attribute-name may include clothes ID and color, size three.
Therefore, the gauge outfit of target matrix can indicate are as follows:
Table four, the gauge outfit example of target matrix
| Data sequence number | Clothes ID | Color | Size |
S204, according to the dynamic coding, determine that each attribute-name is corresponding in the data with identical ID respectively
Attribute value；
It is then possible to determine that each attribute-name is corresponding in the data with identical ID respectively according to above-mentioned dynamic coding
Attribute value.
For example, it includes " color " and " size " that clothes ID, which is " clothes 1 " corresponding attribute-name,.Wherein, " color " is corresponding
Attribute value is " red ", and " size " corresponding attribute value is " 38 ".
S205, field corresponding with the multiple attribute-name in the target matrix is written into the attribute value one by one
In；
In embodiments of the present invention, corresponding number can be inserted into target matrix one by one on the basis of dynamic coding
According to.
I.e., it is first determined the corresponding data record of code1 include the 1st article and the 3rd data, will above-mentioned two record in
Clothes ID is written under the corresponding field of target matrix clothes ID, obtains:
Table five
| Data sequence number | Clothes ID | Color | Size |
| 1st data | Clothes 1 |
Then, it by corresponding color value and size value in the 1st article in determining source data table and the 3rd data record, writes
Enter into target matrix under corresponding field, obtain:
Table six
| Data sequence number | Clothes ID | Color | Size |
| 1st data | Clothes 1 | It is red | 38 |
According to the method described above, the synchronization to all data records is completed, is obtained:
Table seven
S206, in default free time, it is synchronous that full dose made to the source data table, obtains verification tables of data；
In embodiments of the present invention, the synchronization of a full dose data can be completed first on the basis of source data table, i.e., will
Total data in source data table is synchronized in verification tables of data.
Since the synchronous resource occupied of full dose is relatively more, take a long time, therefore can choose during idle time, such as morning
It is synchronous that the equal periods carry out full dose.
S207, each field in the target matrix is compared with corresponding field in the verification tables of data
Compared with；
When being compared to the data in target data and verification tables of data, field can carry out one by one.For example, can be with
Compare first the data that are stored under field Ba in target matrix B whether with the data that are stored under field Ca in verification tables of data C,
After the completion of the data under the field are all compared, then compare the data that are stored under field Bb in target matrix B whether with
The data stored under field Cb in target matrix C, the comparison until completing total data.
If there are corresponding fields in the data and the verification tables of data in either field in S208, the target matrix
In data it is different, then according to the data in the verification tables of data in corresponding field to the data in the target matrix into
Row amendment.
In embodiments of the present invention, if each field in target matrix is remembered with corresponding field in verification tables of data
The data of record are all the same, it may be considered that the data after synchronizing are complete.
If there are the data in corresponding field in the data and verification tables of data in either field in target matrix not
Together, it may be considered that occurring mistake in synchronizing process, needs to be modified data.
It, should be with school when being modified to data since the synchronous verification tables of data of full dose is newest synchronization
It tests subject to tables of data.Therefore, can be by data correcting different in target matrix and corresponding position in verification tables of data
Identical data.
In embodiments of the present invention, when needing to carry out completeness check to data, source data table can be made first complete
Amount synchronizes, and then verification tables of data by comparing the target matrix after increment synchronization after synchronous with full dose carries out integrality
Verification is verified by using two tables of data controls, improves the accuracy rate of verification.
It should be noted that the size of the serial number of each step is not meant that the order of the execution order in above-described embodiment,
The execution sequence of each process should be determined by its function and internal logic, and the implementation process without coping with the embodiment of the present invention, which is constituted, appoints
What is limited.
Referring to Fig. 3, a kind of schematic diagram of the device of data synchronization of one embodiment of the invention is shown, specifically can wrap
Include following module:
Corresponding relationship configuration module 301 is described right for configuring the corresponding relationship between source data table and target matrix
Should be related to including between the source data table itself and the target matrix itself tables of data corresponding relationship and the source
Field corresponding relationship in tables of data in each field and the target matrix between each field；
Entry number determining module 302, for determining the maximum value of data entry number in the target matrix；
Sentence generation module 303 is executed, the maximum value for being numbered according to data entry in the target matrix, and
The tables of data corresponding relationship and field corresponding relationship generate the synchronous execution sentence of data, and the data are synchronous to execute sentence packet
Include query statement and insertion sentence；
Data inquiry module 304 to be synchronized is extracted for executing the query statement from the source data table to be synchronized
Data；
Data insertion module 305 to be synchronized is inserted into for executing the insertion sentence for the data to be synchronized described
In target matrix；
Data check module 306, for carrying out completeness check to the target matrix.
In embodiments of the present invention, the execution sentence generation module 303 can specifically include following submodule:
Field to be synchronized determines submodule, for determining field to be synchronized in the source data table；
Query statement generates submodule, for according to field to be synchronized in the source data table and the target matrix
The maximum value of middle data entry number, generates the query statement for being directed to the source data table；
It is inserted into sentence and generates submodule, for being directed to according to the tables of data corresponding relationship and field corresponding relationship, generation
The insertion sentence of the target matrix.
In embodiments of the present invention, the field in the source data table is more attribute fields, and described device can also include
Following module:
Dynamic coding determining module, for determining the dynamic coding of each field in the source data table；
Data association module, for being associated to the data under each field according to the dynamic coding, so that
Data with identical ID are corresponding with the same dynamic coding；
Data simultaneous module, for the data after association to be synchronized to the target matrix.
In embodiments of the present invention, the data simultaneous module can specifically include following submodule:
Target matrix gauge outfit determines submodule, for more attribute fields according to the source data table, determines the mesh
The gauge outfit of tables of data is marked, the gauge outfit includes multiple attribute-names；
Attribute value determines submodule, for being determined in the data with identical ID respectively according to the dynamic coding
The corresponding attribute value of each attribute-name；
Attribute value be written submodule, for by the attribute value be written one by one in the target matrix with the multiple category
In the property corresponding field of name.
In embodiments of the present invention, the data check module 306 can specifically include following submodule:
Full dose synchronizes submodule, for making full dose to the source data table and synchronizing, obtain check number in default free time
According to table；
Data Comparative sub-module, for each field in the target matrix is corresponding with the verification tables of data
Field be compared.
In embodiments of the present invention, the data check module 306 can also include following submodule:
Data correction submodule, if in the target matrix there are in either field data and the check number
It is different according to the data in corresponding field in table, then according to the data in the verification tables of data in corresponding field to the number of targets
It is modified according to the data in table.
For device embodiment, since it is basically similar to the method embodiment, related so describing fairly simple
Place referring to embodiment of the method part explanation.
Referring to Fig. 4, a kind of schematic diagram of equipment of one embodiment of the invention is shown.As shown in figure 4, the present embodiment
Equipment 400 includes: processor 410, memory 420 and is stored in the memory 420 and can be on the processor 410
The computer program 421 of operation.The processor 410 realizes above-mentioned data synchronous side when executing the computer program 421
Step in each embodiment of method, such as step S101 to S106 shown in FIG. 1.Alternatively, the processor 410 executes the meter
The function of each module/unit in above-mentioned each Installation practice, such as module 301 to 306 shown in Fig. 3 are realized when calculation machine program 421
Function.
Illustratively, the computer program 421 can be divided into one or more module/units, it is one or
Multiple module/the units of person are stored in the memory 420, and are executed by the processor 410, to complete the present invention.Institute
Stating one or more module/units can be the series of computation machine program instruction section that can complete specific function, the instruction segment
It can be used for describing implementation procedure of the computer program 421 in the equipment 400.For example, the computer program 421
Corresponding relationship configuration module can be divided into, entry number determining module, execute sentence generation module, data query to be synchronized
Module, Data insertion module to be synchronized and data correction verification module, each module concrete function are as follows:
Corresponding relationship configuration module, for configuring the corresponding relationship between source data table and target matrix, the correspondence
Relationship includes the tables of data corresponding relationship and the source number between the source data table itself and the target matrix itself
According to the field corresponding relationship between each field in field each in table and the target matrix；
Entry number determining module, for determining the maximum value of data entry number in the target matrix；
Sentence generation module is executed, the maximum value for being numbered according to data entry in the target matrix, Yi Jisuo
Tables of data corresponding relationship and field corresponding relationship are stated, the synchronous execution sentence of data is generated, the synchronous sentence that executes of the data includes
Query statement and insertion sentence；
Data inquiry module to be synchronized extracts number to be synchronized from the source data table for executing the query statement
According to；
The data to be synchronized are inserted into the target for executing the insertion sentence by Data insertion module to be synchronized
In tables of data；
Data check module, for carrying out completeness check to the target matrix.
The equipment 400 can be desktop PC, notebook, palm PC and cloud server etc. and calculate equipment.
The equipment 400 may include, but be not limited only to, processor 410, memory 420.It will be understood by those skilled in the art that Fig. 4 is only
It is only a kind of example of equipment 400, does not constitute the restriction to equipment 400, may include components more more or fewer than diagram,
Perhaps certain components or different components are combined, such as the equipment 400 can also connect including input-output equipment, network
Enter equipment, bus etc..
The processor 410 can be central processing unit (Central Processing Unit, CPU), can also be
Other general processors, digital signal processor (Digital Signal Processor, DSP), specific integrated circuit
(Application Specific Integrated Circuit, ASIC), ready-made programmable gate array (Field-
Programmable Gate Array, FPGA) either other programmable logic device, discrete gate or transistor logic,
Discrete hardware components etc..General processor can be microprocessor or the processor is also possible to any conventional processor
Deng.
The memory 420 can be the internal storage unit of the equipment 400, such as the hard disk or memory of equipment 400.
The memory 420 is also possible to the External memory equipment of the equipment 400, such as the plug-in type being equipped in the equipment 400
Hard disk, intelligent memory card (Smart Media Card, SMC), secure digital (Secure Digital, SD) card, flash card
(Flash Card) etc..Further, the memory 420 can also both include the internal storage unit of the equipment 400
It also include External memory equipment.The memory 420 is for storing needed for the computer program 421 and the equipment 400
Other programs and data.The memory 420 can be also used for temporarily storing the data that has exported or will export.
Those skilled in the art can be understood that, for convenience of description and succinctly, only with above-mentioned each function
The division of unit, module is illustrated.In practical application, it can according to need and by above-mentioned function distribution by different function
Energy unit, module are completed, i.e., described device/terminal device internal structure is divided into different functional unit or module, with
Complete all or part of function described above.Each functional unit in embodiment, module can integrate single in a processing
In member, it is also possible to each unit and physically exists alone, can also be integrated in one unit with two or more units, on
It states integrated unit both and can take the form of hardware realization, can also realize in the form of software functional units.In addition, each
Functional unit, module specific name be also only for convenience of distinguishing each other, the protection scope being not intended to restrict the invention.On
The specific work process for stating unit in system, module, can refer to corresponding processes in the foregoing method embodiment, no longer superfluous herein
It states.
In the above-described embodiments, it all emphasizes particularly on different fields to the description of each embodiment, is not described in detail or remembers in some embodiment
The part of load, reference can be made to the related descriptions of other embodiments.
Those of ordinary skill in the art may be aware that list described in conjunction with the examples disclosed in the embodiments of the present disclosure
Member and algorithm steps can be realized with the combination of electronic hardware or computer software and electronic hardware.These functions are actually
It is implemented in hardware or software, the specific application and design constraint depending on technical solution.Professional technician
Each specific application can be used different methods to achieve the described function, but this realization is it is not considered that exceed
The scope of the present invention.
In embodiment provided by the present invention, it should be understood that disclosed device/terminal device and method, it can be with
It realizes in other way.For example, device described above/terminal device embodiment is only schematical, for example, institute
The division of module or unit is stated, only a kind of logical function partition, there may be another division manner in actual implementation, such as
Multiple units or components can be combined or can be integrated into another system, or some features can be ignored or not executed.Separately
On the one hand, shown or discussed mutual coupling or direct-coupling or communication connection can be through some interfaces, dress
It sets or the INDIRECT COUPLING or communication connection of unit, can be electrical property, mechanical or other forms.
The unit as illustrated by the separation member may or may not be physically separated, aobvious as unit
The component shown may or may not be physical unit, it can and it is in one place, or may be distributed over multiple
In network unit.It can select some or all of unit therein according to the actual needs to realize the mesh of this embodiment scheme
's.
It, can also be in addition, the functional units in various embodiments of the present invention may be integrated into one processing unit
It is that each unit physically exists alone, can also be integrated in one unit with two or more units.Above-mentioned integrated list
Member both can take the form of hardware realization, can also realize in the form of software functional units.
If the integrated module/unit be realized in the form of SFU software functional unit and as independent product sale or
In use, can store in a computer readable storage medium.Based on this understanding, the present invention realizes above-mentioned implementation
All or part of the process in example method, can also instruct relevant hardware to complete, the calculating by computer program
Machine program can be stored in computer readable storage medium, and the computer program is when being executed by processor, it can be achieved that above-mentioned each
The step of a embodiment of the method.Wherein, the computer program includes computer program code, and the computer program code can
Think source code form, object identification code form, executable file or certain intermediate forms etc..The computer readable storage medium
It may include: any entity or device, recording medium, USB flash disk, mobile hard disk, magnetic that can carry the computer program code
Dish, CD, computer storage, read-only memory (ROM, Read-Only Memory), random access memory (RAM,
Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium etc..It should be noted that described
It is appropriate that the content that computer readable storage medium includes can be carried out according to the requirement made laws in jurisdiction with patent practice
Increase and decrease, such as do not include electric carrier wave according to legislation and patent practice, computer readable storage medium in certain jurisdictions
Signal and telecommunication signal.
Embodiment described above is merely illustrative of the technical solution of the present invention, rather than its limitations.Although referring to aforementioned reality
Applying example, invention is explained in detail, those skilled in the art should understand that: it still can be to aforementioned each
Technical solution documented by embodiment is modified or equivalent replacement of some of the technical features；And these are modified
Or replacement, the spirit and scope for technical solution of various embodiments of the present invention that it does not separate the essence of the corresponding technical solution should all
It is included within protection scope of the present invention.
Claims (10)
1. a kind of synchronous method of data characterized by comprising
Configure the corresponding relationship between source data table and target matrix, the corresponding relationship include the source data table itself with
Each field and the number of targets in tables of data corresponding relationship and the source data table between described target matrix itself
According to the field corresponding relationship between field each in table；
Determine the maximum value that data entry is numbered in the target matrix；
The maximum value and the tables of data corresponding relationship and field numbered according to data entry in the target matrix are corresponding
Relationship generates the synchronous execution sentence of data, and the synchronous sentence that executes of the data includes query statement and insertion sentence；
It executes the query statement and extracts data to be synchronized from the source data table；
It executes the insertion sentence data to be synchronized are inserted into the target matrix；
Completeness check is carried out to the target matrix.
2. the method according to claim 1, wherein described number according to data entry in the target matrix
Maximum value and the tables of data corresponding relationship and field corresponding relationship, generating data synchronous the step of executing sentence includes:
Determine field to be synchronized in the source data table；
According to the maximum value that data entry in field to be synchronized in the source data table and the target matrix is numbered, generate
For the query statement of the source data table；
According to the tables of data corresponding relationship and field corresponding relationship, the insertion sentence for being directed to the target matrix is generated.
3. the method according to claim 1, wherein the field in the source data table is more attribute fields, institute
State method further include:
Determine the dynamic coding of each field in the source data table；
According to the dynamic coding, the data under each field are associated, so that data with identical ID and same
One dynamic coding is corresponding；
Data after association are synchronized to the target matrix.
4. according to the method described in claim 3, it is characterized in that, the data by after association are synchronized to the target data
The step of table includes:
According to more attribute fields of the source data table, determine that the gauge outfit of the target matrix, the gauge outfit include multiple categories
Property name；
According to the dynamic coding, the corresponding attribute value of each attribute-name in the data with identical ID is determined respectively；
The attribute value is written in the target matrix one by one in field corresponding with the multiple attribute-name.
5. the method according to claim 1, wherein described carry out completeness check to the target matrix
Step includes:
In default free time, full dose is made to the source data table and is synchronized, obtains verification tables of data；
Each field in the target matrix is compared with corresponding field in the verification tables of data.
6. according to the method described in claim 5, it is characterized by further comprising:
If there are the data in corresponding field in the data and the verification tables of data in either field in the target matrix
Difference is then modified the data in the target matrix according to the data in the verification tables of data in corresponding field.
7. a kind of synchronous device of data characterized by comprising
Corresponding relationship configuration module, for configuring the corresponding relationship between source data table and target matrix, the corresponding relationship
Including the tables of data corresponding relationship and the source data table between the source data table itself and the target matrix itself
In field corresponding relationship in each field and the target matrix between each field；
Entry number determining module, for determining the maximum value of data entry number in the target matrix；
Sentence generation module is executed, maximum value and the number for numbering according to data entry in the target matrix
According to table corresponding relationship and field corresponding relationship, the synchronous execution sentence of data is generated, the synchronous sentence that executes of the data includes inquiry
Sentence and insertion sentence；
Data inquiry module to be synchronized extracts data to be synchronized from the source data table for executing the query statement；
The data to be synchronized are inserted into the target data for executing the insertion sentence by Data insertion module to be synchronized
In table；
Data check module, for carrying out completeness check to the target matrix.
8. device according to claim 7, which is characterized in that the execution sentence generation module includes:
Field to be synchronized determines submodule, for determining field to be synchronized in the source data table；
Query statement generates submodule, for according to number in field to be synchronized in the source data table and the target matrix
According to the maximum value of entry number, the query statement for being directed to the source data table is generated；
It is inserted into sentence and generates submodule, for generating for described according to the tables of data corresponding relationship and field corresponding relationship
The insertion sentence of target matrix.
9. a kind of equipment, including memory, processor and storage can be run in the memory and on the processor
Computer program, which is characterized in that the processor is realized when executing the computer program as claim 1 to 6 is any
The step of method that the item data synchronize.
10. a kind of computer readable storage medium, the computer-readable recording medium storage has computer program, and feature exists
In the step of realization synchronous method of data as described in any one of claim 1 to 6 when the computer program is executed by processor
Suddenly.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910008352.4A CN109885614B (en) | 2019-01-04 | 2019-01-04 | Data synchronization method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910008352.4A CN109885614B (en) | 2019-01-04 | 2019-01-04 | Data synchronization method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109885614A true CN109885614A (en) | 2019-06-14 |
| CN109885614B CN109885614B (en) | 2024-06-28 |
Family
ID=66925527
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910008352.4A Active CN109885614B (en) | 2019-01-04 | 2019-01-04 | Data synchronization method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109885614B (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111984684A (en) * | 2020-08-20 | 2020-11-24 | 北京捷报金峰数据技术有限公司 | Data processing method and device |
| CN112181995A (en) * | 2020-11-06 | 2021-01-05 | 深圳视界信息技术有限公司 | Data processing method, device and equipment of data table and storage medium |
| CN112307124A (en) * | 2020-11-03 | 2021-02-02 | 平安普惠企业管理有限公司 | Database synchronization verification method, device, equipment and storage medium |
| CN112395850A (en) * | 2020-11-05 | 2021-02-23 | 中国人寿保险股份有限公司 | Method and device for checking data definition consistency and electronic equipment |
| CN112749227A (en) * | 2019-10-30 | 2021-05-04 | 北京国双科技有限公司 | Data synchronization method and device |
| CN113407551A (en) * | 2020-03-17 | 2021-09-17 | 北京同邦卓益科技有限公司 | Data consistency determining method, device, equipment and storage medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103164422A (en) * | 2011-12-09 | 2013-06-19 | 百度在线网络技术（北京）有限公司 | Method, equipment and system for conducting data synchronism by adopting general data storage format |
| CN108121757A (en) * | 2017-11-10 | 2018-06-05 | 广州优视网络科技有限公司 | A kind of method of data synchronization, device, system, computing device and storage medium |
| CN108228814A (en) * | 2017-12-29 | 2018-06-29 | 泰康保险集团股份有限公司 | Method of data synchronization and device |
| CN109062952A (en) * | 2018-06-22 | 2018-12-21 | 北京奇艺世纪科技有限公司 | A kind of data query method, apparatus and electronic equipment |
-
2019
- 2019-01-04 CN CN201910008352.4A patent/CN109885614B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103164422A (en) * | 2011-12-09 | 2013-06-19 | 百度在线网络技术（北京）有限公司 | Method, equipment and system for conducting data synchronism by adopting general data storage format |
| CN108121757A (en) * | 2017-11-10 | 2018-06-05 | 广州优视网络科技有限公司 | A kind of method of data synchronization, device, system, computing device and storage medium |
| CN108228814A (en) * | 2017-12-29 | 2018-06-29 | 泰康保险集团股份有限公司 | Method of data synchronization and device |
| CN109062952A (en) * | 2018-06-22 | 2018-12-21 | 北京奇艺世纪科技有限公司 | A kind of data query method, apparatus and electronic equipment |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112749227A (en) * | 2019-10-30 | 2021-05-04 | 北京国双科技有限公司 | Data synchronization method and device |
| CN113407551A (en) * | 2020-03-17 | 2021-09-17 | 北京同邦卓益科技有限公司 | Data consistency determining method, device, equipment and storage medium |
| CN111984684A (en) * | 2020-08-20 | 2020-11-24 | 北京捷报金峰数据技术有限公司 | Data processing method and device |
| CN112307124A (en) * | 2020-11-03 | 2021-02-02 | 平安普惠企业管理有限公司 | Database synchronization verification method, device, equipment and storage medium |
| CN112395850A (en) * | 2020-11-05 | 2021-02-23 | 中国人寿保险股份有限公司 | Method and device for checking data definition consistency and electronic equipment |
| CN112181995A (en) * | 2020-11-06 | 2021-01-05 | 深圳视界信息技术有限公司 | Data processing method, device and equipment of data table and storage medium |
| CN112181995B (en) * | 2020-11-06 | 2023-08-29 | 深圳数阔信息技术有限公司 | Data processing method, device, equipment and storage medium of data table |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109885614B (en) | 2024-06-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109885614A (en) | A kind of method and apparatus that data are synchronous | |
| US20200412526A1 (en) | Method and apparatus for verifying smart contracts in blockchain, and storage medium | |
| CN109325037A (en) | A kind of method of data synchronization, storage medium and server for mass data | |
| CN109508355A (en) | A kind of data pick-up method, system and terminal device | |
| CN110309125A (en) | Data verification method, electronic device and storage medium | |
| TW200931285A (en) | Method, system and apparatus for combining distributed computational data | |
| CN110209728A (en) | A kind of Distributed Heterogeneous Database synchronous method, electronic equipment and storage medium | |
| CN110399368B (en) | Method for customizing data table, data operation method and device | |
| CN107070645A (en) | Compare the method and system of the data of tables of data | |
| CN109522332A (en) | Customer profile data merging method, device, equipment and readable storage medium storing program for executing | |
| CN104871153A (en) | System and method for flexible distributed massively parallel processing (mpp) database | |
| WO2020206952A1 (en) | Graph database data import method and apparatus | |
| CN110209732A (en) | Method of data synchronization and device of the relevant database to Hadoop database | |
| CN110490724A (en) | The storage method and device of account data | |
| CN110399600A (en) | Generate the method and device of wide table | |
| CN104615490A (en) | Method and device for data conversion | |
| CN114416884B (en) | Method and device for connecting partition table | |
| CN109710698A (en) | A kind of data assemblage method, device, electronic equipment and medium | |
| CN110471918A (en) | A kind of relevant database is synchronized to the method and device of block chain | |
| CN111752677B (en) | Cloud resource configuration method, system, terminal and storage medium | |
| CN115718773A (en) | Distributed database query segmentation method and device | |
| CN110362551A (en) | Building Method of Data Warehouse, device, equipment and storage medium | |
| CN115422898A (en) | Visual self-defined report form analysis system based on container cloud | |
| CN108595552A (en) | Data cube dissemination method, device, electronic equipment and storage medium | |
| CN114328486A (en) | Data quality checking method and device based on model |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |